2024-03-26T08:20:44.984 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T08:20:44.984 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} --archive /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868 --name mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --owner mchangir -- /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/orig.config.yaml 2024-03-26T08:20:45.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T08:20:45.128 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868 branch: wip-mchangir-testing-main-20240318.032620 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} email: mchangir@redhat.com first_in_suite: false job_id: '7622868' kernel: client: sha1: distro flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3185 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 155/256 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8Yj+ysP5HHtSGqUEdgYXZC0WFzC+C5FDN3ts7vCWs6KPW/SCyNIvhZoQo1RmS9vK7Q99G/3ZXSHI5yMIbFwlU= smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFo3h6Erm2BsLSnUlOdEi8FvyVwBliJDXsDikSAuHaN01MyvUFkJnD3pi4pwpeHKiBULgLOpNad1W2MxmRrSoGQ= smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA22+X8yiGA/b41MzM9FcrUOdzjl1oWbKgj4gUUFUoyx2SVfSkHRLc5yVXqwwscfXUSrM8EtcWcD9F2LPnBiBEk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-26_07:59:35 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-26T08:20:45.128 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-26T08:20:45.129 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-26T08:20:45.129 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-26T08:20:45.130 INFO:teuthology.task.internal:Checking packages... 2024-03-26T08:20:45.155 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-26T08:20:45.155 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-26T08:20:45.155 INFO:teuthology.packaging:ref: None 2024-03-26T08:20:45.155 INFO:teuthology.packaging:tag: None 2024-03-26T08:20:45.155 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-26T08:20:45.155 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:20:45.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-26T08:20:45.394 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-26T08:20:45.395 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-26T08:20:45.402 INFO:teuthology.task.internal:no buildpackages task found 2024-03-26T08:20:45.402 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-26T08:20:45.410 INFO:teuthology.task.internal:Saving configuration 2024-03-26T08:20:45.432 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-26T08:20:45.439 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T08:20:45.463 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:16:08.375697', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8Yj+ysP5HHtSGqUEdgYXZC0WFzC+C5FDN3ts7vCWs6KPW/SCyNIvhZoQo1RmS9vK7Q99G/3ZXSHI5yMIbFwlU='} 2024-03-26T08:20:45.482 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:16:08.377228', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFo3h6Erm2BsLSnUlOdEi8FvyVwBliJDXsDikSAuHaN01MyvUFkJnD3pi4pwpeHKiBULgLOpNad1W2MxmRrSoGQ='} 2024-03-26T08:20:45.508 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:16:08.378476', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA22+X8yiGA/b41MzM9FcrUOdzjl1oWbKgj4gUUFUoyx2SVfSkHRLc5yVXqwwscfXUSrM8EtcWcD9F2LPnBiBEk='} 2024-03-26T08:20:45.508 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-26T08:20:45.517 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-26T08:20:45.517 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-26T08:20:45.517 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-26T08:20:45.517 INFO:teuthology.run_tasks:Running task console_log... 2024-03-26T08:20:45.628 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faadc0820d0>, signals=[15]) 2024-03-26T08:20:45.629 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-26T08:20:45.639 INFO:teuthology.task.internal:Opening connections... 2024-03-26T08:20:45.640 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-03-26T08:20:45.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:20:45.749 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-03-26T08:20:45.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:20:45.853 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-03-26T08:20:45.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:20:45.956 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-26T08:20:45.968 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-03-26T08:20:45.987 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-03-26T08:20:45.987 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T08:20:46.045 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-03-26T08:20:46.046 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T08:20:46.046 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T08:20:46.046 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T08:20:46.046 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-03-26T08:20:46.071 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-03-26T08:20:46.089 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-03-26T08:20:46.089 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-03-26T08:20:46.147 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T08:20:46.148 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T08:20:46.148 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T08:20:46.148 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-03-26T08:20:46.170 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-03-26T08:20:46.189 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-03-26T08:20:46.189 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-03-26T08:20:46.247 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-03-26T08:20:46.247 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-03-26T08:20:46.247 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-03-26T08:20:46.247 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-03-26T08:20:46.247 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-03-26T08:20:46.247 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-03-26T08:20:46.248 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T08:20:46.248 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-03-26T08:20:46.248 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-03-26T08:20:46.248 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T08:20:46.248 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-03-26T08:20:46.248 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T08:20:46.248 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T08:20:46.248 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T08:20:46.248 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-03-26T08:20:46.270 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-26T08:20:46.283 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-26T08:20:46.293 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-26T08:20:46.294 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-03-26T08:20:46.296 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-03-26T08:20:46.299 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-03-26T08:20:46.315 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-26T08:20:46.324 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-26T08:20:46.324 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-03-26T08:20:46.354 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-03-26T08:20:46.356 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-03-26T08:20:46.382 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-26T08:20:46.459 INFO:teuthology.run_tasks:Running task kernel... 2024-03-26T08:20:46.477 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-26T08:20:46.477 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-26T08:20:46.477 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-26T08:20:46.477 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T08:20:46.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:20:46.495 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-03-26T08:20:46.552 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-361.el9.x86_64 2024-03-26T08:20:46.553 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-361.el9.x86_64 2024-03-26T08:20:46.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-03-26T08:20:48.610 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-26T08:20:50.027 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2024-03-26T08:20:51.259 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-03-26T08:20:53.852 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-03-26T08:21:00.744 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.1 MB 00:03 2024-03-26T08:21:02.178 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-03-26T08:21:03.615 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-26T08:21:08.763 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-03-26T08:21:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-26T08:21:10.986 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-26T08:21:11.082 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-26T08:21:11.098 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-26T08:21:11.098 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-03-26T08:21:11.098 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-26T08:21:11.098 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-432.el9 baseos 66 k 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-432.el9 baseos 15 M 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-432.el9 baseos 33 M 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-432.el9 baseos 27 M 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-03-26T08:21:11.099 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:21:11.100 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 76 M 2024-03-26T08:21:11.100 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 123 M 2024-03-26T08:21:11.100 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-26T08:21:11.589 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-432.el9.x86_64.rpm 187 kB/s | 66 kB 00:00 2024-03-26T08:21:25.096 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-modules-5.14.0-432.el9.x86_64.rpm 2.4 MB/s | 33 MB 00:13 2024-03-26T08:21:27.296 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-core-5.14.0-432.el9.x86_64.rpm 952 kB/s | 15 MB 00:16 2024-03-26T08:21:33.020 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-core-5.14.0-432.el9.x86_6 1.3 MB/s | 27 MB 00:21 2024-03-26T08:21:33.021 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:21:33.021 INFO:teuthology.orchestra.run.smithi007.stdout:Total 3.5 MB/s | 76 MB 00:21 2024-03-26T08:21:33.720 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-26T08:21:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-26T08:21:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-26T08:21:35.181 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-26T08:21:35.181 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-26T08:21:37.788 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-26T08:21:38.457 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-432.el9.x86_64 1/4 2024-03-26T08:21:38.540 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T08:21:41.424 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T08:21:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T08:21:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T08:21:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T08:21:51.793 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T08:22:22.704 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T08:22:22.704 INFO:teuthology.orchestra.run.smithi007.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T08:22:22.704 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:22:22.750 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-432.el9.x86_64 4/4 2024-03-26T08:22:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-432.el9.x86_64 4/4 2024-03-26T08:22:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-432.el9.x86_64 1/4 2024-03-26T08:22:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-432.el9.x86_64 2/4 2024-03-26T08:22:23.107 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-432.el9.x86_64 3/4 2024-03-26T08:22:23.515 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-432.el9.x86_64 4/4 2024-03-26T08:22:23.515 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:22:23.515 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-26T08:22:23.515 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-432.el9.x86_64 2024-03-26T08:22:23.516 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-432.el9.x86_64 2024-03-26T08:22:23.516 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-432.el9.x86_64 2024-03-26T08:22:23.516 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-432.el9.x86_64 2024-03-26T08:22:23.516 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:22:23.516 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-26T08:22:23.868 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T08:22:23.974 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T08:22:23.974 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T08:22:23.974 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-432.el9.x86_64 2024-03-26T08:22:23.974 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-26T08:22:23.974 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-26T08:22:23.975 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-26T08:22:23.975 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-03-26T08:22:23.975 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-03-26T08:22:24.058 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-432.el9.x86_64 2024-03-26T08:22:24.058 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-432.el9.x86_64 2024-03-26T08:22:24.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-03-26T08:22:24.102 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-03-26T08:22:24.102 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-03-26T08:22:24.102 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-03-26T08:22:24.102 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 68.el9 2024-03-26T08:22:24.102 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-03-26T08:22:24.102 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T08:22:24.102 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-03-26T08:22:24.102 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8264254 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T08:22:24.103 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T08:22:24.104 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T08:22:24.104 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-03-26T08:22:24.104 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:22:24.104 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T08:22:24.105 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T08:22:24.105 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-432.el9.x86_64 2024-03-26T08:22:24.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T08:22:24.678 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-03-26T08:22:26.106 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T08:22:26.137 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-03-26T08:22:26.139 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-03-26T08:22:26.154 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.opjx1bhwkh 2024-03-26T08:22:26.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.opjx1bhwkh 2024-03-26T08:22:26.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.opjx1bhwkh 2024-03-26T08:22:26.369 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.opjx1bhwkh is 6KB 2024-03-26T08:22:26.418 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.opjx1bhwkh 2024-03-26T08:22:26.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T08:22:26.501 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T08:22:26.501 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T08:22:26.501 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T08:22:26.501 INFO:teuthology.orchestra.run.smithi007.stdout:15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64.conf 2024-03-26T08:22:26.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-432.el9.x86_64 2024-03-26T08:22:26.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-03-26T08:22:56.669 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-26T08:22:56.669 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-26T08:22:56.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:23:15.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-03-26T08:23:24.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-26T08:23:24.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:23:27.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-03-26T08:23:39.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-26T08:23:39.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:24:11.038 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-03-26T08:24:11.723 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-26T08:24:11.723 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-432.el9.x86_64"... 2024-03-26T08:24:11.723 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-03-26T08:24:11.738 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-432.el9.x86_64 2024-03-26T08:24:11.738 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-432.el9.x86_64 vs 5.14.0-432.el9.x86_64 2024-03-26T08:24:11.738 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T08:24:11.739 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T08:24:12.740 DEBUG:teuthology.parallel:result is None 2024-03-26T08:24:12.740 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-26T08:24:12.748 INFO:teuthology.task.internal:Creating test directory... 2024-03-26T08:24:12.748 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T08:24:12.751 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T08:24:12.753 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T08:24:12.772 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-26T08:24:12.779 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-26T08:24:12.785 INFO:teuthology.task.internal:Creating archive directory... 2024-03-26T08:24:12.785 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T08:24:12.810 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T08:24:12.813 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T08:24:12.851 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-26T08:24:12.857 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-26T08:24:12.857 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T08:24:12.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:24:12.892 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T08:24:12.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:24:12.909 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T08:24:12.911 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T08:24:12.935 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T08:24:12.966 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:24:12.987 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:24:12.993 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:24:13.012 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:24:13.017 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:24:13.019 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T08:24:13.021 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-26T08:24:13.028 INFO:teuthology.task.internal:Configuring sudo... 2024-03-26T08:24:13.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T08:24:13.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T08:24:13.032 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T08:24:13.095 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-26T08:24:13.104 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-26T08:24:13.105 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T08:24:13.107 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T08:24:13.109 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T08:24:13.151 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:24:13.203 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:24:13.263 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:24:13.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:24:13.392 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:24:13.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T08:24:13.462 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:24:13.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:24:13.568 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:24:13.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:24:13.659 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:24:13.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T08:24:13.733 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:24:13.774 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T08:24:13.842 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:24:13.881 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T08:24:13.933 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:24:13.934 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T08:24:14.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-03-26T08:24:14.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-03-26T08:24:14.011 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-03-26T08:24:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T08:24:14.050 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T08:24:14.097 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T08:24:14.610 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-26T08:24:14.619 INFO:teuthology.task.internal:Starting timer... 2024-03-26T08:24:14.619 INFO:teuthology.run_tasks:Running task pcp... 2024-03-26T08:24:14.628 INFO:teuthology.run_tasks:Running task selinux... 2024-03-26T08:24:14.636 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-26T08:24:14.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-03-26T08:24:14.718 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-03-26T08:24:14.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-03-26T08:24:14.775 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-03-26T08:24:14.777 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-03-26T08:24:14.843 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-03-26T08:24:14.844 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-26T08:24:14.845 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-03-26T08:24:14.863 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-03-26T08:24:14.863 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-03-26T08:24:14.887 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-03-26T08:24:14.887 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-03-26T08:24:14.912 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-03-26T08:24:14.912 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi044.front.sepia.ceph.com': 'permissive'} 2024-03-26T08:24:14.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T08:24:14.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:24:14.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T08:24:14.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:24:14.972 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T08:24:15.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:24:15.003 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-26T08:24:15.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-03-26T08:24:15.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-03-26T08:24:15.089 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-03-26T08:24:15.130 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-26T08:24:15.140 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-26T08:24:15.146 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-26T08:24:15.295 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-26T08:24:15.310 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-26T08:24:15.312 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi016.front.sepia.ceph.com,smithi044.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-26T08:29:05.656 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi044.front.sepia.ceph.com')] 2024-03-26T08:29:05.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-26T08:29:05.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:29:05.752 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-03-26T08:29:05.833 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-03-26T08:29:05.833 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-03-26T08:29:05.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:29:05.938 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-03-26T08:29:06.007 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-03-26T08:29:06.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-03-26T08:29:06.009 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T08:29:06.115 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-03-26T08:29:06.187 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-03-26T08:29:06.187 INFO:teuthology.run_tasks:Running task clock... 2024-03-26T08:29:06.198 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-26T08:29:06.199 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T08:29:06.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T08:29:06.202 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T08:29:06.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T08:29:06.204 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T08:29:06.204 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T08:29:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T08:29:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T08:29:06.251 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T08:29:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T08:29:06.266 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T08:29:06.282 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-03-26T08:29:06.287 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T08:29:06.295 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-03-26T08:29:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-03-26T08:29:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T08:29:06.320 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-03-26T08:29:06.325 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-03-26T08:29:06.329 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T08:29:06.340 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T08:29:06.342 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-03-26T08:29:06.360 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T08:29:06.363 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T08:29:06.383 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-03-26T08:29:06.383 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T08:29:06.385 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T08:29:06.386 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-03-26T08:29:06.386 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.386 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.386 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.386 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-03-26T08:29:06.412 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T08:29:06.412 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-03-26T08:29:06.412 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.413 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.413 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.413 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.431 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-03-26T08:29:06.435 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T08:29:06.435 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-03-26T08:29:06.435 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.435 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.435 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.435 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T08:29:06.435 INFO:teuthology.run_tasks:Running task install... 2024-03-26T08:29:06.451 DEBUG:teuthology.task.install:project ceph 2024-03-26T08:29:06.451 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-26T08:29:06.451 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-26T08:29:06.451 INFO:teuthology.task.install:Using flavor: default 2024-03-26T08:29:06.460 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-26T08:29:06.460 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-26T08:29:06.460 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-26T08:29:06.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:29:06.463 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-26T08:29:06.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:29:06.464 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-26T08:29:06.464 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:29:06.606 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-26T08:29:06.607 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-26T08:29:06.682 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-26T08:29:06.683 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-26T08:29:06.715 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-26T08:29:06.716 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-26T08:29:06.779 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T08:29:06.779 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:29:06.779 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T08:29:06.795 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T08:29:06.795 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:29:06.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T08:29:06.812 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T08:29:06.812 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T08:29:06.824 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T08:29:06.825 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T08:29:06.827 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T08:29:06.827 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:29:06.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T08:29:06.860 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T08:29:06.861 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T08:29:06.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T08:29:06.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T08:29:06.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T08:29:06.972 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T08:29:06.973 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T08:29:07.003 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-03-26T08:29:07.005 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-03-26T08:29:07.021 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T08:29:07.040 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-03-26T08:29:07.042 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-03-26T08:29:07.051 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-03-26T08:29:07.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-03-26T08:29:07.252 INFO:teuthology.orchestra.run.smithi044.stdout:74 files removed 2024-03-26T08:29:07.280 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T08:29:07.300 INFO:teuthology.orchestra.run.smithi016.stdout:74 files removed 2024-03-26T08:29:07.301 INFO:teuthology.orchestra.run.smithi007.stdout:74 files removed 2024-03-26T08:29:07.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T08:29:07.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T08:29:08.032 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 188 kB/s | 83 kB 00:00 2024-03-26T08:29:08.071 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 189 kB/s | 83 kB 00:00 2024-03-26T08:29:08.093 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 181 kB/s | 83 kB 00:00 2024-03-26T08:29:08.451 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-03-26T08:29:08.472 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-03-26T08:29:08.473 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-26T08:29:08.824 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-03-26T08:29:08.855 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-03-26T08:29:08.860 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-26T08:29:09.293 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-26T08:29:09.296 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-26T08:29:09.591 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - AppStream 9.6 MB/s | 6.7 MB 00:00 2024-03-26T08:29:10.776 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-26T08:29:10.888 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.0 MB 00:00 2024-03-26T08:29:11.058 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-26T08:29:11.626 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-03-26T08:29:11.870 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-26T08:29:11.882 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-26T08:29:13.783 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-03-26T08:29:16.905 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 19 MB 00:03 2024-03-26T08:29:22.055 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.1 MB 00:01 2024-03-26T08:29:23.191 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 1.1 MB/s | 6.1 MB 00:05 2024-03-26T08:29:23.656 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-03-26T08:29:24.284 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 1.6 MB/s | 19 MB 00:11 2024-03-26T08:29:24.600 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 16 kB 00:00 2024-03-26T08:29:25.336 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 33 MB/s | 21 MB 00:00 2024-03-26T08:29:27.535 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 21 MB 00:03 2024-03-26T08:29:28.560 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.1 MB 00:00 2024-03-26T08:29:29.936 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 16 kB 00:00 2024-03-26T08:29:30.561 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-26T08:29:31.998 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-26T08:29:32.724 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-03-26T08:29:33.078 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:29:33.079 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:29:33.095 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T08:29:33.096 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T08:29:33.097 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T08:29:33.099 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T08:29:33.101 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T08:29:33.104 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T08:29:33.105 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T08:29:33.191 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-03-26T08:29:33.208 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================================ 2024-03-26T08:29:33.208 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-03-26T08:29:33.208 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================================ 2024-03-26T08:29:33.208 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-03-26T08:29:33.208 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T08:29:33.208 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-26T08:29:33.208 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-26T08:29:33.208 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-26T08:29:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-26T08:29:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-26T08:29:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-26T08:29:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-26T08:29:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-26T08:29:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-26T08:29:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-26T08:29:33.209 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-26T08:29:33.210 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-26T08:29:33.210 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-26T08:29:33.210 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-26T08:29:33.210 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-26T08:29:33.210 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-26T08:29:33.210 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T08:29:33.210 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T08:29:33.210 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T08:29:33.210 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-26T08:29:33.211 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T08:29:33.212 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T08:29:33.213 INFO:teuthology.orchestra.run.smithi007.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T08:29:33.213 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T08:29:33.213 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T08:29:33.213 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T08:29:33.213 INFO:teuthology.orchestra.run.smithi007.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T08:29:33.213 INFO:teuthology.orchestra.run.smithi007.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T08:29:33.213 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T08:29:33.213 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T08:29:33.214 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T08:29:33.215 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-26T08:29:33.216 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T08:29:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T08:29:33.218 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T08:29:33.219 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T08:29:33.220 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T08:29:33.221 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T08:29:33.222 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================================ 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout:Install 150 Packages 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 24 Packages 2024-03-26T08:29:33.223 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:29:33.224 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 225 M 2024-03-26T08:29:33.224 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-03-26T08:29:33.803 INFO:teuthology.orchestra.run.smithi007.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-03-26T08:29:34.029 INFO:teuthology.orchestra.run.smithi007.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 3.9 MB/s | 890 kB 00:00 2024-03-26T08:29:34.079 INFO:teuthology.orchestra.run.smithi007.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 2.8 MB/s | 145 kB 00:00 2024-03-26T08:29:34.313 INFO:teuthology.orchestra.run.smithi007.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 9.9 MB/s | 2.3 MB 00:00 2024-03-26T08:29:34.463 INFO:teuthology.orchestra.run.smithi007.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 10 MB/s | 1.5 MB 00:00 2024-03-26T08:29:34.805 INFO:teuthology.orchestra.run.smithi007.stdout:(6/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 4.6 MB/s | 5.4 MB 00:01 2024-03-26T08:29:34.913 INFO:teuthology.orchestra.run.smithi007.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 10 MB/s | 4.7 MB 00:00 2024-03-26T08:29:35.236 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:29:35.237 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:29:35.252 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T08:29:35.253 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T08:29:35.254 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T08:29:35.255 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T08:29:35.257 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T08:29:35.260 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T08:29:35.261 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T08:29:35.347 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-03-26T08:29:35.363 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-26T08:29:35.364 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-26T08:29:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T08:29:35.366 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T08:29:35.367 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T08:29:35.368 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T08:29:35.369 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T08:29:35.370 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T08:29:35.371 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T08:29:35.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T08:29:35.373 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T08:29:35.374 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T08:29:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:29:35.376 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-03-26T08:29:35.377 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2024-03-26T08:29:35.377 INFO:teuthology.orchestra.run.smithi016.stdout:Install 150 Packages 2024-03-26T08:29:35.377 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 24 Packages 2024-03-26T08:29:35.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:29:35.377 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 225 M 2024-03-26T08:29:35.377 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-03-26T08:29:35.830 INFO:teuthology.orchestra.run.smithi007.stdout:(8/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el9 11 MB/s | 9.9 MB 00:00 2024-03-26T08:29:35.905 INFO:teuthology.orchestra.run.smithi007.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 336 kB/s | 25 kB 00:00 2024-03-26T08:29:36.334 INFO:teuthology.orchestra.run.smithi016.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-03-26T08:29:36.598 INFO:teuthology.orchestra.run.smithi007.stdout:(10/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 6.9 MB/s | 20 MB 00:02 2024-03-26T08:29:36.701 INFO:teuthology.orchestra.run.smithi016.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 2.4 MB/s | 890 kB 00:00 2024-03-26T08:29:36.767 INFO:teuthology.orchestra.run.smithi016.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 2.1 MB/s | 145 kB 00:00 2024-03-26T08:29:36.773 INFO:teuthology.orchestra.run.smithi007.stdout:(11/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 8.6 MB/s | 17 MB 00:01 2024-03-26T08:29:36.798 INFO:teuthology.orchestra.run.smithi007.stdout:(12/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 1.1 MB/s | 220 kB 00:00 2024-03-26T08:29:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:(13/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 361 kB/s | 27 kB 00:00 2024-03-26T08:29:36.865 INFO:teuthology.orchestra.run.smithi007.stdout:(14/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 466 kB/s | 31 kB 00:00 2024-03-26T08:29:36.965 INFO:teuthology.orchestra.run.smithi007.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.6 MB/s | 163 kB 00:00 2024-03-26T08:29:37.024 INFO:teuthology.orchestra.run.smithi007.stdout:(16/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 4.2 MB/s | 746 kB 00:00 2024-03-26T08:29:37.066 INFO:teuthology.orchestra.run.smithi007.stdout:(17/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.2 MB/s | 127 kB 00:00 2024-03-26T08:29:37.141 INFO:teuthology.orchestra.run.smithi007.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 4.2 MB/s | 501 kB 00:00 2024-03-26T08:29:37.225 INFO:teuthology.orchestra.run.smithi007.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 541 kB/s | 45 kB 00:00 2024-03-26T08:29:37.257 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-26T08:29:37.325 INFO:teuthology.orchestra.run.smithi007.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.3 MB/s | 131 kB 00:00 2024-03-26T08:29:37.393 INFO:teuthology.orchestra.run.smithi016.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 3.7 MB/s | 2.3 MB 00:00 2024-03-26T08:29:37.417 INFO:teuthology.orchestra.run.smithi007.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.7 MB/s | 162 kB 00:00 2024-03-26T08:29:37.479 INFO:teuthology.orchestra.run.smithi016.stdout:(5/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 3.5 MB/s | 5.4 MB 00:01 2024-03-26T08:29:37.509 INFO:teuthology.orchestra.run.smithi007.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 3.4 MB/s | 321 kB 00:00 2024-03-26T08:29:37.617 INFO:teuthology.orchestra.run.smithi007.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 2.7 MB/s | 298 kB 00:00 2024-03-26T08:29:37.718 INFO:teuthology.orchestra.run.smithi007.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 996 kB/s | 99 kB 00:00 2024-03-26T08:29:37.826 INFO:teuthology.orchestra.run.smithi007.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 790 kB/s | 85 kB 00:00 2024-03-26T08:29:37.835 INFO:teuthology.orchestra.run.smithi016.stdout:(6/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 3.4 MB/s | 1.5 MB 00:00 2024-03-26T08:29:37.910 INFO:teuthology.orchestra.run.smithi007.stdout:(26/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 6.1 MB/s | 5.1 MB 00:00 2024-03-26T08:29:38.019 INFO:teuthology.orchestra.run.smithi007.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.5 MB/s | 170 kB 00:00 2024-03-26T08:29:38.111 INFO:teuthology.orchestra.run.smithi007.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 370 kB/s | 34 kB 00:00 2024-03-26T08:29:38.177 INFO:teuthology.orchestra.run.smithi016.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 6.7 MB/s | 4.7 MB 00:00 2024-03-26T08:29:38.194 INFO:teuthology.orchestra.run.smithi007.stdout:(29/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 8.4 MB/s | 3.1 MB 00:00 2024-03-26T08:29:38.211 INFO:teuthology.orchestra.run.smithi007.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.4 MB/s | 139 kB 00:00 2024-03-26T08:29:38.461 INFO:teuthology.orchestra.run.smithi007.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 6.7 MB/s | 1.8 MB 00:00 2024-03-26T08:29:38.562 INFO:teuthology.orchestra.run.smithi007.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 2.4 MB/s | 248 kB 00:00 2024-03-26T08:29:38.645 INFO:teuthology.orchestra.run.smithi007.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 592 kB/s | 49 kB 00:00 2024-03-26T08:29:38.737 INFO:teuthology.orchestra.run.smithi007.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 180 kB/s | 16 kB 00:00 2024-03-26T08:29:38.846 INFO:teuthology.orchestra.run.smithi007.stdout:(35/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 2.4 MB/s | 264 kB 00:00 2024-03-26T08:29:38.963 INFO:teuthology.orchestra.run.smithi007.stdout:(36/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 6.3 MB/s | 748 kB 00:00 2024-03-26T08:29:39.046 INFO:teuthology.orchestra.run.smithi007.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 219 kB/s | 18 kB 00:00 2024-03-26T08:29:39.180 INFO:teuthology.orchestra.run.smithi007.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 662 kB 00:00 2024-03-26T08:29:39.239 INFO:teuthology.orchestra.run.smithi007.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-03-26T08:29:39.322 INFO:teuthology.orchestra.run.smithi007.stdout:(40/174): ceph-mgr-diskprediction-local-19.0.0- 6.7 MB/s | 7.4 MB 00:01 2024-03-26T08:29:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 976 kB/s | 106 kB 00:00 2024-03-26T08:29:39.373 INFO:teuthology.orchestra.run.smithi007.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-26T08:29:39.398 INFO:teuthology.orchestra.run.smithi007.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 634 kB/s | 32 kB 00:00 2024-03-26T08:29:39.473 INFO:teuthology.orchestra.run.smithi007.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-03-26T08:29:39.499 INFO:teuthology.orchestra.run.smithi007.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 175 kB/s | 17 kB 00:00 2024-03-26T08:29:39.524 INFO:teuthology.orchestra.run.smithi007.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-03-26T08:29:39.549 INFO:teuthology.orchestra.run.smithi007.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-26T08:29:39.574 INFO:teuthology.orchestra.run.smithi007.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-03-26T08:29:39.600 INFO:teuthology.orchestra.run.smithi007.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-26T08:29:39.625 INFO:teuthology.orchestra.run.smithi007.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-26T08:29:39.642 INFO:teuthology.orchestra.run.smithi007.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-26T08:29:39.667 INFO:teuthology.orchestra.run.smithi007.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-26T08:29:39.693 INFO:teuthology.orchestra.run.smithi007.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T08:29:39.718 INFO:teuthology.orchestra.run.smithi007.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-26T08:29:39.735 INFO:teuthology.orchestra.run.smithi007.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 821 kB/s | 34 kB 00:00 2024-03-26T08:29:39.760 INFO:teuthology.orchestra.run.smithi016.stdout:(8/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el9 6.2 MB/s | 9.9 MB 00:01 2024-03-26T08:29:39.761 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:29:39.761 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T08:29:39.776 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T08:29:39.777 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T08:29:39.778 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T08:29:39.779 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T08:29:39.781 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T08:29:39.784 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T08:29:39.785 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T08:29:39.827 INFO:teuthology.orchestra.run.smithi016.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 378 kB/s | 25 kB 00:00 2024-03-26T08:29:39.843 INFO:teuthology.orchestra.run.smithi007.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-03-26T08:29:39.871 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-03-26T08:29:39.877 INFO:teuthology.orchestra.run.smithi007.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 7.2 MB/s | 1.0 MB 00:00 2024-03-26T08:29:39.893 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================ 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================ 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-26T08:29:39.894 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-26T08:29:39.895 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-26T08:29:39.896 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T08:29:39.897 INFO:teuthology.orchestra.run.smithi044.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T08:29:39.898 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T08:29:39.899 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-26T08:29:39.900 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T08:29:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-26T08:29:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T08:29:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T08:29:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T08:29:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T08:29:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T08:29:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T08:29:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T08:29:39.901 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T08:29:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T08:29:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T08:29:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T08:29:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T08:29:39.902 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T08:29:39.903 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T08:29:39.904 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T08:29:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T08:29:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T08:29:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T08:29:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T08:29:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T08:29:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T08:29:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T08:29:39.905 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T08:29:39.906 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T08:29:39.907 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout:============================================================================================ 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout:Install 150 Packages 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 24 Packages 2024-03-26T08:29:39.908 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:29:39.909 INFO:teuthology.orchestra.run.smithi007.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T08:29:39.909 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 225 M 2024-03-26T08:29:39.910 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-03-26T08:29:40.036 INFO:teuthology.orchestra.run.smithi007.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 5.8 MB 00:00 2024-03-26T08:29:40.053 INFO:teuthology.orchestra.run.smithi007.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-03-26T08:29:40.070 INFO:teuthology.orchestra.run.smithi007.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 539 kB/s | 18 kB 00:00 2024-03-26T08:29:40.095 INFO:teuthology.orchestra.run.smithi007.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 144 kB 00:00 2024-03-26T08:29:40.112 INFO:teuthology.orchestra.run.smithi007.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 761 kB/s | 32 kB 00:00 2024-03-26T08:29:40.288 INFO:teuthology.orchestra.run.smithi007.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-03-26T08:29:40.354 INFO:teuthology.orchestra.run.smithi007.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 900 kB/s | 217 kB 00:00 2024-03-26T08:29:40.380 INFO:teuthology.orchestra.run.smithi007.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 174 kB 00:00 2024-03-26T08:29:40.396 INFO:teuthology.orchestra.run.smithi007.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 69 kB 00:00 2024-03-26T08:29:40.422 INFO:teuthology.orchestra.run.smithi007.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 982 kB/s | 41 kB 00:00 2024-03-26T08:29:40.449 INFO:teuthology.orchestra.run.smithi007.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-26T08:29:40.472 INFO:teuthology.orchestra.run.smithi007.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-26T08:29:40.498 INFO:teuthology.orchestra.run.smithi007.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-26T08:29:40.523 INFO:teuthology.orchestra.run.smithi007.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-03-26T08:29:40.548 INFO:teuthology.orchestra.run.smithi007.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 853 kB/s | 43 kB 00:00 2024-03-26T08:29:40.715 INFO:teuthology.orchestra.run.smithi007.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 234 kB/s | 39 kB 00:00 2024-03-26T08:29:40.739 INFO:teuthology.orchestra.run.smithi044.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-03-26T08:29:40.819 INFO:teuthology.orchestra.run.smithi016.stdout:(10/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 4.2 MB/s | 20 MB 00:04 2024-03-26T08:29:40.894 INFO:teuthology.orchestra.run.smithi016.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.9 MB/s | 220 kB 00:00 2024-03-26T08:29:40.907 INFO:teuthology.orchestra.run.smithi007.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-26T08:29:40.932 INFO:teuthology.orchestra.run.smithi007.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 191 kB/s | 41 kB 00:00 2024-03-26T08:29:40.958 INFO:teuthology.orchestra.run.smithi007.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-26T08:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 405 kB/s | 27 kB 00:00 2024-03-26T08:29:40.983 INFO:teuthology.orchestra.run.smithi007.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-26T08:29:41.020 INFO:teuthology.orchestra.run.smithi007.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2024-03-26T08:29:41.036 INFO:teuthology.orchestra.run.smithi016.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 418 kB/s | 31 kB 00:00 2024-03-26T08:29:41.042 INFO:teuthology.orchestra.run.smithi007.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 754 kB/s | 44 kB 00:00 2024-03-26T08:29:41.064 INFO:teuthology.orchestra.run.smithi044.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 2.7 MB/s | 890 kB 00:00 2024-03-26T08:29:41.067 INFO:teuthology.orchestra.run.smithi007.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-26T08:29:41.092 INFO:teuthology.orchestra.run.smithi007.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-26T08:29:41.135 INFO:teuthology.orchestra.run.smithi007.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-03-26T08:29:41.139 INFO:teuthology.orchestra.run.smithi044.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 1.9 MB/s | 145 kB 00:00 2024-03-26T08:29:41.160 INFO:teuthology.orchestra.run.smithi007.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-26T08:29:41.185 INFO:teuthology.orchestra.run.smithi007.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-26T08:29:41.210 INFO:teuthology.orchestra.run.smithi007.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2024-03-26T08:29:41.245 INFO:teuthology.orchestra.run.smithi016.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 3.5 MB/s | 746 kB 00:00 2024-03-26T08:29:41.277 INFO:teuthology.orchestra.run.smithi007.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-03-26T08:29:41.336 INFO:teuthology.orchestra.run.smithi007.stdout:(88/174): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-03-26T08:29:41.337 INFO:teuthology.orchestra.run.smithi016.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.7 MB/s | 163 kB 00:00 2024-03-26T08:29:41.394 INFO:teuthology.orchestra.run.smithi007.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-03-26T08:29:41.420 INFO:teuthology.orchestra.run.smithi007.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-03-26T08:29:41.437 INFO:teuthology.orchestra.run.smithi016.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.2 MB/s | 127 kB 00:00 2024-03-26T08:29:41.520 INFO:teuthology.orchestra.run.smithi007.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 171 kB/s | 21 kB 00:00 2024-03-26T08:29:41.546 INFO:teuthology.orchestra.run.smithi007.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 203 kB/s | 25 kB 00:00 2024-03-26T08:29:41.621 INFO:teuthology.orchestra.run.smithi016.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 2.7 MB/s | 501 kB 00:00 2024-03-26T08:29:41.731 INFO:teuthology.orchestra.run.smithi044.stdout:(4/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 3.9 MB/s | 2.3 MB 00:00 2024-03-26T08:29:41.856 INFO:teuthology.orchestra.run.smithi044.stdout:(5/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 4.0 MB/s | 5.4 MB 00:01 2024-03-26T08:29:41.962 INFO:teuthology.orchestra.run.smithi007.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 3.8 MB/s | 1.6 MB 00:00 2024-03-26T08:29:42.123 INFO:teuthology.orchestra.run.smithi044.stdout:(6/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-03-26T08:29:42.188 INFO:teuthology.orchestra.run.smithi007.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 4.0 MB/s | 911 kB 00:00 2024-03-26T08:29:42.615 INFO:teuthology.orchestra.run.smithi044.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 6.2 MB/s | 4.7 MB 00:00 2024-03-26T08:29:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:(95/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 7.1 MB/s | 48 MB 00:06 2024-03-26T08:29:42.663 INFO:teuthology.orchestra.run.smithi007.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 280 kB/s | 133 kB 00:00 2024-03-26T08:29:42.847 INFO:teuthology.orchestra.run.smithi007.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-03-26T08:29:42.972 INFO:teuthology.orchestra.run.smithi007.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-03-26T08:29:43.122 INFO:teuthology.orchestra.run.smithi007.stdout:(99/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 2.6 MB/s | 4.1 MB 00:01 2024-03-26T08:29:43.148 INFO:teuthology.orchestra.run.smithi007.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-03-26T08:29:43.198 INFO:teuthology.orchestra.run.smithi007.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 477 kB/s | 36 kB 00:00 2024-03-26T08:29:43.223 INFO:teuthology.orchestra.run.smithi007.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-03-26T08:29:43.257 INFO:teuthology.orchestra.run.smithi007.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-26T08:29:43.263 INFO:teuthology.orchestra.run.smithi016.stdout:(18/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 3.1 MB/s | 5.1 MB 00:01 2024-03-26T08:29:43.315 INFO:teuthology.orchestra.run.smithi007.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-03-26T08:29:43.363 INFO:teuthology.orchestra.run.smithi016.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 453 kB/s | 45 kB 00:00 2024-03-26T08:29:43.457 INFO:teuthology.orchestra.run.smithi007.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-03-26T08:29:43.480 INFO:teuthology.orchestra.run.smithi016.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.1 MB/s | 131 kB 00:00 2024-03-26T08:29:43.597 INFO:teuthology.orchestra.run.smithi016.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.4 MB/s | 162 kB 00:00 2024-03-26T08:29:43.641 INFO:teuthology.orchestra.run.smithi007.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-03-26T08:29:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-26T08:29:43.716 INFO:teuthology.orchestra.run.smithi007.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-03-26T08:29:43.789 INFO:teuthology.orchestra.run.smithi016.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 1.6 MB/s | 321 kB 00:00 2024-03-26T08:29:43.808 INFO:teuthology.orchestra.run.smithi007.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-03-26T08:29:43.875 INFO:teuthology.orchestra.run.smithi007.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 732 kB/s | 49 kB 00:00 2024-03-26T08:29:43.956 INFO:teuthology.orchestra.run.smithi016.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 1.8 MB/s | 298 kB 00:00 2024-03-26T08:29:43.959 INFO:teuthology.orchestra.run.smithi007.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 812 kB/s | 67 kB 00:00 2024-03-26T08:29:44.023 INFO:teuthology.orchestra.run.smithi016.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.5 MB/s | 99 kB 00:00 2024-03-26T08:29:44.082 INFO:teuthology.orchestra.run.smithi044.stdout:(8/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 5.7 MB/s | 20 MB 00:03 2024-03-26T08:29:44.098 INFO:teuthology.orchestra.run.smithi016.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-26T08:29:44.149 INFO:teuthology.orchestra.run.smithi044.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 379 kB/s | 25 kB 00:00 2024-03-26T08:29:44.159 INFO:teuthology.orchestra.run.smithi007.stdout:(112/174): postgresql-server-13.14-1.el9.x86_64 3.8 MB/s | 5.8 MB 00:01 2024-03-26T08:29:44.184 INFO:teuthology.orchestra.run.smithi007.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 673 kB/s | 151 kB 00:00 2024-03-26T08:29:44.258 INFO:teuthology.orchestra.run.smithi044.stdout:(10/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 6.0 MB/s | 9.9 MB 00:01 2024-03-26T08:29:44.367 INFO:teuthology.orchestra.run.smithi044.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.0 MB/s | 220 kB 00:00 2024-03-26T08:29:44.426 INFO:teuthology.orchestra.run.smithi044.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 463 kB/s | 27 kB 00:00 2024-03-26T08:29:44.492 INFO:teuthology.orchestra.run.smithi044.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 468 kB/s | 31 kB 00:00 2024-03-26T08:29:44.493 INFO:teuthology.orchestra.run.smithi007.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-03-26T08:29:44.507 INFO:teuthology.orchestra.run.smithi016.stdout:(26/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 2.5 MB/s | 17 MB 00:06 2024-03-26T08:29:44.568 INFO:teuthology.orchestra.run.smithi007.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.0 MB/s | 838 kB 00:00 2024-03-26T08:29:44.593 INFO:teuthology.orchestra.run.smithi007.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 436 kB/s | 43 kB 00:00 2024-03-26T08:29:44.669 INFO:teuthology.orchestra.run.smithi007.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 432 kB/s | 32 kB 00:00 2024-03-26T08:29:44.676 INFO:teuthology.orchestra.run.smithi044.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 4.0 MB/s | 746 kB 00:00 2024-03-26T08:29:44.690 INFO:teuthology.orchestra.run.smithi016.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 932 kB/s | 170 kB 00:00 2024-03-26T08:29:44.694 INFO:teuthology.orchestra.run.smithi007.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-03-26T08:29:44.768 INFO:teuthology.orchestra.run.smithi044.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.7 MB/s | 163 kB 00:00 2024-03-26T08:29:44.782 INFO:teuthology.orchestra.run.smithi016.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 370 kB/s | 34 kB 00:00 2024-03-26T08:29:44.860 INFO:teuthology.orchestra.run.smithi044.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.4 MB/s | 127 kB 00:00 2024-03-26T08:29:44.861 INFO:teuthology.orchestra.run.smithi007.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-03-26T08:29:44.883 INFO:teuthology.orchestra.run.smithi016.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.4 MB/s | 139 kB 00:00 2024-03-26T08:29:44.953 INFO:teuthology.orchestra.run.smithi007.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-03-26T08:29:45.028 INFO:teuthology.orchestra.run.smithi007.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-03-26T08:29:45.043 INFO:teuthology.orchestra.run.smithi044.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 2.7 MB/s | 501 kB 00:00 2024-03-26T08:29:45.129 INFO:teuthology.orchestra.run.smithi007.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-03-26T08:29:45.191 INFO:teuthology.orchestra.run.smithi016.stdout:(30/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 2.8 MB/s | 3.1 MB 00:01 2024-03-26T08:29:45.307 INFO:teuthology.orchestra.run.smithi007.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 729 kB/s | 203 kB 00:00 2024-03-26T08:29:45.421 INFO:teuthology.orchestra.run.smithi007.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-03-26T08:29:45.513 INFO:teuthology.orchestra.run.smithi007.stdout:(125/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 4.4 MB 00:01 2024-03-26T08:29:45.527 INFO:teuthology.orchestra.run.smithi044.stdout:(18/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 4.9 MB/s | 17 MB 00:03 2024-03-26T08:29:45.538 INFO:teuthology.orchestra.run.smithi007.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-03-26T08:29:45.613 INFO:teuthology.orchestra.run.smithi007.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 238 kB/s | 18 kB 00:00 2024-03-26T08:29:45.627 INFO:teuthology.orchestra.run.smithi044.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 452 kB/s | 45 kB 00:00 2024-03-26T08:29:45.638 INFO:teuthology.orchestra.run.smithi007.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 233 kB/s | 23 kB 00:00 2024-03-26T08:29:45.727 INFO:teuthology.orchestra.run.smithi044.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.3 MB/s | 131 kB 00:00 2024-03-26T08:29:45.730 INFO:teuthology.orchestra.run.smithi007.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 213 kB/s | 19 kB 00:00 2024-03-26T08:29:45.747 INFO:teuthology.orchestra.run.smithi007.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-03-26T08:29:45.833 INFO:teuthology.orchestra.run.smithi016.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 1.9 MB/s | 1.8 MB 00:00 2024-03-26T08:29:45.836 INFO:teuthology.orchestra.run.smithi044.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.5 MB/s | 162 kB 00:00 2024-03-26T08:29:45.839 INFO:teuthology.orchestra.run.smithi007.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 509 kB/s | 46 kB 00:00 2024-03-26T08:29:45.939 INFO:teuthology.orchestra.run.smithi007.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 791 kB/s | 79 kB 00:00 2024-03-26T08:29:45.953 INFO:teuthology.orchestra.run.smithi044.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 2.7 MB/s | 321 kB 00:00 2024-03-26T08:29:45.981 INFO:teuthology.orchestra.run.smithi007.stdout:(133/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-03-26T08:29:46.009 INFO:teuthology.orchestra.run.smithi016.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 1.4 MB/s | 248 kB 00:00 2024-03-26T08:29:46.023 INFO:teuthology.orchestra.run.smithi007.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 694 kB/s | 58 kB 00:00 2024-03-26T08:29:46.074 INFO:teuthology.orchestra.run.smithi007.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-03-26T08:29:46.087 INFO:teuthology.orchestra.run.smithi044.stdout:(23/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 4.9 MB/s | 5.1 MB 00:01 2024-03-26T08:29:46.109 INFO:teuthology.orchestra.run.smithi016.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 494 kB/s | 49 kB 00:00 2024-03-26T08:29:46.112 INFO:teuthology.orchestra.run.smithi044.stdout:(24/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 1.8 MB/s | 298 kB 00:00 2024-03-26T08:29:46.123 INFO:teuthology.orchestra.run.smithi007.stdout:(136/174): python3-kubernetes-26.1.0-2.el9.noar 2.7 MB/s | 1.0 MB 00:00 2024-03-26T08:29:46.149 INFO:teuthology.orchestra.run.smithi007.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-03-26T08:29:46.166 INFO:teuthology.orchestra.run.smithi007.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 981 kB/s | 90 kB 00:00 2024-03-26T08:29:46.170 INFO:teuthology.orchestra.run.smithi044.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.2 MB/s | 99 kB 00:00 2024-03-26T08:29:46.196 INFO:teuthology.orchestra.run.smithi044.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-26T08:29:46.199 INFO:teuthology.orchestra.run.smithi007.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-03-26T08:29:46.201 INFO:teuthology.orchestra.run.smithi016.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 179 kB/s | 16 kB 00:00 2024-03-26T08:29:46.216 INFO:teuthology.orchestra.run.smithi007.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 461 kB/s | 31 kB 00:00 2024-03-26T08:29:46.288 INFO:teuthology.orchestra.run.smithi044.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-26T08:29:46.325 INFO:teuthology.orchestra.run.smithi007.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-03-26T08:29:46.350 INFO:teuthology.orchestra.run.smithi007.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 394 kB/s | 59 kB 00:00 2024-03-26T08:29:46.375 INFO:teuthology.orchestra.run.smithi007.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 225 kB/s | 36 kB 00:00 2024-03-26T08:29:46.375 INFO:teuthology.orchestra.run.smithi016.stdout:(35/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 1.5 MB/s | 264 kB 00:00 2024-03-26T08:29:46.388 INFO:teuthology.orchestra.run.smithi044.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 340 kB/s | 34 kB 00:00 2024-03-26T08:29:46.417 INFO:teuthology.orchestra.run.smithi007.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-03-26T08:29:46.484 INFO:teuthology.orchestra.run.smithi007.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 828 kB/s | 90 kB 00:00 2024-03-26T08:29:46.497 INFO:teuthology.orchestra.run.smithi044.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.2 MB/s | 139 kB 00:00 2024-03-26T08:29:46.509 INFO:teuthology.orchestra.run.smithi007.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-26T08:29:46.551 INFO:teuthology.orchestra.run.smithi007.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 301 kB/s | 20 kB 00:00 2024-03-26T08:29:46.585 INFO:teuthology.orchestra.run.smithi007.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-03-26T08:29:46.727 INFO:teuthology.orchestra.run.smithi007.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 884 kB/s | 192 kB 00:00 2024-03-26T08:29:46.797 INFO:teuthology.orchestra.run.smithi044.stdout:(30/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 4.9 MB/s | 3.1 MB 00:00 2024-03-26T08:29:46.926 INFO:teuthology.orchestra.run.smithi016.stdout:(36/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 1.3 MB/s | 748 kB 00:00 2024-03-26T08:29:46.994 INFO:teuthology.orchestra.run.smithi007.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-03-26T08:29:47.036 INFO:teuthology.orchestra.run.smithi007.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-26T08:29:47.043 INFO:teuthology.orchestra.run.smithi016.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 156 kB/s | 18 kB 00:00 2024-03-26T08:29:47.164 INFO:teuthology.orchestra.run.smithi044.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 2.7 MB/s | 1.8 MB 00:00 2024-03-26T08:29:47.185 INFO:teuthology.orchestra.run.smithi016.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.6 MB/s | 662 kB 00:00 2024-03-26T08:29:47.211 INFO:teuthology.orchestra.run.smithi007.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-26T08:29:47.235 INFO:teuthology.orchestra.run.smithi016.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 920 kB 00:00 2024-03-26T08:29:47.269 INFO:teuthology.orchestra.run.smithi016.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-26T08:29:47.311 INFO:teuthology.orchestra.run.smithi016.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-03-26T08:29:47.336 INFO:teuthology.orchestra.run.smithi016.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 1.2 MB/s | 32 kB 00:00 2024-03-26T08:29:47.353 INFO:teuthology.orchestra.run.smithi007.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 13 MB/s | 1.8 MB 00:00 2024-03-26T08:29:47.430 INFO:teuthology.orchestra.run.smithi007.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 561 kB/s | 42 kB 00:00 2024-03-26T08:29:47.430 INFO:teuthology.orchestra.run.smithi016.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-03-26T08:29:47.453 INFO:teuthology.orchestra.run.smithi016.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 689 kB/s | 17 kB 00:00 2024-03-26T08:29:47.478 INFO:teuthology.orchestra.run.smithi016.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 401 kB/s | 10 kB 00:00 2024-03-26T08:29:47.512 INFO:teuthology.orchestra.run.smithi007.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.9 MB/s | 248 kB 00:00 2024-03-26T08:29:47.521 INFO:teuthology.orchestra.run.smithi016.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-26T08:29:47.559 INFO:teuthology.orchestra.run.smithi016.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-26T08:29:47.579 INFO:teuthology.orchestra.run.smithi007.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-03-26T08:29:47.601 INFO:teuthology.orchestra.run.smithi016.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-03-26T08:29:47.643 INFO:teuthology.orchestra.run.smithi016.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 578 kB 00:00 2024-03-26T08:29:47.662 INFO:teuthology.orchestra.run.smithi007.stdout:(157/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 3.4 MB/s | 3.1 MB 00:00 2024-03-26T08:29:47.677 INFO:teuthology.orchestra.run.smithi016.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-26T08:29:47.687 INFO:teuthology.orchestra.run.smithi007.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 336 kB/s | 36 kB 00:00 2024-03-26T08:29:47.746 INFO:teuthology.orchestra.run.smithi007.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.3 MB/s | 192 kB 00:00 2024-03-26T08:29:47.769 INFO:teuthology.orchestra.run.smithi016.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-03-26T08:29:47.772 INFO:teuthology.orchestra.run.smithi044.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 408 kB/s | 248 kB 00:00 2024-03-26T08:29:47.788 INFO:teuthology.orchestra.run.smithi007.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 256 kB 00:00 2024-03-26T08:29:47.813 INFO:teuthology.orchestra.run.smithi007.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 344 kB/s | 23 kB 00:00 2024-03-26T08:29:47.869 INFO:teuthology.orchestra.run.smithi016.stdout:(52/174): ceph-mgr-diskprediction-local-19.0.0- 2.8 MB/s | 7.4 MB 00:02 2024-03-26T08:29:47.895 INFO:teuthology.orchestra.run.smithi016.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2024-03-26T08:29:47.906 INFO:teuthology.orchestra.run.smithi044.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 370 kB/s | 49 kB 00:00 2024-03-26T08:29:47.920 INFO:teuthology.orchestra.run.smithi016.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-26T08:29:47.937 INFO:teuthology.orchestra.run.smithi016.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 824 kB/s | 34 kB 00:00 2024-03-26T08:29:47.998 INFO:teuthology.orchestra.run.smithi044.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 180 kB/s | 16 kB 00:00 2024-03-26T08:29:48.055 INFO:teuthology.orchestra.run.smithi007.stdout:(162/174): librados2-19.0.0-2145.g3be9a5c9.el9. 2.3 MB/s | 3.4 MB 00:01 2024-03-26T08:29:48.062 INFO:teuthology.orchestra.run.smithi016.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-03-26T08:29:48.080 INFO:teuthology.orchestra.run.smithi007.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 371 kB/s | 108 kB 00:00 2024-03-26T08:29:48.096 INFO:teuthology.orchestra.run.smithi016.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-03-26T08:29:48.106 INFO:teuthology.orchestra.run.smithi007.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 530 kB/s | 155 kB 00:00 2024-03-26T08:29:48.121 INFO:teuthology.orchestra.run.smithi016.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T08:29:48.148 INFO:teuthology.orchestra.run.smithi007.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 136 kB 00:00 2024-03-26T08:29:48.173 INFO:teuthology.orchestra.run.smithi007.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 430 kB/s | 29 kB 00:00 2024-03-26T08:29:48.198 INFO:teuthology.orchestra.run.smithi007.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 544 kB/s | 64 kB 00:00 2024-03-26T08:29:48.215 INFO:teuthology.orchestra.run.smithi044.stdout:(35/174): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-03-26T08:29:48.255 INFO:teuthology.orchestra.run.smithi016.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 37 MB/s | 5.8 MB 00:00 2024-03-26T08:29:48.272 INFO:teuthology.orchestra.run.smithi016.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 166 kB/s | 24 kB 00:00 2024-03-26T08:29:48.288 INFO:teuthology.orchestra.run.smithi016.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 542 kB/s | 18 kB 00:00 2024-03-26T08:29:48.290 INFO:teuthology.orchestra.run.smithi007.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 8.5 MB/s | 1.2 MB 00:00 2024-03-26T08:29:48.314 INFO:teuthology.orchestra.run.smithi016.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-26T08:29:48.331 INFO:teuthology.orchestra.run.smithi016.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 759 kB/s | 32 kB 00:00 2024-03-26T08:29:48.332 INFO:teuthology.orchestra.run.smithi044.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 791 kB/s | 264 kB 00:00 2024-03-26T08:29:48.374 INFO:teuthology.orchestra.run.smithi007.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-03-26T08:29:48.416 INFO:teuthology.orchestra.run.smithi044.stdout:(37/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 3.7 MB/s | 748 kB 00:00 2024-03-26T08:29:48.432 INFO:teuthology.orchestra.run.smithi007.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 9.8 MB/s | 2.3 MB 00:00 2024-03-26T08:29:48.433 INFO:teuthology.orchestra.run.smithi044.stdout:(38/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 182 kB/s | 18 kB 00:00 2024-03-26T08:29:48.449 INFO:teuthology.orchestra.run.smithi007.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 414 kB/s | 31 kB 00:00 2024-03-26T08:29:48.464 INFO:teuthology.orchestra.run.smithi016.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-26T08:29:48.474 INFO:teuthology.orchestra.run.smithi007.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 2.5 MB/s | 465 kB 00:00 2024-03-26T08:29:48.490 INFO:teuthology.orchestra.run.smithi016.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-26T08:29:48.515 INFO:teuthology.orchestra.run.smithi016.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-26T08:29:48.540 INFO:teuthology.orchestra.run.smithi016.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-26T08:29:48.557 INFO:teuthology.orchestra.run.smithi016.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 984 kB/s | 41 kB 00:00 2024-03-26T08:29:48.558 INFO:teuthology.orchestra.run.smithi007.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-03-26T08:29:48.583 INFO:teuthology.orchestra.run.smithi016.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 5.5 MB/s | 238 kB 00:00 2024-03-26T08:29:48.583 INFO:teuthology.orchestra.run.smithi044.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 662 kB 00:00 2024-03-26T08:29:48.608 INFO:teuthology.orchestra.run.smithi016.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-26T08:29:48.617 INFO:teuthology.orchestra.run.smithi044.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.9 MB/s | 920 kB 00:00 2024-03-26T08:29:48.633 INFO:teuthology.orchestra.run.smithi016.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-26T08:29:48.633 INFO:teuthology.orchestra.run.smithi007.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 405 kB/s | 74 kB 00:00 2024-03-26T08:29:48.638 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:29:48.638 INFO:teuthology.orchestra.run.smithi007.stdout:Total 15 MB/s | 225 MB 00:15 2024-03-26T08:29:48.642 INFO:teuthology.orchestra.run.smithi044.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-03-26T08:29:48.658 INFO:teuthology.orchestra.run.smithi016.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 945 kB/s | 47 kB 00:00 2024-03-26T08:29:48.667 INFO:teuthology.orchestra.run.smithi044.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-26T08:29:48.684 INFO:teuthology.orchestra.run.smithi016.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 848 kB/s | 43 kB 00:00 2024-03-26T08:29:48.684 INFO:teuthology.orchestra.run.smithi044.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 767 kB/s | 32 kB 00:00 2024-03-26T08:29:48.818 INFO:teuthology.orchestra.run.smithi044.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-03-26T08:29:48.835 INFO:teuthology.orchestra.run.smithi044.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 114 kB/s | 17 kB 00:00 2024-03-26T08:29:48.852 INFO:teuthology.orchestra.run.smithi044.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 300 kB/s | 10 kB 00:00 2024-03-26T08:29:48.867 INFO:teuthology.orchestra.run.smithi016.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 213 kB/s | 39 kB 00:00 2024-03-26T08:29:48.877 INFO:teuthology.orchestra.run.smithi044.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-03-26T08:29:48.894 INFO:teuthology.orchestra.run.smithi044.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-26T08:29:48.953 INFO:teuthology.orchestra.run.smithi044.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-26T08:29:48.978 INFO:teuthology.orchestra.run.smithi044.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-26T08:29:48.995 INFO:teuthology.orchestra.run.smithi044.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-26T08:29:49.020 INFO:teuthology.orchestra.run.smithi044.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-26T08:29:49.046 INFO:teuthology.orchestra.run.smithi044.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T08:29:49.071 INFO:teuthology.orchestra.run.smithi044.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-03-26T08:29:49.088 INFO:teuthology.orchestra.run.smithi044.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 827 kB/s | 34 kB 00:00 2024-03-26T08:29:49.109 INFO:teuthology.orchestra.run.smithi016.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-26T08:29:49.135 INFO:teuthology.orchestra.run.smithi016.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 155 kB/s | 41 kB 00:00 2024-03-26T08:29:49.168 INFO:teuthology.orchestra.run.smithi016.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-26T08:29:49.193 INFO:teuthology.orchestra.run.smithi016.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 963 kB/s | 56 kB 00:00 2024-03-26T08:29:49.213 INFO:teuthology.orchestra.run.smithi044.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-03-26T08:29:49.218 INFO:teuthology.orchestra.run.smithi016.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-26T08:29:49.240 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-03-26T08:29:49.244 INFO:teuthology.orchestra.run.smithi016.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 877 kB/s | 44 kB 00:00 2024-03-26T08:29:49.247 INFO:teuthology.orchestra.run.smithi044.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-03-26T08:29:49.269 INFO:teuthology.orchestra.run.smithi016.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-26T08:29:49.274 INFO:teuthology.orchestra.run.smithi044.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T08:29:49.295 INFO:teuthology.orchestra.run.smithi016.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-26T08:29:49.362 INFO:teuthology.orchestra.run.smithi016.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-26T08:29:49.387 INFO:teuthology.orchestra.run.smithi016.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-26T08:29:49.412 INFO:teuthology.orchestra.run.smithi016.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-26T08:29:49.424 INFO:teuthology.orchestra.run.smithi044.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-03-26T08:29:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-26T08:29:49.450 INFO:teuthology.orchestra.run.smithi044.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 141 kB/s | 24 kB 00:00 2024-03-26T08:29:49.475 INFO:teuthology.orchestra.run.smithi044.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 362 kB/s | 18 kB 00:00 2024-03-26T08:29:49.501 INFO:teuthology.orchestra.run.smithi044.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-26T08:29:49.518 INFO:teuthology.orchestra.run.smithi044.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 749 kB/s | 32 kB 00:00 2024-03-26T08:29:49.522 INFO:teuthology.orchestra.run.smithi016.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-03-26T08:29:49.539 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-03-26T08:29:49.539 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-03-26T08:29:49.563 INFO:teuthology.orchestra.run.smithi016.stdout:(88/174): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-03-26T08:29:49.605 INFO:teuthology.orchestra.run.smithi016.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-03-26T08:29:49.647 INFO:teuthology.orchestra.run.smithi016.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-03-26T08:29:49.652 INFO:teuthology.orchestra.run.smithi044.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-26T08:29:49.677 INFO:teuthology.orchestra.run.smithi044.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-26T08:29:49.702 INFO:teuthology.orchestra.run.smithi044.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-26T08:29:49.719 INFO:teuthology.orchestra.run.smithi044.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 69 kB 00:00 2024-03-26T08:29:49.736 INFO:teuthology.orchestra.run.smithi044.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2024-03-26T08:29:49.761 INFO:teuthology.orchestra.run.smithi044.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 5.6 MB/s | 238 kB 00:00 2024-03-26T08:29:49.772 INFO:teuthology.orchestra.run.smithi016.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 129 kB/s | 21 kB 00:00 2024-03-26T08:29:49.786 INFO:teuthology.orchestra.run.smithi044.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-26T08:29:49.789 INFO:teuthology.orchestra.run.smithi016.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 179 kB/s | 25 kB 00:00 2024-03-26T08:29:49.812 INFO:teuthology.orchestra.run.smithi044.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-26T08:29:49.829 INFO:teuthology.orchestra.run.smithi044.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-26T08:29:49.846 INFO:teuthology.orchestra.run.smithi044.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 43 kB 00:00 2024-03-26T08:29:49.979 INFO:teuthology.orchestra.run.smithi044.stdout:(74/174): python3-toml-0.10.2-6.el9.noarch.rpm 291 kB/s | 39 kB 00:00 2024-03-26T08:29:50.255 INFO:teuthology.orchestra.run.smithi044.stdout:(75/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-26T08:29:50.271 INFO:teuthology.orchestra.run.smithi044.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 142 kB/s | 41 kB 00:00 2024-03-26T08:29:50.297 INFO:teuthology.orchestra.run.smithi044.stdout:(77/174): uuid-1.6.2-55.el9.x86_64.rpm 2.2 MB/s | 56 kB 00:00 2024-03-26T08:29:50.322 INFO:teuthology.orchestra.run.smithi044.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-03-26T08:29:50.339 INFO:teuthology.orchestra.run.smithi044.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-26T08:29:50.356 INFO:teuthology.orchestra.run.smithi044.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-26T08:29:50.373 INFO:teuthology.orchestra.run.smithi016.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-03-26T08:29:50.381 INFO:teuthology.orchestra.run.smithi044.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-26T08:29:50.406 INFO:teuthology.orchestra.run.smithi044.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-26T08:29:50.440 INFO:teuthology.orchestra.run.smithi044.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-26T08:29:50.457 INFO:teuthology.orchestra.run.smithi044.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 103 kB 00:00 2024-03-26T08:29:50.482 INFO:teuthology.orchestra.run.smithi044.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-03-26T08:29:50.508 INFO:teuthology.orchestra.run.smithi044.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-26T08:29:50.515 INFO:teuthology.orchestra.run.smithi016.stdout:(94/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.6 MB/s | 4.1 MB 00:00 2024-03-26T08:29:50.573 INFO:teuthology.orchestra.run.smithi016.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-03-26T08:29:50.633 INFO:teuthology.orchestra.run.smithi044.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-26T08:29:50.640 INFO:teuthology.orchestra.run.smithi016.stdout:(96/174): postgresql-contrib-13.14-1.el9.x86_64 3.3 MB/s | 911 kB 00:00 2024-03-26T08:29:50.683 INFO:teuthology.orchestra.run.smithi044.stdout:(88/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-03-26T08:29:50.700 INFO:teuthology.orchestra.run.smithi044.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-03-26T08:29:50.726 INFO:teuthology.orchestra.run.smithi044.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-03-26T08:29:50.743 INFO:teuthology.orchestra.run.smithi044.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 512 kB/s | 21 kB 00:00 2024-03-26T08:29:50.750 INFO:teuthology.orchestra.run.smithi016.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-03-26T08:29:50.759 INFO:teuthology.orchestra.run.smithi044.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 758 kB/s | 25 kB 00:00 2024-03-26T08:29:50.843 INFO:teuthology.orchestra.run.smithi044.stdout:(93/174): postgresql-13.14-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-03-26T08:29:50.858 INFO:teuthology.orchestra.run.smithi016.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-03-26T08:29:50.918 INFO:teuthology.orchestra.run.smithi044.stdout:(94/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 24 MB/s | 4.1 MB 00:00 2024-03-26T08:29:50.925 INFO:teuthology.orchestra.run.smithi016.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-03-26T08:29:50.960 INFO:teuthology.orchestra.run.smithi044.stdout:(95/174): postgresql-contrib-13.14-1.el9.x86_64 7.6 MB/s | 911 kB 00:00 2024-03-26T08:29:50.986 INFO:teuthology.orchestra.run.smithi044.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-03-26T08:29:51.001 INFO:teuthology.orchestra.run.smithi016.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 478 kB/s | 36 kB 00:00 2024-03-26T08:29:51.028 INFO:teuthology.orchestra.run.smithi044.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-03-26T08:29:51.059 INFO:teuthology.orchestra.run.smithi016.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-03-26T08:29:51.070 INFO:teuthology.orchestra.run.smithi044.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-03-26T08:29:51.145 INFO:teuthology.orchestra.run.smithi044.stdout:(99/174): postgresql-server-13.14-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-03-26T08:29:51.152 INFO:teuthology.orchestra.run.smithi016.stdout:(102/174): postgresql-server-13.14-1.el9.x86_64 10 MB/s | 5.8 MB 00:00 2024-03-26T08:29:51.169 INFO:teuthology.orchestra.run.smithi016.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 860 kB/s | 94 kB 00:00 2024-03-26T08:29:51.170 INFO:teuthology.orchestra.run.smithi044.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-03-26T08:29:51.196 INFO:teuthology.orchestra.run.smithi044.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 716 kB/s | 36 kB 00:00 2024-03-26T08:29:51.211 INFO:teuthology.orchestra.run.smithi016.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 382 kB/s | 22 kB 00:00 2024-03-26T08:29:51.221 INFO:teuthology.orchestra.run.smithi044.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-03-26T08:29:51.246 INFO:teuthology.orchestra.run.smithi044.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-26T08:29:51.288 INFO:teuthology.orchestra.run.smithi044.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 332 kB/s | 22 kB 00:00 2024-03-26T08:29:51.344 INFO:teuthology.orchestra.run.smithi016.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-03-26T08:29:51.364 INFO:teuthology.orchestra.run.smithi044.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-03-26T08:29:51.422 INFO:teuthology.orchestra.run.smithi044.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-03-26T08:29:51.456 INFO:teuthology.orchestra.run.smithi044.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 585 kB/s | 19 kB 00:00 2024-03-26T08:29:51.481 INFO:teuthology.orchestra.run.smithi044.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-03-26T08:29:51.515 INFO:teuthology.orchestra.run.smithi044.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 784 kB/s | 26 kB 00:00 2024-03-26T08:29:51.543 INFO:teuthology.orchestra.run.smithi044.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 1.7 MB/s | 49 kB 00:00 2024-03-26T08:29:51.577 INFO:teuthology.orchestra.run.smithi044.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-03-26T08:29:51.594 INFO:teuthology.orchestra.run.smithi016.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-03-26T08:29:51.629 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-03-26T08:29:51.631 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-03-26T08:29:51.644 INFO:teuthology.orchestra.run.smithi044.stdout:(112/174): libarrow-9.0.0-10.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-03-26T08:29:51.669 INFO:teuthology.orchestra.run.smithi044.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-03-26T08:29:51.678 INFO:teuthology.orchestra.run.smithi016.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 926 kB/s | 309 kB 00:00 2024-03-26T08:29:51.703 INFO:teuthology.orchestra.run.smithi044.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2024-03-26T08:29:51.703 INFO:teuthology.orchestra.run.smithi016.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-03-26T08:29:51.745 INFO:teuthology.orchestra.run.smithi044.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-03-26T08:29:51.770 INFO:teuthology.orchestra.run.smithi044.stdout:(116/174): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-03-26T08:29:51.778 INFO:teuthology.orchestra.run.smithi016.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-03-26T08:29:51.787 INFO:teuthology.orchestra.run.smithi044.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 770 kB/s | 32 kB 00:00 2024-03-26T08:29:51.804 INFO:teuthology.orchestra.run.smithi044.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 448 kB/s | 15 kB 00:00 2024-03-26T08:29:51.829 INFO:teuthology.orchestra.run.smithi044.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-03-26T08:29:51.854 INFO:teuthology.orchestra.run.smithi016.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-03-26T08:29:51.854 INFO:teuthology.orchestra.run.smithi044.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 6.9 MB/s | 358 kB 00:00 2024-03-26T08:29:51.897 INFO:teuthology.orchestra.run.smithi044.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-03-26T08:29:51.922 INFO:teuthology.orchestra.run.smithi044.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 526 kB/s | 35 kB 00:00 2024-03-26T08:29:51.937 INFO:teuthology.orchestra.run.smithi016.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 809 kB/s | 67 kB 00:00 2024-03-26T08:29:51.948 INFO:teuthology.orchestra.run.smithi044.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.9 MB/s | 203 kB 00:00 2024-03-26T08:29:51.981 INFO:teuthology.orchestra.run.smithi044.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-03-26T08:29:52.023 INFO:teuthology.orchestra.run.smithi044.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-03-26T08:29:52.029 INFO:teuthology.orchestra.run.smithi016.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-03-26T08:29:52.049 INFO:teuthology.orchestra.run.smithi044.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-03-26T08:29:52.074 INFO:teuthology.orchestra.run.smithi044.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-03-26T08:29:52.099 INFO:teuthology.orchestra.run.smithi044.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-03-26T08:29:52.126 INFO:teuthology.orchestra.run.smithi044.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-03-26T08:29:52.150 INFO:teuthology.orchestra.run.smithi044.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-03-26T08:29:52.183 INFO:teuthology.orchestra.run.smithi044.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-03-26T08:29:52.205 INFO:teuthology.orchestra.run.smithi016.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-03-26T08:29:52.218 INFO:teuthology.orchestra.run.smithi044.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-03-26T08:29:52.234 INFO:teuthology.orchestra.run.smithi044.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-03-26T08:29:52.238 INFO:teuthology.orchestra.run.smithi016.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-03-26T08:29:52.259 INFO:teuthology.orchestra.run.smithi044.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-03-26T08:29:52.284 INFO:teuthology.orchestra.run.smithi044.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-03-26T08:29:52.305 INFO:teuthology.orchestra.run.smithi016.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-03-26T08:29:52.309 INFO:teuthology.orchestra.run.smithi044.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-03-26T08:29:52.331 INFO:teuthology.orchestra.run.smithi016.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 473 kB/s | 43 kB 00:00 2024-03-26T08:29:52.335 INFO:teuthology.orchestra.run.smithi044.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-26T08:29:52.361 INFO:teuthology.orchestra.run.smithi044.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-03-26T08:29:52.381 INFO:teuthology.orchestra.run.smithi016.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-03-26T08:29:52.386 INFO:teuthology.orchestra.run.smithi044.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-03-26T08:29:52.406 INFO:teuthology.orchestra.run.smithi016.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-03-26T08:29:52.411 INFO:teuthology.orchestra.run.smithi044.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-03-26T08:29:52.428 INFO:teuthology.orchestra.run.smithi044.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-03-26T08:29:52.470 INFO:teuthology.orchestra.run.smithi044.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-03-26T08:29:52.481 INFO:teuthology.orchestra.run.smithi016.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-26T08:29:52.487 INFO:teuthology.orchestra.run.smithi044.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 872 kB/s | 51 kB 00:00 2024-03-26T08:29:52.507 INFO:teuthology.orchestra.run.smithi016.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-26T08:29:52.512 INFO:teuthology.orchestra.run.smithi044.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 5.4 MB/s | 230 kB 00:00 2024-03-26T08:29:52.537 INFO:teuthology.orchestra.run.smithi044.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-03-26T08:29:52.562 INFO:teuthology.orchestra.run.smithi044.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-03-26T08:29:52.574 INFO:teuthology.orchestra.run.smithi016.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-03-26T08:29:52.579 INFO:teuthology.orchestra.run.smithi044.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-03-26T08:29:52.590 INFO:teuthology.orchestra.run.smithi016.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 423 kB/s | 35 kB 00:00 2024-03-26T08:29:52.607 INFO:teuthology.orchestra.run.smithi044.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 4.2 MB/s | 192 kB 00:00 2024-03-26T08:29:52.658 INFO:teuthology.orchestra.run.smithi016.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-26T08:29:52.758 INFO:teuthology.orchestra.run.smithi016.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-03-26T08:29:52.966 INFO:teuthology.orchestra.run.smithi016.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-03-26T08:29:52.983 INFO:teuthology.orchestra.run.smithi016.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 47 kB/s | 11 kB 00:00 2024-03-26T08:29:53.041 INFO:teuthology.orchestra.run.smithi044.stdout:(149/174): ceph-test-19.0.0-2145.g3be9a5c9.el9. 5.4 MB/s | 48 MB 00:08 2024-03-26T08:29:53.075 INFO:teuthology.orchestra.run.smithi016.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-03-26T08:29:53.082 INFO:teuthology.orchestra.run.smithi044.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-03-26T08:29:53.101 INFO:teuthology.orchestra.run.smithi016.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-03-26T08:29:53.125 INFO:teuthology.orchestra.run.smithi044.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-26T08:29:53.193 INFO:teuthology.orchestra.run.smithi016.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-03-26T08:29:53.218 INFO:teuthology.orchestra.run.smithi016.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-03-26T08:29:53.308 INFO:teuthology.orchestra.run.smithi044.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-03-26T08:29:53.344 INFO:teuthology.orchestra.run.smithi016.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-03-26T08:29:53.375 INFO:teuthology.orchestra.run.smithi044.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 28 MB/s | 1.8 MB 00:00 2024-03-26T08:29:53.402 INFO:teuthology.orchestra.run.smithi016.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-03-26T08:29:53.417 INFO:teuthology.orchestra.run.smithi044.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-26T08:29:53.436 INFO:teuthology.orchestra.run.smithi016.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 861 kB/s | 79 kB 00:00 2024-03-26T08:29:53.459 INFO:teuthology.orchestra.run.smithi044.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-03-26T08:29:53.486 INFO:teuthology.orchestra.run.smithi016.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-03-26T08:29:53.501 INFO:teuthology.orchestra.run.smithi044.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-26T08:29:53.528 INFO:teuthology.orchestra.run.smithi016.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-03-26T08:29:53.543 INFO:teuthology.orchestra.run.smithi044.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 877 kB/s | 36 kB 00:00 2024-03-26T08:29:53.570 INFO:teuthology.orchestra.run.smithi016.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-03-26T08:29:53.585 INFO:teuthology.orchestra.run.smithi044.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 192 kB 00:00 2024-03-26T08:29:53.612 INFO:teuthology.orchestra.run.smithi016.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-03-26T08:29:53.627 INFO:teuthology.orchestra.run.smithi044.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 256 kB 00:00 2024-03-26T08:29:53.654 INFO:teuthology.orchestra.run.smithi016.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-03-26T08:29:53.669 INFO:teuthology.orchestra.run.smithi044.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 555 kB/s | 23 kB 00:00 2024-03-26T08:29:53.688 INFO:teuthology.orchestra.run.smithi016.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-03-26T08:29:53.710 INFO:teuthology.orchestra.run.smithi044.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 108 kB 00:00 2024-03-26T08:29:53.738 INFO:teuthology.orchestra.run.smithi016.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-03-26T08:29:53.752 INFO:teuthology.orchestra.run.smithi044.stdout:(162/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 155 kB 00:00 2024-03-26T08:29:53.763 INFO:teuthology.orchestra.run.smithi016.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-03-26T08:29:53.794 INFO:teuthology.orchestra.run.smithi044.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 136 kB 00:00 2024-03-26T08:29:53.813 INFO:teuthology.orchestra.run.smithi016.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-03-26T08:29:53.836 INFO:teuthology.orchestra.run.smithi044.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 64 kB 00:00 2024-03-26T08:29:53.838 INFO:teuthology.orchestra.run.smithi016.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-03-26T08:29:53.878 INFO:teuthology.orchestra.run.smithi044.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 691 kB/s | 29 kB 00:00 2024-03-26T08:29:53.897 INFO:teuthology.orchestra.run.smithi016.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-26T08:29:53.922 INFO:teuthology.orchestra.run.smithi016.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-26T08:29:53.928 INFO:teuthology.orchestra.run.smithi044.stdout:(166/174): openssl-3.0.7-27.el9.x86_64.rpm 24 MB/s | 1.2 MB 00:00 2024-03-26T08:29:53.996 INFO:teuthology.orchestra.run.smithi044.stdout:(167/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 32 MB/s | 2.2 MB 00:00 2024-03-26T08:29:53.998 INFO:teuthology.orchestra.run.smithi016.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-26T08:29:54.023 INFO:teuthology.orchestra.run.smithi016.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-03-26T08:29:54.066 INFO:teuthology.orchestra.run.smithi044.stdout:(168/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-03-26T08:29:54.081 INFO:teuthology.orchestra.run.smithi016.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-03-26T08:29:54.204 INFO:teuthology.orchestra.run.smithi044.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.3 MB 00:00 2024-03-26T08:29:54.223 INFO:teuthology.orchestra.run.smithi016.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-03-26T08:29:54.238 INFO:teuthology.orchestra.run.smithi044.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 942 kB/s | 31 kB 00:00 2024-03-26T08:29:54.297 INFO:teuthology.orchestra.run.smithi044.stdout:(171/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-03-26T08:29:54.322 INFO:teuthology.orchestra.run.smithi044.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 1.8 MB/s | 465 kB 00:00 2024-03-26T08:29:54.364 INFO:teuthology.orchestra.run.smithi044.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 74 kB 00:00 2024-03-26T08:29:54.739 INFO:teuthology.orchestra.run.smithi044.stdout:(174/174): librados2-19.0.0-2145.g3be9a5c9.el9. 1.6 MB/s | 3.4 MB 00:02 2024-03-26T08:29:54.744 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:29:54.744 INFO:teuthology.orchestra.run.smithi044.stdout:Total 15 MB/s | 225 MB 00:14 2024-03-26T08:29:54.816 INFO:teuthology.orchestra.run.smithi016.stdout:(150/174): ceph-test-19.0.0-2145.g3be9a5c9.el9. 3.2 MB/s | 48 MB 00:14 2024-03-26T08:29:54.857 INFO:teuthology.orchestra.run.smithi016.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-26T08:29:55.057 INFO:teuthology.orchestra.run.smithi016.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-26T08:29:55.216 INFO:teuthology.orchestra.run.smithi016.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 12 MB/s | 1.8 MB 00:00 2024-03-26T08:29:55.275 INFO:teuthology.orchestra.run.smithi016.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 722 kB/s | 42 kB 00:00 2024-03-26T08:29:55.333 INFO:teuthology.orchestra.run.smithi016.stdout:(155/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 2.8 MB/s | 3.1 MB 00:01 2024-03-26T08:29:55.347 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-03-26T08:29:55.392 INFO:teuthology.orchestra.run.smithi016.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-26T08:29:55.417 INFO:teuthology.orchestra.run.smithi016.stdout:(157/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.7 MB/s | 248 kB 00:00 2024-03-26T08:29:55.459 INFO:teuthology.orchestra.run.smithi016.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 544 kB/s | 36 kB 00:00 2024-03-26T08:29:55.501 INFO:teuthology.orchestra.run.smithi016.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.2 MB/s | 192 kB 00:00 2024-03-26T08:29:55.543 INFO:teuthology.orchestra.run.smithi016.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 256 kB 00:00 2024-03-26T08:29:55.568 INFO:teuthology.orchestra.run.smithi016.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 345 kB/s | 23 kB 00:00 2024-03-26T08:29:55.619 INFO:teuthology.orchestra.run.smithi016.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-03-26T08:29:55.644 INFO:teuthology.orchestra.run.smithi016.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-26T08:29:55.649 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-03-26T08:29:55.649 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-03-26T08:29:55.694 INFO:teuthology.orchestra.run.smithi016.stdout:(164/174): librados2-19.0.0-2145.g3be9a5c9.el9. 2.1 MB/s | 3.4 MB 00:01 2024-03-26T08:29:55.745 INFO:teuthology.orchestra.run.smithi016.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 136 kB 00:00 2024-03-26T08:29:55.770 INFO:teuthology.orchestra.run.smithi016.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 382 kB/s | 29 kB 00:00 2024-03-26T08:29:55.795 INFO:teuthology.orchestra.run.smithi016.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 424 kB/s | 64 kB 00:00 2024-03-26T08:29:55.904 INFO:teuthology.orchestra.run.smithi016.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 1.2 MB 00:00 2024-03-26T08:29:56.004 INFO:teuthology.orchestra.run.smithi016.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 9.2 MB/s | 2.2 MB 00:00 2024-03-26T08:29:56.054 INFO:teuthology.orchestra.run.smithi016.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 8.8 MB/s | 2.3 MB 00:00 2024-03-26T08:29:56.080 INFO:teuthology.orchestra.run.smithi016.stdout:(171/174): util-linux-core-2.37.4-18.el9.x86_64 2.6 MB/s | 465 kB 00:00 2024-03-26T08:29:56.105 INFO:teuthology.orchestra.run.smithi016.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 310 kB/s | 31 kB 00:00 2024-03-26T08:29:56.205 INFO:teuthology.orchestra.run.smithi016.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.2 MB/s | 1.1 MB 00:00 2024-03-26T08:29:56.230 INFO:teuthology.orchestra.run.smithi016.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 494 kB/s | 74 kB 00:00 2024-03-26T08:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:29:56.235 INFO:teuthology.orchestra.run.smithi016.stdout:Total 11 MB/s | 225 MB 00:20 2024-03-26T08:29:56.819 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-03-26T08:29:57.121 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-03-26T08:29:57.121 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-03-26T08:29:57.493 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-03-26T08:29:57.493 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-03-26T08:29:58.842 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-03-26T08:29:58.929 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T08:29:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T08:29:58.974 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-03-26T08:29:58.974 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-03-26T08:29:59.002 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:29:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:29:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T08:29:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T08:29:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T08:29:59.192 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T08:29:59.242 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T08:29:59.590 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T08:29:59.643 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:29:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:29:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:29:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:29:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:29:59.975 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T08:30:00.161 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T08:30:00.214 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T08:30:00.240 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:30:00.338 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:30:00.366 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:30:00.432 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:30:00.479 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T08:30:00.502 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:30:00.530 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:30:00.719 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:30:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:30:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T08:30:00.971 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T08:30:01.111 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:30:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:30:01.397 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:30:01.622 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:30:01.816 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:30:01.923 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T08:30:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T08:30:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T08:30:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T08:30:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T08:30:02.210 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T08:30:02.261 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T08:30:02.299 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T08:30:02.383 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T08:30:02.423 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T08:30:02.466 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T08:30:02.499 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T08:30:02.526 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T08:30:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T08:30:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-26T08:30:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-26T08:30:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-26T08:30:02.795 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T08:30:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T08:30:02.930 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T08:30:03.083 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T08:30:03.139 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T08:30:03.694 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T08:30:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T08:30:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T08:30:03.969 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:30:03.980 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:30:04.010 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T08:30:04.051 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T08:30:04.088 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-26T08:30:04.119 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T08:30:04.685 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T08:30:04.780 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-03-26T08:30:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T08:30:04.877 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T08:30:04.911 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T08:30:04.941 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:30:04.951 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:30:04.954 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T08:30:04.992 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T08:30:05.015 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T08:30:05.046 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T08:30:05.079 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T08:30:05.112 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T08:30:05.155 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T08:30:05.156 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T08:30:05.201 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T08:30:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T08:30:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T08:30:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T08:30:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T08:30:05.471 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T08:30:05.503 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T08:30:05.505 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T08:30:05.550 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T08:30:05.563 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:30:05.601 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T08:30:05.678 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:30:05.742 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T08:30:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T08:30:05.814 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:30:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T08:30:05.878 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T08:30:05.899 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:30:05.927 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:30:05.967 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T08:30:05.975 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T08:30:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T08:30:06.051 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T08:30:06.076 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T08:30:06.101 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T08:30:06.104 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-03-26T08:30:06.168 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T08:30:06.188 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T08:30:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T08:30:06.217 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T08:30:06.244 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:30:06.260 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T08:30:06.291 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:30:06.301 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T08:30:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T08:30:06.341 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T08:30:06.349 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:30:06.379 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:30:06.396 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T08:30:06.426 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:30:06.430 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T08:30:06.436 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T08:30:06.457 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T08:30:06.462 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T08:30:06.468 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T08:30:06.480 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:30:06.496 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:30:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T08:30:06.582 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T08:30:06.589 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T08:30:06.714 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:30:06.744 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:30:06.774 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T08:30:06.776 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T08:30:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T08:30:06.876 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T08:30:06.970 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:30:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T08:30:07.014 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:30:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T08:30:07.078 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:30:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T08:30:07.095 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T08:30:07.108 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T08:30:07.136 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T08:30:07.139 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:30:07.154 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:30:07.179 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T08:30:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T08:30:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T08:30:07.262 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:30:07.280 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T08:30:07.289 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:30:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T08:30:07.361 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:30:07.382 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:30:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T08:30:07.410 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:30:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T08:30:07.457 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T08:30:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T08:30:07.481 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T08:30:07.500 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T08:30:07.576 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T08:30:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T08:30:07.611 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T08:30:07.637 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T08:30:07.638 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T08:30:07.682 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T08:30:07.685 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T08:30:07.690 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T08:30:07.713 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:30:07.748 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T08:30:07.758 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T08:30:07.779 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T08:30:07.811 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T08:30:07.827 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T08:30:07.843 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:30:07.848 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T08:30:07.872 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:30:07.916 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T08:30:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T08:30:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T08:30:07.956 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T08:30:07.968 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T08:30:07.990 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:30:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T08:30:07.999 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T08:30:08.009 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T08:30:08.031 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T08:30:08.068 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T08:30:08.140 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T08:30:08.173 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-26T08:30:08.200 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:30:08.208 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-26T08:30:08.230 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T08:30:08.236 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-26T08:30:08.262 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T08:30:08.310 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T08:30:08.351 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T08:30:08.362 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T08:30:08.429 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T08:30:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:30:08.508 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T08:30:08.574 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T08:30:08.583 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:30:08.630 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T08:30:08.634 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:30:08.760 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T08:30:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T08:30:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T08:30:08.842 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T08:30:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T08:30:08.952 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T08:30:08.980 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T08:30:09.017 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T08:30:09.047 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T08:30:09.089 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T08:30:09.116 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T08:30:09.160 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T08:30:09.186 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T08:30:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T08:30:09.231 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T08:30:09.237 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T08:30:09.269 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T08:30:09.290 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T08:30:09.312 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T08:30:09.327 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T08:30:09.352 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T08:30:09.385 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:30:09.394 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:30:09.403 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T08:30:09.425 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T08:30:09.443 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T08:30:09.506 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T08:30:09.506 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T08:30:09.512 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-26T08:30:09.519 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T08:30:09.541 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T08:30:09.546 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T08:30:09.618 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T08:30:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-26T08:30:09.695 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-26T08:30:09.723 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-26T08:30:09.784 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T08:30:09.804 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T08:30:09.847 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T08:30:09.925 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T08:30:09.952 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T08:30:09.998 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T08:30:10.019 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T08:30:10.064 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T08:30:10.080 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T08:30:10.117 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T08:30:10.134 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T08:30:10.420 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T08:30:10.500 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T08:30:10.557 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T08:30:10.630 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T08:30:10.667 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T08:30:10.693 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T08:30:10.703 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T08:30:10.751 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T08:30:10.773 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T08:30:10.795 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T08:30:10.838 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T08:30:10.859 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T08:30:10.880 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T08:30:10.914 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:30:10.924 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T08:30:10.955 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T08:30:10.996 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T08:30:11.031 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T08:30:11.035 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-26T08:30:11.063 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T08:30:11.077 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T08:30:11.107 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T08:30:11.150 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T08:30:11.250 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T08:30:11.290 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T08:30:11.317 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T08:30:11.330 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T08:30:11.356 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T08:30:11.380 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T08:30:11.406 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T08:30:11.455 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T08:30:11.498 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T08:30:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T08:30:11.530 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T08:30:11.561 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T08:30:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T08:30:11.586 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T08:30:11.589 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T08:30:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T08:30:11.701 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T08:30:11.748 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T08:30:11.759 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T08:30:11.800 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T08:30:11.805 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T08:30:11.857 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T08:30:11.860 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T08:30:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T08:30:11.932 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T08:30:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:30:11.976 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T08:30:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T08:30:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:30:12.052 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T08:30:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T08:30:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-26T08:30:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T08:30:12.101 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T08:30:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T08:30:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T08:30:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T08:30:12.249 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T08:30:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T08:30:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T08:30:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T08:30:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T08:30:12.492 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T08:30:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T08:30:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T08:30:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T08:30:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T08:30:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:30:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T08:30:12.648 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T08:30:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T08:30:12.679 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T08:30:12.708 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T08:30:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:30:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-26T08:30:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T08:30:12.742 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T08:30:12.781 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T08:30:12.821 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T08:30:12.851 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T08:30:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T08:30:12.932 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T08:30:12.969 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T08:30:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T08:30:13.003 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T08:30:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T08:30:13.044 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T08:30:13.078 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T08:30:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T08:30:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T08:30:13.173 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T08:30:13.262 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T08:30:13.318 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T08:30:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T08:30:13.349 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T08:30:13.387 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T08:30:13.397 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T08:30:13.508 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T08:30:13.530 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T08:30:13.560 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T08:30:13.568 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T08:30:13.600 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T08:30:13.631 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T08:30:13.659 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T08:30:13.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T08:30:13.733 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T08:30:13.764 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T08:30:13.794 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T08:30:13.865 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T08:30:13.897 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T08:30:13.930 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T08:30:13.972 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T08:30:14.006 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T08:30:14.100 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T08:30:14.197 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T08:30:14.246 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T08:30:14.285 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T08:30:14.334 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T08:30:14.335 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T08:30:14.381 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T08:30:14.415 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T08:30:14.442 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T08:30:14.463 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T08:30:14.496 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T08:30:14.555 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T08:30:14.599 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T08:30:14.783 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T08:30:14.813 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T08:30:14.893 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T08:30:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:30:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:30:15.265 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T08:30:15.317 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T08:30:15.359 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T08:30:15.409 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T08:30:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T08:30:15.535 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T08:30:15.574 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T08:30:15.688 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T08:30:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:30:15.720 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T08:30:15.749 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T08:30:15.757 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:30:15.839 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T08:30:15.848 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:30:15.848 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T08:30:15.848 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T08:30:15.848 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:30:16.516 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T08:30:16.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T08:30:16.974 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T08:30:17.047 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T08:30:17.078 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T08:30:17.109 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T08:30:17.136 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T08:30:17.465 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T08:30:17.500 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:30:17.583 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:30:17.612 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-26T08:30:17.650 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T08:30:17.922 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T08:30:18.001 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T08:30:18.032 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T08:30:18.063 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T08:30:18.091 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T08:30:18.136 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T08:30:18.161 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:30:18.216 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:30:18.246 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-26T08:30:18.411 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T08:30:18.437 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:30:18.529 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T08:30:18.559 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-26T08:30:18.596 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T08:30:19.084 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T08:30:19.116 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:30:19.171 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T08:30:19.201 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-26T08:30:20.496 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:30:20.525 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:30:21.092 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:30:21.128 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:30:21.194 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:30:21.194 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T08:30:21.194 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T08:30:21.195 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:21.213 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:30:21.484 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:30:21.530 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:30:22.097 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T08:30:22.142 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:30:22.224 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T08:30:22.225 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T08:30:22.225 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T08:30:22.225 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:22.260 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:30:28.060 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:30:28.060 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-03-26T08:30:28.060 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-03-26T08:30:28.061 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-03-26T08:30:28.061 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-03-26T08:30:28.061 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-03-26T08:30:28.061 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-03-26T08:30:28.061 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-03-26T08:30:28.061 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:28.231 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:30:28.249 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:30:28.249 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:28.249 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T08:30:28.249 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:29.776 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:30:29.801 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:30:29.802 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:29.802 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T08:30:29.802 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:30:29.802 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:30:29.802 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:30.044 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:30:30.066 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:30:30.066 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:30.066 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T08:30:30.066 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:30:30.066 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:30:30.067 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:30.536 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:30:30.559 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:30:30.559 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:30.559 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T08:30:30.559 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:30:30.560 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:30:30.560 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:30.654 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T08:30:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:30:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:30:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-26T08:30:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-26T08:30:31.176 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:30:31.425 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:30:31.452 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:30:32.469 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:30:32.494 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:30:32.599 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:30:32.765 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-26T08:30:32.796 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:30:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:30:32.820 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:32.820 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T08:30:32.820 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:30:32.820 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:30:32.820 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:30:32.883 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:30:32.945 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-26T08:30:33.449 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:30:33.450 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-03-26T08:30:33.450 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-03-26T08:30:33.450 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-03-26T08:30:33.450 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-03-26T08:30:33.450 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-03-26T08:30:33.450 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-03-26T08:30:33.450 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-03-26T08:30:33.450 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:33.626 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:30:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:30:33.645 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:33.645 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T08:30:33.645 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:33.869 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T08:30:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:30:33.922 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:30:33.922 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:33.922 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T08:30:33.922 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:30:33.922 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:30:33.922 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:33.966 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:30:33.989 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:30:33.989 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:33.989 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T08:30:33.989 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:34.033 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:30:34.056 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:30:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:34.056 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T08:30:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:30:34.057 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:30:34.057 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:34.376 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:30:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:30:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T08:30:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:30:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:30:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:34.470 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T08:30:34.470 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-03-26T08:30:34.471 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-03-26T08:30:34.471 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-03-26T08:30:34.471 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-03-26T08:30:34.471 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-03-26T08:30:34.471 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-03-26T08:30:34.471 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-03-26T08:30:34.471 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:34.641 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:30:34.657 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T08:30:34.657 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:34.657 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T08:30:34.657 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:35.188 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:30:35.215 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:30:35.215 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:35.215 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T08:30:35.215 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:30:35.215 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:30:35.215 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:35.455 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:30:35.479 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:30:35.479 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:35.479 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T08:30:35.479 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:30:35.479 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:30:35.479 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:35.939 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:30:35.963 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:30:35.963 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:35.963 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T08:30:35.963 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:30:35.963 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:30:35.963 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:36.066 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T08:30:36.211 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:30:36.238 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T08:30:36.239 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:36.239 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T08:30:36.239 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:30:36.239 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T08:30:36.239 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:36.471 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:30:36.495 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T08:30:36.495 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:36.495 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T08:30:36.496 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:30:36.496 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T08:30:36.496 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:36.511 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:30:36.540 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:30:36.574 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-26T08:30:36.635 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-26T08:30:36.662 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:30:36.910 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:30:36.946 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:30:36.955 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:30:36.979 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T08:30:36.979 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T08:30:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:30:36.980 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T08:30:36.980 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:37.066 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T08:30:37.503 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:30:37.531 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T08:30:37.572 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-26T08:30:37.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-26T08:30:37.663 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:30:37.911 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T08:30:37.955 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:30:37.960 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:30:37.989 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:30:38.097 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:30:38.266 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-26T08:30:38.290 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:30:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:30:38.315 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:38.315 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T08:30:38.315 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:30:38.315 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:30:38.315 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:38.366 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:30:38.377 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:30:38.423 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-26T08:30:38.972 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T08:30:39.005 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:30:39.112 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T08:30:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:30:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-26T08:30:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-26T08:30:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T08:30:39.284 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-26T08:30:39.316 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:30:39.340 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T08:30:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T08:30:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:30:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T08:30:39.340 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:39.345 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T08:30:39.370 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:30:39.392 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:30:39.394 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:30:39.394 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:39.394 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T08:30:39.394 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:30:39.394 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:30:39.394 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:39.403 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T08:30:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-26T08:30:39.437 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:30:39.460 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:30:39.461 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:39.461 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T08:30:39.461 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:39.467 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-26T08:30:39.505 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:30:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:30:39.529 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:39.529 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T08:30:39.529 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:30:39.529 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:30:39.529 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T08:30:39.588 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-26T08:30:39.647 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T08:30:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T08:30:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T08:30:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-26T08:30:39.804 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T08:30:39.834 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T08:30:39.855 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:30:39.879 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:30:39.879 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:39.880 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T08:30:39.880 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:30:39.880 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:30:39.880 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:39.891 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T08:30:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T08:30:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T08:30:40.388 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T08:30:40.419 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:30:40.443 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T08:30:40.443 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:40.443 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T08:30:40.443 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:30:40.444 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T08:30:40.444 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:40.486 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:30:40.509 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T08:30:40.509 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:40.509 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T08:30:40.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:40.552 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:30:40.554 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:30:40.577 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:30:40.578 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:40.578 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T08:30:40.578 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:30:40.578 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T08:30:40.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:40.589 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:30:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:30:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:30:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T08:30:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T08:30:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T08:30:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:30:40.929 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T08:30:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:41.494 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-26T08:30:41.594 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T08:30:41.635 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T08:30:41.729 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T08:30:41.764 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T08:30:41.790 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T08:30:41.815 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T08:30:41.841 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T08:30:41.841 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:30:41.929 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:30:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T08:30:41.959 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:30:42.043 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:30:42.070 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T08:30:42.103 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T08:30:42.128 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T08:30:42.153 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T08:30:42.192 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T08:30:42.230 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T08:30:42.267 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T08:30:42.292 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T08:30:42.317 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T08:30:42.342 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:30:42.368 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T08:30:42.375 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:30:42.376 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:30:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:30:42.562 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:30:42.645 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:30:42.670 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:30:42.702 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T08:30:42.729 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T08:30:42.755 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T08:30:42.824 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:30:44.590 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:30:44.636 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-26T08:30:44.665 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-26T08:30:44.692 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T08:30:44.887 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-26T08:30:45.004 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T08:30:45.049 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-26T08:30:45.088 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T08:30:45.141 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T08:30:45.184 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T08:30:45.216 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-26T08:30:45.240 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T08:30:45.270 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T08:30:45.324 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T08:30:45.386 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T08:30:45.418 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T08:30:45.634 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:30:45.676 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-26T08:30:45.706 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-26T08:30:45.733 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T08:30:45.936 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-26T08:30:46.050 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:30:46.064 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T08:30:46.082 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:30:46.103 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:30:46.109 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-26T08:30:46.147 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T08:30:46.192 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T08:30:46.235 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T08:30:46.267 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-26T08:30:46.308 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T08:30:46.338 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T08:30:46.392 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T08:30:46.446 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T08:30:46.470 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T08:30:47.005 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-26T08:30:47.073 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:30:47.086 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T08:30:47.118 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:30:47.127 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T08:30:47.139 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T08:30:47.212 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T08:30:47.248 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T08:30:47.273 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T08:30:47.298 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T08:30:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T08:30:47.325 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:30:47.413 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:30:47.442 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T08:30:47.442 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:30:47.527 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:30:47.553 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T08:30:47.595 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T08:30:47.620 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T08:30:47.640 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:30:47.641 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-26T08:30:47.642 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-26T08:30:47.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-26T08:30:47.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-26T08:30:47.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-26T08:30:47.643 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-26T08:30:47.649 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T08:30:47.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-26T08:30:47.662 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T08:30:47.663 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T08:30:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T08:30:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T08:30:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T08:30:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T08:30:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T08:30:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T08:30:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T08:30:47.664 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T08:30:47.665 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T08:30:47.665 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T08:30:47.665 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T08:30:47.665 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T08:30:47.665 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T08:30:47.665 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T08:30:47.665 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T08:30:47.665 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T08:30:47.666 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T08:30:47.666 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T08:30:47.666 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T08:30:47.666 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T08:30:47.667 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T08:30:47.667 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T08:30:47.667 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T08:30:47.667 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T08:30:47.667 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T08:30:47.667 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T08:30:47.668 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T08:30:47.668 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T08:30:47.668 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T08:30:47.668 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T08:30:47.668 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T08:30:47.668 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T08:30:47.668 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T08:30:47.669 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T08:30:47.670 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T08:30:47.671 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T08:30:47.672 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T08:30:47.673 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T08:30:47.674 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T08:30:47.675 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T08:30:47.676 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T08:30:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:30:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T08:30:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T08:30:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T08:30:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:30:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T08:30:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T08:30:47.677 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T08:30:47.678 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T08:30:47.726 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T08:30:47.766 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T08:30:47.792 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T08:30:47.817 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T08:30:47.843 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:30:47.867 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T08:30:47.874 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:30:47.874 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:30:48.062 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:30:48.062 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:30:48.062 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-26T08:30:48.145 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:30:48.147 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T08:30:48.169 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:30:48.190 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T08:30:48.202 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T08:30:48.229 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T08:30:48.254 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T08:30:48.274 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T08:30:48.310 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T08:30:48.321 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:30:48.360 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T08:30:48.385 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T08:30:48.412 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T08:30:48.412 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:30:48.500 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T08:30:48.529 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T08:30:48.529 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:30:48.614 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T08:30:48.640 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T08:30:48.673 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T08:30:48.699 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T08:30:48.723 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T08:30:48.758 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T08:30:48.797 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T08:30:48.829 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T08:30:48.854 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T08:30:48.879 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T08:30:48.905 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:30:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T08:30:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:30:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:30:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T08:30:49.125 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:30:49.207 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T08:30:49.232 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:30:49.264 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T08:30:49.292 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T08:30:49.317 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T08:30:49.383 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:30:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T08:30:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:50.805 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-03-26T08:30:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T08:30:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T08:30:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: jq-1.6-16.el9.x86_64 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T08:30:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T08:30:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T08:30:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T08:30:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T08:30:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T08:30:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T08:30:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T08:30:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:30:50.818 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-03-26T08:30:51.250 DEBUG:teuthology.parallel:result is None 2024-03-26T08:30:52.998 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:30:52.998 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-26T08:30:52.998 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-26T08:30:52.998 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-26T08:30:52.998 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-26T08:30:52.998 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-26T08:30:52.998 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-26T08:30:52.998 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-26T08:30:52.999 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-26T08:30:53.000 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T08:30:53.001 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T08:30:53.002 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T08:30:53.003 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T08:30:53.004 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T08:30:53.005 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T08:30:53.006 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T08:30:53.007 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:30:53.008 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T08:30:53.009 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T08:30:53.010 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T08:30:53.011 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T08:30:53.012 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-26T08:30:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-26T08:30:54.231 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-26T08:30:54.232 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-26T08:30:54.232 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-26T08:30:54.248 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T08:30:54.249 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T08:30:54.250 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T08:30:54.251 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T08:30:54.252 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T08:30:54.253 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T08:30:54.254 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T08:30:54.255 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T08:30:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T08:30:54.257 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T08:30:54.258 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T08:30:54.259 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T08:30:54.260 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T08:30:54.260 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T08:30:54.260 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T08:30:54.260 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T08:30:54.260 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T08:30:54.260 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T08:30:54.260 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T08:30:54.260 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T08:30:54.260 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T08:30:56.225 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T08:30:56.226 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.227 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.228 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T08:30:56.229 INFO:teuthology.orchestra.run.smithi044.stdout: jq-1.6-16.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.230 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T08:30:56.231 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.232 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T08:30:56.233 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T08:30:56.234 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.235 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T08:30:56.236 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T08:30:56.237 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T08:30:56.238 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T08:30:56.238 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:30:56.238 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-03-26T08:30:56.686 DEBUG:teuthology.parallel:result is None 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T08:30:57.294 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T08:30:57.295 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.296 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.297 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-16.el9.x86_64 2024-03-26T08:30:57.298 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.299 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T08:30:57.300 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T08:30:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T08:30:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T08:30:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T08:30:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T08:30:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T08:30:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T08:30:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T08:30:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T08:30:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T08:30:57.302 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T08:30:57.303 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T08:30:57.303 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T08:30:57.303 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T08:30:57.303 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T08:30:57.303 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T08:30:57.303 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T08:30:57.304 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T08:30:57.305 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T08:30:57.306 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T08:30:57.307 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T08:30:57.308 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T08:30:57.308 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T08:30:57.308 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T08:30:57.308 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T08:30:57.308 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-03-26T08:30:57.308 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-03-26T08:30:57.779 DEBUG:teuthology.parallel:result is None 2024-03-26T08:30:57.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:30:57.998 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T08:30:58.021 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:30:58.022 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:30:58.022 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-26T08:30:58.024 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:30:58.167 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T08:30:58.190 INFO:teuthology.orchestra.run.smithi016.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:30:58.190 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:30:58.190 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-26T08:30:58.192 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:30:58.405 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T08:30:58.431 INFO:teuthology.orchestra.run.smithi044.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:30:58.431 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-26T08:30:58.431 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-26T08:30:58.433 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-26T08:30:58.434 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:30:58.434 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T08:30:58.467 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:30:58.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T08:30:58.504 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:30:58.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T08:30:58.543 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-26T08:30:58.544 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:30:58.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T08:30:58.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T08:30:58.643 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:30:58.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T08:30:58.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T08:30:58.750 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:30:58.751 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T08:30:58.787 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T08:30:58.863 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-26T08:30:58.863 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:30:58.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T08:30:58.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T08:30:58.963 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:30:58.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T08:30:58.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T08:30:59.071 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:30:59.072 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T08:30:59.107 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T08:30:59.182 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-26T08:30:59.182 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:30:59.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T08:30:59.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T08:30:59.283 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:30:59.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T08:30:59.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T08:30:59.390 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:30:59.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T08:30:59.426 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T08:30:59.502 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-26T08:30:59.601 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-26T08:30:59.601 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:30:59.602 INFO:tasks.cephadm:Cluster fsid is 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:30:59.602 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-26T08:30:59.602 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.7', 'mon.b': '172.21.15.16', 'mon.c': '172.21.15.44'} 2024-03-26T08:30:59.602 INFO:tasks.cephadm:First mon is mon.a on smithi007 2024-03-26T08:30:59.602 INFO:tasks.cephadm:First mgr is x 2024-03-26T08:30:59.602 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-26T08:30:59.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-03-26T08:30:59.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-03-26T08:30:59.670 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname $(hostname -s) 2024-03-26T08:30:59.705 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-26T08:30:59.705 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:30:59.926 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77877/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2145-g3be9a5c9', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2145.g3be9a5c9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 05:51:42.267384', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-main-20240318.032620', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-26T08:31:00.171 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-main-20240318.032620, sha1 3be9a5c9bc793e11e2800a8c0c696e8b46742033 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:31:00.173 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm 2024-03-26T08:31:00.173 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T08:31:00.667 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 26 08:31 /home/ubuntu/cephtest/cephadm 2024-03-26T08:31:00.668 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T08:31:01.126 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 26 08:31 /home/ubuntu/cephtest/cephadm 2024-03-26T08:31:01.126 DEBUG:teuthology.orchestra.run.smithi044:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T08:31:01.587 INFO:teuthology.orchestra.run.smithi044.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 26 08:31 /home/ubuntu/cephtest/cephadm 2024-03-26T08:31:01.587 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T08:31:01.606 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T08:31:01.626 DEBUG:teuthology.orchestra.run.smithi044:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T08:31:01.662 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 on all hosts... 2024-03-26T08:31:01.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-26T08:31:01.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-26T08:31:01.669 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-26T08:31:02.560 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:31:02.567 INFO:teuthology.orchestra.run.smithi044.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:31:02.716 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:31:33.327 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-26T08:31:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T08:31:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-26T08:31:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-03-26T08:31:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-26T08:31:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-03-26T08:31:33.328 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-26T08:31:46.161 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-26T08:31:46.161 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T08:31:46.161 INFO:teuthology.orchestra.run.smithi044.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-26T08:31:46.161 INFO:teuthology.orchestra.run.smithi044.stdout: "repo_digests": [ 2024-03-26T08:31:46.161 INFO:teuthology.orchestra.run.smithi044.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-26T08:31:46.162 INFO:teuthology.orchestra.run.smithi044.stdout: ] 2024-03-26T08:31:46.162 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-26T08:31:46.966 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:31:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T08:31:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-26T08:31:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-03-26T08:31:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-26T08:31:46.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:31:46.967 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:31:46.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-03-26T08:31:47.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-03-26T08:31:47.076 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph 2024-03-26T08:31:47.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-03-26T08:31:47.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-03-26T08:31:47.177 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /etc/ceph 2024-03-26T08:31:47.212 INFO:tasks.cephadm:Writing seed config... 2024-03-26T08:31:47.213 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-26T08:31:47.213 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-26T08:31:47.213 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-26T08:31:47.213 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-26T08:31:47.214 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-26T08:31:47.215 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-26T08:31:47.216 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-26T08:31:47.217 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:31:47.217 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-26T08:31:47.231 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2c753f3e-eb4b-11ee-95d0-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-26T08:31:47.232 DEBUG:teuthology.orchestra.run.smithi007:mon.a> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.a.service 2024-03-26T08:31:47.274 DEBUG:teuthology.orchestra.run.smithi007:mgr.x> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.x.service 2024-03-26T08:31:47.316 INFO:tasks.cephadm:Bootstrapping... 2024-03-26T08:31:47.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 -v bootstrap --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:31:47.512 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-03-26T08:31:47.512 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033', '-v', 'bootstrap', '--fsid', '2c753f3e-eb4b-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-03-26T08:31:47.536 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T08:31:47.536 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-26T08:31:47.537 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-03-26T08:31:47.559 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T08:31:47.560 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-03-26T08:31:47.560 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-03-26T08:31:47.567 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T08:31:47.567 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T08:31:47.573 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T08:31:47.573 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-26T08:31:47.579 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-03-26T08:31:47.585 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-03-26T08:31:47.585 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-03-26T08:31:47.585 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-03-26T08:31:47.607 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T08:31:47.607 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-26T08:31:47.608 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-03-26T08:31:47.608 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-03-26T08:31:47.614 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T08:31:47.614 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T08:31:47.619 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T08:31:47.620 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-26T08:31:47.625 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-03-26T08:31:47.631 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-03-26T08:31:47.632 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-03-26T08:31:47.632 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-03-26T08:31:47.632 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:31:47.632 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 140267336687040 on /run/cephadm/2c753f3e-eb4b-11ee-95d0-87774f69a715.lock 2024-03-26T08:31:47.632 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 140267336687040 acquired on /run/cephadm/2c753f3e-eb4b-11ee-95d0-87774f69a715.lock 2024-03-26T08:31:47.633 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-03-26T08:31:47.633 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-03-26T08:31:47.633 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-03-26T08:31:47.636 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.7 metric 100 2024-03-26T08:31:47.636 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-03-26T08:31:47.638 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-26T08:31:47.638 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-26T08:31:47.638 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-26T08:31:47.641 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-26T08:31:47.641 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-26T08:31:47.641 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T08:31:47.641 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-26T08:31:47.641 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-03-26T08:31:47.641 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T08:31:47.641 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-03-26T08:31:47.642 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-03-26T08:31:47.642 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-26T08:31:47.642 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-26T08:31:47.643 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:31:48.110 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-26T08:31:48.111 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T08:31:48.111 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-03-26T08:31:48.111 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:4dd83c6ee8921fe17396d77c31e205985ae94c87d3f752465b5f04aaf9ae340a 2024-03-26T08:31:48.111 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-26T08:31:48.111 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-26T08:31:48.111 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-26T08:31:49.027 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-26T08:31:49.027 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-26T08:31:49.027 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-03-26T08:31:49.932 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-03-26T08:31:49.932 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-03-26T08:31:50.737 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQD2hwJmtMTEDRAAxsgQhId7bMxp1FwoS56Ypw== 2024-03-26T08:31:51.717 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQD3hwJmpM8+BBAAiT+JUAxBUvxyogZHoT+9dA== 2024-03-26T08:31:52.540 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQD4hwJmfBimARAAhxCF40GmCV0xE+wS3UVnkQ== 2024-03-26T08:31:52.541 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-03-26T08:31:53.479 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T08:31:53.480 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-26T08:31:53.480 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:31:53.480 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T08:31:53.480 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for a [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T08:31:53.480 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-03-26T08:31:53.480 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:31:53.480 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T08:31:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:31:53.480 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-03-26T08:31:54.461 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.a on 2024-03-26T08:31:54.665 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-26T08:31:54.803 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-26T08:31:54.996 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715.target. 2024-03-26T08:31:54.996 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715.target. 2024-03-26T08:31:55.195 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.a 2024-03-26T08:31:55.195 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-26T08:31:55.356 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715.target.wants/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@.service. 2024-03-26T08:31:55.850 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T08:31:55.850 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-26T08:31:55.856 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T08:31:55.856 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-26T08:31:55.856 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-26T08:31:55.857 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T08:31:55.857 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-03-26T08:31:55.857 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.524128s) 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:31:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.273+0000 7fb62bd0c700 1 Processor -- start 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.274+0000 7fb62bd0c700 1 -- start start 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.274+0000 7fb62bd0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.274+0000 7fb62bd0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62407c8f0 con 0x7fb62407dec0 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.274+0000 7fb629aa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.274+0000 7fb629aa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34630/0 (socket says 172.21.15.7:34630) 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.274+0000 7fb629aa8700 1 -- 172.21.15.7:0/3696557105 learned_addr learned my addr 172.21.15.7:0/3696557105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.274+0000 7fb629aa8700 1 -- 172.21.15.7:0/3696557105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62407ca70 con 0x7fb62407dec0 2024-03-26T08:31:56.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb629aa8700 1 --2- 172.21.15.7:0/3696557105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62407c320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6200098a0 tx=0x7fb620009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef8d62d0987f58c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb628aa6700 1 -- 172.21.15.7:0/3696557105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb620004160 con 0x7fb62407dec0 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb628aa6700 1 -- 172.21.15.7:0/3696557105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620004300 con 0x7fb62407dec0 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb628aa6700 1 -- 172.21.15.7:0/3696557105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6200036a0 con 0x7fb62407dec0 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb62bd0c700 1 -- 172.21.15.7:0/3696557105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 msgr2=0x7fb62407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb62bd0c700 1 --2- 172.21.15.7:0/3696557105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62407c320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6200098a0 tx=0x7fb620009bb0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb62bd0c700 1 -- 172.21.15.7:0/3696557105 shutdown_connections 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb62bd0c700 1 --2- 172.21.15.7:0/3696557105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62407c320 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb62bd0c700 1 -- 172.21.15.7:0/3696557105 >> 172.21.15.7:0/3696557105 conn(0x7fb624103150 msgr2=0x7fb624105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb62bd0c700 1 -- 172.21.15.7:0/3696557105 shutdown_connections 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.275+0000 7fb62bd0c700 1 -- 172.21.15.7:0/3696557105 wait complete. 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.276+0000 7fb62bd0c700 1 Processor -- start 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.276+0000 7fb62bd0c700 1 -- start start 2024-03-26T08:31:56.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.276+0000 7fb62bd0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62419d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.276+0000 7fb62bd0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62407cfa0 con 0x7fb62407dec0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.276+0000 7fb629aa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62419d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.276+0000 7fb629aa8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62419d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34634/0 (socket says 172.21.15.7:34634) 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.276+0000 7fb629aa8700 1 -- 172.21.15.7:0/100130757 learned_addr learned my addr 172.21.15.7:0/100130757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.277+0000 7fb629aa8700 1 -- 172.21.15.7:0/100130757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62419da30 con 0x7fb62407dec0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.277+0000 7fb629aa8700 1 --2- 172.21.15.7:0/100130757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62419d4f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb620000c00 tx=0x7fb620004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.277+0000 7fb616ffd700 1 -- 172.21.15.7:0/100130757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb620003db0 con 0x7fb62407dec0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.277+0000 7fb616ffd700 1 -- 172.21.15.7:0/100130757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620011630 con 0x7fb62407dec0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.277+0000 7fb62bd0c700 1 -- 172.21.15.7:0/100130757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62419dcc0 con 0x7fb62407dec0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.277+0000 7fb616ffd700 1 -- 172.21.15.7:0/100130757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb620011950 con 0x7fb62407dec0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.277+0000 7fb62bd0c700 1 -- 172.21.15.7:0/100130757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6241a09b0 con 0x7fb62407dec0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.277+0000 7fb616ffd700 1 -- 172.21.15.7:0/100130757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb62001a460 con 0x7fb62407dec0 2024-03-26T08:31:56.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.277+0000 7fb616ffd700 1 -- 172.21.15.7:0/100130757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb62001b780 con 0x7fb62407dec0 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.278+0000 7fb62bd0c700 1 -- 172.21.15.7:0/100130757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6240637c0 con 0x7fb62407dec0 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.280+0000 7fb616ffd700 1 -- 172.21.15.7:0/100130757 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb620031020 con 0x7fb62407dec0 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.354+0000 7fb62bd0c700 1 -- 172.21.15.7:0/100130757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb62402dc80 con 0x7fb62407dec0 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.354+0000 7fb616ffd700 1 -- 172.21.15.7:0/100130757 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fb62001b9d0 con 0x7fb62407dec0 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.355+0000 7fb62bd0c700 1 -- 172.21.15.7:0/100130757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 msgr2=0x7fb62419d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.355+0000 7fb62bd0c700 1 --2- 172.21.15.7:0/100130757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62419d4f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb620000c00 tx=0x7fb620004750 comp rx=0 tx=0).stop 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.355+0000 7fb62bd0c700 1 -- 172.21.15.7:0/100130757 shutdown_connections 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.355+0000 7fb62bd0c700 1 --2- 172.21.15.7:0/100130757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62407dec0 0x7fb62419d4f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.355+0000 7fb62bd0c700 1 -- 172.21.15.7:0/100130757 >> 172.21.15.7:0/100130757 conn(0x7fb624103150 msgr2=0x7fb624105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.355+0000 7fb62bd0c700 1 -- 172.21.15.7:0/100130757 shutdown_connections 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:56.355+0000 7fb62bd0c700 1 -- 172.21.15.7:0/100130757 wait complete. 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-03-26T08:31:56.940 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-03-26T08:31:57.975 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:31:57 smithi007 ceph-mon[32090]: from='client.? 172.21.15.7:0/3130959929' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-26T08:31:57.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:31:57.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T08:31:57.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:31:57.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T08:31:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.364+0000 7ff38021f700 1 Processor -- start 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.364+0000 7ff38021f700 1 -- start start 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.364+0000 7ff38021f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff3781061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.364+0000 7ff38021f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3781067c0 con 0x7ff378105dd0 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.365+0000 7ff37dfbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff3781061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.365+0000 7ff37dfbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff3781061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56044/0 (socket says 172.21.15.7:56044) 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.365+0000 7ff37dfbb700 1 -- 172.21.15.7:0/2765282883 learned_addr learned my addr 172.21.15.7:0/2765282883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.365+0000 7ff37dfbb700 1 -- 172.21.15.7:0/2765282883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378108d60 con 0x7ff378105dd0 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.365+0000 7ff37dfbb700 1 --2- 172.21.15.7:0/2765282883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff3781061f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff3740098a0 tx=0x7ff374009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcec336d7a9bbbb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.365+0000 7ff37cfb9700 1 -- 172.21.15.7:0/2765282883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff374004160 con 0x7ff378105dd0 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.365+0000 7ff37cfb9700 1 -- 172.21.15.7:0/2765282883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff374004300 con 0x7ff378105dd0 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.365+0000 7ff37cfb9700 1 -- 172.21.15.7:0/2765282883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3740036a0 con 0x7ff378105dd0 2024-03-26T08:31:57.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.366+0000 7ff38021f700 1 -- 172.21.15.7:0/2765282883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 msgr2=0x7ff3781061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.366+0000 7ff38021f700 1 --2- 172.21.15.7:0/2765282883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff3781061f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff3740098a0 tx=0x7ff374009bb0 comp rx=0 tx=0).stop 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.366+0000 7ff38021f700 1 -- 172.21.15.7:0/2765282883 shutdown_connections 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.366+0000 7ff38021f700 1 --2- 172.21.15.7:0/2765282883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff3781061f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.366+0000 7ff38021f700 1 -- 172.21.15.7:0/2765282883 >> 172.21.15.7:0/2765282883 conn(0x7ff378101780 msgr2=0x7ff378103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.366+0000 7ff38021f700 1 -- 172.21.15.7:0/2765282883 shutdown_connections 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.366+0000 7ff38021f700 1 -- 172.21.15.7:0/2765282883 wait complete. 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.366+0000 7ff38021f700 1 Processor -- start 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff38021f700 1 -- start start 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff38021f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff378198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff38021f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff378109290 con 0x7ff378105dd0 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff37dfbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff378198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff37dfbb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff378198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56050/0 (socket says 172.21.15.7:56050) 2024-03-26T08:31:57.986 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff37dfbb700 1 -- 172.21.15.7:0/3130959929 learned_addr learned my addr 172.21.15.7:0/3130959929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff37dfbb700 1 -- 172.21.15.7:0/3130959929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378199530 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff37dfbb700 1 --2- 172.21.15.7:0/3130959929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff378198ff0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff374000c00 tx=0x7ff374004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff36effd700 1 -- 172.21.15.7:0/3130959929 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff374003db0 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff36effd700 1 -- 172.21.15.7:0/3130959929 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff374011630 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff38021f700 1 -- 172.21.15.7:0/3130959929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3781997c0 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff36effd700 1 -- 172.21.15.7:0/3130959929 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff374011950 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.367+0000 7ff38021f700 1 -- 172.21.15.7:0/3130959929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37819c4b0 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.368+0000 7ff36effd700 1 -- 172.21.15.7:0/3130959929 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff374011af0 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.368+0000 7ff36effd700 1 -- 172.21.15.7:0/3130959929 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff37401c450 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.369+0000 7ff38021f700 1 -- 172.21.15.7:0/3130959929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3780637c0 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.371+0000 7ff36effd700 1 -- 172.21.15.7:0/3130959929 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff374004260 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.441+0000 7ff38021f700 1 -- 172.21.15.7:0/3130959929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff37819c680 con 0x7ff378105dd0 2024-03-26T08:31:57.987 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.444+0000 7ff36effd700 1 -- 172.21.15.7:0/3130959929 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7ff37401b450 con 0x7ff378105dd0 2024-03-26T08:31:57.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.444+0000 7ff36effd700 1 -- 172.21.15.7:0/3130959929 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff37401b630 con 0x7ff378105dd0 2024-03-26T08:31:57.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.445+0000 7ff38021f700 1 -- 172.21.15.7:0/3130959929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 msgr2=0x7ff378198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:57.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.445+0000 7ff38021f700 1 --2- 172.21.15.7:0/3130959929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff378198ff0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff374000c00 tx=0x7ff374004750 comp rx=0 tx=0).stop 2024-03-26T08:31:57.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.445+0000 7ff38021f700 1 -- 172.21.15.7:0/3130959929 shutdown_connections 2024-03-26T08:31:57.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.445+0000 7ff38021f700 1 --2- 172.21.15.7:0/3130959929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff378105dd0 0x7ff378198ff0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:57.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.445+0000 7ff38021f700 1 -- 172.21.15.7:0/3130959929 >> 172.21.15.7:0/3130959929 conn(0x7ff378101780 msgr2=0x7ff378103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:57.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.445+0000 7ff38021f700 1 -- 172.21.15.7:0/3130959929 shutdown_connections 2024-03-26T08:31:57.988 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:57.445+0000 7ff38021f700 1 -- 172.21.15.7:0/3130959929 wait complete. 2024-03-26T08:31:57.988 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-03-26T08:31:58.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.359+0000 7fa70e090700 1 Processor -- start 2024-03-26T08:31:58.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.359+0000 7fa70e090700 1 -- start start 2024-03-26T08:31:58.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.359+0000 7fa70e090700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:58.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.359+0000 7fa70e090700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70807c8d0 con 0x7fa70807dea0 2024-03-26T08:31:58.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.360+0000 7fa7077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:58.863 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.360+0000 7fa7077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56062/0 (socket says 172.21.15.7:56062) 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.360+0000 7fa7077fe700 1 -- 172.21.15.7:0/2565015336 learned_addr learned my addr 172.21.15.7:0/2565015336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.360+0000 7fa7077fe700 1 -- 172.21.15.7:0/2565015336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70807ca50 con 0x7fa70807dea0 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.360+0000 7fa7077fe700 1 --2- 172.21.15.7:0/2565015336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70807c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa700009e80 tx=0x7fa70000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7dcf56bb793f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.360+0000 7fa7067fc700 1 -- 172.21.15.7:0/2565015336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa700017070 con 0x7fa70807dea0 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.360+0000 7fa7067fc700 1 -- 172.21.15.7:0/2565015336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa70000b8c0 con 0x7fa70807dea0 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.360+0000 7fa7067fc700 1 -- 172.21.15.7:0/2565015336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7000045c0 con 0x7fa70807dea0 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.361+0000 7fa70e090700 1 -- 172.21.15.7:0/2565015336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 msgr2=0x7fa70807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.361+0000 7fa70e090700 1 --2- 172.21.15.7:0/2565015336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70807c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa700009e80 tx=0x7fa70000b330 comp rx=0 tx=0).stop 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.361+0000 7fa70e090700 1 -- 172.21.15.7:0/2565015336 shutdown_connections 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.361+0000 7fa70e090700 1 --2- 172.21.15.7:0/2565015336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70807c300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.361+0000 7fa70e090700 1 -- 172.21.15.7:0/2565015336 >> 172.21.15.7:0/2565015336 conn(0x7fa708103150 msgr2=0x7fa708105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.361+0000 7fa70e090700 1 -- 172.21.15.7:0/2565015336 shutdown_connections 2024-03-26T08:31:58.864 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.361+0000 7fa70e090700 1 -- 172.21.15.7:0/2565015336 wait complete. 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.361+0000 7fa70e090700 1 Processor -- start 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa70e090700 1 -- start start 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa70e090700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa70e090700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70810e180 con 0x7fa70807dea0 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa7077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa7077fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56072/0 (socket says 172.21.15.7:56072) 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa7077fe700 1 -- 172.21.15.7:0/4201103376 learned_addr learned my addr 172.21.15.7:0/4201103376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa7077fe700 1 -- 172.21.15.7:0/4201103376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70819e210 con 0x7fa70807dea0 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa7077fe700 1 --2- 172.21.15.7:0/4201103376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70819dcd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa700009e50 tx=0x7fa70000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa704ff9700 1 -- 172.21.15.7:0/4201103376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa700020020 con 0x7fa70807dea0 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa704ff9700 1 -- 172.21.15.7:0/4201103376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa700003b50 con 0x7fa70807dea0 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa70e090700 1 -- 172.21.15.7:0/4201103376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70819e4a0 con 0x7fa70807dea0 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.362+0000 7fa704ff9700 1 -- 172.21.15.7:0/4201103376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa700017040 con 0x7fa70807dea0 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.363+0000 7fa70e090700 1 -- 172.21.15.7:0/4201103376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7081a1180 con 0x7fa70807dea0 2024-03-26T08:31:58.865 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.363+0000 7fa704ff9700 1 -- 172.21.15.7:0/4201103376 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa700023020 con 0x7fa70807dea0 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.363+0000 7fa704ff9700 1 -- 172.21.15.7:0/4201103376 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa70000f050 con 0x7fa70807dea0 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.363+0000 7fa70e090700 1 -- 172.21.15.7:0/4201103376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7080637c0 con 0x7fa70807dea0 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.366+0000 7fa704ff9700 1 -- 172.21.15.7:0/4201103376 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa70002a020 con 0x7fa70807dea0 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.437+0000 7fa70e090700 1 -- 172.21.15.7:0/4201103376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa7081a1740 con 0x7fa70807dea0 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.437+0000 7fa704ff9700 1 -- 172.21.15.7:0/4201103376 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7fa700003d00 con 0x7fa70807dea0 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.438+0000 7fa70e090700 1 -- 172.21.15.7:0/4201103376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 msgr2=0x7fa70819dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.438+0000 7fa70e090700 1 --2- 172.21.15.7:0/4201103376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70819dcd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa700009e50 tx=0x7fa70000bd30 comp rx=0 tx=0).stop 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.438+0000 7fa70e090700 1 -- 172.21.15.7:0/4201103376 shutdown_connections 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.438+0000 7fa70e090700 1 --2- 172.21.15.7:0/4201103376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa70807dea0 0x7fa70819dcd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.438+0000 7fa70e090700 1 -- 172.21.15.7:0/4201103376 >> 172.21.15.7:0/4201103376 conn(0x7fa708103150 msgr2=0x7fa708103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.438+0000 7fa70e090700 1 -- 172.21.15.7:0/4201103376 shutdown_connections 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:31:58.438+0000 7fa70e090700 1 -- 172.21.15.7:0/4201103376 wait complete. 2024-03-26T08:31:58.866 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-03-26T08:31:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:31:58 smithi007 ceph-mon[32090]: from='client.? 172.21.15.7:0/4201103376' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:31:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:31:58 smithi007 systemd[1]: Stopping Ceph mon.a for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:31:59.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:31:59 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a[32086]: 2024-03-26T08:31:59.158+0000 7ff4d0a58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:31:59.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:31:59 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a[32086]: 2024-03-26T08:31:59.158+0000 7ff4d0a58700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-26T08:31:59.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:31:59 smithi007 podman[32336]: 2024-03-26 08:31:59.393926157 +0000 UTC m=+0.365047166 container died d1fab79a4ee1d4935f21ab79470433c2b67f61ec00cd3c74530a4d5d403863e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T08:31:59.909 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:31:59 smithi007 podman[32336]: 2024-03-26 08:31:59.733220056 +0000 UTC m=+0.704341065 container cleanup d1fab79a4ee1d4935f21ab79470433c2b67f61ec00cd3c74530a4d5d403863e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-26T08:31:59.910 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:31:59 smithi007 bash[32336]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a 2024-03-26T08:32:00.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:31:59 smithi007 podman[32350]: 2024-03-26 08:31:59.917646453 +0000 UTC m=+0.523893156 container remove d1fab79a4ee1d4935f21ab79470433c2b67f61ec00cd3c74530a4d5d403863e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-26T08:32:00.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 systemd[1]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-26T08:32:00.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 systemd[1]: Stopped Ceph mon.a for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:32:00.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 systemd[1]: Starting Ceph mon.a for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:32:00.600 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-26T08:32:00.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 podman[32442]: 2024-03-26 08:32:00.260268613 +0000 UTC m=+0.017677394 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T08:32:00.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 podman[32442]: 2024-03-26 08:32:00.360541954 +0000 UTC m=+0.117950762 container create e370da5b95266058f482c9dd0fe004a73af9eb054dd1273cd299c27201487c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-26T08:32:00.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 podman[32442]: 2024-03-26 08:32:00.545965785 +0000 UTC m=+0.303374550 container init e370da5b95266058f482c9dd0fe004a73af9eb054dd1273cd299c27201487c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-26T08:32:00.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 podman[32442]: 2024-03-26 08:32:00.548687098 +0000 UTC m=+0.306095870 container start e370da5b95266058f482c9dd0fe004a73af9eb054dd1273cd299c27201487c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:32:00.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: pidfile_write: ignore empty --pid-file 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: load: jerasure load: lrc 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: RocksDB version: 7.9.2 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Git sha 0 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: DB SUMMARY 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: DB Session ID: 7OR0MRZDNKPZL3A4SVSY 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: CURRENT file: CURRENT 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-26T08:32:00.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100720 ; 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.error_if_exists: 0 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.create_if_missing: 0 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.paranoid_checks: 1 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.env: 0x55d0adadab20 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.info_log: 0x55d0af9eb920 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.statistics: (nil) 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.use_fsync: 0 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_log_file_size: 0 2024-03-26T08:32:00.643 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.allow_fallocate: 1 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.use_direct_reads: 0 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.db_log_dir: 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.wal_dir: 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T08:32:00.644 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T08:32:00.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T08:32:00.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T08:32:00.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T08:32:00.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.write_buffer_manager: 0x55d0af9be1e0 2024-03-26T08:32:00.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T08:32:00.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T08:32:00.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T08:32:00.645 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T08:32:00.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T08:32:00.646 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.unordered_write: 0 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.row_cache: None 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.wal_filter: None 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.two_write_queues: 0 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T08:32:00.647 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.wal_compression: 0 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.atomic_flush: 0 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.log_readahead_size: 0 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_background_jobs: 2 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_background_compactions: -1 2024-03-26T08:32:00.648 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_subcompactions: 1 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_open_files: -1 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_background_flushes: -1 2024-03-26T08:32:00.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Compression algorithms supported: 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: kZSTD supported: 0 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: kXpressCompression supported: 0 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: kZlibCompression supported: 1 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: kSnappyCompression supported: 1 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: kLZ4Compression supported: 1 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: kBZip2Compression supported: 0 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T08:32:00.650 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.merge_operator: 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_filter: None 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d0af9eba20) 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_type: 0 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_index_type: 0 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_shortening: 1 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T08:32:00.651 INFO:journalctl@ceph.mon.a.smithi007.stdout: checksum: 4 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: no_block_cache: 0 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache: 0x55d0af9f1090 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_options: 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: capacity : 536870912 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_shard_bits : 4 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: strict_capacity_limit : 0 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_compressed: (nil) 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: persistent_cache: (nil) 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size: 4096 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size_deviation: 10 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_restart_interval: 16 2024-03-26T08:32:00.652 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_block_restart_interval: 1 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: metadata_block_size: 4096 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: partition_filters: 0 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: use_delta_encoding: 1 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: filter_policy: bloomfilter 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: whole_key_filtering: 1 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: verify_compression: 0 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: format_version: 5 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_index_compression: 1 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_align: 0 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: max_auto_readahead_size: 262144 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: prepopulate_block_cache: 0 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: initial_auto_readahead_size: 8192 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T08:32:00.653 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression: NoCompression 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.num_levels: 7 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T08:32:00.654 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T08:32:00.655 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T08:32:00.656 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T08:32:00.657 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.table_properties_collectors: 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.inplace_update_support: 0 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.bloom_locality: 0 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.max_successive_merges: 0 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.ttl: 2592000 2024-03-26T08:32:00.658 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.enable_blob_files: false 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.min_blob_size: 0 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-26T08:32:00.659 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9c755de6-f955-4a10-bf9a-b05e0ff12377 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441920576992, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441920577988, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95839, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93829, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13934, "raw_average_key_size": 50, "raw_value_size": 86257, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711441920, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9c755de6-f955-4a10-bf9a-b05e0ff12377", "db_session_id": "7OR0MRZDNKPZL3A4SVSY", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711441920578070, "job": 1, "event": "recovery_finished"} 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d0afae4000 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: DB pointer 0x55d0aface000 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** DB Stats ** 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T08:32:00.660 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: L0 2/0 95.43 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: Sum 2/0 95.43 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:32:00.661 INFO:journalctl@ceph.mon.a.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative compaction: 0.00 GB write, 30.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval compaction: 0.00 GB write, 30.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache BinnedLRUCache@0x55d0af9f1090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: starting mon.a rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mon.a@-1(???) e1 preinit fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mon.a@-1(???).mds e1 new map 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mon.a@-1(???).mds e1 print_map 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout: e1 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout: legacy client fscid: -1 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout: No filesystems configured 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T08:32:00.663 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 bash[32442]: e370da5b95266058f482c9dd0fe004a73af9eb054dd1273cd299c27201487c52 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 systemd[1]: Started Ceph mon.a for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: monmap e1: 1 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: fsmap 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T08:32:00.664 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:00 smithi007 ceph-mon[32462]: mgrmap e1: no daemons active 2024-03-26T08:32:01.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.098+0000 7fe9f4c69700 1 Processor -- start 2024-03-26T08:32:01.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.098+0000 7fe9f4c69700 1 -- start start 2024-03-26T08:32:01.769 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.099+0000 7fe9f4c69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0106260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.099+0000 7fe9f4c69700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f0106830 con 0x7fe9f0105e40 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.099+0000 7fe9ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0106260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.099+0000 7fe9ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0106260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56082/0 (socket says 172.21.15.7:56082) 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.099+0000 7fe9ee59c700 1 -- 172.21.15.7:0/3621710666 learned_addr learned my addr 172.21.15.7:0/3621710666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.099+0000 7fe9ee59c700 1 -- 172.21.15.7:0/3621710666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0108dd0 con 0x7fe9f0105e40 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.099+0000 7fe9ee59c700 1 --2- 172.21.15.7:0/3621710666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0106260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00098a0 tx=0x7fe9e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42399ec4a1149d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9ed59a700 1 -- 172.21.15.7:0/3621710666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0004160 con 0x7fe9f0105e40 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9ed59a700 1 -- 172.21.15.7:0/3621710666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9e0004300 con 0x7fe9f0105e40 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9ed59a700 1 -- 172.21.15.7:0/3621710666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0011b50 con 0x7fe9f0105e40 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9f4c69700 1 -- 172.21.15.7:0/3621710666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 msgr2=0x7fe9f0106260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9f4c69700 1 --2- 172.21.15.7:0/3621710666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0106260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00098a0 tx=0x7fe9e0009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9f4c69700 1 -- 172.21.15.7:0/3621710666 shutdown_connections 2024-03-26T08:32:01.770 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9f4c69700 1 --2- 172.21.15.7:0/3621710666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0106260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9f4c69700 1 -- 172.21.15.7:0/3621710666 >> 172.21.15.7:0/3621710666 conn(0x7fe9f01017d0 msgr2=0x7fe9f0103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9f4c69700 1 -- 172.21.15.7:0/3621710666 shutdown_connections 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.100+0000 7fe9f4c69700 1 -- 172.21.15.7:0/3621710666 wait complete. 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.101+0000 7fe9f4c69700 1 Processor -- start 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.101+0000 7fe9f4c69700 1 -- start start 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.101+0000 7fe9f4c69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0108650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.101+0000 7fe9f4c69700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f010c180 con 0x7fe9f0105e40 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.101+0000 7fe9ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0108650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.101+0000 7fe9ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0108650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56092/0 (socket says 172.21.15.7:56092) 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.101+0000 7fe9ee59c700 1 -- 172.21.15.7:0/2413911540 learned_addr learned my addr 172.21.15.7:0/2413911540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.102+0000 7fe9ee59c700 1 -- 172.21.15.7:0/2413911540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0108b90 con 0x7fe9f0105e40 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.102+0000 7fe9ee59c700 1 --2- 172.21.15.7:0/2413911540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0108650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009fb0 tx=0x7fe9e0011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.102+0000 7fe9df7fe700 1 -- 172.21.15.7:0/2413911540 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9e00038b0 con 0x7fe9f0105e40 2024-03-26T08:32:01.771 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.102+0000 7fe9f4c69700 1 -- 172.21.15.7:0/2413911540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f0106d80 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.102+0000 7fe9df7fe700 1 -- 172.21.15.7:0/2413911540 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9e0011e50 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.102+0000 7fe9df7fe700 1 -- 172.21.15.7:0/2413911540 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0019550 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.102+0000 7fe9f4c69700 1 -- 172.21.15.7:0/2413911540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f01072c0 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.102+0000 7fe9df7fe700 1 -- 172.21.15.7:0/2413911540 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe9e0019be0 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.102+0000 7fe9df7fe700 1 -- 172.21.15.7:0/2413911540 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe9e0024730 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.103+0000 7fe9f4c69700 1 -- 172.21.15.7:0/2413911540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f00637c0 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.105+0000 7fe9df7fe700 1 -- 172.21.15.7:0/2413911540 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe9e0003a50 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.176+0000 7fe9f4c69700 1 -- 172.21.15.7:0/2413911540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fe9f01077f0 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.178+0000 7fe9df7fe700 1 -- 172.21.15.7:0/2413911540 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe9e0024930 con 0x7fe9f0105e40 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.179+0000 7fe9f4c69700 1 -- 172.21.15.7:0/2413911540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 msgr2=0x7fe9f0108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.179+0000 7fe9f4c69700 1 --2- 172.21.15.7:0/2413911540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0108650 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009fb0 tx=0x7fe9e0011bf0 comp rx=0 tx=0).stop 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.179+0000 7fe9f4c69700 1 -- 172.21.15.7:0/2413911540 shutdown_connections 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.179+0000 7fe9f4c69700 1 --2- 172.21.15.7:0/2413911540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f0105e40 0x7fe9f0108650 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:01.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.179+0000 7fe9f4c69700 1 -- 172.21.15.7:0/2413911540 >> 172.21.15.7:0/2413911540 conn(0x7fe9f01017d0 msgr2=0x7fe9f0102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:01.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.179+0000 7fe9f4c69700 1 -- 172.21.15.7:0/2413911540 shutdown_connections 2024-03-26T08:32:01.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:01.179+0000 7fe9f4c69700 1 -- 172.21.15.7:0/2413911540 wait complete. 2024-03-26T08:32:01.773 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-26T08:32:01.795 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:32:01.795 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-03-26T08:32:01.795 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-26T08:32:01.795 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-26T08:32:02.036 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.x 2024-03-26T08:32:02.037 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-26T08:32:02.177 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715.target.wants/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@.service. 2024-03-26T08:32:02.489 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:02 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2413911540' entity='client.admin' 2024-03-26T08:32:02.679 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T08:32:02.679 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-26T08:32:02.686 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T08:32:02.686 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-26T08:32:02.686 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-26T08:32:02.686 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T08:32:02.694 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T08:32:02.694 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-26T08:32:02.701 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T08:32:02.701 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-26T08:32:02.702 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-26T08:32:02.702 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-26T08:32:02.702 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-03-26T08:32:02.702 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-03-26T08:32:03.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:03 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3626655371' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T08:32:03.544 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:03 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:03.275+0000 7f012ddba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:32:03.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:32:03.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:03.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T08:32:03.804 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T08:32:03.812 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T08:32:03.812 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T08:32:03.812 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T08:32:03.812 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:03.812 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T08:32:03.813 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:03.814 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T08:31:55.832580+0000", 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.128+0000 7f8d8b80d700 1 Processor -- start 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.128+0000 7f8d8b80d700 1 -- start start 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.129+0000 7f8d8b80d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.129+0000 7f8d8b80d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8407c8f0 con 0x7f8d8407dec0 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.129+0000 7f8d895a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.129+0000 7f8d895a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56126/0 (socket says 172.21.15.7:56126) 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.129+0000 7f8d895a9700 1 -- 172.21.15.7:0/349940806 learned_addr learned my addr 172.21.15.7:0/349940806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.129+0000 7f8d895a9700 1 -- 172.21.15.7:0/349940806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8407ca70 con 0x7f8d8407dec0 2024-03-26T08:32:03.815 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.129+0000 7f8d895a9700 1 --2- 172.21.15.7:0/349940806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8407c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8d800098a0 tx=0x7f8d80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe765be81f43688 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d77fff700 1 -- 172.21.15.7:0/349940806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d80004160 con 0x7f8d8407dec0 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d77fff700 1 -- 172.21.15.7:0/349940806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d80004300 con 0x7f8d8407dec0 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d77fff700 1 -- 172.21.15.7:0/349940806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d80011b50 con 0x7f8d8407dec0 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d8b80d700 1 -- 172.21.15.7:0/349940806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 msgr2=0x7f8d8407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d8b80d700 1 --2- 172.21.15.7:0/349940806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8407c320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8d800098a0 tx=0x7f8d80009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d8b80d700 1 -- 172.21.15.7:0/349940806 shutdown_connections 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d8b80d700 1 --2- 172.21.15.7:0/349940806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8407c320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d8b80d700 1 -- 172.21.15.7:0/349940806 >> 172.21.15.7:0/349940806 conn(0x7f8d84103150 msgr2=0x7f8d84105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d8b80d700 1 -- 172.21.15.7:0/349940806 shutdown_connections 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.130+0000 7f8d8b80d700 1 -- 172.21.15.7:0/349940806 wait complete. 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.131+0000 7f8d8b80d700 1 Processor -- start 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.131+0000 7f8d8b80d700 1 -- start start 2024-03-26T08:32:03.816 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.131+0000 7f8d8b80d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.131+0000 7f8d8b80d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8410e160 con 0x7f8d8407dec0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.131+0000 7f8d895a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.131+0000 7f8d895a9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56140/0 (socket says 172.21.15.7:56140) 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.131+0000 7f8d895a9700 1 -- 172.21.15.7:0/3626655371 learned_addr learned my addr 172.21.15.7:0/3626655371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.131+0000 7f8d895a9700 1 -- 172.21.15.7:0/3626655371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8419e2e0 con 0x7f8d8407dec0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.132+0000 7f8d895a9700 1 --2- 172.21.15.7:0/3626655371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8419dda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8d80011850 tx=0x7f8d80011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.132+0000 7f8d767fc700 1 -- 172.21.15.7:0/3626655371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d80003af0 con 0x7f8d8407dec0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.132+0000 7f8d767fc700 1 -- 172.21.15.7:0/3626655371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d80020070 con 0x7f8d8407dec0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.132+0000 7f8d8b80d700 1 -- 172.21.15.7:0/3626655371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8419e570 con 0x7f8d8407dec0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.132+0000 7f8d767fc700 1 -- 172.21.15.7:0/3626655371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d800195a0 con 0x7f8d8407dec0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.132+0000 7f8d8b80d700 1 -- 172.21.15.7:0/3626655371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d841a1220 con 0x7f8d8407dec0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.132+0000 7f8d767fc700 1 -- 172.21.15.7:0/3626655371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8d80019c30 con 0x7f8d8407dec0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.132+0000 7f8d767fc700 1 -- 172.21.15.7:0/3626655371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8d80011e00 con 0x7f8d8407dec0 2024-03-26T08:32:03.817 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.133+0000 7f8d8b80d700 1 -- 172.21.15.7:0/3626655371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d840637c0 con 0x7f8d8407dec0 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.135+0000 7f8d767fc700 1 -- 172.21.15.7:0/3626655371 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8d80003c90 con 0x7f8d8407dec0 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.209+0000 7f8d8b80d700 1 -- 172.21.15.7:0/3626655371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8d841a1490 con 0x7f8d8407dec0 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.210+0000 7f8d767fc700 1 -- 172.21.15.7:0/3626655371 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f8d8001a430 con 0x7f8d8407dec0 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.211+0000 7f8d8b80d700 1 -- 172.21.15.7:0/3626655371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 msgr2=0x7f8d8419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.211+0000 7f8d8b80d700 1 --2- 172.21.15.7:0/3626655371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8419dda0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8d80011850 tx=0x7f8d80011c10 comp rx=0 tx=0).stop 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.211+0000 7f8d8b80d700 1 -- 172.21.15.7:0/3626655371 shutdown_connections 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.211+0000 7f8d8b80d700 1 --2- 172.21.15.7:0/3626655371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d8407dec0 0x7f8d8419dda0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.211+0000 7f8d8b80d700 1 -- 172.21.15.7:0/3626655371 >> 172.21.15.7:0/3626655371 conn(0x7f8d84103150 msgr2=0x7f8d84103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.211+0000 7f8d8b80d700 1 -- 172.21.15.7:0/3626655371 shutdown_connections 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:03.211+0000 7f8d8b80d700 1 -- 172.21.15.7:0/3626655371 wait complete. 2024-03-26T08:32:03.818 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-03-26T08:32:04.275 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:03 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:03.944+0000 7f012ddba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:32:04.275 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:04.040+0000 7f012ddba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:32:04.275 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:04.184+0000 7f012ddba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:32:04.275 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:04.282+0000 7f012ddba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:32:04.543 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:04.367+0000 7f012ddba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:32:04.929 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:04.636+0000 7f012ddba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:32:04.930 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:04.748+0000 7f012ddba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:32:04.930 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:04.838+0000 7f012ddba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:32:04.930 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:04.936+0000 7f012ddba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:32:05.293 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:05 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:05.125+0000 7f012ddba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:32:06.175 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:06.132+0000 7f012ddba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:32:06.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:06 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/109234843' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T08:32:06.543 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:06.270+0000 7f012ddba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:32:06.905 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:06.618+0000 7f012ddba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:32:06.906 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:06.711+0000 7f012ddba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:32:06.906 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:06.912+0000 7f012ddba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:32:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:32:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:32:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:32:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T08:32:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T08:32:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T08:32:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T08:32:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T08:32:06.934 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T08:32:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T08:32:06.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T08:32:06.943 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T08:32:06.944 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T08:31:55.832580+0000", 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.207+0000 7ffa3a3cc700 1 Processor -- start 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.207+0000 7ffa3a3cc700 1 -- start start 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.207+0000 7ffa3a3cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3407b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.207+0000 7ffa3a3cc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3407b800 con 0x7ffa3407cdd0 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.208+0000 7ffa33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3407b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.208+0000 7ffa33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3407b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56150/0 (socket says 172.21.15.7:56150) 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.208+0000 7ffa33fff700 1 -- 172.21.15.7:0/2920811733 learned_addr learned my addr 172.21.15.7:0/2920811733 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.208+0000 7ffa33fff700 1 -- 172.21.15.7:0/2920811733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3407b980 con 0x7ffa3407cdd0 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.208+0000 7ffa33fff700 1 --2- 172.21.15.7:0/2920811733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3407b230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa240098a0 tx=0x7ffa24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01caf37f5f6bdf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.208+0000 7ffa32ffd700 1 -- 172.21.15.7:0/2920811733 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa24004160 con 0x7ffa3407cdd0 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.208+0000 7ffa32ffd700 1 -- 172.21.15.7:0/2920811733 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa24004300 con 0x7ffa3407cdd0 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.208+0000 7ffa32ffd700 1 -- 172.21.15.7:0/2920811733 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa24011b50 con 0x7ffa3407cdd0 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.209+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/2920811733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 msgr2=0x7ffa3407b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.209+0000 7ffa3a3cc700 1 --2- 172.21.15.7:0/2920811733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3407b230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa240098a0 tx=0x7ffa24009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.209+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/2920811733 shutdown_connections 2024-03-26T08:32:06.946 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.209+0000 7ffa3a3cc700 1 --2- 172.21.15.7:0/2920811733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3407b230 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.209+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/2920811733 >> 172.21.15.7:0/2920811733 conn(0x7ffa34103070 msgr2=0x7ffa341054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.209+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/2920811733 shutdown_connections 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.209+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/2920811733 wait complete. 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.210+0000 7ffa3a3cc700 1 Processor -- start 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.210+0000 7ffa3a3cc700 1 -- start start 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.210+0000 7ffa3a3cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.210+0000 7ffa3a3cc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3410e0e0 con 0x7ffa3407cdd0 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.210+0000 7ffa33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.210+0000 7ffa33fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3419dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56160/0 (socket says 172.21.15.7:56160) 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.210+0000 7ffa33fff700 1 -- 172.21.15.7:0/109234843 learned_addr learned my addr 172.21.15.7:0/109234843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.211+0000 7ffa33fff700 1 -- 172.21.15.7:0/109234843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3419e1e0 con 0x7ffa3407cdd0 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.211+0000 7ffa33fff700 1 --2- 172.21.15.7:0/109234843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3419dca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa24011670 tx=0x7ffa24011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.211+0000 7ffa317fa700 1 -- 172.21.15.7:0/109234843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa240039c0 con 0x7ffa3407cdd0 2024-03-26T08:32:06.947 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.211+0000 7ffa317fa700 1 -- 172.21.15.7:0/109234843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa24011e50 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.211+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/109234843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3419e470 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.211+0000 7ffa317fa700 1 -- 172.21.15.7:0/109234843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa2401a5c0 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.211+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/109234843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa341a1160 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.211+0000 7ffa317fa700 1 -- 172.21.15.7:0/109234843 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ffa2401ac00 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.211+0000 7ffa317fa700 1 -- 172.21.15.7:0/109234843 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffa24019700 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.212+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/109234843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3419e950 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.214+0000 7ffa317fa700 1 -- 172.21.15.7:0/109234843 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ffa3419e950 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.288+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/109234843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffa341a1330 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.288+0000 7ffa317fa700 1 -- 172.21.15.7:0/109234843 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ffa24003b60 con 0x7ffa3407cdd0 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.290+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/109234843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 msgr2=0x7ffa3419dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.290+0000 7ffa3a3cc700 1 --2- 172.21.15.7:0/109234843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3419dca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa24011670 tx=0x7ffa24011b60 comp rx=0 tx=0).stop 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.290+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/109234843 shutdown_connections 2024-03-26T08:32:06.948 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.290+0000 7ffa3a3cc700 1 --2- 172.21.15.7:0/109234843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3407cdd0 0x7ffa3419dca0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:06.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.290+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/109234843 >> 172.21.15.7:0/109234843 conn(0x7ffa34103070 msgr2=0x7ffa34103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:06.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.290+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/109234843 shutdown_connections 2024-03-26T08:32:06.949 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:06.290+0000 7ffa3a3cc700 1 -- 172.21.15.7:0/109234843 wait complete. 2024-03-26T08:32:06.949 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-03-26T08:32:07.293 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:07 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:07.102+0000 7f012ddba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:32:07.650 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:07 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:07.396+0000 7f012ddba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:32:07.650 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:07 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:07.657+0000 7f012ddba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:32:08.043 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:07 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:07.901+0000 7f012ddba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:32:09.043 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:08 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:08.732+0000 7f012ddba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:32:09.651 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:09 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3410254215' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T08:32:09.651 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:09.326+0000 7f012ddba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:32:09.651 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:09.430+0000 7f012ddba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:32:09.652 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:09.658+0000 7f012ddba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:32:09.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:32:09.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:32:09.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:32:09.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T08:32:09.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T08:32:09.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T08:32:09.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T08:32:09.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:09.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T08:32:09.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:09.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T08:32:09.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T08:32:09.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T08:32:09.991 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T08:32:09.992 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T08:31:55.832580+0000", 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.330+0000 7f450e387700 1 Processor -- start 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.330+0000 7f450e387700 1 -- start start 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.330+0000 7f450e387700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f4508106e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.330+0000 7f450e387700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45081073e0 con 0x7f45081069f0 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.330+0000 7f4507fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f4508106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.330+0000 7f4507fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f4508106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42628/0 (socket says 172.21.15.7:42628) 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.330+0000 7f4507fff700 1 -- 172.21.15.7:0/2511300084 learned_addr learned my addr 172.21.15.7:0/2511300084 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.331+0000 7f4507fff700 1 -- 172.21.15.7:0/2511300084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4508107bf0 con 0x7f45081069f0 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.331+0000 7f4507fff700 1 --2- 172.21.15.7:0/2511300084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f4508106e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44f80098a0 tx=0x7f44f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55867b3589ed5c4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.331+0000 7f4506ffd700 1 -- 172.21.15.7:0/2511300084 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44f8004160 con 0x7f45081069f0 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.331+0000 7f4506ffd700 1 -- 172.21.15.7:0/2511300084 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44f8004300 con 0x7f45081069f0 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.331+0000 7f4506ffd700 1 -- 172.21.15.7:0/2511300084 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44f8011b50 con 0x7f45081069f0 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.332+0000 7f450e387700 1 -- 172.21.15.7:0/2511300084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 msgr2=0x7f4508106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.332+0000 7f450e387700 1 --2- 172.21.15.7:0/2511300084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f4508106e10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f44f80098a0 tx=0x7f44f8009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.332+0000 7f450e387700 1 -- 172.21.15.7:0/2511300084 shutdown_connections 2024-03-26T08:32:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.332+0000 7f450e387700 1 --2- 172.21.15.7:0/2511300084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f4508106e10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.332+0000 7f450e387700 1 -- 172.21.15.7:0/2511300084 >> 172.21.15.7:0/2511300084 conn(0x7f4508101fb0 msgr2=0x7f45081043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.332+0000 7f450e387700 1 -- 172.21.15.7:0/2511300084 shutdown_connections 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.332+0000 7f450e387700 1 -- 172.21.15.7:0/2511300084 wait complete. 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.332+0000 7f450e387700 1 Processor -- start 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.332+0000 7f450e387700 1 -- start start 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f450e387700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f450819dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f450e387700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450810afa0 con 0x7f45081069f0 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f4507fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f450819dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f4507fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f450819dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42638/0 (socket says 172.21.15.7:42638) 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f4507fff700 1 -- 172.21.15.7:0/3410254215 learned_addr learned my addr 172.21.15.7:0/3410254215 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f4507fff700 1 -- 172.21.15.7:0/3410254215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450819e2e0 con 0x7f45081069f0 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f4507fff700 1 --2- 172.21.15.7:0/3410254215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f450819dda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009fb0 tx=0x7f44f8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f45057fa700 1 -- 172.21.15.7:0/3410254215 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44f80038b0 con 0x7f45081069f0 2024-03-26T08:32:09.995 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f45057fa700 1 -- 172.21.15.7:0/3410254215 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44f8011e50 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.333+0000 7f450e387700 1 -- 172.21.15.7:0/3410254215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450819e570 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.334+0000 7f45057fa700 1 -- 172.21.15.7:0/3410254215 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44f8019550 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.334+0000 7f450e387700 1 -- 172.21.15.7:0/3410254215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45081a1260 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.334+0000 7f45057fa700 1 -- 172.21.15.7:0/3410254215 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f44f8019b90 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.334+0000 7f45057fa700 1 -- 172.21.15.7:0/3410254215 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f44f8024730 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.335+0000 7f450e387700 1 -- 172.21.15.7:0/3410254215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45080637c0 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.337+0000 7f45057fa700 1 -- 172.21.15.7:0/3410254215 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f44f802dda0 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.411+0000 7f450e387700 1 -- 172.21.15.7:0/3410254215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f45080008d0 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.411+0000 7f45057fa700 1 -- 172.21.15.7:0/3410254215 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f44f8032030 con 0x7f45081069f0 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.413+0000 7f450e387700 1 -- 172.21.15.7:0/3410254215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 msgr2=0x7f450819dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.413+0000 7f450e387700 1 --2- 172.21.15.7:0/3410254215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f450819dda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009fb0 tx=0x7f44f8011bf0 comp rx=0 tx=0).stop 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.413+0000 7f450e387700 1 -- 172.21.15.7:0/3410254215 shutdown_connections 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.413+0000 7f450e387700 1 --2- 172.21.15.7:0/3410254215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45081069f0 0x7f450819dda0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.413+0000 7f450e387700 1 -- 172.21.15.7:0/3410254215 >> 172.21.15.7:0/3410254215 conn(0x7f4508101fb0 msgr2=0x7f4508102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:09.996 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.413+0000 7f450e387700 1 -- 172.21.15.7:0/3410254215 shutdown_connections 2024-03-26T08:32:09.997 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:09.413+0000 7f450e387700 1 -- 172.21.15.7:0/3410254215 wait complete. 2024-03-26T08:32:09.997 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (3/15)... 2024-03-26T08:32:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: Activating manager daemon x 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: mgrmap e2: x(active, starting, since 0.00296397s) 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: Manager daemon x is now available 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' 2024-03-26T08:32:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:10 smithi007 ceph-mon[32462]: from='mgr.14100 172.21.15.7:0/2914442397' entity='mgr.x' 2024-03-26T08:32:12.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:11 smithi007 ceph-mon[32462]: mgrmap e3: x(active, since 1.00658s) 2024-03-26T08:32:14.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:13 smithi007 ceph-mon[32462]: mgrmap e4: x(active, since 3s) 2024-03-26T08:32:15.714 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:15 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1079419904' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T08:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T08:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T08:32:21.362 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T08:32:21.363 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T08:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T08:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T08:32:21.364 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T08:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T08:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T08:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T08:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T08:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T08:32:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T08:32:21.373 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T08:31:55.832580+0000", 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T08:32:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.087+0000 7fa3da855700 1 Processor -- start 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.087+0000 7fa3da855700 1 -- start start 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.087+0000 7fa3da855700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d4107f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.087+0000 7fa3da855700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d4108510 con 0x7fa3d4107b20 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.087+0000 7fa3d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d4107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.087+0000 7fa3d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d4107f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42720/0 (socket says 172.21.15.7:42720) 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.087+0000 7fa3d3fff700 1 -- 172.21.15.7:0/1246484996 learned_addr learned my addr 172.21.15.7:0/1246484996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.088+0000 7fa3d3fff700 1 -- 172.21.15.7:0/1246484996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4108d20 con 0x7fa3d4107b20 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.088+0000 7fa3d3fff700 1 --2- 172.21.15.7:0/1246484996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d4107f40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40098a0 tx=0x7fa3c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8e480905fe2dfd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.088+0000 7fa3d2ffd700 1 -- 172.21.15.7:0/1246484996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3c4004160 con 0x7fa3d4107b20 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.088+0000 7fa3d2ffd700 1 -- 172.21.15.7:0/1246484996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3c4004300 con 0x7fa3d4107b20 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.088+0000 7fa3d2ffd700 1 -- 172.21.15.7:0/1246484996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3c4011b50 con 0x7fa3d4107b20 2024-03-26T08:32:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.089+0000 7fa3da855700 1 -- 172.21.15.7:0/1246484996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 msgr2=0x7fa3d4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.089+0000 7fa3da855700 1 --2- 172.21.15.7:0/1246484996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d4107f40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40098a0 tx=0x7fa3c4009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.089+0000 7fa3da855700 1 -- 172.21.15.7:0/1246484996 shutdown_connections 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.089+0000 7fa3da855700 1 --2- 172.21.15.7:0/1246484996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d4107f40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.089+0000 7fa3da855700 1 -- 172.21.15.7:0/1246484996 >> 172.21.15.7:0/1246484996 conn(0x7fa3d41030a0 msgr2=0x7fa3d4105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.089+0000 7fa3da855700 1 -- 172.21.15.7:0/1246484996 shutdown_connections 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.089+0000 7fa3da855700 1 -- 172.21.15.7:0/1246484996 wait complete. 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.089+0000 7fa3da855700 1 Processor -- start 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.090+0000 7fa3da855700 1 -- start start 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.090+0000 7fa3da855700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.090+0000 7fa3da855700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d410c0d0 con 0x7fa3d4107b20 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.090+0000 7fa3d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.090+0000 7fa3d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d419bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42730/0 (socket says 172.21.15.7:42730) 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.090+0000 7fa3d3fff700 1 -- 172.21.15.7:0/1079419904 learned_addr learned my addr 172.21.15.7:0/1079419904 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.090+0000 7fa3d3fff700 1 -- 172.21.15.7:0/1079419904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d419c0b0 con 0x7fa3d4107b20 2024-03-26T08:32:21.376 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.090+0000 7fa3d3fff700 1 --2- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d419bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009fb0 tx=0x7fa3c4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.091+0000 7fa3d17fa700 1 -- 172.21.15.7:0/1079419904 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3c40038b0 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.091+0000 7fa3d17fa700 1 -- 172.21.15.7:0/1079419904 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3c4011e50 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.091+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d419c340 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.091+0000 7fa3d17fa700 1 -- 172.21.15.7:0/1079419904 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3c4019550 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.091+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d419f030 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.091+0000 7fa3d17fa700 1 -- 172.21.15.7:0/1079419904 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7fa3c4019850 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.092+0000 7fa3d17fa700 1 --2- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fa3bc038e80 0x7fa3bc03b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.092+0000 7fa3d17fa700 1 -- 172.21.15.7:0/1079419904 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa3c404d2d0 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.092+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d40637c0 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.092+0000 7fa3d37fe700 1 --2- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fa3bc038e80 0x7fa3bc03b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.092+0000 7fa3d37fe700 1 --2- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fa3bc038e80 0x7fa3bc03b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006fd0 tx=0x7fa3c8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.097+0000 7fa3d17fa700 1 -- 172.21.15.7:0/1079419904 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa3c404cd30 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.391+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa3d419f200 con 0x7fa3d4107b20 2024-03-26T08:32:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.391+0000 7fa3d17fa700 1 -- 172.21.15.7:0/1079419904 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fa3c4032030 con 0x7fa3d4107b20 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fa3bc038e80 msgr2=0x7fa3bc03b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 --2- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fa3bc038e80 0x7fa3bc03b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8006fd0 tx=0x7fa3c8006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 msgr2=0x7fa3d419bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 --2- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d419bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009fb0 tx=0x7fa3c4011bf0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 shutdown_connections 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 --2- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fa3bc038e80 0x7fa3bc03b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 --2- 172.21.15.7:0/1079419904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3d4107b20 0x7fa3d419bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 >> 172.21.15.7:0/1079419904 conn(0x7fa3d41030a0 msgr2=0x7fa3d4103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 shutdown_connections 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:15.395+0000 7fa3da855700 1 -- 172.21.15.7:0/1079419904 wait complete. 2024-03-26T08:32:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-03-26T08:32:22.942 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:22 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3453148675' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T08:32:23.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:32:23.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T08:32:23.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:32:23.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T08:32:23.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.379+0000 7f7135085700 1 Processor -- start 2024-03-26T08:32:23.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.379+0000 7f7135085700 1 -- start start 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.379+0000 7f7135085700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713010aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.379+0000 7f7135085700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713007be10 con 0x7f7130108670 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.379+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713010aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.379+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713010aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44114/0 (socket says 172.21.15.7:44114) 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.379+0000 7f712ed9d700 1 -- 172.21.15.7:0/3936001422 learned_addr learned my addr 172.21.15.7:0/3936001422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.379+0000 7f712ed9d700 1 -- 172.21.15.7:0/3936001422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713010afa0 con 0x7f7130108670 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.380+0000 7f712ed9d700 1 --2- 172.21.15.7:0/3936001422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713010aa60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71200098a0 tx=0x7f7120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5526e58384e4854 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.380+0000 7f712dd9b700 1 -- 172.21.15.7:0/3936001422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7120004160 con 0x7f7130108670 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.380+0000 7f712dd9b700 1 -- 172.21.15.7:0/3936001422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7120004300 con 0x7f7130108670 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.380+0000 7f712dd9b700 1 -- 172.21.15.7:0/3936001422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7120011b50 con 0x7f7130108670 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.380+0000 7f7135085700 1 -- 172.21.15.7:0/3936001422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 msgr2=0x7f713010aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.380+0000 7f7135085700 1 --2- 172.21.15.7:0/3936001422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713010aa60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f71200098a0 tx=0x7f7120009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:23.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.380+0000 7f7135085700 1 -- 172.21.15.7:0/3936001422 shutdown_connections 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.380+0000 7f7135085700 1 --2- 172.21.15.7:0/3936001422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713010aa60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.380+0000 7f7135085700 1 -- 172.21.15.7:0/3936001422 >> 172.21.15.7:0/3936001422 conn(0x7f7130102000 msgr2=0x7f7130104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.381+0000 7f7135085700 1 -- 172.21.15.7:0/3936001422 shutdown_connections 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.381+0000 7f7135085700 1 -- 172.21.15.7:0/3936001422 wait complete. 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.381+0000 7f7135085700 1 Processor -- start 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.381+0000 7f7135085700 1 -- start start 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f7135085700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f7135085700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713010bf10 con 0x7f7130108670 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f712ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44116/0 (socket says 172.21.15.7:44116) 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f712ed9d700 1 -- 172.21.15.7:0/3453148675 learned_addr learned my addr 172.21.15.7:0/3453148675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f712ed9d700 1 -- 172.21.15.7:0/3453148675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713019c050 con 0x7f7130108670 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f712ed9d700 1 --2- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713019bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7120009fb0 tx=0x7f7120011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f7117fff700 1 -- 172.21.15.7:0/3453148675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71200038b0 con 0x7f7130108670 2024-03-26T08:32:23.213 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f7117fff700 1 -- 172.21.15.7:0/3453148675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7120011e50 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713019c2e0 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f7117fff700 1 -- 172.21.15.7:0/3453148675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7120019420 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.382+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713019efd0 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.383+0000 7f7117fff700 1 -- 172.21.15.7:0/3453148675 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f7120019720 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.383+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71300637c0 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.384+0000 7f7117fff700 1 --2- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f7118038e80 0x7f711803b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.384+0000 7f7117fff700 1 -- 172.21.15.7:0/3453148675 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f712004d2c0 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.384+0000 7f712e59c700 1 --2- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f7118038e80 0x7f711803b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.384+0000 7f712e59c700 1 --2- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f7118038e80 0x7f711803b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.389+0000 7f7117fff700 1 -- 172.21.15.7:0/3453148675 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f712000bc70 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.592+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f713019f1a0 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.593+0000 7f7117fff700 1 -- 172.21.15.7:0/3453148675 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f713019f1a0 con 0x7f7130108670 2024-03-26T08:32:23.214 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f7118038e80 msgr2=0x7f711803b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 --2- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f7118038e80 0x7f711803b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7124006fd0 tx=0x7f7124006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 msgr2=0x7f713019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 --2- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713019bb10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7120009fb0 tx=0x7f7120011bf0 comp rx=0 tx=0).stop 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 shutdown_connections 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 --2- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f7118038e80 0x7f711803b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 --2- 172.21.15.7:0/3453148675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7130108670 0x7f713019bb10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 >> 172.21.15.7:0/3453148675 conn(0x7f7130102000 msgr2=0x7f7130102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 shutdown_connections 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:22.595+0000 7f7135085700 1 -- 172.21.15.7:0/3453148675 wait complete. 2024-03-26T08:32:23.215 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-03-26T08:32:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:23 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/397862922' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-26T08:32:24.293 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:24 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: ignoring --setuser ceph since I am not root 2024-03-26T08:32:24.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:24 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: ignoring --setgroup ceph since I am not root 2024-03-26T08:32:24.907 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:24 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:24.521+0000 7f7f3da00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T08:32:24.907 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:24 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:24.615+0000 7f7f3da00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:32:25.207 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:24 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/397862922' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-26T08:32:25.208 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:24 smithi007 ceph-mon[32462]: mgrmap e5: x(active, since 14s) 2024-03-26T08:32:25.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.621+0000 7f633206e700 1 Processor -- start 2024-03-26T08:32:25.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.621+0000 7f633206e700 1 -- start start 2024-03-26T08:32:25.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.621+0000 7f633206e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c07c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:25.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.621+0000 7f633206e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c07c910 con 0x7f632c07dee0 2024-03-26T08:32:25.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.622+0000 7f632b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c07c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:25.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.622+0000 7f632b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c07c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44118/0 (socket says 172.21.15.7:44118) 2024-03-26T08:32:25.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.622+0000 7f632b7fe700 1 -- 172.21.15.7:0/1735954663 learned_addr learned my addr 172.21.15.7:0/1735954663 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:25.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.622+0000 7f632b7fe700 1 -- 172.21.15.7:0/1735954663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c07ca90 con 0x7f632c07dee0 2024-03-26T08:32:25.215 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.622+0000 7f632b7fe700 1 --2- 172.21.15.7:0/1735954663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c07c340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6324009e80 tx=0x7f632400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6765949cde975be server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.622+0000 7f632a7fc700 1 -- 172.21.15.7:0/1735954663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6324017070 con 0x7f632c07dee0 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.622+0000 7f632a7fc700 1 -- 172.21.15.7:0/1735954663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f632400b8c0 con 0x7f632c07dee0 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.622+0000 7f632a7fc700 1 -- 172.21.15.7:0/1735954663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63240045c0 con 0x7f632c07dee0 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.623+0000 7f633206e700 1 -- 172.21.15.7:0/1735954663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 msgr2=0x7f632c07c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.623+0000 7f633206e700 1 --2- 172.21.15.7:0/1735954663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c07c340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6324009e80 tx=0x7f632400b330 comp rx=0 tx=0).stop 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.623+0000 7f633206e700 1 -- 172.21.15.7:0/1735954663 shutdown_connections 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.623+0000 7f633206e700 1 --2- 172.21.15.7:0/1735954663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c07c340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.623+0000 7f633206e700 1 -- 172.21.15.7:0/1735954663 >> 172.21.15.7:0/1735954663 conn(0x7f632c103150 msgr2=0x7f632c105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.623+0000 7f633206e700 1 -- 172.21.15.7:0/1735954663 shutdown_connections 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.623+0000 7f633206e700 1 -- 172.21.15.7:0/1735954663 wait complete. 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.624+0000 7f633206e700 1 Processor -- start 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.624+0000 7f633206e700 1 -- start start 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.624+0000 7f633206e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:25.216 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.624+0000 7f633206e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c10e140 con 0x7f632c07dee0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.624+0000 7f632b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.624+0000 7f632b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c19dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44130/0 (socket says 172.21.15.7:44130) 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.624+0000 7f632b7fe700 1 -- 172.21.15.7:0/397862922 learned_addr learned my addr 172.21.15.7:0/397862922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.624+0000 7f632b7fe700 1 -- 172.21.15.7:0/397862922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c19e260 con 0x7f632c07dee0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.625+0000 7f632b7fe700 1 --2- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c19dd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6324004210 tx=0x7f632400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.625+0000 7f6328ff9700 1 -- 172.21.15.7:0/397862922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6324020020 con 0x7f632c07dee0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.625+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c19e4f0 con 0x7f632c07dee0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.625+0000 7f6328ff9700 1 -- 172.21.15.7:0/397862922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6324003b50 con 0x7f632c07dee0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.625+0000 7f6328ff9700 1 -- 172.21.15.7:0/397862922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6324017040 con 0x7f632c07dee0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.625+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c1a11d0 con 0x7f632c07dee0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.626+0000 7f6328ff9700 1 -- 172.21.15.7:0/397862922 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f6324023020 con 0x7f632c07dee0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.626+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632c0637c0 con 0x7f632c07dee0 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.626+0000 7f6328ff9700 1 --2- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f6318038ed0 0x7f631803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:25.217 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.626+0000 7f6328ff9700 1 -- 172.21.15.7:0/397862922 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f632400f050 con 0x7f632c07dee0 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.626+0000 7f632affd700 1 --2- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f6318038ed0 0x7f631803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.627+0000 7f632affd700 1 --2- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f6318038ed0 0x7f631803b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6314006fd0 tx=0x7f6314006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.631+0000 7f6328ff9700 1 -- 172.21.15.7:0/397862922 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f632402a020 con 0x7f632c07dee0 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.897+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f632c1a18c0 con 0x7f632c07dee0 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.948+0000 7f6328ff9700 1 -- 172.21.15.7:0/397862922 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6324003d70 con 0x7f632c07dee0 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.950+0000 7f6328ff9700 1 -- 172.21.15.7:0/397862922 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f6324025400 con 0x7f632c07dee0 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f6318038ed0 msgr2=0x7f631803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 --2- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f6318038ed0 0x7f631803b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6314006fd0 tx=0x7f6314006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 msgr2=0x7f632c19dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 --2- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c19dd20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6324004210 tx=0x7f632400bd50 comp rx=0 tx=0).stop 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 shutdown_connections 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 --2- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f6318038ed0 0x7f631803b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.218 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 --2- 172.21.15.7:0/397862922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c07dee0 0x7f632c19dd20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:25.219 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 >> 172.21.15.7:0/397862922 conn(0x7f632c103150 msgr2=0x7f632c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:25.219 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 shutdown_connections 2024-03-26T08:32:25.219 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:23.951+0000 7f633206e700 1 -- 172.21.15.7:0/397862922 wait complete. 2024-03-26T08:32:25.487 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:25 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:25.288+0000 7f7f3da00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:32:25.487 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:25 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:25.385+0000 7f7f3da00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:32:25.793 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:25 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:25.532+0000 7f7f3da00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:32:25.794 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:25 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:25.630+0000 7f7f3da00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:32:25.794 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:25 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:25.717+0000 7f7f3da00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:32:26.186 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:26 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2126450036' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T08:32:26.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:25 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:25.989+0000 7f7f3da00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:32:26.186 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:26 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:26.105+0000 7f7f3da00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:32:26.187 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:26 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:26.193+0000 7f7f3da00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:32:26.465 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:26 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:26.292+0000 7f7f3da00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:32:26.466 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:26 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:26.472+0000 7f7f3da00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:32:26.514 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.718+0000 7f897ac96700 1 Processor -- start 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.718+0000 7f897ac96700 1 -- start start 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.718+0000 7f897ac96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897410aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.718+0000 7f897ac96700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8974075840 con 0x7f8974108620 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.719+0000 7f8978a32700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897410aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.719+0000 7f8978a32700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897410aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44152/0 (socket says 172.21.15.7:44152) 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.719+0000 7f8978a32700 1 -- 172.21.15.7:0/884922816 learned_addr learned my addr 172.21.15.7:0/884922816 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.719+0000 7f8978a32700 1 -- 172.21.15.7:0/884922816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897410af50 con 0x7f8974108620 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.719+0000 7f8978a32700 1 --2- 172.21.15.7:0/884922816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897410aa10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f89680098a0 tx=0x7f8968009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272b4ef2fbebd6b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:26.515 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.719+0000 7f89737fe700 1 -- 172.21.15.7:0/884922816 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8968004160 con 0x7f8974108620 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.719+0000 7f89737fe700 1 -- 172.21.15.7:0/884922816 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8968004300 con 0x7f8974108620 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.719+0000 7f89737fe700 1 -- 172.21.15.7:0/884922816 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8968011b50 con 0x7f8974108620 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.720+0000 7f897ac96700 1 -- 172.21.15.7:0/884922816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 msgr2=0x7f897410aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.720+0000 7f897ac96700 1 --2- 172.21.15.7:0/884922816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897410aa10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f89680098a0 tx=0x7f8968009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.720+0000 7f897ac96700 1 -- 172.21.15.7:0/884922816 shutdown_connections 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.720+0000 7f897ac96700 1 --2- 172.21.15.7:0/884922816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897410aa10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.720+0000 7f897ac96700 1 -- 172.21.15.7:0/884922816 >> 172.21.15.7:0/884922816 conn(0x7f8974101fb0 msgr2=0x7f89741043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.720+0000 7f897ac96700 1 -- 172.21.15.7:0/884922816 shutdown_connections 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.720+0000 7f897ac96700 1 -- 172.21.15.7:0/884922816 wait complete. 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.721+0000 7f897ac96700 1 Processor -- start 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.721+0000 7f897ac96700 1 -- start start 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.721+0000 7f897ac96700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.721+0000 7f897ac96700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897410bec0 con 0x7f8974108620 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.721+0000 7f8978a32700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.721+0000 7f8978a32700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44166/0 (socket says 172.21.15.7:44166) 2024-03-26T08:32:26.516 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.721+0000 7f8978a32700 1 -- 172.21.15.7:0/2126450036 learned_addr learned my addr 172.21.15.7:0/2126450036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.721+0000 7f8978a32700 1 -- 172.21.15.7:0/2126450036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897419c060 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.721+0000 7f8978a32700 1 --2- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897419bb20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8968009fb0 tx=0x7f8968011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.722+0000 7f8971ffb700 1 -- 172.21.15.7:0/2126450036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89680038b0 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.722+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897419c2f0 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.722+0000 7f8971ffb700 1 -- 172.21.15.7:0/2126450036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8968011e50 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.722+0000 7f8971ffb700 1 -- 172.21.15.7:0/2126450036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8968019420 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.722+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897419efe0 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.723+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89740637c0 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.723+0000 7f8971ffb700 1 -- 172.21.15.7:0/2126450036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f8968003a50 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.723+0000 7f8971ffb700 1 --2- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f8964038ed0 0x7f896403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.723+0000 7f8971ffb700 1 -- 172.21.15.7:0/2126450036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f896804d040 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.723+0000 7f8973fff700 1 -- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f8964038ed0 msgr2=0x7f896403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3110430694 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.723+0000 7f8973fff700 1 --2- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f8964038ed0 0x7f896403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.728+0000 7f8971ffb700 1 -- 172.21.15.7:0/2126450036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8968019720 con 0x7f8974108620 2024-03-26T08:32:26.517 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.924+0000 7f8973fff700 1 -- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f8964038ed0 msgr2=0x7f896403b390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3110430694 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.924+0000 7f8973fff700 1 --2- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f8964038ed0 0x7f896403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.992+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f897419f620 con 0x7f8974108620 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.992+0000 7f8971ffb700 1 -- 172.21.15.7:0/2126450036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f896804f110 con 0x7f8974108620 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f8964038ed0 msgr2=0x7f896403b390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 --2- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f8964038ed0 0x7f896403b390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 msgr2=0x7f897419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 --2- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897419bb20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8968009fb0 tx=0x7f8968011bf0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 shutdown_connections 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 --2- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7f8964038ed0 0x7f896403b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 --2- 172.21.15.7:0/2126450036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8974108620 0x7f897419bb20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 >> 172.21.15.7:0/2126450036 conn(0x7f8974101fb0 msgr2=0x7f8974102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 shutdown_connections 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:25.995+0000 7f897ac96700 1 -- 172.21.15.7:0/2126450036 wait complete. 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-03-26T08:32:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 5... 2024-03-26T08:32:27.793 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:27 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:27.477+0000 7f7f3da00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:32:27.793 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:27 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:27.611+0000 7f7f3da00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:32:28.234 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:27 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:27.949+0000 7f7f3da00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:32:28.235 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:28 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:28.040+0000 7f7f3da00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:32:28.235 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:28 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:28.241+0000 7f7f3da00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:32:28.543 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:28 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:28.431+0000 7f7f3da00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:32:28.976 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:28 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:28.722+0000 7f7f3da00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:32:28.976 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:28 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:28.983+0000 7f7f3da00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:32:29.293 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:29 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:29.226+0000 7f7f3da00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:32:30.293 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:30 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:30.056+0000 7f7f3da00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:32:30.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:30 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:30.639+0000 7f7f3da00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:32:30.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:30 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:30.739+0000 7f7f3da00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:32:30.955 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:30 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:30.962+0000 7f7f3da00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:32:31.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: Active manager daemon x restarted 2024-03-26T08:32:31.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: Activating manager daemon x 2024-03-26T08:32:31.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: osdmap e2: 0 total, 0 up, 0 in 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: mgrmap e6: x(active, starting, since 0.00326353s) 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: Manager daemon x is now available 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:31.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:31 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:32.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:32 smithi007 ceph-mon[32462]: Found migration_current of "None". Setting to last migration. 2024-03-26T08:32:32.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:32 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T08:32:32.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:32 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T08:32:32.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:32 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:32.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:32 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:32.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:32 smithi007 ceph-mon[32462]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T08:32:32.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:32 smithi007 ceph-mon[32462]: mgrmap e7: x(active, since 1.00524s) 2024-03-26T08:32:32.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:32 smithi007 ceph-mon[32462]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T08:32:32.521 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:32:32.521 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-26T08:32:32.521 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.956+0000 7fe441605700 1 Processor -- start 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.956+0000 7fe441605700 1 -- start start 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.956+0000 7fe441605700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.956+0000 7fe441605700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c07c8f0 con 0x7fe43c07de70 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.956+0000 7fe43affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.956+0000 7fe43affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37264/0 (socket says 172.21.15.7:37264) 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.956+0000 7fe43affd700 1 -- 172.21.15.7:0/1941734968 learned_addr learned my addr 172.21.15.7:0/1941734968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.957+0000 7fe43affd700 1 -- 172.21.15.7:0/1941734968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c07ca70 con 0x7fe43c07de70 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.957+0000 7fe43affd700 1 --2- 172.21.15.7:0/1941734968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c07c320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe4300098a0 tx=0x7fe430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29f73dd3a649af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.957+0000 7fe439ffb700 1 -- 172.21.15.7:0/1941734968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe430004160 con 0x7fe43c07de70 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.957+0000 7fe439ffb700 1 -- 172.21.15.7:0/1941734968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe430004300 con 0x7fe43c07de70 2024-03-26T08:32:32.522 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.957+0000 7fe439ffb700 1 -- 172.21.15.7:0/1941734968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe430011b50 con 0x7fe43c07de70 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.958+0000 7fe441605700 1 -- 172.21.15.7:0/1941734968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 msgr2=0x7fe43c07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.958+0000 7fe441605700 1 --2- 172.21.15.7:0/1941734968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c07c320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe4300098a0 tx=0x7fe430009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.958+0000 7fe441605700 1 -- 172.21.15.7:0/1941734968 shutdown_connections 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.958+0000 7fe441605700 1 --2- 172.21.15.7:0/1941734968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c07c320 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.958+0000 7fe441605700 1 -- 172.21.15.7:0/1941734968 >> 172.21.15.7:0/1941734968 conn(0x7fe43c1030d0 msgr2=0x7fe43c105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.958+0000 7fe441605700 1 -- 172.21.15.7:0/1941734968 shutdown_connections 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.958+0000 7fe441605700 1 -- 172.21.15.7:0/1941734968 wait complete. 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.959+0000 7fe441605700 1 Processor -- start 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.959+0000 7fe441605700 1 -- start start 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.959+0000 7fe441605700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.959+0000 7fe441605700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c10e100 con 0x7fe43c07de70 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.959+0000 7fe43affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.959+0000 7fe43affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c19dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37266/0 (socket says 172.21.15.7:37266) 2024-03-26T08:32:32.523 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.959+0000 7fe43affd700 1 -- 172.21.15.7:0/3643116346 learned_addr learned my addr 172.21.15.7:0/3643116346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.960+0000 7fe43affd700 1 -- 172.21.15.7:0/3643116346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c19e250 con 0x7fe43c07de70 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.960+0000 7fe43affd700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c19dd10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe430011670 tx=0x7fe430011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.960+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4300039c0 con 0x7fe43c07de70 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.960+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe430011e50 con 0x7fe43c07de70 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.960+0000 7fe441605700 1 -- 172.21.15.7:0/3643116346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43c19e4e0 con 0x7fe43c07de70 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.960+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe43001a5c0 con 0x7fe43c07de70 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.960+0000 7fe441605700 1 -- 172.21.15.7:0/3643116346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43c1a11c0 con 0x7fe43c07de70 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.961+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fe43001a870 con 0x7fe43c07de70 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.961+0000 7fe423fff700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 0x7fe42403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.961+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 --> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe42403baf0 con 0x7fe424038f20 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.961+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe43004d190 con 0x7fe43c07de70 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.961+0000 7fe43a7fc700 1 -- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 msgr2=0x7fe42403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3110430694 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:26.961+0000 7fe43a7fc700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 0x7fe42403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:32:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:27.162+0000 7fe43a7fc700 1 -- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 msgr2=0x7fe42403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3110430694 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:27.162+0000 7fe43a7fc700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 0x7fe42403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:27.563+0000 7fe43a7fc700 1 -- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 msgr2=0x7fe42403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3110430694 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:27.563+0000 7fe43a7fc700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 0x7fe42403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:28.364+0000 7fe43a7fc700 1 -- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 msgr2=0x7fe42403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3110430694 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:28.364+0000 7fe43a7fc700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 0x7fe42403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:29.965+0000 7fe43a7fc700 1 -- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 msgr2=0x7fe42403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/3110430694 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:29.965+0000 7fe43a7fc700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 0x7fe42403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:30.977+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7fe43001abb0 con 0x7fe43c07de70 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:30.977+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 msgr2=0x7fe42403b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:30.977+0000 7fe423fff700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 0x7fe42403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.978+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7fe43004ed60 con 0x7fe43c07de70 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.978+0000 7fe423fff700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe42403c810 0x7fe42403ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.978+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe43000f940 con 0x7fe42403c810 2024-03-26T08:32:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.978+0000 7fe43a7fc700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe42403c810 0x7fe42403ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.979+0000 7fe43a7fc700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe42403c810 0x7fe42403ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe42c003a10 tx=0x7fe42c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.979+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 <== mgr.14120 v2:172.21.15.7:6800/1336720039 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fe43000f940 con 0x7fe42403c810 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 -- 172.21.15.7:0/3643116346 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe43c0637c0 con 0x7fe42403c810 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe423fff700 1 -- 172.21.15.7:0/3643116346 <== mgr.14120 v2:172.21.15.7:6800/1336720039 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe43c0637c0 con 0x7fe42403c810 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 -- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe42403c810 msgr2=0x7fe42403ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe42403c810 0x7fe42403ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe42c003a10 tx=0x7fe42c0092b0 comp rx=0 tx=0).stop 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 -- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 msgr2=0x7fe43c19dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c19dd10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe430011670 tx=0x7fe430011b60 comp rx=0 tx=0).stop 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 -- 172.21.15.7:0/3643116346 shutdown_connections 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe42403c810 0x7fe42403ec00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe43c07de70 0x7fe43c19dd10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:32.526 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 --2- 172.21.15.7:0/3643116346 >> [v2:172.21.15.7:6800/3110430694,v1:172.21.15.7:6801/3110430694] conn(0x7fe424038f20 0x7fe42403b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:32.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.986+0000 7fe441605700 1 -- 172.21.15.7:0/3643116346 >> 172.21.15.7:0/3643116346 conn(0x7fe43c1030d0 msgr2=0x7fe43c103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:32.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.987+0000 7fe441605700 1 -- 172.21.15.7:0/3643116346 shutdown_connections 2024-03-26T08:32:32.527 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:31.987+0000 7fe441605700 1 -- 172.21.15.7:0/3643116346 wait complete. 2024-03-26T08:32:32.527 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 5 is available 2024-03-26T08:32:32.527 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-03-26T08:32:33.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:33 smithi007 ceph-mon[32462]: [26/Mar/2024:08:32:32] ENGINE Bus STARTING 2024-03-26T08:32:33.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:33 smithi007 ceph-mon[32462]: [26/Mar/2024:08:32:32] ENGINE Serving on http://172.21.15.7:8765 2024-03-26T08:32:33.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:33 smithi007 ceph-mon[32462]: [26/Mar/2024:08:32:32] ENGINE Serving on https://172.21.15.7:7150 2024-03-26T08:32:33.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:33 smithi007 ceph-mon[32462]: [26/Mar/2024:08:32:32] ENGINE Bus STARTED 2024-03-26T08:32:33.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:33 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:33.784 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.957+0000 7f7ce702c700 1 Processor -- start 2024-03-26T08:32:33.784 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.957+0000 7f7ce702c700 1 -- start start 2024-03-26T08:32:33.784 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.957+0000 7f7ce702c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.957+0000 7f7ce702c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce007c8f0 con 0x7f7ce007dec0 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.957+0000 7f7ce4dc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.957+0000 7f7ce4dc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37352/0 (socket says 172.21.15.7:37352) 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.957+0000 7f7ce4dc8700 1 -- 172.21.15.7:0/3031921508 learned_addr learned my addr 172.21.15.7:0/3031921508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.958+0000 7f7ce4dc8700 1 -- 172.21.15.7:0/3031921508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce007ca70 con 0x7f7ce007dec0 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.958+0000 7f7ce4dc8700 1 --2- 172.21.15.7:0/3031921508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce007c320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40098a0 tx=0x7f7cd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29868aef59fd65a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.958+0000 7f7cdf7fe700 1 -- 172.21.15.7:0/3031921508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd4004160 con 0x7f7ce007dec0 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.958+0000 7f7cdf7fe700 1 -- 172.21.15.7:0/3031921508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cd4004300 con 0x7f7ce007dec0 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.958+0000 7f7cdf7fe700 1 -- 172.21.15.7:0/3031921508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd4011b50 con 0x7f7ce007dec0 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.958+0000 7f7ce702c700 1 -- 172.21.15.7:0/3031921508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 msgr2=0x7f7ce007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.958+0000 7f7ce702c700 1 --2- 172.21.15.7:0/3031921508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce007c320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40098a0 tx=0x7f7cd4009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce702c700 1 -- 172.21.15.7:0/3031921508 shutdown_connections 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce702c700 1 --2- 172.21.15.7:0/3031921508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce007c320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce702c700 1 -- 172.21.15.7:0/3031921508 >> 172.21.15.7:0/3031921508 conn(0x7f7ce0103150 msgr2=0x7f7ce0105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce702c700 1 -- 172.21.15.7:0/3031921508 shutdown_connections 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce702c700 1 -- 172.21.15.7:0/3031921508 wait complete. 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce702c700 1 Processor -- start 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce702c700 1 -- start start 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce702c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce702c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce010e160 con 0x7f7ce007dec0 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce4dc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce4dc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37358/0 (socket says 172.21.15.7:37358) 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.959+0000 7f7ce4dc8700 1 -- 172.21.15.7:0/3030486767 learned_addr learned my addr 172.21.15.7:0/3030486767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.960+0000 7f7ce4dc8700 1 -- 172.21.15.7:0/3030486767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce019e2e0 con 0x7f7ce007dec0 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.960+0000 7f7ce4dc8700 1 --2- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce019dda0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009fb0 tx=0x7f7cd4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.960+0000 7f7cddffb700 1 -- 172.21.15.7:0/3030486767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd40038b0 con 0x7f7ce007dec0 2024-03-26T08:32:33.786 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.960+0000 7f7cddffb700 1 -- 172.21.15.7:0/3030486767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cd4011e50 con 0x7f7ce007dec0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.960+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce019e570 con 0x7f7ce007dec0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.960+0000 7f7cddffb700 1 -- 172.21.15.7:0/3030486767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cd4019550 con 0x7f7ce007dec0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.960+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce01a1220 con 0x7f7ce007dec0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.961+0000 7f7cddffb700 1 -- 172.21.15.7:0/3030486767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7f7cd4019800 con 0x7f7ce007dec0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.961+0000 7f7cddffb700 1 --2- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7cd0038e10 0x7f7cd003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.961+0000 7f7cddffb700 1 -- 172.21.15.7:0/3030486767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7cd404ceb0 con 0x7f7ce007dec0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.961+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce00637c0 con 0x7f7ce007dec0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.961+0000 7f7cdffff700 1 --2- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7cd0038e10 0x7f7cd003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.962+0000 7f7cdffff700 1 --2- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7cd0038e10 0x7f7cd003b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006fd0 tx=0x7f7ccc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:32.967+0000 7f7cddffb700 1 -- 172.21.15.7:0/3030486767 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7cd402d960 con 0x7f7ce007dec0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.029+0000 7f7cddffb700 1 -- 172.21.15.7:0/3030486767 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f7cd402d440 con 0x7f7ce007dec0 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.191+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ce01043c0 con 0x7f7cd0038e10 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.196+0000 7f7cddffb700 1 -- 172.21.15.7:0/3030486767 <== mgr.14120 v2:172.21.15.7:6800/1336720039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7ce01043c0 con 0x7f7cd0038e10 2024-03-26T08:32:33.787 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7cd0038e10 msgr2=0x7f7cd003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:33.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 --2- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7cd0038e10 0x7f7cd003b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc006fd0 tx=0x7f7ccc006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:33.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 msgr2=0x7f7ce019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:33.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 --2- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce019dda0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009fb0 tx=0x7f7cd4011bf0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 shutdown_connections 2024-03-26T08:32:33.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 --2- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7cd0038e10 0x7f7cd003b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 --2- 172.21.15.7:0/3030486767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce007dec0 0x7f7ce019dda0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:33.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 >> 172.21.15.7:0/3030486767 conn(0x7f7ce0103150 msgr2=0x7f7ce0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:33.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 shutdown_connections 2024-03-26T08:32:33.788 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:33.199+0000 7f7ce702c700 1 -- 172.21.15.7:0/3030486767 wait complete. 2024-03-26T08:32:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:34 smithi007 ceph-mon[32462]: mgrmap e8: x(active, since 2s) 2024-03-26T08:32:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:34 smithi007 ceph-mon[32462]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:32:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:34 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:34 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:35.017 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.205+0000 7f9cd0bc6700 1 Processor -- start 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.206+0000 7f9cd0bc6700 1 -- start start 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.206+0000 7f9cd0bc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc10aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.206+0000 7f9cd0bc6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc07be70 con 0x7f9ccc108610 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.206+0000 7f9cca59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc10aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.206+0000 7f9cca59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc10aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37374/0 (socket says 172.21.15.7:37374) 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.206+0000 7f9cca59c700 1 -- 172.21.15.7:0/2204407603 learned_addr learned my addr 172.21.15.7:0/2204407603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.206+0000 7f9cca59c700 1 -- 172.21.15.7:0/2204407603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc10af40 con 0x7f9ccc108610 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.206+0000 7f9cca59c700 1 --2- 172.21.15.7:0/2204407603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc10aa00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40098a0 tx=0x7f9cb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a7bbdb059fadd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.207+0000 7f9cc959a700 1 -- 172.21.15.7:0/2204407603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cb4004160 con 0x7f9ccc108610 2024-03-26T08:32:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.207+0000 7f9cc959a700 1 -- 172.21.15.7:0/2204407603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cb4004300 con 0x7f9ccc108610 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.207+0000 7f9cc959a700 1 -- 172.21.15.7:0/2204407603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cb4011b50 con 0x7f9ccc108610 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.207+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/2204407603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 msgr2=0x7f9ccc10aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.207+0000 7f9cd0bc6700 1 --2- 172.21.15.7:0/2204407603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc10aa00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40098a0 tx=0x7f9cb4009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.207+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/2204407603 shutdown_connections 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.207+0000 7f9cd0bc6700 1 --2- 172.21.15.7:0/2204407603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc10aa00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.207+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/2204407603 >> 172.21.15.7:0/2204407603 conn(0x7f9ccc102000 msgr2=0x7f9ccc104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.207+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/2204407603 shutdown_connections 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.208+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/2204407603 wait complete. 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.208+0000 7f9cd0bc6700 1 Processor -- start 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.208+0000 7f9cd0bc6700 1 -- start start 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.208+0000 7f9cd0bc6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.208+0000 7f9cd0bc6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc10bf10 con 0x7f9ccc108610 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cca59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cca59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc19bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37390/0 (socket says 172.21.15.7:37390) 2024-03-26T08:32:35.019 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cca59c700 1 -- 172.21.15.7:0/3345656415 learned_addr learned my addr 172.21.15.7:0/3345656415 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cca59c700 1 -- 172.21.15.7:0/3345656415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc19c080 con 0x7f9ccc108610 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cca59c700 1 --2- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc19bb40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4011b60 tx=0x7f9cb4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cc37fe700 1 -- 172.21.15.7:0/3345656415 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cb4003970 con 0x7f9ccc108610 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cc37fe700 1 -- 172.21.15.7:0/3345656415 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cb4003b10 con 0x7f9ccc108610 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc19c310 con 0x7f9ccc108610 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cc37fe700 1 -- 172.21.15.7:0/3345656415 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cb4024610 con 0x7f9ccc108610 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.209+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc19f000 con 0x7f9ccc108610 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.210+0000 7f9cc37fe700 1 -- 172.21.15.7:0/3345656415 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f9cb4024910 con 0x7f9ccc108610 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.211+0000 7f9cc37fe700 1 --2- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f9cb8038f20 0x7f9cb803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.211+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc0637c0 con 0x7f9ccc108610 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.211+0000 7f9cc37fe700 1 -- 172.21.15.7:0/3345656415 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9cb4011e00 con 0x7f9ccc108610 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.211+0000 7f9cc9d9b700 1 --2- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f9cb8038f20 0x7f9cb803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.211+0000 7f9cc9d9b700 1 --2- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f9cb8038f20 0x7f9cb803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006fd0 tx=0x7f9cbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.216+0000 7f9cc37fe700 1 -- 172.21.15.7:0/3345656415 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9cb400ba00 con 0x7f9ccc108610 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.420+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ccc1032c0 con 0x7f9cb8038f20 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.421+0000 7f9cc37fe700 1 -- 172.21.15.7:0/3345656415 <== mgr.14120 v2:172.21.15.7:6800/1336720039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f9ccc1032c0 con 0x7f9cb8038f20 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f9cb8038f20 msgr2=0x7f9cb803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 --2- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f9cb8038f20 0x7f9cb803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006fd0 tx=0x7f9cbc006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 msgr2=0x7f9ccc19bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 --2- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc19bb40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4011b60 tx=0x7f9cb4011c40 comp rx=0 tx=0).stop 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 shutdown_connections 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 --2- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f9cb8038f20 0x7f9cb803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:35.021 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 --2- 172.21.15.7:0/3345656415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ccc108610 0x7f9ccc19bb40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:35.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 >> 172.21.15.7:0/3345656415 conn(0x7f9ccc102000 msgr2=0x7f9ccc102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:35.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 shutdown_connections 2024-03-26T08:32:35.022 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:34.423+0000 7f9cd0bc6700 1 -- 172.21.15.7:0/3345656415 wait complete. 2024-03-26T08:32:35.022 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-03-26T08:32:35.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:35 smithi007 ceph-mon[32462]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:32:36.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-mon[32462]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:32:36.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-mon[32462]: Generating ssh key... 2024-03-26T08:32:36.293 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: Generating public/private rsa key pair. 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: Your identification has been saved in /tmp/tmp77jg93j8/key. 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: Your public key has been saved in /tmp/tmp77jg93j8/key.pub. 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: The key fingerprint is: 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: SHA256:Wti9vfnZmC2wPFubhnuOladVku7mOnkZxZ0314hSkUs ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: The key's randomart image is: 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: +---[RSA 3072]----+ 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: | .o | 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: | E | 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: | o o o+| 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: | o .. o .+B| 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: | . S .. o.=| 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: | o o...o.| 2024-03-26T08:32:36.294 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: | . ...==oo| 2024-03-26T08:32:36.295 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: | BBB% | 2024-03-26T08:32:36.295 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: | B#%.o| 2024-03-26T08:32:36.295 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: +----[SHA256]-----+ 2024-03-26T08:32:36.805 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.503+0000 7fde6913d700 1 Processor -- start 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.503+0000 7fde6913d700 1 -- start start 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.503+0000 7fde6913d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde64107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.503+0000 7fde6913d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64108540 con 0x7fde64107b50 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.503+0000 7fde62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde64107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.503+0000 7fde62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde64107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37396/0 (socket says 172.21.15.7:37396) 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.503+0000 7fde62d9d700 1 -- 172.21.15.7:0/2017798583 learned_addr learned my addr 172.21.15.7:0/2017798583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.503+0000 7fde62d9d700 1 -- 172.21.15.7:0/2017798583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde64108d50 con 0x7fde64107b50 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.503+0000 7fde62d9d700 1 --2- 172.21.15.7:0/2017798583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde64107f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fde540098a0 tx=0x7fde54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ad99c777e0ebba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde61d9b700 1 -- 172.21.15.7:0/2017798583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde54004160 con 0x7fde64107b50 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde61d9b700 1 -- 172.21.15.7:0/2017798583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde54004300 con 0x7fde64107b50 2024-03-26T08:32:36.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde61d9b700 1 -- 172.21.15.7:0/2017798583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde54011b50 con 0x7fde64107b50 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde6913d700 1 -- 172.21.15.7:0/2017798583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 msgr2=0x7fde64107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde6913d700 1 --2- 172.21.15.7:0/2017798583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde64107f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fde540098a0 tx=0x7fde54009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde6913d700 1 -- 172.21.15.7:0/2017798583 shutdown_connections 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde6913d700 1 --2- 172.21.15.7:0/2017798583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde64107f70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde6913d700 1 -- 172.21.15.7:0/2017798583 >> 172.21.15.7:0/2017798583 conn(0x7fde64103150 msgr2=0x7fde64105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde6913d700 1 -- 172.21.15.7:0/2017798583 shutdown_connections 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.504+0000 7fde6913d700 1 -- 172.21.15.7:0/2017798583 wait complete. 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.505+0000 7fde6913d700 1 Processor -- start 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.505+0000 7fde6913d700 1 -- start start 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.505+0000 7fde6913d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde6419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde6913d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6410c100 con 0x7fde64107b50 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde6419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde6419bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37402/0 (socket says 172.21.15.7:37402) 2024-03-26T08:32:36.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde62d9d700 1 -- 172.21.15.7:0/897386509 learned_addr learned my addr 172.21.15.7:0/897386509 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde62d9d700 1 -- 172.21.15.7:0/897386509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6419c0b0 con 0x7fde64107b50 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde62d9d700 1 --2- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde6419bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde54009fb0 tx=0x7fde54011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde4bfff700 1 -- 172.21.15.7:0/897386509 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde540038b0 con 0x7fde64107b50 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde4bfff700 1 -- 172.21.15.7:0/897386509 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde54011e50 con 0x7fde64107b50 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6419c340 con 0x7fde64107b50 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde4bfff700 1 -- 172.21.15.7:0/897386509 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde54019550 con 0x7fde64107b50 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.506+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6419f030 con 0x7fde64107b50 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.507+0000 7fde4bfff700 1 -- 172.21.15.7:0/897386509 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fde54019850 con 0x7fde64107b50 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.508+0000 7fde4bfff700 1 --2- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fde4c038ed0 0x7fde4c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.508+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde640637c0 con 0x7fde64107b50 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.508+0000 7fde4bfff700 1 -- 172.21.15.7:0/897386509 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fde5404d120 con 0x7fde64107b50 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.508+0000 7fde6259c700 1 --2- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fde4c038ed0 0x7fde4c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.508+0000 7fde6259c700 1 --2- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fde4c038ed0 0x7fde4c03b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fde58006fd0 tx=0x7fde58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:36.808 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.513+0000 7fde4bfff700 1 -- 172.21.15.7:0/897386509 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde5402d960 con 0x7fde64107b50 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:35.717+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fde641043f0 con 0x7fde4c038ed0 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.190+0000 7fde4bfff700 1 -- 172.21.15.7:0/897386509 <== mgr.14120 v2:172.21.15.7:6800/1336720039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fde641043f0 con 0x7fde4c038ed0 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.192+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fde4c038ed0 msgr2=0x7fde4c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.193+0000 7fde6913d700 1 --2- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fde4c038ed0 0x7fde4c03b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fde58006fd0 tx=0x7fde58006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.193+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 msgr2=0x7fde6419bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.193+0000 7fde6913d700 1 --2- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde6419bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fde54009fb0 tx=0x7fde54011bf0 comp rx=0 tx=0).stop 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.193+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 shutdown_connections 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.193+0000 7fde6913d700 1 --2- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fde4c038ed0 0x7fde4c03b390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.193+0000 7fde6913d700 1 --2- 172.21.15.7:0/897386509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde64107b50 0x7fde6419bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.193+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 >> 172.21.15.7:0/897386509 conn(0x7fde64103150 msgr2=0x7fde64103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.193+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 shutdown_connections 2024-03-26T08:32:36.809 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:36.193+0000 7fde6913d700 1 -- 172.21.15.7:0/897386509 wait complete. 2024-03-26T08:32:37.440 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:37 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:37.441 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:37 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:37.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiKC2ooCq7HBZoXjhgLK6JF3NuoKJKH954cD9E6tu5+zVCzm5dkdubdUMWJkxt+EXqr012AdE67VybhkijY0JnOTmg65aOzqEc5Git76KU+rbWvVI7Vr7XjPyrbLjqngGuRk+JXZT2YUyLExUbaVVn4zxSnfixOPtQN0D/5DQHxC40XroVQOEY3YcDa0O+82KE2ZMy/OB1dbGddRmGIKjzhMaReyEg2hjq5ZpLo5vUqGf0/favTvlq54scRsWqCzGwq4SpIC/2VjfuV3PL884ilD9btK47ygoFAkb0H+xhtA9uOwPSg/q+XNk3S+6le/Zbv9msXw14oWhek4qP0KqmM6TNPwjKRPEEtPsNd+Fg15t1ViImoqbyC09mD0pxXVWfaM/CYrKG8S8W9i4lH3g1HENFd7lSfP2djSu6wDZWUPqXcvVm/Qkqk7+xp/S7kNDKVbKx4ooCT5mEWDh7wO9ciJY9CueJ/AEeOMjrUG7Q4yo7j8MrcR8wMW2rdSNzV2U= ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:32:37.962 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.230+0000 7f39effff700 1 Processor -- start 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.230+0000 7f39effff700 1 -- start start 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f010a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39effff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f007bed0 con 0x7f39f01083c0 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f010a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f010a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52552/0 (socket says 172.21.15.7:52552) 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39eeffd700 1 -- 172.21.15.7:0/3297213973 learned_addr learned my addr 172.21.15.7:0/3297213973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39eeffd700 1 -- 172.21.15.7:0/3297213973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f010acf0 con 0x7f39f01083c0 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39eeffd700 1 --2- 172.21.15.7:0/3297213973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f010a7b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39e00098a0 tx=0x7f39e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad723c8518004c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39edffb700 1 -- 172.21.15.7:0/3297213973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e0004160 con 0x7f39f01083c0 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39edffb700 1 -- 172.21.15.7:0/3297213973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39e0004300 con 0x7f39f01083c0 2024-03-26T08:32:37.963 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.231+0000 7f39edffb700 1 -- 172.21.15.7:0/3297213973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e0011b50 con 0x7f39f01083c0 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.232+0000 7f39effff700 1 -- 172.21.15.7:0/3297213973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 msgr2=0x7f39f010a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.232+0000 7f39effff700 1 --2- 172.21.15.7:0/3297213973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f010a7b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f39e00098a0 tx=0x7f39e0009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.232+0000 7f39effff700 1 -- 172.21.15.7:0/3297213973 shutdown_connections 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.232+0000 7f39effff700 1 --2- 172.21.15.7:0/3297213973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f010a7b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.232+0000 7f39effff700 1 -- 172.21.15.7:0/3297213973 >> 172.21.15.7:0/3297213973 conn(0x7f39f0101e30 msgr2=0x7f39f0104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.232+0000 7f39effff700 1 -- 172.21.15.7:0/3297213973 shutdown_connections 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.232+0000 7f39effff700 1 -- 172.21.15.7:0/3297213973 wait complete. 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.233+0000 7f39effff700 1 Processor -- start 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.233+0000 7f39effff700 1 -- start start 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.233+0000 7f39effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.233+0000 7f39effff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f010bd20 con 0x7f39f01083c0 2024-03-26T08:32:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52566/0 (socket says 172.21.15.7:52566) 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39eeffd700 1 -- 172.21.15.7:0/2941597359 learned_addr learned my addr 172.21.15.7:0/2941597359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39eeffd700 1 -- 172.21.15.7:0/2941597359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f019be40 con 0x7f39f01083c0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39eeffd700 1 --2- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f019b900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39e0011b60 tx=0x7f39e0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39f49ff700 1 -- 172.21.15.7:0/2941597359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e0003b20 con 0x7f39f01083c0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39f49ff700 1 -- 172.21.15.7:0/2941597359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39e0003cc0 con 0x7f39f01083c0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f019c0d0 con 0x7f39f01083c0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39f49ff700 1 -- 172.21.15.7:0/2941597359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39e0024840 con 0x7f39f01083c0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.234+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f019edc0 con 0x7f39f01083c0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.235+0000 7f39f49ff700 1 -- 172.21.15.7:0/2941597359 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f39e0022020 con 0x7f39f01083c0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.235+0000 7f39f49ff700 1 --2- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f39d8038f70 0x7f39d803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.235+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f00637c0 con 0x7f39f01083c0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.235+0000 7f39f49ff700 1 -- 172.21.15.7:0/2941597359 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f39e0011e00 con 0x7f39f01083c0 2024-03-26T08:32:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.235+0000 7f39ee7fc700 1 --2- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f39d8038f70 0x7f39d803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.236+0000 7f39ee7fc700 1 --2- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f39d8038f70 0x7f39d803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f39e4006fd0 tx=0x7f39e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.241+0000 7f39f49ff700 1 -- 172.21.15.7:0/2941597359 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f39e000b7e0 con 0x7f39f01083c0 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.447+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f39f0103050 con 0x7f39d8038f70 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.447+0000 7f39f49ff700 1 -- 172.21.15.7:0/2941597359 <== mgr.14120 v2:172.21.15.7:6800/1336720039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f39f0103050 con 0x7f39d8038f70 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.449+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f39d8038f70 msgr2=0x7f39d803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.449+0000 7f39effff700 1 --2- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f39d8038f70 0x7f39d803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f39e4006fd0 tx=0x7f39e4006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.449+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 msgr2=0x7f39f019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.449+0000 7f39effff700 1 --2- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f019b900 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f39e0011b60 tx=0x7f39e0011c40 comp rx=0 tx=0).stop 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.449+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 shutdown_connections 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.449+0000 7f39effff700 1 --2- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f39d8038f70 0x7f39d803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.449+0000 7f39effff700 1 --2- 172.21.15.7:0/2941597359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39f01083c0 0x7f39f019b900 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.449+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 >> 172.21.15.7:0/2941597359 conn(0x7f39f0101e30 msgr2=0x7f39f0102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:37.966 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.450+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 shutdown_connections 2024-03-26T08:32:37.967 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:37.450+0000 7f39effff700 1 -- 172.21.15.7:0/2941597359 wait complete. 2024-03-26T08:32:37.967 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-26T08:32:37.967 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-03-26T08:32:37.967 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-03-26T08:32:38.205 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:38 smithi007 ceph-mon[32462]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:32:39.483 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:39 smithi007 ceph-mon[32462]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:32:40.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:40 smithi007 ceph-mon[32462]: Deploying cephadm binary to smithi007 2024-03-26T08:32:43.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-03-26T08:32:43.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.414+0000 7f83b4bfe700 1 Processor -- start 2024-03-26T08:32:43.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.414+0000 7f83b4bfe700 1 -- start start 2024-03-26T08:32:43.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.414+0000 7f83b4bfe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b010a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:43.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.414+0000 7f83b4bfe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00758a0 con 0x7f83b01085c0 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.415+0000 7f83ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b010a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.415+0000 7f83ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b010a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52580/0 (socket says 172.21.15.7:52580) 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.415+0000 7f83ae59c700 1 -- 172.21.15.7:0/440671043 learned_addr learned my addr 172.21.15.7:0/440671043 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.415+0000 7f83ae59c700 1 -- 172.21.15.7:0/440671043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b010aef0 con 0x7f83b01085c0 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.415+0000 7f83ae59c700 1 --2- 172.21.15.7:0/440671043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b010a9b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009e80 tx=0x7f83a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f30b73ab1eba49 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.415+0000 7f83ad59a700 1 -- 172.21.15.7:0/440671043 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a0017070 con 0x7f83b01085c0 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.415+0000 7f83ad59a700 1 -- 172.21.15.7:0/440671043 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83a000b8c0 con 0x7f83b01085c0 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.415+0000 7f83ad59a700 1 -- 172.21.15.7:0/440671043 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a00045c0 con 0x7f83b01085c0 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.416+0000 7f83b4bfe700 1 -- 172.21.15.7:0/440671043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 msgr2=0x7f83b010a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.416+0000 7f83b4bfe700 1 --2- 172.21.15.7:0/440671043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b010a9b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009e80 tx=0x7f83a000b330 comp rx=0 tx=0).stop 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.416+0000 7f83b4bfe700 1 -- 172.21.15.7:0/440671043 shutdown_connections 2024-03-26T08:32:43.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.416+0000 7f83b4bfe700 1 --2- 172.21.15.7:0/440671043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b010a9b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.416+0000 7f83b4bfe700 1 -- 172.21.15.7:0/440671043 >> 172.21.15.7:0/440671043 conn(0x7f83b0101fb0 msgr2=0x7f83b01043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.416+0000 7f83b4bfe700 1 -- 172.21.15.7:0/440671043 shutdown_connections 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.416+0000 7f83b4bfe700 1 -- 172.21.15.7:0/440671043 wait complete. 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.417+0000 7f83b4bfe700 1 Processor -- start 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.417+0000 7f83b4bfe700 1 -- start start 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.417+0000 7f83b4bfe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b0107e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.417+0000 7f83b4bfe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b010bec0 con 0x7f83b01085c0 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.417+0000 7f83ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b0107e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.417+0000 7f83ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b0107e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52582/0 (socket says 172.21.15.7:52582) 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.417+0000 7f83ae59c700 1 -- 172.21.15.7:0/2988747686 learned_addr learned my addr 172.21.15.7:0/2988747686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.417+0000 7f83ae59c700 1 -- 172.21.15.7:0/2988747686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b0108380 con 0x7f83b01085c0 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.417+0000 7f83ae59c700 1 --2- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b0107e40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009e50 tx=0x7f83a000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.418+0000 7f83a77fe700 1 -- 172.21.15.7:0/2988747686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a0020020 con 0x7f83b01085c0 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.418+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b0106570 con 0x7f83b01085c0 2024-03-26T08:32:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.418+0000 7f83a77fe700 1 -- 172.21.15.7:0/2988747686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83a0003b50 con 0x7f83b01085c0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.418+0000 7f83a77fe700 1 -- 172.21.15.7:0/2988747686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83a0017040 con 0x7f83b01085c0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.418+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b0106ab0 con 0x7f83b01085c0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.418+0000 7f83a77fe700 1 -- 172.21.15.7:0/2988747686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f83a0023020 con 0x7f83b01085c0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.419+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b00637c0 con 0x7f83b01085c0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.419+0000 7f83a77fe700 1 --2- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f839c038ed0 0x7f839c03b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.419+0000 7f83a77fe700 1 -- 172.21.15.7:0/2988747686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f83a000f050 con 0x7f83b01085c0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.419+0000 7f83add9b700 1 --2- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f839c038ed0 0x7f839c03b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.419+0000 7f83add9b700 1 --2- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f839c038ed0 0x7f839c03b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8398006fd0 tx=0x7f8398006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.424+0000 7f83a77fe700 1 -- 172.21.15.7:0/2988747686 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f83a002a020 con 0x7f83b01085c0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:38.635+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}) v1 -- 0x7f83b0103160 con 0x7f839c038ed0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:42.999+0000 7f83a77fe700 1 -- 172.21.15.7:0/2988747686 <== mgr.14120 v2:172.21.15.7:6800/1336720039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f83b0103160 con 0x7f839c038ed0 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f839c038ed0 msgr2=0x7f839c03b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 --2- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f839c038ed0 0x7f839c03b390 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8398006fd0 tx=0x7f8398006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 msgr2=0x7f83b0107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:43.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 --2- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b0107e40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009e50 tx=0x7f83a000bd30 comp rx=0 tx=0).stop 2024-03-26T08:32:43.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 shutdown_connections 2024-03-26T08:32:43.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 --2- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f839c038ed0 0x7f839c03b390 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:43.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 --2- 172.21.15.7:0/2988747686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b01085c0 0x7f83b0107e40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:43.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 >> 172.21.15.7:0/2988747686 conn(0x7f83b0101fb0 msgr2=0x7f83b0102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:43.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 shutdown_connections 2024-03-26T08:32:43.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:43.002+0000 7f83b4bfe700 1 -- 172.21.15.7:0/2988747686 wait complete. 2024-03-26T08:32:43.557 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mon service... 2024-03-26T08:32:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:43 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:43 smithi007 ceph-mon[32462]: Added host smithi007 2024-03-26T08:32:44.077 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:43 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:45.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.084+0000 7f7e7df4f700 1 Processor -- start 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.084+0000 7f7e7df4f700 1 -- start start 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.084+0000 7f7e7df4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7810a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.084+0000 7f7e7df4f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7807be90 con 0x7f7e78108590 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.085+0000 7f7e777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7810a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.085+0000 7f7e777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7810a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52598/0 (socket says 172.21.15.7:52598) 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.085+0000 7f7e777fe700 1 -- 172.21.15.7:0/2726439357 learned_addr learned my addr 172.21.15.7:0/2726439357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.085+0000 7f7e777fe700 1 -- 172.21.15.7:0/2726439357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7810aec0 con 0x7f7e78108590 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.085+0000 7f7e777fe700 1 --2- 172.21.15.7:0/2726439357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7810a980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009e80 tx=0x7f7e6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc5a42854248926 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.085+0000 7f7e767fc700 1 -- 172.21.15.7:0/2726439357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e68017070 con 0x7f7e78108590 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.085+0000 7f7e767fc700 1 -- 172.21.15.7:0/2726439357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e6800b8c0 con 0x7f7e78108590 2024-03-26T08:32:45.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.085+0000 7f7e767fc700 1 -- 172.21.15.7:0/2726439357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e680045c0 con 0x7f7e78108590 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.086+0000 7f7e7df4f700 1 -- 172.21.15.7:0/2726439357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 msgr2=0x7f7e7810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.086+0000 7f7e7df4f700 1 --2- 172.21.15.7:0/2726439357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7810a980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009e80 tx=0x7f7e6800b330 comp rx=0 tx=0).stop 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.086+0000 7f7e7df4f700 1 -- 172.21.15.7:0/2726439357 shutdown_connections 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.086+0000 7f7e7df4f700 1 --2- 172.21.15.7:0/2726439357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7810a980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.086+0000 7f7e7df4f700 1 -- 172.21.15.7:0/2726439357 >> 172.21.15.7:0/2726439357 conn(0x7f7e78101f80 msgr2=0x7f7e781043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.086+0000 7f7e7df4f700 1 -- 172.21.15.7:0/2726439357 shutdown_connections 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.086+0000 7f7e7df4f700 1 -- 172.21.15.7:0/2726439357 wait complete. 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.086+0000 7f7e7df4f700 1 Processor -- start 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.087+0000 7f7e7df4f700 1 -- start start 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.087+0000 7f7e7df4f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.087+0000 7f7e7df4f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7810bee0 con 0x7f7e78108590 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.087+0000 7f7e777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.087+0000 7f7e777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7819ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52602/0 (socket says 172.21.15.7:52602) 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.087+0000 7f7e777fe700 1 -- 172.21.15.7:0/456833576 learned_addr learned my addr 172.21.15.7:0/456833576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:45.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.087+0000 7f7e777fe700 1 -- 172.21.15.7:0/456833576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7819bf90 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.087+0000 7f7e777fe700 1 --2- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7819ba50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7e68004350 tx=0x7f7e6800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.087+0000 7f7e74ff9700 1 -- 172.21.15.7:0/456833576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e68020020 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.088+0000 7f7e74ff9700 1 -- 172.21.15.7:0/456833576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e68003c10 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.088+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7819c220 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.088+0000 7f7e74ff9700 1 -- 172.21.15.7:0/456833576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e68017040 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.088+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7819ef10 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.088+0000 7f7e74ff9700 1 -- 172.21.15.7:0/456833576 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f7e68023020 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.089+0000 7f7e74ff9700 1 --2- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7e64038f70 0x7f7e6403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.089+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e780637c0 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.089+0000 7f7e74ff9700 1 -- 172.21.15.7:0/456833576 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7e6800f050 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.089+0000 7f7e76ffd700 1 --2- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7e64038f70 0x7f7e6403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.089+0000 7f7e76ffd700 1 --2- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7e64038f70 0x7f7e6403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006fd0 tx=0x7f7e60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.094+0000 7f7e74ff9700 1 -- 172.21.15.7:0/456833576 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e6802a020 con 0x7f7e78108590 2024-03-26T08:32:45.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.316+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7e78103180 con 0x7f7e64038f70 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.320+0000 7f7e74ff9700 1 -- 172.21.15.7:0/456833576 <== mgr.14120 v2:172.21.15.7:6800/1336720039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7e78103180 con 0x7f7e64038f70 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7e64038f70 msgr2=0x7f7e6403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 --2- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7e64038f70 0x7f7e6403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006fd0 tx=0x7f7e60006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 msgr2=0x7f7e7819ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 --2- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7819ba50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7e68004350 tx=0x7f7e6800bd50 comp rx=0 tx=0).stop 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 shutdown_connections 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 --2- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f7e64038f70 0x7f7e6403b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 --2- 172.21.15.7:0/456833576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78108590 0x7f7e7819ba50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 >> 172.21.15.7:0/456833576 conn(0x7f7e78101f80 msgr2=0x7f7e78102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 shutdown_connections 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:44.323+0000 7f7e7df4f700 1 -- 172.21.15.7:0/456833576 wait complete. 2024-03-26T08:32:45.201 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mgr service... 2024-03-26T08:32:45.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:45 smithi007 ceph-mon[32462]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:32:45.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:45 smithi007 ceph-mon[32462]: Saving service mon spec with placement count:5 2024-03-26T08:32:45.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:45 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.181+0000 7fe9df4ad700 1 Processor -- start 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.181+0000 7fe9df4ad700 1 -- start start 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.181+0000 7fe9df4ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d81084f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.181+0000 7fe9df4ad700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8108ac0 con 0x7fe9d81080d0 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.181+0000 7fe9dd249700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d81084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.181+0000 7fe9dd249700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d81084f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52604/0 (socket says 172.21.15.7:52604) 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.181+0000 7fe9dd249700 1 -- 172.21.15.7:0/993809690 learned_addr learned my addr 172.21.15.7:0/993809690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.182+0000 7fe9dd249700 1 -- 172.21.15.7:0/993809690 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d8105bd0 con 0x7fe9d81080d0 2024-03-26T08:32:46.950 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.182+0000 7fe9dd249700 1 --2- 172.21.15.7:0/993809690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d81084f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80098a0 tx=0x7fe9c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3536875249369aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.182+0000 7fe9cffff700 1 -- 172.21.15.7:0/993809690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9c8004160 con 0x7fe9d81080d0 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.182+0000 7fe9cffff700 1 -- 172.21.15.7:0/993809690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9c8004300 con 0x7fe9d81080d0 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.182+0000 7fe9cffff700 1 -- 172.21.15.7:0/993809690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9c8011b20 con 0x7fe9d81080d0 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.183+0000 7fe9df4ad700 1 -- 172.21.15.7:0/993809690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 msgr2=0x7fe9d81084f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.183+0000 7fe9df4ad700 1 --2- 172.21.15.7:0/993809690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d81084f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9c80098a0 tx=0x7fe9c8009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.183+0000 7fe9df4ad700 1 -- 172.21.15.7:0/993809690 shutdown_connections 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.183+0000 7fe9df4ad700 1 --2- 172.21.15.7:0/993809690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d81084f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.183+0000 7fe9df4ad700 1 -- 172.21.15.7:0/993809690 >> 172.21.15.7:0/993809690 conn(0x7fe9d81017d0 msgr2=0x7fe9d8103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.183+0000 7fe9df4ad700 1 -- 172.21.15.7:0/993809690 shutdown_connections 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.183+0000 7fe9df4ad700 1 -- 172.21.15.7:0/993809690 wait complete. 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.184+0000 7fe9df4ad700 1 Processor -- start 2024-03-26T08:32:46.951 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.184+0000 7fe9df4ad700 1 -- start start 2024-03-26T08:32:46.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.184+0000 7fe9df4ad700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:46.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.184+0000 7fe9df4ad700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8106c90 con 0x7fe9d81080d0 2024-03-26T08:32:46.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.184+0000 7fe9dd249700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:46.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.184+0000 7fe9dd249700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52610/0 (socket says 172.21.15.7:52610) 2024-03-26T08:32:46.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.184+0000 7fe9dd249700 1 -- 172.21.15.7:0/552742481 learned_addr learned my addr 172.21.15.7:0/552742481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:46.952 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.184+0000 7fe9dd249700 1 -- 172.21.15.7:0/552742481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d819e210 con 0x7fe9d81080d0 2024-03-26T08:32:46.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.184+0000 7fe9dd249700 1 --2- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d819dcd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8011b60 tx=0x7fe9c8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:46.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.185+0000 7fe9ce7fc700 1 -- 172.21.15.7:0/552742481 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9c8003b60 con 0x7fe9d81080d0 2024-03-26T08:32:46.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.185+0000 7fe9ce7fc700 1 -- 172.21.15.7:0/552742481 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe9c801a910 con 0x7fe9d81080d0 2024-03-26T08:32:46.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.185+0000 7fe9ce7fc700 1 -- 172.21.15.7:0/552742481 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9c8024930 con 0x7fe9d81080d0 2024-03-26T08:32:46.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.185+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d819e4a0 con 0x7fe9d81080d0 2024-03-26T08:32:46.953 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.185+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d819e900 con 0x7fe9d81080d0 2024-03-26T08:32:46.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.186+0000 7fe9ce7fc700 1 -- 172.21.15.7:0/552742481 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fe9c8022020 con 0x7fe9d81080d0 2024-03-26T08:32:46.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.186+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d80637c0 con 0x7fe9d81080d0 2024-03-26T08:32:46.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.186+0000 7fe9ce7fc700 1 --2- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe9c4038f70 0x7fe9c403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:46.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.186+0000 7fe9ce7fc700 1 -- 172.21.15.7:0/552742481 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe9c804d010 con 0x7fe9d81080d0 2024-03-26T08:32:46.954 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.186+0000 7fe9dca48700 1 --2- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe9c4038f70 0x7fe9c403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.187+0000 7fe9dca48700 1 --2- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe9c4038f70 0x7fe9c403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4006fd0 tx=0x7fe9d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.191+0000 7fe9ce7fc700 1 -- 172.21.15.7:0/552742481 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe9c800b3e0 con 0x7fe9d81080d0 2024-03-26T08:32:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.412+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe9d80625d0 con 0x7fe9c4038f70 2024-03-26T08:32:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.416+0000 7fe9ce7fc700 1 -- 172.21.15.7:0/552742481 <== mgr.14120 v2:172.21.15.7:6800/1336720039 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe9d80625d0 con 0x7fe9c4038f70 2024-03-26T08:32:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe9c4038f70 msgr2=0x7fe9c403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 --2- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe9c4038f70 0x7fe9c403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4006fd0 tx=0x7fe9d4006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:46.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 msgr2=0x7fe9d819dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:46.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 --2- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d819dcd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8011b60 tx=0x7fe9c8011c40 comp rx=0 tx=0).stop 2024-03-26T08:32:46.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 shutdown_connections 2024-03-26T08:32:46.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 --2- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe9c4038f70 0x7fe9c403b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:46.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 --2- 172.21.15.7:0/552742481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d81080d0 0x7fe9d819dcd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:46.956 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 >> 172.21.15.7:0/552742481 conn(0x7fe9d81017d0 msgr2=0x7fe9d8194870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:46.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 shutdown_connections 2024-03-26T08:32:46.957 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:46.419+0000 7fe9df4ad700 1 -- 172.21.15.7:0/552742481 wait complete. 2024-03-26T08:32:47.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:46 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:47.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:46 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:47.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:46 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:48.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:47 smithi007 ceph-mon[32462]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:32:48.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:47 smithi007 ceph-mon[32462]: Saving service mgr spec with placement count:2 2024-03-26T08:32:48.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:47 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/342993067' entity='client.admin' 2024-03-26T08:32:48.299 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.390+0000 7fc96d34c700 1 Processor -- start 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.390+0000 7fc96d34c700 1 -- start start 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.390+0000 7fc96d34c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc968107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.390+0000 7fc96d34c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc968108540 con 0x7fc968107b50 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.391+0000 7fc966ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc968107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.391+0000 7fc966ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc968107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45538/0 (socket says 172.21.15.7:45538) 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.391+0000 7fc966ffd700 1 -- 172.21.15.7:0/2600423030 learned_addr learned my addr 172.21.15.7:0/2600423030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.391+0000 7fc966ffd700 1 -- 172.21.15.7:0/2600423030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968108d50 con 0x7fc968107b50 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.391+0000 7fc966ffd700 1 --2- 172.21.15.7:0/2600423030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc968107f70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9600098a0 tx=0x7fc960009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcbb755cfb5594f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.391+0000 7fc965ffb700 1 -- 172.21.15.7:0/2600423030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc960004160 con 0x7fc968107b50 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.391+0000 7fc965ffb700 1 -- 172.21.15.7:0/2600423030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc960004300 con 0x7fc968107b50 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.391+0000 7fc965ffb700 1 -- 172.21.15.7:0/2600423030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc960011b20 con 0x7fc968107b50 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.392+0000 7fc96d34c700 1 -- 172.21.15.7:0/2600423030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 msgr2=0x7fc968107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:48.300 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.392+0000 7fc96d34c700 1 --2- 172.21.15.7:0/2600423030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc968107f70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc9600098a0 tx=0x7fc960009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.392+0000 7fc96d34c700 1 -- 172.21.15.7:0/2600423030 shutdown_connections 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.392+0000 7fc96d34c700 1 --2- 172.21.15.7:0/2600423030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc968107f70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.392+0000 7fc96d34c700 1 -- 172.21.15.7:0/2600423030 >> 172.21.15.7:0/2600423030 conn(0x7fc968103150 msgr2=0x7fc968105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.392+0000 7fc96d34c700 1 -- 172.21.15.7:0/2600423030 shutdown_connections 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.392+0000 7fc96d34c700 1 -- 172.21.15.7:0/2600423030 wait complete. 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.392+0000 7fc96d34c700 1 Processor -- start 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.393+0000 7fc96d34c700 1 -- start start 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.393+0000 7fc96d34c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc96819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.393+0000 7fc96d34c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96810c100 con 0x7fc968107b50 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.393+0000 7fc966ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc96819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.393+0000 7fc966ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc96819bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45544/0 (socket says 172.21.15.7:45544) 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.393+0000 7fc966ffd700 1 -- 172.21.15.7:0/342993067 learned_addr learned my addr 172.21.15.7:0/342993067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.393+0000 7fc966ffd700 1 -- 172.21.15.7:0/342993067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96819c080 con 0x7fc968107b50 2024-03-26T08:32:48.301 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.393+0000 7fc966ffd700 1 --2- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc96819bb40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc960000c00 tx=0x7fc960011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.393+0000 7fc95ffff700 1 -- 172.21.15.7:0/342993067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc960003b60 con 0x7fc968107b50 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.394+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96819c310 con 0x7fc968107b50 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.394+0000 7fc95ffff700 1 -- 172.21.15.7:0/342993067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc96001a910 con 0x7fc968107b50 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.394+0000 7fc95ffff700 1 -- 172.21.15.7:0/342993067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc960024930 con 0x7fc968107b50 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.394+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96819c770 con 0x7fc968107b50 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.394+0000 7fc95ffff700 1 -- 172.21.15.7:0/342993067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fc960022020 con 0x7fc968107b50 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.394+0000 7fc95ffff700 1 --2- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fc954038f20 0x7fc95403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.395+0000 7fc95ffff700 1 -- 172.21.15.7:0/342993067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc96004d190 con 0x7fc968107b50 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.395+0000 7fc9667fc700 1 --2- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fc954038f20 0x7fc95403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.395+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9680637c0 con 0x7fc968107b50 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.395+0000 7fc9667fc700 1 --2- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fc954038f20 0x7fc95403b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc950006fd0 tx=0x7fc950006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.400+0000 7fc95ffff700 1 -- 172.21.15.7:0/342993067 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc96000b3e0 con 0x7fc968107b50 2024-03-26T08:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.612+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fc96819f000 con 0x7fc968107b50 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.614+0000 7fc95ffff700 1 -- 172.21.15.7:0/342993067 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc9600172b0 con 0x7fc968107b50 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fc954038f20 msgr2=0x7fc95403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 --2- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fc954038f20 0x7fc95403b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc950006fd0 tx=0x7fc950006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 msgr2=0x7fc96819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 --2- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc96819bb40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc960000c00 tx=0x7fc960011c10 comp rx=0 tx=0).stop 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 shutdown_connections 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 --2- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fc954038f20 0x7fc95403b3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 --2- 172.21.15.7:0/342993067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc968107b50 0x7fc96819bb40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 >> 172.21.15.7:0/342993067 conn(0x7fc968103150 msgr2=0x7fc968103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 shutdown_connections 2024-03-26T08:32:48.303 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:47.616+0000 7fc96d34c700 1 -- 172.21.15.7:0/342993067 wait complete. 2024-03-26T08:32:49.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.818+0000 7f0bf7053700 1 Processor -- start 2024-03-26T08:32:49.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.818+0000 7f0bf7053700 1 -- start start 2024-03-26T08:32:49.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.818+0000 7f0bf7053700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:49.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.818+0000 7f0bf7053700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf007c8f0 con 0x7f0bf007dec0 2024-03-26T08:32:49.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.818+0000 7f0bf4def700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:49.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.818+0000 7f0bf4def700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45546/0 (socket says 172.21.15.7:45546) 2024-03-26T08:32:49.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.818+0000 7f0bf4def700 1 -- 172.21.15.7:0/1617264697 learned_addr learned my addr 172.21.15.7:0/1617264697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:49.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.819+0000 7f0bf4def700 1 -- 172.21.15.7:0/1617264697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf007ca70 con 0x7f0bf007dec0 2024-03-26T08:32:49.634 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.819+0000 7f0bf4def700 1 --2- 172.21.15.7:0/1617264697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf007c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0be40098a0 tx=0x7f0be4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8831ff84f46e652 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.819+0000 7f0bef7fe700 1 -- 172.21.15.7:0/1617264697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0be4004160 con 0x7f0bf007dec0 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.819+0000 7f0bef7fe700 1 -- 172.21.15.7:0/1617264697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0be4004300 con 0x7f0bf007dec0 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.819+0000 7f0bef7fe700 1 -- 172.21.15.7:0/1617264697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0be4011b20 con 0x7f0bf007dec0 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.820+0000 7f0bf7053700 1 -- 172.21.15.7:0/1617264697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 msgr2=0x7f0bf007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.820+0000 7f0bf7053700 1 --2- 172.21.15.7:0/1617264697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf007c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0be40098a0 tx=0x7f0be4009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.820+0000 7f0bf7053700 1 -- 172.21.15.7:0/1617264697 shutdown_connections 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.820+0000 7f0bf7053700 1 --2- 172.21.15.7:0/1617264697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf007c320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.820+0000 7f0bf7053700 1 -- 172.21.15.7:0/1617264697 >> 172.21.15.7:0/1617264697 conn(0x7f0bf0103150 msgr2=0x7f0bf0105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.820+0000 7f0bf7053700 1 -- 172.21.15.7:0/1617264697 shutdown_connections 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.820+0000 7f0bf7053700 1 -- 172.21.15.7:0/1617264697 wait complete. 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.820+0000 7f0bf7053700 1 Processor -- start 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.820+0000 7f0bf7053700 1 -- start start 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bf7053700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf019dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:49.635 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bf7053700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf010e1b0 con 0x7f0bf007dec0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bf4def700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf019dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bf4def700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf019dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45560/0 (socket says 172.21.15.7:45560) 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bf4def700 1 -- 172.21.15.7:0/2019832018 learned_addr learned my addr 172.21.15.7:0/2019832018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bf4def700 1 -- 172.21.15.7:0/2019832018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf019e2d0 con 0x7f0bf007dec0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bf4def700 1 --2- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf019dd90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0be4000c00 tx=0x7f0be4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bedffb700 1 -- 172.21.15.7:0/2019832018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0be4003b60 con 0x7f0bf007dec0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bedffb700 1 -- 172.21.15.7:0/2019832018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0be401a910 con 0x7f0bf007dec0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf019e560 con 0x7f0bf007dec0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.821+0000 7f0bedffb700 1 -- 172.21.15.7:0/2019832018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0be4024930 con 0x7f0bf007dec0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.822+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf01a1210 con 0x7f0bf007dec0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.822+0000 7f0bedffb700 1 -- 172.21.15.7:0/2019832018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f0be4022020 con 0x7f0bf007dec0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.823+0000 7f0bedffb700 1 --2- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f0be0038f70 0x7f0be003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.823+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf00637c0 con 0x7f0bf007dec0 2024-03-26T08:32:49.636 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.823+0000 7f0bedffb700 1 -- 172.21.15.7:0/2019832018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0be404d200 con 0x7f0bf007dec0 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.823+0000 7f0beffff700 1 --2- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f0be0038f70 0x7f0be003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.823+0000 7f0beffff700 1 --2- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f0be0038f70 0x7f0be003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc006fd0 tx=0x7f0bdc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:48.828+0000 7f0bedffb700 1 -- 172.21.15.7:0/2019832018 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0be401ab30 con 0x7f0bf007dec0 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.034+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f0bf019e9c0 con 0x7f0bf007dec0 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.036+0000 7f0bedffb700 1 -- 172.21.15.7:0/2019832018 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f0be4033020 con 0x7f0bf007dec0 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.038+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f0be0038f70 msgr2=0x7f0be003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.038+0000 7f0bf7053700 1 --2- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f0be0038f70 0x7f0be003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc006fd0 tx=0x7f0bdc006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.038+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 msgr2=0x7f0bf019dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.038+0000 7f0bf7053700 1 --2- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf019dd90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0be4000c00 tx=0x7f0be4011c10 comp rx=0 tx=0).stop 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.039+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 shutdown_connections 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.039+0000 7f0bf7053700 1 --2- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f0be0038f70 0x7f0be003b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.039+0000 7f0bf7053700 1 --2- 172.21.15.7:0/2019832018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bf007dec0 0x7f0bf019dd90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.039+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 >> 172.21.15.7:0/2019832018 conn(0x7f0bf0103150 msgr2=0x7f0bf0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:49.637 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.039+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 shutdown_connections 2024-03-26T08:32:49.638 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:49.039+0000 7f0bf7053700 1 -- 172.21.15.7:0/2019832018 wait complete. 2024-03-26T08:32:49.638 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-03-26T08:32:50.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:50 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2019832018' entity='client.admin' 2024-03-26T08:32:51.457 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:51 smithi007 ceph-mon[32462]: from='mgr.14120 172.21.15.7:0/3100067604' entity='mgr.x' 2024-03-26T08:32:51.457 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:51 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2100100032' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-26T08:32:51.457 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:51 smithi007 ceph-mon[32462]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:32:51.457 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: ignoring --setuser ceph since I am not root 2024-03-26T08:32:51.458 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: ignoring --setgroup ceph since I am not root 2024-03-26T08:32:51.772 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.037+0000 7fd525868700 1 Processor -- start 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.037+0000 7fd525868700 1 -- start start 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.037+0000 7fd525868700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52010a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.037+0000 7fd525868700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd520075880 con 0x7fd5201085e0 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.037+0000 7fd51effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52010a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.037+0000 7fd51effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52010a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45566/0 (socket says 172.21.15.7:45566) 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.038+0000 7fd51effd700 1 -- 172.21.15.7:0/748254050 learned_addr learned my addr 172.21.15.7:0/748254050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.038+0000 7fd51effd700 1 -- 172.21.15.7:0/748254050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52010af10 con 0x7fd5201085e0 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.038+0000 7fd51effd700 1 --2- 172.21.15.7:0/748254050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52010a9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5100098a0 tx=0x7fd510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee67eda998ac362d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.038+0000 7fd51dffb700 1 -- 172.21.15.7:0/748254050 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd510004160 con 0x7fd5201085e0 2024-03-26T08:32:51.773 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.038+0000 7fd51dffb700 1 -- 172.21.15.7:0/748254050 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd510004300 con 0x7fd5201085e0 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.038+0000 7fd51dffb700 1 -- 172.21.15.7:0/748254050 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd510011b20 con 0x7fd5201085e0 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.039+0000 7fd525868700 1 -- 172.21.15.7:0/748254050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 msgr2=0x7fd52010a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.039+0000 7fd525868700 1 --2- 172.21.15.7:0/748254050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52010a9d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5100098a0 tx=0x7fd510009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.039+0000 7fd525868700 1 -- 172.21.15.7:0/748254050 shutdown_connections 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.039+0000 7fd525868700 1 --2- 172.21.15.7:0/748254050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52010a9d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.039+0000 7fd525868700 1 -- 172.21.15.7:0/748254050 >> 172.21.15.7:0/748254050 conn(0x7fd520101fb0 msgr2=0x7fd5201043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.039+0000 7fd525868700 1 -- 172.21.15.7:0/748254050 shutdown_connections 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.039+0000 7fd525868700 1 -- 172.21.15.7:0/748254050 wait complete. 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.039+0000 7fd525868700 1 Processor -- start 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.040+0000 7fd525868700 1 -- start start 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.040+0000 7fd525868700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:51.774 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.040+0000 7fd525868700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52010bf10 con 0x7fd5201085e0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.040+0000 7fd51effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.040+0000 7fd51effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52019ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45578/0 (socket says 172.21.15.7:45578) 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.040+0000 7fd51effd700 1 -- 172.21.15.7:0/2100100032 learned_addr learned my addr 172.21.15.7:0/2100100032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.040+0000 7fd51effd700 1 -- 172.21.15.7:0/2100100032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52019bfb0 con 0x7fd5201085e0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.040+0000 7fd51effd700 1 --2- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52019ba70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd510011b60 tx=0x7fd510011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.040+0000 7fd524866700 1 -- 172.21.15.7:0/2100100032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd510003b60 con 0x7fd5201085e0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.041+0000 7fd524866700 1 -- 172.21.15.7:0/2100100032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd51001a910 con 0x7fd5201085e0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.041+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52019c240 con 0x7fd5201085e0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.041+0000 7fd524866700 1 -- 172.21.15.7:0/2100100032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd510024930 con 0x7fd5201085e0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.041+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52019c6a0 con 0x7fd5201085e0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.041+0000 7fd524866700 1 -- 172.21.15.7:0/2100100032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd510022020 con 0x7fd5201085e0 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.042+0000 7fd524866700 1 --2- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fd508038f20 0x7fd50803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:51.775 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.042+0000 7fd524866700 1 -- 172.21.15.7:0/2100100032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd51004d050 con 0x7fd5201085e0 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.042+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5200637c0 con 0x7fd5201085e0 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.042+0000 7fd51e7fc700 1 --2- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fd508038f20 0x7fd50803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.042+0000 7fd51e7fc700 1 --2- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fd508038f20 0x7fd50803b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd514006fd0 tx=0x7fd514006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.047+0000 7fd524866700 1 -- 172.21.15.7:0/2100100032 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd51002d910 con 0x7fd5201085e0 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:50.332+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fd52019f420 con 0x7fd5201085e0 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.148+0000 7fd524866700 1 -- 172.21.15.7:0/2100100032 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fd51002d3f0 con 0x7fd5201085e0 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.150+0000 7fd524866700 1 -- 172.21.15.7:0/2100100032 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fd51001aac0 con 0x7fd5201085e0 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fd508038f20 msgr2=0x7fd50803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 --2- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fd508038f20 0x7fd50803b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd514006fd0 tx=0x7fd514006e40 comp rx=0 tx=0).stop 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 msgr2=0x7fd52019ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 --2- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52019ba70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd510011b60 tx=0x7fd510011c40 comp rx=0 tx=0).stop 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 shutdown_connections 2024-03-26T08:32:51.776 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 --2- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fd508038f20 0x7fd50803b3e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 --2- 172.21.15.7:0/2100100032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5201085e0 0x7fd52019ba70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:51.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 >> 172.21.15.7:0/2100100032 conn(0x7fd520101fb0 msgr2=0x7fd520102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:51.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 shutdown_connections 2024-03-26T08:32:51.777 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:51.151+0000 7fd525868700 1 -- 172.21.15.7:0/2100100032 wait complete. 2024-03-26T08:32:52.034 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:51.746+0000 7f0ef3101200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T08:32:52.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:51.846+0000 7f0ef3101200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:32:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:52 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2100100032' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-26T08:32:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:52 smithi007 ceph-mon[32462]: mgrmap e9: x(active, since 20s) 2024-03-26T08:32:52.614 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:52.526+0000 7f0ef3101200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:32:52.614 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:52.621+0000 7f0ef3101200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:32:52.949 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:52.772+0000 7f0ef3101200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:32:52.949 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:52.870+0000 7f0ef3101200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:32:52.949 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:52.956+0000 7f0ef3101200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:32:52.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:32:52.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-26T08:32:52.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T08:32:52.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.143+0000 7fe795029700 1 Processor -- start 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.143+0000 7fe795029700 1 -- start start 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.143+0000 7fe795029700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79010a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.143+0000 7fe795029700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900758c0 con 0x7fe7901085a0 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.143+0000 7fe78ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79010a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.143+0000 7fe78ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79010a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45604/0 (socket says 172.21.15.7:45604) 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.143+0000 7fe78ed9d700 1 -- 172.21.15.7:0/2591906780 learned_addr learned my addr 172.21.15.7:0/2591906780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.144+0000 7fe78ed9d700 1 -- 172.21.15.7:0/2591906780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79010aed0 con 0x7fe7901085a0 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.144+0000 7fe78ed9d700 1 --2- 172.21.15.7:0/2591906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79010a990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7800098a0 tx=0x7fe780009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa90b2915958ea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.144+0000 7fe78dd9b700 1 -- 172.21.15.7:0/2591906780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe780004160 con 0x7fe7901085a0 2024-03-26T08:32:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.144+0000 7fe78dd9b700 1 -- 172.21.15.7:0/2591906780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe780004300 con 0x7fe7901085a0 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.144+0000 7fe78dd9b700 1 -- 172.21.15.7:0/2591906780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe780011b20 con 0x7fe7901085a0 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.144+0000 7fe795029700 1 -- 172.21.15.7:0/2591906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 msgr2=0x7fe79010a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.144+0000 7fe795029700 1 --2- 172.21.15.7:0/2591906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79010a990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7800098a0 tx=0x7fe780009bb0 comp rx=0 tx=0).stop 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.145+0000 7fe795029700 1 -- 172.21.15.7:0/2591906780 shutdown_connections 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.145+0000 7fe795029700 1 --2- 172.21.15.7:0/2591906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79010a990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.145+0000 7fe795029700 1 -- 172.21.15.7:0/2591906780 >> 172.21.15.7:0/2591906780 conn(0x7fe790101fb0 msgr2=0x7fe7901043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.145+0000 7fe795029700 1 -- 172.21.15.7:0/2591906780 shutdown_connections 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.145+0000 7fe795029700 1 -- 172.21.15.7:0/2591906780 wait complete. 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.145+0000 7fe795029700 1 Processor -- start 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe795029700 1 -- start start 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe795029700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe795029700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79010bf30 con 0x7fe7901085a0 2024-03-26T08:32:52.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe78ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe78ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79019baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45608/0 (socket says 172.21.15.7:45608) 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe78ed9d700 1 -- 172.21.15.7:0/97362374 learned_addr learned my addr 172.21.15.7:0/97362374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe78ed9d700 1 -- 172.21.15.7:0/97362374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79019bfe0 con 0x7fe7901085a0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe78ed9d700 1 --2- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79019baa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7800116a0 tx=0x7fe780011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe777fff700 1 -- 172.21.15.7:0/97362374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe780003a40 con 0x7fe7901085a0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe777fff700 1 -- 172.21.15.7:0/97362374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe780003be0 con 0x7fe7901085a0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.146+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79019c270 con 0x7fe7901085a0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.147+0000 7fe777fff700 1 -- 172.21.15.7:0/97362374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe78001a750 con 0x7fe7901085a0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.147+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79019c6d0 con 0x7fe7901085a0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.147+0000 7fe777fff700 1 -- 172.21.15.7:0/97362374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fe78001aa50 con 0x7fe7901085a0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.147+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7900637c0 con 0x7fe7901085a0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.147+0000 7fe777fff700 1 --2- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe778038f20 0x7fe77803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.148+0000 7fe777fff700 1 -- 172.21.15.7:0/97362374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe78004d860 con 0x7fe7901085a0 2024-03-26T08:32:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.148+0000 7fe78e59c700 1 -- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe778038f20 msgr2=0x7fe77803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1336720039 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.148+0000 7fe78e59c700 1 --2- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe778038f20 0x7fe77803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.153+0000 7fe777fff700 1 -- 172.21.15.7:0/97362374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe780011e20 con 0x7fe7901085a0 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.349+0000 7fe78e59c700 1 -- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe778038f20 msgr2=0x7fe77803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1336720039 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.349+0000 7fe78e59c700 1 --2- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe778038f20 0x7fe77803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.420+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe79019f080 con 0x7fe7901085a0 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.421+0000 7fe777fff700 1 -- 172.21.15.7:0/97362374 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe78000ba30 con 0x7fe7901085a0 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.423+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe778038f20 msgr2=0x7fe77803b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.423+0000 7fe795029700 1 --2- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe778038f20 0x7fe77803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.423+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 msgr2=0x7fe79019baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.423+0000 7fe795029700 1 --2- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79019baa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe7800116a0 tx=0x7fe780011ba0 comp rx=0 tx=0).stop 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.423+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 shutdown_connections 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.423+0000 7fe795029700 1 --2- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7fe778038f20 0x7fe77803b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.423+0000 7fe795029700 1 --2- 172.21.15.7:0/97362374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7901085a0 0x7fe79019baa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.423+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 >> 172.21.15.7:0/97362374 conn(0x7fe790101fb0 msgr2=0x7fe790102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:32:52.984 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.424+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 shutdown_connections 2024-03-26T08:32:52.985 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:52.424+0000 7fe795029700 1 -- 172.21.15.7:0/97362374 wait complete. 2024-03-26T08:32:52.985 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-03-26T08:32:52.985 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-03-26T08:32:53.225 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:53 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/97362374' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T08:32:53.530 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:53.232+0000 7f0ef3101200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:32:53.530 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:53.347+0000 7f0ef3101200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:32:53.530 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:53.438+0000 7f0ef3101200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:32:53.793 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:53.537+0000 7f0ef3101200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:32:53.793 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:53.717+0000 7f0ef3101200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:32:55.043 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:54.702+0000 7f0ef3101200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:32:55.043 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:54.836+0000 7f0ef3101200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:32:55.465 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:55.176+0000 7f0ef3101200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:32:55.465 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:55.268+0000 7f0ef3101200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:32:55.465 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:55.472+0000 7f0ef3101200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:32:55.793 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:55.663+0000 7f0ef3101200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:32:56.210 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:55.956+0000 7f0ef3101200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:32:56.211 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:56.218+0000 7f0ef3101200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:32:56.543 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:56.463+0000 7f0ef3101200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:32:57.543 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:57 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:57.294+0000 7f0ef3101200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:32:58.197 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:57 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:57.877+0000 7f0ef3101200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:32:58.197 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:57 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:57.979+0000 7f0ef3101200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:32:58.197 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:32:58.204+0000 7f0ef3101200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:32:58.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: Active manager daemon x restarted 2024-03-26T08:32:58.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: Activating manager daemon x 2024-03-26T08:32:58.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: osdmap e3: 0 total, 0 up, 0 in 2024-03-26T08:32:58.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: mgrmap e10: x(active, starting, since 0.00353396s) 2024-03-26T08:32:58.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:32:58.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T08:32:58.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T08:32:58.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T08:32:58.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T08:32:58.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: Manager daemon x is now available 2024-03-26T08:32:58.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:58.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:32:59.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T08:32:59.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T08:32:59.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:59 smithi007 ceph-mon[32462]: [26/Mar/2024:08:32:58] ENGINE Bus STARTING 2024-03-26T08:32:59.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:59 smithi007 ceph-mon[32462]: [26/Mar/2024:08:32:58] ENGINE Serving on http://172.21.15.7:8765 2024-03-26T08:32:59.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:59 smithi007 ceph-mon[32462]: [26/Mar/2024:08:32:58] ENGINE Serving on https://172.21.15.7:7150 2024-03-26T08:32:59.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:59 smithi007 ceph-mon[32462]: [26/Mar/2024:08:32:58] ENGINE Bus STARTED 2024-03-26T08:32:59.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:59 smithi007 ceph-mon[32462]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T08:32:59.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:32:59 smithi007 ceph-mon[32462]: mgrmap e11: x(active, since 1.00573s) 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.452+0000 7f6d08423700 1 Processor -- start 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.452+0000 7f6d08423700 1 -- start start 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.452+0000 7f6d08423700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d00107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.452+0000 7f6d08423700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d00108540 con 0x7f6d00107b50 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.452+0000 7f6d061bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d00107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.452+0000 7f6d061bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d00107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45612/0 (socket says 172.21.15.7:45612) 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.452+0000 7f6d061bf700 1 -- 172.21.15.7:0/2575654803 learned_addr learned my addr 172.21.15.7:0/2575654803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:00.235 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.452+0000 7f6d061bf700 1 -- 172.21.15.7:0/2575654803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d00108d50 con 0x7f6d00107b50 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.453+0000 7f6d061bf700 1 --2- 172.21.15.7:0/2575654803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d00107f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0098a0 tx=0x7f6cfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0abdfb7652c01e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.453+0000 7f6d051bd700 1 -- 172.21.15.7:0/2575654803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cfc004160 con 0x7f6d00107b50 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.453+0000 7f6d051bd700 1 -- 172.21.15.7:0/2575654803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cfc004300 con 0x7f6d00107b50 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.453+0000 7f6d051bd700 1 -- 172.21.15.7:0/2575654803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cfc011b20 con 0x7f6d00107b50 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.454+0000 7f6d08423700 1 -- 172.21.15.7:0/2575654803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 msgr2=0x7f6d00107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.454+0000 7f6d08423700 1 --2- 172.21.15.7:0/2575654803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d00107f70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0098a0 tx=0x7f6cfc009bb0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.454+0000 7f6d08423700 1 -- 172.21.15.7:0/2575654803 shutdown_connections 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.454+0000 7f6d08423700 1 --2- 172.21.15.7:0/2575654803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d00107f70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.454+0000 7f6d08423700 1 -- 172.21.15.7:0/2575654803 >> 172.21.15.7:0/2575654803 conn(0x7f6d00103150 msgr2=0x7f6d00105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.454+0000 7f6d08423700 1 -- 172.21.15.7:0/2575654803 shutdown_connections 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.454+0000 7f6d08423700 1 -- 172.21.15.7:0/2575654803 wait complete. 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.454+0000 7f6d08423700 1 Processor -- start 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.454+0000 7f6d08423700 1 -- start start 2024-03-26T08:33:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6d08423700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d0007db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6d08423700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0010c100 con 0x7f6d00107b50 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6d061bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d0007db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6d061bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d0007db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45618/0 (socket says 172.21.15.7:45618) 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6d061bf700 1 -- 172.21.15.7:0/4235570671 learned_addr learned my addr 172.21.15.7:0/4235570671 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6d061bf700 1 -- 172.21.15.7:0/4235570671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0007e0d0 con 0x7f6d00107b50 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6d061bf700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d0007db90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0116a0 tx=0x7f6cfc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cfc003a40 con 0x7f6d00107b50 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cfc003be0 con 0x7f6d00107b50 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6d08423700 1 -- 172.21.15.7:0/4235570671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0007c2c0 con 0x7f6d00107b50 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cfc01a720 con 0x7f6d00107b50 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.455+0000 7f6d08423700 1 -- 172.21.15.7:0/4235570671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0007c780 con 0x7f6d00107b50 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.456+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f6cfc024aa0 con 0x7f6d00107b50 2024-03-26T08:33:00.237 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.456+0000 7f6cf77fe700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 0x7f6cec03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.456+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 --> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cec03baf0 con 0x7f6cec038f20 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.457+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6cfc04d520 con 0x7f6d00107b50 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.457+0000 7f6d059be700 1 -- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 msgr2=0x7f6cec03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1336720039 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.457+0000 7f6d059be700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 0x7f6cec03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.658+0000 7f6d059be700 1 -- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 msgr2=0x7f6cec03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1336720039 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:53.658+0000 7f6d059be700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 0x7f6cec03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:54.059+0000 7f6d059be700 1 -- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 msgr2=0x7f6cec03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1336720039 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:54.059+0000 7f6d059be700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 0x7f6cec03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:54.860+0000 7f6d059be700 1 -- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 msgr2=0x7f6cec03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1336720039 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:54.860+0000 7f6d059be700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 0x7f6cec03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:56.461+0000 7f6d059be700 1 -- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 msgr2=0x7f6cec03b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/1336720039 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:56.461+0000 7f6d059be700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 0x7f6cec03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:58.219+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f6cfc024430 con 0x7f6d00107b50 2024-03-26T08:33:00.238 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:58.219+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 msgr2=0x7f6cec03b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:58.219+0000 7f6cf77fe700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 0x7f6cec03b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.220+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7f6cfc04efa0 con 0x7f6d00107b50 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.220+0000 7f6cf77fe700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cec03c9a0 0x7f6cec03ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.221+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cec03baf0 con 0x7f6cec03c9a0 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.221+0000 7f6d059be700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cec03c9a0 0x7f6cec03ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.222+0000 7f6d059be700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cec03c9a0 0x7f6cec03ed90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0003a10 tx=0x7f6cf0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.222+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f6cec03baf0 con 0x7f6cec03c9a0 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.229+0000 7f6d08423700 1 -- 172.21.15.7:0/4235570671 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6d000637c0 con 0x7f6cec03c9a0 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.229+0000 7f6cf77fe700 1 -- 172.21.15.7:0/4235570671 <== mgr.14152 v2:172.21.15.7:6800/1233671832 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f6d000637c0 con 0x7f6cec03c9a0 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 -- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cec03c9a0 msgr2=0x7f6cec03ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cec03c9a0 0x7f6cec03ed90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0003a10 tx=0x7f6cf0009270 comp rx=0 tx=0).stop 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 -- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 msgr2=0x7f6d0007db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d0007db90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0116a0 tx=0x7f6cfc011ba0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.239 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 -- 172.21.15.7:0/4235570671 shutdown_connections 2024-03-26T08:33:00.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cec03c9a0 0x7f6cec03ed90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d00107b50 0x7f6d0007db90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 --2- 172.21.15.7:0/4235570671 >> [v2:172.21.15.7:6800/1336720039,v1:172.21.15.7:6801/1336720039] conn(0x7f6cec038f20 0x7f6cec03b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:00.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 -- 172.21.15.7:0/4235570671 >> 172.21.15.7:0/4235570671 conn(0x7f6d00103150 msgr2=0x7f6d00103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:00.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 -- 172.21.15.7:0/4235570671 shutdown_connections 2024-03-26T08:33:00.240 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:32:59.230+0000 7f6d08423700 1 -- 172.21.15.7:0/4235570671 wait complete. 2024-03-26T08:33:00.240 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-03-26T08:33:00.240 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-03-26T08:33:01.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:00 smithi007 ceph-mon[32462]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T08:33:01.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:01.656 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.697+0000 7f3df611d700 1 Processor -- start 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.697+0000 7f3df611d700 1 -- start start 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.697+0000 7f3df611d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df0107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.697+0000 7f3df611d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df0108540 con 0x7f3df0107b50 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.698+0000 7f3def7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df0107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.698+0000 7f3def7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df0107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56128/0 (socket says 172.21.15.7:56128) 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.698+0000 7f3def7fe700 1 -- 172.21.15.7:0/3313191751 learned_addr learned my addr 172.21.15.7:0/3313191751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.698+0000 7f3def7fe700 1 -- 172.21.15.7:0/3313191751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df0108d50 con 0x7f3df0107b50 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.698+0000 7f3def7fe700 1 --2- 172.21.15.7:0/3313191751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df0107f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3de00098a0 tx=0x7f3de0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88acf4ae036172a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.698+0000 7f3dee7fc700 1 -- 172.21.15.7:0/3313191751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de0004160 con 0x7f3df0107b50 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.698+0000 7f3dee7fc700 1 -- 172.21.15.7:0/3313191751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de0004300 con 0x7f3df0107b50 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.698+0000 7f3dee7fc700 1 -- 172.21.15.7:0/3313191751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de0011b20 con 0x7f3df0107b50 2024-03-26T08:33:01.657 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.699+0000 7f3df611d700 1 -- 172.21.15.7:0/3313191751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 msgr2=0x7f3df0107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.699+0000 7f3df611d700 1 --2- 172.21.15.7:0/3313191751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df0107f70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3de00098a0 tx=0x7f3de0009bb0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.699+0000 7f3df611d700 1 -- 172.21.15.7:0/3313191751 shutdown_connections 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.699+0000 7f3df611d700 1 --2- 172.21.15.7:0/3313191751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df0107f70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.699+0000 7f3df611d700 1 -- 172.21.15.7:0/3313191751 >> 172.21.15.7:0/3313191751 conn(0x7f3df0103150 msgr2=0x7f3df0105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.699+0000 7f3df611d700 1 -- 172.21.15.7:0/3313191751 shutdown_connections 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.699+0000 7f3df611d700 1 -- 172.21.15.7:0/3313191751 wait complete. 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.700+0000 7f3df611d700 1 Processor -- start 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.700+0000 7f3df611d700 1 -- start start 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.700+0000 7f3df611d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df007db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.700+0000 7f3df611d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df010c100 con 0x7f3df0107b50 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.700+0000 7f3def7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df007db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.701+0000 7f3def7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df007db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56136/0 (socket says 172.21.15.7:56136) 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.701+0000 7f3def7fe700 1 -- 172.21.15.7:0/947077736 learned_addr learned my addr 172.21.15.7:0/947077736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:01.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.701+0000 7f3def7fe700 1 -- 172.21.15.7:0/947077736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df007e0d0 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.701+0000 7f3def7fe700 1 --2- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df007db90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3de00116a0 tx=0x7f3de0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.701+0000 7f3decff9700 1 -- 172.21.15.7:0/947077736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de0003a40 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.701+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df007c2c0 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.701+0000 7f3decff9700 1 -- 172.21.15.7:0/947077736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de0003be0 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.701+0000 7f3decff9700 1 -- 172.21.15.7:0/947077736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de001a720 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.701+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df007c780 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.702+0000 7f3decff9700 1 -- 172.21.15.7:0/947077736 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7f3de001aa40 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.702+0000 7f3decff9700 1 --2- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc038e10 0x7f3ddc03b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.702+0000 7f3decff9700 1 -- 172.21.15.7:0/947077736 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3de004d410 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.702+0000 7f3deeffd700 1 --2- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc038e10 0x7f3ddc03b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.702+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df00637c0 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.703+0000 7f3deeffd700 1 --2- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc038e10 0x7f3ddc03b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8006fd0 tx=0x7f3dd8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.708+0000 7f3decff9700 1 -- 172.21.15.7:0/947077736 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3de0011e20 con 0x7f3df0107b50 2024-03-26T08:33:01.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.920+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f3df007cd50 con 0x7f3ddc038e10 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:00.939+0000 7f3decff9700 1 -- 172.21.15.7:0/947077736 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f3de002dc90 con 0x7f3df0107b50 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.080+0000 7f3decff9700 1 -- 172.21.15.7:0/947077736 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f3df007cd50 con 0x7f3ddc038e10 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc038e10 msgr2=0x7f3ddc03b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 --2- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc038e10 0x7f3ddc03b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8006fd0 tx=0x7f3dd8006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 msgr2=0x7f3df007db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 --2- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df007db90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3de00116a0 tx=0x7f3de0011ba0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 shutdown_connections 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 --2- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc038e10 0x7f3ddc03b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 --2- 172.21.15.7:0/947077736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0107b50 0x7f3df007db90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 >> 172.21.15.7:0/947077736 conn(0x7f3df0103150 msgr2=0x7f3df0103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 shutdown_connections 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:01.082+0000 7f3df611d700 1 -- 172.21.15.7:0/947077736 wait complete. 2024-03-26T08:33:01.660 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-03-26T08:33:02.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:01 smithi007 ceph-mon[32462]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:02.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:01 smithi007 ceph-mon[32462]: mgrmap e12: x(active, since 2s) 2024-03-26T08:33:02.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:02.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:02.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:03.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:03 smithi007 ceph-mon[32462]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:03.755 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:03.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$7.ML6dWyVlKwbbqou4OgCeHtyOkGePpxVjR1axwyhh0UC/J1rn6oi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711441982, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-26T08:33:03.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.082+0000 7f9173152700 1 Processor -- start 2024-03-26T08:33:03.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.082+0000 7f9173152700 1 -- start start 2024-03-26T08:33:03.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.082+0000 7f9173152700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c107f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:03.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.082+0000 7f9173152700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c108510 con 0x7f916c107b20 2024-03-26T08:33:03.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.082+0000 7f9170eee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:03.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.082+0000 7f9170eee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c107f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56146/0 (socket says 172.21.15.7:56146) 2024-03-26T08:33:03.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.082+0000 7f9170eee700 1 -- 172.21.15.7:0/1965971128 learned_addr learned my addr 172.21.15.7:0/1965971128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:03.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.082+0000 7f9170eee700 1 -- 172.21.15.7:0/1965971128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c108d20 con 0x7f916c107b20 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.082+0000 7f9170eee700 1 --2- 172.21.15.7:0/1965971128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c107f40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91600098a0 tx=0x7f9160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adb497879fe50b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f916b7fe700 1 -- 172.21.15.7:0/1965971128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9160004160 con 0x7f916c107b20 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f916b7fe700 1 -- 172.21.15.7:0/1965971128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9160004300 con 0x7f916c107b20 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f916b7fe700 1 -- 172.21.15.7:0/1965971128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9160011b20 con 0x7f916c107b20 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f9173152700 1 -- 172.21.15.7:0/1965971128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 msgr2=0x7f916c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f9173152700 1 --2- 172.21.15.7:0/1965971128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c107f40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f91600098a0 tx=0x7f9160009bb0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f9173152700 1 -- 172.21.15.7:0/1965971128 shutdown_connections 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f9173152700 1 --2- 172.21.15.7:0/1965971128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c107f40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f9173152700 1 -- 172.21.15.7:0/1965971128 >> 172.21.15.7:0/1965971128 conn(0x7f916c1030a0 msgr2=0x7f916c105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f9173152700 1 -- 172.21.15.7:0/1965971128 shutdown_connections 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.083+0000 7f9173152700 1 -- 172.21.15.7:0/1965971128 wait complete. 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.084+0000 7f9173152700 1 Processor -- start 2024-03-26T08:33:03.764 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.084+0000 7f9173152700 1 -- start start 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.084+0000 7f9173152700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c07db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.084+0000 7f9173152700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c10c0d0 con 0x7f916c107b20 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.084+0000 7f9170eee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c07db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.084+0000 7f9170eee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c07db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56154/0 (socket says 172.21.15.7:56154) 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.084+0000 7f9170eee700 1 -- 172.21.15.7:0/2811776918 learned_addr learned my addr 172.21.15.7:0/2811776918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.085+0000 7f9170eee700 1 -- 172.21.15.7:0/2811776918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c07e0d0 con 0x7f916c107b20 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.085+0000 7f9170eee700 1 --2- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c07db90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91600116a0 tx=0x7f9160011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.085+0000 7f9169ffb700 1 -- 172.21.15.7:0/2811776918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9160003a40 con 0x7f916c107b20 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.085+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c07c2c0 con 0x7f916c107b20 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.085+0000 7f9169ffb700 1 -- 172.21.15.7:0/2811776918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9160003be0 con 0x7f916c107b20 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.085+0000 7f9169ffb700 1 -- 172.21.15.7:0/2811776918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f916001a720 con 0x7f916c107b20 2024-03-26T08:33:03.765 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.085+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c07c780 con 0x7f916c107b20 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.086+0000 7f9169ffb700 1 -- 172.21.15.7:0/2811776918 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f916001a9d0 con 0x7f916c107b20 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.086+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c0637c0 con 0x7f916c107b20 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.086+0000 7f9169ffb700 1 --2- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f915c038f80 0x7f915c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.086+0000 7f9169ffb700 1 -- 172.21.15.7:0/2811776918 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f916004d4f0 con 0x7f916c107b20 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.087+0000 7f916bfff700 1 --2- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f915c038f80 0x7f915c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.087+0000 7f916bfff700 1 --2- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f915c038f80 0x7f915c03b440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9158006fd0 tx=0x7f9158006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.091+0000 7f9169ffb700 1 -- 172.21.15.7:0/2811776918 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9160011e20 con 0x7f916c107b20 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.337+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f916c07ca00 con 0x7f915c038f80 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.583+0000 7f9169ffb700 1 -- 172.21.15.7:0/2811776918 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f916c07ca00 con 0x7f915c038f80 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.585+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f915c038f80 msgr2=0x7f915c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.585+0000 7f9173152700 1 --2- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f915c038f80 0x7f915c03b440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9158006fd0 tx=0x7f9158006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.585+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 msgr2=0x7f916c07db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:03.766 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.585+0000 7f9173152700 1 --2- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c07db90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f91600116a0 tx=0x7f9160011ba0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.586+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 shutdown_connections 2024-03-26T08:33:03.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.586+0000 7f9173152700 1 --2- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f915c038f80 0x7f915c03b440 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.586+0000 7f9173152700 1 --2- 172.21.15.7:0/2811776918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c107b20 0x7f916c07db90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:03.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.586+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 >> 172.21.15.7:0/2811776918 conn(0x7f916c1030a0 msgr2=0x7f916c103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:03.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.586+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 shutdown_connections 2024-03-26T08:33:03.767 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:02.586+0000 7f9173152700 1 -- 172.21.15.7:0/2811776918 wait complete. 2024-03-26T08:33:03.767 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-03-26T08:33:04.652 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:04 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/311828511' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.161+0000 7f81651e5700 1 Processor -- start 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.161+0000 7f81651e5700 1 -- start start 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.161+0000 7f81651e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f8160106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.161+0000 7f81651e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816007be50 con 0x7f8160106430 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.161+0000 7f815ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f8160106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.161+0000 7f815ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f8160106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56166/0 (socket says 172.21.15.7:56166) 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.161+0000 7f815ed9d700 1 -- 172.21.15.7:0/4245206402 learned_addr learned my addr 172.21.15.7:0/4245206402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.162+0000 7f815ed9d700 1 -- 172.21.15.7:0/4245206402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160106d90 con 0x7f8160106430 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.162+0000 7f815ed9d700 1 --2- 172.21.15.7:0/4245206402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f8160106850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81500098a0 tx=0x7f8150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a023e406a811eed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:04.920 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.162+0000 7f815e59c700 1 -- 172.21.15.7:0/4245206402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8150004160 con 0x7f8160106430 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.162+0000 7f815e59c700 1 -- 172.21.15.7:0/4245206402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8150004300 con 0x7f8160106430 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.162+0000 7f815e59c700 1 -- 172.21.15.7:0/4245206402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8150011b20 con 0x7f8160106430 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.163+0000 7f81651e5700 1 -- 172.21.15.7:0/4245206402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 msgr2=0x7f8160106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.163+0000 7f81651e5700 1 --2- 172.21.15.7:0/4245206402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f8160106850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81500098a0 tx=0x7f8150009bb0 comp rx=0 tx=0).stop 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.163+0000 7f81651e5700 1 -- 172.21.15.7:0/4245206402 shutdown_connections 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.163+0000 7f81651e5700 1 --2- 172.21.15.7:0/4245206402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f8160106850 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.163+0000 7f81651e5700 1 -- 172.21.15.7:0/4245206402 >> 172.21.15.7:0/4245206402 conn(0x7f8160102030 msgr2=0x7f8160104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.163+0000 7f81651e5700 1 -- 172.21.15.7:0/4245206402 shutdown_connections 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.163+0000 7f81651e5700 1 -- 172.21.15.7:0/4245206402 wait complete. 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.163+0000 7f81651e5700 1 Processor -- start 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.163+0000 7f81651e5700 1 -- start start 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f81651e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f816019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f81651e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8160107e60 con 0x7f8160106430 2024-03-26T08:33:04.921 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f815ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f816019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f815ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f816019dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56182/0 (socket says 172.21.15.7:56182) 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f815ed9d700 1 -- 172.21.15.7:0/311828511 learned_addr learned my addr 172.21.15.7:0/311828511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f815ed9d700 1 -- 172.21.15.7:0/311828511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816019e210 con 0x7f8160106430 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f815ed9d700 1 --2- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f816019dcd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f81500116a0 tx=0x7f8150011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f815cd99700 1 -- 172.21.15.7:0/311828511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8150003a40 con 0x7f8160106430 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f815cd99700 1 -- 172.21.15.7:0/311828511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8150003be0 con 0x7f8160106430 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816019e4a0 con 0x7f8160106430 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f815cd99700 1 -- 172.21.15.7:0/311828511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f815001a750 con 0x7f8160106430 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.164+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816019e900 con 0x7f8160106430 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.165+0000 7f815cd99700 1 -- 172.21.15.7:0/311828511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f815001aa80 con 0x7f8160106430 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.165+0000 7f815cd99700 1 --2- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f814c038f80 0x7f814c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.165+0000 7f815cd99700 1 -- 172.21.15.7:0/311828511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f815004d6a0 con 0x7f8160106430 2024-03-26T08:33:04.922 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.165+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81600637c0 con 0x7f8160106430 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.166+0000 7f8157fff700 1 --2- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f814c038f80 0x7f814c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.166+0000 7f8157fff700 1 --2- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f814c038f80 0x7f814c03b440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8148006fd0 tx=0x7f8148006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.171+0000 7f815cd99700 1 -- 172.21.15.7:0/311828511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8150011e20 con 0x7f8160106430 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.376+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8160062da0 con 0x7f8160106430 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.376+0000 7f815cd99700 1 -- 172.21.15.7:0/311828511 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8150032020 con 0x7f8160106430 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.378+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f814c038f80 msgr2=0x7f814c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.378+0000 7f81651e5700 1 --2- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f814c038f80 0x7f814c03b440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8148006fd0 tx=0x7f8148006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.378+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 msgr2=0x7f816019dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.378+0000 7f81651e5700 1 --2- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f816019dcd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f81500116a0 tx=0x7f8150011ba0 comp rx=0 tx=0).stop 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.378+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 shutdown_connections 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.378+0000 7f81651e5700 1 --2- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f814c038f80 0x7f814c03b440 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.378+0000 7f81651e5700 1 --2- 172.21.15.7:0/311828511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8160106430 0x7f816019dcd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:04.923 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.378+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 >> 172.21.15.7:0/311828511 conn(0x7f8160102030 msgr2=0x7f8160194940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:04.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.379+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 shutdown_connections 2024-03-26T08:33:04.924 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:04.379+0000 7f81651e5700 1 -- 172.21.15.7:0/311828511 wait complete. 2024-03-26T08:33:04.927 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T08:33:04.927 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-03-26T08:33:04.934 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T08:33:04.934 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-03-26T08:33:04.934 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-03-26T08:33:04.934 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-26T08:33:04.936 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-03-26T08:33:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-03-26T08:33:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-03-26T08:33:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: Password: x124chb97x 2024-03-26T08:33:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:04.937 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config directory 2024-03-26T08:33:04.937 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling autotune for osd_memory_target 2024-03-26T08:33:05.851 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:05 smithi007 ceph-mon[32462]: mgrmap e13: x(active, since 6s) 2024-03-26T08:33:06.110 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.369+0000 7f3224835700 1 Processor -- start 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.370+0000 7f3224835700 1 -- start start 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.370+0000 7f3224835700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f32201061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.370+0000 7f3224835700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32201067c0 con 0x7f3220105dd0 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.370+0000 7f321e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f32201061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.370+0000 7f321e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f32201061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56184/0 (socket says 172.21.15.7:56184) 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.370+0000 7f321e59c700 1 -- 172.21.15.7:0/2200032053 learned_addr learned my addr 172.21.15.7:0/2200032053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.370+0000 7f321e59c700 1 -- 172.21.15.7:0/2200032053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220108d60 con 0x7f3220105dd0 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.371+0000 7f321e59c700 1 --2- 172.21.15.7:0/2200032053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f32201061f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32100098a0 tx=0x7f3210009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9881da7a8880401 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.371+0000 7f321d59a700 1 -- 172.21.15.7:0/2200032053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3210004160 con 0x7f3220105dd0 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.371+0000 7f321d59a700 1 -- 172.21.15.7:0/2200032053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3210004300 con 0x7f3220105dd0 2024-03-26T08:33:06.111 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.371+0000 7f321d59a700 1 -- 172.21.15.7:0/2200032053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3210011b20 con 0x7f3220105dd0 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.371+0000 7f3224835700 1 -- 172.21.15.7:0/2200032053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 msgr2=0x7f32201061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.371+0000 7f3224835700 1 --2- 172.21.15.7:0/2200032053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f32201061f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32100098a0 tx=0x7f3210009bb0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.371+0000 7f3224835700 1 -- 172.21.15.7:0/2200032053 shutdown_connections 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.371+0000 7f3224835700 1 --2- 172.21.15.7:0/2200032053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f32201061f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.371+0000 7f3224835700 1 -- 172.21.15.7:0/2200032053 >> 172.21.15.7:0/2200032053 conn(0x7f3220101780 msgr2=0x7f3220103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.372+0000 7f3224835700 1 -- 172.21.15.7:0/2200032053 shutdown_connections 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.372+0000 7f3224835700 1 -- 172.21.15.7:0/2200032053 wait complete. 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.372+0000 7f3224835700 1 Processor -- start 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.372+0000 7f3224835700 1 -- start start 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.372+0000 7f3224835700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f322019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.372+0000 7f3224835700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322010c110 con 0x7f3220105dd0 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f321e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f322019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f321e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f322019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56198/0 (socket says 172.21.15.7:56198) 2024-03-26T08:33:06.112 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f321e59c700 1 -- 172.21.15.7:0/2007482484 learned_addr learned my addr 172.21.15.7:0/2007482484 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f321e59c700 1 -- 172.21.15.7:0/2007482484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322019c050 con 0x7f3220105dd0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f321e59c700 1 --2- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f322019bb10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f32100116a0 tx=0x7f3210011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f320f7fe700 1 -- 172.21.15.7:0/2007482484 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3210003a40 con 0x7f3220105dd0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f320f7fe700 1 -- 172.21.15.7:0/2007482484 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3210003be0 con 0x7f3220105dd0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322019c2e0 con 0x7f3220105dd0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f320f7fe700 1 -- 172.21.15.7:0/2007482484 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f321001a720 con 0x7f3220105dd0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.373+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322019c740 con 0x7f3220105dd0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.374+0000 7f320f7fe700 1 -- 172.21.15.7:0/2007482484 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f321001a9d0 con 0x7f3220105dd0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.374+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32200637c0 con 0x7f3220105dd0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.374+0000 7f320f7fe700 1 --2- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3208039020 0x7f320803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.374+0000 7f320f7fe700 1 -- 172.21.15.7:0/2007482484 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f321004d690 con 0x7f3220105dd0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.374+0000 7f321dd9b700 1 --2- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3208039020 0x7f320803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.375+0000 7f321dd9b700 1 --2- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3208039020 0x7f320803b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3214006fd0 tx=0x7f3214006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:06.113 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.379+0000 7f320f7fe700 1 -- 172.21.15.7:0/2007482484 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3210011e20 con 0x7f3220105dd0 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.582+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f322019f110 con 0x7f3220105dd0 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.582+0000 7f320f7fe700 1 -- 172.21.15.7:0/2007482484 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f321002d3f0 con 0x7f3220105dd0 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3208039020 msgr2=0x7f320803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 --2- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3208039020 0x7f320803b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3214006fd0 tx=0x7f3214006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 msgr2=0x7f322019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 --2- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f322019bb10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f32100116a0 tx=0x7f3210011ba0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 shutdown_connections 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 --2- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3208039020 0x7f320803b4e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 --2- 172.21.15.7:0/2007482484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3220105dd0 0x7f322019bb10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 >> 172.21.15.7:0/2007482484 conn(0x7f3220101780 msgr2=0x7f3220102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:06.114 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 shutdown_connections 2024-03-26T08:33:06.115 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:05.584+0000 7f3224835700 1 -- 172.21.15.7:0/2007482484 wait complete. 2024-03-26T08:33:07.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.565+0000 7efead9b9700 1 Processor -- start 2024-03-26T08:33:07.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.565+0000 7efead9b9700 1 -- start start 2024-03-26T08:33:07.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.565+0000 7efead9b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea8107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:07.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.565+0000 7efead9b9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea8108540 con 0x7efea8107b50 2024-03-26T08:33:07.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.565+0000 7efea6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea8107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:07.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.565+0000 7efea6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea8107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48714/0 (socket says 172.21.15.7:48714) 2024-03-26T08:33:07.460 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.565+0000 7efea6ffd700 1 -- 172.21.15.7:0/2674134901 learned_addr learned my addr 172.21.15.7:0/2674134901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.565+0000 7efea6ffd700 1 -- 172.21.15.7:0/2674134901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea8108d50 con 0x7efea8107b50 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.566+0000 7efea6ffd700 1 --2- 172.21.15.7:0/2674134901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea8107f70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe980098a0 tx=0x7efe98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf214bf969840793 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.566+0000 7efea67fc700 1 -- 172.21.15.7:0/2674134901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe98004160 con 0x7efea8107b50 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.566+0000 7efea67fc700 1 -- 172.21.15.7:0/2674134901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe98004300 con 0x7efea8107b50 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.566+0000 7efea67fc700 1 -- 172.21.15.7:0/2674134901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe98011b20 con 0x7efea8107b50 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.566+0000 7efead9b9700 1 -- 172.21.15.7:0/2674134901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 msgr2=0x7efea8107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.566+0000 7efead9b9700 1 --2- 172.21.15.7:0/2674134901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea8107f70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efe980098a0 tx=0x7efe98009bb0 comp rx=0 tx=0).stop 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.566+0000 7efead9b9700 1 -- 172.21.15.7:0/2674134901 shutdown_connections 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.566+0000 7efead9b9700 1 --2- 172.21.15.7:0/2674134901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea8107f70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.566+0000 7efead9b9700 1 -- 172.21.15.7:0/2674134901 >> 172.21.15.7:0/2674134901 conn(0x7efea8103150 msgr2=0x7efea8105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.567+0000 7efead9b9700 1 -- 172.21.15.7:0/2674134901 shutdown_connections 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.567+0000 7efead9b9700 1 -- 172.21.15.7:0/2674134901 wait complete. 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.567+0000 7efead9b9700 1 Processor -- start 2024-03-26T08:33:07.461 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.567+0000 7efead9b9700 1 -- start start 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.567+0000 7efead9b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.567+0000 7efead9b9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea810c100 con 0x7efea8107b50 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.567+0000 7efea6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.567+0000 7efea6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48716/0 (socket says 172.21.15.7:48716) 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.567+0000 7efea6ffd700 1 -- 172.21.15.7:0/2520825890 learned_addr learned my addr 172.21.15.7:0/2520825890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.568+0000 7efea6ffd700 1 -- 172.21.15.7:0/2520825890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea807e0d0 con 0x7efea8107b50 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.568+0000 7efea6ffd700 1 --2- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea807db90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe980116a0 tx=0x7efe98011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.568+0000 7efeac9b7700 1 -- 172.21.15.7:0/2520825890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe98003a40 con 0x7efea8107b50 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.568+0000 7efeac9b7700 1 -- 172.21.15.7:0/2520825890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe98003be0 con 0x7efea8107b50 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.568+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea807c2c0 con 0x7efea8107b50 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.568+0000 7efeac9b7700 1 -- 172.21.15.7:0/2520825890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe9801a720 con 0x7efea8107b50 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.568+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea807c780 con 0x7efea8107b50 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.569+0000 7efeac9b7700 1 -- 172.21.15.7:0/2520825890 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7efe9801a9d0 con 0x7efea8107b50 2024-03-26T08:33:07.462 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.569+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea80637c0 con 0x7efea8107b50 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.569+0000 7efeac9b7700 1 --2- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe94039020 0x7efe9403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.569+0000 7efeac9b7700 1 -- 172.21.15.7:0/2520825890 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efe9804d5c0 con 0x7efea8107b50 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.569+0000 7efe9ffff700 1 --2- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe94039020 0x7efe9403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.569+0000 7efe9ffff700 1 --2- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe94039020 0x7efe9403b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe90006fd0 tx=0x7efe90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.574+0000 7efeac9b7700 1 -- 172.21.15.7:0/2520825890 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe98011e20 con 0x7efea8107b50 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.891+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7efea807ca00 con 0x7efea8107b50 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.893+0000 7efeac9b7700 1 -- 172.21.15.7:0/2520825890 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7efe98024b50 con 0x7efea8107b50 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe94039020 msgr2=0x7efe9403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 --2- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe94039020 0x7efe9403b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efe90006fd0 tx=0x7efe90006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 msgr2=0x7efea807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 --2- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea807db90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe980116a0 tx=0x7efe98011ba0 comp rx=0 tx=0).stop 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 shutdown_connections 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 --2- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe94039020 0x7efe9403b4e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:07.463 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 --2- 172.21.15.7:0/2520825890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea8107b50 0x7efea807db90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 >> 172.21.15.7:0/2520825890 conn(0x7efea8103150 msgr2=0x7efea8103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 shutdown_connections 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-03-26T08:33:06.895+0000 7efead9b9700 1 -- 172.21.15.7:0/2520825890 wait complete. 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-03-26T08:33:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:07.465 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-03-26T08:33:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-26T08:33:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:07.465 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-03-26T08:33:07.504 INFO:tasks.cephadm:Fetching config... 2024-03-26T08:33:07.504 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:33:07.505 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-26T08:33:07.520 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-26T08:33:07.520 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:33:07.520 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-26T08:33:07.577 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-26T08:33:07.577 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:33:07.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-26T08:33:07.645 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-26T08:33:07.645 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:33:07.645 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-26T08:33:07.700 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-26T08:33:07.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiKC2ooCq7HBZoXjhgLK6JF3NuoKJKH954cD9E6tu5+zVCzm5dkdubdUMWJkxt+EXqr012AdE67VybhkijY0JnOTmg65aOzqEc5Git76KU+rbWvVI7Vr7XjPyrbLjqngGuRk+JXZT2YUyLExUbaVVn4zxSnfixOPtQN0D/5DQHxC40XroVQOEY3YcDa0O+82KE2ZMy/OB1dbGddRmGIKjzhMaReyEg2hjq5ZpLo5vUqGf0/favTvlq54scRsWqCzGwq4SpIC/2VjfuV3PL884ilD9btK47ygoFAkb0H+xhtA9uOwPSg/q+XNk3S+6le/Zbv9msXw14oWhek4qP0KqmM6TNPwjKRPEEtPsNd+Fg15t1ViImoqbyC09mD0pxXVWfaM/CYrKG8S8W9i4lH3g1HENFd7lSfP2djSu6wDZWUPqXcvVm/Qkqk7+xp/S7kNDKVbKx4ooCT5mEWDh7wO9ciJY9CueJ/AEeOMjrUG7Q4yo7j8MrcR8wMW2rdSNzV2U= ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T08:33:07.779 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiKC2ooCq7HBZoXjhgLK6JF3NuoKJKH954cD9E6tu5+zVCzm5dkdubdUMWJkxt+EXqr012AdE67VybhkijY0JnOTmg65aOzqEc5Git76KU+rbWvVI7Vr7XjPyrbLjqngGuRk+JXZT2YUyLExUbaVVn4zxSnfixOPtQN0D/5DQHxC40XroVQOEY3YcDa0O+82KE2ZMy/OB1dbGddRmGIKjzhMaReyEg2hjq5ZpLo5vUqGf0/favTvlq54scRsWqCzGwq4SpIC/2VjfuV3PL884ilD9btK47ygoFAkb0H+xhtA9uOwPSg/q+XNk3S+6le/Zbv9msXw14oWhek4qP0KqmM6TNPwjKRPEEtPsNd+Fg15t1ViImoqbyC09mD0pxXVWfaM/CYrKG8S8W9i4lH3g1HENFd7lSfP2djSu6wDZWUPqXcvVm/Qkqk7+xp/S7kNDKVbKx4ooCT5mEWDh7wO9ciJY9CueJ/AEeOMjrUG7Q4yo7j8MrcR8wMW2rdSNzV2U= ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:33:07.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiKC2ooCq7HBZoXjhgLK6JF3NuoKJKH954cD9E6tu5+zVCzm5dkdubdUMWJkxt+EXqr012AdE67VybhkijY0JnOTmg65aOzqEc5Git76KU+rbWvVI7Vr7XjPyrbLjqngGuRk+JXZT2YUyLExUbaVVn4zxSnfixOPtQN0D/5DQHxC40XroVQOEY3YcDa0O+82KE2ZMy/OB1dbGddRmGIKjzhMaReyEg2hjq5ZpLo5vUqGf0/favTvlq54scRsWqCzGwq4SpIC/2VjfuV3PL884ilD9btK47ygoFAkb0H+xhtA9uOwPSg/q+XNk3S+6le/Zbv9msXw14oWhek4qP0KqmM6TNPwjKRPEEtPsNd+Fg15t1ViImoqbyC09mD0pxXVWfaM/CYrKG8S8W9i4lH3g1HENFd7lSfP2djSu6wDZWUPqXcvVm/Qkqk7+xp/S7kNDKVbKx4ooCT5mEWDh7wO9ciJY9CueJ/AEeOMjrUG7Q4yo7j8MrcR8wMW2rdSNzV2U= ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T08:33:07.847 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiKC2ooCq7HBZoXjhgLK6JF3NuoKJKH954cD9E6tu5+zVCzm5dkdubdUMWJkxt+EXqr012AdE67VybhkijY0JnOTmg65aOzqEc5Git76KU+rbWvVI7Vr7XjPyrbLjqngGuRk+JXZT2YUyLExUbaVVn4zxSnfixOPtQN0D/5DQHxC40XroVQOEY3YcDa0O+82KE2ZMy/OB1dbGddRmGIKjzhMaReyEg2hjq5ZpLo5vUqGf0/favTvlq54scRsWqCzGwq4SpIC/2VjfuV3PL884ilD9btK47ygoFAkb0H+xhtA9uOwPSg/q+XNk3S+6le/Zbv9msXw14oWhek4qP0KqmM6TNPwjKRPEEtPsNd+Fg15t1ViImoqbyC09mD0pxXVWfaM/CYrKG8S8W9i4lH3g1HENFd7lSfP2djSu6wDZWUPqXcvVm/Qkqk7+xp/S7kNDKVbKx4ooCT5mEWDh7wO9ciJY9CueJ/AEeOMjrUG7Q4yo7j8MrcR8wMW2rdSNzV2U= ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:33:07.866 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiKC2ooCq7HBZoXjhgLK6JF3NuoKJKH954cD9E6tu5+zVCzm5dkdubdUMWJkxt+EXqr012AdE67VybhkijY0JnOTmg65aOzqEc5Git76KU+rbWvVI7Vr7XjPyrbLjqngGuRk+JXZT2YUyLExUbaVVn4zxSnfixOPtQN0D/5DQHxC40XroVQOEY3YcDa0O+82KE2ZMy/OB1dbGddRmGIKjzhMaReyEg2hjq5ZpLo5vUqGf0/favTvlq54scRsWqCzGwq4SpIC/2VjfuV3PL884ilD9btK47ygoFAkb0H+xhtA9uOwPSg/q+XNk3S+6le/Zbv9msXw14oWhek4qP0KqmM6TNPwjKRPEEtPsNd+Fg15t1ViImoqbyC09mD0pxXVWfaM/CYrKG8S8W9i4lH3g1HENFd7lSfP2djSu6wDZWUPqXcvVm/Qkqk7+xp/S7kNDKVbKx4ooCT5mEWDh7wO9ciJY9CueJ/AEeOMjrUG7Q4yo7j8MrcR8wMW2rdSNzV2U= ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T08:33:07.920 INFO:teuthology.orchestra.run.smithi044.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDiKC2ooCq7HBZoXjhgLK6JF3NuoKJKH954cD9E6tu5+zVCzm5dkdubdUMWJkxt+EXqr012AdE67VybhkijY0JnOTmg65aOzqEc5Git76KU+rbWvVI7Vr7XjPyrbLjqngGuRk+JXZT2YUyLExUbaVVn4zxSnfixOPtQN0D/5DQHxC40XroVQOEY3YcDa0O+82KE2ZMy/OB1dbGddRmGIKjzhMaReyEg2hjq5ZpLo5vUqGf0/favTvlq54scRsWqCzGwq4SpIC/2VjfuV3PL884ilD9btK47ygoFAkb0H+xhtA9uOwPSg/q+XNk3S+6le/Zbv9msXw14oWhek4qP0KqmM6TNPwjKRPEEtPsNd+Fg15t1ViImoqbyC09mD0pxXVWfaM/CYrKG8S8W9i4lH3g1HENFd7lSfP2djSu6wDZWUPqXcvVm/Qkqk7+xp/S7kNDKVbKx4ooCT5mEWDh7wO9ciJY9CueJ/AEeOMjrUG7Q4yo7j8MrcR8wMW2rdSNzV2U= ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:33:07.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-26T08:33:07.963 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:07 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2520825890' entity='client.admin' 2024-03-26T08:33:08.182 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.517+0000 7f54e23e0700 1 -- 172.21.15.7:0/482418435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 msgr2=0x7f54dc074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.517+0000 7f54e23e0700 1 --2- 172.21.15.7:0/482418435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 0x7f54dc074620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009910 tx=0x7f54cc009c20 comp rx=0 tx=0).stop 2024-03-26T08:33:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.517+0000 7f54e23e0700 1 -- 172.21.15.7:0/482418435 shutdown_connections 2024-03-26T08:33:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.517+0000 7f54e23e0700 1 --2- 172.21.15.7:0/482418435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 0x7f54dc074620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.517+0000 7f54e23e0700 1 -- 172.21.15.7:0/482418435 >> 172.21.15.7:0/482418435 conn(0x7f54dc0fd180 msgr2=0x7f54dc0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.517+0000 7f54e23e0700 1 -- 172.21.15.7:0/482418435 shutdown_connections 2024-03-26T08:33:10.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.517+0000 7f54e23e0700 1 -- 172.21.15.7:0/482418435 wait complete. 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.517+0000 7f54e23e0700 1 Processor -- start 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.517+0000 7f54e23e0700 1 -- start start 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54e23e0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 0x7f54dc19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54e23e0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc10e340 con 0x7f54dc0761c0 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 0x7f54dc19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 0x7f54dc19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48734/0 (socket says 172.21.15.7:48734) 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54dbfff700 1 -- 172.21.15.7:0/976987965 learned_addr learned my addr 172.21.15.7:0/976987965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54dbfff700 1 -- 172.21.15.7:0/976987965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc19de00 con 0x7f54dc0761c0 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54dbfff700 1 --2- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 0x7f54dc19d8c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f54cc006010 tx=0x7f54cc005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54d97fa700 1 -- 172.21.15.7:0/976987965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54cc027070 con 0x7f54dc0761c0 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54d97fa700 1 -- 172.21.15.7:0/976987965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f54cc00be40 con 0x7f54dc0761c0 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.518+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54dc19e090 con 0x7f54dc0761c0 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.519+0000 7f54d97fa700 1 -- 172.21.15.7:0/976987965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54cc00f460 con 0x7f54dc0761c0 2024-03-26T08:33:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.519+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54dc1a0d80 con 0x7f54dc0761c0 2024-03-26T08:33:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.519+0000 7f54d97fa700 1 -- 172.21.15.7:0/976987965 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f54cc02a070 con 0x7f54dc0761c0 2024-03-26T08:33:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.520+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54dc10e9f0 con 0x7f54dc0761c0 2024-03-26T08:33:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.520+0000 7f54d97fa700 1 --2- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f54c4039020 0x7f54c403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.520+0000 7f54d97fa700 1 -- 172.21.15.7:0/976987965 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f54cc01e030 con 0x7f54dc0761c0 2024-03-26T08:33:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.520+0000 7f54db7fe700 1 --2- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f54c4039020 0x7f54c403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.520+0000 7f54db7fe700 1 --2- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f54c4039020 0x7f54c403b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54d0006fd0 tx=0x7f54d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.525+0000 7f54d97fa700 1 -- 172.21.15.7:0/976987965 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f54cc013070 con 0x7f54dc0761c0 2024-03-26T08:33:10.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.727+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f54dc02db40 con 0x7f54dc0761c0 2024-03-26T08:33:10.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.730+0000 7f54d97fa700 1 -- 172.21.15.7:0/976987965 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f54cc031020 con 0x7f54dc0761c0 2024-03-26T08:33:10.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f54c4039020 msgr2=0x7f54c403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:10.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 --2- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f54c4039020 0x7f54c403b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f54d0006fd0 tx=0x7f54d0006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:10.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 msgr2=0x7f54dc19d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:10.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 --2- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 0x7f54dc19d8c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f54cc006010 tx=0x7f54cc005050 comp rx=0 tx=0).stop 2024-03-26T08:33:10.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 shutdown_connections 2024-03-26T08:33:10.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 --2- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f54c4039020 0x7f54c403b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:10.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 --2- 172.21.15.7:0/976987965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54dc0761c0 0x7f54dc19d8c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:10.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 >> 172.21.15.7:0/976987965 conn(0x7f54dc0fd180 msgr2=0x7f54dc0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:10.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 shutdown_connections 2024-03-26T08:33:10.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:10.732+0000 7f54e23e0700 1 -- 172.21.15.7:0/976987965 wait complete. 2024-03-26T08:33:11.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:11.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:11.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:33:11.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:11.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:11.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:11.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/976987965' entity='client.admin' 2024-03-26T08:33:11.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:11.340 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:11.367 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-26T08:33:11.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-26T08:33:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:13.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.033+0000 7f16c2b9c700 1 -- 172.21.15.7:0/488371341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 msgr2=0x7f16bc104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:13.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.033+0000 7f16c2b9c700 1 --2- 172.21.15.7:0/488371341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 0x7f16bc104ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009910 tx=0x7f16ac009c20 comp rx=0 tx=0).stop 2024-03-26T08:33:13.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.033+0000 7f16c2b9c700 1 -- 172.21.15.7:0/488371341 shutdown_connections 2024-03-26T08:33:13.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.033+0000 7f16c2b9c700 1 --2- 172.21.15.7:0/488371341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 0x7f16bc104ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:13.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.033+0000 7f16c2b9c700 1 -- 172.21.15.7:0/488371341 >> 172.21.15.7:0/488371341 conn(0x7f16bc0fc060 msgr2=0x7f16bc0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:13.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.033+0000 7f16c2b9c700 1 -- 172.21.15.7:0/488371341 shutdown_connections 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.033+0000 7f16c2b9c700 1 -- 172.21.15.7:0/488371341 wait complete. 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.033+0000 7f16c2b9c700 1 Processor -- start 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16c2b9c700 1 -- start start 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16c2b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 0x7f16bc199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16c2b9c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc107c50 con 0x7f16bc1026f0 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16c0938700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 0x7f16bc199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16c0938700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 0x7f16bc199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48748/0 (socket says 172.21.15.7:48748) 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16c0938700 1 -- 172.21.15.7:0/1908363354 learned_addr learned my addr 172.21.15.7:0/1908363354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16c0938700 1 -- 172.21.15.7:0/1908363354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc1999d0 con 0x7f16bc1026f0 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16c0938700 1 --2- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 0x7f16bc199490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16ac004fa0 tx=0x7f16ac005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16b9ffb700 1 -- 172.21.15.7:0/1908363354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16ac027070 con 0x7f16bc1026f0 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16b9ffb700 1 -- 172.21.15.7:0/1908363354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16ac00be40 con 0x7f16bc1026f0 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc199c60 con 0x7f16bc1026f0 2024-03-26T08:33:13.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.034+0000 7f16b9ffb700 1 -- 172.21.15.7:0/1908363354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16ac00f460 con 0x7f16bc1026f0 2024-03-26T08:33:13.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.035+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc19c950 con 0x7f16bc1026f0 2024-03-26T08:33:13.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.036+0000 7f16b9ffb700 1 -- 172.21.15.7:0/1908363354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f16ac02a070 con 0x7f16bc1026f0 2024-03-26T08:33:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.036+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc1083d0 con 0x7f16bc1026f0 2024-03-26T08:33:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.036+0000 7f16b9ffb700 1 --2- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f16a4038fc0 0x7f16a403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.036+0000 7f16b9ffb700 1 -- 172.21.15.7:0/1908363354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f16ac01e030 con 0x7f16bc1026f0 2024-03-26T08:33:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.036+0000 7f16bbfff700 1 --2- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f16a4038fc0 0x7f16a403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.037+0000 7f16bbfff700 1 --2- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f16a4038fc0 0x7f16a403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006fd0 tx=0x7f16b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:13.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.041+0000 7f16b9ffb700 1 -- 172.21.15.7:0/1908363354 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16ac013070 con 0x7f16bc1026f0 2024-03-26T08:33:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.251+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f16bc02db40 con 0x7f16a4038fc0 2024-03-26T08:33:13.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.254+0000 7f16b9ffb700 1 -- 172.21.15.7:0/1908363354 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f16bc02db40 con 0x7f16a4038fc0 2024-03-26T08:33:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f16a4038fc0 msgr2=0x7f16a403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 --2- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f16a4038fc0 0x7f16a403b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006fd0 tx=0x7f16b0006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 msgr2=0x7f16bc199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 --2- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 0x7f16bc199490 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f16ac004fa0 tx=0x7f16ac005080 comp rx=0 tx=0).stop 2024-03-26T08:33:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 shutdown_connections 2024-03-26T08:33:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 --2- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f16a4038fc0 0x7f16a403b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 --2- 172.21.15.7:0/1908363354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc1026f0 0x7f16bc199490 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 >> 172.21.15.7:0/1908363354 conn(0x7f16bc0fc060 msgr2=0x7f16bc0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 shutdown_connections 2024-03-26T08:33:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:13.256+0000 7f16c2b9c700 1 -- 172.21.15.7:0/1908363354 wait complete. 2024-03-26T08:33:14.152 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi016 2024-03-26T08:33:14.152 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:33:14.152 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/etc/ceph/ceph.conf 2024-03-26T08:33:14.171 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:33:14.171 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:14.229 INFO:tasks.cephadm:Adding host smithi016 to orchestrator... 2024-03-26T08:33:14.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch host add smithi016 2024-03-26T08:33:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: Updating smithi007:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: Updating smithi007:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.184+0000 7f82741a5700 1 -- 172.21.15.7:0/1610444909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 msgr2=0x7f826c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.184+0000 7f82741a5700 1 --2- 172.21.15.7:0/1610444909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 0x7f826c1021b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8268009910 tx=0x7f8268009c20 comp rx=0 tx=0).stop 2024-03-26T08:33:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.184+0000 7f82741a5700 1 -- 172.21.15.7:0/1610444909 shutdown_connections 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.184+0000 7f82741a5700 1 --2- 172.21.15.7:0/1610444909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 0x7f826c1021b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.184+0000 7f82741a5700 1 -- 172.21.15.7:0/1610444909 >> 172.21.15.7:0/1610444909 conn(0x7f826c0fd6a0 msgr2=0x7f826c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.184+0000 7f82741a5700 1 -- 172.21.15.7:0/1610444909 shutdown_connections 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.184+0000 7f82741a5700 1 -- 172.21.15.7:0/1610444909 wait complete. 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.184+0000 7f82741a5700 1 Processor -- start 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.184+0000 7f82741a5700 1 -- start start 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f82741a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 0x7f826c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f82741a5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c10c1a0 con 0x7f826c101dd0 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f8271f41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 0x7f826c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f8271f41700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 0x7f826c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48776/0 (socket says 172.21.15.7:48776) 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f8271f41700 1 -- 172.21.15.7:0/1640294086 learned_addr learned my addr 172.21.15.7:0/1640294086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f8271f41700 1 -- 172.21.15.7:0/1640294086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c197c80 con 0x7f826c101dd0 2024-03-26T08:33:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f8271f41700 1 --2- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 0x7f826c197740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8268006010 tx=0x7f8268005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f8262ffd700 1 -- 172.21.15.7:0/1640294086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8268026070 con 0x7f826c101dd0 2024-03-26T08:33:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f8262ffd700 1 -- 172.21.15.7:0/1640294086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f826800bbd0 con 0x7f826c101dd0 2024-03-26T08:33:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c197f70 con 0x7f826c101dd0 2024-03-26T08:33:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f8262ffd700 1 -- 172.21.15.7:0/1640294086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8268003910 con 0x7f826c101dd0 2024-03-26T08:33:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.185+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c19c010 con 0x7f826c101dd0 2024-03-26T08:33:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.186+0000 7f8262ffd700 1 -- 172.21.15.7:0/1640294086 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f8268029070 con 0x7f826c101dd0 2024-03-26T08:33:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.186+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c1032b0 con 0x7f826c101dd0 2024-03-26T08:33:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.186+0000 7f8262ffd700 1 --2- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8258039020 0x7f825803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.186+0000 7f8262ffd700 1 -- 172.21.15.7:0/1640294086 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f826801e070 con 0x7f826c101dd0 2024-03-26T08:33:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.187+0000 7f8271740700 1 --2- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8258039020 0x7f825803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.187+0000 7f8271740700 1 --2- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8258039020 0x7f825803b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f825c006fd0 tx=0x7f825c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.191+0000 7f8262ffd700 1 -- 172.21.15.7:0/1640294086 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8268013070 con 0x7f826c101dd0 2024-03-26T08:33:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:16.404+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "target": ["mon-mgr", ""]}) v1 -- 0x7f826c198aa0 con 0x7f8258039020 2024-03-26T08:33:17.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:17 smithi007 ceph-mon[32462]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:17.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:17 smithi007 ceph-mon[32462]: Deploying cephadm binary to smithi016 2024-03-26T08:33:18.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:18 smithi007 ceph-mon[32462]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:20 smithi007 ceph-mon[32462]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:20.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.785+0000 7f8262ffd700 1 -- 172.21.15.7:0/1640294086 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f826c198aa0 con 0x7f8258039020 2024-03-26T08:33:20.778 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi016' with addr '172.21.15.16' 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8258039020 msgr2=0x7f825803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 --2- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8258039020 0x7f825803b4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f825c006fd0 tx=0x7f825c006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 msgr2=0x7f826c197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 --2- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 0x7f826c197740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8268006010 tx=0x7f8268005030 comp rx=0 tx=0).stop 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 shutdown_connections 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 --2- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8258039020 0x7f825803b4e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 --2- 172.21.15.7:0/1640294086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826c101dd0 0x7f826c197740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 >> 172.21.15.7:0/1640294086 conn(0x7f826c0fd6a0 msgr2=0x7f826c104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 shutdown_connections 2024-03-26T08:33:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:20.788+0000 7f82741a5700 1 -- 172.21.15.7:0/1640294086 wait complete. 2024-03-26T08:33:21.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T08:33:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:21.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:21 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:21.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:21 smithi007 ceph-mon[32462]: Added host smithi016 2024-03-26T08:33:21.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:21 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:21.965 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:21 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:22.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:22.900 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:22 smithi007 ceph-mon[32462]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:23.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.101+0000 7f004d4d4700 1 -- 172.21.15.7:0/2334870731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 msgr2=0x7f0048106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:23.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.101+0000 7f004d4d4700 1 --2- 172.21.15.7:0/2334870731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 0x7f0048106e50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0038009910 tx=0x7f0038009c20 comp rx=0 tx=0).stop 2024-03-26T08:33:23.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.101+0000 7f004d4d4700 1 -- 172.21.15.7:0/2334870731 shutdown_connections 2024-03-26T08:33:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.101+0000 7f004d4d4700 1 --2- 172.21.15.7:0/2334870731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 0x7f0048106e50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.101+0000 7f004d4d4700 1 -- 172.21.15.7:0/2334870731 >> 172.21.15.7:0/2334870731 conn(0x7f00480fc580 msgr2=0x7f00480fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.101+0000 7f004d4d4700 1 -- 172.21.15.7:0/2334870731 shutdown_connections 2024-03-26T08:33:23.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.102+0000 7f004d4d4700 1 -- 172.21.15.7:0/2334870731 wait complete. 2024-03-26T08:33:23.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.102+0000 7f004d4d4700 1 Processor -- start 2024-03-26T08:33:23.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.102+0000 7f004d4d4700 1 -- start start 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.102+0000 7f004d4d4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 0x7f00481976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.102+0000 7f004d4d4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004810b0b0 con 0x7f0048106a70 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f0046ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 0x7f00481976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f0046ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 0x7f00481976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40444/0 (socket says 172.21.15.7:40444) 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f0046ffd700 1 -- 172.21.15.7:0/864433562 learned_addr learned my addr 172.21.15.7:0/864433562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f0046ffd700 1 -- 172.21.15.7:0/864433562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0048197bf0 con 0x7f0048106a70 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f0046ffd700 1 --2- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 0x7f00481976b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0038005850 tx=0x7f0038005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f002ffff700 1 -- 172.21.15.7:0/864433562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0038026070 con 0x7f0048106a70 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f002ffff700 1 -- 172.21.15.7:0/864433562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f003800bbd0 con 0x7f0048106a70 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f002ffff700 1 -- 172.21.15.7:0/864433562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0038003970 con 0x7f0048106a70 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0048197ee0 con 0x7f0048106a70 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.103+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004819bf80 con 0x7f0048106a70 2024-03-26T08:33:23.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.104+0000 7f002ffff700 1 -- 172.21.15.7:0/864433562 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f0038029070 con 0x7f0048106a70 2024-03-26T08:33:23.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.104+0000 7f002ffff700 1 --2- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0030038fc0 0x7f003003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:23.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.104+0000 7f002ffff700 1 -- 172.21.15.7:0/864433562 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f003801e070 con 0x7f0048106a70 2024-03-26T08:33:23.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.104+0000 7f00467fc700 1 --2- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0030038fc0 0x7f003003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:23.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.104+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00481021b0 con 0x7f0048106a70 2024-03-26T08:33:23.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.105+0000 7f00467fc700 1 --2- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0030038fc0 0x7f003003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f003c006fd0 tx=0x7f003c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:23.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.109+0000 7f002ffff700 1 -- 172.21.15.7:0/864433562 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0038030070 con 0x7f0048106a70 2024-03-26T08:33:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.320+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0048063820 con 0x7f0030038fc0 2024-03-26T08:33:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.321+0000 7f002ffff700 1 -- 172.21.15.7:0/864433562 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7f0048063820 con 0x7f0030038fc0 2024-03-26T08:33:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:23.314 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}] 2024-03-26T08:33:23.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.323+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0030038fc0 msgr2=0x7f003003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:23.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.323+0000 7f004d4d4700 1 --2- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0030038fc0 0x7f003003b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f003c006fd0 tx=0x7f003c006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.323+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 msgr2=0x7f00481976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.323+0000 7f004d4d4700 1 --2- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 0x7f00481976b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0038005850 tx=0x7f0038005030 comp rx=0 tx=0).stop 2024-03-26T08:33:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.324+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 shutdown_connections 2024-03-26T08:33:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.324+0000 7f004d4d4700 1 --2- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0030038fc0 0x7f003003b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.324+0000 7f004d4d4700 1 --2- 172.21.15.7:0/864433562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048106a70 0x7f00481976b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.324+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 >> 172.21.15.7:0/864433562 conn(0x7f00480fc580 msgr2=0x7f0048104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.324+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 shutdown_connections 2024-03-26T08:33:23.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:23.324+0000 7f004d4d4700 1 -- 172.21.15.7:0/864433562 wait complete. 2024-03-26T08:33:23.931 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi044 2024-03-26T08:33:23.931 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:33:23.931 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.conf 2024-03-26T08:33:23.951 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:33:23.951 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:24.009 INFO:tasks.cephadm:Adding host smithi044 to orchestrator... 2024-03-26T08:33:24.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch host add smithi044 2024-03-26T08:33:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:24.728 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:24 smithi007 ceph-mon[32462]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:24.728 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:24 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:24.728 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:24 smithi007 ceph-mon[32462]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.748+0000 7ff73063c700 1 -- 172.21.15.7:0/4257813571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 msgr2=0x7ff7281021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.748+0000 7ff73063c700 1 --2- 172.21.15.7:0/4257813571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 0x7ff7281021b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009910 tx=0x7ff71c009c20 comp rx=0 tx=0).stop 2024-03-26T08:33:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff73063c700 1 -- 172.21.15.7:0/4257813571 shutdown_connections 2024-03-26T08:33:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff73063c700 1 --2- 172.21.15.7:0/4257813571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 0x7ff7281021b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff73063c700 1 -- 172.21.15.7:0/4257813571 >> 172.21.15.7:0/4257813571 conn(0x7ff7280fd6a0 msgr2=0x7ff7280ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff73063c700 1 -- 172.21.15.7:0/4257813571 shutdown_connections 2024-03-26T08:33:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff73063c700 1 -- 172.21.15.7:0/4257813571 wait complete. 2024-03-26T08:33:25.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff73063c700 1 Processor -- start 2024-03-26T08:33:25.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff73063c700 1 -- start start 2024-03-26T08:33:25.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff73063c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 0x7ff728074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:25.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff73063c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72810c1a0 con 0x7ff728101dd0 2024-03-26T08:33:25.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff72e3d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 0x7ff728074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff72e3d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 0x7ff728074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40470/0 (socket says 172.21.15.7:40470) 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.749+0000 7ff72e3d8700 1 -- 172.21.15.7:0/1922049447 learned_addr learned my addr 172.21.15.7:0/1922049447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.750+0000 7ff72e3d8700 1 -- 172.21.15.7:0/1922049447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff728076380 con 0x7ff728101dd0 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.750+0000 7ff72e3d8700 1 --2- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 0x7ff728074490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff71c006010 tx=0x7ff71c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.750+0000 7ff71b7fe700 1 -- 172.21.15.7:0/1922049447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff71c026070 con 0x7ff728101dd0 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.750+0000 7ff71b7fe700 1 -- 172.21.15.7:0/1922049447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff71c00bc30 con 0x7ff728101dd0 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.750+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff728074a30 con 0x7ff728101dd0 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.750+0000 7ff71b7fe700 1 -- 172.21.15.7:0/1922049447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff71c0039d0 con 0x7ff728101dd0 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.750+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff728074ef0 con 0x7ff728101dd0 2024-03-26T08:33:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.751+0000 7ff71b7fe700 1 -- 172.21.15.7:0/1922049447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7ff71c029070 con 0x7ff728101dd0 2024-03-26T08:33:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.751+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7281032b0 con 0x7ff728101dd0 2024-03-26T08:33:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.751+0000 7ff71b7fe700 1 --2- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff714038fc0 0x7ff71403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.751+0000 7ff71b7fe700 1 -- 172.21.15.7:0/1922049447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff71c01e070 con 0x7ff728101dd0 2024-03-26T08:33:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.751+0000 7ff72dbd7700 1 --2- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff714038fc0 0x7ff71403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:25.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.752+0000 7ff72dbd7700 1 --2- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff714038fc0 0x7ff71403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff724006fd0 tx=0x7ff724006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:25.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.756+0000 7ff71b7fe700 1 -- 172.21.15.7:0/1922049447 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff71c013070 con 0x7ff728101dd0 2024-03-26T08:33:25.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:25.965+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7280759c0 con 0x7ff714038fc0 2024-03-26T08:33:26.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:26 smithi007 ceph-mon[32462]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:26.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:26 smithi007 ceph-mon[32462]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:27.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:27.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:27.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:27.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:27.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:33:27.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:27.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:27.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-26T08:33:27.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: Deploying cephadm binary to smithi044 2024-03-26T08:33:27.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: Updating smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:33:27.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:27 smithi007 ceph-mon[32462]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:28.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:28 smithi007 ceph-mon[32462]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:30.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.315+0000 7ff71b7fe700 1 -- 172.21.15.7:0/1922049447 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7ff7280759c0 con 0x7ff714038fc0 2024-03-26T08:33:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi044' with addr '172.21.15.44' 2024-03-26T08:33:30.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.317+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff714038fc0 msgr2=0x7ff71403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.317+0000 7ff73063c700 1 --2- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff714038fc0 0x7ff71403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff724006fd0 tx=0x7ff724006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:30.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.317+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 msgr2=0x7ff728074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:30.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.317+0000 7ff73063c700 1 --2- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 0x7ff728074490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff71c006010 tx=0x7ff71c005030 comp rx=0 tx=0).stop 2024-03-26T08:33:30.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.318+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 shutdown_connections 2024-03-26T08:33:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.318+0000 7ff73063c700 1 --2- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff714038fc0 0x7ff71403b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.318+0000 7ff73063c700 1 --2- 172.21.15.7:0/1922049447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff728101dd0 0x7ff728074490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.318+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 >> 172.21.15.7:0/1922049447 conn(0x7ff7280fd6a0 msgr2=0x7ff728104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.318+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 shutdown_connections 2024-03-26T08:33:30.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:30.318+0000 7ff73063c700 1 -- 172.21.15.7:0/1922049447 wait complete. 2024-03-26T08:33:30.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:30 smithi007 ceph-mon[32462]: Updating smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:33:30.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:30 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:30.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:30 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:30.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:30 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:30.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:30 smithi007 ceph-mon[32462]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:30.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:30 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:30.877 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:30 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:30.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T08:33:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:32.289 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:31 smithi007 ceph-mon[32462]: Added host smithi044 2024-03-26T08:33:32.290 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:32.290 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.629+0000 7f82501de700 1 -- 172.21.15.7:0/895971163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 msgr2=0x7f82481021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.629+0000 7f82501de700 1 --2- 172.21.15.7:0/895971163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 0x7f82481021b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f823800b0a0 tx=0x7f823800b3b0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.629+0000 7f82501de700 1 -- 172.21.15.7:0/895971163 shutdown_connections 2024-03-26T08:33:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.629+0000 7f82501de700 1 --2- 172.21.15.7:0/895971163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 0x7f82481021b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.629+0000 7f82501de700 1 -- 172.21.15.7:0/895971163 >> 172.21.15.7:0/895971163 conn(0x7f82480fd6a0 msgr2=0x7f82480ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.629+0000 7f82501de700 1 -- 172.21.15.7:0/895971163 shutdown_connections 2024-03-26T08:33:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.629+0000 7f82501de700 1 -- 172.21.15.7:0/895971163 wait complete. 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.630+0000 7f82501de700 1 Processor -- start 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.630+0000 7f82501de700 1 -- start start 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.630+0000 7f82501de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 0x7f8248197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.630+0000 7f82501de700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824810c1a0 con 0x7f8248101dd0 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.630+0000 7f824df7a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 0x7f8248197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.630+0000 7f824df7a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 0x7f8248197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58010/0 (socket says 172.21.15.7:58010) 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.630+0000 7f824df7a700 1 -- 172.21.15.7:0/3688516036 learned_addr learned my addr 172.21.15.7:0/3688516036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.631+0000 7f824df7a700 1 -- 172.21.15.7:0/3688516036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8248197bd0 con 0x7f8248101dd0 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.631+0000 7f824df7a700 1 --2- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 0x7f8248197690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82380093e0 tx=0x7f8238003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.631+0000 7f8246ffd700 1 -- 172.21.15.7:0/3688516036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8238017040 con 0x7f8248101dd0 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.631+0000 7f8246ffd700 1 -- 172.21.15.7:0/3688516036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8238022070 con 0x7f8248101dd0 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.631+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8248197e60 con 0x7f8248101dd0 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.631+0000 7f8246ffd700 1 -- 172.21.15.7:0/3688516036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f823801d440 con 0x7f8248101dd0 2024-03-26T08:33:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.631+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824819bf60 con 0x7f8248101dd0 2024-03-26T08:33:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.632+0000 7f8246ffd700 1 -- 172.21.15.7:0/3688516036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f823801d6b0 con 0x7f8248101dd0 2024-03-26T08:33:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.632+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82481032b0 con 0x7f8248101dd0 2024-03-26T08:33:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.632+0000 7f8246ffd700 1 --2- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8230039070 0x7f823003b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.632+0000 7f8246ffd700 1 -- 172.21.15.7:0/3688516036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f823800f050 con 0x7f8248101dd0 2024-03-26T08:33:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.632+0000 7f824d779700 1 --2- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8230039070 0x7f823003b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.632+0000 7f824d779700 1 --2- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8230039070 0x7f823003b530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f823c006fd0 tx=0x7f823c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:32.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.637+0000 7f8246ffd700 1 -- 172.21.15.7:0/3688516036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8238007e20 con 0x7f8248101dd0 2024-03-26T08:33:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.846+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82480637c0 con 0x7f8230039070 2024-03-26T08:33:32.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.848+0000 7f8246ffd700 1 -- 172.21.15.7:0/3688516036 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+237 (secure 0 0 0) 0x7f82480637c0 con 0x7f8230039070 2024-03-26T08:33:32.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:33:32.841 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.44", "hostname": "smithi044", "labels": [], "status": ""}] 2024-03-26T08:33:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8230039070 msgr2=0x7f823003b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 --2- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8230039070 0x7f823003b530 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f823c006fd0 tx=0x7f823c006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 msgr2=0x7f8248197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 --2- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 0x7f8248197690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82380093e0 tx=0x7f8238003ce0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 shutdown_connections 2024-03-26T08:33:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 --2- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8230039070 0x7f823003b530 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 --2- 172.21.15.7:0/3688516036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8248101dd0 0x7f8248197690 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 >> 172.21.15.7:0/3688516036 conn(0x7f82480fd6a0 msgr2=0x7f82481037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 shutdown_connections 2024-03-26T08:33:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:32.850+0000 7f82501de700 1 -- 172.21.15.7:0/3688516036 wait complete. 2024-03-26T08:33:33.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:32 smithi007 ceph-mon[32462]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:33.405 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-26T08:33:33.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-26T08:33:33.643 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:33:34.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:34 smithi007 ceph-mon[32462]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:34.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.149+0000 7f3229bc5700 1 -- 172.21.15.7:0/3638614687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 msgr2=0x7f32241006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.149+0000 7f3229bc5700 1 --2- 172.21.15.7:0/3638614687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 0x7f32241006a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3214009910 tx=0x7f3214009c20 comp rx=0 tx=0).stop 2024-03-26T08:33:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.149+0000 7f3229bc5700 1 -- 172.21.15.7:0/3638614687 shutdown_connections 2024-03-26T08:33:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.149+0000 7f3229bc5700 1 --2- 172.21.15.7:0/3638614687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 0x7f32241006a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.149+0000 7f3229bc5700 1 -- 172.21.15.7:0/3638614687 >> 172.21.15.7:0/3638614687 conn(0x7f32240fbce0 msgr2=0x7f32240fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.149+0000 7f3229bc5700 1 -- 172.21.15.7:0/3638614687 shutdown_connections 2024-03-26T08:33:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.149+0000 7f3229bc5700 1 -- 172.21.15.7:0/3638614687 wait complete. 2024-03-26T08:33:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.149+0000 7f3229bc5700 1 Processor -- start 2024-03-26T08:33:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f3229bc5700 1 -- start start 2024-03-26T08:33:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f3229bc5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 0x7f3224197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f3229bc5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322410c1b0 con 0x7f32241002c0 2024-03-26T08:33:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f32237fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 0x7f3224197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f32237fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 0x7f3224197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58032/0 (socket says 172.21.15.7:58032) 2024-03-26T08:33:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f32237fe700 1 -- 172.21.15.7:0/1915112371 learned_addr learned my addr 172.21.15.7:0/1915112371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:33:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f32237fe700 1 -- 172.21.15.7:0/1915112371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224197bb0 con 0x7f32241002c0 2024-03-26T08:33:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f32237fe700 1 --2- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 0x7f3224197670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3214006010 tx=0x7f3214005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f3220ff9700 1 -- 172.21.15.7:0/1915112371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3214026070 con 0x7f32241002c0 2024-03-26T08:33:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f3220ff9700 1 -- 172.21.15.7:0/1915112371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f321400bbd0 con 0x7f32241002c0 2024-03-26T08:33:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3224197ea0 con 0x7f32241002c0 2024-03-26T08:33:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.150+0000 7f3220ff9700 1 -- 172.21.15.7:0/1915112371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3214003910 con 0x7f32241002c0 2024-03-26T08:33:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.151+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322419bf40 con 0x7f32241002c0 2024-03-26T08:33:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.151+0000 7f3220ff9700 1 -- 172.21.15.7:0/1915112371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f3214029070 con 0x7f32241002c0 2024-03-26T08:33:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.151+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3224103380 con 0x7f32241002c0 2024-03-26T08:33:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.152+0000 7f3220ff9700 1 --2- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f320c039020 0x7f320c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.152+0000 7f3220ff9700 1 -- 172.21.15.7:0/1915112371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f321401e070 con 0x7f32241002c0 2024-03-26T08:33:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.152+0000 7f3222ffd700 1 --2- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f320c039020 0x7f320c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.152+0000 7f3222ffd700 1 --2- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f320c039020 0x7f320c03b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3218006fd0 tx=0x7f3218006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:35.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.157+0000 7f3220ff9700 1 -- 172.21.15.7:0/1915112371 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3214013070 con 0x7f32241002c0 2024-03-26T08:33:35.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:35 smithi007 ceph-mon[32462]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:35.366+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f32241989d0 con 0x7f32241002c0 2024-03-26T08:33:36.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.044+0000 7f3220ff9700 1 -- 172.21.15.7:0/1915112371 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f321400fe20 con 0x7f32241002c0 2024-03-26T08:33:36.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f320c039020 msgr2=0x7f320c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 --2- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f320c039020 0x7f320c03b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3218006fd0 tx=0x7f3218006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 msgr2=0x7f3224197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 --2- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 0x7f3224197670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3214006010 tx=0x7f3214005030 comp rx=0 tx=0).stop 2024-03-26T08:33:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 shutdown_connections 2024-03-26T08:33:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 --2- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f320c039020 0x7f320c03b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 --2- 172.21.15.7:0/1915112371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32241002c0 0x7f3224197670 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 >> 172.21.15.7:0/1915112371 conn(0x7f32240fbce0 msgr2=0x7f32240fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 shutdown_connections 2024-03-26T08:33:36.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:33:36.047+0000 7f3229bc5700 1 -- 172.21.15.7:0/1915112371 wait complete. 2024-03-26T08:33:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-03-26T08:33:36.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:36 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1915112371' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T08:33:36.756 INFO:tasks.cephadm:Adding mon.a on smithi007 2024-03-26T08:33:36.756 INFO:tasks.cephadm:Adding mon.b on smithi016 2024-03-26T08:33:36.757 INFO:tasks.cephadm:Adding mon.c on smithi044 2024-03-26T08:33:36.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi044:172.21.15.44=c' 2024-03-26T08:33:37.002 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:33:37.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1915112371' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T08:33:37.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:33:37.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:37.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:37.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:37.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:37.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:33:37.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:37.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:37.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-26T08:33:37.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:37 smithi007 ceph-mon[32462]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:38.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:38 smithi007 ceph-mon[32462]: Updating smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:33:38.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:38 smithi007 ceph-mon[32462]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:38.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:38 smithi007 ceph-mon[32462]: Updating smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:33:38.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:38.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:38.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.604+0000 7fd914464700 1 -- 172.21.15.44:0/1876633553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 msgr2=0x7fd90c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:38.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.604+0000 7fd914464700 1 --2- 172.21.15.44:0/1876633553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 0x7fd90c1021b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd908009910 tx=0x7fd908009c20 comp rx=0 tx=0).stop 2024-03-26T08:33:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.604+0000 7fd914464700 1 -- 172.21.15.44:0/1876633553 shutdown_connections 2024-03-26T08:33:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.604+0000 7fd914464700 1 --2- 172.21.15.44:0/1876633553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 0x7fd90c1021b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.604+0000 7fd914464700 1 -- 172.21.15.44:0/1876633553 >> 172.21.15.44:0/1876633553 conn(0x7fd90c0fd6a0 msgr2=0x7fd90c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.604+0000 7fd914464700 1 -- 172.21.15.44:0/1876633553 shutdown_connections 2024-03-26T08:33:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.604+0000 7fd914464700 1 -- 172.21.15.44:0/1876633553 wait complete. 2024-03-26T08:33:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.605+0000 7fd914464700 1 Processor -- start 2024-03-26T08:33:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.605+0000 7fd914464700 1 -- start start 2024-03-26T08:33:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.605+0000 7fd914464700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 0x7fd90c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:38.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.605+0000 7fd914464700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c10c1a0 con 0x7fd90c101dd0 2024-03-26T08:33:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.605+0000 7fd912200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 0x7fd90c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.606+0000 7fd912200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 0x7fd90c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.44:39598/0 (socket says 172.21.15.44:39598) 2024-03-26T08:33:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.606+0000 7fd912200700 1 -- 172.21.15.44:0/263056293 learned_addr learned my addr 172.21.15.44:0/263056293 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:33:38.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.606+0000 7fd912200700 1 -- 172.21.15.44:0/263056293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c197c80 con 0x7fd90c101dd0 2024-03-26T08:33:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.606+0000 7fd912200700 1 --2- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 0x7fd90c197740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd908005850 tx=0x7fd908005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.606+0000 7fd9037fe700 1 -- 172.21.15.44:0/263056293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd908026070 con 0x7fd90c101dd0 2024-03-26T08:33:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.606+0000 7fd9037fe700 1 -- 172.21.15.44:0/263056293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90800bbd0 con 0x7fd90c101dd0 2024-03-26T08:33:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.606+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c197f70 con 0x7fd90c101dd0 2024-03-26T08:33:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.606+0000 7fd9037fe700 1 -- 172.21.15.44:0/263056293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd908003970 con 0x7fd90c101dd0 2024-03-26T08:33:38.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.606+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c19c010 con 0x7fd90c101dd0 2024-03-26T08:33:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.607+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90c1032b0 con 0x7fd90c101dd0 2024-03-26T08:33:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.607+0000 7fd9037fe700 1 -- 172.21.15.44:0/263056293 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7fd908029070 con 0x7fd90c101dd0 2024-03-26T08:33:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.608+0000 7fd9037fe700 1 --2- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8f8039020 0x7fd8f803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.608+0000 7fd9037fe700 1 -- 172.21.15.44:0/263056293 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd90801e070 con 0x7fd90c101dd0 2024-03-26T08:33:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.608+0000 7fd9119ff700 1 --2- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8f8039020 0x7fd8f803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:38.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.608+0000 7fd9119ff700 1 --2- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8f8039020 0x7fd8f803b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc006fd0 tx=0x7fd8fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:38.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.613+0000 7fd9037fe700 1 -- 172.21.15.44:0/263056293 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd908013070 con 0x7fd90c101dd0 2024-03-26T08:33:38.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.831+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi044:172.21.15.44=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fd90c198aa0 con 0x7fd8f8039020 2024-03-26T08:33:38.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.837+0000 7fd9037fe700 1 -- 172.21.15.44:0/263056293 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd90c198aa0 con 0x7fd8f8039020 2024-03-26T08:33:38.830 INFO:teuthology.orchestra.run.smithi044.stdout:Scheduled mon update... 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8f8039020 msgr2=0x7fd8f803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 --2- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8f8039020 0x7fd8f803b4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc006fd0 tx=0x7fd8fc006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 msgr2=0x7fd90c197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 --2- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 0x7fd90c197740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd908005850 tx=0x7fd908005030 comp rx=0 tx=0).stop 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 shutdown_connections 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 --2- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8f8039020 0x7fd8f803b4e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 --2- 172.21.15.44:0/263056293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd90c101dd0 0x7fd90c197740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 >> 172.21.15.44:0/263056293 conn(0x7fd90c0fd6a0 msgr2=0x7fd90c104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 shutdown_connections 2024-03-26T08:33:38.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:38.839+0000 7fd914464700 1 -- 172.21.15.44:0/263056293 wait complete. 2024-03-26T08:33:39.434 DEBUG:teuthology.orchestra.run.smithi016:mon.b> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.b.service 2024-03-26T08:33:39.437 DEBUG:teuthology.orchestra.run.smithi044:mon.c> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.c.service 2024-03-26T08:33:39.439 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T08:33:39.439 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T08:33:39.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:39 smithi007 ceph-mon[32462]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:39.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:39.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:39.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:39.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:39.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:39.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:33:39.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:40.473 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:33:40.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:40 smithi007 ceph-mon[32462]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi044:172.21.15.44=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:40.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:40 smithi007 ceph-mon[32462]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi044:172.21.15.44=c;count:3 2024-03-26T08:33:40.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:40 smithi007 ceph-mon[32462]: Deploying daemon mon.c on smithi044 2024-03-26T08:33:41.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:41 smithi007 ceph-mon[32462]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:42.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:42 smithi007 ceph-mon[32462]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:43.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 podman[36411]: 2024-03-26 08:33:43.278036596 +0000 UTC m=+0.319208917 container create d20b903c79fca76a44e1453ae9b2ef93404a52ba3ed1dd066fffd913eab4260e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-26T08:33:43.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.467+0000 7f62faef1700 1 -- 172.21.15.44:0/3273574029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 msgr2=0x7f62f4106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:43.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.467+0000 7f62faef1700 1 --2- 172.21.15.44:0/3273574029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 0x7f62f4106e40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009910 tx=0x7f62e8009c20 comp rx=0 tx=0).stop 2024-03-26T08:33:43.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.467+0000 7f62faef1700 1 -- 172.21.15.44:0/3273574029 shutdown_connections 2024-03-26T08:33:43.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.467+0000 7f62faef1700 1 --2- 172.21.15.44:0/3273574029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 0x7f62f4106e40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:43.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.467+0000 7f62faef1700 1 -- 172.21.15.44:0/3273574029 >> 172.21.15.44:0/3273574029 conn(0x7f62f40776d0 msgr2=0x7f62f40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:43.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.467+0000 7f62faef1700 1 -- 172.21.15.44:0/3273574029 shutdown_connections 2024-03-26T08:33:43.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.467+0000 7f62faef1700 1 -- 172.21.15.44:0/3273574029 wait complete. 2024-03-26T08:33:43.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.468+0000 7f62faef1700 1 Processor -- start 2024-03-26T08:33:43.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.468+0000 7f62faef1700 1 -- start start 2024-03-26T08:33:43.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.468+0000 7f62faef1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 0x7f62f4197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:43.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.468+0000 7f62faef1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f410b0a0 con 0x7f62f4106a60 2024-03-26T08:33:43.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.468+0000 7f62f8c8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 0x7f62f4197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:43.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.468+0000 7f62f8c8d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 0x7f62f4197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.44:53864/0 (socket says 172.21.15.44:53864) 2024-03-26T08:33:43.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.469+0000 7f62f8c8d700 1 -- 172.21.15.44:0/1288875670 learned_addr learned my addr 172.21.15.44:0/1288875670 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:33:43.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.469+0000 7f62f8c8d700 1 -- 172.21.15.44:0/1288875670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4197b90 con 0x7f62f4106a60 2024-03-26T08:33:43.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.469+0000 7f62f8c8d700 1 --2- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 0x7f62f4197650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005850 tx=0x7f62e8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:43.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.469+0000 7f62f1ffb700 1 -- 172.21.15.44:0/1288875670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e8026070 con 0x7f62f4106a60 2024-03-26T08:33:43.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.469+0000 7f62f1ffb700 1 -- 172.21.15.44:0/1288875670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f62e800bbd0 con 0x7f62f4106a60 2024-03-26T08:33:43.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.469+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f4197e80 con 0x7f62f4106a60 2024-03-26T08:33:43.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.469+0000 7f62f1ffb700 1 -- 172.21.15.44:0/1288875670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62e8003970 con 0x7f62f4106a60 2024-03-26T08:33:43.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.469+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f419bf20 con 0x7f62f4106a60 2024-03-26T08:33:43.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.470+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f41021a0 con 0x7f62f4106a60 2024-03-26T08:33:43.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.470+0000 7f62f1ffb700 1 -- 172.21.15.44:0/1288875670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f62e8029070 con 0x7f62f4106a60 2024-03-26T08:33:43.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.471+0000 7f62f1ffb700 1 --2- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f62e4038fc0 0x7f62e403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:43.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.471+0000 7f62f1ffb700 1 -- 172.21.15.44:0/1288875670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f62e801e070 con 0x7f62f4106a60 2024-03-26T08:33:43.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.471+0000 7f62f3fff700 1 --2- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f62e4038fc0 0x7f62e403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:43.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.471+0000 7f62f3fff700 1 --2- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f62e4038fc0 0x7f62e403b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f62e0006fd0 tx=0x7f62e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:43.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.476+0000 7f62f1ffb700 1 -- 172.21.15.44:0/1288875670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f62e8013070 con 0x7f62f4106a60 2024-03-26T08:33:43.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.719+0000 7f62f1ffb700 1 -- 172.21.15.44:0/1288875670 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f62e8029360 con 0x7f62f4106a60 2024-03-26T08:33:43.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:43.784+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f62f41989b0 con 0x7f62f4106a60 2024-03-26T08:33:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 podman[36411]: 2024-03-26 08:33:43.664716153 +0000 UTC m=+0.705888468 container init d20b903c79fca76a44e1453ae9b2ef93404a52ba3ed1dd066fffd913eab4260e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:33:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 podman[36411]: 2024-03-26 08:33:43.667808529 +0000 UTC m=+0.708980823 container start d20b903c79fca76a44e1453ae9b2ef93404a52ba3ed1dd066fffd913eab4260e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:33:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T08:33:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pidfile_write: ignore empty --pid-file 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: load: jerasure load: lrc 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: RocksDB version: 7.9.2 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Git sha 0 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: DB SUMMARY 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: DB Session ID: 2Z4U3TDAQ6U7DRA83QQG 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: CURRENT file: CURRENT 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-26T08:33:44.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.error_if_exists: 0 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.create_if_missing: 0 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.paranoid_checks: 1 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.env: 0x55b879a38b20 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.info_log: 0x55b87aca1920 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.statistics: (nil) 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.use_fsync: 0 2024-03-26T08:33:44.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_log_file_size: 0 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.allow_fallocate: 1 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.use_direct_reads: 0 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.db_log_dir: 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.wal_dir: 2024-03-26T08:33:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.write_buffer_manager: 0x55b87ac741e0 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T08:33:44.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T08:33:44.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T08:33:44.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.unordered_write: 0 2024-03-26T08:33:44.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T08:33:44.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T08:33:44.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T08:33:44.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T08:33:44.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.row_cache: None 2024-03-26T08:33:44.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.wal_filter: None 2024-03-26T08:33:44.073 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.two_write_queues: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.wal_compression: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.atomic_flush: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.log_readahead_size: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T08:33:44.074 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_background_jobs: 2 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_background_compactions: -1 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_subcompactions: 1 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T08:33:44.075 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_open_files: -1 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_background_flushes: -1 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Compression algorithms supported: 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: kZSTD supported: 0 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: kXpressCompression supported: 0 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: kZlibCompression supported: 1 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: kSnappyCompression supported: 1 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: kLZ4Compression supported: 1 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: kBZip2Compression supported: 0 2024-03-26T08:33:44.076 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.merge_operator: 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_filter: None 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b87aca1a20) 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout: cache_index_and_filter_blocks: 1 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T08:33:44.077 INFO:journalctl@ceph.mon.c.smithi044.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: pin_top_level_index_and_filter: 1 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: index_type: 0 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: data_block_index_type: 0 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: index_shortening: 1 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: checksum: 4 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: no_block_cache: 0 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: block_cache: 0x55b87aca7090 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: block_cache_name: BinnedLRUCache 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: block_cache_options: 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: capacity : 536870912 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: num_shard_bits : 4 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: strict_capacity_limit : 0 2024-03-26T08:33:44.078 INFO:journalctl@ceph.mon.c.smithi044.stdout: high_pri_pool_ratio: 0.000 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: block_cache_compressed: (nil) 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: persistent_cache: (nil) 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: block_size: 4096 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: block_size_deviation: 10 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: block_restart_interval: 16 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: index_block_restart_interval: 1 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: metadata_block_size: 4096 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: partition_filters: 0 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: use_delta_encoding: 1 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: filter_policy: bloomfilter 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: whole_key_filtering: 1 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: verify_compression: 0 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: read_amp_bytes_per_bit: 0 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: format_version: 5 2024-03-26T08:33:44.079 INFO:journalctl@ceph.mon.c.smithi044.stdout: enable_index_compression: 1 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout: block_align: 0 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout: max_auto_readahead_size: 262144 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout: prepopulate_block_cache: 0 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout: initial_auto_readahead_size: 8192 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression: NoCompression 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.num_levels: 7 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T08:33:44.080 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T08:33:44.081 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T08:33:44.082 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T08:33:44.083 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.table_properties_collectors: 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.inplace_update_support: 0 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.bloom_locality: 0 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.max_successive_merges: 0 2024-03-26T08:33:44.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.ttl: 2592000 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.enable_blob_files: false 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.min_blob_size: 0 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T08:33:44.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 96ec8a35-c03e-4291-b5d1-116f04315c72 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711442023697851, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711442023698302, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711442023, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "96ec8a35-c03e-4291-b5d1-116f04315c72", "db_session_id": "2Z4U3TDAQ6U7DRA83QQG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711442023698384, "job": 1, "event": "recovery_finished"} 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b87ad9a000 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: DB pointer 0x55b87ad84000 2024-03-26T08:33:44.086 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: ** DB Stats ** 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: ** Compaction Stats [default] ** 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T08:33:44.087 INFO:journalctl@ceph.mon.c.smithi044.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: ** Compaction Stats [default] ** 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T08:33:44.088 INFO:journalctl@ceph.mon.c.smithi044.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout: Block cache BinnedLRUCache@0x55b87aca7090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout: 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: using public_addr v2:172.21.15.44:0/0 -> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: starting mon.c rank -1 at public addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] at bind addrs [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(???) e0 preinit fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).mds e1 new map 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout: e1 2024-03-26T08:33:44.089 INFO:journalctl@ceph.mon.c.smithi044.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout: legacy client fscid: -1 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout: 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout: No filesystems configured 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:33:44.090 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Deploying cephadm binary to smithi044 2024-03-26T08:33:44.091 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Updating smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Updating smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Added host smithi044 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:44.092 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1915112371' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1915112371' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Updating smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:33:44.093 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Updating smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi044:172.21.15.44=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi044:172.21.15.44=c;count:3 2024-03-26T08:33:44.094 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: Deploying daemon mon.c on smithi044 2024-03-26T08:33:44.095 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.095 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:44.095 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 ceph-mon[36461]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T08:33:44.095 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 bash[36411]: d20b903c79fca76a44e1453ae9b2ef93404a52ba3ed1dd066fffd913eab4260e 2024-03-26T08:33:44.095 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:43 smithi044 systemd[1]: Started Ceph mon.c for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:33:48.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.767+0000 7f62f1ffb700 1 -- 172.21.15.44:0/1288875670 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1007 (secure 0 0 0) 0x7f62e8030090 con 0x7f62f4106a60 2024-03-26T08:33:48.760 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:33:48.761 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","modified":"2024-03-26T08:33:43.720876Z","created":"2024-03-26T08:31:52.923392Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T08:33:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f62e4038fc0 msgr2=0x7f62e403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 --2- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f62e4038fc0 0x7f62e403b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f62e0006fd0 tx=0x7f62e0006e40 comp rx=0 tx=0).stop 2024-03-26T08:33:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 msgr2=0x7f62f4197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 --2- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 0x7f62f4197650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005850 tx=0x7f62e8005030 comp rx=0 tx=0).stop 2024-03-26T08:33:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 shutdown_connections 2024-03-26T08:33:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 --2- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f62e4038fc0 0x7f62e403b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 --2- 172.21.15.44:0/1288875670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62f4106a60 0x7f62f4197650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:48.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 >> 172.21.15.44:0/1288875670 conn(0x7f62f40776d0 msgr2=0x7f62f4104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 shutdown_connections 2024-03-26T08:33:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:48.770+0000 7f62faef1700 1 -- 172.21.15.44:0/1288875670 wait complete. 2024-03-26T08:33:48.764 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 2 2024-03-26T08:33:49.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:33:49.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: mon.a calling monitor election 2024-03-26T08:33:49.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='client.? 172.21.15.44:0/1288875670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:33:49.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:49.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: mon.c calling monitor election 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: monmap e2: 2 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],c=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: fsmap 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: mgrmap e13: x(active, since 50s) 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: overall HEALTH_OK 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:33:49.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:49.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:33:49.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: mon.a calling monitor election 2024-03-26T08:33:49.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='client.? 172.21.15.44:0/1288875670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:33:49.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: mon.c calling monitor election 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: monmap e2: 2 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],c=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: fsmap 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: mgrmap e13: x(active, since 50s) 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: overall HEALTH_OK 2024-03-26T08:33:49.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:49.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:49.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:49.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:33:49.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:50.135 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:49 smithi007 ceph-mon[32462]: Deploying daemon mon.b on smithi016 2024-03-26T08:33:50.135 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:50.197 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:49 smithi044 ceph-mon[36461]: Deploying daemon mon.b on smithi016 2024-03-26T08:33:50.197 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:50.395 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T08:33:50.396 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T08:33:50.654 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:33:51.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:50 smithi044 ceph-mon[36461]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:51.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:50 smithi007 ceph-mon[32462]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.072+0000 7f6793d4a700 1 -- 172.21.15.44:0/3447797527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 msgr2=0x7f678c1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.072+0000 7f6793d4a700 1 --2- 172.21.15.44:0/3447797527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 0x7f678c1011f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f677c009910 tx=0x7f677c009c20 comp rx=0 tx=0).stop 2024-03-26T08:33:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.072+0000 7f6793d4a700 1 -- 172.21.15.44:0/3447797527 shutdown_connections 2024-03-26T08:33:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.072+0000 7f6793d4a700 1 --2- 172.21.15.44:0/3447797527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 0x7f678c1011f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.072+0000 7f6793d4a700 1 -- 172.21.15.44:0/3447797527 >> 172.21.15.44:0/3447797527 conn(0x7f678c0fc6a0 msgr2=0x7f678c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.072+0000 7f6793d4a700 1 -- 172.21.15.44:0/3447797527 shutdown_connections 2024-03-26T08:33:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.072+0000 7f6793d4a700 1 -- 172.21.15.44:0/3447797527 wait complete. 2024-03-26T08:33:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6793d4a700 1 Processor -- start 2024-03-26T08:33:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6793d4a700 1 -- start start 2024-03-26T08:33:52.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6793d4a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 0x7f678c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6793d4a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 0x7f678c1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6793d4a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c10b1e0 con 0x7f678c100e10 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6793d4a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c10b060 con 0x7f678c19cb60 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6791ae6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 0x7f678c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f67912e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 0x7f678c1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6791ae6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 0x7f678c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.44:59500/0 (socket says 172.21.15.44:59500) 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6791ae6700 1 -- 172.21.15.44:0/2019526334 learned_addr learned my addr 172.21.15.44:0/2019526334 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f67912e5700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 0x7f678c1966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42074/0 (socket says 172.21.15.44:42074) 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f67912e5700 1 -- 172.21.15.44:0/2019526334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 msgr2=0x7f678c1966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f67912e5700 1 -- 172.21.15.44:0/2019526334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 msgr2=0x7f678c1966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f67912e5700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 0x7f678c1966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f67912e5700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 0x7f678c1966f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:33:52.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6791ae6700 1 -- 172.21.15.44:0/2019526334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 msgr2=0x7f678c1966f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:33:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.073+0000 7f6791ae6700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 0x7f678c1966f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.074+0000 7f6791ae6700 1 -- 172.21.15.44:0/2019526334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c19d0e0 con 0x7f678c100e10 2024-03-26T08:33:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.074+0000 7f6791ae6700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 0x7f678c19c620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f677c005b40 tx=0x7f677c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.074+0000 7f6782ffd700 1 -- 172.21.15.44:0/2019526334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f677c005dc0 con 0x7f678c100e10 2024-03-26T08:33:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.074+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678c196c90 con 0x7f678c100e10 2024-03-26T08:33:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.074+0000 7f6782ffd700 1 -- 172.21.15.44:0/2019526334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f677c00be40 con 0x7f678c100e10 2024-03-26T08:33:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.074+0000 7f6782ffd700 1 -- 172.21.15.44:0/2019526334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f677c00f460 con 0x7f678c100e10 2024-03-26T08:33:52.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.074+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c197220 con 0x7f678c100e10 2024-03-26T08:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.075+0000 7f6782ffd700 1 -- 172.21.15.44:0/2019526334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f677c02a070 con 0x7f678c100e10 2024-03-26T08:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.075+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c1022f0 con 0x7f678c100e10 2024-03-26T08:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.075+0000 7f6782ffd700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6778038d60 0x7f677803b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.075+0000 7f6782ffd700 1 -- 172.21.15.44:0/2019526334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f677c01e030 con 0x7f678c100e10 2024-03-26T08:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.075+0000 7f67912e5700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6778038d60 0x7f677803b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:52.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.076+0000 7f67912e5700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6778038d60 0x7f677803b220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6788009850 tx=0x7f6788008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:52.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.081+0000 7f6782ffd700 1 -- 172.21.15.44:0/2019526334 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f677c013070 con 0x7f678c100e10 2024-03-26T08:33:52.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.370+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f678c197590 con 0x7f678c100e10 2024-03-26T08:33:52.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.370+0000 7f6782ffd700 1 -- 172.21.15.44:0/2019526334 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1007 (secure 0 0 0) 0x7f677c031070 con 0x7f678c100e10 2024-03-26T08:33:52.363 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:33:52.363 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","modified":"2024-03-26T08:33:43.720876Z","created":"2024-03-26T08:31:52.923392Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T08:33:52.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6778038d60 msgr2=0x7f677803b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:52.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6778038d60 0x7f677803b220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6788009850 tx=0x7f6788008040 comp rx=0 tx=0).stop 2024-03-26T08:33:52.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 msgr2=0x7f678c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:52.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 0x7f678c19c620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f677c005b40 tx=0x7f677c005050 comp rx=0 tx=0).stop 2024-03-26T08:33:52.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 shutdown_connections 2024-03-26T08:33:52.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6778038d60 0x7f677803b220 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:52.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f678c100e10 0x7f678c19c620 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:52.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 --2- 172.21.15.44:0/2019526334 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f678c19cb60 0x7f678c1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:52.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 >> 172.21.15.44:0/2019526334 conn(0x7f678c0fc6a0 msgr2=0x7f678c102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:52.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.372+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 shutdown_connections 2024-03-26T08:33:52.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:52.373+0000 7f6793d4a700 1 -- 172.21.15.44:0/2019526334 wait complete. 2024-03-26T08:33:52.366 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 2 2024-03-26T08:33:52.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:52 smithi007 ceph-mon[32462]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:52 smithi044 ceph-mon[36461]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.633 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711442032639195, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711442032, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bd127ac3-7a0a-4d61-bc6c-9b43457ec0fd", "db_session_id": "KDAXL8D7JS54UAJP36HF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711442032639278, "job": 1, "event": "recovery_finished"} 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55773419c000 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: rocksdb: DB pointer 0x557734186000 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** DB Stats ** 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T08:33:52.634 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** Compaction Stats [default] ** 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** Compaction Stats [default] ** 2024-03-26T08:33:52.635 INFO:journalctl@ceph.mon.b.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T08:33:52.636 INFO:journalctl@ceph.mon.b.smithi016.stdout: Block cache BinnedLRUCache@0x5577340a9090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-26T08:33:52.637 INFO:journalctl@ceph.mon.b.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-26T08:33:52.637 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-03-26T08:33:52.637 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T08:33:52.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-26T08:33:52.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: using public_addr v2:172.21.15.16:0/0 -> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] 2024-03-26T08:33:52.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: starting mon.b rank -1 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:33:52.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(???) e0 preinit fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:33:52.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).mds e1 new map 2024-03-26T08:33:52.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-26T08:33:52.994 INFO:journalctl@ceph.mon.b.smithi016.stdout: e1 2024-03-26T08:33:52.994 INFO:journalctl@ceph.mon.b.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T08:33:52.994 INFO:journalctl@ceph.mon.b.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout: legacy client fscid: -1 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout: No filesystems configured 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:33:52.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi044", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-26T08:33:52.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Deploying cephadm binary to smithi044 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Updating smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Updating smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Added host smithi044 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1915112371' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1915112371' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi044", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:52.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-26T08:33:52.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:52.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Updating smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:33:52.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Updating smithi044:/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:33:52.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Updating smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T08:33:52.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:52.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi044:172.21.15.44=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi044:172.21.15.44=c;count:3 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Deploying daemon mon.c on smithi044 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:33:53.000 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.a calling monitor election 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='client.? 172.21.15.44:0/1288875670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.c calling monitor election 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: monmap e2: 2 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],c=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: fsmap 2024-03-26T08:33:53.001 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mgrmap e13: x(active, since 50s) 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: overall HEALTH_OK 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: Deploying daemon mon.b on smithi016 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 ceph-mon[35851]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 bash[35832]: d98f5c5ac880a7b44e6be687beee07e02f3926286ac7bc8aa1977bf0d1a1734b 2024-03-26T08:33:53.002 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:52 smithi016 systemd[1]: Started Ceph mon.b for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:33:53.867 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T08:33:53.867 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T08:33:54.125 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:33:58.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: mon.c calling monitor election 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: mon.a calling monitor election 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T08:33:58.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: monmap e3: 3 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],b=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:33:58.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: fsmap 2024-03-26T08:33:58.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:33:58.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: mgrmap e13: x(active, since 59s) 2024-03-26T08:33:58.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-26T08:33:58.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-26T08:33:58.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-26T08:33:58.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: mon.b (rank 2) addr [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] is down (out of quorum) 2024-03-26T08:33:58.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:58.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:58.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: mon.c calling monitor election 2024-03-26T08:33:58.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:33:58.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: mon.a calling monitor election 2024-03-26T08:33:58.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:58.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: monmap e3: 3 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],b=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: fsmap 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: mgrmap e13: x(active, since 59s) 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-26T08:33:58.068 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-26T08:33:58.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: mon.b (rank 2) addr [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] is down (out of quorum) 2024-03-26T08:33:58.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:58.069 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:58.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.648+0000 7f599005d700 1 -- 172.21.15.44:0/266215826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59740035c0 msgr2=0x7f5974005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:58.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.648+0000 7f599005d700 1 --2- 172.21.15.44:0/266215826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59740035c0 0x7f5974005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5988101c10 tx=0x7f598400feb0 comp rx=0 tx=0).stop 2024-03-26T08:33:58.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.648+0000 7f599005d700 1 -- 172.21.15.44:0/266215826 shutdown_connections 2024-03-26T08:33:58.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.648+0000 7f599005d700 1 --2- 172.21.15.44:0/266215826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59740035c0 0x7f5974005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:58.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.648+0000 7f599005d700 1 -- 172.21.15.44:0/266215826 >> 172.21.15.44:0/266215826 conn(0x7f59880fd7b0 msgr2=0x7f59880ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:58.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.648+0000 7f599005d700 1 -- 172.21.15.44:0/266215826 shutdown_connections 2024-03-26T08:33:58.641 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.648+0000 7f599005d700 1 -- 172.21.15.44:0/266215826 wait complete. 2024-03-26T08:33:58.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.649+0000 7f599005d700 1 Processor -- start 2024-03-26T08:33:58.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.649+0000 7f599005d700 1 -- start start 2024-03-26T08:33:58.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f599005d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 0x7f598819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:58.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f599005d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f598819dbd0 0x7f5988197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:58.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f599005d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988197ca0 0x7f5988198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:58.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f599005d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988109d20 con 0x7f598819dbd0 2024-03-26T08:33:58.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f599005d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988109ba0 con 0x7f5988197ca0 2024-03-26T08:33:58.644 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f599005d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988109ea0 con 0x7f59740035c0 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598ddf9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 0x7f598819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598d5f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f598819dbd0 0x7f5988197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598ddf9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 0x7f598819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42082/0 (socket says 172.21.15.44:42082) 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598ddf9700 1 -- 172.21.15.44:0/954588639 learned_addr learned my addr 172.21.15.44:0/954588639 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598ddf9700 1 -- 172.21.15.44:0/954588639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 msgr2=0x7f598819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598ddf9700 1 -- 172.21.15.44:0/954588639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 msgr2=0x7f598819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598ddf9700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 0x7f598819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598ddf9700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 0x7f598819d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598e5fa700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988197ca0 0x7f5988198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598d5f8700 1 -- 172.21.15.44:0/954588639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 msgr2=0x7f598819d690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598d5f8700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 0x7f598819d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598d5f8700 1 -- 172.21.15.44:0/954588639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988197ca0 msgr2=0x7f5988198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598e5fa700 1 -- 172.21.15.44:0/954588639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988197ca0 msgr2=0x7f5988198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-26T08:33:58.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598e5fa700 1 -- 172.21.15.44:0/954588639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988197ca0 msgr2=0x7f5988198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598e5fa700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988197ca0 0x7f5988198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598e5fa700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988197ca0 0x7f5988198150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598d5f8700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988197ca0 0x7f5988198150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598d5f8700 1 -- 172.21.15.44:0/954588639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59881a4440 con 0x7f598819dbd0 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.650+0000 7f598d5f8700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f598819dbd0 0x7f5988197760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f597800b550 tx=0x7f597800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.651+0000 7f597effd700 1 -- 172.21.15.44:0/954588639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978004230 con 0x7f598819dbd0 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.651+0000 7f597effd700 1 -- 172.21.15.44:0/954588639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59780043d0 con 0x7f598819dbd0 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.651+0000 7f597effd700 1 -- 172.21.15.44:0/954588639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978011c20 con 0x7f598819dbd0 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.651+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59881a46d0 con 0x7f598819dbd0 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.651+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59881a4c60 con 0x7f598819dbd0 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.652+0000 7f597effd700 1 -- 172.21.15.44:0/954588639 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f59780036a0 con 0x7f598819dbd0 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.652+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5988103420 con 0x7f598819dbd0 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.652+0000 7f597effd700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f596c039450 0x7f596c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:33:58.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.652+0000 7f597effd700 1 -- 172.21.15.44:0/954588639 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f597804bc50 con 0x7f598819dbd0 2024-03-26T08:33:58.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.652+0000 7f598ddf9700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f596c039450 0x7f596c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:33:58.647 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.652+0000 7f598ddf9700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f596c039450 0x7f596c03b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f598400fa60 tx=0x7f5984005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:33:58.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.657+0000 7f597effd700 1 -- 172.21.15.44:0/954588639 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5978021e00 con 0x7f598819dbd0 2024-03-26T08:33:58.905 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:58.906 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:58.906 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:58.906 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:58 smithi044 ceph-mon[36461]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:58.906 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:58.906 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:33:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:58.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.954+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f598802da30 con 0x7f598819dbd0 2024-03-26T08:33:58.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.954+0000 7f597effd700 1 -- 172.21.15.44:0/954588639 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1271 (secure 0 0 0) 0x7f5978003d30 con 0x7f598819dbd0 2024-03-26T08:33:58.947 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-03-26T08:33:58.947 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":3,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","modified":"2024-03-26T08:33:52.663106Z","created":"2024-03-26T08:31:52.923392Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:3300","nonce":0},{"type":"v1","addr":"172.21.15.44:6789","nonce":0}]},"addr":"172.21.15.44:6789/0","public_addr":"172.21.15.44:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T08:33:58.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.956+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f596c039450 msgr2=0x7f596c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:58.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.956+0000 7f599005d700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f596c039450 0x7f596c03b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f598400fa60 tx=0x7f5984005a90 comp rx=0 tx=0).stop 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.956+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f598819dbd0 msgr2=0x7f5988197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.956+0000 7f599005d700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f598819dbd0 0x7f5988197760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f597800b550 tx=0x7f597800b910 comp rx=0 tx=0).stop 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.956+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 shutdown_connections 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.957+0000 7f599005d700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f596c039450 0x7f596c03b910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.957+0000 7f599005d700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f59740035c0 0x7f598819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.957+0000 7f599005d700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5988197ca0 0x7f5988198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.957+0000 7f599005d700 1 --2- 172.21.15.44:0/954588639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f598819dbd0 0x7f5988197760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.957+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 >> 172.21.15.44:0/954588639 conn(0x7f59880fd7b0 msgr2=0x7f59880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.957+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 shutdown_connections 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:33:58.957+0000 7f599005d700 1 -- 172.21.15.44:0/954588639 wait complete. 2024-03-26T08:33:58.950 INFO:teuthology.orchestra.run.smithi044.stderr:dumped monmap epoch 3 2024-03-26T08:33:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:58 smithi007 ceph-mon[32462]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:33:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:59.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: mon.c calling monitor election 2024-03-26T08:33:59.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:33:59.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: mon.a calling monitor election 2024-03-26T08:33:59.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:59.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:33:59.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:59.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: monmap e3: 3 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],b=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: fsmap 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: mgrmap e13: x(active, since 59s) 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-26T08:33:59.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: mon.b (rank 2) addr [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] is down (out of quorum) 2024-03-26T08:33:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:33:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:33:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:33:59.996 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:33:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:34:00.125 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-26T08:34:00.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-26T08:34:00.365 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: mon.b calling monitor election 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Updating smithi007:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: mon.b calling monitor election 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: mon.c calling monitor election 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: mon.a calling monitor election 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Updating smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Updating smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:34:00.491 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: monmap e3: 3 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],b=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: fsmap 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: mgrmap e13: x(active, since 61s) 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Cluster is now healthy 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: overall HEALTH_OK 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: Reconfiguring daemon mon.a on smithi007 2024-03-26T08:34:00.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:00 smithi007 ceph-mon[32462]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:00.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: mon.b calling monitor election 2024-03-26T08:34:00.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-26T08:34:00.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-26T08:34:00.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-26T08:34:00.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Updating smithi007:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: mon.b calling monitor election 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: mon.c calling monitor election 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: mon.a calling monitor election 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Updating smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Updating smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: monmap e3: 3 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],b=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: fsmap 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: mgrmap e13: x(active, since 61s) 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Cluster is now healthy 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: overall HEALTH_OK 2024-03-26T08:34:00.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:34:00.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:34:00.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:00.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: Reconfiguring daemon mon.a on smithi007 2024-03-26T08:34:00.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:00 smithi016 ceph-mon[35851]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:00.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: mon.b calling monitor election 2024-03-26T08:34:00.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Updating smithi007:/etc/ceph/ceph.conf 2024-03-26T08:34:00.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Updating smithi016:/etc/ceph/ceph.conf 2024-03-26T08:34:00.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Updating smithi044:/etc/ceph/ceph.conf 2024-03-26T08:34:00.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Updating smithi007:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:34:00.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: mon.b calling monitor election 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: mon.c calling monitor election 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: mon.a calling monitor election 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Updating smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Updating smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: monmap e3: 3 mons at {a=[v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0],b=[v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0],c=[v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: fsmap 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: mgrmap e13: x(active, since 61s) 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Cluster is now healthy 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: overall HEALTH_OK 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: Reconfiguring daemon mon.a on smithi007 2024-03-26T08:34:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:00 smithi044 ceph-mon[36461]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:01.446 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:34:01.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:34:01.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:34:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.001+0000 7f34caaf6700 1 -- 172.21.15.7:0/881363543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101070 msgr2=0x7f34c41014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.001+0000 7f34caaf6700 1 --2- 172.21.15.7:0/881363543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101070 0x7f34c41014f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009990 tx=0x7f34b0009ca0 comp rx=0 tx=0).stop 2024-03-26T08:34:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.001+0000 7f34caaf6700 1 -- 172.21.15.7:0/881363543 shutdown_connections 2024-03-26T08:34:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.001+0000 7f34caaf6700 1 --2- 172.21.15.7:0/881363543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c4107120 0x7f34c4107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.001+0000 7f34caaf6700 1 --2- 172.21.15.7:0/881363543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c4101a30 0x7f34c410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.001+0000 7f34caaf6700 1 --2- 172.21.15.7:0/881363543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101070 0x7f34c41014f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.001+0000 7f34caaf6700 1 -- 172.21.15.7:0/881363543 >> 172.21.15.7:0/881363543 conn(0x7f34c40780a0 msgr2=0x7f34c40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 -- 172.21.15.7:0/881363543 shutdown_connections 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 -- 172.21.15.7:0/881363543 wait complete. 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 Processor -- start 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 -- start start 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c4101070 0x7f34c419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101a30 0x7f34c419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c4107120 0x7f34c41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c4111430 con 0x7f34c4101a30 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c41112b0 con 0x7f34c4107120 2024-03-26T08:34:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34caaf6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c41115b0 con 0x7f34c4101070 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101a30 0x7f34c419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101a30 0x7f34c419ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53898/0 (socket says 172.21.15.7:53898) 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34c3fff700 1 -- 172.21.15.7:0/2938316788 learned_addr learned my addr 172.21.15.7:0/2938316788 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.002+0000 7f34c9093700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c4107120 0x7f34c41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c8892700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c4101070 0x7f34c419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c3fff700 1 -- 172.21.15.7:0/2938316788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c4101070 msgr2=0x7f34c419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c3fff700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c4101070 0x7f34c419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c3fff700 1 -- 172.21.15.7:0/2938316788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c4107120 msgr2=0x7f34c41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c3fff700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c4107120 0x7f34c41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c3fff700 1 -- 172.21.15.7:0/2938316788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c4199080 con 0x7f34c4101a30 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c3fff700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101a30 0x7f34c419ec10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f34b0000c00 tx=0x7f34b0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c1ffb700 1 -- 172.21.15.7:0/2938316788 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0027070 con 0x7f34c4101a30 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c1ffb700 1 -- 172.21.15.7:0/2938316788 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f34b0004110 con 0x7f34c4101a30 2024-03-26T08:34:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c4199310 con 0x7f34c4101a30 2024-03-26T08:34:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34c1ffb700 1 -- 172.21.15.7:0/2938316788 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b000f460 con 0x7f34c4101a30 2024-03-26T08:34:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.003+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c41a5530 con 0x7f34c4101a30 2024-03-26T08:34:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.004+0000 7f34c1ffb700 1 -- 172.21.15.7:0/2938316788 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7f34b002a070 con 0x7f34c4101a30 2024-03-26T08:34:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.004+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c4102760 con 0x7f34c4101a30 2024-03-26T08:34:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.004+0000 7f34c1ffb700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f34b4039200 0x7f34b403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.004+0000 7f34c1ffb700 1 -- 172.21.15.7:0/2938316788 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f34b004d1b0 con 0x7f34c4101a30 2024-03-26T08:34:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.004+0000 7f34c8892700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f34b4039200 0x7f34b403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.005+0000 7f34c8892700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f34b4039200 0x7f34b403b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006fd0 tx=0x7f34b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.009+0000 7f34c1ffb700 1 -- 172.21.15.7:0/2938316788 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f34b0013070 con 0x7f34c4101a30 2024-03-26T08:34:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.215+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f34c4199e20 con 0x7f34c4101a30 2024-03-26T08:34:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.216+0000 7f34c1ffb700 1 -- 172.21.15.7:0/2938316788 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+269 (secure 0 0 0) 0x7f34b0031120 con 0x7f34c4101a30 2024-03-26T08:34:03.208 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:34:03.209 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-03-26T08:34:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:34:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] 2024-03-26T08:34:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f34b4039200 msgr2=0x7f34b403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f34b4039200 0x7f34b403b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006fd0 tx=0x7f34b8008040 comp rx=0 tx=0).stop 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101a30 msgr2=0x7f34c419ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101a30 0x7f34c419ec10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f34b0000c00 tx=0x7f34b0003680 comp rx=0 tx=0).stop 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 shutdown_connections 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f34b4039200 0x7f34b403b6c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f34c4101070 0x7f34c419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34c4107120 0x7f34c41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 --2- 172.21.15.7:0/2938316788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34c4101a30 0x7f34c419ec10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 >> 172.21.15.7:0/2938316788 conn(0x7f34c40780a0 msgr2=0x7f34c406a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 shutdown_connections 2024-03-26T08:34:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:03.218+0000 7f34caaf6700 1 -- 172.21.15.7:0/2938316788 wait complete. 2024-03-26T08:34:03.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:03.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:03.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:03 smithi007 ceph-mon[32462]: Reconfiguring mon.b (monmap changed)... 2024-03-26T08:34:03.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:34:03.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:34:03.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:03.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:03 smithi007 ceph-mon[32462]: Reconfiguring daemon mon.b on smithi016 2024-03-26T08:34:03.326 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:03 smithi007 ceph-mon[32462]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:03.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:03.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:03.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:03 smithi016 ceph-mon[35851]: Reconfiguring mon.b (monmap changed)... 2024-03-26T08:34:03.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:34:03.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:34:03.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:03.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:03 smithi016 ceph-mon[35851]: Reconfiguring daemon mon.b on smithi016 2024-03-26T08:34:03.482 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:03 smithi016 ceph-mon[35851]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:03.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:03.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:03.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:03 smithi044 ceph-mon[36461]: Reconfiguring mon.b (monmap changed)... 2024-03-26T08:34:03.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:34:03.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:34:03.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:03.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:03 smithi044 ceph-mon[36461]: Reconfiguring daemon mon.b on smithi016 2024-03-26T08:34:03.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:03 smithi044 ceph-mon[36461]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:03.887 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-26T08:34:03.887 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:34:03.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T08:34:03.916 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:34:03.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:34:03.983 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:34:03.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T08:34:04.021 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:34:04.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:34:04.096 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:34:04.096 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T08:34:04.129 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:34:04.129 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T08:34:04.200 INFO:tasks.cephadm:Adding mgr.x on smithi007 2024-03-26T08:34:04.200 INFO:tasks.cephadm:Adding mgr.y on smithi016 2024-03-26T08:34:04.200 INFO:tasks.cephadm:Adding mgr.z on smithi044 2024-03-26T08:34:04.200 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi007=x;smithi016=y;smithi044=z' 2024-03-26T08:34:04.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:04 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2938316788' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:04.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:04.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:04.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:34:04.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:34:04.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:04.410 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:04 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2938316788' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:04.411 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:04.411 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:04.411 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:34:04.411 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:34:04.411 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:04.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:04 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2938316788' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T08:34:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T08:34:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:05.051 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:34:05.301 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:05 smithi044 ceph-mon[36461]: Reconfiguring mon.c (monmap changed)... 2024-03-26T08:34:05.301 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:05 smithi044 ceph-mon[36461]: Reconfiguring daemon mon.c on smithi044 2024-03-26T08:34:05.301 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:05 smithi044 ceph-mon[36461]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:05.302 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:05 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:05.302 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:05 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:05.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:05 smithi016 ceph-mon[35851]: Reconfiguring mon.c (monmap changed)... 2024-03-26T08:34:05.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:05 smithi016 ceph-mon[35851]: Reconfiguring daemon mon.c on smithi044 2024-03-26T08:34:05.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:05 smithi016 ceph-mon[35851]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:05.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:05 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:05.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:05 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:05.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:05 smithi007 ceph-mon[32462]: Reconfiguring mon.c (monmap changed)... 2024-03-26T08:34:05.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:05 smithi007 ceph-mon[32462]: Reconfiguring daemon mon.c on smithi044 2024-03-26T08:34:05.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:05 smithi007 ceph-mon[32462]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:05.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:05 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:05.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:05 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:06.367 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:06 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:06.367 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:06 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:06.368 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:06 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:06.368 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:06 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:06.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:06 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:06.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:06 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:06.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:06 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:06.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:06 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:06.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:06 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:06.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:06 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:06.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:06 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:06.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:06 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:06.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.570+0000 7fc36d29a700 1 -- 172.21.15.44:0/2784938236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3681022c0 msgr2=0x7fc368102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.570+0000 7fc36d29a700 1 --2- 172.21.15.44:0/2784938236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3681022c0 0x7fc368102720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc350009990 tx=0x7fc350009ca0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.570+0000 7fc36d29a700 1 -- 172.21.15.44:0/2784938236 shutdown_connections 2024-03-26T08:34:06.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.570+0000 7fc36d29a700 1 --2- 172.21.15.44:0/2784938236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc368102c60 0x7fc368109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.570+0000 7fc36d29a700 1 --2- 172.21.15.44:0/2784938236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3681010c0 0x7fc3681014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.570+0000 7fc36d29a700 1 --2- 172.21.15.44:0/2784938236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3681022c0 0x7fc368102720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.570+0000 7fc36d29a700 1 -- 172.21.15.44:0/2784938236 >> 172.21.15.44:0/2784938236 conn(0x7fc3680fc660 msgr2=0x7fc3680feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:06.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.570+0000 7fc36d29a700 1 -- 172.21.15.44:0/2784938236 shutdown_connections 2024-03-26T08:34:06.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.570+0000 7fc36d29a700 1 -- 172.21.15.44:0/2784938236 wait complete. 2024-03-26T08:34:06.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.571+0000 7fc36d29a700 1 Processor -- start 2024-03-26T08:34:06.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.571+0000 7fc36d29a700 1 -- start start 2024-03-26T08:34:06.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.571+0000 7fc36d29a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3681010c0 0x7fc36806aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.571+0000 7fc36d29a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3681022c0 0x7fc36819d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.571+0000 7fc36d29a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc368102c60 0x7fc36819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.571+0000 7fc36d29a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36810ccf0 con 0x7fc3681022c0 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.571+0000 7fc36d29a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36810cb70 con 0x7fc3681010c0 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.571+0000 7fc36d29a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36810ce70 con 0x7fc368102c60 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.571+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc368102c60 0x7fc36819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc368102c60 0x7fc36819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:57256/0 (socket says 172.21.15.44:57256) 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3677fe700 1 -- 172.21.15.44:0/1754086611 learned_addr learned my addr 172.21.15.44:0/1754086611 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3667fc700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3681022c0 0x7fc36819d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc366ffd700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3681010c0 0x7fc36806aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3677fe700 1 -- 172.21.15.44:0/1754086611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3681010c0 msgr2=0x7fc36806aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3677fe700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3681010c0 0x7fc36806aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3677fe700 1 -- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3681022c0 msgr2=0x7fc36819d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3677fe700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3681022c0 0x7fc36819d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3677fe700 1 -- 172.21.15.44:0/1754086611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3681a00c0 con 0x7fc368102c60 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3667fc700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3681022c0 0x7fc36819d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:34:06.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc3677fe700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc368102c60 0x7fc36819f7f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc36000e940 tx=0x7fc36000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:06.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc35ffff700 1 -- 172.21.15.44:0/1754086611 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000cbd0 con 0x7fc368102c60 2024-03-26T08:34:06.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3681a03b0 con 0x7fc368102c60 2024-03-26T08:34:06.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc35ffff700 1 -- 172.21.15.44:0/1754086611 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc36000cd70 con 0x7fc368102c60 2024-03-26T08:34:06.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc35ffff700 1 -- 172.21.15.44:0/1754086611 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360010430 con 0x7fc368102c60 2024-03-26T08:34:06.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.572+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681a08f0 con 0x7fc368102c60 2024-03-26T08:34:06.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.573+0000 7fc35ffff700 1 -- 172.21.15.44:0/1754086611 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7fc3600105d0 con 0x7fc368102c60 2024-03-26T08:34:06.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.573+0000 7fc35ffff700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc354039200 0x7fc35403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:06.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.573+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc368068e50 con 0x7fc368102c60 2024-03-26T08:34:06.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.573+0000 7fc35ffff700 1 -- 172.21.15.44:0/1754086611 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc36004c490 con 0x7fc368102c60 2024-03-26T08:34:06.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.574+0000 7fc366ffd700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc354039200 0x7fc35403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:06.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.574+0000 7fc366ffd700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc354039200 0x7fc35403b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc368102120 tx=0x7fc35800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:06.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.579+0000 7fc35ffff700 1 -- 172.21.15.44:0/1754086611 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc360016bd0 con 0x7fc368102c60 2024-03-26T08:34:06.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.799+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi044=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fc368062660 con 0x7fc354039200 2024-03-26T08:34:06.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.805+0000 7fc35ffff700 1 -- 172.21.15.44:0/1754086611 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc368062660 con 0x7fc354039200 2024-03-26T08:34:06.798 INFO:teuthology.orchestra.run.smithi044.stdout:Scheduled mgr update... 2024-03-26T08:34:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.807+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc354039200 msgr2=0x7fc35403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.807+0000 7fc36d29a700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc354039200 0x7fc35403b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc368102120 tx=0x7fc35800b040 comp rx=0 tx=0).stop 2024-03-26T08:34:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.807+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc368102c60 msgr2=0x7fc36819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.807+0000 7fc36d29a700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc368102c60 0x7fc36819f7f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc36000e940 tx=0x7fc36000ed00 comp rx=0 tx=0).stop 2024-03-26T08:34:06.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.808+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 shutdown_connections 2024-03-26T08:34:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.808+0000 7fc36d29a700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc354039200 0x7fc35403b6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.808+0000 7fc36d29a700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc368102c60 0x7fc36819f7f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.808+0000 7fc36d29a700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3681010c0 0x7fc36806aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.808+0000 7fc36d29a700 1 --2- 172.21.15.44:0/1754086611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3681022c0 0x7fc36819d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.808+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 >> 172.21.15.44:0/1754086611 conn(0x7fc3680fc660 msgr2=0x7fc368105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.808+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 shutdown_connections 2024-03-26T08:34:06.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:34:06.808+0000 7fc36d29a700 1 -- 172.21.15.44:0/1754086611 wait complete. 2024-03-26T08:34:07.402 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:07.402 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:07.402 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:07.402 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:07.402 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:07.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:07.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:34:07.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:34:07.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:34:07.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:07.428 DEBUG:teuthology.orchestra.run.smithi016:mgr.y> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.y.service 2024-03-26T08:34:07.431 DEBUG:teuthology.orchestra.run.smithi044:mgr.z> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.z.service 2024-03-26T08:34:07.433 INFO:tasks.cephadm:Deploying OSDs... 2024-03-26T08:34:07.434 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:34:07.434 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T08:34:07.452 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T08:34:07.453 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:34:07.461 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:07.509 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T08:34:07.509 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:07.509 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-26T08:34:07.509 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:07.509 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:07.509 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-26 08:33:08.567502375 +0000 2024-03-26T08:34:07.509 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-26 08:28:09.115647352 +0000 2024-03-26T08:34:07.510 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-26 08:28:09.115647352 +0000 2024-03-26T08:34:07.510 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-26 08:28:09.115647352 +0000 2024-03-26T08:34:07.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T08:34:07.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:07.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:07.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:07.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:07.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:07.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:07.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:34:07.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:34:07.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:34:07.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:07.572 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-26T08:34:07.572 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-26T08:34:07.572 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000118358 s, 4.3 MB/s 2024-03-26T08:34:07.574 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T08:34:07.633 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-03-26T08:34:07.690 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T08:34:07.690 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:07.690 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-03-26T08:34:07.690 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:07.690 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:07.690 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-26 08:33:08.595501939 +0000 2024-03-26T08:34:07.690 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-26 08:28:09.458641860 +0000 2024-03-26T08:34:07.690 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-26 08:28:09.458641860 +0000 2024-03-26T08:34:07.691 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-26 08:28:09.458641860 +0000 2024-03-26T08:34:07.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T08:34:07.758 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-26T08:34:07.759 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-26T08:34:07.759 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000118201 s, 4.3 MB/s 2024-03-26T08:34:07.760 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T08:34:07.817 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-03-26T08:34:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T08:34:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:07.873 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-26T08:34:07.873 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:07.873 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:07.873 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-26 08:33:08.620501550 +0000 2024-03-26T08:34:07.873 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-26 08:28:09.790636543 +0000 2024-03-26T08:34:07.873 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-26 08:28:09.790636543 +0000 2024-03-26T08:34:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-26 08:28:09.790636543 +0000 2024-03-26T08:34:07.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T08:34:07.940 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-26T08:34:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-26T08:34:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000110142 s, 4.6 MB/s 2024-03-26T08:34:07.942 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T08:34:07.998 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-03-26T08:34:08.053 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T08:34:08.053 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:08.053 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-26T08:34:08.053 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:08.054 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:08.054 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-03-26 08:33:08.650501083 +0000 2024-03-26T08:34:08.054 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-03-26 08:28:10.135631019 +0000 2024-03-26T08:34:08.054 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-03-26 08:28:10.135631019 +0000 2024-03-26T08:34:08.054 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-03-26 08:28:10.135631019 +0000 2024-03-26T08:34:08.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T08:34:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-03-26T08:34:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-03-26T08:34:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00010122 s, 5.1 MB/s 2024-03-26T08:34:08.121 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T08:34:08.177 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-03-26T08:34:08.177 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T08:34:08.196 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T08:34:08.196 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-03-26T08:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T08:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-26T08:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-03-26 08:33:52.102531688 +0000 2024-03-26T08:34:08.255 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-03-26 08:28:30.205707416 +0000 2024-03-26T08:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-03-26 08:28:30.205707416 +0000 2024-03-26T08:34:08.256 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-03-26 08:28:30.205707416 +0000 2024-03-26T08:34:08.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T08:34:08.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:08 smithi007 ceph-mon[32462]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi044=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:34:08.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:08 smithi007 ceph-mon[32462]: Saving service mgr spec with placement smithi007=x;smithi016=y;smithi044=z;count:3 2024-03-26T08:34:08.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:08 smithi007 ceph-mon[32462]: Deploying daemon mgr.z on smithi044 2024-03-26T08:34:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-03-26T08:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-03-26T08:34:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000129601 s, 4.0 MB/s 2024-03-26T08:34:08.330 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T08:34:08.389 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-03-26T08:34:08.433 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:08 smithi044 ceph-mon[36461]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi044=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:34:08.433 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:08 smithi044 ceph-mon[36461]: Saving service mgr spec with placement smithi007=x;smithi016=y;smithi044=z;count:3 2024-03-26T08:34:08.434 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:08 smithi044 ceph-mon[36461]: Deploying daemon mgr.z on smithi044 2024-03-26T08:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T08:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-03-26T08:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-03-26 08:33:52.102531688 +0000 2024-03-26T08:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-03-26 08:28:30.549701851 +0000 2024-03-26T08:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-03-26 08:28:30.549701851 +0000 2024-03-26T08:34:08.447 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-03-26 08:28:30.549701851 +0000 2024-03-26T08:34:08.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T08:34:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:08 smithi016 ceph-mon[35851]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi044=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:34:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:08 smithi016 ceph-mon[35851]: Saving service mgr spec with placement smithi007=x;smithi016=y;smithi044=z;count:3 2024-03-26T08:34:08.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:08 smithi016 ceph-mon[35851]: Deploying daemon mgr.z on smithi044 2024-03-26T08:34:08.520 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-03-26T08:34:08.520 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-03-26T08:34:08.520 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000106843 s, 4.8 MB/s 2024-03-26T08:34:08.521 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T08:34:08.580 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-03-26T08:34:08.636 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T08:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-26T08:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-03-26 08:33:52.102531688 +0000 2024-03-26T08:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-03-26 08:28:30.881696479 +0000 2024-03-26T08:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-03-26 08:28:30.881696479 +0000 2024-03-26T08:34:08.637 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-03-26 08:28:30.881696479 +0000 2024-03-26T08:34:08.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T08:34:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-03-26T08:34:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-03-26T08:34:08.708 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000113164 s, 4.5 MB/s 2024-03-26T08:34:08.709 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T08:34:08.767 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-03-26T08:34:08.823 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T08:34:08.823 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:08.823 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-03-26T08:34:08.823 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:08.823 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:08.823 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-03-26 08:33:52.103531673 +0000 2024-03-26T08:34:08.824 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-03-26 08:28:31.221690979 +0000 2024-03-26T08:34:08.824 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-03-26 08:28:31.221690979 +0000 2024-03-26T08:34:08.824 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-03-26 08:28:31.221690979 +0000 2024-03-26T08:34:08.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T08:34:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-03-26T08:34:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-03-26T08:34:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.8883e-05 s, 5.2 MB/s 2024-03-26T08:34:08.895 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T08:34:08.952 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-03-26T08:34:08.952 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T08:34:08.969 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T08:34:08.969 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-03-26T08:34:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T08:34:09.027 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:09.027 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-26T08:34:09.027 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:09.027 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:09.028 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-03-26 08:34:08.844545313 +0000 2024-03-26T08:34:09.028 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-03-26 08:28:41.788516153 +0000 2024-03-26T08:34:09.028 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-03-26 08:28:41.788516153 +0000 2024-03-26T08:34:09.028 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-03-26 08:28:41.788516153 +0000 2024-03-26T08:34:09.028 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T08:34:09.099 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-03-26T08:34:09.099 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-03-26T08:34:09.099 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000269277 s, 1.9 MB/s 2024-03-26T08:34:09.101 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T08:34:09.157 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-03-26T08:34:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T08:34:09.213 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:09.213 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-03-26T08:34:09.213 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:09.213 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:09.214 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-03-26 08:34:08.844545313 +0000 2024-03-26T08:34:09.214 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-03-26 08:28:42.128510208 +0000 2024-03-26T08:34:09.214 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-03-26 08:28:42.128510208 +0000 2024-03-26T08:34:09.214 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-03-26 08:28:42.128510208 +0000 2024-03-26T08:34:09.214 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T08:34:09.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:09 smithi016 ceph-mon[35851]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:09.281 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:09 smithi044 ceph-mon[36461]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:09.303 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-03-26T08:34:09.303 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-03-26T08:34:09.303 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.0192116 s, 26.7 kB/s 2024-03-26T08:34:09.304 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T08:34:09.361 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-03-26T08:34:09.417 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T08:34:09.418 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:09.418 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-26T08:34:09.418 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:09.418 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:09.418 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-03-26 08:34:08.845545296 +0000 2024-03-26T08:34:09.418 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-03-26 08:28:42.503503652 +0000 2024-03-26T08:34:09.418 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-03-26 08:28:42.503503652 +0000 2024-03-26T08:34:09.418 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-03-26 08:28:42.503503652 +0000 2024-03-26T08:34:09.418 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T08:34:09.489 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-03-26T08:34:09.490 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-03-26T08:34:09.490 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.00020793 s, 2.5 MB/s 2024-03-26T08:34:09.491 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T08:34:09.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:09 smithi007 ceph-mon[32462]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:09.550 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-03-26T08:34:09.566 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:09 smithi044 systemd[1]: Started Ceph mgr.z for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:34:09.581 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T08:34:09.581 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T08:34:09.581 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-03-26T08:34:09.581 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T08:34:09.582 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T08:34:09.582 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-03-26 08:34:08.845545296 +0000 2024-03-26T08:34:09.582 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-03-26 08:28:42.859497428 +0000 2024-03-26T08:34:09.582 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-03-26 08:28:42.859497428 +0000 2024-03-26T08:34:09.582 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-03-26 08:28:42.859497428 +0000 2024-03-26T08:34:09.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T08:34:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-03-26T08:34:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-03-26T08:34:09.652 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000205688 s, 2.5 MB/s 2024-03-26T08:34:09.653 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T08:34:09.711 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-03-26T08:34:09.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T08:34:09.958 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:34:10.285 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:09.990+0000 7f6f7f3ee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T08:34:10.286 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:10.087+0000 7f6f7f3ee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:34:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:10.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:34:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:34:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:34:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:10 smithi007 ceph-mon[32462]: Deploying daemon mgr.y on smithi016 2024-03-26T08:34:10.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:10 smithi007 ceph-mon[32462]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:10.848 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:10.760+0000 7f6f7f3ee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:34:10.848 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:10.855+0000 7f6f7f3ee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:34:10.848 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:10.848 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:10.849 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:10.849 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:34:10.849 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:34:10.849 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:34:10.849 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:10.849 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-mon[36461]: Deploying daemon mgr.y on smithi016 2024-03-26T08:34:10.849 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:10 smithi044 ceph-mon[36461]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:10.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:10.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:10.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:10.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:34:10.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T08:34:10.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:34:10.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:10.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:10 smithi016 ceph-mon[35851]: Deploying daemon mgr.y on smithi016 2024-03-26T08:34:10.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:10 smithi016 ceph-mon[35851]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:11.176 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:11.000+0000 7f6f7f3ee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:34:11.176 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:11.099+0000 7f6f7f3ee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:34:11.176 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:11.184+0000 7f6f7f3ee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:34:11.689 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:11 smithi016 systemd[1]: Starting Ceph mgr.y for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:34:11.746 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:11.455+0000 7f6f7f3ee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:34:11.746 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:11.567+0000 7f6f7f3ee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:34:11.746 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:11.656+0000 7f6f7f3ee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:34:11.746 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:11.754+0000 7f6f7f3ee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:34:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T08:34:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T08:34:11.810 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-26T08:34:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-26T08:34:11.810 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00822017 s, 1.3 GB/s 2024-03-26T08:34:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-26T08:34:11.994 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:11 smithi016 podman[37253]: 2024-03-26 08:34:11.697514553 +0000 UTC m=+0.105569722 container create f11437db8ff5066d2885e21201f6a36105250885625b01b918fefbe652116d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:34:11.994 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:11 smithi016 podman[37253]: 2024-03-26 08:34:11.609714057 +0000 UTC m=+0.017769205 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:34:11.994 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:11 smithi016 podman[37253]: 2024-03-26 08:34:11.85794604 +0000 UTC m=+0.266001166 container init f11437db8ff5066d2885e21201f6a36105250885625b01b918fefbe652116d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-26T08:34:11.994 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:11 smithi016 podman[37253]: 2024-03-26 08:34:11.860712719 +0000 UTC m=+0.268767846 container start f11437db8ff5066d2885e21201f6a36105250885625b01b918fefbe652116d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-26T08:34:11.994 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:11 smithi016 ceph-mgr[37272]: -- 172.21.15.16:0/4020625456 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5633b0e9c5a0 con 0x5633b0e71000 2024-03-26T08:34:11.994 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:11 smithi016 bash[37253]: f11437db8ff5066d2885e21201f6a36105250885625b01b918fefbe652116d06 2024-03-26T08:34:11.994 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:11 smithi016 systemd[1]: Started Ceph mgr.y for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:34:12.066 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:11.935+0000 7f6f7f3ee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:34:12.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-03-26T08:34:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:34:12.744 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:12 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:12.438+0000 7f0f6d52b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T08:34:12.744 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:12 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:12.536+0000 7f0f6d52b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T08:34:13.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:12 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:12 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:12 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:12 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:12 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:13.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:12 smithi007 ceph-mon[32462]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:13.224 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:12 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.225 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:12 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.225 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:12 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.225 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:12 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.225 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:12 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:13.225 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:12 smithi016 ceph-mon[35851]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:13.280 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:13.002+0000 7f6f7f3ee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:34:13.280 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:13.141+0000 7f6f7f3ee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:34:13.280 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:12 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.280 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:12 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.280 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:12 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.280 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:12 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:13.280 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:12 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:13.280 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:12 smithi044 ceph-mon[36461]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:13.488 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:13.231+0000 7f0f6d52b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T08:34:13.488 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:13.327+0000 7f0f6d52b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T08:34:13.566 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:13.492+0000 7f6f7f3ee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:34:13.743 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:13.495+0000 7f0f6d52b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T08:34:13.744 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:13.594+0000 7f0f6d52b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T08:34:13.744 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:13.679+0000 7f0f6d52b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T08:34:13.978 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:13.583+0000 7f6f7f3ee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:34:13.978 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:13.795+0000 7f6f7f3ee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:34:13.978 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:13.986+0000 7f6f7f3ee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:34:14.243 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:13.952+0000 7f0f6d52b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T08:34:14.244 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:14.065+0000 7f0f6d52b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T08:34:14.244 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:14.155+0000 7f0f6d52b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T08:34:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.272+0000 7fbdcd754700 1 -- 172.21.15.7:0/2554696445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81022b0 msgr2=0x7fbdc8102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.272+0000 7fbdcd754700 1 --2- 172.21.15.7:0/2554696445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81022b0 0x7fbdc8102710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00099c0 tx=0x7fbdb0009cd0 comp rx=0 tx=0).stop 2024-03-26T08:34:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.272+0000 7fbdcd754700 1 -- 172.21.15.7:0/2554696445 shutdown_connections 2024-03-26T08:34:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.272+0000 7fbdcd754700 1 --2- 172.21.15.7:0/2554696445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc81010b0 0x7fbdc81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.272+0000 7fbdcd754700 1 --2- 172.21.15.7:0/2554696445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc8102c50 0x7fbdc8109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.272+0000 7fbdcd754700 1 --2- 172.21.15.7:0/2554696445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81022b0 0x7fbdc8102710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.272+0000 7fbdcd754700 1 -- 172.21.15.7:0/2554696445 >> 172.21.15.7:0/2554696445 conn(0x7fbdc80fc650 msgr2=0x7fbdc80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.272+0000 7fbdcd754700 1 -- 172.21.15.7:0/2554696445 shutdown_connections 2024-03-26T08:34:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.272+0000 7fbdcd754700 1 -- 172.21.15.7:0/2554696445 wait complete. 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdcd754700 1 Processor -- start 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdcd754700 1 -- start start 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdcd754700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81010b0 0x7fbdc8105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdcd754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81022b0 0x7fbdc8103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdcd754700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc8102c50 0x7fbdc8103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdcd754700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc810ccc0 con 0x7fbdc81022b0 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdcd754700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc810cb40 con 0x7fbdc81010b0 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdcd754700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc810ce40 con 0x7fbdc8102c50 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdc6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81010b0 0x7fbdc8105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdc6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81010b0 0x7fbdc8105040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48896/0 (socket says 172.21.15.7:48896) 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdc6ffd700 1 -- 172.21.15.7:0/1503172081 learned_addr learned my addr 172.21.15.7:0/1503172081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdc77fe700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc8102c50 0x7fbdc8103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdc67fc700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81022b0 0x7fbdc8103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdc6ffd700 1 -- 172.21.15.7:0/1503172081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc8102c50 msgr2=0x7fbdc8103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.273+0000 7fbdc6ffd700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc8102c50 0x7fbdc8103bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdc6ffd700 1 -- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81022b0 msgr2=0x7fbdc8103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdc6ffd700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81022b0 0x7fbdc8103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdc6ffd700 1 -- 172.21.15.7:0/1503172081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8104480 con 0x7fbdc81010b0 2024-03-26T08:34:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdc77fe700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc8102c50 0x7fbdc8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:34:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdc67fc700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81022b0 0x7fbdc8103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:34:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdc6ffd700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81010b0 0x7fbdc8105040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800e970 tx=0x7fbdb800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdbffff700 1 -- 172.21.15.7:0/1503172081 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8004220 con 0x7fbdc81010b0 2024-03-26T08:34:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdbffff700 1 -- 172.21.15.7:0/1503172081 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdb80043c0 con 0x7fbdc81010b0 2024-03-26T08:34:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdbffff700 1 -- 172.21.15.7:0/1503172081 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8003dc0 con 0x7fbdc81010b0 2024-03-26T08:34:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc81a2620 con 0x7fbdc81010b0 2024-03-26T08:34:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.274+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc81a2ae0 con 0x7fbdc81010b0 2024-03-26T08:34:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.275+0000 7fbdbffff700 1 -- 172.21.15.7:0/1503172081 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) v1 ==== 46161+0+0 (secure 0 0 0) 0x7fbdb8018430 con 0x7fbdc81010b0 2024-03-26T08:34:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.275+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc8068e50 con 0x7fbdc81010b0 2024-03-26T08:34:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.275+0000 7fbdbffff700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb4039200 0x7fbdb403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.275+0000 7fbdbffff700 1 -- 172.21.15.7:0/1503172081 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbdb804c410 con 0x7fbdc81010b0 2024-03-26T08:34:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.276+0000 7fbdc67fc700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb4039200 0x7fbdb403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.276+0000 7fbdc67fc700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb4039200 0x7fbdb403b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00038c0 tx=0x7fbdb0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.281+0000 7fbdbffff700 1 -- 172.21.15.7:0/1503172081 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbdb801fbd0 con 0x7fbdc81010b0 2024-03-26T08:34:14.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:14.505+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdc80fe7e0 con 0x7fbdb4039200 2024-03-26T08:34:14.536 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:14.276+0000 7f6f7f3ee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:34:14.536 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.536 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.536 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: Reconfiguring daemon mgr.x on smithi007 2024-03-26T08:34:14.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-mon[36461]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:34:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: Reconfiguring daemon mgr.x on smithi007 2024-03-26T08:34:14.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:14 smithi007 ceph-mon[32462]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:14.743 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:14.253+0000 7f0f6d52b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T08:34:14.743 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:14.434+0000 7f0f6d52b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: Reconfiguring daemon mgr.x on smithi007 2024-03-26T08:34:14.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:14 smithi016 ceph-mon[35851]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:14.816 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:14.544+0000 7f6f7f3ee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:34:14.816 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:14 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:14.787+0000 7f6f7f3ee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:34:15.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:15 smithi007 ceph-mon[32462]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:34:15.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:15 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:34:15.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:15 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:34:15.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:15 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:15.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:15 smithi044 ceph-mon[36461]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:34:15.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:15 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:34:15.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:15 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:34:15.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:15 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:15.584 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:15 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:15.456+0000 7f0f6d52b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T08:34:15.585 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:15 smithi016 ceph-mon[35851]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:34:15.585 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:15 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:34:15.585 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:15 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:34:15.585 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:15 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:15.927 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:15 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:15.592+0000 7f0f6d52b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T08:34:15.927 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:15 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:15.935+0000 7f0f6d52b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T08:34:16.066 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:15 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:15.617+0000 7f6f7f3ee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:34:16.219 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:16.026+0000 7f0f6d52b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T08:34:16.493 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:16.226+0000 7f0f6d52b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:34:16.494 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:16.417+0000 7f0f6d52b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T08:34:16.510 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:16.206+0000 7f6f7f3ee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:34:16.510 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:16.306+0000 7f6f7f3ee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:34:16.743 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:16.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:16.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:16.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:16.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:16.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:16.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:16 smithi007 ceph-mon[32462]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:16.816 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z[38223]: 2024-03-26T08:34:16.527+0000 7f6f7f3ee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:34:16.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:16.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:16.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:16.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:16.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:16.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:16.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:16 smithi044 ceph-mon[36461]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:16.967 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:16.708+0000 7f0f6d52b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T08:34:16.967 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:16.967 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:16.967 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:16.967 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:16.967 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:16.968 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:16.968 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-mon[35851]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:17.244 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:16.974+0000 7f0f6d52b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T08:34:17.244 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:17 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:17.217+0000 7f0f6d52b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T08:34:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:17.527+0000 7fbdbffff700 1 -- 172.21.15.7:0/1503172081 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 14) v1 ==== 91697+0+0 (secure 0 0 0) 0x7fbdb8017cd0 con 0x7fbdc81010b0 2024-03-26T08:34:17.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:17 smithi007 ceph-mon[32462]: Standby manager daemon z started 2024-03-26T08:34:17.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:17 smithi007 ceph-mon[32462]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T08:34:17.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:17 smithi007 ceph-mon[32462]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:34:17.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:17 smithi007 ceph-mon[32462]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T08:34:17.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:17 smithi007 ceph-mon[32462]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:34:17.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:17 smithi044 ceph-mon[36461]: Standby manager daemon z started 2024-03-26T08:34:17.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:17 smithi044 ceph-mon[36461]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T08:34:17.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:17 smithi044 ceph-mon[36461]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:34:17.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:17 smithi044 ceph-mon[36461]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T08:34:17.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:17 smithi044 ceph-mon[36461]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:34:17.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:17 smithi016 ceph-mon[35851]: Standby manager daemon z started 2024-03-26T08:34:17.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:17 smithi016 ceph-mon[35851]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T08:34:17.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:17 smithi016 ceph-mon[35851]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:34:17.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:17 smithi016 ceph-mon[35851]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T08:34:17.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:17 smithi016 ceph-mon[35851]: from='mgr.? 172.21.15.44:0/2368921896' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:34:18.493 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:18.052+0000 7f0f6d52b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T08:34:18.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:18 smithi007 ceph-mon[32462]: mgrmap e14: x(active, since 79s), standbys: z 2024-03-26T08:34:18.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T08:34:18.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:18 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1440312248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11d5305b-e22f-42b9-a7b6-acc96b92baab"}]: dispatch 2024-03-26T08:34:18.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:18 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1440312248' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11d5305b-e22f-42b9-a7b6-acc96b92baab"}]': finished 2024-03-26T08:34:18.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:18 smithi007 ceph-mon[32462]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T08:34:18.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:18.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:18 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1007714722' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:34:18.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:18 smithi007 ceph-mon[32462]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:18.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:18 smithi044 ceph-mon[36461]: mgrmap e14: x(active, since 79s), standbys: z 2024-03-26T08:34:18.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T08:34:18.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:18 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1440312248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11d5305b-e22f-42b9-a7b6-acc96b92baab"}]: dispatch 2024-03-26T08:34:18.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:18 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1440312248' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11d5305b-e22f-42b9-a7b6-acc96b92baab"}]': finished 2024-03-26T08:34:18.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:18 smithi044 ceph-mon[36461]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T08:34:18.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:18.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:18 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1007714722' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:34:18.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:18 smithi044 ceph-mon[36461]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:18.956 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-mon[35851]: mgrmap e14: x(active, since 79s), standbys: z 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1440312248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11d5305b-e22f-42b9-a7b6-acc96b92baab"}]: dispatch 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1440312248' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11d5305b-e22f-42b9-a7b6-acc96b92baab"}]': finished 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-mon[35851]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1007714722' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-mon[35851]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:18.641+0000 7f0f6d52b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:18.742+0000 7f0f6d52b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T08:34:18.957 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:34:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y[37268]: 2024-03-26T08:34:18.964+0000 7f0f6d52b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T08:34:19.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:19.601+0000 7fbdbffff700 1 -- 172.21.15.7:0/1503172081 <== mon.2 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 15) v1 ==== 137233+0+0 (secure 0 0 0) 0x7fbdb800f3c0 con 0x7fbdc81010b0 2024-03-26T08:34:19.980 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:19 smithi016 ceph-mon[35851]: Standby manager daemon y started 2024-03-26T08:34:19.980 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:19 smithi016 ceph-mon[35851]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T08:34:19.980 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:19 smithi016 ceph-mon[35851]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:34:19.980 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:19 smithi016 ceph-mon[35851]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T08:34:19.980 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:19 smithi016 ceph-mon[35851]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:34:20.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:19 smithi007 ceph-mon[32462]: Standby manager daemon y started 2024-03-26T08:34:20.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:19 smithi007 ceph-mon[32462]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T08:34:20.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:19 smithi007 ceph-mon[32462]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:34:20.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:19 smithi007 ceph-mon[32462]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T08:34:20.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:19 smithi007 ceph-mon[32462]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:34:20.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:19 smithi044 ceph-mon[36461]: Standby manager daemon y started 2024-03-26T08:34:20.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:19 smithi044 ceph-mon[36461]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T08:34:20.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:19 smithi044 ceph-mon[36461]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T08:34:20.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:19 smithi044 ceph-mon[36461]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T08:34:20.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:19 smithi044 ceph-mon[36461]: from='mgr.? 172.21.15.16:0/1658614488' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T08:34:20.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:20 smithi007 ceph-mon[32462]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-03-26T08:34:20.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T08:34:20.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:20 smithi007 ceph-mon[32462]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:20.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:20 smithi016 ceph-mon[35851]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-03-26T08:34:20.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T08:34:20.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:20 smithi016 ceph-mon[35851]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:21.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:20 smithi044 ceph-mon[36461]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-03-26T08:34:21.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T08:34:21.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:20 smithi044 ceph-mon[36461]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:22.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:22 smithi007 ceph-mon[32462]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:22.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:22 smithi044 ceph-mon[36461]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:22.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:22 smithi016 ceph-mon[35851]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:24.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:24 smithi007 ceph-mon[32462]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:24 smithi044 ceph-mon[36461]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:24.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:24 smithi016 ceph-mon[35851]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:25.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:25 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T08:34:25.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:25 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:25.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:25 smithi007 ceph-mon[32462]: Deploying daemon osd.0 on smithi007 2024-03-26T08:34:25.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:25 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T08:34:25.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:25 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:25.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:25 smithi044 ceph-mon[36461]: Deploying daemon osd.0 on smithi007 2024-03-26T08:34:25.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:25 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T08:34:25.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:25 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:25.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:25 smithi016 ceph-mon[35851]: Deploying daemon osd.0 on smithi007 2024-03-26T08:34:26.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:26 smithi007 ceph-mon[32462]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:26 smithi044 ceph-mon[36461]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:26.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:26 smithi016 ceph-mon[35851]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:28.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:28 smithi007 ceph-mon[32462]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:28 smithi044 ceph-mon[36461]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:28.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:28 smithi016 ceph-mon[35851]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:30.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:30 smithi007 ceph-mon[32462]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:30 smithi044 ceph-mon[36461]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:30.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:30 smithi016 ceph-mon[35851]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:31.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:31.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:31.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:31.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:31.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:31.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:31.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:31.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:31.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:32.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:32 smithi007 ceph-mon[32462]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:32.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:32 smithi016 ceph-mon[35851]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:32.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:32 smithi044 ceph-mon[36461]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:33.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.700+0000 7fbdbffff700 1 -- 172.21.15.7:0/1503172081 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbdc80fe7e0 con 0x7fbdb4039200 2024-03-26T08:34:33.693 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb4039200 msgr2=0x7fbdb403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb4039200 0x7fbdb403b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbdb00038c0 tx=0x7fbdb0005fb0 comp rx=0 tx=0).stop 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81010b0 msgr2=0x7fbdc8105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81010b0 0x7fbdc8105040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800e970 tx=0x7fbdb800ec80 comp rx=0 tx=0).stop 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 shutdown_connections 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb4039200 0x7fbdb403b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc8102c50 0x7fbdc8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81010b0 0x7fbdc8105040 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 --2- 172.21.15.7:0/1503172081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81022b0 0x7fbdc8103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.702+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 >> 172.21.15.7:0/1503172081 conn(0x7fbdc80fc650 msgr2=0x7fbdc80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:33.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.703+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 shutdown_connections 2024-03-26T08:34:33.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:34:33.703+0000 7fbdcd754700 1 -- 172.21.15.7:0/1503172081 wait complete. 2024-03-26T08:34:34.282 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.0.service 2024-03-26T08:34:34.284 INFO:tasks.cephadm:Deploying osd.1 on smithi016 with /dev/vg_nvme/lv_4... 2024-03-26T08:34:34.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T08:34:34.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:34.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:34.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:34.467 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-mon[32462]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:34.467 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:34:34 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0[44631]: 2024-03-26T08:34:34.341+0000 7f87b2cd77c0 -1 osd.0 0 log_to_monitors true 2024-03-26T08:34:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:34:34.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:34.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:34.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.653 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.654 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:34.654 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:34 smithi016 ceph-mon[35851]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:34.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:34.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:34.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:34.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:34.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:34 smithi044 ceph-mon[36461]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:35.472 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:35 smithi007 ceph-mon[32462]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T08:34:35.650 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:35 smithi016 ceph-mon[35851]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T08:34:35.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:35 smithi044 ceph-mon[36461]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T08:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T08:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T08:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-03-26T08:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-03-26T08:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00811154 s, 1.3 GB/s 2024-03-26T08:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 2024-03-26T08:34:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-03-26T08:34:36.618 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:36 smithi007 ceph-mon[32462]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T08:34:36.619 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:36 smithi007 ceph-mon[32462]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T08:34:36.619 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:36 smithi007 ceph-mon[32462]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:34:36.619 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:36 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:36.619 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:36 smithi007 ceph-mon[32462]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:36.619 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:34:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0[44631]: 2024-03-26T08:34:36.350+0000 7f87a833c700 -1 osd.0 0 waiting for initial osdmap 2024-03-26T08:34:36.619 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:34:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0[44631]: 2024-03-26T08:34:36.353+0000 7f87a412f700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:34:36.645 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:36 smithi016 ceph-mon[35851]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T08:34:36.646 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:36 smithi016 ceph-mon[35851]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T08:34:36.646 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:36 smithi016 ceph-mon[35851]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:34:36.646 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:36 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:36.646 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:36 smithi016 ceph-mon[35851]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:36.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:36 smithi044 ceph-mon[36461]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T08:34:36.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:36 smithi044 ceph-mon[36461]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T08:34:36.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:36 smithi044 ceph-mon[36461]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:34:36.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:36 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:36.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:36 smithi044 ceph-mon[36461]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:36.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-03-26T08:34:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:34:37.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:37 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:34:37.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:37 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:34:37.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:37 smithi007 ceph-mon[32462]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:34:37.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:37 smithi007 ceph-mon[32462]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T08:34:37.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:37.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:37.550 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:37 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:34:37.550 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:37 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:34:37.550 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:37 smithi016 ceph-mon[35851]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:34:37.550 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:37 smithi016 ceph-mon[35851]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T08:34:37.550 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:37 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:37.550 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:37 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:37.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:37 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:34:37.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:37 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:34:37.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:37 smithi044 ceph-mon[36461]: from='osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:34:37.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:37 smithi044 ceph-mon[36461]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T08:34:37.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:37 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:37.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:37 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:38.443 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi007 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] boot 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi007 to 17093M 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:38 smithi016 ceph-mon[35851]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.666+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2292230230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d20102cb0 msgr2=0x7f3d201050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.666+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2292230230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d20102cb0 0x7f3d201050a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800b0a0 tx=0x7f3d1800b3b0 comp rx=0 tx=0).stop 2024-03-26T08:34:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.666+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2292230230 shutdown_connections 2024-03-26T08:34:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.666+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2292230230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d20107f10 0x7f3d2010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.666+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2292230230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d20102cb0 0x7f3d201050a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.666+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2292230230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d201055e0 0x7f3d201079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.666+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2292230230 >> 172.21.15.16:0/2292230230 conn(0x7f3d200fc660 msgr2=0x7f3d200feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:38.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.666+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2292230230 shutdown_connections 2024-03-26T08:34:38.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.666+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2292230230 wait complete. 2024-03-26T08:34:38.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.667+0000 7f3d24c5b700 1 Processor -- start 2024-03-26T08:34:38.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.667+0000 7f3d24c5b700 1 -- start start 2024-03-26T08:34:38.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.667+0000 7f3d24c5b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d20102cb0 0x7f3d20198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.667+0000 7f3d24c5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d201055e0 0x7f3d201989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d24c5b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d20107f10 0x7f3d2019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d24c5b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2010ddf0 con 0x7f3d20107f10 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d24c5b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2010dc70 con 0x7f3d201055e0 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d24c5b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2010df70 con 0x7f3d20102cb0 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d201055e0 0x7f3d201989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d201055e0 0x7f3d201989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58262/0 (socket says 172.21.15.16:58262) 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1dd9b700 1 -- 172.21.15.16:0/2470175898 learned_addr learned my addr 172.21.15.16:0/2470175898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1ed9d700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d20107f10 0x7f3d2019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1e59c700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d20102cb0 0x7f3d20198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1dd9b700 1 -- 172.21.15.16:0/2470175898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d20102cb0 msgr2=0x7f3d20198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:38.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1dd9b700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d20102cb0 0x7f3d20198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1dd9b700 1 -- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d20107f10 msgr2=0x7f3d2019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1dd9b700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d20107f10 0x7f3d2019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1dd9b700 1 -- 172.21.15.16:0/2470175898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d20100b20 con 0x7f3d201055e0 2024-03-26T08:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d1dd9b700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d201055e0 0x7f3d201989d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800d980 tx=0x7f3d0800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d177fe700 1 -- 172.21.15.16:0/2470175898 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08016070 con 0x7f3d201055e0 2024-03-26T08:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d177fe700 1 -- 172.21.15.16:0/2470175898 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d08004d10 con 0x7f3d201055e0 2024-03-26T08:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d177fe700 1 -- 172.21.15.16:0/2470175898 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d080054e0 con 0x7f3d201055e0 2024-03-26T08:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d20100e10 con 0x7f3d201055e0 2024-03-26T08:34:38.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.668+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d201013a0 con 0x7f3d201055e0 2024-03-26T08:34:38.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.669+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d20068e50 con 0x7f3d201055e0 2024-03-26T08:34:38.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.670+0000 7f3d177fe700 1 -- 172.21.15.16:0/2470175898 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) v1 ==== 137233+0+0 (secure 0 0 0) 0x7f3d08004750 con 0x7f3d201055e0 2024-03-26T08:34:38.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.671+0000 7f3d177fe700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d0c0a2880 0x7f3d0c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:34:38.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.671+0000 7f3d177fe700 1 -- 172.21.15.16:0/2470175898 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f3d080ccd60 con 0x7f3d201055e0 2024-03-26T08:34:38.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.671+0000 7f3d1e59c700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d0c0a2880 0x7f3d0c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:34:38.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.672+0000 7f3d1e59c700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d0c0a2880 0x7f3d0c0a4d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009fd0 tx=0x7f3d1800b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:34:38.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.676+0000 7f3d177fe700 1 -- 172.21.15.16:0/2470175898 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3d0801f020 con 0x7f3d201055e0 2024-03-26T08:34:38.718 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi007 2024-03-26T08:34:38.718 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.718 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:38.718 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] boot 2024-03-26T08:34:38.718 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T08:34:38.719 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:38.719 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.719 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:34:38.719 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi007 to 17093M 2024-03-26T08:34:38.719 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.719 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:38.719 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:38.719 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.719 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:38 smithi044 ceph-mon[36461]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:38.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi007 2024-03-26T08:34:38.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:38.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: osd.0 [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] boot 2024-03-26T08:34:38.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T08:34:38.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T08:34:38.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:34:38.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi007 to 17093M 2024-03-26T08:34:38.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:38.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:38.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:38.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:38 smithi007 ceph-mon[32462]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T08:34:38.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:38.891+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d200fdcd0 con 0x7f3d0c0a2880 2024-03-26T08:34:39.507 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:39 smithi016 ceph-mon[35851]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T08:34:39.507 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:39 smithi016 ceph-mon[35851]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:34:39.507 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:34:39.507 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:34:39.507 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:39.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:39 smithi007 ceph-mon[32462]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T08:34:39.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:39 smithi007 ceph-mon[32462]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:34:39.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:34:39.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:34:39.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:39.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:39 smithi044 ceph-mon[36461]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T08:34:39.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:39 smithi044 ceph-mon[36461]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:34:39.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:34:39.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:34:39.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:40.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:40 smithi016 ceph-mon[35851]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:40.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:40 smithi007 ceph-mon[32462]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:40.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:40 smithi044 ceph-mon[36461]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:41.885 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:41 smithi016 ceph-mon[35851]: from='client.? 172.21.15.16:0/2703089438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b219c6-a22b-4b66-9fa8-967c59f003ab"}]: dispatch 2024-03-26T08:34:41.885 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:41 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b219c6-a22b-4b66-9fa8-967c59f003ab"}]: dispatch 2024-03-26T08:34:41.885 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:41 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95b219c6-a22b-4b66-9fa8-967c59f003ab"}]': finished 2024-03-26T08:34:41.885 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:41 smithi016 ceph-mon[35851]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T08:34:41.886 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:34:42.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:41 smithi007 ceph-mon[32462]: from='client.? 172.21.15.16:0/2703089438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b219c6-a22b-4b66-9fa8-967c59f003ab"}]: dispatch 2024-03-26T08:34:42.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:41 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b219c6-a22b-4b66-9fa8-967c59f003ab"}]: dispatch 2024-03-26T08:34:42.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:41 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95b219c6-a22b-4b66-9fa8-967c59f003ab"}]': finished 2024-03-26T08:34:42.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:41 smithi007 ceph-mon[32462]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T08:34:42.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:34:42.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:41 smithi044 ceph-mon[36461]: from='client.? 172.21.15.16:0/2703089438' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b219c6-a22b-4b66-9fa8-967c59f003ab"}]: dispatch 2024-03-26T08:34:42.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:41 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b219c6-a22b-4b66-9fa8-967c59f003ab"}]: dispatch 2024-03-26T08:34:42.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:41 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95b219c6-a22b-4b66-9fa8-967c59f003ab"}]': finished 2024-03-26T08:34:42.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:41 smithi044 ceph-mon[36461]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T08:34:42.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:34:42.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:42 smithi016 ceph-mon[35851]: from='client.? 172.21.15.16:0/339690477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:34:42.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:42 smithi016 ceph-mon[35851]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:43.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:42 smithi007 ceph-mon[32462]: from='client.? 172.21.15.16:0/339690477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:34:43.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:42 smithi007 ceph-mon[32462]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:43.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:42 smithi044 ceph-mon[36461]: from='client.? 172.21.15.16:0/339690477' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:34:43.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:42 smithi044 ceph-mon[36461]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:44.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:44 smithi007 ceph-mon[32462]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:44.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:44 smithi044 ceph-mon[36461]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:44.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:44 smithi016 ceph-mon[35851]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:46.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:46 smithi007 ceph-mon[32462]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:46.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:46 smithi044 ceph-mon[36461]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:46.577 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:46 smithi016 ceph-mon[35851]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:48.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:48 smithi007 ceph-mon[32462]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:48.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:48 smithi044 ceph-mon[36461]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:48.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:48 smithi016 ceph-mon[35851]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:49.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T08:34:49.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:49.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:49 smithi007 ceph-mon[32462]: Deploying daemon osd.1 on smithi016 2024-03-26T08:34:49.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T08:34:49.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:49.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:49 smithi044 ceph-mon[36461]: Deploying daemon osd.1 on smithi016 2024-03-26T08:34:49.615 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T08:34:49.615 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:49.615 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:49 smithi016 ceph-mon[35851]: Deploying daemon osd.1 on smithi016 2024-03-26T08:34:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:50 smithi044 ceph-mon[36461]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:50.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:50 smithi007 ceph-mon[32462]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:50.679 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:50 smithi016 ceph-mon[35851]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:52.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:52 smithi016 ceph-mon[35851]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:52.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:52 smithi007 ceph-mon[32462]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:52.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:52 smithi044 ceph-mon[36461]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:54.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:54 smithi016 ceph-mon[35851]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:54.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:54 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:54.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:54 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:54.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:54 smithi007 ceph-mon[32462]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:54.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:54 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:54.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:54 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:54.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:54 smithi044 ceph-mon[36461]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:54.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:54 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:54.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:54 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:55.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:55.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:55.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:56.454 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:56 smithi016 ceph-mon[35851]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:56.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:56 smithi007 ceph-mon[32462]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:56.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:56 smithi044 ceph-mon[36461]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.756+0000 7f3d177fe700 1 -- 172.21.15.16:0/2470175898 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3d200fdcd0 con 0x7f3d0c0a2880 2024-03-26T08:34:57.749 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 1 on host 'smithi016' 2024-03-26T08:34:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.758+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d0c0a2880 msgr2=0x7f3d0c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.758+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d0c0a2880 0x7f3d0c0a4d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009fd0 tx=0x7f3d1800b7f0 comp rx=0 tx=0).stop 2024-03-26T08:34:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.758+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d201055e0 msgr2=0x7f3d201989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:34:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.758+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d201055e0 0x7f3d201989d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800d980 tx=0x7f3d0800dc90 comp rx=0 tx=0).stop 2024-03-26T08:34:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.759+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 shutdown_connections 2024-03-26T08:34:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.759+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d0c0a2880 0x7f3d0c0a4d40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.759+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d20102cb0 0x7f3d20198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:57.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.759+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d20107f10 0x7f3d2019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:57.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.759+0000 7f3d24c5b700 1 --2- 172.21.15.16:0/2470175898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d201055e0 0x7f3d201989d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:34:57.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.759+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 >> 172.21.15.16:0/2470175898 conn(0x7f3d200fc660 msgr2=0x7f3d201035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:34:57.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.759+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 shutdown_connections 2024-03-26T08:34:57.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:34:57.759+0000 7f3d24c5b700 1 -- 172.21.15.16:0/2470175898 wait complete. 2024-03-26T08:34:58.399 DEBUG:teuthology.orchestra.run.smithi016:osd.1> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.1.service 2024-03-26T08:34:58.402 INFO:tasks.cephadm:Deploying osd.2 on smithi044 with /dev/vg_nvme/lv_4... 2024-03-26T08:34:58.402 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T08:34:58.663 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:34:58.715 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='osd.1 [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:34:58.716 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:58 smithi016 ceph-mon[35851]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:58.785 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.785 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.785 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:58.786 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:58.786 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.786 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.786 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.786 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:58.786 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='osd.1 [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:34:58.786 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:34:58.786 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:58 smithi044 ceph-mon[36461]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='osd.1 [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:34:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T08:34:58.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:58 smithi007 ceph-mon[32462]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:34:59.874 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:59 smithi016 ceph-mon[35851]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T08:34:59.874 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:59 smithi016 ceph-mon[35851]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T08:34:59.875 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:59 smithi016 ceph-mon[35851]: from='osd.1 [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:34:59.875 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:34:59.875 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:34:59 smithi016 ceph-mon[35851]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:34:59.875 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:34:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1[40475]: 2024-03-26T08:34:59.734+0000 7f2c89df2700 -1 osd.1 0 waiting for initial osdmap 2024-03-26T08:34:59.875 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:34:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1[40475]: 2024-03-26T08:34:59.738+0000 7f2c85be5700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:34:59.945 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:59 smithi044 ceph-mon[36461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T08:34:59.945 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:59 smithi044 ceph-mon[36461]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T08:34:59.945 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:59 smithi044 ceph-mon[36461]: from='osd.1 [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:34:59.945 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:34:59.945 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:34:59 smithi044 ceph-mon[36461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:35:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:59 smithi007 ceph-mon[32462]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T08:35:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:59 smithi007 ceph-mon[32462]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T08:35:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:59 smithi007 ceph-mon[32462]: from='osd.1 [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:35:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:34:59 smithi007 ceph-mon[32462]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:35:00.540 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T08:35:00.540 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T08:35:00.540 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-03-26T08:35:00.540 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-03-26T08:35:00.540 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125901 s, 833 MB/s 2024-03-26T08:35:00.540 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-03-26T08:35:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:00 smithi016 ceph-mon[35851]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:35:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:00 smithi016 ceph-mon[35851]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T08:35:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:00 smithi016 ceph-mon[35851]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:35:01.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:00 smithi007 ceph-mon[32462]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:35:01.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:00 smithi007 ceph-mon[32462]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T08:35:01.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:01.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:01.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:00 smithi007 ceph-mon[32462]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:35:01.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:00 smithi044 ceph-mon[36461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:35:01.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:00 smithi044 ceph-mon[36461]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T08:35:01.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:01.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:01.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:00 smithi044 ceph-mon[36461]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T08:35:01.166 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi044:vg_nvme/lv_4 2024-03-26T08:35:01.418 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: osd.1 [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] boot 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:01.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:01.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:01.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:01.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:02.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:35:02.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:35:02.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:02.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: osd.1 [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] boot 2024-03-26T08:35:02.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T08:35:02.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:02.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:02.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:02.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:02.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:02.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:02.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:02.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:02.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: osd.1 [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] boot 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:02.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:02.964 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:02 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi016 2024-03-26T08:35:02.964 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:02 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi016 to 17084M 2024-03-26T08:35:02.964 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:02 smithi044 ceph-mon[36461]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T08:35:02.964 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:02 smithi044 ceph-mon[36461]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 -- 172.21.15.44:0/1680093999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 msgr2=0x7f003c102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1680093999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 0x7f003c102600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0038009990 tx=0x7f0038009ca0 comp rx=0 tx=0).stop 2024-03-26T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 -- 172.21.15.44:0/1680093999 shutdown_connections 2024-03-26T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1680093999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 0x7f003c102600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1680093999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003c103d80 0x7f003c10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1680093999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c1033e0 0x7f003c103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 -- 172.21.15.44:0/1680093999 >> 172.21.15.44:0/1680093999 conn(0x7f003c0fd7a0 msgr2=0x7f003c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 -- 172.21.15.44:0/1680093999 shutdown_connections 2024-03-26T08:35:02.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 -- 172.21.15.44:0/1680093999 wait complete. 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 Processor -- start 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.994+0000 7f0043eb0700 1 -- start start 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0043eb0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 0x7f003c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0043eb0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c1033e0 0x7f003c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0043eb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003c103d80 0x7f003c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0043eb0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c10ddc0 con 0x7f003c1033e0 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0043eb0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c10dc40 con 0x7f003c103d80 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0043eb0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c10df40 con 0x7f003c1021e0 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0041c4c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 0x7f003c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0041c4c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 0x7f003c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:49408/0 (socket says 172.21.15.44:49408) 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0041c4c700 1 -- 172.21.15.44:0/1356971164 learned_addr learned my addr 172.21.15.44:0/1356971164 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:35:02.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f004144b700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c1033e0 0x7f003c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f004244d700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003c103d80 0x7f003c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0041c4c700 1 -- 172.21.15.44:0/1356971164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003c103d80 msgr2=0x7f003c19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0041c4c700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003c103d80 0x7f003c19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0041c4c700 1 -- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c1033e0 msgr2=0x7f003c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0041c4c700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c1033e0 0x7f003c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0041c4c700 1 -- 172.21.15.44:0/1356971164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c19ff40 con 0x7f003c1021e0 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f004244d700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003c103d80 0x7f003c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0041c4c700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 0x7f003c1982e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0038000c00 tx=0x7f00380057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0032ffd700 1 -- 172.21.15.44:0/1356971164 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038027070 con 0x7f003c1021e0 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c1a01d0 con 0x7f003c1021e0 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0032ffd700 1 -- 172.21.15.44:0/1356971164 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0038004150 con 0x7f003c1021e0 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.995+0000 7f0032ffd700 1 -- 172.21.15.44:0/1356971164 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003800f460 con 0x7f003c1021e0 2024-03-26T08:35:02.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.996+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c1a0620 con 0x7f003c1021e0 2024-03-26T08:35:02.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.997+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c068e50 con 0x7f003c1021e0 2024-03-26T08:35:02.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.998+0000 7f0032ffd700 1 -- 172.21.15.44:0/1356971164 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 15) v1 ==== 137233+0+0 (secure 0 0 0) 0x7f003802a070 con 0x7f003c1021e0 2024-03-26T08:35:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.998+0000 7f0032ffd700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00280a2800 0x7f00280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.998+0000 7f0032ffd700 1 -- 172.21.15.44:0/1356971164 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f00380cdfe0 con 0x7f003c1021e0 2024-03-26T08:35:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.999+0000 7f004144b700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00280a2800 0x7f00280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:02.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:02.999+0000 7f004144b700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00280a2800 0x7f00280a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f002c00ac50 tx=0x7f002c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:35:02.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:02 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi016 2024-03-26T08:35:02.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:02 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi016 to 17084M 2024-03-26T08:35:02.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:02 smithi016 ceph-mon[35851]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T08:35:02.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:02 smithi016 ceph-mon[35851]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:02.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:03.003+0000 7f0032ffd700 1 -- 172.21.15.44:0/1356971164 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0038013070 con 0x7f003c1021e0 2024-03-26T08:35:03.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:02 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi016 2024-03-26T08:35:03.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:02 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi016 to 17084M 2024-03-26T08:35:03.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:02 smithi007 ceph-mon[32462]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T08:35:03.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:02 smithi007 ceph-mon[32462]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:03.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:03.217+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f003c0625d0 con 0x7f00280a2800 2024-03-26T08:35:03.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:03 smithi016 ceph-mon[35851]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:35:03.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:35:03.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:35:03.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:04.041 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:03 smithi044 ceph-mon[36461]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:35:04.041 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:35:04.041 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:35:04.042 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:03 smithi007 ceph-mon[32462]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:35:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:35:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:35:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:04.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:04 smithi016 ceph-mon[35851]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:05.040 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:04 smithi044 ceph-mon[36461]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:05.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:04 smithi007 ceph-mon[32462]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:06.133 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:05 smithi044 ceph-mon[36461]: from='client.? 172.21.15.44:0/368579264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10c4a5af-45a4-4285-b6b4-0bc980ec6faa"}]: dispatch 2024-03-26T08:35:06.133 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:05 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10c4a5af-45a4-4285-b6b4-0bc980ec6faa"}]: dispatch 2024-03-26T08:35:06.133 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:05 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10c4a5af-45a4-4285-b6b4-0bc980ec6faa"}]': finished 2024-03-26T08:35:06.133 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:05 smithi044 ceph-mon[36461]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T08:35:06.133 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:05 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:06.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:05 smithi016 ceph-mon[35851]: from='client.? 172.21.15.44:0/368579264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10c4a5af-45a4-4285-b6b4-0bc980ec6faa"}]: dispatch 2024-03-26T08:35:06.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:05 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10c4a5af-45a4-4285-b6b4-0bc980ec6faa"}]: dispatch 2024-03-26T08:35:06.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:05 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10c4a5af-45a4-4285-b6b4-0bc980ec6faa"}]': finished 2024-03-26T08:35:06.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:05 smithi016 ceph-mon[35851]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T08:35:06.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:05 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:05 smithi007 ceph-mon[32462]: from='client.? 172.21.15.44:0/368579264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10c4a5af-45a4-4285-b6b4-0bc980ec6faa"}]: dispatch 2024-03-26T08:35:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:05 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10c4a5af-45a4-4285-b6b4-0bc980ec6faa"}]: dispatch 2024-03-26T08:35:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:05 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10c4a5af-45a4-4285-b6b4-0bc980ec6faa"}]': finished 2024-03-26T08:35:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:05 smithi007 ceph-mon[32462]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T08:35:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:05 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:07.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:06 smithi016 ceph-mon[35851]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:07.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:06 smithi016 ceph-mon[35851]: from='client.? 172.21.15.44:0/1243869057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:35:07.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:06 smithi007 ceph-mon[32462]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:07.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:06 smithi007 ceph-mon[32462]: from='client.? 172.21.15.44:0/1243869057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:35:07.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:06 smithi044 ceph-mon[36461]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:07.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:06 smithi044 ceph-mon[36461]: from='client.? 172.21.15.44:0/1243869057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:35:09.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:09 smithi007 ceph-mon[32462]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:09.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:09 smithi044 ceph-mon[36461]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:09.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:09 smithi016 ceph-mon[35851]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:11.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:11 smithi007 ceph-mon[32462]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:11 smithi044 ceph-mon[36461]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:11.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:11 smithi016 ceph-mon[35851]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:13 smithi007 ceph-mon[32462]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T08:35:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:13 smithi007 ceph-mon[32462]: Deploying daemon osd.2 on smithi044 2024-03-26T08:35:13.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:13 smithi044 ceph-mon[36461]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:13.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T08:35:13.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:13.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:13 smithi044 ceph-mon[36461]: Deploying daemon osd.2 on smithi044 2024-03-26T08:35:13.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:13 smithi016 ceph-mon[35851]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:13.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T08:35:13.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:13.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:13 smithi016 ceph-mon[35851]: Deploying daemon osd.2 on smithi044 2024-03-26T08:35:15.537 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:15 smithi044 ceph-mon[36461]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:15 smithi016 ceph-mon[35851]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:15.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:15 smithi007 ceph-mon[32462]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:17.554 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:17 smithi044 ceph-mon[36461]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:17.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:17 smithi016 ceph-mon[35851]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:17.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:17 smithi007 ceph-mon[32462]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:19.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:19 smithi016 ceph-mon[35851]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:19.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:19 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:19.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:19 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:19.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:19 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:19.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:19 smithi007 ceph-mon[32462]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:19.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:19 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:19.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:19 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:19.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:19 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:19.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:19 smithi044 ceph-mon[36461]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:19.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:19 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:19.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:19 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:19.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:19 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.785+0000 7f0032ffd700 1 -- 172.21.15.44:0/1356971164 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f003c0625d0 con 0x7f00280a2800 2024-03-26T08:35:21.778 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 2 on host 'smithi044' 2024-03-26T08:35:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00280a2800 msgr2=0x7f00280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00280a2800 0x7f00280a4cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f002c00ac50 tx=0x7f002c00a380 comp rx=0 tx=0).stop 2024-03-26T08:35:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 msgr2=0x7f003c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 0x7f003c1982e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0038000c00 tx=0x7f00380057c0 comp rx=0 tx=0).stop 2024-03-26T08:35:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 shutdown_connections 2024-03-26T08:35:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00280a2800 0x7f00280a4cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003c1021e0 0x7f003c1982e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f003c103d80 0x7f003c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 --2- 172.21.15.44:0/1356971164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c1033e0 0x7f003c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 >> 172.21.15.44:0/1356971164 conn(0x7f003c0fd7a0 msgr2=0x7f003c106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:35:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 shutdown_connections 2024-03-26T08:35:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:35:21.787+0000 7f0043eb0700 1 -- 172.21.15.44:0/1356971164 wait complete. 2024-03-26T08:35:22.402 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:22 smithi044 ceph-mon[36461]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:22.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:22.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:22.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.403 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:22.405 DEBUG:teuthology.orchestra.run.smithi044:osd.2> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.2.service 2024-03-26T08:35:22.407 INFO:tasks.cephadm:Deploying osd.3 on smithi007 with /dev/vg_nvme/lv_3... 2024-03-26T08:35:22.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T08:35:22.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:22 smithi007 ceph-mon[32462]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:22.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:22.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:22.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:22 smithi016 ceph-mon[35851]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:22.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:22.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:22.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:22.660 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:35:23.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:23 smithi016 ceph-mon[35851]: from='osd.2 [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:35:23.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:23 smithi016 ceph-mon[35851]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:35:23.534 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:23 smithi044 ceph-mon[36461]: from='osd.2 [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:35:23.535 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:23 smithi044 ceph-mon[36461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:35:23.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:23 smithi007 ceph-mon[32462]: from='osd.2 [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:35:23.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:23 smithi007 ceph-mon[32462]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T08:35:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:24 smithi007 ceph-mon[32462]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:24 smithi007 ceph-mon[32462]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T08:35:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:24 smithi007 ceph-mon[32462]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T08:35:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:24 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:24 smithi007 ceph-mon[32462]: from='osd.2 [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:35:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:24 smithi007 ceph-mon[32462]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:35:24.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:24 smithi016 ceph-mon[35851]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:24.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:24 smithi016 ceph-mon[35851]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T08:35:24.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:24 smithi016 ceph-mon[35851]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T08:35:24.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:24 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:24.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:24 smithi016 ceph-mon[35851]: from='osd.2 [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:35:24.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:24 smithi016 ceph-mon[35851]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:35:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T08:35:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T08:35:24.551 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-26T08:35:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-26T08:35:24.552 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00844594 s, 1.2 GB/s 2024-03-26T08:35:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-26T08:35:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:24 smithi044 ceph-mon[36461]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:24 smithi044 ceph-mon[36461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T08:35:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:24 smithi044 ceph-mon[36461]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T08:35:24.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:24 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:24.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:24 smithi044 ceph-mon[36461]: from='osd.2 [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:35:24.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:24 smithi044 ceph-mon[36461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:35:25.067 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2[41668]: 2024-03-26T08:35:24.688+0000 7fdf6f3a1700 -1 osd.2 0 waiting for initial osdmap 2024-03-26T08:35:25.067 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2[41668]: 2024-03-26T08:35:24.693+0000 7fdf6718c700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:35:25.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-03-26T08:35:25.450 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:25 smithi044 ceph-mon[36461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:35:25.451 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:25 smithi044 ceph-mon[36461]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T08:35:25.451 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:25 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:25.451 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:25 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:25.451 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:25 smithi044 ceph-mon[36461]: from='osd.2 ' entity='osd.2' 2024-03-26T08:35:25.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:25 smithi016 ceph-mon[35851]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:35:25.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:25 smithi016 ceph-mon[35851]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T08:35:25.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:25 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:25.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:25 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:25.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:25 smithi016 ceph-mon[35851]: from='osd.2 ' entity='osd.2' 2024-03-26T08:35:25.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:25 smithi007 ceph-mon[32462]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:35:25.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:25 smithi007 ceph-mon[32462]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T08:35:25.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:25 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:25.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:25 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:25.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:25 smithi007 ceph-mon[32462]: from='osd.2 ' entity='osd.2' 2024-03-26T08:35:25.515 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:35:26.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: osd.2 [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] boot 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T08:35:26.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:26.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:35:26.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:35:26.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:26.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: osd.2 [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] boot 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T08:35:26.507 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:35:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:35:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: osd.2 [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] boot 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T08:35:26.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T08:35:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.282+0000 7f3780b60700 1 -- 172.21.15.7:0/726399023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102cd0 msgr2=0x7f377c109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.282+0000 7f3780b60700 1 --2- 172.21.15.7:0/726399023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102cd0 0x7f377c109580 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f377400b0d0 tx=0x7f377400b3e0 comp rx=0 tx=0).stop 2024-03-26T08:35:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.282+0000 7f3780b60700 1 -- 172.21.15.7:0/726399023 shutdown_connections 2024-03-26T08:35:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.282+0000 7f3780b60700 1 --2- 172.21.15.7:0/726399023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102cd0 0x7f377c109580 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.282+0000 7f3780b60700 1 --2- 172.21.15.7:0/726399023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c101130 0x7f377c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.282+0000 7f3780b60700 1 --2- 172.21.15.7:0/726399023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377c102330 0x7f377c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.282+0000 7f3780b60700 1 -- 172.21.15.7:0/726399023 >> 172.21.15.7:0/726399023 conn(0x7f377c0fc6f0 msgr2=0x7f377c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 -- 172.21.15.7:0/726399023 shutdown_connections 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 -- 172.21.15.7:0/726399023 wait complete. 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 Processor -- start 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 -- start start 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377c101130 0x7f377c06b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102330 0x7f377c19d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c102cd0 0x7f377c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c10cd50 con 0x7f377c102330 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c10cbd0 con 0x7f377c102cd0 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.283+0000 7f3780b60700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c10ced0 con 0x7f377c101130 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3779d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102330 0x7f377c19d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3779d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102330 0x7f377c19d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55816/0 (socket says 172.21.15.7:55816) 2024-03-26T08:35:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3779d9b700 1 -- 172.21.15.7:0/2657721127 learned_addr learned my addr 172.21.15.7:0/2657721127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f377a59c700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377c101130 0x7f377c06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f377ad9d700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c102cd0 0x7f377c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3779d9b700 1 -- 172.21.15.7:0/2657721127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377c101130 msgr2=0x7f377c06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3779d9b700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377c101130 0x7f377c06b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3779d9b700 1 -- 172.21.15.7:0/2657721127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c102cd0 msgr2=0x7f377c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3779d9b700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c102cd0 0x7f377c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3779d9b700 1 -- 172.21.15.7:0/2657721127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c1a0070 con 0x7f377c102330 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f377ad9d700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c102cd0 0x7f377c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3779d9b700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102330 0x7f377c19d110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f376c00d980 tx=0x7f376c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f37737fe700 1 -- 172.21.15.7:0/2657721127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c017070 con 0x7f377c102330 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f37737fe700 1 -- 172.21.15.7:0/2657721127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f376c005020 con 0x7f377c102330 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c1a0360 con 0x7f377c102330 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f37737fe700 1 -- 172.21.15.7:0/2657721127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c009bc0 con 0x7f377c102330 2024-03-26T08:35:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.284+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c1a08d0 con 0x7f377c102330 2024-03-26T08:35:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.285+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c068e50 con 0x7f377c102330 2024-03-26T08:35:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.286+0000 7f37737fe700 1 -- 172.21.15.7:0/2657721127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 15) v1 ==== 137233+0+0 (secure 0 0 0) 0x7f376c0051d0 con 0x7f377c102330 2024-03-26T08:35:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.287+0000 7f37737fe700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37680a2800 0x7f37680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.287+0000 7f377a59c700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37680a2800 0x7f37680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.287+0000 7f37737fe700 1 -- 172.21.15.7:0/2657721127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f376c0cd900 con 0x7f377c102330 2024-03-26T08:35:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.287+0000 7f377a59c700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37680a2800 0x7f37680a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f377c102190 tx=0x7f3764009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:35:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.291+0000 7f37737fe700 1 -- 172.21.15.7:0/2657721127 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f376c020020 con 0x7f377c102330 2024-03-26T08:35:27.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:27 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi044 2024-03-26T08:35:27.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:27 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi044 to 17084M 2024-03-26T08:35:27.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:35:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:27.507+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f377c0625d0 con 0x7f37680a2800 2024-03-26T08:35:27.499 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:27 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi044 2024-03-26T08:35:27.500 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:27 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi044 to 17084M 2024-03-26T08:35:27.500 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:35:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:27 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi044 2024-03-26T08:35:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:27 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi044 to 17084M 2024-03-26T08:35:27.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:35:28.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:28 smithi007 ceph-mon[32462]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:28.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T08:35:28.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:28 smithi007 ceph-mon[32462]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T08:35:28.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:35:28.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:35:28.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:35:28.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:28 smithi016 ceph-mon[35851]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T08:35:28.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:28 smithi016 ceph-mon[35851]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T08:35:28.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:35:28.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:35:28.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:35:28.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:28 smithi044 ceph-mon[36461]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T08:35:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:28 smithi044 ceph-mon[36461]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T08:35:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:35:28.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:35:28.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:35:28.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:28.793 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:35:28 smithi007 sudo[49195]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:35:28.793 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:35:28 smithi007 sudo[49195]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:28.793 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:35:28 smithi007 sudo[49195]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:28.793 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:35:28 smithi007 sudo[49195]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:29.130 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:35:28 smithi007 sudo[49201]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:35:29.130 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:35:28 smithi007 sudo[49201]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:29.130 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:35:28 smithi007 sudo[49201]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:29.130 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:35:28 smithi007 sudo[49201]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:29.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:29 smithi016 ceph-mon[35851]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:35:29.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:29 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:35:29.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:29 smithi016 ceph-mon[35851]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T08:35:29.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:29 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:29.494 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:35:29 smithi016 sudo[44496]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:35:29.494 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:35:29 smithi016 sudo[44496]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:29.494 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:35:29 smithi016 sudo[44496]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:29.494 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:35:29 smithi016 sudo[44496]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:29.496 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:29 smithi007 ceph-mon[32462]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:35:29.496 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:29 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:35:29.496 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:29 smithi007 ceph-mon[32462]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T08:35:29.496 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:29 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:29.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:29 smithi044 ceph-mon[36461]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:35:29.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:29 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:35:29.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:29 smithi044 ceph-mon[36461]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T08:35:29.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:29 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:29.855+0000 7f37737fe700 1 -- 172.21.15.7:0/2657721127 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f376c097dd0 con 0x7f377c102330 2024-03-26T08:35:29.993 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:35:29 smithi016 sudo[44499]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:35:29.994 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:35:29 smithi016 sudo[44499]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:29.994 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:35:29 smithi016 sudo[44499]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:29.994 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:35:29 smithi016 sudo[44499]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:30.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:30 smithi016 ceph-mon[35851]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:30.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:30 smithi016 ceph-mon[35851]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T08:35:30.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:30 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:30.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:30 smithi016 ceph-mon[35851]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T08:35:30.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:30 smithi007 ceph-mon[32462]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:30.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:30 smithi007 ceph-mon[32462]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T08:35:30.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:30 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:30.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:30 smithi007 ceph-mon[32462]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T08:35:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:30 smithi044 ceph-mon[36461]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:30 smithi044 ceph-mon[36461]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T08:35:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:30 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:30 smithi044 ceph-mon[36461]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-26T08:35:30.567 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:30 smithi044 sudo[45617]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:35:30.567 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:30 smithi044 sudo[45617]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:30.567 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:30 smithi044 sudo[45617]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:30.567 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:30 smithi044 sudo[45617]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:31.066 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:30 smithi044 sudo[45620]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:35:31.066 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:30 smithi044 sudo[45620]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:31.067 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:30 smithi044 sudo[45620]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:31.067 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:35:30 smithi044 sudo[45620]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:31.209 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:30 smithi007 sudo[49311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:35:31.209 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:30 smithi007 sudo[49311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:31.209 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:30 smithi007 sudo[49311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:31.209 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:30 smithi007 sudo[49311]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:31.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:31 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4265880004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46aab34-7b13-4e83-8fbd-7c207cd246e9"}]: dispatch 2024-03-26T08:35:31.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:31 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4265880004' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46aab34-7b13-4e83-8fbd-7c207cd246e9"}]': finished 2024-03-26T08:35:31.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:31 smithi016 ceph-mon[35851]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T08:35:31.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:31.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:31.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:31 smithi016 ceph-mon[35851]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 sudo[49314]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 sudo[49314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 sudo[49314]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 sudo[49314]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4265880004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46aab34-7b13-4e83-8fbd-7c207cd246e9"}]: dispatch 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4265880004' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46aab34-7b13-4e83-8fbd-7c207cd246e9"}]': finished 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 ceph-mon[32462]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:31.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:31 smithi007 ceph-mon[32462]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:35:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4265880004' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b46aab34-7b13-4e83-8fbd-7c207cd246e9"}]: dispatch 2024-03-26T08:35:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4265880004' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46aab34-7b13-4e83-8fbd-7c207cd246e9"}]': finished 2024-03-26T08:35:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 ceph-mon[36461]: osdmap e22: 4 total, 3 up, 4 in 2024-03-26T08:35:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:31.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 ceph-mon[36461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:35:32.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 sudo[45623]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:35:32.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 sudo[45623]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:32.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 sudo[45623]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:32.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:31 smithi044 sudo[45623]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:32.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:32 smithi044 sudo[45626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:35:32.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:32 smithi044 sudo[45626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:32.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:32 smithi044 sudo[45626]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:32.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:32 smithi044 sudo[45626]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:32.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:32 smithi016 sudo[44502]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T08:35:32.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:32 smithi016 sudo[44502]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:32.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:32 smithi016 sudo[44502]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:32.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:32 smithi016 sudo[44502]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:33.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:33.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:35:33.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:35:33.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3602626688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:35:33.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:35:33.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:35:33.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:35:33.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:35:33.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:35:33.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:35:33.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:35:33.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:35:33.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:35:33.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:35:33.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:33.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:35:33.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3602626688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:35:33.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 sudo[44505]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 sudo[44505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 sudo[44505]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 sudo[44505]: pam_unix(sudo:session): session closed for user root 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3602626688' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:35:33.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:35:33.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:35:33.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:35:33.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T08:35:33.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T08:35:33.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T08:35:34.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:34 smithi007 ceph-mon[32462]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:35:34.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:34 smithi007 ceph-mon[32462]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:34.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:34 smithi007 ceph-mon[32462]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:35:34.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:34 smithi044 ceph-mon[36461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:35:34.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:34 smithi044 ceph-mon[36461]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:34.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:34 smithi044 ceph-mon[36461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:35:34.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:34 smithi016 ceph-mon[35851]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T08:35:34.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:34 smithi016 ceph-mon[35851]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:34.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:34 smithi016 ceph-mon[35851]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T08:35:36.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:36 smithi007 ceph-mon[32462]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:36.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:36 smithi044 ceph-mon[36461]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:36.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:36 smithi016 ceph-mon[35851]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:38.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:38 smithi007 ceph-mon[32462]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:38.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T08:35:38.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:38.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:38 smithi044 ceph-mon[36461]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:38.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T08:35:38.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:38.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:38 smithi016 ceph-mon[35851]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:38.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T08:35:38.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:39.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:39 smithi007 ceph-mon[32462]: Deploying daemon osd.3 on smithi007 2024-03-26T08:35:39.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:39 smithi044 ceph-mon[36461]: Deploying daemon osd.3 on smithi007 2024-03-26T08:35:39.394 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:39 smithi016 ceph-mon[35851]: Deploying daemon osd.3 on smithi007 2024-03-26T08:35:40.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:40 smithi007 ceph-mon[32462]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-26T08:35:40.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:40 smithi044 ceph-mon[36461]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-26T08:35:40.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:40 smithi016 ceph-mon[35851]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-03-26T08:35:42.276 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:42 smithi007 ceph-mon[32462]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-26T08:35:42.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:42 smithi044 ceph-mon[36461]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-26T08:35:42.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:42 smithi016 ceph-mon[35851]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-03-26T08:35:44.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:44 smithi007 ceph-mon[32462]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-26T08:35:44.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:44 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:44.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:44 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:44.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:44 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:44.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:44 smithi044 ceph-mon[36461]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-26T08:35:44.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:44 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:44.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:44 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:44.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:44 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:44.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:44 smithi016 ceph-mon[35851]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-26T08:35:44.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:44 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:44.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:44 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:44.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:44 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:46.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:46 smithi007 ceph-mon[32462]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T08:35:46.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:46 smithi044 ceph-mon[36461]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T08:35:46.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:46 smithi016 ceph-mon[35851]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T08:35:47.022 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 3 on host 'smithi007' 2024-03-26T08:35:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.030+0000 7f37737fe700 1 -- 172.21.15.7:0/2657721127 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f377c0625d0 con 0x7f37680a2800 2024-03-26T08:35:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37680a2800 msgr2=0x7f37680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37680a2800 0x7f37680a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f377c102190 tx=0x7f3764009190 comp rx=0 tx=0).stop 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102330 msgr2=0x7f377c19d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102330 0x7f377c19d110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f376c00d980 tx=0x7f376c00dc90 comp rx=0 tx=0).stop 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 shutdown_connections 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37680a2800 0x7f37680a4cc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f377c101130 0x7f377c06b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f377c102cd0 0x7f377c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 --2- 172.21.15.7:0/2657721127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f377c102330 0x7f377c19d110 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 >> 172.21.15.7:0/2657721127 conn(0x7f377c0fc6f0 msgr2=0x7f377c1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 shutdown_connections 2024-03-26T08:35:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:35:47.032+0000 7f3780b60700 1 -- 172.21.15.7:0/2657721127 wait complete. 2024-03-26T08:35:47.691 DEBUG:teuthology.orchestra.run.smithi007:osd.3> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.3.service 2024-03-26T08:35:47.694 INFO:tasks.cephadm:Deploying osd.4 on smithi016 with /dev/vg_nvme/lv_3... 2024-03-26T08:35:47.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T08:35:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:35:48.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:48 smithi007 ceph-mon[32462]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T08:35:48.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:48 smithi007 ceph-mon[32462]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T08:35:48.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:48 smithi044 ceph-mon[36461]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T08:35:48.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:48 smithi044 ceph-mon[36461]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T08:35:48.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.446 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:48 smithi016 ceph-mon[35851]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T08:35:48.446 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.446 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.446 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:48 smithi016 ceph-mon[35851]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T08:35:48.446 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:48.446 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:49.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:49 smithi007 ceph-mon[32462]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T08:35:49.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:49 smithi007 ceph-mon[32462]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T08:35:49.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:49 smithi007 ceph-mon[32462]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:35:49.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:49.294 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:35:49 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3[51327]: 2024-03-26T08:35:49.047+0000 7ffb95076700 -1 osd.3 0 waiting for initial osdmap 2024-03-26T08:35:49.294 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:35:49 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3[51327]: 2024-03-26T08:35:49.053+0000 7ffb90e69700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:35:49.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:49 smithi044 ceph-mon[36461]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T08:35:49.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:49 smithi044 ceph-mon[36461]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T08:35:49.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:49 smithi044 ceph-mon[36461]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:35:49.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:49.392 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:49 smithi016 ceph-mon[35851]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T08:35:49.392 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:49 smithi016 ceph-mon[35851]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T08:35:49.392 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:49 smithi016 ceph-mon[35851]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:35:49.392 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:50.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:50 smithi007 ceph-mon[32462]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:50.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:50 smithi007 ceph-mon[32462]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:35:50.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:50 smithi007 ceph-mon[32462]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T08:35:50.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:50.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:50.298 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T08:35:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T08:35:50.299 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-03-26T08:35:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-03-26T08:35:50.299 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00818757 s, 1.3 GB/s 2024-03-26T08:35:50.299 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-03-26T08:35:50.299 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:50 smithi016 ceph-mon[35851]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:50.299 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:50 smithi016 ceph-mon[35851]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:35:50.299 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:50 smithi016 ceph-mon[35851]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T08:35:50.299 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:50.299 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:50 smithi044 ceph-mon[36461]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:50 smithi044 ceph-mon[36461]: from='osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:35:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:50 smithi044 ceph-mon[36461]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T08:35:50.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:50.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:51.172 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:51 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:35:51.172 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:51 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:35:51.173 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:51.173 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:51 smithi016 ceph-mon[35851]: osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] boot 2024-03-26T08:35:51.173 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:51 smithi016 ceph-mon[35851]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T08:35:51.173 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:51.173 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:51 smithi016 ceph-mon[35851]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T08:35:51.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-03-26T08:35:51.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:51 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:35:51.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:51 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:35:51.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:51.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:51 smithi007 ceph-mon[32462]: osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] boot 2024-03-26T08:35:51.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:51 smithi007 ceph-mon[32462]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T08:35:51.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:51.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:51 smithi007 ceph-mon[32462]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T08:35:51.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:51 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:35:51.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:51 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:35:51.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:51.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:51 smithi044 ceph-mon[36461]: osd.3 [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] boot 2024-03-26T08:35:51.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:51 smithi044 ceph-mon[36461]: osdmap e25: 4 total, 4 up, 4 in 2024-03-26T08:35:51.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T08:35:51.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:51 smithi044 ceph-mon[36461]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T08:35:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:35:52.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:52.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:52.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T08:35:52.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:35:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:35:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.035+0000 7f956d24c700 1 -- 172.21.15.16:0/3647885904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 msgr2=0x7f956810c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.035+0000 7f956d24c700 1 --2- 172.21.15.16:0/3647885904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 0x7f956810c520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f956000b0a0 tx=0x7f956000b3b0 comp rx=0 tx=0).stop 2024-03-26T08:35:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.035+0000 7f956d24c700 1 -- 172.21.15.16:0/3647885904 shutdown_connections 2024-03-26T08:35:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.035+0000 7f956d24c700 1 --2- 172.21.15.16:0/3647885904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95680761c0 0x7f9568074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.035+0000 7f956d24c700 1 --2- 172.21.15.16:0/3647885904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 0x7f956810c520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.035+0000 7f956d24c700 1 --2- 172.21.15.16:0/3647885904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9568074bf0 0x7f9568075070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.035+0000 7f956d24c700 1 -- 172.21.15.16:0/3647885904 >> 172.21.15.16:0/3647885904 conn(0x7f95680fd7a0 msgr2=0x7f95680ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:35:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.035+0000 7f956d24c700 1 -- 172.21.15.16:0/3647885904 shutdown_connections 2024-03-26T08:35:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.035+0000 7f956d24c700 1 -- 172.21.15.16:0/3647885904 wait complete. 2024-03-26T08:35:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.036+0000 7f956d24c700 1 Processor -- start 2024-03-26T08:35:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.036+0000 7f956d24c700 1 -- start start 2024-03-26T08:35:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956d24c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9568074bf0 0x7f956819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956d24c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95680761c0 0x7f956819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956d24c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 0x7f95681a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956d24c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95681100c0 con 0x7f9568074bf0 2024-03-26T08:35:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956d24c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956810ff40 con 0x7f956810a090 2024-03-26T08:35:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956d24c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9568110240 con 0x7f95680761c0 2024-03-26T08:35:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f9566d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9568074bf0 0x7f956819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 0x7f95681a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956659c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95680761c0 0x7f956819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 0x7f95681a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55504/0 (socket says 172.21.15.16:55504) 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956759e700 1 -- 172.21.15.16:0/1049175075 learned_addr learned my addr 172.21.15.16:0/1049175075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f9566d9d700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9568074bf0 0x7f956819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.16:33120/0 (socket says 172.21.15.16:33120) 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956759e700 1 -- 172.21.15.16:0/1049175075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95680761c0 msgr2=0x7f956819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956759e700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95680761c0 0x7f956819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956759e700 1 -- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9568074bf0 msgr2=0x7f956819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956759e700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9568074bf0 0x7f956819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956759e700 1 -- 172.21.15.16:0/1049175075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95681a2320 con 0x7f956810a090 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.037+0000 7f956759e700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 0x7f95681a1c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9560007f00 tx=0x7f956000be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.038+0000 7f954ffff700 1 -- 172.21.15.16:0/1049175075 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560021030 con 0x7f956810a090 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.038+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95681a25b0 con 0x7f956810a090 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.038+0000 7f954ffff700 1 -- 172.21.15.16:0/1049175075 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9560007a80 con 0x7f956810a090 2024-03-26T08:35:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.038+0000 7f954ffff700 1 -- 172.21.15.16:0/1049175075 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956001d470 con 0x7f956810a090 2024-03-26T08:35:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.038+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95681a2ae0 con 0x7f956810a090 2024-03-26T08:35:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.039+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9568068e50 con 0x7f956810a090 2024-03-26T08:35:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.041+0000 7f954ffff700 1 -- 172.21.15.16:0/1049175075 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9560024070 con 0x7f956810a090 2024-03-26T08:35:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.041+0000 7f954ffff700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f95500a2910 0x7f95500a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:35:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.042+0000 7f954ffff700 1 -- 172.21.15.16:0/1049175075 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f95600cecd0 con 0x7f956810a090 2024-03-26T08:35:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.042+0000 7f9566d9d700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f95500a2910 0x7f95500a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:35:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.042+0000 7f9566d9d700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f95500a2910 0x7f95500a4dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95680765a0 tx=0x7f9558009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:35:53.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.045+0000 7f954ffff700 1 -- 172.21.15.16:0/1049175075 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f956009cf60 con 0x7f956810a090 2024-03-26T08:35:53.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:53 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi007 2024-03-26T08:35:53.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:53 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi007 to 8546M 2024-03-26T08:35:53.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:35:53.265+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f95680625d0 con 0x7f95500a2910 2024-03-26T08:35:53.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:53 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi007 2024-03-26T08:35:53.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:53 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi007 to 8546M 2024-03-26T08:35:53.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:53 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi007 2024-03-26T08:35:53.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:53 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi007 to 8546M 2024-03-26T08:35:54.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:54 smithi016 ceph-mon[35851]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T08:35:54.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:54 smithi016 ceph-mon[35851]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T08:35:54.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:54 smithi016 ceph-mon[35851]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T08:35:54.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:54 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:35:54.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:54 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:35:54.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:54 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:54.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:54 smithi044 ceph-mon[36461]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T08:35:54.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:54 smithi044 ceph-mon[36461]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T08:35:54.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:54 smithi044 ceph-mon[36461]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T08:35:54.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:54 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:35:54.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:54 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:35:54.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:54 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:54.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:54 smithi007 ceph-mon[32462]: pgmap v104: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T08:35:54.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:54 smithi007 ceph-mon[32462]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-03-26T08:35:54.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:54 smithi007 ceph-mon[32462]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T08:35:54.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:54 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:35:54.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:54 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:35:54.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:54 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:35:55.216 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:55 smithi016 ceph-mon[35851]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:35:55.216 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:55 smithi016 ceph-mon[35851]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T08:35:55.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:55 smithi044 ceph-mon[36461]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:35:55.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:55 smithi044 ceph-mon[36461]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T08:35:55.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:55 smithi007 ceph-mon[32462]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:35:55.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:55 smithi007 ceph-mon[32462]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T08:35:56.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:56 smithi044 ceph-mon[36461]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T08:35:56.432 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:56 smithi016 ceph-mon[35851]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T08:35:56.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:56 smithi007 ceph-mon[32462]: pgmap v107: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T08:35:57.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:57 smithi016 ceph-mon[35851]: from='client.? 172.21.15.16:0/3912583735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a3ff9e-09b6-4255-96fd-9d155625811b"}]: dispatch 2024-03-26T08:35:57.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:57 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a3ff9e-09b6-4255-96fd-9d155625811b"}]: dispatch 2024-03-26T08:35:57.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:57 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78a3ff9e-09b6-4255-96fd-9d155625811b"}]': finished 2024-03-26T08:35:57.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:57 smithi016 ceph-mon[35851]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T08:35:57.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:57 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:35:57.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:57 smithi016 ceph-mon[35851]: from='client.? 172.21.15.16:0/2670319641' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:35:57.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:57 smithi007 ceph-mon[32462]: from='client.? 172.21.15.16:0/3912583735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a3ff9e-09b6-4255-96fd-9d155625811b"}]: dispatch 2024-03-26T08:35:57.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:57 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a3ff9e-09b6-4255-96fd-9d155625811b"}]: dispatch 2024-03-26T08:35:57.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:57 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78a3ff9e-09b6-4255-96fd-9d155625811b"}]': finished 2024-03-26T08:35:57.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:57 smithi007 ceph-mon[32462]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T08:35:57.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:35:57.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:57 smithi007 ceph-mon[32462]: from='client.? 172.21.15.16:0/2670319641' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:35:57.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:57 smithi044 ceph-mon[36461]: from='client.? 172.21.15.16:0/3912583735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a3ff9e-09b6-4255-96fd-9d155625811b"}]: dispatch 2024-03-26T08:35:57.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:57 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78a3ff9e-09b6-4255-96fd-9d155625811b"}]: dispatch 2024-03-26T08:35:57.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:57 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78a3ff9e-09b6-4255-96fd-9d155625811b"}]': finished 2024-03-26T08:35:57.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:57 smithi044 ceph-mon[36461]: osdmap e30: 5 total, 4 up, 5 in 2024-03-26T08:35:57.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:35:57.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:57 smithi044 ceph-mon[36461]: from='client.? 172.21.15.16:0/2670319641' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:35:58.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:58 smithi016 ceph-mon[35851]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T08:35:58.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:58 smithi007 ceph-mon[32462]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T08:35:58.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:58 smithi044 ceph-mon[36461]: pgmap v109: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-03-26T08:35:59.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:59 smithi016 ceph-mon[35851]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T08:35:59.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:35:59 smithi016 ceph-mon[35851]: Cluster is now healthy 2024-03-26T08:35:59.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:59 smithi007 ceph-mon[32462]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T08:35:59.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:35:59 smithi007 ceph-mon[32462]: Cluster is now healthy 2024-03-26T08:35:59.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:59 smithi044 ceph-mon[36461]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-03-26T08:35:59.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:35:59 smithi044 ceph-mon[36461]: Cluster is now healthy 2024-03-26T08:36:00.348 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:00 smithi016 ceph-mon[35851]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:36:00.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:00 smithi007 ceph-mon[32462]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:36:00.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:00 smithi044 ceph-mon[36461]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:36:02.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:02 smithi016 ceph-mon[35851]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-26T08:36:02.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:02 smithi007 ceph-mon[32462]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-26T08:36:02.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:02 smithi044 ceph-mon[36461]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-26T08:36:04.355 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:04 smithi016 ceph-mon[35851]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T08:36:04.355 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T08:36:04.355 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:04 smithi007 ceph-mon[32462]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T08:36:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T08:36:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:04 smithi044 ceph-mon[36461]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-26T08:36:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T08:36:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:05.373 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:05 smithi016 ceph-mon[35851]: Deploying daemon osd.4 on smithi016 2024-03-26T08:36:05.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:05 smithi007 ceph-mon[32462]: Deploying daemon osd.4 on smithi016 2024-03-26T08:36:05.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:05 smithi044 ceph-mon[36461]: Deploying daemon osd.4 on smithi016 2024-03-26T08:36:06.480 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:06 smithi016 ceph-mon[35851]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:36:06.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:06 smithi007 ceph-mon[32462]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:36:06.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:06 smithi044 ceph-mon[36461]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:36:08.381 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:08 smithi016 ceph-mon[35851]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-26T08:36:08.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:08 smithi007 ceph-mon[32462]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-26T08:36:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:08 smithi044 ceph-mon[36461]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-26T08:36:10.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:10 smithi007 ceph-mon[32462]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T08:36:10.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:10.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:10.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:10.302 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:10 smithi016 ceph-mon[35851]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T08:36:10.302 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:10.303 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:10.303 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:10.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:10 smithi044 ceph-mon[36461]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T08:36:10.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:10.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:10.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:12.237 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:12 smithi016 ceph-mon[35851]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:12.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:12 smithi007 ceph-mon[32462]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:12.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:12 smithi044 ceph-mon[36461]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:12.575 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 4 on host 'smithi016' 2024-03-26T08:36:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.583+0000 7f954ffff700 1 -- 172.21.15.16:0/1049175075 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f95680625d0 con 0x7f95500a2910 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f95500a2910 msgr2=0x7f95500a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f95500a2910 0x7f95500a4dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f95680765a0 tx=0x7f9558009190 comp rx=0 tx=0).stop 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 msgr2=0x7f95681a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 0x7f95681a1c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9560007f00 tx=0x7f956000be60 comp rx=0 tx=0).stop 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 shutdown_connections 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f95500a2910 0x7f95500a4dd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956810a090 0x7f95681a1c80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9568074bf0 0x7f956819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 --2- 172.21.15.16:0/1049175075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f95680761c0 0x7f956819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 >> 172.21.15.16:0/1049175075 conn(0x7f95680fd7a0 msgr2=0x7f9568105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:36:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 shutdown_connections 2024-03-26T08:36:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:36:12.586+0000 7f956d24c700 1 -- 172.21.15.16:0/1049175075 wait complete. 2024-03-26T08:36:13.211 DEBUG:teuthology.orchestra.run.smithi016:osd.4> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.4.service 2024-03-26T08:36:13.213 INFO:tasks.cephadm:Deploying osd.5 on smithi044 with /dev/vg_nvme/lv_3... 2024-03-26T08:36:13.213 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T08:36:13.471 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:36:13.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:13 smithi044 ceph-mon[36461]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:13.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:13.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:13.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:13.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:13.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:13 smithi044 ceph-mon[36461]: from='osd.4 [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:36:13.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:13 smithi044 ceph-mon[36461]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:36:13.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:13 smithi016 ceph-mon[35851]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:13.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:13.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:13.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:13.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:13.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:13 smithi016 ceph-mon[35851]: from='osd.4 [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:36:13.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:13 smithi016 ceph-mon[35851]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:36:14.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:13 smithi007 ceph-mon[32462]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:14.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:14.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:14.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:14.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:14.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:13 smithi007 ceph-mon[32462]: from='osd.4 [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:36:14.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:13 smithi007 ceph-mon[32462]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T08:36:14.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:14 smithi044 ceph-mon[36461]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T08:36:14.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:14 smithi044 ceph-mon[36461]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T08:36:14.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:14 smithi044 ceph-mon[36461]: from='osd.4 [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:36:14.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:14 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:14.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:14 smithi044 ceph-mon[36461]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:36:14.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:14 smithi016 ceph-mon[35851]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T08:36:14.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:14 smithi016 ceph-mon[35851]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T08:36:14.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:14 smithi016 ceph-mon[35851]: from='osd.4 [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:36:14.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:14 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:14.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:14 smithi016 ceph-mon[35851]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:36:14.994 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:36:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4[47315]: 2024-03-26T08:36:14.680+0000 7f0eadee3700 -1 osd.4 0 waiting for initial osdmap 2024-03-26T08:36:14.994 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:36:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4[47315]: 2024-03-26T08:36:14.686+0000 7f0ea9cd6700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:36:15.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:14 smithi007 ceph-mon[32462]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T08:36:15.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:14 smithi007 ceph-mon[32462]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T08:36:15.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:14 smithi007 ceph-mon[32462]: from='osd.4 [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:36:15.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:14 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:15.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:14 smithi007 ceph-mon[32462]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:36:15.457 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T08:36:15.457 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T08:36:15.457 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-03-26T08:36:15.457 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-03-26T08:36:15.457 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109682 s, 956 MB/s 2024-03-26T08:36:15.457 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 2024-03-26T08:36:15.458 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-03-26T08:36:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:15 smithi016 ceph-mon[35851]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:15 smithi016 ceph-mon[35851]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:36:15.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:15 smithi016 ceph-mon[35851]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T08:36:15.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:15 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:15.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:15 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:16.040 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:15 smithi044 ceph-mon[36461]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:16.040 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:15 smithi044 ceph-mon[36461]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:36:16.040 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:15 smithi044 ceph-mon[36461]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T08:36:16.040 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:15 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:16.040 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:15 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:16.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:15 smithi007 ceph-mon[32462]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:16.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:15 smithi007 ceph-mon[32462]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:36:16.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:15 smithi007 ceph-mon[32462]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T08:36:16.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:15 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:16.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:15 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:16.066 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi044:vg_nvme/lv_3 2024-03-26T08:36:16.324 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:36:16.777 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:36:16.777 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:36:16.777 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: osd.4 [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] boot 2024-03-26T08:36:16.777 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T08:36:16.777 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:16.777 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi016 2024-03-26T08:36:16.777 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:16.777 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi016 to 8542M 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:16.778 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:16.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: osd.4 [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] boot 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi016 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:16.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi016 to 8542M 2024-03-26T08:36:16.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:16.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:16.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:16.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:16.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:16.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:16.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:16.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:17.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:36:17.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:36:17.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: osd.4 [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] boot 2024-03-26T08:36:17.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: osdmap e33: 5 total, 5 up, 5 in 2024-03-26T08:36:17.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T08:36:17.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi016 2024-03-26T08:36:17.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi016 to 8542M 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:17.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:17.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.909+0000 7f6bd02de700 1 -- 172.21.15.44:0/2905841354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 msgr2=0x7f6bc8109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.909+0000 7f6bd02de700 1 --2- 172.21.15.44:0/2905841354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 0x7f6bc8109580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b0a0 tx=0x7f6bc000b3b0 comp rx=0 tx=0).stop 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.909+0000 7f6bd02de700 1 -- 172.21.15.44:0/2905841354 shutdown_connections 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.909+0000 7f6bd02de700 1 --2- 172.21.15.44:0/2905841354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 0x7f6bc8109580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.909+0000 7f6bd02de700 1 --2- 172.21.15.44:0/2905841354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc8101130 0x7f6bc8101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.909+0000 7f6bd02de700 1 --2- 172.21.15.44:0/2905841354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc8102330 0x7f6bc8102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.909+0000 7f6bd02de700 1 -- 172.21.15.44:0/2905841354 >> 172.21.15.44:0/2905841354 conn(0x7f6bc80fc6f0 msgr2=0x7f6bc80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.909+0000 7f6bd02de700 1 -- 172.21.15.44:0/2905841354 shutdown_connections 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.909+0000 7f6bd02de700 1 -- 172.21.15.44:0/2905841354 wait complete. 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bd02de700 1 Processor -- start 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bd02de700 1 -- start start 2024-03-26T08:36:17.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bd02de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc8101130 0x7f6bc806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bd02de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc8102330 0x7f6bc819d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bd02de700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 0x7f6bc819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bd02de700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc810ccb0 con 0x7f6bc8101130 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bd02de700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc810cb30 con 0x7f6bc8102330 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bd02de700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc810ce30 con 0x7f6bc8102cd0 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce87b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 0x7f6bc819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce07a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc8101130 0x7f6bc806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce87b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 0x7f6bc819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38024/0 (socket says 172.21.15.44:38024) 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce87b700 1 -- 172.21.15.44:0/435384654 learned_addr learned my addr 172.21.15.44:0/435384654 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bcd879700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc8102330 0x7f6bc819d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce87b700 1 -- 172.21.15.44:0/435384654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc8102330 msgr2=0x7f6bc819d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce87b700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc8102330 0x7f6bc819d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:17.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce87b700 1 -- 172.21.15.44:0/435384654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc8101130 msgr2=0x7f6bc806b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce87b700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc8101130 0x7f6bc806b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce87b700 1 -- 172.21.15.44:0/435384654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc819fff0 con 0x7f6bc8102cd0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.910+0000 7f6bce87b700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 0x7f6bc819f720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b780 tx=0x7f6bc00094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.911+0000 7f6bbf7fe700 1 -- 172.21.15.44:0/435384654 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0021070 con 0x7f6bc8102cd0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.911+0000 7f6bbf7fe700 1 -- 172.21.15.44:0/435384654 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6bc0007c40 con 0x7f6bc8102cd0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.911+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc81a0280 con 0x7f6bc8102cd0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.911+0000 7f6bbf7fe700 1 -- 172.21.15.44:0/435384654 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0005260 con 0x7f6bc8102cd0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.911+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc81a0780 con 0x7f6bc8102cd0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.912+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc8068e50 con 0x7f6bc8102cd0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.913+0000 7f6bbf7fe700 1 -- 172.21.15.44:0/435384654 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6bc0024070 con 0x7f6bc8102cd0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.914+0000 7f6bbf7fe700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6bb40a2910 0x7f6bb40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.914+0000 7f6bbf7fe700 1 -- 172.21.15.44:0/435384654 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f6bc000f020 con 0x7f6bc8102cd0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.914+0000 7f6bce07a700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6bb40a2910 0x7f6bb40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:36:17.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.915+0000 7f6bce07a700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6bb40a2910 0x7f6bb40a4dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8102190 tx=0x7f6bc400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:36:17.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:17.918+0000 7f6bbf7fe700 1 -- 172.21.15.44:0/435384654 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6bc002b020 con 0x7f6bc8102cd0 2024-03-26T08:36:17.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:17 smithi016 ceph-mon[35851]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:17.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:17 smithi016 ceph-mon[35851]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T08:36:18.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:17 smithi007 ceph-mon[32462]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:18.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:17 smithi007 ceph-mon[32462]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T08:36:18.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:17 smithi044 ceph-mon[36461]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T08:36:18.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:17 smithi044 ceph-mon[36461]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T08:36:18.126 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:18.134+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bc80625d0 con 0x7f6bb40a2910 2024-03-26T08:36:18.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:18 smithi016 ceph-mon[35851]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:36:18.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:36:18.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:36:18.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:19.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:18 smithi007 ceph-mon[32462]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:36:19.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:36:19.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:36:19.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:19.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:18 smithi044 ceph-mon[36461]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:36:19.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:36:19.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:36:19.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:19.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:19 smithi044 ceph-mon[36461]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:19.944 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:19 smithi007 ceph-mon[32462]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:19.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:19 smithi016 ceph-mon[35851]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:21.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:21 smithi044 ceph-mon[36461]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:21.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:21 smithi044 ceph-mon[36461]: from='client.? 172.21.15.44:0/283818029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab45e6b1-3006-4af1-90f3-455a09f442f5"}]: dispatch 2024-03-26T08:36:21.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:21 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab45e6b1-3006-4af1-90f3-455a09f442f5"}]: dispatch 2024-03-26T08:36:21.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:21 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab45e6b1-3006-4af1-90f3-455a09f442f5"}]': finished 2024-03-26T08:36:21.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:21 smithi044 ceph-mon[36461]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T08:36:21.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:21 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:21.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:21 smithi016 ceph-mon[35851]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:21.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:21 smithi016 ceph-mon[35851]: from='client.? 172.21.15.44:0/283818029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab45e6b1-3006-4af1-90f3-455a09f442f5"}]: dispatch 2024-03-26T08:36:21.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:21 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab45e6b1-3006-4af1-90f3-455a09f442f5"}]: dispatch 2024-03-26T08:36:21.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:21 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab45e6b1-3006-4af1-90f3-455a09f442f5"}]': finished 2024-03-26T08:36:21.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:21 smithi016 ceph-mon[35851]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T08:36:21.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:21 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:22.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:21 smithi007 ceph-mon[32462]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:22.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:21 smithi007 ceph-mon[32462]: from='client.? 172.21.15.44:0/283818029' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab45e6b1-3006-4af1-90f3-455a09f442f5"}]: dispatch 2024-03-26T08:36:22.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:21 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab45e6b1-3006-4af1-90f3-455a09f442f5"}]: dispatch 2024-03-26T08:36:22.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:21 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab45e6b1-3006-4af1-90f3-455a09f442f5"}]': finished 2024-03-26T08:36:22.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:21 smithi007 ceph-mon[32462]: osdmap e35: 6 total, 5 up, 6 in 2024-03-26T08:36:22.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:21 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:22.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:22 smithi016 ceph-mon[35851]: from='client.? 172.21.15.44:0/1490967438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:36:23.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:22 smithi007 ceph-mon[32462]: from='client.? 172.21.15.44:0/1490967438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:36:23.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:22 smithi044 ceph-mon[36461]: from='client.? 172.21.15.44:0/1490967438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:36:23.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:23 smithi016 ceph-mon[35851]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:24.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:23 smithi007 ceph-mon[32462]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:24.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:23 smithi044 ceph-mon[36461]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:25.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:25 smithi044 ceph-mon[36461]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:25.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:25 smithi016 ceph-mon[35851]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:26.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:25 smithi007 ceph-mon[32462]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:27.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:27 smithi044 ceph-mon[36461]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:27.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:27 smithi016 ceph-mon[35851]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:28.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:27 smithi007 ceph-mon[32462]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:28.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T08:36:28.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:29.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T08:36:29.045 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:29.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T08:36:29.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:29.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:29 smithi044 ceph-mon[36461]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:29.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:29 smithi044 ceph-mon[36461]: Deploying daemon osd.5 on smithi044 2024-03-26T08:36:29.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:29 smithi016 ceph-mon[35851]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:29.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:29 smithi016 ceph-mon[35851]: Deploying daemon osd.5 on smithi044 2024-03-26T08:36:30.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:29 smithi007 ceph-mon[32462]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:30.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:29 smithi007 ceph-mon[32462]: Deploying daemon osd.5 on smithi044 2024-03-26T08:36:31.970 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:31 smithi044 ceph-mon[36461]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:31.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:31 smithi016 ceph-mon[35851]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:32.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:31 smithi007 ceph-mon[32462]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:33.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:33 smithi044 ceph-mon[36461]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:33.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:33 smithi016 ceph-mon[35851]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:34.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:33 smithi007 ceph-mon[32462]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:34.943 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:34.943 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:34.943 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:34.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:35.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:35.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:35 smithi016 ceph-mon[35851]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:36.027 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:35 smithi044 ceph-mon[36461]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:36.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:35 smithi007 ceph-mon[32462]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:37.839 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:37 smithi044 ceph-mon[36461]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:37.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.900+0000 7f6bbf7fe700 1 -- 172.21.15.44:0/435384654 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6bc80625d0 con 0x7f6bb40a2910 2024-03-26T08:36:37.892 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 5 on host 'smithi044' 2024-03-26T08:36:37.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6bb40a2910 msgr2=0x7f6bb40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:37.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6bb40a2910 0x7f6bb40a4dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8102190 tx=0x7f6bc400b410 comp rx=0 tx=0).stop 2024-03-26T08:36:37.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 msgr2=0x7f6bc819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:37.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 0x7f6bc819f720 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b780 tx=0x7f6bc00094f0 comp rx=0 tx=0).stop 2024-03-26T08:36:37.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 shutdown_connections 2024-03-26T08:36:37.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6bb40a2910 0x7f6bb40a4dd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:37.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6bc8102cd0 0x7f6bc819f720 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:37.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bc8101130 0x7f6bc806b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:37.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 --2- 172.21.15.44:0/435384654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc8102330 0x7f6bc819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:37.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 >> 172.21.15.44:0/435384654 conn(0x7f6bc80fc6f0 msgr2=0x7f6bc8105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:36:37.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 shutdown_connections 2024-03-26T08:36:37.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:36:37.904+0000 7f6bd02de700 1 -- 172.21.15.44:0/435384654 wait complete. 2024-03-26T08:36:37.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:37 smithi016 ceph-mon[35851]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:38.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:37 smithi007 ceph-mon[32462]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:38.525 DEBUG:teuthology.orchestra.run.smithi044:osd.5> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.5.service 2024-03-26T08:36:38.527 INFO:tasks.cephadm:Deploying osd.6 on smithi007 with /dev/vg_nvme/lv_2... 2024-03-26T08:36:38.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T08:36:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:36:39.216 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.217 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.217 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.217 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.217 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:38 smithi007 ceph-mon[32462]: from='osd.5 [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:36:39.217 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:38 smithi007 ceph-mon[32462]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:36:39.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:38 smithi016 ceph-mon[35851]: from='osd.5 [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:36:39.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:38 smithi016 ceph-mon[35851]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:36:39.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:39.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:38 smithi044 ceph-mon[36461]: from='osd.5 [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:36:39.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:38 smithi044 ceph-mon[36461]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T08:36:40.159 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:39 smithi007 ceph-mon[32462]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:40.159 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:39 smithi007 ceph-mon[32462]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T08:36:40.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:39 smithi007 ceph-mon[32462]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T08:36:40.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:40.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:39 smithi007 ceph-mon[32462]: from='osd.5 [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:36:40.160 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:39 smithi007 ceph-mon[32462]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:36:40.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:39 smithi016 ceph-mon[35851]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:40.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:39 smithi016 ceph-mon[35851]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T08:36:40.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:39 smithi016 ceph-mon[35851]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T08:36:40.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:40.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:39 smithi016 ceph-mon[35851]: from='osd.5 [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:36:40.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:39 smithi016 ceph-mon[35851]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:36:40.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:39 smithi044 ceph-mon[36461]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:40.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:39 smithi044 ceph-mon[36461]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T08:36:40.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:39 smithi044 ceph-mon[36461]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T08:36:40.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:40.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:39 smithi044 ceph-mon[36461]: from='osd.5 [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:36:40.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:39 smithi044 ceph-mon[36461]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:36:40.317 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:36:39 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:36:39.972+0000 7faf5e052700 -1 osd.5 0 waiting for initial osdmap 2024-03-26T08:36:40.317 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:36:39 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:36:39.979+0000 7faf59e45700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:36:40.854 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T08:36:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T08:36:40.855 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-26T08:36:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-26T08:36:40.855 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00849929 s, 1.2 GB/s 2024-03-26T08:36:40.855 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-26T08:36:41.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:40 smithi044 ceph-mon[36461]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:36:41.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:40 smithi044 ceph-mon[36461]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T08:36:41.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:41.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:41.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:40 smithi044 ceph-mon[36461]: osd.5 [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] boot 2024-03-26T08:36:41.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:40 smithi044 ceph-mon[36461]: osdmap e38: 6 total, 6 up, 6 in 2024-03-26T08:36:41.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:41.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:40 smithi016 ceph-mon[35851]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:36:41.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:40 smithi016 ceph-mon[35851]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T08:36:41.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:41.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:41.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:40 smithi016 ceph-mon[35851]: osd.5 [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] boot 2024-03-26T08:36:41.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:40 smithi016 ceph-mon[35851]: osdmap e38: 6 total, 6 up, 6 in 2024-03-26T08:36:41.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:41.244 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:40 smithi007 ceph-mon[32462]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:36:41.244 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:40 smithi007 ceph-mon[32462]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T08:36:41.244 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:41.244 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:41.244 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:40 smithi007 ceph-mon[32462]: osd.5 [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] boot 2024-03-26T08:36:41.244 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:40 smithi007 ceph-mon[32462]: osdmap e38: 6 total, 6 up, 6 in 2024-03-26T08:36:41.244 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T08:36:41.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-03-26T08:36:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:36:42.148 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.149 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:42.150 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:42.150 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:42.150 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:36:42.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:36:42.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:36:42.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:36:42.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:36:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:36:43.231 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:42 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi044 2024-03-26T08:36:43.231 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:42 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi044 to 8542M 2024-03-26T08:36:43.232 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:42 smithi007 ceph-mon[32462]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T08:36:43.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:42 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi044 2024-03-26T08:36:43.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:42 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi044 to 8542M 2024-03-26T08:36:43.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:42 smithi016 ceph-mon[35851]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T08:36:43.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:42 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi044 2024-03-26T08:36:43.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:42 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi044 to 8542M 2024-03-26T08:36:43.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:42 smithi044 ceph-mon[36461]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T08:36:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.344+0000 7f2af85af700 1 -- 172.21.15.7:0/3699998944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 msgr2=0x7f2af0109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.344+0000 7f2af85af700 1 --2- 172.21.15.7:0/3699998944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 0x7f2af0109510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800b0d0 tx=0x7f2ae800b3e0 comp rx=0 tx=0).stop 2024-03-26T08:36:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.344+0000 7f2af85af700 1 -- 172.21.15.7:0/3699998944 shutdown_connections 2024-03-26T08:36:43.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.344+0000 7f2af85af700 1 --2- 172.21.15.7:0/3699998944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af01010c0 0x7f2af01014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.344+0000 7f2af85af700 1 --2- 172.21.15.7:0/3699998944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 0x7f2af0109510 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.344+0000 7f2af85af700 1 --2- 172.21.15.7:0/3699998944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2af01022c0 0x7f2af0102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.344+0000 7f2af85af700 1 -- 172.21.15.7:0/3699998944 >> 172.21.15.7:0/3699998944 conn(0x7f2af00fc680 msgr2=0x7f2af00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.344+0000 7f2af85af700 1 -- 172.21.15.7:0/3699998944 shutdown_connections 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.344+0000 7f2af85af700 1 -- 172.21.15.7:0/3699998944 wait complete. 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af85af700 1 Processor -- start 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af85af700 1 -- start start 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af85af700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af01010c0 0x7f2af0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af85af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2af01022c0 0x7f2af0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af85af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 0x7f2af019f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af85af700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af010cce0 con 0x7f2af0102c60 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af85af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af010cb60 con 0x7f2af01022c0 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af85af700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af010ce60 con 0x7f2af01010c0 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af6b4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 0x7f2af019f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:36:43.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af6b4c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 0x7f2af019f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37556/0 (socket says 172.21.15.7:37556) 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af6b4c700 1 -- 172.21.15.7:0/1961007868 learned_addr learned my addr 172.21.15.7:0/1961007868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.345+0000 7f2af5b4a700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2af01022c0 0x7f2af0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af634b700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af01010c0 0x7f2af0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af6b4c700 1 -- 172.21.15.7:0/1961007868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af01010c0 msgr2=0x7f2af0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af6b4c700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af01010c0 0x7f2af0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af6b4c700 1 -- 172.21.15.7:0/1961007868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2af01022c0 msgr2=0x7f2af0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af6b4c700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2af01022c0 0x7f2af0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af6b4c700 1 -- 172.21.15.7:0/1961007868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af01a0070 con 0x7f2af0102c60 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af5b4a700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2af01022c0 0x7f2af0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af6b4c700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 0x7f2af019f9d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8004da0 tx=0x7f2ae8004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2ae77fe700 1 -- 172.21.15.7:0/1961007868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8021020 con 0x7f2af0102c60 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2ae77fe700 1 -- 172.21.15.7:0/1961007868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8007c80 con 0x7f2af0102c60 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af01a0300 con 0x7f2af0102c60 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2ae77fe700 1 -- 172.21.15.7:0/1961007868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8005400 con 0x7f2af0102c60 2024-03-26T08:36:43.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.346+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af01a07b0 con 0x7f2af0102c60 2024-03-26T08:36:43.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.347+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af0068e50 con 0x7f2af0102c60 2024-03-26T08:36:43.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.349+0000 7f2ae77fe700 1 -- 172.21.15.7:0/1961007868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2ae8024020 con 0x7f2af0102c60 2024-03-26T08:36:43.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.349+0000 7f2ae77fe700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2adc0a2890 0x7f2adc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:36:43.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.350+0000 7f2af634b700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2adc0a2890 0x7f2adc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:36:43.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.350+0000 7f2ae77fe700 1 -- 172.21.15.7:0/1961007868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f2ae800f020 con 0x7f2af0102c60 2024-03-26T08:36:43.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.350+0000 7f2af634b700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2adc0a2890 0x7f2adc0a4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005950 tx=0x7f2aec0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:36:43.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.353+0000 7f2ae77fe700 1 -- 172.21.15.7:0/1961007868 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2ae809d2d0 con 0x7f2af0102c60 2024-03-26T08:36:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:36:43.568+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2af00fe780 con 0x7f2adc0a2890 2024-03-26T08:36:44.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:43 smithi016 ceph-mon[35851]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:36:44.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:36:44.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:36:44.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:44.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:43 smithi007 ceph-mon[32462]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:36:44.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:36:44.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:36:44.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:44.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:43 smithi044 ceph-mon[36461]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:36:44.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:36:44.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:36:44.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:45.230 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:44 smithi007 ceph-mon[32462]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:36:45.231 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:44 smithi007 ceph-mon[32462]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T08:36:45.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:44 smithi016 ceph-mon[35851]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:36:45.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:44 smithi016 ceph-mon[35851]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T08:36:45.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:44 smithi044 ceph-mon[36461]: from='client.14379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:36:45.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:44 smithi044 ceph-mon[36461]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T08:36:46.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:45 smithi016 ceph-mon[35851]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:36:46.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:45 smithi016 ceph-mon[35851]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T08:36:46.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:45 smithi007 ceph-mon[32462]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:36:46.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:45 smithi007 ceph-mon[32462]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T08:36:46.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:45 smithi044 ceph-mon[36461]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:36:46.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:45 smithi044 ceph-mon[36461]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T08:36:47.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:46 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4158451784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44d3581c-a611-48f8-92c7-4273e6639781"}]: dispatch 2024-03-26T08:36:47.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:46 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4158451784' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44d3581c-a611-48f8-92c7-4273e6639781"}]': finished 2024-03-26T08:36:47.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:46 smithi016 ceph-mon[35851]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T08:36:47.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:36:47.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:46 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1343898713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:36:47.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:46 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4158451784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44d3581c-a611-48f8-92c7-4273e6639781"}]: dispatch 2024-03-26T08:36:47.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:46 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4158451784' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44d3581c-a611-48f8-92c7-4273e6639781"}]': finished 2024-03-26T08:36:47.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:46 smithi007 ceph-mon[32462]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T08:36:47.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:36:47.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:46 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1343898713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:36:47.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:46 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4158451784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44d3581c-a611-48f8-92c7-4273e6639781"}]: dispatch 2024-03-26T08:36:47.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:46 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4158451784' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44d3581c-a611-48f8-92c7-4273e6639781"}]': finished 2024-03-26T08:36:47.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:46 smithi044 ceph-mon[36461]: osdmap e43: 7 total, 6 up, 7 in 2024-03-26T08:36:47.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:36:47.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:46 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1343898713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:36:48.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:47 smithi016 ceph-mon[35851]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:36:48.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:47 smithi007 ceph-mon[32462]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:36:48.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:47 smithi044 ceph-mon[36461]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:36:50.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:49 smithi016 ceph-mon[35851]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T08:36:50.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:49 smithi007 ceph-mon[32462]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T08:36:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:49 smithi044 ceph-mon[36461]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T08:36:52.106 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:52 smithi007 ceph-mon[32462]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-26T08:36:52.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:52 smithi016 ceph-mon[35851]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-26T08:36:52.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:52 smithi044 ceph-mon[36461]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-26T08:36:54.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:54 smithi007 ceph-mon[32462]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:36:54.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:54 smithi044 ceph-mon[36461]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:36:54.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:54 smithi016 ceph-mon[35851]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:36:55.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T08:36:55.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:55.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:55 smithi007 ceph-mon[32462]: Deploying daemon osd.6 on smithi007 2024-03-26T08:36:55.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T08:36:55.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:55.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:55 smithi044 ceph-mon[36461]: Deploying daemon osd.6 on smithi007 2024-03-26T08:36:55.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T08:36:55.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:36:55.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:55 smithi016 ceph-mon[35851]: Deploying daemon osd.6 on smithi007 2024-03-26T08:36:56.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:56 smithi007 ceph-mon[32462]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T08:36:56.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:56 smithi044 ceph-mon[36461]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T08:36:56.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:56 smithi016 ceph-mon[35851]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-26T08:36:58.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:36:58 smithi007 ceph-mon[32462]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:36:58.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:36:58 smithi044 ceph-mon[36461]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:36:58.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:36:58 smithi016 ceph-mon[35851]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:37:00.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:00 smithi007 ceph-mon[32462]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T08:37:00.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:00.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:00.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:00.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:00 smithi044 ceph-mon[36461]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T08:37:00.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:00.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:00.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:00.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:00 smithi016 ceph-mon[35851]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-26T08:37:00.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:00.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:00.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:02.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:02 smithi007 ceph-mon[32462]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:02.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:02 smithi044 ceph-mon[36461]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:02.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:02 smithi016 ceph-mon[35851]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:03.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.975+0000 7f2ae77fe700 1 -- 172.21.15.7:0/1961007868 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2af00fe780 con 0x7f2adc0a2890 2024-03-26T08:37:03.968 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 6 on host 'smithi007' 2024-03-26T08:37:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2adc0a2890 msgr2=0x7f2adc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2adc0a2890 0x7f2adc0a4d50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005950 tx=0x7f2aec0058e0 comp rx=0 tx=0).stop 2024-03-26T08:37:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 msgr2=0x7f2af019f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 0x7f2af019f9d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8004da0 tx=0x7f2ae8004e80 comp rx=0 tx=0).stop 2024-03-26T08:37:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 shutdown_connections 2024-03-26T08:37:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2adc0a2890 0x7f2adc0a4d50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2af01010c0 0x7f2af0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af0102c60 0x7f2af019f9d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 --2- 172.21.15.7:0/1961007868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2af01022c0 0x7f2af0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 >> 172.21.15.7:0/1961007868 conn(0x7f2af00fc680 msgr2=0x7f2af00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:37:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 shutdown_connections 2024-03-26T08:37:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:37:03.978+0000 7f2af85af700 1 -- 172.21.15.7:0/1961007868 wait complete. 2024-03-26T08:37:04.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:04.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T08:37:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:04.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:03 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:04.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:04.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:04.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T08:37:04.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:04.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:03 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:04.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:04.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:04.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T08:37:04.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:04.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:03 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:04.674 DEBUG:teuthology.orchestra.run.smithi007:osd.6> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.6.service 2024-03-26T08:37:04.676 INFO:tasks.cephadm:Deploying osd.7 on smithi016 with /dev/vg_nvme/lv_2... 2024-03-26T08:37:04.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T08:37:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:37:05.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:05 smithi007 ceph-mon[32462]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T08:37:05.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:05 smithi007 ceph-mon[32462]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T08:37:05.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:05 smithi007 ceph-mon[32462]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:37:05.242 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:05 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:05.243 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:37:05 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:37:05.028+0000 7fc035b3d700 -1 osd.6 0 waiting for initial osdmap 2024-03-26T08:37:05.243 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:37:05 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:37:05.039+0000 7fc031930700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:37:05.315 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:05 smithi016 ceph-mon[35851]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T08:37:05.315 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:05 smithi016 ceph-mon[35851]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T08:37:05.315 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:05 smithi016 ceph-mon[35851]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:37:05.315 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:05 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:05.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:05 smithi044 ceph-mon[36461]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T08:37:05.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:05 smithi044 ceph-mon[36461]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T08:37:05.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:05 smithi044 ceph-mon[36461]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:37:05.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:05 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:06 smithi007 ceph-mon[32462]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:06 smithi007 ceph-mon[32462]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:37:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:06 smithi007 ceph-mon[32462]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T08:37:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:06 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:06 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:06.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:06 smithi044 ceph-mon[36461]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:06.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:06 smithi044 ceph-mon[36461]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:37:06.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:06 smithi044 ceph-mon[36461]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T08:37:06.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:06 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:06.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:06 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:06.367 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:06 smithi016 ceph-mon[35851]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:06.367 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:06 smithi016 ceph-mon[35851]: from='osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:37:06.367 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:06 smithi016 ceph-mon[35851]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T08:37:06.367 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:06 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:06.367 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:06 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T08:37:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T08:37:07.100 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-03-26T08:37:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-03-26T08:37:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00822904 s, 1.3 GB/s 2024-03-26T08:37:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-03-26T08:37:07.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:07 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:37:07.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:07 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:37:07.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:07.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:07 smithi007 ceph-mon[32462]: osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] boot 2024-03-26T08:37:07.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:07 smithi007 ceph-mon[32462]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T08:37:07.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:07.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:07 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:37:07.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:07 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:37:07.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:07.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:07 smithi044 ceph-mon[36461]: osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] boot 2024-03-26T08:37:07.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:07 smithi044 ceph-mon[36461]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T08:37:07.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:07.409 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:07 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:37:07.409 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:07 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:37:07.409 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:07.409 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:07 smithi016 ceph-mon[35851]: osd.6 [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] boot 2024-03-26T08:37:07.409 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:07 smithi016 ceph-mon[35851]: osdmap e46: 7 total, 7 up, 7 in 2024-03-26T08:37:07.409 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T08:37:07.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-03-26T08:37:08.032 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:37:08.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:08 smithi007 ceph-mon[32462]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:08.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:08 smithi007 ceph-mon[32462]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T08:37:08.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:08 smithi044 ceph-mon[36461]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:08.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:08 smithi044 ceph-mon[36461]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T08:37:08.387 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:08 smithi016 ceph-mon[35851]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-03-26T08:37:08.387 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:08 smithi016 ceph-mon[35851]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T08:37:09.480 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi007 2024-03-26T08:37:09.480 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.480 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.480 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:09.480 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:09.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:09.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi007 to 5697M 2024-03-26T08:37:09.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:09.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:09.481 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi007 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi007 to 5697M 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:09.581 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.582 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi007 2024-03-26T08:37:09.582 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.582 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.582 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:09.582 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:09.582 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:09.583 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi007 to 5697M 2024-03-26T08:37:09.583 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.583 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:09.583 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:09.583 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:09.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.706+0000 7fa949ede700 1 -- 172.21.15.16:0/2260528798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944105620 msgr2=0x7fa944107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.706+0000 7fa949ede700 1 --2- 172.21.15.16:0/2260528798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944105620 0x7fa944107a10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009990 tx=0x7fa92c009ca0 comp rx=0 tx=0).stop 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.706+0000 7fa949ede700 1 -- 172.21.15.16:0/2260528798 shutdown_connections 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.706+0000 7fa949ede700 1 --2- 172.21.15.16:0/2260528798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa944102cf0 0x7fa9441050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.706+0000 7fa949ede700 1 --2- 172.21.15.16:0/2260528798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa944107f50 0x7fa94410a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.706+0000 7fa949ede700 1 --2- 172.21.15.16:0/2260528798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944105620 0x7fa944107a10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.706+0000 7fa949ede700 1 -- 172.21.15.16:0/2260528798 >> 172.21.15.16:0/2260528798 conn(0x7fa9440fc660 msgr2=0x7fa9440feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.706+0000 7fa949ede700 1 -- 172.21.15.16:0/2260528798 shutdown_connections 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.706+0000 7fa949ede700 1 -- 172.21.15.16:0/2260528798 wait complete. 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa949ede700 1 Processor -- start 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa949ede700 1 -- start start 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa949ede700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa944102cf0 0x7fa944198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa949ede700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa944105620 0x7fa944198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa949ede700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944107f50 0x7fa94419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:37:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa949ede700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94410ddd0 con 0x7fa944102cf0 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa949ede700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94410dc50 con 0x7fa944107f50 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa949ede700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94410df50 con 0x7fa944105620 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa943fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944107f50 0x7fa94419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa943fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944107f50 0x7fa94419f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55002/0 (socket says 172.21.15.16:55002) 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa943fff700 1 -- 172.21.15.16:0/403486328 learned_addr learned my addr 172.21.15.16:0/403486328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa942ffd700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa944105620 0x7fa944198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.707+0000 7fa9437fe700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa944102cf0 0x7fa944198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa943fff700 1 -- 172.21.15.16:0/403486328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa944105620 msgr2=0x7fa944198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa943fff700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa944105620 0x7fa944198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa943fff700 1 -- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa944102cf0 msgr2=0x7fa944198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa943fff700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa944102cf0 0x7fa944198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa943fff700 1 -- 172.21.15.16:0/403486328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9441a0060 con 0x7fa944107f50 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa943fff700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944107f50 0x7fa94419f9c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00e970 tx=0x7fa93c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:37:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa940ff9700 1 -- 172.21.15.16:0/403486328 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c004a30 con 0x7fa944107f50 2024-03-26T08:37:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9441a0350 con 0x7fa944107f50 2024-03-26T08:37:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa940ff9700 1 -- 172.21.15.16:0/403486328 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa93c005500 con 0x7fa944107f50 2024-03-26T08:37:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa940ff9700 1 -- 172.21.15.16:0/403486328 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00cd20 con 0x7fa944107f50 2024-03-26T08:37:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.708+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9441a0890 con 0x7fa944107f50 2024-03-26T08:37:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.709+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa944068e50 con 0x7fa944107f50 2024-03-26T08:37:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.710+0000 7fa940ff9700 1 -- 172.21.15.16:0/403486328 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa93c004bd0 con 0x7fa944107f50 2024-03-26T08:37:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.711+0000 7fa940ff9700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9300a2890 0x7fa9300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:37:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.711+0000 7fa940ff9700 1 -- 172.21.15.16:0/403486328 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fa93c0ce2a0 con 0x7fa944107f50 2024-03-26T08:37:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.711+0000 7fa9437fe700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9300a2890 0x7fa9300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:37:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.712+0000 7fa9437fe700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9300a2890 0x7fa9300a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa934003f90 tx=0x7fa934009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:37:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.715+0000 7fa940ff9700 1 -- 172.21.15.16:0/403486328 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa93c053740 con 0x7fa944107f50 2024-03-26T08:37:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:09.931+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9440625d0 con 0x7fa9300a2890 2024-03-26T08:37:10.467 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:10 smithi044 ceph-mon[36461]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:10.468 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:37:10.468 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:37:10.468 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:10 smithi016 ceph-mon[35851]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:37:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:37:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:10.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:10 smithi007 ceph-mon[32462]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:10.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:37:10.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:37:10.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:11.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:11 smithi016 ceph-mon[35851]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:37:11.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:11 smithi007 ceph-mon[32462]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:37:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:11 smithi044 ceph-mon[36461]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:37:12.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:12 smithi016 ceph-mon[35851]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:12.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:12 smithi007 ceph-mon[32462]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:12.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:12 smithi044 ceph-mon[36461]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:13 smithi016 ceph-mon[35851]: from='client.? 172.21.15.16:0/3934194904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e87d365c-1fcf-40cd-93f6-5fc972c25217"}]: dispatch 2024-03-26T08:37:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:13 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e87d365c-1fcf-40cd-93f6-5fc972c25217"}]: dispatch 2024-03-26T08:37:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:13 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e87d365c-1fcf-40cd-93f6-5fc972c25217"}]': finished 2024-03-26T08:37:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:13 smithi016 ceph-mon[35851]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T08:37:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:13 smithi007 ceph-mon[32462]: from='client.? 172.21.15.16:0/3934194904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e87d365c-1fcf-40cd-93f6-5fc972c25217"}]: dispatch 2024-03-26T08:37:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:13 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e87d365c-1fcf-40cd-93f6-5fc972c25217"}]: dispatch 2024-03-26T08:37:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:13 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e87d365c-1fcf-40cd-93f6-5fc972c25217"}]': finished 2024-03-26T08:37:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:13 smithi007 ceph-mon[32462]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T08:37:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:13.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:13 smithi044 ceph-mon[36461]: from='client.? 172.21.15.16:0/3934194904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e87d365c-1fcf-40cd-93f6-5fc972c25217"}]: dispatch 2024-03-26T08:37:13.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:13 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e87d365c-1fcf-40cd-93f6-5fc972c25217"}]: dispatch 2024-03-26T08:37:13.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:13 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e87d365c-1fcf-40cd-93f6-5fc972c25217"}]': finished 2024-03-26T08:37:13.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:13 smithi044 ceph-mon[36461]: osdmap e48: 8 total, 7 up, 8 in 2024-03-26T08:37:13.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:14.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:14 smithi016 ceph-mon[35851]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:14.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:14 smithi016 ceph-mon[35851]: from='client.? 172.21.15.16:0/2945817730' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:37:14.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:14 smithi007 ceph-mon[32462]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:14 smithi007 ceph-mon[32462]: from='client.? 172.21.15.16:0/2945817730' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:37:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:14 smithi044 ceph-mon[36461]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:14 smithi044 ceph-mon[36461]: from='client.? 172.21.15.16:0/2945817730' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:37:16.467 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:16 smithi016 ceph-mon[35851]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:16.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:16 smithi007 ceph-mon[32462]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:16 smithi044 ceph-mon[36461]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:18.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:18 smithi016 ceph-mon[35851]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:18.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:18 smithi007 ceph-mon[32462]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:18 smithi044 ceph-mon[36461]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:20.325 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:20 smithi016 ceph-mon[35851]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:20.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:20 smithi007 ceph-mon[32462]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:20 smithi044 ceph-mon[36461]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:21.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:21 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T08:37:21.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:21 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:21 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T08:37:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:21 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:21.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:21 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T08:37:21.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:21 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:22 smithi016 ceph-mon[35851]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:22 smithi016 ceph-mon[35851]: Deploying daemon osd.7 on smithi016 2024-03-26T08:37:22.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:22 smithi007 ceph-mon[32462]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:22.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:22 smithi007 ceph-mon[32462]: Deploying daemon osd.7 on smithi016 2024-03-26T08:37:22.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:22 smithi044 ceph-mon[36461]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:22.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:22 smithi044 ceph-mon[36461]: Deploying daemon osd.7 on smithi016 2024-03-26T08:37:24.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:24 smithi016 ceph-mon[35851]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:24.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:24 smithi007 ceph-mon[32462]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:24 smithi044 ceph-mon[36461]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:26.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:26 smithi016 ceph-mon[35851]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:26.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:26.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:26.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:26 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:26.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:26 smithi007 ceph-mon[32462]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:26.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:26.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:26.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:26 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:26 smithi044 ceph-mon[36461]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:26 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:28.288 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:28 smithi016 ceph-mon[35851]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:28 smithi007 ceph-mon[32462]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:28 smithi044 ceph-mon[36461]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:29.638 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 7 on host 'smithi016' 2024-03-26T08:37:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.646+0000 7fa940ff9700 1 -- 172.21.15.16:0/403486328 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa9440625d0 con 0x7fa9300a2890 2024-03-26T08:37:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9300a2890 msgr2=0x7fa9300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9300a2890 0x7fa9300a4d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa934003f90 tx=0x7fa934009180 comp rx=0 tx=0).stop 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944107f50 msgr2=0x7fa94419f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944107f50 0x7fa94419f9c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00e970 tx=0x7fa93c00ec80 comp rx=0 tx=0).stop 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 shutdown_connections 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9300a2890 0x7fa9300a4d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa944107f50 0x7fa94419f9c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa944102cf0 0x7fa944198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 --2- 172.21.15.16:0/403486328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa944105620 0x7fa944198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.648+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 >> 172.21.15.16:0/403486328 conn(0x7fa9440fc660 msgr2=0x7fa944103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.649+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 shutdown_connections 2024-03-26T08:37:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:37:29.649+0000 7fa949ede700 1 -- 172.21.15.16:0/403486328 wait complete. 2024-03-26T08:37:30.278 DEBUG:teuthology.orchestra.run.smithi016:osd.7> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.7.service 2024-03-26T08:37:30.281 INFO:tasks.cephadm:Deploying osd.8 on smithi044 with /dev/vg_nvme/lv_2... 2024-03-26T08:37:30.281 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T08:37:30.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:30 smithi016 ceph-mon[35851]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:30.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:30 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:30.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:30 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:30.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:30 smithi016 ceph-mon[35851]: from='osd.7 [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:37:30.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:30 smithi016 ceph-mon[35851]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:37:30.542 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:37:30.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:30 smithi007 ceph-mon[32462]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:30.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:30 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:30.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:30 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:30.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:30 smithi007 ceph-mon[32462]: from='osd.7 [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:37:30.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:30 smithi007 ceph-mon[32462]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:37:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:30 smithi044 ceph-mon[36461]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:30 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:30 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:30 smithi044 ceph-mon[36461]: from='osd.7 [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:37:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:30 smithi044 ceph-mon[36461]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T08:37:31.739 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:31 smithi044 ceph-mon[36461]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:31.740 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:31.740 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:31.740 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:31 smithi044 ceph-mon[36461]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T08:37:31.740 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:31 smithi044 ceph-mon[36461]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T08:37:31.740 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:31 smithi044 ceph-mon[36461]: from='osd.7 [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:37:31.740 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:31.740 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:31 smithi044 ceph-mon[36461]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:37:31.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:31 smithi016 ceph-mon[35851]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:31.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:31.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:31.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:31 smithi016 ceph-mon[35851]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T08:37:31.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:31 smithi016 ceph-mon[35851]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T08:37:31.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:31 smithi016 ceph-mon[35851]: from='osd.7 [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:37:31.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:31.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:31 smithi016 ceph-mon[35851]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:37:31.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:31 smithi007 ceph-mon[32462]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:31.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:31.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:31.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:31 smithi007 ceph-mon[32462]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T08:37:31.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:31 smithi007 ceph-mon[32462]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T08:37:31.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:31 smithi007 ceph-mon[32462]: from='osd.7 [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:37:31.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:31.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:31 smithi007 ceph-mon[32462]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:37:32.399 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:37:32 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:37:32.128+0000 7f24bc859700 -1 osd.7 0 waiting for initial osdmap 2024-03-26T08:37:32.399 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:37:32 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:37:32.139+0000 7f24b3e43700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:37:32.670 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T08:37:32.670 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T08:37:32.670 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-03-26T08:37:32.670 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-03-26T08:37:32.670 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111232 s, 943 MB/s 2024-03-26T08:37:32.670 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-03-26T08:37:32.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:32 smithi016 ceph-mon[35851]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:37:32.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:32 smithi016 ceph-mon[35851]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T08:37:32.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:32.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:32.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:32 smithi016 ceph-mon[35851]: from='osd.7 ' entity='osd.7' 2024-03-26T08:37:33.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:32 smithi007 ceph-mon[32462]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:37:33.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:32 smithi007 ceph-mon[32462]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T08:37:33.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:33.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:33.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:32 smithi007 ceph-mon[32462]: from='osd.7 ' entity='osd.7' 2024-03-26T08:37:33.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:32 smithi044 ceph-mon[36461]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:37:33.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:32 smithi044 ceph-mon[36461]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T08:37:33.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:33.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:33.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:32 smithi044 ceph-mon[36461]: from='osd.7 ' entity='osd.7' 2024-03-26T08:37:33.449 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi044:vg_nvme/lv_2 2024-03-26T08:37:33.700 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:37:33.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:33 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:37:33.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:33 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:37:33.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:33 smithi044 ceph-mon[36461]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:33.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:33 smithi044 ceph-mon[36461]: osd.7 [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] boot 2024-03-26T08:37:33.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:33 smithi044 ceph-mon[36461]: osdmap e51: 8 total, 8 up, 8 in 2024-03-26T08:37:33.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:33.907 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:33 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:37:33.907 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:33 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:37:33.907 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:33 smithi016 ceph-mon[35851]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:33.907 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:33 smithi016 ceph-mon[35851]: osd.7 [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] boot 2024-03-26T08:37:33.907 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:33 smithi016 ceph-mon[35851]: osdmap e51: 8 total, 8 up, 8 in 2024-03-26T08:37:33.907 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:34.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:33 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:37:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:33 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:37:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:33 smithi007 ceph-mon[32462]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 192 MiB used, 626 GiB / 626 GiB avail 2024-03-26T08:37:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:33 smithi007 ceph-mon[32462]: osd.7 [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] boot 2024-03-26T08:37:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:33 smithi007 ceph-mon[32462]: osdmap e51: 8 total, 8 up, 8 in 2024-03-26T08:37:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T08:37:34.804 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi016 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi016 to 5694M 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:34.805 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:34.806 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:34.806 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:34.806 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:34.806 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:34.806 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:34.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T08:37:34.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi016 2024-03-26T08:37:34.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi016 to 5694M 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:34.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:35.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T08:37:35.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi016 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi016 to 5694M 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:37:35.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.506+0000 7fd46223d700 1 -- 172.21.15.44:0/2192526438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c107f50 msgr2=0x7fd45c10a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.506+0000 7fd46223d700 1 --2- 172.21.15.44:0/2192526438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c107f50 0x7fd45c10a340 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b0a0 tx=0x7fd45400b3b0 comp rx=0 tx=0).stop 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.507+0000 7fd46223d700 1 -- 172.21.15.44:0/2192526438 shutdown_connections 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.507+0000 7fd46223d700 1 --2- 172.21.15.44:0/2192526438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c107f50 0x7fd45c10a340 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.507+0000 7fd46223d700 1 --2- 172.21.15.44:0/2192526438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd45c102cf0 0x7fd45c1050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.507+0000 7fd46223d700 1 --2- 172.21.15.44:0/2192526438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c105620 0x7fd45c107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.507+0000 7fd46223d700 1 -- 172.21.15.44:0/2192526438 >> 172.21.15.44:0/2192526438 conn(0x7fd45c0fc680 msgr2=0x7fd45c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.507+0000 7fd46223d700 1 -- 172.21.15.44:0/2192526438 shutdown_connections 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.507+0000 7fd46223d700 1 -- 172.21.15.44:0/2192526438 wait complete. 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.507+0000 7fd46223d700 1 Processor -- start 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.507+0000 7fd46223d700 1 -- start start 2024-03-26T08:37:35.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd46223d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c102cf0 0x7fd45c102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd46223d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c105620 0x7fd45c100ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd46223d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd45c107f50 0x7fd45c1011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd46223d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c10ddc0 con 0x7fd45c107f50 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd46223d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c10dc40 con 0x7fd45c105620 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd46223d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c10df40 con 0x7fd45c102cf0 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c102cf0 0x7fd45c102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45b7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c102cf0 0x7fd45c102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:51148/0 (socket says 172.21.15.44:51148) 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45b7fe700 1 -- 172.21.15.44:0/632348042 learned_addr learned my addr 172.21.15.44:0/632348042 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45affd700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c105620 0x7fd45c100ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45bfff700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd45c107f50 0x7fd45c1011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45b7fe700 1 -- 172.21.15.44:0/632348042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c105620 msgr2=0x7fd45c100ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45b7fe700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c105620 0x7fd45c100ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:35.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45b7fe700 1 -- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd45c107f50 msgr2=0x7fd45c1011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45b7fe700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd45c107f50 0x7fd45c1011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45b7fe700 1 -- 172.21.15.44:0/632348042 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c101a90 con 0x7fd45c102cf0 2024-03-26T08:37:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd45b7fe700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c102cf0 0x7fd45c102650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00d980 tx=0x7fd44c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:37:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd458ff9700 1 -- 172.21.15.44:0/632348042 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c016070 con 0x7fd45c102cf0 2024-03-26T08:37:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd458ff9700 1 -- 172.21.15.44:0/632348042 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd44c004d10 con 0x7fd45c102cf0 2024-03-26T08:37:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c1a2530 con 0x7fd45c102cf0 2024-03-26T08:37:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd458ff9700 1 -- 172.21.15.44:0/632348042 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0055f0 con 0x7fd45c102cf0 2024-03-26T08:37:35.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.508+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c1a2a70 con 0x7fd45c102cf0 2024-03-26T08:37:35.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.509+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c068e50 con 0x7fd45c102cf0 2024-03-26T08:37:35.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.511+0000 7fd458ff9700 1 -- 172.21.15.44:0/632348042 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd44c004750 con 0x7fd45c102cf0 2024-03-26T08:37:35.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.511+0000 7fd458ff9700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4440a2910 0x7fd4440a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:37:35.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.511+0000 7fd458ff9700 1 -- 172.21.15.44:0/632348042 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7fd44c0ce280 con 0x7fd45c102cf0 2024-03-26T08:37:35.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.512+0000 7fd45affd700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4440a2910 0x7fd4440a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:37:35.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.512+0000 7fd45affd700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4440a2910 0x7fd4440a4dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd450005950 tx=0x7fd45000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:37:35.507 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.515+0000 7fd458ff9700 1 -- 172.21.15.44:0/632348042 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd44c01f020 con 0x7fd45c102cf0 2024-03-26T08:37:35.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:35.732+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd45c0625d0 con 0x7fd4440a2910 2024-03-26T08:37:35.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:35 smithi016 ceph-mon[35851]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:35.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:35 smithi016 ceph-mon[35851]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T08:37:35.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:35 smithi016 ceph-mon[35851]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T08:37:36.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:35 smithi007 ceph-mon[32462]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:36.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:35 smithi007 ceph-mon[32462]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T08:37:36.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:35 smithi007 ceph-mon[32462]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T08:37:36.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:35 smithi044 ceph-mon[36461]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:36.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:35 smithi044 ceph-mon[36461]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T08:37:36.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:35 smithi044 ceph-mon[36461]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T08:37:36.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:36 smithi016 ceph-mon[35851]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:37:36.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:36 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:37:36.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:36 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:37:36.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:36 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:36.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:36 smithi016 ceph-mon[35851]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T08:37:37.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:36 smithi007 ceph-mon[32462]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:37:37.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:36 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:37:37.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:36 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:37:37.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:36 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:37.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:36 smithi007 ceph-mon[32462]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T08:37:37.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:36 smithi044 ceph-mon[36461]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:37:37.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:36 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:37:37.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:36 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:37:37.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:36 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:37.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:36 smithi044 ceph-mon[36461]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T08:37:37.710 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:37 smithi044 ceph-mon[36461]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:37.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:37 smithi016 ceph-mon[35851]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:38.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:37 smithi007 ceph-mon[32462]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:38.940 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:38 smithi044 ceph-mon[36461]: from='client.? 172.21.15.44:0/2022456312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f51bbc-b176-435d-bf69-4269bb0a6102"}]: dispatch 2024-03-26T08:37:38.940 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:38 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f51bbc-b176-435d-bf69-4269bb0a6102"}]: dispatch 2024-03-26T08:37:38.940 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:38 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59f51bbc-b176-435d-bf69-4269bb0a6102"}]': finished 2024-03-26T08:37:38.940 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:38 smithi044 ceph-mon[36461]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T08:37:38.940 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:38.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:38 smithi016 ceph-mon[35851]: from='client.? 172.21.15.44:0/2022456312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f51bbc-b176-435d-bf69-4269bb0a6102"}]: dispatch 2024-03-26T08:37:38.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:38 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f51bbc-b176-435d-bf69-4269bb0a6102"}]: dispatch 2024-03-26T08:37:38.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:38 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59f51bbc-b176-435d-bf69-4269bb0a6102"}]': finished 2024-03-26T08:37:38.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:38 smithi016 ceph-mon[35851]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T08:37:38.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:39.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:38 smithi007 ceph-mon[32462]: from='client.? 172.21.15.44:0/2022456312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f51bbc-b176-435d-bf69-4269bb0a6102"}]: dispatch 2024-03-26T08:37:39.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:38 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59f51bbc-b176-435d-bf69-4269bb0a6102"}]: dispatch 2024-03-26T08:37:39.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:38 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59f51bbc-b176-435d-bf69-4269bb0a6102"}]': finished 2024-03-26T08:37:39.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:38 smithi007 ceph-mon[32462]: osdmap e56: 9 total, 8 up, 9 in 2024-03-26T08:37:39.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:39.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:39 smithi016 ceph-mon[35851]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:39.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:39 smithi016 ceph-mon[35851]: from='client.? 172.21.15.44:0/3658818634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:37:40.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:39 smithi007 ceph-mon[32462]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:40.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:39 smithi007 ceph-mon[32462]: from='client.? 172.21.15.44:0/3658818634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:37:40.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:39 smithi044 ceph-mon[36461]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:40.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:39 smithi044 ceph-mon[36461]: from='client.? 172.21.15.44:0/3658818634' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:37:41.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:41 smithi016 ceph-mon[35851]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:42.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:41 smithi007 ceph-mon[32462]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:42.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:41 smithi044 ceph-mon[36461]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:43.853 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:43 smithi044 ceph-mon[36461]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:43 smithi016 ceph-mon[35851]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:44.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:43 smithi007 ceph-mon[32462]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:45.931 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:45 smithi044 ceph-mon[36461]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:45.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:45 smithi016 ceph-mon[35851]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:46.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:45 smithi007 ceph-mon[32462]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:46.876 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T08:37:46.876 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:46.876 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:46 smithi044 ceph-mon[36461]: Deploying daemon osd.8 on smithi044 2024-03-26T08:37:46.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T08:37:46.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:46.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:46 smithi016 ceph-mon[35851]: Deploying daemon osd.8 on smithi044 2024-03-26T08:37:47.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T08:37:47.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:37:47.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:46 smithi007 ceph-mon[32462]: Deploying daemon osd.8 on smithi044 2024-03-26T08:37:47.789 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:47 smithi044 ceph-mon[36461]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:47.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:47 smithi016 ceph-mon[35851]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:48.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:47 smithi007 ceph-mon[32462]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:49 smithi044 ceph-mon[36461]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:49 smithi016 ceph-mon[35851]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:50.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:49 smithi007 ceph-mon[32462]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:51.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:51 smithi044 ceph-mon[36461]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:51.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:51 smithi016 ceph-mon[35851]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:52.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:51 smithi007 ceph-mon[32462]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:52.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:52.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:52.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:53.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:53.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:53.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:53.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:53.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:53.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:37:53.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:53 smithi016 ceph-mon[35851]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:54.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:53 smithi007 ceph-mon[32462]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:54.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:53 smithi044 ceph-mon[36461]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:54.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.855+0000 7fd458ff9700 1 -- 172.21.15.44:0/632348042 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd45c0625d0 con 0x7fd4440a2910 2024-03-26T08:37:54.848 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 8 on host 'smithi044' 2024-03-26T08:37:54.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4440a2910 msgr2=0x7fd4440a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4440a2910 0x7fd4440a4dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd450005950 tx=0x7fd45000b410 comp rx=0 tx=0).stop 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c102cf0 msgr2=0x7fd45c102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c102cf0 0x7fd45c102650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00d980 tx=0x7fd44c00dc90 comp rx=0 tx=0).stop 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 shutdown_connections 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4440a2910 0x7fd4440a4dd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd45c102cf0 0x7fd45c102650 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd45c107f50 0x7fd45c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 --2- 172.21.15.44:0/632348042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45c105620 0x7fd45c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 >> 172.21.15.44:0/632348042 conn(0x7fd45c0fc680 msgr2=0x7fd45c103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 shutdown_connections 2024-03-26T08:37:54.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:37:54.858+0000 7fd46223d700 1 -- 172.21.15.44:0/632348042 wait complete. 2024-03-26T08:37:55.702 DEBUG:teuthology.orchestra.run.smithi044:osd.8> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.8.service 2024-03-26T08:37:55.706 INFO:tasks.cephadm:Deploying osd.9 on smithi007 with /dev/vg_nvme/lv_1... 2024-03-26T08:37:55.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T08:37:55.962 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:37:56.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:55 smithi007 ceph-mon[32462]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:56.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:56.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:56.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:55 smithi007 ceph-mon[32462]: from='osd.8 [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:37:56.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:55 smithi007 ceph-mon[32462]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:37:56.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:55 smithi044 ceph-mon[36461]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:56.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:56.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:56.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:55 smithi044 ceph-mon[36461]: from='osd.8 [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:37:56.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:55 smithi044 ceph-mon[36461]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:37:56.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:55 smithi016 ceph-mon[35851]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:56.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:56.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:56.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:55 smithi016 ceph-mon[35851]: from='osd.8 [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:37:56.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:55 smithi016 ceph-mon[35851]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T08:37:57.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:56 smithi044 ceph-mon[36461]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T08:37:57.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:56 smithi044 ceph-mon[36461]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T08:37:57.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:56 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:57.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:56 smithi044 ceph-mon[36461]: from='osd.8 [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:37:57.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:56 smithi044 ceph-mon[36461]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:37:57.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:56 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:57.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:56 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:57.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:56 smithi007 ceph-mon[32462]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T08:37:57.073 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:56 smithi007 ceph-mon[32462]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T08:37:57.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:56 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:57.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:56 smithi007 ceph-mon[32462]: from='osd.8 [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:37:57.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:56 smithi007 ceph-mon[32462]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:37:57.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:56 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:57.074 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:56 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:57.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:56 smithi016 ceph-mon[35851]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T08:37:57.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:56 smithi016 ceph-mon[35851]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T08:37:57.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:56 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:57.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:56 smithi016 ceph-mon[35851]: from='osd.8 [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:37:57.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:56 smithi016 ceph-mon[35851]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:37:57.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:56 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:57.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:56 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:37:57.547 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:37:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:37:57.246+0000 7f667b919700 -1 osd.8 0 waiting for initial osdmap 2024-03-26T08:37:57.548 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:37:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:37:57.257+0000 7f6672f03700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:37:58.084 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:57 smithi044 ceph-mon[36461]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:58.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:57 smithi044 ceph-mon[36461]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:37:58.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:57 smithi044 ceph-mon[36461]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T08:37:58.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:58.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:57 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:58.085 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:57 smithi044 ceph-mon[36461]: from='osd.8 ' entity='osd.8' 2024-03-26T08:37:58.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:57 smithi016 ceph-mon[35851]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:58.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:57 smithi016 ceph-mon[35851]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:37:58.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:57 smithi016 ceph-mon[35851]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T08:37:58.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:57 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:58.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:57 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:58.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:57 smithi016 ceph-mon[35851]: from='osd.8 ' entity='osd.8' 2024-03-26T08:37:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T08:37:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T08:37:58.254 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-03-26T08:37:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-03-26T08:37:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00827098 s, 1.3 GB/s 2024-03-26T08:37:58.254 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 2024-03-26T08:37:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-03-26T08:37:58.254 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:57 smithi007 ceph-mon[32462]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:37:58.254 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:57 smithi007 ceph-mon[32462]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:37:58.255 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:57 smithi007 ceph-mon[32462]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T08:37:58.255 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:58.255 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:57 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:58.255 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:57 smithi007 ceph-mon[32462]: from='osd.8 ' entity='osd.8' 2024-03-26T08:37:58.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi007:vg_nvme/lv_1 2024-03-26T08:37:59.046 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:58 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:37:59.046 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:58 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:37:59.046 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:59.046 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:58 smithi044 ceph-mon[36461]: osd.8 [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] boot 2024-03-26T08:37:59.046 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:58 smithi044 ceph-mon[36461]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T08:37:59.046 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:59.205 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:58 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:58 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:58 smithi016 ceph-mon[35851]: osd.8 [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] boot 2024-03-26T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:58 smithi016 ceph-mon[35851]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T08:37:59.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:59.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:58 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:37:59.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:58 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:37:59.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:37:59.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:58 smithi007 ceph-mon[32462]: osd.8 [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] boot 2024-03-26T08:37:59.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:58 smithi007 ceph-mon[32462]: osdmap e59: 9 total, 9 up, 9 in 2024-03-26T08:37:59.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:00.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:00.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:00.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:00.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:37:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:38:00.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T08:38:00.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:00.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:37:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-03-26T08:38:00.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T08:38:00.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:00.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:37:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:00.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:00 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi044 2024-03-26T08:38:00.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:00 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi044 to 5694M 2024-03-26T08:38:00.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:00 smithi007 ceph-mon[32462]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T08:38:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.988+0000 7f2b932b2700 1 -- 172.21.15.7:0/4196465621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c102c50 msgr2=0x7f2b8c109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.988+0000 7f2b932b2700 1 --2- 172.21.15.7:0/4196465621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c102c50 0x7f2b8c109500 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400b0d0 tx=0x7f2b8400b3e0 comp rx=0 tx=0).stop 2024-03-26T08:38:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 -- 172.21.15.7:0/4196465621 shutdown_connections 2024-03-26T08:38:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 --2- 172.21.15.7:0/4196465621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b8c1010b0 0x7f2b8c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 --2- 172.21.15.7:0/4196465621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c102c50 0x7f2b8c109500 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 --2- 172.21.15.7:0/4196465621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b8c1022b0 0x7f2b8c102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 -- 172.21.15.7:0/4196465621 >> 172.21.15.7:0/4196465621 conn(0x7f2b8c0fc690 msgr2=0x7f2b8c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 -- 172.21.15.7:0/4196465621 shutdown_connections 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 -- 172.21.15.7:0/4196465621 wait complete. 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 Processor -- start 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 -- start start 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b8c1010b0 0x7f2b8c105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c1022b0 0x7f2b8c103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b8c102c50 0x7f2b8c103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c10cc90 con 0x7f2b8c1022b0 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c10cb10 con 0x7f2b8c102c50 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b932b2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c10ce10 con 0x7f2b8c1010b0 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.989+0000 7f2b9084d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c1022b0 0x7f2b8c103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9084d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c1022b0 0x7f2b8c103690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40024/0 (socket says 172.21.15.7:40024) 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9084d700 1 -- 172.21.15.7:0/1221534511 learned_addr learned my addr 172.21.15.7:0/1221534511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9104e700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b8c1010b0 0x7f2b8c105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9184f700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b8c102c50 0x7f2b8c103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9084d700 1 -- 172.21.15.7:0/1221534511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b8c1010b0 msgr2=0x7f2b8c105040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9084d700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b8c1010b0 0x7f2b8c105040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9084d700 1 -- 172.21.15.7:0/1221534511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b8c102c50 msgr2=0x7f2b8c103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9084d700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b8c102c50 0x7f2b8c103bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9084d700 1 -- 172.21.15.7:0/1221534511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c104480 con 0x7f2b8c1022b0 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b9084d700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c1022b0 0x7f2b8c103690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00d950 tx=0x7f2b7c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:38:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b827fc700 1 -- 172.21.15.7:0/1221534511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c016070 con 0x7f2b8c1022b0 2024-03-26T08:38:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b827fc700 1 -- 172.21.15.7:0/1221534511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b7c00fd70 con 0x7f2b8c1022b0 2024-03-26T08:38:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c1a2550 con 0x7f2b8c1022b0 2024-03-26T08:38:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b827fc700 1 -- 172.21.15.7:0/1221534511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00e610 con 0x7f2b8c1022b0 2024-03-26T08:38:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.990+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c1a2a60 con 0x7f2b8c1022b0 2024-03-26T08:38:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.991+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c068e50 con 0x7f2b8c1022b0 2024-03-26T08:38:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.993+0000 7f2b827fc700 1 -- 172.21.15.7:0/1221534511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2b7c019070 con 0x7f2b8c1022b0 2024-03-26T08:38:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.994+0000 7f2b827fc700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2b780a2890 0x7f2b780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.994+0000 7f2b9104e700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2b780a2890 0x7f2b780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.994+0000 7f2b827fc700 1 -- 172.21.15.7:0/1221534511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f2b7c0cdb20 con 0x7f2b8c1022b0 2024-03-26T08:38:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.994+0000 7f2b9104e700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2b780a2890 0x7f2b780a4d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c102110 tx=0x7f2b8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:38:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:00.997+0000 7f2b827fc700 1 -- 172.21.15.7:0/1221534511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2b7c0cde20 con 0x7f2b8c1022b0 2024-03-26T08:38:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:01.211+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b8c0fe770 con 0x7f2b780a2890 2024-03-26T08:38:01.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:00 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi044 2024-03-26T08:38:01.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:00 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi044 to 5694M 2024-03-26T08:38:01.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:00 smithi016 ceph-mon[35851]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T08:38:01.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:00 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi044 2024-03-26T08:38:01.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:00 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi044 to 5694M 2024-03-26T08:38:01.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:00 smithi044 ceph-mon[36461]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T08:38:02.110 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:01 smithi007 ceph-mon[32462]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:02.110 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:01 smithi007 ceph-mon[32462]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T08:38:02.110 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:01 smithi007 ceph-mon[32462]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:38:02.110 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:38:02.110 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:38:02.110 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:02.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:01 smithi016 ceph-mon[35851]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:02.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:01 smithi016 ceph-mon[35851]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T08:38:02.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:01 smithi016 ceph-mon[35851]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:38:02.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:38:02.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:38:02.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:02.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:01 smithi044 ceph-mon[36461]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:02.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:01 smithi044 ceph-mon[36461]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T08:38:02.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:01 smithi044 ceph-mon[36461]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:38:02.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:38:02.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:38:02.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:03.229 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:02 smithi007 ceph-mon[32462]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T08:38:03.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:02 smithi016 ceph-mon[35851]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T08:38:03.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:02 smithi044 ceph-mon[36461]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T08:38:04.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:03 smithi016 ceph-mon[35851]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:04.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:03 smithi007 ceph-mon[32462]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:04.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:03 smithi044 ceph-mon[36461]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:05.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:04 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3372432983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "048f7ae9-18c6-4128-815b-082a9297ed68"}]: dispatch 2024-03-26T08:38:05.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:04 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3372432983' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "048f7ae9-18c6-4128-815b-082a9297ed68"}]': finished 2024-03-26T08:38:05.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:04 smithi016 ceph-mon[35851]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T08:38:05.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:05.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:04 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2066269505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:38:05.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:04 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3372432983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "048f7ae9-18c6-4128-815b-082a9297ed68"}]: dispatch 2024-03-26T08:38:05.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:04 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3372432983' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "048f7ae9-18c6-4128-815b-082a9297ed68"}]': finished 2024-03-26T08:38:05.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:04 smithi007 ceph-mon[32462]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T08:38:05.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:05.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:04 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2066269505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:38:05.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:04 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3372432983' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "048f7ae9-18c6-4128-815b-082a9297ed68"}]: dispatch 2024-03-26T08:38:05.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:04 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3372432983' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "048f7ae9-18c6-4128-815b-082a9297ed68"}]': finished 2024-03-26T08:38:05.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:04 smithi044 ceph-mon[36461]: osdmap e64: 10 total, 9 up, 10 in 2024-03-26T08:38:05.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:05.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:04 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2066269505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:38:06.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:06 smithi016 ceph-mon[35851]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-03-26T08:38:06.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:06 smithi007 ceph-mon[32462]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-03-26T08:38:06.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:06 smithi044 ceph-mon[36461]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-03-26T08:38:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:08 smithi016 ceph-mon[35851]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:38:08.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:08 smithi007 ceph-mon[32462]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:38:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:08 smithi044 ceph-mon[36461]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T08:38:10.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:10 smithi007 ceph-mon[32462]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-26T08:38:10.369 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:10 smithi044 ceph-mon[36461]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-26T08:38:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:10 smithi016 ceph-mon[35851]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-26T08:38:12.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:12 smithi016 ceph-mon[35851]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-26T08:38:12.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:12 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T08:38:12.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:12 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:12.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:12 smithi007 ceph-mon[32462]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-26T08:38:12.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:12 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T08:38:12.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:12 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:12.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:12 smithi044 ceph-mon[36461]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-03-26T08:38:12.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:12 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T08:38:12.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:12 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:13 smithi016 ceph-mon[35851]: Deploying daemon osd.9 on smithi007 2024-03-26T08:38:13.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:13 smithi007 ceph-mon[32462]: Deploying daemon osd.9 on smithi007 2024-03-26T08:38:13.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:13 smithi044 ceph-mon[36461]: Deploying daemon osd.9 on smithi007 2024-03-26T08:38:14.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:14 smithi007 ceph-mon[32462]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:38:14.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:14 smithi016 ceph-mon[35851]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:38:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:14 smithi044 ceph-mon[36461]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:38:16.314 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:16 smithi007 ceph-mon[32462]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:16.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:16 smithi016 ceph-mon[35851]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:16 smithi044 ceph-mon[36461]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:18.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:18 smithi007 ceph-mon[32462]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:18.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:18.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:18.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:18.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:18 smithi016 ceph-mon[35851]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:18.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:18.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:18.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:18 smithi044 ceph-mon[36461]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:20.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:20 smithi016 ceph-mon[35851]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:20.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:20 smithi007 ceph-mon[32462]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:20 smithi044 ceph-mon[36461]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:22.014 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 9 on host 'smithi007' 2024-03-26T08:38:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.022+0000 7f2b827fc700 1 -- 172.21.15.7:0/1221534511 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2b8c0fe770 con 0x7f2b780a2890 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2b780a2890 msgr2=0x7f2b780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2b780a2890 0x7f2b780a4d50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c102110 tx=0x7f2b8800b410 comp rx=0 tx=0).stop 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c1022b0 msgr2=0x7f2b8c103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c1022b0 0x7f2b8c103690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00d950 tx=0x7f2b7c00dc60 comp rx=0 tx=0).stop 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 shutdown_connections 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2b780a2890 0x7f2b780a4d50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2b8c1010b0 0x7f2b8c105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b8c102c50 0x7f2b8c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 --2- 172.21.15.7:0/1221534511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8c1022b0 0x7f2b8c103690 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 >> 172.21.15.7:0/1221534511 conn(0x7f2b8c0fc690 msgr2=0x7f2b8c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 shutdown_connections 2024-03-26T08:38:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:38:22.025+0000 7f2b932b2700 1 -- 172.21.15.7:0/1221534511 wait complete. 2024-03-26T08:38:22.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:22.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:22.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:22.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T08:38:22.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:22.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:22 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:22 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:22 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:22.723 DEBUG:teuthology.orchestra.run.smithi007:osd.9> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.9.service 2024-03-26T08:38:22.725 INFO:tasks.cephadm:Deploying osd.10 on smithi016 with /dev/vg_nvme/lv_1... 2024-03-26T08:38:22.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T08:38:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:38:23.442 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:23 smithi016 ceph-mon[35851]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T08:38:23.442 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:23 smithi016 ceph-mon[35851]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T08:38:23.442 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:23 smithi016 ceph-mon[35851]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:38:23.442 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:23 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:23.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:23 smithi007 ceph-mon[32462]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T08:38:23.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:23 smithi007 ceph-mon[32462]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T08:38:23.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:23 smithi007 ceph-mon[32462]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:38:23.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:23 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:23.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:23 smithi044 ceph-mon[36461]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T08:38:23.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:23 smithi044 ceph-mon[36461]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T08:38:23.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:23 smithi044 ceph-mon[36461]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-03-26T08:38:23.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:23 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:23.943 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:38:23 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:38:23.592+0000 7fc24aacf700 -1 osd.9 0 waiting for initial osdmap 2024-03-26T08:38:23.943 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:38:23 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:38:23.604+0000 7fc2428ba700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:38:24.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:24 smithi007 ceph-mon[32462]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:24 smithi007 ceph-mon[32462]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:38:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:24 smithi007 ceph-mon[32462]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T08:38:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:24 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:24 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:24.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:24 smithi007 ceph-mon[32462]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' 2024-03-26T08:38:24.426 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:24 smithi016 ceph-mon[35851]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:24.426 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:24 smithi016 ceph-mon[35851]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:38:24.426 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:24 smithi016 ceph-mon[35851]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T08:38:24.426 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:24 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:24.426 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:24 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:24.426 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:24 smithi016 ceph-mon[35851]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' 2024-03-26T08:38:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:24 smithi044 ceph-mon[36461]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:24 smithi044 ceph-mon[36461]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-03-26T08:38:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:24 smithi044 ceph-mon[36461]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T08:38:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:24 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:24 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:24 smithi044 ceph-mon[36461]: from='osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612]' entity='osd.9' 2024-03-26T08:38:24.995 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T08:38:24.995 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T08:38:24.995 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-03-26T08:38:24.995 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-03-26T08:38:24.995 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00828415 s, 1.3 GB/s 2024-03-26T08:38:24.996 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-03-26T08:38:25.305 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:25 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:38:25.305 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:25 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:38:25.305 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:25 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:25.305 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:25 smithi016 ceph-mon[35851]: osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] boot 2024-03-26T08:38:25.305 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:25 smithi016 ceph-mon[35851]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T08:38:25.306 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:25 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:25.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:25 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:38:25.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:25 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:38:25.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:25 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:25.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:25 smithi007 ceph-mon[32462]: osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] boot 2024-03-26T08:38:25.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:25 smithi007 ceph-mon[32462]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T08:38:25.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:25 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:25 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:25 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:25 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:25 smithi044 ceph-mon[36461]: osd.9 [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] boot 2024-03-26T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:25 smithi044 ceph-mon[36461]: osdmap e67: 10 total, 10 up, 10 in 2024-03-26T08:38:25.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:25 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T08:38:25.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-03-26T08:38:26.076 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:38:26.459 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:26 smithi016 ceph-mon[35851]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:26.459 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:26 smithi016 ceph-mon[35851]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T08:38:26.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:26 smithi007 ceph-mon[32462]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:26.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:26 smithi007 ceph-mon[32462]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T08:38:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:26 smithi044 ceph-mon[36461]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-26T08:38:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:26 smithi044 ceph-mon[36461]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T08:38:27.475 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi007 2024-03-26T08:38:27.475 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.475 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.475 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.475 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.475 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.475 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.475 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi007 to 4273M 2024-03-26T08:38:27.475 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.476 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:27.476 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:27.476 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:27 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi007 2024-03-26T08:38:27.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi007 to 4273M 2024-03-26T08:38:27.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:27.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:27.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:27 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi007 2024-03-26T08:38:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi007 to 4273M 2024-03-26T08:38:27.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:27.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:27.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:27 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.709+0000 7f818f5cb700 1 -- 172.21.15.16:0/3858212445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102cc0 msgr2=0x7f81881050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.709+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3858212445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102cc0 0x7f81881050b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8184009990 tx=0x7f8184009ca0 comp rx=0 tx=0).stop 2024-03-26T08:38:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 -- 172.21.15.16:0/3858212445 shutdown_connections 2024-03-26T08:38:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3858212445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8188107f20 0x7f818810a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3858212445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188102cc0 0x7f81881050b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3858212445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81881055f0 0x7f81881079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 -- 172.21.15.16:0/3858212445 >> 172.21.15.16:0/3858212445 conn(0x7f81880fc6f0 msgr2=0x7f81880feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 -- 172.21.15.16:0/3858212445 shutdown_connections 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 -- 172.21.15.16:0/3858212445 wait complete. 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 Processor -- start 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 -- start start 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8188102cc0 0x7f81881983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81881055f0 0x7f8188198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188107f20 0x7f818819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.710+0000 7f818f5cb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818810de10 con 0x7f8188102cc0 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818f5cb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818810dc90 con 0x7f8188107f20 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818f5cb700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818810df90 con 0x7f81881055f0 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818db68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188107f20 0x7f818819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818db68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188107f20 0x7f818819f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58284/0 (socket says 172.21.15.16:58284) 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818db68700 1 -- 172.21.15.16:0/3342062954 learned_addr learned my addr 172.21.15.16:0/3342062954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818cb66700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81881055f0 0x7f8188198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818d367700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8188102cc0 0x7f81881983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818db68700 1 -- 172.21.15.16:0/3342062954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81881055f0 msgr2=0x7f8188198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818db68700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81881055f0 0x7f8188198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818db68700 1 -- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8188102cc0 msgr2=0x7f81881983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818db68700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8188102cc0 0x7f81881983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818db68700 1 -- 172.21.15.16:0/3342062954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81881a0000 con 0x7f8188107f20 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818db68700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188107f20 0x7f818819f960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f818000b9e0 tx=0x7f818000bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f817e7fc700 1 -- 172.21.15.16:0/3342062954 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180019070 con 0x7f8188107f20 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81881a02f0 con 0x7f8188107f20 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f817e7fc700 1 -- 172.21.15.16:0/3342062954 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f818000ce10 con 0x7f8188107f20 2024-03-26T08:38:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f817e7fc700 1 -- 172.21.15.16:0/3342062954 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180012360 con 0x7f8188107f20 2024-03-26T08:38:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.711+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81881a0900 con 0x7f8188107f20 2024-03-26T08:38:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.712+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8188068e50 con 0x7f8188107f20 2024-03-26T08:38:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.713+0000 7f817e7fc700 1 -- 172.21.15.16:0/3342062954 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f818001c070 con 0x7f8188107f20 2024-03-26T08:38:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.714+0000 7f817e7fc700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f81740a2890 0x7f81740a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.714+0000 7f817e7fc700 1 -- 172.21.15.16:0/3342062954 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f81800cd530 con 0x7f8188107f20 2024-03-26T08:38:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.714+0000 7f818d367700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f81740a2890 0x7f81740a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.715+0000 7f818d367700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f81740a2890 0x7f81740a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f818400b5c0 tx=0x7f8184005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:38:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.719+0000 7f817e7fc700 1 -- 172.21.15.16:0/3342062954 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f818009add0 con 0x7f8188107f20 2024-03-26T08:38:27.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:27.934+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f81880625d0 con 0x7f81740a2890 2024-03-26T08:38:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:28 smithi016 ceph-mon[35851]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:38:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:38:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:28 smithi007 ceph-mon[32462]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:38:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:38:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:28 smithi044 ceph-mon[36461]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:38:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:38:28.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:29.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:29 smithi016 ceph-mon[35851]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:38:29.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:29 smithi007 ceph-mon[32462]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:38:29.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:29 smithi044 ceph-mon[36461]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:38:30.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:30 smithi016 ceph-mon[35851]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:30.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:30 smithi007 ceph-mon[32462]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:30.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:30 smithi044 ceph-mon[36461]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:31.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:31 smithi016 ceph-mon[35851]: from='client.? 172.21.15.16:0/3815244044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40e2915e-efad-4dbd-a244-cad31df0cd80"}]: dispatch 2024-03-26T08:38:31.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:31 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40e2915e-efad-4dbd-a244-cad31df0cd80"}]: dispatch 2024-03-26T08:38:31.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:31 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40e2915e-efad-4dbd-a244-cad31df0cd80"}]': finished 2024-03-26T08:38:31.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:31 smithi016 ceph-mon[35851]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T08:38:31.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:31.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:31 smithi007 ceph-mon[32462]: from='client.? 172.21.15.16:0/3815244044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40e2915e-efad-4dbd-a244-cad31df0cd80"}]: dispatch 2024-03-26T08:38:31.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:31 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40e2915e-efad-4dbd-a244-cad31df0cd80"}]: dispatch 2024-03-26T08:38:31.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:31 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40e2915e-efad-4dbd-a244-cad31df0cd80"}]': finished 2024-03-26T08:38:31.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:31 smithi007 ceph-mon[32462]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T08:38:31.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:31 smithi044 ceph-mon[36461]: from='client.? 172.21.15.16:0/3815244044' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40e2915e-efad-4dbd-a244-cad31df0cd80"}]: dispatch 2024-03-26T08:38:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:31 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40e2915e-efad-4dbd-a244-cad31df0cd80"}]: dispatch 2024-03-26T08:38:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:31 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40e2915e-efad-4dbd-a244-cad31df0cd80"}]': finished 2024-03-26T08:38:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:31 smithi044 ceph-mon[36461]: osdmap e69: 11 total, 10 up, 11 in 2024-03-26T08:38:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:32 smithi016 ceph-mon[35851]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:32 smithi016 ceph-mon[35851]: from='client.? 172.21.15.16:0/3756530202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:38:32.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:32 smithi007 ceph-mon[32462]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:32.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:32 smithi007 ceph-mon[32462]: from='client.? 172.21.15.16:0/3756530202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:38:32.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:32 smithi044 ceph-mon[36461]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:32.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:32 smithi044 ceph-mon[36461]: from='client.? 172.21.15.16:0/3756530202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:38:34.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:34 smithi016 ceph-mon[35851]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:34.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:34 smithi007 ceph-mon[32462]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:34.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:34 smithi044 ceph-mon[36461]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:36.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:36 smithi016 ceph-mon[35851]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:36.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:36 smithi007 ceph-mon[32462]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:36.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:36 smithi044 ceph-mon[36461]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:38.408 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:38 smithi016 ceph-mon[35851]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:38.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:38 smithi007 ceph-mon[32462]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:38 smithi044 ceph-mon[36461]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:39.336 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T08:38:39.336 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:39.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T08:38:39.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:39.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T08:38:39.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:40.357 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:40 smithi016 ceph-mon[35851]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:40.357 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:40 smithi016 ceph-mon[35851]: Deploying daemon osd.10 on smithi016 2024-03-26T08:38:40.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:40 smithi007 ceph-mon[32462]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:40.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:40 smithi007 ceph-mon[32462]: Deploying daemon osd.10 on smithi016 2024-03-26T08:38:40.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:40 smithi044 ceph-mon[36461]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:40.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:40 smithi044 ceph-mon[36461]: Deploying daemon osd.10 on smithi016 2024-03-26T08:38:42.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:42 smithi007 ceph-mon[32462]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:42.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:42 smithi044 ceph-mon[36461]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:42.603 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:42 smithi016 ceph-mon[35851]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:44.395 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:44 smithi016 ceph-mon[35851]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:44.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:44 smithi007 ceph-mon[32462]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:44.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:44 smithi044 ceph-mon[36461]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:45.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:45 smithi016 ceph-mon[35851]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:45.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:45 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:45.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:45 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:45.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:45 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:45 smithi007 ceph-mon[32462]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:45 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:45 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:45 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:45.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:45 smithi044 ceph-mon[36461]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:45.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:45 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:45.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:45 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:45.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:45 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:47.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:47 smithi016 ceph-mon[35851]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:47.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:47 smithi007 ceph-mon[32462]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:47.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:47 smithi044 ceph-mon[36461]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.553+0000 7f817e7fc700 1 -- 172.21.15.16:0/3342062954 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f81880625d0 con 0x7f81740a2890 2024-03-26T08:38:48.546 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 10 on host 'smithi016' 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f81740a2890 msgr2=0x7f81740a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f81740a2890 0x7f81740a4d50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f818400b5c0 tx=0x7f8184005fb0 comp rx=0 tx=0).stop 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188107f20 msgr2=0x7f818819f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188107f20 0x7f818819f960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f818000b9e0 tx=0x7f818000bcf0 comp rx=0 tx=0).stop 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 shutdown_connections 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f81740a2890 0x7f81740a4d50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8188107f20 0x7f818819f960 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8188102cc0 0x7f81881983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 --2- 172.21.15.16:0/3342062954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81881055f0 0x7f8188198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 >> 172.21.15.16:0/3342062954 conn(0x7f81880fc6f0 msgr2=0x7f8188103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 shutdown_connections 2024-03-26T08:38:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-03-26T08:38:48.555+0000 7f818f5cb700 1 -- 172.21.15.16:0/3342062954 wait complete. 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='osd.10 [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:48.912 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:48 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:49.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:49.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='osd.10 [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:38:49.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:38:49.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:49.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:48 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:49.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:49.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='osd.10 [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:38:49.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T08:38:49.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:38:49.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:48 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:49.301 DEBUG:teuthology.orchestra.run.smithi016:osd.10> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.10.service 2024-03-26T08:38:49.304 INFO:tasks.cephadm:Deploying osd.11 on smithi044 with /dev/vg_nvme/lv_1... 2024-03-26T08:38:49.304 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T08:38:49.564 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:38:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:49 smithi016 ceph-mon[35851]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:49 smithi016 ceph-mon[35851]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T08:38:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:49 smithi016 ceph-mon[35851]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T08:38:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:49 smithi016 ceph-mon[35851]: from='osd.10 [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:38:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:49 smithi016 ceph-mon[35851]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:38:50.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:49 smithi007 ceph-mon[32462]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:50.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:49 smithi007 ceph-mon[32462]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T08:38:50.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:49 smithi007 ceph-mon[32462]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T08:38:50.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:49 smithi007 ceph-mon[32462]: from='osd.10 [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:38:50.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:50.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:49 smithi007 ceph-mon[32462]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:38:50.049 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:49 smithi044 ceph-mon[36461]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:50.049 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:49 smithi044 ceph-mon[36461]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T08:38:50.049 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:49 smithi044 ceph-mon[36461]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T08:38:50.050 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:49 smithi044 ceph-mon[36461]: from='osd.10 [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:38:50.050 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:50.050 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:49 smithi044 ceph-mon[36461]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-03-26T08:38:50.492 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:38:50.232+0000 7f071e248700 -1 osd.10 0 waiting for initial osdmap 2024-03-26T08:38:50.492 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:38:50.243+0000 7f0716033700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:38:50.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:50 smithi044 ceph-mon[36461]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:38:50.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:50 smithi044 ceph-mon[36461]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T08:38:50.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:50.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:50.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:50 smithi044 ceph-mon[36461]: from='osd.10 ' entity='osd.10' 2024-03-26T08:38:50.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:50 smithi044 ceph-mon[36461]: osd.10 [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] boot 2024-03-26T08:38:50.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:50 smithi044 ceph-mon[36461]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T08:38:50.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:51.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:50 smithi007 ceph-mon[32462]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:38:51.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:50 smithi007 ceph-mon[32462]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T08:38:51.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:51.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:51.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:50 smithi007 ceph-mon[32462]: from='osd.10 ' entity='osd.10' 2024-03-26T08:38:51.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:50 smithi007 ceph-mon[32462]: osd.10 [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] boot 2024-03-26T08:38:51.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:50 smithi007 ceph-mon[32462]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T08:38:51.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:51.093 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-mon[35851]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-03-26T08:38:51.093 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-mon[35851]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T08:38:51.093 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:51.093 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:51.093 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-mon[35851]: from='osd.10 ' entity='osd.10' 2024-03-26T08:38:51.093 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-mon[35851]: osd.10 [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] boot 2024-03-26T08:38:51.093 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-mon[35851]: osdmap e72: 11 total, 11 up, 11 in 2024-03-26T08:38:51.093 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T08:38:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T08:38:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T08:38:51.843 INFO:teuthology.orchestra.run.smithi044.stderr: stderr: 10+0 records in 2024-03-26T08:38:51.843 INFO:teuthology.orchestra.run.smithi044.stderr:10+0 records out 2024-03-26T08:38:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113823 s, 921 MB/s 2024-03-26T08:38:51.844 INFO:teuthology.orchestra.run.smithi044.stderr:--> Zapping successful for: 2024-03-26T08:38:52.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:52 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:38:52.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:52 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:38:52.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:52 smithi016 ceph-mon[35851]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:52.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:52 smithi016 ceph-mon[35851]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T08:38:52.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:52 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:38:52.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:52 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:38:52.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:52 smithi007 ceph-mon[32462]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:52.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:52 smithi007 ceph-mon[32462]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T08:38:52.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:52 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:38:52.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:52 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:38:52.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:52 smithi044 ceph-mon[36461]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-26T08:38:52.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:52 smithi044 ceph-mon[36461]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T08:38:52.635 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi044:vg_nvme/lv_1 2024-03-26T08:38:52.907 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:38:53.933 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:38:53.933 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T08:38:53.933 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi016 2024-03-26T08:38:53.933 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:53.933 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:53.933 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:53.933 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:53.934 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:53.934 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:53.934 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi016 to 4271M 2024-03-26T08:38:53.934 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:53.934 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:53.934 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:53.934 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:53.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:38:53.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi016 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi016 to 4271M 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:53.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:53.995 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:54.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:38:54.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi016 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi016 to 4271M 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:38:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:38:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.718+0000 7ff5a2619700 1 -- 172.21.15.44:0/3048994354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102330 msgr2=0x7ff59c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.718+0000 7ff5a2619700 1 --2- 172.21.15.44:0/3048994354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102330 0x7ff59c102790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009990 tx=0x7ff58c009ca0 comp rx=0 tx=0).stop 2024-03-26T08:38:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.718+0000 7ff5a2619700 1 -- 172.21.15.44:0/3048994354 shutdown_connections 2024-03-26T08:38:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.718+0000 7ff5a2619700 1 --2- 172.21.15.44:0/3048994354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c102cd0 0x7ff59c109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.718+0000 7ff5a2619700 1 --2- 172.21.15.44:0/3048994354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c101130 0x7ff59c101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.718+0000 7ff5a2619700 1 --2- 172.21.15.44:0/3048994354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102330 0x7ff59c102790 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.718+0000 7ff5a2619700 1 -- 172.21.15.44:0/3048994354 >> 172.21.15.44:0/3048994354 conn(0x7ff59c0fc6f0 msgr2=0x7ff59c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:38:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.718+0000 7ff5a2619700 1 -- 172.21.15.44:0/3048994354 shutdown_connections 2024-03-26T08:38:54.710 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.718+0000 7ff5a2619700 1 -- 172.21.15.44:0/3048994354 wait complete. 2024-03-26T08:38:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a2619700 1 Processor -- start 2024-03-26T08:38:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a2619700 1 -- start start 2024-03-26T08:38:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a2619700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c101130 0x7ff59c06b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a2619700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102330 0x7ff59c19d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a2619700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102cd0 0x7ff59c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a2619700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c10cca0 con 0x7ff59c101130 2024-03-26T08:38:54.712 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a2619700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c10cb20 con 0x7ff59c102330 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a2619700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c10ce20 con 0x7ff59c102cd0 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a0bb6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102cd0 0x7ff59c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a0bb6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102cd0 0x7ff59c19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36094/0 (socket says 172.21.15.44:36094) 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a0bb6700 1 -- 172.21.15.44:0/1532866433 learned_addr learned my addr 172.21.15.44:0/1532866433 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff59bfff700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c101130 0x7ff59c06b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff59b7fe700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102330 0x7ff59c19d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a0bb6700 1 -- 172.21.15.44:0/1532866433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102330 msgr2=0x7ff59c19d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a0bb6700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102330 0x7ff59c19d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a0bb6700 1 -- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c101130 msgr2=0x7ff59c06b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a0bb6700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c101130 0x7ff59c06b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.719+0000 7ff5a0bb6700 1 -- 172.21.15.44:0/1532866433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c1a0040 con 0x7ff59c102cd0 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.720+0000 7ff5a0bb6700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102cd0 0x7ff59c19f770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff59400ada0 tx=0x7ff59400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.720+0000 7ff5997fa700 1 -- 172.21.15.44:0/1532866433 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594017020 con 0x7ff59c102cd0 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.720+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c1a0330 con 0x7ff59c102cd0 2024-03-26T08:38:54.713 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.720+0000 7ff5997fa700 1 -- 172.21.15.44:0/1532866433 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff59400a0d0 con 0x7ff59c102cd0 2024-03-26T08:38:54.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.720+0000 7ff5997fa700 1 -- 172.21.15.44:0/1532866433 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff594005480 con 0x7ff59c102cd0 2024-03-26T08:38:54.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.720+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c1a0820 con 0x7ff59c102cd0 2024-03-26T08:38:54.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.721+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c068e50 con 0x7ff59c102cd0 2024-03-26T08:38:54.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.722+0000 7ff5997fa700 1 -- 172.21.15.44:0/1532866433 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff594005620 con 0x7ff59c102cd0 2024-03-26T08:38:54.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.723+0000 7ff5997fa700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff5840a2890 0x7ff5840a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:38:54.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.723+0000 7ff5997fa700 1 -- 172.21.15.44:0/1532866433 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6694+0+0 (secure 0 0 0) 0x7ff5940ce190 con 0x7ff59c102cd0 2024-03-26T08:38:54.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.723+0000 7ff59bfff700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff5840a2890 0x7ff5840a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:38:54.715 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.724+0000 7ff59bfff700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff5840a2890 0x7ff5840a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff590005950 tx=0x7ff59000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:38:54.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.727+0000 7ff5997fa700 1 -- 172.21.15.44:0/1532866433 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff594021020 con 0x7ff59c102cd0 2024-03-26T08:38:54.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:38:54.940+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff59c0625d0 con 0x7ff5840a2890 2024-03-26T08:38:54.932 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:54 smithi044 ceph-mon[36461]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T08:38:54.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:54 smithi016 ceph-mon[35851]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T08:38:55.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:54 smithi007 ceph-mon[32462]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T08:38:55.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:55 smithi044 ceph-mon[36461]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:38:55.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:55 smithi044 ceph-mon[36461]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T08:38:55.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:55 smithi044 ceph-mon[36461]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:38:55.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:38:55.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:38:55.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:55.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:55 smithi016 ceph-mon[35851]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:38:55.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:55 smithi016 ceph-mon[35851]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T08:38:55.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:55 smithi016 ceph-mon[35851]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:38:55.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:38:55.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:38:55.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:56.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:55 smithi007 ceph-mon[32462]: pgmap v243: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:38:56.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:55 smithi007 ceph-mon[32462]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T08:38:56.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:55 smithi007 ceph-mon[32462]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi044:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:38:56.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T08:38:56.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T08:38:56.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:38:57.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:57 smithi016 ceph-mon[35851]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:38:58.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:57 smithi007 ceph-mon[32462]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:38:58.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:57 smithi044 ceph-mon[36461]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:38:58.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:58 smithi016 ceph-mon[35851]: from='client.? 172.21.15.44:0/3682438099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3e817ae-4259-46ec-bbe3-91c24adbeb96"}]: dispatch 2024-03-26T08:38:58.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:58 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3e817ae-4259-46ec-bbe3-91c24adbeb96"}]: dispatch 2024-03-26T08:38:58.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:58 smithi016 ceph-mon[35851]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3e817ae-4259-46ec-bbe3-91c24adbeb96"}]': finished 2024-03-26T08:38:58.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:58 smithi016 ceph-mon[35851]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T08:38:58.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:38:58.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:58 smithi016 ceph-mon[35851]: from='client.? 172.21.15.44:0/1410048908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:38:59.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:58 smithi007 ceph-mon[32462]: from='client.? 172.21.15.44:0/3682438099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3e817ae-4259-46ec-bbe3-91c24adbeb96"}]: dispatch 2024-03-26T08:38:59.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:58 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3e817ae-4259-46ec-bbe3-91c24adbeb96"}]: dispatch 2024-03-26T08:38:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:58 smithi007 ceph-mon[32462]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3e817ae-4259-46ec-bbe3-91c24adbeb96"}]': finished 2024-03-26T08:38:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:58 smithi007 ceph-mon[32462]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T08:38:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:38:59.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:58 smithi007 ceph-mon[32462]: from='client.? 172.21.15.44:0/1410048908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:38:59.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:58 smithi044 ceph-mon[36461]: from='client.? 172.21.15.44:0/3682438099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3e817ae-4259-46ec-bbe3-91c24adbeb96"}]: dispatch 2024-03-26T08:38:59.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:58 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3e817ae-4259-46ec-bbe3-91c24adbeb96"}]: dispatch 2024-03-26T08:38:59.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:58 smithi044 ceph-mon[36461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3e817ae-4259-46ec-bbe3-91c24adbeb96"}]': finished 2024-03-26T08:38:59.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:58 smithi044 ceph-mon[36461]: osdmap e77: 12 total, 11 up, 12 in 2024-03-26T08:38:59.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:38:59.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:58 smithi044 ceph-mon[36461]: from='client.? 172.21.15.44:0/1410048908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T08:39:00.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:38:59 smithi016 ceph-mon[35851]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T08:39:00.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:38:59 smithi007 ceph-mon[32462]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T08:39:00.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:38:59 smithi044 ceph-mon[36461]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T08:39:02.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:01 smithi016 ceph-mon[35851]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-26T08:39:02.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:01 smithi007 ceph-mon[32462]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-26T08:39:02.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:01 smithi044 ceph-mon[36461]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-26T08:39:04.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:03 smithi016 ceph-mon[35851]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:39:04.292 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:03 smithi044 ceph-mon[36461]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:39:04.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:03 smithi007 ceph-mon[32462]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T08:39:06.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:05 smithi044 ceph-mon[36461]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-26T08:39:06.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:05 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T08:39:06.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:05 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:06.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:05 smithi016 ceph-mon[35851]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-26T08:39:06.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:05 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T08:39:06.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:05 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:06.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:05 smithi007 ceph-mon[32462]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-26T08:39:06.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:05 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T08:39:06.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:05 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:07.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:06 smithi016 ceph-mon[35851]: Deploying daemon osd.11 on smithi044 2024-03-26T08:39:07.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:06 smithi007 ceph-mon[32462]: Deploying daemon osd.11 on smithi044 2024-03-26T08:39:07.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:06 smithi044 ceph-mon[36461]: Deploying daemon osd.11 on smithi044 2024-03-26T08:39:07.932 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:07 smithi044 ceph-mon[36461]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:39:08.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:07 smithi016 ceph-mon[35851]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:39:08.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:07 smithi007 ceph-mon[32462]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T08:39:09.941 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:09 smithi044 ceph-mon[36461]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:10.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:09 smithi016 ceph-mon[35851]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:10.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:09 smithi007 ceph-mon[32462]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:12.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:11 smithi016 ceph-mon[35851]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:12.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:11 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:12.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:11 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:12.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:11 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:12.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:11 smithi007 ceph-mon[32462]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:12.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:12.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:12.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:11 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:12.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:11 smithi044 ceph-mon[36461]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:12.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:11 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:12.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:11 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:12.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:11 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:14.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:13 smithi016 ceph-mon[35851]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:14.282 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:13 smithi044 ceph-mon[36461]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:14.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:13 smithi007 ceph-mon[32462]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:15.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.261+0000 7ff5997fa700 1 -- 172.21.15.44:0/1532866433 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff59c0625d0 con 0x7ff5840a2890 2024-03-26T08:39:15.253 INFO:teuthology.orchestra.run.smithi044.stdout:Created osd(s) 11 on host 'smithi044' 2024-03-26T08:39:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff5840a2890 msgr2=0x7ff5840a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff5840a2890 0x7ff5840a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff590005950 tx=0x7ff59000a300 comp rx=0 tx=0).stop 2024-03-26T08:39:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102cd0 msgr2=0x7ff59c19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102cd0 0x7ff59c19f770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff59400ada0 tx=0x7ff59400c330 comp rx=0 tx=0).stop 2024-03-26T08:39:15.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 shutdown_connections 2024-03-26T08:39:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff5840a2890 0x7ff5840a4d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff59c102cd0 0x7ff59c19f770 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c101130 0x7ff59c06b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 --2- 172.21.15.44:0/1532866433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff59c102330 0x7ff59c19d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 >> 172.21.15.44:0/1532866433 conn(0x7ff59c0fc6f0 msgr2=0x7ff59c105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.263+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 shutdown_connections 2024-03-26T08:39:15.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:15.264+0000 7ff5a2619700 1 -- 172.21.15.44:0/1532866433 wait complete. 2024-03-26T08:39:15.909 DEBUG:teuthology.orchestra.run.smithi044:osd.11> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.11.service 2024-03-26T08:39:15.912 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-26T08:39:15.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T08:39:16.165 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:39:16.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:16 smithi007 ceph-mon[32462]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:16.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:16 smithi007 ceph-mon[32462]: from='osd.11 [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:39:16.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:16 smithi007 ceph-mon[32462]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:39:16.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:16 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:16 smithi044 ceph-mon[36461]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:16 smithi044 ceph-mon[36461]: from='osd.11 [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:39:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:16 smithi044 ceph-mon[36461]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:39:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:16 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:16 smithi016 ceph-mon[35851]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:16 smithi016 ceph-mon[35851]: from='osd.11 [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:39:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:16 smithi016 ceph-mon[35851]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T08:39:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:16 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:17.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:17 smithi007 ceph-mon[32462]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T08:39:17.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:17 smithi007 ceph-mon[32462]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T08:39:17.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:17 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:17.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:17 smithi007 ceph-mon[32462]: from='osd.11 [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:39:17.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:17 smithi007 ceph-mon[32462]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:39:17.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:17 smithi044 ceph-mon[36461]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T08:39:17.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:17 smithi044 ceph-mon[36461]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T08:39:17.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:17 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:17.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:17 smithi044 ceph-mon[36461]: from='osd.11 [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:39:17.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:17 smithi044 ceph-mon[36461]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:39:17.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:17 smithi016 ceph-mon[35851]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T08:39:17.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:17 smithi016 ceph-mon[35851]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T08:39:17.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:17 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:17.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:17 smithi016 ceph-mon[35851]: from='osd.11 [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:39:17.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:17 smithi016 ceph-mon[35851]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]: dispatch 2024-03-26T08:39:17.954 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:39:17 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:39:17.707+0000 7ff4d852d700 -1 osd.11 0 waiting for initial osdmap 2024-03-26T08:39:17.954 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:39:17 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:39:17.719+0000 7ff4cfb17700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T08:39:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.163+0000 7fa86d94c700 1 -- 172.21.15.7:0/2998157495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868103e00 msgr2=0x7fa86810a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.163+0000 7fa86d94c700 1 --2- 172.21.15.7:0/2998157495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868103e00 0x7fa86810a6b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa86000b0a0 tx=0x7fa86000b3b0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.163+0000 7fa86d94c700 1 -- 172.21.15.7:0/2998157495 shutdown_connections 2024-03-26T08:39:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.163+0000 7fa86d94c700 1 --2- 172.21.15.7:0/2998157495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868103e00 0x7fa86810a6b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.163+0000 7fa86d94c700 1 --2- 172.21.15.7:0/2998157495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868100480 0x7fa8681008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.163+0000 7fa86d94c700 1 --2- 172.21.15.7:0/2998157495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa868103460 0x7fa8681038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.163+0000 7fa86d94c700 1 -- 172.21.15.7:0/2998157495 >> 172.21.15.7:0/2998157495 conn(0x7fa8680fbda0 msgr2=0x7fa8680fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.164+0000 7fa86d94c700 1 -- 172.21.15.7:0/2998157495 shutdown_connections 2024-03-26T08:39:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.164+0000 7fa86d94c700 1 -- 172.21.15.7:0/2998157495 wait complete. 2024-03-26T08:39:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.164+0000 7fa86d94c700 1 Processor -- start 2024-03-26T08:39:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa86d94c700 1 -- start start 2024-03-26T08:39:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa86d94c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868100480 0x7fa86819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa86d94c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868103460 0x7fa86819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa86d94c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa868103e00 0x7fa8681a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa86d94c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86810dda0 con 0x7fa868100480 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa86d94c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86810dc20 con 0x7fa868103460 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa86d94c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86810df20 con 0x7fa868103e00 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa866ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868100480 0x7fa86819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa866ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868100480 0x7fa86819e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54424/0 (socket says 172.21.15.7:54424) 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa866ffd700 1 -- 172.21.15.7:0/1576495147 learned_addr learned my addr 172.21.15.7:0/1576495147 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa8667fc700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868103460 0x7fa86819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa8677fe700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa868103e00 0x7fa8681a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa866ffd700 1 -- 172.21.15.7:0/1576495147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa868103e00 msgr2=0x7fa8681a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa866ffd700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa868103e00 0x7fa8681a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa866ffd700 1 -- 172.21.15.7:0/1576495147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868103460 msgr2=0x7fa86819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa866ffd700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868103460 0x7fa86819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa866ffd700 1 -- 172.21.15.7:0/1576495147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8681a65c0 con 0x7fa868100480 2024-03-26T08:39:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.165+0000 7fa866ffd700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868100480 0x7fa86819e960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa85800c9a0 tx=0x7fa85800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.166+0000 7fa86c94a700 1 -- 172.21.15.7:0/1576495147 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858004220 con 0x7fa868100480 2024-03-26T08:39:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.166+0000 7fa86c94a700 1 -- 172.21.15.7:0/1576495147 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa858007d40 con 0x7fa868100480 2024-03-26T08:39:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.166+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8681a68b0 con 0x7fa868100480 2024-03-26T08:39:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.166+0000 7fa86c94a700 1 -- 172.21.15.7:0/1576495147 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858003da0 con 0x7fa868100480 2024-03-26T08:39:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.166+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8681a6df0 con 0x7fa868100480 2024-03-26T08:39:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.167+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa868068e50 con 0x7fa868100480 2024-03-26T08:39:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.168+0000 7fa86c94a700 1 -- 172.21.15.7:0/1576495147 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa8580043c0 con 0x7fa868100480 2024-03-26T08:39:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.168+0000 7fa86c94a700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8500a2890 0x7fa8500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.168+0000 7fa8667fc700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8500a2890 0x7fa8500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.168+0000 7fa86c94a700 1 -- 172.21.15.7:0/1576495147 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fa8580ce060 con 0x7fa868100480 2024-03-26T08:39:18.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.169+0000 7fa8667fc700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8500a2890 0x7fa8500a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa85c005950 tx=0x7fa85c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:18.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.173+0000 7fa86c94a700 1 -- 172.21.15.7:0/1576495147 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa858014030 con 0x7fa868100480 2024-03-26T08:39:18.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.380+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fa868062da0 con 0x7fa868100480 2024-03-26T08:39:18.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.381+0000 7fa86c94a700 1 -- 172.21.15.7:0/1576495147 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7fa85809b6c0 con 0x7fa868100480 2024-03-26T08:39:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8500a2890 msgr2=0x7fa8500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8500a2890 0x7fa8500a4d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa85c005950 tx=0x7fa85c00a300 comp rx=0 tx=0).stop 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868100480 msgr2=0x7fa86819e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868100480 0x7fa86819e960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa85800c9a0 tx=0x7fa85800ccb0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 shutdown_connections 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8500a2890 0x7fa8500a4d50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa868103e00 0x7fa8681a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa868100480 0x7fa86819e960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 --2- 172.21.15.7:0/1576495147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa868103460 0x7fa86819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 >> 172.21.15.7:0/1576495147 conn(0x7fa8680fbda0 msgr2=0x7fa868104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 shutdown_connections 2024-03-26T08:39:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:18.383+0000 7fa86d94c700 1 -- 172.21.15.7:0/1576495147 wait complete. 2024-03-26T08:39:18.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:18 smithi007 ceph-mon[32462]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:18.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:18 smithi007 ceph-mon[32462]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:39:18.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:18 smithi007 ceph-mon[32462]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T08:39:18.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:18.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:18 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:18.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:18 smithi007 ceph-mon[32462]: from='osd.11 ' entity='osd.11' 2024-03-26T08:39:18.545 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:18 smithi044 ceph-mon[36461]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:18.545 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:18 smithi044 ceph-mon[36461]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:39:18.546 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:18 smithi044 ceph-mon[36461]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T08:39:18.546 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:18.546 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:18 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:18.546 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:18 smithi044 ceph-mon[36461]: from='osd.11 ' entity='osd.11' 2024-03-26T08:39:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:18 smithi016 ceph-mon[35851]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:18 smithi016 ceph-mon[35851]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi044", "root=default"]}]': finished 2024-03-26T08:39:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:18 smithi016 ceph-mon[35851]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T08:39:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:18 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:18 smithi016 ceph-mon[35851]: from='osd.11 ' entity='osd.11' 2024-03-26T08:39:19.196 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1711442330,"num_in_osds":12,"osd_in_since":1711442337,"num_remapped_pgs":0} 2024-03-26T08:39:19.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:19 smithi007 ceph-mon[32462]: purged_snaps scrub starts 2024-03-26T08:39:19.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:19 smithi007 ceph-mon[32462]: purged_snaps scrub ok 2024-03-26T08:39:19.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:19 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:19.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:19 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1576495147' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T08:39:19.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:19 smithi007 ceph-mon[32462]: osd.11 [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] boot 2024-03-26T08:39:19.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:19 smithi007 ceph-mon[32462]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T08:39:19.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:19 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:19.563 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:19 smithi044 ceph-mon[36461]: purged_snaps scrub starts 2024-03-26T08:39:19.564 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:19 smithi044 ceph-mon[36461]: purged_snaps scrub ok 2024-03-26T08:39:19.564 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:19 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:19.564 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:19 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1576495147' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T08:39:19.564 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:19 smithi044 ceph-mon[36461]: osd.11 [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] boot 2024-03-26T08:39:19.564 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:19 smithi044 ceph-mon[36461]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T08:39:19.564 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:19 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:19.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:19 smithi016 ceph-mon[35851]: purged_snaps scrub starts 2024-03-26T08:39:19.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:19 smithi016 ceph-mon[35851]: purged_snaps scrub ok 2024-03-26T08:39:19.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:19 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:19.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:19 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1576495147' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T08:39:19.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:19 smithi016 ceph-mon[35851]: osd.11 [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] boot 2024-03-26T08:39:19.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:19 smithi016 ceph-mon[35851]: osdmap e80: 12 total, 12 up, 12 in 2024-03-26T08:39:19.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:19 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T08:39:20.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T08:39:20.441 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:39:20.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:20.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:20 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:20 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:20.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 331 MiB used, 983 GiB / 983 GiB avail 2024-03-26T08:39:20.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:39:20.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:20.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:20.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:20.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:20 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:21 smithi007 ceph-mon[32462]: Detected new or changed devices on smithi044 2024-03-26T08:39:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:21 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi044 to 4271M 2024-03-26T08:39:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:21 smithi007 ceph-mon[32462]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T08:39:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:21 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:21 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:21 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:21.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:21 smithi044 ceph-mon[36461]: Detected new or changed devices on smithi044 2024-03-26T08:39:21.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:21 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi044 to 4271M 2024-03-26T08:39:21.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:21 smithi044 ceph-mon[36461]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T08:39:21.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:21 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:21.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:21 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:21.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:21 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:21.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:21 smithi016 ceph-mon[35851]: Detected new or changed devices on smithi044 2024-03-26T08:39:21.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:21 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi044 to 4271M 2024-03-26T08:39:21.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:21 smithi016 ceph-mon[35851]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T08:39:21.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:21 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:21.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:21 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:21.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:21 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:22.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.192+0000 7f5b4d133700 1 -- 172.21.15.7:0/3596104480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481033c0 msgr2=0x7f5b48103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:22.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.192+0000 7f5b4d133700 1 --2- 172.21.15.7:0/3596104480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481033c0 0x7f5b48103820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5b380099c0 tx=0x7f5b38009cd0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.192+0000 7f5b4d133700 1 -- 172.21.15.7:0/3596104480 shutdown_connections 2024-03-26T08:39:22.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.192+0000 7f5b4d133700 1 --2- 172.21.15.7:0/3596104480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b48103d60 0x7f5b4810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.192+0000 7f5b4d133700 1 --2- 172.21.15.7:0/3596104480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b481021c0 0x7f5b481025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.192+0000 7f5b4d133700 1 --2- 172.21.15.7:0/3596104480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481033c0 0x7f5b48103820 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.192+0000 7f5b4d133700 1 -- 172.21.15.7:0/3596104480 >> 172.21.15.7:0/3596104480 conn(0x7f5b480fd7a0 msgr2=0x7f5b480ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:22.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.192+0000 7f5b4d133700 1 -- 172.21.15.7:0/3596104480 shutdown_connections 2024-03-26T08:39:22.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.192+0000 7f5b4d133700 1 -- 172.21.15.7:0/3596104480 wait complete. 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.193+0000 7f5b4d133700 1 Processor -- start 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.193+0000 7f5b4d133700 1 -- start start 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.193+0000 7f5b4d133700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481021c0 0x7f5b48198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.193+0000 7f5b4d133700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b481033c0 0x7f5b48198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.193+0000 7f5b4d133700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b48103d60 0x7f5b4819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.193+0000 7f5b4d133700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4810de10 con 0x7f5b481021c0 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.193+0000 7f5b4d133700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4810dc90 con 0x7f5b48103d60 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.193+0000 7f5b4d133700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4810df90 con 0x7f5b481033c0 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.193+0000 7f5b46d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481021c0 0x7f5b48198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b46d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481021c0 0x7f5b48198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54448/0 (socket says 172.21.15.7:54448) 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b46d9d700 1 -- 172.21.15.7:0/2652101802 learned_addr learned my addr 172.21.15.7:0/2652101802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b4659c700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b481033c0 0x7f5b48198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:22.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b4759e700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b48103d60 0x7f5b4819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b46d9d700 1 -- 172.21.15.7:0/2652101802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b481033c0 msgr2=0x7f5b48198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b46d9d700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b481033c0 0x7f5b48198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b46d9d700 1 -- 172.21.15.7:0/2652101802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b48103d60 msgr2=0x7f5b4819f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b46d9d700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b48103d60 0x7f5b4819f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b46d9d700 1 -- 172.21.15.7:0/2652101802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b481a0060 con 0x7f5b481021c0 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b46d9d700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481021c0 0x7f5b48198400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00c9a0 tx=0x7f5b3c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b2ffff700 1 -- 172.21.15.7:0/2652101802 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c004220 con 0x7f5b481021c0 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b2ffff700 1 -- 172.21.15.7:0/2652101802 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b3c007d40 con 0x7f5b481021c0 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b481a0350 con 0x7f5b481021c0 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b2ffff700 1 -- 172.21.15.7:0/2652101802 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00f450 con 0x7f5b481021c0 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.194+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b481a0910 con 0x7f5b481021c0 2024-03-26T08:39:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.195+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b48068e50 con 0x7f5b481021c0 2024-03-26T08:39:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.196+0000 7f5b2ffff700 1 -- 172.21.15.7:0/2652101802 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5b3c007860 con 0x7f5b481021c0 2024-03-26T08:39:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.197+0000 7f5b2ffff700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5b300a2890 0x7f5b300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.197+0000 7f5b4659c700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5b300a2890 0x7f5b300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.197+0000 7f5b2ffff700 1 -- 172.21.15.7:0/2652101802 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5b3c0ce5e0 con 0x7f5b481021c0 2024-03-26T08:39:22.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.197+0000 7f5b4659c700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5b300a2890 0x7f5b300a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c00 tx=0x7f5b38005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:22.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.200+0000 7f5b2ffff700 1 -- 172.21.15.7:0/2652101802 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5b3c014030 con 0x7f5b481021c0 2024-03-26T08:39:22.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.408+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5b4802da30 con 0x7f5b481021c0 2024-03-26T08:39:22.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.408+0000 7f5b2ffff700 1 -- 172.21.15.7:0/2652101802 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f5b4802da30 con 0x7f5b481021c0 2024-03-26T08:39:22.400 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5b300a2890 msgr2=0x7f5b300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5b300a2890 0x7f5b300a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c00 tx=0x7f5b38005fd0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481021c0 msgr2=0x7f5b48198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481021c0 0x7f5b48198400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00c9a0 tx=0x7f5b3c00cd60 comp rx=0 tx=0).stop 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 shutdown_connections 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5b300a2890 0x7f5b300a4d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b48103d60 0x7f5b4819f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481021c0 0x7f5b48198400 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 --2- 172.21.15.7:0/2652101802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5b481033c0 0x7f5b48198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 >> 172.21.15.7:0/2652101802 conn(0x7f5b480fd7a0 msgr2=0x7f5b48106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 shutdown_connections 2024-03-26T08:39:22.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:22.410+0000 7f5b4d133700 1 -- 172.21.15.7:0/2652101802 wait complete. 2024-03-26T08:39:22.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:22 smithi007 ceph-mon[32462]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:22.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:22 smithi044 ceph-mon[36461]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:22.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:22 smithi016 ceph-mon[35851]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:23.108 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711442358,"num_in_osds":12,"osd_in_since":1711442337,"num_remapped_pgs":0} 2024-03-26T08:39:23.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T08:39:23.353 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:39:23.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:23 smithi007 ceph-mon[32462]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:23.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:23 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2652101802' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T08:39:23.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:23 smithi044 ceph-mon[36461]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:23.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:23 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2652101802' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T08:39:23.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:23 smithi016 ceph-mon[35851]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:23.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:23 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2652101802' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T08:39:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.291+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2593711900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 msgr2=0x7f1c28105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.291+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2593711900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 0x7f1c28105cd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009990 tx=0x7f1c10009ca0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.291+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2593711900 shutdown_connections 2024-03-26T08:39:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.291+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2593711900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c28101320 0x7f1c28101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.291+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2593711900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c281063b0 0x7f1c2810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.291+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2593711900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 0x7f1c28105cd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.291+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2593711900 >> 172.21.15.7:0/2593711900 conn(0x7f1c280770b0 msgr2=0x7f1c280774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:25.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.291+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2593711900 shutdown_connections 2024-03-26T08:39:25.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.291+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2593711900 wait complete. 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c2d6ea700 1 Processor -- start 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c2d6ea700 1 -- start start 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c2d6ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28101320 0x7f1c28102d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c2d6ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 0x7f1c281032b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c2d6ea700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c281063b0 0x7f1c281037f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c2d6ea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28111370 con 0x7f1c28101cd0 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c2d6ea700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c281111f0 con 0x7f1c28101320 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c2d6ea700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c281114f0 con 0x7f1c281063b0 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c267fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 0x7f1c281032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c267fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 0x7f1c281032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54464/0 (socket says 172.21.15.7:54464) 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.292+0000 7f1c267fc700 1 -- 172.21.15.7:0/2940723645 learned_addr learned my addr 172.21.15.7:0/2940723645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c26ffd700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28101320 0x7f1c28102d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c277fe700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c281063b0 0x7f1c281037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c267fc700 1 -- 172.21.15.7:0/2940723645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c281063b0 msgr2=0x7f1c281037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c267fc700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c281063b0 0x7f1c281037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c267fc700 1 -- 172.21.15.7:0/2940723645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28101320 msgr2=0x7f1c28102d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c267fc700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28101320 0x7f1c28102d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c267fc700 1 -- 172.21.15.7:0/2940723645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c281a6df0 con 0x7f1c28101cd0 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c267fc700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 0x7f1c281032b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000c00 tx=0x7f1c100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c1ffff700 1 -- 172.21.15.7:0/2940723645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10027070 con 0x7f1c28101cd0 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c1ffff700 1 -- 172.21.15.7:0/2940723645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c10003780 con 0x7f1c28101cd0 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c1ffff700 1 -- 172.21.15.7:0/2940723645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10022db0 con 0x7f1c28101cd0 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c281a7080 con 0x7f1c28101cd0 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.293+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c281a7480 con 0x7f1c28101cd0 2024-03-26T08:39:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.294+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c281027b0 con 0x7f1c28101cd0 2024-03-26T08:39:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.296+0000 7f1c1ffff700 1 -- 172.21.15.7:0/2940723645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1c1002a070 con 0x7f1c28101cd0 2024-03-26T08:39:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.296+0000 7f1c1ffff700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c140a2890 0x7f1c140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.296+0000 7f1c26ffd700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c140a2890 0x7f1c140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.297+0000 7f1c1ffff700 1 -- 172.21.15.7:0/2940723645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1c100cf6b0 con 0x7f1c28101cd0 2024-03-26T08:39:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.297+0000 7f1c26ffd700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c140a2890 0x7f1c140a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c20004630 tx=0x7f1c2000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.300+0000 7f1c1ffff700 1 -- 172.21.15.7:0/2940723645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c10013070 con 0x7f1c28101cd0 2024-03-26T08:39:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.506+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c2802dbf0 con 0x7f1c28101cd0 2024-03-26T08:39:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.507+0000 7f1c1ffff700 1 -- 172.21.15.7:0/2940723645 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18326 (secure 0 0 0) 0x7f1c1009cbe0 con 0x7f1c28101cd0 2024-03-26T08:39:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:25.500 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:39:20.275865+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c140a2890 msgr2=0x7f1c140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c140a2890 0x7f1c140a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c20004630 tx=0x7f1c2000b3c0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 msgr2=0x7f1c281032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 0x7f1c281032b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c10000c00 tx=0x7f1c100047b0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 shutdown_connections 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c140a2890 0x7f1c140a4d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c281063b0 0x7f1c281037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c28101320 0x7f1c28102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 --2- 172.21.15.7:0/2940723645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c28101cd0 0x7f1c281032b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 >> 172.21.15.7:0/2940723645 conn(0x7f1c280770b0 msgr2=0x7f1c2806abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 shutdown_connections 2024-03-26T08:39:25.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:25.510+0000 7f1c2d6ea700 1 -- 172.21.15.7:0/2940723645 wait complete. 2024-03-26T08:39:25.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:25 smithi007 ceph-mon[32462]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:25.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:25 smithi016 ceph-mon[35851]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:25.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:25 smithi044 ceph-mon[36461]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:26.172 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:39:26.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-26T08:39:26.417 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:39:26.538 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:26 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2940723645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:39:26.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:26 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2940723645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:39:26.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:26 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2940723645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:39:27.634 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:27 smithi007 ceph-mon[32462]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:27.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:27 smithi016 ceph-mon[35851]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:27.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:27 smithi044 ceph-mon[36461]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:28.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.238+0000 7facd7092700 1 -- 172.21.15.7:0/3681684508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0102ba0 msgr2=0x7facd010efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:28.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.238+0000 7facd7092700 1 --2- 172.21.15.7:0/3681684508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0102ba0 0x7facd010efe0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7facc800b0d0 tx=0x7facc800b3e0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 -- 172.21.15.7:0/3681684508 shutdown_connections 2024-03-26T08:39:28.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 --2- 172.21.15.7:0/3681684508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd01081c0 0x7facd01085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 --2- 172.21.15.7:0/3681684508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd0102ba0 0x7facd010efe0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 --2- 172.21.15.7:0/3681684508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd01021e0 0x7facd0102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 -- 172.21.15.7:0/3681684508 >> 172.21.15.7:0/3681684508 conn(0x7facd00792c0 msgr2=0x7facd00796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 -- 172.21.15.7:0/3681684508 shutdown_connections 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 -- 172.21.15.7:0/3681684508 wait complete. 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 Processor -- start 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 -- start start 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd01021e0 0x7facd019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd0102ba0 0x7facd019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd01081c0 0x7facd0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd0112460 con 0x7facd01021e0 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd01122e0 con 0x7facd01081c0 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.239+0000 7facd7092700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd01125e0 con 0x7facd0102ba0 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd4e2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd01021e0 0x7facd019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd4e2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd01021e0 0x7facd019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36400/0 (socket says 172.21.15.7:36400) 2024-03-26T08:39:28.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd4e2e700 1 -- 172.21.15.7:0/2918071380 learned_addr learned my addr 172.21.15.7:0/2918071380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7faccffff700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd0102ba0 0x7facd019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd562f700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd01081c0 0x7facd0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd4e2e700 1 -- 172.21.15.7:0/2918071380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd0102ba0 msgr2=0x7facd019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd4e2e700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd0102ba0 0x7facd019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd4e2e700 1 -- 172.21.15.7:0/2918071380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd01081c0 msgr2=0x7facd0198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd4e2e700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd01081c0 0x7facd0198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd4e2e700 1 -- 172.21.15.7:0/2918071380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd0198de0 con 0x7facd01021e0 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd4e2e700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd01021e0 0x7facd019e420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7facc400d980 tx=0x7facc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7faccdffb700 1 -- 172.21.15.7:0/2918071380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4017070 con 0x7facd01021e0 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7faccdffb700 1 -- 172.21.15.7:0/2918071380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facc4005020 con 0x7facd01021e0 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd01990d0 con 0x7facd01021e0 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.240+0000 7faccdffb700 1 -- 172.21.15.7:0/2918071380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc400fbf0 con 0x7facd01021e0 2024-03-26T08:39:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.241+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd01a52f0 con 0x7facd01021e0 2024-03-26T08:39:28.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.242+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd0068e50 con 0x7facd01021e0 2024-03-26T08:39:28.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.242+0000 7faccdffb700 1 -- 172.21.15.7:0/2918071380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7facc40051d0 con 0x7facd01021e0 2024-03-26T08:39:28.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.243+0000 7faccdffb700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7facc00a2910 0x7facc00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.243+0000 7faccffff700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7facc00a2910 0x7facc00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.243+0000 7faccdffb700 1 -- 172.21.15.7:0/2918071380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7facc40ce340 con 0x7facd01021e0 2024-03-26T08:39:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.244+0000 7faccffff700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7facc00a2910 0x7facc00a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7facd0079830 tx=0x7facbc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:28.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.248+0000 7faccdffb700 1 -- 172.21.15.7:0/2918071380 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7facc4020020 con 0x7facd01021e0 2024-03-26T08:39:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.460+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7facd002da30 con 0x7facd01021e0 2024-03-26T08:39:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.460+0000 7faccdffb700 1 -- 172.21.15.7:0/2918071380 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7facc409b880 con 0x7facd01021e0 2024-03-26T08:39:28.452 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:39:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7facc00a2910 msgr2=0x7facc00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7facc00a2910 0x7facc00a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7facd0079830 tx=0x7facbc00a380 comp rx=0 tx=0).stop 2024-03-26T08:39:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd01021e0 msgr2=0x7facd019e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd01021e0 0x7facd019e420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7facc400d980 tx=0x7facc400dc90 comp rx=0 tx=0).stop 2024-03-26T08:39:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 shutdown_connections 2024-03-26T08:39:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7facc00a2910 0x7facc00a4dd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facd01081c0 0x7facd0198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd01021e0 0x7facd019e420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 --2- 172.21.15.7:0/2918071380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facd0102ba0 0x7facd019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 >> 172.21.15.7:0/2918071380 conn(0x7facd00792c0 msgr2=0x7facd0100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 shutdown_connections 2024-03-26T08:39:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:28.462+0000 7facd7092700 1 -- 172.21.15.7:0/2918071380 wait complete. 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.a on smithi007 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.d on smithi007 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.g on smithi007 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.j on smithi007 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.b on smithi016 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.e on smithi016 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.h on smithi016 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.k on smithi016 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.c on smithi044 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.f on smithi044 2024-03-26T08:39:29.177 INFO:tasks.cephadm:Adding mds.i on smithi044 2024-03-26T08:39:29.178 INFO:tasks.cephadm:Adding mds.l on smithi044 2024-03-26T08:39:29.178 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi044=c;smithi044=f;smithi044=i;smithi044=l' 2024-03-26T08:39:29.440 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:39:29.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:29 smithi007 ceph-mon[32462]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:29.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:29 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2918071380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:39:29.563 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:29 smithi044 ceph-mon[36461]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:29.563 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:29 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2918071380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:39:29.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:29 smithi016 ceph-mon[35851]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:29.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:29 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2918071380' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:39:31.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.193+0000 7f724f301700 1 -- 172.21.15.44:0/559756792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248101d50 msgr2=0x7f7248109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:31.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.193+0000 7f724f301700 1 --2- 172.21.15.44:0/559756792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248101d50 0x7f7248109610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f724000b0d0 tx=0x7f724000b3e0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.193+0000 7f724f301700 1 -- 172.21.15.44:0/559756792 shutdown_connections 2024-03-26T08:39:31.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.193+0000 7f724f301700 1 --2- 172.21.15.44:0/559756792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248101d50 0x7f7248109610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.193+0000 7f724f301700 1 --2- 172.21.15.44:0/559756792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7248102670 0x7f7248102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.193+0000 7f724f301700 1 --2- 172.21.15.44:0/559756792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72481013b0 0x7f7248101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.193+0000 7f724f301700 1 -- 172.21.15.44:0/559756792 >> 172.21.15.44:0/559756792 conn(0x7f72480fbde0 msgr2=0x7f72480fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:31.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 -- 172.21.15.44:0/559756792 shutdown_connections 2024-03-26T08:39:31.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 -- 172.21.15.44:0/559756792 wait complete. 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 Processor -- start 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 -- start start 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72481013b0 0x7f7248105150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7248101d50 0x7f72481037a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248102670 0x7f7248103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724810ccb0 con 0x7f7248101d50 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724810cb30 con 0x7f72481013b0 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.194+0000 7f724f301700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724810ce30 con 0x7f7248102670 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d89e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248102670 0x7f7248103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d89e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248102670 0x7f7248103ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:60600/0 (socket says 172.21.15.44:60600) 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d89e700 1 -- 172.21.15.44:0/2941129833 learned_addr learned my addr 172.21.15.44:0/2941129833 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-03-26T08:39:31.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724c89c700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7248101d50 0x7f72481037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d09d700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72481013b0 0x7f7248105150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d89e700 1 -- 172.21.15.44:0/2941129833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72481013b0 msgr2=0x7f7248105150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d89e700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72481013b0 0x7f7248105150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d89e700 1 -- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7248101d50 msgr2=0x7f72481037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d89e700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7248101d50 0x7f72481037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d89e700 1 -- 172.21.15.44:0/2941129833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7248104590 con 0x7f7248102670 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d09d700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72481013b0 0x7f7248105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724d89e700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248102670 0x7f7248103ce0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7240006070 tx=0x7f7240007d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f723e7fc700 1 -- 172.21.15.44:0/2941129833 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240021020 con 0x7f7248102670 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f723e7fc700 1 -- 172.21.15.44:0/2941129833 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f724001d950 con 0x7f7248102670 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f723e7fc700 1 -- 172.21.15.44:0/2941129833 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240005390 con 0x7f7248102670 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72481a24c0 con 0x7f7248102670 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.195+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72481a2a00 con 0x7f7248102670 2024-03-26T08:39:31.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.196+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7248068e50 con 0x7f7248102670 2024-03-26T08:39:31.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.197+0000 7f723e7fc700 1 -- 172.21.15.44:0/2941129833 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7240024020 con 0x7f7248102670 2024-03-26T08:39:31.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.198+0000 7f723e7fc700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72340a2910 0x7f72340a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:31.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.198+0000 7f723e7fc700 1 -- 172.21.15.44:0/2941129833 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f724000f020 con 0x7f7248102670 2024-03-26T08:39:31.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.198+0000 7f724d09d700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72340a2910 0x7f72340a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:31.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.199+0000 7f724d09d700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72340a2910 0x7f72340a4dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7244005950 tx=0x7f72440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:31.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.204+0000 7f723e7fc700 1 -- 172.21.15.44:0/2941129833 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f724009d500 con 0x7f7248102670 2024-03-26T08:39:31.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.426+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi044=c;smithi044=f;smithi044=i;smithi044=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f72480fe040 con 0x7f72340a2910 2024-03-26T08:39:31.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.433+0000 7f723e7fc700 1 -- 172.21.15.44:0/2941129833 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f72480fe040 con 0x7f72340a2910 2024-03-26T08:39:31.425 INFO:teuthology.orchestra.run.smithi044.stdout:Scheduled mds.all update... 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72340a2910 msgr2=0x7f72340a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72340a2910 0x7f72340a4dd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7244005950 tx=0x7f72440058e0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248102670 msgr2=0x7f7248103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248102670 0x7f7248103ce0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7240006070 tx=0x7f7240007d10 comp rx=0 tx=0).stop 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 shutdown_connections 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72340a2910 0x7f72340a4dd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7248102670 0x7f7248103ce0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72481013b0 0x7f7248105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 --2- 172.21.15.44:0/2941129833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7248101d50 0x7f72481037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 >> 172.21.15.44:0/2941129833 conn(0x7f72480fbde0 msgr2=0x7f72480fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 shutdown_connections 2024-03-26T08:39:31.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-03-26T08:39:31.436+0000 7f724f301700 1 -- 172.21.15.44:0/2941129833 wait complete. 2024-03-26T08:39:31.470 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:31 smithi044 ceph-mon[36461]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:31.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:31 smithi016 ceph-mon[35851]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:31.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:31 smithi007 ceph-mon[32462]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:32.073 DEBUG:teuthology.orchestra.run.smithi007:mds.a> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.a.service 2024-03-26T08:39:32.076 DEBUG:teuthology.orchestra.run.smithi007:mds.d> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.d.service 2024-03-26T08:39:32.118 DEBUG:teuthology.orchestra.run.smithi007:mds.g> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.g.service 2024-03-26T08:39:32.161 DEBUG:teuthology.orchestra.run.smithi007:mds.j> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.j.service 2024-03-26T08:39:32.204 DEBUG:teuthology.orchestra.run.smithi016:mds.b> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.b.service 2024-03-26T08:39:32.207 DEBUG:teuthology.orchestra.run.smithi016:mds.e> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.e.service 2024-03-26T08:39:32.251 DEBUG:teuthology.orchestra.run.smithi016:mds.h> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.h.service 2024-03-26T08:39:32.294 DEBUG:teuthology.orchestra.run.smithi016:mds.k> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.k.service 2024-03-26T08:39:32.338 DEBUG:teuthology.orchestra.run.smithi044:mds.c> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.c.service 2024-03-26T08:39:32.340 DEBUG:teuthology.orchestra.run.smithi044:mds.f> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.f.service 2024-03-26T08:39:32.383 DEBUG:teuthology.orchestra.run.smithi044:mds.i> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.i.service 2024-03-26T08:39:32.425 DEBUG:teuthology.orchestra.run.smithi044:mds.l> sudo journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.l.service 2024-03-26T08:39:32.468 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-26T08:39:32.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:39:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi044=c;smithi044=f;smithi044=i;smithi044=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:39:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi044=c;smithi044=f;smithi044=i;smithi044=l;count:12 2024-03-26T08:39:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:32.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:32.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:32.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:32 smithi016 ceph-mon[35851]: Deploying daemon mds.l on smithi044 2024-03-26T08:39:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.594+0000 7f1586d09640 1 -- 172.21.15.7:0/2326814876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801017c0 msgr2=0x7f1580101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.594+0000 7f1586d09640 1 --2- 172.21.15.7:0/2326814876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801017c0 0x7f1580101c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1570009900 tx=0x7f157002bc30 comp rx=0 tx=0).stop 2024-03-26T08:39:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.594+0000 7f1586d09640 1 -- 172.21.15.7:0/2326814876 shutdown_connections 2024-03-26T08:39:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.594+0000 7f1586d09640 1 --2- 172.21.15.7:0/2326814876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1580102180 0x7f1580108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.594+0000 7f1586d09640 1 --2- 172.21.15.7:0/2326814876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801017c0 0x7f1580101c40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.594+0000 7f1586d09640 1 --2- 172.21.15.7:0/2326814876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15801005d0 0x7f15801009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.594+0000 7f1586d09640 1 -- 172.21.15.7:0/2326814876 >> 172.21.15.7:0/2326814876 conn(0x7f15800fbb40 msgr2=0x7f15800fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.594+0000 7f1586d09640 1 -- 172.21.15.7:0/2326814876 shutdown_connections 2024-03-26T08:39:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.594+0000 7f1586d09640 1 -- 172.21.15.7:0/2326814876 wait complete. 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1586d09640 1 Processor -- start 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1586d09640 1 -- start start 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1586d09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801005d0 0x7f15800694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1586d09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15801017c0 0x7f158019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1586d09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1580102180 0x7f15801a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1586d09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158010c250 con 0x7f15801005d0 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1586d09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158010c0d0 con 0x7f15801017c0 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1586d09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158010c3d0 con 0x7f1580102180 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1584aa5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801005d0 0x7f15800694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1584aa5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801005d0 0x7f15800694e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36422/0 (socket says 172.21.15.7:36422) 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1584aa5640 1 -- 172.21.15.7:0/3868391836 learned_addr learned my addr 172.21.15.7:0/3868391836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f15852a6640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1580102180 0x7f15801a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1584aa5640 1 -- 172.21.15.7:0/3868391836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1580102180 msgr2=0x7f15801a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1584aa5640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1580102180 0x7f15801a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1584aa5640 1 -- 172.21.15.7:0/3868391836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15801017c0 msgr2=0x7f158019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1584aa5640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15801017c0 0x7f158019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1584aa5640 1 -- 172.21.15.7:0/3868391836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15801a1820 con 0x7f15801005d0 2024-03-26T08:39:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.595+0000 7f1584aa5640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801005d0 0x7f15800694e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f157c00c970 tx=0x7f157c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.596+0000 7f1575ffb640 1 -- 172.21.15.7:0/3868391836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c004220 con 0x7f15801005d0 2024-03-26T08:39:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.596+0000 7f1575ffb640 1 -- 172.21.15.7:0/3868391836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f157c0043c0 con 0x7f15801005d0 2024-03-26T08:39:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.596+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15801a1ab0 con 0x7f15801005d0 2024-03-26T08:39:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.596+0000 7f1575ffb640 1 -- 172.21.15.7:0/3868391836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00f450 con 0x7f15801005d0 2024-03-26T08:39:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.596+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15801a2050 con 0x7f15801005d0 2024-03-26T08:39:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.597+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15801078f0 con 0x7f15801005d0 2024-03-26T08:39:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.598+0000 7f1575ffb640 1 -- 172.21.15.7:0/3868391836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f157c0079b0 con 0x7f15801005d0 2024-03-26T08:39:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.598+0000 7f1575ffb640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15680a28b0 0x7f15680a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.598+0000 7f1577fff640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15680a28b0 0x7f15680a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.599+0000 7f1575ffb640 1 -- 172.21.15.7:0/3868391836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f157c0ce3d0 con 0x7f15801005d0 2024-03-26T08:39:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.599+0000 7f1577fff640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15680a28b0 0x7f15680a4d70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1570036040 tx=0x7f1570000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.603+0000 7f1575ffb640 1 -- 172.21.15.7:0/3868391836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f157c014030 con 0x7f15801005d0 2024-03-26T08:39:32.638 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi044=c;smithi044=f;smithi044=i;smithi044=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi044=c;smithi044=f;smithi044=i;smithi044=l;count:12 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:32.639 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:32 smithi044 ceph-mon[36461]: Deploying daemon mds.l on smithi044 2024-03-26T08:39:32.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.787+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15800622a0 con 0x7f15801005d0 2024-03-26T08:39:32.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.788+0000 7f1575ffb640 1 -- 172.21.15.7:0/3868391836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18326 (secure 0 0 0) 0x7f157c09b870 con 0x7f15801005d0 2024-03-26T08:39:32.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:32.780 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:39:20.275865+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:39:32.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15680a28b0 msgr2=0x7f15680a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:32.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15680a28b0 0x7f15680a4d70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1570036040 tx=0x7f1570000bc0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801005d0 msgr2=0x7f15800694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:32.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801005d0 0x7f15800694e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f157c00c970 tx=0x7f157c00ce30 comp rx=0 tx=0).stop 2024-03-26T08:39:32.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 shutdown_connections 2024-03-26T08:39:32.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15680a28b0 0x7f15680a4d70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1580102180 0x7f15801a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15801017c0 0x7f158019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 --2- 172.21.15.7:0/3868391836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15801005d0 0x7f15800694e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 >> 172.21.15.7:0/3868391836 conn(0x7f15800fbb40 msgr2=0x7f1580104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:32.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 shutdown_connections 2024-03-26T08:39:32.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.790+0000 7f1586d09640 1 -- 172.21.15.7:0/3868391836 wait complete. 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi044=c;smithi044=f;smithi044=i;smithi044=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi044=c;smithi044=f;smithi044=i;smithi044=l;count:12 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:32.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:32 smithi007 ceph-mon[32462]: Deploying daemon mds.l on smithi044 2024-03-26T08:39:32.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:39:32.794 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:39:32.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 -- 172.21.15.7:0/35226916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201005b0 msgr2=0x7f84201009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 --2- 172.21.15.7:0/35226916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201005b0 0x7f84201009d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8410009900 tx=0x7f841002bc30 comp rx=0 tx=0).stop 2024-03-26T08:39:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 -- 172.21.15.7:0/35226916 shutdown_connections 2024-03-26T08:39:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 --2- 172.21.15.7:0/35226916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420102160 0x7f84201089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 --2- 172.21.15.7:0/35226916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201017a0 0x7f8420101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 --2- 172.21.15.7:0/35226916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201005b0 0x7f84201009d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 -- 172.21.15.7:0/35226916 >> 172.21.15.7:0/35226916 conn(0x7f84200fbb00 msgr2=0x7f84200fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:32.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 -- 172.21.15.7:0/35226916 shutdown_connections 2024-03-26T08:39:32.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 -- 172.21.15.7:0/35226916 wait complete. 2024-03-26T08:39:32.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.907+0000 7f84251b9640 1 Processor -- start 2024-03-26T08:39:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f84251b9640 1 -- start start 2024-03-26T08:39:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f84251b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201005b0 0x7f842006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f84251b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201017a0 0x7f842019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f84251b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420102160 0x7f842019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f84251b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842010c200 con 0x7f84201017a0 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f84251b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842010c080 con 0x7f84201005b0 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f84251b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842010c380 con 0x7f8420102160 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201017a0 0x7f842019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201017a0 0x7f842019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36452/0 (socket says 172.21.15.7:36452) 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841e59c640 1 -- 172.21.15.7:0/3552659656 learned_addr learned my addr 172.21.15.7:0/3552659656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841f59e640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420102160 0x7f842019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841ed9d640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201005b0 0x7f842006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841e59c640 1 -- 172.21.15.7:0/3552659656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420102160 msgr2=0x7f842019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841e59c640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420102160 0x7f842019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841e59c640 1 -- 172.21.15.7:0/3552659656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201005b0 msgr2=0x7f842006a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841e59c640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201005b0 0x7f842006a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841e59c640 1 -- 172.21.15.7:0/3552659656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842019f640 con 0x7f84201017a0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841ed9d640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201005b0 0x7f842006a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f841e59c640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201017a0 0x7f842019c710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f841400d950 tx=0x7f841400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f8407fff640 1 -- 172.21.15.7:0/3552659656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8414017070 con 0x7f84201017a0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842019f8d0 con 0x7f84201017a0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f8407fff640 1 -- 172.21.15.7:0/3552659656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8414009c30 con 0x7f84201017a0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.908+0000 7f8407fff640 1 -- 172.21.15.7:0/3552659656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841400f6b0 con 0x7f84201017a0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.909+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842019fe70 con 0x7f84201017a0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.909+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84201078d0 con 0x7f84201017a0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.910+0000 7f8407fff640 1 -- 172.21.15.7:0/3552659656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f84140109d0 con 0x7f84201017a0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.911+0000 7f8407fff640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84080a2800 0x7f84080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.911+0000 7f841ed9d640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84080a2800 0x7f84080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.911+0000 7f8407fff640 1 -- 172.21.15.7:0/3552659656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f84140ce230 con 0x7f84201017a0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.911+0000 7f841ed9d640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84080a2800 0x7f84080a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8410036040 tx=0x7f8410000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:32.915+0000 7f8407fff640 1 -- 172.21.15.7:0/3552659656 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8414009de0 con 0x7f84201017a0 2024-03-26T08:39:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.097+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f842002d270 con 0x7f84201017a0 2024-03-26T08:39:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.097+0000 7f8407fff640 1 -- 172.21.15.7:0/3552659656 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f841409b770 con 0x7f84201017a0 2024-03-26T08:39:33.089 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:39:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84080a2800 msgr2=0x7f84080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84080a2800 0x7f84080a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8410036040 tx=0x7f8410000bc0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201017a0 msgr2=0x7f842019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201017a0 0x7f842019c710 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f841400d950 tx=0x7f841400de10 comp rx=0 tx=0).stop 2024-03-26T08:39:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 shutdown_connections 2024-03-26T08:39:33.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84080a2800 0x7f84080a4cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8420102160 0x7f842019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84201017a0 0x7f842019c710 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 --2- 172.21.15.7:0/3552659656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201005b0 0x7f842006a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 >> 172.21.15.7:0/3552659656 conn(0x7f84200fbb00 msgr2=0x7f8420104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:33.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.099+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 shutdown_connections 2024-03-26T08:39:33.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.100+0000 7f84251b9640 1 -- 172.21.15.7:0/3552659656 wait complete. 2024-03-26T08:39:33.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.213+0000 7f09e969a640 1 -- 172.21.15.7:0/981331104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 msgr2=0x7f09e4101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.213+0000 7f09e969a640 1 --2- 172.21.15.7:0/981331104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 0x7f09e4101c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009900 tx=0x7f09d802bc30 comp rx=0 tx=0).stop 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.213+0000 7f09e969a640 1 -- 172.21.15.7:0/981331104 shutdown_connections 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.213+0000 7f09e969a640 1 --2- 172.21.15.7:0/981331104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e4102180 0x7f09e4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.213+0000 7f09e969a640 1 --2- 172.21.15.7:0/981331104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 0x7f09e4101c40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.213+0000 7f09e969a640 1 --2- 172.21.15.7:0/981331104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e41005d0 0x7f09e41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.213+0000 7f09e969a640 1 -- 172.21.15.7:0/981331104 >> 172.21.15.7:0/981331104 conn(0x7f09e40fbb60 msgr2=0x7f09e40fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.213+0000 7f09e969a640 1 -- 172.21.15.7:0/981331104 shutdown_connections 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.213+0000 7f09e969a640 1 -- 172.21.15.7:0/981331104 wait complete. 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e969a640 1 Processor -- start 2024-03-26T08:39:33.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e969a640 1 -- start start 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e969a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e41005d0 0x7f09e406a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e969a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 0x7f09e419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e969a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e4102180 0x7f09e419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e969a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e410c230 con 0x7f09e41017c0 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e969a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e410c0b0 con 0x7f09e4102180 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e969a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e410c3b0 con 0x7f09e41005d0 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 0x7f09e419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 0x7f09e419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36464/0 (socket says 172.21.15.7:36464) 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e27fc640 1 -- 172.21.15.7:0/943718466 learned_addr learned my addr 172.21.15.7:0/943718466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.214+0000 7f09e2ffd640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e41005d0 0x7f09e406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e37fe640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e4102180 0x7f09e419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e27fc640 1 -- 172.21.15.7:0/943718466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e41005d0 msgr2=0x7f09e406a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e27fc640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e41005d0 0x7f09e406a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e27fc640 1 -- 172.21.15.7:0/943718466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e4102180 msgr2=0x7f09e419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e27fc640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e4102180 0x7f09e419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e27fc640 1 -- 172.21.15.7:0/943718466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e419f5f0 con 0x7f09e41017c0 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e37fe640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e4102180 0x7f09e419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e27fc640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 0x7f09e419c6c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009230 tx=0x7f09d8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09cbfff640 1 -- 172.21.15.7:0/943718466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8041070 con 0x7f09e41017c0 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e419f880 con 0x7f09e41017c0 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09cbfff640 1 -- 172.21.15.7:0/943718466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09d80054e0 con 0x7f09e41017c0 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09cbfff640 1 -- 172.21.15.7:0/943718466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8034440 con 0x7f09e41017c0 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.215+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e419fd70 con 0x7f09e41017c0 2024-03-26T08:39:33.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.216+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e41078f0 con 0x7f09e41017c0 2024-03-26T08:39:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.217+0000 7f09cbfff640 1 -- 172.21.15.7:0/943718466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f09d8044070 con 0x7f09e41017c0 2024-03-26T08:39:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.218+0000 7f09cbfff640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09cc0a2800 0x7f09cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.218+0000 7f09e2ffd640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09cc0a2800 0x7f09cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.218+0000 7f09cbfff640 1 -- 172.21.15.7:0/943718466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09d80f0360 con 0x7f09e41017c0 2024-03-26T08:39:33.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.218+0000 7f09e2ffd640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09cc0a2800 0x7f09cc0a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f09e4101620 tx=0x7f09dc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:33.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.222+0000 7f09cbfff640 1 -- 172.21.15.7:0/943718466 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f09d80bd800 con 0x7f09e41017c0 2024-03-26T08:39:33.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.448+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09e402d070 con 0x7f09e41017c0 2024-03-26T08:39:33.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.449+0000 7f09cbfff640 1 -- 172.21.15.7:0/943718466 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f09d80bd100 con 0x7f09e41017c0 2024-03-26T08:39:33.441 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:33.441 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-26T08:39:33.441 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 1 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09cc0a2800 msgr2=0x7f09cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09cc0a2800 0x7f09cc0a4cc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f09e4101620 tx=0x7f09dc00b390 comp rx=0 tx=0).stop 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 msgr2=0x7f09e419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 0x7f09e419c6c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009230 tx=0x7f09d8005dd0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 shutdown_connections 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09cc0a2800 0x7f09cc0a4cc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09e4102180 0x7f09e419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e41017c0 0x7f09e419c6c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 --2- 172.21.15.7:0/943718466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09e41005d0 0x7f09e406a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 >> 172.21.15.7:0/943718466 conn(0x7f09e40fbb60 msgr2=0x7f09e4104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:33.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 shutdown_connections 2024-03-26T08:39:33.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:33.451+0000 7f09e969a640 1 -- 172.21.15.7:0/943718466 wait complete. 2024-03-26T08:39:33.562 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:33 smithi044 ceph-mon[36461]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:33.562 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:33 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3868391836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:39:33.562 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:33 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3552659656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:39:33.562 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:33 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/943718466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:33.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:33 smithi016 ceph-mon[35851]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:33.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:33 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3868391836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:39:33.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:33 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3552659656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:39:33.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:33 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/943718466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:33.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:33 smithi007 ceph-mon[32462]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:33.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:33 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3868391836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:39:33.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:33 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3552659656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:39:33.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:33 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/943718466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:33.815 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 08:39:33 smithi044 systemd[1]: Starting Ceph mds.l for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:39:34.223 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 08:39:33 smithi044 podman[66709]: 2024-03-26 08:39:33.880112295 +0000 UTC m=+0.018520390 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:39:34.224 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 08:39:34 smithi044 podman[66709]: 2024-03-26 08:39:34.005032058 +0000 UTC m=+0.143440142 container create 3cc654354be32b3d3dc4c40943e75b821f7bbffb190517ccffc3d0b9f10d80b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:39:34.224 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 08:39:34 smithi044 podman[66709]: 2024-03-26 08:39:34.232583173 +0000 UTC m=+0.370991258 container init 3cc654354be32b3d3dc4c40943e75b821f7bbffb190517ccffc3d0b9f10d80b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-26T08:39:34.566 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 08:39:34 smithi044 podman[66709]: 2024-03-26 08:39:34.235856132 +0000 UTC m=+0.374264232 container start 3cc654354be32b3d3dc4c40943e75b821f7bbffb190517ccffc3d0b9f10d80b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True) 2024-03-26T08:39:34.566 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 08:39:34 smithi044 ceph-mds[66728]: -- 172.21.15.44:0/642894426 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55efd52dda40 con 0x55efd52b0800 2024-03-26T08:39:34.566 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 08:39:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l[66724]: starting mds.l at 2024-03-26T08:39:34.566 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 08:39:34 smithi044 bash[66709]: 3cc654354be32b3d3dc4c40943e75b821f7bbffb190517ccffc3d0b9f10d80b5 2024-03-26T08:39:34.566 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 08:39:34 smithi044 systemd[1]: Started Ceph mds.l for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:39:35.367 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:35 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.44:6832/3400459626,v1:172.21.15.44:6833/3400459626] up:boot 2024-03-26T08:39:35.368 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:35 smithi016 ceph-mon[35851]: fsmap 1 up:standby 2024-03-26T08:39:35.368 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T08:39:35.368 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:35.368 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:35.368 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:35.368 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:35.368 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:35.368 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:35.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:35.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:35 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.44:6832/3400459626,v1:172.21.15.44:6833/3400459626] up:boot 2024-03-26T08:39:35.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:35 smithi007 ceph-mon[32462]: fsmap 1 up:standby 2024-03-26T08:39:35.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T08:39:35.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:35.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:35.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:35.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:35.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:35.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:35 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.44:6832/3400459626,v1:172.21.15.44:6833/3400459626] up:boot 2024-03-26T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:35 smithi044 ceph-mon[36461]: fsmap 1 up:standby 2024-03-26T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:35.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:35.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 -- 172.21.15.7:0/3782084896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 msgr2=0x7f17e01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:35.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/3782084896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 0x7f17e01009f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f17d8009900 tx=0x7f17d802bc30 comp rx=0 tx=0).stop 2024-03-26T08:39:35.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 -- 172.21.15.7:0/3782084896 shutdown_connections 2024-03-26T08:39:35.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/3782084896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e0102180 0x7f17e0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/3782084896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e01017c0 0x7f17e0101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/3782084896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 0x7f17e01009f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 -- 172.21.15.7:0/3782084896 >> 172.21.15.7:0/3782084896 conn(0x7f17e00fbb40 msgr2=0x7f17e00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:35.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 -- 172.21.15.7:0/3782084896 shutdown_connections 2024-03-26T08:39:35.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 -- 172.21.15.7:0/3782084896 wait complete. 2024-03-26T08:39:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 Processor -- start 2024-03-26T08:39:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 -- start start 2024-03-26T08:39:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.578+0000 7f17e7f1d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 0x7f17e00694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e7f1d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e01017c0 0x7f17e019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e7f1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e0102180 0x7f17e019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e7f1d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e010c2a0 con 0x7f17e01005d0 2024-03-26T08:39:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e7f1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e010c120 con 0x7f17e0102180 2024-03-26T08:39:35.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e7f1d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e010c420 con 0x7f17e01017c0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e5cb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 0x7f17e00694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e5cb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 0x7f17e00694e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36482/0 (socket says 172.21.15.7:36482) 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e5cb9640 1 -- 172.21.15.7:0/309737513 learned_addr learned my addr 172.21.15.7:0/309737513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e54b8640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e01017c0 0x7f17e019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e64ba640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e0102180 0x7f17e019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e5cb9640 1 -- 172.21.15.7:0/309737513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e01017c0 msgr2=0x7f17e019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e5cb9640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e01017c0 0x7f17e019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e5cb9640 1 -- 172.21.15.7:0/309737513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e0102180 msgr2=0x7f17e019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e5cb9640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e0102180 0x7f17e019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e5cb9640 1 -- 172.21.15.7:0/309737513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e019f640 con 0x7f17e01005d0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e5cb9640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 0x7f17e00694e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f17d8000c00 tx=0x7f17d8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17deffd640 1 -- 172.21.15.7:0/309737513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8043020 con 0x7f17e01005d0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e019f870 con 0x7f17e01005d0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17deffd640 1 -- 172.21.15.7:0/309737513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17d80054e0 con 0x7f17e01005d0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17deffd640 1 -- 172.21.15.7:0/309737513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d8003e60 con 0x7f17e01005d0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.579+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e019fdc0 con 0x7f17e01005d0 2024-03-26T08:39:35.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.580+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e0068520 con 0x7f17e01005d0 2024-03-26T08:39:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.581+0000 7f17deffd640 1 -- 172.21.15.7:0/309737513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f17d8046020 con 0x7f17e01005d0 2024-03-26T08:39:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.582+0000 7f17deffd640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f17c80a2800 0x7f17c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.582+0000 7f17e54b8640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f17c80a2800 0x7f17c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:35.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.582+0000 7f17deffd640 1 -- 172.21.15.7:0/309737513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f17d80f07b0 con 0x7f17e01005d0 2024-03-26T08:39:35.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.583+0000 7f17e54b8640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f17c80a2800 0x7f17c80a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f17e0069250 tx=0x7f17d400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:35.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.586+0000 7f17deffd640 1 -- 172.21.15.7:0/309737513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f17d804d020 con 0x7f17e01005d0 2024-03-26T08:39:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.809+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17e01078f0 con 0x7f17e01005d0 2024-03-26T08:39:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.810+0000 7f17deffd640 1 -- 172.21.15.7:0/309737513 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f17d80bdc50 con 0x7f17e01005d0 2024-03-26T08:39:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:35.802 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T08:39:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-03-26T08:39:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f17c80a2800 msgr2=0x7f17c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f17c80a2800 0x7f17c80a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f17e0069250 tx=0x7f17d400b390 comp rx=0 tx=0).stop 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 msgr2=0x7f17e00694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 0x7f17e00694e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f17d8000c00 tx=0x7f17d8005dd0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 shutdown_connections 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f17c80a2800 0x7f17c80a4cc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e0102180 0x7f17e019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f17e01017c0 0x7f17e019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 --2- 172.21.15.7:0/309737513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17e01005d0 0x7f17e00694e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 >> 172.21.15.7:0/309737513 conn(0x7f17e00fbb40 msgr2=0x7f17e0104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 shutdown_connections 2024-03-26T08:39:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:35.812+0000 7f17e7f1d640 1 -- 172.21.15.7:0/309737513 wait complete. 2024-03-26T08:39:36.376 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:36 smithi016 ceph-mon[35851]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:36.376 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:36 smithi016 ceph-mon[35851]: Deploying daemon mds.k on smithi016 2024-03-26T08:39:36.376 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:36 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/309737513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:36.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:36 smithi007 ceph-mon[32462]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:36.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:36 smithi007 ceph-mon[32462]: Deploying daemon mds.k on smithi016 2024-03-26T08:39:36.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:36 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/309737513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:36.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:36 smithi044 ceph-mon[36461]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:36.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:36 smithi044 ceph-mon[36461]: Deploying daemon mds.k on smithi016 2024-03-26T08:39:36.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:36 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/309737513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:36.744 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 08:39:36 smithi016 systemd[1]: Starting Ceph mds.k for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:39:37.052 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 08:39:36 smithi016 podman[65268]: 2024-03-26 08:39:36.686746841 +0000 UTC m=+0.018329133 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:39:37.052 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 08:39:36 smithi016 podman[65268]: 2024-03-26 08:39:36.800091462 +0000 UTC m=+0.131673781 container create fcec085315304577cd4263d34564370219c1ee26e41b02270a5d01bdd7caa5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:39:37.052 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 08:39:37 smithi016 podman[65268]: 2024-03-26 08:39:37.061142735 +0000 UTC m=+0.392725015 container init fcec085315304577cd4263d34564370219c1ee26e41b02270a5d01bdd7caa5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T08:39:37.493 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 08:39:37 smithi016 podman[65268]: 2024-03-26 08:39:37.06426417 +0000 UTC m=+0.395846469 container start fcec085315304577cd4263d34564370219c1ee26e41b02270a5d01bdd7caa5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-03-26T08:39:37.493 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 08:39:37 smithi016 ceph-mds[65287]: -- 172.21.15.16:0/1339950110 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d3882f1a40 con 0x55d3882c5000 2024-03-26T08:39:37.493 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 08:39:37 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k[65283]: starting mds.k at 2024-03-26T08:39:37.493 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 08:39:37 smithi016 bash[65268]: fcec085315304577cd4263d34564370219c1ee26e41b02270a5d01bdd7caa5e2 2024-03-26T08:39:37.494 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 08:39:37 smithi016 systemd[1]: Started Ceph mds.k for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:39:37.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:37.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.928+0000 7fc1278dc640 1 -- 172.21.15.7:0/1694104013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc120102190 msgr2=0x7fc120108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:37.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.928+0000 7fc1278dc640 1 --2- 172.21.15.7:0/1694104013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc120102190 0x7fc120108a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc118009ca0 tx=0x7fc11802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:39:37.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.928+0000 7fc1278dc640 1 -- 172.21.15.7:0/1694104013 shutdown_connections 2024-03-26T08:39:37.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.928+0000 7fc1278dc640 1 --2- 172.21.15.7:0/1694104013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc120102190 0x7fc120108a20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.928+0000 7fc1278dc640 1 --2- 172.21.15.7:0/1694104013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1201017d0 0x7fc120101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.928+0000 7fc1278dc640 1 --2- 172.21.15.7:0/1694104013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1201005e0 0x7fc120100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.928+0000 7fc1278dc640 1 -- 172.21.15.7:0/1694104013 >> 172.21.15.7:0/1694104013 conn(0x7fc1200fbb30 msgr2=0x7fc1200fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.928+0000 7fc1278dc640 1 -- 172.21.15.7:0/1694104013 shutdown_connections 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.928+0000 7fc1278dc640 1 -- 172.21.15.7:0/1694104013 wait complete. 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1278dc640 1 Processor -- start 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1278dc640 1 -- start start 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1278dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1201005e0 0x7fc120104570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1278dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1201017d0 0x7fc120102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1278dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc120102190 0x7fc1201030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1278dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12010c260 con 0x7fc1201005e0 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1278dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12010c0e0 con 0x7fc1201017d0 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1278dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12010c3e0 con 0x7fc120102190 2024-03-26T08:39:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc125678640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1201005e0 0x7fc120104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc125678640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1201005e0 0x7fc120104570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44392/0 (socket says 172.21.15.7:44392) 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc125678640 1 -- 172.21.15.7:0/613163510 learned_addr learned my addr 172.21.15.7:0/613163510 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc124e77640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1201017d0 0x7fc120102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc125678640 1 -- 172.21.15.7:0/613163510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc120102190 msgr2=0x7fc1201030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc125678640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc120102190 0x7fc1201030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc125678640 1 -- 172.21.15.7:0/613163510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1201017d0 msgr2=0x7fc120102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc125678640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1201017d0 0x7fc120102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc125678640 1 -- 172.21.15.7:0/613163510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc120103980 con 0x7fc1201005e0 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc125678640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1201005e0 0x7fc120104570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00d770 tx=0x7fc11c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1167fc640 1 -- 172.21.15.7:0/613163510 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c017070 con 0x7fc1201005e0 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1167fc640 1 -- 172.21.15.7:0/613163510 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc11c009c30 con 0x7fc1201005e0 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1201a1a70 con 0x7fc1201005e0 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.929+0000 7fc1167fc640 1 -- 172.21.15.7:0/613163510 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c003dd0 con 0x7fc1201005e0 2024-03-26T08:39:37.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.930+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1201a2010 con 0x7fc1201005e0 2024-03-26T08:39:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.931+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1201a21e0 con 0x7fc1201005e0 2024-03-26T08:39:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.932+0000 7fc1167fc640 1 -- 172.21.15.7:0/613163510 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc11c003fb0 con 0x7fc1201005e0 2024-03-26T08:39:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.932+0000 7fc1167fc640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc10c0a2880 0x7fc10c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.932+0000 7fc124e77640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc10c0a2880 0x7fc10c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.933+0000 7fc1167fc640 1 -- 172.21.15.7:0/613163510 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc11c0cdf90 con 0x7fc1201005e0 2024-03-26T08:39:37.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.933+0000 7fc124e77640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc10c0a2880 0x7fc10c0a4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc110005fd0 tx=0x7fc110005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:37.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:37.938+0000 7fc1167fc640 1 -- 172.21.15.7:0/613163510 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc11c009de0 con 0x7fc1201005e0 2024-03-26T08:39:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.163+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1201a23b0 con 0x7fc1201005e0 2024-03-26T08:39:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.164+0000 7fc1167fc640 1 -- 172.21.15.7:0/613163510 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2110 (secure 0 0 0) 0x7fc11c09b4d0 con 0x7fc1201005e0 2024-03-26T08:39:38.156 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:38.156 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T08:39:38.156 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-03-26T08:39:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc10c0a2880 msgr2=0x7fc10c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc10c0a2880 0x7fc10c0a4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc110005fd0 tx=0x7fc110005850 comp rx=0 tx=0).stop 2024-03-26T08:39:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1201005e0 msgr2=0x7fc120104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1201005e0 0x7fc120104570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00d770 tx=0x7fc11c00dc30 comp rx=0 tx=0).stop 2024-03-26T08:39:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 shutdown_connections 2024-03-26T08:39:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc10c0a2880 0x7fc10c0a4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc120102190 0x7fc1201030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1201017d0 0x7fc120102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:38.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 --2- 172.21.15.7:0/613163510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1201005e0 0x7fc120104570 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 >> 172.21.15.7:0/613163510 conn(0x7fc1200fbb30 msgr2=0x7fc1200fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 shutdown_connections 2024-03-26T08:39:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:38.166+0000 7fc1278dc640 1 -- 172.21.15.7:0/613163510 wait complete. 2024-03-26T08:39:38.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:38.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:38.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:38.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:38.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:38.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:38.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.16:6832/76809036,v1:172.21.15.16:6833/76809036] up:boot 2024-03-26T08:39:38.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: fsmap 2 up:standby 2024-03-26T08:39:38.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T08:39:38.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:38.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:38 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/613163510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:38.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:38.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:38.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:38.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:38.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:38.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:38.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.16:6832/76809036,v1:172.21.15.16:6833/76809036] up:boot 2024-03-26T08:39:38.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: fsmap 2 up:standby 2024-03-26T08:39:38.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T08:39:38.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:38.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:38 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/613163510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.16:6832/76809036,v1:172.21.15.16:6833/76809036] up:boot 2024-03-26T08:39:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: fsmap 2 up:standby 2024-03-26T08:39:38.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T08:39:38.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:38.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:38 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/613163510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:39.476 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:39 smithi016 ceph-mon[35851]: Deploying daemon mds.j on smithi007 2024-03-26T08:39:39.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:39 smithi007 ceph-mon[32462]: Deploying daemon mds.j on smithi007 2024-03-26T08:39:39.542 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 08:39:39 smithi007 systemd[1]: Starting Ceph mds.j for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:39:39.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:39 smithi044 ceph-mon[36461]: Deploying daemon mds.j on smithi007 2024-03-26T08:39:39.941 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 08:39:39 smithi007 podman[70349]: 2024-03-26 08:39:39.578072389 +0000 UTC m=+0.017502924 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:39:39.941 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 08:39:39 smithi007 podman[70349]: 2024-03-26 08:39:39.680600079 +0000 UTC m=+0.120030655 container create 91d9c52035e81bf66c2b1f5853e34323143e67ef1ea1e35f83d8961c3734e773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T08:39:39.941 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 08:39:39 smithi007 podman[70349]: 2024-03-26 08:39:39.950230541 +0000 UTC m=+0.389661078 container init 91d9c52035e81bf66c2b1f5853e34323143e67ef1ea1e35f83d8961c3734e773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T08:39:40.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:40.194 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 08:39:39 smithi007 podman[70349]: 2024-03-26 08:39:39.952925811 +0000 UTC m=+0.392356354 container start 91d9c52035e81bf66c2b1f5853e34323143e67ef1ea1e35f83d8961c3734e773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-26T08:39:40.195 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 08:39:39 smithi007 ceph-mds[70368]: -- 172.21.15.7:0/3194550947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653615eba40 con 0x5653615be800 2024-03-26T08:39:40.195 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 08:39:39 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j[70364]: starting mds.j at 2024-03-26T08:39:40.195 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 08:39:40 smithi007 bash[70349]: 91d9c52035e81bf66c2b1f5853e34323143e67ef1ea1e35f83d8961c3734e773 2024-03-26T08:39:40.195 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 08:39:40 smithi007 systemd[1]: Started Ceph mds.j for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:39:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.287+0000 7fe56cdda640 1 -- 172.21.15.7:0/1558522126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe568107410 msgr2=0x7fe568109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.287+0000 7fe56cdda640 1 --2- 172.21.15.7:0/1558522126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe568107410 0x7fe568109800 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe560009ca0 tx=0x7fe56002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.287+0000 7fe56cdda640 1 -- 172.21.15.7:0/1558522126 shutdown_connections 2024-03-26T08:39:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.287+0000 7fe56cdda640 1 --2- 172.21.15.7:0/1558522126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe568107410 0x7fe568109800 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.287+0000 7fe56cdda640 1 --2- 172.21.15.7:0/1558522126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568104ae0 0x7fe568106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.287+0000 7fe56cdda640 1 --2- 172.21.15.7:0/1558522126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5681021b0 0x7fe5681045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.287+0000 7fe56cdda640 1 -- 172.21.15.7:0/1558522126 >> 172.21.15.7:0/1558522126 conn(0x7fe5680fbb40 msgr2=0x7fe5680fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.287+0000 7fe56cdda640 1 -- 172.21.15.7:0/1558522126 shutdown_connections 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.287+0000 7fe56cdda640 1 -- 172.21.15.7:0/1558522126 wait complete. 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56cdda640 1 Processor -- start 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56cdda640 1 -- start start 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56cdda640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5681021b0 0x7fe568101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56cdda640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568104ae0 0x7fe568100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56cdda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568107410 0x7fe5681006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56cdda640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56810d3c0 con 0x7fe5681021b0 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56cdda640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56810d240 con 0x7fe568107410 2024-03-26T08:39:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56cdda640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56810d540 con 0x7fe568104ae0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5681021b0 0x7fe568101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5681021b0 0x7fe568101b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44428/0 (socket says 172.21.15.7:44428) 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.288+0000 7fe56659c640 1 -- 172.21.15.7:0/3453722695 learned_addr learned my addr 172.21.15.7:0/3453722695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe566d9d640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568107410 0x7fe5681006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe55ffff640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568104ae0 0x7fe568100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe56659c640 1 -- 172.21.15.7:0/3453722695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568104ae0 msgr2=0x7fe568100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe56659c640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568104ae0 0x7fe568100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe56659c640 1 -- 172.21.15.7:0/3453722695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568107410 msgr2=0x7fe5681006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe56659c640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568107410 0x7fe5681006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe56659c640 1 -- 172.21.15.7:0/3453722695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568100f30 con 0x7fe5681021b0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe56659c640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5681021b0 0x7fe568101b20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe55800d950 tx=0x7fe55800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe55f7fe640 1 -- 172.21.15.7:0/3453722695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558017070 con 0x7fe5681021b0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe55f7fe640 1 -- 172.21.15.7:0/3453722695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe558009c30 con 0x7fe5681021b0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5681a1af0 con 0x7fe5681021b0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe55f7fe640 1 -- 172.21.15.7:0/3453722695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800f6b0 con 0x7fe5681021b0 2024-03-26T08:39:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.289+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5681a2040 con 0x7fe5681021b0 2024-03-26T08:39:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.290+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5680fd630 con 0x7fe5681021b0 2024-03-26T08:39:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.292+0000 7fe55f7fe640 1 -- 172.21.15.7:0/3453722695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe5580109d0 con 0x7fe5681021b0 2024-03-26T08:39:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.293+0000 7fe55f7fe640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5540a2800 0x7fe5540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.293+0000 7fe55ffff640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5540a2800 0x7fe5540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.293+0000 7fe55f7fe640 1 -- 172.21.15.7:0/3453722695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe5580ce1c0 con 0x7fe5681021b0 2024-03-26T08:39:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.293+0000 7fe55ffff640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5540a2800 0x7fe5540a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe568101890 tx=0x7fe550008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.296+0000 7fe55f7fe640 1 -- 172.21.15.7:0/3453722695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe558009de0 con 0x7fe5681021b0 2024-03-26T08:39:40.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:40 smithi016 ceph-mon[35851]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:40.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:40.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:40.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:40.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:40.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:40.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.519+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe56802d2e0 con 0x7fe5681021b0 2024-03-26T08:39:40.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.520+0000 7fe55f7fe640 1 -- 172.21.15.7:0/3453722695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2898 (secure 0 0 0) 0x7fe55809b700 con 0x7fe5681021b0 2024-03-26T08:39:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T08:39:40.512 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 4 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5540a2800 msgr2=0x7fe5540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5540a2800 0x7fe5540a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe568101890 tx=0x7fe550008040 comp rx=0 tx=0).stop 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5681021b0 msgr2=0x7fe568101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5681021b0 0x7fe568101b20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe55800d950 tx=0x7fe55800de10 comp rx=0 tx=0).stop 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 shutdown_connections 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5540a2800 0x7fe5540a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe568107410 0x7fe5681006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe568104ae0 0x7fe568100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 --2- 172.21.15.7:0/3453722695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5681021b0 0x7fe568101b20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 >> 172.21.15.7:0/3453722695 conn(0x7fe5680fbb40 msgr2=0x7fe568102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 shutdown_connections 2024-03-26T08:39:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:40.522+0000 7fe56cdda640 1 -- 172.21.15.7:0/3453722695 wait complete. 2024-03-26T08:39:40.523 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:40 smithi007 ceph-mon[32462]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:40.523 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:40.523 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:40.523 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:40.523 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:40.524 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:40.524 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:40.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:40 smithi044 ceph-mon[36461]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:40.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:40.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:40.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:40.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:40.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:40.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:41.490 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:41 smithi044 ceph-mon[36461]: Deploying daemon mds.d on smithi007 2024-03-26T08:39:41.491 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:41 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.7:6834/2764125554,v1:172.21.15.7:6835/2764125554] up:boot 2024-03-26T08:39:41.491 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:41 smithi044 ceph-mon[36461]: fsmap 3 up:standby 2024-03-26T08:39:41.491 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T08:39:41.491 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:41 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3453722695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:41.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:41 smithi016 ceph-mon[35851]: Deploying daemon mds.d on smithi007 2024-03-26T08:39:41.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:41 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.7:6834/2764125554,v1:172.21.15.7:6835/2764125554] up:boot 2024-03-26T08:39:41.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:41 smithi016 ceph-mon[35851]: fsmap 3 up:standby 2024-03-26T08:39:41.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T08:39:41.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:41 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3453722695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:41.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:41 smithi007 ceph-mon[32462]: Deploying daemon mds.d on smithi007 2024-03-26T08:39:41.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:41 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.7:6834/2764125554,v1:172.21.15.7:6835/2764125554] up:boot 2024-03-26T08:39:41.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:41 smithi007 ceph-mon[32462]: fsmap 3 up:standby 2024-03-26T08:39:41.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T08:39:41.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:41 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3453722695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:42.235 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 08:39:42 smithi007 systemd[1]: Starting Ceph mds.d for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:39:42.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:42 smithi016 ceph-mon[35851]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:42.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:42 smithi007 ceph-mon[32462]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:42.494 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 08:39:42 smithi007 podman[70697]: 2024-03-26 08:39:42.23914621 +0000 UTC m=+0.017402782 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:39:42.494 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 08:39:42 smithi007 podman[70697]: 2024-03-26 08:39:42.361997004 +0000 UTC m=+0.140253546 container create eb1102f0da13a665ef7eee105a4e06b052e592df2fced68ee5079ab56d595a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-26T08:39:42.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:42.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:42 smithi044 ceph-mon[36461]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 -- 172.21.15.7:0/2272539272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e81021f0 msgr2=0x7f40e81045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 --2- 172.21.15.7:0/2272539272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e81021f0 0x7f40e81045e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009900 tx=0x7f40d802bc30 comp rx=0 tx=0).stop 2024-03-26T08:39:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 -- 172.21.15.7:0/2272539272 shutdown_connections 2024-03-26T08:39:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 --2- 172.21.15.7:0/2272539272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e8107450 0x7f40e8109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 --2- 172.21.15.7:0/2272539272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8104b20 0x7f40e8106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 --2- 172.21.15.7:0/2272539272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e81021f0 0x7f40e81045e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 -- 172.21.15.7:0/2272539272 >> 172.21.15.7:0/2272539272 conn(0x7f40e80fbb40 msgr2=0x7f40e80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 -- 172.21.15.7:0/2272539272 shutdown_connections 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 -- 172.21.15.7:0/2272539272 wait complete. 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 Processor -- start 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.642+0000 7f40effc6640 1 -- start start 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40effc6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e81021f0 0x7f40e8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40effc6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e8104b20 0x7f40e8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40effc6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e8107450 0x7f40e819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40effc6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e810d390 con 0x7f40e8107450 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40effc6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e810d210 con 0x7f40e81021f0 2024-03-26T08:39:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40effc6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e810d510 con 0x7f40e8104b20 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ee563640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e8107450 0x7f40e819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ee563640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e8107450 0x7f40e819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44456/0 (socket says 172.21.15.7:44456) 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ee563640 1 -- 172.21.15.7:0/1188869038 learned_addr learned my addr 172.21.15.7:0/1188869038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40edd62640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e81021f0 0x7f40e8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ed561640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e8104b20 0x7f40e8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ee563640 1 -- 172.21.15.7:0/1188869038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e8104b20 msgr2=0x7f40e8197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ee563640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e8104b20 0x7f40e8197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ee563640 1 -- 172.21.15.7:0/1188869038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e81021f0 msgr2=0x7f40e8197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ee563640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e81021f0 0x7f40e8197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ee563640 1 -- 172.21.15.7:0/1188869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e819f570 con 0x7f40e8107450 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40ee563640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e8107450 0x7f40e819ef00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f40e000eaa0 tx=0x7f40e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40deffd640 1 -- 172.21.15.7:0/1188869038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000cdb0 con 0x7f40e8107450 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e819f860 con 0x7f40e8107450 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40deffd640 1 -- 172.21.15.7:0/1188869038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40e0005020 con 0x7f40e8107450 2024-03-26T08:39:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40deffd640 1 -- 172.21.15.7:0/1188869038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e00116b0 con 0x7f40e8107450 2024-03-26T08:39:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.643+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e819fdb0 con 0x7f40e8107450 2024-03-26T08:39:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.644+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e80fd630 con 0x7f40e8107450 2024-03-26T08:39:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.645+0000 7f40deffd640 1 -- 172.21.15.7:0/1188869038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f40e0005560 con 0x7f40e8107450 2024-03-26T08:39:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.646+0000 7f40deffd640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40d40a2800 0x7f40d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.646+0000 7f40edd62640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40d40a2800 0x7f40d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.646+0000 7f40deffd640 1 -- 172.21.15.7:0/1188869038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f40e00ce970 con 0x7f40e8107450 2024-03-26T08:39:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.646+0000 7f40edd62640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40d40a2800 0x7f40d40a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f40d802c240 tx=0x7f40d802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.650+0000 7f40deffd640 1 -- 172.21.15.7:0/1188869038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f40e0016030 con 0x7f40e8107450 2024-03-26T08:39:42.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.904+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40e802d450 con 0x7f40e8107450 2024-03-26T08:39:42.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.904+0000 7f40deffd640 1 -- 172.21.15.7:0/1188869038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2898 (secure 0 0 0) 0x7f40e802d450 con 0x7f40e8107450 2024-03-26T08:39:42.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:42.896 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T08:39:42.896 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 4 2024-03-26T08:39:42.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40d40a2800 msgr2=0x7f40d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:42.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40d40a2800 0x7f40d40a4cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f40d802c240 tx=0x7f40d802df90 comp rx=0 tx=0).stop 2024-03-26T08:39:42.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e8107450 msgr2=0x7f40e819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:42.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e8107450 0x7f40e819ef00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f40e000eaa0 tx=0x7f40e000ef60 comp rx=0 tx=0).stop 2024-03-26T08:39:42.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 shutdown_connections 2024-03-26T08:39:42.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40d40a2800 0x7f40d40a4cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:42.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e8107450 0x7f40e819ef00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:42.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40e8104b20 0x7f40e8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:42.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 --2- 172.21.15.7:0/1188869038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e81021f0 0x7f40e8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:42.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 >> 172.21.15.7:0/1188869038 conn(0x7f40e80fbb40 msgr2=0x7f40e8102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:42.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 shutdown_connections 2024-03-26T08:39:42.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:42.906+0000 7f40effc6640 1 -- 172.21.15.7:0/1188869038 wait complete. 2024-03-26T08:39:42.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 08:39:42 smithi007 podman[70697]: 2024-03-26 08:39:42.607252458 +0000 UTC m=+0.385509014 container init eb1102f0da13a665ef7eee105a4e06b052e592df2fced68ee5079ab56d595a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-26T08:39:42.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 08:39:42 smithi007 podman[70697]: 2024-03-26 08:39:42.610240053 +0000 UTC m=+0.388496595 container start eb1102f0da13a665ef7eee105a4e06b052e592df2fced68ee5079ab56d595a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:39:42.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 08:39:42 smithi007 ceph-mds[70736]: -- 172.21.15.7:0/2117430214 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562107007a40 con 0x562106fdac00 2024-03-26T08:39:42.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 08:39:42 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d[70712]: starting mds.d at 2024-03-26T08:39:42.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 08:39:42 smithi007 bash[70697]: eb1102f0da13a665ef7eee105a4e06b052e592df2fced68ee5079ab56d595a72 2024-03-26T08:39:42.908 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 08:39:42 smithi007 systemd[1]: Started Ceph mds.d for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:39:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: Deploying daemon mds.e on smithi016 2024-03-26T08:39:43.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1188869038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:43.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.7:6836/1591045564,v1:172.21.15.7:6837/1591045564] up:boot 2024-03-26T08:39:43.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: fsmap 4 up:standby 2024-03-26T08:39:43.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T08:39:44.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:44.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:44.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:44.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:44.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:44.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:44.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:44.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: Deploying daemon mds.e on smithi016 2024-03-26T08:39:44.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1188869038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:44.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.7:6836/1591045564,v1:172.21.15.7:6837/1591045564] up:boot 2024-03-26T08:39:44.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: fsmap 4 up:standby 2024-03-26T08:39:44.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T08:39:44.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:44.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:44.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:44.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:44.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: Deploying daemon mds.e on smithi016 2024-03-26T08:39:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1188869038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.7:6836/1591045564,v1:172.21.15.7:6837/1591045564] up:boot 2024-03-26T08:39:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: fsmap 4 up:standby 2024-03-26T08:39:44.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T08:39:44.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.034+0000 7ff6896b2640 1 -- 172.21.15.7:0/1703753260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841017d0 msgr2=0x7ff684101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.034+0000 7ff6896b2640 1 --2- 172.21.15.7:0/1703753260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841017d0 0x7ff684101c50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff678009900 tx=0x7ff67802bc30 comp rx=0 tx=0).stop 2024-03-26T08:39:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.034+0000 7ff6896b2640 1 -- 172.21.15.7:0/1703753260 shutdown_connections 2024-03-26T08:39:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.034+0000 7ff6896b2640 1 --2- 172.21.15.7:0/1703753260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684102190 0x7ff684108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.034+0000 7ff6896b2640 1 --2- 172.21.15.7:0/1703753260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841017d0 0x7ff684101c50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.034+0000 7ff6896b2640 1 --2- 172.21.15.7:0/1703753260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6841005e0 0x7ff684100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.034+0000 7ff6896b2640 1 -- 172.21.15.7:0/1703753260 >> 172.21.15.7:0/1703753260 conn(0x7ff6840fbb30 msgr2=0x7ff6840fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.034+0000 7ff6896b2640 1 -- 172.21.15.7:0/1703753260 shutdown_connections 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.034+0000 7ff6896b2640 1 -- 172.21.15.7:0/1703753260 wait complete. 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff6896b2640 1 Processor -- start 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff6896b2640 1 -- start start 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff6896b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841005e0 0x7ff68406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff6896b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6841017d0 0x7ff68419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff6896b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684102190 0x7ff68419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff6896b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68410c240 con 0x7ff6841005e0 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff6896b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68410c0c0 con 0x7ff6841017d0 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff6896b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68410c3c0 con 0x7ff684102190 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff682ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841005e0 0x7ff68406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff682ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841005e0 0x7ff68406a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44482/0 (socket says 172.21.15.7:44482) 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff682ffd640 1 -- 172.21.15.7:0/874208375 learned_addr learned my addr 172.21.15.7:0/874208375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.035+0000 7ff6827fc640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6841017d0 0x7ff68419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff6837fe640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684102190 0x7ff68419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff682ffd640 1 -- 172.21.15.7:0/874208375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684102190 msgr2=0x7ff68419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff682ffd640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684102190 0x7ff68419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff682ffd640 1 -- 172.21.15.7:0/874208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6841017d0 msgr2=0x7ff68419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff682ffd640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6841017d0 0x7ff68419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff682ffd640 1 -- 172.21.15.7:0/874208375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68419f570 con 0x7ff6841005e0 2024-03-26T08:39:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff6837fe640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684102190 0x7ff68419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:39:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff682ffd640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841005e0 0x7ff68406a3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff674009d20 tx=0x7ff67400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff66bfff640 1 -- 172.21.15.7:0/874208375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674010630 con 0x7ff6841005e0 2024-03-26T08:39:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff66bfff640 1 -- 172.21.15.7:0/874208375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff674010cb0 con 0x7ff6841005e0 2024-03-26T08:39:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68419f860 con 0x7ff6841005e0 2024-03-26T08:39:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff66bfff640 1 -- 172.21.15.7:0/874208375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674018de0 con 0x7ff6841005e0 2024-03-26T08:39:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.036+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68419fd80 con 0x7ff6841005e0 2024-03-26T08:39:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.037+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6841078a0 con 0x7ff6841005e0 2024-03-26T08:39:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.038+0000 7ff66bfff640 1 -- 172.21.15.7:0/874208375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff67401f020 con 0x7ff6841005e0 2024-03-26T08:39:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.039+0000 7ff66bfff640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff66c0a2800 0x7ff66c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.039+0000 7ff6827fc640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff66c0a2800 0x7ff66c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.039+0000 7ff66bfff640 1 -- 172.21.15.7:0/874208375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6740ceb80 con 0x7ff6841005e0 2024-03-26T08:39:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.039+0000 7ff6827fc640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff66c0a2800 0x7ff66c0a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff67802c7b0 tx=0x7ff678005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:45.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.043+0000 7ff66bfff640 1 -- 172.21.15.7:0/874208375 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff67409c020 con 0x7ff6841005e0 2024-03-26T08:39:45.243 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:39:45 smithi016 systemd[1]: Starting Ceph mds.e for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:39:45.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.266+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff68402d070 con 0x7ff6841005e0 2024-03-26T08:39:45.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.267+0000 7ff66bfff640 1 -- 172.21.15.7:0/874208375 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3686 (secure 0 0 0) 0x7ff67409b920 con 0x7ff6841005e0 2024-03-26T08:39:45.258 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:45.258 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T08:39:45.259 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 5 2024-03-26T08:39:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff66c0a2800 msgr2=0x7ff66c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff66c0a2800 0x7ff66c0a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff67802c7b0 tx=0x7ff678005fd0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841005e0 msgr2=0x7ff68406a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841005e0 0x7ff68406a3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff674009d20 tx=0x7ff67400c6a0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 shutdown_connections 2024-03-26T08:39:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff66c0a2800 0x7ff66c0a4cc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff684102190 0x7ff68419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6841017d0 0x7ff68419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 --2- 172.21.15.7:0/874208375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6841005e0 0x7ff68406a3f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 >> 172.21.15.7:0/874208375 conn(0x7ff6840fbb30 msgr2=0x7ff684104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 shutdown_connections 2024-03-26T08:39:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:45.269+0000 7ff6896b2640 1 -- 172.21.15.7:0/874208375 wait complete. 2024-03-26T08:39:45.583 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:39:45 smithi016 podman[65593]: 2024-03-26 08:39:45.220469584 +0000 UTC m=+0.018549390 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:39:45.583 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:39:45 smithi016 podman[65593]: 2024-03-26 08:39:45.330660281 +0000 UTC m=+0.128740079 container create 3dc18f6b035a2b7258f4ddf9229631b623623c58d90f9c887204b7701df4b965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-26T08:39:45.583 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:39:45 smithi016 podman[65593]: 2024-03-26 08:39:45.591781745 +0000 UTC m=+0.389861546 container init 3dc18f6b035a2b7258f4ddf9229631b623623c58d90f9c887204b7701df4b965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T08:39:45.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:45 smithi016 ceph-mon[35851]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:45.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:45 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/874208375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:45.993 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:39:45 smithi016 podman[65593]: 2024-03-26 08:39:45.59453453 +0000 UTC m=+0.392614323 container start 3dc18f6b035a2b7258f4ddf9229631b623623c58d90f9c887204b7701df4b965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-03-26T08:39:45.993 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:39:45 smithi016 ceph-mds[65612]: -- 172.21.15.16:0/3381103668 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5640a5a4ba40 con 0x5640a5a1f000 2024-03-26T08:39:45.994 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:39:45 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e[65608]: starting mds.e at 2024-03-26T08:39:45.994 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:39:45 smithi016 bash[65593]: 3dc18f6b035a2b7258f4ddf9229631b623623c58d90f9c887204b7701df4b965 2024-03-26T08:39:45.994 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:39:45 smithi016 systemd[1]: Started Ceph mds.e for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:39:46.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:45 smithi007 ceph-mon[32462]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:46.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:45 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/874208375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:46.065 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:45 smithi044 ceph-mon[36461]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:46.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:45 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/874208375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:47.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.16:6834/1837116746,v1:172.21.15.16:6835/1837116746] up:boot 2024-03-26T08:39:47.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: fsmap 5 up:standby 2024-03-26T08:39:47.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T08:39:47.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:47.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:47.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:47.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:47.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:47.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:47.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:46 smithi007 ceph-mon[32462]: Deploying daemon mds.f on smithi044 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.16:6834/1837116746,v1:172.21.15.16:6835/1837116746] up:boot 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: fsmap 5 up:standby 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:46 smithi044 ceph-mon[36461]: Deploying daemon mds.f on smithi044 2024-03-26T08:39:47.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.16:6834/1837116746,v1:172.21.15.16:6835/1837116746] up:boot 2024-03-26T08:39:47.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: fsmap 5 up:standby 2024-03-26T08:39:47.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T08:39:47.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:47.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:47.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:47.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:47.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:47.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:47.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:46 smithi016 ceph-mon[35851]: Deploying daemon mds.f on smithi044 2024-03-26T08:39:47.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.395+0000 7f0e91f10640 1 -- 172.21.15.7:0/3576220202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c1032e0 msgr2=0x7f0e8c109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.395+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3576220202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c1032e0 0x7f0e8c109b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009ca0 tx=0x7f0e8402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.395+0000 7f0e91f10640 1 -- 172.21.15.7:0/3576220202 shutdown_connections 2024-03-26T08:39:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.395+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3576220202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c1032e0 0x7f0e8c109b70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.395+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3576220202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e8c102920 0x7f0e8c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.395+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3576220202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c101730 0x7f0e8c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.395+0000 7f0e91f10640 1 -- 172.21.15.7:0/3576220202 >> 172.21.15.7:0/3576220202 conn(0x7f0e8c0fccc0 msgr2=0x7f0e8c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.395+0000 7f0e91f10640 1 -- 172.21.15.7:0/3576220202 shutdown_connections 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.395+0000 7f0e91f10640 1 -- 172.21.15.7:0/3576220202 wait complete. 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e91f10640 1 Processor -- start 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e91f10640 1 -- start start 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e91f10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c101730 0x7f0e8c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e91f10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c102920 0x7f0e8c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e91f10640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e8c1032e0 0x7f0e8c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e91f10640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c10d320 con 0x7f0e8c102920 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e91f10640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c10d1a0 con 0x7f0e8c101730 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e91f10640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c10d4a0 con 0x7f0e8c1032e0 2024-03-26T08:39:47.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e8affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c102920 0x7f0e8c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e8affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c102920 0x7f0e8c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32792/0 (socket says 172.21.15.7:32792) 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e8affd640 1 -- 172.21.15.7:0/3862953840 learned_addr learned my addr 172.21.15.7:0/3862953840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e8b7fe640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c101730 0x7f0e8c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.396+0000 7f0e8bfff640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e8c1032e0 0x7f0e8c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e8affd640 1 -- 172.21.15.7:0/3862953840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e8c1032e0 msgr2=0x7f0e8c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e8affd640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e8c1032e0 0x7f0e8c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e8affd640 1 -- 172.21.15.7:0/3862953840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c101730 msgr2=0x7f0e8c1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e8affd640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c101730 0x7f0e8c1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e8affd640 1 -- 172.21.15.7:0/3862953840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c19f4d0 con 0x7f0e8c102920 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e8affd640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c102920 0x7f0e8c197df0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c920 tx=0x7f0e7c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e88ff9640 1 -- 172.21.15.7:0/3862953840 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c007c70 con 0x7f0e8c102920 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e88ff9640 1 -- 172.21.15.7:0/3862953840 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c007e10 con 0x7f0e8c102920 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c19f7c0 con 0x7f0e8c102920 2024-03-26T08:39:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e88ff9640 1 -- 172.21.15.7:0/3862953840 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c018890 con 0x7f0e8c102920 2024-03-26T08:39:47.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.397+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c19fd10 con 0x7f0e8c102920 2024-03-26T08:39:47.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.398+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8c108980 con 0x7f0e8c102920 2024-03-26T08:39:47.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.399+0000 7f0e88ff9640 1 -- 172.21.15.7:0/3862953840 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0e7c018a70 con 0x7f0e8c102920 2024-03-26T08:39:47.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.400+0000 7f0e88ff9640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e740a2800 0x7f0e740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.400+0000 7f0e8b7fe640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e740a2800 0x7f0e740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.400+0000 7f0e88ff9640 1 -- 172.21.15.7:0/3862953840 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0e7c0ce7b0 con 0x7f0e8c102920 2024-03-26T08:39:47.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.400+0000 7f0e8b7fe640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e740a2800 0x7f0e740a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c102780 tx=0x7f0e8000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:47.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.405+0000 7f0e88ff9640 1 -- 172.21.15.7:0/3862953840 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e7c014030 con 0x7f0e8c102920 2024-03-26T08:39:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.627+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e8c02d070 con 0x7f0e8c102920 2024-03-26T08:39:47.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.627+0000 7f0e88ff9640 1 -- 172.21.15.7:0/3862953840 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4477 (secure 0 0 0) 0x7f0e7c09bc50 con 0x7f0e8c102920 2024-03-26T08:39:47.619 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:47.619 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T08:39:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 6 2024-03-26T08:39:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e740a2800 msgr2=0x7f0e740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e740a2800 0x7f0e740a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c102780 tx=0x7f0e8000b390 comp rx=0 tx=0).stop 2024-03-26T08:39:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c102920 msgr2=0x7f0e8c197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c102920 0x7f0e8c197df0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c920 tx=0x7f0e7c00cde0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 shutdown_connections 2024-03-26T08:39:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e740a2800 0x7f0e740a4cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e8c1032e0 0x7f0e8c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c102920 0x7f0e8c197df0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 --2- 172.21.15.7:0/3862953840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c101730 0x7f0e8c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:47.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.629+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 >> 172.21.15.7:0/3862953840 conn(0x7f0e8c0fccc0 msgr2=0x7f0e8c105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:47.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.630+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 shutdown_connections 2024-03-26T08:39:47.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:47.630+0000 7f0e91f10640 1 -- 172.21.15.7:0/3862953840 wait complete. 2024-03-26T08:39:48.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:47 smithi007 ceph-mon[32462]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:48.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:47 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3862953840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:48.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:47 smithi044 ceph-mon[36461]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:48.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:47 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3862953840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:48.066 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:39:48 smithi044 systemd[1]: Starting Ceph mds.f for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:39:48.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:47 smithi016 ceph-mon[35851]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:48.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:47 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3862953840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:48.470 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:39:48 smithi044 podman[67020]: 2024-03-26 08:39:48.124232503 +0000 UTC m=+0.018259915 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:39:48.470 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:39:48 smithi044 podman[67020]: 2024-03-26 08:39:48.23527435 +0000 UTC m=+0.129301767 container create 8c77619fc916570c7b2091a7f43eb47f4aab06bffffbd6751c27176f48c5b834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-26T08:39:48.816 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:39:48 smithi044 podman[67020]: 2024-03-26 08:39:48.479485032 +0000 UTC m=+0.373512430 container init 8c77619fc916570c7b2091a7f43eb47f4aab06bffffbd6751c27176f48c5b834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:39:48.816 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:39:48 smithi044 podman[67020]: 2024-03-26 08:39:48.482261453 +0000 UTC m=+0.376288859 container start 8c77619fc916570c7b2091a7f43eb47f4aab06bffffbd6751c27176f48c5b834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T08:39:48.816 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:39:48 smithi044 ceph-mds[67040]: -- 172.21.15.44:0/4138221476 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562b764cda40 con 0x562b764a0c00 2024-03-26T08:39:48.816 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:39:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f[67036]: starting mds.f at 2024-03-26T08:39:48.816 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:39:48 smithi044 bash[67020]: 8c77619fc916570c7b2091a7f43eb47f4aab06bffffbd6751c27176f48c5b834 2024-03-26T08:39:48.816 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:39:48 smithi044 systemd[1]: Started Ceph mds.f for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:39:49.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.754+0000 7f9a2d79f640 1 -- 172.21.15.7:0/4056003765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ffff0 msgr2=0x7f9a28109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.754+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/4056003765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ffff0 0x7f9a28109ba0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009ca0 tx=0x7f9a2002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.754+0000 7f9a2d79f640 1 -- 172.21.15.7:0/4056003765 shutdown_connections 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.754+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/4056003765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ffff0 0x7f9a28109ba0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.754+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/4056003765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a280ff630 0x7f9a280ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.754+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/4056003765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a28102bc0 0x7f9a28102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.754+0000 7f9a2d79f640 1 -- 172.21.15.7:0/4056003765 >> 172.21.15.7:0/4056003765 conn(0x7f9a280fb1d0 msgr2=0x7f9a280fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.754+0000 7f9a2d79f640 1 -- 172.21.15.7:0/4056003765 shutdown_connections 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.755+0000 7f9a2d79f640 1 -- 172.21.15.7:0/4056003765 wait complete. 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.755+0000 7f9a2d79f640 1 Processor -- start 2024-03-26T08:39:49.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.755+0000 7f9a2d79f640 1 -- start start 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a2d79f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ff630 0x7f9a28197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a2d79f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a280ffff0 0x7f9a28197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a2d79f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a28102bc0 0x7f9a2819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a2d79f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2810d270 con 0x7f9a280ff630 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a2d79f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2810d0f0 con 0x7f9a28102bc0 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a2d79f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2810d3f0 con 0x7f9a280ffff0 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ff630 0x7f9a28197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ff630 0x7f9a28197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32806/0 (socket says 172.21.15.7:32806) 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a27fff640 1 -- 172.21.15.7:0/3028546526 learned_addr learned my addr 172.21.15.7:0/3028546526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a280ffff0 0x7f9a28197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:49.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a2cf9e640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a28102bc0 0x7f9a2819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a27fff640 1 -- 172.21.15.7:0/3028546526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a280ffff0 msgr2=0x7f9a28197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a27fff640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a280ffff0 0x7f9a28197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a27fff640 1 -- 172.21.15.7:0/3028546526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a28102bc0 msgr2=0x7f9a2819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a27fff640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a28102bc0 0x7f9a2819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a27fff640 1 -- 172.21.15.7:0/3028546526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2819f440 con 0x7f9a280ff630 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a27fff640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ff630 0x7f9a28197820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800e9c0 tx=0x7f9a1800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.756+0000 7f9a257fa640 1 -- 172.21.15.7:0/3028546526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800cd80 con 0x7f9a280ff630 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.757+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2819f730 con 0x7f9a280ff630 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.757+0000 7f9a257fa640 1 -- 172.21.15.7:0/3028546526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a180035f0 con 0x7f9a280ff630 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.757+0000 7f9a257fa640 1 -- 172.21.15.7:0/3028546526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18010690 con 0x7f9a280ff630 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.757+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2819fc80 con 0x7f9a280ff630 2024-03-26T08:39:49.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.758+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a28108a20 con 0x7f9a280ff630 2024-03-26T08:39:49.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.759+0000 7f9a257fa640 1 -- 172.21.15.7:0/3028546526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9a180037a0 con 0x7f9a280ff630 2024-03-26T08:39:49.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.759+0000 7f9a257fa640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a140a27f0 0x7f9a140a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:49.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.760+0000 7f9a277fe640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a140a27f0 0x7f9a140a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:49.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.760+0000 7f9a257fa640 1 -- 172.21.15.7:0/3028546526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9a180ce920 con 0x7f9a280ff630 2024-03-26T08:39:49.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.760+0000 7f9a277fe640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a140a27f0 0x7f9a140a4cb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9a28198d30 tx=0x7f9a10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:49.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.764+0000 7f9a257fa640 1 -- 172.21.15.7:0/3028546526 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9a1809be60 con 0x7f9a280ff630 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: Deploying daemon mds.c on smithi044 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.44:6834/2525817262,v1:172.21.15.44:6835/2525817262] up:boot 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: fsmap 6 up:standby 2024-03-26T08:39:49.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T08:39:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.986+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a2802d070 con 0x7f9a280ff630 2024-03-26T08:39:49.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.986+0000 7f9a257fa640 1 -- 172.21.15.7:0/3028546526 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5268 (secure 0 0 0) 0x7f9a1809b760 con 0x7f9a280ff630 2024-03-26T08:39:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:49.978 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-26T08:39:49.978 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2024-03-26T08:39:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a140a27f0 msgr2=0x7f9a140a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a140a27f0 0x7f9a140a4cb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9a28198d30 tx=0x7f9a10005f90 comp rx=0 tx=0).stop 2024-03-26T08:39:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ff630 msgr2=0x7f9a28197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ff630 0x7f9a28197820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800e9c0 tx=0x7f9a1800ee80 comp rx=0 tx=0).stop 2024-03-26T08:39:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 shutdown_connections 2024-03-26T08:39:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a140a27f0 0x7f9a140a4cb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a28102bc0 0x7f9a2819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a280ffff0 0x7f9a28197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 --2- 172.21.15.7:0/3028546526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a280ff630 0x7f9a28197820 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 >> 172.21.15.7:0/3028546526 conn(0x7f9a280fb1d0 msgr2=0x7f9a28105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 shutdown_connections 2024-03-26T08:39:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:49.989+0000 7f9a2d79f640 1 -- 172.21.15.7:0/3028546526 wait complete. 2024-03-26T08:39:49.990 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:49.990 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:49.990 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:49.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:49.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:49.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:49.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:49.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: Deploying daemon mds.c on smithi044 2024-03-26T08:39:49.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.44:6834/2525817262,v1:172.21.15.44:6835/2525817262] up:boot 2024-03-26T08:39:49.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: fsmap 6 up:standby 2024-03-26T08:39:49.991 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T08:39:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: Deploying daemon mds.c on smithi044 2024-03-26T08:39:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.44:6834/2525817262,v1:172.21.15.44:6835/2525817262] up:boot 2024-03-26T08:39:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: fsmap 6 up:standby 2024-03-26T08:39:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T08:39:50.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:50 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3028546526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:51.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:50 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3028546526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:51.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:50 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3028546526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:51.066 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:39:50 smithi044 systemd[1]: Starting Ceph mds.c for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:39:51.335 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:39:51 smithi044 podman[67330]: 2024-03-26 08:39:50.987700603 +0000 UTC m=+0.018027292 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:39:51.335 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:39:51 smithi044 podman[67330]: 2024-03-26 08:39:51.090687919 +0000 UTC m=+0.121014635 container create 4eede49de1663048a087b74290ced867515694100dbee4e51241d1c95e8958d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T08:39:51.639 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:39:51 smithi044 podman[67330]: 2024-03-26 08:39:51.344302429 +0000 UTC m=+0.374629104 container init 4eede49de1663048a087b74290ced867515694100dbee4e51241d1c95e8958d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-03-26T08:39:51.639 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:39:51 smithi044 podman[67330]: 2024-03-26 08:39:51.347440828 +0000 UTC m=+0.377767513 container start 4eede49de1663048a087b74290ced867515694100dbee4e51241d1c95e8958d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-26T08:39:51.640 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:39:51 smithi044 ceph-mds[67350]: -- 172.21.15.44:0/1060980101 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561e114e7a40 con 0x561e114bac00 2024-03-26T08:39:51.640 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:39:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c[67346]: starting mds.c at 2024-03-26T08:39:51.640 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:39:51 smithi044 bash[67330]: 4eede49de1663048a087b74290ced867515694100dbee4e51241d1c95e8958d5 2024-03-26T08:39:51.640 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:39:51 smithi044 systemd[1]: Started Ceph mds.c for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:39:51.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:51.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:51 smithi016 ceph-mon[35851]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:51.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:51.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:51.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:51.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:51.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:51.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:52.021 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:51 smithi007 ceph-mon[32462]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:52.021 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:52.021 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:52.021 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:52.021 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:52.022 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:52.022 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:52.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:51 smithi044 ceph-mon[36461]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:52.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:52.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:52.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:52.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:52.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:52.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:52.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 -- 172.21.15.7:0/3923807845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454101750 msgr2=0x7f9454101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:52.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 --2- 172.21.15.7:0/3923807845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454101750 0x7f9454101b70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9444009ca0 tx=0x7f944402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 -- 172.21.15.7:0/3923807845 shutdown_connections 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 --2- 172.21.15.7:0/3923807845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9454103300 0x7f9454109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 --2- 172.21.15.7:0/3923807845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9454102940 0x7f9454102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 --2- 172.21.15.7:0/3923807845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454101750 0x7f9454101b70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 -- 172.21.15.7:0/3923807845 >> 172.21.15.7:0/3923807845 conn(0x7f94540fccc0 msgr2=0x7f94540ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 -- 172.21.15.7:0/3923807845 shutdown_connections 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 -- 172.21.15.7:0/3923807845 wait complete. 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.117+0000 7f9458d97640 1 Processor -- start 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9458d97640 1 -- start start 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9458d97640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9454101750 0x7f94541979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9458d97640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454102940 0x7f9454197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9458d97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9454103300 0x7f945419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9458d97640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945410d330 con 0x7f9454102940 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9458d97640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945410d1b0 con 0x7f9454103300 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9458d97640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945410d4b0 con 0x7f9454101750 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9451d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454102940 0x7f9454197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9452d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9454103300 0x7f945419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9451d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454102940 0x7f9454197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32826/0 (socket says 172.21.15.7:32826) 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9452d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9454103300 0x7f945419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49672/0 (socket says 172.21.15.7:49672) 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9451d9b640 1 -- 172.21.15.7:0/3572095834 learned_addr learned my addr 172.21.15.7:0/3572095834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f945259c640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9454101750 0x7f94541979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9452d9d640 1 -- 172.21.15.7:0/3572095834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9454101750 msgr2=0x7f94541979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9452d9d640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9454101750 0x7f94541979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9452d9d640 1 -- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454102940 msgr2=0x7f9454197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9452d9d640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454102940 0x7f9454197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9452d9d640 1 -- 172.21.15.7:0/3572095834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945419f5f0 con 0x7f9454103300 2024-03-26T08:39:52.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9451d9b640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454102940 0x7f9454197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:39:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f9452d9d640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9454103300 0x7f945419ef80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f944c00e970 tx=0x7f944c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f94437fe640 1 -- 172.21.15.7:0/3572095834 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c00ccc0 con 0x7f9454103300 2024-03-26T08:39:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f94437fe640 1 -- 172.21.15.7:0/3572095834 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f944c0035f0 con 0x7f9454103300 2024-03-26T08:39:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.118+0000 7f94437fe640 1 -- 172.21.15.7:0/3572095834 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c00ce60 con 0x7f9454103300 2024-03-26T08:39:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.119+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945419f8e0 con 0x7f9454103300 2024-03-26T08:39:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.119+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945419fde0 con 0x7f9454103300 2024-03-26T08:39:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.119+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9454108af0 con 0x7f9454103300 2024-03-26T08:39:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.121+0000 7f94437fe640 1 -- 172.21.15.7:0/3572095834 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f944c0037a0 con 0x7f9454103300 2024-03-26T08:39:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.121+0000 7f94437fe640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f943c0a2800 0x7f943c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.121+0000 7f945259c640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f943c0a2800 0x7f943c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.121+0000 7f94437fe640 1 -- 172.21.15.7:0/3572095834 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f944c0ce880 con 0x7f9454103300 2024-03-26T08:39:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.122+0000 7f945259c640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f943c0a2800 0x7f943c0a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f944402c700 tx=0x7f9444032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:52.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.125+0000 7f94437fe640 1 -- 172.21.15.7:0/3572095834 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f944c09bd20 con 0x7f9454103300 2024-03-26T08:39:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.348+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f945402d070 con 0x7f9454103300 2024-03-26T08:39:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.349+0000 7f94437fe640 1 -- 172.21.15.7:0/3572095834 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6059 (secure 0 0 0) 0x7f944c09b620 con 0x7f9454103300 2024-03-26T08:39:52.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:52.341 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-26T08:39:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-03-26T08:39:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f943c0a2800 msgr2=0x7f943c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f943c0a2800 0x7f943c0a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f944402c700 tx=0x7f9444032590 comp rx=0 tx=0).stop 2024-03-26T08:39:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9454103300 msgr2=0x7f945419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9454103300 0x7f945419ef80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f944c00e970 tx=0x7f944c00ee30 comp rx=0 tx=0).stop 2024-03-26T08:39:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 shutdown_connections 2024-03-26T08:39:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f943c0a2800 0x7f943c0a4cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9454103300 0x7f945419ef80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9454102940 0x7f9454197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 --2- 172.21.15.7:0/3572095834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9454101750 0x7f94541979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 >> 172.21.15.7:0/3572095834 conn(0x7f94540fccc0 msgr2=0x7f9454105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 shutdown_connections 2024-03-26T08:39:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:52.351+0000 7f9458d97640 1 -- 172.21.15.7:0/3572095834 wait complete. 2024-03-26T08:39:52.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:52 smithi016 ceph-mon[35851]: Deploying daemon mds.h on smithi016 2024-03-26T08:39:52.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:52 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] up:boot 2024-03-26T08:39:52.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:52 smithi016 ceph-mon[35851]: fsmap 7 up:standby 2024-03-26T08:39:52.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T08:39:52.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:52 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3572095834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:53.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:52 smithi007 ceph-mon[32462]: Deploying daemon mds.h on smithi016 2024-03-26T08:39:53.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:52 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] up:boot 2024-03-26T08:39:53.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:52 smithi007 ceph-mon[32462]: fsmap 7 up:standby 2024-03-26T08:39:53.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T08:39:53.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:52 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3572095834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:53.065 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:52 smithi044 ceph-mon[36461]: Deploying daemon mds.h on smithi016 2024-03-26T08:39:53.065 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:52 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] up:boot 2024-03-26T08:39:53.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:52 smithi044 ceph-mon[36461]: fsmap 7 up:standby 2024-03-26T08:39:53.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T08:39:53.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:52 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3572095834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:53.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:53 smithi007 ceph-mon[32462]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:53.906 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:53 smithi016 ceph-mon[35851]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:53.906 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:39:53 smithi016 systemd[1]: Starting Ceph mds.h for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:39:54.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:53 smithi044 ceph-mon[36461]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:54.158 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:39:53 smithi016 podman[65904]: 2024-03-26 08:39:53.817591738 +0000 UTC m=+0.018465190 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:39:54.158 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:39:53 smithi016 podman[65904]: 2024-03-26 08:39:53.936405342 +0000 UTC m=+0.137278830 container create 9a5e1bdb2be06e2b2cd79c7eed70bec6c6310ad35f9a600929f42008ffcd7533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T08:39:54.158 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:39:54 smithi016 podman[65904]: 2024-03-26 08:39:54.164017605 +0000 UTC m=+0.364891042 container init 9a5e1bdb2be06e2b2cd79c7eed70bec6c6310ad35f9a600929f42008ffcd7533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:39:54.158 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:39:54 smithi016 podman[65904]: 2024-03-26 08:39:54.166742325 +0000 UTC m=+0.367615762 container start 9a5e1bdb2be06e2b2cd79c7eed70bec6c6310ad35f9a600929f42008ffcd7533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T08:39:54.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 -- 172.21.15.7:0/1040827942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe41021f0 msgr2=0x7f5fe4108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1040827942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe41021f0 0x7f5fe4108a80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009ca0 tx=0x7f5fd402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 -- 172.21.15.7:0/1040827942 shutdown_connections 2024-03-26T08:39:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1040827942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe41021f0 0x7f5fe4108a80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1040827942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe4101830 0x7f5fe4101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1040827942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe4100640 0x7f5fe4100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 -- 172.21.15.7:0/1040827942 >> 172.21.15.7:0/1040827942 conn(0x7f5fe40fbbd0 msgr2=0x7f5fe40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 -- 172.21.15.7:0/1040827942 shutdown_connections 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 -- 172.21.15.7:0/1040827942 wait complete. 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 Processor -- start 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.480+0000 7f5fec51b640 1 -- start start 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fec51b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe4100640 0x7f5fe4196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fec51b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe4101830 0x7f5fe4196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fec51b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe41021f0 0x7f5fe419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fec51b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe410c230 con 0x7f5fe4100640 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fec51b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe410c0b0 con 0x7f5fe41021f0 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fec51b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe410c3b0 con 0x7f5fe4101830 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fea2b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe4100640 0x7f5fe4196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fea2b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe4100640 0x7f5fe4196890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32846/0 (socket says 172.21.15.7:32846) 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fea2b7640 1 -- 172.21.15.7:0/1902732460 learned_addr learned my addr 172.21.15.7:0/1902732460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fe9ab6640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe4101830 0x7f5fe4196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5feaab8640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe41021f0 0x7f5fe419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fea2b7640 1 -- 172.21.15.7:0/1902732460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe4101830 msgr2=0x7f5fe4196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fea2b7640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe4101830 0x7f5fe4196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fea2b7640 1 -- 172.21.15.7:0/1902732460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe41021f0 msgr2=0x7f5fe419de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fea2b7640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe41021f0 0x7f5fe419de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fea2b7640 1 -- 172.21.15.7:0/1902732460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe419e4b0 con 0x7f5fe4100640 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fea2b7640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe4100640 0x7f5fe4196890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9c0 tx=0x7f5fdc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fe37fe640 1 -- 172.21.15.7:0/1902732460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00cd80 con 0x7f5fe4100640 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fe37fe640 1 -- 172.21.15.7:0/1902732460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fdc0035f0 con 0x7f5fe4100640 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe419e7a0 con 0x7f5fe4100640 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fe37fe640 1 -- 172.21.15.7:0/1902732460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc010690 con 0x7f5fe4100640 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.481+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe419ed40 con 0x7f5fe4100640 2024-03-26T08:39:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.482+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe4107900 con 0x7f5fe4100640 2024-03-26T08:39:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.483+0000 7f5fe37fe640 1 -- 172.21.15.7:0/1902732460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5fdc0037a0 con 0x7f5fe4100640 2024-03-26T08:39:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.484+0000 7f5fe37fe640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5fcc0a2880 0x7f5fcc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.484+0000 7f5fe9ab6640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5fcc0a2880 0x7f5fcc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.484+0000 7f5fe37fe640 1 -- 172.21.15.7:0/1902732460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5fdc0ce9c0 con 0x7f5fe4100640 2024-03-26T08:39:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.485+0000 7f5fe9ab6640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5fcc0a2880 0x7f5fcc0a4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4197da0 tx=0x7f5fd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.488+0000 7f5fe37fe640 1 -- 172.21.15.7:0/1902732460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5fdc09bf90 con 0x7f5fe4100640 2024-03-26T08:39:54.493 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:39:54 smithi016 ceph-mds[65923]: -- 172.21.15.16:0/1191489456 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fd8ea05a40 con 0x55fd8e9d8800 2024-03-26T08:39:54.493 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:39:54 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h[65919]: starting mds.h at 2024-03-26T08:39:54.493 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:39:54 smithi016 bash[65904]: 9a5e1bdb2be06e2b2cd79c7eed70bec6c6310ad35f9a600929f42008ffcd7533 2024-03-26T08:39:54.493 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:39:54 smithi016 systemd[1]: Started Ceph mds.h for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:39:54.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.711+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fe402d070 con 0x7f5fe4100640 2024-03-26T08:39:54.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.711+0000 7f5fe37fe640 1 -- 172.21.15.7:0/1902732460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6841 (secure 0 0 0) 0x7f5fdc09b890 con 0x7f5fe4100640 2024-03-26T08:39:54.703 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:54.704 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-26T08:39:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-03-26T08:39:54.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5fcc0a2880 msgr2=0x7f5fcc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:54.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5fcc0a2880 0x7f5fcc0a4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4197da0 tx=0x7f5fd8005f90 comp rx=0 tx=0).stop 2024-03-26T08:39:54.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe4100640 msgr2=0x7f5fe4196890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:54.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe4100640 0x7f5fe4196890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9c0 tx=0x7f5fdc00ee80 comp rx=0 tx=0).stop 2024-03-26T08:39:54.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 shutdown_connections 2024-03-26T08:39:54.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5fcc0a2880 0x7f5fcc0a4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe41021f0 0x7f5fe419de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fe4101830 0x7f5fe4196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 --2- 172.21.15.7:0/1902732460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe4100640 0x7f5fe4196890 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:54.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.713+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 >> 172.21.15.7:0/1902732460 conn(0x7f5fe40fbbd0 msgr2=0x7f5fe4104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:54.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.714+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 shutdown_connections 2024-03-26T08:39:54.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:54.714+0000 7f5fec51b640 1 -- 172.21.15.7:0/1902732460 wait complete. 2024-03-26T08:39:55.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:55.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:55.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:55.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:55.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:55.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:55.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:55.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: Deploying daemon mds.i on smithi044 2024-03-26T08:39:55.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.16:6836/7790244,v1:172.21.15.16:6837/7790244] up:boot 2024-03-26T08:39:55.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: fsmap 8 up:standby 2024-03-26T08:39:55.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:39:55.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:55 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1902732460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:55.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:55.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:55.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:55.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:55.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:55.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:55.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:55.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: Deploying daemon mds.i on smithi044 2024-03-26T08:39:55.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.16:6836/7790244,v1:172.21.15.16:6837/7790244] up:boot 2024-03-26T08:39:55.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: fsmap 8 up:standby 2024-03-26T08:39:55.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:39:55.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:55 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1902732460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:55.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:55.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:55.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:55.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:55.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:55.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:55.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:55.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: Deploying daemon mds.i on smithi044 2024-03-26T08:39:55.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.16:6836/7790244,v1:172.21.15.16:6837/7790244] up:boot 2024-03-26T08:39:55.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: fsmap 8 up:standby 2024-03-26T08:39:55.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:39:55.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:55 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1902732460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:56.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:56.816 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:39:56 smithi044 systemd[1]: Starting Ceph mds.i for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:39:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.839+0000 7fc24f087640 1 -- 172.21.15.7:0/2638985944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc248104af0 msgr2=0x7fc248106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.839+0000 7fc24f087640 1 --2- 172.21.15.7:0/2638985944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc248104af0 0x7fc248106ee0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009900 tx=0x7fc23c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:39:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.839+0000 7fc24f087640 1 -- 172.21.15.7:0/2638985944 shutdown_connections 2024-03-26T08:39:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.839+0000 7fc24f087640 1 --2- 172.21.15.7:0/2638985944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc248107420 0x7fc248109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.839+0000 7fc24f087640 1 --2- 172.21.15.7:0/2638985944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc248104af0 0x7fc248106ee0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.839+0000 7fc24f087640 1 --2- 172.21.15.7:0/2638985944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc2481021c0 0x7fc2481045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.839+0000 7fc24f087640 1 -- 172.21.15.7:0/2638985944 >> 172.21.15.7:0/2638985944 conn(0x7fc2480fbbd0 msgr2=0x7fc2480fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.839+0000 7fc24f087640 1 -- 172.21.15.7:0/2638985944 shutdown_connections 2024-03-26T08:39:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.839+0000 7fc24f087640 1 -- 172.21.15.7:0/2638985944 wait complete. 2024-03-26T08:39:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24f087640 1 Processor -- start 2024-03-26T08:39:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24f087640 1 -- start start 2024-03-26T08:39:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24f087640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2481021c0 0x7fc248197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24f087640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc248104af0 0x7fc248197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24f087640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc248107420 0x7fc24819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24f087640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24810d420 con 0x7fc2481021c0 2024-03-26T08:39:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24f087640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24810d2a0 con 0x7fc248104af0 2024-03-26T08:39:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24f087640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24810d5a0 con 0x7fc248107420 2024-03-26T08:39:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24ce23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2481021c0 0x7fc248197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24ce23640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2481021c0 0x7fc248197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56726/0 (socket says 172.21.15.7:56726) 2024-03-26T08:39:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24ce23640 1 -- 172.21.15.7:0/2470044460 learned_addr learned my addr 172.21.15.7:0/2470044460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.840+0000 7fc24d624640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc248107420 0x7fc24819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc24ce23640 1 -- 172.21.15.7:0/2470044460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc248107420 msgr2=0x7fc24819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc24ce23640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc248107420 0x7fc24819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc24ce23640 1 -- 172.21.15.7:0/2470044460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc248104af0 msgr2=0x7fc248197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc24ce23640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc248104af0 0x7fc248197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc24ce23640 1 -- 172.21.15.7:0/2470044460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24819f6c0 con 0x7fc2481021c0 2024-03-26T08:39:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc24d624640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc248107420 0x7fc24819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:39:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc24ce23640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2481021c0 0x7fc248197aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc24000ba50 tx=0x7fc24000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc245ffb640 1 -- 172.21.15.7:0/2470044460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240019070 con 0x7fc2481021c0 2024-03-26T08:39:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc245ffb640 1 -- 172.21.15.7:0/2470044460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc24000cbe0 con 0x7fc2481021c0 2024-03-26T08:39:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24819f9b0 con 0x7fc2481021c0 2024-03-26T08:39:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc245ffb640 1 -- 172.21.15.7:0/2470044460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2400076e0 con 0x7fc2481021c0 2024-03-26T08:39:56.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.841+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24819ff00 con 0x7fc2481021c0 2024-03-26T08:39:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.842+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2480fd6c0 con 0x7fc2481021c0 2024-03-26T08:39:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.843+0000 7fc245ffb640 1 -- 172.21.15.7:0/2470044460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc24001c070 con 0x7fc2481021c0 2024-03-26T08:39:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.844+0000 7fc245ffb640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2300a2800 0x7fc2300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.844+0000 7fc247fff640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2300a2800 0x7fc2300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.844+0000 7fc245ffb640 1 -- 172.21.15.7:0/2470044460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc2400cdcd0 con 0x7fc2481021c0 2024-03-26T08:39:56.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.844+0000 7fc247fff640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2300a2800 0x7fc2300a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc23c036040 tx=0x7fc23c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:56.848+0000 7fc245ffb640 1 -- 172.21.15.7:0/2470044460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc24009b170 con 0x7fc2481021c0 2024-03-26T08:39:57.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.071+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc248062340 con 0x7fc2481021c0 2024-03-26T08:39:57.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.071+0000 7fc245ffb640 1 -- 172.21.15.7:0/2470044460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6841 (secure 0 0 0) 0x7fc24009aa70 con 0x7fc2481021c0 2024-03-26T08:39:57.063 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:57.063 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-26T08:39:57.064 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-03-26T08:39:57.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2300a2800 msgr2=0x7fc2300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:57.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2300a2800 0x7fc2300a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc23c036040 tx=0x7fc23c000bc0 comp rx=0 tx=0).stop 2024-03-26T08:39:57.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2481021c0 msgr2=0x7fc248197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:57.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2481021c0 0x7fc248197aa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc24000ba50 tx=0x7fc24000bf10 comp rx=0 tx=0).stop 2024-03-26T08:39:57.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 shutdown_connections 2024-03-26T08:39:57.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2300a2800 0x7fc2300a4cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:57.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc248107420 0x7fc24819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:57.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc248104af0 0x7fc248197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:57.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 --2- 172.21.15.7:0/2470044460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2481021c0 0x7fc248197aa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:57.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 >> 172.21.15.7:0/2470044460 conn(0x7fc2480fbbd0 msgr2=0x7fc248102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:57.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 shutdown_connections 2024-03-26T08:39:57.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:57.074+0000 7fc24f087640 1 -- 172.21.15.7:0/2470044460 wait complete. 2024-03-26T08:39:57.179 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:39:56 smithi044 podman[67634]: 2024-03-26 08:39:56.808040511 +0000 UTC m=+0.018876111 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:39:57.180 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:39:57 smithi044 podman[67634]: 2024-03-26 08:39:57.049249695 +0000 UTC m=+0.260085274 container create 6be81398e8b113889b2801054aeec2b4b77bdad47aa63ae6dc5a1b5170f0dac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:39:57.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:57 smithi007 ceph-mon[32462]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:57.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:57 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2470044460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:57.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:57 smithi044 ceph-mon[36461]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:57.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:57 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2470044460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:57.566 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:39:57 smithi044 podman[67634]: 2024-03-26 08:39:57.298563634 +0000 UTC m=+0.509399227 container init 6be81398e8b113889b2801054aeec2b4b77bdad47aa63ae6dc5a1b5170f0dac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-26T08:39:57.566 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:39:57 smithi044 podman[67634]: 2024-03-26 08:39:57.301262843 +0000 UTC m=+0.512098428 container start 6be81398e8b113889b2801054aeec2b4b77bdad47aa63ae6dc5a1b5170f0dac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-26T08:39:57.566 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:39:57 smithi044 ceph-mds[67653]: -- 172.21.15.44:0/104552937 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f27e90da40 con 0x55f27e8e0800 2024-03-26T08:39:57.566 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:39:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i[67649]: starting mds.i at 2024-03-26T08:39:57.566 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:39:57 smithi044 bash[67634]: 6be81398e8b113889b2801054aeec2b4b77bdad47aa63ae6dc5a1b5170f0dac7 2024-03-26T08:39:57.566 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:39:57 smithi044 systemd[1]: Started Ceph mds.i for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:39:57.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:57 smithi016 ceph-mon[35851]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:57.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:57 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2470044460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: Deploying daemon mds.g on smithi007 2024-03-26T08:39:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] up:boot 2024-03-26T08:39:58.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: fsmap 9 up:standby 2024-03-26T08:39:58.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T08:39:58.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:58.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: Deploying daemon mds.g on smithi007 2024-03-26T08:39:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] up:boot 2024-03-26T08:39:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: fsmap 9 up:standby 2024-03-26T08:39:58.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T08:39:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:39:58.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:39:58.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:39:58.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:39:58.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: Deploying daemon mds.g on smithi007 2024-03-26T08:39:58.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] up:boot 2024-03-26T08:39:58.817 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: fsmap 9 up:standby 2024-03-26T08:39:58.818 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T08:39:59.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:39:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 -- 172.21.15.7:0/3095669082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c103290 msgr2=0x7f1c5c109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 --2- 172.21.15.7:0/3095669082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c103290 0x7f1c5c109b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009900 tx=0x7f1c5402bc30 comp rx=0 tx=0).stop 2024-03-26T08:39:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 -- 172.21.15.7:0/3095669082 shutdown_connections 2024-03-26T08:39:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 --2- 172.21.15.7:0/3095669082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c103290 0x7f1c5c109b20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 --2- 172.21.15.7:0/3095669082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c1028d0 0x7f1c5c102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 --2- 172.21.15.7:0/3095669082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c1016e0 0x7f1c5c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 -- 172.21.15.7:0/3095669082 >> 172.21.15.7:0/3095669082 conn(0x7f1c5c0fcc30 msgr2=0x7f1c5c0ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 -- 172.21.15.7:0/3095669082 shutdown_connections 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 -- 172.21.15.7:0/3095669082 wait complete. 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 Processor -- start 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 -- start start 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c1016e0 0x7f1c5c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.201+0000 7f1c6431b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c1028d0 0x7f1c5c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c6431b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c103290 0x7f1c5c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c6431b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c10d360 con 0x7f1c5c1016e0 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c6431b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c10d1e0 con 0x7f1c5c103290 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c6431b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c10d4e0 con 0x7f1c5c1028d0 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c620b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c1016e0 0x7f1c5c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c620b7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c1016e0 0x7f1c5c197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56742/0 (socket says 172.21.15.7:56742) 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c620b7640 1 -- 172.21.15.7:0/64500293 learned_addr learned my addr 172.21.15.7:0/64500293 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c618b6640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c1028d0 0x7f1c5c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c628b8640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c103290 0x7f1c5c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c620b7640 1 -- 172.21.15.7:0/64500293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c1028d0 msgr2=0x7f1c5c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c620b7640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c1028d0 0x7f1c5c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c620b7640 1 -- 172.21.15.7:0/64500293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c103290 msgr2=0x7f1c5c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c620b7640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c103290 0x7f1c5c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c620b7640 1 -- 172.21.15.7:0/64500293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c19f590 con 0x7f1c5c1016e0 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c628b8640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c103290 0x7f1c5c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c620b7640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c1016e0 0x7f1c5c197970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00eb70 tx=0x7f1c4c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c5b7fe640 1 -- 172.21.15.7:0/64500293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c003b70 con 0x7f1c5c1016e0 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c5b7fe640 1 -- 172.21.15.7:0/64500293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1c4c00ee70 con 0x7f1c5c1016e0 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c5b7fe640 1 -- 172.21.15.7:0/64500293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c010830 con 0x7f1c5c1016e0 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c19f880 con 0x7f1c5c1016e0 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.202+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c19fdd0 con 0x7f1c5c1016e0 2024-03-26T08:39:59.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.203+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c1089a0 con 0x7f1c5c1016e0 2024-03-26T08:39:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.204+0000 7f1c5b7fe640 1 -- 172.21.15.7:0/64500293 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1c4c004a40 con 0x7f1c5c1016e0 2024-03-26T08:39:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.205+0000 7f1c5b7fe640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c440a2880 0x7f1c440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:39:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.205+0000 7f1c618b6640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c440a2880 0x7f1c440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:39:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.205+0000 7f1c5b7fe640 1 -- 172.21.15.7:0/64500293 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1c4c0cedd0 con 0x7f1c5c1016e0 2024-03-26T08:39:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.206+0000 7f1c618b6640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c440a2880 0x7f1c440a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c198e80 tx=0x7f1c5000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:39:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.209+0000 7f1c5b7fe640 1 -- 172.21.15.7:0/64500293 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c4c09c270 con 0x7f1c5c1016e0 2024-03-26T08:39:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.449+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c5c02d070 con 0x7f1c5c1016e0 2024-03-26T08:39:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.450+0000 7f1c5b7fe640 1 -- 172.21.15.7:0/64500293 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7634 (secure 0 0 0) 0x7f1c4c09bb70 con 0x7f1c5c1016e0 2024-03-26T08:39:59.442 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:39:59.442 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-26T08:39:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-03-26T08:39:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c440a2880 msgr2=0x7f1c440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c440a2880 0x7f1c440a4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c198e80 tx=0x7f1c5000b410 comp rx=0 tx=0).stop 2024-03-26T08:39:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c1016e0 msgr2=0x7f1c5c197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:39:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c1016e0 0x7f1c5c197970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00eb70 tx=0x7f1c4c00c6a0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 shutdown_connections 2024-03-26T08:39:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1c440a2880 0x7f1c440a4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c5c103290 0x7f1c5c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1c5c1028d0 0x7f1c5c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 --2- 172.21.15.7:0/64500293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c1016e0 0x7f1c5c197970 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:39:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 >> 172.21.15.7:0/64500293 conn(0x7f1c5c0fcc30 msgr2=0x7f1c5c105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:39:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.452+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 shutdown_connections 2024-03-26T08:39:59.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:39:59.453+0000 7f1c6431b640 1 -- 172.21.15.7:0/64500293 wait complete. 2024-03-26T08:39:59.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:59 smithi007 ceph-mon[32462]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:59.774 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:39:59 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/64500293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:59 smithi044 ceph-mon[36461]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:39:59 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/64500293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:39:59.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:59 smithi016 ceph-mon[35851]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:39:59.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:39:59 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/64500293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:00.512 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 26 08:40:00 smithi007 systemd[1]: Started Ceph mds.g for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:40:00.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:00 smithi007 ceph-mon[32462]: overall HEALTH_OK 2024-03-26T08:40:00.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:00.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:00.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:00.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:40:00.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:40:00.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:00 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:00.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:00 smithi044 ceph-mon[36461]: overall HEALTH_OK 2024-03-26T08:40:00.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:00.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:00.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:00.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:40:00.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:40:00.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:00 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:00.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:00 smithi016 ceph-mon[35851]: overall HEALTH_OK 2024-03-26T08:40:00.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:40:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:40:00.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:00 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:01.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:40:01.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.572+0000 7fee91b77640 1 -- 172.21.15.7:0/1783128119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 msgr2=0x7fee8c0ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.572+0000 7fee91b77640 1 --2- 172.21.15.7:0/1783128119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 0x7fee8c0ffbd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fee80009900 tx=0x7fee8002bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:01.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 -- 172.21.15.7:0/1783128119 shutdown_connections 2024-03-26T08:40:01.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 --2- 172.21.15.7:0/1783128119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee8c100110 0x7fee8c109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 --2- 172.21.15.7:0/1783128119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 0x7fee8c0ffbd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 --2- 172.21.15.7:0/1783128119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8c102ce0 0x7fee8c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 -- 172.21.15.7:0/1783128119 >> 172.21.15.7:0/1783128119 conn(0x7fee8c0fb2d0 msgr2=0x7fee8c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:01.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 -- 172.21.15.7:0/1783128119 shutdown_connections 2024-03-26T08:40:01.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 -- 172.21.15.7:0/1783128119 wait complete. 2024-03-26T08:40:01.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 Processor -- start 2024-03-26T08:40:01.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 -- start start 2024-03-26T08:40:01.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 0x7fee8c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8c100110 0x7fee8c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee8c102ce0 0x7fee8c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.573+0000 7fee91b77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c10d3c0 con 0x7fee8c0ff750 2024-03-26T08:40:01.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee91b77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c10d240 con 0x7fee8c100110 2024-03-26T08:40:01.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee91b77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c10d540 con 0x7fee8c102ce0 2024-03-26T08:40:01.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 0x7fee8c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 0x7fee8c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56784/0 (socket says 172.21.15.7:56784) 2024-03-26T08:40:01.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8b7fe640 1 -- 172.21.15.7:0/678530389 learned_addr learned my addr 172.21.15.7:0/678530389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:01.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8affd640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8c100110 0x7fee8c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8bfff640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee8c102ce0 0x7fee8c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8b7fe640 1 -- 172.21.15.7:0/678530389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee8c102ce0 msgr2=0x7fee8c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8b7fe640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee8c102ce0 0x7fee8c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8b7fe640 1 -- 172.21.15.7:0/678530389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8c100110 msgr2=0x7fee8c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8b7fe640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8c100110 0x7fee8c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8b7fe640 1 -- 172.21.15.7:0/678530389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c19f650 con 0x7fee8c0ff750 2024-03-26T08:40:01.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee8b7fe640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 0x7fee8c197a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00d770 tx=0x7fee7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:01.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee88ff9640 1 -- 172.21.15.7:0/678530389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c017070 con 0x7fee8c0ff750 2024-03-26T08:40:01.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee88ff9640 1 -- 172.21.15.7:0/678530389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee7c009c30 con 0x7fee8c0ff750 2024-03-26T08:40:01.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8c19f940 con 0x7fee8c0ff750 2024-03-26T08:40:01.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee88ff9640 1 -- 172.21.15.7:0/678530389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c003dd0 con 0x7fee8c0ff750 2024-03-26T08:40:01.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.574+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8c19feb0 con 0x7fee8c0ff750 2024-03-26T08:40:01.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.575+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8c108ba0 con 0x7fee8c0ff750 2024-03-26T08:40:01.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.576+0000 7fee88ff9640 1 -- 172.21.15.7:0/678530389 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fee7c003fb0 con 0x7fee8c0ff750 2024-03-26T08:40:01.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.577+0000 7fee88ff9640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee740a2880 0x7fee740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:01.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.577+0000 7fee8affd640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee740a2880 0x7fee740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:01.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.577+0000 7fee88ff9640 1 -- 172.21.15.7:0/678530389 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fee7c0cdf80 con 0x7fee8c0ff750 2024-03-26T08:40:01.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.577+0000 7fee8affd640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee740a2880 0x7fee740a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee8002c7b0 tx=0x7fee80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:01.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.581+0000 7fee88ff9640 1 -- 172.21.15.7:0/678530389 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee7c009de0 con 0x7fee8c0ff750 2024-03-26T08:40:01.739 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:01 smithi007 ceph-mon[32462]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:01.739 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:01 smithi007 ceph-mon[32462]: Deploying daemon mds.a on smithi007 2024-03-26T08:40:01.739 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:01 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.7:6838/2041082244,v1:172.21.15.7:6839/2041082244] up:boot 2024-03-26T08:40:01.739 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:01 smithi007 ceph-mon[32462]: fsmap 10 up:standby 2024-03-26T08:40:01.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:01 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T08:40:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.807+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee8c0622a0 con 0x7fee8c0ff750 2024-03-26T08:40:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.808+0000 7fee88ff9640 1 -- 172.21.15.7:0/678530389 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8423 (secure 0 0 0) 0x7fee7c09b420 con 0x7fee8c0ff750 2024-03-26T08:40:01.800 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:01.800 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-26T08:40:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 11 2024-03-26T08:40:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee740a2880 msgr2=0x7fee740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee740a2880 0x7fee740a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fee8002c7b0 tx=0x7fee80005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 msgr2=0x7fee8c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 0x7fee8c197a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00d770 tx=0x7fee7c00dc30 comp rx=0 tx=0).stop 2024-03-26T08:40:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 shutdown_connections 2024-03-26T08:40:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee740a2880 0x7fee740a4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee8c102ce0 0x7fee8c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee8c100110 0x7fee8c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 --2- 172.21.15.7:0/678530389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee8c0ff750 0x7fee8c197a30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 >> 172.21.15.7:0/678530389 conn(0x7fee8c0fb2d0 msgr2=0x7fee8c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 shutdown_connections 2024-03-26T08:40:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:01.810+0000 7fee91b77640 1 -- 172.21.15.7:0/678530389 wait complete. 2024-03-26T08:40:01.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:01 smithi044 ceph-mon[36461]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:01.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:01 smithi044 ceph-mon[36461]: Deploying daemon mds.a on smithi007 2024-03-26T08:40:01.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:01 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.7:6838/2041082244,v1:172.21.15.7:6839/2041082244] up:boot 2024-03-26T08:40:01.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:01 smithi044 ceph-mon[36461]: fsmap 10 up:standby 2024-03-26T08:40:01.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:01 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T08:40:01.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:01 smithi016 ceph-mon[35851]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:01.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:01 smithi016 ceph-mon[35851]: Deploying daemon mds.a on smithi007 2024-03-26T08:40:01.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:01 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.7:6838/2041082244,v1:172.21.15.7:6839/2041082244] up:boot 2024-03-26T08:40:01.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:01 smithi016 ceph-mon[35851]: fsmap 10 up:standby 2024-03-26T08:40:01.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:01 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T08:40:02.725 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:02 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/678530389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:02.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:02 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/678530389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:02.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:02 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/678530389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:03.292 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 26 08:40:03 smithi007 systemd[1]: Started Ceph mds.a for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:40:03.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 -- 172.21.15.7:0/495390291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0101730 msgr2=0x7f0ce0101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/495390291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0101730 0x7f0ce0101b50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009ca0 tx=0x7f0cd802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 -- 172.21.15.7:0/495390291 shutdown_connections 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/495390291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ce01032e0 0x7f0ce0109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/495390291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce0102920 0x7f0ce0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/495390291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0101730 0x7f0ce0101b50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 -- 172.21.15.7:0/495390291 >> 172.21.15.7:0/495390291 conn(0x7f0ce00fccc0 msgr2=0x7f0ce00ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 -- 172.21.15.7:0/495390291 shutdown_connections 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 -- 172.21.15.7:0/495390291 wait complete. 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 Processor -- start 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.936+0000 7f0ce80ea640 1 -- start start 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce80ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ce0101730 0x7f0ce0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce80ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0102920 0x7f0ce0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce80ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce01032e0 0x7f0ce019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce80ea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce010d390 con 0x7f0ce0102920 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce80ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce010d210 con 0x7f0ce01032e0 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce80ea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce010d510 con 0x7f0ce0101730 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0102920 0x7f0ce0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0102920 0x7f0ce0197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56820/0 (socket says 172.21.15.7:56820) 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5685640 1 -- 172.21.15.7:0/4004781616 learned_addr learned my addr 172.21.15.7:0/4004781616 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce6687640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce01032e0 0x7f0ce019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5e86640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ce0101730 0x7f0ce0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5685640 1 -- 172.21.15.7:0/4004781616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ce0101730 msgr2=0x7f0ce0197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5685640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ce0101730 0x7f0ce0197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5685640 1 -- 172.21.15.7:0/4004781616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce01032e0 msgr2=0x7f0ce019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5685640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce01032e0 0x7f0ce019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5685640 1 -- 172.21.15.7:0/4004781616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce019f650 con 0x7f0ce0102920 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce5685640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0102920 0x7f0ce0197f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400d950 tx=0x7f0cd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0cdeffd640 1 -- 172.21.15.7:0/4004781616 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4017070 con 0x7f0ce0102920 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce019f940 con 0x7f0ce0102920 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0cdeffd640 1 -- 172.21.15.7:0/4004781616 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4009c30 con 0x7f0ce0102920 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0cdeffd640 1 -- 172.21.15.7:0/4004781616 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400f6b0 con 0x7f0ce0102920 2024-03-26T08:40:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.937+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce019fee0 con 0x7f0ce0102920 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.938+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce0108a50 con 0x7f0ce0102920 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.939+0000 7f0cdeffd640 1 -- 172.21.15.7:0/4004781616 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0cd40109d0 con 0x7f0ce0102920 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.940+0000 7f0cdeffd640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cc80a2800 0x7f0cc80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.940+0000 7f0ce5e86640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cc80a2800 0x7f0cc80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.940+0000 7f0cdeffd640 1 -- 172.21.15.7:0/4004781616 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0cd40ce120 con 0x7f0ce0102920 2024-03-26T08:40:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.940+0000 7f0ce5e86640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cc80a2800 0x7f0cc80a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0cd802c700 tx=0x7f0cd8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:03.945+0000 7f0cdeffd640 1 -- 172.21.15.7:0/4004781616 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0cd4009de0 con 0x7f0ce0102920 2024-03-26T08:40:04.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.167+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ce002d270 con 0x7f0ce0102920 2024-03-26T08:40:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.167+0000 7f0cdeffd640 1 -- 172.21.15.7:0/4004781616 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9212 (secure 0 0 0) 0x7f0cd409b010 con 0x7f0ce0102920 2024-03-26T08:40:04.159 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:04.159 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-26T08:40:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2024-03-26T08:40:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.169+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cc80a2800 msgr2=0x7f0cc80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.169+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cc80a2800 0x7f0cc80a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0cd802c700 tx=0x7f0cd8032590 comp rx=0 tx=0).stop 2024-03-26T08:40:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0102920 msgr2=0x7f0ce0197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0102920 0x7f0ce0197f70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400d950 tx=0x7f0cd400de10 comp rx=0 tx=0).stop 2024-03-26T08:40:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 shutdown_connections 2024-03-26T08:40:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cc80a2800 0x7f0cc80a4cc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce01032e0 0x7f0ce019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ce0102920 0x7f0ce0197f70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 --2- 172.21.15.7:0/4004781616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ce0101730 0x7f0ce0197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 >> 172.21.15.7:0/4004781616 conn(0x7f0ce00fccc0 msgr2=0x7f0ce0105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 shutdown_connections 2024-03-26T08:40:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:04.170+0000 7f0ce80ea640 1 -- 172.21.15.7:0/4004781616 wait complete. 2024-03-26T08:40:04.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:04.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:04.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:04.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:40:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:40:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: Deploying daemon mds.b on smithi016 2024-03-26T08:40:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.7:6840/3829840154,v1:172.21.15.7:6841/3829840154] up:boot 2024-03-26T08:40:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: fsmap 11 up:standby 2024-03-26T08:40:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T08:40:04.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:04 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4004781616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:04.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:04.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: Deploying daemon mds.b on smithi016 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.7:6840/3829840154,v1:172.21.15.7:6841/3829840154] up:boot 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: fsmap 11 up:standby 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T08:40:04.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:04 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4004781616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T08:40:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T08:40:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:04.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: Deploying daemon mds.b on smithi016 2024-03-26T08:40:04.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.7:6840/3829840154,v1:172.21.15.7:6841/3829840154] up:boot 2024-03-26T08:40:04.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: fsmap 11 up:standby 2024-03-26T08:40:04.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T08:40:04.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:04 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4004781616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:05.743 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:05 smithi016 systemd[1]: Starting Ceph mds.b for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:40:06.045 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:05 smithi016 podman[66212]: 2024-03-26 08:40:05.700158936 +0000 UTC m=+0.018690173 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:40:06.045 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:05 smithi016 podman[66212]: 2024-03-26 08:40:05.800681227 +0000 UTC m=+0.119212449 container create 19cd65b0137d56ba868c99ebf378bff18efc9fa7373483d44816e9d84eb52471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-26T08:40:06.045 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:06 smithi016 podman[66212]: 2024-03-26 08:40:06.028265199 +0000 UTC m=+0.346796419 container init 19cd65b0137d56ba868c99ebf378bff18efc9fa7373483d44816e9d84eb52471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-26T08:40:06.045 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:06 smithi016 podman[66212]: 2024-03-26 08:40:06.030860853 +0000 UTC m=+0.349392073 container start 19cd65b0137d56ba868c99ebf378bff18efc9fa7373483d44816e9d84eb52471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-26T08:40:06.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:40:06.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.299+0000 7f781e145640 1 -- 172.21.15.7:0/954531538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7818103390 msgr2=0x7f7818109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.299+0000 7f781e145640 1 --2- 172.21.15.7:0/954531538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7818103390 0x7f7818109c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7810009cd0 tx=0x7f781002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.299+0000 7f781e145640 1 -- 172.21.15.7:0/954531538 shutdown_connections 2024-03-26T08:40:06.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.299+0000 7f781e145640 1 --2- 172.21.15.7:0/954531538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7818103390 0x7f7818109c20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.299+0000 7f781e145640 1 --2- 172.21.15.7:0/954531538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78181029d0 0x7f7818102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.299+0000 7f781e145640 1 --2- 172.21.15.7:0/954531538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78180ffa00 0x7f78180ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.299+0000 7f781e145640 1 -- 172.21.15.7:0/954531538 >> 172.21.15.7:0/954531538 conn(0x7f78180fb310 msgr2=0x7f78180fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:06.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.299+0000 7f781e145640 1 -- 172.21.15.7:0/954531538 shutdown_connections 2024-03-26T08:40:06.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.299+0000 7f781e145640 1 -- 172.21.15.7:0/954531538 wait complete. 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f781e145640 1 Processor -- start 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f781e145640 1 -- start start 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f781e145640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78180ffa00 0x7f78181979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f781e145640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78181029d0 0x7f7818197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f781e145640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7818103390 0x7f781819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f781e145640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781810d360 con 0x7f78180ffa00 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f781e145640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781810d1e0 con 0x7f7818103390 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f781e145640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781810d4e0 con 0x7f78181029d0 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f78177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78180ffa00 0x7f78181979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f78177fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78180ffa00 0x7f78181979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56834/0 (socket says 172.21.15.7:56834) 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f78177fe640 1 -- 172.21.15.7:0/1583417202 learned_addr learned my addr 172.21.15.7:0/1583417202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f7817fff640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7818103390 0x7f781819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f7816ffd640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78181029d0 0x7f7818197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f78177fe640 1 -- 172.21.15.7:0/1583417202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78181029d0 msgr2=0x7f7818197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f78177fe640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78181029d0 0x7f7818197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f78177fe640 1 -- 172.21.15.7:0/1583417202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7818103390 msgr2=0x7f781819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f78177fe640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7818103390 0x7f781819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.300+0000 7f78177fe640 1 -- 172.21.15.7:0/1583417202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781819f5c0 con 0x7f78180ffa00 2024-03-26T08:40:06.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.301+0000 7f78177fe640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78180ffa00 0x7f78181979a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f780800c970 tx=0x7f780800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.301+0000 7f7814ff9640 1 -- 172.21.15.7:0/1583417202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808004220 con 0x7f78180ffa00 2024-03-26T08:40:06.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.301+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781819f8b0 con 0x7f78180ffa00 2024-03-26T08:40:06.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.301+0000 7f7814ff9640 1 -- 172.21.15.7:0/1583417202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78080043c0 con 0x7f78180ffa00 2024-03-26T08:40:06.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.301+0000 7f7814ff9640 1 -- 172.21.15.7:0/1583417202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808004a00 con 0x7f78180ffa00 2024-03-26T08:40:06.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.301+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781819fdb0 con 0x7f78180ffa00 2024-03-26T08:40:06.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.302+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7818108b00 con 0x7f78180ffa00 2024-03-26T08:40:06.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.303+0000 7f7814ff9640 1 -- 172.21.15.7:0/1583417202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f78080079b0 con 0x7f78180ffa00 2024-03-26T08:40:06.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.303+0000 7f7814ff9640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78000a2800 0x7f78000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.304+0000 7f7816ffd640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78000a2800 0x7f78000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.304+0000 7f7814ff9640 1 -- 172.21.15.7:0/1583417202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f78080ce340 con 0x7f78180ffa00 2024-03-26T08:40:06.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.304+0000 7f7816ffd640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78000a2800 0x7f78000a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7818198eb0 tx=0x7f780c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.308+0000 7f7814ff9640 1 -- 172.21.15.7:0/1583417202 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7808014030 con 0x7f78180ffa00 2024-03-26T08:40:06.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:06 smithi016 ceph-mon[35851]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:06.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:06 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:06.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:06 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:06.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:06 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:06.493 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:06 smithi016 ceph-mds[66231]: -- 172.21.15.16:0/2535550900 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d87749ba40 con 0x55d87746ec00 2024-03-26T08:40:06.493 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b[66227]: starting mds.b at 2024-03-26T08:40:06.493 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:06 smithi016 bash[66212]: 19cd65b0137d56ba868c99ebf378bff18efc9fa7373483d44816e9d84eb52471 2024-03-26T08:40:06.494 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:06 smithi016 systemd[1]: Started Ceph mds.b for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T08:40:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.534+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f781802d070 con 0x7f78180ffa00 2024-03-26T08:40:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.535+0000 7f7814ff9640 1 -- 172.21.15.7:0/1583417202 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10004 (secure 0 0 0) 0x7f780809b880 con 0x7f78180ffa00 2024-03-26T08:40:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:06.528 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24466,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24476,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-26T08:40:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2024-03-26T08:40:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78000a2800 msgr2=0x7f78000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78000a2800 0x7f78000a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7818198eb0 tx=0x7f780c005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78180ffa00 msgr2=0x7f78181979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78180ffa00 0x7f78181979a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f780800c970 tx=0x7f780800ce30 comp rx=0 tx=0).stop 2024-03-26T08:40:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 shutdown_connections 2024-03-26T08:40:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78000a2800 0x7f78000a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7818103390 0x7f781819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78181029d0 0x7f7818197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 --2- 172.21.15.7:0/1583417202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78180ffa00 0x7f78181979a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 >> 172.21.15.7:0/1583417202 conn(0x7f78180fb310 msgr2=0x7f7818105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.537+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 shutdown_connections 2024-03-26T08:40:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.538+0000 7f781e145640 1 -- 172.21.15.7:0/1583417202 wait complete. 2024-03-26T08:40:06.540 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:06 smithi007 ceph-mon[32462]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:06.540 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:06 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:06.540 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:06 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:06.540 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:06 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:06.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:40:06.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:06 smithi044 ceph-mon[36461]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:06.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:06 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:06.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:06 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:06.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:06 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.652+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/817564440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d4102180 msgr2=0x7fb2d4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.652+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/817564440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d4102180 0x7fb2d4108a10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009ca0 tx=0x7fb2cc02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.653+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/817564440 shutdown_connections 2024-03-26T08:40:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.653+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/817564440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d4102180 0x7fb2d4108a10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.653+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/817564440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d41017c0 0x7fb2d4101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.653+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/817564440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2d41005d0 0x7fb2d41009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.653+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/817564440 >> 172.21.15.7:0/817564440 conn(0x7fb2d40fbb40 msgr2=0x7fb2d40fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.653+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/817564440 shutdown_connections 2024-03-26T08:40:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.653+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/817564440 wait complete. 2024-03-26T08:40:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.653+0000 7fb2d8d7e640 1 Processor -- start 2024-03-26T08:40:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.653+0000 7fb2d8d7e640 1 -- start start 2024-03-26T08:40:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d8d7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d41005d0 0x7fb2d4197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d8d7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d41017c0 0x7fb2d4197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d8d7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2d4102180 0x7fb2d419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d8d7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d410c250 con 0x7fb2d41005d0 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d8d7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d410c0d0 con 0x7fb2d41017c0 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d8d7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d410c3d0 con 0x7fb2d4102180 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d41005d0 0x7fb2d4197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d41005d0 0x7fb2d4197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60348/0 (socket says 172.21.15.7:60348) 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d1d9b640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d41017c0 0x7fb2d4197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d259c640 1 -- 172.21.15.7:0/1792259682 learned_addr learned my addr 172.21.15.7:0/1792259682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d2d9d640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2d4102180 0x7fb2d419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d259c640 1 -- 172.21.15.7:0/1792259682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2d4102180 msgr2=0x7fb2d419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d259c640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2d4102180 0x7fb2d419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d259c640 1 -- 172.21.15.7:0/1792259682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d41017c0 msgr2=0x7fb2d4197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d259c640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d41017c0 0x7fb2d4197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d259c640 1 -- 172.21.15.7:0/1792259682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d419f5a0 con 0x7fb2d41005d0 2024-03-26T08:40:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d1d9b640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d41017c0 0x7fb2d4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d259c640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d41005d0 0x7fb2d4197980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400e9c0 tx=0x7fb2c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2c37fe640 1 -- 172.21.15.7:0/1792259682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c400cd80 con 0x7fb2d41005d0 2024-03-26T08:40:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d419f890 con 0x7fb2d41005d0 2024-03-26T08:40:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2c37fe640 1 -- 172.21.15.7:0/1792259682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2c40035f0 con 0x7fb2d41005d0 2024-03-26T08:40:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.654+0000 7fb2c37fe640 1 -- 172.21.15.7:0/1792259682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4010690 con 0x7fb2d41005d0 2024-03-26T08:40:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.655+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d419fd90 con 0x7fb2d41005d0 2024-03-26T08:40:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.655+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d419fff0 con 0x7fb2d41005d0 2024-03-26T08:40:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.657+0000 7fb2c37fe640 1 -- 172.21.15.7:0/1792259682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb2c40037a0 con 0x7fb2d41005d0 2024-03-26T08:40:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.657+0000 7fb2c37fe640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2bc0a2800 0x7fb2bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.657+0000 7fb2d1d9b640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2bc0a2800 0x7fb2bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.657+0000 7fb2c37fe640 1 -- 172.21.15.7:0/1792259682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb2c40ce9f0 con 0x7fb2d41005d0 2024-03-26T08:40:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.658+0000 7fb2d1d9b640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2bc0a2800 0x7fb2bc0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4198e90 tx=0x7fb2c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.661+0000 7fb2c37fe640 1 -- 172.21.15.7:0/1792259682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb2c409bf30 con 0x7fb2d41005d0 2024-03-26T08:40:06.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.850+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2d41a01c0 con 0x7fb2d41005d0 2024-03-26T08:40:06.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.851+0000 7fb2c37fe640 1 -- 172.21.15.7:0/1792259682 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18326 (secure 0 0 0) 0x7fb2c409b830 con 0x7fb2d41005d0 2024-03-26T08:40:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:06.844 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:39:20.275865+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2bc0a2800 msgr2=0x7fb2bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2bc0a2800 0x7fb2bc0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4198e90 tx=0x7fb2c8005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d41005d0 msgr2=0x7fb2d4197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d41005d0 0x7fb2d4197980 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400e9c0 tx=0x7fb2c400ee80 comp rx=0 tx=0).stop 2024-03-26T08:40:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 shutdown_connections 2024-03-26T08:40:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2bc0a2800 0x7fb2bc0a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2d4102180 0x7fb2d419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2d41017c0 0x7fb2d4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 --2- 172.21.15.7:0/1792259682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2d41005d0 0x7fb2d4197980 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 >> 172.21.15.7:0/1792259682 conn(0x7fb2d40fbb40 msgr2=0x7fb2d40fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:06.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 shutdown_connections 2024-03-26T08:40:06.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.853+0000 7fb2d8d7e640 1 -- 172.21.15.7:0/1792259682 wait complete. 2024-03-26T08:40:06.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:40:06.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:40:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.980+0000 7f0299de8640 1 -- 172.21.15.7:0/1894928673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294102130 msgr2=0x7f0294104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.980+0000 7f0299de8640 1 --2- 172.21.15.7:0/1894928673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294102130 0x7f0294104520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0284009900 tx=0x7f028402bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.980+0000 7f0299de8640 1 -- 172.21.15.7:0/1894928673 shutdown_connections 2024-03-26T08:40:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.980+0000 7f0299de8640 1 --2- 172.21.15.7:0/1894928673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0294107390 0x7f02941097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.980+0000 7f0299de8640 1 --2- 172.21.15.7:0/1894928673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294104a60 0x7f0294106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.980+0000 7f0299de8640 1 --2- 172.21.15.7:0/1894928673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294102130 0x7f0294104520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.980+0000 7f0299de8640 1 -- 172.21.15.7:0/1894928673 >> 172.21.15.7:0/1894928673 conn(0x7f02940fbb60 msgr2=0x7f02940fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.980+0000 7f0299de8640 1 -- 172.21.15.7:0/1894928673 shutdown_connections 2024-03-26T08:40:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.980+0000 7f0299de8640 1 -- 172.21.15.7:0/1894928673 wait complete. 2024-03-26T08:40:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f0299de8640 1 Processor -- start 2024-03-26T08:40:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f0299de8640 1 -- start start 2024-03-26T08:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f0299de8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0294102130 0x7f02941979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f0299de8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294104a60 0x7f0294197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f0299de8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294107390 0x7f029419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f0299de8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029410d340 con 0x7f0294104a60 2024-03-26T08:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f0299de8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029410d1c0 con 0x7f0294107390 2024-03-26T08:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f0299de8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029410d4c0 con 0x7f0294102130 2024-03-26T08:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f028affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294104a60 0x7f0294197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f028affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294104a60 0x7f0294197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60368/0 (socket says 172.21.15.7:60368) 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f028affd640 1 -- 172.21.15.7:0/4220105647 learned_addr learned my addr 172.21.15.7:0/4220105647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f02937fe640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0294102130 0x7f02941979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.981+0000 7f0293fff640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294107390 0x7f029419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f028affd640 1 -- 172.21.15.7:0/4220105647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0294102130 msgr2=0x7f02941979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f028affd640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0294102130 0x7f02941979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f028affd640 1 -- 172.21.15.7:0/4220105647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294107390 msgr2=0x7f029419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f028affd640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294107390 0x7f029419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f028affd640 1 -- 172.21.15.7:0/4220105647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029419f610 con 0x7f0294104a60 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f02937fe640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0294102130 0x7f02941979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f028affd640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294104a60 0x7f0294197f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9a0 tx=0x7f027c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f02917fa640 1 -- 172.21.15.7:0/4220105647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c00cd80 con 0x7f0294104a60 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029419f900 con 0x7f0294104a60 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f02917fa640 1 -- 172.21.15.7:0/4220105647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f027c0035f0 con 0x7f0294104a60 2024-03-26T08:40:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f02917fa640 1 -- 172.21.15.7:0/4220105647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c010670 con 0x7f0294104a60 2024-03-26T08:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.982+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029419fea0 con 0x7f0294104a60 2024-03-26T08:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.983+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02940fd650 con 0x7f0294104a60 2024-03-26T08:40:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.984+0000 7f02917fa640 1 -- 172.21.15.7:0/4220105647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f027c0037a0 con 0x7f0294104a60 2024-03-26T08:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.985+0000 7f02917fa640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02800a2800 0x7f02800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.985+0000 7f02937fe640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02800a2800 0x7f02800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.985+0000 7f02917fa640 1 -- 172.21.15.7:0/4220105647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f027c0ce810 con 0x7f0294104a60 2024-03-26T08:40:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.986+0000 7f02937fe640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02800a2800 0x7f02800a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f028402c7b0 tx=0x7f0284005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:06.989+0000 7f02917fa640 1 -- 172.21.15.7:0/4220105647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f027c09bd50 con 0x7f0294104a60 2024-03-26T08:40:07.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.182+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f029402d220 con 0x7f0294104a60 2024-03-26T08:40:07.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.182+0000 7f02917fa640 1 -- 172.21.15.7:0/4220105647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f027c09b650 con 0x7f0294104a60 2024-03-26T08:40:07.175 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:40:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.184+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02800a2800 msgr2=0x7f02800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.184+0000 7f0299de8640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02800a2800 0x7f02800a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f028402c7b0 tx=0x7f0284005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294104a60 msgr2=0x7f0294197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294104a60 0x7f0294197f30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9a0 tx=0x7f027c00ee60 comp rx=0 tx=0).stop 2024-03-26T08:40:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 shutdown_connections 2024-03-26T08:40:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02800a2800 0x7f02800a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0294107390 0x7f029419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0294104a60 0x7f0294197f30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 --2- 172.21.15.7:0/4220105647 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0294102130 0x7f02941979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 >> 172.21.15.7:0/4220105647 conn(0x7f02940fbb60 msgr2=0x7f0294102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 shutdown_connections 2024-03-26T08:40:07.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.185+0000 7f0299de8640 1 -- 172.21.15.7:0/4220105647 wait complete. 2024-03-26T08:40:07.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-26T08:40:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.301+0000 7fb8c7026640 1 -- 172.21.15.7:0/1183721214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0102180 msgr2=0x7fb8c0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.301+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1183721214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0102180 0x7fb8c0108a10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009ca0 tx=0x7fb8b802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.301+0000 7fb8c7026640 1 -- 172.21.15.7:0/1183721214 shutdown_connections 2024-03-26T08:40:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.301+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1183721214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c0102180 0x7fb8c0108a10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.301+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1183721214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8c01017c0 0x7fb8c0101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.301+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1183721214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c01005d0 0x7fb8c01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.301+0000 7fb8c7026640 1 -- 172.21.15.7:0/1183721214 >> 172.21.15.7:0/1183721214 conn(0x7fb8c00fbb40 msgr2=0x7fb8c00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.301+0000 7fb8c7026640 1 -- 172.21.15.7:0/1183721214 shutdown_connections 2024-03-26T08:40:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.301+0000 7fb8c7026640 1 -- 172.21.15.7:0/1183721214 wait complete. 2024-03-26T08:40:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c7026640 1 Processor -- start 2024-03-26T08:40:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c7026640 1 -- start start 2024-03-26T08:40:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c7026640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c01005d0 0x7fb8c0104560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c7026640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c01017c0 0x7fb8c0102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c7026640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8c0102180 0x7fb8c01030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c7026640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c010c230 con 0x7fb8c01017c0 2024-03-26T08:40:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c7026640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c010c0b0 con 0x7fb8c01005d0 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c7026640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c010c3b0 con 0x7fb8c0102180 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8bffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c01017c0 0x7fb8c0102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8bffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c01017c0 0x7fb8c0102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60382/0 (socket says 172.21.15.7:60382) 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8bffff640 1 -- 172.21.15.7:0/1108513732 learned_addr learned my addr 172.21.15.7:0/1108513732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c4dc2640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c01005d0 0x7fb8c0104560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.302+0000 7fb8c55c3640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8c0102180 0x7fb8c01030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8bffff640 1 -- 172.21.15.7:0/1108513732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8c0102180 msgr2=0x7fb8c01030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8bffff640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8c0102180 0x7fb8c01030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8bffff640 1 -- 172.21.15.7:0/1108513732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c01005d0 msgr2=0x7fb8c0104560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8bffff640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c01005d0 0x7fb8c0104560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8bffff640 1 -- 172.21.15.7:0/1108513732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c0103970 con 0x7fb8c01017c0 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8bffff640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c01017c0 0x7fb8c0102ba0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d950 tx=0x7fb8ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8bdffb640 1 -- 172.21.15.7:0/1108513732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac017030 con 0x7fb8c01017c0 2024-03-26T08:40:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c01a1a80 con 0x7fb8c01017c0 2024-03-26T08:40:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8bdffb640 1 -- 172.21.15.7:0/1108513732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb8ac004020 con 0x7fb8c01017c0 2024-03-26T08:40:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8bdffb640 1 -- 172.21.15.7:0/1108513732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0047a0 con 0x7fb8c01017c0 2024-03-26T08:40:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.303+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c01a2020 con 0x7fb8c01017c0 2024-03-26T08:40:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.304+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c01a2250 con 0x7fb8c01017c0 2024-03-26T08:40:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.305+0000 7fb8bdffb640 1 -- 172.21.15.7:0/1108513732 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb8ac0041d0 con 0x7fb8c01017c0 2024-03-26T08:40:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.306+0000 7fb8bdffb640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8b00a2800 0x7fb8b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.306+0000 7fb8bdffb640 1 -- 172.21.15.7:0/1108513732 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb8ac0cdf90 con 0x7fb8c01017c0 2024-03-26T08:40:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.306+0000 7fb8c4dc2640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8b00a2800 0x7fb8b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.306+0000 7fb8c4dc2640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8b00a2800 0x7fb8b00a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0101620 tx=0x7fb8b400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:07.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.310+0000 7fb8bdffb640 1 -- 172.21.15.7:0/1108513732 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb8ac09b4d0 con 0x7fb8c01017c0 2024-03-26T08:40:07.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:07.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:07 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] up:boot 2024-03-26T08:40:07.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:07 smithi007 ceph-mon[32462]: fsmap 12 up:standby 2024-03-26T08:40:07.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:40:07.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:07.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:07 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:40:07.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:07 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1583417202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:07.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:07 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1792259682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:07.458 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:07 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4220105647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.498+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb8c01a25e0 con 0x7fb8c01017c0 2024-03-26T08:40:07.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.498+0000 7fb8bdffb640 1 -- 172.21.15.7:0/1108513732 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb8ac09add0 con 0x7fb8c01017c0 2024-03-26T08:40:07.491 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:07.491 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-03-26T08:40:07.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:40:07.491 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-03-26T08:40:07.491 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-03-26T08:40:07.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:40:07.492 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-03-26T08:40:07.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8b00a2800 msgr2=0x7fb8b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8b00a2800 0x7fb8b00a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0101620 tx=0x7fb8b400b390 comp rx=0 tx=0).stop 2024-03-26T08:40:07.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c01017c0 msgr2=0x7fb8c0102ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c01017c0 0x7fb8c0102ba0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00d950 tx=0x7fb8ac00de10 comp rx=0 tx=0).stop 2024-03-26T08:40:07.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 shutdown_connections 2024-03-26T08:40:07.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8b00a2800 0x7fb8b00a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8c0102180 0x7fb8c01030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c01017c0 0x7fb8c0102ba0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 --2- 172.21.15.7:0/1108513732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c01005d0 0x7fb8c0104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.500+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 >> 172.21.15.7:0/1108513732 conn(0x7fb8c00fbb40 msgr2=0x7fb8c00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.501+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 shutdown_connections 2024-03-26T08:40:07.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.501+0000 7fb8c7026640 1 -- 172.21.15.7:0/1108513732 wait complete. 2024-03-26T08:40:07.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:07.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:07 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] up:boot 2024-03-26T08:40:07.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:07 smithi016 ceph-mon[35851]: fsmap 12 up:standby 2024-03-26T08:40:07.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:40:07.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:07.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:07 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:40:07.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:07 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1583417202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:07.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:07 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1792259682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:07.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:07 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4220105647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:07.504 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-26T08:40:07.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-26T08:40:07.510 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:07.510 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:07 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] up:boot 2024-03-26T08:40:07.510 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:07 smithi044 ceph-mon[36461]: fsmap 12 up:standby 2024-03-26T08:40:07.510 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:40:07.510 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:07.510 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:07 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:40:07.510 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:07 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1583417202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:07.510 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:07 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1792259682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:07.510 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:07 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4220105647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:40:07.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.619+0000 7f9040859640 1 -- 172.21.15.7:0/3620275733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c104a60 msgr2=0x7f903c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.619+0000 7f9040859640 1 --2- 172.21.15.7:0/3620275733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c104a60 0x7f903c106e50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9024009900 tx=0x7f902402bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.619+0000 7f9040859640 1 -- 172.21.15.7:0/3620275733 shutdown_connections 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.619+0000 7f9040859640 1 --2- 172.21.15.7:0/3620275733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f903c107390 0x7f903c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.619+0000 7f9040859640 1 --2- 172.21.15.7:0/3620275733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c104a60 0x7f903c106e50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.619+0000 7f9040859640 1 --2- 172.21.15.7:0/3620275733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c102130 0x7f903c104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.619+0000 7f9040859640 1 -- 172.21.15.7:0/3620275733 >> 172.21.15.7:0/3620275733 conn(0x7f903c0fbb00 msgr2=0x7f903c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.619+0000 7f9040859640 1 -- 172.21.15.7:0/3620275733 shutdown_connections 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.619+0000 7f9040859640 1 -- 172.21.15.7:0/3620275733 wait complete. 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f9040859640 1 Processor -- start 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f9040859640 1 -- start start 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f9040859640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c102130 0x7f903c101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f9040859640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f903c104a60 0x7f903c1000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f9040859640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c107390 0x7f903c100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f9040859640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c10d3d0 con 0x7f903c102130 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f9040859640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c10d250 con 0x7f903c107390 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f9040859640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c10d550 con 0x7f903c104a60 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f903a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c102130 0x7f903c101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f903a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c102130 0x7f903c101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60400/0 (socket says 172.21.15.7:60400) 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f903a59c640 1 -- 172.21.15.7:0/2865269869 learned_addr learned my addr 172.21.15.7:0/2865269869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f903ad9d640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c107390 0x7f903c100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f9033fff640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f903c104a60 0x7f903c1000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f903a59c640 1 -- 172.21.15.7:0/2865269869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f903c104a60 msgr2=0x7f903c1000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f903a59c640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f903c104a60 0x7f903c1000e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f903a59c640 1 -- 172.21.15.7:0/2865269869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c107390 msgr2=0x7f903c100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f903a59c640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c107390 0x7f903c100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:07.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.620+0000 7f903a59c640 1 -- 172.21.15.7:0/2865269869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c100eb0 con 0x7f903c102130 2024-03-26T08:40:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.621+0000 7f903a59c640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c102130 0x7f903c101aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f902c00d950 tx=0x7f902c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.621+0000 7f90337fe640 1 -- 172.21.15.7:0/2865269869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c017070 con 0x7f903c102130 2024-03-26T08:40:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.621+0000 7f90337fe640 1 -- 172.21.15.7:0/2865269869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f902c009c30 con 0x7f903c102130 2024-03-26T08:40:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.621+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903c1a1bc0 con 0x7f903c102130 2024-03-26T08:40:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.621+0000 7f90337fe640 1 -- 172.21.15.7:0/2865269869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c00f6b0 con 0x7f903c102130 2024-03-26T08:40:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.621+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c1a2110 con 0x7f903c102130 2024-03-26T08:40:07.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.622+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903c0fd5f0 con 0x7f903c102130 2024-03-26T08:40:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.623+0000 7f90337fe640 1 -- 172.21.15.7:0/2865269869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f902c0109d0 con 0x7f903c102130 2024-03-26T08:40:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.623+0000 7f90337fe640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90280a2800 0x7f90280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.624+0000 7f9033fff640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90280a2800 0x7f90280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.624+0000 7f90337fe640 1 -- 172.21.15.7:0/2865269869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f902c0ce1e0 con 0x7f903c102130 2024-03-26T08:40:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.624+0000 7f9033fff640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90280a2800 0x7f90280a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9024036040 tx=0x7f9024005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.628+0000 7f90337fe640 1 -- 172.21.15.7:0/2865269869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f902c009de0 con 0x7f903c102130 2024-03-26T08:40:07.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:07.844+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f903c02d070 con 0x7f903c102130 2024-03-26T08:40:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.207+0000 7f90337fe640 1 -- 172.21.15.7:0/2865269869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f902c09b720 con 0x7f903c102130 2024-03-26T08:40:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-03-26T08:40:08.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90280a2800 msgr2=0x7f90280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90280a2800 0x7f90280a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9024036040 tx=0x7f9024005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c102130 msgr2=0x7f903c101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c102130 0x7f903c101aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f902c00d950 tx=0x7f902c00de10 comp rx=0 tx=0).stop 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 shutdown_connections 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90280a2800 0x7f90280a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c107390 0x7f903c100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f903c104a60 0x7f903c1000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 --2- 172.21.15.7:0/2865269869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f903c102130 0x7f903c101aa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 >> 172.21.15.7:0/2865269869 conn(0x7f903c0fbb00 msgr2=0x7f903c102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 shutdown_connections 2024-03-26T08:40:08.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.210+0000 7f9040859640 1 -- 172.21.15.7:0/2865269869 wait complete. 2024-03-26T08:40:08.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-26T08:40:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.336+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/4030209653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1032a0 msgr2=0x7fc1bc109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.336+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/4030209653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1032a0 0x7fc1bc109b30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009900 tx=0x7fc1ac02bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.336+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/4030209653 shutdown_connections 2024-03-26T08:40:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.336+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/4030209653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1032a0 0x7fc1bc109b30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.336+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/4030209653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1028e0 0x7fc1bc102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.336+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/4030209653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1bc1016f0 0x7fc1bc101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.336+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/4030209653 >> 172.21.15.7:0/4030209653 conn(0x7fc1bc0fcc60 msgr2=0x7fc1bc0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.336+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/4030209653 shutdown_connections 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/4030209653 wait complete. 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1c0ab2640 1 Processor -- start 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1c0ab2640 1 -- start start 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1c0ab2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1016f0 0x7fc1bc197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1c0ab2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1bc1028e0 0x7fc1bc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1c0ab2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1032a0 0x7fc1bc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1c0ab2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc10d350 con 0x7fc1bc1016f0 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1c0ab2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc10d1d0 con 0x7fc1bc1028e0 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1c0ab2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc10d4d0 con 0x7fc1bc1032a0 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.337+0000 7fc1ba59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1016f0 0x7fc1bc197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1ba59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1016f0 0x7fc1bc197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60422/0 (socket says 172.21.15.7:60422) 2024-03-26T08:40:08.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1ba59c640 1 -- 172.21.15.7:0/743408970 learned_addr learned my addr 172.21.15.7:0/743408970 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1b9d9b640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1bc1028e0 0x7fc1bc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1bad9d640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1032a0 0x7fc1bc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1ba59c640 1 -- 172.21.15.7:0/743408970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1032a0 msgr2=0x7fc1bc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1ba59c640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1032a0 0x7fc1bc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1ba59c640 1 -- 172.21.15.7:0/743408970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1bc1028e0 msgr2=0x7fc1bc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1ba59c640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1bc1028e0 0x7fc1bc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1ba59c640 1 -- 172.21.15.7:0/743408970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc19f5b0 con 0x7fc1bc1016f0 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1ba59c640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1016f0 0x7fc1bc197990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400ccb0 tx=0x7fc1b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1b37fe640 1 -- 172.21.15.7:0/743408970 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4003990 con 0x7fc1bc1016f0 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc19f8a0 con 0x7fc1bc1016f0 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1b37fe640 1 -- 172.21.15.7:0/743408970 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1b4003b30 con 0x7fc1bc1016f0 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1b37fe640 1 -- 172.21.15.7:0/743408970 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400f7b0 con 0x7fc1bc1016f0 2024-03-26T08:40:08.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.338+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc19fe40 con 0x7fc1bc1016f0 2024-03-26T08:40:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.339+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc068520 con 0x7fc1bc1016f0 2024-03-26T08:40:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.340+0000 7fc1b37fe640 1 -- 172.21.15.7:0/743408970 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc1b400f950 con 0x7fc1bc1016f0 2024-03-26T08:40:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.341+0000 7fc1b37fe640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1a80a2800 0x7fc1a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.341+0000 7fc1b9d9b640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1a80a2800 0x7fc1a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:08.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.341+0000 7fc1b37fe640 1 -- 172.21.15.7:0/743408970 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fc1b40ce810 con 0x7fc1bc1016f0 2024-03-26T08:40:08.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.341+0000 7fc1b9d9b640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1a80a2800 0x7fc1a80a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc198ea0 tx=0x7fc1a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:08.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.345+0000 7fc1b37fe640 1 -- 172.21.15.7:0/743408970 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc1b4014030 con 0x7fc1bc1016f0 2024-03-26T08:40:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1108513732' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T08:40:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2865269869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T08:40:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:08.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:40:08.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:08 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:08.536+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc1bc108a10 con 0x7fc1bc1016f0 2024-03-26T08:40:08.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:08.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1108513732' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T08:40:08.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2865269869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T08:40:08.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:08.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:40:08.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:08 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1108513732' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T08:40:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2865269869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T08:40:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:08.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:08.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:40:08.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:08 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:09.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.209+0000 7fc1b37fe640 1 -- 172.21.15.7:0/743408970 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc1b409b4f0 con 0x7fc1bc1016f0 2024-03-26T08:40:09.202 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-03-26T08:40:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.214+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1a80a2800 msgr2=0x7fc1a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1a80a2800 0x7fc1a80a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc198ea0 tx=0x7fc1a400b390 comp rx=0 tx=0).stop 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1016f0 msgr2=0x7fc1bc197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1016f0 0x7fc1bc197990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400ccb0 tx=0x7fc1b400a620 comp rx=0 tx=0).stop 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 shutdown_connections 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1a80a2800 0x7fc1a80a4cc0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc198ea0 tx=0x7fc1a400b390 comp rx=0 tx=0).stop 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1bc1032a0 0x7fc1bc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1bc1028e0 0x7fc1bc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 --2- 172.21.15.7:0/743408970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1bc1016f0 0x7fc1bc197990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 >> 172.21.15.7:0/743408970 conn(0x7fc1bc0fcc60 msgr2=0x7fc1bc105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 shutdown_connections 2024-03-26T08:40:09.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.215+0000 7fc1c0ab2640 1 -- 172.21.15.7:0/743408970 wait complete. 2024-03-26T08:40:09.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-26T08:40:09.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.344+0000 7f71c8fc4640 1 -- 172.21.15.7:0/2491291745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 msgr2=0x7f71c41097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.344+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/2491291745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 0x7f71c41097f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009ca0 tx=0x7f71b402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.344+0000 7f71c8fc4640 1 -- 172.21.15.7:0/2491291745 shutdown_connections 2024-03-26T08:40:09.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.344+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/2491291745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 0x7f71c41097f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.344+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/2491291745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c4104a70 0x7f71c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.344+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/2491291745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c4102140 0x7f71c4104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.344+0000 7f71c8fc4640 1 -- 172.21.15.7:0/2491291745 >> 172.21.15.7:0/2491291745 conn(0x7f71c40fbb30 msgr2=0x7f71c40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.345+0000 7f71c8fc4640 1 -- 172.21.15.7:0/2491291745 shutdown_connections 2024-03-26T08:40:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.345+0000 7f71c8fc4640 1 -- 172.21.15.7:0/2491291745 wait complete. 2024-03-26T08:40:09.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.345+0000 7f71c8fc4640 1 Processor -- start 2024-03-26T08:40:09.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.345+0000 7f71c8fc4640 1 -- start start 2024-03-26T08:40:09.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.345+0000 7f71c8fc4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c4102140 0x7f71c4101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.345+0000 7f71c8fc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c4104a70 0x7f71c41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c8fc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 0x7f71c4100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c8fc4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c410d360 con 0x7f71c41073a0 2024-03-26T08:40:09.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c8fc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c410d1e0 con 0x7f71c4104a70 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c8fc4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c410d4e0 con 0x7f71c4102140 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 0x7f71c4100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c4102140 0x7f71c4101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 0x7f71c4100630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60452/0 (socket says 172.21.15.7:60452) 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c4102140 0x7f71c4101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35038/0 (socket says 172.21.15.7:35038) 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c2d9d640 1 -- 172.21.15.7:0/276852797 learned_addr learned my addr 172.21.15.7:0/276852797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c1d9b640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c4104a70 0x7f71c41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c2d9d640 1 -- 172.21.15.7:0/276852797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c4102140 msgr2=0x7f71c4101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c2d9d640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c4102140 0x7f71c4101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c2d9d640 1 -- 172.21.15.7:0/276852797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c4104a70 msgr2=0x7f71c41000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c2d9d640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c4104a70 0x7f71c41000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c2d9d640 1 -- 172.21.15.7:0/276852797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c4100ec0 con 0x7f71c41073a0 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c259c640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c4102140 0x7f71c4101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c2d9d640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 0x7f71c4100630 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71b4037040 tx=0x7f71b4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:09.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71bb7fe640 1 -- 172.21.15.7:0/276852797 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4041020 con 0x7f71c41073a0 2024-03-26T08:40:09.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c41a1a70 con 0x7f71c41073a0 2024-03-26T08:40:09.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71bb7fe640 1 -- 172.21.15.7:0/276852797 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71b4034cd0 con 0x7f71c41073a0 2024-03-26T08:40:09.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71bb7fe640 1 -- 172.21.15.7:0/276852797 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4039040 con 0x7f71c41073a0 2024-03-26T08:40:09.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.346+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c41a1fc0 con 0x7f71c41073a0 2024-03-26T08:40:09.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.347+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c40fd620 con 0x7f71c41073a0 2024-03-26T08:40:09.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.348+0000 7f71bb7fe640 1 -- 172.21.15.7:0/276852797 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f71b4044020 con 0x7f71c41073a0 2024-03-26T08:40:09.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.349+0000 7f71bb7fe640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b00a2800 0x7f71b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:09.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.349+0000 7f71c259c640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b00a2800 0x7f71b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:09.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.349+0000 7f71bb7fe640 1 -- 172.21.15.7:0/276852797 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f71b40f0c60 con 0x7f71c41073a0 2024-03-26T08:40:09.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.350+0000 7f71c259c640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b00a2800 0x7f71b00a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009fc0 tx=0x7f71bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:09.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.353+0000 7f71bb7fe640 1 -- 172.21.15.7:0/276852797 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71b404b020 con 0x7f71c41073a0 2024-03-26T08:40:09.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:09 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2865269869' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T08:40:09.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:09 smithi016 ceph-mon[35851]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T08:40:09.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:09.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:09 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:40:09.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:09 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/743408970' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T08:40:09.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:09 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/743408970' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T08:40:09.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:09 smithi016 ceph-mon[35851]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T08:40:09.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:09 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2865269869' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T08:40:09.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:09 smithi007 ceph-mon[32462]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T08:40:09.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:09.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:09 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:40:09.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:09 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/743408970' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T08:40:09.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:09 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/743408970' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T08:40:09.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:09 smithi007 ceph-mon[32462]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T08:40:09.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:09 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2865269869' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T08:40:09.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:09 smithi044 ceph-mon[36461]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T08:40:09.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:09.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:09 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:40:09.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:09 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/743408970' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T08:40:09.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:09 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/743408970' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T08:40:09.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:09 smithi044 ceph-mon[36461]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T08:40:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:09.580+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f71c4062310 con 0x7f71c41073a0 2024-03-26T08:40:10.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.244+0000 7f71bb7fe640 1 -- 172.21.15.7:0/276852797 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f71b40bdea0 con 0x7f71c41073a0 2024-03-26T08:40:10.237 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-03-26T08:40:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b00a2800 msgr2=0x7f71b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b00a2800 0x7f71b00a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009fc0 tx=0x7f71bc005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 msgr2=0x7f71c4100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 0x7f71c4100630 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71b4037040 tx=0x7f71b4032f40 comp rx=0 tx=0).stop 2024-03-26T08:40:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 shutdown_connections 2024-03-26T08:40:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b00a2800 0x7f71b00a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c41073a0 0x7f71c4100630 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c4104a70 0x7f71c41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 --2- 172.21.15.7:0/276852797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71c4102140 0x7f71c4101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 >> 172.21.15.7:0/276852797 conn(0x7f71c40fbb30 msgr2=0x7f71c4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 shutdown_connections 2024-03-26T08:40:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.247+0000 7f71c8fc4640 1 -- 172.21.15.7:0/276852797 wait complete. 2024-03-26T08:40:10.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:40:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.363+0000 7fb54ed7e640 1 -- 172.21.15.7:0/1243281946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548101830 msgr2=0x7fb548101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.363+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/1243281946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548101830 0x7fb548101cb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb534009900 tx=0x7fb53402bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.363+0000 7fb54ed7e640 1 -- 172.21.15.7:0/1243281946 shutdown_connections 2024-03-26T08:40:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.363+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/1243281946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481021f0 0x7fb548108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.363+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/1243281946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548101830 0x7fb548101cb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.363+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/1243281946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb548100640 0x7fb548100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.363+0000 7fb54ed7e640 1 -- 172.21.15.7:0/1243281946 >> 172.21.15.7:0/1243281946 conn(0x7fb5480fbbd0 msgr2=0x7fb5480fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.363+0000 7fb54ed7e640 1 -- 172.21.15.7:0/1243281946 shutdown_connections 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.363+0000 7fb54ed7e640 1 -- 172.21.15.7:0/1243281946 wait complete. 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54ed7e640 1 Processor -- start 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54ed7e640 1 -- start start 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54ed7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548100640 0x7fb548197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54ed7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb548101830 0x7fb548197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54ed7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481021f0 0x7fb54819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54ed7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54810c250 con 0x7fb548100640 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54ed7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54810c0d0 con 0x7fb5481021f0 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54ed7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54810c3d0 con 0x7fb548101830 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54cb1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548100640 0x7fb548197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54cb1a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548100640 0x7fb548197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60478/0 (socket says 172.21.15.7:60478) 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54cb1a640 1 -- 172.21.15.7:0/160931047 learned_addr learned my addr 172.21.15.7:0/160931047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb547fff640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb548101830 0x7fb548197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54cb1a640 1 -- 172.21.15.7:0/160931047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb548101830 msgr2=0x7fb548197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54cb1a640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb548101830 0x7fb548197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54d31b640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481021f0 0x7fb54819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54cb1a640 1 -- 172.21.15.7:0/160931047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481021f0 msgr2=0x7fb54819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54cb1a640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481021f0 0x7fb54819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54cb1a640 1 -- 172.21.15.7:0/160931047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54819f620 con 0x7fb548100640 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb54cb1a640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548100640 0x7fb548197a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d950 tx=0x7fb53c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.364+0000 7fb545ffb640 1 -- 172.21.15.7:0/160931047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c017070 con 0x7fb548100640 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.365+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54819f910 con 0x7fb548100640 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.365+0000 7fb545ffb640 1 -- 172.21.15.7:0/160931047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb53c009c30 con 0x7fb548100640 2024-03-26T08:40:10.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.365+0000 7fb545ffb640 1 -- 172.21.15.7:0/160931047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00f600 con 0x7fb548100640 2024-03-26T08:40:10.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.365+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54819fe30 con 0x7fb548100640 2024-03-26T08:40:10.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.366+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548068970 con 0x7fb548100640 2024-03-26T08:40:10.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.367+0000 7fb545ffb640 1 -- 172.21.15.7:0/160931047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb53c01a070 con 0x7fb548100640 2024-03-26T08:40:10.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.367+0000 7fb545ffb640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5380a2800 0x7fb5380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.367+0000 7fb547fff640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5380a2800 0x7fb5380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.367+0000 7fb545ffb640 1 -- 172.21.15.7:0/160931047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb53c0ce350 con 0x7fb548100640 2024-03-26T08:40:10.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.368+0000 7fb547fff640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5380a2800 0x7fb5380a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb53402c7b0 tx=0x7fb534005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.371+0000 7fb545ffb640 1 -- 172.21.15.7:0/160931047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb53c009de0 con 0x7fb548100640 2024-03-26T08:40:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T08:40:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/276852797' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T08:40:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T08:40:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T08:40:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:10.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:40:10.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:10.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T08:40:10.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/276852797' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T08:40:10.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T08:40:10.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T08:40:10.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:10.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:40:10.544 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.556+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5481a0290 con 0x7fb548100640 2024-03-26T08:40:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.557+0000 7fb545ffb640 1 -- 172.21.15.7:0/160931047 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22102 (secure 0 0 0) 0x7fb53c09b540 con 0x7fb548100640 2024-03-26T08:40:10.550 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:10.550 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:09.582943+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:10.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5380a2800 msgr2=0x7fb5380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5380a2800 0x7fb5380a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb53402c7b0 tx=0x7fb534005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548100640 msgr2=0x7fb548197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548100640 0x7fb548197a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d950 tx=0x7fb53c00de10 comp rx=0 tx=0).stop 2024-03-26T08:40:10.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 shutdown_connections 2024-03-26T08:40:10.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5380a2800 0x7fb5380a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481021f0 0x7fb54819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb548101830 0x7fb548197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 --2- 172.21.15.7:0/160931047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548100640 0x7fb548197a50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 >> 172.21.15.7:0/160931047 conn(0x7fb5480fbbd0 msgr2=0x7fb5480fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 shutdown_connections 2024-03-26T08:40:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.559+0000 7fb54ed7e640 1 -- 172.21.15.7:0/160931047 wait complete. 2024-03-26T08:40:10.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:40:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: pgmap v287: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T08:40:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/276852797' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T08:40:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T08:40:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T08:40:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:40:10.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:40:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.672+0000 7efde511d640 1 -- 172.21.15.7:0/1373554038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde0101c50 msgr2=0x7efde0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.672+0000 7efde511d640 1 --2- 172.21.15.7:0/1373554038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde0101c50 0x7efde0102070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009930 tx=0x7efdd802bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.672+0000 7efde511d640 1 -- 172.21.15.7:0/1373554038 shutdown_connections 2024-03-26T08:40:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.672+0000 7efde511d640 1 --2- 172.21.15.7:0/1373554038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde01012b0 0x7efde0108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.672+0000 7efde511d640 1 --2- 172.21.15.7:0/1373554038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde01008f0 0x7efde0100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.672+0000 7efde511d640 1 --2- 172.21.15.7:0/1373554038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde0101c50 0x7efde0102070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.672+0000 7efde511d640 1 -- 172.21.15.7:0/1373554038 >> 172.21.15.7:0/1373554038 conn(0x7efde00fb320 msgr2=0x7efde00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.672+0000 7efde511d640 1 -- 172.21.15.7:0/1373554038 shutdown_connections 2024-03-26T08:40:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.672+0000 7efde511d640 1 -- 172.21.15.7:0/1373554038 wait complete. 2024-03-26T08:40:10.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.673+0000 7efde511d640 1 Processor -- start 2024-03-26T08:40:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.673+0000 7efde511d640 1 -- start start 2024-03-26T08:40:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.673+0000 7efde511d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde01008f0 0x7efde0197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.673+0000 7efde511d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde01012b0 0x7efde0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.673+0000 7efde511d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde0101c50 0x7efde019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.673+0000 7efde511d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde010c270 con 0x7efde01008f0 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.673+0000 7efde511d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde010c0f0 con 0x7efde0101c50 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.673+0000 7efde511d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde010c3f0 con 0x7efde01012b0 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdded9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde01008f0 0x7efde0197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdded9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde01008f0 0x7efde0197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60504/0 (socket says 172.21.15.7:60504) 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdded9d640 1 -- 172.21.15.7:0/3712617982 learned_addr learned my addr 172.21.15.7:0/3712617982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdde59c640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde01012b0 0x7efde0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efddf59e640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde0101c50 0x7efde019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdded9d640 1 -- 172.21.15.7:0/3712617982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde01012b0 msgr2=0x7efde0197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdded9d640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde01012b0 0x7efde0197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdded9d640 1 -- 172.21.15.7:0/3712617982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde0101c50 msgr2=0x7efde019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdded9d640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde0101c50 0x7efde019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdded9d640 1 -- 172.21.15.7:0/3712617982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde019f580 con 0x7efde01008f0 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efddf59e640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde0101c50 0x7efde019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdded9d640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde01008f0 0x7efde0197960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efdd802c7b0 tx=0x7efdd802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdd7fff640 1 -- 172.21.15.7:0/3712617982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8041070 con 0x7efde01008f0 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdd7fff640 1 -- 172.21.15.7:0/3712617982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdd8004450 con 0x7efde01008f0 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde019f810 con 0x7efde01008f0 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efdd7fff640 1 -- 172.21.15.7:0/3712617982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8034440 con 0x7efde01008f0 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.674+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde019fd00 con 0x7efde01008f0 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.675+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde019ff60 con 0x7efde01008f0 2024-03-26T08:40:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.676+0000 7efdd7fff640 1 -- 172.21.15.7:0/3712617982 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7efdd8044070 con 0x7efde01008f0 2024-03-26T08:40:10.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.677+0000 7efdd7fff640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efdcc0a2800 0x7efdcc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.677+0000 7efdde59c640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efdcc0a2800 0x7efdcc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.677+0000 7efdd7fff640 1 -- 172.21.15.7:0/3712617982 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efdd80f0600 con 0x7efde01008f0 2024-03-26T08:40:10.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.677+0000 7efdde59c640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efdcc0a2800 0x7efdcc0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efde0198e70 tx=0x7efdc800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.681+0000 7efdd7fff640 1 -- 172.21.15.7:0/3712617982 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efdd80bd750 con 0x7efde01008f0 2024-03-26T08:40:10.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.867+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efde01a0130 con 0x7efde01008f0 2024-03-26T08:40:10.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.869+0000 7efdd7fff640 1 -- 172.21.15.7:0/3712617982 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22102 (secure 0 0 0) 0x7efdd80bd050 con 0x7efde01008f0 2024-03-26T08:40:10.861 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:10.861 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:09.582943+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:10.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efdcc0a2800 msgr2=0x7efdcc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efdcc0a2800 0x7efdcc0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efde0198e70 tx=0x7efdc800a2d0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde01008f0 msgr2=0x7efde0197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde01008f0 0x7efde0197960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efdd802c7b0 tx=0x7efdd802c6c0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 shutdown_connections 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efdcc0a2800 0x7efdcc0a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efde0101c50 0x7efde019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efde01012b0 0x7efde0197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 --2- 172.21.15.7:0/3712617982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde01008f0 0x7efde0197960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 >> 172.21.15.7:0/3712617982 conn(0x7efde00fb320 msgr2=0x7efde00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 shutdown_connections 2024-03-26T08:40:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.871+0000 7efde511d640 1 -- 172.21.15.7:0/3712617982 wait complete. 2024-03-26T08:40:10.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-26T08:40:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.988+0000 7fa7782b2640 1 -- 172.21.15.7:0/2682145337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701021f0 msgr2=0x7fa770108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.988+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2682145337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701021f0 0x7fa770108a80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa760009ca0 tx=0x7fa76002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.988+0000 7fa7782b2640 1 -- 172.21.15.7:0/2682145337 shutdown_connections 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.988+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2682145337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7701021f0 0x7fa770108a80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.988+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2682145337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa770101830 0x7fa770101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.988+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2682145337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa770100b20 0x7fa770100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.988+0000 7fa7782b2640 1 -- 172.21.15.7:0/2682145337 >> 172.21.15.7:0/2682145337 conn(0x7fa7700fb2f0 msgr2=0x7fa7700fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.988+0000 7fa7782b2640 1 -- 172.21.15.7:0/2682145337 shutdown_connections 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.988+0000 7fa7782b2640 1 -- 172.21.15.7:0/2682145337 wait complete. 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa7782b2640 1 Processor -- start 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa7782b2640 1 -- start start 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa7782b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa770100b20 0x7fa770197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa7782b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa770101830 0x7fa770197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa7782b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7701021f0 0x7fa77019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa7782b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77010d3e0 con 0x7fa770100b20 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa7782b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77010d260 con 0x7fa7701021f0 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa7782b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77010d560 con 0x7fa770101830 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77604e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa770100b20 0x7fa770197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77604e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa770100b20 0x7fa770197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60516/0 (socket says 172.21.15.7:60516) 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77604e640 1 -- 172.21.15.7:0/2681248710 learned_addr learned my addr 172.21.15.7:0/2681248710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77584d640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa770101830 0x7fa770197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77684f640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7701021f0 0x7fa77019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77604e640 1 -- 172.21.15.7:0/2681248710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa770101830 msgr2=0x7fa770197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77604e640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa770101830 0x7fa770197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77604e640 1 -- 172.21.15.7:0/2681248710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7701021f0 msgr2=0x7fa77019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:10.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77604e640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7701021f0 0x7fa77019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:10.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.989+0000 7fa77604e640 1 -- 172.21.15.7:0/2681248710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77019f5a0 con 0x7fa770100b20 2024-03-26T08:40:10.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.990+0000 7fa77604e640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa770100b20 0x7fa770197980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa76800e9c0 tx=0x7fa76800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:10.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.990+0000 7fa76f7fe640 1 -- 172.21.15.7:0/2681248710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800cd80 con 0x7fa770100b20 2024-03-26T08:40:10.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.990+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77019f890 con 0x7fa770100b20 2024-03-26T08:40:10.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.990+0000 7fa76f7fe640 1 -- 172.21.15.7:0/2681248710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7680035f0 con 0x7fa770100b20 2024-03-26T08:40:10.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.990+0000 7fa76f7fe640 1 -- 172.21.15.7:0/2681248710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768010690 con 0x7fa770100b20 2024-03-26T08:40:10.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.990+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77019fd90 con 0x7fa770100b20 2024-03-26T08:40:10.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.991+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa770107900 con 0x7fa770100b20 2024-03-26T08:40:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.992+0000 7fa76f7fe640 1 -- 172.21.15.7:0/2681248710 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa7680037a0 con 0x7fa770100b20 2024-03-26T08:40:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.992+0000 7fa76f7fe640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa7580a2800 0x7fa7580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.992+0000 7fa77584d640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa7580a2800 0x7fa7580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.992+0000 7fa76f7fe640 1 -- 172.21.15.7:0/2681248710 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa7680cec80 con 0x7fa770100b20 2024-03-26T08:40:10.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.993+0000 7fa77584d640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa7580a2800 0x7fa7580a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa770198e90 tx=0x7fa764005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:10.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:10.998+0000 7fa76f7fe640 1 -- 172.21.15.7:0/2681248710 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa76809be70 con 0x7fa770100b20 2024-03-26T08:40:11.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.239+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa77002d070 con 0x7fa770100b20 2024-03-26T08:40:11.493 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b[66227]: 2024-03-26T08:40:11.203+0000 7f832bbf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-26T08:40:11.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/276852797' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T08:40:11.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-mon[35851]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T08:40:11.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-mon[35851]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T08:40:11.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/160931047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:11.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] up:active 2024-03-26T08:40:11.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-mon[35851]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:40:11.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3712617982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:11.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-mon[35851]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T08:40:11.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2681248710' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T08:40:11.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/276852797' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T08:40:11.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:11 smithi007 ceph-mon[32462]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T08:40:11.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:11 smithi007 ceph-mon[32462]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T08:40:11.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/160931047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:11.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:11 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] up:active 2024-03-26T08:40:11.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:11 smithi007 ceph-mon[32462]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:40:11.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3712617982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:11.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:11 smithi007 ceph-mon[32462]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T08:40:11.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2681248710' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T08:40:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/276852797' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T08:40:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:11 smithi044 ceph-mon[36461]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T08:40:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:11 smithi044 ceph-mon[36461]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T08:40:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/160931047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:11 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] up:active 2024-03-26T08:40:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:11 smithi044 ceph-mon[36461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:40:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3712617982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:11 smithi044 ceph-mon[36461]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T08:40:11.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2681248710' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T08:40:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.921+0000 7fa76f7fe640 1 -- 172.21.15.7:0/2681248710 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa76809b770 con 0x7fa770100b20 2024-03-26T08:40:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa7580a2800 msgr2=0x7fa7580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa7580a2800 0x7fa7580a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa770198e90 tx=0x7fa764005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa770100b20 msgr2=0x7fa770197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa770100b20 0x7fa770197980 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa76800e9c0 tx=0x7fa76800ee80 comp rx=0 tx=0).stop 2024-03-26T08:40:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 shutdown_connections 2024-03-26T08:40:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa7580a2800 0x7fa7580a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7701021f0 0x7fa77019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa770101830 0x7fa770197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 --2- 172.21.15.7:0/2681248710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa770100b20 0x7fa770197980 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 >> 172.21.15.7:0/2681248710 conn(0x7fa7700fb2f0 msgr2=0x7fa770104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 shutdown_connections 2024-03-26T08:40:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:11.924+0000 7fa7782b2640 1 -- 172.21.15.7:0/2681248710 wait complete. 2024-03-26T08:40:11.929 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-26T08:40:11.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-26T08:40:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.050+0000 7f3c31c63640 1 -- 172.21.15.7:0/1802205539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 msgr2=0x7f3c2c1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.050+0000 7f3c31c63640 1 --2- 172.21.15.7:0/1802205539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 0x7f3c2c1009c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009ca0 tx=0x7f3c1c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.050+0000 7f3c31c63640 1 -- 172.21.15.7:0/1802205539 shutdown_connections 2024-03-26T08:40:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.050+0000 7f3c31c63640 1 --2- 172.21.15.7:0/1802205539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c102150 0x7f3c2c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.050+0000 7f3c31c63640 1 --2- 172.21.15.7:0/1802205539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2c101790 0x7f3c2c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.051+0000 7f3c31c63640 1 --2- 172.21.15.7:0/1802205539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 0x7f3c2c1009c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.051+0000 7f3c31c63640 1 -- 172.21.15.7:0/1802205539 >> 172.21.15.7:0/1802205539 conn(0x7f3c2c0fbb10 msgr2=0x7f3c2c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:12.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.051+0000 7f3c31c63640 1 -- 172.21.15.7:0/1802205539 shutdown_connections 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.051+0000 7f3c31c63640 1 -- 172.21.15.7:0/1802205539 wait complete. 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.051+0000 7f3c31c63640 1 Processor -- start 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.051+0000 7f3c31c63640 1 -- start start 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.051+0000 7f3c31c63640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 0x7f3c2c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.051+0000 7f3c31c63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c101790 0x7f3c2c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.051+0000 7f3c31c63640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2c102150 0x7f3c2c19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c31c63640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c10c230 con 0x7f3c2c1005a0 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c31c63640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c10c0b0 con 0x7f3c2c101790 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c31c63640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c10c3b0 con 0x7f3c2c102150 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 0x7f3c2c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 0x7f3c2c1968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60544/0 (socket says 172.21.15.7:60544) 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2b7fe640 1 -- 172.21.15.7:0/3583143609 learned_addr learned my addr 172.21.15.7:0/3583143609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2affd640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c101790 0x7f3c2c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2bfff640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2c102150 0x7f3c2c19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2b7fe640 1 -- 172.21.15.7:0/3583143609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2c102150 msgr2=0x7f3c2c19de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2b7fe640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2c102150 0x7f3c2c19de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2b7fe640 1 -- 172.21.15.7:0/3583143609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c101790 msgr2=0x7f3c2c196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2b7fe640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c101790 0x7f3c2c196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2b7fe640 1 -- 172.21.15.7:0/3583143609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c19e4d0 con 0x7f3c2c1005a0 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c2b7fe640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 0x7f3c2c1968b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0324c0 tx=0x7f3c1c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c28ff9640 1 -- 172.21.15.7:0/3583143609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c041020 con 0x7f3c2c1005a0 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c19e760 con 0x7f3c2c1005a0 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c28ff9640 1 -- 172.21.15.7:0/3583143609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c034cd0 con 0x7f3c2c1005a0 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c28ff9640 1 -- 172.21.15.7:0/3583143609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c039040 con 0x7f3c2c1005a0 2024-03-26T08:40:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.052+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2c19ec50 con 0x7f3c2c1005a0 2024-03-26T08:40:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.053+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2c107940 con 0x7f3c2c1005a0 2024-03-26T08:40:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.054+0000 7f3c28ff9640 1 -- 172.21.15.7:0/3583143609 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3c1c044020 con 0x7f3c2c1005a0 2024-03-26T08:40:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.055+0000 7f3c28ff9640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c140a2800 0x7f3c140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.055+0000 7f3c2affd640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c140a2800 0x7f3c140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.055+0000 7f3c28ff9640 1 -- 172.21.15.7:0/3583143609 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3c1c0f0800 con 0x7f3c2c1005a0 2024-03-26T08:40:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.055+0000 7f3c2affd640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c140a2800 0x7f3c140a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c197dc0 tx=0x7f3c2000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.059+0000 7f3c28ff9640 1 -- 172.21.15.7:0/3583143609 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c1c04b020 con 0x7f3c2c1005a0 2024-03-26T08:40:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:12.285+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f3c2c0622a0 con 0x7f3c2c1005a0 2024-03-26T08:40:12.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:12 smithi016 ceph-mon[35851]: pgmap v290: 97 pgs: 13 creating+peering, 6 active+clean, 78 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:12.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:12 smithi016 ceph-mon[35851]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T08:40:12.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:12 smithi016 ceph-mon[35851]: Cluster is now healthy 2024-03-26T08:40:12.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:12 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2681248710' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T08:40:12.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:12 smithi016 ceph-mon[35851]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:40:12.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:12 smithi007 ceph-mon[32462]: pgmap v290: 97 pgs: 13 creating+peering, 6 active+clean, 78 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:12.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:12 smithi007 ceph-mon[32462]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T08:40:12.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:12 smithi007 ceph-mon[32462]: Cluster is now healthy 2024-03-26T08:40:12.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:12 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2681248710' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T08:40:12.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:12 smithi007 ceph-mon[32462]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:40:12.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:12 smithi044 ceph-mon[36461]: pgmap v290: 97 pgs: 13 creating+peering, 6 active+clean, 78 unknown; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T08:40:12.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:12 smithi044 ceph-mon[36461]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T08:40:12.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:12 smithi044 ceph-mon[36461]: Cluster is now healthy 2024-03-26T08:40:12.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:12 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2681248710' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T08:40:12.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:12 smithi044 ceph-mon[36461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:40:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.257+0000 7f3c28ff9640 1 -- 172.21.15.7:0/3583143609 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3c1c0bd9e0 con 0x7f3c2c1005a0 2024-03-26T08:40:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c140a2800 msgr2=0x7f3c140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c140a2800 0x7f3c140a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c197dc0 tx=0x7f3c2000b390 comp rx=0 tx=0).stop 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 msgr2=0x7f3c2c1968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 0x7f3c2c1968b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0324c0 tx=0x7f3c1c032f40 comp rx=0 tx=0).stop 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 shutdown_connections 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c140a2800 0x7f3c140a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c2c102150 0x7f3c2c19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c101790 0x7f3c2c196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 --2- 172.21.15.7:0/3583143609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2c1005a0 0x7f3c2c1968b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 >> 172.21.15.7:0/3583143609 conn(0x7f3c2c0fbb10 msgr2=0x7f3c2c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 shutdown_connections 2024-03-26T08:40:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.260+0000 7f3c31c63640 1 -- 172.21.15.7:0/3583143609 wait complete. 2024-03-26T08:40:13.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-26T08:40:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.388+0000 7f79382b2640 1 -- 172.21.15.7:0/500203066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301016a0 msgr2=0x7f7930101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.388+0000 7f79382b2640 1 --2- 172.21.15.7:0/500203066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301016a0 0x7f7930101ac0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f792c009900 tx=0x7f792c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.388+0000 7f79382b2640 1 -- 172.21.15.7:0/500203066 shutdown_connections 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.388+0000 7f79382b2640 1 --2- 172.21.15.7:0/500203066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7930103250 0x7f7930109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.388+0000 7f79382b2640 1 --2- 172.21.15.7:0/500203066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7930102890 0x7f7930102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.388+0000 7f79382b2640 1 --2- 172.21.15.7:0/500203066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79301016a0 0x7f7930101ac0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.388+0000 7f79382b2640 1 -- 172.21.15.7:0/500203066 >> 172.21.15.7:0/500203066 conn(0x7f79300fcbd0 msgr2=0x7f79300ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.388+0000 7f79382b2640 1 -- 172.21.15.7:0/500203066 shutdown_connections 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.388+0000 7f79382b2640 1 -- 172.21.15.7:0/500203066 wait complete. 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f79382b2640 1 Processor -- start 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f79382b2640 1 -- start start 2024-03-26T08:40:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f79382b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79301016a0 0x7f79301979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f79382b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7930102890 0x7f7930197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f79382b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7930103250 0x7f793019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f79382b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793010d320 con 0x7f7930102890 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f79382b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793010d1a0 con 0x7f7930103250 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f79382b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793010d4a0 con 0x7f79301016a0 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793584d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7930102890 0x7f7930197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793584d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7930102890 0x7f7930197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60562/0 (socket says 172.21.15.7:60562) 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793584d640 1 -- 172.21.15.7:0/1151512787 learned_addr learned my addr 172.21.15.7:0/1151512787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793684f640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7930103250 0x7f793019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793584d640 1 -- 172.21.15.7:0/1151512787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79301016a0 msgr2=0x7f79301979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793604e640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79301016a0 0x7f79301979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793584d640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79301016a0 0x7f79301979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793584d640 1 -- 172.21.15.7:0/1151512787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7930103250 msgr2=0x7f793019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793584d640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7930103250 0x7f793019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793584d640 1 -- 172.21.15.7:0/1151512787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793019f5f0 con 0x7f7930102890 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.389+0000 7f793584d640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7930102890 0x7f7930197f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f792000e9f0 tx=0x7f792000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.390+0000 7f79277fe640 1 -- 172.21.15.7:0/1151512787 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792000cd80 con 0x7f7930102890 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.390+0000 7f79277fe640 1 -- 172.21.15.7:0/1151512787 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7920010910 con 0x7f7930102890 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.390+0000 7f79277fe640 1 -- 172.21.15.7:0/1151512787 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920018990 con 0x7f7930102890 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.390+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793019f8e0 con 0x7f7930102890 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.390+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793019fe80 con 0x7f7930102890 2024-03-26T08:40:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.391+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79301089c0 con 0x7f7930102890 2024-03-26T08:40:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.392+0000 7f79277fe640 1 -- 172.21.15.7:0/1151512787 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7920010ac0 con 0x7f7930102890 2024-03-26T08:40:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.392+0000 7f79277fe640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f791c0a2880 0x7f791c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.392+0000 7f793604e640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f791c0a2880 0x7f791c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.392+0000 7f79277fe640 1 -- 172.21.15.7:0/1151512787 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f79200cec50 con 0x7f7930102890 2024-03-26T08:40:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.393+0000 7f793604e640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f791c0a2880 0x7f791c0a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f792c036040 tx=0x7f792c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.397+0000 7f79277fe640 1 -- 172.21.15.7:0/1151512787 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f79200d4020 con 0x7f7930102890 2024-03-26T08:40:13.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:13 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3583143609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-26T08:40:13.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:13 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3583143609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-26T08:40:13.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:13.620+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f793002d070 con 0x7f7930102890 2024-03-26T08:40:13.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:13 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3583143609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-26T08:40:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.263+0000 7f79277fe640 1 -- 172.21.15.7:0/1151512787 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f792009be40 con 0x7f7930102890 2024-03-26T08:40:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f791c0a2880 msgr2=0x7f791c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f791c0a2880 0x7f791c0a4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f792c036040 tx=0x7f792c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7930102890 msgr2=0x7f7930197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7930102890 0x7f7930197f10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f792000e9f0 tx=0x7f792000eeb0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 shutdown_connections 2024-03-26T08:40:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f791c0a2880 0x7f791c0a4d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7930103250 0x7f793019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7930102890 0x7f7930197f10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 --2- 172.21.15.7:0/1151512787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79301016a0 0x7f79301979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 >> 172.21.15.7:0/1151512787 conn(0x7f79300fcbd0 msgr2=0x7f7930105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 shutdown_connections 2024-03-26T08:40:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.266+0000 7f79382b2640 1 -- 172.21.15.7:0/1151512787 wait complete. 2024-03-26T08:40:14.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-26T08:40:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.394+0000 7f82e0b90640 1 -- 172.21.15.7:0/2887703337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 msgr2=0x7f82dc109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.394+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2887703337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 0x7f82dc109c90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009ca0 tx=0x7f82cc02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.394+0000 7f82e0b90640 1 -- 172.21.15.7:0/2887703337 shutdown_connections 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.394+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2887703337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 0x7f82dc109c90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.394+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2887703337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82dc0ff720 0x7f82dc0ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.394+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2887703337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc102cb0 0x7f82dc1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.394+0000 7f82e0b90640 1 -- 172.21.15.7:0/2887703337 >> 172.21.15.7:0/2887703337 conn(0x7f82dc0fb2c0 msgr2=0x7f82dc0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.394+0000 7f82e0b90640 1 -- 172.21.15.7:0/2887703337 shutdown_connections 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.394+0000 7f82e0b90640 1 -- 172.21.15.7:0/2887703337 wait complete. 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82e0b90640 1 Processor -- start 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82e0b90640 1 -- start start 2024-03-26T08:40:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82e0b90640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82dc0ff720 0x7f82dc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82e0b90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 0x7f82dc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82e0b90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc102cb0 0x7f82dc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82e0b90640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc10d360 con 0x7f82dc1000e0 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82e0b90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc10d1e0 con 0x7f82dc102cb0 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82e0b90640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc10d4e0 con 0x7f82dc0ff720 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82d9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 0x7f82dc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82d9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 0x7f82dc197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60582/0 (socket says 172.21.15.7:60582) 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82d9d9b640 1 -- 172.21.15.7:0/2294200513 learned_addr learned my addr 172.21.15.7:0/2294200513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82da59c640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82dc0ff720 0x7f82dc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82dad9d640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc102cb0 0x7f82dc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82d9d9b640 1 -- 172.21.15.7:0/2294200513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82dc0ff720 msgr2=0x7f82dc1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82d9d9b640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82dc0ff720 0x7f82dc1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82d9d9b640 1 -- 172.21.15.7:0/2294200513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc102cb0 msgr2=0x7f82dc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82d9d9b640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc102cb0 0x7f82dc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82d9d9b640 1 -- 172.21.15.7:0/2294200513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc19f5c0 con 0x7f82dc1000e0 2024-03-26T08:40:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.395+0000 7f82d9d9b640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 0x7f82dc197ee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f82d000e9f0 tx=0x7f82d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.396+0000 7f82cb7fe640 1 -- 172.21.15.7:0/2294200513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d000cd80 con 0x7f82dc1000e0 2024-03-26T08:40:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.396+0000 7f82cb7fe640 1 -- 172.21.15.7:0/2294200513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82d0005500 con 0x7f82dc1000e0 2024-03-26T08:40:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.396+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc19f8b0 con 0x7f82dc1000e0 2024-03-26T08:40:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.396+0000 7f82cb7fe640 1 -- 172.21.15.7:0/2294200513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0010690 con 0x7f82dc1000e0 2024-03-26T08:40:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.396+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc19fdb0 con 0x7f82dc1000e0 2024-03-26T08:40:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.397+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc108b10 con 0x7f82dc1000e0 2024-03-26T08:40:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.398+0000 7f82cb7fe640 1 -- 172.21.15.7:0/2294200513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f82d00046c0 con 0x7f82dc1000e0 2024-03-26T08:40:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.398+0000 7f82cb7fe640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82c40a2800 0x7f82c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.398+0000 7f82da59c640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82c40a2800 0x7f82c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.398+0000 7f82cb7fe640 1 -- 172.21.15.7:0/2294200513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f82d00ced00 con 0x7f82dc1000e0 2024-03-26T08:40:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.399+0000 7f82da59c640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82c40a2800 0x7f82c40a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f82dc069a60 tx=0x7f82d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:14.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.403+0000 7f82cb7fe640 1 -- 172.21.15.7:0/2294200513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f82d009bef0 con 0x7f82dc1000e0 2024-03-26T08:40:14.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:14 smithi007 ceph-mon[32462]: pgmap v292: 97 pgs: 13 creating+peering, 61 active+clean, 23 unknown; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 4 op/s 2024-03-26T08:40:14.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:14 smithi007 ceph-mon[32462]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T08:40:14.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:14 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3583143609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-26T08:40:14.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:14 smithi007 ceph-mon[32462]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T08:40:14.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:14 smithi007 ceph-mon[32462]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:40:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:14 smithi007 ceph-mon[32462]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-26T08:40:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:14 smithi007 ceph-mon[32462]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-26T08:40:14.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:14 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1151512787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T08:40:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:14 smithi044 ceph-mon[36461]: pgmap v292: 97 pgs: 13 creating+peering, 61 active+clean, 23 unknown; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 4 op/s 2024-03-26T08:40:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:14 smithi044 ceph-mon[36461]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T08:40:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:14 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3583143609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-26T08:40:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:14 smithi044 ceph-mon[36461]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T08:40:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:14 smithi044 ceph-mon[36461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:40:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:14 smithi044 ceph-mon[36461]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-26T08:40:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:14 smithi044 ceph-mon[36461]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-26T08:40:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:14 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1151512787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T08:40:14.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:14.625+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f82dc02d070 con 0x7f82dc1000e0 2024-03-26T08:40:14.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:14 smithi016 ceph-mon[35851]: pgmap v292: 97 pgs: 13 creating+peering, 61 active+clean, 23 unknown; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 4 op/s 2024-03-26T08:40:14.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:14 smithi016 ceph-mon[35851]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T08:40:14.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:14 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3583143609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-26T08:40:14.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:14 smithi016 ceph-mon[35851]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T08:40:14.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:14 smithi016 ceph-mon[35851]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T08:40:14.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:14 smithi016 ceph-mon[35851]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-03-26T08:40:14.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:14 smithi016 ceph-mon[35851]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-03-26T08:40:14.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:14 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1151512787' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T08:40:15.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.273+0000 7f82cb7fe640 1 -- 172.21.15.7:0/2294200513 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f82d009b7f0 con 0x7f82dc1000e0 2024-03-26T08:40:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82c40a2800 msgr2=0x7f82c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82c40a2800 0x7f82c40a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f82dc069a60 tx=0x7f82d4005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:15.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 msgr2=0x7f82dc197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 0x7f82dc197ee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f82d000e9f0 tx=0x7f82d000edf0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 shutdown_connections 2024-03-26T08:40:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82c40a2800 0x7f82c40a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc102cb0 0x7f82dc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc1000e0 0x7f82dc197ee0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 --2- 172.21.15.7:0/2294200513 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82dc0ff720 0x7f82dc1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 >> 172.21.15.7:0/2294200513 conn(0x7f82dc0fb2c0 msgr2=0x7f82dc105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 shutdown_connections 2024-03-26T08:40:15.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.276+0000 7f82e0b90640 1 -- 172.21.15.7:0/2294200513 wait complete. 2024-03-26T08:40:15.282 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-26T08:40:15.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.402+0000 7f1381819640 1 -- 172.21.15.7:0/3159818208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c1005c0 msgr2=0x7f137c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.402+0000 7f1381819640 1 --2- 172.21.15.7:0/3159818208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c1005c0 0x7f137c1009e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1374009860 tx=0x7f137402bb70 comp rx=0 tx=0).stop 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.402+0000 7f1381819640 1 -- 172.21.15.7:0/3159818208 shutdown_connections 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.402+0000 7f1381819640 1 --2- 172.21.15.7:0/3159818208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c102170 0x7f137c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.402+0000 7f1381819640 1 --2- 172.21.15.7:0/3159818208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137c1017b0 0x7f137c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.402+0000 7f1381819640 1 --2- 172.21.15.7:0/3159818208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c1005c0 0x7f137c1009e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.402+0000 7f1381819640 1 -- 172.21.15.7:0/3159818208 >> 172.21.15.7:0/3159818208 conn(0x7f137c0fbb50 msgr2=0x7f137c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.402+0000 7f1381819640 1 -- 172.21.15.7:0/3159818208 shutdown_connections 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.402+0000 7f1381819640 1 -- 172.21.15.7:0/3159818208 wait complete. 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f1381819640 1 Processor -- start 2024-03-26T08:40:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f1381819640 1 -- start start 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f1381819640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c1005c0 0x7f137c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f1381819640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137c1017b0 0x7f137c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f1381819640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c102170 0x7f137c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f1381819640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c10c210 con 0x7f137c102170 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f1381819640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c10c090 con 0x7f137c1017b0 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f1381819640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c10c390 con 0x7f137c1005c0 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c102170 0x7f137c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c102170 0x7f137c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60592/0 (socket says 172.21.15.7:60592) 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137b7fe640 1 -- 172.21.15.7:0/1765046822 learned_addr learned my addr 172.21.15.7:0/1765046822 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137affd640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c1005c0 0x7f137c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137a7fc640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137c1017b0 0x7f137c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137affd640 1 -- 172.21.15.7:0/1765046822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137c1017b0 msgr2=0x7f137c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137affd640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137c1017b0 0x7f137c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137affd640 1 -- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c102170 msgr2=0x7f137c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137affd640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c102170 0x7f137c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137affd640 1 -- 172.21.15.7:0/1765046822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c19f600 con 0x7f137c1005c0 2024-03-26T08:40:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.403+0000 7f137b7fe640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c102170 0x7f137c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.404+0000 7f137affd640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c1005c0 0x7f137c197a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f137402c0b0 tx=0x7f1374032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.404+0000 7f1380817640 1 -- 172.21.15.7:0/1765046822 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374041020 con 0x7f137c1005c0 2024-03-26T08:40:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.404+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c19f890 con 0x7f137c1005c0 2024-03-26T08:40:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.404+0000 7f1380817640 1 -- 172.21.15.7:0/1765046822 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f137402c9f0 con 0x7f137c1005c0 2024-03-26T08:40:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.404+0000 7f1380817640 1 -- 172.21.15.7:0/1765046822 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374039040 con 0x7f137c1005c0 2024-03-26T08:40:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.404+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c19fdd0 con 0x7f137c1005c0 2024-03-26T08:40:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.405+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137c1a0090 con 0x7f137c1005c0 2024-03-26T08:40:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.406+0000 7f1380817640 1 -- 172.21.15.7:0/1765046822 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1374044020 con 0x7f137c1005c0 2024-03-26T08:40:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.406+0000 7f1380817640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13640a2800 0x7f13640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.407+0000 7f137a7fc640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13640a2800 0x7f13640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.407+0000 7f1380817640 1 -- 172.21.15.7:0/1765046822 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f13740f0530 con 0x7f137c1005c0 2024-03-26T08:40:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.407+0000 7f137a7fc640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13640a2800 0x7f13640a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13700095f0 tx=0x7f1370005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.410+0000 7f1380817640 1 -- 172.21.15.7:0/1765046822 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f137404b020 con 0x7f137c1005c0 2024-03-26T08:40:15.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1151512787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T08:40:15.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] up:active 2024-03-26T08:40:15.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-mon[32462]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T08:40:15.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-mon[32462]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T08:40:15.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-mon[32462]: Cluster is now healthy 2024-03-26T08:40:15.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-mon[32462]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-26T08:40:15.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-mon[32462]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-03-26T08:40:15.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-mon[32462]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-26T08:40:15.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2294200513' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T08:40:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:15 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1151512787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T08:40:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:15 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] up:active 2024-03-26T08:40:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:15 smithi044 ceph-mon[36461]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T08:40:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:15 smithi044 ceph-mon[36461]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T08:40:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:15 smithi044 ceph-mon[36461]: Cluster is now healthy 2024-03-26T08:40:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:15 smithi044 ceph-mon[36461]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-26T08:40:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:15 smithi044 ceph-mon[36461]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-03-26T08:40:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:15 smithi044 ceph-mon[36461]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-26T08:40:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:15 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2294200513' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T08:40:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.620+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f137c0fdd40 con 0x7f13640a2800 2024-03-26T08:40:15.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.643+0000 7f1380817640 1 -- 172.21.15.7:0/1765046822 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f137c0fdd40 con 0x7f13640a2800 2024-03-26T08:40:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13640a2800 msgr2=0x7f13640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13640a2800 0x7f13640a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13700095f0 tx=0x7f1370005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c1005c0 msgr2=0x7f137c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c1005c0 0x7f137c197a30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f137402c0b0 tx=0x7f1374032430 comp rx=0 tx=0).stop 2024-03-26T08:40:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 shutdown_connections 2024-03-26T08:40:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13640a2800 0x7f13640a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f137c102170 0x7f137c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137c1017b0 0x7f137c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 --2- 172.21.15.7:0/1765046822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f137c1005c0 0x7f137c197a30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 >> 172.21.15.7:0/1765046822 conn(0x7f137c0fbb50 msgr2=0x7f137c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:15.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 shutdown_connections 2024-03-26T08:40:15.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.645+0000 7f1381819640 1 -- 172.21.15.7:0/1765046822 wait complete. 2024-03-26T08:40:15.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-26T08:40:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:15 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1151512787' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T08:40:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:15 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] up:active 2024-03-26T08:40:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:15 smithi016 ceph-mon[35851]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T08:40:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:15 smithi016 ceph-mon[35851]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T08:40:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:15 smithi016 ceph-mon[35851]: Cluster is now healthy 2024-03-26T08:40:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:15 smithi016 ceph-mon[35851]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-03-26T08:40:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:15 smithi016 ceph-mon[35851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-03-26T08:40:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:15 smithi016 ceph-mon[35851]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-26T08:40:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:15 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2294200513' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.764+0000 7f91c1576640 1 -- 172.21.15.7:0/292245043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc1016d0 msgr2=0x7f91bc101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.764+0000 7f91c1576640 1 --2- 172.21.15.7:0/292245043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc1016d0 0x7f91bc101af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009900 tx=0x7f91b402bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.764+0000 7f91c1576640 1 -- 172.21.15.7:0/292245043 shutdown_connections 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.764+0000 7f91c1576640 1 --2- 172.21.15.7:0/292245043 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc103280 0x7f91bc109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.764+0000 7f91c1576640 1 --2- 172.21.15.7:0/292245043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc1028c0 0x7f91bc102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.764+0000 7f91c1576640 1 --2- 172.21.15.7:0/292245043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc1016d0 0x7f91bc101af0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.764+0000 7f91c1576640 1 -- 172.21.15.7:0/292245043 >> 172.21.15.7:0/292245043 conn(0x7f91bc0fcc60 msgr2=0x7f91bc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.764+0000 7f91c1576640 1 -- 172.21.15.7:0/292245043 shutdown_connections 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.764+0000 7f91c1576640 1 -- 172.21.15.7:0/292245043 wait complete. 2024-03-26T08:40:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91c1576640 1 Processor -- start 2024-03-26T08:40:15.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91c1576640 1 -- start start 2024-03-26T08:40:15.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91c1576640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc1016d0 0x7f91bc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91c1576640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc1028c0 0x7f91bc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91c1576640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc103280 0x7f91bc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91c1576640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc10d320 con 0x7f91bc103280 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91c1576640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc10d1a0 con 0x7f91bc1028c0 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91c1576640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc10d4a0 con 0x7f91bc1016d0 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc103280 0x7f91bc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91baffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc1016d0 0x7f91bc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91bb7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc103280 0x7f91bc19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60632/0 (socket says 172.21.15.7:60632) 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91bb7fe640 1 -- 172.21.15.7:0/2357700241 learned_addr learned my addr 172.21.15.7:0/2357700241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91ba7fc640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc1028c0 0x7f91bc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91bb7fe640 1 -- 172.21.15.7:0/2357700241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc1016d0 msgr2=0x7f91bc197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91bb7fe640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc1016d0 0x7f91bc197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91bb7fe640 1 -- 172.21.15.7:0/2357700241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc1028c0 msgr2=0x7f91bc197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91bb7fe640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc1028c0 0x7f91bc197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.765+0000 7f91bb7fe640 1 -- 172.21.15.7:0/2357700241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc19f640 con 0x7f91bc103280 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.766+0000 7f91bb7fe640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc103280 0x7f91bc19efd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00efd0 tx=0x7f91ac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.766+0000 7f91b3fff640 1 -- 172.21.15.7:0/2357700241 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac003b70 con 0x7f91bc103280 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.766+0000 7f91b3fff640 1 -- 172.21.15.7:0/2357700241 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91ac003d10 con 0x7f91bc103280 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.766+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc19f930 con 0x7f91bc103280 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.766+0000 7f91b3fff640 1 -- 172.21.15.7:0/2357700241 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0109f0 con 0x7f91bc103280 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.766+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc19fe30 con 0x7f91bc103280 2024-03-26T08:40:15.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.767+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc108a60 con 0x7f91bc103280 2024-03-26T08:40:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.768+0000 7f91b3fff640 1 -- 172.21.15.7:0/2357700241 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f91ac004020 con 0x7f91bc103280 2024-03-26T08:40:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.768+0000 7f91b3fff640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f91a80a2800 0x7f91a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.768+0000 7f91baffd640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f91a80a2800 0x7f91a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.769+0000 7f91b3fff640 1 -- 172.21.15.7:0/2357700241 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f91ac0cee20 con 0x7f91bc103280 2024-03-26T08:40:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.769+0000 7f91baffd640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f91a80a2800 0x7f91a80a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f91b4000c00 tx=0x7f91b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.773+0000 7f91b3fff640 1 -- 172.21.15.7:0/2357700241 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f91ac09bf70 con 0x7f91bc103280 2024-03-26T08:40:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:15.997+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f91bc061ae0 con 0x7f91a80a2800 2024-03-26T08:40:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.027+0000 7f91b3fff640 1 -- 172.21.15.7:0/2357700241 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f91bc061ae0 con 0x7f91a80a2800 2024-03-26T08:40:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f91a80a2800 msgr2=0x7f91a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f91a80a2800 0x7f91a80a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f91b4000c00 tx=0x7f91b4005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc103280 msgr2=0x7f91bc19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc103280 0x7f91bc19efd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00efd0 tx=0x7f91ac00c6a0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 shutdown_connections 2024-03-26T08:40:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f91a80a2800 0x7f91a80a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91bc103280 0x7f91bc19efd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91bc1028c0 0x7f91bc197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 --2- 172.21.15.7:0/2357700241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91bc1016d0 0x7f91bc197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.029+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 >> 172.21.15.7:0/2357700241 conn(0x7f91bc0fcc60 msgr2=0x7f91bc105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.030+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 shutdown_connections 2024-03-26T08:40:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.030+0000 7f91c1576640 1 -- 172.21.15.7:0/2357700241 wait complete. 2024-03-26T08:40:16.031 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:40:15.624+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:40:16.031 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:40:15.624+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:40:16.031 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:40:15.624+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:40:16.031 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:40:15.624+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:40:16.031 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:40:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:40:15.624+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:40:16.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-26T08:40:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.148+0000 7f1334bc6640 1 -- 172.21.15.7:0/3027037884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1330107420 msgr2=0x7f1330109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.148+0000 7f1334bc6640 1 --2- 172.21.15.7:0/3027037884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1330107420 0x7f1330109810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1320009ca0 tx=0x7f132002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.148+0000 7f1334bc6640 1 -- 172.21.15.7:0/3027037884 shutdown_connections 2024-03-26T08:40:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.148+0000 7f1334bc6640 1 --2- 172.21.15.7:0/3027037884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1330107420 0x7f1330109810 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.148+0000 7f1334bc6640 1 --2- 172.21.15.7:0/3027037884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1330104af0 0x7f1330106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.148+0000 7f1334bc6640 1 --2- 172.21.15.7:0/3027037884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13301021c0 0x7f13301045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.148+0000 7f1334bc6640 1 -- 172.21.15.7:0/3027037884 >> 172.21.15.7:0/3027037884 conn(0x7f13300fbb30 msgr2=0x7f13300fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.148+0000 7f1334bc6640 1 -- 172.21.15.7:0/3027037884 shutdown_connections 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f1334bc6640 1 -- 172.21.15.7:0/3027037884 wait complete. 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f1334bc6640 1 Processor -- start 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f1334bc6640 1 -- start start 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f1334bc6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13301021c0 0x7f13301979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f1334bc6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1330104af0 0x7f1330197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f1334bc6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1330107420 0x7f133019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f1334bc6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133010d360 con 0x7f13301021c0 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f1334bc6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133010d1e0 con 0x7f1330104af0 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f1334bc6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133010d4e0 con 0x7f1330107420 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f132e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13301021c0 0x7f13301979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f132e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13301021c0 0x7f13301979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60650/0 (socket says 172.21.15.7:60650) 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.149+0000 7f132e59c640 1 -- 172.21.15.7:0/1174043193 learned_addr learned my addr 172.21.15.7:0/1174043193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f132ed9d640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1330107420 0x7f133019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f132dd9b640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1330104af0 0x7f1330197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f132e59c640 1 -- 172.21.15.7:0/1174043193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1330107420 msgr2=0x7f133019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f132e59c640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1330107420 0x7f133019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f132e59c640 1 -- 172.21.15.7:0/1174043193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1330104af0 msgr2=0x7f1330197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f132e59c640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1330104af0 0x7f1330197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f132e59c640 1 -- 172.21.15.7:0/1174043193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133019f5f0 con 0x7f13301021c0 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f132e59c640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13301021c0 0x7f13301979d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f132800e9f0 tx=0x7f132800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f131f7fe640 1 -- 172.21.15.7:0/1174043193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132800cd80 con 0x7f13301021c0 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133019f8e0 con 0x7f13301021c0 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f131f7fe640 1 -- 172.21.15.7:0/1174043193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1328005500 con 0x7f13301021c0 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f131f7fe640 1 -- 172.21.15.7:0/1174043193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328010690 con 0x7f13301021c0 2024-03-26T08:40:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.150+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133019fe30 con 0x7f13301021c0 2024-03-26T08:40:16.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.151+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13300fd620 con 0x7f13301021c0 2024-03-26T08:40:16.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.152+0000 7f131f7fe640 1 -- 172.21.15.7:0/1174043193 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f13280046c0 con 0x7f13301021c0 2024-03-26T08:40:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.153+0000 7f131f7fe640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13180a2800 0x7f13180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.153+0000 7f132dd9b640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13180a2800 0x7f13180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.153+0000 7f131f7fe640 1 -- 172.21.15.7:0/1174043193 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f13280ceda0 con 0x7f13301021c0 2024-03-26T08:40:16.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.153+0000 7f132dd9b640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13180a2800 0x7f13180a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1330198ee0 tx=0x7f1324005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:16.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.157+0000 7f131f7fe640 1 -- 172.21.15.7:0/1174043193 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f132809bef0 con 0x7f13301021c0 2024-03-26T08:40:16.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:16 smithi007 ceph-mon[32462]: pgmap v293: 97 pgs: 97 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-26T08:40:16.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:16 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2294200513' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T08:40:16.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:16 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] up:active 2024-03-26T08:40:16.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:16 smithi007 ceph-mon[32462]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-03-26T08:40:16.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:16 smithi007 ceph-mon[32462]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:40:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.371+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f1330061ae0 con 0x7f13180a2800 2024-03-26T08:40:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.379+0000 7f131f7fe640 1 -- 172.21.15.7:0/1174043193 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f1330061ae0 con 0x7f13180a2800 2024-03-26T08:40:16.371 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_0/e4e86d2d-0f86-42b9-a59a-7bb96dc24e83 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13180a2800 msgr2=0x7f13180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13180a2800 0x7f13180a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1330198ee0 tx=0x7f1324005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13301021c0 msgr2=0x7f13301979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13301021c0 0x7f13301979d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f132800e9f0 tx=0x7f132800edf0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 shutdown_connections 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13180a2800 0x7f13180a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1330107420 0x7f133019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1330104af0 0x7f1330197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 --2- 172.21.15.7:0/1174043193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13301021c0 0x7f13301979d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 >> 172.21.15.7:0/1174043193 conn(0x7f13300fbb30 msgr2=0x7f1330102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 shutdown_connections 2024-03-26T08:40:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.381+0000 7f1334bc6640 1 -- 172.21.15.7:0/1174043193 wait complete. 2024-03-26T08:40:16.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-26T08:40:16.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.499+0000 7f2ff27cd640 1 -- 172.21.15.7:0/2979346998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 msgr2=0x7f2fec108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.499+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/2979346998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 0x7f2fec108a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009ca0 tx=0x7f2fe402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.499+0000 7f2ff27cd640 1 -- 172.21.15.7:0/2979346998 shutdown_connections 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.499+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/2979346998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 0x7f2fec108a00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.499+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/2979346998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fec1017b0 0x7f2fec101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.499+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/2979346998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fec1005c0 0x7f2fec1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.499+0000 7f2ff27cd640 1 -- 172.21.15.7:0/2979346998 >> 172.21.15.7:0/2979346998 conn(0x7f2fec0fbb50 msgr2=0x7f2fec0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.499+0000 7f2ff27cd640 1 -- 172.21.15.7:0/2979346998 shutdown_connections 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.499+0000 7f2ff27cd640 1 -- 172.21.15.7:0/2979346998 wait complete. 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff27cd640 1 Processor -- start 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff27cd640 1 -- start start 2024-03-26T08:40:16.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff27cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fec1005c0 0x7f2fec197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff27cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fec1017b0 0x7f2fec197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff27cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 0x7f2fec19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff27cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec10c240 con 0x7f2fec102170 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff27cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec10c0c0 con 0x7f2fec1017b0 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff27cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec10c3c0 con 0x7f2fec1005c0 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff0d6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 0x7f2fec19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff0d6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 0x7f2fec19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60668/0 (socket says 172.21.15.7:60668) 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff0d6a640 1 -- 172.21.15.7:0/3342043003 learned_addr learned my addr 172.21.15.7:0/3342043003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2febfff640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fec1005c0 0x7f2fec197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2feb7fe640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fec1017b0 0x7f2fec197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff0d6a640 1 -- 172.21.15.7:0/3342043003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fec1005c0 msgr2=0x7f2fec197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff0d6a640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fec1005c0 0x7f2fec197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff0d6a640 1 -- 172.21.15.7:0/3342043003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fec1017b0 msgr2=0x7f2fec197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff0d6a640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fec1017b0 0x7f2fec197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.500+0000 7f2ff0d6a640 1 -- 172.21.15.7:0/3342043003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec19f580 con 0x7f2fec102170 2024-03-26T08:40:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.501+0000 7f2ff0d6a640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 0x7f2fec19ef10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fe402c630 tx=0x7f2fe4032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.501+0000 7f2fe97fa640 1 -- 172.21.15.7:0/3342043003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4041020 con 0x7f2fec102170 2024-03-26T08:40:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.501+0000 7f2fe97fa640 1 -- 172.21.15.7:0/3342043003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fe4034cd0 con 0x7f2fec102170 2024-03-26T08:40:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.501+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fec19f810 con 0x7f2fec102170 2024-03-26T08:40:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.501+0000 7f2fe97fa640 1 -- 172.21.15.7:0/3342043003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4039040 con 0x7f2fec102170 2024-03-26T08:40:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.501+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec19fd50 con 0x7f2fec102170 2024-03-26T08:40:16.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.502+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fec1a0070 con 0x7f2fec102170 2024-03-26T08:40:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.503+0000 7f2fe97fa640 1 -- 172.21.15.7:0/3342043003 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2fe4044020 con 0x7f2fec102170 2024-03-26T08:40:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.503+0000 7f2fe97fa640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2fd80a2880 0x7f2fd80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.504+0000 7f2febfff640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2fd80a2880 0x7f2fd80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.504+0000 7f2fe97fa640 1 -- 172.21.15.7:0/3342043003 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2fe40f06e0 con 0x7f2fec102170 2024-03-26T08:40:16.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.504+0000 7f2febfff640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2fd80a2880 0x7f2fd80a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2fec101610 tx=0x7f2fd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.508+0000 7f2fe97fa640 1 -- 172.21.15.7:0/3342043003 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2fe404b020 con 0x7f2fec102170 2024-03-26T08:40:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:16 smithi044 ceph-mon[36461]: pgmap v293: 97 pgs: 97 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-26T08:40:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:16 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2294200513' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T08:40:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:16 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] up:active 2024-03-26T08:40:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:16 smithi044 ceph-mon[36461]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-03-26T08:40:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:16 smithi044 ceph-mon[36461]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:40:16.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.718+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fec0fdcb0 con 0x7f2fd80a2880 2024-03-26T08:40:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.742+0000 7f2fe97fa640 1 -- 172.21.15.7:0/3342043003 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2fec0fdcb0 con 0x7f2fd80a2880 2024-03-26T08:40:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2fd80a2880 msgr2=0x7f2fd80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2fd80a2880 0x7f2fd80a4d40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2fec101610 tx=0x7f2fd4005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 msgr2=0x7f2fec19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 0x7f2fec19ef10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2fe402c630 tx=0x7f2fe4032ea0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 shutdown_connections 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2fd80a2880 0x7f2fd80a4d40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fec102170 0x7f2fec19ef10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fec1017b0 0x7f2fec197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 --2- 172.21.15.7:0/3342043003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fec1005c0 0x7f2fec197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 >> 172.21.15.7:0/3342043003 conn(0x7f2fec0fbb50 msgr2=0x7f2fec0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 shutdown_connections 2024-03-26T08:40:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.744+0000 7f2ff27cd640 1 -- 172.21.15.7:0/3342043003 wait complete. 2024-03-26T08:40:16.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:16 smithi016 ceph-mon[35851]: pgmap v293: 97 pgs: 97 active+clean; 578 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-03-26T08:40:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:16 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2294200513' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T08:40:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:16 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] up:active 2024-03-26T08:40:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:16 smithi016 ceph-mon[35851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-03-26T08:40:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:16 smithi016 ceph-mon[35851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:40:16.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-26T08:40:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.863+0000 7f79596ff640 1 -- 172.21.15.7:0/3633854842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541028a0 msgr2=0x7f7954102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.863+0000 7f79596ff640 1 --2- 172.21.15.7:0/3633854842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541028a0 0x7f7954102d20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f793c009900 tx=0x7f793c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.863+0000 7f79596ff640 1 -- 172.21.15.7:0/3633854842 shutdown_connections 2024-03-26T08:40:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.863+0000 7f79596ff640 1 --2- 172.21.15.7:0/3633854842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7954103260 0x7f7954109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.863+0000 7f79596ff640 1 --2- 172.21.15.7:0/3633854842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541028a0 0x7f7954102d20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.863+0000 7f79596ff640 1 --2- 172.21.15.7:0/3633854842 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f79541016b0 0x7f7954101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.863+0000 7f79596ff640 1 -- 172.21.15.7:0/3633854842 >> 172.21.15.7:0/3633854842 conn(0x7f79540fcc20 msgr2=0x7f79540ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.863+0000 7f79596ff640 1 -- 172.21.15.7:0/3633854842 shutdown_connections 2024-03-26T08:40:16.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.863+0000 7f79596ff640 1 -- 172.21.15.7:0/3633854842 wait complete. 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f79596ff640 1 Processor -- start 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f79596ff640 1 -- start start 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f79596ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541016b0 0x7f79541979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f79596ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541028a0 0x7f7954197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f79596ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7954103260 0x7f795419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f79596ff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795410d330 con 0x7f79541016b0 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f79596ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795410d1b0 con 0x7f79541028a0 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f79596ff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795410d4b0 con 0x7f7954103260 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541016b0 0x7f79541979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7952ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541016b0 0x7f79541979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49616/0 (socket says 172.21.15.7:49616) 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7952ffd640 1 -- 172.21.15.7:0/3028283326 learned_addr learned my addr 172.21.15.7:0/3028283326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f794bfff640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541028a0 0x7f7954197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f79537fe640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7954103260 0x7f795419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7952ffd640 1 -- 172.21.15.7:0/3028283326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7954103260 msgr2=0x7f795419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7952ffd640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7954103260 0x7f795419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7952ffd640 1 -- 172.21.15.7:0/3028283326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541028a0 msgr2=0x7f7954197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7952ffd640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541028a0 0x7f7954197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7952ffd640 1 -- 172.21.15.7:0/3028283326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795419f600 con 0x7f79541016b0 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7952ffd640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541016b0 0x7f79541979e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9c0 tx=0x7f794400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.864+0000 7f7950ff9640 1 -- 172.21.15.7:0/3028283326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794400cd80 con 0x7f79541016b0 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.865+0000 7f7950ff9640 1 -- 172.21.15.7:0/3028283326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79440035f0 con 0x7f79541016b0 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.865+0000 7f7950ff9640 1 -- 172.21.15.7:0/3028283326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7944010690 con 0x7f79541016b0 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.865+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795419f8f0 con 0x7f79541016b0 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.865+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795419fdc0 con 0x7f79541016b0 2024-03-26T08:40:16.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.866+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79541089d0 con 0x7f79541016b0 2024-03-26T08:40:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.867+0000 7f7950ff9640 1 -- 172.21.15.7:0/3028283326 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f79440037a0 con 0x7f79541016b0 2024-03-26T08:40:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.867+0000 7f7950ff9640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f79400a2800 0x7f79400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.867+0000 7f794bfff640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f79400a2800 0x7f79400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.867+0000 7f7950ff9640 1 -- 172.21.15.7:0/3028283326 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f79440cec70 con 0x7f79541016b0 2024-03-26T08:40:16.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.868+0000 7f794bfff640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f79400a2800 0x7f79400a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f793c02c7b0 tx=0x7f793c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:16.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:16.872+0000 7f7950ff9640 1 -- 172.21.15.7:0/3028283326 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f794409be60 con 0x7f79541016b0 2024-03-26T08:40:17.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.083+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7954061ae0 con 0x7f79400a2800 2024-03-26T08:40:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.090+0000 7f7950ff9640 1 -- 172.21.15.7:0/3028283326 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f7954061ae0 con 0x7f79400a2800 2024-03-26T08:40:17.081 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_1/e0c5a2ed-a3a8-4daf-9d6d-978951487586 2024-03-26T08:40:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.091+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f79400a2800 msgr2=0x7f79400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.091+0000 7f79596ff640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f79400a2800 0x7f79400a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f793c02c7b0 tx=0x7f793c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.091+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541016b0 msgr2=0x7f79541979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.091+0000 7f79596ff640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541016b0 0x7f79541979e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f794400e9c0 tx=0x7f794400ee80 comp rx=0 tx=0).stop 2024-03-26T08:40:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.092+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 shutdown_connections 2024-03-26T08:40:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.092+0000 7f79596ff640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f79400a2800 0x7f79400a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.092+0000 7f79596ff640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7954103260 0x7f795419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.092+0000 7f79596ff640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79541028a0 0x7f7954197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.092+0000 7f79596ff640 1 --2- 172.21.15.7:0/3028283326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79541016b0 0x7f79541979e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.092+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 >> 172.21.15.7:0/3028283326 conn(0x7f79540fcc20 msgr2=0x7f7954105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.092+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 shutdown_connections 2024-03-26T08:40:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.092+0000 7f79596ff640 1 -- 172.21.15.7:0/3028283326 wait complete. 2024-03-26T08:40:17.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.205+0000 7f559b3b2640 1 -- 172.21.15.7:0/139200545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 msgr2=0x7f5594101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.205+0000 7f559b3b2640 1 --2- 172.21.15.7:0/139200545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 0x7f5594101c40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5584009900 tx=0x7f558402bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.205+0000 7f559b3b2640 1 -- 172.21.15.7:0/139200545 shutdown_connections 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.205+0000 7f559b3b2640 1 --2- 172.21.15.7:0/139200545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5594102180 0x7f5594108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.205+0000 7f559b3b2640 1 --2- 172.21.15.7:0/139200545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 0x7f5594101c40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.205+0000 7f559b3b2640 1 --2- 172.21.15.7:0/139200545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55941005d0 0x7f55941009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.205+0000 7f559b3b2640 1 -- 172.21.15.7:0/139200545 >> 172.21.15.7:0/139200545 conn(0x7f55940fbb60 msgr2=0x7f55940fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.205+0000 7f559b3b2640 1 -- 172.21.15.7:0/139200545 shutdown_connections 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.205+0000 7f559b3b2640 1 -- 172.21.15.7:0/139200545 wait complete. 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559b3b2640 1 Processor -- start 2024-03-26T08:40:17.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559b3b2640 1 -- start start 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559b3b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55941005d0 0x7f5594197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559b3b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 0x7f5594197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559b3b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594102180 0x7f559419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559b3b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559410c230 con 0x7f55941017c0 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559b3b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559410c0b0 con 0x7f5594102180 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559b3b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559410c3b0 con 0x7f55941005d0 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559894d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 0x7f5594197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559894d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 0x7f5594197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49638/0 (socket says 172.21.15.7:49638) 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559894d640 1 -- 172.21.15.7:0/211303810 learned_addr learned my addr 172.21.15.7:0/211303810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559994f640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594102180 0x7f559419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559914e640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55941005d0 0x7f5594197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559894d640 1 -- 172.21.15.7:0/211303810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55941005d0 msgr2=0x7f5594197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559894d640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55941005d0 0x7f5594197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559894d640 1 -- 172.21.15.7:0/211303810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594102180 msgr2=0x7f559419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559894d640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594102180 0x7f559419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559894d640 1 -- 172.21.15.7:0/211303810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559419f6a0 con 0x7f55941017c0 2024-03-26T08:40:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.206+0000 7f559894d640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 0x7f5594197fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5584005ab0 tx=0x7f558402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.207+0000 7f558a7fc640 1 -- 172.21.15.7:0/211303810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584041070 con 0x7f55941017c0 2024-03-26T08:40:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.207+0000 7f558a7fc640 1 -- 172.21.15.7:0/211303810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5584004450 con 0x7f55941017c0 2024-03-26T08:40:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.207+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559419f930 con 0x7f55941017c0 2024-03-26T08:40:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.207+0000 7f558a7fc640 1 -- 172.21.15.7:0/211303810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584003d70 con 0x7f55941017c0 2024-03-26T08:40:17.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.207+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559419fe20 con 0x7f55941017c0 2024-03-26T08:40:17.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.208+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55941a0050 con 0x7f55941017c0 2024-03-26T08:40:17.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.209+0000 7f558a7fc640 1 -- 172.21.15.7:0/211303810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5584044070 con 0x7f55941017c0 2024-03-26T08:40:17.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.209+0000 7f558a7fc640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55800a2800 0x7f55800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.209+0000 7f559914e640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55800a2800 0x7f55800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.209+0000 7f558a7fc640 1 -- 172.21.15.7:0/211303810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f55840f0690 con 0x7f55941017c0 2024-03-26T08:40:17.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.210+0000 7f559914e640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55800a2800 0x7f55800a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5594101620 tx=0x7f5590005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.213+0000 7f558a7fc640 1 -- 172.21.15.7:0/211303810 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f55840bd880 con 0x7f55941017c0 2024-03-26T08:40:17.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.292+0000 7f558a7fc640 1 -- 172.21.15.7:0/211303810 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f55840b97f0 con 0x7f55941017c0 2024-03-26T08:40:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.444+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55941a0220 con 0x7f55941017c0 2024-03-26T08:40:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.445+0000 7f558a7fc640 1 -- 172.21.15.7:0/211303810 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f55840ddb00 con 0x7f55941017c0 2024-03-26T08:40:17.437 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:40:15.277649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:40:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-03-26T08:40:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55800a2800 msgr2=0x7f55800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55800a2800 0x7f55800a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5594101620 tx=0x7f5590005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 msgr2=0x7f5594197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 0x7f5594197fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5584005ab0 tx=0x7f558402c6c0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 shutdown_connections 2024-03-26T08:40:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55800a2800 0x7f55800a4cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594102180 0x7f559419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55941017c0 0x7f5594197fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 --2- 172.21.15.7:0/211303810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f55941005d0 0x7f5594197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 >> 172.21.15.7:0/211303810 conn(0x7f55940fbb60 msgr2=0x7f55940fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 shutdown_connections 2024-03-26T08:40:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.447+0000 7f559b3b2640 1 -- 172.21.15.7:0/211303810 wait complete. 2024-03-26T08:40:17.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:17 smithi007 ceph-mon[32462]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:17.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:17 smithi007 ceph-mon[32462]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:17.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:40:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.560+0000 7f37acac6640 1 -- 172.21.15.7:0/3357628721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a81005d0 msgr2=0x7f37a81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.560+0000 7f37acac6640 1 --2- 172.21.15.7:0/3357628721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a81005d0 0x7f37a81009f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3798009ca0 tx=0x7f379802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.560+0000 7f37acac6640 1 -- 172.21.15.7:0/3357628721 shutdown_connections 2024-03-26T08:40:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.560+0000 7f37acac6640 1 --2- 172.21.15.7:0/3357628721 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a8102180 0x7f37a8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.560+0000 7f37acac6640 1 --2- 172.21.15.7:0/3357628721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a81017c0 0x7f37a8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.560+0000 7f37acac6640 1 --2- 172.21.15.7:0/3357628721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a81005d0 0x7f37a81009f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.560+0000 7f37acac6640 1 -- 172.21.15.7:0/3357628721 >> 172.21.15.7:0/3357628721 conn(0x7f37a80fbb40 msgr2=0x7f37a80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.560+0000 7f37acac6640 1 -- 172.21.15.7:0/3357628721 shutdown_connections 2024-03-26T08:40:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.560+0000 7f37acac6640 1 -- 172.21.15.7:0/3357628721 wait complete. 2024-03-26T08:40:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37acac6640 1 Processor -- start 2024-03-26T08:40:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37acac6640 1 -- start start 2024-03-26T08:40:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37acac6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a81005d0 0x7f37a8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37acac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a81017c0 0x7f37a8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37acac6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8102180 0x7f37a819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37acac6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a810d370 con 0x7f37a8102180 2024-03-26T08:40:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37acac6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a810d1f0 con 0x7f37a81017c0 2024-03-26T08:40:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37acac6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a810d4f0 con 0x7f37a81005d0 2024-03-26T08:40:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37a6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8102180 0x7f37a819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37a659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a81005d0 0x7f37a8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37a6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8102180 0x7f37a819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49666/0 (socket says 172.21.15.7:49666) 2024-03-26T08:40:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37a6d9d640 1 -- 172.21.15.7:0/3119901134 learned_addr learned my addr 172.21.15.7:0/3119901134 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.561+0000 7f37a5d9b640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a81017c0 0x7f37a8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f37a659c640 1 -- 172.21.15.7:0/3119901134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a81017c0 msgr2=0x7f37a8197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f37a659c640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a81017c0 0x7f37a8197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f37a659c640 1 -- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8102180 msgr2=0x7f37a819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f37a659c640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8102180 0x7f37a819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f37a659c640 1 -- 172.21.15.7:0/3119901134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a819f5b0 con 0x7f37a81005d0 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f37a6d9d640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8102180 0x7f37a819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f37a659c640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a81005d0 0x7f37a8197990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f379802c630 tx=0x7f3798032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f379f7fe640 1 -- 172.21.15.7:0/3119901134 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798041020 con 0x7f37a81005d0 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a819f840 con 0x7f37a81005d0 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f379f7fe640 1 -- 172.21.15.7:0/3119901134 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3798034cd0 con 0x7f37a81005d0 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f379f7fe640 1 -- 172.21.15.7:0/3119901134 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798039040 con 0x7f37a81005d0 2024-03-26T08:40:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.562+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a819fd80 con 0x7f37a81005d0 2024-03-26T08:40:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.563+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a8107970 con 0x7f37a81005d0 2024-03-26T08:40:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.564+0000 7f379f7fe640 1 -- 172.21.15.7:0/3119901134 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f3798044020 con 0x7f37a81005d0 2024-03-26T08:40:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.565+0000 7f379f7fe640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37940a2890 0x7f37940a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.565+0000 7f37a5d9b640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37940a2890 0x7f37940a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.565+0000 7f379f7fe640 1 -- 172.21.15.7:0/3119901134 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f37980f0bb0 con 0x7f37a81005d0 2024-03-26T08:40:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.565+0000 7f37a5d9b640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37940a2890 0x7f37940a4d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37a8198ea0 tx=0x7f3790005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.568+0000 7f379f7fe640 1 -- 172.21.15.7:0/3119901134 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f379804b020 con 0x7f37a81005d0 2024-03-26T08:40:17.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:17 smithi044 ceph-mon[36461]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:17.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:17 smithi044 ceph-mon[36461]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:17.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:17 smithi016 ceph-mon[35851]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:17.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:17 smithi016 ceph-mon[35851]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:17.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.753+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37a802d1f0 con 0x7f37a81005d0 2024-03-26T08:40:17.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.755+0000 7f379f7fe640 1 -- 172.21.15.7:0/3119901134 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22089 (secure 0 0 0) 0x7f37980b9030 con 0x7f37a81005d0 2024-03-26T08:40:17.747 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:17.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:10.919648+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37940a2890 msgr2=0x7f37940a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37940a2890 0x7f37940a4d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37a8198ea0 tx=0x7f3790005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a81005d0 msgr2=0x7f37a8197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a81005d0 0x7f37a8197990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f379802c630 tx=0x7f3798032f20 comp rx=0 tx=0).stop 2024-03-26T08:40:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 shutdown_connections 2024-03-26T08:40:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37940a2890 0x7f37940a4d50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37a8102180 0x7f37a819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37a81017c0 0x7f37a8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 --2- 172.21.15.7:0/3119901134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37a81005d0 0x7f37a8197990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 >> 172.21.15.7:0/3119901134 conn(0x7f37a80fbb40 msgr2=0x7f37a8104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 shutdown_connections 2024-03-26T08:40:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.757+0000 7f37acac6640 1 -- 172.21.15.7:0/3119901134 wait complete. 2024-03-26T08:40:17.760 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-26T08:40:17.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T08:40:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.871+0000 7f7be5214640 1 -- 172.21.15.7:0/252414961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be01001a0 msgr2=0x7f7be0109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.871+0000 7f7be5214640 1 --2- 172.21.15.7:0/252414961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be01001a0 0x7f7be0109d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009900 tx=0x7f7bd802bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.871+0000 7f7be5214640 1 -- 172.21.15.7:0/252414961 shutdown_connections 2024-03-26T08:40:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.871+0000 7f7be5214640 1 --2- 172.21.15.7:0/252414961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be01001a0 0x7f7be0109d50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.871+0000 7f7be5214640 1 --2- 172.21.15.7:0/252414961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7be00ff7e0 0x7f7be00ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.871+0000 7f7be5214640 1 --2- 172.21.15.7:0/252414961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be0102d70 0x7f7be0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.871+0000 7f7be5214640 1 -- 172.21.15.7:0/252414961 >> 172.21.15.7:0/252414961 conn(0x7f7be00fb320 msgr2=0x7f7be00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.871+0000 7f7be5214640 1 -- 172.21.15.7:0/252414961 shutdown_connections 2024-03-26T08:40:17.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.871+0000 7f7be5214640 1 -- 172.21.15.7:0/252414961 wait complete. 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7be5214640 1 Processor -- start 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7be5214640 1 -- start start 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7be5214640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7be00ff7e0 0x7f7be0197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7be5214640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be01001a0 0x7f7be0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7be5214640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be0102d70 0x7f7be019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7be5214640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be010d350 con 0x7f7be0102d70 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7be5214640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be010d1d0 con 0x7f7be00ff7e0 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7be5214640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be010d4d0 con 0x7f7be01001a0 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7bdf59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be0102d70 0x7f7be019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7bdf59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be0102d70 0x7f7be019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49692/0 (socket says 172.21.15.7:49692) 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7bdf59e640 1 -- 172.21.15.7:0/333343441 learned_addr learned my addr 172.21.15.7:0/333343441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7bded9d640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7be00ff7e0 0x7f7be0197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.872+0000 7f7bde59c640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be01001a0 0x7f7be0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bdf59e640 1 -- 172.21.15.7:0/333343441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be01001a0 msgr2=0x7f7be0197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bdf59e640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be01001a0 0x7f7be0197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bdf59e640 1 -- 172.21.15.7:0/333343441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7be00ff7e0 msgr2=0x7f7be0197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bdf59e640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7be00ff7e0 0x7f7be0197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bdf59e640 1 -- 172.21.15.7:0/333343441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be019f550 con 0x7f7be0102d70 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bded9d640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7be00ff7e0 0x7f7be0197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bdf59e640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be0102d70 0x7f7be019eee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8005ab0 tx=0x7f7bd802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bd7fff640 1 -- 172.21.15.7:0/333343441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8041070 con 0x7f7be0102d70 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be019f7e0 con 0x7f7be0102d70 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bd7fff640 1 -- 172.21.15.7:0/333343441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bd8004450 con 0x7f7be0102d70 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7bd7fff640 1 -- 172.21.15.7:0/333343441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8003e60 con 0x7f7be0102d70 2024-03-26T08:40:17.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.873+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be019fcd0 con 0x7f7be0102d70 2024-03-26T08:40:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.874+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be0108b60 con 0x7f7be0102d70 2024-03-26T08:40:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.875+0000 7f7bd7fff640 1 -- 172.21.15.7:0/333343441 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f7bd8044070 con 0x7f7be0102d70 2024-03-26T08:40:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.875+0000 7f7bd7fff640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bcc0a2890 0x7f7bcc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:17.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.876+0000 7f7bded9d640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bcc0a2890 0x7f7bcc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.876+0000 7f7bd7fff640 1 -- 172.21.15.7:0/333343441 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7bd80f0550 con 0x7f7be0102d70 2024-03-26T08:40:17.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.876+0000 7f7bded9d640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bcc0a2890 0x7f7bcc0a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7be0069ae0 tx=0x7f7bd0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:17.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:17.880+0000 7f7bd7fff640 1 -- 172.21.15.7:0/333343441 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7bd80bd100 con 0x7f7be0102d70 2024-03-26T08:40:18.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.064+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7be0061ae0 con 0x7f7bcc0a2890 2024-03-26T08:40:18.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.072+0000 7f7bd7fff640 1 -- 172.21.15.7:0/333343441 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258912 (secure 0 0 0) 0x7f7be0061ae0 con 0x7f7bcc0a2890 2024-03-26T08:40:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:18.071 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-26T08:40:18.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.074+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bcc0a2890 msgr2=0x7f7bcc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.074+0000 7f7be5214640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bcc0a2890 0x7f7bcc0a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7be0069ae0 tx=0x7f7bd0009500 comp rx=0 tx=0).stop 2024-03-26T08:40:18.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.074+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be0102d70 msgr2=0x7f7be019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.074+0000 7f7be5214640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be0102d70 0x7f7be019eee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8005ab0 tx=0x7f7bd802c6c0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.074+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 shutdown_connections 2024-03-26T08:40:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.074+0000 7f7be5214640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bcc0a2890 0x7f7bcc0a4d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.074+0000 7f7be5214640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7be0102d70 0x7f7be019eee0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.074+0000 7f7be5214640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7be01001a0 0x7f7be0197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.074+0000 7f7be5214640 1 --2- 172.21.15.7:0/333343441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7be00ff7e0 0x7f7be0197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.075+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 >> 172.21.15.7:0/333343441 conn(0x7f7be00fb320 msgr2=0x7f7be0105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.075+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 shutdown_connections 2024-03-26T08:40:18.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.075+0000 7f7be5214640 1 -- 172.21.15.7:0/333343441 wait complete. 2024-03-26T08:40:18.077 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-03-26T08:40:16.300860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601757,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":138,"num_read_kb":119,"num_write":277,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":265,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372688,"kb_used_data":11020,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569136,"statfs":{"total":1151940427776,"available":1151558795264,"internally_reserved":0,"allocated":11284480,"data_stored":7043830,"data_compressed":107721,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11389,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":96,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.088599"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153654+0000","last_change":"2024-03-26T08:40:09.939078+0000","last_active":"2024-03-26T08:40:11.153654+0000","last_peered":"2024-03-26T08:40:11.153654+0000","last_clean":"2024-03-26T08:40:11.153654+0000","last_became_active":"2024-03-26T08:40:09.937626+0000","last_became_peered":"2024-03-26T08:40:09.937626+0000","last_unstale":"2024-03-26T08:40:11.153654+0000","last_undegraded":"2024-03-26T08:40:11.153654+0000","last_fullsized":"2024-03-26T08:40:11.153654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:52:59.254181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094283+0000","last_change":"2024-03-26T08:40:09.924207+0000","last_active":"2024-03-26T08:40:11.094283+0000","last_peered":"2024-03-26T08:40:11.094283+0000","last_clean":"2024-03-26T08:40:11.094283+0000","last_became_active":"2024-03-26T08:40:09.923881+0000","last_became_peered":"2024-03-26T08:40:09.923881+0000","last_unstale":"2024-03-26T08:40:11.094283+0000","last_undegraded":"2024-03-26T08:40:11.094283+0000","last_fullsized":"2024-03-26T08:40:11.094283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:16:45.265521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094443+0000","last_change":"2024-03-26T08:40:09.929154+0000","last_active":"2024-03-26T08:40:11.094443+0000","last_peered":"2024-03-26T08:40:11.094443+0000","last_clean":"2024-03-26T08:40:11.094443+0000","last_became_active":"2024-03-26T08:40:09.928833+0000","last_became_peered":"2024-03-26T08:40:09.928833+0000","last_unstale":"2024-03-26T08:40:11.094443+0000","last_undegraded":"2024-03-26T08:40:11.094443+0000","last_fullsized":"2024-03-26T08:40:11.094443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:22.972210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152692+0000","last_change":"2024-03-26T08:40:09.930713+0000","last_active":"2024-03-26T08:40:11.152692+0000","last_peered":"2024-03-26T08:40:11.152692+0000","last_clean":"2024-03-26T08:40:11.152692+0000","last_became_active":"2024-03-26T08:40:09.930420+0000","last_became_peered":"2024-03-26T08:40:09.930420+0000","last_unstale":"2024-03-26T08:40:11.152692+0000","last_undegraded":"2024-03-26T08:40:11.152692+0000","last_fullsized":"2024-03-26T08:40:11.152692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:17:03.015703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153736+0000","last_change":"2024-03-26T08:40:09.923935+0000","last_active":"2024-03-26T08:40:11.153736+0000","last_peered":"2024-03-26T08:40:11.153736+0000","last_clean":"2024-03-26T08:40:11.153736+0000","last_became_active":"2024-03-26T08:40:09.923587+0000","last_became_peered":"2024-03-26T08:40:09.923587+0000","last_unstale":"2024-03-26T08:40:11.153736+0000","last_undegraded":"2024-03-26T08:40:11.153736+0000","last_fullsized":"2024-03-26T08:40:11.153736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:19:38.831448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152667+0000","last_change":"2024-03-26T08:40:09.922361+0000","last_active":"2024-03-26T08:40:11.152667+0000","last_peered":"2024-03-26T08:40:11.152667+0000","last_clean":"2024-03-26T08:40:11.152667+0000","last_became_active":"2024-03-26T08:40:09.921643+0000","last_became_peered":"2024-03-26T08:40:09.921643+0000","last_unstale":"2024-03-26T08:40:11.152667+0000","last_undegraded":"2024-03-26T08:40:11.152667+0000","last_fullsized":"2024-03-26T08:40:11.152667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:37:23.048778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280800+0000","last_change":"2024-03-26T08:40:09.932428+0000","last_active":"2024-03-26T08:40:11.280800+0000","last_peered":"2024-03-26T08:40:11.280800+0000","last_clean":"2024-03-26T08:40:11.280800+0000","last_became_active":"2024-03-26T08:40:09.932120+0000","last_became_peered":"2024-03-26T08:40:09.932120+0000","last_unstale":"2024-03-26T08:40:11.280800+0000","last_undegraded":"2024-03-26T08:40:11.280800+0000","last_fullsized":"2024-03-26T08:40:11.280800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:24.362568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152021+0000","last_change":"2024-03-26T08:40:09.928251+0000","last_active":"2024-03-26T08:40:11.152021+0000","last_peered":"2024-03-26T08:40:11.152021+0000","last_clean":"2024-03-26T08:40:11.152021+0000","last_became_active":"2024-03-26T08:40:09.927961+0000","last_became_peered":"2024-03-26T08:40:09.927961+0000","last_unstale":"2024-03-26T08:40:11.152021+0000","last_undegraded":"2024-03-26T08:40:11.152021+0000","last_fullsized":"2024-03-26T08:40:11.152021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:25:09.348033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923263+0000","last_change":"2024-03-26T08:40:09.930888+0000","last_active":"2024-03-26T08:40:10.923263+0000","last_peered":"2024-03-26T08:40:10.923263+0000","last_clean":"2024-03-26T08:40:10.923263+0000","last_became_active":"2024-03-26T08:40:09.930513+0000","last_became_peered":"2024-03-26T08:40:09.930513+0000","last_unstale":"2024-03-26T08:40:10.923263+0000","last_undegraded":"2024-03-26T08:40:10.923263+0000","last_fullsized":"2024-03-26T08:40:10.923263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:21.023569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153219+0000","last_change":"2024-03-26T08:40:09.935534+0000","last_active":"2024-03-26T08:40:11.153219+0000","last_peered":"2024-03-26T08:40:11.153219+0000","last_clean":"2024-03-26T08:40:11.153219+0000","last_became_active":"2024-03-26T08:40:09.934959+0000","last_became_peered":"2024-03-26T08:40:09.934959+0000","last_unstale":"2024-03-26T08:40:11.153219+0000","last_undegraded":"2024-03-26T08:40:11.153219+0000","last_fullsized":"2024-03-26T08:40:11.153219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:18:59.311180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281138+0000","last_change":"2024-03-26T08:40:09.930486+0000","last_active":"2024-03-26T08:40:11.281138+0000","last_peered":"2024-03-26T08:40:11.281138+0000","last_clean":"2024-03-26T08:40:11.281138+0000","last_became_active":"2024-03-26T08:40:09.930184+0000","last_became_peered":"2024-03-26T08:40:09.930184+0000","last_unstale":"2024-03-26T08:40:11.281138+0000","last_undegraded":"2024-03-26T08:40:11.281138+0000","last_fullsized":"2024-03-26T08:40:11.281138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:07.246878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152530+0000","last_change":"2024-03-26T08:40:09.924305+0000","last_active":"2024-03-26T08:40:11.152530+0000","last_peered":"2024-03-26T08:40:11.152530+0000","last_clean":"2024-03-26T08:40:11.152530+0000","last_became_active":"2024-03-26T08:40:09.923985+0000","last_became_peered":"2024-03-26T08:40:09.923985+0000","last_unstale":"2024-03-26T08:40:11.152530+0000","last_undegraded":"2024-03-26T08:40:11.152530+0000","last_fullsized":"2024-03-26T08:40:11.152530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:42:39.894245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922581+0000","last_change":"2024-03-26T08:40:09.916637+0000","last_active":"2024-03-26T08:40:10.922581+0000","last_peered":"2024-03-26T08:40:10.922581+0000","last_clean":"2024-03-26T08:40:10.922581+0000","last_became_active":"2024-03-26T08:40:09.916314+0000","last_became_peered":"2024-03-26T08:40:09.916314+0000","last_unstale":"2024-03-26T08:40:10.922581+0000","last_undegraded":"2024-03-26T08:40:10.922581+0000","last_fullsized":"2024-03-26T08:40:10.922581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:45:10.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094242+0000","last_change":"2024-03-26T08:40:09.923753+0000","last_active":"2024-03-26T08:40:11.094242+0000","last_peered":"2024-03-26T08:40:11.094242+0000","last_clean":"2024-03-26T08:40:11.094242+0000","last_became_active":"2024-03-26T08:40:09.923427+0000","last_became_peered":"2024-03-26T08:40:09.923427+0000","last_unstale":"2024-03-26T08:40:11.094242+0000","last_undegraded":"2024-03-26T08:40:11.094242+0000","last_fullsized":"2024-03-26T08:40:11.094242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:13:16.711556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094190+0000","last_change":"2024-03-26T08:40:09.931879+0000","last_active":"2024-03-26T08:40:11.094190+0000","last_peered":"2024-03-26T08:40:11.094190+0000","last_clean":"2024-03-26T08:40:11.094190+0000","last_became_active":"2024-03-26T08:40:09.931589+0000","last_became_peered":"2024-03-26T08:40:09.931589+0000","last_unstale":"2024-03-26T08:40:11.094190+0000","last_undegraded":"2024-03-26T08:40:11.094190+0000","last_fullsized":"2024-03-26T08:40:11.094190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:22:01.038745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922718+0000","last_change":"2024-03-26T08:40:09.930500+0000","last_active":"2024-03-26T08:40:10.922718+0000","last_peered":"2024-03-26T08:40:10.922718+0000","last_clean":"2024-03-26T08:40:10.922718+0000","last_became_active":"2024-03-26T08:40:09.930101+0000","last_became_peered":"2024-03-26T08:40:09.930101+0000","last_unstale":"2024-03-26T08:40:10.922718+0000","last_undegraded":"2024-03-26T08:40:10.922718+0000","last_fullsized":"2024-03-26T08:40:10.922718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:54.229004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153660+0000","last_change":"2024-03-26T08:40:09.939177+0000","last_active":"2024-03-26T08:40:11.153660+0000","last_peered":"2024-03-26T08:40:11.153660+0000","last_clean":"2024-03-26T08:40:11.153660+0000","last_became_active":"2024-03-26T08:40:09.937727+0000","last_became_peered":"2024-03-26T08:40:09.937727+0000","last_unstale":"2024-03-26T08:40:11.153660+0000","last_undegraded":"2024-03-26T08:40:11.153660+0000","last_fullsized":"2024-03-26T08:40:11.153660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:40:44.040015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152641+0000","last_change":"2024-03-26T08:40:09.928429+0000","last_active":"2024-03-26T08:40:11.152641+0000","last_peered":"2024-03-26T08:40:11.152641+0000","last_clean":"2024-03-26T08:40:11.152641+0000","last_became_active":"2024-03-26T08:40:09.928025+0000","last_became_peered":"2024-03-26T08:40:09.928025+0000","last_unstale":"2024-03-26T08:40:11.152641+0000","last_undegraded":"2024-03-26T08:40:11.152641+0000","last_fullsized":"2024-03-26T08:40:11.152641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:52:55.450291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923944+0000","last_change":"2024-03-26T08:40:09.930856+0000","last_active":"2024-03-26T08:40:10.923944+0000","last_peered":"2024-03-26T08:40:10.923944+0000","last_clean":"2024-03-26T08:40:10.923944+0000","last_became_active":"2024-03-26T08:40:09.930552+0000","last_became_peered":"2024-03-26T08:40:09.930552+0000","last_unstale":"2024-03-26T08:40:10.923944+0000","last_undegraded":"2024-03-26T08:40:10.923944+0000","last_fullsized":"2024-03-26T08:40:10.923944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:52.851287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.283522+0000","last_change":"2024-03-26T08:40:09.230609+0000","last_active":"2024-03-26T08:40:14.283522+0000","last_peered":"2024-03-26T08:40:14.283522+0000","last_clean":"2024-03-26T08:40:14.283522+0000","last_became_active":"2024-03-26T08:40:09.230284+0000","last_became_peered":"2024-03-26T08:40:09.230284+0000","last_unstale":"2024-03-26T08:40:14.283522+0000","last_undegraded":"2024-03-26T08:40:14.283522+0000","last_fullsized":"2024-03-26T08:40:14.283522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:06.681725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152561+0000","last_change":"2024-03-26T08:40:09.930498+0000","last_active":"2024-03-26T08:40:11.152561+0000","last_peered":"2024-03-26T08:40:11.152561+0000","last_clean":"2024-03-26T08:40:11.152561+0000","last_became_active":"2024-03-26T08:40:09.930209+0000","last_became_peered":"2024-03-26T08:40:09.930209+0000","last_unstale":"2024-03-26T08:40:11.152561+0000","last_undegraded":"2024-03-26T08:40:11.152561+0000","last_fullsized":"2024-03-26T08:40:11.152561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:12:59.437553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152886+0000","last_change":"2024-03-26T08:40:09.235451+0000","last_active":"2024-03-26T08:40:11.152886+0000","last_peered":"2024-03-26T08:40:11.152886+0000","last_clean":"2024-03-26T08:40:11.152886+0000","last_became_active":"2024-03-26T08:40:09.235154+0000","last_became_peered":"2024-03-26T08:40:09.235154+0000","last_unstale":"2024-03-26T08:40:11.152886+0000","last_undegraded":"2024-03-26T08:40:11.152886+0000","last_fullsized":"2024-03-26T08:40:11.152886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:37:38.418888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094433+0000","last_change":"2024-03-26T08:40:09.931728+0000","last_active":"2024-03-26T08:40:11.094433+0000","last_peered":"2024-03-26T08:40:11.094433+0000","last_clean":"2024-03-26T08:40:11.094433+0000","last_became_active":"2024-03-26T08:40:09.931401+0000","last_became_peered":"2024-03-26T08:40:09.931401+0000","last_unstale":"2024-03-26T08:40:11.094433+0000","last_undegraded":"2024-03-26T08:40:11.094433+0000","last_fullsized":"2024-03-26T08:40:11.094433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:02.229763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153427+0000","last_change":"2024-03-26T08:40:09.231247+0000","last_active":"2024-03-26T08:40:11.153427+0000","last_peered":"2024-03-26T08:40:11.153427+0000","last_clean":"2024-03-26T08:40:11.153427+0000","last_became_active":"2024-03-26T08:40:09.230914+0000","last_became_peered":"2024-03-26T08:40:09.230914+0000","last_unstale":"2024-03-26T08:40:11.153427+0000","last_undegraded":"2024-03-26T08:40:11.153427+0000","last_fullsized":"2024-03-26T08:40:11.153427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:03.956444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922935+0000","last_change":"2024-03-26T08:40:09.925174+0000","last_active":"2024-03-26T08:40:10.922935+0000","last_peered":"2024-03-26T08:40:10.922935+0000","last_clean":"2024-03-26T08:40:10.922935+0000","last_became_active":"2024-03-26T08:40:09.924840+0000","last_became_peered":"2024-03-26T08:40:09.924840+0000","last_unstale":"2024-03-26T08:40:10.922935+0000","last_undegraded":"2024-03-26T08:40:10.922935+0000","last_fullsized":"2024-03-26T08:40:10.922935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:57:19.041245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152847+0000","last_change":"2024-03-26T08:40:09.229504+0000","last_active":"2024-03-26T08:40:11.152847+0000","last_peered":"2024-03-26T08:40:11.152847+0000","last_clean":"2024-03-26T08:40:11.152847+0000","last_became_active":"2024-03-26T08:40:09.226449+0000","last_became_peered":"2024-03-26T08:40:09.226449+0000","last_unstale":"2024-03-26T08:40:11.152847+0000","last_undegraded":"2024-03-26T08:40:11.152847+0000","last_fullsized":"2024-03-26T08:40:11.152847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:38.047424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153068+0000","last_change":"2024-03-26T08:40:09.930184+0000","last_active":"2024-03-26T08:40:11.153068+0000","last_peered":"2024-03-26T08:40:11.153068+0000","last_clean":"2024-03-26T08:40:11.153068+0000","last_became_active":"2024-03-26T08:40:09.927679+0000","last_became_peered":"2024-03-26T08:40:09.927679+0000","last_unstale":"2024-03-26T08:40:11.153068+0000","last_undegraded":"2024-03-26T08:40:11.153068+0000","last_fullsized":"2024-03-26T08:40:11.153068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:05:52.949814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:15.288057+0000","last_change":"2024-03-26T08:40:09.233025+0000","last_active":"2024-03-26T08:40:15.288057+0000","last_peered":"2024-03-26T08:40:15.288057+0000","last_clean":"2024-03-26T08:40:15.288057+0000","last_became_active":"2024-03-26T08:40:09.232654+0000","last_became_peered":"2024-03-26T08:40:09.232654+0000","last_unstale":"2024-03-26T08:40:15.288057+0000","last_undegraded":"2024-03-26T08:40:15.288057+0000","last_fullsized":"2024-03-26T08:40:15.288057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:01.745583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152492+0000","last_change":"2024-03-26T08:40:09.924187+0000","last_active":"2024-03-26T08:40:11.152492+0000","last_peered":"2024-03-26T08:40:11.152492+0000","last_clean":"2024-03-26T08:40:11.152492+0000","last_became_active":"2024-03-26T08:40:09.923861+0000","last_became_peered":"2024-03-26T08:40:09.923861+0000","last_unstale":"2024-03-26T08:40:11.152492+0000","last_undegraded":"2024-03-26T08:40:11.152492+0000","last_fullsized":"2024-03-26T08:40:11.152492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:12:17.284186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.283592+0000","last_change":"2024-03-26T08:40:09.231368+0000","last_active":"2024-03-26T08:40:14.283592+0000","last_peered":"2024-03-26T08:40:14.283592+0000","last_clean":"2024-03-26T08:40:14.283592+0000","last_became_active":"2024-03-26T08:40:09.231040+0000","last_became_peered":"2024-03-26T08:40:09.231040+0000","last_unstale":"2024-03-26T08:40:14.283592+0000","last_undegraded":"2024-03-26T08:40:14.283592+0000","last_fullsized":"2024-03-26T08:40:14.283592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:14:20.681225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152685+0000","last_change":"2024-03-26T08:40:09.928125+0000","last_active":"2024-03-26T08:40:11.152685+0000","last_peered":"2024-03-26T08:40:11.152685+0000","last_clean":"2024-03-26T08:40:11.152685+0000","last_became_active":"2024-03-26T08:40:09.927616+0000","last_became_peered":"2024-03-26T08:40:09.927616+0000","last_unstale":"2024-03-26T08:40:11.152685+0000","last_undegraded":"2024-03-26T08:40:11.152685+0000","last_fullsized":"2024-03-26T08:40:11.152685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:12.764054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922333+0000","last_change":"2024-03-26T08:40:09.229976+0000","last_active":"2024-03-26T08:40:10.922333+0000","last_peered":"2024-03-26T08:40:10.922333+0000","last_clean":"2024-03-26T08:40:10.922333+0000","last_became_active":"2024-03-26T08:40:09.229594+0000","last_became_peered":"2024-03-26T08:40:09.229594+0000","last_unstale":"2024-03-26T08:40:10.922333+0000","last_undegraded":"2024-03-26T08:40:10.922333+0000","last_fullsized":"2024-03-26T08:40:10.922333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:31:24.204853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.151949+0000","last_change":"2024-03-26T08:40:09.931290+0000","last_active":"2024-03-26T08:40:11.151949+0000","last_peered":"2024-03-26T08:40:11.151949+0000","last_clean":"2024-03-26T08:40:11.151949+0000","last_became_active":"2024-03-26T08:40:09.930961+0000","last_became_peered":"2024-03-26T08:40:09.930961+0000","last_unstale":"2024-03-26T08:40:11.151949+0000","last_undegraded":"2024-03-26T08:40:11.151949+0000","last_fullsized":"2024-03-26T08:40:11.151949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:36.460807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.151949+0000","last_change":"2024-03-26T08:40:09.232166+0000","last_active":"2024-03-26T08:40:11.151949+0000","last_peered":"2024-03-26T08:40:11.151949+0000","last_clean":"2024-03-26T08:40:11.151949+0000","last_became_active":"2024-03-26T08:40:09.231840+0000","last_became_peered":"2024-03-26T08:40:09.231840+0000","last_unstale":"2024-03-26T08:40:11.151949+0000","last_undegraded":"2024-03-26T08:40:11.151949+0000","last_fullsized":"2024-03-26T08:40:11.151949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:51.636322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152163+0000","last_change":"2024-03-26T08:40:09.924680+0000","last_active":"2024-03-26T08:40:11.152163+0000","last_peered":"2024-03-26T08:40:11.152163+0000","last_clean":"2024-03-26T08:40:11.152163+0000","last_became_active":"2024-03-26T08:40:09.924314+0000","last_became_peered":"2024-03-26T08:40:09.924314+0000","last_unstale":"2024-03-26T08:40:11.152163+0000","last_undegraded":"2024-03-26T08:40:11.152163+0000","last_fullsized":"2024-03-26T08:40:11.152163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:49:20.780950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.273704+0000","last_change":"2024-03-26T08:40:09.235997+0000","last_active":"2024-03-26T08:40:13.273704+0000","last_peered":"2024-03-26T08:40:13.273704+0000","last_clean":"2024-03-26T08:40:13.273704+0000","last_became_active":"2024-03-26T08:40:09.235729+0000","last_became_peered":"2024-03-26T08:40:09.235729+0000","last_unstale":"2024-03-26T08:40:13.273704+0000","last_undegraded":"2024-03-26T08:40:13.273704+0000","last_fullsized":"2024-03-26T08:40:13.273704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:46:12.322059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923227+0000","last_change":"2024-03-26T08:40:09.926371+0000","last_active":"2024-03-26T08:40:10.923227+0000","last_peered":"2024-03-26T08:40:10.923227+0000","last_clean":"2024-03-26T08:40:10.923227+0000","last_became_active":"2024-03-26T08:40:09.925536+0000","last_became_peered":"2024-03-26T08:40:09.925536+0000","last_unstale":"2024-03-26T08:40:10.923227+0000","last_undegraded":"2024-03-26T08:40:10.923227+0000","last_fullsized":"2024-03-26T08:40:10.923227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:50:39.101684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281069+0000","last_change":"2024-03-26T08:40:09.229018+0000","last_active":"2024-03-26T08:40:11.281069+0000","last_peered":"2024-03-26T08:40:11.281069+0000","last_clean":"2024-03-26T08:40:11.281069+0000","last_became_active":"2024-03-26T08:40:09.228680+0000","last_became_peered":"2024-03-26T08:40:09.228680+0000","last_unstale":"2024-03-26T08:40:11.281069+0000","last_undegraded":"2024-03-26T08:40:11.281069+0000","last_fullsized":"2024-03-26T08:40:11.281069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:31:10.065127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094419+0000","last_change":"2024-03-26T08:40:09.926152+0000","last_active":"2024-03-26T08:40:11.094419+0000","last_peered":"2024-03-26T08:40:11.094419+0000","last_clean":"2024-03-26T08:40:11.094419+0000","last_became_active":"2024-03-26T08:40:09.925825+0000","last_became_peered":"2024-03-26T08:40:09.925825+0000","last_unstale":"2024-03-26T08:40:11.094419+0000","last_undegraded":"2024-03-26T08:40:11.094419+0000","last_fullsized":"2024-03-26T08:40:11.094419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:28:24.287706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.273102+0000","last_change":"2024-03-26T08:40:09.234699+0000","last_active":"2024-03-26T08:40:13.273102+0000","last_peered":"2024-03-26T08:40:13.273102+0000","last_clean":"2024-03-26T08:40:13.273102+0000","last_became_active":"2024-03-26T08:40:09.234451+0000","last_became_peered":"2024-03-26T08:40:09.234451+0000","last_unstale":"2024-03-26T08:40:13.273102+0000","last_undegraded":"2024-03-26T08:40:13.273102+0000","last_fullsized":"2024-03-26T08:40:13.273102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:53.337418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923055+0000","last_change":"2024-03-26T08:40:09.927023+0000","last_active":"2024-03-26T08:40:10.923055+0000","last_peered":"2024-03-26T08:40:10.923055+0000","last_clean":"2024-03-26T08:40:10.923055+0000","last_became_active":"2024-03-26T08:40:09.926734+0000","last_became_peered":"2024-03-26T08:40:09.926734+0000","last_unstale":"2024-03-26T08:40:10.923055+0000","last_undegraded":"2024-03-26T08:40:10.923055+0000","last_fullsized":"2024-03-26T08:40:10.923055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:19:44.137513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:15.636414+0000","last_change":"2024-03-26T08:40:09.230059+0000","last_active":"2024-03-26T08:40:15.636414+0000","last_peered":"2024-03-26T08:40:15.636414+0000","last_clean":"2024-03-26T08:40:15.636414+0000","last_became_active":"2024-03-26T08:40:09.229776+0000","last_became_peered":"2024-03-26T08:40:09.229776+0000","last_unstale":"2024-03-26T08:40:15.636414+0000","last_undegraded":"2024-03-26T08:40:15.636414+0000","last_fullsized":"2024-03-26T08:40:15.636414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:00.917694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153177+0000","last_change":"2024-03-26T08:40:09.930702+0000","last_active":"2024-03-26T08:40:11.153177+0000","last_peered":"2024-03-26T08:40:11.153177+0000","last_clean":"2024-03-26T08:40:11.153177+0000","last_became_active":"2024-03-26T08:40:09.930400+0000","last_became_peered":"2024-03-26T08:40:09.930400+0000","last_unstale":"2024-03-26T08:40:11.153177+0000","last_undegraded":"2024-03-26T08:40:11.153177+0000","last_fullsized":"2024-03-26T08:40:11.153177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:03:12.897812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.283373+0000","last_change":"2024-03-26T08:40:09.225972+0000","last_active":"2024-03-26T08:40:14.283373+0000","last_peered":"2024-03-26T08:40:14.283373+0000","last_clean":"2024-03-26T08:40:14.283373+0000","last_became_active":"2024-03-26T08:40:09.225524+0000","last_became_peered":"2024-03-26T08:40:09.225524+0000","last_unstale":"2024-03-26T08:40:14.283373+0000","last_undegraded":"2024-03-26T08:40:14.283373+0000","last_fullsized":"2024-03-26T08:40:14.283373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:08:17.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922439+0000","last_change":"2024-03-26T08:40:09.920628+0000","last_active":"2024-03-26T08:40:10.922439+0000","last_peered":"2024-03-26T08:40:10.922439+0000","last_clean":"2024-03-26T08:40:10.922439+0000","last_became_active":"2024-03-26T08:40:09.920300+0000","last_became_peered":"2024-03-26T08:40:09.920300+0000","last_unstale":"2024-03-26T08:40:10.922439+0000","last_undegraded":"2024-03-26T08:40:10.922439+0000","last_fullsized":"2024-03-26T08:40:10.922439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:02:16.503881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.273465+0000","last_change":"2024-03-26T08:40:09.237802+0000","last_active":"2024-03-26T08:40:13.273465+0000","last_peered":"2024-03-26T08:40:13.273465+0000","last_clean":"2024-03-26T08:40:13.273465+0000","last_became_active":"2024-03-26T08:40:09.237425+0000","last_became_peered":"2024-03-26T08:40:09.237425+0000","last_unstale":"2024-03-26T08:40:13.273465+0000","last_undegraded":"2024-03-26T08:40:13.273465+0000","last_fullsized":"2024-03-26T08:40:13.273465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:05.953882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281102+0000","last_change":"2024-03-26T08:40:09.923221+0000","last_active":"2024-03-26T08:40:11.281102+0000","last_peered":"2024-03-26T08:40:11.281102+0000","last_clean":"2024-03-26T08:40:11.281102+0000","last_became_active":"2024-03-26T08:40:09.922894+0000","last_became_peered":"2024-03-26T08:40:09.922894+0000","last_unstale":"2024-03-26T08:40:11.281102+0000","last_undegraded":"2024-03-26T08:40:11.281102+0000","last_fullsized":"2024-03-26T08:40:11.281102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:24.887625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152200+0000","last_change":"2024-03-26T08:40:09.225179+0000","last_active":"2024-03-26T08:40:11.152200+0000","last_peered":"2024-03-26T08:40:11.152200+0000","last_clean":"2024-03-26T08:40:11.152200+0000","last_became_active":"2024-03-26T08:40:09.224861+0000","last_became_peered":"2024-03-26T08:40:09.224861+0000","last_unstale":"2024-03-26T08:40:11.152200+0000","last_undegraded":"2024-03-26T08:40:11.152200+0000","last_fullsized":"2024-03-26T08:40:11.152200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:37:54.843284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280729+0000","last_change":"2024-03-26T08:40:09.932590+0000","last_active":"2024-03-26T08:40:11.280729+0000","last_peered":"2024-03-26T08:40:11.280729+0000","last_clean":"2024-03-26T08:40:11.280729+0000","last_became_active":"2024-03-26T08:40:09.932283+0000","last_became_peered":"2024-03-26T08:40:09.932283+0000","last_unstale":"2024-03-26T08:40:11.280729+0000","last_undegraded":"2024-03-26T08:40:11.280729+0000","last_fullsized":"2024-03-26T08:40:11.280729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:23:41.449789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.274347+0000","last_change":"2024-03-26T08:40:09.232024+0000","last_active":"2024-03-26T08:40:13.274347+0000","last_peered":"2024-03-26T08:40:13.274347+0000","last_clean":"2024-03-26T08:40:13.274347+0000","last_became_active":"2024-03-26T08:40:09.231715+0000","last_became_peered":"2024-03-26T08:40:09.231715+0000","last_unstale":"2024-03-26T08:40:13.274347+0000","last_undegraded":"2024-03-26T08:40:13.274347+0000","last_fullsized":"2024-03-26T08:40:13.274347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:02.229042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152508+0000","last_change":"2024-03-26T08:40:09.926893+0000","last_active":"2024-03-26T08:40:11.152508+0000","last_peered":"2024-03-26T08:40:11.152508+0000","last_clean":"2024-03-26T08:40:11.152508+0000","last_became_active":"2024-03-26T08:40:09.926529+0000","last_became_peered":"2024-03-26T08:40:09.926529+0000","last_unstale":"2024-03-26T08:40:11.152508+0000","last_undegraded":"2024-03-26T08:40:11.152508+0000","last_fullsized":"2024-03-26T08:40:11.152508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:10.189407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153214+0000","last_change":"2024-03-26T08:40:09.227966+0000","last_active":"2024-03-26T08:40:11.153214+0000","last_peered":"2024-03-26T08:40:11.153214+0000","last_clean":"2024-03-26T08:40:11.153214+0000","last_became_active":"2024-03-26T08:40:09.227638+0000","last_became_peered":"2024-03-26T08:40:09.227638+0000","last_unstale":"2024-03-26T08:40:11.153214+0000","last_undegraded":"2024-03-26T08:40:11.153214+0000","last_fullsized":"2024-03-26T08:40:11.153214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:47:08.131999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153171+0000","last_change":"2024-03-26T08:40:09.935942+0000","last_active":"2024-03-26T08:40:11.153171+0000","last_peered":"2024-03-26T08:40:11.153171+0000","last_clean":"2024-03-26T08:40:11.153171+0000","last_became_active":"2024-03-26T08:40:09.935607+0000","last_became_peered":"2024-03-26T08:40:09.935607+0000","last_unstale":"2024-03-26T08:40:11.153171+0000","last_undegraded":"2024-03-26T08:40:11.153171+0000","last_fullsized":"2024-03-26T08:40:11.153171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:22:59.132614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152487+0000","last_change":"2024-03-26T08:40:09.224385+0000","last_active":"2024-03-26T08:40:11.152487+0000","last_peered":"2024-03-26T08:40:11.152487+0000","last_clean":"2024-03-26T08:40:11.152487+0000","last_became_active":"2024-03-26T08:40:09.223991+0000","last_became_peered":"2024-03-26T08:40:09.223991+0000","last_unstale":"2024-03-26T08:40:11.152487+0000","last_undegraded":"2024-03-26T08:40:11.152487+0000","last_fullsized":"2024-03-26T08:40:11.152487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:55.808922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153073+0000","last_change":"2024-03-26T08:40:09.921072+0000","last_active":"2024-03-26T08:40:11.153073+0000","last_peered":"2024-03-26T08:40:11.153073+0000","last_clean":"2024-03-26T08:40:11.153073+0000","last_became_active":"2024-03-26T08:40:09.920747+0000","last_became_peered":"2024-03-26T08:40:09.920747+0000","last_unstale":"2024-03-26T08:40:11.153073+0000","last_undegraded":"2024-03-26T08:40:11.153073+0000","last_fullsized":"2024-03-26T08:40:11.153073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:20.862765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280809+0000","last_change":"2024-03-26T08:40:09.231344+0000","last_active":"2024-03-26T08:40:11.280809+0000","last_peered":"2024-03-26T08:40:11.280809+0000","last_clean":"2024-03-26T08:40:11.280809+0000","last_became_active":"2024-03-26T08:40:09.229313+0000","last_became_peered":"2024-03-26T08:40:09.229313+0000","last_unstale":"2024-03-26T08:40:11.280809+0000","last_undegraded":"2024-03-26T08:40:11.280809+0000","last_fullsized":"2024-03-26T08:40:11.280809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:37.130884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152698+0000","last_change":"2024-03-26T08:40:09.926355+0000","last_active":"2024-03-26T08:40:11.152698+0000","last_peered":"2024-03-26T08:40:11.152698+0000","last_clean":"2024-03-26T08:40:11.152698+0000","last_became_active":"2024-03-26T08:40:09.922889+0000","last_became_peered":"2024-03-26T08:40:09.922889+0000","last_unstale":"2024-03-26T08:40:11.152698+0000","last_undegraded":"2024-03-26T08:40:11.152698+0000","last_fullsized":"2024-03-26T08:40:11.152698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:42:22.784811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152070+0000","last_change":"2024-03-26T08:40:09.919577+0000","last_active":"2024-03-26T08:40:11.152070+0000","last_peered":"2024-03-26T08:40:11.152070+0000","last_clean":"2024-03-26T08:40:11.152070+0000","last_became_active":"2024-03-26T08:40:09.919234+0000","last_became_peered":"2024-03-26T08:40:09.919234+0000","last_unstale":"2024-03-26T08:40:11.152070+0000","last_undegraded":"2024-03-26T08:40:11.152070+0000","last_fullsized":"2024-03-26T08:40:11.152070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:18.894052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922214+0000","last_change":"2024-03-26T08:40:09.237190+0000","last_active":"2024-03-26T08:40:10.922214+0000","last_peered":"2024-03-26T08:40:10.922214+0000","last_clean":"2024-03-26T08:40:10.922214+0000","last_became_active":"2024-03-26T08:40:09.236862+0000","last_became_peered":"2024-03-26T08:40:09.236862+0000","last_unstale":"2024-03-26T08:40:10.922214+0000","last_undegraded":"2024-03-26T08:40:10.922214+0000","last_fullsized":"2024-03-26T08:40:10.922214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:29.900711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152249+0000","last_change":"2024-03-26T08:40:09.927444+0000","last_active":"2024-03-26T08:40:11.152249+0000","last_peered":"2024-03-26T08:40:11.152249+0000","last_clean":"2024-03-26T08:40:11.152249+0000","last_became_active":"2024-03-26T08:40:09.927152+0000","last_became_peered":"2024-03-26T08:40:09.927152+0000","last_unstale":"2024-03-26T08:40:11.152249+0000","last_undegraded":"2024-03-26T08:40:11.152249+0000","last_fullsized":"2024-03-26T08:40:11.152249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:44.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152240+0000","last_change":"2024-03-26T08:40:09.224831+0000","last_active":"2024-03-26T08:40:11.152240+0000","last_peered":"2024-03-26T08:40:11.152240+0000","last_clean":"2024-03-26T08:40:11.152240+0000","last_became_active":"2024-03-26T08:40:09.224447+0000","last_became_peered":"2024-03-26T08:40:09.224447+0000","last_unstale":"2024-03-26T08:40:11.152240+0000","last_undegraded":"2024-03-26T08:40:11.152240+0000","last_fullsized":"2024-03-26T08:40:11.152240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:25:50.935342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280663+0000","last_change":"2024-03-26T08:38:54.696342+0000","last_active":"2024-03-26T08:40:11.280663+0000","last_peered":"2024-03-26T08:40:11.280663+0000","last_clean":"2024-03-26T08:40:11.280663+0000","last_became_active":"2024-03-26T08:38:54.695839+0000","last_became_peered":"2024-03-26T08:38:54.695839+0000","last_unstale":"2024-03-26T08:40:11.280663+0000","last_undegraded":"2024-03-26T08:40:11.280663+0000","last_fullsized":"2024-03-26T08:40:11.280663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_clean_scrub_stamp":"2024-03-26T08:35:27.189917+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:13.219778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922304+0000","last_change":"2024-03-26T08:40:09.930683+0000","last_active":"2024-03-26T08:40:10.922304+0000","last_peered":"2024-03-26T08:40:10.922304+0000","last_clean":"2024-03-26T08:40:10.922304+0000","last_became_active":"2024-03-26T08:40:09.930341+0000","last_became_peered":"2024-03-26T08:40:09.930341+0000","last_unstale":"2024-03-26T08:40:10.922304+0000","last_undegraded":"2024-03-26T08:40:10.922304+0000","last_fullsized":"2024-03-26T08:40:10.922304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:39:20.232007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280852+0000","last_change":"2024-03-26T08:40:09.228548+0000","last_active":"2024-03-26T08:40:11.280852+0000","last_peered":"2024-03-26T08:40:11.280852+0000","last_clean":"2024-03-26T08:40:11.280852+0000","last_became_active":"2024-03-26T08:40:09.228076+0000","last_became_peered":"2024-03-26T08:40:09.228076+0000","last_unstale":"2024-03-26T08:40:11.280852+0000","last_undegraded":"2024-03-26T08:40:11.280852+0000","last_fullsized":"2024-03-26T08:40:11.280852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:52:51.641128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153279+0000","last_change":"2024-03-26T08:40:09.235632+0000","last_active":"2024-03-26T08:40:11.153279+0000","last_peered":"2024-03-26T08:40:11.153279+0000","last_clean":"2024-03-26T08:40:11.153279+0000","last_became_active":"2024-03-26T08:40:09.235334+0000","last_became_peered":"2024-03-26T08:40:09.235334+0000","last_unstale":"2024-03-26T08:40:11.153279+0000","last_undegraded":"2024-03-26T08:40:11.153279+0000","last_fullsized":"2024-03-26T08:40:11.153279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:38:27.064431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922721+0000","last_change":"2024-03-26T08:40:09.929137+0000","last_active":"2024-03-26T08:40:10.922721+0000","last_peered":"2024-03-26T08:40:10.922721+0000","last_clean":"2024-03-26T08:40:10.922721+0000","last_became_active":"2024-03-26T08:40:09.928831+0000","last_became_peered":"2024-03-26T08:40:09.928831+0000","last_unstale":"2024-03-26T08:40:10.922721+0000","last_undegraded":"2024-03-26T08:40:10.922721+0000","last_fullsized":"2024-03-26T08:40:10.922721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:19.235168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.282379+0000","last_change":"2024-03-26T08:40:09.227153+0000","last_active":"2024-03-26T08:40:14.282379+0000","last_peered":"2024-03-26T08:40:14.282379+0000","last_clean":"2024-03-26T08:40:14.282379+0000","last_became_active":"2024-03-26T08:40:09.226772+0000","last_became_peered":"2024-03-26T08:40:09.226772+0000","last_unstale":"2024-03-26T08:40:14.282379+0000","last_undegraded":"2024-03-26T08:40:14.282379+0000","last_fullsized":"2024-03-26T08:40:14.282379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:33.159538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152953+0000","last_change":"2024-03-26T08:40:09.934589+0000","last_active":"2024-03-26T08:40:11.152953+0000","last_peered":"2024-03-26T08:40:11.152953+0000","last_clean":"2024-03-26T08:40:11.152953+0000","last_became_active":"2024-03-26T08:40:09.934312+0000","last_became_peered":"2024-03-26T08:40:09.934312+0000","last_unstale":"2024-03-26T08:40:11.152953+0000","last_undegraded":"2024-03-26T08:40:11.152953+0000","last_fullsized":"2024-03-26T08:40:11.152953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:19:27.737506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152436+0000","last_change":"2024-03-26T08:40:09.230014+0000","last_active":"2024-03-26T08:40:11.152436+0000","last_peered":"2024-03-26T08:40:11.152436+0000","last_clean":"2024-03-26T08:40:11.152436+0000","last_became_active":"2024-03-26T08:40:09.229682+0000","last_became_peered":"2024-03-26T08:40:09.229682+0000","last_unstale":"2024-03-26T08:40:11.152436+0000","last_undegraded":"2024-03-26T08:40:11.152436+0000","last_fullsized":"2024-03-26T08:40:11.152436+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:52.725810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094436+0000","last_change":"2024-03-26T08:40:09.925593+0000","last_active":"2024-03-26T08:40:11.094436+0000","last_peered":"2024-03-26T08:40:11.094436+0000","last_clean":"2024-03-26T08:40:11.094436+0000","last_became_active":"2024-03-26T08:40:09.925334+0000","last_became_peered":"2024-03-26T08:40:09.925334+0000","last_unstale":"2024-03-26T08:40:11.094436+0000","last_undegraded":"2024-03-26T08:40:11.094436+0000","last_fullsized":"2024-03-26T08:40:11.094436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:25:12.110345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922662+0000","last_change":"2024-03-26T08:40:09.226634+0000","last_active":"2024-03-26T08:40:10.922662+0000","last_peered":"2024-03-26T08:40:10.922662+0000","last_clean":"2024-03-26T08:40:10.922662+0000","last_became_active":"2024-03-26T08:40:09.226248+0000","last_became_peered":"2024-03-26T08:40:09.226248+0000","last_unstale":"2024-03-26T08:40:10.922662+0000","last_undegraded":"2024-03-26T08:40:10.922662+0000","last_fullsized":"2024-03-26T08:40:10.922662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:24:00.390573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152278+0000","last_change":"2024-03-26T08:40:09.934848+0000","last_active":"2024-03-26T08:40:11.152278+0000","last_peered":"2024-03-26T08:40:11.152278+0000","last_clean":"2024-03-26T08:40:11.152278+0000","last_became_active":"2024-03-26T08:40:09.934521+0000","last_became_peered":"2024-03-26T08:40:09.934521+0000","last_unstale":"2024-03-26T08:40:11.152278+0000","last_undegraded":"2024-03-26T08:40:11.152278+0000","last_fullsized":"2024-03-26T08:40:11.152278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:24:25.994961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.272785+0000","last_change":"2024-03-26T08:40:09.222945+0000","last_active":"2024-03-26T08:40:13.272785+0000","last_peered":"2024-03-26T08:40:13.272785+0000","last_clean":"2024-03-26T08:40:13.272785+0000","last_became_active":"2024-03-26T08:40:09.222564+0000","last_became_peered":"2024-03-26T08:40:09.222564+0000","last_unstale":"2024-03-26T08:40:13.272785+0000","last_undegraded":"2024-03-26T08:40:13.272785+0000","last_fullsized":"2024-03-26T08:40:13.272785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:35:11.181812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152602+0000","last_change":"2024-03-26T08:40:09.932674+0000","last_active":"2024-03-26T08:40:11.152602+0000","last_peered":"2024-03-26T08:40:11.152602+0000","last_clean":"2024-03-26T08:40:11.152602+0000","last_became_active":"2024-03-26T08:40:09.932346+0000","last_became_peered":"2024-03-26T08:40:09.932346+0000","last_unstale":"2024-03-26T08:40:11.152602+0000","last_undegraded":"2024-03-26T08:40:11.152602+0000","last_fullsized":"2024-03-26T08:40:11.152602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:12:45.871467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.212432+0000","last_change":"2024-03-26T08:40:09.228605+0000","last_active":"2024-03-26T08:40:11.212432+0000","last_peered":"2024-03-26T08:40:11.212432+0000","last_clean":"2024-03-26T08:40:11.212432+0000","last_became_active":"2024-03-26T08:40:09.228233+0000","last_became_peered":"2024-03-26T08:40:09.228233+0000","last_unstale":"2024-03-26T08:40:11.212432+0000","last_undegraded":"2024-03-26T08:40:11.212432+0000","last_fullsized":"2024-03-26T08:40:11.212432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:32:25.977866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152347+0000","last_change":"2024-03-26T08:40:09.922766+0000","last_active":"2024-03-26T08:40:11.152347+0000","last_peered":"2024-03-26T08:40:11.152347+0000","last_clean":"2024-03-26T08:40:11.152347+0000","last_became_active":"2024-03-26T08:40:09.922453+0000","last_became_peered":"2024-03-26T08:40:09.922453+0000","last_unstale":"2024-03-26T08:40:11.152347+0000","last_undegraded":"2024-03-26T08:40:11.152347+0000","last_fullsized":"2024-03-26T08:40:11.152347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:13:28.740291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.274801+0000","last_change":"2024-03-26T08:40:09.228114+0000","last_active":"2024-03-26T08:40:13.274801+0000","last_peered":"2024-03-26T08:40:13.274801+0000","last_clean":"2024-03-26T08:40:13.274801+0000","last_became_active":"2024-03-26T08:40:09.227803+0000","last_became_peered":"2024-03-26T08:40:09.227803+0000","last_unstale":"2024-03-26T08:40:13.274801+0000","last_undegraded":"2024-03-26T08:40:13.274801+0000","last_fullsized":"2024-03-26T08:40:13.274801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:31.717981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280945+0000","last_change":"2024-03-26T08:40:09.929671+0000","last_active":"2024-03-26T08:40:11.280945+0000","last_peered":"2024-03-26T08:40:11.280945+0000","last_clean":"2024-03-26T08:40:11.280945+0000","last_became_active":"2024-03-26T08:40:09.929222+0000","last_became_peered":"2024-03-26T08:40:09.929222+0000","last_unstale":"2024-03-26T08:40:11.280945+0000","last_undegraded":"2024-03-26T08:40:11.280945+0000","last_fullsized":"2024-03-26T08:40:11.280945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:00:50.579569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.282891+0000","last_change":"2024-03-26T08:40:09.228994+0000","last_active":"2024-03-26T08:40:14.282891+0000","last_peered":"2024-03-26T08:40:14.282891+0000","last_clean":"2024-03-26T08:40:14.282891+0000","last_became_active":"2024-03-26T08:40:09.227138+0000","last_became_peered":"2024-03-26T08:40:09.227138+0000","last_unstale":"2024-03-26T08:40:14.282891+0000","last_undegraded":"2024-03-26T08:40:14.282891+0000","last_fullsized":"2024-03-26T08:40:14.282891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:06.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281218+0000","last_change":"2024-03-26T08:40:09.931752+0000","last_active":"2024-03-26T08:40:11.281218+0000","last_peered":"2024-03-26T08:40:11.281218+0000","last_clean":"2024-03-26T08:40:11.281218+0000","last_became_active":"2024-03-26T08:40:09.931436+0000","last_became_peered":"2024-03-26T08:40:09.931436+0000","last_unstale":"2024-03-26T08:40:11.281218+0000","last_undegraded":"2024-03-26T08:40:11.281218+0000","last_fullsized":"2024-03-26T08:40:11.281218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:40:54.864614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152958+0000","last_change":"2024-03-26T08:40:09.238356+0000","last_active":"2024-03-26T08:40:11.152958+0000","last_peered":"2024-03-26T08:40:11.152958+0000","last_clean":"2024-03-26T08:40:11.152958+0000","last_became_active":"2024-03-26T08:40:09.238066+0000","last_became_peered":"2024-03-26T08:40:09.238066+0000","last_unstale":"2024-03-26T08:40:11.152958+0000","last_undegraded":"2024-03-26T08:40:11.152958+0000","last_fullsized":"2024-03-26T08:40:11.152958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:13.851490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281049+0000","last_change":"2024-03-26T08:40:09.933223+0000","last_active":"2024-03-26T08:40:11.281049+0000","last_peered":"2024-03-26T08:40:11.281049+0000","last_clean":"2024-03-26T08:40:11.281049+0000","last_became_active":"2024-03-26T08:40:09.932903+0000","last_became_peered":"2024-03-26T08:40:09.932903+0000","last_unstale":"2024-03-26T08:40:11.281049+0000","last_undegraded":"2024-03-26T08:40:11.281049+0000","last_fullsized":"2024-03-26T08:40:11.281049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:02:16.079139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922655+0000","last_change":"2024-03-26T08:40:09.238009+0000","last_active":"2024-03-26T08:40:10.922655+0000","last_peered":"2024-03-26T08:40:10.922655+0000","last_clean":"2024-03-26T08:40:10.922655+0000","last_became_active":"2024-03-26T08:40:09.237697+0000","last_became_peered":"2024-03-26T08:40:09.237697+0000","last_unstale":"2024-03-26T08:40:10.922655+0000","last_undegraded":"2024-03-26T08:40:10.922655+0000","last_fullsized":"2024-03-26T08:40:10.922655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:17:33.179285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281229+0000","last_change":"2024-03-26T08:40:09.934672+0000","last_active":"2024-03-26T08:40:11.281229+0000","last_peered":"2024-03-26T08:40:11.281229+0000","last_clean":"2024-03-26T08:40:11.281229+0000","last_became_active":"2024-03-26T08:40:09.934365+0000","last_became_peered":"2024-03-26T08:40:09.934365+0000","last_unstale":"2024-03-26T08:40:11.281229+0000","last_undegraded":"2024-03-26T08:40:11.281229+0000","last_fullsized":"2024-03-26T08:40:11.281229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:41:22.318481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152766+0000","last_change":"2024-03-26T08:40:09.933910+0000","last_active":"2024-03-26T08:40:11.152766+0000","last_peered":"2024-03-26T08:40:11.152766+0000","last_clean":"2024-03-26T08:40:11.152766+0000","last_became_active":"2024-03-26T08:40:09.933598+0000","last_became_peered":"2024-03-26T08:40:09.933598+0000","last_unstale":"2024-03-26T08:40:11.152766+0000","last_undegraded":"2024-03-26T08:40:11.152766+0000","last_fullsized":"2024-03-26T08:40:11.152766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:19.877984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923662+0000","last_change":"2024-03-26T08:40:09.935809+0000","last_active":"2024-03-26T08:40:10.923662+0000","last_peered":"2024-03-26T08:40:10.923662+0000","last_clean":"2024-03-26T08:40:10.923662+0000","last_became_active":"2024-03-26T08:40:09.935500+0000","last_became_peered":"2024-03-26T08:40:09.935500+0000","last_unstale":"2024-03-26T08:40:10.923662+0000","last_undegraded":"2024-03-26T08:40:10.923662+0000","last_fullsized":"2024-03-26T08:40:10.923662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:23.717845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923609+0000","last_change":"2024-03-26T08:40:09.928093+0000","last_active":"2024-03-26T08:40:10.923609+0000","last_peered":"2024-03-26T08:40:10.923609+0000","last_clean":"2024-03-26T08:40:10.923609+0000","last_became_active":"2024-03-26T08:40:09.927732+0000","last_became_peered":"2024-03-26T08:40:09.927732+0000","last_unstale":"2024-03-26T08:40:10.923609+0000","last_undegraded":"2024-03-26T08:40:10.923609+0000","last_fullsized":"2024-03-26T08:40:10.923609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:35:34.972435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094249+0000","last_change":"2024-03-26T08:40:09.925439+0000","last_active":"2024-03-26T08:40:11.094249+0000","last_peered":"2024-03-26T08:40:11.094249+0000","last_clean":"2024-03-26T08:40:11.094249+0000","last_became_active":"2024-03-26T08:40:09.925123+0000","last_became_peered":"2024-03-26T08:40:09.925123+0000","last_unstale":"2024-03-26T08:40:11.094249+0000","last_undegraded":"2024-03-26T08:40:11.094249+0000","last_fullsized":"2024-03-26T08:40:11.094249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:02:17.934610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922648+0000","last_change":"2024-03-26T08:40:09.919466+0000","last_active":"2024-03-26T08:40:10.922648+0000","last_peered":"2024-03-26T08:40:10.922648+0000","last_clean":"2024-03-26T08:40:10.922648+0000","last_became_active":"2024-03-26T08:40:09.919152+0000","last_became_peered":"2024-03-26T08:40:09.919152+0000","last_unstale":"2024-03-26T08:40:10.922648+0000","last_undegraded":"2024-03-26T08:40:10.922648+0000","last_fullsized":"2024-03-26T08:40:10.922648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:37.928485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153433+0000","last_change":"2024-03-26T08:40:09.927140+0000","last_active":"2024-03-26T08:40:11.153433+0000","last_peered":"2024-03-26T08:40:11.153433+0000","last_clean":"2024-03-26T08:40:11.153433+0000","last_became_active":"2024-03-26T08:40:09.926823+0000","last_became_peered":"2024-03-26T08:40:09.926823+0000","last_unstale":"2024-03-26T08:40:11.153433+0000","last_undegraded":"2024-03-26T08:40:11.153433+0000","last_fullsized":"2024-03-26T08:40:11.153433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:29:38.477123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.924081+0000","last_change":"2024-03-26T08:40:09.934202+0000","last_active":"2024-03-26T08:40:10.924081+0000","last_peered":"2024-03-26T08:40:10.924081+0000","last_clean":"2024-03-26T08:40:10.924081+0000","last_became_active":"2024-03-26T08:40:09.933878+0000","last_became_peered":"2024-03-26T08:40:09.933878+0000","last_unstale":"2024-03-26T08:40:10.924081+0000","last_undegraded":"2024-03-26T08:40:10.924081+0000","last_fullsized":"2024-03-26T08:40:10.924081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:44:26.576140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152943+0000","last_change":"2024-03-26T08:40:09.929019+0000","last_active":"2024-03-26T08:40:11.152943+0000","last_peered":"2024-03-26T08:40:11.152943+0000","last_clean":"2024-03-26T08:40:11.152943+0000","last_became_active":"2024-03-26T08:40:09.928711+0000","last_became_peered":"2024-03-26T08:40:09.928711+0000","last_unstale":"2024-03-26T08:40:11.152943+0000","last_undegraded":"2024-03-26T08:40:11.152943+0000","last_fullsized":"2024-03-26T08:40:11.152943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:47.120826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152040+0000","last_change":"2024-03-26T08:40:09.927606+0000","last_active":"2024-03-26T08:40:11.152040+0000","last_peered":"2024-03-26T08:40:11.152040+0000","last_clean":"2024-03-26T08:40:11.152040+0000","last_became_active":"2024-03-26T08:40:09.926717+0000","last_became_peered":"2024-03-26T08:40:09.926717+0000","last_unstale":"2024-03-26T08:40:11.152040+0000","last_undegraded":"2024-03-26T08:40:11.152040+0000","last_fullsized":"2024-03-26T08:40:11.152040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:51:52.567252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152160+0000","last_change":"2024-03-26T08:40:09.934659+0000","last_active":"2024-03-26T08:40:11.152160+0000","last_peered":"2024-03-26T08:40:11.152160+0000","last_clean":"2024-03-26T08:40:11.152160+0000","last_became_active":"2024-03-26T08:40:09.934346+0000","last_became_peered":"2024-03-26T08:40:09.934346+0000","last_unstale":"2024-03-26T08:40:11.152160+0000","last_undegraded":"2024-03-26T08:40:11.152160+0000","last_fullsized":"2024-03-26T08:40:11.152160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:35.009264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922950+0000","last_change":"2024-03-26T08:40:09.925007+0000","last_active":"2024-03-26T08:40:10.922950+0000","last_peered":"2024-03-26T08:40:10.922950+0000","last_clean":"2024-03-26T08:40:10.922950+0000","last_became_active":"2024-03-26T08:40:09.923439+0000","last_became_peered":"2024-03-26T08:40:09.923439+0000","last_unstale":"2024-03-26T08:40:10.922950+0000","last_undegraded":"2024-03-26T08:40:10.922950+0000","last_fullsized":"2024-03-26T08:40:10.922950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:18.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922546+0000","last_change":"2024-03-26T08:40:09.920795+0000","last_active":"2024-03-26T08:40:10.922546+0000","last_peered":"2024-03-26T08:40:10.922546+0000","last_clean":"2024-03-26T08:40:10.922546+0000","last_became_active":"2024-03-26T08:40:09.920378+0000","last_became_peered":"2024-03-26T08:40:09.920378+0000","last_unstale":"2024-03-26T08:40:10.922546+0000","last_undegraded":"2024-03-26T08:40:10.922546+0000","last_fullsized":"2024-03-26T08:40:10.922546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:19:04.027727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153339+0000","last_change":"2024-03-26T08:40:09.929426+0000","last_active":"2024-03-26T08:40:11.153339+0000","last_peered":"2024-03-26T08:40:11.153339+0000","last_clean":"2024-03-26T08:40:11.153339+0000","last_became_active":"2024-03-26T08:40:09.927267+0000","last_became_peered":"2024-03-26T08:40:09.927267+0000","last_unstale":"2024-03-26T08:40:11.153339+0000","last_undegraded":"2024-03-26T08:40:11.153339+0000","last_fullsized":"2024-03-26T08:40:11.153339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:58:55.064053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282339,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11389,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":96,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":45489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332130,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":966656,"data_stored":1029471,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":448474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":429973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":1030756,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":29194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":9790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:40:18.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.228+0000 7f004d885640 1 -- 172.21.15.7:0/3731273426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00481017c0 msgr2=0x7f0048101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.228+0000 7f004d885640 1 --2- 172.21.15.7:0/3731273426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00481017c0 0x7f0048101c40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f003c009900 tx=0x7f003c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.228+0000 7f004d885640 1 -- 172.21.15.7:0/3731273426 shutdown_connections 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.228+0000 7f004d885640 1 --2- 172.21.15.7:0/3731273426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0048102180 0x7f0048108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.228+0000 7f004d885640 1 --2- 172.21.15.7:0/3731273426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00481017c0 0x7f0048101c40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.228+0000 7f004d885640 1 --2- 172.21.15.7:0/3731273426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00481005d0 0x7f00481009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.228+0000 7f004d885640 1 -- 172.21.15.7:0/3731273426 >> 172.21.15.7:0/3731273426 conn(0x7f00480fbb40 msgr2=0x7f00480fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.228+0000 7f004d885640 1 -- 172.21.15.7:0/3731273426 shutdown_connections 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.228+0000 7f004d885640 1 -- 172.21.15.7:0/3731273426 wait complete. 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f004d885640 1 Processor -- start 2024-03-26T08:40:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f004d885640 1 -- start start 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f004d885640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00481005d0 0x7f0048196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f004d885640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00481017c0 0x7f0048196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f004d885640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048102180 0x7f004819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f004d885640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004810c230 con 0x7f0048102180 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f004d885640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004810c0b0 con 0x7f00481005d0 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f004d885640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004810c3b0 con 0x7f00481017c0 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f00477fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048102180 0x7f004819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f00477fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048102180 0x7f004819de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49716/0 (socket says 172.21.15.7:49716) 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f00477fe640 1 -- 172.21.15.7:0/728040796 learned_addr learned my addr 172.21.15.7:0/728040796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f00467fc640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00481017c0 0x7f0048196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f0046ffd640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00481005d0 0x7f0048196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f00477fe640 1 -- 172.21.15.7:0/728040796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00481017c0 msgr2=0x7f0048196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f00477fe640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00481017c0 0x7f0048196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f00477fe640 1 -- 172.21.15.7:0/728040796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00481005d0 msgr2=0x7f0048196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f00477fe640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00481005d0 0x7f0048196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.229+0000 7f00477fe640 1 -- 172.21.15.7:0/728040796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004819e490 con 0x7f0048102180 2024-03-26T08:40:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.230+0000 7f00467fc640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00481017c0 0x7f0048196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.230+0000 7f00477fe640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048102180 0x7f004819de20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f003800eb10 tx=0x7f003800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.230+0000 7f004c883640 1 -- 172.21.15.7:0/728040796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038004b20 con 0x7f0048102180 2024-03-26T08:40:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.230+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004819e780 con 0x7f0048102180 2024-03-26T08:40:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.230+0000 7f004c883640 1 -- 172.21.15.7:0/728040796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0038004cc0 con 0x7f0048102180 2024-03-26T08:40:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.230+0000 7f004c883640 1 -- 172.21.15.7:0/728040796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038010760 con 0x7f0048102180 2024-03-26T08:40:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.230+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004819ec50 con 0x7f0048102180 2024-03-26T08:40:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.231+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00481078f0 con 0x7f0048102180 2024-03-26T08:40:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.232+0000 7f004c883640 1 -- 172.21.15.7:0/728040796 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f0038005020 con 0x7f0048102180 2024-03-26T08:40:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.232+0000 7f004c883640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00300a2890 0x7f00300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.233+0000 7f0046ffd640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00300a2890 0x7f00300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.233+0000 7f004c883640 1 -- 172.21.15.7:0/728040796 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f00380cefb0 con 0x7f0048102180 2024-03-26T08:40:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.233+0000 7f0046ffd640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00300a2890 0x7f00300a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0048101620 tx=0x7f0040005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.237+0000 7f004c883640 1 -- 172.21.15.7:0/728040796 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f003809c0a0 con 0x7f0048102180 2024-03-26T08:40:18.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.419+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0048061ae0 con 0x7f00300a2890 2024-03-26T08:40:18.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.427+0000 7f004c883640 1 -- 172.21.15.7:0/728040796 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258915 (secure 0 0 0) 0x7f0048061ae0 con 0x7f00300a2890 2024-03-26T08:40:18.421 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-26T08:40:18.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00300a2890 msgr2=0x7f00300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00300a2890 0x7f00300a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0048101620 tx=0x7f0040005f90 comp rx=0 tx=0).stop 2024-03-26T08:40:18.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048102180 msgr2=0x7f004819de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:18.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048102180 0x7f004819de20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f003800eb10 tx=0x7f003800c2d0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 shutdown_connections 2024-03-26T08:40:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f00300a2890 0x7f00300a4d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0048102180 0x7f004819de20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f00481017c0 0x7f0048196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 --2- 172.21.15.7:0/728040796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00481005d0 0x7f0048196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 >> 172.21.15.7:0/728040796 conn(0x7f00480fbb40 msgr2=0x7f0048104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 shutdown_connections 2024-03-26T08:40:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:18.430+0000 7f004d885640 1 -- 172.21.15.7:0/728040796 wait complete. 2024-03-26T08:40:18.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:18 smithi007 ceph-mon[32462]: pgmap v294: 97 pgs: 97 active+clean; 588 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2024-03-26T08:40:18.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:18 smithi007 ceph-mon[32462]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:18.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:18 smithi007 ceph-mon[32462]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:18.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:18 smithi007 ceph-mon[32462]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:18.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:18 smithi007 ceph-mon[32462]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T08:40:18.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:18 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/211303810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:18 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3119901134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-03-26T08:40:18.301284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":602586,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":148,"num_read_kb":127,"num_write":303,"num_write_kb":2978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372724,"kb_used_data":11088,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124569100,"statfs":{"total":1151940427776,"available":1151558758400,"internally_reserved":0,"allocated":11354112,"data_stored":7080788,"data_compressed":107721,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12218,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":22,"num_read_kb":18,"num_write":122,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.394032"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153654+0000","last_change":"2024-03-26T08:40:09.939078+0000","last_active":"2024-03-26T08:40:11.153654+0000","last_peered":"2024-03-26T08:40:11.153654+0000","last_clean":"2024-03-26T08:40:11.153654+0000","last_became_active":"2024-03-26T08:40:09.937626+0000","last_became_peered":"2024-03-26T08:40:09.937626+0000","last_unstale":"2024-03-26T08:40:11.153654+0000","last_undegraded":"2024-03-26T08:40:11.153654+0000","last_fullsized":"2024-03-26T08:40:11.153654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:52:59.254181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094283+0000","last_change":"2024-03-26T08:40:09.924207+0000","last_active":"2024-03-26T08:40:11.094283+0000","last_peered":"2024-03-26T08:40:11.094283+0000","last_clean":"2024-03-26T08:40:11.094283+0000","last_became_active":"2024-03-26T08:40:09.923881+0000","last_became_peered":"2024-03-26T08:40:09.923881+0000","last_unstale":"2024-03-26T08:40:11.094283+0000","last_undegraded":"2024-03-26T08:40:11.094283+0000","last_fullsized":"2024-03-26T08:40:11.094283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:16:45.265521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094443+0000","last_change":"2024-03-26T08:40:09.929154+0000","last_active":"2024-03-26T08:40:11.094443+0000","last_peered":"2024-03-26T08:40:11.094443+0000","last_clean":"2024-03-26T08:40:11.094443+0000","last_became_active":"2024-03-26T08:40:09.928833+0000","last_became_peered":"2024-03-26T08:40:09.928833+0000","last_unstale":"2024-03-26T08:40:11.094443+0000","last_undegraded":"2024-03-26T08:40:11.094443+0000","last_fullsized":"2024-03-26T08:40:11.094443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:22.972210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152692+0000","last_change":"2024-03-26T08:40:09.930713+0000","last_active":"2024-03-26T08:40:11.152692+0000","last_peered":"2024-03-26T08:40:11.152692+0000","last_clean":"2024-03-26T08:40:11.152692+0000","last_became_active":"2024-03-26T08:40:09.930420+0000","last_became_peered":"2024-03-26T08:40:09.930420+0000","last_unstale":"2024-03-26T08:40:11.152692+0000","last_undegraded":"2024-03-26T08:40:11.152692+0000","last_fullsized":"2024-03-26T08:40:11.152692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:17:03.015703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153736+0000","last_change":"2024-03-26T08:40:09.923935+0000","last_active":"2024-03-26T08:40:11.153736+0000","last_peered":"2024-03-26T08:40:11.153736+0000","last_clean":"2024-03-26T08:40:11.153736+0000","last_became_active":"2024-03-26T08:40:09.923587+0000","last_became_peered":"2024-03-26T08:40:09.923587+0000","last_unstale":"2024-03-26T08:40:11.153736+0000","last_undegraded":"2024-03-26T08:40:11.153736+0000","last_fullsized":"2024-03-26T08:40:11.153736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:19:38.831448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152667+0000","last_change":"2024-03-26T08:40:09.922361+0000","last_active":"2024-03-26T08:40:11.152667+0000","last_peered":"2024-03-26T08:40:11.152667+0000","last_clean":"2024-03-26T08:40:11.152667+0000","last_became_active":"2024-03-26T08:40:09.921643+0000","last_became_peered":"2024-03-26T08:40:09.921643+0000","last_unstale":"2024-03-26T08:40:11.152667+0000","last_undegraded":"2024-03-26T08:40:11.152667+0000","last_fullsized":"2024-03-26T08:40:11.152667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:37:23.048778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280800+0000","last_change":"2024-03-26T08:40:09.932428+0000","last_active":"2024-03-26T08:40:11.280800+0000","last_peered":"2024-03-26T08:40:11.280800+0000","last_clean":"2024-03-26T08:40:11.280800+0000","last_became_active":"2024-03-26T08:40:09.932120+0000","last_became_peered":"2024-03-26T08:40:09.932120+0000","last_unstale":"2024-03-26T08:40:11.280800+0000","last_undegraded":"2024-03-26T08:40:11.280800+0000","last_fullsized":"2024-03-26T08:40:11.280800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:24.362568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152021+0000","last_change":"2024-03-26T08:40:09.928251+0000","last_active":"2024-03-26T08:40:11.152021+0000","last_peered":"2024-03-26T08:40:11.152021+0000","last_clean":"2024-03-26T08:40:11.152021+0000","last_became_active":"2024-03-26T08:40:09.927961+0000","last_became_peered":"2024-03-26T08:40:09.927961+0000","last_unstale":"2024-03-26T08:40:11.152021+0000","last_undegraded":"2024-03-26T08:40:11.152021+0000","last_fullsized":"2024-03-26T08:40:11.152021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:25:09.348033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923263+0000","last_change":"2024-03-26T08:40:09.930888+0000","last_active":"2024-03-26T08:40:10.923263+0000","last_peered":"2024-03-26T08:40:10.923263+0000","last_clean":"2024-03-26T08:40:10.923263+0000","last_became_active":"2024-03-26T08:40:09.930513+0000","last_became_peered":"2024-03-26T08:40:09.930513+0000","last_unstale":"2024-03-26T08:40:10.923263+0000","last_undegraded":"2024-03-26T08:40:10.923263+0000","last_fullsized":"2024-03-26T08:40:10.923263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:21.023569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153219+0000","last_change":"2024-03-26T08:40:09.935534+0000","last_active":"2024-03-26T08:40:11.153219+0000","last_peered":"2024-03-26T08:40:11.153219+0000","last_clean":"2024-03-26T08:40:11.153219+0000","last_became_active":"2024-03-26T08:40:09.934959+0000","last_became_peered":"2024-03-26T08:40:09.934959+0000","last_unstale":"2024-03-26T08:40:11.153219+0000","last_undegraded":"2024-03-26T08:40:11.153219+0000","last_fullsized":"2024-03-26T08:40:11.153219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:18:59.311180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281138+0000","last_change":"2024-03-26T08:40:09.930486+0000","last_active":"2024-03-26T08:40:11.281138+0000","last_peered":"2024-03-26T08:40:11.281138+0000","last_clean":"2024-03-26T08:40:11.281138+0000","last_became_active":"2024-03-26T08:40:09.930184+0000","last_became_peered":"2024-03-26T08:40:09.930184+0000","last_unstale":"2024-03-26T08:40:11.281138+0000","last_undegraded":"2024-03-26T08:40:11.281138+0000","last_fullsized":"2024-03-26T08:40:11.281138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:07.246878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152530+0000","last_change":"2024-03-26T08:40:09.924305+0000","last_active":"2024-03-26T08:40:11.152530+0000","last_peered":"2024-03-26T08:40:11.152530+0000","last_clean":"2024-03-26T08:40:11.152530+0000","last_became_active":"2024-03-26T08:40:09.923985+0000","last_became_peered":"2024-03-26T08:40:09.923985+0000","last_unstale":"2024-03-26T08:40:11.152530+0000","last_undegraded":"2024-03-26T08:40:11.152530+0000","last_fullsized":"2024-03-26T08:40:11.152530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:42:39.894245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922581+0000","last_change":"2024-03-26T08:40:09.916637+0000","last_active":"2024-03-26T08:40:10.922581+0000","last_peered":"2024-03-26T08:40:10.922581+0000","last_clean":"2024-03-26T08:40:10.922581+0000","last_became_active":"2024-03-26T08:40:09.916314+0000","last_became_peered":"2024-03-26T08:40:09.916314+0000","last_unstale":"2024-03-26T08:40:10.922581+0000","last_undegraded":"2024-03-26T08:40:10.922581+0000","last_fullsized":"2024-03-26T08:40:10.922581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:45:10.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094242+0000","last_change":"2024-03-26T08:40:09.923753+0000","last_active":"2024-03-26T08:40:11.094242+0000","last_peered":"2024-03-26T08:40:11.094242+0000","last_clean":"2024-03-26T08:40:11.094242+0000","last_became_active":"2024-03-26T08:40:09.923427+0000","last_became_peered":"2024-03-26T08:40:09.923427+0000","last_unstale":"2024-03-26T08:40:11.094242+0000","last_undegraded":"2024-03-26T08:40:11.094242+0000","last_fullsized":"2024-03-26T08:40:11.094242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:13:16.711556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094190+0000","last_change":"2024-03-26T08:40:09.931879+0000","last_active":"2024-03-26T08:40:11.094190+0000","last_peered":"2024-03-26T08:40:11.094190+0000","last_clean":"2024-03-26T08:40:11.094190+0000","last_became_active":"2024-03-26T08:40:09.931589+0000","last_became_peered":"2024-03-26T08:40:09.931589+0000","last_unstale":"2024-03-26T08:40:11.094190+0000","last_undegraded":"2024-03-26T08:40:11.094190+0000","last_fullsized":"2024-03-26T08:40:11.094190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:22:01.038745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922718+0000","last_change":"2024-03-26T08:40:09.930500+0000","last_active":"2024-03-26T08:40:10.922718+0000","last_peered":"2024-03-26T08:40:10.922718+0000","last_clean":"2024-03-26T08:40:10.922718+0000","last_became_active":"2024-03-26T08:40:09.930101+0000","last_became_peered":"2024-03-26T08:40:09.930101+0000","last_unstale":"2024-03-26T08:40:10.922718+0000","last_undegraded":"2024-03-26T08:40:10.922718+0000","last_fullsized":"2024-03-26T08:40:10.922718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:54.229004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153660+0000","last_change":"2024-03-26T08:40:09.939177+0000","last_active":"2024-03-26T08:40:11.153660+0000","last_peered":"2024-03-26T08:40:11.153660+0000","last_clean":"2024-03-26T08:40:11.153660+0000","last_became_active":"2024-03-26T08:40:09.937727+0000","last_became_peered":"2024-03-26T08:40:09.937727+0000","last_unstale":"2024-03-26T08:40:11.153660+0000","last_undegraded":"2024-03-26T08:40:11.153660+0000","last_fullsized":"2024-03-26T08:40:11.153660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:40:44.040015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152641+0000","last_change":"2024-03-26T08:40:09.928429+0000","last_active":"2024-03-26T08:40:11.152641+0000","last_peered":"2024-03-26T08:40:11.152641+0000","last_clean":"2024-03-26T08:40:11.152641+0000","last_became_active":"2024-03-26T08:40:09.928025+0000","last_became_peered":"2024-03-26T08:40:09.928025+0000","last_unstale":"2024-03-26T08:40:11.152641+0000","last_undegraded":"2024-03-26T08:40:11.152641+0000","last_fullsized":"2024-03-26T08:40:11.152641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:52:55.450291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923944+0000","last_change":"2024-03-26T08:40:09.930856+0000","last_active":"2024-03-26T08:40:10.923944+0000","last_peered":"2024-03-26T08:40:10.923944+0000","last_clean":"2024-03-26T08:40:10.923944+0000","last_became_active":"2024-03-26T08:40:09.930552+0000","last_became_peered":"2024-03-26T08:40:09.930552+0000","last_unstale":"2024-03-26T08:40:10.923944+0000","last_undegraded":"2024-03-26T08:40:10.923944+0000","last_fullsized":"2024-03-26T08:40:10.923944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:52.851287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.283522+0000","last_change":"2024-03-26T08:40:09.230609+0000","last_active":"2024-03-26T08:40:14.283522+0000","last_peered":"2024-03-26T08:40:14.283522+0000","last_clean":"2024-03-26T08:40:14.283522+0000","last_became_active":"2024-03-26T08:40:09.230284+0000","last_became_peered":"2024-03-26T08:40:09.230284+0000","last_unstale":"2024-03-26T08:40:14.283522+0000","last_undegraded":"2024-03-26T08:40:14.283522+0000","last_fullsized":"2024-03-26T08:40:14.283522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:06.681725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152561+0000","last_change":"2024-03-26T08:40:09.930498+0000","last_active":"2024-03-26T08:40:11.152561+0000","last_peered":"2024-03-26T08:40:11.152561+0000","last_clean":"2024-03-26T08:40:11.152561+0000","last_became_active":"2024-03-26T08:40:09.930209+0000","last_became_peered":"2024-03-26T08:40:09.930209+0000","last_unstale":"2024-03-26T08:40:11.152561+0000","last_undegraded":"2024-03-26T08:40:11.152561+0000","last_fullsized":"2024-03-26T08:40:11.152561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:12:59.437553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152886+0000","last_change":"2024-03-26T08:40:09.235451+0000","last_active":"2024-03-26T08:40:11.152886+0000","last_peered":"2024-03-26T08:40:11.152886+0000","last_clean":"2024-03-26T08:40:11.152886+0000","last_became_active":"2024-03-26T08:40:09.235154+0000","last_became_peered":"2024-03-26T08:40:09.235154+0000","last_unstale":"2024-03-26T08:40:11.152886+0000","last_undegraded":"2024-03-26T08:40:11.152886+0000","last_fullsized":"2024-03-26T08:40:11.152886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:37:38.418888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094433+0000","last_change":"2024-03-26T08:40:09.931728+0000","last_active":"2024-03-26T08:40:11.094433+0000","last_peered":"2024-03-26T08:40:11.094433+0000","last_clean":"2024-03-26T08:40:11.094433+0000","last_became_active":"2024-03-26T08:40:09.931401+0000","last_became_peered":"2024-03-26T08:40:09.931401+0000","last_unstale":"2024-03-26T08:40:11.094433+0000","last_undegraded":"2024-03-26T08:40:11.094433+0000","last_fullsized":"2024-03-26T08:40:11.094433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:02.229763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153427+0000","last_change":"2024-03-26T08:40:09.231247+0000","last_active":"2024-03-26T08:40:11.153427+0000","last_peered":"2024-03-26T08:40:11.153427+0000","last_clean":"2024-03-26T08:40:11.153427+0000","last_became_active":"2024-03-26T08:40:09.230914+0000","last_became_peered":"2024-03-26T08:40:09.230914+0000","last_unstale":"2024-03-26T08:40:11.153427+0000","last_undegraded":"2024-03-26T08:40:11.153427+0000","last_fullsized":"2024-03-26T08:40:11.153427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:03.956444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922935+0000","last_change":"2024-03-26T08:40:09.925174+0000","last_active":"2024-03-26T08:40:10.922935+0000","last_peered":"2024-03-26T08:40:10.922935+0000","last_clean":"2024-03-26T08:40:10.922935+0000","last_became_active":"2024-03-26T08:40:09.924840+0000","last_became_peered":"2024-03-26T08:40:09.924840+0000","last_unstale":"2024-03-26T08:40:10.922935+0000","last_undegraded":"2024-03-26T08:40:10.922935+0000","last_fullsized":"2024-03-26T08:40:10.922935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:57:19.041245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152847+0000","last_change":"2024-03-26T08:40:09.229504+0000","last_active":"2024-03-26T08:40:11.152847+0000","last_peered":"2024-03-26T08:40:11.152847+0000","last_clean":"2024-03-26T08:40:11.152847+0000","last_became_active":"2024-03-26T08:40:09.226449+0000","last_became_peered":"2024-03-26T08:40:09.226449+0000","last_unstale":"2024-03-26T08:40:11.152847+0000","last_undegraded":"2024-03-26T08:40:11.152847+0000","last_fullsized":"2024-03-26T08:40:11.152847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:38.047424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153068+0000","last_change":"2024-03-26T08:40:09.930184+0000","last_active":"2024-03-26T08:40:11.153068+0000","last_peered":"2024-03-26T08:40:11.153068+0000","last_clean":"2024-03-26T08:40:11.153068+0000","last_became_active":"2024-03-26T08:40:09.927679+0000","last_became_peered":"2024-03-26T08:40:09.927679+0000","last_unstale":"2024-03-26T08:40:11.153068+0000","last_undegraded":"2024-03-26T08:40:11.153068+0000","last_fullsized":"2024-03-26T08:40:11.153068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:05:52.949814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:15.288057+0000","last_change":"2024-03-26T08:40:09.233025+0000","last_active":"2024-03-26T08:40:15.288057+0000","last_peered":"2024-03-26T08:40:15.288057+0000","last_clean":"2024-03-26T08:40:15.288057+0000","last_became_active":"2024-03-26T08:40:09.232654+0000","last_became_peered":"2024-03-26T08:40:09.232654+0000","last_unstale":"2024-03-26T08:40:15.288057+0000","last_undegraded":"2024-03-26T08:40:15.288057+0000","last_fullsized":"2024-03-26T08:40:15.288057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:01.745583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152492+0000","last_change":"2024-03-26T08:40:09.924187+0000","last_active":"2024-03-26T08:40:11.152492+0000","last_peered":"2024-03-26T08:40:11.152492+0000","last_clean":"2024-03-26T08:40:11.152492+0000","last_became_active":"2024-03-26T08:40:09.923861+0000","last_became_peered":"2024-03-26T08:40:09.923861+0000","last_unstale":"2024-03-26T08:40:11.152492+0000","last_undegraded":"2024-03-26T08:40:11.152492+0000","last_fullsized":"2024-03-26T08:40:11.152492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:12:17.284186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.283592+0000","last_change":"2024-03-26T08:40:09.231368+0000","last_active":"2024-03-26T08:40:14.283592+0000","last_peered":"2024-03-26T08:40:14.283592+0000","last_clean":"2024-03-26T08:40:14.283592+0000","last_became_active":"2024-03-26T08:40:09.231040+0000","last_became_peered":"2024-03-26T08:40:09.231040+0000","last_unstale":"2024-03-26T08:40:14.283592+0000","last_undegraded":"2024-03-26T08:40:14.283592+0000","last_fullsized":"2024-03-26T08:40:14.283592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:14:20.681225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152685+0000","last_change":"2024-03-26T08:40:09.928125+0000","last_active":"2024-03-26T08:40:11.152685+0000","last_peered":"2024-03-26T08:40:11.152685+0000","last_clean":"2024-03-26T08:40:11.152685+0000","last_became_active":"2024-03-26T08:40:09.927616+0000","last_became_peered":"2024-03-26T08:40:09.927616+0000","last_unstale":"2024-03-26T08:40:11.152685+0000","last_undegraded":"2024-03-26T08:40:11.152685+0000","last_fullsized":"2024-03-26T08:40:11.152685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:12.764054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.282793+0000","last_change":"2024-03-26T08:40:09.229976+0000","last_active":"2024-03-26T08:40:14.282793+0000","last_peered":"2024-03-26T08:40:14.282793+0000","last_clean":"2024-03-26T08:40:14.282793+0000","last_became_active":"2024-03-26T08:40:09.229594+0000","last_became_peered":"2024-03-26T08:40:09.229594+0000","last_unstale":"2024-03-26T08:40:14.282793+0000","last_undegraded":"2024-03-26T08:40:14.282793+0000","last_fullsized":"2024-03-26T08:40:14.282793+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:31:24.204853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.151949+0000","last_change":"2024-03-26T08:40:09.931290+0000","last_active":"2024-03-26T08:40:11.151949+0000","last_peered":"2024-03-26T08:40:11.151949+0000","last_clean":"2024-03-26T08:40:11.151949+0000","last_became_active":"2024-03-26T08:40:09.930961+0000","last_became_peered":"2024-03-26T08:40:09.930961+0000","last_unstale":"2024-03-26T08:40:11.151949+0000","last_undegraded":"2024-03-26T08:40:11.151949+0000","last_fullsized":"2024-03-26T08:40:11.151949+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:36.460807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.151949+0000","last_change":"2024-03-26T08:40:09.232166+0000","last_active":"2024-03-26T08:40:11.151949+0000","last_peered":"2024-03-26T08:40:11.151949+0000","last_clean":"2024-03-26T08:40:11.151949+0000","last_became_active":"2024-03-26T08:40:09.231840+0000","last_became_peered":"2024-03-26T08:40:09.231840+0000","last_unstale":"2024-03-26T08:40:11.151949+0000","last_undegraded":"2024-03-26T08:40:11.151949+0000","last_fullsized":"2024-03-26T08:40:11.151949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:51.636322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152163+0000","last_change":"2024-03-26T08:40:09.924680+0000","last_active":"2024-03-26T08:40:11.152163+0000","last_peered":"2024-03-26T08:40:11.152163+0000","last_clean":"2024-03-26T08:40:11.152163+0000","last_became_active":"2024-03-26T08:40:09.924314+0000","last_became_peered":"2024-03-26T08:40:09.924314+0000","last_unstale":"2024-03-26T08:40:11.152163+0000","last_undegraded":"2024-03-26T08:40:11.152163+0000","last_fullsized":"2024-03-26T08:40:11.152163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:49:20.780950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.273704+0000","last_change":"2024-03-26T08:40:09.235997+0000","last_active":"2024-03-26T08:40:13.273704+0000","last_peered":"2024-03-26T08:40:13.273704+0000","last_clean":"2024-03-26T08:40:13.273704+0000","last_became_active":"2024-03-26T08:40:09.235729+0000","last_became_peered":"2024-03-26T08:40:09.235729+0000","last_unstale":"2024-03-26T08:40:13.273704+0000","last_undegraded":"2024-03-26T08:40:13.273704+0000","last_fullsized":"2024-03-26T08:40:13.273704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:46:12.322059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923227+0000","last_change":"2024-03-26T08:40:09.926371+0000","last_active":"2024-03-26T08:40:10.923227+0000","last_peered":"2024-03-26T08:40:10.923227+0000","last_clean":"2024-03-26T08:40:10.923227+0000","last_became_active":"2024-03-26T08:40:09.925536+0000","last_became_peered":"2024-03-26T08:40:09.925536+0000","last_unstale":"2024-03-26T08:40:10.923227+0000","last_undegraded":"2024-03-26T08:40:10.923227+0000","last_fullsized":"2024-03-26T08:40:10.923227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:50:39.101684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.274273+0000","last_change":"2024-03-26T08:40:09.229018+0000","last_active":"2024-03-26T08:40:13.274273+0000","last_peered":"2024-03-26T08:40:13.274273+0000","last_clean":"2024-03-26T08:40:13.274273+0000","last_became_active":"2024-03-26T08:40:09.228680+0000","last_became_peered":"2024-03-26T08:40:09.228680+0000","last_unstale":"2024-03-26T08:40:13.274273+0000","last_undegraded":"2024-03-26T08:40:13.274273+0000","last_fullsized":"2024-03-26T08:40:13.274273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:31:10.065127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094419+0000","last_change":"2024-03-26T08:40:09.926152+0000","last_active":"2024-03-26T08:40:11.094419+0000","last_peered":"2024-03-26T08:40:11.094419+0000","last_clean":"2024-03-26T08:40:11.094419+0000","last_became_active":"2024-03-26T08:40:09.925825+0000","last_became_peered":"2024-03-26T08:40:09.925825+0000","last_unstale":"2024-03-26T08:40:11.094419+0000","last_undegraded":"2024-03-26T08:40:11.094419+0000","last_fullsized":"2024-03-26T08:40:11.094419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:28:24.287706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.273102+0000","last_change":"2024-03-26T08:40:09.234699+0000","last_active":"2024-03-26T08:40:13.273102+0000","last_peered":"2024-03-26T08:40:13.273102+0000","last_clean":"2024-03-26T08:40:13.273102+0000","last_became_active":"2024-03-26T08:40:09.234451+0000","last_became_peered":"2024-03-26T08:40:09.234451+0000","last_unstale":"2024-03-26T08:40:13.273102+0000","last_undegraded":"2024-03-26T08:40:13.273102+0000","last_fullsized":"2024-03-26T08:40:13.273102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:53.337418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923055+0000","last_change":"2024-03-26T08:40:09.927023+0000","last_active":"2024-03-26T08:40:10.923055+0000","last_peered":"2024-03-26T08:40:10.923055+0000","last_clean":"2024-03-26T08:40:10.923055+0000","last_became_active":"2024-03-26T08:40:09.926734+0000","last_became_peered":"2024-03-26T08:40:09.926734+0000","last_unstale":"2024-03-26T08:40:10.923055+0000","last_undegraded":"2024-03-26T08:40:10.923055+0000","last_fullsized":"2024-03-26T08:40:10.923055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:19:44.137513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:15.636414+0000","last_change":"2024-03-26T08:40:09.230059+0000","last_active":"2024-03-26T08:40:15.636414+0000","last_peered":"2024-03-26T08:40:15.636414+0000","last_clean":"2024-03-26T08:40:15.636414+0000","last_became_active":"2024-03-26T08:40:09.229776+0000","last_became_peered":"2024-03-26T08:40:09.229776+0000","last_unstale":"2024-03-26T08:40:15.636414+0000","last_undegraded":"2024-03-26T08:40:15.636414+0000","last_fullsized":"2024-03-26T08:40:15.636414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:00.917694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153177+0000","last_change":"2024-03-26T08:40:09.930702+0000","last_active":"2024-03-26T08:40:11.153177+0000","last_peered":"2024-03-26T08:40:11.153177+0000","last_clean":"2024-03-26T08:40:11.153177+0000","last_became_active":"2024-03-26T08:40:09.930400+0000","last_became_peered":"2024-03-26T08:40:09.930400+0000","last_unstale":"2024-03-26T08:40:11.153177+0000","last_undegraded":"2024-03-26T08:40:11.153177+0000","last_fullsized":"2024-03-26T08:40:11.153177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:03:12.897812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.283373+0000","last_change":"2024-03-26T08:40:09.225972+0000","last_active":"2024-03-26T08:40:14.283373+0000","last_peered":"2024-03-26T08:40:14.283373+0000","last_clean":"2024-03-26T08:40:14.283373+0000","last_became_active":"2024-03-26T08:40:09.225524+0000","last_became_peered":"2024-03-26T08:40:09.225524+0000","last_unstale":"2024-03-26T08:40:14.283373+0000","last_undegraded":"2024-03-26T08:40:14.283373+0000","last_fullsized":"2024-03-26T08:40:14.283373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:08:17.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922439+0000","last_change":"2024-03-26T08:40:09.920628+0000","last_active":"2024-03-26T08:40:10.922439+0000","last_peered":"2024-03-26T08:40:10.922439+0000","last_clean":"2024-03-26T08:40:10.922439+0000","last_became_active":"2024-03-26T08:40:09.920300+0000","last_became_peered":"2024-03-26T08:40:09.920300+0000","last_unstale":"2024-03-26T08:40:10.922439+0000","last_undegraded":"2024-03-26T08:40:10.922439+0000","last_fullsized":"2024-03-26T08:40:10.922439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:02:16.503881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.273465+0000","last_change":"2024-03-26T08:40:09.237802+0000","last_active":"2024-03-26T08:40:13.273465+0000","last_peered":"2024-03-26T08:40:13.273465+0000","last_clean":"2024-03-26T08:40:13.273465+0000","last_became_active":"2024-03-26T08:40:09.237425+0000","last_became_peered":"2024-03-26T08:40:09.237425+0000","last_unstale":"2024-03-26T08:40:13.273465+0000","last_undegraded":"2024-03-26T08:40:13.273465+0000","last_fullsized":"2024-03-26T08:40:13.273465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:05.953882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281102+0000","last_change":"2024-03-26T08:40:09.923221+0000","last_active":"2024-03-26T08:40:11.281102+0000","last_peered":"2024-03-26T08:40:11.281102+0000","last_clean":"2024-03-26T08:40:11.281102+0000","last_became_active":"2024-03-26T08:40:09.922894+0000","last_became_peered":"2024-03-26T08:40:09.922894+0000","last_unstale":"2024-03-26T08:40:11.281102+0000","last_undegraded":"2024-03-26T08:40:11.281102+0000","last_fullsized":"2024-03-26T08:40:11.281102+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:24.887625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:15.287212+0000","last_change":"2024-03-26T08:40:09.225179+0000","last_active":"2024-03-26T08:40:15.287212+0000","last_peered":"2024-03-26T08:40:15.287212+0000","last_clean":"2024-03-26T08:40:15.287212+0000","last_became_active":"2024-03-26T08:40:09.224861+0000","last_became_peered":"2024-03-26T08:40:09.224861+0000","last_unstale":"2024-03-26T08:40:15.287212+0000","last_undegraded":"2024-03-26T08:40:15.287212+0000","last_fullsized":"2024-03-26T08:40:15.287212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:37:54.843284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280729+0000","last_change":"2024-03-26T08:40:09.932590+0000","last_active":"2024-03-26T08:40:11.280729+0000","last_peered":"2024-03-26T08:40:11.280729+0000","last_clean":"2024-03-26T08:40:11.280729+0000","last_became_active":"2024-03-26T08:40:09.932283+0000","last_became_peered":"2024-03-26T08:40:09.932283+0000","last_unstale":"2024-03-26T08:40:11.280729+0000","last_undegraded":"2024-03-26T08:40:11.280729+0000","last_fullsized":"2024-03-26T08:40:11.280729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:23:41.449789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.274347+0000","last_change":"2024-03-26T08:40:09.232024+0000","last_active":"2024-03-26T08:40:13.274347+0000","last_peered":"2024-03-26T08:40:13.274347+0000","last_clean":"2024-03-26T08:40:13.274347+0000","last_became_active":"2024-03-26T08:40:09.231715+0000","last_became_peered":"2024-03-26T08:40:09.231715+0000","last_unstale":"2024-03-26T08:40:13.274347+0000","last_undegraded":"2024-03-26T08:40:13.274347+0000","last_fullsized":"2024-03-26T08:40:13.274347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:02.229042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152508+0000","last_change":"2024-03-26T08:40:09.926893+0000","last_active":"2024-03-26T08:40:11.152508+0000","last_peered":"2024-03-26T08:40:11.152508+0000","last_clean":"2024-03-26T08:40:11.152508+0000","last_became_active":"2024-03-26T08:40:09.926529+0000","last_became_peered":"2024-03-26T08:40:09.926529+0000","last_unstale":"2024-03-26T08:40:11.152508+0000","last_undegraded":"2024-03-26T08:40:11.152508+0000","last_fullsized":"2024-03-26T08:40:11.152508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:10.189407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153214+0000","last_change":"2024-03-26T08:40:09.227966+0000","last_active":"2024-03-26T08:40:11.153214+0000","last_peered":"2024-03-26T08:40:11.153214+0000","last_clean":"2024-03-26T08:40:11.153214+0000","last_became_active":"2024-03-26T08:40:09.227638+0000","last_became_peered":"2024-03-26T08:40:09.227638+0000","last_unstale":"2024-03-26T08:40:11.153214+0000","last_undegraded":"2024-03-26T08:40:11.153214+0000","last_fullsized":"2024-03-26T08:40:11.153214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:47:08.131999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153171+0000","last_change":"2024-03-26T08:40:09.935942+0000","last_active":"2024-03-26T08:40:11.153171+0000","last_peered":"2024-03-26T08:40:11.153171+0000","last_clean":"2024-03-26T08:40:11.153171+0000","last_became_active":"2024-03-26T08:40:09.935607+0000","last_became_peered":"2024-03-26T08:40:09.935607+0000","last_unstale":"2024-03-26T08:40:11.153171+0000","last_undegraded":"2024-03-26T08:40:11.153171+0000","last_fullsized":"2024-03-26T08:40:11.153171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:22:59.132614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152487+0000","last_change":"2024-03-26T08:40:09.224385+0000","last_active":"2024-03-26T08:40:11.152487+0000","last_peered":"2024-03-26T08:40:11.152487+0000","last_clean":"2024-03-26T08:40:11.152487+0000","last_became_active":"2024-03-26T08:40:09.223991+0000","last_became_peered":"2024-03-26T08:40:09.223991+0000","last_unstale":"2024-03-26T08:40:11.152487+0000","last_undegraded":"2024-03-26T08:40:11.152487+0000","last_fullsized":"2024-03-26T08:40:11.152487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:55.808922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153073+0000","last_change":"2024-03-26T08:40:09.921072+0000","last_active":"2024-03-26T08:40:11.153073+0000","last_peered":"2024-03-26T08:40:11.153073+0000","last_clean":"2024-03-26T08:40:11.153073+0000","last_became_active":"2024-03-26T08:40:09.920747+0000","last_became_peered":"2024-03-26T08:40:09.920747+0000","last_unstale":"2024-03-26T08:40:11.153073+0000","last_undegraded":"2024-03-26T08:40:11.153073+0000","last_fullsized":"2024-03-26T08:40:11.153073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:20.862765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:12.678157+0000","last_change":"2024-03-26T08:40:09.231344+0000","last_active":"2024-03-26T08:40:12.678157+0000","last_peered":"2024-03-26T08:40:12.678157+0000","last_clean":"2024-03-26T08:40:12.678157+0000","last_became_active":"2024-03-26T08:40:09.229313+0000","last_became_peered":"2024-03-26T08:40:09.229313+0000","last_unstale":"2024-03-26T08:40:12.678157+0000","last_undegraded":"2024-03-26T08:40:12.678157+0000","last_fullsized":"2024-03-26T08:40:12.678157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:37.130884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152698+0000","last_change":"2024-03-26T08:40:09.926355+0000","last_active":"2024-03-26T08:40:11.152698+0000","last_peered":"2024-03-26T08:40:11.152698+0000","last_clean":"2024-03-26T08:40:11.152698+0000","last_became_active":"2024-03-26T08:40:09.922889+0000","last_became_peered":"2024-03-26T08:40:09.922889+0000","last_unstale":"2024-03-26T08:40:11.152698+0000","last_undegraded":"2024-03-26T08:40:11.152698+0000","last_fullsized":"2024-03-26T08:40:11.152698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:42:22.784811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152070+0000","last_change":"2024-03-26T08:40:09.919577+0000","last_active":"2024-03-26T08:40:11.152070+0000","last_peered":"2024-03-26T08:40:11.152070+0000","last_clean":"2024-03-26T08:40:11.152070+0000","last_became_active":"2024-03-26T08:40:09.919234+0000","last_became_peered":"2024-03-26T08:40:09.919234+0000","last_unstale":"2024-03-26T08:40:11.152070+0000","last_undegraded":"2024-03-26T08:40:11.152070+0000","last_fullsized":"2024-03-26T08:40:11.152070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:18.894052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:15.285182+0000","last_change":"2024-03-26T08:40:09.237190+0000","last_active":"2024-03-26T08:40:15.285182+0000","last_peered":"2024-03-26T08:40:15.285182+0000","last_clean":"2024-03-26T08:40:15.285182+0000","last_became_active":"2024-03-26T08:40:09.236862+0000","last_became_peered":"2024-03-26T08:40:09.236862+0000","last_unstale":"2024-03-26T08:40:15.285182+0000","last_undegraded":"2024-03-26T08:40:15.285182+0000","last_fullsized":"2024-03-26T08:40:15.285182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:29.900711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152249+0000","last_change":"2024-03-26T08:40:09.927444+0000","last_active":"2024-03-26T08:40:11.152249+0000","last_peered":"2024-03-26T08:40:11.152249+0000","last_clean":"2024-03-26T08:40:11.152249+0000","last_became_active":"2024-03-26T08:40:09.927152+0000","last_became_peered":"2024-03-26T08:40:09.927152+0000","last_unstale":"2024-03-26T08:40:11.152249+0000","last_undegraded":"2024-03-26T08:40:11.152249+0000","last_fullsized":"2024-03-26T08:40:11.152249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:44.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:15.283512+0000","last_change":"2024-03-26T08:40:09.224831+0000","last_active":"2024-03-26T08:40:15.283512+0000","last_peered":"2024-03-26T08:40:15.283512+0000","last_clean":"2024-03-26T08:40:15.283512+0000","last_became_active":"2024-03-26T08:40:09.224447+0000","last_became_peered":"2024-03-26T08:40:09.224447+0000","last_unstale":"2024-03-26T08:40:15.283512+0000","last_undegraded":"2024-03-26T08:40:15.283512+0000","last_fullsized":"2024-03-26T08:40:15.283512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:25:50.935342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":337,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280663+0000","last_change":"2024-03-26T08:38:54.696342+0000","last_active":"2024-03-26T08:40:11.280663+0000","last_peered":"2024-03-26T08:40:11.280663+0000","last_clean":"2024-03-26T08:40:11.280663+0000","last_became_active":"2024-03-26T08:38:54.695839+0000","last_became_peered":"2024-03-26T08:38:54.695839+0000","last_unstale":"2024-03-26T08:40:11.280663+0000","last_undegraded":"2024-03-26T08:40:11.280663+0000","last_fullsized":"2024-03-26T08:40:11.280663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_clean_scrub_stamp":"2024-03-26T08:35:27.189917+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:13.219778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922304+0000","last_change":"2024-03-26T08:40:09.930683+0000","last_active":"2024-03-26T08:40:10.922304+0000","last_peered":"2024-03-26T08:40:10.922304+0000","last_clean":"2024-03-26T08:40:10.922304+0000","last_became_active":"2024-03-26T08:40:09.930341+0000","last_became_peered":"2024-03-26T08:40:09.930341+0000","last_unstale":"2024-03-26T08:40:10.922304+0000","last_undegraded":"2024-03-26T08:40:10.922304+0000","last_fullsized":"2024-03-26T08:40:10.922304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:39:20.232007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.282626+0000","last_change":"2024-03-26T08:40:09.228548+0000","last_active":"2024-03-26T08:40:14.282626+0000","last_peered":"2024-03-26T08:40:14.282626+0000","last_clean":"2024-03-26T08:40:14.282626+0000","last_became_active":"2024-03-26T08:40:09.228076+0000","last_became_peered":"2024-03-26T08:40:09.228076+0000","last_unstale":"2024-03-26T08:40:14.282626+0000","last_undegraded":"2024-03-26T08:40:14.282626+0000","last_fullsized":"2024-03-26T08:40:14.282626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:52:51.641128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153279+0000","last_change":"2024-03-26T08:40:09.235632+0000","last_active":"2024-03-26T08:40:11.153279+0000","last_peered":"2024-03-26T08:40:11.153279+0000","last_clean":"2024-03-26T08:40:11.153279+0000","last_became_active":"2024-03-26T08:40:09.235334+0000","last_became_peered":"2024-03-26T08:40:09.235334+0000","last_unstale":"2024-03-26T08:40:11.153279+0000","last_undegraded":"2024-03-26T08:40:11.153279+0000","last_fullsized":"2024-03-26T08:40:11.153279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:38:27.064431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922721+0000","last_change":"2024-03-26T08:40:09.929137+0000","last_active":"2024-03-26T08:40:10.922721+0000","last_peered":"2024-03-26T08:40:10.922721+0000","last_clean":"2024-03-26T08:40:10.922721+0000","last_became_active":"2024-03-26T08:40:09.928831+0000","last_became_peered":"2024-03-26T08:40:09.928831+0000","last_unstale":"2024-03-26T08:40:10.922721+0000","last_undegraded":"2024-03-26T08:40:10.922721+0000","last_fullsized":"2024-03-26T08:40:10.922721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:19.235168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.282379+0000","last_change":"2024-03-26T08:40:09.227153+0000","last_active":"2024-03-26T08:40:14.282379+0000","last_peered":"2024-03-26T08:40:14.282379+0000","last_clean":"2024-03-26T08:40:14.282379+0000","last_became_active":"2024-03-26T08:40:09.226772+0000","last_became_peered":"2024-03-26T08:40:09.226772+0000","last_unstale":"2024-03-26T08:40:14.282379+0000","last_undegraded":"2024-03-26T08:40:14.282379+0000","last_fullsized":"2024-03-26T08:40:14.282379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:33.159538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152953+0000","last_change":"2024-03-26T08:40:09.934589+0000","last_active":"2024-03-26T08:40:11.152953+0000","last_peered":"2024-03-26T08:40:11.152953+0000","last_clean":"2024-03-26T08:40:11.152953+0000","last_became_active":"2024-03-26T08:40:09.934312+0000","last_became_peered":"2024-03-26T08:40:09.934312+0000","last_unstale":"2024-03-26T08:40:11.152953+0000","last_undegraded":"2024-03-26T08:40:11.152953+0000","last_fullsized":"2024-03-26T08:40:11.152953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:19:27.737506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.273413+0000","last_change":"2024-03-26T08:40:09.230014+0000","last_active":"2024-03-26T08:40:13.273413+0000","last_peered":"2024-03-26T08:40:13.273413+0000","last_clean":"2024-03-26T08:40:13.273413+0000","last_became_active":"2024-03-26T08:40:09.229682+0000","last_became_peered":"2024-03-26T08:40:09.229682+0000","last_unstale":"2024-03-26T08:40:13.273413+0000","last_undegraded":"2024-03-26T08:40:13.273413+0000","last_fullsized":"2024-03-26T08:40:13.273413+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:52.725810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094436+0000","last_change":"2024-03-26T08:40:09.925593+0000","last_active":"2024-03-26T08:40:11.094436+0000","last_peered":"2024-03-26T08:40:11.094436+0000","last_clean":"2024-03-26T08:40:11.094436+0000","last_became_active":"2024-03-26T08:40:09.925334+0000","last_became_peered":"2024-03-26T08:40:09.925334+0000","last_unstale":"2024-03-26T08:40:11.094436+0000","last_undegraded":"2024-03-26T08:40:11.094436+0000","last_fullsized":"2024-03-26T08:40:11.094436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:25:12.110345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.274316+0000","last_change":"2024-03-26T08:40:09.226634+0000","last_active":"2024-03-26T08:40:13.274316+0000","last_peered":"2024-03-26T08:40:13.274316+0000","last_clean":"2024-03-26T08:40:13.274316+0000","last_became_active":"2024-03-26T08:40:09.226248+0000","last_became_peered":"2024-03-26T08:40:09.226248+0000","last_unstale":"2024-03-26T08:40:13.274316+0000","last_undegraded":"2024-03-26T08:40:13.274316+0000","last_fullsized":"2024-03-26T08:40:13.274316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:24:00.390573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152278+0000","last_change":"2024-03-26T08:40:09.934848+0000","last_active":"2024-03-26T08:40:11.152278+0000","last_peered":"2024-03-26T08:40:11.152278+0000","last_clean":"2024-03-26T08:40:11.152278+0000","last_became_active":"2024-03-26T08:40:09.934521+0000","last_became_peered":"2024-03-26T08:40:09.934521+0000","last_unstale":"2024-03-26T08:40:11.152278+0000","last_undegraded":"2024-03-26T08:40:11.152278+0000","last_fullsized":"2024-03-26T08:40:11.152278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:24:25.994961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.272785+0000","last_change":"2024-03-26T08:40:09.222945+0000","last_active":"2024-03-26T08:40:13.272785+0000","last_peered":"2024-03-26T08:40:13.272785+0000","last_clean":"2024-03-26T08:40:13.272785+0000","last_became_active":"2024-03-26T08:40:09.222564+0000","last_became_peered":"2024-03-26T08:40:09.222564+0000","last_unstale":"2024-03-26T08:40:13.272785+0000","last_undegraded":"2024-03-26T08:40:13.272785+0000","last_fullsized":"2024-03-26T08:40:13.272785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:35:11.181812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152602+0000","last_change":"2024-03-26T08:40:09.932674+0000","last_active":"2024-03-26T08:40:11.152602+0000","last_peered":"2024-03-26T08:40:11.152602+0000","last_clean":"2024-03-26T08:40:11.152602+0000","last_became_active":"2024-03-26T08:40:09.932346+0000","last_became_peered":"2024-03-26T08:40:09.932346+0000","last_unstale":"2024-03-26T08:40:11.152602+0000","last_undegraded":"2024-03-26T08:40:11.152602+0000","last_fullsized":"2024-03-26T08:40:11.152602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:12:45.871467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.212432+0000","last_change":"2024-03-26T08:40:09.228605+0000","last_active":"2024-03-26T08:40:11.212432+0000","last_peered":"2024-03-26T08:40:11.212432+0000","last_clean":"2024-03-26T08:40:11.212432+0000","last_became_active":"2024-03-26T08:40:09.228233+0000","last_became_peered":"2024-03-26T08:40:09.228233+0000","last_unstale":"2024-03-26T08:40:11.212432+0000","last_undegraded":"2024-03-26T08:40:11.212432+0000","last_fullsized":"2024-03-26T08:40:11.212432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:32:25.977866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152347+0000","last_change":"2024-03-26T08:40:09.922766+0000","last_active":"2024-03-26T08:40:11.152347+0000","last_peered":"2024-03-26T08:40:11.152347+0000","last_clean":"2024-03-26T08:40:11.152347+0000","last_became_active":"2024-03-26T08:40:09.922453+0000","last_became_peered":"2024-03-26T08:40:09.922453+0000","last_unstale":"2024-03-26T08:40:11.152347+0000","last_undegraded":"2024-03-26T08:40:11.152347+0000","last_fullsized":"2024-03-26T08:40:11.152347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:13:28.740291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:13.274801+0000","last_change":"2024-03-26T08:40:09.228114+0000","last_active":"2024-03-26T08:40:13.274801+0000","last_peered":"2024-03-26T08:40:13.274801+0000","last_clean":"2024-03-26T08:40:13.274801+0000","last_became_active":"2024-03-26T08:40:09.227803+0000","last_became_peered":"2024-03-26T08:40:09.227803+0000","last_unstale":"2024-03-26T08:40:13.274801+0000","last_undegraded":"2024-03-26T08:40:13.274801+0000","last_fullsized":"2024-03-26T08:40:13.274801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:31.717981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.280945+0000","last_change":"2024-03-26T08:40:09.929671+0000","last_active":"2024-03-26T08:40:11.280945+0000","last_peered":"2024-03-26T08:40:11.280945+0000","last_clean":"2024-03-26T08:40:11.280945+0000","last_became_active":"2024-03-26T08:40:09.929222+0000","last_became_peered":"2024-03-26T08:40:09.929222+0000","last_unstale":"2024-03-26T08:40:11.280945+0000","last_undegraded":"2024-03-26T08:40:11.280945+0000","last_fullsized":"2024-03-26T08:40:11.280945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:00:50.579569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:14.282891+0000","last_change":"2024-03-26T08:40:09.228994+0000","last_active":"2024-03-26T08:40:14.282891+0000","last_peered":"2024-03-26T08:40:14.282891+0000","last_clean":"2024-03-26T08:40:14.282891+0000","last_became_active":"2024-03-26T08:40:09.227138+0000","last_became_peered":"2024-03-26T08:40:09.227138+0000","last_unstale":"2024-03-26T08:40:14.282891+0000","last_undegraded":"2024-03-26T08:40:14.282891+0000","last_fullsized":"2024-03-26T08:40:14.282891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:06.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281218+0000","last_change":"2024-03-26T08:40:09.931752+0000","last_active":"2024-03-26T08:40:11.281218+0000","last_peered":"2024-03-26T08:40:11.281218+0000","last_clean":"2024-03-26T08:40:11.281218+0000","last_became_active":"2024-03-26T08:40:09.931436+0000","last_became_peered":"2024-03-26T08:40:09.931436+0000","last_unstale":"2024-03-26T08:40:11.281218+0000","last_undegraded":"2024-03-26T08:40:11.281218+0000","last_fullsized":"2024-03-26T08:40:11.281218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:40:54.864614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152958+0000","last_change":"2024-03-26T08:40:09.238356+0000","last_active":"2024-03-26T08:40:11.152958+0000","last_peered":"2024-03-26T08:40:11.152958+0000","last_clean":"2024-03-26T08:40:11.152958+0000","last_became_active":"2024-03-26T08:40:09.238066+0000","last_became_peered":"2024-03-26T08:40:09.238066+0000","last_unstale":"2024-03-26T08:40:11.152958+0000","last_undegraded":"2024-03-26T08:40:11.152958+0000","last_fullsized":"2024-03-26T08:40:11.152958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:13.851490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:12.678183+0000","last_change":"2024-03-26T08:40:09.933223+0000","last_active":"2024-03-26T08:40:12.678183+0000","last_peered":"2024-03-26T08:40:12.678183+0000","last_clean":"2024-03-26T08:40:12.678183+0000","last_became_active":"2024-03-26T08:40:09.932903+0000","last_became_peered":"2024-03-26T08:40:09.932903+0000","last_unstale":"2024-03-26T08:40:12.678183+0000","last_undegraded":"2024-03-26T08:40:12.678183+0000","last_fullsized":"2024-03-26T08:40:12.678183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:02:16.079139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922655+0000","last_change":"2024-03-26T08:40:09.238009+0000","last_active":"2024-03-26T08:40:10.922655+0000","last_peered":"2024-03-26T08:40:10.922655+0000","last_clean":"2024-03-26T08:40:10.922655+0000","last_became_active":"2024-03-26T08:40:09.237697+0000","last_became_peered":"2024-03-26T08:40:09.237697+0000","last_unstale":"2024-03-26T08:40:10.922655+0000","last_undegraded":"2024-03-26T08:40:10.922655+0000","last_fullsized":"2024-03-26T08:40:10.922655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:17:33.179285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.281229+0000","last_change":"2024-03-26T08:40:09.934672+0000","last_active":"2024-03-26T08:40:11.281229+0000","last_peered":"2024-03-26T08:40:11.281229+0000","last_clean":"2024-03-26T08:40:11.281229+0000","last_became_active":"2024-03-26T08:40:09.934365+0000","last_became_peered":"2024-03-26T08:40:09.934365+0000","last_unstale":"2024-03-26T08:40:11.281229+0000","last_undegraded":"2024-03-26T08:40:11.281229+0000","last_fullsized":"2024-03-26T08:40:11.281229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:41:22.318481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152766+0000","last_change":"2024-03-26T08:40:09.933910+0000","last_active":"2024-03-26T08:40:11.152766+0000","last_peered":"2024-03-26T08:40:11.152766+0000","last_clean":"2024-03-26T08:40:11.152766+0000","last_became_active":"2024-03-26T08:40:09.933598+0000","last_became_peered":"2024-03-26T08:40:09.933598+0000","last_unstale":"2024-03-26T08:40:11.152766+0000","last_undegraded":"2024-03-26T08:40:11.152766+0000","last_fullsized":"2024-03-26T08:40:11.152766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:19.877984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923662+0000","last_change":"2024-03-26T08:40:09.935809+0000","last_active":"2024-03-26T08:40:10.923662+0000","last_peered":"2024-03-26T08:40:10.923662+0000","last_clean":"2024-03-26T08:40:10.923662+0000","last_became_active":"2024-03-26T08:40:09.935500+0000","last_became_peered":"2024-03-26T08:40:09.935500+0000","last_unstale":"2024-03-26T08:40:10.923662+0000","last_undegraded":"2024-03-26T08:40:10.923662+0000","last_fullsized":"2024-03-26T08:40:10.923662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:23.717845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.923609+0000","last_change":"2024-03-26T08:40:09.928093+0000","last_active":"2024-03-26T08:40:10.923609+0000","last_peered":"2024-03-26T08:40:10.923609+0000","last_clean":"2024-03-26T08:40:10.923609+0000","last_became_active":"2024-03-26T08:40:09.927732+0000","last_became_peered":"2024-03-26T08:40:09.927732+0000","last_unstale":"2024-03-26T08:40:10.923609+0000","last_undegraded":"2024-03-26T08:40:10.923609+0000","last_fullsized":"2024-03-26T08:40:10.923609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:35:34.972435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.094249+0000","last_change":"2024-03-26T08:40:09.925439+0000","last_active":"2024-03-26T08:40:11.094249+0000","last_peered":"2024-03-26T08:40:11.094249+0000","last_clean":"2024-03-26T08:40:11.094249+0000","last_became_active":"2024-03-26T08:40:09.925123+0000","last_became_peered":"2024-03-26T08:40:09.925123+0000","last_unstale":"2024-03-26T08:40:11.094249+0000","last_undegraded":"2024-03-26T08:40:11.094249+0000","last_fullsized":"2024-03-26T08:40:11.094249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:02:17.934610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922648+0000","last_change":"2024-03-26T08:40:09.919466+0000","last_active":"2024-03-26T08:40:10.922648+0000","last_peered":"2024-03-26T08:40:10.922648+0000","last_clean":"2024-03-26T08:40:10.922648+0000","last_became_active":"2024-03-26T08:40:09.919152+0000","last_became_peered":"2024-03-26T08:40:09.919152+0000","last_unstale":"2024-03-26T08:40:10.922648+0000","last_undegraded":"2024-03-26T08:40:10.922648+0000","last_fullsized":"2024-03-26T08:40:10.922648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:37.928485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153433+0000","last_change":"2024-03-26T08:40:09.927140+0000","last_active":"2024-03-26T08:40:11.153433+0000","last_peered":"2024-03-26T08:40:11.153433+0000","last_clean":"2024-03-26T08:40:11.153433+0000","last_became_active":"2024-03-26T08:40:09.926823+0000","last_became_peered":"2024-03-26T08:40:09.926823+0000","last_unstale":"2024-03-26T08:40:11.153433+0000","last_undegraded":"2024-03-26T08:40:11.153433+0000","last_fullsized":"2024-03-26T08:40:11.153433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:29:38.477123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.924081+0000","last_change":"2024-03-26T08:40:09.934202+0000","last_active":"2024-03-26T08:40:10.924081+0000","last_peered":"2024-03-26T08:40:10.924081+0000","last_clean":"2024-03-26T08:40:10.924081+0000","last_became_active":"2024-03-26T08:40:09.933878+0000","last_became_peered":"2024-03-26T08:40:09.933878+0000","last_unstale":"2024-03-26T08:40:10.924081+0000","last_undegraded":"2024-03-26T08:40:10.924081+0000","last_fullsized":"2024-03-26T08:40:10.924081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:44:26.576140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152943+0000","last_change":"2024-03-26T08:40:09.929019+0000","last_active":"2024-03-26T08:40:11.152943+0000","last_peered":"2024-03-26T08:40:11.152943+0000","last_clean":"2024-03-26T08:40:11.152943+0000","last_became_active":"2024-03-26T08:40:09.928711+0000","last_became_peered":"2024-03-26T08:40:09.928711+0000","last_unstale":"2024-03-26T08:40:11.152943+0000","last_undegraded":"2024-03-26T08:40:11.152943+0000","last_fullsized":"2024-03-26T08:40:11.152943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:47.120826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152040+0000","last_change":"2024-03-26T08:40:09.927606+0000","last_active":"2024-03-26T08:40:11.152040+0000","last_peered":"2024-03-26T08:40:11.152040+0000","last_clean":"2024-03-26T08:40:11.152040+0000","last_became_active":"2024-03-26T08:40:09.926717+0000","last_became_peered":"2024-03-26T08:40:09.926717+0000","last_unstale":"2024-03-26T08:40:11.152040+0000","last_undegraded":"2024-03-26T08:40:11.152040+0000","last_fullsized":"2024-03-26T08:40:11.152040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:51:52.567252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.152160+0000","last_change":"2024-03-26T08:40:09.934659+0000","last_active":"2024-03-26T08:40:11.152160+0000","last_peered":"2024-03-26T08:40:11.152160+0000","last_clean":"2024-03-26T08:40:11.152160+0000","last_became_active":"2024-03-26T08:40:09.934346+0000","last_became_peered":"2024-03-26T08:40:09.934346+0000","last_unstale":"2024-03-26T08:40:11.152160+0000","last_undegraded":"2024-03-26T08:40:11.152160+0000","last_fullsized":"2024-03-26T08:40:11.152160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:35.009264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922950+0000","last_change":"2024-03-26T08:40:09.925007+0000","last_active":"2024-03-26T08:40:10.922950+0000","last_peered":"2024-03-26T08:40:10.922950+0000","last_clean":"2024-03-26T08:40:10.922950+0000","last_became_active":"2024-03-26T08:40:09.923439+0000","last_became_peered":"2024-03-26T08:40:09.923439+0000","last_unstale":"2024-03-26T08:40:10.922950+0000","last_undegraded":"2024-03-26T08:40:10.922950+0000","last_fullsized":"2024-03-26T08:40:10.922950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:18.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:10.922546+0000","last_change":"2024-03-26T08:40:09.920795+0000","last_active":"2024-03-26T08:40:10.922546+0000","last_peered":"2024-03-26T08:40:10.922546+0000","last_clean":"2024-03-26T08:40:10.922546+0000","last_became_active":"2024-03-26T08:40:09.920378+0000","last_became_peered":"2024-03-26T08:40:09.920378+0000","last_unstale":"2024-03-26T08:40:10.922546+0000","last_undegraded":"2024-03-26T08:40:10.922546+0000","last_fullsized":"2024-03-26T08:40:10.922546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:19:04.027727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-26T08:40:11.153339+0000","last_change":"2024-03-26T08:40:09.929426+0000","last_active":"2024-03-26T08:40:11.153339+0000","last_peered":"2024-03-26T08:40:11.153339+0000","last_clean":"2024-03-26T08:40:11.153339+0000","last_became_active":"2024-03-26T08:40:09.927267+0000","last_became_peered":"2024-03-26T08:40:09.927267+0000","last_unstale":"2024-03-26T08:40:11.153339+0000","last_undegraded":"2024-03-26T08:40:11.153339+0000","last_fullsized":"2024-03-26T08:40:11.153339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:58:55.064053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282339,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":12101,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":22,"num_read_kb":18,"num_write":121,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":90318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383693,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1032079,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":439339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":439147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":1031189,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":438728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":465974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":448474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":1030756,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":44128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":29194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":9790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:40:18.441 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-26T08:40:18.441 INFO:tasks.cephadm:Setting up client nodes... 2024-03-26T08:40:18.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-26T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:18 smithi044 ceph-mon[36461]: pgmap v294: 97 pgs: 97 active+clean; 588 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2024-03-26T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:18 smithi044 ceph-mon[36461]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:18 smithi044 ceph-mon[36461]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:18 smithi044 ceph-mon[36461]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:18 smithi044 ceph-mon[36461]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T08:40:18.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:18 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/211303810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.567 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:18 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3119901134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:18 smithi016 ceph-mon[35851]: pgmap v294: 97 pgs: 97 active+clean; 588 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2024-03-26T08:40:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:18 smithi016 ceph-mon[35851]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:18 smithi016 ceph-mon[35851]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:18 smithi016 ceph-mon[35851]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:40:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:18 smithi016 ceph-mon[35851]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-26T08:40:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:18 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/211303810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:40:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:18 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3119901134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:19.538 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:19 smithi007 ceph-mon[32462]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:40:19.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:19 smithi044 ceph-mon[36461]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:40:19.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:19 smithi016 ceph-mon[35851]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:40:20.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:20 smithi007 ceph-mon[32462]: pgmap v295: 97 pgs: 97 active+clean; 588 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.8 KiB/s wr, 17 op/s 2024-03-26T08:40:20.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:20 smithi007 ceph-mon[32462]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:40:20.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.404+0000 7f875e9dc700 1 -- 172.21.15.7:0/3776513028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 msgr2=0x7f8758102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.404+0000 7f875e9dc700 1 --2- 172.21.15.7:0/3776513028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 0x7f8758102770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f874c0099c0 tx=0x7f874c009cd0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.404+0000 7f875e9dc700 1 -- 172.21.15.7:0/3776513028 shutdown_connections 2024-03-26T08:40:20.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.404+0000 7f875e9dc700 1 --2- 172.21.15.7:0/3776513028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8758102cb0 0x7f8758109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.404+0000 7f875e9dc700 1 --2- 172.21.15.7:0/3776513028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8758101110 0x7f8758101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.404+0000 7f875e9dc700 1 --2- 172.21.15.7:0/3776513028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 0x7f8758102770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.404+0000 7f875e9dc700 1 -- 172.21.15.7:0/3776513028 >> 172.21.15.7:0/3776513028 conn(0x7f87580fc6b0 msgr2=0x7f87580feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:20.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.404+0000 7f875e9dc700 1 -- 172.21.15.7:0/3776513028 shutdown_connections 2024-03-26T08:40:20.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.404+0000 7f875e9dc700 1 -- 172.21.15.7:0/3776513028 wait complete. 2024-03-26T08:40:20.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f875e9dc700 1 Processor -- start 2024-03-26T08:40:20.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f875e9dc700 1 -- start start 2024-03-26T08:40:20.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f875e9dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8758101110 0x7f8758105050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:20.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f875e9dc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 0x7f87581036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:20.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f875e9dc700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8758102cb0 0x7f8758103c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f875e9dc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875810cd00 con 0x7f8758102310 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f875e9dc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875810cb80 con 0x7f8758101110 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f875e9dc700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875810ce80 con 0x7f8758102cb0 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f87577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 0x7f87581036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f87577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 0x7f87581036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49732/0 (socket says 172.21.15.7:49732) 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f8757fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8758101110 0x7f8758105050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f87577fe700 1 -- 172.21.15.7:0/4260654794 learned_addr learned my addr 172.21.15.7:0/4260654794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.405+0000 7f875cf79700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8758102cb0 0x7f8758103c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f87577fe700 1 -- 172.21.15.7:0/4260654794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8758102cb0 msgr2=0x7f8758103c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f87577fe700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8758102cb0 0x7f8758103c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f87577fe700 1 -- 172.21.15.7:0/4260654794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8758101110 msgr2=0x7f8758105050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f87577fe700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8758101110 0x7f8758105050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f87577fe700 1 -- 172.21.15.7:0/4260654794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758104500 con 0x7f8758102310 2024-03-26T08:40:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f8757fff700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8758101110 0x7f8758105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f875cf79700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8758102cb0 0x7f8758103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f87577fe700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 0x7f87581036f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f874c003790 tx=0x7f874c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f87557fa700 1 -- 172.21.15.7:0/4260654794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c009e60 con 0x7f8758102310 2024-03-26T08:40:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f87557fa700 1 -- 172.21.15.7:0/4260654794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f874c003960 con 0x7f8758102310 2024-03-26T08:40:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87581a2400 con 0x7f8758102310 2024-03-26T08:40:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f87557fa700 1 -- 172.21.15.7:0/4260654794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c0173f0 con 0x7f8758102310 2024-03-26T08:40:20.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.406+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87581a28b0 con 0x7f8758102310 2024-03-26T08:40:20.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.407+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87580506a0 con 0x7f8758102310 2024-03-26T08:40:20.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.408+0000 7f87557fa700 1 -- 172.21.15.7:0/4260654794 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f874c003b10 con 0x7f8758102310 2024-03-26T08:40:20.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.409+0000 7f87557fa700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87400a29d0 0x7f87400a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:20.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.409+0000 7f8757fff700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87400a29d0 0x7f87400a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:20.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.409+0000 7f87557fa700 1 -- 172.21.15.7:0/4260654794 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f874c0cf7e0 con 0x7f8758102310 2024-03-26T08:40:20.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.409+0000 7f8757fff700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87400a29d0 0x7f87400a4e90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8758102170 tx=0x7f8748009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:20.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.413+0000 7f87557fa700 1 -- 172.21.15.7:0/4260654794 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f874c013070 con 0x7f8758102310 2024-03-26T08:40:20.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:20 smithi044 ceph-mon[36461]: pgmap v295: 97 pgs: 97 active+clean; 588 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.8 KiB/s wr, 17 op/s 2024-03-26T08:40:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:20 smithi044 ceph-mon[36461]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:40:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.701+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f87581a2cf0 con 0x7f8758102310 2024-03-26T08:40:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.703+0000 7f87557fa700 1 -- 172.21.15.7:0/4260654794 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f87581a2cf0 con 0x7f8758102310 2024-03-26T08:40:20.695 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-03-26T08:40:20.695 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQD0iQJmZIvjKRAA1bIJdeWZNQxgixSuH1q5UA== 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87400a29d0 msgr2=0x7f87400a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87400a29d0 0x7f87400a4e90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8758102170 tx=0x7f8748009260 comp rx=0 tx=0).stop 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 msgr2=0x7f87581036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 0x7f87581036f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f874c003790 tx=0x7f874c00f690 comp rx=0 tx=0).stop 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 shutdown_connections 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87400a29d0 0x7f87400a4e90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8758102cb0 0x7f8758103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8758101110 0x7f8758105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 --2- 172.21.15.7:0/4260654794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8758102310 0x7f87581036f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 >> 172.21.15.7:0/4260654794 conn(0x7f87580fc6b0 msgr2=0x7f87580fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:20.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 shutdown_connections 2024-03-26T08:40:20.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:20.705+0000 7f875e9dc700 1 -- 172.21.15.7:0/4260654794 wait complete. 2024-03-26T08:40:20.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:20 smithi016 ceph-mon[35851]: pgmap v295: 97 pgs: 97 active+clean; 588 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.8 KiB/s wr, 17 op/s 2024-03-26T08:40:20.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:20 smithi016 ceph-mon[35851]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:40:21.317 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:40:21.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-26T08:40:21.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-26T08:40:21.359 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-26T08:40:21.360 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-26T08:40:21.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-26T08:40:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:21 smithi007 ceph-mon[32462]: pgmap v296: 97 pgs: 97 active+clean; 589 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-03-26T08:40:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:21 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4260654794' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T08:40:21.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:21 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4260654794' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T08:40:21.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:21 smithi044 ceph-mon[36461]: pgmap v296: 97 pgs: 97 active+clean; 589 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-03-26T08:40:21.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:21 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4260654794' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T08:40:21.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:21 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4260654794' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T08:40:21.640 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:21.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:21 smithi016 ceph-mon[35851]: pgmap v296: 97 pgs: 97 active+clean; 589 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-03-26T08:40:21.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:21 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4260654794' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T08:40:21.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:21 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4260654794' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.300+0000 7f0c35eff700 1 -- 172.21.15.7:0/612174626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c30107510 msgr2=0x7f0c3010f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.300+0000 7f0c35eff700 1 --2- 172.21.15.7:0/612174626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c30107510 0x7f0c3010f090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800b0d0 tx=0x7f0c2800b3e0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.300+0000 7f0c35eff700 1 -- 172.21.15.7:0/612174626 shutdown_connections 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.300+0000 7f0c35eff700 1 --2- 172.21.15.7:0/612174626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c3006a930 0x7f0c3006ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.300+0000 7f0c35eff700 1 --2- 172.21.15.7:0/612174626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c30107510 0x7f0c3010f090 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.300+0000 7f0c35eff700 1 --2- 172.21.15.7:0/612174626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3006b2e0 0x7f0c30106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.300+0000 7f0c35eff700 1 -- 172.21.15.7:0/612174626 >> 172.21.15.7:0/612174626 conn(0x7f0c300770e0 msgr2=0x7f0c300774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.300+0000 7f0c35eff700 1 -- 172.21.15.7:0/612174626 shutdown_connections 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.300+0000 7f0c35eff700 1 -- 172.21.15.7:0/612174626 wait complete. 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c35eff700 1 Processor -- start 2024-03-26T08:40:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c35eff700 1 -- start start 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c35eff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3006a930 0x7f0c3019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c35eff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3006b2e0 0x7f0c3019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c35eff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c30107510 0x7f0c30198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c35eff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c301124d0 con 0x7f0c3006b2e0 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c35eff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c30112350 con 0x7f0c3006a930 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c35eff700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c30112650 con 0x7f0c30107510 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3006b2e0 0x7f0c3019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3006b2e0 0x7f0c3019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49754/0 (socket says 172.21.15.7:49754) 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c2effd700 1 -- 172.21.15.7:0/4252394572 learned_addr learned my addr 172.21.15.7:0/4252394572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c2f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3006a930 0x7f0c3019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.301+0000 7f0c2ffff700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c30107510 0x7f0c30198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c2effd700 1 -- 172.21.15.7:0/4252394572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c30107510 msgr2=0x7f0c30198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c2effd700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c30107510 0x7f0c30198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c2effd700 1 -- 172.21.15.7:0/4252394572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3006a930 msgr2=0x7f0c3019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c2effd700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3006a930 0x7f0c3019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c2effd700 1 -- 172.21.15.7:0/4252394572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c30199010 con 0x7f0c3006b2e0 2024-03-26T08:40:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c2effd700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3006b2e0 0x7f0c3019eba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e940 tx=0x7f0c2400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c2cff9700 1 -- 172.21.15.7:0/4252394572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400c9e0 con 0x7f0c3006b2e0 2024-03-26T08:40:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c2cff9700 1 -- 172.21.15.7:0/4252394572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c2400cb80 con 0x7f0c3006b2e0 2024-03-26T08:40:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c2cff9700 1 -- 172.21.15.7:0/4252394572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c24018660 con 0x7f0c3006b2e0 2024-03-26T08:40:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c30199300 con 0x7f0c3006b2e0 2024-03-26T08:40:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.302+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c301a55e0 con 0x7f0c3006b2e0 2024-03-26T08:40:23.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.303+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3010a6c0 con 0x7f0c3006b2e0 2024-03-26T08:40:23.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.304+0000 7f0c2cff9700 1 -- 172.21.15.7:0/4252394572 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f0c24010430 con 0x7f0c3006b2e0 2024-03-26T08:40:23.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.305+0000 7f0c2cff9700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c180a2920 0x7f0c180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:23.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.305+0000 7f0c2f7fe700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c180a2920 0x7f0c180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:23.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.305+0000 7f0c2cff9700 1 -- 172.21.15.7:0/4252394572 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0c240cec30 con 0x7f0c3006b2e0 2024-03-26T08:40:23.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.305+0000 7f0c2f7fe700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c180a2920 0x7f0c180a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009c60 tx=0x7f0c20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:23.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.309+0000 7f0c2cff9700 1 -- 172.21.15.7:0/4252394572 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0c2409bc20 con 0x7f0c3006b2e0 2024-03-26T08:40:23.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:23 smithi016 ceph-mon[35851]: pgmap v297: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.3 KiB/s wr, 17 op/s 2024-03-26T08:40:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.577+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0c30068e80 con 0x7f0c3006b2e0 2024-03-26T08:40:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.582+0000 7f0c2cff9700 1 -- 172.21.15.7:0/4252394572 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262652 (secure 0 0 0) 0x7f0c30068e80 con 0x7f0c3006b2e0 2024-03-26T08:40:23.577 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c180a2920 msgr2=0x7f0c180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c180a2920 0x7f0c180a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009c60 tx=0x7f0c20009260 comp rx=0 tx=0).stop 2024-03-26T08:40:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3006b2e0 msgr2=0x7f0c3019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3006b2e0 0x7f0c3019eba0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400e940 tx=0x7f0c2400ec50 comp rx=0 tx=0).stop 2024-03-26T08:40:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 shutdown_connections 2024-03-26T08:40:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c180a2920 0x7f0c180a4de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c30107510 0x7f0c30198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3006a930 0x7f0c3019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 --2- 172.21.15.7:0/4252394572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c3006b2e0 0x7f0c3019eba0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.585+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 >> 172.21.15.7:0/4252394572 conn(0x7f0c300770e0 msgr2=0x7f0c300ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.586+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 shutdown_connections 2024-03-26T08:40:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:23.586+0000 7f0c35eff700 1 -- 172.21.15.7:0/4252394572 wait complete. 2024-03-26T08:40:23.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:23 smithi007 ceph-mon[32462]: pgmap v297: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.3 KiB/s wr, 17 op/s 2024-03-26T08:40:23.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:23 smithi044 ceph-mon[36461]: pgmap v297: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.3 KiB/s wr, 17 op/s 2024-03-26T08:40:24.394 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1233671832},{"type":"v1","addr":"172.21.15.7:6801","nonce":1233671832}]},"active_addr":"172.21.15.7:6801/1233671832","active_change":"2024-03-26T08:32:58.217914+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1348143777}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2409591608}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2353302922}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3588192525}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2458618019}]}]} 2024-03-26T08:40:24.407 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-26T08:40:24.407 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-26T08:40:24.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T08:40:24.660 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:24.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:24 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4252394572' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T08:40:24.781 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:24 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4252394572' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T08:40:24.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:24 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4252394572' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T08:40:25.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:25 smithi016 ceph-mon[35851]: pgmap v298: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-26T08:40:25.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:25 smithi007 ceph-mon[32462]: pgmap v298: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-26T08:40:25.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:25 smithi044 ceph-mon[36461]: pgmap v298: 97 pgs: 97 active+clean; 633 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-03-26T08:40:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.351+0000 7fb0d5d12700 1 -- 172.21.15.7:0/4294546083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 msgr2=0x7fb0d010ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.351+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/4294546083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 0x7fb0d010ef90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800b0a0 tx=0x7fb0c800b3b0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.351+0000 7fb0d5d12700 1 -- 172.21.15.7:0/4294546083 shutdown_connections 2024-03-26T08:40:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.351+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/4294546083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d01082e0 0x7fb0d01086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.351+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/4294546083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 0x7fb0d010ef90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.351+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/4294546083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d006a910 0x7fb0d006ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.351+0000 7fb0d5d12700 1 -- 172.21.15.7:0/4294546083 >> 172.21.15.7:0/4294546083 conn(0x7fb0d0078190 msgr2=0x7fb0d00785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.351+0000 7fb0d5d12700 1 -- 172.21.15.7:0/4294546083 shutdown_connections 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.351+0000 7fb0d5d12700 1 -- 172.21.15.7:0/4294546083 wait complete. 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0d5d12700 1 Processor -- start 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0d5d12700 1 -- start start 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0d5d12700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d006a910 0x7fb0d019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0d5d12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 0x7fb0d019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0d5d12700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d01082e0 0x7fb0d0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0d5d12700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d01124a0 con 0x7fb0d006b2d0 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0d5d12700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0112320 con 0x7fb0d006a910 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0d5d12700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0112620 con 0x7fb0d01082e0 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 0x7fb0d019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 0x7fb0d019ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49774/0 (socket says 172.21.15.7:49774) 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0ceffd700 1 -- 172.21.15.7:0/843001314 learned_addr learned my addr 172.21.15.7:0/843001314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0cf7fe700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d006a910 0x7fb0d019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0cffff700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d01082e0 0x7fb0d0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0ceffd700 1 -- 172.21.15.7:0/843001314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d01082e0 msgr2=0x7fb0d0198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.352+0000 7fb0ceffd700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d01082e0 0x7fb0d0198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0ceffd700 1 -- 172.21.15.7:0/843001314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d006a910 msgr2=0x7fb0d019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0ceffd700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d006a910 0x7fb0d019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0ceffd700 1 -- 172.21.15.7:0/843001314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0198f40 con 0x7fb0d006b2d0 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0cf7fe700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d006a910 0x7fb0d019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0ceffd700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 0x7fb0d019ead0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400e9a0 tx=0x7fb0c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0ccff9700 1 -- 172.21.15.7:0/843001314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400ca10 con 0x7fb0d006b2d0 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d0199230 con 0x7fb0d006b2d0 2024-03-26T08:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0ccff9700 1 -- 172.21.15.7:0/843001314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0c400cbb0 con 0x7fb0d006b2d0 2024-03-26T08:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0ccff9700 1 -- 172.21.15.7:0/843001314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4018680 con 0x7fb0d006b2d0 2024-03-26T08:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.353+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d01a5510 con 0x7fb0d006b2d0 2024-03-26T08:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.354+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d0103890 con 0x7fb0d006b2d0 2024-03-26T08:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.356+0000 7fb0ccff9700 1 -- 172.21.15.7:0/843001314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fb0c4010430 con 0x7fb0d006b2d0 2024-03-26T08:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.356+0000 7fb0ccff9700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b80a2920 0x7fb0b80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.356+0000 7fb0cf7fe700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b80a2920 0x7fb0b80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.356+0000 7fb0ccff9700 1 -- 172.21.15.7:0/843001314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb0c40cf0f0 con 0x7fb0d006b2d0 2024-03-26T08:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.357+0000 7fb0cf7fe700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b80a2920 0x7fb0b80a4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0011490 tx=0x7fb0c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.360+0000 7fb0ccff9700 1 -- 172.21.15.7:0/843001314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb0c409c1e0 con 0x7fb0d006b2d0 2024-03-26T08:40:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.565+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0d0199b90 con 0x7fb0d006b2d0 2024-03-26T08:40:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.567+0000 7fb0ccff9700 1 -- 172.21.15.7:0/843001314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22089 (secure 0 0 0) 0x7fb0c409bae0 con 0x7fb0d006b2d0 2024-03-26T08:40:26.559 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:26.559 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:10.919648+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b80a2920 msgr2=0x7fb0b80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b80a2920 0x7fb0b80a4de0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0011490 tx=0x7fb0c0009260 comp rx=0 tx=0).stop 2024-03-26T08:40:26.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 msgr2=0x7fb0d019ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 0x7fb0d019ead0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400e9a0 tx=0x7fb0c400ecb0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 shutdown_connections 2024-03-26T08:40:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b80a2920 0x7fb0b80a4de0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0d01082e0 0x7fb0d0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0d006a910 0x7fb0d019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 --2- 172.21.15.7:0/843001314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d006b2d0 0x7fb0d019ead0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 >> 172.21.15.7:0/843001314 conn(0x7fb0d0078190 msgr2=0x7fb0d00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 shutdown_connections 2024-03-26T08:40:26.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:26.569+0000 7fb0d5d12700 1 -- 172.21.15.7:0/843001314 wait complete. 2024-03-26T08:40:27.375 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-26T08:40:27.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T08:40:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:27.741 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:27 smithi007 ceph-mon[32462]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.6 KiB/s wr, 15 op/s 2024-03-26T08:40:27.741 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:27 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/843001314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:27.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:27 smithi016 ceph-mon[35851]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.6 KiB/s wr, 15 op/s 2024-03-26T08:40:27.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:27 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/843001314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:27.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:27 smithi044 ceph-mon[36461]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.6 KiB/s wr, 15 op/s 2024-03-26T08:40:27.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:27 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/843001314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.387+0000 7f77a8db3700 1 -- 172.21.15.7:0/1969423308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 msgr2=0x7f77a41064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.387+0000 7f77a8db3700 1 --2- 172.21.15.7:0/1969423308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 0x7f77a41064a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f77940099c0 tx=0x7f7794009cd0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.387+0000 7f77a8db3700 1 -- 172.21.15.7:0/1969423308 shutdown_connections 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.387+0000 7f77a8db3700 1 --2- 172.21.15.7:0/1969423308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102600 0x7f77a410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.387+0000 7f77a8db3700 1 --2- 172.21.15.7:0/1969423308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 0x7f77a41064a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.387+0000 7f77a8db3700 1 --2- 172.21.15.7:0/1969423308 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a4101c40 0x7f77a41020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.387+0000 7f77a8db3700 1 -- 172.21.15.7:0/1969423308 >> 172.21.15.7:0/1969423308 conn(0x7f77a4076410 msgr2=0x7f77a4076820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.387+0000 7f77a8db3700 1 -- 172.21.15.7:0/1969423308 shutdown_connections 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.387+0000 7f77a8db3700 1 -- 172.21.15.7:0/1969423308 wait complete. 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a8db3700 1 Processor -- start 2024-03-26T08:40:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a8db3700 1 -- start start 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a8db3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a4101c40 0x7f77a419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a8db3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102600 0x7f77a419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a8db3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 0x7f77a4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a8db3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a41113e0 con 0x7f77a41060c0 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a8db3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4111260 con 0x7f77a4102600 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a8db3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a4111560 con 0x7f77a4101c40 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 0x7f77a4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a4101c40 0x7f77a419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a259c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a4101c40 0x7f77a419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34202/0 (socket says 172.21.15.7:34202) 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 0x7f77a4198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56836/0 (socket says 172.21.15.7:56836) 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a259c700 1 -- 172.21.15.7:0/3438708990 learned_addr learned my addr 172.21.15.7:0/3438708990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.388+0000 7f77a1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102600 0x7f77a419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77a259c700 1 -- 172.21.15.7:0/3438708990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102600 msgr2=0x7f77a419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77a259c700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102600 0x7f77a419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77a259c700 1 -- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 msgr2=0x7f77a4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77a259c700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 0x7f77a4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77a259c700 1 -- 172.21.15.7:0/3438708990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a4199020 con 0x7f77a4101c40 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77a2d9d700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 0x7f77a4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77a259c700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a4101c40 0x7f77a419e670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f779400f690 tx=0x7f7794005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77937fe700 1 -- 172.21.15.7:0/3438708990 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794027070 con 0x7f77a4101c40 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77937fe700 1 -- 172.21.15.7:0/3438708990 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7794003680 con 0x7f77a4101c40 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a41992b0 con 0x7f77a4101c40 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77937fe700 1 -- 172.21.15.7:0/3438708990 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940173f0 con 0x7f77a4101c40 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.389+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a41a55f0 con 0x7f77a4101c40 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.390+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a4199610 con 0x7f77a4101c40 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.391+0000 7f77937fe700 1 -- 172.21.15.7:0/3438708990 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f779402a070 con 0x7f77a4101c40 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.391+0000 7f77937fe700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f778c0a29a0 0x7f778c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.392+0000 7f77a1d9b700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f778c0a29a0 0x7f778c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.392+0000 7f77937fe700 1 -- 172.21.15.7:0/3438708990 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f77940cf820 con 0x7f77a4101c40 2024-03-26T08:40:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.392+0000 7f77a1d9b700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f778c0a29a0 0x7f778c0a4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f77a4199f00 tx=0x7f7798009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.398+0000 7f77937fe700 1 -- 172.21.15.7:0/3438708990 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7794013070 con 0x7f77a4101c40 2024-03-26T08:40:29.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:29 smithi007 ceph-mon[32462]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-03-26T08:40:29.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.604+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77a4199c70 con 0x7f77a4101c40 2024-03-26T08:40:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.605+0000 7f77937fe700 1 -- 172.21.15.7:0/3438708990 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22089 (secure 0 0 0) 0x7f7794096e20 con 0x7f77a4101c40 2024-03-26T08:40:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:40:29.598 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:10.919648+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:40:29.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f778c0a29a0 msgr2=0x7f778c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:29.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f778c0a29a0 0x7f778c0a4e60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f77a4199f00 tx=0x7f7798009260 comp rx=0 tx=0).stop 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a4101c40 msgr2=0x7f77a419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a4101c40 0x7f77a419e670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f779400f690 tx=0x7f7794005f10 comp rx=0 tx=0).stop 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 shutdown_connections 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f778c0a29a0 0x7f778c0a4e60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f77a4101c40 0x7f77a419e670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a41060c0 0x7f77a4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 --2- 172.21.15.7:0/3438708990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77a4102600 0x7f77a419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 >> 172.21.15.7:0/3438708990 conn(0x7f77a4076410 msgr2=0x7f77a4105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 shutdown_connections 2024-03-26T08:40:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:29.608+0000 7f77a8db3700 1 -- 172.21.15.7:0/3438708990 wait complete. 2024-03-26T08:40:29.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:29 smithi016 ceph-mon[35851]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-03-26T08:40:29.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:29 smithi044 ceph-mon[36461]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-03-26T08:40:30.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-26T08:40:30.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-26T08:40:30.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-26T08:40:30.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-26T08:40:30.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-26T08:40:30.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-26T08:40:30.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-26T08:40:30.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-26T08:40:30.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-26T08:40:30.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-26T08:40:30.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-26T08:40:30.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-26T08:40:30.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:30 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3438708990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:30.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:30 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3438708990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:30.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:30 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3438708990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:40:30.981 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:31 smithi007 ceph-mon[32462]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-03-26T08:40:31.721 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.721 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.741 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:31 smithi016 ceph-mon[35851]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-03-26T08:40:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:31.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:31 smithi044 ceph-mon[36461]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.4 KiB/s wr, 8 op/s 2024-03-26T08:40:33.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:33 smithi007 ceph-mon[32462]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-03-26T08:40:33.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:33 smithi016 ceph-mon[35851]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-03-26T08:40:33.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:33 smithi044 ceph-mon[36461]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-03-26T08:40:34.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.882+0000 7fe4ec3db700 1 -- 172.21.15.7:0/912199484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e4102ba0 msgr2=0x7fe4e410efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:34.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.882+0000 7fe4ec3db700 1 --2- 172.21.15.7:0/912199484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e4102ba0 0x7fe4e410efe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b0d0 tx=0x7fe4dc00b3e0 comp rx=0 tx=0).stop 2024-03-26T08:40:34.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.882+0000 7fe4ec3db700 1 -- 172.21.15.7:0/912199484 shutdown_connections 2024-03-26T08:40:34.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.882+0000 7fe4ec3db700 1 --2- 172.21.15.7:0/912199484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e4102ba0 0x7fe4e410efe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:34.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.882+0000 7fe4ec3db700 1 --2- 172.21.15.7:0/912199484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e41081c0 0x7fe4e41085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:34.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.882+0000 7fe4ec3db700 1 --2- 172.21.15.7:0/912199484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4e41021e0 0x7fe4e4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:34.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.882+0000 7fe4ec3db700 1 -- 172.21.15.7:0/912199484 >> 172.21.15.7:0/912199484 conn(0x7fe4e40792c0 msgr2=0x7fe4e40796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:34.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.882+0000 7fe4ec3db700 1 -- 172.21.15.7:0/912199484 shutdown_connections 2024-03-26T08:40:34.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.882+0000 7fe4ec3db700 1 -- 172.21.15.7:0/912199484 wait complete. 2024-03-26T08:40:34.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ec3db700 1 Processor -- start 2024-03-26T08:40:34.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ec3db700 1 -- start start 2024-03-26T08:40:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ec3db700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e41021e0 0x7fe4e419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ec3db700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4e4102ba0 0x7fe4e419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ec3db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e41081c0 0x7fe4e41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ec3db700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e4112470 con 0x7fe4e41021e0 2024-03-26T08:40:34.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ec3db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e41122f0 con 0x7fe4e41081c0 2024-03-26T08:40:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ec3db700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e41125f0 con 0x7fe4e4102ba0 2024-03-26T08:40:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea177700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e41021e0 0x7fe4e419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea177700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e41021e0 0x7fe4e419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56860/0 (socket says 172.21.15.7:56860) 2024-03-26T08:40:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea177700 1 -- 172.21.15.7:0/2715667285 learned_addr learned my addr 172.21.15.7:0/2715667285 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea978700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e41081c0 0x7fe4e41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4e9976700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4e4102ba0 0x7fe4e419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:34.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea978700 1 -- 172.21.15.7:0/2715667285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4e4102ba0 msgr2=0x7fe4e419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea978700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4e4102ba0 0x7fe4e419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea978700 1 -- 172.21.15.7:0/2715667285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e41021e0 msgr2=0x7fe4e419e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea978700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e41021e0 0x7fe4e419e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea978700 1 -- 172.21.15.7:0/2715667285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4199090 con 0x7fe4e41081c0 2024-03-26T08:40:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4e9976700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4e4102ba0 0x7fe4e419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.883+0000 7fe4ea177700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e41021e0 0x7fe4e419e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:34.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.884+0000 7fe4ea978700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e41081c0 0x7fe4e41987b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b770 tx=0x7fe4dc007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.884+0000 7fe4db7fe700 1 -- 172.21.15.7:0/2715667285 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc021020 con 0x7fe4e41081c0 2024-03-26T08:40:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.884+0000 7fe4db7fe700 1 -- 172.21.15.7:0/2715667285 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc004c50 con 0x7fe4e41081c0 2024-03-26T08:40:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.884+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e4199320 con 0x7fe4e41081c0 2024-03-26T08:40:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.884+0000 7fe4db7fe700 1 -- 172.21.15.7:0/2715667285 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0052b0 con 0x7fe4e41081c0 2024-03-26T08:40:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.884+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e41a5440 con 0x7fe4e41081c0 2024-03-26T08:40:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.885+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4e4062820 con 0x7fe4e41081c0 2024-03-26T08:40:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.886+0000 7fe4db7fe700 1 -- 172.21.15.7:0/2715667285 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fe4dc024020 con 0x7fe4e41081c0 2024-03-26T08:40:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.886+0000 7fe4db7fe700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe4d00a2920 0x7fe4d00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.886+0000 7fe4ea177700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe4d00a2920 0x7fe4d00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.886+0000 7fe4db7fe700 1 -- 172.21.15.7:0/2715667285 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe4dc0cfa70 con 0x7fe4e41081c0 2024-03-26T08:40:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.886+0000 7fe4db7fe700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] conn(0x7fe4d00ac100 0x7fe4d00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.886+0000 7fe4db7fe700 1 -- 172.21.15.7:0/2715667285 --> [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4dc00ae50 con 0x7fe4d00ac100 2024-03-26T08:40:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.886+0000 7fe4db7fe700 1 -- 172.21.15.7:0/2715667285 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4dc0d42c0 con 0x7fe4e41081c0 2024-03-26T08:40:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.887+0000 7fe4ea177700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe4d00a2920 0x7fe4d00a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0011550 tx=0x7fe4e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.887+0000 7fe4e9976700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] conn(0x7fe4d00ac100 0x7fe4d00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.887+0000 7fe4e9976700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] conn(0x7fe4d00ac100 0x7fe4d00ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.888+0000 7fe4db7fe700 1 -- 172.21.15.7:0/2715667285 <== osd.4 v2:172.21.15.16:6808/3280306354 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe4dc00ae50 con 0x7fe4d00ac100 2024-03-26T08:40:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.904+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 --> [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe4e4062cd0 con 0x7fe4d00ac100 2024-03-26T08:40:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.904+0000 7fe4db7fe700 1 -- 172.21.15.7:0/2715667285 <== osd.4 v2:172.21.15.16:6808/3280306354 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe4e4062cd0 con 0x7fe4d00ac100 2024-03-26T08:40:34.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.904+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 >> [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] conn(0x7fe4d00ac100 msgr2=0x7fe4d00ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ec3db700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] conn(0x7fe4d00ac100 0x7fe4d00ae520 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe4d00a2920 msgr2=0x7fe4d00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ec3db700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe4d00a2920 0x7fe4d00a4de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0011550 tx=0x7fe4e0009260 comp rx=0 tx=0).stop 2024-03-26T08:40:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e41081c0 msgr2=0x7fe4e41987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ec3db700 1 --2- 172.21.15.7:0/2715667285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e41081c0 0x7fe4e41987b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00b770 tx=0x7fe4dc007a20 comp rx=0 tx=0).stop 2024-03-26T08:40:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ea978700 1 -- 172.21.15.7:0/2715667285 reap_dead start 2024-03-26T08:40:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 shutdown_connections 2024-03-26T08:40:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 >> 172.21.15.7:0/2715667285 conn(0x7fe4e40792c0 msgr2=0x7fe4e4100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 shutdown_connections 2024-03-26T08:40:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:34.905+0000 7fe4ec3db700 1 -- 172.21.15.7:0/2715667285 wait complete. 2024-03-26T08:40:35.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:35 smithi016 ceph-mon[35851]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T08:40:35.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:35 smithi007 ceph-mon[32462]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T08:40:35.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:35 smithi044 ceph-mon[36461]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T08:40:36.462 INFO:teuthology.orchestra.run.smithi007.stdout:141733920823 2024-03-26T08:40:36.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-26T08:40:37.634 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:37 smithi007 ceph-mon[32462]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-03-26T08:40:37.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:37 smithi016 ceph-mon[35851]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-03-26T08:40:37.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:37 smithi044 ceph-mon[36461]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-03-26T08:40:39.618 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:39 smithi007 ceph-mon[32462]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T08:40:39.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:39 smithi016 ceph-mon[35851]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T08:40:39.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:39 smithi044 ceph-mon[36461]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T08:40:41.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:41 smithi007 ceph-mon[32462]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:40:41.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:41 smithi016 ceph-mon[35851]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:40:41.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:41 smithi044 ceph-mon[36461]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:40:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.239+0000 7f87c3d05700 1 -- 172.21.15.7:0/1283972728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87bc06a930 msgr2=0x7f87bc06ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.239+0000 7f87c3d05700 1 --2- 172.21.15.7:0/1283972728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87bc06a930 0x7f87bc06ad10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00b0d0 tx=0x7f87ac00b3e0 comp rx=0 tx=0).stop 2024-03-26T08:40:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.239+0000 7f87c3d05700 1 -- 172.21.15.7:0/1283972728 shutdown_connections 2024-03-26T08:40:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.239+0000 7f87c3d05700 1 --2- 172.21.15.7:0/1283972728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87bc107510 0x7f87bc10f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.239+0000 7f87c3d05700 1 --2- 172.21.15.7:0/1283972728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87bc06a930 0x7f87bc06ad10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.239+0000 7f87c3d05700 1 --2- 172.21.15.7:0/1283972728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87bc06b2e0 0x7f87bc106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.240+0000 7f87c3d05700 1 -- 172.21.15.7:0/1283972728 >> 172.21.15.7:0/1283972728 conn(0x7f87bc0770e0 msgr2=0x7f87bc0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.240+0000 7f87c3d05700 1 -- 172.21.15.7:0/1283972728 shutdown_connections 2024-03-26T08:40:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.240+0000 7f87c3d05700 1 -- 172.21.15.7:0/1283972728 wait complete. 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.240+0000 7f87c3d05700 1 Processor -- start 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.240+0000 7f87c3d05700 1 -- start start 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c3d05700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87bc06a930 0x7f87bc19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c3d05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87bc06b2e0 0x7f87bc19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c3d05700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87bc107510 0x7f87bc1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c3d05700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc1124d0 con 0x7f87bc06a930 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c3d05700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc112350 con 0x7f87bc06b2e0 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c3d05700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc112650 con 0x7f87bc107510 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c1aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87bc06a930 0x7f87bc19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c1aa1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87bc06a930 0x7f87bc19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45208/0 (socket says 172.21.15.7:45208) 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c1aa1700 1 -- 172.21.15.7:0/1161441860 learned_addr learned my addr 172.21.15.7:0/1161441860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c12a0700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87bc06b2e0 0x7f87bc19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c22a2700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87bc107510 0x7f87bc1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c1aa1700 1 -- 172.21.15.7:0/1161441860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87bc107510 msgr2=0x7f87bc1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c1aa1700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f87bc107510 0x7f87bc1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c1aa1700 1 -- 172.21.15.7:0/1161441860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87bc06b2e0 msgr2=0x7f87bc19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c1aa1700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87bc06b2e0 0x7f87bc19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c1aa1700 1 -- 172.21.15.7:0/1161441860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc198fb0 con 0x7f87bc06a930 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c12a0700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87bc06b2e0 0x7f87bc19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87c1aa1700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87bc06a930 0x7f87bc19e650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00d010 tx=0x7f87ac007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.241+0000 7f87baffd700 1 -- 172.21.15.7:0/1161441860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac021070 con 0x7f87bc06a930 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.242+0000 7f87baffd700 1 -- 172.21.15.7:0/1161441860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87ac01db30 con 0x7f87bc06a930 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.242+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc199240 con 0x7f87bc06a930 2024-03-26T08:40:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.242+0000 7f87baffd700 1 -- 172.21.15.7:0/1161441860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac01bb00 con 0x7f87bc06a930 2024-03-26T08:40:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.242+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc1a54b0 con 0x7f87bc06a930 2024-03-26T08:40:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.242+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f87bc062860 con 0x7f87bc06a930 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.244+0000 7f87baffd700 1 -- 172.21.15.7:0/1161441860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f87ac024070 con 0x7f87bc06a930 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.244+0000 7f87baffd700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87a40a2920 0x7f87a40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.244+0000 7f87c12a0700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87a40a2920 0x7f87a40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.244+0000 7f87baffd700 1 -- 172.21.15.7:0/1161441860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f87ac00f020 con 0x7f87bc06a930 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.244+0000 7f87baffd700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] conn(0x7f87a40abfd0 0x7f87a40ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.244+0000 7f87baffd700 1 -- 172.21.15.7:0/1161441860 --> [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87a40aeaa0 con 0x7f87a40abfd0 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.244+0000 7f87baffd700 1 -- 172.21.15.7:0/1161441860 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87ac0d0910 con 0x7f87bc06a930 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.245+0000 7f87c22a2700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] conn(0x7f87a40abfd0 0x7f87a40ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.245+0000 7f87c12a0700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87a40a2920 0x7f87a40a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87bc100ce0 tx=0x7f87b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.245+0000 7f87c22a2700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] conn(0x7f87a40abfd0 0x7f87a40ae3f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.246+0000 7f87baffd700 1 -- 172.21.15.7:0/1161441860 <== osd.9 v2:172.21.15.7:6826/387825612 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f87a40aeaa0 con 0x7f87a40abfd0 2024-03-26T08:40:42.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.262+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 --> [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f87bc062ce0 con 0x7f87a40abfd0 2024-03-26T08:40:42.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.262+0000 7f87baffd700 1 -- 172.21.15.7:0/1161441860 <== osd.9 v2:172.21.15.7:6826/387825612 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f87bc062ce0 con 0x7f87a40abfd0 2024-03-26T08:40:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.262+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] conn(0x7f87a40abfd0 msgr2=0x7f87a40ae3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.262+0000 7f87c3d05700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] conn(0x7f87a40abfd0 0x7f87a40ae3f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.263+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87a40a2920 msgr2=0x7f87a40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.263+0000 7f87c3d05700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87a40a2920 0x7f87a40a4de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f87bc100ce0 tx=0x7f87b0009260 comp rx=0 tx=0).stop 2024-03-26T08:40:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.263+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87bc06a930 msgr2=0x7f87bc19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.263+0000 7f87c3d05700 1 --2- 172.21.15.7:0/1161441860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87bc06a930 0x7f87bc19e650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00d010 tx=0x7f87ac007a20 comp rx=0 tx=0).stop 2024-03-26T08:40:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.263+0000 7f87c22a2700 1 -- 172.21.15.7:0/1161441860 reap_dead start 2024-03-26T08:40:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.263+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 shutdown_connections 2024-03-26T08:40:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.263+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 >> 172.21.15.7:0/1161441860 conn(0x7f87bc0770e0 msgr2=0x7f87bc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.263+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 shutdown_connections 2024-03-26T08:40:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:42.263+0000 7f87c3d05700 1 -- 172.21.15.7:0/1161441860 wait complete. 2024-03-26T08:40:43.025 INFO:teuthology.orchestra.run.smithi007.stdout:287762808863 2024-03-26T08:40:43.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-26T08:40:43.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:43 smithi016 ceph-mon[35851]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T08:40:43.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:43 smithi007 ceph-mon[32462]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T08:40:43.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:43 smithi044 ceph-mon[36461]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T08:40:44.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 -- 172.21.15.7:0/370818439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c54101a30 msgr2=0x7f6c5410df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 --2- 172.21.15.7:0/370818439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c54101a30 0x7f6c5410df10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00b0a0 tx=0x7f6c4c00b3b0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 -- 172.21.15.7:0/370818439 shutdown_connections 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 --2- 172.21.15.7:0/370818439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c541070d0 0x7f6c541074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 --2- 172.21.15.7:0/370818439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c54101a30 0x7f6c5410df10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 --2- 172.21.15.7:0/370818439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c54101090 0x7f6c541014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 -- 172.21.15.7:0/370818439 >> 172.21.15.7:0/370818439 conn(0x7f6c54077fc0 msgr2=0x7f6c540783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 -- 172.21.15.7:0/370818439 shutdown_connections 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 -- 172.21.15.7:0/370818439 wait complete. 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 Processor -- start 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.438+0000 7f6c5b082700 1 -- start start 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c5b082700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c54101090 0x7f6c54199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c5b082700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c54101a30 0x7f6c54199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c5b082700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c541070d0 0x7f6c541a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c5b082700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54111350 con 0x7f6c54101090 2024-03-26T08:40:44.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c5b082700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541111d0 con 0x7f6c54101a30 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c5b082700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541114d0 con 0x7f6c541070d0 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c58e1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c54101090 0x7f6c54199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c53fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c54101a30 0x7f6c54199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c58e1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c54101090 0x7f6c54199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45222/0 (socket says 172.21.15.7:45222) 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c58e1e700 1 -- 172.21.15.7:0/1594236981 learned_addr learned my addr 172.21.15.7:0/1594236981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c5961f700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c541070d0 0x7f6c541a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c58e1e700 1 -- 172.21.15.7:0/1594236981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c541070d0 msgr2=0x7f6c541a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c58e1e700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c541070d0 0x7f6c541a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c58e1e700 1 -- 172.21.15.7:0/1594236981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c54101a30 msgr2=0x7f6c54199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c58e1e700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c54101a30 0x7f6c54199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c58e1e700 1 -- 172.21.15.7:0/1594236981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c541a4d10 con 0x7f6c54101090 2024-03-26T08:40:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c5961f700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c541070d0 0x7f6c541a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c58e1e700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c54101090 0x7f6c54199750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9c0 tx=0x7f6c4800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c51ffb700 1 -- 172.21.15.7:0/1594236981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800cbd0 con 0x7f6c54101090 2024-03-26T08:40:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c541a4ee0 con 0x7f6c54101090 2024-03-26T08:40:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c51ffb700 1 -- 172.21.15.7:0/1594236981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c4800ee60 con 0x7f6c54101090 2024-03-26T08:40:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.439+0000 7f6c51ffb700 1 -- 172.21.15.7:0/1594236981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48005630 con 0x7f6c54101090 2024-03-26T08:40:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.440+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541a5390 con 0x7f6c54101090 2024-03-26T08:40:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.440+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c540628a0 con 0x7f6c54101090 2024-03-26T08:40:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.441+0000 7f6c51ffb700 1 -- 172.21.15.7:0/1594236981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f6c48004750 con 0x7f6c54101090 2024-03-26T08:40:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.442+0000 7f6c51ffb700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c440a2920 0x7f6c440a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.442+0000 7f6c53fff700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c440a2920 0x7f6c440a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.442+0000 7f6c51ffb700 1 -- 172.21.15.7:0/1594236981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6c480cec40 con 0x7f6c54101090 2024-03-26T08:40:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.442+0000 7f6c51ffb700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] conn(0x7f6c440ac100 0x7f6c440ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.442+0000 7f6c51ffb700 1 -- 172.21.15.7:0/1594236981 --> [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c440aebd0 con 0x7f6c440ac100 2024-03-26T08:40:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.442+0000 7f6c51ffb700 1 -- 172.21.15.7:0/1594236981 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c480d1390 con 0x7f6c54101090 2024-03-26T08:40:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.442+0000 7f6c5961f700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] conn(0x7f6c440ac100 0x7f6c440ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.443+0000 7f6c53fff700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c440a2920 0x7f6c440a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c54078630 tx=0x7f6c40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.443+0000 7f6c5961f700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] conn(0x7f6c440ac100 0x7f6c440ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.443+0000 7f6c51ffb700 1 -- 172.21.15.7:0/1594236981 <== osd.6 v2:172.21.15.7:6818/2859951867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6c440aebd0 con 0x7f6c440ac100 2024-03-26T08:40:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.460+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 --> [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6c54062d50 con 0x7f6c440ac100 2024-03-26T08:40:44.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.460+0000 7f6c51ffb700 1 -- 172.21.15.7:0/1594236981 <== osd.6 v2:172.21.15.7:6818/2859951867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6c54062d50 con 0x7f6c440ac100 2024-03-26T08:40:44.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] conn(0x7f6c440ac100 msgr2=0x7f6c440ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] conn(0x7f6c440ac100 0x7f6c440ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c440a2920 msgr2=0x7f6c440a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c440a2920 0x7f6c440a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6c54078630 tx=0x7f6c40009260 comp rx=0 tx=0).stop 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c54101090 msgr2=0x7f6c54199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 --2- 172.21.15.7:0/1594236981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c54101090 0x7f6c54199750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800e9c0 tx=0x7f6c4800ecd0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5961f700 1 -- 172.21.15.7:0/1594236981 reap_dead start 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 shutdown_connections 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 >> 172.21.15.7:0/1594236981 conn(0x7f6c54077fc0 msgr2=0x7f6c540ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 shutdown_connections 2024-03-26T08:40:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.461+0000 7f6c5b082700 1 -- 172.21.15.7:0/1594236981 wait complete. 2024-03-26T08:40:44.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.655+0000 7fd334a5f700 1 -- 172.21.15.7:0/3795153832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd33006ab90 msgr2=0x7fd33006af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.655+0000 7fd334a5f700 1 --2- 172.21.15.7:0/3795153832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd33006ab90 0x7fd33006af70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd320009990 tx=0x7fd320009ca0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.655+0000 7fd334a5f700 1 -- 172.21.15.7:0/3795153832 shutdown_connections 2024-03-26T08:40:44.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.655+0000 7fd334a5f700 1 --2- 172.21.15.7:0/3795153832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd330107550 0x7fd33010f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.655+0000 7fd334a5f700 1 --2- 172.21.15.7:0/3795153832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd33006ab90 0x7fd33006af70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.655+0000 7fd334a5f700 1 --2- 172.21.15.7:0/3795153832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd33006b540 0x7fd330106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.655+0000 7fd334a5f700 1 -- 172.21.15.7:0/3795153832 >> 172.21.15.7:0/3795153832 conn(0x7fd330078160 msgr2=0x7fd330078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.655+0000 7fd334a5f700 1 -- 172.21.15.7:0/3795153832 shutdown_connections 2024-03-26T08:40:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.656+0000 7fd334a5f700 1 -- 172.21.15.7:0/3795153832 wait complete. 2024-03-26T08:40:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.656+0000 7fd334a5f700 1 Processor -- start 2024-03-26T08:40:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.656+0000 7fd334a5f700 1 -- start start 2024-03-26T08:40:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.656+0000 7fd334a5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd33006ab90 0x7fd33019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.656+0000 7fd334a5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd33006b540 0x7fd33019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.656+0000 7fd334a5f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd330107550 0x7fd330198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd334a5f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3301124b0 con 0x7fd33006b540 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd334a5f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd330112330 con 0x7fd33006ab90 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd334a5f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd330112630 con 0x7fd330107550 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd33006b540 0x7fd33019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd33006ab90 0x7fd33019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd33006ab90 0x7fd33019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47042/0 (socket says 172.21.15.7:47042) 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd33006b540 0x7fd33019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45238/0 (socket says 172.21.15.7:45238) 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32e59c700 1 -- 172.21.15.7:0/2112878200 learned_addr learned my addr 172.21.15.7:0/2112878200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32ed9d700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd330107550 0x7fd330198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32e59c700 1 -- 172.21.15.7:0/2112878200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd330107550 msgr2=0x7fd330198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32e59c700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd330107550 0x7fd330198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32e59c700 1 -- 172.21.15.7:0/2112878200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd33006b540 msgr2=0x7fd33019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32e59c700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd33006b540 0x7fd33019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32e59c700 1 -- 172.21.15.7:0/2112878200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd330199010 con 0x7fd33006ab90 2024-03-26T08:40:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd32e59c700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd33006ab90 0x7fd33019e610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd320000c00 tx=0x7fd32000bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd31f7fe700 1 -- 172.21.15.7:0/2112878200 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320027070 con 0x7fd33006ab90 2024-03-26T08:40:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd31f7fe700 1 -- 172.21.15.7:0/2112878200 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd320003d70 con 0x7fd33006ab90 2024-03-26T08:40:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3301992a0 con 0x7fd33006ab90 2024-03-26T08:40:44.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd31f7fe700 1 -- 172.21.15.7:0/2112878200 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32000f460 con 0x7fd33006ab90 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.657+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3301a53c0 con 0x7fd33006ab90 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.658+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd3300628e0 con 0x7fd33006ab90 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.659+0000 7fd31f7fe700 1 -- 172.21.15.7:0/2112878200 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fd32002a070 con 0x7fd33006ab90 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.660+0000 7fd31f7fe700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3180a2920 0x7fd3180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.660+0000 7fd32dd9b700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3180a2920 0x7fd3180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.660+0000 7fd31f7fe700 1 -- 172.21.15.7:0/2112878200 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd3200cfa70 con 0x7fd33006ab90 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.660+0000 7fd31f7fe700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] conn(0x7fd3180abfd0 0x7fd3180ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.660+0000 7fd31f7fe700 1 -- 172.21.15.7:0/2112878200 --> [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3180aeaa0 con 0x7fd3180abfd0 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.660+0000 7fd31f7fe700 1 -- 172.21.15.7:0/2112878200 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3200cfe20 con 0x7fd33006ab90 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.660+0000 7fd32ed9d700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] conn(0x7fd3180abfd0 0x7fd3180ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.660+0000 7fd32dd9b700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3180a2920 0x7fd3180a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd330199ea0 tx=0x7fd324009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.661+0000 7fd32ed9d700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] conn(0x7fd3180abfd0 0x7fd3180ae3f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.661+0000 7fd31f7fe700 1 -- 172.21.15.7:0/2112878200 <== osd.5 v2:172.21.15.44:6808/2190779473 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd3180aeaa0 con 0x7fd3180abfd0 2024-03-26T08:40:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.682+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 --> [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd330062d60 con 0x7fd3180abfd0 2024-03-26T08:40:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.682+0000 7fd31f7fe700 1 -- 172.21.15.7:0/2112878200 <== osd.5 v2:172.21.15.44:6808/2190779473 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd330062d60 con 0x7fd3180abfd0 2024-03-26T08:40:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 >> [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] conn(0x7fd3180abfd0 msgr2=0x7fd3180ae3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] conn(0x7fd3180abfd0 0x7fd3180ae3f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3180a2920 msgr2=0x7fd3180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3180a2920 0x7fd3180a4de0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd330199ea0 tx=0x7fd324009260 comp rx=0 tx=0).stop 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd33006ab90 msgr2=0x7fd33019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 --2- 172.21.15.7:0/2112878200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd33006ab90 0x7fd33019e610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd320000c00 tx=0x7fd32000bab0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd32ed9d700 1 -- 172.21.15.7:0/2112878200 reap_dead start 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 shutdown_connections 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 >> 172.21.15.7:0/2112878200 conn(0x7fd330078160 msgr2=0x7fd3301007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 shutdown_connections 2024-03-26T08:40:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:44.683+0000 7fd334a5f700 1 -- 172.21.15.7:0/2112878200 wait complete. 2024-03-26T08:40:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.738+0000 7f7a3999a700 1 -- 172.21.15.7:0/1046120937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a34107120 msgr2=0x7f7a34107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.738+0000 7f7a3999a700 1 --2- 172.21.15.7:0/1046120937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a34107120 0x7f7a34107500 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009990 tx=0x7f7a24009ca0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.738+0000 7f7a3999a700 1 -- 172.21.15.7:0/1046120937 shutdown_connections 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.738+0000 7f7a3999a700 1 --2- 172.21.15.7:0/1046120937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34101a30 0x7f7a3410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.738+0000 7f7a3999a700 1 --2- 172.21.15.7:0/1046120937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a34107120 0x7f7a34107500 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.738+0000 7f7a3999a700 1 --2- 172.21.15.7:0/1046120937 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a34101070 0x7f7a341014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.738+0000 7f7a3999a700 1 -- 172.21.15.7:0/1046120937 >> 172.21.15.7:0/1046120937 conn(0x7f7a340780a0 msgr2=0x7f7a340784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.738+0000 7f7a3999a700 1 -- 172.21.15.7:0/1046120937 shutdown_connections 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.738+0000 7f7a3999a700 1 -- 172.21.15.7:0/1046120937 wait complete. 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a3999a700 1 Processor -- start 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a3999a700 1 -- start start 2024-03-26T08:40:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a3999a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a34101070 0x7f7a3419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a3999a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34101a30 0x7f7a3419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a3999a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a34107120 0x7f7a34197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a3999a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34111360 con 0x7f7a34107120 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a3999a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341111e0 con 0x7f7a34101a30 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a3999a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341114e0 con 0x7f7a34101070 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a34107120 0x7f7a34197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a34107120 0x7f7a34197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45186/0 (socket says 172.21.15.7:45186) 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a32ffd700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a34101070 0x7f7a3419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a337fe700 1 -- 172.21.15.7:0/264619644 learned_addr learned my addr 172.21.15.7:0/264619644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.739+0000 7f7a327fc700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34101a30 0x7f7a3419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a32ffd700 1 -- 172.21.15.7:0/264619644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34101a30 msgr2=0x7f7a3419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a32ffd700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34101a30 0x7f7a3419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a32ffd700 1 -- 172.21.15.7:0/264619644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a34107120 msgr2=0x7f7a34197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a32ffd700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a34107120 0x7f7a34197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a32ffd700 1 -- 172.21.15.7:0/264619644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a34197d80 con 0x7f7a34101070 2024-03-26T08:40:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a327fc700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34101a30 0x7f7a3419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a32ffd700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a34101070 0x7f7a3419d410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7a240059d0 tx=0x7f7a2400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a38998700 1 -- 172.21.15.7:0/264619644 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24027070 con 0x7f7a34101070 2024-03-26T08:40:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a34198010 con 0x7f7a34101070 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a38998700 1 -- 172.21.15.7:0/264619644 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a24003d70 con 0x7f7a34101070 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a38998700 1 -- 172.21.15.7:0/264619644 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24022e30 con 0x7f7a34101070 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.740+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3406a8b0 con 0x7f7a34101070 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.741+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a3406ae20 con 0x7f7a34101070 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.742+0000 7f7a38998700 1 -- 172.21.15.7:0/264619644 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f7a2402a070 con 0x7f7a34101070 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.743+0000 7f7a38998700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a1c0a2920 0x7f7a1c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.743+0000 7f7a327fc700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a1c0a2920 0x7f7a1c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.743+0000 7f7a38998700 1 -- 172.21.15.7:0/264619644 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7a240d00b0 con 0x7f7a34101070 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.743+0000 7f7a38998700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] conn(0x7f7a1c0ac100 0x7f7a1c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.743+0000 7f7a38998700 1 -- 172.21.15.7:0/264619644 --> [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a1c0aebd0 con 0x7f7a1c0ac100 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.743+0000 7f7a38998700 1 -- 172.21.15.7:0/264619644 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a240d0460 con 0x7f7a34101070 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.743+0000 7f7a327fc700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a1c0a2920 0x7f7a1c0a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7a34078710 tx=0x7f7a28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.744+0000 7f7a337fe700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] conn(0x7f7a1c0ac100 0x7f7a1c0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.744+0000 7f7a337fe700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] conn(0x7f7a1c0ac100 0x7f7a1c0ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.745+0000 7f7a38998700 1 -- 172.21.15.7:0/264619644 <== osd.8 v2:172.21.15.44:6816/3863516143 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7a1c0aebd0 con 0x7f7a1c0ac100 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.761+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 --> [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7a3406b220 con 0x7f7a1c0ac100 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.761+0000 7f7a38998700 1 -- 172.21.15.7:0/264619644 <== osd.8 v2:172.21.15.44:6816/3863516143 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7a3406b220 con 0x7f7a1c0ac100 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 >> [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] conn(0x7f7a1c0ac100 msgr2=0x7f7a1c0ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] conn(0x7f7a1c0ac100 0x7f7a1c0ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a1c0a2920 msgr2=0x7f7a1c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a1c0a2920 0x7f7a1c0a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7a34078710 tx=0x7f7a28009260 comp rx=0 tx=0).stop 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a34101070 msgr2=0x7f7a3419d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 --2- 172.21.15.7:0/264619644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a34101070 0x7f7a3419d410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7a240059d0 tx=0x7f7a2400bab0 comp rx=0 tx=0).stop 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a337fe700 1 -- 172.21.15.7:0/264619644 reap_dead start 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 shutdown_connections 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 >> 172.21.15.7:0/264619644 conn(0x7f7a340780a0 msgr2=0x7f7a340ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 shutdown_connections 2024-03-26T08:40:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:39.762+0000 7f7a3999a700 1 -- 172.21.15.7:0/264619644 wait complete. 2024-03-26T08:40:45.590 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:45 smithi007 ceph-mon[32462]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:40:45.591 INFO:teuthology.orchestra.run.smithi007.stdout:253403070500 2024-03-26T08:40:45.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-26T08:40:45.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:45 smithi016 ceph-mon[35851]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:40:45.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:45 smithi044 ceph-mon[36461]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:40:47.124 INFO:teuthology.orchestra.run.smithi007.stdout:197568495663 2024-03-26T08:40:47.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-26T08:40:47.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:47 smithi016 ceph-mon[35851]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T08:40:47.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:47 smithi007 ceph-mon[32462]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T08:40:47.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:47 smithi044 ceph-mon[36461]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T08:40:49.432 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:49 smithi007 ceph-mon[32462]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T08:40:49.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.724+0000 7f3e0c63d700 1 -- 172.21.15.7:0/3023276593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e04069e40 msgr2=0x7f3e0406a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:49.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.724+0000 7f3e0c63d700 1 --2- 172.21.15.7:0/3023276593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e04069e40 0x7f3e0406a220 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009990 tx=0x7f3e00009ca0 comp rx=0 tx=0).stop 2024-03-26T08:40:49.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.724+0000 7f3e0c63d700 1 -- 172.21.15.7:0/3023276593 shutdown_connections 2024-03-26T08:40:49.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.724+0000 7f3e0c63d700 1 --2- 172.21.15.7:0/3023276593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e0410ec40 0x7f3e04111030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:49.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.724+0000 7f3e0c63d700 1 --2- 172.21.15.7:0/3023276593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e04069e40 0x7f3e0406a220 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:49.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.724+0000 7f3e0c63d700 1 --2- 172.21.15.7:0/3023276593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e0406a760 0x7f3e0410e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:49.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.724+0000 7f3e0c63d700 1 -- 172.21.15.7:0/3023276593 >> 172.21.15.7:0/3023276593 conn(0x7f3e04077fc0 msgr2=0x7f3e040783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.724+0000 7f3e0c63d700 1 -- 172.21.15.7:0/3023276593 shutdown_connections 2024-03-26T08:40:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.724+0000 7f3e0c63d700 1 -- 172.21.15.7:0/3023276593 wait complete. 2024-03-26T08:40:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0c63d700 1 Processor -- start 2024-03-26T08:40:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0c63d700 1 -- start start 2024-03-26T08:40:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0c63d700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e04069e40 0x7f3e041a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0c63d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e0406a760 0x7f3e041a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0c63d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0410ec40 0x7f3e0419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0c63d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e041146c0 con 0x7f3e0410ec40 2024-03-26T08:40:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0c63d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04114540 con 0x7f3e0406a760 2024-03-26T08:40:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0c63d700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04114840 con 0x7f3e04069e40 2024-03-26T08:40:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e09bd8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e0406a760 0x7f3e041a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:49.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0a3d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e04069e40 0x7f3e041a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0abda700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0410ec40 0x7f3e0419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e09bd8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e0406a760 0x7f3e041a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55256/0 (socket says 172.21.15.7:55256) 2024-03-26T08:40:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0a3d9700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e04069e40 0x7f3e041a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:55566/0 (socket says 172.21.15.7:55566) 2024-03-26T08:40:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e09bd8700 1 -- 172.21.15.7:0/2396690648 learned_addr learned my addr 172.21.15.7:0/2396690648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e09bd8700 1 -- 172.21.15.7:0/2396690648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e04069e40 msgr2=0x7f3e041a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e09bd8700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e04069e40 0x7f3e041a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e09bd8700 1 -- 172.21.15.7:0/2396690648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0410ec40 msgr2=0x7f3e0419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e09bd8700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0410ec40 0x7f3e0419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e09bd8700 1 -- 172.21.15.7:0/2396690648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0419b160 con 0x7f3e0406a760 2024-03-26T08:40:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0a3d9700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3e04069e40 0x7f3e041a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.725+0000 7f3e0abda700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0410ec40 0x7f3e0419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.726+0000 7f3e09bd8700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e0406a760 0x7f3e041a0d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3df400e9a0 tx=0x7f3df400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.726+0000 7f3dfb7fe700 1 -- 172.21.15.7:0/2396690648 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df400ca10 con 0x7f3e0406a760 2024-03-26T08:40:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.726+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0419b450 con 0x7f3e0406a760 2024-03-26T08:40:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.726+0000 7f3dfb7fe700 1 -- 172.21.15.7:0/2396690648 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3df400cbb0 con 0x7f3e0406a760 2024-03-26T08:40:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.726+0000 7f3dfb7fe700 1 -- 172.21.15.7:0/2396690648 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4018680 con 0x7f3e0406a760 2024-03-26T08:40:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.726+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e041a7710 con 0x7f3e0406a760 2024-03-26T08:40:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.727+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e040628a0 con 0x7f3e0406a760 2024-03-26T08:40:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.728+0000 7f3dfb7fe700 1 -- 172.21.15.7:0/2396690648 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f3df4010430 con 0x7f3e0406a760 2024-03-26T08:40:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.728+0000 7f3dfb7fe700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3df00a2920 0x7f3df00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.728+0000 7f3e0a3d9700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3df00a2920 0x7f3df00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.729+0000 7f3dfb7fe700 1 -- 172.21.15.7:0/2396690648 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3df40cf550 con 0x7f3e0406a760 2024-03-26T08:40:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.729+0000 7f3dfb7fe700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] conn(0x7f3df00ac100 0x7f3df00ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.729+0000 7f3dfb7fe700 1 -- 172.21.15.7:0/2396690648 --> [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3df401bec0 con 0x7f3df00ac100 2024-03-26T08:40:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.729+0000 7f3dfb7fe700 1 -- 172.21.15.7:0/2396690648 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3df40cf940 con 0x7f3e0406a760 2024-03-26T08:40:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.729+0000 7f3e0abda700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] conn(0x7f3df00ac100 0x7f3df00ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.729+0000 7f3e0a3d9700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3df00a2920 0x7f3df00a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3e0419bb40 tx=0x7f3e00005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.729+0000 7f3e0abda700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] conn(0x7f3df00ac100 0x7f3df00ae520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:49.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.730+0000 7f3dfb7fe700 1 -- 172.21.15.7:0/2396690648 <== osd.7 v2:172.21.15.16:6816/3902462229 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3df401bec0 con 0x7f3df00ac100 2024-03-26T08:40:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.746+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 --> [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3e04062d20 con 0x7f3df00ac100 2024-03-26T08:40:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3dfb7fe700 1 -- 172.21.15.7:0/2396690648 <== osd.7 v2:172.21.15.16:6816/3902462229 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3e04062d20 con 0x7f3df00ac100 2024-03-26T08:40:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 >> [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] conn(0x7f3df00ac100 msgr2=0x7f3df00ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] conn(0x7f3df00ac100 0x7f3df00ae520 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3df00a2920 msgr2=0x7f3df00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3df00a2920 0x7f3df00a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3e0419bb40 tx=0x7f3e00005fd0 comp rx=0 tx=0).stop 2024-03-26T08:40:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e0406a760 msgr2=0x7f3e041a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 --2- 172.21.15.7:0/2396690648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e0406a760 0x7f3e041a0d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3df400e9a0 tx=0x7f3df400ecb0 comp rx=0 tx=0).stop 2024-03-26T08:40:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0abda700 1 -- 172.21.15.7:0/2396690648 reap_dead start 2024-03-26T08:40:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 shutdown_connections 2024-03-26T08:40:49.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 >> 172.21.15.7:0/2396690648 conn(0x7f3e04077fc0 msgr2=0x7f3e041102b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:49.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 shutdown_connections 2024-03-26T08:40:49.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:49.747+0000 7f3e0c63d700 1 -- 172.21.15.7:0/2396690648 wait complete. 2024-03-26T08:40:49.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:49 smithi016 ceph-mon[35851]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T08:40:49.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:49 smithi044 ceph-mon[36461]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T08:40:50.569 INFO:teuthology.orchestra.run.smithi007.stdout:163208757299 2024-03-26T08:40:50.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-26T08:40:51.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.485+0000 7f8c3a155700 1 -- 172.21.15.7:0/2363304984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341024a0 msgr2=0x7f8c34105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:51.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.485+0000 7f8c3a155700 1 --2- 172.21.15.7:0/2363304984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341024a0 0x7f8c34105d30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8c280099c0 tx=0x7f8c28009cd0 comp rx=0 tx=0).stop 2024-03-26T08:40:51.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.486+0000 7f8c3a155700 1 -- 172.21.15.7:0/2363304984 shutdown_connections 2024-03-26T08:40:51.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.486+0000 7f8c3a155700 1 --2- 172.21.15.7:0/2363304984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c34101af0 0x7f8c34101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:51.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.486+0000 7f8c3a155700 1 --2- 172.21.15.7:0/2363304984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c34106410 0x7f8c3410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:51.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.486+0000 7f8c3a155700 1 --2- 172.21.15.7:0/2363304984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c341024a0 0x7f8c34105d30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:51.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.486+0000 7f8c3a155700 1 -- 172.21.15.7:0/2363304984 >> 172.21.15.7:0/2363304984 conn(0x7f8c34077690 msgr2=0x7f8c340feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:51.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.486+0000 7f8c3a155700 1 -- 172.21.15.7:0/2363304984 shutdown_connections 2024-03-26T08:40:51.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.486+0000 7f8c3a155700 1 -- 172.21.15.7:0/2363304984 wait complete. 2024-03-26T08:40:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.486+0000 7f8c3a155700 1 Processor -- start 2024-03-26T08:40:51.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c3a155700 1 -- start start 2024-03-26T08:40:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c3a155700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c34101af0 0x7f8c3419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:51.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c3a155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c341024a0 0x7f8c3419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c3a155700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c34106410 0x7f8c34198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c3a155700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c341113d0 con 0x7f8c34101af0 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c3a155700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c34111250 con 0x7f8c341024a0 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c3a155700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c34111550 con 0x7f8c34106410 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c34101af0 0x7f8c3419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c34101af0 0x7f8c3419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39532/0 (socket says 172.21.15.7:39532) 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c337fe700 1 -- 172.21.15.7:0/63027489 learned_addr learned my addr 172.21.15.7:0/63027489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c32ffd700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c341024a0 0x7f8c3419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c33fff700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c34106410 0x7f8c34198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c33fff700 1 -- 172.21.15.7:0/63027489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c341024a0 msgr2=0x7f8c3419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c33fff700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c341024a0 0x7f8c3419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:51.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c33fff700 1 -- 172.21.15.7:0/63027489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c34101af0 msgr2=0x7f8c3419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c33fff700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c34101af0 0x7f8c3419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c33fff700 1 -- 172.21.15.7:0/63027489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c34198ff0 con 0x7f8c34106410 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c32ffd700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c341024a0 0x7f8c3419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.487+0000 7f8c337fe700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c34101af0 0x7f8c3419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.488+0000 7f8c33fff700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c34106410 0x7f8c34198810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00efc0 tx=0x7f8c2c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.488+0000 7f8c30ff9700 1 -- 172.21.15.7:0/63027489 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c0092e0 con 0x7f8c34106410 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.488+0000 7f8c30ff9700 1 -- 172.21.15.7:0/63027489 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c004bb0 con 0x7f8c34106410 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.488+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c34199280 con 0x7f8c34106410 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.488+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c341a54a0 con 0x7f8c34106410 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.488+0000 7f8c30ff9700 1 -- 172.21.15.7:0/63027489 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c004280 con 0x7f8c34106410 2024-03-26T08:40:51.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.489+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c341055b0 con 0x7f8c34106410 2024-03-26T08:40:51.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.490+0000 7f8c30ff9700 1 -- 172.21.15.7:0/63027489 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f8c2c01b030 con 0x7f8c34106410 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.490+0000 7f8c30ff9700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c1c0a29a0 0x7f8c1c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.490+0000 7f8c337fe700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c1c0a29a0 0x7f8c1c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.490+0000 7f8c30ff9700 1 -- 172.21.15.7:0/63027489 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8c2c0ce6d0 con 0x7f8c34106410 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.491+0000 7f8c30ff9700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] conn(0x7f8c1c0ac180 0x7f8c1c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.491+0000 7f8c30ff9700 1 -- 172.21.15.7:0/63027489 --> [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c2c018ec0 con 0x7f8c1c0ac180 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.491+0000 7f8c30ff9700 1 -- 172.21.15.7:0/63027489 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c2c0cea80 con 0x7f8c34106410 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.491+0000 7f8c32ffd700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] conn(0x7f8c1c0ac180 0x7f8c1c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.491+0000 7f8c337fe700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c1c0a29a0 0x7f8c1c0a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c24011550 tx=0x7f8c24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.491+0000 7f8c32ffd700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] conn(0x7f8c1c0ac180 0x7f8c1c0ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:51.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.492+0000 7f8c30ff9700 1 -- 172.21.15.7:0/63027489 <== osd.10 v2:172.21.15.16:6824/1979394150 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8c2c018ec0 con 0x7f8c1c0ac180 2024-03-26T08:40:51.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.508+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 --> [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8c34068e50 con 0x7f8c1c0ac180 2024-03-26T08:40:51.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.508+0000 7f8c30ff9700 1 -- 172.21.15.7:0/63027489 <== osd.10 v2:172.21.15.16:6824/1979394150 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8c34068e50 con 0x7f8c1c0ac180 2024-03-26T08:40:51.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 >> [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] conn(0x7f8c1c0ac180 msgr2=0x7f8c1c0ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] conn(0x7f8c1c0ac180 0x7f8c1c0ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c1c0a29a0 msgr2=0x7f8c1c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c1c0a29a0 0x7f8c1c0a4e60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8c24011550 tx=0x7f8c24009260 comp rx=0 tx=0).stop 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c34106410 msgr2=0x7f8c34198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 --2- 172.21.15.7:0/63027489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c34106410 0x7f8c34198810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00efc0 tx=0x7f8c2c00c3a0 comp rx=0 tx=0).stop 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c33fff700 1 -- 172.21.15.7:0/63027489 reap_dead start 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 shutdown_connections 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 >> 172.21.15.7:0/63027489 conn(0x7f8c34077690 msgr2=0x7f8c34100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 shutdown_connections 2024-03-26T08:40:51.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:51.509+0000 7f8c3a155700 1 -- 172.21.15.7:0/63027489 wait complete. 2024-03-26T08:40:51.510 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:51 smithi007 ceph-mon[32462]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T08:40:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:51 smithi016 ceph-mon[35851]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T08:40:51.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:51 smithi044 ceph-mon[36461]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T08:40:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.007+0000 7f572085c700 1 -- 172.21.15.7:0/2729679183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f571c06ad70 msgr2=0x7f571c06b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.007+0000 7f572085c700 1 --2- 172.21.15.7:0/2729679183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f571c06ad70 0x7f571c06b1d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5710009990 tx=0x7f5710009ca0 comp rx=0 tx=0).stop 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.008+0000 7f572085c700 1 -- 172.21.15.7:0/2729679183 shutdown_connections 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.008+0000 7f572085c700 1 --2- 172.21.15.7:0/2729679183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f571c10ea70 0x7f571c110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.008+0000 7f572085c700 1 --2- 172.21.15.7:0/2729679183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571c1030f0 0x7f571c06a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.008+0000 7f572085c700 1 --2- 172.21.15.7:0/2729679183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f571c06ad70 0x7f571c06b1d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.008+0000 7f572085c700 1 -- 172.21.15.7:0/2729679183 >> 172.21.15.7:0/2729679183 conn(0x7f571c077f50 msgr2=0x7f571c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.008+0000 7f572085c700 1 -- 172.21.15.7:0/2729679183 shutdown_connections 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.008+0000 7f572085c700 1 -- 172.21.15.7:0/2729679183 wait complete. 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.008+0000 7f572085c700 1 Processor -- start 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.008+0000 7f572085c700 1 -- start start 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f572085c700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f571c06ad70 0x7f571c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f572085c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f571c1030f0 0x7f571c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f572085c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571c10ea70 0x7f571c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f572085c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c1146f0 con 0x7f571c1030f0 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f572085c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c114570 con 0x7f571c10ea70 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f572085c700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c114870 con 0x7f571c06ad70 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f5719d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f571c1030f0 0x7f571c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f5719d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f571c1030f0 0x7f571c1a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39560/0 (socket says 172.21.15.7:39560) 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f5719d9b700 1 -- 172.21.15.7:0/3917313649 learned_addr learned my addr 172.21.15.7:0/3917313649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f571ad9d700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571c10ea70 0x7f571c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f571a59c700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f571c06ad70 0x7f571c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f5719d9b700 1 -- 172.21.15.7:0/3917313649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f571c06ad70 msgr2=0x7f571c1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f5719d9b700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f571c06ad70 0x7f571c1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f5719d9b700 1 -- 172.21.15.7:0/3917313649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571c10ea70 msgr2=0x7f571c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f5719d9b700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571c10ea70 0x7f571c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f5719d9b700 1 -- 172.21.15.7:0/3917313649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c19b0d0 con 0x7f571c1030f0 2024-03-26T08:40:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f571ad9d700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571c10ea70 0x7f571c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f5719d9b700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f571c1030f0 0x7f571c1a0ca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5710000c00 tx=0x7f571000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f570b7fe700 1 -- 172.21.15.7:0/3917313649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710027070 con 0x7f571c1030f0 2024-03-26T08:40:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c19b360 con 0x7f571c1030f0 2024-03-26T08:40:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f570b7fe700 1 -- 172.21.15.7:0/3917313649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5710003d70 con 0x7f571c1030f0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c1a7630 con 0x7f571c1030f0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.009+0000 7f570b7fe700 1 -- 172.21.15.7:0/3917313649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000f460 con 0x7f571c1030f0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.010+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f571c100dc0 con 0x7f571c1030f0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.012+0000 7f570b7fe700 1 -- 172.21.15.7:0/3917313649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f571002a070 con 0x7f571c1030f0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.012+0000 7f570b7fe700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57040a2920 0x7f57040a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.012+0000 7f571a59c700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57040a2920 0x7f57040a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.012+0000 7f570b7fe700 1 -- 172.21.15.7:0/3917313649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f57100cfdd0 con 0x7f571c1030f0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.012+0000 7f570b7fe700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] conn(0x7f57040ac100 0x7f57040ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.012+0000 7f570b7fe700 1 -- 172.21.15.7:0/3917313649 --> [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57100056d0 con 0x7f57040ac100 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.012+0000 7f570b7fe700 1 -- 172.21.15.7:0/3917313649 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57100d02b0 con 0x7f571c1030f0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.013+0000 7f571ad9d700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] conn(0x7f57040ac100 0x7f57040ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.013+0000 7f571a59c700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57040a2920 0x7f57040a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f571c102490 tx=0x7f570c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.013+0000 7f571ad9d700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] conn(0x7f57040ac100 0x7f57040ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.014+0000 7f570b7fe700 1 -- 172.21.15.7:0/3917313649 <== osd.2 v2:172.21.15.44:6800/2047416654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f57100056d0 con 0x7f57040ac100 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.030+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 --> [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f571c1016f0 con 0x7f57040ac100 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.030+0000 7f570b7fe700 1 -- 172.21.15.7:0/3917313649 <== osd.2 v2:172.21.15.44:6800/2047416654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f571c1016f0 con 0x7f57040ac100 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 >> [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] conn(0x7f57040ac100 msgr2=0x7f57040ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] conn(0x7f57040ac100 0x7f57040ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57040a2920 msgr2=0x7f57040a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57040a2920 0x7f57040a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f571c102490 tx=0x7f570c009260 comp rx=0 tx=0).stop 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f571c1030f0 msgr2=0x7f571c1a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 --2- 172.21.15.7:0/3917313649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f571c1030f0 0x7f571c1a0ca0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5710000c00 tx=0x7f571000bab0 comp rx=0 tx=0).stop 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f571ad9d700 1 -- 172.21.15.7:0/3917313649 reap_dead start 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 shutdown_connections 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 >> 172.21.15.7:0/3917313649 conn(0x7f571c077f50 msgr2=0x7f571c100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 shutdown_connections 2024-03-26T08:40:52.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:52.031+0000 7f572085c700 1 -- 172.21.15.7:0/3917313649 wait complete. 2024-03-26T08:40:52.853 INFO:teuthology.orchestra.run.smithi007.stdout:77309411397 2024-03-26T08:40:52.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-26T08:40:53.488 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:53 smithi007 ceph-mon[32462]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-26T08:40:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/1620567056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e68101d30 msgr2=0x7f7e6810df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 --2- 172.21.15.7:0/1620567056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e68101d30 0x7f7e6810df60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b0a0 tx=0x7f7e6000b3b0 comp rx=0 tx=0).stop 2024-03-26T08:40:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/1620567056 shutdown_connections 2024-03-26T08:40:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 --2- 172.21.15.7:0/1620567056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e68101d30 0x7f7e6810df60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 --2- 172.21.15.7:0/1620567056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68107280 0x7f7e68107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 --2- 172.21.15.7:0/1620567056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e68101370 0x7f7e681017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/1620567056 >> 172.21.15.7:0/1620567056 conn(0x7f7e68077ef0 msgr2=0x7f7e68078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/1620567056 shutdown_connections 2024-03-26T08:40:53.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/1620567056 wait complete. 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.497+0000 7f7e6d5f6700 1 Processor -- start 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e6d5f6700 1 -- start start 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e6d5f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e68101370 0x7f7e6819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e6d5f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68101d30 0x7f7e6819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e6d5f6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e68107280 0x7f7e68197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e6d5f6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68111410 con 0x7f7e68101370 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e6d5f6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68111290 con 0x7f7e68101d30 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e6d5f6700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68111590 con 0x7f7e68107280 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e66ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e68101370 0x7f7e6819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e66ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e68101370 0x7f7e6819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39564/0 (socket says 172.21.15.7:39564) 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2525178590 learned_addr learned my addr 172.21.15.7:0/2525178590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e667fc700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68101d30 0x7f7e6819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e677fe700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e68107280 0x7f7e68197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2525178590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e68107280 msgr2=0x7f7e68197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:53.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e66ffd700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7e68107280 0x7f7e68197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2525178590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68101d30 msgr2=0x7f7e6819dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e66ffd700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68101d30 0x7f7e6819dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2525178590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68197f30 con 0x7f7e68101370 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.498+0000 7f7e66ffd700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e68101370 0x7f7e6819d580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800e9a0 tx=0x7f7e5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.499+0000 7f7e4ffff700 1 -- 172.21.15.7:0/2525178590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800ca10 con 0x7f7e68101370 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.499+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e68198220 con 0x7f7e68101370 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.499+0000 7f7e4ffff700 1 -- 172.21.15.7:0/2525178590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e5800cbb0 con 0x7f7e68101370 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.499+0000 7f7e4ffff700 1 -- 172.21.15.7:0/2525178590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58018680 con 0x7f7e68101370 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.499+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e681a4500 con 0x7f7e68101370 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.499+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e68062820 con 0x7f7e68101370 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.501+0000 7f7e4ffff700 1 -- 172.21.15.7:0/2525178590 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f7e58010430 con 0x7f7e68101370 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.501+0000 7f7e4ffff700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e500a2920 0x7f7e500a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.501+0000 7f7e4ffff700 1 -- 172.21.15.7:0/2525178590 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7e580cf3f0 con 0x7f7e68101370 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.501+0000 7f7e667fc700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e500a2920 0x7f7e500a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.502+0000 7f7e4ffff700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] conn(0x7f7e500ac100 0x7f7e500ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.502+0000 7f7e4ffff700 1 -- 172.21.15.7:0/2525178590 --> [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e5801bec0 con 0x7f7e500ac100 2024-03-26T08:40:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.502+0000 7f7e4ffff700 1 -- 172.21.15.7:0/2525178590 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e580cf8d0 con 0x7f7e68101370 2024-03-26T08:40:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.502+0000 7f7e677fe700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] conn(0x7f7e500ac100 0x7f7e500ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.502+0000 7f7e667fc700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e500a2920 0x7f7e500a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e68198e10 tx=0x7f7e5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.502+0000 7f7e677fe700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] conn(0x7f7e500ac100 0x7f7e500ae520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.503+0000 7f7e4ffff700 1 -- 172.21.15.7:0/2525178590 <== osd.11 v2:172.21.15.44:6824/2818540789 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7e5801bec0 con 0x7f7e500ac100 2024-03-26T08:40:53.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.522+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 --> [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e68062c50 con 0x7f7e500ac100 2024-03-26T08:40:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.522+0000 7f7e4ffff700 1 -- 172.21.15.7:0/2525178590 <== osd.11 v2:172.21.15.44:6824/2818540789 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7e68062c50 con 0x7f7e500ac100 2024-03-26T08:40:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.522+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 >> [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] conn(0x7f7e500ac100 msgr2=0x7f7e500ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.522+0000 7f7e6d5f6700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] conn(0x7f7e500ac100 0x7f7e500ae520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.523+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e500a2920 msgr2=0x7f7e500a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.523+0000 7f7e6d5f6700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e500a2920 0x7f7e500a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e68198e10 tx=0x7f7e5c009260 comp rx=0 tx=0).stop 2024-03-26T08:40:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.523+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e68101370 msgr2=0x7f7e6819d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.523+0000 7f7e6d5f6700 1 --2- 172.21.15.7:0/2525178590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e68101370 0x7f7e6819d580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800e9a0 tx=0x7f7e5800ecb0 comp rx=0 tx=0).stop 2024-03-26T08:40:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.523+0000 7f7e677fe700 1 -- 172.21.15.7:0/2525178590 reap_dead start 2024-03-26T08:40:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.523+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 shutdown_connections 2024-03-26T08:40:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.523+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 >> 172.21.15.7:0/2525178590 conn(0x7f7e68077ef0 msgr2=0x7f7e680ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.523+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 shutdown_connections 2024-03-26T08:40:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:53.523+0000 7f7e6d5f6700 1 -- 172.21.15.7:0/2525178590 wait complete. 2024-03-26T08:40:53.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:53 smithi044 ceph-mon[36461]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-26T08:40:53.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:53 smithi016 ceph-mon[35851]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-03-26T08:40:54.584 INFO:teuthology.orchestra.run.smithi007.stdout:343597383701 2024-03-26T08:40:54.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-26T08:40:55.186 INFO:teuthology.orchestra.run.smithi007.stdout:309237645339 2024-03-26T08:40:55.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-26T08:40:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.411+0000 7ff8421d0700 1 -- 172.21.15.7:0/279765805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c102210 msgr2=0x7ff83c102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.411+0000 7ff8421d0700 1 --2- 172.21.15.7:0/279765805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c102210 0x7ff83c102690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff824009990 tx=0x7ff824009ca0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.411+0000 7ff8421d0700 1 -- 172.21.15.7:0/279765805 shutdown_connections 2024-03-26T08:40:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.411+0000 7ff8421d0700 1 --2- 172.21.15.7:0/279765805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff83c1082f0 0x7ff83c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.411+0000 7ff8421d0700 1 --2- 172.21.15.7:0/279765805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83c102bd0 0x7ff83c10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.411+0000 7ff8421d0700 1 --2- 172.21.15.7:0/279765805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c102210 0x7ff83c102690 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.411+0000 7ff8421d0700 1 -- 172.21.15.7:0/279765805 >> 172.21.15.7:0/279765805 conn(0x7ff83c079250 msgr2=0x7ff83c079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:55.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.411+0000 7ff8421d0700 1 -- 172.21.15.7:0/279765805 shutdown_connections 2024-03-26T08:40:55.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.411+0000 7ff8421d0700 1 -- 172.21.15.7:0/279765805 wait complete. 2024-03-26T08:40:55.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff8421d0700 1 Processor -- start 2024-03-26T08:40:55.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff8421d0700 1 -- start start 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff8421d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83c102210 0x7ff83c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff8421d0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff83c102bd0 0x7ff83c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff8421d0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c1082f0 0x7ff83c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff8421d0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c112470 con 0x7ff83c1082f0 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff8421d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c1122f0 con 0x7ff83c102210 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff8421d0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c1125f0 con 0x7ff83c102bd0 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c1082f0 0x7ff83c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff83affd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff83c102bd0 0x7ff83c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff83bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c1082f0 0x7ff83c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39586/0 (socket says 172.21.15.7:39586) 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff83b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83c102210 0x7ff83c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.412+0000 7ff83bfff700 1 -- 172.21.15.7:0/3538415958 learned_addr learned my addr 172.21.15.7:0/3538415958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff83affd700 1 -- 172.21.15.7:0/3538415958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83c102210 msgr2=0x7ff83c19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff83affd700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83c102210 0x7ff83c19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff83affd700 1 -- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c1082f0 msgr2=0x7ff83c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff83affd700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c1082f0 0x7ff83c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff83affd700 1 -- 172.21.15.7:0/3538415958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c198f90 con 0x7ff83c102bd0 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83c102210 0x7ff83c19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff83bfff700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff83c1082f0 0x7ff83c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff83affd700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff83c102bd0 0x7ff83c19eb70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff8240059d0 tx=0x7ff82400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff838ff9700 1 -- 172.21.15.7:0/3538415958 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824027070 con 0x7ff83c102bd0 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff838ff9700 1 -- 172.21.15.7:0/3538415958 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff824003d70 con 0x7ff83c102bd0 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff838ff9700 1 -- 172.21.15.7:0/3538415958 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824022e30 con 0x7ff83c102bd0 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83c199220 con 0x7ff83c102bd0 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.413+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83c0743c0 con 0x7ff83c102bd0 2024-03-26T08:40:55.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.414+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff83c0628a0 con 0x7ff83c102bd0 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.415+0000 7ff838ff9700 1 -- 172.21.15.7:0/3538415958 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7ff82402a070 con 0x7ff83c102bd0 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.416+0000 7ff838ff9700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff8280a6db0 0x7ff8280a9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.416+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff8280a6db0 0x7ff8280a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.416+0000 7ff838ff9700 1 -- 172.21.15.7:0/3538415958 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff8240cf930 con 0x7ff83c102bd0 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.416+0000 7ff838ff9700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] conn(0x7ff8280b0590 0x7ff8280b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.416+0000 7ff838ff9700 1 -- 172.21.15.7:0/3538415958 --> [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8280b3060 con 0x7ff8280b0590 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.416+0000 7ff838ff9700 1 -- 172.21.15.7:0/3538415958 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8240cfd90 con 0x7ff83c102bd0 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.417+0000 7ff83bfff700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] conn(0x7ff8280b0590 0x7ff8280b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.417+0000 7ff83b7fe700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff8280a6db0 0x7ff8280a9270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff82c003e30 tx=0x7ff82c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.417+0000 7ff83bfff700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] conn(0x7ff8280b0590 0x7ff8280b29b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:55.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.418+0000 7ff838ff9700 1 -- 172.21.15.7:0/3538415958 <== osd.0 v2:172.21.15.7:6802/2166020682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff8280b3060 con 0x7ff8280b0590 2024-03-26T08:40:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.435+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 --> [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff83c062d70 con 0x7ff8280b0590 2024-03-26T08:40:55.426 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:55 smithi007 ceph-mon[32462]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T08:40:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.435+0000 7ff838ff9700 1 -- 172.21.15.7:0/3538415958 <== osd.0 v2:172.21.15.7:6802/2166020682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff83c062d70 con 0x7ff8280b0590 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] conn(0x7ff8280b0590 msgr2=0x7ff8280b29b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] conn(0x7ff8280b0590 0x7ff8280b29b0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff8280a6db0 msgr2=0x7ff8280a9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff8280a6db0 0x7ff8280a9270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff82c003e30 tx=0x7ff82c00b040 comp rx=0 tx=0).stop 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff83c102bd0 msgr2=0x7ff83c19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 --2- 172.21.15.7:0/3538415958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff83c102bd0 0x7ff83c19eb70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff8240059d0 tx=0x7ff82400bab0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff83bfff700 1 -- 172.21.15.7:0/3538415958 reap_dead start 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 shutdown_connections 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 >> 172.21.15.7:0/3538415958 conn(0x7ff83c079250 msgr2=0x7ff83c100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 shutdown_connections 2024-03-26T08:40:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.436+0000 7ff8421d0700 1 -- 172.21.15.7:0/3538415958 wait complete. 2024-03-26T08:40:55.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:55 smithi016 ceph-mon[35851]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 -- 172.21.15.7:0/2990588115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41074b0 msgr2=0x7f13b410f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 --2- 172.21.15.7:0/2990588115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41074b0 0x7f13b410f080 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00b0d0 tx=0x7f13ac00b3e0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 -- 172.21.15.7:0/2990588115 shutdown_connections 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 --2- 172.21.15.7:0/2990588115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13b41023d0 0x7f13b41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 --2- 172.21.15.7:0/2990588115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41074b0 0x7f13b410f080 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 --2- 172.21.15.7:0/2990588115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b4102d80 0x7f13b4106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 -- 172.21.15.7:0/2990588115 >> 172.21.15.7:0/2990588115 conn(0x7f13b40fdc40 msgr2=0x7f13b4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 -- 172.21.15.7:0/2990588115 shutdown_connections 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 -- 172.21.15.7:0/2990588115 wait complete. 2024-03-26T08:40:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.758+0000 7f13bc694700 1 Processor -- start 2024-03-26T08:40:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bc694700 1 -- start start 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bc694700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b41023d0 0x7f13b41a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bc694700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13b4102d80 0x7f13b41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bc694700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41074b0 0x7f13b419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bc694700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b41124f0 con 0x7f13b41074b0 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bc694700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4112370 con 0x7f13b41023d0 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bc694700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4112670 con 0x7f13b4102d80 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bac31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41074b0 0x7f13b419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13ba430700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b41023d0 0x7f13b41a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13ba430700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b41023d0 0x7f13b41a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55340/0 (socket says 172.21.15.7:55340) 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13b9c2f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13b4102d80 0x7f13b41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13ba430700 1 -- 172.21.15.7:0/2015508447 learned_addr learned my addr 172.21.15.7:0/2015508447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bac31700 1 -- 172.21.15.7:0/2015508447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13b4102d80 msgr2=0x7f13b41a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bac31700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13b4102d80 0x7f13b41a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bac31700 1 -- 172.21.15.7:0/2015508447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b41023d0 msgr2=0x7f13b41a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bac31700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b41023d0 0x7f13b41a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bac31700 1 -- 172.21.15.7:0/2015508447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b419b080 con 0x7f13b41074b0 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13ba430700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b41023d0 0x7f13b41a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.759+0000 7f13bac31700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41074b0 0x7f13b419a920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00d010 tx=0x7f13ac009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.760+0000 7f13ab7fe700 1 -- 172.21.15.7:0/2015508447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac003a00 con 0x7f13b41074b0 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.760+0000 7f13ab7fe700 1 -- 172.21.15.7:0/2015508447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13ac022070 con 0x7f13b41074b0 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.760+0000 7f13ab7fe700 1 -- 172.21.15.7:0/2015508447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac01b6c0 con 0x7f13b41074b0 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.760+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b419b310 con 0x7f13b41074b0 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.760+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b41a7630 con 0x7f13b41074b0 2024-03-26T08:40:55.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.761+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13b4075720 con 0x7f13b41074b0 2024-03-26T08:40:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.762+0000 7f13ab7fe700 1 -- 172.21.15.7:0/2015508447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f13ac003ba0 con 0x7f13b41074b0 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.762+0000 7f13ab7fe700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13a00a29a0 0x7f13a00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.762+0000 7f13ba430700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13a00a29a0 0x7f13a00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.762+0000 7f13ab7fe700 1 -- 172.21.15.7:0/2015508447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f13ac00f020 con 0x7f13b41074b0 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.763+0000 7f13ab7fe700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] conn(0x7f13a00ac180 0x7f13a00ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.763+0000 7f13ab7fe700 1 -- 172.21.15.7:0/2015508447 --> [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13a00aec50 con 0x7f13a00ac180 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.763+0000 7f13ab7fe700 1 -- 172.21.15.7:0/2015508447 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13ac0cff40 con 0x7f13b41074b0 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.763+0000 7f13b9c2f700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] conn(0x7f13a00ac180 0x7f13a00ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.763+0000 7f13ba430700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13a00a29a0 0x7f13a00a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f13b419ba60 tx=0x7f13b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.763+0000 7f13b9c2f700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] conn(0x7f13a00ac180 0x7f13a00ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.764+0000 7f13ab7fe700 1 -- 172.21.15.7:0/2015508447 <== osd.1 v2:172.21.15.16:6800/1464490925 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f13a00aec50 con 0x7f13a00ac180 2024-03-26T08:40:55.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.794+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 --> [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f13b41038b0 con 0x7f13a00ac180 2024-03-26T08:40:55.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13ab7fe700 1 -- 172.21.15.7:0/2015508447 <== osd.1 v2:172.21.15.16:6800/1464490925 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f13b41038b0 con 0x7f13a00ac180 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 >> [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] conn(0x7f13a00ac180 msgr2=0x7f13a00ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] conn(0x7f13a00ac180 0x7f13a00ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13a00a29a0 msgr2=0x7f13a00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13a00a29a0 0x7f13a00a4e60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f13b419ba60 tx=0x7f13b0009260 comp rx=0 tx=0).stop 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41074b0 msgr2=0x7f13b419a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 --2- 172.21.15.7:0/2015508447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b41074b0 0x7f13b419a920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00d010 tx=0x7f13ac009810 comp rx=0 tx=0).stop 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bac31700 1 -- 172.21.15.7:0/2015508447 reap_dead start 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 shutdown_connections 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 >> 172.21.15.7:0/2015508447 conn(0x7f13b40fdc40 msgr2=0x7f13b410d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 shutdown_connections 2024-03-26T08:40:55.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.795+0000 7f13bc694700 1 -- 172.21.15.7:0/2015508447 wait complete. 2024-03-26T08:40:55.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:55 smithi044 ceph-mon[36461]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.936+0000 7f861b273700 1 -- 172.21.15.7:0/2778021051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614101140 msgr2=0x7f861410f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.936+0000 7f861b273700 1 --2- 172.21.15.7:0/2778021051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614101140 0x7f861410f0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f860c009f90 tx=0x7f860c00b330 comp rx=0 tx=0).stop 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.936+0000 7f861b273700 1 -- 172.21.15.7:0/2778021051 shutdown_connections 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.936+0000 7f861b273700 1 --2- 172.21.15.7:0/2778021051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614101140 0x7f861410f0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.936+0000 7f861b273700 1 --2- 172.21.15.7:0/2778021051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8614107200 0x7f86141075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.936+0000 7f861b273700 1 --2- 172.21.15.7:0/2778021051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614100780 0x7f8614100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.936+0000 7f861b273700 1 -- 172.21.15.7:0/2778021051 >> 172.21.15.7:0/2778021051 conn(0x7f8614077690 msgr2=0x7f86140feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.936+0000 7f861b273700 1 -- 172.21.15.7:0/2778021051 shutdown_connections 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.936+0000 7f861b273700 1 -- 172.21.15.7:0/2778021051 wait complete. 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861b273700 1 Processor -- start 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861b273700 1 -- start start 2024-03-26T08:40:55.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861b273700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8614100780 0x7f861419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861b273700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614101140 0x7f861419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861b273700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614107200 0x7f8614198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861b273700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86141125e0 con 0x7f8614101140 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861b273700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8614112460 con 0x7f8614100780 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861b273700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8614112760 con 0x7f8614107200 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861880e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614101140 0x7f861419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861880e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614101140 0x7f861419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39620/0 (socket says 172.21.15.7:39620) 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861880e700 1 -- 172.21.15.7:0/2048014897 learned_addr learned my addr 172.21.15.7:0/2048014897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f8619810700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614107200 0x7f8614198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861900f700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8614100780 0x7f861419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861880e700 1 -- 172.21.15.7:0/2048014897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614107200 msgr2=0x7f8614198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861880e700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8614107200 0x7f8614198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861880e700 1 -- 172.21.15.7:0/2048014897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8614100780 msgr2=0x7f861419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861880e700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8614100780 0x7f861419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861880e700 1 -- 172.21.15.7:0/2048014897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8614198fe0 con 0x7f8614101140 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.937+0000 7f861900f700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8614100780 0x7f861419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.938+0000 7f861880e700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614101140 0x7f861419ebc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f860400ec10 tx=0x7f860400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.938+0000 7f860a7fc700 1 -- 172.21.15.7:0/2048014897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860400cc90 con 0x7f8614101140 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.938+0000 7f860a7fc700 1 -- 172.21.15.7:0/2048014897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f860400ce30 con 0x7f8614101140 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.938+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86141992d0 con 0x7f8614101140 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.938+0000 7f860a7fc700 1 -- 172.21.15.7:0/2048014897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604010690 con 0x7f8614101140 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.938+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86141a54e0 con 0x7f8614101140 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.939+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f86140628a0 con 0x7f8614101140 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.940+0000 7f860a7fc700 1 -- 172.21.15.7:0/2048014897 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f8604003680 con 0x7f8614101140 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.940+0000 7f860a7fc700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86000a2a20 0x7f86000a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.940+0000 7f861900f700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86000a2a20 0x7f86000a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.940+0000 7f860a7fc700 1 -- 172.21.15.7:0/2048014897 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f86040cf0e0 con 0x7f8614101140 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.941+0000 7f860a7fc700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] conn(0x7f86000ac200 0x7f86000ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:40:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.941+0000 7f860a7fc700 1 -- 172.21.15.7:0/2048014897 --> [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8604012ec0 con 0x7f86000ac200 2024-03-26T08:40:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.941+0000 7f860a7fc700 1 -- 172.21.15.7:0/2048014897 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86040cf4d0 con 0x7f8614101140 2024-03-26T08:40:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.941+0000 7f8619810700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] conn(0x7f86000ac200 0x7f86000ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:40:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.941+0000 7f861900f700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86000a2a20 0x7f86000a4ee0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8610011490 tx=0x7f8610009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.941+0000 7f8619810700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] conn(0x7f86000ac200 0x7f86000ae620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:40:55.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.942+0000 7f860a7fc700 1 -- 172.21.15.7:0/2048014897 <== osd.3 v2:172.21.15.7:6810/4236464716 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8604012ec0 con 0x7f86000ac200 2024-03-26T08:40:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.962+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 --> [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8614062ca0 con 0x7f86000ac200 2024-03-26T08:40:55.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.962+0000 7f860a7fc700 1 -- 172.21.15.7:0/2048014897 <== osd.3 v2:172.21.15.7:6810/4236464716 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8614062ca0 con 0x7f86000ac200 2024-03-26T08:40:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] conn(0x7f86000ac200 msgr2=0x7f86000ae620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] conn(0x7f86000ac200 0x7f86000ae620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:40:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86000a2a20 msgr2=0x7f86000a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86000a2a20 0x7f86000a4ee0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8610011490 tx=0x7f8610009260 comp rx=0 tx=0).stop 2024-03-26T08:40:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614101140 msgr2=0x7f861419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:40:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 --2- 172.21.15.7:0/2048014897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8614101140 0x7f861419ebc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f860400ec10 tx=0x7f860400ef20 comp rx=0 tx=0).stop 2024-03-26T08:40:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f8619810700 1 -- 172.21.15.7:0/2048014897 reap_dead start 2024-03-26T08:40:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 shutdown_connections 2024-03-26T08:40:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 >> 172.21.15.7:0/2048014897 conn(0x7f8614077690 msgr2=0x7f8614105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:40:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 shutdown_connections 2024-03-26T08:40:55.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:40:55.963+0000 7f861b273700 1 -- 172.21.15.7:0/2048014897 wait complete. 2024-03-26T08:40:56.836 INFO:teuthology.orchestra.run.smithi007.stdout:219043332138 2024-03-26T08:40:56.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-26T08:40:57.064 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:57.478 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:57 smithi007 ceph-mon[32462]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-26T08:40:57.502 INFO:teuthology.orchestra.run.smithi007.stdout:55834574922 2024-03-26T08:40:57.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-26T08:40:57.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:57 smithi016 ceph-mon[35851]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-26T08:40:57.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:57 smithi044 ceph-mon[36461]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-03-26T08:40:58.051 INFO:teuthology.orchestra.run.smithi007.stdout:107374182464 2024-03-26T08:40:58.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-26T08:40:58.349 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:58.360 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:58.420 INFO:teuthology.orchestra.run.smithi007.stdout:34359738447 2024-03-26T08:40:58.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-26T08:40:58.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-26T08:40:58.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-26T08:40:58.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-03-26T08:40:58.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]: dispatch 2024-03-26T08:40:58.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-26T08:40:58.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-26T08:40:58.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-26T08:40:58.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-26T08:40:58.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:58 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-26T08:40:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-26T08:40:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-26T08:40:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-03-26T08:40:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]: dispatch 2024-03-26T08:40:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-26T08:40:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-26T08:40:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-26T08:40:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-26T08:40:58.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:58 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-26T08:40:58.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-03-26T08:40:58.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-26T08:40:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-03-26T08:40:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]: dispatch 2024-03-26T08:40:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-26T08:40:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-03-26T08:40:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-26T08:40:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-03-26T08:40:58.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:58 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-26T08:40:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:59.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T08:40:59.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-26T08:40:59.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-26T08:40:59.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-03-26T08:40:59.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]': finished 2024-03-26T08:40:59.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-26T08:40:59.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-26T08:40:59.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-26T08:40:59.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-26T08:40:59.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-26T08:40:59.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:40:59 smithi007 ceph-mon[32462]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T08:40:59.613 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:59.718 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:40:59.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]': finished 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-26T08:40:59.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:40:59 smithi016 ceph-mon[35851]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]': finished 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-26T08:40:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:40:59 smithi044 ceph-mon[36461]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T08:41:00.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:00 smithi016 ceph-mon[35851]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T08:41:00.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:00 smithi007 ceph-mon[32462]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T08:41:00.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:00 smithi044 ceph-mon[36461]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T08:41:01.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:01 smithi007 ceph-mon[32462]: pgmap v318: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-03-26T08:41:01.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:01 smithi016 ceph-mon[35851]: pgmap v318: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-03-26T08:41:01.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:01 smithi044 ceph-mon[36461]: pgmap v318: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3 op/s 2024-03-26T08:41:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.331+0000 7f3599893700 1 -- 172.21.15.7:0/1829118371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35941012f0 msgr2=0x7f3594106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.331+0000 7f3599893700 1 --2- 172.21.15.7:0/1829118371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35941012f0 0x7f3594106e30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f35840099c0 tx=0x7f3584009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.331+0000 7f3599893700 1 -- 172.21.15.7:0/1829118371 shutdown_connections 2024-03-26T08:41:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.331+0000 7f3599893700 1 --2- 172.21.15.7:0/1829118371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3594100940 0x7f3594100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.331+0000 7f3599893700 1 --2- 172.21.15.7:0/1829118371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3594107560 0x7f359410f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.331+0000 7f3599893700 1 --2- 172.21.15.7:0/1829118371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35941012f0 0x7f3594106e30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.331+0000 7f3599893700 1 -- 172.21.15.7:0/1829118371 >> 172.21.15.7:0/1829118371 conn(0x7f3594077690 msgr2=0x7f35940fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.331+0000 7f3599893700 1 -- 172.21.15.7:0/1829118371 shutdown_connections 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.331+0000 7f3599893700 1 -- 172.21.15.7:0/1829118371 wait complete. 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3599893700 1 Processor -- start 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3599893700 1 -- start start 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3599893700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3594100940 0x7f35941018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3599893700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35941012f0 0x7f3594101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3599893700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3594107560 0x7f3594102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3599893700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594112450 con 0x7f3594100940 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3599893700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35941122d0 con 0x7f3594107560 2024-03-26T08:41:03.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3599893700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35941125d0 con 0x7f35941012f0 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3592ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3594100940 0x7f35941018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3592ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3594100940 0x7f35941018c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48596/0 (socket says 172.21.15.7:48596) 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f3592ffd700 1 -- 172.21.15.7:0/3261334862 learned_addr learned my addr 172.21.15.7:0/3261334862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f35927fc700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35941012f0 0x7f3594101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.332+0000 7f35937fe700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3594107560 0x7f3594102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3592ffd700 1 -- 172.21.15.7:0/3261334862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35941012f0 msgr2=0x7f3594101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3592ffd700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35941012f0 0x7f3594101e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3592ffd700 1 -- 172.21.15.7:0/3261334862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3594107560 msgr2=0x7f3594102340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3592ffd700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3594107560 0x7f3594102340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3592ffd700 1 -- 172.21.15.7:0/3261334862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35941a6dc0 con 0x7f3594100940 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3592ffd700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3594100940 0x7f35941018c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f357c00d980 tx=0x7f357c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3598891700 1 -- 172.21.15.7:0/3261334862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c016070 con 0x7f3594100940 2024-03-26T08:41:03.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3598891700 1 -- 172.21.15.7:0/3261334862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f357c004d10 con 0x7f3594100940 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35941a70b0 con 0x7f3594100940 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3598891700 1 -- 172.21.15.7:0/3261334862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c005670 con 0x7f3594100940 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.333+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35941a7610 con 0x7f3594100940 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.334+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3594068e50 con 0x7f3594100940 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.335+0000 7f3598891700 1 -- 172.21.15.7:0/3261334862 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f357c004750 con 0x7f3594100940 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.336+0000 7f3598891700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35800a29a0 0x7f35800a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.336+0000 7f35927fc700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35800a29a0 0x7f35800a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.336+0000 7f3598891700 1 -- 172.21.15.7:0/3261334862 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f357c0ce960 con 0x7f3594100940 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.336+0000 7f35927fc700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35800a29a0 0x7f35800a4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3594103300 tx=0x7f3584005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:03.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.340+0000 7f3598891700 1 -- 172.21.15.7:0/3261334862 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f357c01f020 con 0x7f3594100940 2024-03-26T08:41:03.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.549+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f359402da30 con 0x7f3594100940 2024-03-26T08:41:03.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.550+0000 7f3598891700 1 -- 172.21.15.7:0/3261334862 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f357c022350 con 0x7f3594100940 2024-03-26T08:41:03.541 INFO:teuthology.orchestra.run.smithi007.stdout:253403070504 2024-03-26T08:41:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35800a29a0 msgr2=0x7f35800a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35800a29a0 0x7f35800a4e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3594103300 tx=0x7f3584005fb0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3594100940 msgr2=0x7f35941018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3594100940 0x7f35941018c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f357c00d980 tx=0x7f357c00dc90 comp rx=0 tx=0).stop 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 shutdown_connections 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35800a29a0 0x7f35800a4e60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3594107560 0x7f3594102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3594100940 0x7f35941018c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 --2- 172.21.15.7:0/3261334862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35941012f0 0x7f3594101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 >> 172.21.15.7:0/3261334862 conn(0x7f3594077690 msgr2=0x7f35940fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 shutdown_connections 2024-03-26T08:41:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:03.552+0000 7f3599893700 1 -- 172.21.15.7:0/3261334862 wait complete. 2024-03-26T08:41:03.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:03 smithi007 ceph-mon[32462]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T08:41:03.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:03 smithi016 ceph-mon[35851]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T08:41:03.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:03 smithi044 ceph-mon[36461]: pgmap v319: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T08:41:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.398+0000 7f590250a700 1 -- 172.21.15.7:0/134203330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 msgr2=0x7f58fc06aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.398+0000 7f590250a700 1 --2- 172.21.15.7:0/134203330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 0x7f58fc06aeb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f58ec0099c0 tx=0x7f58ec009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.398+0000 7f590250a700 1 -- 172.21.15.7:0/134203330 shutdown_connections 2024-03-26T08:41:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.398+0000 7f590250a700 1 --2- 172.21.15.7:0/134203330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc107540 0x7f58fc10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.398+0000 7f590250a700 1 --2- 172.21.15.7:0/134203330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 0x7f58fc06aeb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.398+0000 7f590250a700 1 --2- 172.21.15.7:0/134203330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc06b480 0x7f58fc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.398+0000 7f590250a700 1 -- 172.21.15.7:0/134203330 >> 172.21.15.7:0/134203330 conn(0x7f58fc0770e0 msgr2=0x7f58fc0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.398+0000 7f590250a700 1 -- 172.21.15.7:0/134203330 shutdown_connections 2024-03-26T08:41:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.398+0000 7f590250a700 1 -- 172.21.15.7:0/134203330 wait complete. 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f590250a700 1 Processor -- start 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f590250a700 1 -- start start 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f590250a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 0x7f58fc1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f590250a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc06b480 0x7f58fc1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f590250a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc107540 0x7f58fc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f590250a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc1124f0 con 0x7f58fc06aad0 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f590250a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc112370 con 0x7f58fc107540 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f590250a700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc112670 con 0x7f58fc06b480 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f58fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 0x7f58fc1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f58fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 0x7f58fc1a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48614/0 (socket says 172.21.15.7:48614) 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f58fb7fe700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc06b480 0x7f58fc1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f58fbfff700 1 -- 172.21.15.7:0/965593701 learned_addr learned my addr 172.21.15.7:0/965593701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.399+0000 7f5900aa7700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc107540 0x7f58fc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f58fbfff700 1 -- 172.21.15.7:0/965593701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc06b480 msgr2=0x7f58fc1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f58fbfff700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc06b480 0x7f58fc1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f58fbfff700 1 -- 172.21.15.7:0/965593701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc107540 msgr2=0x7f58fc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f58fbfff700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc107540 0x7f58fc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f58fbfff700 1 -- 172.21.15.7:0/965593701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc19b0e0 con 0x7f58fc06aad0 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f58fbfff700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 0x7f58fc1a07c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58ec006010 tx=0x7f58ec00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f58f97fa700 1 -- 172.21.15.7:0/965593701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec003680 con 0x7f58fc06aad0 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58fc19b370 con 0x7f58fc06aad0 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f58f97fa700 1 -- 172.21.15.7:0/965593701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f58ec0050d0 con 0x7f58fc06aad0 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f58f97fa700 1 -- 172.21.15.7:0/965593701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec003db0 con 0x7f58fc06aad0 2024-03-26T08:41:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.400+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58fc1a7690 con 0x7f58fc06aad0 2024-03-26T08:41:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.401+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58fc06a0a0 con 0x7f58fc06aad0 2024-03-26T08:41:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.402+0000 7f58f97fa700 1 -- 172.21.15.7:0/965593701 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f58ec005280 con 0x7f58fc06aad0 2024-03-26T08:41:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.402+0000 7f58f97fa700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58e40a2ac0 0x7f58e40a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.403+0000 7f58fb7fe700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58e40a2ac0 0x7f58e40a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.403+0000 7f58f97fa700 1 -- 172.21.15.7:0/965593701 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f58ec0cf900 con 0x7f58fc06aad0 2024-03-26T08:41:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.403+0000 7f58fb7fe700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58e40a2ac0 0x7f58e40a4f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f58fc10f010 tx=0x7f58f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.407+0000 7f58f97fa700 1 -- 172.21.15.7:0/965593701 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f58ec013070 con 0x7f58fc06aad0 2024-03-26T08:41:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.630+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f58fc1038f0 con 0x7f58fc06aad0 2024-03-26T08:41:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.631+0000 7f58f97fa700 1 -- 172.21.15.7:0/965593701 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58ec09c890 con 0x7f58fc06aad0 2024-03-26T08:41:04.622 INFO:teuthology.orchestra.run.smithi007.stdout:287762808867 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58e40a2ac0 msgr2=0x7f58e40a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58e40a2ac0 0x7f58e40a4f80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f58fc10f010 tx=0x7f58f0009260 comp rx=0 tx=0).stop 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 msgr2=0x7f58fc1a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 0x7f58fc1a07c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58ec006010 tx=0x7f58ec00f690 comp rx=0 tx=0).stop 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 shutdown_connections 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58e40a2ac0 0x7f58e40a4f80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc107540 0x7f58fc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc06aad0 0x7f58fc1a07c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 --2- 172.21.15.7:0/965593701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f58fc06b480 0x7f58fc1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 >> 172.21.15.7:0/965593701 conn(0x7f58fc0770e0 msgr2=0x7f58fc10d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 shutdown_connections 2024-03-26T08:41:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:04.633+0000 7f590250a700 1 -- 172.21.15.7:0/965593701 wait complete. 2024-03-26T08:41:04.732 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:04 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3261334862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T08:41:04.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:04 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3261334862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T08:41:04.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:04 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3261334862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T08:41:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.053+0000 7f6ac37c6700 1 -- 172.21.15.7:0/805327068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc069e40 msgr2=0x7f6abc06a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.053+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/805327068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc069e40 0x7f6abc06a2a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00099c0 tx=0x7f6ab0009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.053+0000 7f6ac37c6700 1 -- 172.21.15.7:0/805327068 shutdown_connections 2024-03-26T08:41:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.053+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/805327068 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6abc1081e0 0x7f6abc1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.053+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/805327068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6abc06a7e0 0x7f6abc10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.053+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/805327068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc069e40 0x7f6abc06a2a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.053+0000 7f6ac37c6700 1 -- 172.21.15.7:0/805327068 >> 172.21.15.7:0/805327068 conn(0x7f6abc077fc0 msgr2=0x7f6abc0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.054+0000 7f6ac37c6700 1 -- 172.21.15.7:0/805327068 shutdown_connections 2024-03-26T08:41:05.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.054+0000 7f6ac37c6700 1 -- 172.21.15.7:0/805327068 wait complete. 2024-03-26T08:41:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.054+0000 7f6ac37c6700 1 Processor -- start 2024-03-26T08:41:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.054+0000 7f6ac37c6700 1 -- start start 2024-03-26T08:41:05.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.054+0000 7f6ac37c6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6abc069e40 0x7f6abc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.054+0000 7f6ac37c6700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6abc06a7e0 0x7f6abc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.054+0000 7f6ac37c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc1081e0 0x7f6abc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac37c6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc112440 con 0x7f6abc1081e0 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1d63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc1081e0 0x7f6abc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1562700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6abc069e40 0x7f6abc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1d63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc1081e0 0x7f6abc198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48642/0 (socket says 172.21.15.7:48642) 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1562700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6abc069e40 0x7f6abc19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45904/0 (socket says 172.21.15.7:45904) 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1d63700 1 -- 172.21.15.7:0/4006457690 learned_addr learned my addr 172.21.15.7:0/4006457690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac0d61700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6abc06a7e0 0x7f6abc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac37c6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc1122c0 con 0x7f6abc069e40 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc1125c0 con 0x7f6abc06a7e0 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1562700 1 -- 172.21.15.7:0/4006457690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6abc06a7e0 msgr2=0x7f6abc19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1562700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6abc06a7e0 0x7f6abc19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1562700 1 -- 172.21.15.7:0/4006457690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc1081e0 msgr2=0x7f6abc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1562700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc1081e0 0x7f6abc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1562700 1 -- 172.21.15.7:0/4006457690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc198fe0 con 0x7f6abc069e40 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac0d61700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6abc06a7e0 0x7f6abc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac1562700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6abc069e40 0x7f6abc19e630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00efc0 tx=0x7f6aac00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6aba7fc700 1 -- 172.21.15.7:0/4006457690 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac0092e0 con 0x7f6abc069e40 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc199270 con 0x7f6abc069e40 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6aba7fc700 1 -- 172.21.15.7:0/4006457690 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6aac004bb0 con 0x7f6abc069e40 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.055+0000 7f6aba7fc700 1 -- 172.21.15.7:0/4006457690 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac004280 con 0x7f6abc069e40 2024-03-26T08:41:05.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.056+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc1a55b0 con 0x7f6abc069e40 2024-03-26T08:41:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.056+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc103890 con 0x7f6abc069e40 2024-03-26T08:41:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.057+0000 7f6aba7fc700 1 -- 172.21.15.7:0/4006457690 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f6aac01b030 con 0x7f6abc069e40 2024-03-26T08:41:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.058+0000 7f6aba7fc700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6aa40a29a0 0x7f6aa40a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.058+0000 7f6ac0d61700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6aa40a29a0 0x7f6aa40a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.058+0000 7f6aba7fc700 1 -- 172.21.15.7:0/4006457690 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6aac0ce820 con 0x7f6abc069e40 2024-03-26T08:41:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.058+0000 7f6ac0d61700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6aa40a29a0 0x7f6aa40a4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6abc199f50 tx=0x7f6ab0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.062+0000 7f6aba7fc700 1 -- 172.21.15.7:0/4006457690 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6aac022020 con 0x7f6abc069e40 2024-03-26T08:41:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.270+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6abc1995b0 con 0x7f6abc069e40 2024-03-26T08:41:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.270+0000 7f6aba7fc700 1 -- 172.21.15.7:0/4006457690 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6aac010040 con 0x7f6abc069e40 2024-03-26T08:41:05.262 INFO:teuthology.orchestra.run.smithi007.stdout:141733920829 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6aa40a29a0 msgr2=0x7f6aa40a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6aa40a29a0 0x7f6aa40a4e60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6abc199f50 tx=0x7f6ab0005fb0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6abc069e40 msgr2=0x7f6abc19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6abc069e40 0x7f6abc19e630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6aac00efc0 tx=0x7f6aac00c3a0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 shutdown_connections 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6aa40a29a0 0x7f6aa40a4e60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6abc1081e0 0x7f6abc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6abc069e40 0x7f6abc19e630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 --2- 172.21.15.7:0/4006457690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6abc06a7e0 0x7f6abc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 >> 172.21.15.7:0/4006457690 conn(0x7f6abc077fc0 msgr2=0x7f6abc0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 shutdown_connections 2024-03-26T08:41:05.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:05.272+0000 7f6ac37c6700 1 -- 172.21.15.7:0/4006457690 wait complete. 2024-03-26T08:41:05.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:05 smithi007 ceph-mon[32462]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 17 B/s, 0 objects/s recovering 2024-03-26T08:41:05.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:05 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/965593701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T08:41:05.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:05 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4006457690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T08:41:05.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:05 smithi016 ceph-mon[35851]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 17 B/s, 0 objects/s recovering 2024-03-26T08:41:05.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:05 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/965593701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T08:41:05.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:05 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4006457690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T08:41:05.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:05 smithi044 ceph-mon[36461]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 17 B/s, 0 objects/s recovering 2024-03-26T08:41:05.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:05 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/965593701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T08:41:05.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:05 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4006457690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T08:41:07.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:07 smithi007 ceph-mon[32462]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 17 B/s, 0 objects/s recovering 2024-03-26T08:41:07.644 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808863 got 287762808867 for osd.9 2024-03-26T08:41:07.644 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:07.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:07 smithi016 ceph-mon[35851]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 17 B/s, 0 objects/s recovering 2024-03-26T08:41:07.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:07 smithi044 ceph-mon[36461]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 17 B/s, 0 objects/s recovering 2024-03-26T08:41:08.855 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070500 got 253403070504 for osd.8 2024-03-26T08:41:08.855 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:09.715 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:09 smithi007 ceph-mon[32462]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-03-26T08:41:09.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:09 smithi016 ceph-mon[35851]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-03-26T08:41:09.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:09 smithi044 ceph-mon[36461]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-03-26T08:41:09.989 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920829 for osd.4 2024-03-26T08:41:09.990 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.313+0000 7f7fd79be700 1 -- 172.21.15.7:0/3745086540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fd01081e0 msgr2=0x7f7fd01085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.313+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3745086540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fd01081e0 0x7f7fd01085c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009920 tx=0x7f7fcc009c30 comp rx=0 tx=0).stop 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.313+0000 7f7fd79be700 1 -- 172.21.15.7:0/3745086540 shutdown_connections 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.313+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3745086540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fd01081e0 0x7f7fd01085c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.313+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3745086540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd006a7e0 0x7f7fd010f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.313+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3745086540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fd0069e40 0x7f7fd006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.314+0000 7f7fd79be700 1 -- 172.21.15.7:0/3745086540 >> 172.21.15.7:0/3745086540 conn(0x7f7fd0077fc0 msgr2=0x7f7fd00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.314+0000 7f7fd79be700 1 -- 172.21.15.7:0/3745086540 shutdown_connections 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.314+0000 7f7fd79be700 1 -- 172.21.15.7:0/3745086540 wait complete. 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.314+0000 7f7fd79be700 1 Processor -- start 2024-03-26T08:41:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.314+0000 7f7fd79be700 1 -- start start 2024-03-26T08:41:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd79be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fd0069e40 0x7f7fd01a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd79be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd006a7e0 0x7f7fd01a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd79be700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fd01081e0 0x7f7fd019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd79be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd0112490 con 0x7f7fd0069e40 2024-03-26T08:41:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd79be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd0112310 con 0x7f7fd006a7e0 2024-03-26T08:41:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd79be700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd0112610 con 0x7f7fd01081e0 2024-03-26T08:41:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd4f59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd006a7e0 0x7f7fd01a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd575a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fd0069e40 0x7f7fd01a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd4f59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd006a7e0 0x7f7fd01a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57682/0 (socket says 172.21.15.7:57682) 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd4f59700 1 -- 172.21.15.7:0/3040289119 learned_addr learned my addr 172.21.15.7:0/3040289119 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd5f5b700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fd01081e0 0x7f7fd019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd4f59700 1 -- 172.21.15.7:0/3040289119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fd01081e0 msgr2=0x7f7fd019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd4f59700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fd01081e0 0x7f7fd019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd4f59700 1 -- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fd0069e40 msgr2=0x7f7fd01a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd4f59700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fd0069e40 0x7f7fd01a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd4f59700 1 -- 172.21.15.7:0/3040289119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd019b170 con 0x7f7fd006a7e0 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.315+0000 7f7fd575a700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fd0069e40 0x7f7fd01a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.316+0000 7f7fd4f59700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd006a7e0 0x7f7fd01a0d90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000ec10 tx=0x7f7fc000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.316+0000 7f7fc67fc700 1 -- 172.21.15.7:0/3040289119 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000cc90 con 0x7f7fd006a7e0 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.316+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd019b460 con 0x7f7fd006a7e0 2024-03-26T08:41:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.316+0000 7f7fc67fc700 1 -- 172.21.15.7:0/3040289119 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc000ce30 con 0x7f7fd006a7e0 2024-03-26T08:41:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.316+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd01a7720 con 0x7f7fd006a7e0 2024-03-26T08:41:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.316+0000 7f7fc67fc700 1 -- 172.21.15.7:0/3040289119 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0010690 con 0x7f7fd006a7e0 2024-03-26T08:41:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.317+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb4005330 con 0x7f7fd006a7e0 2024-03-26T08:41:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.318+0000 7f7fc67fc700 1 -- 172.21.15.7:0/3040289119 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f7fc0004020 con 0x7f7fd006a7e0 2024-03-26T08:41:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.319+0000 7f7fc67fc700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fbc0a29a0 0x7f7fbc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.319+0000 7f7fd575a700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fbc0a29a0 0x7f7fbc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.319+0000 7f7fc67fc700 1 -- 172.21.15.7:0/3040289119 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fc00ced70 con 0x7f7fd006a7e0 2024-03-26T08:41:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.319+0000 7f7fd575a700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fbc0a29a0 0x7f7fbc0a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7fd019bb50 tx=0x7f7fcc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:10.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.327+0000 7f7fc67fc700 1 -- 172.21.15.7:0/3040289119 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7fc00d3070 con 0x7f7fd006a7e0 2024-03-26T08:41:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.541+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7fb40058f0 con 0x7f7fd006a7e0 2024-03-26T08:41:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.541+0000 7f7fc67fc700 1 -- 172.21.15.7:0/3040289119 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7fc009bcb0 con 0x7f7fd006a7e0 2024-03-26T08:41:10.534 INFO:teuthology.orchestra.run.smithi007.stdout:197568495668 2024-03-26T08:41:10.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fbc0a29a0 msgr2=0x7f7fbc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fbc0a29a0 0x7f7fbc0a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7fd019bb50 tx=0x7f7fcc019040 comp rx=0 tx=0).stop 2024-03-26T08:41:10.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd006a7e0 msgr2=0x7f7fd01a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd006a7e0 0x7f7fd01a0d90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000ec10 tx=0x7f7fc000ef20 comp rx=0 tx=0).stop 2024-03-26T08:41:10.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 shutdown_connections 2024-03-26T08:41:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fbc0a29a0 0x7f7fbc0a4e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fd01081e0 0x7f7fd019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fd0069e40 0x7f7fd01a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 --2- 172.21.15.7:0/3040289119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd006a7e0 0x7f7fd01a0d90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 >> 172.21.15.7:0/3040289119 conn(0x7f7fd0077fc0 msgr2=0x7f7fd010d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 shutdown_connections 2024-03-26T08:41:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:10.545+0000 7f7fd79be700 1 -- 172.21.15.7:0/3040289119 wait complete. 2024-03-26T08:41:10.552 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:10 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2581568063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T08:41:10.553 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:10 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:41:10.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:10 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2581568063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T08:41:10.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:10 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:41:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.722+0000 7f83b56b2700 1 -- 172.21.15.7:0/2705973401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0102e20 msgr2=0x7f83b0106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.722+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2705973401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0102e20 0x7f83b0106df0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f83980099c0 tx=0x7f8398009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.722+0000 7f83b56b2700 1 -- 172.21.15.7:0/2705973401 shutdown_connections 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.722+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2705973401 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b0107520 0x7f83b010f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.722+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2705973401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b0102470 0x7f83b0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.722+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2705973401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0102e20 0x7f83b0106df0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.722+0000 7f83b56b2700 1 -- 172.21.15.7:0/2705973401 >> 172.21.15.7:0/2705973401 conn(0x7f83b00792b0 msgr2=0x7f83b00796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.722+0000 7f83b56b2700 1 -- 172.21.15.7:0/2705973401 shutdown_connections 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.722+0000 7f83b56b2700 1 -- 172.21.15.7:0/2705973401 wait complete. 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.723+0000 7f83b56b2700 1 Processor -- start 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.723+0000 7f83b56b2700 1 -- start start 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.723+0000 7f83b56b2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b0102470 0x7f83b0074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.723+0000 7f83b56b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0102e20 0x7f83b00749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.723+0000 7f83b56b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b0107520 0x7f83b0074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.723+0000 7f83b56b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b0112490 con 0x7f83b0102e20 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.723+0000 7f83b56b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b0112310 con 0x7f83b0107520 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.723+0000 7f83b56b2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b0112610 con 0x7f83b0102470 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83ae7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0102e20 0x7f83b00749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b0102470 0x7f83b0074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83aeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b0102470 0x7f83b0074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36614/0 (socket says 172.21.15.7:36614) 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83aeffd700 1 -- 172.21.15.7:0/2581568063 learned_addr learned my addr 172.21.15.7:0/2581568063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b0107520 0x7f83b0074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83ae7fc700 1 -- 172.21.15.7:0/2581568063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b0102470 msgr2=0x7f83b0074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b0102470 0x7f83b0074490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83ae7fc700 1 -- 172.21.15.7:0/2581568063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b0107520 msgr2=0x7f83b0074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b0107520 0x7f83b0074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83ae7fc700 1 -- 172.21.15.7:0/2581568063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b01a6e30 con 0x7f83b0102e20 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83af7fe700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b0107520 0x7f83b0074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83ae7fc700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0102e20 0x7f83b00749d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f83980098e0 tx=0x7f8398005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83a7fff700 1 -- 172.21.15.7:0/2581568063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398027070 con 0x7f83b0102e20 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b01a70c0 con 0x7f83b0102e20 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83a7fff700 1 -- 172.21.15.7:0/2581568063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8398003680 con 0x7f83b0102e20 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83a7fff700 1 -- 172.21.15.7:0/2581568063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398022de0 con 0x7f83b0102e20 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.724+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b01a7520 con 0x7f83b0102e20 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.725+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b0068e50 con 0x7f83b0102e20 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.726+0000 7f83a7fff700 1 -- 172.21.15.7:0/2581568063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f839802a070 con 0x7f83b0102e20 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.727+0000 7f83a7fff700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f839c0a2920 0x7f839c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.727+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f839c0a2920 0x7f839c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.727+0000 7f83a7fff700 1 -- 172.21.15.7:0/2581568063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f83980cf670 con 0x7f83b0102e20 2024-03-26T08:41:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.727+0000 7f83aeffd700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f839c0a2920 0x7f839c0a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009440 tx=0x7f83a000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.732+0000 7f83a7fff700 1 -- 172.21.15.7:0/2581568063 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f839805e740 con 0x7f83b0102e20 2024-03-26T08:41:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.938+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f83b002da30 con 0x7f83b0102e20 2024-03-26T08:41:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.938+0000 7f83a7fff700 1 -- 172.21.15.7:0/2581568063 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8398013070 con 0x7f83b0102e20 2024-03-26T08:41:10.750 INFO:teuthology.orchestra.run.smithi007.stdout:163208757304 2024-03-26T08:41:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.940+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f839c0a2920 msgr2=0x7f839c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.940+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f839c0a2920 0x7f839c0a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009440 tx=0x7f83a000b040 comp rx=0 tx=0).stop 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.940+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0102e20 msgr2=0x7f83b00749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.940+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0102e20 0x7f83b00749d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f83980098e0 tx=0x7f8398005f10 comp rx=0 tx=0).stop 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.940+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 shutdown_connections 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.940+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f839c0a2920 0x7f839c0a4de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.940+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f83b0102470 0x7f83b0074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.940+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b0107520 0x7f83b0074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.940+0000 7f83b56b2700 1 --2- 172.21.15.7:0/2581568063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83b0102e20 0x7f83b00749d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.941+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 >> 172.21.15.7:0/2581568063 conn(0x7f83b00792b0 msgr2=0x7f83b0108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.941+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 shutdown_connections 2024-03-26T08:41:10.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:09.941+0000 7f83b56b2700 1 -- 172.21.15.7:0/2581568063 wait complete. 2024-03-26T08:41:10.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:10 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2581568063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T08:41:10.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:10 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:41:11.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757304 for osd.5 2024-03-26T08:41:11.379 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:11.601 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:11 smithi007 ceph-mon[32462]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 12 B/s, 0 objects/s recovering 2024-03-26T08:41:11.601 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3040289119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T08:41:11.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:11 smithi016 ceph-mon[35851]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 12 B/s, 0 objects/s recovering 2024-03-26T08:41:11.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3040289119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T08:41:11.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:11 smithi044 ceph-mon[36461]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 12 B/s, 0 objects/s recovering 2024-03-26T08:41:11.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3040289119' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T08:41:12.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.128+0000 7f43e7048700 1 -- 172.21.15.7:0/1864043986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 msgr2=0x7f43e01014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:12.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.128+0000 7f43e7048700 1 --2- 172.21.15.7:0/1864043986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 0x7f43e01014f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009990 tx=0x7f43cc009ca0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.128+0000 7f43e7048700 1 -- 172.21.15.7:0/1864043986 shutdown_connections 2024-03-26T08:41:12.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.128+0000 7f43e7048700 1 --2- 172.21.15.7:0/1864043986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e0101a30 0x7f43e010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.128+0000 7f43e7048700 1 --2- 172.21.15.7:0/1864043986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e01070d0 0x7f43e01074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.128+0000 7f43e7048700 1 --2- 172.21.15.7:0/1864043986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 0x7f43e01014f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.128+0000 7f43e7048700 1 -- 172.21.15.7:0/1864043986 >> 172.21.15.7:0/1864043986 conn(0x7f43e0077fc0 msgr2=0x7f43e00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:12.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.128+0000 7f43e7048700 1 -- 172.21.15.7:0/1864043986 shutdown_connections 2024-03-26T08:41:12.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e7048700 1 -- 172.21.15.7:0/1864043986 wait complete. 2024-03-26T08:41:12.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e7048700 1 Processor -- start 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e7048700 1 -- start start 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e7048700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 0x7f43e0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e7048700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e0101a30 0x7f43e0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e7048700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e01070d0 0x7f43e01a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e7048700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e0111380 con 0x7f43e0101090 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e7048700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e0111200 con 0x7f43e01070d0 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e7048700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e0111500 con 0x7f43e0101a30 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.129+0000 7f43e4de4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 0x7f43e0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e4de4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 0x7f43e0199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33458/0 (socket says 172.21.15.7:33458) 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e4de4700 1 -- 172.21.15.7:0/2777048917 learned_addr learned my addr 172.21.15.7:0/2777048917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43dffff700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e0101a30 0x7f43e0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:12.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e55e5700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e01070d0 0x7f43e01a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e4de4700 1 -- 172.21.15.7:0/2777048917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e0101a30 msgr2=0x7f43e0199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e4de4700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e0101a30 0x7f43e0199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e4de4700 1 -- 172.21.15.7:0/2777048917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e01070d0 msgr2=0x7f43e01a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e4de4700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e01070d0 0x7f43e01a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e4de4700 1 -- 172.21.15.7:0/2777048917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e01a4d40 con 0x7f43e0101090 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43dffff700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e0101a30 0x7f43e0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e4de4700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 0x7f43e0199780 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f43d400e9a0 tx=0x7f43d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43ddffb700 1 -- 172.21.15.7:0/2777048917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d400ca10 con 0x7f43e0101090 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43ddffb700 1 -- 172.21.15.7:0/2777048917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43d400cbb0 con 0x7f43e0101090 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43ddffb700 1 -- 172.21.15.7:0/2777048917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4018680 con 0x7f43e0101090 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.130+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e01a4f70 con 0x7f43e0101090 2024-03-26T08:41:12.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.131+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e01a53d0 con 0x7f43e0101090 2024-03-26T08:41:12.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.132+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e0068e50 con 0x7f43e0101090 2024-03-26T08:41:12.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.133+0000 7f43ddffb700 1 -- 172.21.15.7:0/2777048917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f43d4010430 con 0x7f43e0101090 2024-03-26T08:41:12.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.133+0000 7f43ddffb700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43d00a29a0 0x7f43d00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:12.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.133+0000 7f43dffff700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43d00a29a0 0x7f43d00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:12.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.133+0000 7f43ddffb700 1 -- 172.21.15.7:0/2777048917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f43d40ced00 con 0x7f43e0101090 2024-03-26T08:41:12.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.134+0000 7f43dffff700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43d00a29a0 0x7f43d00a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f43e0078610 tx=0x7f43cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:12.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.140+0000 7f43ddffb700 1 -- 172.21.15.7:0/2777048917 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f43d409bc40 con 0x7f43e0101090 2024-03-26T08:41:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.349+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f43e002dc90 con 0x7f43e0101090 2024-03-26T08:41:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.350+0000 7f43ddffb700 1 -- 172.21.15.7:0/2777048917 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f43d409b540 con 0x7f43e0101090 2024-03-26T08:41:12.342 INFO:teuthology.orchestra.run.smithi007.stdout:309237645342 2024-03-26T08:41:12.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43d00a29a0 msgr2=0x7f43d00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43d00a29a0 0x7f43d00a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f43e0078610 tx=0x7f43cc005fd0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 msgr2=0x7f43e0199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 0x7f43e0199780 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f43d400e9a0 tx=0x7f43d400ecb0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 shutdown_connections 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43d00a29a0 0x7f43d00a4e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e01070d0 0x7f43e01a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43e0101090 0x7f43e0199780 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 --2- 172.21.15.7:0/2777048917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43e0101a30 0x7f43e0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 >> 172.21.15.7:0/2777048917 conn(0x7f43e0077fc0 msgr2=0x7f43e00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 shutdown_connections 2024-03-26T08:41:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:12.352+0000 7f43e7048700 1 -- 172.21.15.7:0/2777048917 wait complete. 2024-03-26T08:41:13.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:13 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:13 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2777048917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T08:41:13.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:13 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:13 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2777048917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T08:41:13.358 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:13.358 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:13.358 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:13.395 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495663 got 197568495668 for osd.6 2024-03-26T08:41:13.395 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:13 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:13.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:13 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2777048917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T08:41:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:14 smithi007 ceph-mon[32462]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-03-26T08:41:14.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:14 smithi016 ceph-mon[35851]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-03-26T08:41:14.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:14 smithi044 ceph-mon[36461]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 11 B/s, 0 objects/s recovering 2024-03-26T08:41:16.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:16 smithi007 ceph-mon[32462]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 11 B/s, 0 objects/s recovering 2024-03-26T08:41:16.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:16 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3216271720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T08:41:16.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:16 smithi016 ceph-mon[35851]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 11 B/s, 0 objects/s recovering 2024-03-26T08:41:16.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:16 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3216271720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T08:41:16.519 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645339 got 309237645342 for osd.10 2024-03-26T08:41:16.520 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:16 smithi044 ceph-mon[36461]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 11 B/s, 0 objects/s recovering 2024-03-26T08:41:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:16 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3216271720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T08:41:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.888+0000 7f089fa6f700 1 -- 172.21.15.7:0/2146192023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0898101af0 msgr2=0x7f0898101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.888+0000 7f089fa6f700 1 --2- 172.21.15.7:0/2146192023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0898101af0 0x7f0898101ed0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08940099c0 tx=0x7f0894009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.888+0000 7f089fa6f700 1 -- 172.21.15.7:0/2146192023 shutdown_connections 2024-03-26T08:41:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.888+0000 7f089fa6f700 1 --2- 172.21.15.7:0/2146192023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898106410 0x7f089810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.888+0000 7f089fa6f700 1 --2- 172.21.15.7:0/2146192023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0898101af0 0x7f0898101ed0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.888+0000 7f089fa6f700 1 --2- 172.21.15.7:0/2146192023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08981024a0 0x7f0898105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.888+0000 7f089fa6f700 1 -- 172.21.15.7:0/2146192023 >> 172.21.15.7:0/2146192023 conn(0x7f0898077690 msgr2=0x7f08980feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.888+0000 7f089fa6f700 1 -- 172.21.15.7:0/2146192023 shutdown_connections 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.888+0000 7f089fa6f700 1 -- 172.21.15.7:0/2146192023 wait complete. 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.889+0000 7f089fa6f700 1 Processor -- start 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.889+0000 7f089fa6f700 1 -- start start 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.889+0000 7f089fa6f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898101af0 0x7f08980fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.889+0000 7f089fa6f700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08981024a0 0x7f08980fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.889+0000 7f089fa6f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0898106410 0x7f08980fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.889+0000 7f089fa6f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0898111410 con 0x7f0898106410 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.889+0000 7f089fa6f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0898111290 con 0x7f0898101af0 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.889+0000 7f089fa6f700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0898111590 con 0x7f08981024a0 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d80b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898101af0 0x7f08980fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089e00c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0898106410 0x7f08980fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d00a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08981024a0 0x7f08980fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d80b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898101af0 0x7f08980fcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57738/0 (socket says 172.21.15.7:57738) 2024-03-26T08:41:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:343597383706 2024-03-26T08:41:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089e00c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0898106410 0x7f08980fd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33476/0 (socket says 172.21.15.7:33476) 2024-03-26T08:41:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d00a700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08981024a0 0x7f08980fd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36686/0 (socket says 172.21.15.7:36686) 2024-03-26T08:41:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d80b700 1 -- 172.21.15.7:0/3216271720 learned_addr learned my addr 172.21.15.7:0/3216271720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d80b700 1 -- 172.21.15.7:0/3216271720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08981024a0 msgr2=0x7f08980fd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d80b700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08981024a0 0x7f08980fd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d80b700 1 -- 172.21.15.7:0/3216271720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0898106410 msgr2=0x7f08980fd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d80b700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0898106410 0x7f08980fd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d80b700 1 -- 172.21.15.7:0/3216271720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08981a6ef0 con 0x7f0898101af0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089d80b700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898101af0 0x7f08980fcb70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0894006010 tx=0x7f089400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f088effd700 1 -- 172.21.15.7:0/3216271720 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894003680 con 0x7f0898101af0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f088effd700 1 -- 172.21.15.7:0/3216271720 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08940050d0 con 0x7f0898101af0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08981a7180 con 0x7f0898101af0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f088effd700 1 -- 172.21.15.7:0/3216271720 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894003db0 con 0x7f0898101af0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.890+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08981a75d0 con 0x7f0898101af0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.891+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0898068e50 con 0x7f0898101af0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.892+0000 7f088effd700 1 -- 172.21.15.7:0/3216271720 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f0894005280 con 0x7f0898101af0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.893+0000 7f088effd700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f08840a2ac0 0x7f08840a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.893+0000 7f089d00a700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f08840a2ac0 0x7f08840a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.893+0000 7f088effd700 1 -- 172.21.15.7:0/3216271720 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08940cfaf0 con 0x7f0898101af0 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.893+0000 7f089d00a700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f08840a2ac0 0x7f08840a4f80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08980ffe70 tx=0x7f0888009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:15.897+0000 7f088effd700 1 -- 172.21.15.7:0/3216271720 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f089405d740 con 0x7f0898101af0 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.101+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f089802da30 con 0x7f0898101af0 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.102+0000 7f088effd700 1 -- 172.21.15.7:0/3216271720 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f0894013070 con 0x7f0898101af0 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f08840a2ac0 msgr2=0x7f08840a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f08840a2ac0 0x7f08840a4f80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08980ffe70 tx=0x7f0888009260 comp rx=0 tx=0).stop 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898101af0 msgr2=0x7f08980fcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898101af0 0x7f08980fcb70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0894006010 tx=0x7f089400f690 comp rx=0 tx=0).stop 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 shutdown_connections 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f08840a2ac0 0x7f08840a4f80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0898106410 0x7f08980fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0898101af0 0x7f08980fcb70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 --2- 172.21.15.7:0/3216271720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08981024a0 0x7f08980fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 >> 172.21.15.7:0/3216271720 conn(0x7f0898077690 msgr2=0x7f08980ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 shutdown_connections 2024-03-26T08:41:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:16.104+0000 7f089fa6f700 1 -- 172.21.15.7:0/3216271720 wait complete. 2024-03-26T08:41:18.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:18 smithi016 ceph-mon[35851]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:18.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:18 smithi007 ceph-mon[32462]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:18.545 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383701 got 343597383706 for osd.11 2024-03-26T08:41:18.546 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:18.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:18 smithi044 ceph-mon[36461]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/636614490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01074b0 msgr2=0x7f9ce010f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/636614490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01074b0 0x7f9ce010f080 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9cd80099c0 tx=0x7f9cd8009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/636614490 shutdown_connections 2024-03-26T08:41:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/636614490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce01023d0 0x7f9ce01027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/636614490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01074b0 0x7f9ce010f080 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/636614490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce0102d80 0x7f9ce0106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/636614490 >> 172.21.15.7:0/636614490 conn(0x7f9ce00fdc40 msgr2=0x7f9ce0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/636614490 shutdown_connections 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/636614490 wait complete. 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.001+0000 7f9ce5cb0700 1 Processor -- start 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9ce5cb0700 1 -- start start 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9ce5cb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce01023d0 0x7f9ce019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9ce5cb0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce0102d80 0x7f9ce019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9ce5cb0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01074b0 0x7f9ce0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9ce5cb0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce01124c0 con 0x7f9ce01074b0 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9ce5cb0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce0112340 con 0x7f9ce01023d0 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9ce5cb0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce0112640 con 0x7f9ce0102d80 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce0102d80 0x7f9ce019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce01023d0 0x7f9ce019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01074b0 0x7f9ce0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:19.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdeffd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce0102d80 0x7f9ce019eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42234/0 (socket says 172.21.15.7:42234) 2024-03-26T08:41:19.215 INFO:teuthology.orchestra.run.smithi007.stdout:34359738451 2024-03-26T08:41:19.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdeffd700 1 -- 172.21.15.7:0/2736803715 learned_addr learned my addr 172.21.15.7:0/2736803715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:19.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce01023d0 0x7f9ce019e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44312/0 (socket says 172.21.15.7:44312) 2024-03-26T08:41:19.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdf7fe700 1 -- 172.21.15.7:0/2736803715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce0102d80 msgr2=0x7f9ce019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:19.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdf7fe700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce0102d80 0x7f9ce019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdf7fe700 1 -- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01074b0 msgr2=0x7f9ce0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdf7fe700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01074b0 0x7f9ce0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdf7fe700 1 -- 172.21.15.7:0/2736803715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce0198fe0 con 0x7f9ce01023d0 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdffff700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01074b0 0x7f9ce0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.002+0000 7f9cdf7fe700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce01023d0 0x7f9ce019e630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000efc0 tx=0x7f9cd000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.003+0000 7f9cdcff9700 1 -- 172.21.15.7:0/2736803715 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd00092e0 con 0x7f9ce01023d0 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.003+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce0199270 con 0x7f9ce01023d0 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.003+0000 7f9cdcff9700 1 -- 172.21.15.7:0/2736803715 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cd0004bb0 con 0x7f9ce01023d0 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.003+0000 7f9cdcff9700 1 -- 172.21.15.7:0/2736803715 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0004280 con 0x7f9ce01023d0 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.003+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce01a55b0 con 0x7f9ce01023d0 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.004+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce01995e0 con 0x7f9ce01023d0 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.005+0000 7f9cdcff9700 1 -- 172.21.15.7:0/2736803715 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f9cd001b030 con 0x7f9ce01023d0 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.005+0000 7f9cdcff9700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9ccc0a2920 0x7f9ccc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:19.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.005+0000 7f9cdeffd700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9ccc0a2920 0x7f9ccc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.005+0000 7f9cdcff9700 1 -- 172.21.15.7:0/2736803715 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cd00ce7f0 con 0x7f9ce01023d0 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.006+0000 7f9cdeffd700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9ccc0a2920 0x7f9ccc0a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0199ec0 tx=0x7f9cc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.010+0000 7f9cdcff9700 1 -- 172.21.15.7:0/2736803715 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9ce01995e0 con 0x7f9ce01023d0 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.219+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9ce0199c30 con 0x7f9ce01023d0 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.219+0000 7f9cdcff9700 1 -- 172.21.15.7:0/2736803715 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9cd0022020 con 0x7f9ce01023d0 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9ccc0a2920 msgr2=0x7f9ccc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9ccc0a2920 0x7f9ccc0a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0199ec0 tx=0x7f9cc8009260 comp rx=0 tx=0).stop 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce01023d0 msgr2=0x7f9ce019e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce01023d0 0x7f9ce019e630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000efc0 tx=0x7f9cd000c3a0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 shutdown_connections 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9ccc0a2920 0x7f9ccc0a4de0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ce01074b0 0x7f9ce0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce01023d0 0x7f9ce019e630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 --2- 172.21.15.7:0/2736803715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ce0102d80 0x7f9ce019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:19.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 >> 172.21.15.7:0/2736803715 conn(0x7f9ce00fdc40 msgr2=0x7f9ce00fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:19.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 shutdown_connections 2024-03-26T08:41:19.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:19.221+0000 7f9ce5cb0700 1 -- 172.21.15.7:0/2736803715 wait complete. 2024-03-26T08:41:20.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:20 smithi007 ceph-mon[32462]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:41:20.414 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:20 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2736803715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T08:41:20.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:20 smithi016 ceph-mon[35851]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:41:20.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:20 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2736803715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T08:41:20.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:20 smithi044 ceph-mon[36461]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:41:20.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:20 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2736803715' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T08:41:21.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.133+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3249338819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 msgr2=0x7f5a48102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.133+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3249338819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 0x7f5a48102660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0099c0 tx=0x7f5a3c009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.133+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3249338819 shutdown_connections 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.133+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3249338819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a48102ba0 0x7f5a4810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.133+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3249338819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a481081c0 0x7f5a481085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.133+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3249338819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 0x7f5a48102660 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.133+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3249338819 >> 172.21.15.7:0/3249338819 conn(0x7f5a480792c0 msgr2=0x7f5a480796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.133+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3249338819 shutdown_connections 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.133+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3249338819 wait complete. 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a4dd50700 1 Processor -- start 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a4dd50700 1 -- start start 2024-03-26T08:41:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a4dd50700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 0x7f5a481a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a4dd50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a48102ba0 0x7f5a481a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a4dd50700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a481081c0 0x7f5a4819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a4dd50700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48112470 con 0x7f5a481021e0 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a4dd50700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a481122f0 con 0x7f5a48102ba0 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a4dd50700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a481125f0 con 0x7f5a481081c0 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 0x7f5a481a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a46ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a48102ba0 0x7f5a481a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a47fff700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a481081c0 0x7f5a4819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.134+0000 7f5a477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 0x7f5a481a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49798/0 (socket says 172.21.15.7:49798) 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a477fe700 1 -- 172.21.15.7:0/3688715112 learned_addr learned my addr 172.21.15.7:0/3688715112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a46ffd700 1 -- 172.21.15.7:0/3688715112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a481081c0 msgr2=0x7f5a4819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a46ffd700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a481081c0 0x7f5a4819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a46ffd700 1 -- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 msgr2=0x7f5a481a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a46ffd700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 0x7f5a481a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a46ffd700 1 -- 172.21.15.7:0/3688715112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4819b030 con 0x7f5a48102ba0 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a477fe700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 0x7f5a481a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a46ffd700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a48102ba0 0x7f5a481a0c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0098e0 tx=0x7f5a3c005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a44ff9700 1 -- 172.21.15.7:0/3688715112 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c027070 con 0x7f5a48102ba0 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a44ff9700 1 -- 172.21.15.7:0/3688715112 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c003680 con 0x7f5a48102ba0 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4819b2c0 con 0x7f5a48102ba0 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a44ff9700 1 -- 172.21.15.7:0/3688715112 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c022e10 con 0x7f5a48102ba0 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.135+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a481a75e0 con 0x7f5a48102ba0 2024-03-26T08:41:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.136+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a48103870 con 0x7f5a48102ba0 2024-03-26T08:41:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.137+0000 7f5a44ff9700 1 -- 172.21.15.7:0/3688715112 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f5a3c02a070 con 0x7f5a48102ba0 2024-03-26T08:41:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.138+0000 7f5a44ff9700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a300a2920 0x7f5a300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.138+0000 7f5a477fe700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a300a2920 0x7f5a300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.138+0000 7f5a44ff9700 1 -- 172.21.15.7:0/3688715112 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5a3c0cf680 con 0x7f5a48102ba0 2024-03-26T08:41:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.138+0000 7f5a477fe700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a300a2920 0x7f5a300a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5a4819ba10 tx=0x7f5a38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.141+0000 7f5a44ff9700 1 -- 172.21.15.7:0/3688715112 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a3c013070 con 0x7f5a48102ba0 2024-03-26T08:41:21.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.352+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5a481a77b0 con 0x7f5a48102ba0 2024-03-26T08:41:21.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.353+0000 7f5a44ff9700 1 -- 172.21.15.7:0/3688715112 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a3c09c730 con 0x7f5a48102ba0 2024-03-26T08:41:21.344 INFO:teuthology.orchestra.run.smithi007.stdout:219043332144 2024-03-26T08:41:21.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a300a2920 msgr2=0x7f5a300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:21.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a300a2920 0x7f5a300a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5a4819ba10 tx=0x7f5a38009260 comp rx=0 tx=0).stop 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a48102ba0 msgr2=0x7f5a481a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a48102ba0 0x7f5a481a0c50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0098e0 tx=0x7f5a3c005f10 comp rx=0 tx=0).stop 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 shutdown_connections 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a300a2920 0x7f5a300a4de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a481081c0 0x7f5a4819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a481021e0 0x7f5a481a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 --2- 172.21.15.7:0/3688715112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a48102ba0 0x7f5a481a0c50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 >> 172.21.15.7:0/3688715112 conn(0x7f5a480792c0 msgr2=0x7f5a48074650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 shutdown_connections 2024-03-26T08:41:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.355+0000 7f5a4dd50700 1 -- 172.21.15.7:0/3688715112 wait complete. 2024-03-26T08:41:21.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 -- 172.21.15.7:0/4135990959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e001024a0 msgr2=0x7f9e00105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 --2- 172.21.15.7:0/4135990959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e001024a0 0x7f9e00105d30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009920 tx=0x7f9df0009c30 comp rx=0 tx=0).stop 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 -- 172.21.15.7:0/4135990959 shutdown_connections 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 --2- 172.21.15.7:0/4135990959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e00106410 0x7f9e0010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 --2- 172.21.15.7:0/4135990959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e00101af0 0x7f9e00101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 --2- 172.21.15.7:0/4135990959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e001024a0 0x7f9e00105d30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 -- 172.21.15.7:0/4135990959 >> 172.21.15.7:0/4135990959 conn(0x7f9e00077690 msgr2=0x7f9e000feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 -- 172.21.15.7:0/4135990959 shutdown_connections 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 -- 172.21.15.7:0/4135990959 wait complete. 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 Processor -- start 2024-03-26T08:41:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.492+0000 7f9e07c56700 1 -- start start 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e07c56700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e00101af0 0x7f9e001a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e07c56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e001024a0 0x7f9e001a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e07c56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e00106410 0x7f9e0019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e07c56700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e00110300 con 0x7f9e001024a0 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e07c56700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e00110180 con 0x7f9e00106410 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e07c56700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e00110480 con 0x7f9e00101af0 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e059f2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e00101af0 0x7f9e001a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e059f2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e00101af0 0x7f9e001a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42248/0 (socket says 172.21.15.7:42248) 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e059f2700 1 -- 172.21.15.7:0/1845291730 learned_addr learned my addr 172.21.15.7:0/1845291730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e061f3700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e00106410 0x7f9e0019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e059f2700 1 -- 172.21.15.7:0/1845291730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e00106410 msgr2=0x7f9e0019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e059f2700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e00106410 0x7f9e0019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e059f2700 1 -- 172.21.15.7:0/1845291730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e001024a0 msgr2=0x7f9e001a0e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e059f2700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e001024a0 0x7f9e001a0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.493+0000 7f9e059f2700 1 -- 172.21.15.7:0/1845291730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0019b1f0 con 0x7f9e00101af0 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.494+0000 7f9e059f2700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e00101af0 0x7f9e001a08d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00d980 tx=0x7f9dfc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.494+0000 7f9df6ffd700 1 -- 172.21.15.7:0/1845291730 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc016070 con 0x7f9e00101af0 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.494+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0019b4e0 con 0x7f9e00101af0 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.494+0000 7f9df6ffd700 1 -- 172.21.15.7:0/1845291730 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9dfc004d10 con 0x7f9e00101af0 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.494+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e001a77a0 con 0x7f9e00101af0 2024-03-26T08:41:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.494+0000 7f9df6ffd700 1 -- 172.21.15.7:0/1845291730 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0055f0 con 0x7f9e00101af0 2024-03-26T08:41:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.495+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e001028a0 con 0x7f9e00101af0 2024-03-26T08:41:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.496+0000 7f9df6ffd700 1 -- 172.21.15.7:0/1845291730 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f9dfc004750 con 0x7f9e00101af0 2024-03-26T08:41:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.497+0000 7f9df6ffd700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9dec0a2900 0x7f9dec0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.497+0000 7f9e051f1700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9dec0a2900 0x7f9dec0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.497+0000 7f9df6ffd700 1 -- 172.21.15.7:0/1845291730 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9dfc0cef80 con 0x7f9e00101af0 2024-03-26T08:41:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.497+0000 7f9e051f1700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9dec0a2900 0x7f9dec0a4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009230 tx=0x7f9df0009f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:21.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.505+0000 7f9df6ffd700 1 -- 172.21.15.7:0/1845291730 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9dfc01f020 con 0x7f9e00101af0 2024-03-26T08:41:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.711+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9e0002da30 con 0x7f9e00101af0 2024-03-26T08:41:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.711+0000 7f9df6ffd700 1 -- 172.21.15.7:0/1845291730 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9dfc09bec0 con 0x7f9e00101af0 2024-03-26T08:41:21.703 INFO:teuthology.orchestra.run.smithi007.stdout:55834574927 2024-03-26T08:41:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9dec0a2900 msgr2=0x7f9dec0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9dec0a2900 0x7f9dec0a4dc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009230 tx=0x7f9df0009f80 comp rx=0 tx=0).stop 2024-03-26T08:41:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e00101af0 msgr2=0x7f9e001a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e00101af0 0x7f9e001a08d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00d980 tx=0x7f9dfc00dc90 comp rx=0 tx=0).stop 2024-03-26T08:41:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 shutdown_connections 2024-03-26T08:41:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9dec0a2900 0x7f9dec0a4dc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e00101af0 0x7f9e001a08d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e00106410 0x7f9e0019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 --2- 172.21.15.7:0/1845291730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e001024a0 0x7f9e001a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 >> 172.21.15.7:0/1845291730 conn(0x7f9e00077690 msgr2=0x7f9e000feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 shutdown_connections 2024-03-26T08:41:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:21.714+0000 7f9e07c56700 1 -- 172.21.15.7:0/1845291730 wait complete. 2024-03-26T08:41:22.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:22 smithi007 ceph-mon[32462]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:22.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:22 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3688715112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T08:41:22.221 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:22 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1845291730' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T08:41:22.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:22 smithi016 ceph-mon[35851]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:22 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3688715112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T08:41:22.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:22 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1845291730' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T08:41:22.559 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738451 for osd.0 2024-03-26T08:41:22.560 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:22.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:22 smithi044 ceph-mon[36461]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:22.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:22 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3688715112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T08:41:22.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:22 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1845291730' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T08:41:22.577 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332138 got 219043332144 for osd.7 2024-03-26T08:41:22.578 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:23.567 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574927 for osd.1 2024-03-26T08:41:23.568 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.761+0000 7fdb3efd0700 1 -- 172.21.15.7:0/887795628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 msgr2=0x7fdb38105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.761+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/887795628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 0x7fdb38105d30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdb240099c0 tx=0x7fdb24009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.761+0000 7fdb3efd0700 1 -- 172.21.15.7:0/887795628 shutdown_connections 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.761+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/887795628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb38106410 0x7fdb3810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.761+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/887795628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101af0 0x7fdb38101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.761+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/887795628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 0x7fdb38105d30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.761+0000 7fdb3efd0700 1 -- 172.21.15.7:0/887795628 >> 172.21.15.7:0/887795628 conn(0x7fdb38077690 msgr2=0x7fdb380feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.761+0000 7fdb3efd0700 1 -- 172.21.15.7:0/887795628 shutdown_connections 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.761+0000 7fdb3efd0700 1 -- 172.21.15.7:0/887795628 wait complete. 2024-03-26T08:41:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3efd0700 1 Processor -- start 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3efd0700 1 -- start start 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3efd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101af0 0x7fdb380fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3efd0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 0x7fdb380fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3efd0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb38106410 0x7fdb380fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3efd0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb381113d0 con 0x7fdb381024a0 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3efd0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38111250 con 0x7fdb38101af0 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3efd0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38111550 con 0x7fdb38106410 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb37fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 0x7fdb380fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb37fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 0x7fdb380fd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49842/0 (socket says 172.21.15.7:49842) 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb37fff700 1 -- 172.21.15.7:0/582968585 learned_addr learned my addr 172.21.15.7:0/582968585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3cd6c700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101af0 0x7fdb380fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3d56d700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb38106410 0x7fdb380fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb37fff700 1 -- 172.21.15.7:0/582968585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb38106410 msgr2=0x7fdb380fd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb37fff700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb38106410 0x7fdb380fd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb37fff700 1 -- 172.21.15.7:0/582968585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101af0 msgr2=0x7fdb380fcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb37fff700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101af0 0x7fdb380fcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb37fff700 1 -- 172.21.15.7:0/582968585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb381a6ed0 con 0x7fdb381024a0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.762+0000 7fdb3cd6c700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101af0 0x7fdb380fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.763+0000 7fdb37fff700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 0x7fdb380fd0b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400f690 tx=0x7fdb24005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.763+0000 7fdb35ffb700 1 -- 172.21.15.7:0/582968585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24027070 con 0x7fdb381024a0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.763+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb381a7160 con 0x7fdb381024a0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.763+0000 7fdb35ffb700 1 -- 172.21.15.7:0/582968585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb24003680 con 0x7fdb381024a0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.763+0000 7fdb35ffb700 1 -- 172.21.15.7:0/582968585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb240173f0 con 0x7fdb381024a0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.763+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb381a7610 con 0x7fdb381024a0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.764+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb38068e50 con 0x7fdb381024a0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.766+0000 7fdb35ffb700 1 -- 172.21.15.7:0/582968585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fdb2402a070 con 0x7fdb381024a0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.766+0000 7fdb35ffb700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb280a2920 0x7fdb280a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.766+0000 7fdb3cd6c700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb280a2920 0x7fdb280a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.766+0000 7fdb35ffb700 1 -- 172.21.15.7:0/582968585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb240cff20 con 0x7fdb381024a0 2024-03-26T08:41:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.767+0000 7fdb3cd6c700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb280a2920 0x7fdb280a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c011490 tx=0x7fdb2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.770+0000 7fdb35ffb700 1 -- 172.21.15.7:0/582968585 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb24031070 con 0x7fdb381024a0 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.012+0000 7f659dfc0700 1 -- 172.21.15.7:0/2873552069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598107510 msgr2=0x7f659810f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.012+0000 7f659dfc0700 1 --2- 172.21.15.7:0/2873552069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598107510 0x7f659810f090 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f659000b0d0 tx=0x7f659000b3e0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.012+0000 7f659dfc0700 1 -- 172.21.15.7:0/2873552069 shutdown_connections 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.012+0000 7f659dfc0700 1 --2- 172.21.15.7:0/2873552069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598107510 0x7f659810f090 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.012+0000 7f659dfc0700 1 --2- 172.21.15.7:0/2873552069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659806a930 0x7f659806ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.012+0000 7f659dfc0700 1 --2- 172.21.15.7:0/2873552069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f659806b2e0 0x7f6598106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.012+0000 7f659dfc0700 1 -- 172.21.15.7:0/2873552069 >> 172.21.15.7:0/2873552069 conn(0x7f65980770e0 msgr2=0x7f65980774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.012+0000 7f659dfc0700 1 -- 172.21.15.7:0/2873552069 shutdown_connections 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.012+0000 7f659dfc0700 1 -- 172.21.15.7:0/2873552069 wait complete. 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.013+0000 7f659dfc0700 1 Processor -- start 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.013+0000 7f659dfc0700 1 -- start start 2024-03-26T08:41:23.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f659dfc0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659806a930 0x7f659819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f659dfc0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659806b2e0 0x7f659819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f659dfc0700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6598107510 0x7f6598198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f659dfc0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6598112520 con 0x7f659806a930 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f659dfc0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65981123a0 con 0x7f659806b2e0 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f659dfc0700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65981126a0 con 0x7f6598107510 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f6596ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659806b2e0 0x7f659819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f6596ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659806b2e0 0x7f659819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44360/0 (socket says 172.21.15.7:44360) 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f6596ffd700 1 -- 172.21.15.7:0/922856852 learned_addr learned my addr 172.21.15.7:0/922856852 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f65977fe700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659806a930 0x7f659819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f6597fff700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6598107510 0x7f6598198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f6596ffd700 1 -- 172.21.15.7:0/922856852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6598107510 msgr2=0x7f6598198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f6596ffd700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6598107510 0x7f6598198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f6596ffd700 1 -- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659806a930 msgr2=0x7f659819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f6596ffd700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659806a930 0x7f659819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.014+0000 7f6596ffd700 1 -- 172.21.15.7:0/922856852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598199000 con 0x7f659806b2e0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.015+0000 7f6596ffd700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659806b2e0 0x7f659819ebe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f658c00e9c0 tx=0x7f658c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.015+0000 7f6594ff9700 1 -- 172.21.15.7:0/922856852 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c004a30 con 0x7f659806b2e0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.015+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65981992f0 con 0x7f659806b2e0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.015+0000 7f6594ff9700 1 -- 172.21.15.7:0/922856852 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f658c00cca0 con 0x7f659806b2e0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.015+0000 7f6594ff9700 1 -- 172.21.15.7:0/922856852 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c005690 con 0x7f659806b2e0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.015+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65981010c0 con 0x7f659806b2e0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.016+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65981038c0 con 0x7f659806b2e0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.017+0000 7f6594ff9700 1 -- 172.21.15.7:0/922856852 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f658c004cf0 con 0x7f659806b2e0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.017+0000 7f6594ff9700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65800a2920 0x7f65800a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.018+0000 7f65977fe700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65800a2920 0x7f65800a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.018+0000 7f6594ff9700 1 -- 172.21.15.7:0/922856852 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f658c0ced50 con 0x7f659806b2e0 2024-03-26T08:41:23.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.018+0000 7f65977fe700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65800a2920 0x7f65800a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6588010d80 tx=0x7f6588012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.022+0000 7f6594ff9700 1 -- 172.21.15.7:0/922856852 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f658c09bc90 con 0x7f659806b2e0 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.237+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f659802da30 con 0x7f659806b2e0 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.238+0000 7f6594ff9700 1 -- 172.21.15.7:0/922856852 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f658c09b590 con 0x7f659806b2e0 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65800a2920 msgr2=0x7f65800a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65800a2920 0x7f65800a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6588010d80 tx=0x7f6588012040 comp rx=0 tx=0).stop 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659806b2e0 msgr2=0x7f659819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659806b2e0 0x7f659819ebe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f658c00e9c0 tx=0x7f658c00ecd0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 shutdown_connections 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65800a2920 0x7f65800a4de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6598107510 0x7f6598198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659806a930 0x7f659819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 --2- 172.21.15.7:0/922856852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f659806b2e0 0x7f659819ebe0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.240+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 >> 172.21.15.7:0/922856852 conn(0x7f65980770e0 msgr2=0x7f65980ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:23.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.241+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 shutdown_connections 2024-03-26T08:41:23.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.241+0000 7f659dfc0700 1 -- 172.21.15.7:0/922856852 wait complete. 2024-03-26T08:41:23.930 INFO:teuthology.orchestra.run.smithi007.stdout:77309411403 2024-03-26T08:41:23.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.999+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdb380fe0a0 con 0x7fdb381024a0 2024-03-26T08:41:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:23.999+0000 7fdb35ffb700 1 -- 172.21.15.7:0/582968585 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdb24013070 con 0x7fdb381024a0 2024-03-26T08:41:23.991 INFO:teuthology.orchestra.run.smithi007.stdout:107374182470 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb280a2920 msgr2=0x7fdb280a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb280a2920 0x7fdb280a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c011490 tx=0x7fdb2c009260 comp rx=0 tx=0).stop 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 msgr2=0x7fdb380fd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 0x7fdb380fd0b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400f690 tx=0x7fdb24005f10 comp rx=0 tx=0).stop 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 shutdown_connections 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb280a2920 0x7fdb280a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb38106410 0x7fdb380fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb38101af0 0x7fdb380fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 --2- 172.21.15.7:0/582968585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb381024a0 0x7fdb380fd0b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 >> 172.21.15.7:0/582968585 conn(0x7fdb38077690 msgr2=0x7fdb38100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 shutdown_connections 2024-03-26T08:41:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:24.001+0000 7fdb3efd0700 1 -- 172.21.15.7:0/582968585 wait complete. 2024-03-26T08:41:24.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:24 smithi007 ceph-mon[32462]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T08:41:24.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:24 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/922856852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T08:41:24.147 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:24 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/582968585' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T08:41:24.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:24 smithi016 ceph-mon[35851]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T08:41:24.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:24 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/922856852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T08:41:24.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:24 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/582968585' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T08:41:24.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:24 smithi044 ceph-mon[36461]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T08:41:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:24 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/922856852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T08:41:24.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:24 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/582968585' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T08:41:24.758 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182464 got 107374182470 for osd.3 2024-03-26T08:41:24.758 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:25.165 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411403 for osd.2 2024-03-26T08:41:25.165 DEBUG:teuthology.parallel:result is None 2024-03-26T08:41:25.165 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-26T08:41:25.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T08:41:25.966 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:26.384 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:26 smithi007 ceph-mon[32462]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:26.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:26 smithi016 ceph-mon[35851]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:26.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:26 smithi044 ceph-mon[36461]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.720+0000 7f7eac44e700 1 -- 172.21.15.7:0/584739646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea4103050 msgr2=0x7f7ea4103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.720+0000 7f7eac44e700 1 --2- 172.21.15.7:0/584739646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea4103050 0x7f7ea4103430 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009990 tx=0x7f7ea0009ca0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.720+0000 7f7eac44e700 1 -- 172.21.15.7:0/584739646 shutdown_connections 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.720+0000 7f7eac44e700 1 --2- 172.21.15.7:0/584739646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4101290 0x7f7ea4110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.720+0000 7f7eac44e700 1 --2- 172.21.15.7:0/584739646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea4103050 0x7f7ea4103430 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.720+0000 7f7eac44e700 1 --2- 172.21.15.7:0/584739646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea41006e0 0x7f7ea4100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.720+0000 7f7eac44e700 1 -- 172.21.15.7:0/584739646 >> 172.21.15.7:0/584739646 conn(0x7f7ea4077660 msgr2=0x7f7ea40fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.720+0000 7f7eac44e700 1 -- 172.21.15.7:0/584739646 shutdown_connections 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.720+0000 7f7eac44e700 1 -- 172.21.15.7:0/584739646 wait complete. 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eac44e700 1 Processor -- start 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eac44e700 1 -- start start 2024-03-26T08:41:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eac44e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea41006e0 0x7f7ea41a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eac44e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4101290 0x7f7ea41a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eac44e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea4103050 0x7f7ea419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eac44e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41146d0 con 0x7f7ea41006e0 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eac44e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea4114550 con 0x7f7ea4101290 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eac44e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea4114850 con 0x7f7ea4103050 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa1ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea41006e0 0x7f7ea41a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa1ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea41006e0 0x7f7ea41a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35768/0 (socket says 172.21.15.7:35768) 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa1ea700 1 -- 172.21.15.7:0/1306070267 learned_addr learned my addr 172.21.15.7:0/1306070267 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7ea99e9700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4101290 0x7f7ea41a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa9eb700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea4103050 0x7f7ea419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa1ea700 1 -- 172.21.15.7:0/1306070267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea4103050 msgr2=0x7f7ea419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa1ea700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea4103050 0x7f7ea419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa1ea700 1 -- 172.21.15.7:0/1306070267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4101290 msgr2=0x7f7ea41a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa1ea700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4101290 0x7f7ea41a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa1ea700 1 -- 172.21.15.7:0/1306070267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea419b130 con 0x7f7ea41006e0 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7ea99e9700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4101290 0x7f7ea41a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.721+0000 7f7eaa1ea700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea41006e0 0x7f7ea41a07c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000c00 tx=0x7f7ea000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.722+0000 7f7e9b7fe700 1 -- 172.21.15.7:0/1306070267 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0027070 con 0x7f7ea41006e0 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.722+0000 7f7e9b7fe700 1 -- 172.21.15.7:0/1306070267 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ea0003d70 con 0x7f7ea41006e0 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.722+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea419b3c0 con 0x7f7ea41006e0 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.722+0000 7f7e9b7fe700 1 -- 172.21.15.7:0/1306070267 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000f460 con 0x7f7ea41006e0 2024-03-26T08:41:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.722+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea41a7690 con 0x7f7ea41006e0 2024-03-26T08:41:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.723+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea41016f0 con 0x7f7ea41006e0 2024-03-26T08:41:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.724+0000 7f7e9b7fe700 1 -- 172.21.15.7:0/1306070267 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f7ea002a070 con 0x7f7ea41006e0 2024-03-26T08:41:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.724+0000 7f7e9b7fe700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e900a2920 0x7f7e900a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.724+0000 7f7ea99e9700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e900a2920 0x7f7e900a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:27.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.724+0000 7f7e9b7fe700 1 -- 172.21.15.7:0/1306070267 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7ea00cf970 con 0x7f7ea41006e0 2024-03-26T08:41:27.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.725+0000 7f7ea99e9700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e900a2920 0x7f7e900a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40ffc40 tx=0x7f7e94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:27.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.729+0000 7f7e9b7fe700 1 -- 172.21.15.7:0/1306070267 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ea0013070 con 0x7f7ea41006e0 2024-03-26T08:41:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.980+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ea419beb0 con 0x7f7e900a2920 2024-03-26T08:41:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.988+0000 7f7e9b7fe700 1 -- 172.21.15.7:0/1306070267 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258970 (secure 0 0 0) 0x7f7ea419beb0 con 0x7f7e900a2920 2024-03-26T08:41:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e900a2920 msgr2=0x7f7e900a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e900a2920 0x7f7e900a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40ffc40 tx=0x7f7e94009260 comp rx=0 tx=0).stop 2024-03-26T08:41:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea41006e0 msgr2=0x7f7ea41a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea41006e0 0x7f7ea41a07c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0000c00 tx=0x7f7ea000bab0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 shutdown_connections 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7e900a2920 0x7f7e900a4de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ea4103050 0x7f7ea419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea41006e0 0x7f7ea41a07c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 --2- 172.21.15.7:0/1306070267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea4101290 0x7f7ea41a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 >> 172.21.15.7:0/1306070267 conn(0x7f7ea4077660 msgr2=0x7f7ea40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 shutdown_connections 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:27.991+0000 7f7eac44e700 1 -- 172.21.15.7:0/1306070267 wait complete. 2024-03-26T08:41:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-26T08:41:28.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:28 smithi007 ceph-mon[32462]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:28.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:28.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:28 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:28.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:28 smithi016 ceph-mon[35851]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:28 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:28 smithi044 ceph-mon[36461]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:28 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:28.634 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-03-26T08:41:26.318439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659005,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":569,"num_read_kb":402,"num_write":339,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373296,"kb_used_data":11580,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568528,"statfs":{"total":1151940427776,"available":1151558172672,"internally_reserved":0,"allocated":11857920,"data_stored":7445343,"data_compressed":177645,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003188"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420515+0000","last_change":"2024-03-26T08:40:09.939078+0000","last_active":"2024-03-26T08:40:59.420515+0000","last_peered":"2024-03-26T08:40:59.420515+0000","last_clean":"2024-03-26T08:40:59.420515+0000","last_became_active":"2024-03-26T08:40:09.937626+0000","last_became_peered":"2024-03-26T08:40:09.937626+0000","last_unstale":"2024-03-26T08:40:59.420515+0000","last_undegraded":"2024-03-26T08:40:59.420515+0000","last_fullsized":"2024-03-26T08:40:59.420515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:52:59.254181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416493+0000","last_change":"2024-03-26T08:40:09.924207+0000","last_active":"2024-03-26T08:40:59.416493+0000","last_peered":"2024-03-26T08:40:59.416493+0000","last_clean":"2024-03-26T08:40:59.416493+0000","last_became_active":"2024-03-26T08:40:09.923881+0000","last_became_peered":"2024-03-26T08:40:09.923881+0000","last_unstale":"2024-03-26T08:40:59.416493+0000","last_undegraded":"2024-03-26T08:40:59.416493+0000","last_fullsized":"2024-03-26T08:40:59.416493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:16:45.265521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416463+0000","last_change":"2024-03-26T08:40:09.929154+0000","last_active":"2024-03-26T08:40:59.416463+0000","last_peered":"2024-03-26T08:40:59.416463+0000","last_clean":"2024-03-26T08:40:59.416463+0000","last_became_active":"2024-03-26T08:40:09.928833+0000","last_became_peered":"2024-03-26T08:40:09.928833+0000","last_unstale":"2024-03-26T08:40:59.416463+0000","last_undegraded":"2024-03-26T08:40:59.416463+0000","last_fullsized":"2024-03-26T08:40:59.416463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:22.972210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420773+0000","last_change":"2024-03-26T08:40:09.930713+0000","last_active":"2024-03-26T08:40:59.420773+0000","last_peered":"2024-03-26T08:40:59.420773+0000","last_clean":"2024-03-26T08:40:59.420773+0000","last_became_active":"2024-03-26T08:40:09.930420+0000","last_became_peered":"2024-03-26T08:40:09.930420+0000","last_unstale":"2024-03-26T08:40:59.420773+0000","last_undegraded":"2024-03-26T08:40:59.420773+0000","last_fullsized":"2024-03-26T08:40:59.420773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:17:03.015703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416634+0000","last_change":"2024-03-26T08:40:09.923935+0000","last_active":"2024-03-26T08:40:59.416634+0000","last_peered":"2024-03-26T08:40:59.416634+0000","last_clean":"2024-03-26T08:40:59.416634+0000","last_became_active":"2024-03-26T08:40:09.923587+0000","last_became_peered":"2024-03-26T08:40:09.923587+0000","last_unstale":"2024-03-26T08:40:59.416634+0000","last_undegraded":"2024-03-26T08:40:59.416634+0000","last_fullsized":"2024-03-26T08:40:59.416634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:19:38.831448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502235+0000","last_change":"2024-03-26T08:40:09.922361+0000","last_active":"2024-03-26T08:40:59.502235+0000","last_peered":"2024-03-26T08:40:59.502235+0000","last_clean":"2024-03-26T08:40:59.502235+0000","last_became_active":"2024-03-26T08:40:09.921643+0000","last_became_peered":"2024-03-26T08:40:09.921643+0000","last_unstale":"2024-03-26T08:40:59.502235+0000","last_undegraded":"2024-03-26T08:40:59.502235+0000","last_fullsized":"2024-03-26T08:40:59.502235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:37:23.048778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416687+0000","last_change":"2024-03-26T08:40:09.932428+0000","last_active":"2024-03-26T08:40:59.416687+0000","last_peered":"2024-03-26T08:40:59.416687+0000","last_clean":"2024-03-26T08:40:59.416687+0000","last_became_active":"2024-03-26T08:40:09.932120+0000","last_became_peered":"2024-03-26T08:40:09.932120+0000","last_unstale":"2024-03-26T08:40:59.416687+0000","last_undegraded":"2024-03-26T08:40:59.416687+0000","last_fullsized":"2024-03-26T08:40:59.416687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:24.362568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419004+0000","last_change":"2024-03-26T08:40:09.928251+0000","last_active":"2024-03-26T08:40:59.419004+0000","last_peered":"2024-03-26T08:40:59.419004+0000","last_clean":"2024-03-26T08:40:59.419004+0000","last_became_active":"2024-03-26T08:40:09.927961+0000","last_became_peered":"2024-03-26T08:40:09.927961+0000","last_unstale":"2024-03-26T08:40:59.419004+0000","last_undegraded":"2024-03-26T08:40:59.419004+0000","last_fullsized":"2024-03-26T08:40:59.419004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:25:09.348033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502820+0000","last_change":"2024-03-26T08:40:09.930888+0000","last_active":"2024-03-26T08:40:59.502820+0000","last_peered":"2024-03-26T08:40:59.502820+0000","last_clean":"2024-03-26T08:40:59.502820+0000","last_became_active":"2024-03-26T08:40:09.930513+0000","last_became_peered":"2024-03-26T08:40:09.930513+0000","last_unstale":"2024-03-26T08:40:59.502820+0000","last_undegraded":"2024-03-26T08:40:59.502820+0000","last_fullsized":"2024-03-26T08:40:59.502820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:21.023569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502684+0000","last_change":"2024-03-26T08:40:09.935534+0000","last_active":"2024-03-26T08:40:59.502684+0000","last_peered":"2024-03-26T08:40:59.502684+0000","last_clean":"2024-03-26T08:40:59.502684+0000","last_became_active":"2024-03-26T08:40:09.934959+0000","last_became_peered":"2024-03-26T08:40:09.934959+0000","last_unstale":"2024-03-26T08:40:59.502684+0000","last_undegraded":"2024-03-26T08:40:59.502684+0000","last_fullsized":"2024-03-26T08:40:59.502684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:18:59.311180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417244+0000","last_change":"2024-03-26T08:40:09.930486+0000","last_active":"2024-03-26T08:40:59.417244+0000","last_peered":"2024-03-26T08:40:59.417244+0000","last_clean":"2024-03-26T08:40:59.417244+0000","last_became_active":"2024-03-26T08:40:09.930184+0000","last_became_peered":"2024-03-26T08:40:09.930184+0000","last_unstale":"2024-03-26T08:40:59.417244+0000","last_undegraded":"2024-03-26T08:40:59.417244+0000","last_fullsized":"2024-03-26T08:40:59.417244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:07.246878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420629+0000","last_change":"2024-03-26T08:40:09.924305+0000","last_active":"2024-03-26T08:40:59.420629+0000","last_peered":"2024-03-26T08:40:59.420629+0000","last_clean":"2024-03-26T08:40:59.420629+0000","last_became_active":"2024-03-26T08:40:09.923985+0000","last_became_peered":"2024-03-26T08:40:09.923985+0000","last_unstale":"2024-03-26T08:40:59.420629+0000","last_undegraded":"2024-03-26T08:40:59.420629+0000","last_fullsized":"2024-03-26T08:40:59.420629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:42:39.894245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.462998+0000","last_change":"2024-03-26T08:40:09.916637+0000","last_active":"2024-03-26T08:40:59.462998+0000","last_peered":"2024-03-26T08:40:59.462998+0000","last_clean":"2024-03-26T08:40:59.462998+0000","last_became_active":"2024-03-26T08:40:09.916314+0000","last_became_peered":"2024-03-26T08:40:09.916314+0000","last_unstale":"2024-03-26T08:40:59.462998+0000","last_undegraded":"2024-03-26T08:40:59.462998+0000","last_fullsized":"2024-03-26T08:40:59.462998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:45:10.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416204+0000","last_change":"2024-03-26T08:40:09.923753+0000","last_active":"2024-03-26T08:40:59.416204+0000","last_peered":"2024-03-26T08:40:59.416204+0000","last_clean":"2024-03-26T08:40:59.416204+0000","last_became_active":"2024-03-26T08:40:09.923427+0000","last_became_peered":"2024-03-26T08:40:09.923427+0000","last_unstale":"2024-03-26T08:40:59.416204+0000","last_undegraded":"2024-03-26T08:40:59.416204+0000","last_fullsized":"2024-03-26T08:40:59.416204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:13:16.711556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416255+0000","last_change":"2024-03-26T08:40:09.931879+0000","last_active":"2024-03-26T08:40:59.416255+0000","last_peered":"2024-03-26T08:40:59.416255+0000","last_clean":"2024-03-26T08:40:59.416255+0000","last_became_active":"2024-03-26T08:40:09.931589+0000","last_became_peered":"2024-03-26T08:40:09.931589+0000","last_unstale":"2024-03-26T08:40:59.416255+0000","last_undegraded":"2024-03-26T08:40:59.416255+0000","last_fullsized":"2024-03-26T08:40:59.416255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:22:01.038745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.501778+0000","last_change":"2024-03-26T08:40:09.930500+0000","last_active":"2024-03-26T08:40:59.501778+0000","last_peered":"2024-03-26T08:40:59.501778+0000","last_clean":"2024-03-26T08:40:59.501778+0000","last_became_active":"2024-03-26T08:40:09.930101+0000","last_became_peered":"2024-03-26T08:40:09.930101+0000","last_unstale":"2024-03-26T08:40:59.501778+0000","last_undegraded":"2024-03-26T08:40:59.501778+0000","last_fullsized":"2024-03-26T08:40:59.501778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:54.229004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420418+0000","last_change":"2024-03-26T08:40:09.939177+0000","last_active":"2024-03-26T08:40:59.420418+0000","last_peered":"2024-03-26T08:40:59.420418+0000","last_clean":"2024-03-26T08:40:59.420418+0000","last_became_active":"2024-03-26T08:40:09.937727+0000","last_became_peered":"2024-03-26T08:40:09.937727+0000","last_unstale":"2024-03-26T08:40:59.420418+0000","last_undegraded":"2024-03-26T08:40:59.420418+0000","last_fullsized":"2024-03-26T08:40:59.420418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:40:44.040015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421457+0000","last_change":"2024-03-26T08:40:09.928429+0000","last_active":"2024-03-26T08:40:59.421457+0000","last_peered":"2024-03-26T08:40:59.421457+0000","last_clean":"2024-03-26T08:40:59.421457+0000","last_became_active":"2024-03-26T08:40:09.928025+0000","last_became_peered":"2024-03-26T08:40:09.928025+0000","last_unstale":"2024-03-26T08:40:59.421457+0000","last_undegraded":"2024-03-26T08:40:59.421457+0000","last_fullsized":"2024-03-26T08:40:59.421457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:52:55.450291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422006+0000","last_change":"2024-03-26T08:40:09.930856+0000","last_active":"2024-03-26T08:40:59.422006+0000","last_peered":"2024-03-26T08:40:59.422006+0000","last_clean":"2024-03-26T08:40:59.422006+0000","last_became_active":"2024-03-26T08:40:09.930552+0000","last_became_peered":"2024-03-26T08:40:09.930552+0000","last_unstale":"2024-03-26T08:40:59.422006+0000","last_undegraded":"2024-03-26T08:40:59.422006+0000","last_fullsized":"2024-03-26T08:40:59.422006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:52.851287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421132+0000","last_change":"2024-03-26T08:40:09.230609+0000","last_active":"2024-03-26T08:40:59.421132+0000","last_peered":"2024-03-26T08:40:59.421132+0000","last_clean":"2024-03-26T08:40:59.421132+0000","last_became_active":"2024-03-26T08:40:09.230284+0000","last_became_peered":"2024-03-26T08:40:09.230284+0000","last_unstale":"2024-03-26T08:40:59.421132+0000","last_undegraded":"2024-03-26T08:40:59.421132+0000","last_fullsized":"2024-03-26T08:40:59.421132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:06.681725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.425767+0000","last_change":"2024-03-26T08:40:59.425449+0000","last_active":"2024-03-26T08:40:59.425767+0000","last_peered":"2024-03-26T08:40:59.425767+0000","last_clean":"2024-03-26T08:40:59.425767+0000","last_became_active":"2024-03-26T08:40:59.425164+0000","last_became_peered":"2024-03-26T08:40:59.425164+0000","last_unstale":"2024-03-26T08:40:59.425767+0000","last_undegraded":"2024-03-26T08:40:59.425767+0000","last_fullsized":"2024-03-26T08:40:59.425767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:09.846252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419617+0000","last_change":"2024-03-26T08:40:09.235451+0000","last_active":"2024-03-26T08:40:59.419617+0000","last_peered":"2024-03-26T08:40:59.419617+0000","last_clean":"2024-03-26T08:40:59.419617+0000","last_became_active":"2024-03-26T08:40:09.235154+0000","last_became_peered":"2024-03-26T08:40:09.235154+0000","last_unstale":"2024-03-26T08:40:59.419617+0000","last_undegraded":"2024-03-26T08:40:59.419617+0000","last_fullsized":"2024-03-26T08:40:59.419617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:37:38.418888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416541+0000","last_change":"2024-03-26T08:40:09.931728+0000","last_active":"2024-03-26T08:40:59.416541+0000","last_peered":"2024-03-26T08:40:59.416541+0000","last_clean":"2024-03-26T08:40:59.416541+0000","last_became_active":"2024-03-26T08:40:09.931401+0000","last_became_peered":"2024-03-26T08:40:09.931401+0000","last_unstale":"2024-03-26T08:40:59.416541+0000","last_undegraded":"2024-03-26T08:40:59.416541+0000","last_fullsized":"2024-03-26T08:40:59.416541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:02.229763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417066+0000","last_change":"2024-03-26T08:40:09.231247+0000","last_active":"2024-03-26T08:40:59.417066+0000","last_peered":"2024-03-26T08:40:59.417066+0000","last_clean":"2024-03-26T08:40:59.417066+0000","last_became_active":"2024-03-26T08:40:09.230914+0000","last_became_peered":"2024-03-26T08:40:09.230914+0000","last_unstale":"2024-03-26T08:40:59.417066+0000","last_undegraded":"2024-03-26T08:40:59.417066+0000","last_fullsized":"2024-03-26T08:40:59.417066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:03.956444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.501978+0000","last_change":"2024-03-26T08:40:09.925174+0000","last_active":"2024-03-26T08:40:59.501978+0000","last_peered":"2024-03-26T08:40:59.501978+0000","last_clean":"2024-03-26T08:40:59.501978+0000","last_became_active":"2024-03-26T08:40:09.924840+0000","last_became_peered":"2024-03-26T08:40:09.924840+0000","last_unstale":"2024-03-26T08:40:59.501978+0000","last_undegraded":"2024-03-26T08:40:59.501978+0000","last_fullsized":"2024-03-26T08:40:59.501978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:57:19.041245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416818+0000","last_change":"2024-03-26T08:40:09.229504+0000","last_active":"2024-03-26T08:40:59.416818+0000","last_peered":"2024-03-26T08:40:59.416818+0000","last_clean":"2024-03-26T08:40:59.416818+0000","last_became_active":"2024-03-26T08:40:09.226449+0000","last_became_peered":"2024-03-26T08:40:09.226449+0000","last_unstale":"2024-03-26T08:40:59.416818+0000","last_undegraded":"2024-03-26T08:40:59.416818+0000","last_fullsized":"2024-03-26T08:40:59.416818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:38.047424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419075+0000","last_change":"2024-03-26T08:40:09.930184+0000","last_active":"2024-03-26T08:40:59.419075+0000","last_peered":"2024-03-26T08:40:59.419075+0000","last_clean":"2024-03-26T08:40:59.419075+0000","last_became_active":"2024-03-26T08:40:09.927679+0000","last_became_peered":"2024-03-26T08:40:09.927679+0000","last_unstale":"2024-03-26T08:40:59.419075+0000","last_undegraded":"2024-03-26T08:40:59.419075+0000","last_fullsized":"2024-03-26T08:40:59.419075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:05:52.949814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":99,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:24.357997+0000","last_change":"2024-03-26T08:40:09.233025+0000","last_active":"2024-03-26T08:41:24.357997+0000","last_peered":"2024-03-26T08:41:24.357997+0000","last_clean":"2024-03-26T08:41:24.357997+0000","last_became_active":"2024-03-26T08:40:09.232654+0000","last_became_peered":"2024-03-26T08:40:09.232654+0000","last_unstale":"2024-03-26T08:41:24.357997+0000","last_undegraded":"2024-03-26T08:41:24.357997+0000","last_fullsized":"2024-03-26T08:41:24.357997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:01.745583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416612+0000","last_change":"2024-03-26T08:40:09.924187+0000","last_active":"2024-03-26T08:40:59.416612+0000","last_peered":"2024-03-26T08:40:59.416612+0000","last_clean":"2024-03-26T08:40:59.416612+0000","last_became_active":"2024-03-26T08:40:09.923861+0000","last_became_peered":"2024-03-26T08:40:09.923861+0000","last_unstale":"2024-03-26T08:40:59.416612+0000","last_undegraded":"2024-03-26T08:40:59.416612+0000","last_fullsized":"2024-03-26T08:40:59.416612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:12:17.284186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502123+0000","last_change":"2024-03-26T08:40:09.231368+0000","last_active":"2024-03-26T08:40:59.502123+0000","last_peered":"2024-03-26T08:40:59.502123+0000","last_clean":"2024-03-26T08:40:59.502123+0000","last_became_active":"2024-03-26T08:40:09.231040+0000","last_became_peered":"2024-03-26T08:40:09.231040+0000","last_unstale":"2024-03-26T08:40:59.502123+0000","last_undegraded":"2024-03-26T08:40:59.502123+0000","last_fullsized":"2024-03-26T08:40:59.502123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:14:20.681225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419928+0000","last_change":"2024-03-26T08:40:09.928125+0000","last_active":"2024-03-26T08:40:59.419928+0000","last_peered":"2024-03-26T08:40:59.419928+0000","last_clean":"2024-03-26T08:40:59.419928+0000","last_became_active":"2024-03-26T08:40:09.927616+0000","last_became_peered":"2024-03-26T08:40:09.927616+0000","last_unstale":"2024-03-26T08:40:59.419928+0000","last_undegraded":"2024-03-26T08:40:59.419928+0000","last_fullsized":"2024-03-26T08:40:59.419928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:12.764054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.462999+0000","last_change":"2024-03-26T08:40:09.229976+0000","last_active":"2024-03-26T08:40:59.462999+0000","last_peered":"2024-03-26T08:40:59.462999+0000","last_clean":"2024-03-26T08:40:59.462999+0000","last_became_active":"2024-03-26T08:40:09.229594+0000","last_became_peered":"2024-03-26T08:40:09.229594+0000","last_unstale":"2024-03-26T08:40:59.462999+0000","last_undegraded":"2024-03-26T08:40:59.462999+0000","last_fullsized":"2024-03-26T08:40:59.462999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:31:24.204853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.418918+0000","last_change":"2024-03-26T08:40:09.931290+0000","last_active":"2024-03-26T08:40:59.418918+0000","last_peered":"2024-03-26T08:40:59.418918+0000","last_clean":"2024-03-26T08:40:59.418918+0000","last_became_active":"2024-03-26T08:40:09.930961+0000","last_became_peered":"2024-03-26T08:40:09.930961+0000","last_unstale":"2024-03-26T08:40:59.418918+0000","last_undegraded":"2024-03-26T08:40:59.418918+0000","last_fullsized":"2024-03-26T08:40:59.418918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:36.460807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419243+0000","last_change":"2024-03-26T08:40:09.232166+0000","last_active":"2024-03-26T08:40:59.419243+0000","last_peered":"2024-03-26T08:40:59.419243+0000","last_clean":"2024-03-26T08:40:59.419243+0000","last_became_active":"2024-03-26T08:40:09.231840+0000","last_became_peered":"2024-03-26T08:40:09.231840+0000","last_unstale":"2024-03-26T08:40:59.419243+0000","last_undegraded":"2024-03-26T08:40:59.419243+0000","last_fullsized":"2024-03-26T08:40:59.419243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:51.636322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420606+0000","last_change":"2024-03-26T08:40:09.924680+0000","last_active":"2024-03-26T08:40:59.420606+0000","last_peered":"2024-03-26T08:40:59.420606+0000","last_clean":"2024-03-26T08:40:59.420606+0000","last_became_active":"2024-03-26T08:40:09.924314+0000","last_became_peered":"2024-03-26T08:40:09.924314+0000","last_unstale":"2024-03-26T08:40:59.420606+0000","last_undegraded":"2024-03-26T08:40:59.420606+0000","last_fullsized":"2024-03-26T08:40:59.420606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:49:20.780950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:24.357493+0000","last_change":"2024-03-26T08:40:09.235997+0000","last_active":"2024-03-26T08:41:24.357493+0000","last_peered":"2024-03-26T08:41:24.357493+0000","last_clean":"2024-03-26T08:41:24.357493+0000","last_became_active":"2024-03-26T08:40:09.235729+0000","last_became_peered":"2024-03-26T08:40:09.235729+0000","last_unstale":"2024-03-26T08:41:24.357493+0000","last_undegraded":"2024-03-26T08:41:24.357493+0000","last_fullsized":"2024-03-26T08:41:24.357493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:46:12.322059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502552+0000","last_change":"2024-03-26T08:40:09.926371+0000","last_active":"2024-03-26T08:40:59.502552+0000","last_peered":"2024-03-26T08:40:59.502552+0000","last_clean":"2024-03-26T08:40:59.502552+0000","last_became_active":"2024-03-26T08:40:09.925536+0000","last_became_peered":"2024-03-26T08:40:09.925536+0000","last_unstale":"2024-03-26T08:40:59.502552+0000","last_undegraded":"2024-03-26T08:40:59.502552+0000","last_fullsized":"2024-03-26T08:40:59.502552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:50:39.101684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417249+0000","last_change":"2024-03-26T08:40:09.229018+0000","last_active":"2024-03-26T08:40:59.417249+0000","last_peered":"2024-03-26T08:40:59.417249+0000","last_clean":"2024-03-26T08:40:59.417249+0000","last_became_active":"2024-03-26T08:40:09.228680+0000","last_became_peered":"2024-03-26T08:40:09.228680+0000","last_unstale":"2024-03-26T08:40:59.417249+0000","last_undegraded":"2024-03-26T08:40:59.417249+0000","last_fullsized":"2024-03-26T08:40:59.417249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:31:10.065127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416587+0000","last_change":"2024-03-26T08:40:09.926152+0000","last_active":"2024-03-26T08:40:59.416587+0000","last_peered":"2024-03-26T08:40:59.416587+0000","last_clean":"2024-03-26T08:40:59.416587+0000","last_became_active":"2024-03-26T08:40:09.925825+0000","last_became_peered":"2024-03-26T08:40:09.925825+0000","last_unstale":"2024-03-26T08:40:59.416587+0000","last_undegraded":"2024-03-26T08:40:59.416587+0000","last_fullsized":"2024-03-26T08:40:59.416587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:28:24.287706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419184+0000","last_change":"2024-03-26T08:40:09.234699+0000","last_active":"2024-03-26T08:40:59.419184+0000","last_peered":"2024-03-26T08:40:59.419184+0000","last_clean":"2024-03-26T08:40:59.419184+0000","last_became_active":"2024-03-26T08:40:09.234451+0000","last_became_peered":"2024-03-26T08:40:09.234451+0000","last_unstale":"2024-03-26T08:40:59.419184+0000","last_undegraded":"2024-03-26T08:40:59.419184+0000","last_fullsized":"2024-03-26T08:40:59.419184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:53.337418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502112+0000","last_change":"2024-03-26T08:40:09.927023+0000","last_active":"2024-03-26T08:40:59.502112+0000","last_peered":"2024-03-26T08:40:59.502112+0000","last_clean":"2024-03-26T08:40:59.502112+0000","last_became_active":"2024-03-26T08:40:09.926734+0000","last_became_peered":"2024-03-26T08:40:09.926734+0000","last_unstale":"2024-03-26T08:40:59.502112+0000","last_undegraded":"2024-03-26T08:40:59.502112+0000","last_fullsized":"2024-03-26T08:40:59.502112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:19:44.137513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":198,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:25.358474+0000","last_change":"2024-03-26T08:40:09.230059+0000","last_active":"2024-03-26T08:41:25.358474+0000","last_peered":"2024-03-26T08:41:25.358474+0000","last_clean":"2024-03-26T08:41:25.358474+0000","last_became_active":"2024-03-26T08:40:09.229776+0000","last_became_peered":"2024-03-26T08:40:09.229776+0000","last_unstale":"2024-03-26T08:41:25.358474+0000","last_undegraded":"2024-03-26T08:41:25.358474+0000","last_fullsized":"2024-03-26T08:41:25.358474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:00.917694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64402,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":149,"num_read_kb":137,"num_write":31,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416899+0000","last_change":"2024-03-26T08:40:09.930702+0000","last_active":"2024-03-26T08:40:59.416899+0000","last_peered":"2024-03-26T08:40:59.416899+0000","last_clean":"2024-03-26T08:40:59.416899+0000","last_became_active":"2024-03-26T08:40:09.930400+0000","last_became_peered":"2024-03-26T08:40:09.930400+0000","last_unstale":"2024-03-26T08:40:59.416899+0000","last_undegraded":"2024-03-26T08:40:59.416899+0000","last_fullsized":"2024-03-26T08:40:59.416899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:03:12.897812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422276+0000","last_change":"2024-03-26T08:40:09.225972+0000","last_active":"2024-03-26T08:40:59.422276+0000","last_peered":"2024-03-26T08:40:59.422276+0000","last_clean":"2024-03-26T08:40:59.422276+0000","last_became_active":"2024-03-26T08:40:09.225524+0000","last_became_peered":"2024-03-26T08:40:09.225524+0000","last_unstale":"2024-03-26T08:40:59.422276+0000","last_undegraded":"2024-03-26T08:40:59.422276+0000","last_fullsized":"2024-03-26T08:40:59.422276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:08:17.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463062+0000","last_change":"2024-03-26T08:40:09.920628+0000","last_active":"2024-03-26T08:40:59.463062+0000","last_peered":"2024-03-26T08:40:59.463062+0000","last_clean":"2024-03-26T08:40:59.463062+0000","last_became_active":"2024-03-26T08:40:09.920300+0000","last_became_peered":"2024-03-26T08:40:09.920300+0000","last_unstale":"2024-03-26T08:40:59.463062+0000","last_undegraded":"2024-03-26T08:40:59.463062+0000","last_fullsized":"2024-03-26T08:40:59.463062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:02:16.503881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419684+0000","last_change":"2024-03-26T08:40:09.237802+0000","last_active":"2024-03-26T08:40:59.419684+0000","last_peered":"2024-03-26T08:40:59.419684+0000","last_clean":"2024-03-26T08:40:59.419684+0000","last_became_active":"2024-03-26T08:40:09.237425+0000","last_became_peered":"2024-03-26T08:40:09.237425+0000","last_unstale":"2024-03-26T08:40:59.419684+0000","last_undegraded":"2024-03-26T08:40:59.419684+0000","last_fullsized":"2024-03-26T08:40:59.419684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:05.953882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417003+0000","last_change":"2024-03-26T08:40:09.923221+0000","last_active":"2024-03-26T08:40:59.417003+0000","last_peered":"2024-03-26T08:40:59.417003+0000","last_clean":"2024-03-26T08:40:59.417003+0000","last_became_active":"2024-03-26T08:40:09.922894+0000","last_became_peered":"2024-03-26T08:40:09.922894+0000","last_unstale":"2024-03-26T08:40:59.417003+0000","last_undegraded":"2024-03-26T08:40:59.417003+0000","last_fullsized":"2024-03-26T08:40:59.417003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:24.887625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":91,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:20.355732+0000","last_change":"2024-03-26T08:40:09.225179+0000","last_active":"2024-03-26T08:41:20.355732+0000","last_peered":"2024-03-26T08:41:20.355732+0000","last_clean":"2024-03-26T08:41:20.355732+0000","last_became_active":"2024-03-26T08:40:09.224861+0000","last_became_peered":"2024-03-26T08:40:09.224861+0000","last_unstale":"2024-03-26T08:41:20.355732+0000","last_undegraded":"2024-03-26T08:41:20.355732+0000","last_fullsized":"2024-03-26T08:41:20.355732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:37:54.843284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.503355+0000","last_change":"2024-03-26T08:40:59.503355+0000","last_active":"2024-03-26T08:40:59.503355+0000","last_peered":"2024-03-26T08:40:59.503355+0000","last_clean":"2024-03-26T08:40:59.503355+0000","last_became_active":"2024-03-26T08:40:59.503009+0000","last_became_peered":"2024-03-26T08:40:59.503009+0000","last_unstale":"2024-03-26T08:40:59.503355+0000","last_undegraded":"2024-03-26T08:40:59.503355+0000","last_fullsized":"2024-03-26T08:40:59.503355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:08:20.694763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416469+0000","last_change":"2024-03-26T08:40:09.232024+0000","last_active":"2024-03-26T08:40:59.416469+0000","last_peered":"2024-03-26T08:40:59.416469+0000","last_clean":"2024-03-26T08:40:59.416469+0000","last_became_active":"2024-03-26T08:40:09.231715+0000","last_became_peered":"2024-03-26T08:40:09.231715+0000","last_unstale":"2024-03-26T08:40:59.416469+0000","last_undegraded":"2024-03-26T08:40:59.416469+0000","last_fullsized":"2024-03-26T08:40:59.416469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:02.229042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419447+0000","last_change":"2024-03-26T08:40:09.926893+0000","last_active":"2024-03-26T08:40:59.419447+0000","last_peered":"2024-03-26T08:40:59.419447+0000","last_clean":"2024-03-26T08:40:59.419447+0000","last_became_active":"2024-03-26T08:40:09.926529+0000","last_became_peered":"2024-03-26T08:40:09.926529+0000","last_unstale":"2024-03-26T08:40:59.419447+0000","last_undegraded":"2024-03-26T08:40:59.419447+0000","last_fullsized":"2024-03-26T08:40:59.419447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:10.189407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.428905+0000","last_change":"2024-03-26T08:40:59.428905+0000","last_active":"2024-03-26T08:40:59.428905+0000","last_peered":"2024-03-26T08:40:59.428905+0000","last_clean":"2024-03-26T08:40:59.428905+0000","last_became_active":"2024-03-26T08:40:59.423345+0000","last_became_peered":"2024-03-26T08:40:59.423345+0000","last_unstale":"2024-03-26T08:40:59.428905+0000","last_undegraded":"2024-03-26T08:40:59.428905+0000","last_fullsized":"2024-03-26T08:40:59.428905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:49.069421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.464333+0000","last_change":"2024-03-26T08:40:59.464333+0000","last_active":"2024-03-26T08:40:59.464333+0000","last_peered":"2024-03-26T08:40:59.464333+0000","last_clean":"2024-03-26T08:40:59.464333+0000","last_became_active":"2024-03-26T08:40:59.464016+0000","last_became_peered":"2024-03-26T08:40:59.464016+0000","last_unstale":"2024-03-26T08:40:59.464333+0000","last_undegraded":"2024-03-26T08:40:59.464333+0000","last_fullsized":"2024-03-26T08:40:59.464333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:50:05.770568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420974+0000","last_change":"2024-03-26T08:40:09.224385+0000","last_active":"2024-03-26T08:40:59.420974+0000","last_peered":"2024-03-26T08:40:59.420974+0000","last_clean":"2024-03-26T08:40:59.420974+0000","last_became_active":"2024-03-26T08:40:09.223991+0000","last_became_peered":"2024-03-26T08:40:09.223991+0000","last_unstale":"2024-03-26T08:40:59.420974+0000","last_undegraded":"2024-03-26T08:40:59.420974+0000","last_fullsized":"2024-03-26T08:40:59.420974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:55.808922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502351+0000","last_change":"2024-03-26T08:40:09.921072+0000","last_active":"2024-03-26T08:40:59.502351+0000","last_peered":"2024-03-26T08:40:59.502351+0000","last_clean":"2024-03-26T08:40:59.502351+0000","last_became_active":"2024-03-26T08:40:09.920747+0000","last_became_peered":"2024-03-26T08:40:09.920747+0000","last_unstale":"2024-03-26T08:40:59.502351+0000","last_undegraded":"2024-03-26T08:40:59.502351+0000","last_fullsized":"2024-03-26T08:40:59.502351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:20.862765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416822+0000","last_change":"2024-03-26T08:40:09.231344+0000","last_active":"2024-03-26T08:40:59.416822+0000","last_peered":"2024-03-26T08:40:59.416822+0000","last_clean":"2024-03-26T08:40:59.416822+0000","last_became_active":"2024-03-26T08:40:09.229313+0000","last_became_peered":"2024-03-26T08:40:09.229313+0000","last_unstale":"2024-03-26T08:40:59.416822+0000","last_undegraded":"2024-03-26T08:40:59.416822+0000","last_fullsized":"2024-03-26T08:40:59.416822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:37.130884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502402+0000","last_change":"2024-03-26T08:40:09.926355+0000","last_active":"2024-03-26T08:40:59.502402+0000","last_peered":"2024-03-26T08:40:59.502402+0000","last_clean":"2024-03-26T08:40:59.502402+0000","last_became_active":"2024-03-26T08:40:09.922889+0000","last_became_peered":"2024-03-26T08:40:09.922889+0000","last_unstale":"2024-03-26T08:40:59.502402+0000","last_undegraded":"2024-03-26T08:40:59.502402+0000","last_fullsized":"2024-03-26T08:40:59.502402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:42:22.784811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419357+0000","last_change":"2024-03-26T08:40:09.919577+0000","last_active":"2024-03-26T08:40:59.419357+0000","last_peered":"2024-03-26T08:40:59.419357+0000","last_clean":"2024-03-26T08:40:59.419357+0000","last_became_active":"2024-03-26T08:40:09.919234+0000","last_became_peered":"2024-03-26T08:40:09.919234+0000","last_unstale":"2024-03-26T08:40:59.419357+0000","last_undegraded":"2024-03-26T08:40:59.419357+0000","last_fullsized":"2024-03-26T08:40:59.419357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:18.894052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:20.355351+0000","last_change":"2024-03-26T08:40:09.237190+0000","last_active":"2024-03-26T08:41:20.355351+0000","last_peered":"2024-03-26T08:41:20.355351+0000","last_clean":"2024-03-26T08:41:20.355351+0000","last_became_active":"2024-03-26T08:40:09.236862+0000","last_became_peered":"2024-03-26T08:40:09.236862+0000","last_unstale":"2024-03-26T08:41:20.355351+0000","last_undegraded":"2024-03-26T08:41:20.355351+0000","last_fullsized":"2024-03-26T08:41:20.355351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:29.900711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419315+0000","last_change":"2024-03-26T08:40:09.927444+0000","last_active":"2024-03-26T08:40:59.419315+0000","last_peered":"2024-03-26T08:40:59.419315+0000","last_clean":"2024-03-26T08:40:59.419315+0000","last_became_active":"2024-03-26T08:40:09.927152+0000","last_became_peered":"2024-03-26T08:40:09.927152+0000","last_unstale":"2024-03-26T08:40:59.419315+0000","last_undegraded":"2024-03-26T08:40:59.419315+0000","last_fullsized":"2024-03-26T08:40:59.419315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:44.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420563+0000","last_change":"2024-03-26T08:40:09.224831+0000","last_active":"2024-03-26T08:40:59.420563+0000","last_peered":"2024-03-26T08:40:59.420563+0000","last_clean":"2024-03-26T08:40:59.420563+0000","last_became_active":"2024-03-26T08:40:09.224447+0000","last_became_peered":"2024-03-26T08:40:09.224447+0000","last_unstale":"2024-03-26T08:40:59.420563+0000","last_undegraded":"2024-03-26T08:40:59.420563+0000","last_fullsized":"2024-03-26T08:40:59.420563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:25:50.935342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416700+0000","last_change":"2024-03-26T08:38:54.696342+0000","last_active":"2024-03-26T08:40:59.416700+0000","last_peered":"2024-03-26T08:40:59.416700+0000","last_clean":"2024-03-26T08:40:59.416700+0000","last_became_active":"2024-03-26T08:38:54.695839+0000","last_became_peered":"2024-03-26T08:38:54.695839+0000","last_unstale":"2024-03-26T08:40:59.416700+0000","last_undegraded":"2024-03-26T08:40:59.416700+0000","last_fullsized":"2024-03-26T08:40:59.416700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_clean_scrub_stamp":"2024-03-26T08:35:27.189917+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:13.219778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463034+0000","last_change":"2024-03-26T08:40:09.930683+0000","last_active":"2024-03-26T08:40:59.463034+0000","last_peered":"2024-03-26T08:40:59.463034+0000","last_clean":"2024-03-26T08:40:59.463034+0000","last_became_active":"2024-03-26T08:40:09.930341+0000","last_became_peered":"2024-03-26T08:40:09.930341+0000","last_unstale":"2024-03-26T08:40:59.463034+0000","last_undegraded":"2024-03-26T08:40:59.463034+0000","last_fullsized":"2024-03-26T08:40:59.463034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:39:20.232007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416894+0000","last_change":"2024-03-26T08:40:09.228548+0000","last_active":"2024-03-26T08:40:59.416894+0000","last_peered":"2024-03-26T08:40:59.416894+0000","last_clean":"2024-03-26T08:40:59.416894+0000","last_became_active":"2024-03-26T08:40:09.228076+0000","last_became_peered":"2024-03-26T08:40:09.228076+0000","last_unstale":"2024-03-26T08:40:59.416894+0000","last_undegraded":"2024-03-26T08:40:59.416894+0000","last_fullsized":"2024-03-26T08:40:59.416894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:52:51.641128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416941+0000","last_change":"2024-03-26T08:40:09.235632+0000","last_active":"2024-03-26T08:40:59.416941+0000","last_peered":"2024-03-26T08:40:59.416941+0000","last_clean":"2024-03-26T08:40:59.416941+0000","last_became_active":"2024-03-26T08:40:09.235334+0000","last_became_peered":"2024-03-26T08:40:09.235334+0000","last_unstale":"2024-03-26T08:40:59.416941+0000","last_undegraded":"2024-03-26T08:40:59.416941+0000","last_fullsized":"2024-03-26T08:40:59.416941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:38:27.064431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463180+0000","last_change":"2024-03-26T08:40:09.929137+0000","last_active":"2024-03-26T08:40:59.463180+0000","last_peered":"2024-03-26T08:40:59.463180+0000","last_clean":"2024-03-26T08:40:59.463180+0000","last_became_active":"2024-03-26T08:40:09.928831+0000","last_became_peered":"2024-03-26T08:40:09.928831+0000","last_unstale":"2024-03-26T08:40:59.463180+0000","last_undegraded":"2024-03-26T08:40:59.463180+0000","last_fullsized":"2024-03-26T08:40:59.463180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:19.235168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421503+0000","last_change":"2024-03-26T08:40:09.227153+0000","last_active":"2024-03-26T08:40:59.421503+0000","last_peered":"2024-03-26T08:40:59.421503+0000","last_clean":"2024-03-26T08:40:59.421503+0000","last_became_active":"2024-03-26T08:40:09.226772+0000","last_became_peered":"2024-03-26T08:40:09.226772+0000","last_unstale":"2024-03-26T08:40:59.421503+0000","last_undegraded":"2024-03-26T08:40:59.421503+0000","last_fullsized":"2024-03-26T08:40:59.421503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:33.159538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419240+0000","last_change":"2024-03-26T08:40:09.934589+0000","last_active":"2024-03-26T08:40:59.419240+0000","last_peered":"2024-03-26T08:40:59.419240+0000","last_clean":"2024-03-26T08:40:59.419240+0000","last_became_active":"2024-03-26T08:40:09.934312+0000","last_became_peered":"2024-03-26T08:40:09.934312+0000","last_unstale":"2024-03-26T08:40:59.419240+0000","last_undegraded":"2024-03-26T08:40:59.419240+0000","last_fullsized":"2024-03-26T08:40:59.419240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:19:27.737506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420385+0000","last_change":"2024-03-26T08:40:09.230014+0000","last_active":"2024-03-26T08:40:59.420385+0000","last_peered":"2024-03-26T08:40:59.420385+0000","last_clean":"2024-03-26T08:40:59.420385+0000","last_became_active":"2024-03-26T08:40:09.229682+0000","last_became_peered":"2024-03-26T08:40:09.229682+0000","last_unstale":"2024-03-26T08:40:59.420385+0000","last_undegraded":"2024-03-26T08:40:59.420385+0000","last_fullsized":"2024-03-26T08:40:59.420385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:52.725810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.464419+0000","last_change":"2024-03-26T08:40:59.464419+0000","last_active":"2024-03-26T08:40:59.464419+0000","last_peered":"2024-03-26T08:40:59.464419+0000","last_clean":"2024-03-26T08:40:59.464419+0000","last_became_active":"2024-03-26T08:40:59.464051+0000","last_became_peered":"2024-03-26T08:40:59.464051+0000","last_unstale":"2024-03-26T08:40:59.464419+0000","last_undegraded":"2024-03-26T08:40:59.464419+0000","last_fullsized":"2024-03-26T08:40:59.464419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:01:47.975380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.501822+0000","last_change":"2024-03-26T08:40:09.226634+0000","last_active":"2024-03-26T08:40:59.501822+0000","last_peered":"2024-03-26T08:40:59.501822+0000","last_clean":"2024-03-26T08:40:59.501822+0000","last_became_active":"2024-03-26T08:40:09.226248+0000","last_became_peered":"2024-03-26T08:40:09.226248+0000","last_unstale":"2024-03-26T08:40:59.501822+0000","last_undegraded":"2024-03-26T08:40:59.501822+0000","last_fullsized":"2024-03-26T08:40:59.501822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:24:00.390573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416466+0000","last_change":"2024-03-26T08:40:09.934848+0000","last_active":"2024-03-26T08:40:59.416466+0000","last_peered":"2024-03-26T08:40:59.416466+0000","last_clean":"2024-03-26T08:40:59.416466+0000","last_became_active":"2024-03-26T08:40:09.934521+0000","last_became_peered":"2024-03-26T08:40:09.934521+0000","last_unstale":"2024-03-26T08:40:59.416466+0000","last_undegraded":"2024-03-26T08:40:59.416466+0000","last_fullsized":"2024-03-26T08:40:59.416466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:24:25.994961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419371+0000","last_change":"2024-03-26T08:40:09.222945+0000","last_active":"2024-03-26T08:40:59.419371+0000","last_peered":"2024-03-26T08:40:59.419371+0000","last_clean":"2024-03-26T08:40:59.419371+0000","last_became_active":"2024-03-26T08:40:09.222564+0000","last_became_peered":"2024-03-26T08:40:09.222564+0000","last_unstale":"2024-03-26T08:40:59.419371+0000","last_undegraded":"2024-03-26T08:40:59.419371+0000","last_fullsized":"2024-03-26T08:40:59.419371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:35:11.181812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419131+0000","last_change":"2024-03-26T08:40:09.932674+0000","last_active":"2024-03-26T08:40:59.419131+0000","last_peered":"2024-03-26T08:40:59.419131+0000","last_clean":"2024-03-26T08:40:59.419131+0000","last_became_active":"2024-03-26T08:40:09.932346+0000","last_became_peered":"2024-03-26T08:40:09.932346+0000","last_unstale":"2024-03-26T08:40:59.419131+0000","last_undegraded":"2024-03-26T08:40:59.419131+0000","last_fullsized":"2024-03-26T08:40:59.419131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:12:45.871467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.431219+0000","last_change":"2024-03-26T08:40:59.431219+0000","last_active":"2024-03-26T08:40:59.431219+0000","last_peered":"2024-03-26T08:40:59.431219+0000","last_clean":"2024-03-26T08:40:59.431219+0000","last_became_active":"2024-03-26T08:40:59.424868+0000","last_became_peered":"2024-03-26T08:40:59.424868+0000","last_unstale":"2024-03-26T08:40:59.431219+0000","last_undegraded":"2024-03-26T08:40:59.431219+0000","last_fullsized":"2024-03-26T08:40:59.431219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:02.837090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419228+0000","last_change":"2024-03-26T08:40:09.922766+0000","last_active":"2024-03-26T08:40:59.419228+0000","last_peered":"2024-03-26T08:40:59.419228+0000","last_clean":"2024-03-26T08:40:59.419228+0000","last_became_active":"2024-03-26T08:40:09.922453+0000","last_became_peered":"2024-03-26T08:40:09.922453+0000","last_unstale":"2024-03-26T08:40:59.419228+0000","last_undegraded":"2024-03-26T08:40:59.419228+0000","last_fullsized":"2024-03-26T08:40:59.419228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:13:28.740291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416510+0000","last_change":"2024-03-26T08:40:09.228114+0000","last_active":"2024-03-26T08:40:59.416510+0000","last_peered":"2024-03-26T08:40:59.416510+0000","last_clean":"2024-03-26T08:40:59.416510+0000","last_became_active":"2024-03-26T08:40:09.227803+0000","last_became_peered":"2024-03-26T08:40:09.227803+0000","last_unstale":"2024-03-26T08:40:59.416510+0000","last_undegraded":"2024-03-26T08:40:59.416510+0000","last_fullsized":"2024-03-26T08:40:59.416510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:31.717981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416928+0000","last_change":"2024-03-26T08:40:09.929671+0000","last_active":"2024-03-26T08:40:59.416928+0000","last_peered":"2024-03-26T08:40:59.416928+0000","last_clean":"2024-03-26T08:40:59.416928+0000","last_became_active":"2024-03-26T08:40:09.929222+0000","last_became_peered":"2024-03-26T08:40:09.929222+0000","last_unstale":"2024-03-26T08:40:59.416928+0000","last_undegraded":"2024-03-26T08:40:59.416928+0000","last_fullsized":"2024-03-26T08:40:59.416928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:00:50.579569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422136+0000","last_change":"2024-03-26T08:40:09.228994+0000","last_active":"2024-03-26T08:40:59.422136+0000","last_peered":"2024-03-26T08:40:59.422136+0000","last_clean":"2024-03-26T08:40:59.422136+0000","last_became_active":"2024-03-26T08:40:09.227138+0000","last_became_peered":"2024-03-26T08:40:09.227138+0000","last_unstale":"2024-03-26T08:40:59.422136+0000","last_undegraded":"2024-03-26T08:40:59.422136+0000","last_fullsized":"2024-03-26T08:40:59.422136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:06.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417219+0000","last_change":"2024-03-26T08:40:09.931752+0000","last_active":"2024-03-26T08:40:59.417219+0000","last_peered":"2024-03-26T08:40:59.417219+0000","last_clean":"2024-03-26T08:40:59.417219+0000","last_became_active":"2024-03-26T08:40:09.931436+0000","last_became_peered":"2024-03-26T08:40:09.931436+0000","last_unstale":"2024-03-26T08:40:59.417219+0000","last_undegraded":"2024-03-26T08:40:59.417219+0000","last_fullsized":"2024-03-26T08:40:59.417219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:40:54.864614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.418961+0000","last_change":"2024-03-26T08:40:09.238356+0000","last_active":"2024-03-26T08:40:59.418961+0000","last_peered":"2024-03-26T08:40:59.418961+0000","last_clean":"2024-03-26T08:40:59.418961+0000","last_became_active":"2024-03-26T08:40:09.238066+0000","last_became_peered":"2024-03-26T08:40:09.238066+0000","last_unstale":"2024-03-26T08:40:59.418961+0000","last_undegraded":"2024-03-26T08:40:59.418961+0000","last_fullsized":"2024-03-26T08:40:59.418961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:13.851490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.427153+0000","last_change":"2024-03-26T08:40:59.427153+0000","last_active":"2024-03-26T08:40:59.427153+0000","last_peered":"2024-03-26T08:40:59.427153+0000","last_clean":"2024-03-26T08:40:59.427153+0000","last_became_active":"2024-03-26T08:40:59.423044+0000","last_became_peered":"2024-03-26T08:40:59.423044+0000","last_unstale":"2024-03-26T08:40:59.427153+0000","last_undegraded":"2024-03-26T08:40:59.427153+0000","last_fullsized":"2024-03-26T08:40:59.427153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:20.181241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463434+0000","last_change":"2024-03-26T08:40:09.238009+0000","last_active":"2024-03-26T08:40:59.463434+0000","last_peered":"2024-03-26T08:40:59.463434+0000","last_clean":"2024-03-26T08:40:59.463434+0000","last_became_active":"2024-03-26T08:40:09.237697+0000","last_became_peered":"2024-03-26T08:40:09.237697+0000","last_unstale":"2024-03-26T08:40:59.463434+0000","last_undegraded":"2024-03-26T08:40:59.463434+0000","last_fullsized":"2024-03-26T08:40:59.463434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:17:33.179285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417423+0000","last_change":"2024-03-26T08:40:09.934672+0000","last_active":"2024-03-26T08:40:59.417423+0000","last_peered":"2024-03-26T08:40:59.417423+0000","last_clean":"2024-03-26T08:40:59.417423+0000","last_became_active":"2024-03-26T08:40:09.934365+0000","last_became_peered":"2024-03-26T08:40:09.934365+0000","last_unstale":"2024-03-26T08:40:59.417423+0000","last_undegraded":"2024-03-26T08:40:59.417423+0000","last_fullsized":"2024-03-26T08:40:59.417423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:41:22.318481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419226+0000","last_change":"2024-03-26T08:40:09.933910+0000","last_active":"2024-03-26T08:40:59.419226+0000","last_peered":"2024-03-26T08:40:59.419226+0000","last_clean":"2024-03-26T08:40:59.419226+0000","last_became_active":"2024-03-26T08:40:09.933598+0000","last_became_peered":"2024-03-26T08:40:09.933598+0000","last_unstale":"2024-03-26T08:40:59.419226+0000","last_undegraded":"2024-03-26T08:40:59.419226+0000","last_fullsized":"2024-03-26T08:40:59.419226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:19.877984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422139+0000","last_change":"2024-03-26T08:40:09.935809+0000","last_active":"2024-03-26T08:40:59.422139+0000","last_peered":"2024-03-26T08:40:59.422139+0000","last_clean":"2024-03-26T08:40:59.422139+0000","last_became_active":"2024-03-26T08:40:09.935500+0000","last_became_peered":"2024-03-26T08:40:09.935500+0000","last_unstale":"2024-03-26T08:40:59.422139+0000","last_undegraded":"2024-03-26T08:40:59.422139+0000","last_fullsized":"2024-03-26T08:40:59.422139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:23.717845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421745+0000","last_change":"2024-03-26T08:40:09.928093+0000","last_active":"2024-03-26T08:40:59.421745+0000","last_peered":"2024-03-26T08:40:59.421745+0000","last_clean":"2024-03-26T08:40:59.421745+0000","last_became_active":"2024-03-26T08:40:09.927732+0000","last_became_peered":"2024-03-26T08:40:09.927732+0000","last_unstale":"2024-03-26T08:40:59.421745+0000","last_undegraded":"2024-03-26T08:40:59.421745+0000","last_fullsized":"2024-03-26T08:40:59.421745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:35:34.972435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463368+0000","last_change":"2024-03-26T08:40:59.463368+0000","last_active":"2024-03-26T08:40:59.463368+0000","last_peered":"2024-03-26T08:40:59.463368+0000","last_clean":"2024-03-26T08:40:59.463368+0000","last_became_active":"2024-03-26T08:40:59.462979+0000","last_became_peered":"2024-03-26T08:40:59.462979+0000","last_unstale":"2024-03-26T08:40:59.463368+0000","last_undegraded":"2024-03-26T08:40:59.463368+0000","last_fullsized":"2024-03-26T08:40:59.463368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:13:01.413240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463076+0000","last_change":"2024-03-26T08:40:09.919466+0000","last_active":"2024-03-26T08:40:59.463076+0000","last_peered":"2024-03-26T08:40:59.463076+0000","last_clean":"2024-03-26T08:40:59.463076+0000","last_became_active":"2024-03-26T08:40:09.919152+0000","last_became_peered":"2024-03-26T08:40:09.919152+0000","last_unstale":"2024-03-26T08:40:59.463076+0000","last_undegraded":"2024-03-26T08:40:59.463076+0000","last_fullsized":"2024-03-26T08:40:59.463076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:37.928485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419604+0000","last_change":"2024-03-26T08:40:09.927140+0000","last_active":"2024-03-26T08:40:59.419604+0000","last_peered":"2024-03-26T08:40:59.419604+0000","last_clean":"2024-03-26T08:40:59.419604+0000","last_became_active":"2024-03-26T08:40:09.926823+0000","last_became_peered":"2024-03-26T08:40:09.926823+0000","last_unstale":"2024-03-26T08:40:59.419604+0000","last_undegraded":"2024-03-26T08:40:59.419604+0000","last_fullsized":"2024-03-26T08:40:59.419604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:29:38.477123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422522+0000","last_change":"2024-03-26T08:40:09.934202+0000","last_active":"2024-03-26T08:40:59.422522+0000","last_peered":"2024-03-26T08:40:59.422522+0000","last_clean":"2024-03-26T08:40:59.422522+0000","last_became_active":"2024-03-26T08:40:09.933878+0000","last_became_peered":"2024-03-26T08:40:09.933878+0000","last_unstale":"2024-03-26T08:40:59.422522+0000","last_undegraded":"2024-03-26T08:40:59.422522+0000","last_fullsized":"2024-03-26T08:40:59.422522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:44:26.576140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419198+0000","last_change":"2024-03-26T08:40:09.929019+0000","last_active":"2024-03-26T08:40:59.419198+0000","last_peered":"2024-03-26T08:40:59.419198+0000","last_clean":"2024-03-26T08:40:59.419198+0000","last_became_active":"2024-03-26T08:40:09.928711+0000","last_became_peered":"2024-03-26T08:40:09.928711+0000","last_unstale":"2024-03-26T08:40:59.419198+0000","last_undegraded":"2024-03-26T08:40:59.419198+0000","last_fullsized":"2024-03-26T08:40:59.419198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:47.120826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419311+0000","last_change":"2024-03-26T08:40:09.927606+0000","last_active":"2024-03-26T08:40:59.419311+0000","last_peered":"2024-03-26T08:40:59.419311+0000","last_clean":"2024-03-26T08:40:59.419311+0000","last_became_active":"2024-03-26T08:40:09.926717+0000","last_became_peered":"2024-03-26T08:40:09.926717+0000","last_unstale":"2024-03-26T08:40:59.419311+0000","last_undegraded":"2024-03-26T08:40:59.419311+0000","last_fullsized":"2024-03-26T08:40:59.419311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:51:52.567252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419449+0000","last_change":"2024-03-26T08:40:09.934659+0000","last_active":"2024-03-26T08:40:59.419449+0000","last_peered":"2024-03-26T08:40:59.419449+0000","last_clean":"2024-03-26T08:40:59.419449+0000","last_became_active":"2024-03-26T08:40:09.934346+0000","last_became_peered":"2024-03-26T08:40:09.934346+0000","last_unstale":"2024-03-26T08:40:59.419449+0000","last_undegraded":"2024-03-26T08:40:59.419449+0000","last_fullsized":"2024-03-26T08:40:59.419449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:35.009264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502074+0000","last_change":"2024-03-26T08:40:09.925007+0000","last_active":"2024-03-26T08:40:59.502074+0000","last_peered":"2024-03-26T08:40:59.502074+0000","last_clean":"2024-03-26T08:40:59.502074+0000","last_became_active":"2024-03-26T08:40:09.923439+0000","last_became_peered":"2024-03-26T08:40:09.923439+0000","last_unstale":"2024-03-26T08:40:59.502074+0000","last_undegraded":"2024-03-26T08:40:59.502074+0000","last_fullsized":"2024-03-26T08:40:59.502074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:18.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463053+0000","last_change":"2024-03-26T08:40:09.920795+0000","last_active":"2024-03-26T08:40:59.463053+0000","last_peered":"2024-03-26T08:40:59.463053+0000","last_clean":"2024-03-26T08:40:59.463053+0000","last_became_active":"2024-03-26T08:40:09.920378+0000","last_became_peered":"2024-03-26T08:40:09.920378+0000","last_unstale":"2024-03-26T08:40:59.463053+0000","last_undegraded":"2024-03-26T08:40:59.463053+0000","last_fullsized":"2024-03-26T08:40:59.463053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:19:04.027727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.425704+0000","last_change":"2024-03-26T08:40:59.425704+0000","last_active":"2024-03-26T08:40:59.425704+0000","last_peered":"2024-03-26T08:40:59.425704+0000","last_clean":"2024-03-26T08:40:59.425704+0000","last_became_active":"2024-03-26T08:40:59.425014+0000","last_became_peered":"2024-03-26T08:40:59.425014+0000","last_unstale":"2024-03-26T08:40:59.425704+0000","last_undegraded":"2024-03-26T08:40:59.425704+0000","last_fullsized":"2024-03-26T08:40:59.425704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:08:14.847536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282339,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68403,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":443,"num_read_kb":293,"num_write":156,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":456298,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":1112159,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":456512,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456527,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1048362,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":455901,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":520096,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":457172,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":521837,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":456212,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":456338,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1047929,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:41:28.640 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T08:41:28.885 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:29.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:29 smithi007 ceph-mon[32462]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:41:29.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:29 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:41:29.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:29 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:41:29.260 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:29 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:29.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:29 smithi016 ceph-mon[35851]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:41:29.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:29 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:41:29.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:29 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:41:29.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:29 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:29.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:29 smithi044 ceph-mon[36461]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:41:29.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:29 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:41:29.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:29 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:41:29.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:29 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:41:30.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:30 smithi007 ceph-mon[32462]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:41:30.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:30 smithi016 ceph-mon[35851]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:41:30.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:30 smithi044 ceph-mon[36461]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:41:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 -- 172.21.15.7:0/4227537493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70101a30 msgr2=0x7f7d7010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 --2- 172.21.15.7:0/4227537493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70101a30 0x7f7d7010ded0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800b0a0 tx=0x7f7d6800b3b0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 -- 172.21.15.7:0/4227537493 shutdown_connections 2024-03-26T08:41:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 --2- 172.21.15.7:0/4227537493 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d70107120 0x7f7d70107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 --2- 172.21.15.7:0/4227537493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70101a30 0x7f7d7010ded0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 --2- 172.21.15.7:0/4227537493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70101070 0x7f7d701014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 -- 172.21.15.7:0/4227537493 >> 172.21.15.7:0/4227537493 conn(0x7f7d700780a0 msgr2=0x7f7d700784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 -- 172.21.15.7:0/4227537493 shutdown_connections 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 -- 172.21.15.7:0/4227537493 wait complete. 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 Processor -- start 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.615+0000 7f7d765c2700 1 -- start start 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d765c2700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d70101070 0x7f7d70199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d765c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70101a30 0x7f7d70199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d765c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70107120 0x7f7d701a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d765c2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d70111350 con 0x7f7d70107120 2024-03-26T08:41:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d765c2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701111d0 con 0x7f7d70101a30 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d765c2700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701114d0 con 0x7f7d70101070 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d74b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70107120 0x7f7d701a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d74b5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70107120 0x7f7d701a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35788/0 (socket says 172.21.15.7:35788) 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d74b5f700 1 -- 172.21.15.7:0/955641925 learned_addr learned my addr 172.21.15.7:0/955641925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d6f7fe700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70101a30 0x7f7d70199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d6ffff700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d70101070 0x7f7d70199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d74b5f700 1 -- 172.21.15.7:0/955641925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d70101070 msgr2=0x7f7d70199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d74b5f700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d70101070 0x7f7d70199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d74b5f700 1 -- 172.21.15.7:0/955641925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70101a30 msgr2=0x7f7d70199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d74b5f700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70101a30 0x7f7d70199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d74b5f700 1 -- 172.21.15.7:0/955641925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d701a4c50 con 0x7f7d70107120 2024-03-26T08:41:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d6f7fe700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70101a30 0x7f7d70199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d74b5f700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70107120 0x7f7d701a4710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800b780 tx=0x7f7d68004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d6d7fa700 1 -- 172.21.15.7:0/955641925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68021070 con 0x7f7d70107120 2024-03-26T08:41:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d6d7fa700 1 -- 172.21.15.7:0/955641925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d68007b80 con 0x7f7d70107120 2024-03-26T08:41:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.616+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d701a4e20 con 0x7f7d70107120 2024-03-26T08:41:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.617+0000 7f7d6d7fa700 1 -- 172.21.15.7:0/955641925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d680052a0 con 0x7f7d70107120 2024-03-26T08:41:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.617+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d701a5330 con 0x7f7d70107120 2024-03-26T08:41:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.618+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d70109470 con 0x7f7d70107120 2024-03-26T08:41:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.618+0000 7f7d6d7fa700 1 -- 172.21.15.7:0/955641925 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7d68024070 con 0x7f7d70107120 2024-03-26T08:41:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.619+0000 7f7d6d7fa700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d5c0a2890 0x7f7d5c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.619+0000 7f7d6ffff700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d5c0a2890 0x7f7d5c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.619+0000 7f7d6d7fa700 1 -- 172.21.15.7:0/955641925 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7d6800f020 con 0x7f7d70107120 2024-03-26T08:41:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.619+0000 7f7d6ffff700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d5c0a2890 0x7f7d5c0a4d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009be0 tx=0x7f7d60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.623+0000 7f7d6d7fa700 1 -- 172.21.15.7:0/955641925 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d6802b020 con 0x7f7d70107120 2024-03-26T08:41:30.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.823+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d700625d0 con 0x7f7d5c0a2890 2024-03-26T08:41:30.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.830+0000 7f7d6d7fa700 1 -- 172.21.15.7:0/955641925 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259050 (secure 0 0 0) 0x7f7d700625d0 con 0x7f7d5c0a2890 2024-03-26T08:41:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:30.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.833+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d5c0a2890 msgr2=0x7f7d5c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:30.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.833+0000 7f7d765c2700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d5c0a2890 0x7f7d5c0a4d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009be0 tx=0x7f7d60009260 comp rx=0 tx=0).stop 2024-03-26T08:41:30.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.833+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70107120 msgr2=0x7f7d701a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.833+0000 7f7d765c2700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70107120 0x7f7d701a4710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800b780 tx=0x7f7d68004d00 comp rx=0 tx=0).stop 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.834+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 shutdown_connections 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.834+0000 7f7d765c2700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d5c0a2890 0x7f7d5c0a4d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.834+0000 7f7d765c2700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d70101070 0x7f7d70199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.834+0000 7f7d765c2700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d70107120 0x7f7d701a4710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.834+0000 7f7d765c2700 1 --2- 172.21.15.7:0/955641925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d70101a30 0x7f7d70199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.834+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 >> 172.21.15.7:0/955641925 conn(0x7f7d700780a0 msgr2=0x7f7d700ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.834+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 shutdown_connections 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:30.834+0000 7f7d765c2700 1 -- 172.21.15.7:0/955641925 wait complete. 2024-03-26T08:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-26T08:41:31.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:31 smithi007 ceph-mon[32462]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T08:41:31.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:31 smithi016 ceph-mon[35851]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T08:41:31.509 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-03-26T08:41:30.319779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659005,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":579,"num_read_kb":407,"num_write":341,"num_write_kb":3050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373272,"kb_used_data":11580,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568552,"statfs":{"total":1151940427776,"available":1151558197248,"internally_reserved":0,"allocated":11857920,"data_stored":7445343,"data_compressed":177645,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003380"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353784+0000","last_change":"2024-03-26T08:40:09.939078+0000","last_active":"2024-03-26T08:41:28.353784+0000","last_peered":"2024-03-26T08:41:28.353784+0000","last_clean":"2024-03-26T08:41:28.353784+0000","last_became_active":"2024-03-26T08:40:09.937626+0000","last_became_peered":"2024-03-26T08:40:09.937626+0000","last_unstale":"2024-03-26T08:41:28.353784+0000","last_undegraded":"2024-03-26T08:41:28.353784+0000","last_fullsized":"2024-03-26T08:41:28.353784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:52:59.254181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345392+0000","last_change":"2024-03-26T08:40:09.924207+0000","last_active":"2024-03-26T08:41:28.345392+0000","last_peered":"2024-03-26T08:41:28.345392+0000","last_clean":"2024-03-26T08:41:28.345392+0000","last_became_active":"2024-03-26T08:40:09.923881+0000","last_became_peered":"2024-03-26T08:40:09.923881+0000","last_unstale":"2024-03-26T08:41:28.345392+0000","last_undegraded":"2024-03-26T08:41:28.345392+0000","last_fullsized":"2024-03-26T08:41:28.345392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:16:45.265521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349461+0000","last_change":"2024-03-26T08:40:09.929154+0000","last_active":"2024-03-26T08:41:28.349461+0000","last_peered":"2024-03-26T08:41:28.349461+0000","last_clean":"2024-03-26T08:41:28.349461+0000","last_became_active":"2024-03-26T08:40:09.928833+0000","last_became_peered":"2024-03-26T08:40:09.928833+0000","last_unstale":"2024-03-26T08:41:28.349461+0000","last_undegraded":"2024-03-26T08:41:28.349461+0000","last_fullsized":"2024-03-26T08:41:28.349461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:22.972210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420773+0000","last_change":"2024-03-26T08:40:09.930713+0000","last_active":"2024-03-26T08:40:59.420773+0000","last_peered":"2024-03-26T08:40:59.420773+0000","last_clean":"2024-03-26T08:40:59.420773+0000","last_became_active":"2024-03-26T08:40:09.930420+0000","last_became_peered":"2024-03-26T08:40:09.930420+0000","last_unstale":"2024-03-26T08:40:59.420773+0000","last_undegraded":"2024-03-26T08:40:59.420773+0000","last_fullsized":"2024-03-26T08:40:59.420773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:17:03.015703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.336520+0000","last_change":"2024-03-26T08:40:09.923935+0000","last_active":"2024-03-26T08:41:28.336520+0000","last_peered":"2024-03-26T08:41:28.336520+0000","last_clean":"2024-03-26T08:41:28.336520+0000","last_became_active":"2024-03-26T08:40:09.923587+0000","last_became_peered":"2024-03-26T08:40:09.923587+0000","last_unstale":"2024-03-26T08:41:28.336520+0000","last_undegraded":"2024-03-26T08:41:28.336520+0000","last_fullsized":"2024-03-26T08:41:28.336520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:19:38.831448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502235+0000","last_change":"2024-03-26T08:40:09.922361+0000","last_active":"2024-03-26T08:40:59.502235+0000","last_peered":"2024-03-26T08:40:59.502235+0000","last_clean":"2024-03-26T08:40:59.502235+0000","last_became_active":"2024-03-26T08:40:09.921643+0000","last_became_peered":"2024-03-26T08:40:09.921643+0000","last_unstale":"2024-03-26T08:40:59.502235+0000","last_undegraded":"2024-03-26T08:40:59.502235+0000","last_fullsized":"2024-03-26T08:40:59.502235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:37:23.048778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416687+0000","last_change":"2024-03-26T08:40:09.932428+0000","last_active":"2024-03-26T08:40:59.416687+0000","last_peered":"2024-03-26T08:40:59.416687+0000","last_clean":"2024-03-26T08:40:59.416687+0000","last_became_active":"2024-03-26T08:40:09.932120+0000","last_became_peered":"2024-03-26T08:40:09.932120+0000","last_unstale":"2024-03-26T08:40:59.416687+0000","last_undegraded":"2024-03-26T08:40:59.416687+0000","last_fullsized":"2024-03-26T08:40:59.416687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:24.362568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419004+0000","last_change":"2024-03-26T08:40:09.928251+0000","last_active":"2024-03-26T08:40:59.419004+0000","last_peered":"2024-03-26T08:40:59.419004+0000","last_clean":"2024-03-26T08:40:59.419004+0000","last_became_active":"2024-03-26T08:40:09.927961+0000","last_became_peered":"2024-03-26T08:40:09.927961+0000","last_unstale":"2024-03-26T08:40:59.419004+0000","last_undegraded":"2024-03-26T08:40:59.419004+0000","last_fullsized":"2024-03-26T08:40:59.419004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:25:09.348033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502820+0000","last_change":"2024-03-26T08:40:09.930888+0000","last_active":"2024-03-26T08:40:59.502820+0000","last_peered":"2024-03-26T08:40:59.502820+0000","last_clean":"2024-03-26T08:40:59.502820+0000","last_became_active":"2024-03-26T08:40:09.930513+0000","last_became_peered":"2024-03-26T08:40:09.930513+0000","last_unstale":"2024-03-26T08:40:59.502820+0000","last_undegraded":"2024-03-26T08:40:59.502820+0000","last_fullsized":"2024-03-26T08:40:59.502820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:21.023569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502684+0000","last_change":"2024-03-26T08:40:09.935534+0000","last_active":"2024-03-26T08:40:59.502684+0000","last_peered":"2024-03-26T08:40:59.502684+0000","last_clean":"2024-03-26T08:40:59.502684+0000","last_became_active":"2024-03-26T08:40:09.934959+0000","last_became_peered":"2024-03-26T08:40:09.934959+0000","last_unstale":"2024-03-26T08:40:59.502684+0000","last_undegraded":"2024-03-26T08:40:59.502684+0000","last_fullsized":"2024-03-26T08:40:59.502684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:18:59.311180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417244+0000","last_change":"2024-03-26T08:40:09.930486+0000","last_active":"2024-03-26T08:40:59.417244+0000","last_peered":"2024-03-26T08:40:59.417244+0000","last_clean":"2024-03-26T08:40:59.417244+0000","last_became_active":"2024-03-26T08:40:09.930184+0000","last_became_peered":"2024-03-26T08:40:09.930184+0000","last_unstale":"2024-03-26T08:40:59.417244+0000","last_undegraded":"2024-03-26T08:40:59.417244+0000","last_fullsized":"2024-03-26T08:40:59.417244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:07.246878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420629+0000","last_change":"2024-03-26T08:40:09.924305+0000","last_active":"2024-03-26T08:40:59.420629+0000","last_peered":"2024-03-26T08:40:59.420629+0000","last_clean":"2024-03-26T08:40:59.420629+0000","last_became_active":"2024-03-26T08:40:09.923985+0000","last_became_peered":"2024-03-26T08:40:09.923985+0000","last_unstale":"2024-03-26T08:40:59.420629+0000","last_undegraded":"2024-03-26T08:40:59.420629+0000","last_fullsized":"2024-03-26T08:40:59.420629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:42:39.894245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.462998+0000","last_change":"2024-03-26T08:40:09.916637+0000","last_active":"2024-03-26T08:40:59.462998+0000","last_peered":"2024-03-26T08:40:59.462998+0000","last_clean":"2024-03-26T08:40:59.462998+0000","last_became_active":"2024-03-26T08:40:09.916314+0000","last_became_peered":"2024-03-26T08:40:09.916314+0000","last_unstale":"2024-03-26T08:40:59.462998+0000","last_undegraded":"2024-03-26T08:40:59.462998+0000","last_fullsized":"2024-03-26T08:40:59.462998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:45:10.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345246+0000","last_change":"2024-03-26T08:40:09.923753+0000","last_active":"2024-03-26T08:41:28.345246+0000","last_peered":"2024-03-26T08:41:28.345246+0000","last_clean":"2024-03-26T08:41:28.345246+0000","last_became_active":"2024-03-26T08:40:09.923427+0000","last_became_peered":"2024-03-26T08:40:09.923427+0000","last_unstale":"2024-03-26T08:41:28.345246+0000","last_undegraded":"2024-03-26T08:41:28.345246+0000","last_fullsized":"2024-03-26T08:41:28.345246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:13:16.711556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338474+0000","last_change":"2024-03-26T08:40:09.931879+0000","last_active":"2024-03-26T08:41:28.338474+0000","last_peered":"2024-03-26T08:41:28.338474+0000","last_clean":"2024-03-26T08:41:28.338474+0000","last_became_active":"2024-03-26T08:40:09.931589+0000","last_became_peered":"2024-03-26T08:40:09.931589+0000","last_unstale":"2024-03-26T08:41:28.338474+0000","last_undegraded":"2024-03-26T08:41:28.338474+0000","last_fullsized":"2024-03-26T08:41:28.338474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:22:01.038745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.501778+0000","last_change":"2024-03-26T08:40:09.930500+0000","last_active":"2024-03-26T08:40:59.501778+0000","last_peered":"2024-03-26T08:40:59.501778+0000","last_clean":"2024-03-26T08:40:59.501778+0000","last_became_active":"2024-03-26T08:40:09.930101+0000","last_became_peered":"2024-03-26T08:40:09.930101+0000","last_unstale":"2024-03-26T08:40:59.501778+0000","last_undegraded":"2024-03-26T08:40:59.501778+0000","last_fullsized":"2024-03-26T08:40:59.501778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:54.229004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.354043+0000","last_change":"2024-03-26T08:40:09.939177+0000","last_active":"2024-03-26T08:41:28.354043+0000","last_peered":"2024-03-26T08:41:28.354043+0000","last_clean":"2024-03-26T08:41:28.354043+0000","last_became_active":"2024-03-26T08:40:09.937727+0000","last_became_peered":"2024-03-26T08:40:09.937727+0000","last_unstale":"2024-03-26T08:41:28.354043+0000","last_undegraded":"2024-03-26T08:41:28.354043+0000","last_fullsized":"2024-03-26T08:41:28.354043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:40:44.040015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350313+0000","last_change":"2024-03-26T08:40:09.928429+0000","last_active":"2024-03-26T08:41:28.350313+0000","last_peered":"2024-03-26T08:41:28.350313+0000","last_clean":"2024-03-26T08:41:28.350313+0000","last_became_active":"2024-03-26T08:40:09.928025+0000","last_became_peered":"2024-03-26T08:40:09.928025+0000","last_unstale":"2024-03-26T08:41:28.350313+0000","last_undegraded":"2024-03-26T08:41:28.350313+0000","last_fullsized":"2024-03-26T08:41:28.350313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:52:55.450291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422006+0000","last_change":"2024-03-26T08:40:09.930856+0000","last_active":"2024-03-26T08:40:59.422006+0000","last_peered":"2024-03-26T08:40:59.422006+0000","last_clean":"2024-03-26T08:40:59.422006+0000","last_became_active":"2024-03-26T08:40:09.930552+0000","last_became_peered":"2024-03-26T08:40:09.930552+0000","last_unstale":"2024-03-26T08:40:59.422006+0000","last_undegraded":"2024-03-26T08:40:59.422006+0000","last_fullsized":"2024-03-26T08:40:59.422006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:52.851287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421132+0000","last_change":"2024-03-26T08:40:09.230609+0000","last_active":"2024-03-26T08:40:59.421132+0000","last_peered":"2024-03-26T08:40:59.421132+0000","last_clean":"2024-03-26T08:40:59.421132+0000","last_became_active":"2024-03-26T08:40:09.230284+0000","last_became_peered":"2024-03-26T08:40:09.230284+0000","last_unstale":"2024-03-26T08:40:59.421132+0000","last_undegraded":"2024-03-26T08:40:59.421132+0000","last_fullsized":"2024-03-26T08:40:59.421132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:06.681725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.337071+0000","last_change":"2024-03-26T08:40:59.425449+0000","last_active":"2024-03-26T08:41:28.337071+0000","last_peered":"2024-03-26T08:41:28.337071+0000","last_clean":"2024-03-26T08:41:28.337071+0000","last_became_active":"2024-03-26T08:40:59.425164+0000","last_became_peered":"2024-03-26T08:40:59.425164+0000","last_unstale":"2024-03-26T08:41:28.337071+0000","last_undegraded":"2024-03-26T08:41:28.337071+0000","last_fullsized":"2024-03-26T08:41:28.337071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:09.846252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419617+0000","last_change":"2024-03-26T08:40:09.235451+0000","last_active":"2024-03-26T08:40:59.419617+0000","last_peered":"2024-03-26T08:40:59.419617+0000","last_clean":"2024-03-26T08:40:59.419617+0000","last_became_active":"2024-03-26T08:40:09.235154+0000","last_became_peered":"2024-03-26T08:40:09.235154+0000","last_unstale":"2024-03-26T08:40:59.419617+0000","last_undegraded":"2024-03-26T08:40:59.419617+0000","last_fullsized":"2024-03-26T08:40:59.419617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:37:38.418888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349428+0000","last_change":"2024-03-26T08:40:09.931728+0000","last_active":"2024-03-26T08:41:28.349428+0000","last_peered":"2024-03-26T08:41:28.349428+0000","last_clean":"2024-03-26T08:41:28.349428+0000","last_became_active":"2024-03-26T08:40:09.931401+0000","last_became_peered":"2024-03-26T08:40:09.931401+0000","last_unstale":"2024-03-26T08:41:28.349428+0000","last_undegraded":"2024-03-26T08:41:28.349428+0000","last_fullsized":"2024-03-26T08:41:28.349428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:02.229763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417066+0000","last_change":"2024-03-26T08:40:09.231247+0000","last_active":"2024-03-26T08:40:59.417066+0000","last_peered":"2024-03-26T08:40:59.417066+0000","last_clean":"2024-03-26T08:40:59.417066+0000","last_became_active":"2024-03-26T08:40:09.230914+0000","last_became_peered":"2024-03-26T08:40:09.230914+0000","last_unstale":"2024-03-26T08:40:59.417066+0000","last_undegraded":"2024-03-26T08:40:59.417066+0000","last_fullsized":"2024-03-26T08:40:59.417066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:03.956444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.501978+0000","last_change":"2024-03-26T08:40:09.925174+0000","last_active":"2024-03-26T08:40:59.501978+0000","last_peered":"2024-03-26T08:40:59.501978+0000","last_clean":"2024-03-26T08:40:59.501978+0000","last_became_active":"2024-03-26T08:40:09.924840+0000","last_became_peered":"2024-03-26T08:40:09.924840+0000","last_unstale":"2024-03-26T08:40:59.501978+0000","last_undegraded":"2024-03-26T08:40:59.501978+0000","last_fullsized":"2024-03-26T08:40:59.501978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:57:19.041245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416818+0000","last_change":"2024-03-26T08:40:09.229504+0000","last_active":"2024-03-26T08:40:59.416818+0000","last_peered":"2024-03-26T08:40:59.416818+0000","last_clean":"2024-03-26T08:40:59.416818+0000","last_became_active":"2024-03-26T08:40:09.226449+0000","last_became_peered":"2024-03-26T08:40:09.226449+0000","last_unstale":"2024-03-26T08:40:59.416818+0000","last_undegraded":"2024-03-26T08:40:59.416818+0000","last_fullsized":"2024-03-26T08:40:59.416818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:38.047424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419075+0000","last_change":"2024-03-26T08:40:09.930184+0000","last_active":"2024-03-26T08:40:59.419075+0000","last_peered":"2024-03-26T08:40:59.419075+0000","last_clean":"2024-03-26T08:40:59.419075+0000","last_became_active":"2024-03-26T08:40:09.927679+0000","last_became_peered":"2024-03-26T08:40:09.927679+0000","last_unstale":"2024-03-26T08:40:59.419075+0000","last_undegraded":"2024-03-26T08:40:59.419075+0000","last_fullsized":"2024-03-26T08:40:59.419075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:05:52.949814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":99,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:24.357997+0000","last_change":"2024-03-26T08:40:09.233025+0000","last_active":"2024-03-26T08:41:24.357997+0000","last_peered":"2024-03-26T08:41:24.357997+0000","last_clean":"2024-03-26T08:41:24.357997+0000","last_became_active":"2024-03-26T08:40:09.232654+0000","last_became_peered":"2024-03-26T08:40:09.232654+0000","last_unstale":"2024-03-26T08:41:24.357997+0000","last_undegraded":"2024-03-26T08:41:24.357997+0000","last_fullsized":"2024-03-26T08:41:24.357997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:01.745583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.333641+0000","last_change":"2024-03-26T08:40:09.924187+0000","last_active":"2024-03-26T08:41:28.333641+0000","last_peered":"2024-03-26T08:41:28.333641+0000","last_clean":"2024-03-26T08:41:28.333641+0000","last_became_active":"2024-03-26T08:40:09.923861+0000","last_became_peered":"2024-03-26T08:40:09.923861+0000","last_unstale":"2024-03-26T08:41:28.333641+0000","last_undegraded":"2024-03-26T08:41:28.333641+0000","last_fullsized":"2024-03-26T08:41:28.333641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:12:17.284186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502123+0000","last_change":"2024-03-26T08:40:09.231368+0000","last_active":"2024-03-26T08:40:59.502123+0000","last_peered":"2024-03-26T08:40:59.502123+0000","last_clean":"2024-03-26T08:40:59.502123+0000","last_became_active":"2024-03-26T08:40:09.231040+0000","last_became_peered":"2024-03-26T08:40:09.231040+0000","last_unstale":"2024-03-26T08:40:59.502123+0000","last_undegraded":"2024-03-26T08:40:59.502123+0000","last_fullsized":"2024-03-26T08:40:59.502123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:14:20.681225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419928+0000","last_change":"2024-03-26T08:40:09.928125+0000","last_active":"2024-03-26T08:40:59.419928+0000","last_peered":"2024-03-26T08:40:59.419928+0000","last_clean":"2024-03-26T08:40:59.419928+0000","last_became_active":"2024-03-26T08:40:09.927616+0000","last_became_peered":"2024-03-26T08:40:09.927616+0000","last_unstale":"2024-03-26T08:40:59.419928+0000","last_undegraded":"2024-03-26T08:40:59.419928+0000","last_fullsized":"2024-03-26T08:40:59.419928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:12.764054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.462999+0000","last_change":"2024-03-26T08:40:09.229976+0000","last_active":"2024-03-26T08:40:59.462999+0000","last_peered":"2024-03-26T08:40:59.462999+0000","last_clean":"2024-03-26T08:40:59.462999+0000","last_became_active":"2024-03-26T08:40:09.229594+0000","last_became_peered":"2024-03-26T08:40:09.229594+0000","last_unstale":"2024-03-26T08:40:59.462999+0000","last_undegraded":"2024-03-26T08:40:59.462999+0000","last_fullsized":"2024-03-26T08:40:59.462999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:31:24.204853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.418918+0000","last_change":"2024-03-26T08:40:09.931290+0000","last_active":"2024-03-26T08:40:59.418918+0000","last_peered":"2024-03-26T08:40:59.418918+0000","last_clean":"2024-03-26T08:40:59.418918+0000","last_became_active":"2024-03-26T08:40:09.930961+0000","last_became_peered":"2024-03-26T08:40:09.930961+0000","last_unstale":"2024-03-26T08:40:59.418918+0000","last_undegraded":"2024-03-26T08:40:59.418918+0000","last_fullsized":"2024-03-26T08:40:59.418918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:36.460807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419243+0000","last_change":"2024-03-26T08:40:09.232166+0000","last_active":"2024-03-26T08:40:59.419243+0000","last_peered":"2024-03-26T08:40:59.419243+0000","last_clean":"2024-03-26T08:40:59.419243+0000","last_became_active":"2024-03-26T08:40:09.231840+0000","last_became_peered":"2024-03-26T08:40:09.231840+0000","last_unstale":"2024-03-26T08:40:59.419243+0000","last_undegraded":"2024-03-26T08:40:59.419243+0000","last_fullsized":"2024-03-26T08:40:59.419243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:51.636322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352211+0000","last_change":"2024-03-26T08:40:09.924680+0000","last_active":"2024-03-26T08:41:28.352211+0000","last_peered":"2024-03-26T08:41:28.352211+0000","last_clean":"2024-03-26T08:41:28.352211+0000","last_became_active":"2024-03-26T08:40:09.924314+0000","last_became_peered":"2024-03-26T08:40:09.924314+0000","last_unstale":"2024-03-26T08:41:28.352211+0000","last_undegraded":"2024-03-26T08:41:28.352211+0000","last_fullsized":"2024-03-26T08:41:28.352211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:49:20.780950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:24.357493+0000","last_change":"2024-03-26T08:40:09.235997+0000","last_active":"2024-03-26T08:41:24.357493+0000","last_peered":"2024-03-26T08:41:24.357493+0000","last_clean":"2024-03-26T08:41:24.357493+0000","last_became_active":"2024-03-26T08:40:09.235729+0000","last_became_peered":"2024-03-26T08:40:09.235729+0000","last_unstale":"2024-03-26T08:41:24.357493+0000","last_undegraded":"2024-03-26T08:41:24.357493+0000","last_fullsized":"2024-03-26T08:41:24.357493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:46:12.322059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502552+0000","last_change":"2024-03-26T08:40:09.926371+0000","last_active":"2024-03-26T08:40:59.502552+0000","last_peered":"2024-03-26T08:40:59.502552+0000","last_clean":"2024-03-26T08:40:59.502552+0000","last_became_active":"2024-03-26T08:40:09.925536+0000","last_became_peered":"2024-03-26T08:40:09.925536+0000","last_unstale":"2024-03-26T08:40:59.502552+0000","last_undegraded":"2024-03-26T08:40:59.502552+0000","last_fullsized":"2024-03-26T08:40:59.502552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:50:39.101684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417249+0000","last_change":"2024-03-26T08:40:09.229018+0000","last_active":"2024-03-26T08:40:59.417249+0000","last_peered":"2024-03-26T08:40:59.417249+0000","last_clean":"2024-03-26T08:40:59.417249+0000","last_became_active":"2024-03-26T08:40:09.228680+0000","last_became_peered":"2024-03-26T08:40:09.228680+0000","last_unstale":"2024-03-26T08:40:59.417249+0000","last_undegraded":"2024-03-26T08:40:59.417249+0000","last_fullsized":"2024-03-26T08:40:59.417249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:31:10.065127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338799+0000","last_change":"2024-03-26T08:40:09.926152+0000","last_active":"2024-03-26T08:41:28.338799+0000","last_peered":"2024-03-26T08:41:28.338799+0000","last_clean":"2024-03-26T08:41:28.338799+0000","last_became_active":"2024-03-26T08:40:09.925825+0000","last_became_peered":"2024-03-26T08:40:09.925825+0000","last_unstale":"2024-03-26T08:41:28.338799+0000","last_undegraded":"2024-03-26T08:41:28.338799+0000","last_fullsized":"2024-03-26T08:41:28.338799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:28:24.287706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419184+0000","last_change":"2024-03-26T08:40:09.234699+0000","last_active":"2024-03-26T08:40:59.419184+0000","last_peered":"2024-03-26T08:40:59.419184+0000","last_clean":"2024-03-26T08:40:59.419184+0000","last_became_active":"2024-03-26T08:40:09.234451+0000","last_became_peered":"2024-03-26T08:40:09.234451+0000","last_unstale":"2024-03-26T08:40:59.419184+0000","last_undegraded":"2024-03-26T08:40:59.419184+0000","last_fullsized":"2024-03-26T08:40:59.419184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:53.337418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502112+0000","last_change":"2024-03-26T08:40:09.927023+0000","last_active":"2024-03-26T08:40:59.502112+0000","last_peered":"2024-03-26T08:40:59.502112+0000","last_clean":"2024-03-26T08:40:59.502112+0000","last_became_active":"2024-03-26T08:40:09.926734+0000","last_became_peered":"2024-03-26T08:40:09.926734+0000","last_unstale":"2024-03-26T08:40:59.502112+0000","last_undegraded":"2024-03-26T08:40:59.502112+0000","last_fullsized":"2024-03-26T08:40:59.502112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:19:44.137513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'30","reported_seq":198,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:25.358474+0000","last_change":"2024-03-26T08:40:09.230059+0000","last_active":"2024-03-26T08:41:25.358474+0000","last_peered":"2024-03-26T08:41:25.358474+0000","last_clean":"2024-03-26T08:41:25.358474+0000","last_became_active":"2024-03-26T08:40:09.229776+0000","last_became_peered":"2024-03-26T08:40:09.229776+0000","last_unstale":"2024-03-26T08:41:25.358474+0000","last_undegraded":"2024-03-26T08:41:25.358474+0000","last_fullsized":"2024-03-26T08:41:25.358474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:00.917694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64402,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":149,"num_read_kb":137,"num_write":31,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.334293+0000","last_change":"2024-03-26T08:40:09.930702+0000","last_active":"2024-03-26T08:41:28.334293+0000","last_peered":"2024-03-26T08:41:28.334293+0000","last_clean":"2024-03-26T08:41:28.334293+0000","last_became_active":"2024-03-26T08:40:09.930400+0000","last_became_peered":"2024-03-26T08:40:09.930400+0000","last_unstale":"2024-03-26T08:41:28.334293+0000","last_undegraded":"2024-03-26T08:41:28.334293+0000","last_fullsized":"2024-03-26T08:41:28.334293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:03:12.897812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422276+0000","last_change":"2024-03-26T08:40:09.225972+0000","last_active":"2024-03-26T08:40:59.422276+0000","last_peered":"2024-03-26T08:40:59.422276+0000","last_clean":"2024-03-26T08:40:59.422276+0000","last_became_active":"2024-03-26T08:40:09.225524+0000","last_became_peered":"2024-03-26T08:40:09.225524+0000","last_unstale":"2024-03-26T08:40:59.422276+0000","last_undegraded":"2024-03-26T08:40:59.422276+0000","last_fullsized":"2024-03-26T08:40:59.422276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:08:17.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463062+0000","last_change":"2024-03-26T08:40:09.920628+0000","last_active":"2024-03-26T08:40:59.463062+0000","last_peered":"2024-03-26T08:40:59.463062+0000","last_clean":"2024-03-26T08:40:59.463062+0000","last_became_active":"2024-03-26T08:40:09.920300+0000","last_became_peered":"2024-03-26T08:40:09.920300+0000","last_unstale":"2024-03-26T08:40:59.463062+0000","last_undegraded":"2024-03-26T08:40:59.463062+0000","last_fullsized":"2024-03-26T08:40:59.463062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:02:16.503881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419684+0000","last_change":"2024-03-26T08:40:09.237802+0000","last_active":"2024-03-26T08:40:59.419684+0000","last_peered":"2024-03-26T08:40:59.419684+0000","last_clean":"2024-03-26T08:40:59.419684+0000","last_became_active":"2024-03-26T08:40:09.237425+0000","last_became_peered":"2024-03-26T08:40:09.237425+0000","last_unstale":"2024-03-26T08:40:59.419684+0000","last_undegraded":"2024-03-26T08:40:59.419684+0000","last_fullsized":"2024-03-26T08:40:59.419684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:05.953882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417003+0000","last_change":"2024-03-26T08:40:09.923221+0000","last_active":"2024-03-26T08:40:59.417003+0000","last_peered":"2024-03-26T08:40:59.417003+0000","last_clean":"2024-03-26T08:40:59.417003+0000","last_became_active":"2024-03-26T08:40:09.922894+0000","last_became_peered":"2024-03-26T08:40:09.922894+0000","last_unstale":"2024-03-26T08:40:59.417003+0000","last_undegraded":"2024-03-26T08:40:59.417003+0000","last_fullsized":"2024-03-26T08:40:59.417003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:24.887625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:25.360920+0000","last_change":"2024-03-26T08:40:09.225179+0000","last_active":"2024-03-26T08:41:25.360920+0000","last_peered":"2024-03-26T08:41:25.360920+0000","last_clean":"2024-03-26T08:41:25.360920+0000","last_became_active":"2024-03-26T08:40:09.224861+0000","last_became_peered":"2024-03-26T08:40:09.224861+0000","last_unstale":"2024-03-26T08:41:25.360920+0000","last_undegraded":"2024-03-26T08:41:25.360920+0000","last_fullsized":"2024-03-26T08:41:25.360920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:37:54.843284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.503355+0000","last_change":"2024-03-26T08:40:59.503355+0000","last_active":"2024-03-26T08:40:59.503355+0000","last_peered":"2024-03-26T08:40:59.503355+0000","last_clean":"2024-03-26T08:40:59.503355+0000","last_became_active":"2024-03-26T08:40:59.503009+0000","last_became_peered":"2024-03-26T08:40:59.503009+0000","last_unstale":"2024-03-26T08:40:59.503355+0000","last_undegraded":"2024-03-26T08:40:59.503355+0000","last_fullsized":"2024-03-26T08:40:59.503355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:08:20.694763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416469+0000","last_change":"2024-03-26T08:40:09.232024+0000","last_active":"2024-03-26T08:40:59.416469+0000","last_peered":"2024-03-26T08:40:59.416469+0000","last_clean":"2024-03-26T08:40:59.416469+0000","last_became_active":"2024-03-26T08:40:09.231715+0000","last_became_peered":"2024-03-26T08:40:09.231715+0000","last_unstale":"2024-03-26T08:40:59.416469+0000","last_undegraded":"2024-03-26T08:40:59.416469+0000","last_fullsized":"2024-03-26T08:40:59.416469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:02.229042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419447+0000","last_change":"2024-03-26T08:40:09.926893+0000","last_active":"2024-03-26T08:40:59.419447+0000","last_peered":"2024-03-26T08:40:59.419447+0000","last_clean":"2024-03-26T08:40:59.419447+0000","last_became_active":"2024-03-26T08:40:09.926529+0000","last_became_peered":"2024-03-26T08:40:09.926529+0000","last_unstale":"2024-03-26T08:40:59.419447+0000","last_undegraded":"2024-03-26T08:40:59.419447+0000","last_fullsized":"2024-03-26T08:40:59.419447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:10.189407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.428905+0000","last_change":"2024-03-26T08:40:59.428905+0000","last_active":"2024-03-26T08:40:59.428905+0000","last_peered":"2024-03-26T08:40:59.428905+0000","last_clean":"2024-03-26T08:40:59.428905+0000","last_became_active":"2024-03-26T08:40:59.423345+0000","last_became_peered":"2024-03-26T08:40:59.423345+0000","last_unstale":"2024-03-26T08:40:59.428905+0000","last_undegraded":"2024-03-26T08:40:59.428905+0000","last_fullsized":"2024-03-26T08:40:59.428905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:49.069421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.346046+0000","last_change":"2024-03-26T08:40:59.464333+0000","last_active":"2024-03-26T08:41:28.346046+0000","last_peered":"2024-03-26T08:41:28.346046+0000","last_clean":"2024-03-26T08:41:28.346046+0000","last_became_active":"2024-03-26T08:40:59.464016+0000","last_became_peered":"2024-03-26T08:40:59.464016+0000","last_unstale":"2024-03-26T08:41:28.346046+0000","last_undegraded":"2024-03-26T08:41:28.346046+0000","last_fullsized":"2024-03-26T08:41:28.346046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:50:05.770568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420974+0000","last_change":"2024-03-26T08:40:09.224385+0000","last_active":"2024-03-26T08:40:59.420974+0000","last_peered":"2024-03-26T08:40:59.420974+0000","last_clean":"2024-03-26T08:40:59.420974+0000","last_became_active":"2024-03-26T08:40:09.223991+0000","last_became_peered":"2024-03-26T08:40:09.223991+0000","last_unstale":"2024-03-26T08:40:59.420974+0000","last_undegraded":"2024-03-26T08:40:59.420974+0000","last_fullsized":"2024-03-26T08:40:59.420974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:55.808922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502351+0000","last_change":"2024-03-26T08:40:09.921072+0000","last_active":"2024-03-26T08:40:59.502351+0000","last_peered":"2024-03-26T08:40:59.502351+0000","last_clean":"2024-03-26T08:40:59.502351+0000","last_became_active":"2024-03-26T08:40:09.920747+0000","last_became_peered":"2024-03-26T08:40:09.920747+0000","last_unstale":"2024-03-26T08:40:59.502351+0000","last_undegraded":"2024-03-26T08:40:59.502351+0000","last_fullsized":"2024-03-26T08:40:59.502351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:20.862765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416822+0000","last_change":"2024-03-26T08:40:09.231344+0000","last_active":"2024-03-26T08:40:59.416822+0000","last_peered":"2024-03-26T08:40:59.416822+0000","last_clean":"2024-03-26T08:40:59.416822+0000","last_became_active":"2024-03-26T08:40:09.229313+0000","last_became_peered":"2024-03-26T08:40:09.229313+0000","last_unstale":"2024-03-26T08:40:59.416822+0000","last_undegraded":"2024-03-26T08:40:59.416822+0000","last_fullsized":"2024-03-26T08:40:59.416822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:37.130884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502402+0000","last_change":"2024-03-26T08:40:09.926355+0000","last_active":"2024-03-26T08:40:59.502402+0000","last_peered":"2024-03-26T08:40:59.502402+0000","last_clean":"2024-03-26T08:40:59.502402+0000","last_became_active":"2024-03-26T08:40:09.922889+0000","last_became_peered":"2024-03-26T08:40:09.922889+0000","last_unstale":"2024-03-26T08:40:59.502402+0000","last_undegraded":"2024-03-26T08:40:59.502402+0000","last_fullsized":"2024-03-26T08:40:59.502402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:42:22.784811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419357+0000","last_change":"2024-03-26T08:40:09.919577+0000","last_active":"2024-03-26T08:40:59.419357+0000","last_peered":"2024-03-26T08:40:59.419357+0000","last_clean":"2024-03-26T08:40:59.419357+0000","last_became_active":"2024-03-26T08:40:09.919234+0000","last_became_peered":"2024-03-26T08:40:09.919234+0000","last_unstale":"2024-03-26T08:40:59.419357+0000","last_undegraded":"2024-03-26T08:40:59.419357+0000","last_fullsized":"2024-03-26T08:40:59.419357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:18.894052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:25.360524+0000","last_change":"2024-03-26T08:40:09.237190+0000","last_active":"2024-03-26T08:41:25.360524+0000","last_peered":"2024-03-26T08:41:25.360524+0000","last_clean":"2024-03-26T08:41:25.360524+0000","last_became_active":"2024-03-26T08:40:09.236862+0000","last_became_peered":"2024-03-26T08:40:09.236862+0000","last_unstale":"2024-03-26T08:41:25.360524+0000","last_undegraded":"2024-03-26T08:41:25.360524+0000","last_fullsized":"2024-03-26T08:41:25.360524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:29.900711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419315+0000","last_change":"2024-03-26T08:40:09.927444+0000","last_active":"2024-03-26T08:40:59.419315+0000","last_peered":"2024-03-26T08:40:59.419315+0000","last_clean":"2024-03-26T08:40:59.419315+0000","last_became_active":"2024-03-26T08:40:09.927152+0000","last_became_peered":"2024-03-26T08:40:09.927152+0000","last_unstale":"2024-03-26T08:40:59.419315+0000","last_undegraded":"2024-03-26T08:40:59.419315+0000","last_fullsized":"2024-03-26T08:40:59.419315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:44.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420563+0000","last_change":"2024-03-26T08:40:09.224831+0000","last_active":"2024-03-26T08:40:59.420563+0000","last_peered":"2024-03-26T08:40:59.420563+0000","last_clean":"2024-03-26T08:40:59.420563+0000","last_became_active":"2024-03-26T08:40:09.224447+0000","last_became_peered":"2024-03-26T08:40:09.224447+0000","last_unstale":"2024-03-26T08:40:59.420563+0000","last_undegraded":"2024-03-26T08:40:59.420563+0000","last_fullsized":"2024-03-26T08:40:59.420563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:25:50.935342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416700+0000","last_change":"2024-03-26T08:38:54.696342+0000","last_active":"2024-03-26T08:40:59.416700+0000","last_peered":"2024-03-26T08:40:59.416700+0000","last_clean":"2024-03-26T08:40:59.416700+0000","last_became_active":"2024-03-26T08:38:54.695839+0000","last_became_peered":"2024-03-26T08:38:54.695839+0000","last_unstale":"2024-03-26T08:40:59.416700+0000","last_undegraded":"2024-03-26T08:40:59.416700+0000","last_fullsized":"2024-03-26T08:40:59.416700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_clean_scrub_stamp":"2024-03-26T08:35:27.189917+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:13.219778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463034+0000","last_change":"2024-03-26T08:40:09.930683+0000","last_active":"2024-03-26T08:40:59.463034+0000","last_peered":"2024-03-26T08:40:59.463034+0000","last_clean":"2024-03-26T08:40:59.463034+0000","last_became_active":"2024-03-26T08:40:09.930341+0000","last_became_peered":"2024-03-26T08:40:09.930341+0000","last_unstale":"2024-03-26T08:40:59.463034+0000","last_undegraded":"2024-03-26T08:40:59.463034+0000","last_fullsized":"2024-03-26T08:40:59.463034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:39:20.232007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416894+0000","last_change":"2024-03-26T08:40:09.228548+0000","last_active":"2024-03-26T08:40:59.416894+0000","last_peered":"2024-03-26T08:40:59.416894+0000","last_clean":"2024-03-26T08:40:59.416894+0000","last_became_active":"2024-03-26T08:40:09.228076+0000","last_became_peered":"2024-03-26T08:40:09.228076+0000","last_unstale":"2024-03-26T08:40:59.416894+0000","last_undegraded":"2024-03-26T08:40:59.416894+0000","last_fullsized":"2024-03-26T08:40:59.416894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:52:51.641128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416941+0000","last_change":"2024-03-26T08:40:09.235632+0000","last_active":"2024-03-26T08:40:59.416941+0000","last_peered":"2024-03-26T08:40:59.416941+0000","last_clean":"2024-03-26T08:40:59.416941+0000","last_became_active":"2024-03-26T08:40:09.235334+0000","last_became_peered":"2024-03-26T08:40:09.235334+0000","last_unstale":"2024-03-26T08:40:59.416941+0000","last_undegraded":"2024-03-26T08:40:59.416941+0000","last_fullsized":"2024-03-26T08:40:59.416941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:38:27.064431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463180+0000","last_change":"2024-03-26T08:40:09.929137+0000","last_active":"2024-03-26T08:40:59.463180+0000","last_peered":"2024-03-26T08:40:59.463180+0000","last_clean":"2024-03-26T08:40:59.463180+0000","last_became_active":"2024-03-26T08:40:09.928831+0000","last_became_peered":"2024-03-26T08:40:09.928831+0000","last_unstale":"2024-03-26T08:40:59.463180+0000","last_undegraded":"2024-03-26T08:40:59.463180+0000","last_fullsized":"2024-03-26T08:40:59.463180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:19.235168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421563+0000","last_change":"2024-03-26T08:40:09.227153+0000","last_active":"2024-03-26T08:40:59.421563+0000","last_peered":"2024-03-26T08:40:59.421563+0000","last_clean":"2024-03-26T08:40:59.421563+0000","last_became_active":"2024-03-26T08:40:09.226772+0000","last_became_peered":"2024-03-26T08:40:09.226772+0000","last_unstale":"2024-03-26T08:40:59.421563+0000","last_undegraded":"2024-03-26T08:40:59.421563+0000","last_fullsized":"2024-03-26T08:40:59.421563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:33.159538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419240+0000","last_change":"2024-03-26T08:40:09.934589+0000","last_active":"2024-03-26T08:40:59.419240+0000","last_peered":"2024-03-26T08:40:59.419240+0000","last_clean":"2024-03-26T08:40:59.419240+0000","last_became_active":"2024-03-26T08:40:09.934312+0000","last_became_peered":"2024-03-26T08:40:09.934312+0000","last_unstale":"2024-03-26T08:40:59.419240+0000","last_undegraded":"2024-03-26T08:40:59.419240+0000","last_fullsized":"2024-03-26T08:40:59.419240+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:19:27.737506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420385+0000","last_change":"2024-03-26T08:40:09.230014+0000","last_active":"2024-03-26T08:40:59.420385+0000","last_peered":"2024-03-26T08:40:59.420385+0000","last_clean":"2024-03-26T08:40:59.420385+0000","last_became_active":"2024-03-26T08:40:09.229682+0000","last_became_peered":"2024-03-26T08:40:09.229682+0000","last_unstale":"2024-03-26T08:40:59.420385+0000","last_undegraded":"2024-03-26T08:40:59.420385+0000","last_fullsized":"2024-03-26T08:40:59.420385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:52.725810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.337290+0000","last_change":"2024-03-26T08:40:59.464419+0000","last_active":"2024-03-26T08:41:28.337290+0000","last_peered":"2024-03-26T08:41:28.337290+0000","last_clean":"2024-03-26T08:41:28.337290+0000","last_became_active":"2024-03-26T08:40:59.464051+0000","last_became_peered":"2024-03-26T08:40:59.464051+0000","last_unstale":"2024-03-26T08:41:28.337290+0000","last_undegraded":"2024-03-26T08:41:28.337290+0000","last_fullsized":"2024-03-26T08:41:28.337290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:01:47.975380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.501822+0000","last_change":"2024-03-26T08:40:09.226634+0000","last_active":"2024-03-26T08:40:59.501822+0000","last_peered":"2024-03-26T08:40:59.501822+0000","last_clean":"2024-03-26T08:40:59.501822+0000","last_became_active":"2024-03-26T08:40:09.226248+0000","last_became_peered":"2024-03-26T08:40:09.226248+0000","last_unstale":"2024-03-26T08:40:59.501822+0000","last_undegraded":"2024-03-26T08:40:59.501822+0000","last_fullsized":"2024-03-26T08:40:59.501822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:24:00.390573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.303390+0000","last_change":"2024-03-26T08:40:09.934848+0000","last_active":"2024-03-26T08:41:28.303390+0000","last_peered":"2024-03-26T08:41:28.303390+0000","last_clean":"2024-03-26T08:41:28.303390+0000","last_became_active":"2024-03-26T08:40:09.934521+0000","last_became_peered":"2024-03-26T08:40:09.934521+0000","last_unstale":"2024-03-26T08:41:28.303390+0000","last_undegraded":"2024-03-26T08:41:28.303390+0000","last_fullsized":"2024-03-26T08:41:28.303390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:24:25.994961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419371+0000","last_change":"2024-03-26T08:40:09.222945+0000","last_active":"2024-03-26T08:40:59.419371+0000","last_peered":"2024-03-26T08:40:59.419371+0000","last_clean":"2024-03-26T08:40:59.419371+0000","last_became_active":"2024-03-26T08:40:09.222564+0000","last_became_peered":"2024-03-26T08:40:09.222564+0000","last_unstale":"2024-03-26T08:40:59.419371+0000","last_undegraded":"2024-03-26T08:40:59.419371+0000","last_fullsized":"2024-03-26T08:40:59.419371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:35:11.181812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.347366+0000","last_change":"2024-03-26T08:40:09.932674+0000","last_active":"2024-03-26T08:41:28.347366+0000","last_peered":"2024-03-26T08:41:28.347366+0000","last_clean":"2024-03-26T08:41:28.347366+0000","last_became_active":"2024-03-26T08:40:09.932346+0000","last_became_peered":"2024-03-26T08:40:09.932346+0000","last_unstale":"2024-03-26T08:41:28.347366+0000","last_undegraded":"2024-03-26T08:41:28.347366+0000","last_fullsized":"2024-03-26T08:41:28.347366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:12:45.871467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.431219+0000","last_change":"2024-03-26T08:40:59.431219+0000","last_active":"2024-03-26T08:40:59.431219+0000","last_peered":"2024-03-26T08:40:59.431219+0000","last_clean":"2024-03-26T08:40:59.431219+0000","last_became_active":"2024-03-26T08:40:59.424868+0000","last_became_peered":"2024-03-26T08:40:59.424868+0000","last_unstale":"2024-03-26T08:40:59.431219+0000","last_undegraded":"2024-03-26T08:40:59.431219+0000","last_fullsized":"2024-03-26T08:40:59.431219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:02.837090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419228+0000","last_change":"2024-03-26T08:40:09.922766+0000","last_active":"2024-03-26T08:40:59.419228+0000","last_peered":"2024-03-26T08:40:59.419228+0000","last_clean":"2024-03-26T08:40:59.419228+0000","last_became_active":"2024-03-26T08:40:09.922453+0000","last_became_peered":"2024-03-26T08:40:09.922453+0000","last_unstale":"2024-03-26T08:40:59.419228+0000","last_undegraded":"2024-03-26T08:40:59.419228+0000","last_fullsized":"2024-03-26T08:40:59.419228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:13:28.740291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416510+0000","last_change":"2024-03-26T08:40:09.228114+0000","last_active":"2024-03-26T08:40:59.416510+0000","last_peered":"2024-03-26T08:40:59.416510+0000","last_clean":"2024-03-26T08:40:59.416510+0000","last_became_active":"2024-03-26T08:40:09.227803+0000","last_became_peered":"2024-03-26T08:40:09.227803+0000","last_unstale":"2024-03-26T08:40:59.416510+0000","last_undegraded":"2024-03-26T08:40:59.416510+0000","last_fullsized":"2024-03-26T08:40:59.416510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:31.717981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416928+0000","last_change":"2024-03-26T08:40:09.929671+0000","last_active":"2024-03-26T08:40:59.416928+0000","last_peered":"2024-03-26T08:40:59.416928+0000","last_clean":"2024-03-26T08:40:59.416928+0000","last_became_active":"2024-03-26T08:40:09.929222+0000","last_became_peered":"2024-03-26T08:40:09.929222+0000","last_unstale":"2024-03-26T08:40:59.416928+0000","last_undegraded":"2024-03-26T08:40:59.416928+0000","last_fullsized":"2024-03-26T08:40:59.416928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:00:50.579569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422136+0000","last_change":"2024-03-26T08:40:09.228994+0000","last_active":"2024-03-26T08:40:59.422136+0000","last_peered":"2024-03-26T08:40:59.422136+0000","last_clean":"2024-03-26T08:40:59.422136+0000","last_became_active":"2024-03-26T08:40:09.227138+0000","last_became_peered":"2024-03-26T08:40:09.227138+0000","last_unstale":"2024-03-26T08:40:59.422136+0000","last_undegraded":"2024-03-26T08:40:59.422136+0000","last_fullsized":"2024-03-26T08:40:59.422136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:06.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417219+0000","last_change":"2024-03-26T08:40:09.931752+0000","last_active":"2024-03-26T08:40:59.417219+0000","last_peered":"2024-03-26T08:40:59.417219+0000","last_clean":"2024-03-26T08:40:59.417219+0000","last_became_active":"2024-03-26T08:40:09.931436+0000","last_became_peered":"2024-03-26T08:40:09.931436+0000","last_unstale":"2024-03-26T08:40:59.417219+0000","last_undegraded":"2024-03-26T08:40:59.417219+0000","last_fullsized":"2024-03-26T08:40:59.417219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:40:54.864614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.418961+0000","last_change":"2024-03-26T08:40:09.238356+0000","last_active":"2024-03-26T08:40:59.418961+0000","last_peered":"2024-03-26T08:40:59.418961+0000","last_clean":"2024-03-26T08:40:59.418961+0000","last_became_active":"2024-03-26T08:40:09.238066+0000","last_became_peered":"2024-03-26T08:40:09.238066+0000","last_unstale":"2024-03-26T08:40:59.418961+0000","last_undegraded":"2024-03-26T08:40:59.418961+0000","last_fullsized":"2024-03-26T08:40:59.418961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:13.851490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.427153+0000","last_change":"2024-03-26T08:40:59.427153+0000","last_active":"2024-03-26T08:40:59.427153+0000","last_peered":"2024-03-26T08:40:59.427153+0000","last_clean":"2024-03-26T08:40:59.427153+0000","last_became_active":"2024-03-26T08:40:59.423044+0000","last_became_peered":"2024-03-26T08:40:59.423044+0000","last_unstale":"2024-03-26T08:40:59.427153+0000","last_undegraded":"2024-03-26T08:40:59.427153+0000","last_fullsized":"2024-03-26T08:40:59.427153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:20.181241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463434+0000","last_change":"2024-03-26T08:40:09.238009+0000","last_active":"2024-03-26T08:40:59.463434+0000","last_peered":"2024-03-26T08:40:59.463434+0000","last_clean":"2024-03-26T08:40:59.463434+0000","last_became_active":"2024-03-26T08:40:09.237697+0000","last_became_peered":"2024-03-26T08:40:09.237697+0000","last_unstale":"2024-03-26T08:40:59.463434+0000","last_undegraded":"2024-03-26T08:40:59.463434+0000","last_fullsized":"2024-03-26T08:40:59.463434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:17:33.179285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417423+0000","last_change":"2024-03-26T08:40:09.934672+0000","last_active":"2024-03-26T08:40:59.417423+0000","last_peered":"2024-03-26T08:40:59.417423+0000","last_clean":"2024-03-26T08:40:59.417423+0000","last_became_active":"2024-03-26T08:40:09.934365+0000","last_became_peered":"2024-03-26T08:40:09.934365+0000","last_unstale":"2024-03-26T08:40:59.417423+0000","last_undegraded":"2024-03-26T08:40:59.417423+0000","last_fullsized":"2024-03-26T08:40:59.417423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:41:22.318481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.346142+0000","last_change":"2024-03-26T08:40:09.933910+0000","last_active":"2024-03-26T08:41:28.346142+0000","last_peered":"2024-03-26T08:41:28.346142+0000","last_clean":"2024-03-26T08:41:28.346142+0000","last_became_active":"2024-03-26T08:40:09.933598+0000","last_became_peered":"2024-03-26T08:40:09.933598+0000","last_unstale":"2024-03-26T08:41:28.346142+0000","last_undegraded":"2024-03-26T08:41:28.346142+0000","last_fullsized":"2024-03-26T08:41:28.346142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:19.877984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422139+0000","last_change":"2024-03-26T08:40:09.935809+0000","last_active":"2024-03-26T08:40:59.422139+0000","last_peered":"2024-03-26T08:40:59.422139+0000","last_clean":"2024-03-26T08:40:59.422139+0000","last_became_active":"2024-03-26T08:40:09.935500+0000","last_became_peered":"2024-03-26T08:40:09.935500+0000","last_unstale":"2024-03-26T08:40:59.422139+0000","last_undegraded":"2024-03-26T08:40:59.422139+0000","last_fullsized":"2024-03-26T08:40:59.422139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:23.717845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421745+0000","last_change":"2024-03-26T08:40:09.928093+0000","last_active":"2024-03-26T08:40:59.421745+0000","last_peered":"2024-03-26T08:40:59.421745+0000","last_clean":"2024-03-26T08:40:59.421745+0000","last_became_active":"2024-03-26T08:40:09.927732+0000","last_became_peered":"2024-03-26T08:40:09.927732+0000","last_unstale":"2024-03-26T08:40:59.421745+0000","last_undegraded":"2024-03-26T08:40:59.421745+0000","last_fullsized":"2024-03-26T08:40:59.421745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:35:34.972435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338020+0000","last_change":"2024-03-26T08:40:59.463368+0000","last_active":"2024-03-26T08:41:28.338020+0000","last_peered":"2024-03-26T08:41:28.338020+0000","last_clean":"2024-03-26T08:41:28.338020+0000","last_became_active":"2024-03-26T08:40:59.462979+0000","last_became_peered":"2024-03-26T08:40:59.462979+0000","last_unstale":"2024-03-26T08:41:28.338020+0000","last_undegraded":"2024-03-26T08:41:28.338020+0000","last_fullsized":"2024-03-26T08:41:28.338020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:13:01.413240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463076+0000","last_change":"2024-03-26T08:40:09.919466+0000","last_active":"2024-03-26T08:40:59.463076+0000","last_peered":"2024-03-26T08:40:59.463076+0000","last_clean":"2024-03-26T08:40:59.463076+0000","last_became_active":"2024-03-26T08:40:09.919152+0000","last_became_peered":"2024-03-26T08:40:09.919152+0000","last_unstale":"2024-03-26T08:40:59.463076+0000","last_undegraded":"2024-03-26T08:40:59.463076+0000","last_fullsized":"2024-03-26T08:40:59.463076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:37.928485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345579+0000","last_change":"2024-03-26T08:40:09.927140+0000","last_active":"2024-03-26T08:41:28.345579+0000","last_peered":"2024-03-26T08:41:28.345579+0000","last_clean":"2024-03-26T08:41:28.345579+0000","last_became_active":"2024-03-26T08:40:09.926823+0000","last_became_peered":"2024-03-26T08:40:09.926823+0000","last_unstale":"2024-03-26T08:41:28.345579+0000","last_undegraded":"2024-03-26T08:41:28.345579+0000","last_fullsized":"2024-03-26T08:41:28.345579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:29:38.477123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422522+0000","last_change":"2024-03-26T08:40:09.934202+0000","last_active":"2024-03-26T08:40:59.422522+0000","last_peered":"2024-03-26T08:40:59.422522+0000","last_clean":"2024-03-26T08:40:59.422522+0000","last_became_active":"2024-03-26T08:40:09.933878+0000","last_became_peered":"2024-03-26T08:40:09.933878+0000","last_unstale":"2024-03-26T08:40:59.422522+0000","last_undegraded":"2024-03-26T08:40:59.422522+0000","last_fullsized":"2024-03-26T08:40:59.422522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:44:26.576140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419198+0000","last_change":"2024-03-26T08:40:09.929019+0000","last_active":"2024-03-26T08:40:59.419198+0000","last_peered":"2024-03-26T08:40:59.419198+0000","last_clean":"2024-03-26T08:40:59.419198+0000","last_became_active":"2024-03-26T08:40:09.928711+0000","last_became_peered":"2024-03-26T08:40:09.928711+0000","last_unstale":"2024-03-26T08:40:59.419198+0000","last_undegraded":"2024-03-26T08:40:59.419198+0000","last_fullsized":"2024-03-26T08:40:59.419198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:47.120826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419311+0000","last_change":"2024-03-26T08:40:09.927606+0000","last_active":"2024-03-26T08:40:59.419311+0000","last_peered":"2024-03-26T08:40:59.419311+0000","last_clean":"2024-03-26T08:40:59.419311+0000","last_became_active":"2024-03-26T08:40:09.926717+0000","last_became_peered":"2024-03-26T08:40:09.926717+0000","last_unstale":"2024-03-26T08:40:59.419311+0000","last_undegraded":"2024-03-26T08:40:59.419311+0000","last_fullsized":"2024-03-26T08:40:59.419311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:51:52.567252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419449+0000","last_change":"2024-03-26T08:40:09.934659+0000","last_active":"2024-03-26T08:40:59.419449+0000","last_peered":"2024-03-26T08:40:59.419449+0000","last_clean":"2024-03-26T08:40:59.419449+0000","last_became_active":"2024-03-26T08:40:09.934346+0000","last_became_peered":"2024-03-26T08:40:09.934346+0000","last_unstale":"2024-03-26T08:40:59.419449+0000","last_undegraded":"2024-03-26T08:40:59.419449+0000","last_fullsized":"2024-03-26T08:40:59.419449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:35.009264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502074+0000","last_change":"2024-03-26T08:40:09.925007+0000","last_active":"2024-03-26T08:40:59.502074+0000","last_peered":"2024-03-26T08:40:59.502074+0000","last_clean":"2024-03-26T08:40:59.502074+0000","last_became_active":"2024-03-26T08:40:09.923439+0000","last_became_peered":"2024-03-26T08:40:09.923439+0000","last_unstale":"2024-03-26T08:40:59.502074+0000","last_undegraded":"2024-03-26T08:40:59.502074+0000","last_fullsized":"2024-03-26T08:40:59.502074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:18.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463053+0000","last_change":"2024-03-26T08:40:09.920795+0000","last_active":"2024-03-26T08:40:59.463053+0000","last_peered":"2024-03-26T08:40:59.463053+0000","last_clean":"2024-03-26T08:40:59.463053+0000","last_became_active":"2024-03-26T08:40:09.920378+0000","last_became_peered":"2024-03-26T08:40:09.920378+0000","last_unstale":"2024-03-26T08:40:59.463053+0000","last_undegraded":"2024-03-26T08:40:59.463053+0000","last_fullsized":"2024-03-26T08:40:59.463053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:19:04.027727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344142+0000","last_change":"2024-03-26T08:40:59.425704+0000","last_active":"2024-03-26T08:41:28.344142+0000","last_peered":"2024-03-26T08:41:28.344142+0000","last_clean":"2024-03-26T08:41:28.344142+0000","last_became_active":"2024-03-26T08:40:59.425014+0000","last_became_peered":"2024-03-26T08:40:59.425014+0000","last_unstale":"2024-03-26T08:41:28.344142+0000","last_undegraded":"2024-03-26T08:41:28.344142+0000","last_fullsized":"2024-03-26T08:41:28.344142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:08:14.847536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282339,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68403,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":453,"num_read_kb":298,"num_write":158,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":456298,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":1112159,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":456512,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456527,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1048362,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":455901,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":520096,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":457172,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":521837,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":456212,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":456338,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1047929,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:41:31.513 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-26T08:41:31.513 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-26T08:41:31.513 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-26T08:41:31.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-26T08:41:31.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:31 smithi044 ceph-mon[36461]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-26T08:41:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:32.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:32 smithi016 ceph-mon[35851]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-26T08:41:32.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:32 smithi016 ceph-mon[35851]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:41:32.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:32 smithi007 ceph-mon[32462]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-26T08:41:32.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:32 smithi007 ceph-mon[32462]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:41:32.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:32 smithi044 ceph-mon[36461]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-03-26T08:41:32.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:32 smithi044 ceph-mon[36461]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:41:33.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.382+0000 7fd510780700 1 -- 172.21.15.7:0/4080455819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508107120 msgr2=0x7fd508107500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.382+0000 7fd510780700 1 --2- 172.21.15.7:0/4080455819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508107120 0x7fd508107500 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b0a0 tx=0x7fd4f800b3b0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.382+0000 7fd510780700 1 -- 172.21.15.7:0/4080455819 shutdown_connections 2024-03-26T08:41:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.382+0000 7fd510780700 1 --2- 172.21.15.7:0/4080455819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508101a30 0x7fd50810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.382+0000 7fd510780700 1 --2- 172.21.15.7:0/4080455819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508107120 0x7fd508107500 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.382+0000 7fd510780700 1 --2- 172.21.15.7:0/4080455819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508101070 0x7fd5081014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.382+0000 7fd510780700 1 -- 172.21.15.7:0/4080455819 >> 172.21.15.7:0/4080455819 conn(0x7fd5080780a0 msgr2=0x7fd5080784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.382+0000 7fd510780700 1 -- 172.21.15.7:0/4080455819 shutdown_connections 2024-03-26T08:41:33.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.382+0000 7fd510780700 1 -- 172.21.15.7:0/4080455819 wait complete. 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd510780700 1 Processor -- start 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd510780700 1 -- start start 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd510780700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508101070 0x7fd50819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd510780700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508101a30 0x7fd50819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd510780700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508107120 0x7fd508197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd510780700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd508111360 con 0x7fd508101070 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd510780700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081111e0 con 0x7fd508101a30 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd510780700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5081114e0 con 0x7fd508107120 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50e51c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508101070 0x7fd50819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:33.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50e51c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508101070 0x7fd50819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35820/0 (socket says 172.21.15.7:35820) 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50e51c700 1 -- 172.21.15.7:0/3283193109 learned_addr learned my addr 172.21.15.7:0/3283193109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50dd1b700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508101a30 0x7fd50819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50ed1d700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508107120 0x7fd508197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50e51c700 1 -- 172.21.15.7:0/3283193109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508107120 msgr2=0x7fd508197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50e51c700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508107120 0x7fd508197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50e51c700 1 -- 172.21.15.7:0/3283193109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508101a30 msgr2=0x7fd50819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50e51c700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508101a30 0x7fd50819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50e51c700 1 -- 172.21.15.7:0/3283193109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd508197f60 con 0x7fd508101070 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.383+0000 7fd50dd1b700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508101a30 0x7fd50819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.384+0000 7fd50e51c700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508101070 0x7fd50819d5b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80062a0 tx=0x7fd4f8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.384+0000 7fd5077fe700 1 -- 172.21.15.7:0/3283193109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8021070 con 0x7fd508101070 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.384+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5081981f0 con 0x7fd508101070 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.384+0000 7fd5077fe700 1 -- 172.21.15.7:0/3283193109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4f8007b80 con 0x7fd508101070 2024-03-26T08:41:33.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.384+0000 7fd5077fe700 1 -- 172.21.15.7:0/3283193109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f80052c0 con 0x7fd508101070 2024-03-26T08:41:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.384+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5081a4530 con 0x7fd508101070 2024-03-26T08:41:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.385+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd508198560 con 0x7fd508101070 2024-03-26T08:41:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.386+0000 7fd5077fe700 1 -- 172.21.15.7:0/3283193109 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd4f8024070 con 0x7fd508101070 2024-03-26T08:41:33.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.387+0000 7fd5077fe700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4f00a2890 0x7fd4f00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.387+0000 7fd50dd1b700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4f00a2890 0x7fd4f00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.387+0000 7fd5077fe700 1 -- 172.21.15.7:0/3283193109 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd4f800f020 con 0x7fd508101070 2024-03-26T08:41:33.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.387+0000 7fd50dd1b700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4f00a2890 0x7fd4f00a4d50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd508198e40 tx=0x7fd4fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:33.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.390+0000 7fd5077fe700 1 -- 172.21.15.7:0/3283193109 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd508198560 con 0x7fd508101070 2024-03-26T08:41:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.641+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd508068e80 con 0x7fd508101070 2024-03-26T08:41:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.641+0000 7fd5077fe700 1 -- 172.21.15.7:0/3283193109 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd508068e80 con 0x7fd508101070 2024-03-26T08:41:33.633 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:33.633 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-26T08:41:33.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4f00a2890 msgr2=0x7fd4f00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4f00a2890 0x7fd4f00a4d50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd508198e40 tx=0x7fd4fc009260 comp rx=0 tx=0).stop 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508101070 msgr2=0x7fd50819d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508101070 0x7fd50819d5b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80062a0 tx=0x7fd4f8004ce0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 shutdown_connections 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4f00a2890 0x7fd4f00a4d50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd508107120 0x7fd508197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd508101070 0x7fd50819d5b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 --2- 172.21.15.7:0/3283193109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd508101a30 0x7fd50819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 >> 172.21.15.7:0/3283193109 conn(0x7fd5080780a0 msgr2=0x7fd5080ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 shutdown_connections 2024-03-26T08:41:33.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:33.644+0000 7fd510780700 1 -- 172.21.15.7:0/3283193109 wait complete. 2024-03-26T08:41:34.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:34 smithi007 ceph-mon[32462]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-26T08:41:34.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:34 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3283193109' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T08:41:34.342 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-26T08:41:34.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:41:34.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/2076883561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81016f0 msgr2=0x7fa8c8101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/2076883561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81016f0 0x7fa8c8101b10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009900 tx=0x7fa8bc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/2076883561 shutdown_connections 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/2076883561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81032a0 0x7fa8c8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/2076883561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c81028e0 0x7fa8c8102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/2076883561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81016f0 0x7fa8c8101b10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/2076883561 >> 172.21.15.7:0/2076883561 conn(0x7fa8c80fcc60 msgr2=0x7fa8c80ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/2076883561 shutdown_connections 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/2076883561 wait complete. 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 Processor -- start 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.460+0000 7fa8cc8ff640 1 -- start start 2024-03-26T08:41:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8cc8ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c81016f0 0x7fa8c8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8cc8ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81028e0 0x7fa8c8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8cc8ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81032a0 0x7fa8c8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8cc8ff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c810d3c0 con 0x7fa8c81032a0 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8cc8ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c810d240 con 0x7fa8c81016f0 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8cc8ff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c810d540 con 0x7fa8c81028e0 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81032a0 0x7fa8c8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81032a0 0x7fa8c8073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35840/0 (socket says 172.21.15.7:35840) 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c6d9d640 1 -- 172.21.15.7:0/3730911428 learned_addr learned my addr 172.21.15.7:0/3730911428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c659c640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c81016f0 0x7fa8c8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c5d9b640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81028e0 0x7fa8c8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c6d9d640 1 -- 172.21.15.7:0/3730911428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81028e0 msgr2=0x7fa8c8073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c6d9d640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81028e0 0x7fa8c8073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c6d9d640 1 -- 172.21.15.7:0/3730911428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c81016f0 msgr2=0x7fa8c8075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c6d9d640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c81016f0 0x7fa8c8075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c6d9d640 1 -- 172.21.15.7:0/3730911428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8074860 con 0x7fa8c81032a0 2024-03-26T08:41:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c659c640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c81016f0 0x7fa8c8075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c5d9b640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81028e0 0x7fa8c8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8c6d9d640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81032a0 0x7fa8c8073fd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000eb10 tx=0x7fa8c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8b77fe640 1 -- 172.21.15.7:0/3730911428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0004b20 con 0x7fa8c81032a0 2024-03-26T08:41:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8b77fe640 1 -- 172.21.15.7:0/3730911428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8c0004cc0 con 0x7fa8c81032a0 2024-03-26T08:41:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c81a1b00 con 0x7fa8c81032a0 2024-03-26T08:41:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.461+0000 7fa8b77fe640 1 -- 172.21.15.7:0/3730911428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0010760 con 0x7fa8c81032a0 2024-03-26T08:41:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.462+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c81a2050 con 0x7fa8c81032a0 2024-03-26T08:41:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.463+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c81089b0 con 0x7fa8c81032a0 2024-03-26T08:41:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.464+0000 7fa8b77fe640 1 -- 172.21.15.7:0/3730911428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa8c0005020 con 0x7fa8c81032a0 2024-03-26T08:41:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.464+0000 7fa8b77fe640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8b00a2800 0x7fa8b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.464+0000 7fa8c659c640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8b00a2800 0x7fa8b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.464+0000 7fa8b77fe640 1 -- 172.21.15.7:0/3730911428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa8c00cf160 con 0x7fa8c81032a0 2024-03-26T08:41:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.465+0000 7fa8c659c640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8b00a2800 0x7fa8b00a4cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc02c170 tx=0x7fa8bc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.469+0000 7fa8b77fe640 1 -- 172.21.15.7:0/3730911428 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa8c009c200 con 0x7fa8c81032a0 2024-03-26T08:41:34.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:34 smithi016 ceph-mon[35851]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-26T08:41:34.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:34 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3283193109' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T08:41:34.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:34 smithi044 ceph-mon[36461]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-26T08:41:34.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:34 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3283193109' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T08:41:34.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.654+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c80622a0 con 0x7fa8c81032a0 2024-03-26T08:41:34.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.656+0000 7fa8b77fe640 1 -- 172.21.15.7:0/3730911428 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7fa8c009bb00 con 0x7fa8c81032a0 2024-03-26T08:41:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:34.647 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:41:34.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8b00a2800 msgr2=0x7fa8b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8b00a2800 0x7fa8b00a4cc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc02c170 tx=0x7fa8bc02df90 comp rx=0 tx=0).stop 2024-03-26T08:41:34.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81032a0 msgr2=0x7fa8c8073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81032a0 0x7fa8c8073fd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000eb10 tx=0x7fa8c000c2d0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 shutdown_connections 2024-03-26T08:41:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8b00a2800 0x7fa8b00a4cc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81032a0 0x7fa8c8073fd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8c81028e0 0x7fa8c8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 --2- 172.21.15.7:0/3730911428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c81016f0 0x7fa8c8075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 >> 172.21.15.7:0/3730911428 conn(0x7fa8c80fcc60 msgr2=0x7fa8c8105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 shutdown_connections 2024-03-26T08:41:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.658+0000 7fa8cc8ff640 1 -- 172.21.15.7:0/3730911428 wait complete. 2024-03-26T08:41:34.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:40:08.100906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:40:08.784972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:41:34.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:41:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.775+0000 7f15f672f640 1 -- 172.21.15.7:0/1067047924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0103260 msgr2=0x7f15f0109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.775+0000 7f15f672f640 1 --2- 172.21.15.7:0/1067047924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0103260 0x7f15f0109af0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009ca0 tx=0x7f15e802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.775+0000 7f15f672f640 1 -- 172.21.15.7:0/1067047924 shutdown_connections 2024-03-26T08:41:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.775+0000 7f15f672f640 1 --2- 172.21.15.7:0/1067047924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0103260 0x7f15f0109af0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.775+0000 7f15f672f640 1 --2- 172.21.15.7:0/1067047924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f01028a0 0x7f15f0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.775+0000 7f15f672f640 1 --2- 172.21.15.7:0/1067047924 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15f01016b0 0x7f15f0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.775+0000 7f15f672f640 1 -- 172.21.15.7:0/1067047924 >> 172.21.15.7:0/1067047924 conn(0x7f15f00fcc20 msgr2=0x7f15f00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.775+0000 7f15f672f640 1 -- 172.21.15.7:0/1067047924 shutdown_connections 2024-03-26T08:41:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.775+0000 7f15f672f640 1 -- 172.21.15.7:0/1067047924 wait complete. 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15f672f640 1 Processor -- start 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15f672f640 1 -- start start 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15f672f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15f01016b0 0x7f15f0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15f672f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f01028a0 0x7f15f0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15f672f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f0103260 0x7f15f019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15f672f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f010d330 con 0x7f15f01028a0 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15f672f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f010d1b0 con 0x7f15f0103260 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15f672f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f010d4b0 con 0x7f15f01016b0 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f01028a0 0x7f15f0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f01028a0 0x7f15f0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35850/0 (socket says 172.21.15.7:35850) 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15ef7fe640 1 -- 172.21.15.7:0/1264206754 learned_addr learned my addr 172.21.15.7:0/1264206754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15effff640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15f01016b0 0x7f15f0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15f4ccc640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f0103260 0x7f15f019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15ef7fe640 1 -- 172.21.15.7:0/1264206754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15f01016b0 msgr2=0x7f15f0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15ef7fe640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15f01016b0 0x7f15f0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15ef7fe640 1 -- 172.21.15.7:0/1264206754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f0103260 msgr2=0x7f15f019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15ef7fe640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f0103260 0x7f15f019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15ef7fe640 1 -- 172.21.15.7:0/1264206754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f019f590 con 0x7f15f01028a0 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.776+0000 7f15ef7fe640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f01028a0 0x7f15f0197eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f15e400ea60 tx=0x7f15e400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.777+0000 7f15ed7fa640 1 -- 172.21.15.7:0/1264206754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e400cde0 con 0x7f15f01028a0 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.777+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f019f880 con 0x7f15f01028a0 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.777+0000 7f15ed7fa640 1 -- 172.21.15.7:0/1264206754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15e40035f0 con 0x7f15f01028a0 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.777+0000 7f15ed7fa640 1 -- 172.21.15.7:0/1264206754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4010670 con 0x7f15f01028a0 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.777+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f019fdd0 con 0x7f15f01028a0 2024-03-26T08:41:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.778+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f0108970 con 0x7f15f01028a0 2024-03-26T08:41:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.779+0000 7f15ed7fa640 1 -- 172.21.15.7:0/1264206754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f15e40037a0 con 0x7f15f01028a0 2024-03-26T08:41:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.779+0000 7f15ed7fa640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15d80a2800 0x7f15d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.779+0000 7f15effff640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15d80a2800 0x7f15d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.779+0000 7f15ed7fa640 1 -- 172.21.15.7:0/1264206754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f15e40cee50 con 0x7f15f01028a0 2024-03-26T08:41:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.780+0000 7f15effff640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15d80a2800 0x7f15d80a4cc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f15f0102700 tx=0x7f15e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:34.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.783+0000 7f15ed7fa640 1 -- 172.21.15.7:0/1264206754 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f15e409bf40 con 0x7f15f01028a0 2024-03-26T08:41:34.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.969+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15f002d200 con 0x7f15f01028a0 2024-03-26T08:41:34.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.969+0000 7f15ed7fa640 1 -- 172.21.15.7:0/1264206754 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15e409b840 con 0x7f15f01028a0 2024-03-26T08:41:34.960 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15d80a2800 msgr2=0x7f15d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15d80a2800 0x7f15d80a4cc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f15f0102700 tx=0x7f15e0005f90 comp rx=0 tx=0).stop 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f01028a0 msgr2=0x7f15f0197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f01028a0 0x7f15f0197eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f15e400ea60 tx=0x7f15e400ef20 comp rx=0 tx=0).stop 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 shutdown_connections 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15d80a2800 0x7f15d80a4cc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f0103260 0x7f15f019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f01028a0 0x7f15f0197eb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 --2- 172.21.15.7:0/1264206754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15f01016b0 0x7f15f0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 >> 172.21.15.7:0/1264206754 conn(0x7f15f00fcc20 msgr2=0x7f15f0105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 shutdown_connections 2024-03-26T08:41:34.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:34.971+0000 7f15f672f640 1 -- 172.21.15.7:0/1264206754 wait complete. 2024-03-26T08:41:34.974 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:41:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.088+0000 7f1a64705640 1 -- 172.21.15.7:0/1815591746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 msgr2=0x7f1a5c101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.088+0000 7f1a64705640 1 --2- 172.21.15.7:0/1815591746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 0x7f1a5c101c30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009900 tx=0x7f1a4c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.088+0000 7f1a64705640 1 -- 172.21.15.7:0/1815591746 shutdown_connections 2024-03-26T08:41:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.088+0000 7f1a64705640 1 --2- 172.21.15.7:0/1815591746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c102170 0x7f1a5c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.088+0000 7f1a64705640 1 --2- 172.21.15.7:0/1815591746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 0x7f1a5c101c30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.088+0000 7f1a64705640 1 --2- 172.21.15.7:0/1815591746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a5c1005c0 0x7f1a5c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.088+0000 7f1a64705640 1 -- 172.21.15.7:0/1815591746 >> 172.21.15.7:0/1815591746 conn(0x7f1a5c0fbb30 msgr2=0x7f1a5c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.088+0000 7f1a64705640 1 -- 172.21.15.7:0/1815591746 shutdown_connections 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.088+0000 7f1a64705640 1 -- 172.21.15.7:0/1815591746 wait complete. 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a64705640 1 Processor -- start 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a64705640 1 -- start start 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a64705640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a5c1005c0 0x7f1a5c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a64705640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 0x7f1a5c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a64705640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c102170 0x7f1a5c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a64705640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c10d360 con 0x7f1a5c1017b0 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a64705640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c10d1e0 con 0x7f1a5c102170 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a64705640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c10d4e0 con 0x7f1a5c1005c0 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a61ca0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 0x7f1a5c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a61ca0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 0x7f1a5c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35874/0 (socket says 172.21.15.7:35874) 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a61ca0640 1 -- 172.21.15.7:0/700112993 learned_addr learned my addr 172.21.15.7:0/700112993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a62ca2640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c102170 0x7f1a5c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a624a1640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a5c1005c0 0x7f1a5c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a61ca0640 1 -- 172.21.15.7:0/700112993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a5c1005c0 msgr2=0x7f1a5c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a61ca0640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a5c1005c0 0x7f1a5c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a61ca0640 1 -- 172.21.15.7:0/700112993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c102170 msgr2=0x7f1a5c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a61ca0640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c102170 0x7f1a5c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.089+0000 7f1a61ca0640 1 -- 172.21.15.7:0/700112993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c19f620 con 0x7f1a5c1017b0 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.090+0000 7f1a61ca0640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 0x7f1a5c197f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009230 tx=0x7f1a4c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.090+0000 7f1a537fe640 1 -- 172.21.15.7:0/700112993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c041070 con 0x7f1a5c1017b0 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.090+0000 7f1a537fe640 1 -- 172.21.15.7:0/700112993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a4c004450 con 0x7f1a5c1017b0 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.090+0000 7f1a537fe640 1 -- 172.21.15.7:0/700112993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c003e20 con 0x7f1a5c1017b0 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.090+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c19f8b0 con 0x7f1a5c1017b0 2024-03-26T08:41:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.090+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c19fdf0 con 0x7f1a5c1017b0 2024-03-26T08:41:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.091+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c1078e0 con 0x7f1a5c1017b0 2024-03-26T08:41:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.092+0000 7f1a537fe640 1 -- 172.21.15.7:0/700112993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1a4c044070 con 0x7f1a5c1017b0 2024-03-26T08:41:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.093+0000 7f1a537fe640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a480a2880 0x7f1a480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.093+0000 7f1a624a1640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a480a2880 0x7f1a480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.093+0000 7f1a537fe640 1 -- 172.21.15.7:0/700112993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1a4c0f0570 con 0x7f1a5c1017b0 2024-03-26T08:41:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.093+0000 7f1a624a1640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a480a2880 0x7f1a480a4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c101610 tx=0x7f1a58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:35.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.097+0000 7f1a537fe640 1 -- 172.21.15.7:0/700112993 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1a4c0bcfc0 con 0x7f1a5c1017b0 2024-03-26T08:41:35.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.281+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a5c0622a0 con 0x7f1a5c1017b0 2024-03-26T08:41:35.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.282+0000 7f1a537fe640 1 -- 172.21.15.7:0/700112993 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a4c0c1060 con 0x7f1a5c1017b0 2024-03-26T08:41:35.273 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-26T08:41:35.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a480a2880 msgr2=0x7f1a480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a480a2880 0x7f1a480a4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c101610 tx=0x7f1a58005f90 comp rx=0 tx=0).stop 2024-03-26T08:41:35.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 msgr2=0x7f1a5c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 0x7f1a5c197f40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009230 tx=0x7f1a4c02c650 comp rx=0 tx=0).stop 2024-03-26T08:41:35.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 shutdown_connections 2024-03-26T08:41:35.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a480a2880 0x7f1a480a4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c102170 0x7f1a5c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a5c1017b0 0x7f1a5c197f40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 --2- 172.21.15.7:0/700112993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a5c1005c0 0x7f1a5c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 >> 172.21.15.7:0/700112993 conn(0x7f1a5c0fbb30 msgr2=0x7f1a5c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:35.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 shutdown_connections 2024-03-26T08:41:35.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.284+0000 7f1a64705640 1 -- 172.21.15.7:0/700112993 wait complete. 2024-03-26T08:41:35.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:35 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3730911428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:41:35.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:35 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1264206754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:41:35.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:41:35.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 -- 172.21.15.7:0/33588549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881017b0 msgr2=0x7f5e88101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/33588549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881017b0 0x7f5e88101c30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009900 tx=0x7f5e7002bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:35.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 -- 172.21.15.7:0/33588549 shutdown_connections 2024-03-26T08:41:35.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/33588549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e88102170 0x7f5e88108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/33588549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881017b0 0x7f5e88101c30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/33588549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e881005c0 0x7f5e881009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 -- 172.21.15.7:0/33588549 >> 172.21.15.7:0/33588549 conn(0x7f5e880fbb30 msgr2=0x7f5e880fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:35.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 -- 172.21.15.7:0/33588549 shutdown_connections 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 -- 172.21.15.7:0/33588549 wait complete. 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 Processor -- start 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 -- start start 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881005c0 0x7f5e88199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e881017b0 0x7f5e8819a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88102170 0x7f5e881a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8810c280 con 0x7f5e881005c0 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8810c100 con 0x7f5e881017b0 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8ed53640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8810c400 con 0x7f5e88102170 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.401+0000 7f5e8caef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881005c0 0x7f5e88199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8caef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881005c0 0x7f5e88199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35892/0 (socket says 172.21.15.7:35892) 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8caef640 1 -- 172.21.15.7:0/1741038341 learned_addr learned my addr 172.21.15.7:0/1741038341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e87fff640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e881017b0 0x7f5e8819a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8d2f0640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88102170 0x7f5e881a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8caef640 1 -- 172.21.15.7:0/1741038341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88102170 msgr2=0x7f5e881a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8caef640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88102170 0x7f5e881a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8caef640 1 -- 172.21.15.7:0/1741038341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e881017b0 msgr2=0x7f5e8819a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8caef640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e881017b0 0x7f5e8819a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8caef640 1 -- 172.21.15.7:0/1741038341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e881a1810 con 0x7f5e881005c0 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e87fff640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e881017b0 0x7f5e8819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8caef640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881005c0 0x7f5e88199bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009d20 tx=0x7f5e8000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e85ffb640 1 -- 172.21.15.7:0/1741038341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80010630 con 0x7f5e881005c0 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e881a1b00 con 0x7f5e881005c0 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e85ffb640 1 -- 172.21.15.7:0/1741038341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e80010cb0 con 0x7f5e881005c0 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e85ffb640 1 -- 172.21.15.7:0/1741038341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80018de0 con 0x7f5e881005c0 2024-03-26T08:41:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.402+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e881a2020 con 0x7f5e881005c0 2024-03-26T08:41:35.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.403+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e881a21f0 con 0x7f5e881005c0 2024-03-26T08:41:35.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.404+0000 7f5e85ffb640 1 -- 172.21.15.7:0/1741038341 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5e8001f020 con 0x7f5e881005c0 2024-03-26T08:41:35.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.405+0000 7f5e85ffb640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5e740a2800 0x7f5e740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.405+0000 7f5e87fff640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5e740a2800 0x7f5e740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.405+0000 7f5e85ffb640 1 -- 172.21.15.7:0/1741038341 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5e800ceea0 con 0x7f5e881005c0 2024-03-26T08:41:35.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.405+0000 7f5e87fff640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5e740a2800 0x7f5e740a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e7002c240 tx=0x7f5e7002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:35.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.409+0000 7f5e85ffb640 1 -- 172.21.15.7:0/1741038341 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5e8009bf40 con 0x7f5e881005c0 2024-03-26T08:41:35.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:35 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3730911428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:41:35.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:35 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1264206754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:41:35.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:35 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3730911428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:41:35.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:35 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1264206754' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:41:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.593+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e881a2460 con 0x7f5e881005c0 2024-03-26T08:41:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.593+0000 7f5e85ffb640 1 -- 172.21.15.7:0/1741038341 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e8009b840 con 0x7f5e881005c0 2024-03-26T08:41:35.585 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-26T08:41:35.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5e740a2800 msgr2=0x7f5e740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5e740a2800 0x7f5e740a4cc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e7002c240 tx=0x7f5e7002df90 comp rx=0 tx=0).stop 2024-03-26T08:41:35.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881005c0 msgr2=0x7f5e88199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881005c0 0x7f5e88199bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009d20 tx=0x7f5e8000c6a0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 shutdown_connections 2024-03-26T08:41:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5e740a2800 0x7f5e740a4cc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5e88102170 0x7f5e881a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e881017b0 0x7f5e8819a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 --2- 172.21.15.7:0/1741038341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e881005c0 0x7f5e88199bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.595+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 >> 172.21.15.7:0/1741038341 conn(0x7f5e880fbb30 msgr2=0x7f5e88068970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.596+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 shutdown_connections 2024-03-26T08:41:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.596+0000 7f5e8ed53640 1 -- 172.21.15.7:0/1741038341 wait complete. 2024-03-26T08:41:35.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:41:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.708+0000 7fb02073b640 1 -- 172.21.15.7:0/1266759074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181016f0 msgr2=0x7fb018101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.708+0000 7fb02073b640 1 --2- 172.21.15.7:0/1266759074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181016f0 0x7fb018101b10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009900 tx=0x7fb00c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.708+0000 7fb02073b640 1 -- 172.21.15.7:0/1266759074 shutdown_connections 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.708+0000 7fb02073b640 1 --2- 172.21.15.7:0/1266759074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0181032a0 0x7fb018109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.708+0000 7fb02073b640 1 --2- 172.21.15.7:0/1266759074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0181028e0 0x7fb018102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.708+0000 7fb02073b640 1 --2- 172.21.15.7:0/1266759074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181016f0 0x7fb018101b10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.708+0000 7fb02073b640 1 -- 172.21.15.7:0/1266759074 >> 172.21.15.7:0/1266759074 conn(0x7fb0180fcc60 msgr2=0x7fb0180ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.708+0000 7fb02073b640 1 -- 172.21.15.7:0/1266759074 shutdown_connections 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.708+0000 7fb02073b640 1 -- 172.21.15.7:0/1266759074 wait complete. 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb02073b640 1 Processor -- start 2024-03-26T08:41:35.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb02073b640 1 -- start start 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb02073b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0181016f0 0x7fb018197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb02073b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181028e0 0x7fb018197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb02073b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0181032a0 0x7fb01819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb02073b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01810d3c0 con 0x7fb0181028e0 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb02073b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01810d240 con 0x7fb0181016f0 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb02073b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01810d540 con 0x7fb0181032a0 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb01dcd6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181028e0 0x7fb018197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb01dcd6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181028e0 0x7fb018197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35912/0 (socket says 172.21.15.7:35912) 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb01dcd6640 1 -- 172.21.15.7:0/4217689421 learned_addr learned my addr 172.21.15.7:0/4217689421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb01e4d7640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0181016f0 0x7fb018197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb01ecd8640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0181032a0 0x7fb01819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb01dcd6640 1 -- 172.21.15.7:0/4217689421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0181032a0 msgr2=0x7fb01819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.709+0000 7fb01dcd6640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0181032a0 0x7fb01819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.710+0000 7fb01dcd6640 1 -- 172.21.15.7:0/4217689421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0181016f0 msgr2=0x7fb018197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.710+0000 7fb01dcd6640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0181016f0 0x7fb018197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.710+0000 7fb01dcd6640 1 -- 172.21.15.7:0/4217689421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01819f620 con 0x7fb0181028e0 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.710+0000 7fb01dcd6640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181028e0 0x7fb018197f40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb01400ea60 tx=0x7fb01400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.710+0000 7fb00b7fe640 1 -- 172.21.15.7:0/4217689421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400cde0 con 0x7fb0181028e0 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.710+0000 7fb00b7fe640 1 -- 172.21.15.7:0/4217689421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0140035f0 con 0x7fb0181028e0 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.710+0000 7fb00b7fe640 1 -- 172.21.15.7:0/4217689421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014010670 con 0x7fb0181028e0 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.710+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01819f910 con 0x7fb0181028e0 2024-03-26T08:41:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.710+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01819fe60 con 0x7fb0181028e0 2024-03-26T08:41:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.711+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0181089b0 con 0x7fb0181028e0 2024-03-26T08:41:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.712+0000 7fb00b7fe640 1 -- 172.21.15.7:0/4217689421 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb0140037a0 con 0x7fb0181028e0 2024-03-26T08:41:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.712+0000 7fb00b7fe640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0040a2800 0x7fb0040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.713+0000 7fb01e4d7640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0040a2800 0x7fb0040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.713+0000 7fb00b7fe640 1 -- 172.21.15.7:0/4217689421 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb0140ced90 con 0x7fb0181028e0 2024-03-26T08:41:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.713+0000 7fb01e4d7640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0040a2800 0x7fb0040a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb00c02c170 tx=0x7fb00c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:35.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.717+0000 7fb00b7fe640 1 -- 172.21.15.7:0/4217689421 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb01409bdf0 con 0x7fb0181028e0 2024-03-26T08:41:35.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.941+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0180622a0 con 0x7fb0181028e0 2024-03-26T08:41:35.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.942+0000 7fb00b7fe640 1 -- 172.21.15.7:0/4217689421 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fb01409b6f0 con 0x7fb0181028e0 2024-03-26T08:41:35.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:35.934 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:40:15.277649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:41:35.934 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0040a2800 msgr2=0x7fb0040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0040a2800 0x7fb0040a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb00c02c170 tx=0x7fb00c02df90 comp rx=0 tx=0).stop 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181028e0 msgr2=0x7fb018197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181028e0 0x7fb018197f40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb01400ea60 tx=0x7fb01400ef20 comp rx=0 tx=0).stop 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 shutdown_connections 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0040a2800 0x7fb0040a4cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0181032a0 0x7fb01819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0181028e0 0x7fb018197f40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 --2- 172.21.15.7:0/4217689421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0181016f0 0x7fb018197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 >> 172.21.15.7:0/4217689421 conn(0x7fb0180fcc60 msgr2=0x7fb018105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:35.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 shutdown_connections 2024-03-26T08:41:35.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:35.944+0000 7fb02073b640 1 -- 172.21.15.7:0/4217689421 wait complete. 2024-03-26T08:41:35.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:41:36.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 -- 172.21.15.7:0/1201730962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 msgr2=0x7f1d14109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 --2- 172.21.15.7:0/1201730962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 0x7f1d14109860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009ca0 tx=0x7f1d0c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 -- 172.21.15.7:0/1201730962 shutdown_connections 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 --2- 172.21.15.7:0/1201730962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 0x7f1d14109860 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 --2- 172.21.15.7:0/1201730962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d14104ae0 0x7f1d14106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 --2- 172.21.15.7:0/1201730962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d141021b0 0x7f1d141045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 -- 172.21.15.7:0/1201730962 >> 172.21.15.7:0/1201730962 conn(0x7f1d140fbb80 msgr2=0x7f1d140fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 -- 172.21.15.7:0/1201730962 shutdown_connections 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 -- 172.21.15.7:0/1201730962 wait complete. 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 Processor -- start 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.059+0000 7f1d18d90640 1 -- start start 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d18d90640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d141021b0 0x7f1d141979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d18d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d14104ae0 0x7f1d14197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d18d90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 0x7f1d1419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d18d90640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1410d360 con 0x7f1d14107410 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d18d90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1410d1e0 con 0x7f1d14104ae0 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d18d90640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1410d4e0 con 0x7f1d141021b0 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d12d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 0x7f1d1419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d1259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d141021b0 0x7f1d141979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d12d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 0x7f1d1419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35928/0 (socket says 172.21.15.7:35928) 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d1259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d141021b0 0x7f1d141979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46970/0 (socket says 172.21.15.7:46970) 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d12d9d640 1 -- 172.21.15.7:0/157995362 learned_addr learned my addr 172.21.15.7:0/157995362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d11d9b640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d14104ae0 0x7f1d14197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d12d9d640 1 -- 172.21.15.7:0/157995362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d141021b0 msgr2=0x7f1d141979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d12d9d640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d141021b0 0x7f1d141979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d12d9d640 1 -- 172.21.15.7:0/157995362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d14104ae0 msgr2=0x7f1d14197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d12d9d640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d14104ae0 0x7f1d14197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d12d9d640 1 -- 172.21.15.7:0/157995362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1419f5c0 con 0x7f1d14107410 2024-03-26T08:41:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d1259c640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d141021b0 0x7f1d141979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d12d9d640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 0x7f1d1419ef50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c02c630 tx=0x7f1d0c032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d037fe640 1 -- 172.21.15.7:0/157995362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c041020 con 0x7f1d14107410 2024-03-26T08:41:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d037fe640 1 -- 172.21.15.7:0/157995362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d0c034cd0 con 0x7f1d14107410 2024-03-26T08:41:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1419f850 con 0x7f1d14107410 2024-03-26T08:41:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d037fe640 1 -- 172.21.15.7:0/157995362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c039040 con 0x7f1d14107410 2024-03-26T08:41:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.060+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1419fd40 con 0x7f1d14107410 2024-03-26T08:41:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.061+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d140fd670 con 0x7f1d14107410 2024-03-26T08:41:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.062+0000 7f1d037fe640 1 -- 172.21.15.7:0/157995362 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1d0c044020 con 0x7f1d14107410 2024-03-26T08:41:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.063+0000 7f1d037fe640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1cfc0a2800 0x7f1cfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.063+0000 7f1d037fe640 1 -- 172.21.15.7:0/157995362 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d0c0f0880 con 0x7f1d14107410 2024-03-26T08:41:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.063+0000 7f1d1259c640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1cfc0a2800 0x7f1cfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.063+0000 7f1d1259c640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1cfc0a2800 0x7f1cfc0a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d040095f0 tx=0x7f1d04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:36.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.067+0000 7f1d037fe640 1 -- 172.21.15.7:0/157995362 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d0c04b020 con 0x7f1d14107410 2024-03-26T08:41:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.253+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d140622a0 con 0x7f1d14107410 2024-03-26T08:41:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.255+0000 7f1d037fe640 1 -- 172.21.15.7:0/157995362 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7f1d0c0b7e20 con 0x7f1d14107410 2024-03-26T08:41:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:36.246 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:41:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1cfc0a2800 msgr2=0x7f1cfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1cfc0a2800 0x7f1cfc0a4cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1d040095f0 tx=0x7f1d04005f90 comp rx=0 tx=0).stop 2024-03-26T08:41:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 msgr2=0x7f1d1419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 0x7f1d1419ef50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c02c630 tx=0x7f1d0c032ea0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 shutdown_connections 2024-03-26T08:41:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1cfc0a2800 0x7f1cfc0a4cc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d14107410 0x7f1d1419ef50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d14104ae0 0x7f1d14197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 --2- 172.21.15.7:0/157995362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d141021b0 0x7f1d141979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 >> 172.21.15.7:0/157995362 conn(0x7f1d140fbb80 msgr2=0x7f1d14102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 shutdown_connections 2024-03-26T08:41:36.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.257+0000 7f1d18d90640 1 -- 172.21.15.7:0/157995362 wait complete. 2024-03-26T08:41:36.258 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:36 smithi007 ceph-mon[32462]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:41:36.258 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:36 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/700112993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:41:36.258 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:36 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1741038341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:41:36.258 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:36 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4217689421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:41:36.260 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:40:08.100906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:40:08.784972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:41:36.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:41:36.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 -- 172.21.15.7:0/2629954551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c1005b0 msgr2=0x7f364c1009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 --2- 172.21.15.7:0/2629954551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c1005b0 0x7f364c1009d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3640009900 tx=0x7f364002bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:36.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 -- 172.21.15.7:0/2629954551 shutdown_connections 2024-03-26T08:41:36.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 --2- 172.21.15.7:0/2629954551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c102160 0x7f364c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 --2- 172.21.15.7:0/2629954551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f364c1017a0 0x7f364c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 --2- 172.21.15.7:0/2629954551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c1005b0 0x7f364c1009d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 -- 172.21.15.7:0/2629954551 >> 172.21.15.7:0/2629954551 conn(0x7f364c0fbb20 msgr2=0x7f364c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 -- 172.21.15.7:0/2629954551 shutdown_connections 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 -- 172.21.15.7:0/2629954551 wait complete. 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.374+0000 7f3653026640 1 Processor -- start 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3653026640 1 -- start start 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3653026640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f364c1005b0 0x7f364c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3653026640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c1017a0 0x7f364c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3653026640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c102160 0x7f364c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3653026640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c10c220 con 0x7f364c102160 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3653026640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c10c0a0 con 0x7f364c1017a0 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3653026640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364c10c3a0 con 0x7f364c1005b0 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f36515c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c102160 0x7f364c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f36515c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c102160 0x7f364c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35940/0 (socket says 172.21.15.7:35940) 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f36515c3640 1 -- 172.21.15.7:0/666999246 learned_addr learned my addr 172.21.15.7:0/666999246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f364bfff640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c1017a0 0x7f364c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3650dc2640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f364c1005b0 0x7f364c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f36515c3640 1 -- 172.21.15.7:0/666999246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f364c1005b0 msgr2=0x7f364c1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f36515c3640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f364c1005b0 0x7f364c1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f36515c3640 1 -- 172.21.15.7:0/666999246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c1017a0 msgr2=0x7f364c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f36515c3640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c1017a0 0x7f364c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f36515c3640 1 -- 172.21.15.7:0/666999246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c19f610 con 0x7f364c102160 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f36515c3640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c102160 0x7f364c19efa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f364400eb10 tx=0x7f364400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3649ffb640 1 -- 172.21.15.7:0/666999246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644004b20 con 0x7f364c102160 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364c19f900 con 0x7f364c102160 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3649ffb640 1 -- 172.21.15.7:0/666999246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3644004cc0 con 0x7f364c102160 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.375+0000 7f3649ffb640 1 -- 172.21.15.7:0/666999246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3644010760 con 0x7f364c102160 2024-03-26T08:41:36.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.376+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364c19fe00 con 0x7f364c102160 2024-03-26T08:41:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.377+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364c1a0090 con 0x7f364c102160 2024-03-26T08:41:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.377+0000 7f3649ffb640 1 -- 172.21.15.7:0/666999246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3644005000 con 0x7f364c102160 2024-03-26T08:41:36.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.378+0000 7f3649ffb640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f363c0a2800 0x7f363c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.378+0000 7f3650dc2640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f363c0a2800 0x7f363c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.378+0000 7f3649ffb640 1 -- 172.21.15.7:0/666999246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f36440ceff0 con 0x7f364c102160 2024-03-26T08:41:36.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.378+0000 7f3650dc2640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f363c0a2800 0x7f363c0a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3640000c00 tx=0x7f3640005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:36.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.382+0000 7f3649ffb640 1 -- 172.21.15.7:0/666999246 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f364409c070 con 0x7f364c102160 2024-03-26T08:41:36.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:36 smithi016 ceph-mon[35851]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:41:36.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:36 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/700112993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:41:36.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:36 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1741038341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:41:36.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:36 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4217689421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:41:36.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.565+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f364c1a0260 con 0x7f364c102160 2024-03-26T08:41:36.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.565+0000 7f3649ffb640 1 -- 172.21.15.7:0/666999246 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f364409b970 con 0x7f364c102160 2024-03-26T08:41:36.557 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:41:36.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.567+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f363c0a2800 msgr2=0x7f363c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.567+0000 7f3653026640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f363c0a2800 0x7f363c0a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3640000c00 tx=0x7f3640005fd0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.567+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c102160 msgr2=0x7f364c19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.567+0000 7f3653026640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c102160 0x7f364c19efa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f364400eb10 tx=0x7f364400c2d0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.568+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 shutdown_connections 2024-03-26T08:41:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.568+0000 7f3653026640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f363c0a2800 0x7f363c0a4cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.568+0000 7f3653026640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f364c102160 0x7f364c19efa0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.568+0000 7f3653026640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c1017a0 0x7f364c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.568+0000 7f3653026640 1 --2- 172.21.15.7:0/666999246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f364c1005b0 0x7f364c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.568+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 >> 172.21.15.7:0/666999246 conn(0x7f364c0fbb20 msgr2=0x7f364c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.568+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 shutdown_connections 2024-03-26T08:41:36.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.568+0000 7f3653026640 1 -- 172.21.15.7:0/666999246 wait complete. 2024-03-26T08:41:36.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:36 smithi044 ceph-mon[36461]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:41:36.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:36 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/700112993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:41:36.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:36 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1741038341' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:41:36.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:36 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4217689421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:41:36.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:41:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.684+0000 7fa7a3243640 1 -- 172.21.15.7:0/579228472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c103300 msgr2=0x7fa79c109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.684+0000 7fa7a3243640 1 --2- 172.21.15.7:0/579228472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c103300 0x7fa79c109b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa794009cd0 tx=0x7fa79402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.684+0000 7fa7a3243640 1 -- 172.21.15.7:0/579228472 shutdown_connections 2024-03-26T08:41:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.684+0000 7fa7a3243640 1 --2- 172.21.15.7:0/579228472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c103300 0x7fa79c109b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.684+0000 7fa7a3243640 1 --2- 172.21.15.7:0/579228472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79c102940 0x7fa79c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.684+0000 7fa7a3243640 1 --2- 172.21.15.7:0/579228472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79c0ff970 0x7fa79c0ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.684+0000 7fa7a3243640 1 -- 172.21.15.7:0/579228472 >> 172.21.15.7:0/579228472 conn(0x7fa79c0fb2c0 msgr2=0x7fa79c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.684+0000 7fa7a3243640 1 -- 172.21.15.7:0/579228472 shutdown_connections 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.684+0000 7fa7a3243640 1 -- 172.21.15.7:0/579228472 wait complete. 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a3243640 1 Processor -- start 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a3243640 1 -- start start 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a3243640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c0ff970 0x7fa79c1022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a3243640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79c102940 0x7fa79c1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a3243640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79c103300 0x7fa79c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a3243640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c10d320 con 0x7fa79c0ff970 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a3243640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c10d1a0 con 0x7fa79c103300 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a3243640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c10d4a0 con 0x7fa79c102940 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a0fdf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c0ff970 0x7fa79c1022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a0fdf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c0ff970 0x7fa79c1022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37980/0 (socket says 172.21.15.7:37980) 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a0fdf640 1 -- 172.21.15.7:0/3567633035 learned_addr learned my addr 172.21.15.7:0/3567633035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a17e0640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79c103300 0x7fa79c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa79bfff640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79c102940 0x7fa79c1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a0fdf640 1 -- 172.21.15.7:0/3567633035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79c102940 msgr2=0x7fa79c1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a0fdf640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79c102940 0x7fa79c1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a0fdf640 1 -- 172.21.15.7:0/3567633035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79c103300 msgr2=0x7fa79c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a0fdf640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79c103300 0x7fa79c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a0fdf640 1 -- 172.21.15.7:0/3567633035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c1016c0 con 0x7fa79c0ff970 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.685+0000 7fa7a0fdf640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c0ff970 0x7fa79c1022b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa79000e9c0 tx=0x7fa79000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.686+0000 7fa799ffb640 1 -- 172.21.15.7:0/3567633035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79000cde0 con 0x7fa79c0ff970 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.686+0000 7fa799ffb640 1 -- 172.21.15.7:0/3567633035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa790010910 con 0x7fa79c0ff970 2024-03-26T08:41:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.686+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c1a1ae0 con 0x7fa79c0ff970 2024-03-26T08:41:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.686+0000 7fa799ffb640 1 -- 172.21.15.7:0/3567633035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7900189c0 con 0x7fa79c0ff970 2024-03-26T08:41:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.686+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c1a2080 con 0x7fa79c0ff970 2024-03-26T08:41:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.687+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79c108ac0 con 0x7fa79c0ff970 2024-03-26T08:41:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.688+0000 7fa799ffb640 1 -- 172.21.15.7:0/3567633035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa790010ac0 con 0x7fa79c0ff970 2024-03-26T08:41:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.688+0000 7fa799ffb640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa78c0a2880 0x7fa78c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.688+0000 7fa79bfff640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa78c0a2880 0x7fa78c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.688+0000 7fa799ffb640 1 -- 172.21.15.7:0/3567633035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa7900cef00 con 0x7fa79c0ff970 2024-03-26T08:41:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.689+0000 7fa79bfff640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa78c0a2880 0x7fa78c0a4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa7880095f0 tx=0x7fa788005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:36.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.693+0000 7fa799ffb640 1 -- 172.21.15.7:0/3567633035 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa790053740 con 0x7fa79c0ff970 2024-03-26T08:41:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.876+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa79c0622a0 con 0x7fa79c0ff970 2024-03-26T08:41:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.877+0000 7fa799ffb640 1 -- 172.21.15.7:0/3567633035 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa79009bfa0 con 0x7fa79c0ff970 2024-03-26T08:41:36.868 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-26T08:41:36.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa78c0a2880 msgr2=0x7fa78c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa78c0a2880 0x7fa78c0a4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa7880095f0 tx=0x7fa788005f90 comp rx=0 tx=0).stop 2024-03-26T08:41:36.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c0ff970 msgr2=0x7fa79c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c0ff970 0x7fa79c1022b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa79000e9c0 tx=0x7fa79000ee80 comp rx=0 tx=0).stop 2024-03-26T08:41:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 shutdown_connections 2024-03-26T08:41:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa78c0a2880 0x7fa78c0a4d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa79c103300 0x7fa79c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa79c102940 0x7fa79c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 --2- 172.21.15.7:0/3567633035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa79c0ff970 0x7fa79c1022b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 >> 172.21.15.7:0/3567633035 conn(0x7fa79c0fb2c0 msgr2=0x7fa79c105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 shutdown_connections 2024-03-26T08:41:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.879+0000 7fa7a3243640 1 -- 172.21.15.7:0/3567633035 wait complete. 2024-03-26T08:41:36.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.995+0000 7f5715dcd640 1 -- 172.21.15.7:0/3808518760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 msgr2=0x7f5710106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.995+0000 7f5715dcd640 1 --2- 172.21.15.7:0/3808518760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 0x7f5710106ee0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f570402bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.995+0000 7f5715dcd640 1 -- 172.21.15.7:0/3808518760 shutdown_connections 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.995+0000 7f5715dcd640 1 --2- 172.21.15.7:0/3808518760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5710107420 0x7f5710109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.995+0000 7f5715dcd640 1 --2- 172.21.15.7:0/3808518760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 0x7f5710106ee0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.995+0000 7f5715dcd640 1 --2- 172.21.15.7:0/3808518760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57101021c0 0x7f57101045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.995+0000 7f5715dcd640 1 -- 172.21.15.7:0/3808518760 >> 172.21.15.7:0/3808518760 conn(0x7f57100fbb30 msgr2=0x7f57100fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.995+0000 7f5715dcd640 1 -- 172.21.15.7:0/3808518760 shutdown_connections 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.995+0000 7f5715dcd640 1 -- 172.21.15.7:0/3808518760 wait complete. 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f5715dcd640 1 Processor -- start 2024-03-26T08:41:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f5715dcd640 1 -- start start 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f5715dcd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57101021c0 0x7f5710197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f5715dcd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 0x7f5710197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f5715dcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5710107420 0x7f571019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f5715dcd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571010d330 con 0x7f5710104af0 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f5715dcd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571010d1b0 con 0x7f5710107420 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f5715dcd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571010d4b0 con 0x7f57101021c0 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 0x7f5710197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57101021c0 0x7f5710197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 0x7f5710197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37998/0 (socket says 172.21.15.7:37998) 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570effd640 1 -- 172.21.15.7:0/958617742 learned_addr learned my addr 172.21.15.7:0/958617742 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570ffff640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5710107420 0x7f571019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570f7fe640 1 -- 172.21.15.7:0/958617742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5710107420 msgr2=0x7f571019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570f7fe640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5710107420 0x7f571019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570f7fe640 1 -- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 msgr2=0x7f5710197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570f7fe640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 0x7f5710197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570f7fe640 1 -- 172.21.15.7:0/958617742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571019f650 con 0x7f57101021c0 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.996+0000 7f570effd640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 0x7f5710197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.997+0000 7f570f7fe640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57101021c0 0x7f5710197a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f570000e9f0 tx=0x7f570000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.997+0000 7f570cff9640 1 -- 172.21.15.7:0/958617742 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570000ccc0 con 0x7f57101021c0 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.997+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571019f940 con 0x7f57101021c0 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.997+0000 7f570cff9640 1 -- 172.21.15.7:0/958617742 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5700005020 con 0x7f57101021c0 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.997+0000 7f570cff9640 1 -- 172.21.15.7:0/958617742 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57000049e0 con 0x7f57101021c0 2024-03-26T08:41:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.997+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571019fe60 con 0x7f57101021c0 2024-03-26T08:41:36.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.998+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57100fd620 con 0x7f57101021c0 2024-03-26T08:41:36.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.999+0000 7f570cff9640 1 -- 172.21.15.7:0/958617742 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5700005560 con 0x7f57101021c0 2024-03-26T08:41:36.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.999+0000 7f570cff9640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f56f80a2800 0x7f56f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:36.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.999+0000 7f570effd640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f56f80a2800 0x7f56f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:36.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:36.999+0000 7f570cff9640 1 -- 172.21.15.7:0/958617742 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57000ce9b0 con 0x7f57101021c0 2024-03-26T08:41:36.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.000+0000 7f570effd640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f56f80a2800 0x7f56f80a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f570402c8e0 tx=0x7f5704005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:36.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.003+0000 7f570cff9640 1 -- 172.21.15.7:0/958617742 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f57000051d0 con 0x7f57101021c0 2024-03-26T08:41:37.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.186+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f571002d070 con 0x7f57101021c0 2024-03-26T08:41:37.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.187+0000 7f570cff9640 1 -- 172.21.15.7:0/958617742 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f570009ba50 con 0x7f57101021c0 2024-03-26T08:41:37.178 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-26T08:41:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f56f80a2800 msgr2=0x7f56f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f56f80a2800 0x7f56f80a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f570402c8e0 tx=0x7f5704005b50 comp rx=0 tx=0).stop 2024-03-26T08:41:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57101021c0 msgr2=0x7f5710197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57101021c0 0x7f5710197a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f570000e9f0 tx=0x7f570000eeb0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 shutdown_connections 2024-03-26T08:41:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f56f80a2800 0x7f56f80a4cc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5710107420 0x7f571019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5710104af0 0x7f5710197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 --2- 172.21.15.7:0/958617742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f57101021c0 0x7f5710197a30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 >> 172.21.15.7:0/958617742 conn(0x7f57100fbb30 msgr2=0x7f5710102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 shutdown_connections 2024-03-26T08:41:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.189+0000 7f5715dcd640 1 -- 172.21.15.7:0/958617742 wait complete. 2024-03-26T08:41:37.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:41:37.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 -- 172.21.15.7:0/2757149744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c1017d0 msgr2=0x7f576c101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 --2- 172.21.15.7:0/2757149744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c1017d0 0x7f576c101c50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5754009900 tx=0x7f575402bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 -- 172.21.15.7:0/2757149744 shutdown_connections 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 --2- 172.21.15.7:0/2757149744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f576c102190 0x7f576c108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 --2- 172.21.15.7:0/2757149744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c1017d0 0x7f576c101c50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 --2- 172.21.15.7:0/2757149744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f576c1005e0 0x7f576c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 -- 172.21.15.7:0/2757149744 >> 172.21.15.7:0/2757149744 conn(0x7f576c0fbb30 msgr2=0x7f576c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 -- 172.21.15.7:0/2757149744 shutdown_connections 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 -- 172.21.15.7:0/2757149744 wait complete. 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 Processor -- start 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.306+0000 7f5772348640 1 -- start start 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f5772348640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f576c1005e0 0x7f576c1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f5772348640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f576c1017d0 0x7f576c196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f5772348640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c102190 0x7f576c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f5772348640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c10c260 con 0x7f576c102190 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f5772348640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c10c0e0 con 0x7f576c1005e0 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f5772348640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c10c3e0 con 0x7f576c1017d0 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57708e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c102190 0x7f576c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57708e5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c102190 0x7f576c19de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38028/0 (socket says 172.21.15.7:38028) 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57708e5640 1 -- 172.21.15.7:0/2408277181 learned_addr learned my addr 172.21.15.7:0/2408277181 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f576bfff640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f576c1005e0 0x7f576c1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f576b7fe640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f576c1017d0 0x7f576c196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57708e5640 1 -- 172.21.15.7:0/2408277181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f576c1017d0 msgr2=0x7f576c196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57708e5640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f576c1017d0 0x7f576c196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57708e5640 1 -- 172.21.15.7:0/2408277181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f576c1005e0 msgr2=0x7f576c1968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57708e5640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f576c1005e0 0x7f576c1968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57708e5640 1 -- 172.21.15.7:0/2408277181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c19e4c0 con 0x7f576c102190 2024-03-26T08:41:37.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57708e5640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c102190 0x7f576c19de50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f576400eca0 tx=0x7f576400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:37.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.307+0000 7f57697fa640 1 -- 172.21.15.7:0/2408277181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764004b20 con 0x7f576c102190 2024-03-26T08:41:37.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.308+0000 7f57697fa640 1 -- 172.21.15.7:0/2408277181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5764004cc0 con 0x7f576c102190 2024-03-26T08:41:37.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.308+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c069920 con 0x7f576c102190 2024-03-26T08:41:37.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.308+0000 7f57697fa640 1 -- 172.21.15.7:0/2408277181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764017650 con 0x7f576c102190 2024-03-26T08:41:37.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.308+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c069e40 con 0x7f576c102190 2024-03-26T08:41:37.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.308+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c067c90 con 0x7f576c102190 2024-03-26T08:41:37.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.310+0000 7f57697fa640 1 -- 172.21.15.7:0/2408277181 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5764005020 con 0x7f576c102190 2024-03-26T08:41:37.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.310+0000 7f57697fa640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57580a2800 0x7f57580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.310+0000 7f576bfff640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57580a2800 0x7f57580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.310+0000 7f57697fa640 1 -- 172.21.15.7:0/2408277181 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57640d0cb0 con 0x7f576c102190 2024-03-26T08:41:37.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.311+0000 7f576bfff640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57580a2800 0x7f57580a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f576c101630 tx=0x7f575c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:37.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.314+0000 7f57697fa640 1 -- 172.21.15.7:0/2408277181 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f576409d6c0 con 0x7f576c102190 2024-03-26T08:41:37.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:37 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/157995362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:41:37.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:37 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/666999246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:41:37.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:37 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3567633035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:41:37.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:37 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/958617742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:41:37.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.537+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f576c107310 con 0x7f576c102190 2024-03-26T08:41:37.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.537+0000 7f57697fa640 1 -- 172.21.15.7:0/2408277181 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f57640a1760 con 0x7f576c102190 2024-03-26T08:41:37.529 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:37.529 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:40:15.277649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:41:37.529 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-03-26T08:41:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57580a2800 msgr2=0x7f57580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57580a2800 0x7f57580a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f576c101630 tx=0x7f575c005f90 comp rx=0 tx=0).stop 2024-03-26T08:41:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c102190 msgr2=0x7f576c19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c102190 0x7f576c19de50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f576400eca0 tx=0x7f576400b690 comp rx=0 tx=0).stop 2024-03-26T08:41:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 shutdown_connections 2024-03-26T08:41:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f57580a2800 0x7f57580a4cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576c102190 0x7f576c19de50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f576c1017d0 0x7f576c196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 --2- 172.21.15.7:0/2408277181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f576c1005e0 0x7f576c1968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.539+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 >> 172.21.15.7:0/2408277181 conn(0x7f576c0fbb30 msgr2=0x7f576c104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.540+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 shutdown_connections 2024-03-26T08:41:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.540+0000 7f5772348640 1 -- 172.21.15.7:0/2408277181 wait complete. 2024-03-26T08:41:37.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:37 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/157995362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:41:37.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:37 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/666999246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:41:37.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:37 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3567633035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:41:37.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:37 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/958617742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:41:37.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:41:37.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:37 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/157995362' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:41:37.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:37 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/666999246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:41:37.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:37 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3567633035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:41:37.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:37 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/958617742' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:41:37.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.653+0000 7fc70a377640 1 -- 172.21.15.7:0/1335338630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704107420 msgr2=0x7fc704109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.653+0000 7fc70a377640 1 --2- 172.21.15.7:0/1335338630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704107420 0x7fc704109870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009ca0 tx=0x7fc6fc02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 -- 172.21.15.7:0/1335338630 shutdown_connections 2024-03-26T08:41:37.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 --2- 172.21.15.7:0/1335338630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704107420 0x7fc704109870 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 --2- 172.21.15.7:0/1335338630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc704104af0 0x7fc704106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 --2- 172.21.15.7:0/1335338630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7041021c0 0x7fc7041045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 -- 172.21.15.7:0/1335338630 >> 172.21.15.7:0/1335338630 conn(0x7fc7040fbbb0 msgr2=0x7fc7040fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 -- 172.21.15.7:0/1335338630 shutdown_connections 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 -- 172.21.15.7:0/1335338630 wait complete. 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 Processor -- start 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 -- start start 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7041021c0 0x7fc704197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704104af0 0x7fc704197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc704107420 0x7fc70419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.654+0000 7fc70a377640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70410d350 con 0x7fc704104af0 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc70a377640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70410d1d0 con 0x7fc704107420 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc70a377640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70410d4d0 con 0x7fc7041021c0 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704104af0 0x7fc704197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc703fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7041021c0 0x7fc704197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704104af0 0x7fc704197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38046/0 (socket says 172.21.15.7:38046) 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7037fe640 1 -- 172.21.15.7:0/535611673 learned_addr learned my addr 172.21.15.7:0/535611673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc708914640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc704107420 0x7fc70419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7037fe640 1 -- 172.21.15.7:0/535611673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7041021c0 msgr2=0x7fc704197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7037fe640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7041021c0 0x7fc704197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7037fe640 1 -- 172.21.15.7:0/535611673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc704107420 msgr2=0x7fc70419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7037fe640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc704107420 0x7fc70419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7037fe640 1 -- 172.21.15.7:0/535611673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70419f560 con 0x7fc704104af0 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc703fff640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7041021c0 0x7fc704197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7037fe640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704104af0 0x7fc704197e80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9a0 tx=0x7fc6f400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7017fa640 1 -- 172.21.15.7:0/535611673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400cd80 con 0x7fc704104af0 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7017fa640 1 -- 172.21.15.7:0/535611673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f40035f0 con 0x7fc704104af0 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70419f850 con 0x7fc704104af0 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc7017fa640 1 -- 172.21.15.7:0/535611673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4010670 con 0x7fc704104af0 2024-03-26T08:41:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.655+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70419fdf0 con 0x7fc704104af0 2024-03-26T08:41:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.656+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7040fd6a0 con 0x7fc704104af0 2024-03-26T08:41:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.657+0000 7fc7017fa640 1 -- 172.21.15.7:0/535611673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc6f40037a0 con 0x7fc704104af0 2024-03-26T08:41:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.658+0000 7fc7017fa640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.658+0000 7fc703fff640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.658+0000 7fc7017fa640 1 -- 172.21.15.7:0/535611673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc6f40cec90 con 0x7fc704104af0 2024-03-26T08:41:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.658+0000 7fc703fff640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009fc0 tx=0x7fc6f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:37.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.662+0000 7fc7017fa640 1 -- 172.21.15.7:0/535611673 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6f409bcf0 con 0x7fc704104af0 2024-03-26T08:41:37.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.845+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7040622a0 con 0x7fc704104af0 2024-03-26T08:41:37.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.847+0000 7fc7017fa640 1 -- 172.21.15.7:0/535611673 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7fc6f409b5f0 con 0x7fc704104af0 2024-03-26T08:41:37.838 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:37.838 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:41:37.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6ec0a2880 msgr2=0x7fc6ec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009fc0 tx=0x7fc6f8005f90 comp rx=0 tx=0).stop 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704104af0 msgr2=0x7fc704197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704104af0 0x7fc704197e80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9a0 tx=0x7fc6f400ee60 comp rx=0 tx=0).stop 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 shutdown_connections 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc704107420 0x7fc70419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc704104af0 0x7fc704197e80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 --2- 172.21.15.7:0/535611673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7041021c0 0x7fc704197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 >> 172.21.15.7:0/535611673 conn(0x7fc7040fbbb0 msgr2=0x7fc704102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 shutdown_connections 2024-03-26T08:41:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.849+0000 7fc70a377640 1 -- 172.21.15.7:0/535611673 wait complete. 2024-03-26T08:41:37.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:41:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 -- 172.21.15.7:0/3617463702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff080104af0 msgr2=0x7ff080106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3617463702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff080104af0 0x7ff080106ee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff070009900 tx=0x7ff07002bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 -- 172.21.15.7:0/3617463702 shutdown_connections 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3617463702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080107420 0x7ff080109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3617463702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff080104af0 0x7ff080106ee0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3617463702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff0801021c0 0x7ff0801045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 -- 172.21.15.7:0/3617463702 >> 172.21.15.7:0/3617463702 conn(0x7ff0800fbbb0 msgr2=0x7ff0800fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 -- 172.21.15.7:0/3617463702 shutdown_connections 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 -- 172.21.15.7:0/3617463702 wait complete. 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 Processor -- start 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.965+0000 7ff0863f3640 1 -- start start 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff0863f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0801021c0 0x7ff080197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff0863f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080104af0 0x7ff080197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff0863f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080107420 0x7ff08019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff0863f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08010d360 con 0x7ff0801021c0 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff0863f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08010d1e0 con 0x7ff080107420 2024-03-26T08:41:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff0863f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08010d4e0 con 0x7ff080104af0 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0801021c0 0x7ff080197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0801021c0 0x7ff080197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38068/0 (socket says 172.21.15.7:38068) 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07ffff640 1 -- 172.21.15.7:0/3794365503 learned_addr learned my addr 172.21.15.7:0/3794365503 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07f7fe640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080104af0 0x7ff080197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff084990640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080107420 0x7ff08019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07ffff640 1 -- 172.21.15.7:0/3794365503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080104af0 msgr2=0x7ff080197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07ffff640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080104af0 0x7ff080197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07ffff640 1 -- 172.21.15.7:0/3794365503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080107420 msgr2=0x7ff08019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07ffff640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080107420 0x7ff08019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07ffff640 1 -- 172.21.15.7:0/3794365503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08019f630 con 0x7ff0801021c0 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07ffff640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0801021c0 0x7ff080197a10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e9f0 tx=0x7ff06800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07d7fa640 1 -- 172.21.15.7:0/3794365503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06800cd80 con 0x7ff0801021c0 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07d7fa640 1 -- 172.21.15.7:0/3794365503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff068005500 con 0x7ff0801021c0 2024-03-26T08:41:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08019f920 con 0x7ff0801021c0 2024-03-26T08:41:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff07d7fa640 1 -- 172.21.15.7:0/3794365503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068010690 con 0x7ff0801021c0 2024-03-26T08:41:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.966+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08019fdf0 con 0x7ff0801021c0 2024-03-26T08:41:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.968+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0800fd6a0 con 0x7ff0801021c0 2024-03-26T08:41:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.968+0000 7ff07d7fa640 1 -- 172.21.15.7:0/3794365503 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff0680046c0 con 0x7ff0801021c0 2024-03-26T08:41:37.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.969+0000 7ff07d7fa640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff06c0a2800 0x7ff06c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:37.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.969+0000 7ff07f7fe640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff06c0a2800 0x7ff06c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:37.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.969+0000 7ff07d7fa640 1 -- 172.21.15.7:0/3794365503 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff0680cee50 con 0x7ff0801021c0 2024-03-26T08:41:37.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.970+0000 7ff07f7fe640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff06c0a2800 0x7ff06c0a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff070000c00 tx=0x7ff070005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:37.973+0000 7ff07d7fa640 1 -- 172.21.15.7:0/3794365503 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff06809bef0 con 0x7ff0801021c0 2024-03-26T08:41:38.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.195+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff08002d070 con 0x7ff0801021c0 2024-03-26T08:41:38.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.196+0000 7ff07d7fa640 1 -- 172.21.15.7:0/3794365503 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11357 (secure 0 0 0) 0x7ff06809b7f0 con 0x7ff0801021c0 2024-03-26T08:41:38.187 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:38.188 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:40:15.277649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:41:38.188 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff06c0a2800 msgr2=0x7ff06c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff06c0a2800 0x7ff06c0a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff070000c00 tx=0x7ff070005fd0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0801021c0 msgr2=0x7ff080197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0801021c0 0x7ff080197a10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e9f0 tx=0x7ff06800edf0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 shutdown_connections 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff06c0a2800 0x7ff06c0a4cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff080107420 0x7ff08019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff080104af0 0x7ff080197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 --2- 172.21.15.7:0/3794365503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0801021c0 0x7ff080197a10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 >> 172.21.15.7:0/3794365503 conn(0x7ff0800fbbb0 msgr2=0x7ff080102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 shutdown_connections 2024-03-26T08:41:38.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.199+0000 7ff0863f3640 1 -- 172.21.15.7:0/3794365503 wait complete. 2024-03-26T08:41:38.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T08:40:09.583324+0000', 'modified': '2024-03-26T08:40:15.277649+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24502, 'mds_2': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/1837116746', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 1837116746}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1837116746}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6835/2525817262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 2525817262}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2525817262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/7790244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 7790244}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 7790244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.44:6837/1604095881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 1604095881}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 1604095881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.44:6839/2441928071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 2441928071}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 2441928071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6839/3011473533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3011473533}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3011473533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24478, 24502, 24512]} 2024-03-26T08:41:38.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-26T08:41:38.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-26T08:41:38.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.314+0000 7fe145ec6640 1 -- 172.21.15.7:0/4023801938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1401021e0 msgr2=0x7fe1401045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.314+0000 7fe145ec6640 1 --2- 172.21.15.7:0/4023801938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1401021e0 0x7fe1401045d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe130009ca0 tx=0x7fe13002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.314+0000 7fe145ec6640 1 -- 172.21.15.7:0/4023801938 shutdown_connections 2024-03-26T08:41:38.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.314+0000 7fe145ec6640 1 --2- 172.21.15.7:0/4023801938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe140107440 0x7fe140109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.314+0000 7fe145ec6640 1 --2- 172.21.15.7:0/4023801938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe140104b10 0x7fe140106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.314+0000 7fe145ec6640 1 --2- 172.21.15.7:0/4023801938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1401021e0 0x7fe1401045d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.314+0000 7fe145ec6640 1 -- 172.21.15.7:0/4023801938 >> 172.21.15.7:0/4023801938 conn(0x7fe1400fbb30 msgr2=0x7fe1400fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.314+0000 7fe145ec6640 1 -- 172.21.15.7:0/4023801938 shutdown_connections 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.315+0000 7fe145ec6640 1 -- 172.21.15.7:0/4023801938 wait complete. 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.315+0000 7fe145ec6640 1 Processor -- start 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.315+0000 7fe145ec6640 1 -- start start 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.315+0000 7fe145ec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1401021e0 0x7fe140197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.315+0000 7fe145ec6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe140104b10 0x7fe140197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.315+0000 7fe145ec6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe140107440 0x7fe14019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.315+0000 7fe145ec6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14010d330 con 0x7fe140107440 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.315+0000 7fe145ec6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14010d1b0 con 0x7fe1401021e0 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.315+0000 7fe145ec6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14010d4b0 con 0x7fe140104b10 2024-03-26T08:41:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe140107440 0x7fe14019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1401021e0 0x7fe140197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1401021e0 0x7fe140197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34338/0 (socket says 172.21.15.7:34338) 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13f7fe640 1 -- 172.21.15.7:0/1073377976 learned_addr learned my addr 172.21.15.7:0/1073377976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13effd640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe140104b10 0x7fe140197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13f7fe640 1 -- 172.21.15.7:0/1073377976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe140104b10 msgr2=0x7fe140197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe140104b10 0x7fe140197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13f7fe640 1 -- 172.21.15.7:0/1073377976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe140107440 msgr2=0x7fe14019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe140107440 0x7fe14019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13f7fe640 1 -- 172.21.15.7:0/1073377976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14019f650 con 0x7fe1401021e0 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1401021e0 0x7fe140197a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe130009dc0 tx=0x7fe130032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13cff9640 1 -- 172.21.15.7:0/1073377976 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130041020 con 0x7fe1401021e0 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe145ec6640 1 -- 172.21.15.7:0/1073377976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14019f8e0 con 0x7fe1401021e0 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13cff9640 1 -- 172.21.15.7:0/1073377976 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe130034cd0 con 0x7fe1401021e0 2024-03-26T08:41:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe13cff9640 1 -- 172.21.15.7:0/1073377976 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130039040 con 0x7fe1401021e0 2024-03-26T08:41:38.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.316+0000 7fe145ec6640 1 -- 172.21.15.7:0/1073377976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14019fdd0 con 0x7fe1401021e0 2024-03-26T08:41:38.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.318+0000 7fe13cff9640 1 -- 172.21.15.7:0/1073377976 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe130044020 con 0x7fe1401021e0 2024-03-26T08:41:38.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.318+0000 7fe148b93740 1 Processor -- start 2024-03-26T08:41:38.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe148b93740 1 -- start start 2024-03-26T08:41:38.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13cff9640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe12c0a2800 0x7fe12c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe148b93740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 0x563b79be7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe148b93740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79becc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe148b93740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 0x563b79bef730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe148b93740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79b4cc20 con 0x563b79bed2a0 2024-03-26T08:41:38.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe148b93740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79af5a40 con 0x563b79be8130 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe148b93740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79b48cc0 con 0x563b79be7740 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13ffff640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe12c0a2800 0x7fe12c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79becc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 0x563b79be7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79becc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34354/0 (socket says 172.21.15.7:34354) 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13effd640 1 -- 172.21.15.7:0/3566427452 learned_addr learned my addr 172.21.15.7:0/3566427452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13cff9640 1 -- 172.21.15.7:0/1073377976 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe1300f08b0 con 0x7fe1401021e0 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13ffff640 1 --2- 172.21.15.7:0/3566427452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 0x563b79bef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13effd640 1 -- 172.21.15.7:0/3566427452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 msgr2=0x563b79be7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13effd640 1 --2- 172.21.15.7:0/3566427452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 0x563b79be7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13effd640 1 -- 172.21.15.7:0/3566427452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 msgr2=0x563b79bef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13effd640 1 --2- 172.21.15.7:0/3566427452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 0x563b79bef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13effd640 1 -- 172.21.15.7:0/3566427452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b79b17f90 con 0x563b79be8130 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13effd640 1 --2- 172.21.15.7:0/3566427452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79becc30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe128009c60 tx=0x7fe128011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca82a5c364aa7d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.319+0000 7fe13ffff640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe12c0a2800 0x7fe12c0a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe138009fd0 tx=0x7fe138009b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe135ffb640 1 -- 172.21.15.7:0/3566427452 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128004030 con 0x563b79be8130 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe135ffb640 1 -- 172.21.15.7:0/3566427452 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1280041d0 con 0x563b79be8130 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe135ffb640 1 -- 172.21.15.7:0/3566427452 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128019760 con 0x563b79be8130 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe148b93740 1 -- 172.21.15.7:0/3566427452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 msgr2=0x563b79becc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe148b93740 1 --2- 172.21.15.7:0/3566427452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79becc30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe128009c60 tx=0x7fe128011810 comp rx=0 tx=0).stop 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe148b93740 1 -- 172.21.15.7:0/3566427452 shutdown_connections 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe148b93740 1 --2- 172.21.15.7:0/3566427452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 0x563b79bef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe148b93740 1 --2- 172.21.15.7:0/3566427452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79becc30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe148b93740 1 --2- 172.21.15.7:0/3566427452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 0x563b79be7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe148b93740 1 -- 172.21.15.7:0/3566427452 >> 172.21.15.7:0/3566427452 conn(0x563b79bafa60 msgr2=0x563b79be30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe148b93740 1 -- 172.21.15.7:0/3566427452 shutdown_connections 2024-03-26T08:41:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.320+0000 7fe148b93740 1 -- 172.21.15.7:0/3566427452 wait complete. 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe148b93740 1 -- start start 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe148b93740 1 Processor -- start 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe148b93740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 0x563b79bf1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe148b93740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be8130 0x563b79bf2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe148b93740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79bed2a0 0x563b79bf9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe148b93740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79bf0250 con 0x563b79be8130 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe148b93740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79b08e90 con 0x563b79bed2a0 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe148b93740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79bf03d0 con 0x563b79be7740 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe148b93740 10 client.0 _ll_register_callbacks cb 0x563b79860ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be8130 0x563b79bf2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79bed2a0 0x563b79bf9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be8130 0x563b79bf2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38116/0 (socket says 172.21.15.7:38116) 2024-03-26T08:41:38.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13effd640 1 -- 172.21.15.7:0/1475010820 learned_addr learned my addr 172.21.15.7:0/1475010820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 0x563b79bf1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13effd640 1 -- 172.21.15.7:0/1475010820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 msgr2=0x563b79bf1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13effd640 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 0x563b79bf1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13effd640 1 -- 172.21.15.7:0/1475010820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79bed2a0 msgr2=0x563b79bf9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13effd640 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79bed2a0 0x563b79bf9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13effd640 1 -- 172.21.15.7:0/1475010820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b79b159d0 con 0x563b79be8130 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 0x563b79bf1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.321+0000 7fe13ffff640 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79bed2a0 0x563b79bf9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe13effd640 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be8130 0x563b79bf2130 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe128009c30 tx=0x7fe128003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae72099747e20c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe11effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe11effd640 1 -- 172.21.15.7:0/1475010820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128028020 con 0x563b79be8130 2024-03-26T08:41:38.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe11effd640 1 -- 172.21.15.7:0/1475010820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe128004380 con 0x563b79be8130 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe148b93740 1 -- 172.21.15.7:0/1475010820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1280041d0 con 0x563b79be8130 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe11effd640 1 -- 172.21.15.7:0/1475010820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe12801f040 con 0x563b79be8130 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe148b93740 10 client.15096 fetch_fsmap learned FSMap version 22 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe148b93740 1 -- 172.21.15.7:0/1475010820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563b798612d0 con 0x563b79be8130 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe11effd640 1 -- 172.21.15.7:0/1475010820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1280041d0 con 0x563b79be8130 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe11effd640 1 -- 172.21.15.7:0/1475010820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe128018070 con 0x563b79be8130 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe148b93740 10 client.15096 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe148b93740 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe148b93740 20 client.15096 populate_metadata read hostname 'smithi007' 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe148b93740 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563b79bfe110 0x563b79bfd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe148b93740 4 client.15096 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.322+0000 7fe148b93740 1 -- 172.21.15.7:0/1475010820 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b79b5a160 con 0x563b79bfe110 2024-03-26T08:41:38.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.323+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563b79bfe110 0x563b79bfd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.323+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563b79bfe110 0x563b79bfd330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3eb685a307260f64 server_cookie=f53953dd38806434 in_seq=0 out_seq=0 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.323+0000 7fe11effd640 10 client.15096 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe11effd640 1 -- 172.21.15.7:0/1475010820 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563b79b5a160 con 0x563b79bfe110 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe11effd640 10 client.15096 handle_command_reply: tid=0 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 client.15096 shutdown 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 -- 172.21.15.7:0/1475010820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be8130 msgr2=0x563b79bf2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be8130 0x563b79bf2130 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe128009c30 tx=0x7fe128003cf0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 -- 172.21.15.7:0/1475010820 shutdown_connections 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563b79bfe110 0x563b79bfd330 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79bed2a0 0x563b79bf9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be8130 0x563b79bf2130 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 --2- 172.21.15.7:0/1475010820 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be7740 0x563b79bf1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe11effd640 0 client.15096 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:41:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 -- 172.21.15.7:0/1475010820 >> 172.21.15.7:0/1475010820 conn(0x563b79bafa60 msgr2=0x563b79be22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 -- 172.21.15.7:0/1475010820 shutdown_connections 2024-03-26T08:41:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 1 -- 172.21.15.7:0/1475010820 wait complete. 2024-03-26T08:41:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.324+0000 7fe148b93740 20 client.15096 trim_cache size 0 max 16384 2024-03-26T08:41:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe148b93740 1 Processor -- start 2024-03-26T08:41:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe148b93740 1 -- start start 2024-03-26T08:41:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe148b93740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be7740 0x563b79be32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe148b93740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be8130 0x563b79bf9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe148b93740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 0x563b79be62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe148b93740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79bfd930 con 0x563b79bed2a0 2024-03-26T08:41:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe148b93740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79bfdf90 con 0x563b79be7740 2024-03-26T08:41:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe148b93740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79bfe5f0 con 0x563b79be8130 2024-03-26T08:41:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 0x563b79be62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be7740 0x563b79be32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 0x563b79be62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38130/0 (socket says 172.21.15.7:38130) 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13ffff640 1 -- 172.21.15.7:0/2858745141 learned_addr learned my addr 172.21.15.7:0/2858745141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be7740 0x563b79be32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34374/0 (socket says 172.21.15.7:34374) 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13effd640 1 --2- 172.21.15.7:0/2858745141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be8130 0x563b79bf9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13f7fe640 1 -- 172.21.15.7:0/2858745141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be8130 msgr2=0x563b79bf9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/2858745141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be8130 0x563b79bf9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13f7fe640 1 -- 172.21.15.7:0/2858745141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 msgr2=0x563b79be62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/2858745141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 0x563b79be62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13f7fe640 1 -- 172.21.15.7:0/2858745141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b79bfd6a0 con 0x563b79be7740 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.341+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/2858745141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be7740 0x563b79be32e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe130000f80 tx=0x7fe13003a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d84baed7407fe45 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe1357fa640 1 -- 172.21.15.7:0/2858745141 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300c1170 con 0x563b79be7740 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe1357fa640 1 -- 172.21.15.7:0/2858745141 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1300bd950 con 0x563b79be7740 2024-03-26T08:41:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe1357fa640 1 -- 172.21.15.7:0/2858745141 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13004c260 con 0x563b79be7740 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 -- 172.21.15.7:0/2858745141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be7740 msgr2=0x563b79be32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 --2- 172.21.15.7:0/2858745141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be7740 0x563b79be32e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe130000f80 tx=0x7fe13003a890 comp rx=0 tx=0).stop 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 -- 172.21.15.7:0/2858745141 shutdown_connections 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 --2- 172.21.15.7:0/2858745141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79bed2a0 0x563b79be62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 --2- 172.21.15.7:0/2858745141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79be8130 0x563b79bf9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 --2- 172.21.15.7:0/2858745141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be7740 0x563b79be32e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 -- 172.21.15.7:0/2858745141 >> 172.21.15.7:0/2858745141 conn(0x563b79bafa60 msgr2=0x563b79be2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 -- 172.21.15.7:0/2858745141 shutdown_connections 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 -- 172.21.15.7:0/2858745141 wait complete. 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.342+0000 7fe148b93740 1 -- start start 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe148b93740 1 Processor -- start 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe148b93740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be7740 0x563b79bf4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe148b93740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79bf51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe148b93740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79bed2a0 0x563b79bf8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe148b93740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79bff2b0 con 0x563b79be7740 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe148b93740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79bfec50 con 0x563b79be8130 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe148b93740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b79bff910 con 0x563b79bed2a0 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe148b93740 10 client.0 _ll_register_callbacks cb 0x7fe138011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be7740 0x563b79bf4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be7740 0x563b79bf4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38134/0 (socket says 172.21.15.7:38134) 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13f7fe640 1 -- 172.21.15.7:0/1680915923 learned_addr learned my addr 172.21.15.7:0/1680915923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13effd640 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79bf51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13ffff640 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79bed2a0 0x563b79bf8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13f7fe640 1 -- 172.21.15.7:0/1680915923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79bed2a0 msgr2=0x563b79bf8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79bed2a0 0x563b79bf8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13f7fe640 1 -- 172.21.15.7:0/1680915923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 msgr2=0x563b79bf51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79bf51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13f7fe640 1 -- 172.21.15.7:0/1680915923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b7992b6b0 con 0x563b79be7740 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13ffff640 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79bed2a0 0x563b79bf8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe13f7fe640 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be7740 0x563b79bf4c70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe1300bd1e0 tx=0x7fe1300c1470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec7fd769c8545c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.343+0000 7fe11effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe11effd640 1 -- 172.21.15.7:0/1680915923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300dd070 con 0x563b79be7740 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe11effd640 1 -- 172.21.15.7:0/1680915923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe130003a70 con 0x563b79be7740 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe148b93740 1 -- 172.21.15.7:0/1680915923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1300bd950 con 0x563b79be7740 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe11effd640 1 -- 172.21.15.7:0/1680915923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1300dd210 con 0x563b79be7740 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe148b93740 10 client.15102 fetch_fsmap learned FSMap version 22 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe148b93740 1 -- 172.21.15.7:0/1680915923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563b79c593a0 con 0x563b79be7740 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe11effd640 1 -- 172.21.15.7:0/1680915923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe1300bd950 con 0x563b79be7740 2024-03-26T08:41:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe11effd640 1 -- 172.21.15.7:0/1680915923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe13003c030 con 0x563b79be7740 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe148b93740 10 client.15102 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe148b93740 10 client.15102 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe148b93740 20 client.15102 populate_metadata read hostname 'smithi007' 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe148b93740 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563b79c5d7e0 0x563b79c59570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe148b93740 4 client.15102 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe148b93740 1 -- 172.21.15.7:0/1680915923 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563b79b544b0 con 0x563b79c5d7e0 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.344+0000 7fe13effd640 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563b79c5d7e0 0x563b79c59570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe13effd640 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563b79c5d7e0 0x563b79c59570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9b69b8a00634f2a server_cookie=3284816fe88a717e in_seq=0 out_seq=0 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe11effd640 10 client.15102 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe11effd640 1 -- 172.21.15.7:0/1680915923 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563b79b544b0 con 0x563b79c5d7e0 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe11effd640 10 client.15102 handle_command_reply: tid=0 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 client.15102 shutdown 2024-03-26T08:41:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 -- 172.21.15.7:0/1680915923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be7740 msgr2=0x563b79bf4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be7740 0x563b79bf4c70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe1300bd1e0 tx=0x7fe1300c1470 comp rx=0 tx=0).stop 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 -- 172.21.15.7:0/1680915923 shutdown_connections 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563b79c5d7e0 0x563b79c59570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b79bed2a0 0x563b79bf8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b79be8130 0x563b79bf51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 --2- 172.21.15.7:0/1680915923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b79be7740 0x563b79bf4c70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe11effd640 0 client.15102 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 -- 172.21.15.7:0/1680915923 >> 172.21.15.7:0/1680915923 conn(0x563b79bafa60 msgr2=0x563b79be62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 -- 172.21.15.7:0/1680915923 shutdown_connections 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 1 -- 172.21.15.7:0/1680915923 wait complete. 2024-03-26T08:41:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.345+0000 7fe148b93740 20 client.15102 trim_cache size 0 max 16384 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24478, 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 84.073705892000007, 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T08:41:38.345 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 106.97244680599999 2024-03-26T08:41:38.346 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:41:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 -- 172.21.15.7:0/1073377976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe12c0a2800 msgr2=0x7fe12c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe12c0a2800 0x7fe12c0a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe138009fd0 tx=0x7fe138009b80 comp rx=0 tx=0).stop 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 -- 172.21.15.7:0/1073377976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1401021e0 msgr2=0x7fe140197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1401021e0 0x7fe140197a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe130009dc0 tx=0x7fe130032f20 comp rx=0 tx=0).stop 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 -- 172.21.15.7:0/1073377976 shutdown_connections 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe12c0a2800 0x7fe12c0a4cc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe140107440 0x7fe14019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe140104b10 0x7fe140197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 --2- 172.21.15.7:0/1073377976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1401021e0 0x7fe140197a30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 -- 172.21.15.7:0/1073377976 >> 172.21.15.7:0/1073377976 conn(0x7fe1400fbb30 msgr2=0x7fe140102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 -- 172.21.15.7:0/1073377976 shutdown_connections 2024-03-26T08:41:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.346+0000 7fe145ec6640 1 -- 172.21.15.7:0/1073377976 wait complete. 2024-03-26T08:41:38.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T08:41:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.460+0000 7f8240636640 1 -- 172.21.15.7:0/4154383309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82381005c0 msgr2=0x7f82381009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.460+0000 7f8240636640 1 --2- 172.21.15.7:0/4154383309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82381005c0 0x7f82381009e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8234009900 tx=0x7f823402bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 -- 172.21.15.7:0/4154383309 shutdown_connections 2024-03-26T08:41:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 --2- 172.21.15.7:0/4154383309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8238102170 0x7f8238108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 --2- 172.21.15.7:0/4154383309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82381017b0 0x7f8238101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 --2- 172.21.15.7:0/4154383309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82381005c0 0x7f82381009e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 -- 172.21.15.7:0/4154383309 >> 172.21.15.7:0/4154383309 conn(0x7f82380fbb30 msgr2=0x7f82380fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 -- 172.21.15.7:0/4154383309 shutdown_connections 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 -- 172.21.15.7:0/4154383309 wait complete. 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 Processor -- start 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 -- start start 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82381005c0 0x7f8238199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82381017b0 0x7f823819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238102170 0x7f82381a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823810c210 con 0x7f8238102170 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823810c090 con 0x7f82381005c0 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f8240636640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823810c390 con 0x7f82381017b0 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f823ebd3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238102170 0x7f82381a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f823ebd3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238102170 0x7f82381a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38160/0 (socket says 172.21.15.7:38160) 2024-03-26T08:41:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.461+0000 7f823ebd3640 1 -- 172.21.15.7:0/3021412299 learned_addr learned my addr 172.21.15.7:0/3021412299 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f823e3d2640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82381005c0 0x7f8238199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f823dbd1640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82381017b0 0x7f823819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f823ebd3640 1 -- 172.21.15.7:0/3021412299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82381017b0 msgr2=0x7f823819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f823ebd3640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82381017b0 0x7f823819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f823ebd3640 1 -- 172.21.15.7:0/3021412299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82381005c0 msgr2=0x7f8238199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f823ebd3640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82381005c0 0x7f8238199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f823ebd3640 1 -- 172.21.15.7:0/3021412299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82381a1830 con 0x7f8238102170 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f823ebd3640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238102170 0x7f82381a1190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f823000eb10 tx=0x7f823000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f822f7fe640 1 -- 172.21.15.7:0/3021412299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230004b20 con 0x7f8238102170 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f822f7fe640 1 -- 172.21.15.7:0/3021412299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8230004cc0 con 0x7f8238102170 2024-03-26T08:41:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f8240636640 1 -- 172.21.15.7:0/3021412299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82381a1b20 con 0x7f8238102170 2024-03-26T08:41:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f822f7fe640 1 -- 172.21.15.7:0/3021412299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230010760 con 0x7f8238102170 2024-03-26T08:41:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.462+0000 7f8240636640 1 -- 172.21.15.7:0/3021412299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82381a2040 con 0x7f8238102170 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.464+0000 7f822f7fe640 1 -- 172.21.15.7:0/3021412299 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8230005020 con 0x7f8238102170 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.464+0000 7f8243227740 1 Processor -- start 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f8243227740 1 -- start start 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f8243227740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daacf740 0x55c8daacfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f8243227740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 0x55c8daad4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f8243227740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f8243227740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daa34c20 con 0x55c8daad52a0 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f8243227740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8da9dda40 con 0x55c8daacf740 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f8243227740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daa30cc0 con 0x55c8daad0130 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823dbd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823dbd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38164/0 (socket says 172.21.15.7:38164) 2024-03-26T08:41:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823e3d2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 0x55c8daad4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823dbd1640 1 -- 172.21.15.7:0/2370384247 learned_addr learned my addr 172.21.15.7:0/2370384247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823dbd1640 1 --2- 172.21.15.7:0/2370384247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daacf740 0x55c8daacfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f822f7fe640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82240a2880 0x7f82240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823ebd3640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82240a2880 0x7f82240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823dbd1640 1 -- 172.21.15.7:0/2370384247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 msgr2=0x55c8daad4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823dbd1640 1 --2- 172.21.15.7:0/2370384247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 0x55c8daad4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823dbd1640 1 -- 172.21.15.7:0/2370384247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daacf740 msgr2=0x55c8daacfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823dbd1640 1 --2- 172.21.15.7:0/2370384247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daacf740 0x55c8daacfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823dbd1640 1 -- 172.21.15.7:0/2370384247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8da9fff90 con 0x55c8daad52a0 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f823e3d2640 1 --2- 172.21.15.7:0/2370384247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 0x55c8daad4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.465+0000 7f822f7fe640 1 -- 172.21.15.7:0/3021412299 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f82300cefd0 con 0x7f8238102170 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f823dbd1640 1 --2- 172.21.15.7:0/2370384247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad7730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55c8daad0990 tx=0x7f8228009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32de7bd88db04d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f822cbfe640 1 -- 172.21.15.7:0/2370384247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228004030 con 0x55c8daad52a0 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f822cbfe640 1 -- 172.21.15.7:0/2370384247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f822800fc70 con 0x55c8daad52a0 2024-03-26T08:41:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f823ebd3640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82240a2880 0x7f82240a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8230097d40 tx=0x7f823006a740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f822cbfe640 1 -- 172.21.15.7:0/2370384247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280039d0 con 0x55c8daad52a0 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f8243227740 1 -- 172.21.15.7:0/2370384247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 msgr2=0x55c8daad7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f8243227740 1 --2- 172.21.15.7:0/2370384247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad7730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55c8daad0990 tx=0x7f8228009370 comp rx=0 tx=0).stop 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f8243227740 1 -- 172.21.15.7:0/2370384247 shutdown_connections 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f8243227740 1 --2- 172.21.15.7:0/2370384247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad7730 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f8243227740 1 --2- 172.21.15.7:0/2370384247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 0x55c8daad4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f8243227740 1 --2- 172.21.15.7:0/2370384247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daacf740 0x55c8daacfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.466+0000 7f8243227740 1 -- 172.21.15.7:0/2370384247 >> 172.21.15.7:0/2370384247 conn(0x55c8daa97a60 msgr2=0x55c8daacb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 -- 172.21.15.7:0/2370384247 shutdown_connections 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 -- 172.21.15.7:0/2370384247 wait complete. 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 -- start start 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 Processor -- start 2024-03-26T08:41:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 0x55c8daad9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 0x55c8daada170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daae11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daad8250 con 0x55c8daad52a0 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8da9f0e90 con 0x55c8daad0130 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daad83d0 con 0x55c8daacf740 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f8243227740 10 client.0 _ll_register_callbacks cb 0x55c8da748ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823dbd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daae11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823dbd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daae11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38176/0 (socket says 172.21.15.7:38176) 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823dbd1640 1 -- 172.21.15.7:0/903604518 learned_addr learned my addr 172.21.15.7:0/903604518 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823dbd1640 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 0x55c8daad9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823e3d2640 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 0x55c8daada170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823dbd1640 1 -- 172.21.15.7:0/903604518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 msgr2=0x55c8daad9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823dbd1640 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 0x55c8daad9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823dbd1640 1 -- 172.21.15.7:0/903604518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 msgr2=0x55c8daada170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823dbd1640 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 0x55c8daada170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.467+0000 7f823dbd1640 1 -- 172.21.15.7:0/903604518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8da9fd9d0 con 0x55c8daad52a0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f823dbd1640 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daae11b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8228009d30 tx=0x7f8228009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e755d3684a01137d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8221ffb640 1 -- 172.21.15.7:0/903604518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228005020 con 0x55c8daad52a0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8221ffb640 1 -- 172.21.15.7:0/903604518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8228019040 con 0x55c8daad52a0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8243227740 1 -- 172.21.15.7:0/903604518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f822800fc70 con 0x55c8daad52a0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8221ffb640 1 -- 172.21.15.7:0/903604518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8228035050 con 0x55c8daad52a0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8243227740 10 client.15114 fetch_fsmap learned FSMap version 22 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8243227740 1 -- 172.21.15.7:0/903604518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c8da7492d0 con 0x55c8daad52a0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8221ffb640 1 -- 172.21.15.7:0/903604518 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f822800fc70 con 0x55c8daad52a0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8221ffb640 1 -- 172.21.15.7:0/903604518 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f822801f4b0 con 0x55c8daad52a0 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8243227740 10 client.15114 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8243227740 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8243227740 20 client.15114 populate_metadata read hostname 'smithi007' 2024-03-26T08:41:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8243227740 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c8daae61a0 0x55c8daae5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8243227740 4 client.15114 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.468+0000 7f8243227740 1 -- 172.21.15.7:0/903604518 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8daa42160 con 0x55c8daae61a0 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.469+0000 7f823e3d2640 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c8daae61a0 0x55c8daae5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.469+0000 7f823e3d2640 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c8daae61a0 0x55c8daae5310 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d6815c8233e0067 server_cookie=9546cb4be0810ca2 in_seq=0 out_seq=0 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.469+0000 7f8221ffb640 10 client.15114 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.469+0000 7f8221ffb640 1 -- 172.21.15.7:0/903604518 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c8daa42160 con 0x55c8daae61a0 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.469+0000 7f8221ffb640 10 client.15114 handle_command_reply: tid=0 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 client.15114 shutdown 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 -- 172.21.15.7:0/903604518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 msgr2=0x55c8daae11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daae11b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8228009d30 tx=0x7f8228009770 comp rx=0 tx=0).stop 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 -- 172.21.15.7:0/903604518 shutdown_connections 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daae11b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 0x55c8daada170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c8daae61a0 0x55c8daae5310 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 --2- 172.21.15.7:0/903604518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 0x55c8daad9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8221ffb640 0 client.15114 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:41:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 -- 172.21.15.7:0/903604518 >> 172.21.15.7:0/903604518 conn(0x55c8daa97a60 msgr2=0x55c8daaca2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 -- 172.21.15.7:0/903604518 shutdown_connections 2024-03-26T08:41:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 1 -- 172.21.15.7:0/903604518 wait complete. 2024-03-26T08:41:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.470+0000 7f8243227740 20 client.15114 trim_cache size 0 max 16384 2024-03-26T08:41:38.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.486+0000 7f8243227740 1 Processor -- start 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.486+0000 7f8243227740 1 -- start start 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f8243227740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 0x55c8daad9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f8243227740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 0x55c8daae10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f8243227740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f8243227740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daae59c0 con 0x55c8daad52a0 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f8243227740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daae6020 con 0x55c8daad0130 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f8243227740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daae6680 con 0x55c8daacf740 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38192/0 (socket says 172.21.15.7:38192) 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 -- 172.21.15.7:0/2587014245 learned_addr learned my addr 172.21.15.7:0/2587014245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823e3d2640 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 0x55c8daae10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 0x55c8daad9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 -- 172.21.15.7:0/2587014245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 msgr2=0x55c8daad9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 0x55c8daad9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 -- 172.21.15.7:0/2587014245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 msgr2=0x55c8daae10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 0x55c8daae10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 -- 172.21.15.7:0/2587014245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8da8136b0 con 0x55c8daad52a0 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823e3d2640 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 0x55c8daae10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f823dbd1640 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad9d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8228009b10 tx=0x7f8228011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645d1c8e28889a3d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f8223fff640 1 -- 172.21.15.7:0/2587014245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822801fbc0 con 0x55c8daad52a0 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f8223fff640 1 -- 172.21.15.7:0/2587014245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f822801f4b0 con 0x55c8daad52a0 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.487+0000 7f8223fff640 1 -- 172.21.15.7:0/2587014245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228003e60 con 0x55c8daad52a0 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 -- 172.21.15.7:0/2587014245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 msgr2=0x55c8daad9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad9d80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8228009b10 tx=0x7f8228011560 comp rx=0 tx=0).stop 2024-03-26T08:41:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 -- 172.21.15.7:0/2587014245 shutdown_connections 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daad52a0 0x55c8daad9d80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad0130 0x55c8daae10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 --2- 172.21.15.7:0/2587014245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daacf740 0x55c8daad9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 -- 172.21.15.7:0/2587014245 >> 172.21.15.7:0/2587014245 conn(0x55c8daa97a60 msgr2=0x55c8daad1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 -- 172.21.15.7:0/2587014245 shutdown_connections 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 -- 172.21.15.7:0/2587014245 wait complete. 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 -- start start 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 Processor -- start 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daacf740 0x55c8daae0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 0x55c8daadd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad52a0 0x55c8daaddc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daae7340 con 0x55c8daacf740 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daae6ce0 con 0x55c8daad52a0 2024-03-26T08:41:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8daae79a0 con 0x55c8daad0130 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f8243227740 10 client.0 _ll_register_callbacks cb 0x7f8228009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f823dbd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daacf740 0x55c8daae0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f823dbd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daacf740 0x55c8daae0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38206/0 (socket says 172.21.15.7:38206) 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f823dbd1640 1 -- 172.21.15.7:0/1728095303 learned_addr learned my addr 172.21.15.7:0/1728095303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.488+0000 7f823e3d2640 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 0x55c8daadd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f823dbd1640 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad52a0 0x55c8daaddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f823dbd1640 1 -- 172.21.15.7:0/1728095303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 msgr2=0x55c8daadd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f823dbd1640 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 0x55c8daadd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f823dbd1640 1 -- 172.21.15.7:0/1728095303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad52a0 msgr2=0x55c8daaddc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f823dbd1640 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad52a0 0x55c8daaddc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f823dbd1640 1 -- 172.21.15.7:0/1728095303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8daae1270 con 0x55c8daacf740 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f823dbd1640 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daacf740 0x55c8daae0c40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8228006010 tx=0x7f8228009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426e1c0a0c2e83a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8221ffb640 1 -- 172.21.15.7:0/1728095303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228041020 con 0x55c8daacf740 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8221ffb640 1 -- 172.21.15.7:0/1728095303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8228003de0 con 0x55c8daacf740 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8243227740 1 -- 172.21.15.7:0/1728095303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f822801f4b0 con 0x55c8daacf740 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8221ffb640 1 -- 172.21.15.7:0/1728095303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8228023070 con 0x55c8daacf740 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8243227740 10 client.15120 fetch_fsmap learned FSMap version 22 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8243227740 1 -- 172.21.15.7:0/1728095303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55c8dab420c0 con 0x55c8daacf740 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8221ffb640 1 -- 172.21.15.7:0/1728095303 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f822801f4b0 con 0x55c8daacf740 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8221ffb640 1 -- 172.21.15.7:0/1728095303 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8228048430 con 0x55c8daacf740 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8243227740 10 client.15120 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8243227740 10 client.15120 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8243227740 20 client.15120 populate_metadata read hostname 'smithi007' 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.489+0000 7f8243227740 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c8dab46350 0x55c8dab42290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 4 client.15120 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 -- 172.21.15.7:0/1728095303 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c8daa3c4b0 con 0x55c8dab46350 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f823e3d2640 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c8dab46350 0x55c8dab42290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8221ffb640 10 client.15120 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f823e3d2640 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c8dab46350 0x55c8dab42290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9e2be6de300b55f server_cookie=703b8da76ab56449 in_seq=0 out_seq=0 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8221ffb640 1 -- 172.21.15.7:0/1728095303 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c8daa3c4b0 con 0x55c8dab46350 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8221ffb640 10 client.15120 handle_command_reply: tid=0 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 client.15120 shutdown 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 -- 172.21.15.7:0/1728095303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daacf740 msgr2=0x55c8daae0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daacf740 0x55c8daae0c40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8228006010 tx=0x7f8228009de0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 -- 172.21.15.7:0/1728095303 shutdown_connections 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8daad52a0 0x55c8daaddc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c8dab46350 0x55c8dab42290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8daad0130 0x55c8daadd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8221ffb640 0 client.15120 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 --2- 172.21.15.7:0/1728095303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8daacf740 0x55c8daae0c40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 -- 172.21.15.7:0/1728095303 >> 172.21.15.7:0/1728095303 conn(0x55c8daa97a60 msgr2=0x55c8daad9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 -- 172.21.15.7:0/1728095303 shutdown_connections 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 1 -- 172.21.15.7:0/1728095303 wait complete. 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.490+0000 7f8243227740 20 client.15120 trim_cache size 0 max 16384 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24502, 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-26T08:41:38.490 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-26T08:41:38.491 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-26T08:41:38.491 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 85.227342725, 2024-03-26T08:41:38.491 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-03-26T08:41:38.491 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-26T08:41:38.491 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T08:41:38.491 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 101.16401723600001 2024-03-26T08:41:38.491 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 -- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82240a2880 msgr2=0x7f82240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82240a2880 0x7f82240a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8230097d40 tx=0x7f823006a740 comp rx=0 tx=0).stop 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 -- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238102170 msgr2=0x7f82381a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238102170 0x7f82381a1190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f823000eb10 tx=0x7f823000c2d0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 -- 172.21.15.7:0/3021412299 shutdown_connections 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82240a2880 0x7f82240a4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238102170 0x7f82381a1190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f82381017b0 0x7f823819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 --2- 172.21.15.7:0/3021412299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82381005c0 0x7f8238199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 -- 172.21.15.7:0/3021412299 >> 172.21.15.7:0/3021412299 conn(0x7f82380fbb30 msgr2=0x7f82380fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 -- 172.21.15.7:0/3021412299 shutdown_connections 2024-03-26T08:41:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.491+0000 7f8240636640 1 -- 172.21.15.7:0/3021412299 wait complete. 2024-03-26T08:41:38.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:38 smithi007 ceph-mon[32462]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T08:41:38.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:38 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2408277181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:41:38.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:38 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/535611673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:41:38.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T08:41:38.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:38 smithi016 ceph-mon[35851]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T08:41:38.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:38 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2408277181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:41:38.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:38 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/535611673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:38 smithi044 ceph-mon[36461]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T08:41:38.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:38 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2408277181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:41:38.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:38 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/535611673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:41:38.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.606+0000 7fea36922640 1 -- 172.21.15.7:0/1529061625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301005c0 msgr2=0x7fea301009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.606+0000 7fea36922640 1 --2- 172.21.15.7:0/1529061625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301005c0 0x7fea301009e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fea20009860 tx=0x7fea2002bb70 comp rx=0 tx=0).stop 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.606+0000 7fea36922640 1 -- 172.21.15.7:0/1529061625 shutdown_connections 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.606+0000 7fea36922640 1 --2- 172.21.15.7:0/1529061625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea30102170 0x7fea30108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.606+0000 7fea36922640 1 --2- 172.21.15.7:0/1529061625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea301017b0 0x7fea30101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.606+0000 7fea36922640 1 --2- 172.21.15.7:0/1529061625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301005c0 0x7fea301009e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.606+0000 7fea36922640 1 -- 172.21.15.7:0/1529061625 >> 172.21.15.7:0/1529061625 conn(0x7fea300fbb50 msgr2=0x7fea300fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.606+0000 7fea36922640 1 -- 172.21.15.7:0/1529061625 shutdown_connections 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.606+0000 7fea36922640 1 -- 172.21.15.7:0/1529061625 wait complete. 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.607+0000 7fea36922640 1 Processor -- start 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.607+0000 7fea36922640 1 -- start start 2024-03-26T08:41:38.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.607+0000 7fea36922640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea301005c0 0x7fea30104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.607+0000 7fea36922640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301017b0 0x7fea30102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.607+0000 7fea36922640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea30102170 0x7fea301030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.607+0000 7fea36922640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3010c210 con 0x7fea30102170 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.607+0000 7fea36922640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3010c090 con 0x7fea301017b0 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.607+0000 7fea36922640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3010c390 con 0x7fea301005c0 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301017b0 0x7fea30102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea34ebf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea30102170 0x7fea301030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea34ebf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea30102170 0x7fea301030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38228/0 (socket says 172.21.15.7:38228) 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301017b0 0x7fea30102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34474/0 (socket says 172.21.15.7:34474) 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea34ebf640 1 -- 172.21.15.7:0/2798118433 learned_addr learned my addr 172.21.15.7:0/2798118433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea2ffff640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea301005c0 0x7fea30104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea34ebf640 1 -- 172.21.15.7:0/2798118433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea301005c0 msgr2=0x7fea30104500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea34ebf640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea301005c0 0x7fea30104500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea34ebf640 1 -- 172.21.15.7:0/2798118433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301017b0 msgr2=0x7fea30102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea34ebf640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301017b0 0x7fea30102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea34ebf640 1 -- 172.21.15.7:0/2798118433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea30103990 con 0x7fea30102170 2024-03-26T08:41:38.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301017b0 0x7fea30102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:38.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea34ebf640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea30102170 0x7fea301030d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fea2800efc0 tx=0x7fea2800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea2d7fa640 1 -- 172.21.15.7:0/2798118433 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28003970 con 0x7fea30102170 2024-03-26T08:41:38.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea2d7fa640 1 -- 172.21.15.7:0/2798118433 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea28004020 con 0x7fea30102170 2024-03-26T08:41:38.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea36922640 1 -- 172.21.15.7:0/2798118433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea301a1bb0 con 0x7fea30102170 2024-03-26T08:41:38.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea2d7fa640 1 -- 172.21.15.7:0/2798118433 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2800b9f0 con 0x7fea30102170 2024-03-26T08:41:38.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.608+0000 7fea36922640 1 -- 172.21.15.7:0/2798118433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea301a2080 con 0x7fea30102170 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea2d7fa640 1 -- 172.21.15.7:0/2798118433 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fea2801d020 con 0x7fea30102170 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea395ee740 1 Processor -- start 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea395ee740 1 -- start start 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea395ee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea395ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d73c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea395ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d76730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea395ee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050cd3c20 con 0x55b050d6e740 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea395ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050c7ca40 con 0x55b050d742a0 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea395ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050ccfcc0 con 0x55b050d6f130 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38232/0 (socket says 172.21.15.7:38232) 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea2f7fe640 1 -- 172.21.15.7:0/3751025587 learned_addr learned my addr 172.21.15.7:0/3751025587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea2ffff640 1 --2- 172.21.15.7:0/3751025587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d73c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea2d7fa640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea180a2800 0x7fea180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.611+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/3751025587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d76730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea2f7fe640 1 -- 172.21.15.7:0/3751025587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 msgr2=0x55b050d73c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/3751025587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d73c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea2f7fe640 1 -- 172.21.15.7:0/3751025587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 msgr2=0x55b050d76730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/3751025587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d76730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea2f7fe640 1 -- 172.21.15.7:0/3751025587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b050c9ef90 con 0x55b050d6e740 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea34ebf640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea180a2800 0x7fea180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea2d7fa640 1 -- 172.21.15.7:0/2798118433 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea280ce8b0 con 0x7fea30102170 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/3751025587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6eb60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fea24013a20 tx=0x7fea24009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da16c37ab09fe2c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea167fc640 1 -- 172.21.15.7:0/3751025587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24004030 con 0x55b050d6e740 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea167fc640 1 -- 172.21.15.7:0/3751025587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea2400fe60 con 0x55b050d6e740 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.612+0000 7fea167fc640 1 -- 172.21.15.7:0/3751025587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24003b00 con 0x55b050d6e740 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 -- 172.21.15.7:0/3751025587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 msgr2=0x55b050d6eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea34ebf640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea180a2800 0x7fea180a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fea280976b0 tx=0x7fea28066010 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 --2- 172.21.15.7:0/3751025587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6eb60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fea24013a20 tx=0x7fea24009c00 comp rx=0 tx=0).stop 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 -- 172.21.15.7:0/3751025587 shutdown_connections 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 --2- 172.21.15.7:0/3751025587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d76730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 --2- 172.21.15.7:0/3751025587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d73c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 --2- 172.21.15.7:0/3751025587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6eb60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 -- 172.21.15.7:0/3751025587 >> 172.21.15.7:0/3751025587 conn(0x55b050d36a60 msgr2=0x55b050d6a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 -- 172.21.15.7:0/3751025587 shutdown_connections 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 -- 172.21.15.7:0/3751025587 wait complete. 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 -- start start 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 Processor -- start 2024-03-26T08:41:38.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d78b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d790d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d80110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050d771f0 con 0x55b050d6e740 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050c8fe90 con 0x55b050d742a0 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050d77370 con 0x55b050d6f130 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.613+0000 7fea395ee740 10 client.0 _ll_register_callbacks cb 0x55b0509e7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d78b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d78b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38244/0 (socket says 172.21.15.7:38244) 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 -- 172.21.15.7:0/4209352868 learned_addr learned my addr 172.21.15.7:0/4209352868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2ffff640 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d80110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 -- 172.21.15.7:0/4209352868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 msgr2=0x55b050d790d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d790d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 -- 172.21.15.7:0/4209352868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 msgr2=0x55b050d80110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d80110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 -- 172.21.15.7:0/4209352868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b050c9c9d0 con 0x55b050d6e740 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d78b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fea24014fd0 tx=0x7fea24005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89046494dabf6883 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea0f7fe640 1 -- 172.21.15.7:0/4209352868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24023750 con 0x55b050d6e740 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea395ee740 1 -- 172.21.15.7:0/4209352868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea2400fe60 con 0x55b050d6e740 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea0f7fe640 1 -- 172.21.15.7:0/4209352868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea240238f0 con 0x55b050d6e740 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea0f7fe640 1 -- 172.21.15.7:0/4209352868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea24036050 con 0x55b050d6e740 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea395ee740 10 client.15132 fetch_fsmap learned FSMap version 22 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea395ee740 1 -- 172.21.15.7:0/4209352868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b0509e82d0 con 0x55b050d6e740 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.614+0000 7fea0f7fe640 1 -- 172.21.15.7:0/4209352868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea2400fe60 con 0x55b050d6e740 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea0f7fe640 1 -- 172.21.15.7:0/4209352868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fea2401e070 con 0x55b050d6e740 2024-03-26T08:41:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea395ee740 10 client.15132 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea395ee740 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea395ee740 20 client.15132 populate_metadata read hostname 'smithi007' 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea395ee740 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b050d851a0 0x55b050d84270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea395ee740 4 client.15132 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea395ee740 1 -- 172.21.15.7:0/4209352868 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b050ce1160 con 0x55b050d851a0 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea2ffff640 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b050d851a0 0x55b050d84270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea0f7fe640 10 client.15132 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.615+0000 7fea2ffff640 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b050d851a0 0x55b050d84270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbcb0d09b082cc0 server_cookie=5e914f57b19c5f0a in_seq=0 out_seq=0 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea0f7fe640 1 -- 172.21.15.7:0/4209352868 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b050ce1160 con 0x55b050d851a0 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea0f7fe640 10 client.15132 handle_command_reply: tid=0 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 client.15132 shutdown 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 -- 172.21.15.7:0/4209352868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 msgr2=0x55b050d78b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d78b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fea24014fd0 tx=0x7fea24005d30 comp rx=0 tx=0).stop 2024-03-26T08:41:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 -- 172.21.15.7:0/4209352868 shutdown_connections 2024-03-26T08:41:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b050d851a0 0x55b050d84270 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d80110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d790d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea0f7fe640 0 client.15132 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:41:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 --2- 172.21.15.7:0/4209352868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d78b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 -- 172.21.15.7:0/4209352868 >> 172.21.15.7:0/4209352868 conn(0x55b050d36a60 msgr2=0x55b050d692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 -- 172.21.15.7:0/4209352868 shutdown_connections 2024-03-26T08:41:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 1 -- 172.21.15.7:0/4209352868 wait complete. 2024-03-26T08:41:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.616+0000 7fea395ee740 20 client.15132 trim_cache size 0 max 16384 2024-03-26T08:41:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea395ee740 1 Processor -- start 2024-03-26T08:41:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea395ee740 1 -- start start 2024-03-26T08:41:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea395ee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea395ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d800d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea395ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d6d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea395ee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050d849c0 con 0x55b050d6e740 2024-03-26T08:41:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea395ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050d85020 con 0x55b050d742a0 2024-03-26T08:41:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea395ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050d85680 con 0x55b050d6f130 2024-03-26T08:41:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38256/0 (socket says 172.21.15.7:38256) 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 -- 172.21.15.7:0/3797803174 learned_addr learned my addr 172.21.15.7:0/3797803174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2ffff640 1 --2- 172.21.15.7:0/3797803174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d800d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/3797803174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d6d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 -- 172.21.15.7:0/3797803174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 msgr2=0x55b050d800d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/3797803174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d800d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 -- 172.21.15.7:0/3797803174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 msgr2=0x55b050d6d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/3797803174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d6d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 -- 172.21.15.7:0/3797803174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b050ab26b0 con 0x55b050d6e740 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.633+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/3797803174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6d2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fea24006010 tx=0x7fea24013850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a85322a7356382 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea15ffb640 1 -- 172.21.15.7:0/3797803174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24005750 con 0x55b050d6e740 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea15ffb640 1 -- 172.21.15.7:0/3797803174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea2400fe60 con 0x55b050d6e740 2024-03-26T08:41:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea15ffb640 1 -- 172.21.15.7:0/3797803174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2401f690 con 0x55b050d6e740 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea395ee740 1 -- 172.21.15.7:0/3797803174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 msgr2=0x55b050d6d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea395ee740 1 --2- 172.21.15.7:0/3797803174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6d2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fea24006010 tx=0x7fea24013850 comp rx=0 tx=0).stop 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea395ee740 1 -- 172.21.15.7:0/3797803174 shutdown_connections 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea395ee740 1 --2- 172.21.15.7:0/3797803174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d742a0 0x55b050d6d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea395ee740 1 --2- 172.21.15.7:0/3797803174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d6f130 0x55b050d800d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea395ee740 1 --2- 172.21.15.7:0/3797803174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d6d2f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea395ee740 1 -- 172.21.15.7:0/3797803174 >> 172.21.15.7:0/3797803174 conn(0x55b050d36a60 msgr2=0x55b050d69a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea395ee740 1 -- 172.21.15.7:0/3797803174 shutdown_connections 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.634+0000 7fea395ee740 1 -- 172.21.15.7:0/3797803174 wait complete. 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea395ee740 1 -- start start 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea395ee740 1 Processor -- start 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea395ee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d7f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea395ee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d6f130 0x55b050d7fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea395ee740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d742a0 0x55b050d7c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea395ee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050d86340 con 0x55b050d6e740 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea395ee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050d85ce0 con 0x55b050d6f130 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea395ee740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b050d869a0 con 0x55b050d742a0 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea395ee740 10 client.0 _ll_register_callbacks cb 0x7fea28009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d7f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d7f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38264/0 (socket says 172.21.15.7:38264) 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 -- 172.21.15.7:0/885577587 learned_addr learned my addr 172.21.15.7:0/885577587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2ffff640 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d6f130 0x55b050d7fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d742a0 0x55b050d7c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 -- 172.21.15.7:0/885577587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d742a0 msgr2=0x55b050d7c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d742a0 0x55b050d7c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 -- 172.21.15.7:0/885577587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d6f130 msgr2=0x55b050d7fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:41:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d6f130 0x55b050d7fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 -- 172.21.15.7:0/885577587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b050d7c940 con 0x55b050d6e740 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea2f7fe640 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d7f860 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fea240092e0 tx=0x7fea240096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b740e8ed33dd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.635+0000 7fea0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea0f7fe640 1 -- 172.21.15.7:0/885577587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24005110 con 0x55b050d6e740 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea0f7fe640 1 -- 172.21.15.7:0/885577587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea240052b0 con 0x55b050d6e740 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea395ee740 1 -- 172.21.15.7:0/885577587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea2400fe60 con 0x55b050d6e740 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea0f7fe640 1 -- 172.21.15.7:0/885577587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea2401f430 con 0x55b050d6e740 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea395ee740 10 client.15138 fetch_fsmap learned FSMap version 22 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea395ee740 1 -- 172.21.15.7:0/885577587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55b050de0d40 con 0x55b050d6e740 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea0f7fe640 1 -- 172.21.15.7:0/885577587 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea2400fe60 con 0x55b050d6e740 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea0f7fe640 1 -- 172.21.15.7:0/885577587 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fea24042070 con 0x55b050d6e740 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea395ee740 10 client.15138 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea395ee740 10 client.15138 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:41:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea395ee740 20 client.15138 populate_metadata read hostname 'smithi007' 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea395ee740 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b050de4ee0 0x55b050de0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea395ee740 4 client.15138 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea395ee740 1 -- 172.21.15.7:0/885577587 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b050cdb4b0 con 0x55b050de4ee0 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.636+0000 7fea2ffff640 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b050de4ee0 0x55b050de0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea2ffff640 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b050de4ee0 0x55b050de0f10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31eb2a9444c502ee server_cookie=d1d3cad788085083 in_seq=0 out_seq=0 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea0f7fe640 10 client.15138 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea0f7fe640 1 -- 172.21.15.7:0/885577587 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55b050cdb4b0 con 0x55b050de4ee0 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea0f7fe640 10 client.15138 handle_command_reply: tid=0 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 client.15138 shutdown 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 -- 172.21.15.7:0/885577587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 msgr2=0x55b050d7f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d7f860 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fea240092e0 tx=0x7fea240096c0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 -- 172.21.15.7:0/885577587 shutdown_connections 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b050de4ee0 0x55b050de0f10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b050d742a0 0x55b050d7c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea0f7fe640 0 client.15138 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b050d6f130 0x55b050d7fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 --2- 172.21.15.7:0/885577587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b050d6e740 0x55b050d7f860 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 -- 172.21.15.7:0/885577587 >> 172.21.15.7:0/885577587 conn(0x55b050d36a60 msgr2=0x55b050d787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 -- 172.21.15.7:0/885577587 shutdown_connections 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 1 -- 172.21.15.7:0/885577587 wait complete. 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.637+0000 7fea395ee740 20 client.15138 trim_cache size 0 max 16384 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:41:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24512, 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 88.390873748000004, 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 92.581030489 2024-03-26T08:41:38.637 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:41:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 -- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea180a2800 msgr2=0x7fea180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea180a2800 0x7fea180a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fea280976b0 tx=0x7fea28066010 comp rx=0 tx=0).stop 2024-03-26T08:41:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 -- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea30102170 msgr2=0x7fea301030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea30102170 0x7fea301030d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fea2800efc0 tx=0x7fea2800c800 comp rx=0 tx=0).stop 2024-03-26T08:41:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 -- 172.21.15.7:0/2798118433 shutdown_connections 2024-03-26T08:41:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea180a2800 0x7fea180a4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea30102170 0x7fea301030d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea301017b0 0x7fea30102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 --2- 172.21.15.7:0/2798118433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea301005c0 0x7fea30104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 -- 172.21.15.7:0/2798118433 >> 172.21.15.7:0/2798118433 conn(0x7fea300fbb50 msgr2=0x7fea30069d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 -- 172.21.15.7:0/2798118433 shutdown_connections 2024-03-26T08:41:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:38.638+0000 7fea36922640 1 -- 172.21.15.7:0/2798118433 wait complete. 2024-03-26T08:41:38.640 INFO:tasks.cephadm:Setup complete, yielding 2024-03-26T08:41:38.640 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T08:41:38.651 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-26T08:41:38.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-26T08:41:38.892 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:39.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:39 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3794365503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:41:39.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:39 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3794365503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:41:39.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:39 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3794365503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:41:40.269 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:40 smithi007 ceph-mon[32462]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T08:41:40.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:40 smithi016 ceph-mon[35851]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T08:41:40.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:40 smithi044 ceph-mon[36461]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 -- 172.21.15.7:0/1963985847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf8102c60 msgr2=0x7f9bf8109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 --2- 172.21.15.7:0/1963985847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf8102c60 0x7f9bf8109510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00099c0 tx=0x7f9bf0009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 -- 172.21.15.7:0/1963985847 shutdown_connections 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 --2- 172.21.15.7:0/1963985847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf81010c0 0x7f9bf81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 --2- 172.21.15.7:0/1963985847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf8102c60 0x7f9bf8109510 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 --2- 172.21.15.7:0/1963985847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf81022c0 0x7f9bf8102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 -- 172.21.15.7:0/1963985847 >> 172.21.15.7:0/1963985847 conn(0x7f9bf80fc660 msgr2=0x7f9bf80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 -- 172.21.15.7:0/1963985847 shutdown_connections 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 -- 172.21.15.7:0/1963985847 wait complete. 2024-03-26T08:41:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.680+0000 7f9bffea7700 1 Processor -- start 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bffea7700 1 -- start start 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bffea7700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf81010c0 0x7f9bf806a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bffea7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf81022c0 0x7f9bf819d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bffea7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8102c60 0x7f9bf819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bffea7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf810cd00 con 0x7f9bf81022c0 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bffea7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf810cb80 con 0x7f9bf8102c60 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bffea7700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf810ce80 con 0x7f9bf81010c0 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfd442700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf81022c0 0x7f9bf819d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfd442700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf81022c0 0x7f9bf819d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38284/0 (socket says 172.21.15.7:38284) 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfd442700 1 -- 172.21.15.7:0/804963636 learned_addr learned my addr 172.21.15.7:0/804963636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfdc43700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf81010c0 0x7f9bf806a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfe444700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8102c60 0x7f9bf819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfdc43700 1 -- 172.21.15.7:0/804963636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8102c60 msgr2=0x7f9bf819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfdc43700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8102c60 0x7f9bf819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfdc43700 1 -- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf81022c0 msgr2=0x7f9bf819d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfdc43700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf81022c0 0x7f9bf819d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfdc43700 1 -- 172.21.15.7:0/804963636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf81a0050 con 0x7f9bf81010c0 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfd442700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf81022c0 0x7f9bf819d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfe444700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8102c60 0x7f9bf819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bfdc43700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf81010c0 0x7f9bf806a9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9be800ef90 tx=0x7f9be800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.681+0000 7f9bf6ffd700 1 -- 172.21.15.7:0/804963636 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8003a00 con 0x7f9bf81010c0 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.682+0000 7f9bf6ffd700 1 -- 172.21.15.7:0/804963636 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9be8003ba0 con 0x7f9bf81010c0 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.682+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf81a0340 con 0x7f9bf81010c0 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.682+0000 7f9bf6ffd700 1 -- 172.21.15.7:0/804963636 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be80048a0 con 0x7f9bf81010c0 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.682+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf81a0830 con 0x7f9bf81010c0 2024-03-26T08:41:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.682+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf8068e50 con 0x7f9bf81010c0 2024-03-26T08:41:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.684+0000 7f9bf6ffd700 1 -- 172.21.15.7:0/804963636 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9be8012050 con 0x7f9bf81010c0 2024-03-26T08:41:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.684+0000 7f9bf6ffd700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9be00a2890 0x7f9be00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.684+0000 7f9bfd442700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9be00a2890 0x7f9be00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.684+0000 7f9bf6ffd700 1 -- 172.21.15.7:0/804963636 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9be80ce8e0 con 0x7f9bf81010c0 2024-03-26T08:41:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.685+0000 7f9bfd442700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9be00a2890 0x7f9be00a4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806af20 tx=0x7f9bec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:40.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.691+0000 7f9bf6ffd700 1 -- 172.21.15.7:0/804963636 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9be8010040 con 0x7f9bf81010c0 2024-03-26T08:41:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.914+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bf8062690 con 0x7f9be00a2890 2024-03-26T08:41:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.915+0000 7f9bf6ffd700 1 -- 172.21.15.7:0/804963636 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9bf8062690 con 0x7f9be00a2890 2024-03-26T08:41:40.907 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-03-26T08:41:40.907 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-03-26T08:41:40.907 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-03-26T08:41:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9be00a2890 msgr2=0x7f9be00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9be00a2890 0x7f9be00a4d50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9bf806af20 tx=0x7f9bec009260 comp rx=0 tx=0).stop 2024-03-26T08:41:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf81010c0 msgr2=0x7f9bf806a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf81010c0 0x7f9bf806a9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9be800ef90 tx=0x7f9be800c3a0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 shutdown_connections 2024-03-26T08:41:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9be00a2890 0x7f9be00a4d50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bf81010c0 0x7f9bf806a9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bf8102c60 0x7f9bf819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 --2- 172.21.15.7:0/804963636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bf81022c0 0x7f9bf819d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:40.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 >> 172.21.15.7:0/804963636 conn(0x7f9bf80fc660 msgr2=0x7f9bf81055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:40.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 shutdown_connections 2024-03-26T08:41:40.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:40.918+0000 7f9bffea7700 1 -- 172.21.15.7:0/804963636 wait complete. 2024-03-26T08:41:41.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-26T08:41:41.966 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:42.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:42 smithi007 ceph-mon[32462]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:41:42.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:42 smithi007 ceph-mon[32462]: from='client.24970 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:42.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:42 smithi016 ceph-mon[35851]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:41:42.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:42 smithi016 ceph-mon[35851]: from='client.24970 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:42.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:42 smithi044 ceph-mon[36461]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T08:41:42.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:42 smithi044 ceph-mon[36461]: from='client.24970 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 -- 172.21.15.7:0/2957629272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c102360 msgr2=0x7f715c1027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 --2- 172.21.15.7:0/2957629272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c102360 0x7f715c1027c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f714c0099c0 tx=0x7f714c009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 -- 172.21.15.7:0/2957629272 shutdown_connections 2024-03-26T08:41:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 --2- 172.21.15.7:0/2957629272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f715c102d00 0x7f715c1095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 --2- 172.21.15.7:0/2957629272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c101160 0x7f715c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 --2- 172.21.15.7:0/2957629272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c102360 0x7f715c1027c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 -- 172.21.15.7:0/2957629272 >> 172.21.15.7:0/2957629272 conn(0x7f715c0fc720 msgr2=0x7f715c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 -- 172.21.15.7:0/2957629272 shutdown_connections 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 -- 172.21.15.7:0/2957629272 wait complete. 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 Processor -- start 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.714+0000 7f7163020700 1 -- start start 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7163020700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c101160 0x7f715c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7163020700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c102360 0x7f715c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7163020700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f715c102d00 0x7f715c1a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7163020700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c10ccb0 con 0x7f715c101160 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7163020700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c10cb30 con 0x7f715c102360 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7163020700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c10ce30 con 0x7f715c102d00 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7160dbc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c101160 0x7f715c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7160dbc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c101160 0x7f715c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38294/0 (socket says 172.21.15.7:38294) 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7160dbc700 1 -- 172.21.15.7:0/1532840139 learned_addr learned my addr 172.21.15.7:0/1532840139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f715bfff700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c102360 0x7f715c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f71615bd700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f715c102d00 0x7f715c1a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7160dbc700 1 -- 172.21.15.7:0/1532840139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f715c102d00 msgr2=0x7f715c1a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7160dbc700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f715c102d00 0x7f715c1a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7160dbc700 1 -- 172.21.15.7:0/1532840139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c102360 msgr2=0x7f715c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7160dbc700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c102360 0x7f715c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7160dbc700 1 -- 172.21.15.7:0/1532840139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c1a2210 con 0x7f715c101160 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f715bfff700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c102360 0x7f715c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7160dbc700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c101160 0x7f715c19a5b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f715400c170 tx=0x7f715400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7159ffb700 1 -- 172.21.15.7:0/1532840139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400cd00 con 0x7f715c101160 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c1a24a0 con 0x7f715c101160 2024-03-26T08:41:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7159ffb700 1 -- 172.21.15.7:0/1532840139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7154018070 con 0x7f715c101160 2024-03-26T08:41:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.715+0000 7f7159ffb700 1 -- 172.21.15.7:0/1532840139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154013660 con 0x7f715c101160 2024-03-26T08:41:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.716+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c1a2a30 con 0x7f715c101160 2024-03-26T08:41:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.717+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c068e50 con 0x7f715c101160 2024-03-26T08:41:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.718+0000 7f7159ffb700 1 -- 172.21.15.7:0/1532840139 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7154012070 con 0x7f715c101160 2024-03-26T08:41:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.718+0000 7f7159ffb700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71480a2890 0x7f71480a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.718+0000 7f715bfff700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71480a2890 0x7f71480a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.718+0000 7f7159ffb700 1 -- 172.21.15.7:0/1532840139 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f71540ce670 con 0x7f715c101160 2024-03-26T08:41:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.719+0000 7f715bfff700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71480a2890 0x7f71480a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f714c000c00 tx=0x7f714c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.723+0000 7f7159ffb700 1 -- 172.21.15.7:0/1532840139 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7154022020 con 0x7f715c101160 2024-03-26T08:41:43.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.937+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f715c0fe920 con 0x7f71480a2890 2024-03-26T08:41:43.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.951+0000 7f7159ffb700 1 -- 172.21.15.7:0/1532840139 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f715c0fe920 con 0x7f71480a2890 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.a smithi007 running (101s) 16s ago 100s 13.8M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 436cb35c9619 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.b smithi016 running (98s) 31s ago 97s 25.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 19cd65b0137d 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.c smithi044 running (112s) 31s ago 112s 19.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 4eede49de166 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.d smithi007 running (2m) 16s ago 2m 16.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 eb1102f0da13 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.e smithi016 running (118s) 31s ago 118s 16.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3dc18f6b035a 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.f smithi044 running (115s) 31s ago 115s 18.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 8c77619fc916 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.g smithi007 running (104s) 16s ago 103s 13.7M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 6a3f25f2be0a 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.h smithi016 running (110s) 31s ago 109s 20.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9a5e1bdb2be0 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.i smithi044 running (107s) 31s ago 106s 17.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 6be81398e8b1 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.j smithi007 running (2m) 16s ago 2m 16.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 91d9c52035e8 2024-03-26T08:41:43.943 INFO:teuthology.orchestra.run.smithi007.stdout:mds.k smithi016 running (2m) 31s ago 2m 13.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 fcec08531530 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:mds.l smithi044 running (2m) 31s ago 2m 15.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 3cc654354be3 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.x smithi007 *:9283,8765 running (9m) 16s ago 9m 515M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 4a674aaa0482 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.y smithi016 *:8443,8765 running (7m) 31s ago 7m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 f11437db8ff5 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.z smithi044 *:8443,8765 running (7m) 31s ago 7m 453M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9426886c12da 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a smithi007 running (9m) 16s ago 9m 50.0M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e370da5b9526 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b smithi016 running (7m) 31s ago 7m 41.0M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 d98f5c5ac880 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c smithi044 running (8m) 31s ago 8m 40.1M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 d20b903c79fc 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (7m) 16s ago 7m 45.6M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 079acd1a3dd0 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi016 running (6m) 31s ago 6m 46.2M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 d4884ea2e923 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi044 running (6m) 31s ago 6m 57.8M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 5d8f8d55638e 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi007 running (6m) 16s ago 6m 43.6M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 954a0ee870de 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi016 running (5m) 31s ago 5m 42.3M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 5f6657b803ac 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi044 running (5m) 31s ago 5m 43.2M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 cc38895a1bbe 2024-03-26T08:41:43.944 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 smithi007 running (4m) 16s ago 4m 41.4M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 afb491690921 2024-03-26T08:41:43.945 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 smithi016 running (4m) 31s ago 4m 54.5M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 5b29a5048f87 2024-03-26T08:41:43.945 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 smithi044 running (3m) 31s ago 3m 53.7M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 d45e75f140b2 2024-03-26T08:41:43.945 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 smithi007 running (3m) 16s ago 3m 55.1M 4273M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c8dd7f144d0d 2024-03-26T08:41:43.945 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 smithi016 running (3m) 31s ago 2m 54.8M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 b16d787d9cd6 2024-03-26T08:41:43.945 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 smithi044 running (2m) 31s ago 2m 52.2M 4271M 19.0.0-2145-g3be9a5c9 c664afe56bb0 1bce0b947910 2024-03-26T08:41:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71480a2890 msgr2=0x7f71480a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71480a2890 0x7f71480a4d50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f714c000c00 tx=0x7f714c005fb0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c101160 msgr2=0x7f715c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c101160 0x7f715c19a5b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f715400c170 tx=0x7f715400c480 comp rx=0 tx=0).stop 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 shutdown_connections 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71480a2890 0x7f71480a4d50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f715c102d00 0x7f715c1a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c101160 0x7f715c19a5b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 --2- 172.21.15.7:0/1532840139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715c102360 0x7f715c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 >> 172.21.15.7:0/1532840139 conn(0x7f715c0fc720 msgr2=0x7f715c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 shutdown_connections 2024-03-26T08:41:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:43.953+0000 7f7163020700 1 -- 172.21.15.7:0/1532840139 wait complete. 2024-03-26T08:41:44.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:44 smithi016 ceph-mon[35851]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-26T08:41:44.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:44 smithi007 ceph-mon[32462]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-26T08:41:44.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:44 smithi044 ceph-mon[36461]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-26T08:41:44.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-26T08:41:45.002 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:45.379 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:45 smithi007 ceph-mon[32462]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:45.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:45 smithi016 ceph-mon[35851]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:45.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:45 smithi044 ceph-mon[36461]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:46.459 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:46 smithi007 ceph-mon[32462]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:46.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:46 smithi016 ceph-mon[35851]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:46.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:46 smithi044 ceph-mon[36461]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.714+0000 7f7b5a91e700 1 -- 172.21.15.7:0/1507278274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b541022c0 msgr2=0x7f7b54102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.714+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/1507278274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b541022c0 0x7f7b54102720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7b480099c0 tx=0x7f7b48009cd0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.714+0000 7f7b5a91e700 1 -- 172.21.15.7:0/1507278274 shutdown_connections 2024-03-26T08:41:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.714+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/1507278274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b541010c0 0x7f7b541014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.714+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/1507278274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b54102c60 0x7f7b54109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.714+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/1507278274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b541022c0 0x7f7b54102720 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.714+0000 7f7b5a91e700 1 -- 172.21.15.7:0/1507278274 >> 172.21.15.7:0/1507278274 conn(0x7f7b540fc660 msgr2=0x7f7b540feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.715+0000 7f7b5a91e700 1 -- 172.21.15.7:0/1507278274 shutdown_connections 2024-03-26T08:41:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.715+0000 7f7b5a91e700 1 -- 172.21.15.7:0/1507278274 wait complete. 2024-03-26T08:41:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.715+0000 7f7b5a91e700 1 Processor -- start 2024-03-26T08:41:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.715+0000 7f7b5a91e700 1 -- start start 2024-03-26T08:41:46.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.715+0000 7f7b5a91e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b541010c0 0x7f7b541972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.715+0000 7f7b5a91e700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b541022c0 0x7f7b54197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.715+0000 7f7b5a91e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b54102c60 0x7f7b5419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b5a91e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5410cc80 con 0x7f7b54102c60 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b5a91e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5410cb00 con 0x7f7b541010c0 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b5a91e700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5410ce00 con 0x7f7b541022c0 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b58ebb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b54102c60 0x7f7b5419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b53fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b541010c0 0x7f7b541972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b58ebb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b54102c60 0x7f7b5419e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60840/0 (socket says 172.21.15.7:60840) 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b58ebb700 1 -- 172.21.15.7:0/407856631 learned_addr learned my addr 172.21.15.7:0/407856631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b537fe700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b541022c0 0x7f7b54197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b53fff700 1 -- 172.21.15.7:0/407856631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b541022c0 msgr2=0x7f7b54197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b53fff700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b541022c0 0x7f7b54197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b53fff700 1 -- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b54102c60 msgr2=0x7f7b5419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b53fff700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b54102c60 0x7f7b5419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b53fff700 1 -- 172.21.15.7:0/407856631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5419ef50 con 0x7f7b541010c0 2024-03-26T08:41:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b58ebb700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b54102c60 0x7f7b5419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b53fff700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b541010c0 0x7f7b541972f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400e9c0 tx=0x7f7b4400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b517fa700 1 -- 172.21.15.7:0/407856631 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44004220 con 0x7f7b541010c0 2024-03-26T08:41:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b517fa700 1 -- 172.21.15.7:0/407856631 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b440043c0 con 0x7f7b541010c0 2024-03-26T08:41:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5419f240 con 0x7f7b541010c0 2024-03-26T08:41:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b517fa700 1 -- 172.21.15.7:0/407856631 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44003d80 con 0x7f7b541010c0 2024-03-26T08:41:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.716+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5419f7d0 con 0x7f7b541010c0 2024-03-26T08:41:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.717+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b54068e50 con 0x7f7b541010c0 2024-03-26T08:41:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.718+0000 7f7b517fa700 1 -- 172.21.15.7:0/407856631 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7b4400c7c0 con 0x7f7b541010c0 2024-03-26T08:41:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.719+0000 7f7b517fa700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b3c0a2910 0x7f7b3c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.719+0000 7f7b537fe700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b3c0a2910 0x7f7b3c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.719+0000 7f7b517fa700 1 -- 172.21.15.7:0/407856631 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7b440cebc0 con 0x7f7b541010c0 2024-03-26T08:41:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.720+0000 7f7b537fe700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b3c0a2910 0x7f7b3c0a4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b48000c00 tx=0x7f7b48005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.723+0000 7f7b517fa700 1 -- 172.21.15.7:0/407856631 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b4409bc60 con 0x7f7b541010c0 2024-03-26T08:41:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.937+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b540625d0 con 0x7f7b3c0a2910 2024-03-26T08:41:46.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.943+0000 7f7b517fa700 1 -- 172.21.15.7:0/407856631 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f7b540625d0 con 0x7f7b3c0a2910 2024-03-26T08:41:46.934 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-26T08:41:46.934 INFO:teuthology.orchestra.run.smithi007.stdout:mds.all 12/12 34s ago 2m smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi044=c;smithi044=f;smithi044=i;smithi044=l;count:12 2024-03-26T08:41:46.934 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 3/3 34s ago 7m smithi007=x;smithi016=y;smithi044=z;count:3 2024-03-26T08:41:46.935 INFO:teuthology.orchestra.run.smithi007.stdout:mon 3/3 34s ago 8m smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi044:172.21.15.44=c;count:3 2024-03-26T08:41:46.935 INFO:teuthology.orchestra.run.smithi007.stdout:osd 12 34s ago - 2024-03-26T08:41:46.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b3c0a2910 msgr2=0x7f7b3c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:46.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b3c0a2910 0x7f7b3c0a4dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b48000c00 tx=0x7f7b48005fb0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b541010c0 msgr2=0x7f7b541972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b541010c0 0x7f7b541972f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400e9c0 tx=0x7f7b4400ecd0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 shutdown_connections 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b3c0a2910 0x7f7b3c0a4dd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b54102c60 0x7f7b5419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b541010c0 0x7f7b541972f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 --2- 172.21.15.7:0/407856631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7b541022c0 0x7f7b54197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 >> 172.21.15.7:0/407856631 conn(0x7f7b540fc660 msgr2=0x7f7b54105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 shutdown_connections 2024-03-26T08:41:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:46.945+0000 7f7b5a91e700 1 -- 172.21.15.7:0/407856631 wait complete. 2024-03-26T08:41:47.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-26T08:41:47.886 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:48.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:48 smithi007 ceph-mon[32462]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:48.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:48 smithi007 ceph-mon[32462]: from='client.24968 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:48.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:48 smithi016 ceph-mon[35851]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:48.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:48 smithi016 ceph-mon[35851]: from='client.24968 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:48.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:48 smithi044 ceph-mon[36461]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:48.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:48 smithi044 ceph-mon[36461]: from='client.24968 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.609+0000 7fe96bfbd700 1 -- 172.21.15.7:0/588555560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9641002a0 msgr2=0x7fe964100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.609+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/588555560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9641002a0 0x7fe964100700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe958009990 tx=0x7fe958009ca0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.609+0000 7fe96bfbd700 1 -- 172.21.15.7:0/588555560 shutdown_connections 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.609+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/588555560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe964103820 0x7fe964103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.609+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/588555560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964100c40 0x7fe96410a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.609+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/588555560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9641002a0 0x7fe964100700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.609+0000 7fe96bfbd700 1 -- 172.21.15.7:0/588555560 >> 172.21.15.7:0/588555560 conn(0x7fe9640fbe20 msgr2=0x7fe9640fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.609+0000 7fe96bfbd700 1 -- 172.21.15.7:0/588555560 shutdown_connections 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.609+0000 7fe96bfbd700 1 -- 172.21.15.7:0/588555560 wait complete. 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe96bfbd700 1 Processor -- start 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe96bfbd700 1 -- start start 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe96bfbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9641002a0 0x7fe9641983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe96bfbd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964100c40 0x7fe9641988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe96bfbd700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe964103820 0x7fe96419f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe96bfbd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96410ddf0 con 0x7fe964100c40 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe96bfbd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96410dc70 con 0x7fe9641002a0 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe96bfbd700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96410df70 con 0x7fe964103820 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe969558700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964100c40 0x7fe9641988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe969d59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9641002a0 0x7fe9641983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe969558700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964100c40 0x7fe9641988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60850/0 (socket says 172.21.15.7:60850) 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe969558700 1 -- 172.21.15.7:0/2120479022 learned_addr learned my addr 172.21.15.7:0/2120479022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.610+0000 7fe96a55a700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe964103820 0x7fe96419f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe969d59700 1 -- 172.21.15.7:0/2120479022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe964103820 msgr2=0x7fe96419f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe969d59700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe964103820 0x7fe96419f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe969d59700 1 -- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964100c40 msgr2=0x7fe9641988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe969d59700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964100c40 0x7fe9641988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe969d59700 1 -- 172.21.15.7:0/2120479022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9641a0000 con 0x7fe9641002a0 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe969558700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964100c40 0x7fe9641988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe969d59700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9641002a0 0x7fe9641983a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe95400ef90 tx=0x7fe95400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe962ffd700 1 -- 172.21.15.7:0/2120479022 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954003a00 con 0x7fe9641002a0 2024-03-26T08:41:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe962ffd700 1 -- 172.21.15.7:0/2120479022 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe954003ba0 con 0x7fe9641002a0 2024-03-26T08:41:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9641a02f0 con 0x7fe9641002a0 2024-03-26T08:41:49.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe962ffd700 1 -- 172.21.15.7:0/2120479022 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9540048a0 con 0x7fe9641002a0 2024-03-26T08:41:49.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.611+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9641a0830 con 0x7fe9641002a0 2024-03-26T08:41:49.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.612+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe964068e50 con 0x7fe9641002a0 2024-03-26T08:41:49.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.613+0000 7fe962ffd700 1 -- 172.21.15.7:0/2120479022 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe954012050 con 0x7fe9641002a0 2024-03-26T08:41:49.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.614+0000 7fe962ffd700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe94c0a2910 0x7fe94c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:49.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.614+0000 7fe969558700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe94c0a2910 0x7fe94c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:49.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.614+0000 7fe962ffd700 1 -- 172.21.15.7:0/2120479022 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe9540ce8e0 con 0x7fe9641002a0 2024-03-26T08:41:49.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.614+0000 7fe969558700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe94c0a2910 0x7fe94c0a4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe95800b2e0 tx=0x7fe95800b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.618+0000 7fe962ffd700 1 -- 172.21.15.7:0/2120479022 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe954010040 con 0x7fe9641002a0 2024-03-26T08:41:49.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.831+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9640625d0 con 0x7fe94c0a2910 2024-03-26T08:41:49.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.832+0000 7fe962ffd700 1 -- 172.21.15.7:0/2120479022 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7fe9640625d0 con 0x7fe94c0a2910 2024-03-26T08:41:49.824 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-03-26T08:41:49.824 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-03-26T08:41:49.824 INFO:teuthology.orchestra.run.smithi007.stdout:smithi016 172.21.15.16 2024-03-26T08:41:49.824 INFO:teuthology.orchestra.run.smithi007.stdout:smithi044 172.21.15.44 2024-03-26T08:41:49.824 INFO:teuthology.orchestra.run.smithi007.stdout:3 hosts in cluster 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe94c0a2910 msgr2=0x7fe94c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe94c0a2910 0x7fe94c0a4dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe95800b2e0 tx=0x7fe95800b880 comp rx=0 tx=0).stop 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9641002a0 msgr2=0x7fe9641983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9641002a0 0x7fe9641983a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe95400ef90 tx=0x7fe95400c3a0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 shutdown_connections 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe94c0a2910 0x7fe94c0a4dd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe964103820 0x7fe96419f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9641002a0 0x7fe9641983a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 --2- 172.21.15.7:0/2120479022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964100c40 0x7fe9641988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 >> 172.21.15.7:0/2120479022 conn(0x7fe9640fbe20 msgr2=0x7fe964106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 shutdown_connections 2024-03-26T08:41:49.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:49.834+0000 7fe96bfbd700 1 -- 172.21.15.7:0/2120479022 wait complete. 2024-03-26T08:41:50.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:50 smithi016 ceph-mon[35851]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:41:50.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:50 smithi007 ceph-mon[32462]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:41:50.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:50 smithi044 ceph-mon[36461]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:41:50.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-26T08:41:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:51.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:51 smithi007 ceph-mon[32462]: from='client.24974 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:51.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:51 smithi016 ceph-mon[35851]: from='client.24974 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:51.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:51 smithi044 ceph-mon[36461]: from='client.24974 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:52.445 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:52 smithi007 ceph-mon[32462]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:52.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:52 smithi016 ceph-mon[35851]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:52.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:52 smithi044 ceph-mon[36461]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.740+0000 7f98e4647700 1 -- 172.21.15.7:0/3179169572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 msgr2=0x7f98dc107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.740+0000 7f98e4647700 1 --2- 172.21.15.7:0/3179169572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 0x7f98dc107a20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009990 tx=0x7f98cc009ca0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.740+0000 7f98e4647700 1 -- 172.21.15.7:0/3179169572 shutdown_connections 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.740+0000 7f98e4647700 1 --2- 172.21.15.7:0/3179169572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98dc102d00 0x7f98dc1050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.740+0000 7f98e4647700 1 --2- 172.21.15.7:0/3179169572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98dc107f60 0x7f98dc10a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.740+0000 7f98e4647700 1 --2- 172.21.15.7:0/3179169572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 0x7f98dc107a20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.740+0000 7f98e4647700 1 -- 172.21.15.7:0/3179169572 >> 172.21.15.7:0/3179169572 conn(0x7f98dc0fc6b0 msgr2=0x7f98dc0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.740+0000 7f98e4647700 1 -- 172.21.15.7:0/3179169572 shutdown_connections 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.740+0000 7f98e4647700 1 -- 172.21.15.7:0/3179169572 wait complete. 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e4647700 1 Processor -- start 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e4647700 1 -- start start 2024-03-26T08:41:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e4647700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98dc102d00 0x7f98dc102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e4647700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 0x7f98dc100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e4647700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98dc107f60 0x7f98dc1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e4647700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc10de20 con 0x7f98dc105630 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e4647700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc10dca0 con 0x7f98dc107f60 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e4647700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc10dfa0 con 0x7f98dc102d00 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e1be2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 0x7f98dc100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e23e3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98dc102d00 0x7f98dc102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e1be2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 0x7f98dc100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60858/0 (socket says 172.21.15.7:60858) 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e1be2700 1 -- 172.21.15.7:0/3608904443 learned_addr learned my addr 172.21.15.7:0/3608904443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.741+0000 7f98e2be4700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98dc107f60 0x7f98dc1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98e1be2700 1 -- 172.21.15.7:0/3608904443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98dc102d00 msgr2=0x7f98dc102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98e1be2700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98dc102d00 0x7f98dc102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98e1be2700 1 -- 172.21.15.7:0/3608904443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98dc107f60 msgr2=0x7f98dc1011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98e1be2700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98dc107f60 0x7f98dc1011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98e1be2700 1 -- 172.21.15.7:0/3608904443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc101aa0 con 0x7f98dc105630 2024-03-26T08:41:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98e2be4700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98dc107f60 0x7f98dc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:41:52.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98e1be2700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 0x7f98dc100cb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f98cc005910 tx=0x7f98cc00bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:52.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98d37fe700 1 -- 172.21.15.7:0/3608904443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc026070 con 0x7f98dc105630 2024-03-26T08:41:52.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98d37fe700 1 -- 172.21.15.7:0/3608904443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f98cc003d70 con 0x7f98dc105630 2024-03-26T08:41:52.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc1a24f0 con 0x7f98dc105630 2024-03-26T08:41:52.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98d37fe700 1 -- 172.21.15.7:0/3608904443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc021410 con 0x7f98dc105630 2024-03-26T08:41:52.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.742+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc1a2940 con 0x7f98dc105630 2024-03-26T08:41:52.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.743+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc068e50 con 0x7f98dc105630 2024-03-26T08:41:52.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.744+0000 7f98d37fe700 1 -- 172.21.15.7:0/3608904443 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f98cc029070 con 0x7f98dc105630 2024-03-26T08:41:52.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.745+0000 7f98d37fe700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f98c80a2990 0x7f98c80a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:52.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.745+0000 7f98e23e3700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f98c80a2990 0x7f98c80a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:52.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.745+0000 7f98d37fe700 1 -- 172.21.15.7:0/3608904443 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f98cc0cf9f0 con 0x7f98dc105630 2024-03-26T08:41:52.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.745+0000 7f98e23e3700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f98c80a2990 0x7f98c80a4e50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009c60 tx=0x7f98d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:52.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.748+0000 7f98d37fe700 1 -- 172.21.15.7:0/3608904443 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f98cc030080 con 0x7f98dc105630 2024-03-26T08:41:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.959+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f98dc0625d0 con 0x7f98c80a2990 2024-03-26T08:41:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.961+0000 7f98d37fe700 1 -- 172.21.15.7:0/3608904443 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f98dc0625d0 con 0x7f98c80a2990 2024-03-26T08:41:52.952 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-26T08:41:52.952 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-26T08:41:52.953 INFO:teuthology.orchestra.run.smithi007.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-03-26T08:41:52.953 INFO:teuthology.orchestra.run.smithi007.stdout:smithi044 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001K400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-26T08:41:52.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f98c80a2990 msgr2=0x7f98c80a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:52.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f98c80a2990 0x7f98c80a4e50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009c60 tx=0x7f98d8009260 comp rx=0 tx=0).stop 2024-03-26T08:41:52.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 msgr2=0x7f98dc100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 0x7f98dc100cb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f98cc005910 tx=0x7f98cc00bbb0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 shutdown_connections 2024-03-26T08:41:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f98c80a2990 0x7f98c80a4e50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f98dc102d00 0x7f98dc102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98dc107f60 0x7f98dc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 --2- 172.21.15.7:0/3608904443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98dc105630 0x7f98dc100cb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 >> 172.21.15.7:0/3608904443 conn(0x7f98dc0fc6b0 msgr2=0x7f98dc103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 shutdown_connections 2024-03-26T08:41:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:52.963+0000 7f98e4647700 1 -- 172.21.15.7:0/3608904443 wait complete. 2024-03-26T08:41:53.752 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T08:41:53.764 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-26T08:41:53.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-26T08:41:54.014 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:54.472 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:54 smithi007 ceph-mon[32462]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T08:41:54.472 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:54 smithi007 ceph-mon[32462]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:54.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:54 smithi016 ceph-mon[35851]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T08:41:54.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:54 smithi016 ceph-mon[35851]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:54.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:54 smithi044 ceph-mon[36461]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T08:41:54.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:54 smithi044 ceph-mon[36461]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.679+0000 7f5ed382b700 1 -- 172.21.15.7:0/2736770479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc100280 msgr2=0x7f5ecc100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.679+0000 7f5ed382b700 1 --2- 172.21.15.7:0/2736770479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc100280 0x7f5ecc100700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009990 tx=0x7f5ebc009ca0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 -- 172.21.15.7:0/2736770479 shutdown_connections 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 --2- 172.21.15.7:0/2736770479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ecc102780 0x7f5ecc102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 --2- 172.21.15.7:0/2736770479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ecc100c40 0x7f5ecc1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 --2- 172.21.15.7:0/2736770479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc100280 0x7f5ecc100700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 -- 172.21.15.7:0/2736770479 >> 172.21.15.7:0/2736770479 conn(0x7f5ecc0fbe20 msgr2=0x7f5ecc0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 -- 172.21.15.7:0/2736770479 shutdown_connections 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 -- 172.21.15.7:0/2736770479 wait complete. 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 Processor -- start 2024-03-26T08:41:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 -- start start 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ecc100280 0x7f5ecc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ecc100c40 0x7f5ecc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc102780 0x7f5ecc1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc10ffc0 con 0x7f5ecc102780 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc10fe40 con 0x7f5ecc100c40 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.680+0000 7f5ed382b700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc110140 con 0x7f5ecc100280 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed1dc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc102780 0x7f5ecc1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed1dc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc102780 0x7f5ecc1a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60888/0 (socket says 172.21.15.7:60888) 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed1dc8700 1 -- 172.21.15.7:0/430219801 learned_addr learned my addr 172.21.15.7:0/430219801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed15c7700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ecc100280 0x7f5ecc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed0dc6700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ecc100c40 0x7f5ecc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed1dc8700 1 -- 172.21.15.7:0/430219801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ecc100280 msgr2=0x7f5ecc19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed1dc8700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ecc100280 0x7f5ecc19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed1dc8700 1 -- 172.21.15.7:0/430219801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ecc100c40 msgr2=0x7f5ecc19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed1dc8700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ecc100c40 0x7f5ecc19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed1dc8700 1 -- 172.21.15.7:0/430219801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc1a21e0 con 0x7f5ecc102780 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed1dc8700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc102780 0x7f5ecc1a1b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400ef60 tx=0x7f5ec400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ec27fc700 1 -- 172.21.15.7:0/430219801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40092e0 con 0x7f5ecc102780 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc1a2470 con 0x7f5ecc102780 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ec27fc700 1 -- 172.21.15.7:0/430219801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ec4004bb0 con 0x7f5ecc102780 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ec27fc700 1 -- 172.21.15.7:0/430219801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4004280 con 0x7f5ecc102780 2024-03-26T08:41:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.681+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc1a29b0 con 0x7f5ecc102780 2024-03-26T08:41:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.682+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc068e50 con 0x7f5ecc102780 2024-03-26T08:41:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.683+0000 7f5ec27fc700 1 -- 172.21.15.7:0/430219801 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5ec401b030 con 0x7f5ecc102780 2024-03-26T08:41:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.684+0000 7f5ec27fc700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5eb80a2890 0x7f5eb80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.684+0000 7f5ed15c7700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5eb80a2890 0x7f5eb80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.684+0000 7f5ec27fc700 1 -- 172.21.15.7:0/430219801 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5ec40ce4d0 con 0x7f5ecc102780 2024-03-26T08:41:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.684+0000 7f5ed15c7700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5eb80a2890 0x7f5eb80a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8005950 tx=0x7f5ec800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.688+0000 7f5ec27fc700 1 -- 172.21.15.7:0/430219801 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5ec4022020 con 0x7f5ecc102780 2024-03-26T08:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.943+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5ecc062da0 con 0x7f5ecc102780 2024-03-26T08:41:55.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.943+0000 7f5ec27fc700 1 -- 172.21.15.7:0/430219801 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2968 (secure 0 0 0) 0x7f5ec4010040 con 0x7f5ecc102780 2024-03-26T08:41:55.935 INFO:teuthology.orchestra.run.smithi007.stdout:e22 2024-03-26T08:41:55.935 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T08:41:55.935 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:41:55.935 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-03-26T08:41:55.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 22 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-26T08:40:09.583324+0000 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-26T08:40:15.277649+0000 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 300 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-03-26T08:41:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-03-26T08:41:55.937 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-03-26T08:41:55.937 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T08:41:55.937 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 3 2024-03-26T08:41:55.937 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1,2 2024-03-26T08:41:55.937 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=24512,1=24502,2=24478} 2024-03-26T08:41:55.937 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-03-26T08:41:55.937 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 24512 members: 24478,24502,24512 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b{0:24512} state up:active seq 3 addr [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.h{0:24476} state up:standby-replay seq 1 addr [v2:172.21.15.16:6836/7790244,v1:172.21.15.16:6837/7790244] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.938 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.i{1:24502} state up:active seq 5 addr [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f{1:24466} state up:standby-replay seq 1 addr [v2:172.21.15.44:6834/2525817262,v1:172.21.15.44:6835/2525817262] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c{2:24478} state up:active seq 7 addr [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e{2:24443} state up:standby-replay seq 1 addr [v2:172.21.15.16:6834/1837116746,v1:172.21.15.16:6835/1837116746] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.j{-1:14580} state up:standby seq 1 addr [v2:172.21.15.7:6834/2764125554,v1:172.21.15.7:6835/2764125554] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d{-1:14592} state up:standby seq 1 addr [v2:172.21.15.7:6836/1591045564,v1:172.21.15.7:6837/1591045564] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.g{-1:14667} state up:standby seq 1 addr [v2:172.21.15.7:6838/2041082244,v1:172.21.15.7:6839/2041082244] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a{-1:14679} state up:standby seq 1 addr [v2:172.21.15.7:6840/3829840154,v1:172.21.15.7:6841/3829840154] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.k{-1:24410} state up:standby seq 1 addr [v2:172.21.15.16:6832/76809036,v1:172.21.15.16:6833/76809036] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.939 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.l{-1:24424} state up:standby seq 1 addr [v2:172.21.15.44:6832/3400459626,v1:172.21.15.44:6833/3400459626] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T08:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5eb80a2890 msgr2=0x7f5eb80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5eb80a2890 0x7f5eb80a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8005950 tx=0x7f5ec800b410 comp rx=0 tx=0).stop 2024-03-26T08:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc102780 msgr2=0x7f5ecc1a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc102780 0x7f5ecc1a1b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400ef60 tx=0x7f5ec400ef90 comp rx=0 tx=0).stop 2024-03-26T08:41:55.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 shutdown_connections 2024-03-26T08:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5eb80a2890 0x7f5eb80a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ecc100280 0x7f5ecc19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ecc102780 0x7f5ecc1a1b40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 --2- 172.21.15.7:0/430219801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ecc100c40 0x7f5ecc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.946+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 >> 172.21.15.7:0/430219801 conn(0x7f5ecc0fbe20 msgr2=0x7f5ecc0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.947+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 shutdown_connections 2024-03-26T08:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:55.947+0000 7f5ed382b700 1 -- 172.21.15.7:0/430219801 wait complete. 2024-03-26T08:41:55.941 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-03-26T08:41:56.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:56 smithi016 ceph-mon[35851]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:56.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:56 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/430219801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T08:41:56.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:56 smithi007 ceph-mon[32462]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:56.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:56 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/430219801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T08:41:56.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:56 smithi044 ceph-mon[36461]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:41:56.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:56 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/430219801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T08:41:56.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-26T08:41:56.961 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:41:58.382 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:58 smithi007 ceph-mon[32462]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:58.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:58 smithi016 ceph-mon[35851]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:58.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:58 smithi044 ceph-mon[36461]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:41:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.685+0000 7ff9c0089700 1 -- 172.21.15.7:0/4169447684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9b8100a40 msgr2=0x7ff9b8100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.685+0000 7ff9c0089700 1 --2- 172.21.15.7:0/4169447684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9b8100a40 0x7ff9b8100e60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40098f0 tx=0x7ff9b4009c00 comp rx=0 tx=0).stop 2024-03-26T08:41:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.685+0000 7ff9c0089700 1 -- 172.21.15.7:0/4169447684 shutdown_connections 2024-03-26T08:41:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.685+0000 7ff9c0089700 1 --2- 172.21.15.7:0/4169447684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9b8100a40 0x7ff9b8100e60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.685+0000 7ff9c0089700 1 --2- 172.21.15.7:0/4169447684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8101d60 0x7ff9b81021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.685+0000 7ff9c0089700 1 --2- 172.21.15.7:0/4169447684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b81013a0 0x7ff9b8101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.685+0000 7ff9c0089700 1 -- 172.21.15.7:0/4169447684 >> 172.21.15.7:0/4169447684 conn(0x7ff9b80fc660 msgr2=0x7ff9b80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.685+0000 7ff9c0089700 1 -- 172.21.15.7:0/4169447684 shutdown_connections 2024-03-26T08:41:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.685+0000 7ff9c0089700 1 -- 172.21.15.7:0/4169447684 wait complete. 2024-03-26T08:41:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9c0089700 1 Processor -- start 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9c0089700 1 -- start start 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9c0089700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8100a40 0x7ff9b8106570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9c0089700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9b81013a0 0x7ff9b8106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9c0089700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8101d60 0x7ff9b81030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9c0089700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b810ff60 con 0x7ff9b8100a40 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9c0089700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b810fde0 con 0x7ff9b8101d60 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9c0089700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81100e0 con 0x7ff9b81013a0 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9bde25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8100a40 0x7ff9b8106570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9bde25700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8100a40 0x7ff9b8106570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46420/0 (socket says 172.21.15.7:46420) 2024-03-26T08:41:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9bde25700 1 -- 172.21.15.7:0/1190715110 learned_addr learned my addr 172.21.15.7:0/1190715110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9bd624700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9b81013a0 0x7ff9b8106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.686+0000 7ff9be626700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8101d60 0x7ff9b81030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9bde25700 1 -- 172.21.15.7:0/1190715110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9b81013a0 msgr2=0x7ff9b8106ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9bde25700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9b81013a0 0x7ff9b8106ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9bde25700 1 -- 172.21.15.7:0/1190715110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8101d60 msgr2=0x7ff9b81030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9bde25700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8101d60 0x7ff9b81030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9bde25700 1 -- 172.21.15.7:0/1190715110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b81039c0 con 0x7ff9b8100a40 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9bde25700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8100a40 0x7ff9b8106570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4010ff0 tx=0x7ff9b4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9aeffd700 1 -- 172.21.15.7:0/1190715110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4027070 con 0x7ff9b8100a40 2024-03-26T08:41:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b8103c50 con 0x7ff9b8100a40 2024-03-26T08:41:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9aeffd700 1 -- 172.21.15.7:0/1190715110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9b400b560 con 0x7ff9b8100a40 2024-03-26T08:41:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9aeffd700 1 -- 172.21.15.7:0/1190715110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4004b90 con 0x7ff9b8100a40 2024-03-26T08:41:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.687+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b81a6e10 con 0x7ff9b8100a40 2024-03-26T08:41:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.688+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b8068e50 con 0x7ff9b8100a40 2024-03-26T08:41:58.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.690+0000 7ff9aeffd700 1 -- 172.21.15.7:0/1190715110 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff9b402a070 con 0x7ff9b8100a40 2024-03-26T08:41:58.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.690+0000 7ff9aeffd700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9a40a2890 0x7ff9a40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:58.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.691+0000 7ff9bd624700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9a40a2890 0x7ff9a40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:58.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.691+0000 7ff9aeffd700 1 -- 172.21.15.7:0/1190715110 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9b40cfbe0 con 0x7ff9b8100a40 2024-03-26T08:41:58.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.691+0000 7ff9bd624700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9a40a2890 0x7ff9a40a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009680 tx=0x7ff9a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.694+0000 7ff9aeffd700 1 -- 172.21.15.7:0/1190715110 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9b4031070 con 0x7ff9b8100a40 2024-03-26T08:41:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.898+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7ff9b802da30 con 0x7ff9b8100a40 2024-03-26T08:41:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.899+0000 7ff9aeffd700 1 -- 172.21.15.7:0/1190715110 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5257 (secure 0 0 0) 0x7ff9b4013030 con 0x7ff9b8100a40 2024-03-26T08:41:58.891 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 87 2024-03-26T08:41:58.891 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T08:41:58.891 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-03-26T08:31:55.831994+0000 2024-03-26T08:41:58.891 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-03-26T08:40:59.413729+0000 2024-03-26T08:41:58.891 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-26T08:41:58.892 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 26 2024-03-26T08:41:58.892 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-03-26T08:41:58.892 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-03-26T08:41:58.892 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-03-26T08:41:58.892 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-03-26T08:41:58.892 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client luminous 2024-03-26T08:41:58.893 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-03-26T08:41:58.893 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-03-26T08:41:58.893 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-26T08:41:58.893 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-26T08:41:58.893 INFO:teuthology.orchestra.run.smithi007.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-26T08:41:58.893 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 12 2024-03-26T08:41:58.894 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6802/2166020682,v1:172.21.15.7:6803/2166020682] [v2:172.21.15.7:6804/2166020682,v1:172.21.15.7:6805/2166020682] exists,up 11d5305b-e22f-42b9-a7b6-acc96b92baab 2024-03-26T08:41:58.894 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6800/1464490925,v1:172.21.15.16:6801/1464490925] [v2:172.21.15.16:6802/1464490925,v1:172.21.15.16:6803/1464490925] exists,up 95b219c6-a22b-4b66-9fa8-967c59f003ab 2024-03-26T08:41:58.894 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6800/2047416654,v1:172.21.15.44:6801/2047416654] [v2:172.21.15.44:6802/2047416654,v1:172.21.15.44:6803/2047416654] exists,up 10c4a5af-45a4-4285-b6b4-0bc980ec6faa 2024-03-26T08:41:58.894 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6810/4236464716,v1:172.21.15.7:6811/4236464716] [v2:172.21.15.7:6812/4236464716,v1:172.21.15.7:6813/4236464716] exists,up b46aab34-7b13-4e83-8fbd-7c207cd246e9 2024-03-26T08:41:58.894 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6808/3280306354,v1:172.21.15.16:6809/3280306354] [v2:172.21.15.16:6810/3280306354,v1:172.21.15.16:6811/3280306354] exists,up 78a3ff9e-09b6-4255-96fd-9d155625811b 2024-03-26T08:41:58.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6808/2190779473,v1:172.21.15.44:6809/2190779473] [v2:172.21.15.44:6810/2190779473,v1:172.21.15.44:6811/2190779473] exists,up ab45e6b1-3006-4af1-90f3-455a09f442f5 2024-03-26T08:41:58.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6818/2859951867,v1:172.21.15.7:6819/2859951867] [v2:172.21.15.7:6820/2859951867,v1:172.21.15.7:6821/2859951867] exists,up 44d3581c-a611-48f8-92c7-4273e6639781 2024-03-26T08:41:58.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6816/3902462229,v1:172.21.15.16:6817/3902462229] [v2:172.21.15.16:6818/3902462229,v1:172.21.15.16:6819/3902462229] exists,up e87d365c-1fcf-40cd-93f6-5fc972c25217 2024-03-26T08:41:58.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6816/3863516143,v1:172.21.15.44:6817/3863516143] [v2:172.21.15.44:6818/3863516143,v1:172.21.15.44:6819/3863516143] exists,up 59f51bbc-b176-435d-bf69-4269bb0a6102 2024-03-26T08:41:58.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6826/387825612,v1:172.21.15.7:6827/387825612] [v2:172.21.15.7:6828/387825612,v1:172.21.15.7:6829/387825612] exists,up 048f7ae9-18c6-4128-815b-082a9297ed68 2024-03-26T08:41:58.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6824/1979394150,v1:172.21.15.16:6825/1979394150] [v2:172.21.15.16:6826/1979394150,v1:172.21.15.16:6827/1979394150] exists,up 40e2915e-efad-4dbd-a244-cad31df0cd80 2024-03-26T08:41:58.896 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.44:6824/2818540789,v1:172.21.15.44:6825/2818540789] [v2:172.21.15.44:6826/2818540789,v1:172.21.15.44:6827/2818540789] exists,up e3e817ae-4259-46ec-bbe3-91c24adbeb96 2024-03-26T08:41:58.896 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.a [2,11] 2024-03-26T08:41:58.896 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.f [8,11] 2024-03-26T08:41:58.896 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.6 [4,6] 2024-03-26T08:41:58.896 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.c [5,8] 2024-03-26T08:41:58.896 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.e [4,6] 2024-03-26T08:41:58.896 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.10 [5,9] 2024-03-26T08:41:58.897 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.1e [3,9] 2024-03-26T08:41:58.897 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.23 [4,8] 2024-03-26T08:41:58.897 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.2c [4,9] 2024-03-26T08:41:58.897 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1981295866 expires 2024-03-27T08:32:58.217753+0000 2024-03-26T08:41:58.897 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/1336720039 expires 2024-03-27T08:32:58.217753+0000 2024-03-26T08:41:58.897 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/444923255 expires 2024-03-27T08:32:58.217753+0000 2024-03-26T08:41:58.898 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/3914280225 expires 2024-03-27T08:32:30.975912+0000 2024-03-26T08:41:58.898 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/2786531355 expires 2024-03-27T08:32:30.975912+0000 2024-03-26T08:41:58.898 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/3110430694 expires 2024-03-27T08:32:30.975912+0000 2024-03-26T08:41:58.898 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/3110430694 expires 2024-03-27T08:32:30.975912+0000 2024-03-26T08:41:58.898 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/804148713 expires 2024-03-27T08:32:30.975912+0000 2024-03-26T08:41:58.898 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/1336720039 expires 2024-03-27T08:32:58.217753+0000 2024-03-26T08:41:58.899 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/2665937259 expires 2024-03-27T08:32:58.217753+0000 2024-03-26T08:41:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9a40a2890 msgr2=0x7ff9a40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9a40a2890 0x7ff9a40a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009680 tx=0x7ff9a8009260 comp rx=0 tx=0).stop 2024-03-26T08:41:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8100a40 msgr2=0x7ff9b8106570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:58.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8100a40 0x7ff9b8106570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4010ff0 tx=0x7ff9b4005040 comp rx=0 tx=0).stop 2024-03-26T08:41:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 shutdown_connections 2024-03-26T08:41:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9a40a2890 0x7ff9a40a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b8101d60 0x7ff9b81030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b8100a40 0x7ff9b8106570 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 --2- 172.21.15.7:0/1190715110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9b81013a0 0x7ff9b8106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:58.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 >> 172.21.15.7:0/1190715110 conn(0x7ff9b80fc660 msgr2=0x7ff9b80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 shutdown_connections 2024-03-26T08:41:58.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:58.902+0000 7ff9c0089700 1 -- 172.21.15.7:0/1190715110 wait complete. 2024-03-26T08:41:59.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:41:59 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1190715110' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T08:41:59.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:41:59 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1190715110' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T08:41:59.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:41:59 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1190715110' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T08:41:59.822 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-26T08:41:59.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:41:59.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.953+0000 7fefab47e640 1 -- 172.21.15.7:0/3829664541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa41021c0 msgr2=0x7fefa41045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.953+0000 7fefab47e640 1 --2- 172.21.15.7:0/3829664541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa41021c0 0x7fefa41045b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009900 tx=0x7fefa002bc30 comp rx=0 tx=0).stop 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.953+0000 7fefab47e640 1 -- 172.21.15.7:0/3829664541 shutdown_connections 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.953+0000 7fefab47e640 1 --2- 172.21.15.7:0/3829664541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4107420 0x7fefa4109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.953+0000 7fefab47e640 1 --2- 172.21.15.7:0/3829664541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefa4104af0 0x7fefa4106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.953+0000 7fefab47e640 1 --2- 172.21.15.7:0/3829664541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa41021c0 0x7fefa41045b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.953+0000 7fefab47e640 1 -- 172.21.15.7:0/3829664541 >> 172.21.15.7:0/3829664541 conn(0x7fefa40fbbd0 msgr2=0x7fefa40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.953+0000 7fefab47e640 1 -- 172.21.15.7:0/3829664541 shutdown_connections 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.953+0000 7fefab47e640 1 -- 172.21.15.7:0/3829664541 wait complete. 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefab47e640 1 Processor -- start 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefab47e640 1 -- start start 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefab47e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefa41021c0 0x7fefa41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefab47e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4104af0 0x7fefa4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefab47e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa4107420 0x7fefa419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefab47e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa410d370 con 0x7fefa4107420 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefab47e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa410d1f0 con 0x7fefa4104af0 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefab47e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa410d4f0 con 0x7fefa41021c0 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa9a1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa4107420 0x7fefa419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa9a1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa4107420 0x7fefa419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46432/0 (socket says 172.21.15.7:46432) 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa9a1b640 1 -- 172.21.15.7:0/288585450 learned_addr learned my addr 172.21.15.7:0/288585450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa8a19640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4104af0 0x7fefa4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa921a640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefa41021c0 0x7fefa41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa9a1b640 1 -- 172.21.15.7:0/288585450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefa41021c0 msgr2=0x7fefa41979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa9a1b640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefa41021c0 0x7fefa41979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa9a1b640 1 -- 172.21.15.7:0/288585450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4104af0 msgr2=0x7fefa4197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa9a1b640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4104af0 0x7fefa4197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:41:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.954+0000 7fefa9a1b640 1 -- 172.21.15.7:0/288585450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa419f600 con 0x7fefa4107420 2024-03-26T08:41:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.955+0000 7fefa9a1b640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa4107420 0x7fefa419ef90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00eb10 tx=0x7fef9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.955+0000 7fef9a7fc640 1 -- 172.21.15.7:0/288585450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c004b20 con 0x7fefa4107420 2024-03-26T08:41:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.955+0000 7fef9a7fc640 1 -- 172.21.15.7:0/288585450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef9c004cc0 con 0x7fefa4107420 2024-03-26T08:41:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.955+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa419f8f0 con 0x7fefa4107420 2024-03-26T08:41:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.955+0000 7fef9a7fc640 1 -- 172.21.15.7:0/288585450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c010760 con 0x7fefa4107420 2024-03-26T08:41:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.955+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa419fdf0 con 0x7fefa4107420 2024-03-26T08:41:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.956+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa40fd6c0 con 0x7fefa4107420 2024-03-26T08:41:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.957+0000 7fef9a7fc640 1 -- 172.21.15.7:0/288585450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fef9c005020 con 0x7fefa4107420 2024-03-26T08:41:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.957+0000 7fef9a7fc640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fef900a2800 0x7fef900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:41:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.958+0000 7fefa921a640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fef900a2800 0x7fef900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:41:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.958+0000 7fef9a7fc640 1 -- 172.21.15.7:0/288585450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fef9c0cefd0 con 0x7fefa4107420 2024-03-26T08:41:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.958+0000 7fefa921a640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fef900a2800 0x7fef900a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fefa002c7b0 tx=0x7fefa0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:41:59.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:41:59.962+0000 7fef9a7fc640 1 -- 172.21.15.7:0/288585450 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef9c09c070 con 0x7fefa4107420 2024-03-26T08:42:00.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.147+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefa40622a0 con 0x7fefa4107420 2024-03-26T08:42:00.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.148+0000 7fef9a7fc640 1 -- 172.21.15.7:0/288585450 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7fef9c09b970 con 0x7fefa4107420 2024-03-26T08:42:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:00.140 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:42:00.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fef900a2800 msgr2=0x7fef900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fef900a2800 0x7fef900a4cc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fefa002c7b0 tx=0x7fefa0005fd0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa4107420 msgr2=0x7fefa419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa4107420 0x7fefa419ef90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00eb10 tx=0x7fef9c00c2d0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 shutdown_connections 2024-03-26T08:42:00.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fef900a2800 0x7fef900a4cc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefa4107420 0x7fefa419ef90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefa4104af0 0x7fefa4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 --2- 172.21.15.7:0/288585450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fefa41021c0 0x7fefa41979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.150+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 >> 172.21.15.7:0/288585450 conn(0x7fefa40fbbd0 msgr2=0x7fefa4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:00.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.151+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 shutdown_connections 2024-03-26T08:42:00.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.151+0000 7fefab47e640 1 -- 172.21.15.7:0/288585450 wait complete. 2024-03-26T08:42:00.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:40:08.100906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:40:08.784972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:42:00.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:42:00.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.268+0000 7f5665463640 1 -- 172.21.15.7:0/2859431035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 msgr2=0x7f5660109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.268+0000 7f5665463640 1 --2- 172.21.15.7:0/2859431035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 0x7f5660109810 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5650009ca0 tx=0x7f565002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.268+0000 7f5665463640 1 -- 172.21.15.7:0/2859431035 shutdown_connections 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.268+0000 7f5665463640 1 --2- 172.21.15.7:0/2859431035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 0x7f5660109810 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.268+0000 7f5665463640 1 --2- 172.21.15.7:0/2859431035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660104af0 0x7f5660106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.268+0000 7f5665463640 1 --2- 172.21.15.7:0/2859431035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56601021c0 0x7f56601045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.268+0000 7f5665463640 1 -- 172.21.15.7:0/2859431035 >> 172.21.15.7:0/2859431035 conn(0x7f56600fbb30 msgr2=0x7f56600fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.268+0000 7f5665463640 1 -- 172.21.15.7:0/2859431035 shutdown_connections 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f5665463640 1 -- 172.21.15.7:0/2859431035 wait complete. 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f5665463640 1 Processor -- start 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f5665463640 1 -- start start 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f5665463640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56601021c0 0x7f5660101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f5665463640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660104af0 0x7f5660100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f5665463640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 0x7f56601006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f5665463640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566010d360 con 0x7f5660107420 2024-03-26T08:42:00.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f5665463640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566010d1e0 con 0x7f5660104af0 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f5665463640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566010d4e0 con 0x7f56601021c0 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.269+0000 7f565f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 0x7f56601006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 0x7f56601006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46440/0 (socket says 172.21.15.7:46440) 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565f7fe640 1 -- 172.21.15.7:0/2590756664 learned_addr learned my addr 172.21.15.7:0/2590756664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565effd640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56601021c0 0x7f5660101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565e7fc640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660104af0 0x7f5660100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565f7fe640 1 -- 172.21.15.7:0/2590756664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56601021c0 msgr2=0x7f5660101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565f7fe640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56601021c0 0x7f5660101ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565f7fe640 1 -- 172.21.15.7:0/2590756664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660104af0 msgr2=0x7f5660100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565f7fe640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660104af0 0x7f5660100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565f7fe640 1 -- 172.21.15.7:0/2590756664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5660100f70 con 0x7f5660107420 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565effd640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56601021c0 0x7f5660101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f565f7fe640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 0x7f56601006b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5650037040 tx=0x7f5650032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:00.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f5657fff640 1 -- 172.21.15.7:0/2590756664 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650041020 con 0x7f5660107420 2024-03-26T08:42:00.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56601a1ac0 con 0x7f5660107420 2024-03-26T08:42:00.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f5657fff640 1 -- 172.21.15.7:0/2590756664 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5650034cd0 con 0x7f5660107420 2024-03-26T08:42:00.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f5657fff640 1 -- 172.21.15.7:0/2590756664 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5650039040 con 0x7f5660107420 2024-03-26T08:42:00.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.270+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56601a1f80 con 0x7f5660107420 2024-03-26T08:42:00.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.271+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56600fd620 con 0x7f5660107420 2024-03-26T08:42:00.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.272+0000 7f5657fff640 1 -- 172.21.15.7:0/2590756664 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5650044020 con 0x7f5660107420 2024-03-26T08:42:00.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.273+0000 7f5657fff640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f564c0a2800 0x7f564c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.273+0000 7f565effd640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f564c0a2800 0x7f564c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.273+0000 7f5657fff640 1 -- 172.21.15.7:0/2590756664 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f56500f08c0 con 0x7f5660107420 2024-03-26T08:42:00.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.273+0000 7f565effd640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f564c0a2800 0x7f564c0a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5658009fc0 tx=0x7f5658005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:00.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.277+0000 7f5657fff640 1 -- 172.21.15.7:0/2590756664 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f565004b020 con 0x7f5660107420 2024-03-26T08:42:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.458+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56600622a0 con 0x7f5660107420 2024-03-26T08:42:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.459+0000 7f5657fff640 1 -- 172.21.15.7:0/2590756664 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56500b7e20 con 0x7f5660107420 2024-03-26T08:42:00.451 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:42:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f564c0a2800 msgr2=0x7f564c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f564c0a2800 0x7f564c0a4cc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5658009fc0 tx=0x7f5658005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 msgr2=0x7f56601006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 0x7f56601006b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5650037040 tx=0x7f5650032f40 comp rx=0 tx=0).stop 2024-03-26T08:42:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 shutdown_connections 2024-03-26T08:42:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f564c0a2800 0x7f564c0a4cc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5660107420 0x7f56601006b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660104af0 0x7f5660100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 --2- 172.21.15.7:0/2590756664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f56601021c0 0x7f5660101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.460+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 >> 172.21.15.7:0/2590756664 conn(0x7f56600fbb30 msgr2=0x7f5660102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.461+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 shutdown_connections 2024-03-26T08:42:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.461+0000 7f5665463640 1 -- 172.21.15.7:0/2590756664 wait complete. 2024-03-26T08:42:00.462 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:00 smithi007 ceph-mon[32462]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:42:00.462 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:00 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/288585450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:00.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:42:00.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:00 smithi016 ceph-mon[35851]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:42:00.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:00 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/288585450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:00.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:00 smithi044 ceph-mon[36461]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:42:00.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:00 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/288585450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/1990783335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f44103260 msgr2=0x7f8f44109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/1990783335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f44103260 0x7f8f44109af0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009be0 tx=0x7f8f3c02bef0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/1990783335 shutdown_connections 2024-03-26T08:42:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/1990783335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f44103260 0x7f8f44109af0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/1990783335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f441028a0 0x7f8f44102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/1990783335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f441016b0 0x7f8f44101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/1990783335 >> 172.21.15.7:0/1990783335 conn(0x7f8f440fcc20 msgr2=0x7f8f440ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/1990783335 shutdown_connections 2024-03-26T08:42:00.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/1990783335 wait complete. 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.577+0000 7f8f4bbf4640 1 Processor -- start 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4bbf4640 1 -- start start 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4bbf4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f441016b0 0x7f8f441979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4bbf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f441028a0 0x7f8f44197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4bbf4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f44103260 0x7f8f4419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4bbf4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4410d330 con 0x7f8f441016b0 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4bbf4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4410d1b0 con 0x7f8f441028a0 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4bbf4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4410d4b0 con 0x7f8f44103260 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f49990640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f441016b0 0x7f8f441979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f49990640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f441016b0 0x7f8f441979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46452/0 (socket says 172.21.15.7:46452) 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f49990640 1 -- 172.21.15.7:0/3617872708 learned_addr learned my addr 172.21.15.7:0/3617872708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4918f640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f441028a0 0x7f8f44197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4a191640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f44103260 0x7f8f4419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f49990640 1 -- 172.21.15.7:0/3617872708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f44103260 msgr2=0x7f8f4419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f49990640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f44103260 0x7f8f4419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f49990640 1 -- 172.21.15.7:0/3617872708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f441028a0 msgr2=0x7f8f44197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f49990640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f441028a0 0x7f8f44197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f49990640 1 -- 172.21.15.7:0/3617872708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4419f5c0 con 0x7f8f441016b0 2024-03-26T08:42:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f49990640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f441016b0 0x7f8f441979a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000e9c0 tx=0x7f8f4000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f3affd640 1 -- 172.21.15.7:0/3617872708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000ccc0 con 0x7f8f441016b0 2024-03-26T08:42:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f3affd640 1 -- 172.21.15.7:0/3617872708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f400035f0 con 0x7f8f441016b0 2024-03-26T08:42:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4419f8b0 con 0x7f8f441016b0 2024-03-26T08:42:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.578+0000 7f8f3affd640 1 -- 172.21.15.7:0/3617872708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000ce60 con 0x7f8f441016b0 2024-03-26T08:42:00.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.579+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4419fdb0 con 0x7f8f441016b0 2024-03-26T08:42:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.579+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f441089d0 con 0x7f8f441016b0 2024-03-26T08:42:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.581+0000 7f8f3affd640 1 -- 172.21.15.7:0/3617872708 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8f400037a0 con 0x7f8f441016b0 2024-03-26T08:42:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.581+0000 7f8f3affd640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8f300a2800 0x7f8f300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.581+0000 7f8f3affd640 1 -- 172.21.15.7:0/3617872708 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8f400cee90 con 0x7f8f441016b0 2024-03-26T08:42:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.581+0000 7f8f4918f640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8f300a2800 0x7f8f300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.582+0000 7f8f4918f640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8f300a2800 0x7f8f300a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8f44198eb0 tx=0x7f8f34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:00.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.585+0000 7f8f3affd640 1 -- 172.21.15.7:0/3617872708 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f4009bf30 con 0x7f8f441016b0 2024-03-26T08:42:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.768+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f4402d220 con 0x7f8f441016b0 2024-03-26T08:42:00.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.768+0000 7f8f3affd640 1 -- 172.21.15.7:0/3617872708 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f4009b830 con 0x7f8f441016b0 2024-03-26T08:42:00.760 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-26T08:42:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8f300a2800 msgr2=0x7f8f300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8f300a2800 0x7f8f300a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8f44198eb0 tx=0x7f8f34009260 comp rx=0 tx=0).stop 2024-03-26T08:42:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f441016b0 msgr2=0x7f8f441979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f441016b0 0x7f8f441979a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000e9c0 tx=0x7f8f4000edc0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 shutdown_connections 2024-03-26T08:42:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8f300a2800 0x7f8f300a4cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8f44103260 0x7f8f4419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f441028a0 0x7f8f44197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 --2- 172.21.15.7:0/3617872708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f441016b0 0x7f8f441979a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 >> 172.21.15.7:0/3617872708 conn(0x7f8f440fcc20 msgr2=0x7f8f44105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 shutdown_connections 2024-03-26T08:42:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.770+0000 7f8f4bbf4640 1 -- 172.21.15.7:0/3617872708 wait complete. 2024-03-26T08:42:00.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.886+0000 7fcce65f3640 1 -- 172.21.15.7:0/771395823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 msgr2=0x7fcce0104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.886+0000 7fcce65f3640 1 --2- 172.21.15.7:0/771395823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 0x7fcce0104530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009900 tx=0x7fccc802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.886+0000 7fcce65f3640 1 -- 172.21.15.7:0/771395823 shutdown_connections 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.886+0000 7fcce65f3640 1 --2- 172.21.15.7:0/771395823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce01073a0 0x7fcce01097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.886+0000 7fcce65f3640 1 --2- 172.21.15.7:0/771395823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce0104a70 0x7fcce0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.886+0000 7fcce65f3640 1 --2- 172.21.15.7:0/771395823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 0x7fcce0104530 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.886+0000 7fcce65f3640 1 -- 172.21.15.7:0/771395823 >> 172.21.15.7:0/771395823 conn(0x7fcce00fbb30 msgr2=0x7fcce00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.886+0000 7fcce65f3640 1 -- 172.21.15.7:0/771395823 shutdown_connections 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.886+0000 7fcce65f3640 1 -- 172.21.15.7:0/771395823 wait complete. 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fcce65f3640 1 Processor -- start 2024-03-26T08:42:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fcce65f3640 1 -- start start 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fcce65f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 0x7fcce0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fcce65f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce0104a70 0x7fcce0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fcce65f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce01073a0 0x7fcce019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fcce65f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce010d370 con 0x7fcce0102140 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fcce65f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce010d1f0 con 0x7fcce01073a0 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fcce65f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce010d4f0 con 0x7fcce0104a70 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fccdffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 0x7fcce0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fccdffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 0x7fcce0197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46472/0 (socket says 172.21.15.7:46472) 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fccdffff640 1 -- 172.21.15.7:0/238374677 learned_addr learned my addr 172.21.15.7:0/238374677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fccdf7fe640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce0104a70 0x7fcce0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.887+0000 7fcce4b90640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce01073a0 0x7fcce019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fccdffff640 1 -- 172.21.15.7:0/238374677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce0104a70 msgr2=0x7fcce0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fccdffff640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce0104a70 0x7fcce0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fccdffff640 1 -- 172.21.15.7:0/238374677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce01073a0 msgr2=0x7fcce019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fccdffff640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce01073a0 0x7fcce019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fccdffff640 1 -- 172.21.15.7:0/238374677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce019f630 con 0x7fcce0102140 2024-03-26T08:42:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fcce4b90640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce01073a0 0x7fcce019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fccdffff640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 0x7fcce0197a10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009230 tx=0x7fccc802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fccdd7fa640 1 -- 172.21.15.7:0/238374677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8041070 con 0x7fcce0102140 2024-03-26T08:42:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fccdd7fa640 1 -- 172.21.15.7:0/238374677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccc8004450 con 0x7fcce0102140 2024-03-26T08:42:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce019f8c0 con 0x7fcce0102140 2024-03-26T08:42:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fccdd7fa640 1 -- 172.21.15.7:0/238374677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8003e20 con 0x7fcce0102140 2024-03-26T08:42:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.888+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce019fdb0 con 0x7fcce0102140 2024-03-26T08:42:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.889+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce00fd620 con 0x7fcce0102140 2024-03-26T08:42:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.890+0000 7fccdd7fa640 1 -- 172.21.15.7:0/238374677 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fccc8044070 con 0x7fcce0102140 2024-03-26T08:42:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.891+0000 7fccdd7fa640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcccc0a2800 0x7fcccc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.891+0000 7fccdf7fe640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcccc0a2800 0x7fcccc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.891+0000 7fccdd7fa640 1 -- 172.21.15.7:0/238374677 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fccc80f0600 con 0x7fcce0102140 2024-03-26T08:42:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.892+0000 7fccdf7fe640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcccc0a2800 0x7fcccc0a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcce0198f20 tx=0x7fccd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:00.896+0000 7fccdd7fa640 1 -- 172.21.15.7:0/238374677 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fccc80bd050 con 0x7fcce0102140 2024-03-26T08:42:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.077+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcce002d290 con 0x7fcce0102140 2024-03-26T08:42:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.078+0000 7fccdd7fa640 1 -- 172.21.15.7:0/238374677 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fccc80c10f0 con 0x7fcce0102140 2024-03-26T08:42:01.069 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-26T08:42:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcccc0a2800 msgr2=0x7fcccc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcccc0a2800 0x7fcccc0a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcce0198f20 tx=0x7fccd0005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 msgr2=0x7fcce0197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 0x7fcce0197a10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009230 tx=0x7fccc802c650 comp rx=0 tx=0).stop 2024-03-26T08:42:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 shutdown_connections 2024-03-26T08:42:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcccc0a2800 0x7fcccc0a4cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce01073a0 0x7fcce019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce0104a70 0x7fcce0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 --2- 172.21.15.7:0/238374677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce0102140 0x7fcce0197a10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 >> 172.21.15.7:0/238374677 conn(0x7fcce00fbb30 msgr2=0x7fcce0102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 shutdown_connections 2024-03-26T08:42:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.080+0000 7fcce65f3640 1 -- 172.21.15.7:0/238374677 wait complete. 2024-03-26T08:42:01.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:42:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 -- 172.21.15.7:0/4030699322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8101830 msgr2=0x7f02e8101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 --2- 172.21.15.7:0/4030699322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8101830 0x7f02e8101cb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009900 tx=0x7f02d802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 -- 172.21.15.7:0/4030699322 shutdown_connections 2024-03-26T08:42:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 --2- 172.21.15.7:0/4030699322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e81021f0 0x7f02e8108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 --2- 172.21.15.7:0/4030699322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e8101830 0x7f02e8101cb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 --2- 172.21.15.7:0/4030699322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8100640 0x7f02e8100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 -- 172.21.15.7:0/4030699322 >> 172.21.15.7:0/4030699322 conn(0x7f02e80fbbb0 msgr2=0x7f02e80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 -- 172.21.15.7:0/4030699322 shutdown_connections 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 -- 172.21.15.7:0/4030699322 wait complete. 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 Processor -- start 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.193+0000 7f02efd53640 1 -- start start 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02efd53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8100640 0x7f02e81979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02efd53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 0x7f02e8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02efd53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81021f0 0x7f02e819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02efd53640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e810c250 con 0x7f02e81021f0 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02efd53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e810c0d0 con 0x7f02e8100640 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02efd53640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e810c3d0 con 0x7f02e8101830 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02ee2f0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81021f0 0x7f02e819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02edaef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8100640 0x7f02e81979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02edaef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8100640 0x7f02e81979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41976/0 (socket says 172.21.15.7:41976) 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02ed2ee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 0x7f02e8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02edaef640 1 -- 172.21.15.7:0/2824067194 learned_addr learned my addr 172.21.15.7:0/2824067194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02ee2f0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81021f0 0x7f02e819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46484/0 (socket says 172.21.15.7:46484) 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02edaef640 1 -- 172.21.15.7:0/2824067194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 msgr2=0x7f02e8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02edaef640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 0x7f02e8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02edaef640 1 -- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81021f0 msgr2=0x7f02e819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02edaef640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81021f0 0x7f02e819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02edaef640 1 -- 172.21.15.7:0/2824067194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e819f5d0 con 0x7f02e8100640 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02ee2f0640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81021f0 0x7f02e819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02edaef640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8100640 0x7f02e81979b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9c0 tx=0x7f02e400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02deffd640 1 -- 172.21.15.7:0/2824067194 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e400cde0 con 0x7f02e8100640 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02deffd640 1 -- 172.21.15.7:0/2824067194 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02e4010910 con 0x7f02e8100640 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e819f8c0 con 0x7f02e8100640 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02deffd640 1 -- 172.21.15.7:0/2824067194 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4018a00 con 0x7f02e8100640 2024-03-26T08:42:01.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.194+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e819fe30 con 0x7f02e8100640 2024-03-26T08:42:01.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.195+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e81a0060 con 0x7f02e8100640 2024-03-26T08:42:01.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.196+0000 7f02deffd640 1 -- 172.21.15.7:0/2824067194 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f02e401f020 con 0x7f02e8100640 2024-03-26T08:42:01.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.197+0000 7f02deffd640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02d40a2880 0x7f02d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.197+0000 7f02ed2ee640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02d40a2880 0x7f02d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.197+0000 7f02deffd640 1 -- 172.21.15.7:0/2824067194 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f02e40cedb0 con 0x7f02e8100640 2024-03-26T08:42:01.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.197+0000 7f02ed2ee640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02d40a2880 0x7f02d40a4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f02d802c7b0 tx=0x7f02d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:01.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.201+0000 7f02deffd640 1 -- 172.21.15.7:0/2824067194 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02e409be10 con 0x7f02e8100640 2024-03-26T08:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.426+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02e81a0230 con 0x7f02e8100640 2024-03-26T08:42:01.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.427+0000 7f02deffd640 1 -- 172.21.15.7:0/2824067194 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f02e409b710 con 0x7f02e8100640 2024-03-26T08:42:01.419 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:01.419 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:40:15.277649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:42:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-03-26T08:42:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02d40a2880 msgr2=0x7f02d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02d40a2880 0x7f02d40a4d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f02d802c7b0 tx=0x7f02d8005fd0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8100640 msgr2=0x7f02e81979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8100640 0x7f02e81979b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9c0 tx=0x7f02e400ee80 comp rx=0 tx=0).stop 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 shutdown_connections 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f02d40a2880 0x7f02d40a4d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02e81021f0 0x7f02e819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f02e8101830 0x7f02e8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 --2- 172.21.15.7:0/2824067194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e8100640 0x7f02e81979b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 >> 172.21.15.7:0/2824067194 conn(0x7f02e80fbbb0 msgr2=0x7f02e80fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 shutdown_connections 2024-03-26T08:42:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.429+0000 7f02efd53640 1 -- 172.21.15.7:0/2824067194 wait complete. 2024-03-26T08:42:01.430 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:01 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2590756664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:01.430 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:01 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3617872708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:01.430 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:01 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/238374677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:01.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:42:01.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:01 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2590756664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:01.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:01 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3617872708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:01.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:01 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/238374677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 -- 172.21.15.7:0/1340854624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810100640 msgr2=0x7f0810100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 --2- 172.21.15.7:0/1340854624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810100640 0x7f0810100a60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0808009900 tx=0x7f080802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 -- 172.21.15.7:0/1340854624 shutdown_connections 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 --2- 172.21.15.7:0/1340854624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08101021f0 0x7f0810108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 --2- 172.21.15.7:0/1340854624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0810101830 0x7f0810101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 --2- 172.21.15.7:0/1340854624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810100640 0x7f0810100a60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 -- 172.21.15.7:0/1340854624 >> 172.21.15.7:0/1340854624 conn(0x7f08100fbbb0 msgr2=0x7f08100fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 -- 172.21.15.7:0/1340854624 shutdown_connections 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 -- 172.21.15.7:0/1340854624 wait complete. 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 Processor -- start 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 -- start start 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0810100640 0x7f081006a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810101830 0x7f081019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08101021f0 0x7f081019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081010c240 con 0x7f0810101830 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081010c0c0 con 0x7f0810100640 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.544+0000 7f08180bf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081010c3c0 con 0x7f08101021f0 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081565a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810101830 0x7f081019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081565a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810101830 0x7f081019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46504/0 (socket says 172.21.15.7:46504) 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081565a640 1 -- 172.21.15.7:0/573476189 learned_addr learned my addr 172.21.15.7:0/573476189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f0815e5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0810100640 0x7f081006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081665c640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08101021f0 0x7f081019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081565a640 1 -- 172.21.15.7:0/573476189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08101021f0 msgr2=0x7f081019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081565a640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08101021f0 0x7f081019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081565a640 1 -- 172.21.15.7:0/573476189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0810100640 msgr2=0x7f081006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081565a640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0810100640 0x7f081006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081565a640 1 -- 172.21.15.7:0/573476189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081019f5e0 con 0x7f0810101830 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f0815e5b640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0810100640 0x7f081006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:01.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f081565a640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810101830 0x7f081019c6e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f080400e9f0 tx=0x7f080400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f080effd640 1 -- 172.21.15.7:0/573476189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080400cd80 con 0x7f0810101830 2024-03-26T08:42:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081019f8d0 con 0x7f0810101830 2024-03-26T08:42:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f080effd640 1 -- 172.21.15.7:0/573476189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0804010910 con 0x7f0810101830 2024-03-26T08:42:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f080effd640 1 -- 172.21.15.7:0/573476189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0804018990 con 0x7f0810101830 2024-03-26T08:42:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.545+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081019fe20 con 0x7f0810101830 2024-03-26T08:42:01.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.546+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0810107980 con 0x7f0810101830 2024-03-26T08:42:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.547+0000 7f080effd640 1 -- 172.21.15.7:0/573476189 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0804010ac0 con 0x7f0810101830 2024-03-26T08:42:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.548+0000 7f080effd640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07f80a2800 0x7f07f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.548+0000 7f0815e5b640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07f80a2800 0x7f07f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.548+0000 7f080effd640 1 -- 172.21.15.7:0/573476189 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08040cedf0 con 0x7f0810101830 2024-03-26T08:42:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.548+0000 7f0815e5b640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07f80a2800 0x7f07f80a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0808000c00 tx=0x7f0808005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:01.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.552+0000 7f080effd640 1 -- 172.21.15.7:0/573476189 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f080409b800 con 0x7f0810101830 2024-03-26T08:42:01.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:01 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2590756664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:01.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:01 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3617872708' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:01.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:01 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/238374677' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.736+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08100622a0 con 0x7f0810101830 2024-03-26T08:42:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.738+0000 7f080effd640 1 -- 172.21.15.7:0/573476189 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7f080409f8a0 con 0x7f0810101830 2024-03-26T08:42:01.729 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:01.729 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:42:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07f80a2800 msgr2=0x7f07f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07f80a2800 0x7f07f80a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0808000c00 tx=0x7f0808005fd0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810101830 msgr2=0x7f081019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810101830 0x7f081019c6e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f080400e9f0 tx=0x7f080400eeb0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 shutdown_connections 2024-03-26T08:42:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07f80a2800 0x7f07f80a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f08101021f0 0x7f081019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810101830 0x7f081019c6e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 --2- 172.21.15.7:0/573476189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0810100640 0x7f081006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 >> 172.21.15.7:0/573476189 conn(0x7f08100fbbb0 msgr2=0x7f0810104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 shutdown_connections 2024-03-26T08:42:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.740+0000 7f08180bf640 1 -- 172.21.15.7:0/573476189 wait complete. 2024-03-26T08:42:01.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:40:08.100906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:40:08.784972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:42:01.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:42:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 -- 172.21.15.7:0/2148378260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3688107430 msgr2=0x7f3688109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 --2- 172.21.15.7:0/2148378260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3688107430 0x7f3688109820 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3680009ca0 tx=0x7f368002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 -- 172.21.15.7:0/2148378260 shutdown_connections 2024-03-26T08:42:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 --2- 172.21.15.7:0/2148378260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3688107430 0x7f3688109820 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 --2- 172.21.15.7:0/2148378260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3688104b00 0x7f3688106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 --2- 172.21.15.7:0/2148378260 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36881021d0 0x7f36881045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 -- 172.21.15.7:0/2148378260 >> 172.21.15.7:0/2148378260 conn(0x7f36880fbb20 msgr2=0x7f36880fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 -- 172.21.15.7:0/2148378260 shutdown_connections 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 -- 172.21.15.7:0/2148378260 wait complete. 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 Processor -- start 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.857+0000 7f368e0de640 1 -- start start 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f368e0de640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36881021d0 0x7f36881979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f368e0de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3688104b00 0x7f3688197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f368e0de640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3688107430 0x7f368819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f368e0de640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368810d350 con 0x7f36881021d0 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f368e0de640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368810d1d0 con 0x7f3688104b00 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f368e0de640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368810d4d0 con 0x7f3688107430 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f36877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36881021d0 0x7f36881979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f36877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36881021d0 0x7f36881979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46534/0 (socket says 172.21.15.7:46534) 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f36877fe640 1 -- 172.21.15.7:0/1692328024 learned_addr learned my addr 172.21.15.7:0/1692328024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f3686ffd640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3688104b00 0x7f3688197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f3687fff640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3688107430 0x7f368819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f36877fe640 1 -- 172.21.15.7:0/1692328024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3688107430 msgr2=0x7f368819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f36877fe640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3688107430 0x7f368819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f36877fe640 1 -- 172.21.15.7:0/1692328024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3688104b00 msgr2=0x7f3688197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f36877fe640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3688104b00 0x7f3688197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f36877fe640 1 -- 172.21.15.7:0/1692328024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368819f610 con 0x7f36881021d0 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f36877fe640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36881021d0 0x7f36881979f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f367800e9f0 tx=0x7f367800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f3684ff9640 1 -- 172.21.15.7:0/1692328024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800cd80 con 0x7f36881021d0 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f3684ff9640 1 -- 172.21.15.7:0/1692328024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3678005500 con 0x7f36881021d0 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368819f900 con 0x7f36881021d0 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f3684ff9640 1 -- 172.21.15.7:0/1692328024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678010690 con 0x7f36881021d0 2024-03-26T08:42:01.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.858+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368819fe00 con 0x7f36881021d0 2024-03-26T08:42:01.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.859+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36880fd610 con 0x7f36881021d0 2024-03-26T08:42:01.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.860+0000 7f3684ff9640 1 -- 172.21.15.7:0/1692328024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f36780046c0 con 0x7f36881021d0 2024-03-26T08:42:01.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.861+0000 7f3684ff9640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2800 0x7f36700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:01.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.861+0000 7f3686ffd640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2800 0x7f36700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:01.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.861+0000 7f3684ff9640 1 -- 172.21.15.7:0/1692328024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f36780cee50 con 0x7f36881021d0 2024-03-26T08:42:01.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.861+0000 7f3686ffd640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2800 0x7f36700a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3688198f00 tx=0x7f367c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:01.865+0000 7f3684ff9640 1 -- 172.21.15.7:0/1692328024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f367809bef0 con 0x7f36881021d0 2024-03-26T08:42:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.051+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36880622a0 con 0x7f36881021d0 2024-03-26T08:42:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.051+0000 7f3684ff9640 1 -- 172.21.15.7:0/1692328024 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f367809b7f0 con 0x7f36881021d0 2024-03-26T08:42:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2800 msgr2=0x7f36700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2800 0x7f36700a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3688198f00 tx=0x7f367c005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36881021d0 msgr2=0x7f36881979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36881021d0 0x7f36881979f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f367800e9f0 tx=0x7f367800edf0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 shutdown_connections 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2800 0x7f36700a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3688107430 0x7f368819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3688104b00 0x7f3688197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 --2- 172.21.15.7:0/1692328024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36881021d0 0x7f36881979f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 >> 172.21.15.7:0/1692328024 conn(0x7f36880fbb20 msgr2=0x7f3688102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 shutdown_connections 2024-03-26T08:42:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.053+0000 7f368e0de640 1 -- 172.21.15.7:0/1692328024 wait complete. 2024-03-26T08:42:02.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:42:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 -- 172.21.15.7:0/747805504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c107380 msgr2=0x7fcd9c1097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 --2- 172.21.15.7:0/747805504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c107380 0x7fcd9c1097d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009900 tx=0x7fcd8402bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 -- 172.21.15.7:0/747805504 shutdown_connections 2024-03-26T08:42:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 --2- 172.21.15.7:0/747805504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c107380 0x7fcd9c1097d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 --2- 172.21.15.7:0/747805504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c104a50 0x7fcd9c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 --2- 172.21.15.7:0/747805504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9c102120 0x7fcd9c104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 -- 172.21.15.7:0/747805504 >> 172.21.15.7:0/747805504 conn(0x7fcd9c0fbb30 msgr2=0x7fcd9c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 -- 172.21.15.7:0/747805504 shutdown_connections 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 -- 172.21.15.7:0/747805504 wait complete. 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 Processor -- start 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 -- start start 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9c102120 0x7fcd9c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c104a50 0x7fcd9c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c107380 0x7fcd9c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c10d390 con 0x7fcd9c104a50 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c10d210 con 0x7fcd9c102120 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcda30dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c10d510 con 0x7fcd9c107380 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcd9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c104a50 0x7fcd9c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcd9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c104a50 0x7fcd9c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46548/0 (socket says 172.21.15.7:46548) 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.170+0000 7fcd9bfff640 1 -- 172.21.15.7:0/23864131 learned_addr learned my addr 172.21.15.7:0/23864131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcda0e78640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9c102120 0x7fcd9c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcda1679640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c107380 0x7fcd9c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcd9bfff640 1 -- 172.21.15.7:0/23864131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c107380 msgr2=0x7fcd9c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcd9bfff640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c107380 0x7fcd9c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcd9bfff640 1 -- 172.21.15.7:0/23864131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9c102120 msgr2=0x7fcd9c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcd9bfff640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9c102120 0x7fcd9c1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcd9bfff640 1 -- 172.21.15.7:0/23864131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c19f5f0 con 0x7fcd9c104a50 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcda0e78640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9c102120 0x7fcd9c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcd9bfff640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c104a50 0x7fcd9c197f10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00ebc0 tx=0x7fcd8c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcd99ffb640 1 -- 172.21.15.7:0/23864131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c004d20 con 0x7fcd9c104a50 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcd99ffb640 1 -- 172.21.15.7:0/23864131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd8c005500 con 0x7fcd9c104a50 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c19f8e0 con 0x7fcd9c104a50 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcd99ffb640 1 -- 172.21.15.7:0/23864131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c010890 con 0x7fcd9c104a50 2024-03-26T08:42:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.171+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c19fde0 con 0x7fcd9c104a50 2024-03-26T08:42:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.172+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c0fd620 con 0x7fcd9c104a50 2024-03-26T08:42:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.173+0000 7fcd99ffb640 1 -- 172.21.15.7:0/23864131 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcd8c005020 con 0x7fcd9c104a50 2024-03-26T08:42:02.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.174+0000 7fcd99ffb640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd880a2800 0x7fcd880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.174+0000 7fcda0e78640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd880a2800 0x7fcd880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.174+0000 7fcd99ffb640 1 -- 172.21.15.7:0/23864131 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcd8c0cf280 con 0x7fcd9c104a50 2024-03-26T08:42:02.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.174+0000 7fcda0e78640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd880a2800 0x7fcd880a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009fc0 tx=0x7fcd94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:02.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.178+0000 7fcd99ffb640 1 -- 172.21.15.7:0/23864131 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcd8c09c2e0 con 0x7fcd9c104a50 2024-03-26T08:42:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.362+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd9c0622a0 con 0x7fcd9c104a50 2024-03-26T08:42:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.362+0000 7fcd99ffb640 1 -- 172.21.15.7:0/23864131 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd8c09bbe0 con 0x7fcd9c104a50 2024-03-26T08:42:02.354 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd880a2800 msgr2=0x7fcd880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd880a2800 0x7fcd880a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009fc0 tx=0x7fcd94005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c104a50 msgr2=0x7fcd9c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c104a50 0x7fcd9c197f10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00ebc0 tx=0x7fcd8c00c6a0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 shutdown_connections 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd880a2800 0x7fcd880a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd9c107380 0x7fcd9c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c104a50 0x7fcd9c197f10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 --2- 172.21.15.7:0/23864131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd9c102120 0x7fcd9c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 >> 172.21.15.7:0/23864131 conn(0x7fcd9c0fbb30 msgr2=0x7fcd9c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 shutdown_connections 2024-03-26T08:42:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.364+0000 7fcda30dc640 1 -- 172.21.15.7:0/23864131 wait complete. 2024-03-26T08:42:02.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:02 smithi007 ceph-mon[32462]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:42:02.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:02 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2824067194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:02.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:02 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/573476189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:02.365 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:02 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1692328024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:02.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:42:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 -- 172.21.15.7:0/49473025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c1028a0 msgr2=0x7fb27c102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 --2- 172.21.15.7:0/49473025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c1028a0 0x7fb27c102d20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb270009900 tx=0x7fb27002bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 -- 172.21.15.7:0/49473025 shutdown_connections 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 --2- 172.21.15.7:0/49473025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c103260 0x7fb27c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 --2- 172.21.15.7:0/49473025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c1028a0 0x7fb27c102d20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 --2- 172.21.15.7:0/49473025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb27c1016b0 0x7fb27c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 -- 172.21.15.7:0/49473025 >> 172.21.15.7:0/49473025 conn(0x7fb27c0fcc20 msgr2=0x7fb27c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 -- 172.21.15.7:0/49473025 shutdown_connections 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 -- 172.21.15.7:0/49473025 wait complete. 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.481+0000 7fb281af8640 1 Processor -- start 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb281af8640 1 -- start start 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb281af8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb27c1016b0 0x7fb27c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb281af8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1028a0 0x7fb27c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb281af8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c103260 0x7fb27c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb281af8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c10d330 con 0x7fb27c103260 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb281af8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c10d1b0 con 0x7fb27c1016b0 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb281af8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c10d4b0 con 0x7fb27c1028a0 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb27c1016b0 0x7fb27c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c103260 0x7fb27c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1028a0 0x7fb27c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb27c1016b0 0x7fb27c1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42046/0 (socket says 172.21.15.7:42046) 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27b7fe640 1 -- 172.21.15.7:0/466135654 learned_addr learned my addr 172.21.15.7:0/466135654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27bfff640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c103260 0x7fb27c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46556/0 (socket says 172.21.15.7:46556) 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27bfff640 1 -- 172.21.15.7:0/466135654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1028a0 msgr2=0x7fb27c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27bfff640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1028a0 0x7fb27c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27bfff640 1 -- 172.21.15.7:0/466135654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb27c1016b0 msgr2=0x7fb27c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27bfff640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb27c1016b0 0x7fb27c1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27bfff640 1 -- 172.21.15.7:0/466135654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c19f5f0 con 0x7fb27c103260 2024-03-26T08:42:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27b7fe640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb27c1016b0 0x7fb27c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27affd640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1028a0 0x7fb27c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb27bfff640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c103260 0x7fb27c19ef80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb274009fd0 tx=0x7fb27400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.482+0000 7fb278ff9640 1 -- 172.21.15.7:0/466135654 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27400ce20 con 0x7fb27c103260 2024-03-26T08:42:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.483+0000 7fb278ff9640 1 -- 172.21.15.7:0/466135654 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb274005020 con 0x7fb27c103260 2024-03-26T08:42:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.483+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27c19f8e0 con 0x7fb27c103260 2024-03-26T08:42:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.483+0000 7fb278ff9640 1 -- 172.21.15.7:0/466135654 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274010720 con 0x7fb27c103260 2024-03-26T08:42:02.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.483+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27c19fe00 con 0x7fb27c103260 2024-03-26T08:42:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.484+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27c1089d0 con 0x7fb27c103260 2024-03-26T08:42:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.485+0000 7fb278ff9640 1 -- 172.21.15.7:0/466135654 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb274005540 con 0x7fb27c103260 2024-03-26T08:42:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.485+0000 7fb278ff9640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2640a2800 0x7fb2640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.485+0000 7fb27b7fe640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2640a2800 0x7fb2640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.485+0000 7fb278ff9640 1 -- 172.21.15.7:0/466135654 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb2740cefd0 con 0x7fb27c103260 2024-03-26T08:42:02.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.486+0000 7fb27b7fe640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2640a2800 0x7fb2640a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0058c0 tx=0x7fb26c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.490+0000 7fb278ff9640 1 -- 172.21.15.7:0/466135654 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb27409c110 con 0x7fb27c103260 2024-03-26T08:42:02.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:02 smithi016 ceph-mon[35851]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:42:02.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:02 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2824067194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:02.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:02 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/573476189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:02.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:02 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1692328024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:02.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:02 smithi044 ceph-mon[36461]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:42:02.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:02 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2824067194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:02.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:02 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/573476189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:02.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:02 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1692328024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:02.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.672+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb27c02d0c0 con 0x7fb27c103260 2024-03-26T08:42:02.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.673+0000 7fb278ff9640 1 -- 172.21.15.7:0/466135654 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb27409ba10 con 0x7fb27c103260 2024-03-26T08:42:02.664 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-26T08:42:02.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2640a2800 msgr2=0x7fb2640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2640a2800 0x7fb2640a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb26c0058c0 tx=0x7fb26c005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c103260 msgr2=0x7fb27c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c103260 0x7fb27c19ef80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb274009fd0 tx=0x7fb27400ef10 comp rx=0 tx=0).stop 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 shutdown_connections 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2640a2800 0x7fb2640a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb27c103260 0x7fb27c19ef80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb27c1028a0 0x7fb27c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 --2- 172.21.15.7:0/466135654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb27c1016b0 0x7fb27c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 >> 172.21.15.7:0/466135654 conn(0x7fb27c0fcc20 msgr2=0x7fb27c105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 shutdown_connections 2024-03-26T08:42:02.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.675+0000 7fb281af8640 1 -- 172.21.15.7:0/466135654 wait complete. 2024-03-26T08:42:02.677 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:42:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.788+0000 7f4de04b2640 1 -- 172.21.15.7:0/3802748768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd8102180 msgr2=0x7f4dd8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.788+0000 7f4de04b2640 1 --2- 172.21.15.7:0/3802748768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd8102180 0x7f4dd8108a10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009900 tx=0x7f4dc802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 -- 172.21.15.7:0/3802748768 shutdown_connections 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 --2- 172.21.15.7:0/3802748768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd8102180 0x7f4dd8108a10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 --2- 172.21.15.7:0/3802748768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dd81017c0 0x7f4dd8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 --2- 172.21.15.7:0/3802748768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd81005d0 0x7f4dd81009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 -- 172.21.15.7:0/3802748768 >> 172.21.15.7:0/3802748768 conn(0x7f4dd80fbb40 msgr2=0x7f4dd80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 -- 172.21.15.7:0/3802748768 shutdown_connections 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 -- 172.21.15.7:0/3802748768 wait complete. 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 Processor -- start 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 -- start start 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd81005d0 0x7f4dd81979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd81017c0 0x7f4dd8197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dd8102180 0x7f4dd819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd810c280 con 0x7f4dd81017c0 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd810c100 con 0x7f4dd8102180 2024-03-26T08:42:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4de04b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd810c400 con 0x7f4dd81005d0 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4ddda4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd81017c0 0x7f4dd8197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4ddda4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd81017c0 0x7f4dd8197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46570/0 (socket says 172.21.15.7:46570) 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4ddda4d640 1 -- 172.21.15.7:0/2156378097 learned_addr learned my addr 172.21.15.7:0/2156378097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4dde24e640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd81005d0 0x7f4dd81979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.789+0000 7f4ddea4f640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dd8102180 0x7f4dd819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4ddda4d640 1 -- 172.21.15.7:0/2156378097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd81005d0 msgr2=0x7f4dd81979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4ddda4d640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd81005d0 0x7f4dd81979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4ddda4d640 1 -- 172.21.15.7:0/2156378097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dd8102180 msgr2=0x7f4dd819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4ddda4d640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dd8102180 0x7f4dd819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4ddda4d640 1 -- 172.21.15.7:0/2156378097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd819f600 con 0x7f4dd81017c0 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4dde24e640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd81005d0 0x7f4dd81979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4ddda4d640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd81017c0 0x7f4dd8197f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e9f0 tx=0x7f4dcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4dd77fe640 1 -- 172.21.15.7:0/2156378097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc00cd80 con 0x7f4dd81017c0 2024-03-26T08:42:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4dd77fe640 1 -- 172.21.15.7:0/2156378097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dcc010910 con 0x7f4dd81017c0 2024-03-26T08:42:02.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd819f8f0 con 0x7f4dd81017c0 2024-03-26T08:42:02.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4dd77fe640 1 -- 172.21.15.7:0/2156378097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc018990 con 0x7f4dd81017c0 2024-03-26T08:42:02.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.790+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd819fe90 con 0x7f4dd81017c0 2024-03-26T08:42:02.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.791+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd8068520 con 0x7f4dd81017c0 2024-03-26T08:42:02.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.792+0000 7f4dd77fe640 1 -- 172.21.15.7:0/2156378097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4dcc010ac0 con 0x7f4dd81017c0 2024-03-26T08:42:02.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.793+0000 7f4dd77fe640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4dc00a2880 0x7f4dc00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:02.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.793+0000 7f4dde24e640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4dc00a2880 0x7f4dc00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:02.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.793+0000 7f4dd77fe640 1 -- 172.21.15.7:0/2156378097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4dcc0cf0f0 con 0x7f4dd81017c0 2024-03-26T08:42:02.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.793+0000 7f4dde24e640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4dc00a2880 0x7f4dc00a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8101620 tx=0x7f4dd0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:02.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:02.797+0000 7f4dd77fe640 1 -- 172.21.15.7:0/2156378097 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4dcc09bb40 con 0x7f4dd81017c0 2024-03-26T08:42:03.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.019+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dd81a0310 con 0x7f4dd81017c0 2024-03-26T08:42:03.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.020+0000 7f4dd77fe640 1 -- 172.21.15.7:0/2156378097 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f4dcc09fbe0 con 0x7f4dd81017c0 2024-03-26T08:42:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:03.011 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:40:15.277649+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:42:03.011 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-03-26T08:42:03.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4dc00a2880 msgr2=0x7f4dc00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4dc00a2880 0x7f4dc00a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8101620 tx=0x7f4dd0009500 comp rx=0 tx=0).stop 2024-03-26T08:42:03.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd81017c0 msgr2=0x7f4dd8197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd81017c0 0x7f4dd8197f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00e9f0 tx=0x7f4dcc00eeb0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 shutdown_connections 2024-03-26T08:42:03.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4dc00a2880 0x7f4dc00a4d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dd8102180 0x7f4dd819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dd81017c0 0x7f4dd8197f20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 --2- 172.21.15.7:0/2156378097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4dd81005d0 0x7f4dd81979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 >> 172.21.15.7:0/2156378097 conn(0x7f4dd80fbb40 msgr2=0x7f4dd80fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 shutdown_connections 2024-03-26T08:42:03.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.022+0000 7f4de04b2640 1 -- 172.21.15.7:0/2156378097 wait complete. 2024-03-26T08:42:03.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:42:03.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.135+0000 7f483ac0b640 1 -- 172.21.15.7:0/3148546714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 msgr2=0x7f4834108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.135+0000 7f483ac0b640 1 --2- 172.21.15.7:0/3148546714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 0x7f4834108a10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f482c009ca0 tx=0x7f482c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.135+0000 7f483ac0b640 1 -- 172.21.15.7:0/3148546714 shutdown_connections 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.135+0000 7f483ac0b640 1 --2- 172.21.15.7:0/3148546714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 0x7f4834108a10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.135+0000 7f483ac0b640 1 --2- 172.21.15.7:0/3148546714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48341017c0 0x7f4834101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.135+0000 7f483ac0b640 1 --2- 172.21.15.7:0/3148546714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48341005d0 0x7f48341009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.135+0000 7f483ac0b640 1 -- 172.21.15.7:0/3148546714 >> 172.21.15.7:0/3148546714 conn(0x7f48340fbb40 msgr2=0x7f48340fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.135+0000 7f483ac0b640 1 -- 172.21.15.7:0/3148546714 shutdown_connections 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f483ac0b640 1 -- 172.21.15.7:0/3148546714 wait complete. 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f483ac0b640 1 Processor -- start 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f483ac0b640 1 -- start start 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f483ac0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48341005d0 0x7f4834196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f483ac0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48341017c0 0x7f4834196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f483ac0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 0x7f483419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f483ac0b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483410c250 con 0x7f4834102180 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f483ac0b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483410c0d0 con 0x7f48341005d0 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f483ac0b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483410c3d0 con 0x7f48341017c0 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f48391a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 0x7f483419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f48389a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48341005d0 0x7f4834196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f48391a8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 0x7f483419de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46586/0 (socket says 172.21.15.7:46586) 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f48391a8640 1 -- 172.21.15.7:0/2171679712 learned_addr learned my addr 172.21.15.7:0/2171679712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.136+0000 7f4833fff640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48341017c0 0x7f4834196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f48391a8640 1 -- 172.21.15.7:0/2171679712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48341017c0 msgr2=0x7f4834196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f48391a8640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48341017c0 0x7f4834196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f48391a8640 1 -- 172.21.15.7:0/2171679712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48341005d0 msgr2=0x7f4834196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f48391a8640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48341005d0 0x7f4834196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f48391a8640 1 -- 172.21.15.7:0/2171679712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483419e4b0 con 0x7f4834102180 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f48389a7640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48341005d0 0x7f4834196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f4833fff640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48341017c0 0x7f4834196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f48391a8640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 0x7f483419de40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f482c037040 tx=0x7f482c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f4831ffb640 1 -- 172.21.15.7:0/2171679712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c041020 con 0x7f4834102180 2024-03-26T08:42:03.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f4831ffb640 1 -- 172.21.15.7:0/2171679712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f482c034cd0 con 0x7f4834102180 2024-03-26T08:42:03.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483419e740 con 0x7f4834102180 2024-03-26T08:42:03.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f4831ffb640 1 -- 172.21.15.7:0/2171679712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c039040 con 0x7f4834102180 2024-03-26T08:42:03.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.137+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483419ec80 con 0x7f4834102180 2024-03-26T08:42:03.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.138+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48341078f0 con 0x7f4834102180 2024-03-26T08:42:03.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.139+0000 7f4831ffb640 1 -- 172.21.15.7:0/2171679712 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f482c044020 con 0x7f4834102180 2024-03-26T08:42:03.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.140+0000 7f4831ffb640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f48240a2880 0x7f48240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.140+0000 7f48389a7640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f48240a2880 0x7f48240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.140+0000 7f4831ffb640 1 -- 172.21.15.7:0/2171679712 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f482c0f08c0 con 0x7f4834102180 2024-03-26T08:42:03.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.140+0000 7f48389a7640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f48240a2880 0x7f48240a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4834101620 tx=0x7f4828005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.144+0000 7f4831ffb640 1 -- 172.21.15.7:0/2171679712 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f482c04b020 con 0x7f4834102180 2024-03-26T08:42:03.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:03 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/23864131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:03.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:03 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/466135654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:03.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:03 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2156378097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:03.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.330+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48340622a0 con 0x7f4834102180 2024-03-26T08:42:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.332+0000 7f4831ffb640 1 -- 172.21.15.7:0/2171679712 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7f482c0b7e20 con 0x7f4834102180 2024-03-26T08:42:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:03.323 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:42:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f48240a2880 msgr2=0x7f48240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f48240a2880 0x7f48240a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4834101620 tx=0x7f4828005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 msgr2=0x7f483419de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 0x7f483419de40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f482c037040 tx=0x7f482c032f40 comp rx=0 tx=0).stop 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 shutdown_connections 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f48240a2880 0x7f48240a4d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4834102180 0x7f483419de40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f48341017c0 0x7f4834196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 --2- 172.21.15.7:0/2171679712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48341005d0 0x7f4834196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 >> 172.21.15.7:0/2171679712 conn(0x7f48340fbb40 msgr2=0x7f4834104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 shutdown_connections 2024-03-26T08:42:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.334+0000 7f483ac0b640 1 -- 172.21.15.7:0/2171679712 wait complete. 2024-03-26T08:42:03.337 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T08:40:09.583324+0000', 'modified': '2024-03-26T08:40:15.277649+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24512, 'mds_1': 24502, 'mds_2': 24478}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/1837116746', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 1837116746}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1837116746}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24466': {'gid': 24466, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.44:6835/2525817262', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6834', 'nonce': 2525817262}, {'type': 'v1', 'addr': '172.21.15.44:6835', 'nonce': 2525817262}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24476': {'gid': 24476, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/7790244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 7790244}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 7790244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24478': {'gid': 24478, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.44:6837/1604095881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 1604095881}, {'type': 'v1', 'addr': '172.21.15.44:6837', 'nonce': 1604095881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.44:6839/2441928071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6838', 'nonce': 2441928071}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 2441928071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6839/3011473533', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 3011473533}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 3011473533}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24478, 24502, 24512]} 2024-03-26T08:42:03.337 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-26T08:42:03.337 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.449+0000 7fd84993b640 1 -- 172.21.15.7:0/327098584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8441032a0 msgr2=0x7fd844109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.449+0000 7fd84993b640 1 --2- 172.21.15.7:0/327098584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8441032a0 0x7fd844109b30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd834009ca0 tx=0x7fd83402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.449+0000 7fd84993b640 1 -- 172.21.15.7:0/327098584 shutdown_connections 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.449+0000 7fd84993b640 1 --2- 172.21.15.7:0/327098584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8441032a0 0x7fd844109b30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.449+0000 7fd84993b640 1 --2- 172.21.15.7:0/327098584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8441028e0 0x7fd844102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.449+0000 7fd84993b640 1 --2- 172.21.15.7:0/327098584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8441016f0 0x7fd844101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.449+0000 7fd84993b640 1 -- 172.21.15.7:0/327098584 >> 172.21.15.7:0/327098584 conn(0x7fd8440fcc60 msgr2=0x7fd8440ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.449+0000 7fd84993b640 1 -- 172.21.15.7:0/327098584 shutdown_connections 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.449+0000 7fd84993b640 1 -- 172.21.15.7:0/327098584 wait complete. 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd84993b640 1 Processor -- start 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd84993b640 1 -- start start 2024-03-26T08:42:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd84993b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8441016f0 0x7fd8441979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd84993b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8441028e0 0x7fd844197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd84993b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8441032a0 0x7fd84419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd84993b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84410d370 con 0x7fd8441032a0 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd84993b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84410d1f0 con 0x7fd8441016f0 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd84993b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84410d4f0 con 0x7fd8441028e0 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8441032a0 0x7fd84419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8441032a0 0x7fd84419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46600/0 (socket says 172.21.15.7:46600) 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd8437fe640 1 -- 172.21.15.7:0/423599363 learned_addr learned my addr 172.21.15.7:0/423599363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd842ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8441016f0 0x7fd8441979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.450+0000 7fd8427fc640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8441028e0 0x7fd844197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd842ffd640 1 -- 172.21.15.7:0/423599363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8441028e0 msgr2=0x7fd844197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd842ffd640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8441028e0 0x7fd844197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd842ffd640 1 -- 172.21.15.7:0/423599363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8441032a0 msgr2=0x7fd84419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd842ffd640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8441032a0 0x7fd84419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd842ffd640 1 -- 172.21.15.7:0/423599363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84419f5d0 con 0x7fd8441016f0 2024-03-26T08:42:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd8427fc640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8441028e0 0x7fd844197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd842ffd640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8441016f0 0x7fd8441979b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e9c0 tx=0x7fd83c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd848939640 1 -- 172.21.15.7:0/423599363 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00ccc0 con 0x7fd8441016f0 2024-03-26T08:42:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd848939640 1 -- 172.21.15.7:0/423599363 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd83c0035f0 con 0x7fd8441016f0 2024-03-26T08:42:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd84993b640 1 -- 172.21.15.7:0/423599363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84419f8c0 con 0x7fd8441016f0 2024-03-26T08:42:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd848939640 1 -- 172.21.15.7:0/423599363 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00ce60 con 0x7fd8441016f0 2024-03-26T08:42:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.451+0000 7fd84993b640 1 -- 172.21.15.7:0/423599363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84419fdc0 con 0x7fd8441016f0 2024-03-26T08:42:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.453+0000 7fd848939640 1 -- 172.21.15.7:0/423599363 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd83c0037a0 con 0x7fd8441016f0 2024-03-26T08:42:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.453+0000 7fd84c606740 1 Processor -- start 2024-03-26T08:42:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.453+0000 7fd84c606740 1 -- start start 2024-03-26T08:42:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd84c606740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd84c606740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 0x557dfd1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd84c606740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd1a12a0 0x557dfd1a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd848939640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8300a2800 0x7fd8300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd84c606740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd100c20 con 0x557dfd19b740 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd84c606740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd0a9a40 con 0x557dfd19c130 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd84c606740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd0fccc0 con 0x557dfd1a12a0 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd19bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46610/0 (socket says 172.21.15.7:46610) 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 -- 172.21.15.7:0/2000531953 learned_addr learned my addr 172.21.15.7:0/2000531953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd842ffd640 1 --2- 172.21.15.7:0/2000531953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 0x557dfd1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8427fc640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8300a2800 0x7fd8300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 --2- 172.21.15.7:0/2000531953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd1a12a0 0x557dfd1a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd848939640 1 -- 172.21.15.7:0/423599363 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd83c0cefb0 con 0x7fd8441016f0 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 -- 172.21.15.7:0/2000531953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd1a12a0 msgr2=0x557dfd1a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 --2- 172.21.15.7:0/2000531953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd1a12a0 0x557dfd1a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 -- 172.21.15.7:0/2000531953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 msgr2=0x557dfd1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 --2- 172.21.15.7:0/2000531953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 0x557dfd1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 -- 172.21.15.7:0/2000531953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dfd0cbf90 con 0x557dfd19b740 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8427fc640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8300a2800 0x7fd8300a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0095f0 tx=0x7fd82c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd8437fe640 1 --2- 172.21.15.7:0/2000531953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd19bb60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd834009a60 tx=0x7fd834032380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423447416394264b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd839ffb640 1 -- 172.21.15.7:0/2000531953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834048070 con 0x557dfd19b740 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd839ffb640 1 -- 172.21.15.7:0/2000531953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd83402ce50 con 0x557dfd19b740 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.454+0000 7fd839ffb640 1 -- 172.21.15.7:0/2000531953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834039040 con 0x557dfd19b740 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 -- 172.21.15.7:0/2000531953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 msgr2=0x557dfd19bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 --2- 172.21.15.7:0/2000531953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd19bb60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd834009a60 tx=0x7fd834032380 comp rx=0 tx=0).stop 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 -- 172.21.15.7:0/2000531953 shutdown_connections 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 --2- 172.21.15.7:0/2000531953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd1a12a0 0x557dfd1a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 --2- 172.21.15.7:0/2000531953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 0x557dfd1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 --2- 172.21.15.7:0/2000531953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd19bb60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 -- 172.21.15.7:0/2000531953 >> 172.21.15.7:0/2000531953 conn(0x557dfd163a60 msgr2=0x557dfd1970b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 -- 172.21.15.7:0/2000531953 shutdown_connections 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 -- 172.21.15.7:0/2000531953 wait complete. 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.455+0000 7fd84c606740 1 -- start start 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd84c606740 1 Processor -- start 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd84c606740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19b740 0x557dfd1a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd84c606740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 0x557dfd1a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd84c606740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd84c606740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd1a41f0 con 0x557dfd1a12a0 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd84c606740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd0bce90 con 0x557dfd19c130 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd84c606740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd1a4370 con 0x557dfd19b740 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd84c606740 10 client.0 _ll_register_callbacks cb 0x557dfce14ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46614/0 (socket says 172.21.15.7:46614) 2024-03-26T08:42:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8437fe640 1 -- 172.21.15.7:0/3380019519 learned_addr learned my addr 172.21.15.7:0/3380019519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8427fc640 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 0x557dfd1a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd842ffd640 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19b740 0x557dfd1a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8437fe640 1 -- 172.21.15.7:0/3380019519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19b740 msgr2=0x557dfd1a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8437fe640 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19b740 0x557dfd1a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8437fe640 1 -- 172.21.15.7:0/3380019519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 msgr2=0x557dfd1a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8437fe640 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 0x557dfd1a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8437fe640 1 -- 172.21.15.7:0/3380019519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dfd0c99d0 con 0x557dfd1a12a0 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd8437fe640 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ad210 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd834037040 tx=0x7fd834005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3ea9c386feb81 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.456+0000 7fd822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd822ffd640 1 -- 172.21.15.7:0/3380019519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834052060 con 0x557dfd1a12a0 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd822ffd640 1 -- 172.21.15.7:0/3380019519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd834003b70 con 0x557dfd1a12a0 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd84c606740 1 -- 172.21.15.7:0/3380019519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd83402ce50 con 0x557dfd1a12a0 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd822ffd640 1 -- 172.21.15.7:0/3380019519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd834048070 con 0x557dfd1a12a0 2024-03-26T08:42:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd84c606740 10 client.15258 fetch_fsmap learned FSMap version 22 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd84c606740 1 -- 172.21.15.7:0/3380019519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557dfce152d0 con 0x557dfd1a12a0 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd822ffd640 1 -- 172.21.15.7:0/3380019519 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd83402ce50 con 0x557dfd1a12a0 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd822ffd640 1 -- 172.21.15.7:0/3380019519 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd834057070 con 0x557dfd1a12a0 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd84c606740 10 client.15258 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd84c606740 10 client.15258 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd84c606740 20 client.15258 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd84c606740 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557dfd1b1f10 0x557dfd1b6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd84c606740 4 client.15258 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd84c606740 1 -- 172.21.15.7:0/3380019519 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dfd10e160 con 0x557dfd1b1f10 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.457+0000 7fd842ffd640 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557dfd1b1f10 0x557dfd1b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.458+0000 7fd842ffd640 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557dfd1b1f10 0x557dfd1b6390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3095455c6ddcbd2 server_cookie=a73c8381cecc00aa in_seq=0 out_seq=0 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.458+0000 7fd822ffd640 10 client.15258 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd822ffd640 1 -- 172.21.15.7:0/3380019519 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557dfd10e160 con 0x557dfd1b1f10 2024-03-26T08:42:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd822ffd640 10 client.15258 handle_command_reply: tid=0 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 client.15258 shutdown 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 -- 172.21.15.7:0/3380019519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 msgr2=0x557dfd1ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ad210 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd834037040 tx=0x7fd834005bd0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 -- 172.21.15.7:0/3380019519 shutdown_connections 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557dfd1b1f10 0x557dfd1b6390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ad210 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19c130 0x557dfd1a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd822ffd640 0 client.15258 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 --2- 172.21.15.7:0/3380019519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19b740 0x557dfd1a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 -- 172.21.15.7:0/3380019519 >> 172.21.15.7:0/3380019519 conn(0x557dfd163a60 msgr2=0x557dfd1962f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 -- 172.21.15.7:0/3380019519 shutdown_connections 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 1 -- 172.21.15.7:0/3380019519 wait complete. 2024-03-26T08:42:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.459+0000 7fd84c606740 20 client.15258 trim_cache size 0 max 16384 2024-03-26T08:42:03.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd84c606740 1 Processor -- start 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd84c606740 1 -- start start 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd84c606740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd1a3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd84c606740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 0x557dfd1ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd84c606740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd1a12a0 0x557dfd19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd84c606740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd1b1730 con 0x557dfd19b740 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd84c606740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd1b1d90 con 0x557dfd1a12a0 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd84c606740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd1b23f0 con 0x557dfd19c130 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd842ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd1a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd842ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd1a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46616/0 (socket says 172.21.15.7:46616) 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd842ffd640 1 -- 172.21.15.7:0/2937409080 learned_addr learned my addr 172.21.15.7:0/2937409080 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd8427fc640 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 0x557dfd1ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd8437fe640 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd1a12a0 0x557dfd19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd842ffd640 1 -- 172.21.15.7:0/2937409080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 msgr2=0x557dfd1ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd842ffd640 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 0x557dfd1ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd842ffd640 1 -- 172.21.15.7:0/2937409080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd1a12a0 msgr2=0x557dfd19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd842ffd640 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd1a12a0 0x557dfd19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.476+0000 7fd842ffd640 1 -- 172.21.15.7:0/2937409080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dfcedf6b0 con 0x557dfd19b740 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd8437fe640 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd1a12a0 0x557dfd19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd8427fc640 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 0x557dfd1ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd842ffd640 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd1a3690 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd83c09b610 tx=0x7fd83c06d590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d76bcae66e18e78 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd8397fa640 1 -- 172.21.15.7:0/2937409080 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c09c010 con 0x557dfd19b740 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd8397fa640 1 -- 172.21.15.7:0/2937409080 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd83c09b910 con 0x557dfd19b740 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd8397fa640 1 -- 172.21.15.7:0/2937409080 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c097f80 con 0x557dfd19b740 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd84c606740 1 -- 172.21.15.7:0/2937409080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 msgr2=0x557dfd1a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd84c606740 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd1a3690 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd83c09b610 tx=0x7fd83c06d590 comp rx=0 tx=0).stop 2024-03-26T08:42:03.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd84c606740 1 -- 172.21.15.7:0/2937409080 shutdown_connections 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd84c606740 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd1a12a0 0x557dfd19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd84c606740 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 0x557dfd1ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd84c606740 1 --2- 172.21.15.7:0/2937409080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd19b740 0x557dfd1a3690 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd84c606740 1 -- 172.21.15.7:0/2937409080 >> 172.21.15.7:0/2937409080 conn(0x557dfd163a60 msgr2=0x557dfd19a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd84c606740 1 -- 172.21.15.7:0/2937409080 shutdown_connections 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.477+0000 7fd84c606740 1 -- 172.21.15.7:0/2937409080 wait complete. 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd84c606740 1 -- start start 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd84c606740 1 Processor -- start 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd84c606740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19b740 0x557dfd1a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd84c606740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 0x557dfd1a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd84c606740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd84c606740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd1b30b0 con 0x557dfd1a12a0 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd84c606740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd1b2a50 con 0x557dfd19b740 2024-03-26T08:42:03.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd84c606740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dfd1b3710 con 0x557dfd19c130 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd84c606740 10 client.0 _ll_register_callbacks cb 0x7fd834046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46632/0 (socket says 172.21.15.7:46632) 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd8437fe640 1 -- 172.21.15.7:0/525601502 learned_addr learned my addr 172.21.15.7:0/525601502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd842ffd640 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19b740 0x557dfd1a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd8427fc640 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 0x557dfd1a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd8437fe640 1 -- 172.21.15.7:0/525601502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 msgr2=0x557dfd1a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd8437fe640 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 0x557dfd1a9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd8437fe640 1 -- 172.21.15.7:0/525601502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19b740 msgr2=0x557dfd1a90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd8437fe640 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19b740 0x557dfd1a90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.478+0000 7fd8437fe640 1 -- 172.21.15.7:0/525601502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dfd1acbc0 con 0x557dfd1a12a0 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd8437fe640 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ac5d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd834037040 tx=0x7fd83402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1487aa32db4fa274 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:03.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd822ffd640 1 -- 172.21.15.7:0/525601502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83405d070 con 0x557dfd1a12a0 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd822ffd640 1 -- 172.21.15.7:0/525601502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd834003d80 con 0x557dfd1a12a0 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd84c606740 1 -- 172.21.15.7:0/525601502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd83c09b910 con 0x557dfd1a12a0 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd822ffd640 1 -- 172.21.15.7:0/525601502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd834039040 con 0x557dfd1a12a0 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd84c606740 10 client.15264 fetch_fsmap learned FSMap version 22 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd84c606740 1 -- 172.21.15.7:0/525601502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557dfd20dc70 con 0x557dfd1a12a0 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd822ffd640 1 -- 172.21.15.7:0/525601502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd83c09b910 con 0x557dfd1a12a0 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd822ffd640 1 -- 172.21.15.7:0/525601502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd834057040 con 0x557dfd1a12a0 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd84c606740 10 client.15264 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd84c606740 10 client.15264 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd84c606740 20 client.15264 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd84c606740 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557dfd212310 0x557dfd20df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd84c606740 4 client.15264 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd84c606740 1 -- 172.21.15.7:0/525601502 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557dfd1084b0 con 0x557dfd212310 2024-03-26T08:42:03.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.479+0000 7fd842ffd640 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557dfd212310 0x557dfd20df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd842ffd640 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557dfd212310 0x557dfd20df50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=418ffa36a0a9811a server_cookie=3d750b25a88a6d53 in_seq=0 out_seq=0 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd822ffd640 10 client.15264 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd822ffd640 1 -- 172.21.15.7:0/525601502 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x557dfd1084b0 con 0x557dfd212310 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd822ffd640 10 client.15264 handle_command_reply: tid=0 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 client.15264 shutdown 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 -- 172.21.15.7:0/525601502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 msgr2=0x557dfd1ac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ac5d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd834037040 tx=0x7fd83402c890 comp rx=0 tx=0).stop 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 -- 172.21.15.7:0/525601502 shutdown_connections 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557dfd212310 0x557dfd20df50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557dfd1a12a0 0x557dfd1ac5d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557dfd19c130 0x557dfd1a9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd822ffd640 0 client.15264 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:42:03.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 --2- 172.21.15.7:0/525601502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557dfd19b740 0x557dfd1a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 -- 172.21.15.7:0/525601502 >> 172.21.15.7:0/525601502 conn(0x557dfd163a60 msgr2=0x557dfd1a5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 -- 172.21.15.7:0/525601502 shutdown_connections 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 1 -- 172.21.15.7:0/525601502 wait complete. 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.480+0000 7fd84c606740 20 client.15264 trim_cache size 0 max 16384 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24478, 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-26T08:42:03.479 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-26T08:42:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 109.208861961, 2024-03-26T08:42:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-03-26T08:42:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-26T08:42:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T08:42:03.480 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 132.107603377 2024-03-26T08:42:03.480 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 -- 172.21.15.7:0/423599363 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8300a2800 msgr2=0x7fd8300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8300a2800 0x7fd8300a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0095f0 tx=0x7fd82c005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 -- 172.21.15.7:0/423599363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8441016f0 msgr2=0x7fd8441979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8441016f0 0x7fd8441979b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00e9c0 tx=0x7fd83c00ee80 comp rx=0 tx=0).stop 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 -- 172.21.15.7:0/423599363 shutdown_connections 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8300a2800 0x7fd8300a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8441032a0 0x7fd84419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8441028e0 0x7fd844197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 --2- 172.21.15.7:0/423599363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8441016f0 0x7fd8441979b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 -- 172.21.15.7:0/423599363 >> 172.21.15.7:0/423599363 conn(0x7fd8440fcc60 msgr2=0x7fd844105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 -- 172.21.15.7:0/423599363 shutdown_connections 2024-03-26T08:42:03.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.481+0000 7fd84993b640 1 -- 172.21.15.7:0/423599363 wait complete. 2024-03-26T08:42:03.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-26T08:42:03.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:03 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/23864131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:03.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:03 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/466135654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:03.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:03 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2156378097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:03.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:03 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/23864131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:03.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:03 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/466135654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:03.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:03 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2156378097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 -- 172.21.15.7:0/104232465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 msgr2=0x7f1b88101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 --2- 172.21.15.7:0/104232465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 0x7f1b88101c30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009900 tx=0x7f1b7c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 -- 172.21.15.7:0/104232465 shutdown_connections 2024-03-26T08:42:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 --2- 172.21.15.7:0/104232465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b88102170 0x7f1b88108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 --2- 172.21.15.7:0/104232465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 0x7f1b88101c30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 --2- 172.21.15.7:0/104232465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b881005c0 0x7f1b881009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 -- 172.21.15.7:0/104232465 >> 172.21.15.7:0/104232465 conn(0x7f1b880fbb30 msgr2=0x7f1b880fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 -- 172.21.15.7:0/104232465 shutdown_connections 2024-03-26T08:42:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 -- 172.21.15.7:0/104232465 wait complete. 2024-03-26T08:42:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 Processor -- start 2024-03-26T08:42:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.596+0000 7f1b8e220640 1 -- start start 2024-03-26T08:42:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b8e220640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b881005c0 0x7f1b88104550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b8e220640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 0x7f1b88102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b8e220640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b88102170 0x7f1b881030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b8e220640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8810c240 con 0x7f1b881017b0 2024-03-26T08:42:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b8e220640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8810c0c0 con 0x7f1b88102170 2024-03-26T08:42:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b8e220640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8810c3c0 con 0x7f1b881005c0 2024-03-26T08:42:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 0x7f1b88102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 0x7f1b88102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46658/0 (socket says 172.21.15.7:46658) 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b86ffd640 1 -- 172.21.15.7:0/2360166668 learned_addr learned my addr 172.21.15.7:0/2360166668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b877fe640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b881005c0 0x7f1b88104550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b87fff640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b88102170 0x7f1b881030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b86ffd640 1 -- 172.21.15.7:0/2360166668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b881005c0 msgr2=0x7f1b88104550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b86ffd640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b881005c0 0x7f1b88104550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b86ffd640 1 -- 172.21.15.7:0/2360166668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b88102170 msgr2=0x7f1b881030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b86ffd640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b88102170 0x7f1b881030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b86ffd640 1 -- 172.21.15.7:0/2360166668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b88103960 con 0x7f1b881017b0 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b86ffd640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 0x7f1b88102b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c005ab0 tx=0x7f1b7c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b84ff9640 1 -- 172.21.15.7:0/2360166668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c041070 con 0x7f1b881017b0 2024-03-26T08:42:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b8e220640 1 -- 172.21.15.7:0/2360166668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b881a1bc0 con 0x7f1b881017b0 2024-03-26T08:42:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b84ff9640 1 -- 172.21.15.7:0/2360166668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b7c004450 con 0x7f1b881017b0 2024-03-26T08:42:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b84ff9640 1 -- 172.21.15.7:0/2360166668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c003d70 con 0x7f1b881017b0 2024-03-26T08:42:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.597+0000 7f1b8e220640 1 -- 172.21.15.7:0/2360166668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b881a2110 con 0x7f1b881017b0 2024-03-26T08:42:03.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.599+0000 7f1b84ff9640 1 -- 172.21.15.7:0/2360166668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1b7c044070 con 0x7f1b881017b0 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b90e0d740 1 Processor -- start 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b90e0d740 1 -- start start 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b84ff9640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1b700a2800 0x7f1b700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dcec9db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9e130 0x561dceca2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 0x561dceca5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcec02c20 con 0x561dcec9e130 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcebaba40 con 0x561dcec9d740 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcebfecc0 con 0x561dceca32a0 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b87fff640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1b700a2800 0x7f1b700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dcec9db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dcec9db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42182/0 (socket says 172.21.15.7:42182) 2024-03-26T08:42:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9e130 0x561dceca2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b877fe640 1 -- 172.21.15.7:0/3592977271 learned_addr learned my addr 172.21.15.7:0/3592977271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b87fff640 1 --2- 172.21.15.7:0/3592977271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 0x561dceca5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.600+0000 7f1b84ff9640 1 -- 172.21.15.7:0/2360166668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1b7c0f05c0 con 0x7f1b881017b0 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b877fe640 1 -- 172.21.15.7:0/3592977271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 msgr2=0x561dceca5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b877fe640 1 --2- 172.21.15.7:0/3592977271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 0x561dceca5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b877fe640 1 -- 172.21.15.7:0/3592977271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9e130 msgr2=0x561dceca2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b877fe640 1 --2- 172.21.15.7:0/3592977271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9e130 0x561dceca2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b877fe640 1 -- 172.21.15.7:0/3592977271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dcebcdf90 con 0x561dcec9d740 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b87fff640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1b700a2800 0x7f1b700a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1b80005fd0 tx=0x7f1b80009bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b877fe640 1 --2- 172.21.15.7:0/3592977271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dcec9db60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009ca0 tx=0x7f1b78011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f3756c9f76a2eb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b6dffb640 1 -- 172.21.15.7:0/3592977271 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78004220 con 0x561dcec9d740 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b6dffb640 1 -- 172.21.15.7:0/3592977271 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b78009e50 con 0x561dcec9d740 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.601+0000 7f1b6dffb640 1 -- 172.21.15.7:0/3592977271 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78018630 con 0x561dcec9d740 2024-03-26T08:42:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3592977271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 msgr2=0x561dcec9db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3592977271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dcec9db60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009ca0 tx=0x7f1b78011810 comp rx=0 tx=0).stop 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3592977271 shutdown_connections 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3592977271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 0x561dceca5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3592977271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9e130 0x561dceca2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3592977271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dcec9db60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3592977271 >> 172.21.15.7:0/3592977271 conn(0x561dcec65a60 msgr2=0x561dcec990b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3592977271 shutdown_connections 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3592977271 wait complete. 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 -- start start 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 Processor -- start 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 0x561dceca80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dceca32a0 0x561dcecaf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dceca6250 con 0x561dceca32a0 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcebbee90 con 0x561dcec9d740 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.602+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dceca63d0 con 0x561dcec9e130 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b90e0d740 10 client.0 _ll_register_callbacks cb 0x561dce916ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b87fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dceca32a0 0x561dcecaf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42198/0 (socket says 172.21.15.7:42198) 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b86ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 0x561dceca80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b877fe640 1 -- 172.21.15.7:0/4069363955 learned_addr learned my addr 172.21.15.7:0/4069363955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b87fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dceca32a0 0x561dcecaf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46672/0 (socket says 172.21.15.7:46672) 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b87fff640 1 -- 172.21.15.7:0/4069363955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 msgr2=0x561dceca80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b87fff640 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 0x561dceca80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b87fff640 1 -- 172.21.15.7:0/4069363955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 msgr2=0x561dceca7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b87fff640 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b87fff640 1 -- 172.21.15.7:0/4069363955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dcebcb9d0 con 0x561dceca32a0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b877fe640 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b86ffd640 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 0x561dceca80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b87fff640 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dceca32a0 0x561dcecaf100 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000fc20 tx=0x7f1b80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f630b09f94787b45 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b66ffd640 1 -- 172.21.15.7:0/4069363955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80004240 con 0x561dceca32a0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b66ffd640 1 -- 172.21.15.7:0/4069363955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1b800043e0 con 0x561dceca32a0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b90e0d740 1 -- 172.21.15.7:0/4069363955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b78009e50 con 0x561dceca32a0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b66ffd640 1 -- 172.21.15.7:0/4069363955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b800045e0 con 0x561dceca32a0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b90e0d740 10 client.15276 fetch_fsmap learned FSMap version 22 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.603+0000 7f1b90e0d740 1 -- 172.21.15.7:0/4069363955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x561dce9172d0 con 0x561dceca32a0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b66ffd640 1 -- 172.21.15.7:0/4069363955 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b78009e50 con 0x561dceca32a0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b66ffd640 1 -- 172.21.15.7:0/4069363955 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1b80020050 con 0x561dceca32a0 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b90e0d740 10 client.15276 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b90e0d740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b90e0d740 20 client.15276 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:03.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x561dcecb43b0 0x561dcecb3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b90e0d740 4 client.15276 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b90e0d740 1 -- 172.21.15.7:0/4069363955 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561dcec10160 con 0x561dcecb43b0 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b877fe640 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x561dcecb43b0 0x561dcecb3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b66ffd640 10 client.15276 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.604+0000 7f1b877fe640 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x561dcecb43b0 0x561dcecb3440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e63ff67949a7d4d server_cookie=3efd040b2e35d1bc in_seq=0 out_seq=0 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b66ffd640 1 -- 172.21.15.7:0/4069363955 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561dcec10160 con 0x561dcecb43b0 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b66ffd640 10 client.15276 handle_command_reply: tid=0 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 client.15276 shutdown 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 -- 172.21.15.7:0/4069363955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dceca32a0 msgr2=0x561dcecaf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dceca32a0 0x561dcecaf100 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000fc20 tx=0x7f1b80004ab0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 -- 172.21.15.7:0/4069363955 shutdown_connections 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dceca32a0 0x561dcecaf100 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x561dcecb43b0 0x561dcecb3440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 0x561dceca80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b66ffd640 0 client.15276 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:42:03.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/4069363955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 -- 172.21.15.7:0/4069363955 >> 172.21.15.7:0/4069363955 conn(0x561dcec65a60 msgr2=0x561dcec982f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 -- 172.21.15.7:0/4069363955 shutdown_connections 2024-03-26T08:42:03.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 1 -- 172.21.15.7:0/4069363955 wait complete. 2024-03-26T08:42:03.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.605+0000 7f1b90e0d740 20 client.15276 trim_cache size 0 max 16384 2024-03-26T08:42:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b90e0d740 1 Processor -- start 2024-03-26T08:42:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b90e0d740 1 -- start start 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 0x561dceca80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dceca32a0 0x561dcecaf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcecb3bd0 con 0x561dcec9d740 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcecb4230 con 0x561dceca32a0 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcecb4890 con 0x561dcec9e130 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46686/0 (socket says 172.21.15.7:46686) 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b877fe640 1 -- 172.21.15.7:0/696232255 learned_addr learned my addr 172.21.15.7:0/696232255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b86ffd640 1 --2- 172.21.15.7:0/696232255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 0x561dceca80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b87fff640 1 --2- 172.21.15.7:0/696232255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dceca32a0 0x561dcecaf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b877fe640 1 -- 172.21.15.7:0/696232255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 msgr2=0x561dceca80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b877fe640 1 --2- 172.21.15.7:0/696232255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 0x561dceca80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b877fe640 1 -- 172.21.15.7:0/696232255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dceca32a0 msgr2=0x561dcecaf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b877fe640 1 --2- 172.21.15.7:0/696232255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dceca32a0 0x561dcecaf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.622+0000 7f1b877fe640 1 -- 172.21.15.7:0/696232255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dce9e16b0 con 0x561dcec9d740 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b877fe640 1 --2- 172.21.15.7:0/696232255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dceca7b80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1b78003610 tx=0x7f1b78009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a39e27cb3a1cbb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b6d7fa640 1 -- 172.21.15.7:0/696232255 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7801e040 con 0x561dcec9d740 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b6d7fa640 1 -- 172.21.15.7:0/696232255 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b78005530 con 0x561dcec9d740 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b6d7fa640 1 -- 172.21.15.7:0/696232255 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78029c50 con 0x561dcec9d740 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b90e0d740 1 -- 172.21.15.7:0/696232255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 msgr2=0x561dceca7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/696232255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dceca7b80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1b78003610 tx=0x7f1b78009f40 comp rx=0 tx=0).stop 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b90e0d740 1 -- 172.21.15.7:0/696232255 shutdown_connections 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/696232255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dceca32a0 0x561dcecaf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/696232255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dcec9e130 0x561dceca80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/696232255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dceca7b80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b90e0d740 1 -- 172.21.15.7:0/696232255 >> 172.21.15.7:0/696232255 conn(0x561dcec65a60 msgr2=0x561dcecb3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b90e0d740 1 -- 172.21.15.7:0/696232255 shutdown_connections 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.623+0000 7f1b90e0d740 1 -- 172.21.15.7:0/696232255 wait complete. 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b90e0d740 1 -- start start 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b90e0d740 1 Processor -- start 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dcecab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9e130 0x561dcecab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b90e0d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 0x561dcecae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcecb5550 con 0x561dcec9d740 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcecb4ef0 con 0x561dcec9e130 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b90e0d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dcecb5bb0 con 0x561dceca32a0 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b90e0d740 10 client.0 _ll_register_callbacks cb 0x7f1b80009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dcecab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dcecab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46694/0 (socket says 172.21.15.7:46694) 2024-03-26T08:42:03.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b877fe640 1 -- 172.21.15.7:0/3070383553 learned_addr learned my addr 172.21.15.7:0/3070383553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b86ffd640 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9e130 0x561dcecab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b87fff640 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 0x561dcecae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b877fe640 1 -- 172.21.15.7:0/3070383553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 msgr2=0x561dcecae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b877fe640 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 0x561dcecae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b877fe640 1 -- 172.21.15.7:0/3070383553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9e130 msgr2=0x561dcecab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b877fe640 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9e130 0x561dcecab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.624+0000 7f1b877fe640 1 -- 172.21.15.7:0/3070383553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dcecaebd0 con 0x561dcec9d740 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b877fe640 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dcecab250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1b78011850 tx=0x7f1b78005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37673c7dd7f4307 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b66ffd640 1 -- 172.21.15.7:0/3070383553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78036020 con 0x561dcec9d740 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b66ffd640 1 -- 172.21.15.7:0/3070383553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1b78029180 con 0x561dcec9d740 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3070383553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b78005530 con 0x561dcec9d740 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b66ffd640 1 -- 172.21.15.7:0/3070383553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b78009d90 con 0x561dcec9d740 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b90e0d740 10 client.15282 fetch_fsmap learned FSMap version 22 2024-03-26T08:42:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3070383553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x561dced0fd10 con 0x561dcec9d740 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b66ffd640 1 -- 172.21.15.7:0/3070383553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1b78005530 con 0x561dcec9d740 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b66ffd640 1 -- 172.21.15.7:0/3070383553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1b78026440 con 0x561dcec9d740 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b90e0d740 10 client.15282 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b90e0d740 10 client.15282 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b90e0d740 20 client.15282 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x561dced13d80 0x561dced0fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b90e0d740 4 client.15282 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3070383553 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561dcec0a4b0 con 0x561dced13d80 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.625+0000 7f1b86ffd640 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x561dced13d80 0x561dced0fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b86ffd640 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x561dced13d80 0x561dced0fee0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75139284dba429c8 server_cookie=67b87a6961f911a2 in_seq=0 out_seq=0 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b66ffd640 10 client.15282 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b66ffd640 1 -- 172.21.15.7:0/3070383553 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x561dcec0a4b0 con 0x561dced13d80 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b66ffd640 10 client.15282 handle_command_reply: tid=0 2024-03-26T08:42:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 client.15282 shutdown 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3070383553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 msgr2=0x561dcecab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dcecab250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1b78011850 tx=0x7f1b78005750 comp rx=0 tx=0).stop 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3070383553 shutdown_connections 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x561dced13d80 0x561dced0fee0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561dceca32a0 0x561dcecae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b66ffd640 0 client.15282 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561dcec9e130 0x561dcecab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 --2- 172.21.15.7:0/3070383553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561dcec9d740 0x561dcecab250 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3070383553 >> 172.21.15.7:0/3070383553 conn(0x561dcec65a60 msgr2=0x561dceca7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3070383553 shutdown_connections 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 1 -- 172.21.15.7:0/3070383553 wait complete. 2024-03-26T08:42:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.626+0000 7f1b90e0d740 20 client.15282 trim_cache size 0 max 16384 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24502, 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 110.363062126, 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 126.299737948 2024-03-26T08:42:03.626 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:42:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 -- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1b700a2800 msgr2=0x7f1b700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1b700a2800 0x7f1b700a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1b80005fd0 tx=0x7f1b80009bf0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 -- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 msgr2=0x7f1b88102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 0x7f1b88102b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c005ab0 tx=0x7f1b7c02c6c0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 -- 172.21.15.7:0/2360166668 shutdown_connections 2024-03-26T08:42:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1b700a2800 0x7f1b700a4cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b88102170 0x7f1b881030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b881017b0 0x7f1b88102b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 --2- 172.21.15.7:0/2360166668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b881005c0 0x7f1b88104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 -- 172.21.15.7:0/2360166668 >> 172.21.15.7:0/2360166668 conn(0x7f1b880fbb30 msgr2=0x7f1b88068a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 -- 172.21.15.7:0/2360166668 shutdown_connections 2024-03-26T08:42:03.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.627+0000 7f1b8e220640 1 -- 172.21.15.7:0/2360166668 wait complete. 2024-03-26T08:42:03.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.747+0000 7f5f054b8640 1 -- 172.21.15.7:0/2306850013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f000ffd50 msgr2=0x7f5f00100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.747+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2306850013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f000ffd50 0x7f5f00100170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000a970 tx=0x7f5ef002cea0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.747+0000 7f5f054b8640 1 -- 172.21.15.7:0/2306850013 shutdown_connections 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.747+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2306850013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f00101900 0x7f5f00108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.747+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2306850013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f00100f40 0x7f5f001013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.747+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2306850013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f000ffd50 0x7f5f00100170 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.747+0000 7f5f054b8640 1 -- 172.21.15.7:0/2306850013 >> 172.21.15.7:0/2306850013 conn(0x7f5f000fb2c0 msgr2=0x7f5f000fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.747+0000 7f5f054b8640 1 -- 172.21.15.7:0/2306850013 shutdown_connections 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.747+0000 7f5f054b8640 1 -- 172.21.15.7:0/2306850013 wait complete. 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5f054b8640 1 Processor -- start 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5f054b8640 1 -- start start 2024-03-26T08:42:03.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5f054b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f000ffd50 0x7f5f00103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5f054b8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f00100f40 0x7f5f00102320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5f054b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f00101900 0x7f5f00102860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5f054b8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0010b120 con 0x7f5f000ffd50 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5f054b8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0010afa0 con 0x7f5f00101900 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5f054b8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0010b2a0 con 0x7f5f00100f40 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5efeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f000ffd50 0x7f5f00103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5eff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f00101900 0x7f5f00102860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5efe7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f00100f40 0x7f5f00102320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5efeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f000ffd50 0x7f5f00103c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46720/0 (socket says 172.21.15.7:46720) 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.748+0000 7f5efeffd640 1 -- 172.21.15.7:0/2183610871 learned_addr learned my addr 172.21.15.7:0/2183610871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5eff7fe640 1 -- 172.21.15.7:0/2183610871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f00100f40 msgr2=0x7f5f00102320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f00100f40 0x7f5f00102320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5eff7fe640 1 -- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f000ffd50 msgr2=0x7f5f00103c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f000ffd50 0x7f5f00103c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5eff7fe640 1 -- 172.21.15.7:0/2183610871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f00103060 con 0x7f5f00101900 2024-03-26T08:42:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5efeffd640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f000ffd50 0x7f5f00103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f00101900 0x7f5f00102860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800efc0 tx=0x7f5ef800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5ee7fff640 1 -- 172.21.15.7:0/2183610871 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8003970 con 0x7f5f00101900 2024-03-26T08:42:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5ee7fff640 1 -- 172.21.15.7:0/2183610871 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ef8004020 con 0x7f5f00101900 2024-03-26T08:42:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5ee7fff640 1 -- 172.21.15.7:0/2183610871 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800b9f0 con 0x7f5f00101900 2024-03-26T08:42:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5f054b8640 1 -- 172.21.15.7:0/2183610871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f001a1b60 con 0x7f5f00101900 2024-03-26T08:42:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.749+0000 7f5f054b8640 1 -- 172.21.15.7:0/2183610871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f001a2030 con 0x7f5f00101900 2024-03-26T08:42:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.751+0000 7f5ee7fff640 1 -- 172.21.15.7:0/2183610871 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5ef801d020 con 0x7f5f00101900 2024-03-26T08:42:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.752+0000 7f5ee7fff640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ee80a2800 0x7f5ee80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.752+0000 7f5ee7fff640 1 -- 172.21.15.7:0/2183610871 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5ef80ce940 con 0x7f5f00101900 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.752+0000 7f5efeffd640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ee80a2800 0x7f5ee80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.752+0000 7f5efeffd640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ee80a2800 0x7f5ee80a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00057c0 tx=0x7f5ef0035f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5f08178740 1 Processor -- start 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5f08178740 1 -- start start 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5f08178740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5b740 0x555c2cf5bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5f08178740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5c130 0x555c2cf60c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5f08178740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf612a0 0x555c2cf63730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5f08178740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cec0c20 con 0x555c2cf5b740 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5f08178740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2ce69a40 con 0x555c2cf5c130 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5f08178740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cebccc0 con 0x555c2cf612a0 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5efeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5b740 0x555c2cf5bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5efeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5b740 0x555c2cf5bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46722/0 (socket says 172.21.15.7:46722) 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5efeffd640 1 -- 172.21.15.7:0/3660951516 learned_addr learned my addr 172.21.15.7:0/3660951516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf612a0 0x555c2cf63730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5efe7fc640 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5c130 0x555c2cf60c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.753+0000 7f5efeffd640 1 -- 172.21.15.7:0/3660951516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf612a0 msgr2=0x555c2cf63730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5efeffd640 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf612a0 0x555c2cf63730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5efeffd640 1 -- 172.21.15.7:0/3660951516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5c130 msgr2=0x555c2cf60c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5efeffd640 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5c130 0x555c2cf60c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5efeffd640 1 -- 172.21.15.7:0/3660951516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c2ce8bf90 con 0x555c2cf5b740 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf612a0 0x555c2cf63730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5efe7fc640 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5c130 0x555c2cf60c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5efeffd640 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5b740 0x555c2cf5bb60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0038040 tx=0x7f5ef003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cc94221ef08cc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5ee57fa640 1 -- 172.21.15.7:0/3660951516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef003a040 con 0x555c2cf5b740 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5ee57fa640 1 -- 172.21.15.7:0/3660951516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ef002dc70 con 0x555c2cf5b740 2024-03-26T08:42:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.754+0000 7f5ee57fa640 1 -- 172.21.15.7:0/3660951516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0003e50 con 0x555c2cf5b740 2024-03-26T08:42:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 -- 172.21.15.7:0/3660951516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5b740 msgr2=0x555c2cf5bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5b740 0x555c2cf5bb60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0038040 tx=0x7f5ef003e000 comp rx=0 tx=0).stop 2024-03-26T08:42:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 -- 172.21.15.7:0/3660951516 shutdown_connections 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf612a0 0x555c2cf63730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5c130 0x555c2cf60c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 --2- 172.21.15.7:0/3660951516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5b740 0x555c2cf5bb60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 -- 172.21.15.7:0/3660951516 >> 172.21.15.7:0/3660951516 conn(0x555c2cf23a60 msgr2=0x555c2cf570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 -- 172.21.15.7:0/3660951516 shutdown_connections 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 -- 172.21.15.7:0/3660951516 wait complete. 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 -- start start 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 Processor -- start 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5b740 0x555c2cf65c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5c130 0x555c2cf66140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf612a0 0x555c2cf6d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cf641f0 con 0x555c2cf612a0 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2ce7ce90 con 0x555c2cf5b740 2024-03-26T08:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cf64370 con 0x555c2cf5c130 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5f08178740 10 client.0 _ll_register_callbacks cb 0x555c2cbd4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.755+0000 7f5eff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf612a0 0x555c2cf6d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5eff7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf612a0 0x555c2cf6d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46736/0 (socket says 172.21.15.7:46736) 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5eff7fe640 1 -- 172.21.15.7:0/449040624 learned_addr learned my addr 172.21.15.7:0/449040624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5efe7fc640 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5c130 0x555c2cf66140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5efeffd640 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5b740 0x555c2cf65c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5eff7fe640 1 -- 172.21.15.7:0/449040624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5c130 msgr2=0x555c2cf66140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5c130 0x555c2cf66140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5eff7fe640 1 -- 172.21.15.7:0/449040624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5b740 msgr2=0x555c2cf65c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5b740 0x555c2cf65c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5eff7fe640 1 -- 172.21.15.7:0/449040624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c2ce899d0 con 0x555c2cf612a0 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf612a0 0x555c2cf6d180 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ef809afa0 tx=0x7f5ef809f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4289b269b21ad9d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5ee2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/449040624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8004560 con 0x555c2cf612a0 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/449040624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5ef80bb0c0 con 0x555c2cf612a0 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5f08178740 1 -- 172.21.15.7:0/449040624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ef002dc70 con 0x555c2cf612a0 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/449040624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ef80041d0 con 0x555c2cf612a0 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5f08178740 10 client.15294 fetch_fsmap learned FSMap version 22 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5f08178740 1 -- 172.21.15.7:0/449040624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555c2cbd52d0 con 0x555c2cf612a0 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.756+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/449040624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ef002dc70 con 0x555c2cf612a0 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/449040624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5ef800b040 con 0x555c2cf612a0 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5f08178740 10 client.15294 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5f08178740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5f08178740 20 client.15294 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5f08178740 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555c2cf723b0 0x555c2cf71470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5f08178740 4 client.15294 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5f08178740 1 -- 172.21.15.7:0/449040624 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c2cece160 con 0x555c2cf723b0 2024-03-26T08:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5efeffd640 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555c2cf723b0 0x555c2cf71470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5ee2ff5640 10 client.15294 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.757+0000 7f5efeffd640 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555c2cf723b0 0x555c2cf71470 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68e5de10f31697c2 server_cookie=898f2720046907e5 in_seq=0 out_seq=0 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/449040624 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555c2cece160 con 0x555c2cf723b0 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5ee2ff5640 10 client.15294 handle_command_reply: tid=0 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 client.15294 shutdown 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 -- 172.21.15.7:0/449040624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf612a0 msgr2=0x555c2cf6d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf612a0 0x555c2cf6d180 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ef809afa0 tx=0x7f5ef809f4b0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 -- 172.21.15.7:0/449040624 shutdown_connections 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555c2cf723b0 0x555c2cf71470 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf612a0 0x555c2cf6d180 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5c130 0x555c2cf66140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5ee2ff5640 0 client.15294 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 --2- 172.21.15.7:0/449040624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf5b740 0x555c2cf65c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 -- 172.21.15.7:0/449040624 >> 172.21.15.7:0/449040624 conn(0x555c2cf23a60 msgr2=0x555c2cf562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 -- 172.21.15.7:0/449040624 shutdown_connections 2024-03-26T08:42:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 1 -- 172.21.15.7:0/449040624 wait complete. 2024-03-26T08:42:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.758+0000 7f5f08178740 20 client.15294 trim_cache size 0 max 16384 2024-03-26T08:42:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5f08178740 1 Processor -- start 2024-03-26T08:42:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5f08178740 1 -- start start 2024-03-26T08:42:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5f08178740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 0x555c2cf65f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5f08178740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5f08178740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cf7a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5f08178740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cf71bd0 con 0x555c2cf5c130 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5f08178740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cf72230 con 0x555c2cf612a0 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5f08178740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cf72890 con 0x555c2cf5b740 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5efe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5eff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cf7a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5eff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cf7a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42280/0 (socket says 172.21.15.7:42280) 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5eff7fe640 1 -- 172.21.15.7:0/138556136 learned_addr learned my addr 172.21.15.7:0/138556136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.775+0000 7f5efeffd640 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 0x555c2cf65f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5eff7fe640 1 -- 172.21.15.7:0/138556136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 msgr2=0x555c2cf65f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 0x555c2cf65f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5eff7fe640 1 -- 172.21.15.7:0/138556136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 msgr2=0x555c2cf6d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5eff7fe640 1 -- 172.21.15.7:0/138556136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c2cc9f6b0 con 0x555c2cf612a0 2024-03-26T08:42:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5efe7fc640 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5efeffd640 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 0x555c2cf65f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cf7a850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8097700 tx=0x7f5ef809b100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232b0471adb604ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5ee4ff9640 1 -- 172.21.15.7:0/138556136 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8004370 con 0x555c2cf612a0 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5ee4ff9640 1 -- 172.21.15.7:0/138556136 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ef809b9a0 con 0x555c2cf612a0 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.776+0000 7f5ee4ff9640 1 -- 172.21.15.7:0/138556136 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800b040 con 0x555c2cf612a0 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 -- 172.21.15.7:0/138556136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 msgr2=0x555c2cf7a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cf7a850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8097700 tx=0x7f5ef809b100 comp rx=0 tx=0).stop 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 -- 172.21.15.7:0/138556136 shutdown_connections 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cf7a850 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 --2- 172.21.15.7:0/138556136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 0x555c2cf65f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 -- 172.21.15.7:0/138556136 >> 172.21.15.7:0/138556136 conn(0x555c2cf23a60 msgr2=0x555c2cf65810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 -- 172.21.15.7:0/138556136 shutdown_connections 2024-03-26T08:42:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 -- 172.21.15.7:0/138556136 wait complete. 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 -- start start 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 Processor -- start 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 0x555c2cf69f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cfcd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cf73550 con 0x555c2cf5c130 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cf72ef0 con 0x555c2cf612a0 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c2cf73bb0 con 0x555c2cf5b740 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5f08178740 10 client.0 _ll_register_callbacks cb 0x7f5ef809ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5efe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.777+0000 7f5efe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46748/0 (socket says 172.21.15.7:46748) 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5efe7fc640 1 -- 172.21.15.7:0/2685958545 learned_addr learned my addr 172.21.15.7:0/2685958545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5efeffd640 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 0x555c2cf69f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5eff7fe640 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cfcd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5efe7fc640 1 -- 172.21.15.7:0/2685958545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 msgr2=0x555c2cf69f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5efe7fc640 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 0x555c2cf69f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5efe7fc640 1 -- 172.21.15.7:0/2685958545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 msgr2=0x555c2cfcd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5efe7fc640 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cfcd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5efe7fc640 1 -- 172.21.15.7:0/2685958545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c2cf6ab10 con 0x555c2cf5c130 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5efe7fc640 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6a470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x555c2cf5cba0 tx=0x7f5ef4012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3d1f9d1cda81a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5ee2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/2685958545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4023060 con 0x555c2cf5c130 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/2685958545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5ef4004230 con 0x555c2cf5c130 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5f08178740 1 -- 172.21.15.7:0/2685958545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ef809b9a0 con 0x555c2cf5c130 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/2685958545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ef4004430 con 0x555c2cf5c130 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5f08178740 10 client.15300 fetch_fsmap learned FSMap version 22 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5f08178740 1 -- 172.21.15.7:0/2685958545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x555c2cfceb80 con 0x555c2cf5c130 2024-03-26T08:42:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.778+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/2685958545 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ef809b9a0 con 0x555c2cf5c130 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/2685958545 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5ef4028070 con 0x555c2cf5c130 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 10 client.15300 fetch_fsmap finished waiting for FSMap version 22 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 10 client.15300 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 20 client.15300 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555c2cfd3270 0x555c2cfceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 4 client.15300 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 1 -- 172.21.15.7:0/2685958545 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555c2cec84b0 con 0x555c2cfd3270 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5efeffd640 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555c2cfd3270 0x555c2cfceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5efeffd640 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555c2cfd3270 0x555c2cfceec0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbef3218f9f7da7 server_cookie=faf548d70f30e6b4 in_seq=0 out_seq=0 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5ee2ff5640 10 client.15300 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5ee2ff5640 1 -- 172.21.15.7:0/2685958545 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555c2cec84b0 con 0x555c2cfd3270 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5ee2ff5640 10 client.15300 handle_command_reply: tid=0 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 1 client.15300 shutdown 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 1 -- 172.21.15.7:0/2685958545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 msgr2=0x555c2cf6a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6a470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x555c2cf5cba0 tx=0x7f5ef4012d70 comp rx=0 tx=0).stop 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 1 -- 172.21.15.7:0/2685958545 shutdown_connections 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.779+0000 7f5f08178740 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555c2cfd3270 0x555c2cfceec0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f08178740 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555c2cf612a0 0x555c2cfcd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5ee2ff5640 0 client.15300 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f08178740 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555c2cf5c130 0x555c2cf6a470 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f08178740 1 --2- 172.21.15.7:0/2685958545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555c2cf5b740 0x555c2cf69f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f08178740 1 -- 172.21.15.7:0/2685958545 >> 172.21.15.7:0/2685958545 conn(0x555c2cf23a60 msgr2=0x555c2cf5a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f08178740 1 -- 172.21.15.7:0/2685958545 shutdown_connections 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f08178740 1 -- 172.21.15.7:0/2685958545 wait complete. 2024-03-26T08:42:03.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f08178740 20 client.15300 trim_cache size 0 max 16384 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2c753f3e-eb4b-11ee-95d0-87774f69a715", 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24512, 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 113.533303283, 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 87, 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 117.72346029099999 2024-03-26T08:42:03.779 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:42:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 -- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ee80a2800 msgr2=0x7f5ee80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ee80a2800 0x7f5ee80a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00057c0 tx=0x7f5ef0035f90 comp rx=0 tx=0).stop 2024-03-26T08:42:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 -- 172.21.15.7:0/2183610871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f00101900 msgr2=0x7f5f00102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f00101900 0x7f5f00102860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800efc0 tx=0x7f5ef800c800 comp rx=0 tx=0).stop 2024-03-26T08:42:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 -- 172.21.15.7:0/2183610871 shutdown_connections 2024-03-26T08:42:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ee80a2800 0x7f5ee80a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f00101900 0x7f5f00102860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5f00100f40 0x7f5f00102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 --2- 172.21.15.7:0/2183610871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f000ffd50 0x7f5f00103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 -- 172.21.15.7:0/2183610871 >> 172.21.15.7:0/2183610871 conn(0x7f5f000fb2c0 msgr2=0x7f5f00106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 -- 172.21.15.7:0/2183610871 shutdown_connections 2024-03-26T08:42:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:03.780+0000 7f5f054b8640 1 -- 172.21.15.7:0/2183610871 wait complete. 2024-03-26T08:42:03.782 INFO:teuthology.run_tasks:Running task kclient... 2024-03-26T08:42:03.806 INFO:tasks.kclient:Mounting kernel clients... 2024-03-26T08:42:03.806 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-26T08:42:03.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:03.806 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-26T08:42:03.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:03.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-03-26T08:42:03.889 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-03-26T08:42:03.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T08:42:03.891 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-03-26T08:42:03.891 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T08:42:03.891 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-03-26T08:42:03.891 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-03-26T08:42:03.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T08:42:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:42:04.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:04 smithi016 ceph-mon[35851]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T08:42:04.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:04 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2171679712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:04.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:04 smithi007 ceph-mon[32462]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T08:42:04.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:04 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2171679712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:04.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:04 smithi044 ceph-mon[36461]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T08:42:04.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:04 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2171679712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 -- 172.21.15.7:0/557386835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758101320 msgr2=0x7f4758101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 --2- 172.21.15.7:0/557386835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758101320 0x7f4758101700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f474c0099c0 tx=0x7f474c009cd0 comp rx=0 tx=0).stop 2024-03-26T08:42:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 -- 172.21.15.7:0/557386835 shutdown_connections 2024-03-26T08:42:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 --2- 172.21.15.7:0/557386835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4758106370 0x7f475810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 --2- 172.21.15.7:0/557386835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758101320 0x7f4758101700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 --2- 172.21.15.7:0/557386835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4758101cd0 0x7f4758105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 -- 172.21.15.7:0/557386835 >> 172.21.15.7:0/557386835 conn(0x7f47580770e0 msgr2=0x7f47580774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 -- 172.21.15.7:0/557386835 shutdown_connections 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 -- 172.21.15.7:0/557386835 wait complete. 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 Processor -- start 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.968+0000 7f475f0d3700 1 -- start start 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475f0d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4758101320 0x7f475819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475f0d3700 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4758101cd0 0x7f475819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475f0d3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758106370 0x7f4758198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475f0d3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4758111370 con 0x7f4758106370 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475f0d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47581111f0 con 0x7f4758101320 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475f0d3700 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47581114f0 con 0x7f4758101cd0 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475ce6f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4758101320 0x7f475819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475d670700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758106370 0x7f4758198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475d670700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758106370 0x7f4758198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46776/0 (socket says 172.21.15.7:46776) 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475d670700 1 -- 172.21.15.7:0/3775722011 learned_addr learned my addr 172.21.15.7:0/3775722011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4757fff700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4758101cd0 0x7f475819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4757fff700 1 -- 172.21.15.7:0/3775722011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4758101320 msgr2=0x7f475819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4757fff700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4758101320 0x7f475819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4757fff700 1 -- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758106370 msgr2=0x7f4758198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4757fff700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758106370 0x7f4758198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4757fff700 1 -- 172.21.15.7:0/3775722011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4758199070 con 0x7f4758101cd0 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475ce6f700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4758101320 0x7f475819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475d670700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758106370 0x7f4758198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4757fff700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4758101cd0 0x7f475819ebb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f474400e9c0 tx=0x7f474400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4755ffb700 1 -- 172.21.15.7:0/3775722011 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744004220 con 0x7f4758101cd0 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4758199360 con 0x7f4758101cd0 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4755ffb700 1 -- 172.21.15.7:0/3775722011 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4744004d10 con 0x7f4758101cd0 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.969+0000 7f4755ffb700 1 -- 172.21.15.7:0/3775722011 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744005020 con 0x7f4758101cd0 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.970+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47581a55f0 con 0x7f4758101cd0 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.970+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4758102770 con 0x7f4758101cd0 2024-03-26T08:42:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.971+0000 7f4755ffb700 1 -- 172.21.15.7:0/3775722011 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4744004750 con 0x7f4758101cd0 2024-03-26T08:42:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.972+0000 7f4755ffb700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47480a2910 0x7f47480a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.972+0000 7f475ce6f700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47480a2910 0x7f47480a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.972+0000 7f4755ffb700 1 -- 172.21.15.7:0/3775722011 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f47440ce8f0 con 0x7f4758101cd0 2024-03-26T08:42:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.973+0000 7f475ce6f700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47480a2910 0x7f47480a4dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f474c000c00 tx=0x7f474c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:05.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:05.977+0000 7f4755ffb700 1 -- 172.21.15.7:0/3775722011 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f474409b950 con 0x7f4758101cd0 2024-03-26T08:42:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.230+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4758199c70 con 0x7f4758101cd0 2024-03-26T08:42:06.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.230+0000 7f4755ffb700 1 -- 172.21.15.7:0/3775722011 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7f474409b250 con 0x7f4758101cd0 2024-03-26T08:42:06.222 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47480a2910 msgr2=0x7f47480a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47480a2910 0x7f47480a4dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f474c000c00 tx=0x7f474c005fb0 comp rx=0 tx=0).stop 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4758101cd0 msgr2=0x7f475819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4758101cd0 0x7f475819ebb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f474400e9c0 tx=0x7f474400ecd0 comp rx=0 tx=0).stop 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 shutdown_connections 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47480a2910 0x7f47480a4dd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4758106370 0x7f4758198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4758101320 0x7f475819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 --2- 172.21.15.7:0/3775722011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4758101cd0 0x7f475819ebb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 >> 172.21.15.7:0/3775722011 conn(0x7f47580770e0 msgr2=0x7f4758105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 shutdown_connections 2024-03-26T08:42:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:06.233+0000 7f475f0d3700 1 -- 172.21.15.7:0/3775722011 wait complete. 2024-03-26T08:42:06.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:06 smithi016 ceph-mon[35851]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:42:06.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:06 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3775722011' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T08:42:06.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:06 smithi007 ceph-mon[32462]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:42:06.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:06 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3775722011' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T08:42:06.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:06 smithi044 ceph-mon[36461]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T08:42:06.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:06 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3775722011' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T08:42:06.881 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-26T08:42:06.881 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T08:42:06.881 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-03-26T08:42:06.881 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-26T08:42:06.881 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-03-26T08:42:06.881 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-26T08:42:06.881 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T08:42:06.881 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T08:42:06.881 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-03-26T08:42:06.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:06.882 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-03-26T08:42:06.898 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T08:42:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T08:42:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T08:42:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-26T08:42:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-03-26T08:42:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-26T08:42:06.898 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T08:42:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42116sec preferred_lft 42116sec 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-03-26T08:42:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-03-26T08:42:06.900 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T08:42:06.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:06.900 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-03-26T08:42:06.900 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-26T08:42:06.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-03-26T08:42:06.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-03-26T08:42:06.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-03-26T08:42:06.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T08:42:06.900 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-26T08:42:07.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:07.065 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T08:42:07.093 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-03-26T08:42:07.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:07.095 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-03-26T08:42:07.168 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-03-26T08:42:07.169 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T08:42:07.169 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-26T08:42:07.169 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-26T08:42:07.169 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T08:42:07.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:07.169 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-03-26T08:42:07.169 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-26T08:42:07.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-26T08:42:07.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-26T08:42:07.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-26T08:42:07.170 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-26T08:42:07.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:07.446 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-26T08:42:07.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:07.460 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-03-26T08:42:07.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:07.515 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-03-26T08:42:07.515 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-26T08:42:07.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--tmp-cephfs 2024-03-26T08:42:07.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-03-26T08:42:07.515 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-26T08:42:07.596 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-03-26T08:42:07.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:07.597 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-03-26T08:42:07.597 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-26T08:42:07.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-03-26T08:42:07.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T08:42:07.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-03-26T08:42:07.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-03-26T08:42:07.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-03-26T08:42:07.597 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-26T08:42:07.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:07.747 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-03-26T08:42:07.747 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-03-26T08:42:07.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-03-26T08:42:07.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T08:42:07.747 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-03-26T08:42:07.788 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T08:42:07.788 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /tmp/cephfs 2024-03-26T08:42:07.844 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/tmp/cephfs' 2024-03-26T08:42:07.844 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T08:42:07.844 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /tmp/cephfs 2024-03-26T08:42:07.898 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-26T08:42:07.898 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-26T08:42:07.899 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/e0c5a2ed-a3a8-4daf-9d6d-978951487586 2024-03-26T08:42:07.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:07.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/e0c5a2ed-a3a8-4daf-9d6d-978951487586 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-03-26T08:42:07.983 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-03-26T08:42:07.983 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-26T08:42:08.090 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-03-26T08:42:08.090 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-03-26T08:42:08.091 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.16:3300,172.21.15.44:3300" 2024-03-26T08:42:08.091 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2c753f3e-eb4b-11ee-95d0-87774f69a715.cephfs=/volumes/qa/sv_1/e0c5a2ed-a3a8-4daf-9d6d-978951487586 2024-03-26T08:42:08.091 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.7:3300/172.21.15.16:3300/172.21.15.44:3300" will pass to kernel 2024-03-26T08:42:08.093 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-26T08:42:08.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:08.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /tmp/cephfs 2024-03-26T08:42:08.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:08.164 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T08:42:08.220 INFO:teuthology.orchestra.run.smithi007.stdout:client25135 2024-03-26T08:42:08.220 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:42:08.221 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T08:42:08.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:08.276 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T08:42:08.332 INFO:teuthology.orchestra.run.smithi007.stdout:client25135 2024-03-26T08:42:08.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:08.332 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-26T08:42:08.388 INFO:teuthology.orchestra.run.smithi007.stdout:client25135 2024-03-26T08:42:08.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:08.388 DEBUG:teuthology.orchestra.run.smithi007:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715.client25135/status') 2024-03-26T08:42:08.455 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.25135 (3)172.21.15.7:0/1228958764 2024-03-26T08:42:08.455 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-03-26T08:42:08.455 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-03-26T08:42:08.455 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-03-26T08:42:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:70 bytes copied, 2.4405e-05 s, 2.9 MB/s 2024-03-26T08:42:08.457 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T08:42:08.468 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T08:42:08.468 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-26T08:42:08.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-26T08:42:08.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:08 smithi016 ceph-mon[35851]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:42:08.520 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:08 smithi007 ceph-mon[32462]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:42:08.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:08 smithi044 ceph-mon[36461]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T08:42:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 -- 172.21.15.7:0/935980945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 msgr2=0x7f9fc0101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 --2- 172.21.15.7:0/935980945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 0x7f9fc0101c40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009900 tx=0x7f9fb402bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 -- 172.21.15.7:0/935980945 shutdown_connections 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 --2- 172.21.15.7:0/935980945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc0102180 0x7f9fc0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 --2- 172.21.15.7:0/935980945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 0x7f9fc0101c40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 --2- 172.21.15.7:0/935980945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc01005d0 0x7f9fc01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 -- 172.21.15.7:0/935980945 >> 172.21.15.7:0/935980945 conn(0x7f9fc00fbb40 msgr2=0x7f9fc00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 -- 172.21.15.7:0/935980945 shutdown_connections 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 -- 172.21.15.7:0/935980945 wait complete. 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.610+0000 7f9fc588c640 1 Processor -- start 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fc588c640 1 -- start start 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fc588c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc01005d0 0x7f9fc01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fc588c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 0x7f9fc0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fc588c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc0102180 0x7f9fc019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fc588c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc010c250 con 0x7f9fc01017c0 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fc588c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc010c0d0 con 0x7f9fc01005d0 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fc588c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc010c3d0 con 0x7f9fc0102180 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 0x7f9fc0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 0x7f9fc0197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53174/0 (socket says 172.21.15.7:53174) 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbe7fc640 1 -- 172.21.15.7:0/485115313 learned_addr learned my addr 172.21.15.7:0/485115313 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbeffd640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc01005d0 0x7f9fc01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbf7fe640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc0102180 0x7f9fc019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbeffd640 1 -- 172.21.15.7:0/485115313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc0102180 msgr2=0x7f9fc019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbeffd640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc0102180 0x7f9fc019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbeffd640 1 -- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 msgr2=0x7f9fc0197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbeffd640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 0x7f9fc0197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbeffd640 1 -- 172.21.15.7:0/485115313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc019f5e0 con 0x7f9fc01005d0 2024-03-26T08:42:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbe7fc640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 0x7f9fc0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.611+0000 7f9fbeffd640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc01005d0 0x7f9fc01979c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000e9c0 tx=0x7f9fb000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.612+0000 7f9fc488a640 1 -- 172.21.15.7:0/485115313 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000ccc0 con 0x7f9fc01005d0 2024-03-26T08:42:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.612+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc019f8d0 con 0x7f9fc01005d0 2024-03-26T08:42:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.612+0000 7f9fc488a640 1 -- 172.21.15.7:0/485115313 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fb00035f0 con 0x7f9fc01005d0 2024-03-26T08:42:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.612+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc019fe40 con 0x7f9fc01005d0 2024-03-26T08:42:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.612+0000 7f9fc488a640 1 -- 172.21.15.7:0/485115313 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000ce60 con 0x7f9fc01005d0 2024-03-26T08:42:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.613+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc004fc50 con 0x7f9fc01005d0 2024-03-26T08:42:08.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.614+0000 7f9fc488a640 1 -- 172.21.15.7:0/485115313 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9fb00037a0 con 0x7f9fc01005d0 2024-03-26T08:42:08.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.614+0000 7f9fc488a640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fa80a27f0 0x7f9fa80a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:08.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.614+0000 7f9fbe7fc640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fa80a27f0 0x7f9fa80a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:08.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.614+0000 7f9fc488a640 1 -- 172.21.15.7:0/485115313 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9fb00cee30 con 0x7f9fc01005d0 2024-03-26T08:42:08.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.615+0000 7f9fbe7fc640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fa80a27f0 0x7f9fa80a4cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000c00 tx=0x7f9fb402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:08.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.618+0000 7f9fc488a640 1 -- 172.21.15.7:0/485115313 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9fb009bed0 con 0x7f9fc01005d0 2024-03-26T08:42:08.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:08.842+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f9fc01a02f0 con 0x7f9fc01005d0 2024-03-26T08:42:09.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.254+0000 7f9fc488a640 1 -- 172.21.15.7:0/485115313 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9fb009b7d0 con 0x7f9fc01005d0 2024-03-26T08:42:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fa80a27f0 msgr2=0x7f9fa80a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fa80a27f0 0x7f9fa80a4cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000c00 tx=0x7f9fb402c880 comp rx=0 tx=0).stop 2024-03-26T08:42:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc01005d0 msgr2=0x7f9fc01979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc01005d0 0x7f9fc01979c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000e9c0 tx=0x7f9fb000ee80 comp rx=0 tx=0).stop 2024-03-26T08:42:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 shutdown_connections 2024-03-26T08:42:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fa80a27f0 0x7f9fa80a4cb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fc0102180 0x7f9fc019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fc01017c0 0x7f9fc0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 --2- 172.21.15.7:0/485115313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fc01005d0 0x7f9fc01979c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.256+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 >> 172.21.15.7:0/485115313 conn(0x7f9fc00fbb40 msgr2=0x7f9fc0069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.257+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 shutdown_connections 2024-03-26T08:42:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.257+0000 7f9fc588c640 1 -- 172.21.15.7:0/485115313 wait complete. 2024-03-26T08:42:09.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.417+0000 7f7740485640 1 -- 172.21.15.7:0/1967689213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381005d0 msgr2=0x7f77381009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.417+0000 7f7740485640 1 --2- 172.21.15.7:0/1967689213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381005d0 0x7f77381009f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7730009ca0 tx=0x7f773002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.417+0000 7f7740485640 1 -- 172.21.15.7:0/1967689213 shutdown_connections 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.417+0000 7f7740485640 1 --2- 172.21.15.7:0/1967689213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7738102180 0x7f7738108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.417+0000 7f7740485640 1 --2- 172.21.15.7:0/1967689213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77381017c0 0x7f7738101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.417+0000 7f7740485640 1 --2- 172.21.15.7:0/1967689213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381005d0 0x7f77381009f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.417+0000 7f7740485640 1 -- 172.21.15.7:0/1967689213 >> 172.21.15.7:0/1967689213 conn(0x7f77380fbb40 msgr2=0x7f77380fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.417+0000 7f7740485640 1 -- 172.21.15.7:0/1967689213 shutdown_connections 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.417+0000 7f7740485640 1 -- 172.21.15.7:0/1967689213 wait complete. 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f7740485640 1 Processor -- start 2024-03-26T08:42:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f7740485640 1 -- start start 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f7740485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77381005d0 0x7f7738197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f7740485640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381017c0 0x7f7738197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f7740485640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7738102180 0x7f773819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f7740485640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773810d370 con 0x7f77381017c0 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f7740485640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773810d1f0 con 0x7f77381005d0 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f7740485640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773810d4f0 con 0x7f7738102180 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773da20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381017c0 0x7f7738197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773da20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381017c0 0x7f7738197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53186/0 (socket says 172.21.15.7:53186) 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773da20640 1 -- 172.21.15.7:0/552645733 learned_addr learned my addr 172.21.15.7:0/552645733 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773e221640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77381005d0 0x7f7738197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773ea22640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7738102180 0x7f773819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773da20640 1 -- 172.21.15.7:0/552645733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7738102180 msgr2=0x7f773819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773da20640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7738102180 0x7f773819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773da20640 1 -- 172.21.15.7:0/552645733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77381005d0 msgr2=0x7f7738197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773da20640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77381005d0 0x7f7738197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.418+0000 7f773da20640 1 -- 172.21.15.7:0/552645733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773819f670 con 0x7f77381017c0 2024-03-26T08:42:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.419+0000 7f773da20640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381017c0 0x7f7738197f90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f772c00e9f0 tx=0x7f772c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.419+0000 7f77377fe640 1 -- 172.21.15.7:0/552645733 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c00cd80 con 0x7f77381017c0 2024-03-26T08:42:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.419+0000 7f77377fe640 1 -- 172.21.15.7:0/552645733 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f772c010910 con 0x7f77381017c0 2024-03-26T08:42:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.419+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773819f960 con 0x7f77381017c0 2024-03-26T08:42:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.419+0000 7f77377fe640 1 -- 172.21.15.7:0/552645733 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c018990 con 0x7f77381017c0 2024-03-26T08:42:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.419+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773819feb0 con 0x7f77381017c0 2024-03-26T08:42:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.420+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7738107960 con 0x7f77381017c0 2024-03-26T08:42:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.421+0000 7f77377fe640 1 -- 172.21.15.7:0/552645733 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f772c010ac0 con 0x7f77381017c0 2024-03-26T08:42:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.422+0000 7f77377fe640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f77200a27f0 0x7f77200a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.422+0000 7f773e221640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f77200a27f0 0x7f77200a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.422+0000 7f77377fe640 1 -- 172.21.15.7:0/552645733 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f772c0cedf0 con 0x7f77381017c0 2024-03-26T08:42:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.422+0000 7f773e221640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f77200a27f0 0x7f77200a4cb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f773002c700 tx=0x7f7730032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.427+0000 7f77377fe640 1 -- 172.21.15.7:0/552645733 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f772c09b800 con 0x7f77381017c0 2024-03-26T08:42:09.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:09 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/485115313' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T08:42:09.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:09 smithi016 ceph-mon[35851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T08:42:09.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:09 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/485115313' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T08:42:09.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:09 smithi007 ceph-mon[32462]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T08:42:09.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:09 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/485115313' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T08:42:09.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:09 smithi044 ceph-mon[36461]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T08:42:09.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.638+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f7738061b80 con 0x7f77200a27f0 2024-03-26T08:42:09.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.654+0000 7f77377fe640 1 -- 172.21.15.7:0/552645733 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f7738061b80 con 0x7f77200a27f0 2024-03-26T08:42:09.646 INFO:teuthology.orchestra.run.smithi007.stdout:{} 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f77200a27f0 msgr2=0x7f77200a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f77200a27f0 0x7f77200a4cb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f773002c700 tx=0x7f7730032590 comp rx=0 tx=0).stop 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381017c0 msgr2=0x7f7738197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381017c0 0x7f7738197f90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f772c00e9f0 tx=0x7f772c00eeb0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 shutdown_connections 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f77200a27f0 0x7f77200a4cb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7738102180 0x7f773819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77381017c0 0x7f7738197f90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 --2- 172.21.15.7:0/552645733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77381005d0 0x7f7738197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 >> 172.21.15.7:0/552645733 conn(0x7f77380fbb40 msgr2=0x7f7738104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 shutdown_connections 2024-03-26T08:42:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.656+0000 7f7740485640 1 -- 172.21.15.7:0/552645733 wait complete. 2024-03-26T08:42:09.658 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-26T08:42:09.670 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-26T08:42:09.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.792+0000 7f586cd77640 1 -- 172.21.15.7:0/2716066388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868102920 msgr2=0x7f5868102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.792+0000 7f586cd77640 1 --2- 172.21.15.7:0/2716066388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868102920 0x7f5868102da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5850009900 tx=0x7f585002bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.792+0000 7f586cd77640 1 -- 172.21.15.7:0/2716066388 shutdown_connections 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.792+0000 7f586cd77640 1 --2- 172.21.15.7:0/2716066388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58681032e0 0x7f5868109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.792+0000 7f586cd77640 1 --2- 172.21.15.7:0/2716066388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868102920 0x7f5868102da0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.792+0000 7f586cd77640 1 --2- 172.21.15.7:0/2716066388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5868101730 0x7f5868101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.792+0000 7f586cd77640 1 -- 172.21.15.7:0/2716066388 >> 172.21.15.7:0/2716066388 conn(0x7f58680fcc80 msgr2=0x7f58680ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.792+0000 7f586cd77640 1 -- 172.21.15.7:0/2716066388 shutdown_connections 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.792+0000 7f586cd77640 1 -- 172.21.15.7:0/2716066388 wait complete. 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586cd77640 1 Processor -- start 2024-03-26T08:42:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586cd77640 1 -- start start 2024-03-26T08:42:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586cd77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868101730 0x7f5868075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586cd77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868102920 0x7f5868073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586cd77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58681032e0 0x7f5868073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586cd77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586810d320 con 0x7f5868101730 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586cd77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586810d1a0 con 0x7f58681032e0 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586cd77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586810d4a0 con 0x7f5868102920 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868101730 0x7f5868075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f585dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868102920 0x7f5868073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868101730 0x7f5868075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53224/0 (socket says 172.21.15.7:53224) 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f586659c640 1 -- 172.21.15.7:0/1351552380 learned_addr learned my addr 172.21.15.7:0/1351552380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.793+0000 7f5866d9d640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58681032e0 0x7f5868073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f585dd9b640 1 -- 172.21.15.7:0/1351552380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58681032e0 msgr2=0x7f5868073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f585dd9b640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58681032e0 0x7f5868073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f585dd9b640 1 -- 172.21.15.7:0/1351552380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868101730 msgr2=0x7f5868075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f585dd9b640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868101730 0x7f5868075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f585dd9b640 1 -- 172.21.15.7:0/1351552380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5868074860 con 0x7f5868102920 2024-03-26T08:42:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f585dd9b640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868102920 0x7f5868073a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5850009230 tx=0x7f585002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f585ffff640 1 -- 172.21.15.7:0/1351552380 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850041070 con 0x7f5868102920 2024-03-26T08:42:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f585ffff640 1 -- 172.21.15.7:0/1351552380 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5850004450 con 0x7f5868102920 2024-03-26T08:42:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58681a1a20 con 0x7f5868102920 2024-03-26T08:42:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f585ffff640 1 -- 172.21.15.7:0/1351552380 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5850003df0 con 0x7f5868102920 2024-03-26T08:42:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.794+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58681a1f70 con 0x7f5868102920 2024-03-26T08:42:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.795+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58681089f0 con 0x7f5868102920 2024-03-26T08:42:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.796+0000 7f585ffff640 1 -- 172.21.15.7:0/1351552380 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5850044070 con 0x7f5868102920 2024-03-26T08:42:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.796+0000 7f585ffff640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58540a2800 0x7f58540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.797+0000 7f586659c640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58540a2800 0x7f58540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.797+0000 7f585ffff640 1 -- 172.21.15.7:0/1351552380 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f58500f0460 con 0x7f5868102920 2024-03-26T08:42:09.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.797+0000 7f586659c640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58540a2800 0x7f58540a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58580058c0 tx=0x7f5858005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.800+0000 7f585ffff640 1 -- 172.21.15.7:0/1351552380 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f58500bce70 con 0x7f5868102920 2024-03-26T08:42:09.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.989+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f586802d280 con 0x7f5868102920 2024-03-26T08:42:09.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.991+0000 7f585ffff640 1 -- 172.21.15.7:0/1351552380 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7f58500c0f10 con 0x7f5868102920 2024-03-26T08:42:09.983 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:09.983 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:42:09.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58540a2800 msgr2=0x7f58540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58540a2800 0x7f58540a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f58580058c0 tx=0x7f5858005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868102920 msgr2=0x7f5868073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868102920 0x7f5868073a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5850009230 tx=0x7f585002c650 comp rx=0 tx=0).stop 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 shutdown_connections 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58540a2800 0x7f58540a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58681032e0 0x7f5868073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5868102920 0x7f5868073a90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 --2- 172.21.15.7:0/1351552380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5868101730 0x7f5868075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 >> 172.21.15.7:0/1351552380 conn(0x7f58680fcc80 msgr2=0x7f5868105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 shutdown_connections 2024-03-26T08:42:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:09.994+0000 7f586cd77640 1 -- 172.21.15.7:0/1351552380 wait complete. 2024-03-26T08:42:09.996 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:42:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:42:09.643+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:42:09.996 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:42:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:42:09.643+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:42:09.996 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:42:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:42:09.643+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:42:09.996 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:42:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:42:09.643+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:42:09.996 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:42:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:42:09.643+0000 7f0e555d5700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T08:42:09.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:40:08.100906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:40:08.784972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:42:09.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.112+0000 7ff4c784b640 1 -- 172.21.15.7:0/498162186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c01011c0 msgr2=0x7ff4c0108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.112+0000 7ff4c784b640 1 --2- 172.21.15.7:0/498162186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c01011c0 0x7ff4c0108a60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009ca0 tx=0x7ff4b802c140 comp rx=0 tx=0).stop 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.112+0000 7ff4c784b640 1 -- 172.21.15.7:0/498162186 shutdown_connections 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.112+0000 7ff4c784b640 1 --2- 172.21.15.7:0/498162186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c01011c0 0x7ff4c0108a60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.112+0000 7ff4c784b640 1 --2- 172.21.15.7:0/498162186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c0100800 0x7ff4c0100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.112+0000 7ff4c784b640 1 --2- 172.21.15.7:0/498162186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c0101c40 0x7ff4c0102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.112+0000 7ff4c784b640 1 -- 172.21.15.7:0/498162186 >> 172.21.15.7:0/498162186 conn(0x7ff4c00fb2b0 msgr2=0x7ff4c00fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.112+0000 7ff4c784b640 1 -- 172.21.15.7:0/498162186 shutdown_connections 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.112+0000 7ff4c784b640 1 -- 172.21.15.7:0/498162186 wait complete. 2024-03-26T08:42:10.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c784b640 1 Processor -- start 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c784b640 1 -- start start 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c784b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c0100800 0x7ff4c0195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c784b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c01011c0 0x7ff4c0195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c784b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c0101c40 0x7ff4c019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c784b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c010b180 con 0x7ff4c0101c40 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c784b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c010b000 con 0x7ff4c01011c0 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c784b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c010b300 con 0x7ff4c0100800 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c5de8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c0101c40 0x7ff4c019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c5de8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c0101c40 0x7ff4c019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53252/0 (socket says 172.21.15.7:53252) 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c55e7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c0100800 0x7ff4c0195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c5de8640 1 -- 172.21.15.7:0/2796682460 learned_addr learned my addr 172.21.15.7:0/2796682460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c4de6640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c01011c0 0x7ff4c0195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c5de8640 1 -- 172.21.15.7:0/2796682460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c0100800 msgr2=0x7ff4c0195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c5de8640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c0100800 0x7ff4c0195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c5de8640 1 -- 172.21.15.7:0/2796682460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c01011c0 msgr2=0x7ff4c0195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c5de8640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c01011c0 0x7ff4c0195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c5de8640 1 -- 172.21.15.7:0/2796682460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c019d350 con 0x7ff4c0101c40 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.113+0000 7ff4c55e7640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c0100800 0x7ff4c0195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.114+0000 7ff4c5de8640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c0101c40 0x7ff4c019cce0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8005cc0 tx=0x7ff4b8035ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.114+0000 7ff4b67fc640 1 -- 172.21.15.7:0/2796682460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8039040 con 0x7ff4c0101c40 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.114+0000 7ff4b67fc640 1 -- 172.21.15.7:0/2796682460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff4b8044070 con 0x7ff4c0101c40 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.114+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c019d5e0 con 0x7ff4c0101c40 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.114+0000 7ff4b67fc640 1 -- 172.21.15.7:0/2796682460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80043c0 con 0x7ff4c0101c40 2024-03-26T08:42:10.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.114+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c019db20 con 0x7ff4c0101c40 2024-03-26T08:42:10.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.115+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c0107940 con 0x7ff4c0101c40 2024-03-26T08:42:10.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.116+0000 7ff4b67fc640 1 -- 172.21.15.7:0/2796682460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff4b8004560 con 0x7ff4c0101c40 2024-03-26T08:42:10.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.116+0000 7ff4b67fc640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff4ac0a2880 0x7ff4ac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.116+0000 7ff4b67fc640 1 -- 172.21.15.7:0/2796682460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff4b80f0fa0 con 0x7ff4c0101c40 2024-03-26T08:42:10.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.116+0000 7ff4c55e7640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff4ac0a2880 0x7ff4ac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.117+0000 7ff4c55e7640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff4ac0a2880 0x7ff4ac0a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009fc0 tx=0x7ff4bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:10.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.121+0000 7ff4b67fc640 1 -- 172.21.15.7:0/2796682460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff4b80be0e0 con 0x7ff4c0101c40 2024-03-26T08:42:10.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:10 smithi007 ceph-mon[32462]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:42:10.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:10 smithi007 ceph-mon[32462]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T08:42:10.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:10 smithi007 ceph-mon[32462]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:42:10.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:10 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1351552380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.306+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4c00622a0 con 0x7ff4c0101c40 2024-03-26T08:42:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.307+0000 7ff4b67fc640 1 -- 172.21.15.7:0/2796682460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff4b80bd9e0 con 0x7ff4c0101c40 2024-03-26T08:42:10.298 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:42:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff4ac0a2880 msgr2=0x7ff4ac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff4ac0a2880 0x7ff4ac0a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009fc0 tx=0x7ff4bc005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c0101c40 msgr2=0x7ff4c019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c0101c40 0x7ff4c019cce0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8005cc0 tx=0x7ff4b8035ae0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 shutdown_connections 2024-03-26T08:42:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff4ac0a2880 0x7ff4ac0a4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4c0101c40 0x7ff4c019cce0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c01011c0 0x7ff4c0195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 --2- 172.21.15.7:0/2796682460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c0100800 0x7ff4c0195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 >> 172.21.15.7:0/2796682460 conn(0x7ff4c00fb2b0 msgr2=0x7ff4c0104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 shutdown_connections 2024-03-26T08:42:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.309+0000 7ff4c784b640 1 -- 172.21.15.7:0/2796682460 wait complete. 2024-03-26T08:42:10.311 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:42:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 -- 172.21.15.7:0/1793218952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c102940 msgr2=0x7f554c102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 --2- 172.21.15.7:0/1793218952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c102940 0x7f554c102dc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5540009930 tx=0x7f554002bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 -- 172.21.15.7:0/1793218952 shutdown_connections 2024-03-26T08:42:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 --2- 172.21.15.7:0/1793218952 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554c103300 0x7f554c109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 --2- 172.21.15.7:0/1793218952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c102940 0x7f554c102dc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 --2- 172.21.15.7:0/1793218952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554c0ff970 0x7f554c0ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 -- 172.21.15.7:0/1793218952 >> 172.21.15.7:0/1793218952 conn(0x7f554c0fb2c0 msgr2=0x7f554c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 -- 172.21.15.7:0/1793218952 shutdown_connections 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 -- 172.21.15.7:0/1793218952 wait complete. 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 Processor -- start 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.426+0000 7f55525f4640 1 -- start start 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f55525f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c0ff970 0x7f554c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f55525f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554c102940 0x7f554c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f55525f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554c103300 0x7f554c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f55525f4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c10d350 con 0x7f554c0ff970 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f55525f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c10d1d0 con 0x7f554c102940 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f55525f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c10d4d0 con 0x7f554c103300 2024-03-26T08:42:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c0ff970 0x7f554c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c0ff970 0x7f554c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53274/0 (socket says 172.21.15.7:53274) 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554bfff640 1 -- 172.21.15.7:0/3875161971 learned_addr learned my addr 172.21.15.7:0/3875161971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f5550b91640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554c103300 0x7f554c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554b7fe640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554c102940 0x7f554c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554bfff640 1 -- 172.21.15.7:0/3875161971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554c103300 msgr2=0x7f554c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554bfff640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554c103300 0x7f554c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554bfff640 1 -- 172.21.15.7:0/3875161971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554c102940 msgr2=0x7f554c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554bfff640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554c102940 0x7f554c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554bfff640 1 -- 172.21.15.7:0/3875161971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c19f650 con 0x7f554c0ff970 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f554bfff640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c0ff970 0x7f554c197a30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f553c00e9c0 tx=0x7f553c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f55497fa640 1 -- 172.21.15.7:0/3875161971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00cd80 con 0x7f554c0ff970 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.427+0000 7f55497fa640 1 -- 172.21.15.7:0/3875161971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f553c0035f0 con 0x7f554c0ff970 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.428+0000 7f55497fa640 1 -- 172.21.15.7:0/3875161971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c010690 con 0x7f554c0ff970 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.428+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554c19f940 con 0x7f554c0ff970 2024-03-26T08:42:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.428+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554c19fe60 con 0x7f554c0ff970 2024-03-26T08:42:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.428+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554c108be0 con 0x7f554c0ff970 2024-03-26T08:42:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.430+0000 7f55497fa640 1 -- 172.21.15.7:0/3875161971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f553c0037a0 con 0x7f554c0ff970 2024-03-26T08:42:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.430+0000 7f55497fa640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55340a2800 0x7f55340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.430+0000 7f55497fa640 1 -- 172.21.15.7:0/3875161971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f553c0ceef0 con 0x7f554c0ff970 2024-03-26T08:42:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.430+0000 7f554b7fe640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55340a2800 0x7f55340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.431+0000 7f554b7fe640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55340a2800 0x7f55340a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f554002efd0 tx=0x7f5540000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:10.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.434+0000 7f55497fa640 1 -- 172.21.15.7:0/3875161971 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f553c09c020 con 0x7f554c0ff970 2024-03-26T08:42:10.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:10 smithi044 ceph-mon[36461]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:42:10.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:10 smithi044 ceph-mon[36461]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T08:42:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:10 smithi044 ceph-mon[36461]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:42:10.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:10 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1351552380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.618+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f554c02d0f0 con 0x7f554c0ff970 2024-03-26T08:42:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.618+0000 7f55497fa640 1 -- 172.21.15.7:0/3875161971 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f553c09b920 con 0x7f554c0ff970 2024-03-26T08:42:10.610 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-26T08:42:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.620+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55340a2800 msgr2=0x7f55340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.620+0000 7f55525f4640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55340a2800 0x7f55340a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f554002efd0 tx=0x7f5540000bc0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.620+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c0ff970 msgr2=0x7f554c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.620+0000 7f55525f4640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c0ff970 0x7f554c197a30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f553c00e9c0 tx=0x7f553c00ee80 comp rx=0 tx=0).stop 2024-03-26T08:42:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.620+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 shutdown_connections 2024-03-26T08:42:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.621+0000 7f55525f4640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f55340a2800 0x7f55340a4cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.621+0000 7f55525f4640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f554c103300 0x7f554c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.621+0000 7f55525f4640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f554c102940 0x7f554c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.621+0000 7f55525f4640 1 --2- 172.21.15.7:0/3875161971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f554c0ff970 0x7f554c197a30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.621+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 >> 172.21.15.7:0/3875161971 conn(0x7f554c0fb2c0 msgr2=0x7f554c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.621+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 shutdown_connections 2024-03-26T08:42:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.621+0000 7f55525f4640 1 -- 172.21.15.7:0/3875161971 wait complete. 2024-03-26T08:42:10.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:42:10.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.736+0000 7f24d2e3f640 1 -- 172.21.15.7:0/1790961966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1005c0 msgr2=0x7f24cc1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.736+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/1790961966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1005c0 0x7f24cc1009e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009900 tx=0x7f24c402bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:10.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.736+0000 7f24d2e3f640 1 -- 172.21.15.7:0/1790961966 shutdown_connections 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.736+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/1790961966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24cc102170 0x7f24cc108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.736+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/1790961966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc1017b0 0x7f24cc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.736+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/1790961966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1005c0 0x7f24cc1009e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.736+0000 7f24d2e3f640 1 -- 172.21.15.7:0/1790961966 >> 172.21.15.7:0/1790961966 conn(0x7f24cc0fbb30 msgr2=0x7f24cc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 -- 172.21.15.7:0/1790961966 shutdown_connections 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 -- 172.21.15.7:0/1790961966 wait complete. 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 Processor -- start 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 -- start start 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc1005c0 0x7f24cc197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1017b0 0x7f24cc197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24cc102170 0x7f24cc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc10c280 con 0x7f24cc1017b0 2024-03-26T08:42:10.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc10c100 con 0x7f24cc1005c0 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d2e3f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc10c400 con 0x7f24cc102170 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1017b0 0x7f24cc197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1017b0 0x7f24cc197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53294/0 (socket says 172.21.15.7:53294) 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24cbfff640 1 -- 172.21.15.7:0/2248032274 learned_addr learned my addr 172.21.15.7:0/2248032274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d0bdb640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc1005c0 0x7f24cc197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.737+0000 7f24d13dc640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24cc102170 0x7f24cc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24cbfff640 1 -- 172.21.15.7:0/2248032274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24cc102170 msgr2=0x7f24cc19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24cbfff640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24cc102170 0x7f24cc19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24cbfff640 1 -- 172.21.15.7:0/2248032274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc1005c0 msgr2=0x7f24cc197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24cbfff640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc1005c0 0x7f24cc197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24cbfff640 1 -- 172.21.15.7:0/2248032274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc19f650 con 0x7f24cc1017b0 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24cbfff640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1017b0 0x7f24cc197fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00ebc0 tx=0x7f24bc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:10.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24c9ffb640 1 -- 172.21.15.7:0/2248032274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc010630 con 0x7f24cc1017b0 2024-03-26T08:42:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24c9ffb640 1 -- 172.21.15.7:0/2248032274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24bc010cb0 con 0x7f24cc1017b0 2024-03-26T08:42:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24cc19f940 con 0x7f24cc1017b0 2024-03-26T08:42:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24c9ffb640 1 -- 172.21.15.7:0/2248032274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc018bd0 con 0x7f24cc1017b0 2024-03-26T08:42:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.738+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24cc19fee0 con 0x7f24cc1017b0 2024-03-26T08:42:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.739+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24cc1a0110 con 0x7f24cc1017b0 2024-03-26T08:42:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.740+0000 7f24c9ffb640 1 -- 172.21.15.7:0/2248032274 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f24bc0107d0 con 0x7f24cc1017b0 2024-03-26T08:42:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.741+0000 7f24c9ffb640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24b80a2880 0x7f24b80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.741+0000 7f24d0bdb640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24b80a2880 0x7f24b80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.741+0000 7f24c9ffb640 1 -- 172.21.15.7:0/2248032274 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f24bc0cefc0 con 0x7f24cc1017b0 2024-03-26T08:42:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.741+0000 7f24d0bdb640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24b80a2880 0x7f24b80a4d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f24c402c170 tx=0x7f24c402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:10.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.745+0000 7f24c9ffb640 1 -- 172.21.15.7:0/2248032274 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f24bc09ba10 con 0x7f24cc1017b0 2024-03-26T08:42:10.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:10 smithi016 ceph-mon[35851]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T08:42:10.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:10 smithi016 ceph-mon[35851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T08:42:10.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:10 smithi016 ceph-mon[35851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:42:10.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:10 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1351552380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.927+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24cc1a0380 con 0x7f24cc1017b0 2024-03-26T08:42:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.927+0000 7f24c9ffb640 1 -- 172.21.15.7:0/2248032274 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24bc09fab0 con 0x7f24cc1017b0 2024-03-26T08:42:10.919 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-26T08:42:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24b80a2880 msgr2=0x7f24b80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24b80a2880 0x7f24b80a4d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f24c402c170 tx=0x7f24c402df90 comp rx=0 tx=0).stop 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1017b0 msgr2=0x7f24cc197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1017b0 0x7f24cc197fc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00ebc0 tx=0x7f24bc00c6a0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 shutdown_connections 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24b80a2880 0x7f24b80a4d40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f24cc102170 0x7f24cc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24cc1017b0 0x7f24cc197fc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 --2- 172.21.15.7:0/2248032274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc1005c0 0x7f24cc197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 >> 172.21.15.7:0/2248032274 conn(0x7f24cc0fbb30 msgr2=0x7f24cc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 shutdown_connections 2024-03-26T08:42:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:10.929+0000 7f24d2e3f640 1 -- 172.21.15.7:0/2248032274 wait complete. 2024-03-26T08:42:10.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:42:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.042+0000 7faf270a1640 1 -- 172.21.15.7:0/593780219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf201017c0 msgr2=0x7faf20101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.042+0000 7faf270a1640 1 --2- 172.21.15.7:0/593780219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf201017c0 0x7faf20101c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faf10009900 tx=0x7faf1002bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.042+0000 7faf270a1640 1 -- 172.21.15.7:0/593780219 shutdown_connections 2024-03-26T08:42:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.042+0000 7faf270a1640 1 --2- 172.21.15.7:0/593780219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf20102180 0x7faf20108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.042+0000 7faf270a1640 1 --2- 172.21.15.7:0/593780219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf201017c0 0x7faf20101c40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.042+0000 7faf270a1640 1 --2- 172.21.15.7:0/593780219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf201005d0 0x7faf201009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.042+0000 7faf270a1640 1 -- 172.21.15.7:0/593780219 >> 172.21.15.7:0/593780219 conn(0x7faf200fbb40 msgr2=0x7faf200fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.042+0000 7faf270a1640 1 -- 172.21.15.7:0/593780219 shutdown_connections 2024-03-26T08:42:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.042+0000 7faf270a1640 1 -- 172.21.15.7:0/593780219 wait complete. 2024-03-26T08:42:11.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf270a1640 1 Processor -- start 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf270a1640 1 -- start start 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf270a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf201005d0 0x7faf20199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf270a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf201017c0 0x7faf2019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf270a1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf20102180 0x7faf201a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf270a1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2010c230 con 0x7faf20102180 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf270a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2010c0b0 con 0x7faf201017c0 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf270a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2010c3b0 con 0x7faf201005d0 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf2563e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf20102180 0x7faf201a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf2563e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf20102180 0x7faf201a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53320/0 (socket says 172.21.15.7:53320) 2024-03-26T08:42:11.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf2563e640 1 -- 172.21.15.7:0/288791824 learned_addr learned my addr 172.21.15.7:0/288791824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf24e3d640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf201005d0 0x7faf20199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.043+0000 7faf1ffff640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf201017c0 0x7faf2019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf2563e640 1 -- 172.21.15.7:0/288791824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf201005d0 msgr2=0x7faf20199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf2563e640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf201005d0 0x7faf20199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf2563e640 1 -- 172.21.15.7:0/288791824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf201017c0 msgr2=0x7faf2019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf2563e640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf201017c0 0x7faf2019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf2563e640 1 -- 172.21.15.7:0/288791824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf201a17e0 con 0x7faf20102180 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf2563e640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf20102180 0x7faf201a1170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e970 tx=0x7faf0800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf1dffb640 1 -- 172.21.15.7:0/288791824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800ccc0 con 0x7faf20102180 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf1dffb640 1 -- 172.21.15.7:0/288791824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf0800ce60 con 0x7faf20102180 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf201a1ad0 con 0x7faf20102180 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf1dffb640 1 -- 172.21.15.7:0/288791824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08010490 con 0x7faf20102180 2024-03-26T08:42:11.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.044+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf201a1fa0 con 0x7faf20102180 2024-03-26T08:42:11.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.045+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf201a2290 con 0x7faf20102180 2024-03-26T08:42:11.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.046+0000 7faf1dffb640 1 -- 172.21.15.7:0/288791824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7faf08005020 con 0x7faf20102180 2024-03-26T08:42:11.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.047+0000 7faf1dffb640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faf0c0a2800 0x7faf0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.047+0000 7faf1dffb640 1 -- 172.21.15.7:0/288791824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faf080ceee0 con 0x7faf20102180 2024-03-26T08:42:11.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.047+0000 7faf24e3d640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faf0c0a2800 0x7faf0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.047+0000 7faf24e3d640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faf0c0a2800 0x7faf0c0a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf20101620 tx=0x7faf18009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:11.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.051+0000 7faf1dffb640 1 -- 172.21.15.7:0/288791824 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faf0809bf80 con 0x7faf20102180 2024-03-26T08:42:11.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.237+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf201a24b0 con 0x7faf20102180 2024-03-26T08:42:11.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.239+0000 7faf1dffb640 1 -- 172.21.15.7:0/288791824 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7faf0809b880 con 0x7faf20102180 2024-03-26T08:42:11.231 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:11.232 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faf0c0a2800 msgr2=0x7faf0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faf0c0a2800 0x7faf0c0a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf20101620 tx=0x7faf18009500 comp rx=0 tx=0).stop 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf20102180 msgr2=0x7faf201a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf20102180 0x7faf201a1170 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e970 tx=0x7faf0800ed70 comp rx=0 tx=0).stop 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 shutdown_connections 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faf0c0a2800 0x7faf0c0a4cc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf20102180 0x7faf201a1170 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf201017c0 0x7faf2019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 --2- 172.21.15.7:0/288791824 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf201005d0 0x7faf20199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 >> 172.21.15.7:0/288791824 conn(0x7faf200fbb40 msgr2=0x7faf200fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:11.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 shutdown_connections 2024-03-26T08:42:11.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.241+0000 7faf270a1640 1 -- 172.21.15.7:0/288791824 wait complete. 2024-03-26T08:42:11.245 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:40:08.100906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:40:08.784972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:42:11.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:42:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.359+0000 7fc6b3275640 1 -- 172.21.15.7:0/2668650575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 msgr2=0x7fc6ac108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.359+0000 7fc6b3275640 1 --2- 172.21.15.7:0/2668650575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 0x7fc6ac108a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009900 tx=0x7fc69c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.359+0000 7fc6b3275640 1 -- 172.21.15.7:0/2668650575 shutdown_connections 2024-03-26T08:42:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 --2- 172.21.15.7:0/2668650575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 0x7fc6ac108a00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 --2- 172.21.15.7:0/2668650575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ac1017b0 0x7fc6ac101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 --2- 172.21.15.7:0/2668650575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6ac1005c0 0x7fc6ac1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 -- 172.21.15.7:0/2668650575 >> 172.21.15.7:0/2668650575 conn(0x7fc6ac0fbb30 msgr2=0x7fc6ac0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 -- 172.21.15.7:0/2668650575 shutdown_connections 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 -- 172.21.15.7:0/2668650575 wait complete. 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 Processor -- start 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 -- start start 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ac1005c0 0x7fc6ac0694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6ac1017b0 0x7fc6ac19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 0x7fc6ac19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac10c240 con 0x7fc6ac102170 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac10c0c0 con 0x7fc6ac1017b0 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b3275640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac10c3c0 con 0x7fc6ac1005c0 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.360+0000 7fc6b1812640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 0x7fc6ac19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b1011640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ac1005c0 0x7fc6ac0694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b1812640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 0x7fc6ac19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53334/0 (socket says 172.21.15.7:53334) 2024-03-26T08:42:11.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b1812640 1 -- 172.21.15.7:0/461184687 learned_addr learned my addr 172.21.15.7:0/461184687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b0810640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6ac1017b0 0x7fc6ac19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b1812640 1 -- 172.21.15.7:0/461184687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ac1005c0 msgr2=0x7fc6ac0694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b1812640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ac1005c0 0x7fc6ac0694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b1812640 1 -- 172.21.15.7:0/461184687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6ac1017b0 msgr2=0x7fc6ac19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b1812640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6ac1017b0 0x7fc6ac19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b1812640 1 -- 172.21.15.7:0/461184687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac19f630 con 0x7fc6ac102170 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b1812640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 0x7fc6ac19ed20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc69c005ab0 tx=0x7fc69c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6aa7fc640 1 -- 172.21.15.7:0/461184687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c041070 con 0x7fc6ac102170 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6aa7fc640 1 -- 172.21.15.7:0/461184687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc69c004450 con 0x7fc6ac102170 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac19f860 con 0x7fc6ac102170 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6aa7fc640 1 -- 172.21.15.7:0/461184687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c003e60 con 0x7fc6ac102170 2024-03-26T08:42:11.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.361+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac19fda0 con 0x7fc6ac102170 2024-03-26T08:42:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.362+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ac1078e0 con 0x7fc6ac102170 2024-03-26T08:42:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.363+0000 7fc6aa7fc640 1 -- 172.21.15.7:0/461184687 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc69c044070 con 0x7fc6ac102170 2024-03-26T08:42:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.364+0000 7fc6aa7fc640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6940a2880 0x7fc6940a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.364+0000 7fc6b1011640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6940a2880 0x7fc6940a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.364+0000 7fc6aa7fc640 1 -- 172.21.15.7:0/461184687 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc69c0f09e0 con 0x7fc6ac102170 2024-03-26T08:42:11.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.365+0000 7fc6b1011640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6940a2880 0x7fc6940a4d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac101610 tx=0x7fc6a4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.367+0000 7fc6aa7fc640 1 -- 172.21.15.7:0/461184687 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc69c0bd3f0 con 0x7fc6ac102170 2024-03-26T08:42:11.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:11 smithi007 ceph-mon[32462]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:42:11.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2796682460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:11.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3875161971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:11.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2248032274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:11.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/288791824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.552+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6ac02d200 con 0x7fc6ac102170 2024-03-26T08:42:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.553+0000 7fc6aa7fc640 1 -- 172.21.15.7:0/461184687 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc69c0c1490 con 0x7fc6ac102170 2024-03-26T08:42:11.545 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:42:11.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6940a2880 msgr2=0x7fc6940a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6940a2880 0x7fc6940a4d40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac101610 tx=0x7fc6a4009500 comp rx=0 tx=0).stop 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 msgr2=0x7fc6ac19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 0x7fc6ac19ed20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc69c005ab0 tx=0x7fc69c02c6c0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 shutdown_connections 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6940a2880 0x7fc6940a4d40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6ac102170 0x7fc6ac19ed20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6ac1017b0 0x7fc6ac19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 --2- 172.21.15.7:0/461184687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6ac1005c0 0x7fc6ac0694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 >> 172.21.15.7:0/461184687 conn(0x7fc6ac0fbb30 msgr2=0x7fc6ac104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 shutdown_connections 2024-03-26T08:42:11.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.555+0000 7fc6b3275640 1 -- 172.21.15.7:0/461184687 wait complete. 2024-03-26T08:42:11.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:42:11.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:11 smithi044 ceph-mon[36461]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:42:11.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2796682460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:11.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3875161971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:11.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2248032274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:11.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/288791824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.672+0000 7f0192fba640 1 -- 172.21.15.7:0/3591931539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c1016b0 msgr2=0x7f018c101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.672+0000 7f0192fba640 1 --2- 172.21.15.7:0/3591931539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c1016b0 0x7f018c101ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0180009900 tx=0x7f018002bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.672+0000 7f0192fba640 1 -- 172.21.15.7:0/3591931539 shutdown_connections 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.672+0000 7f0192fba640 1 --2- 172.21.15.7:0/3591931539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c103260 0x7f018c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.672+0000 7f0192fba640 1 --2- 172.21.15.7:0/3591931539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018c1028a0 0x7f018c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.672+0000 7f0192fba640 1 --2- 172.21.15.7:0/3591931539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c1016b0 0x7f018c101ad0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.672+0000 7f0192fba640 1 -- 172.21.15.7:0/3591931539 >> 172.21.15.7:0/3591931539 conn(0x7f018c0fcc20 msgr2=0x7f018c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.672+0000 7f0192fba640 1 -- 172.21.15.7:0/3591931539 shutdown_connections 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.672+0000 7f0192fba640 1 -- 172.21.15.7:0/3591931539 wait complete. 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0192fba640 1 Processor -- start 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0192fba640 1 -- start start 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0192fba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c1016b0 0x7f018c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0192fba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018c1028a0 0x7f018c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0192fba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c103260 0x7f018c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0192fba640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c10d330 con 0x7f018c103260 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0192fba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c10d1b0 con 0x7f018c1016b0 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0192fba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c10d4b0 con 0x7f018c1028a0 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0191557640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c103260 0x7f018c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0191557640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c103260 0x7f018c19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53344/0 (socket says 172.21.15.7:53344) 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0191557640 1 -- 172.21.15.7:0/1110115946 learned_addr learned my addr 172.21.15.7:0/1110115946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0190d56640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c1016b0 0x7f018c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f018bfff640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018c1028a0 0x7f018c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0191557640 1 -- 172.21.15.7:0/1110115946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018c1028a0 msgr2=0x7f018c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0191557640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018c1028a0 0x7f018c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0191557640 1 -- 172.21.15.7:0/1110115946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c1016b0 msgr2=0x7f018c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0191557640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c1016b0 0x7f018c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0191557640 1 -- 172.21.15.7:0/1110115946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c19f620 con 0x7f018c103260 2024-03-26T08:42:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.673+0000 7f0191557640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c103260 0x7f018c19efb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f018400eb10 tx=0x7f018400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.674+0000 7f0189ffb640 1 -- 172.21.15.7:0/1110115946 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184004b20 con 0x7f018c103260 2024-03-26T08:42:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.674+0000 7f0189ffb640 1 -- 172.21.15.7:0/1110115946 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0184004cc0 con 0x7f018c103260 2024-03-26T08:42:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.674+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018c19f910 con 0x7f018c103260 2024-03-26T08:42:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.674+0000 7f0189ffb640 1 -- 172.21.15.7:0/1110115946 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184010760 con 0x7f018c103260 2024-03-26T08:42:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.674+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018c19fe60 con 0x7f018c103260 2024-03-26T08:42:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.675+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018c108a50 con 0x7f018c103260 2024-03-26T08:42:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.676+0000 7f0189ffb640 1 -- 172.21.15.7:0/1110115946 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0184005020 con 0x7f018c103260 2024-03-26T08:42:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.677+0000 7f0189ffb640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f017c0a2880 0x7f017c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.677+0000 7f0190d56640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f017c0a2880 0x7f017c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.677+0000 7f0189ffb640 1 -- 172.21.15.7:0/1110115946 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f01840cef80 con 0x7f018c103260 2024-03-26T08:42:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.677+0000 7f0190d56640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f017c0a2880 0x7f017c0a4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0180000c00 tx=0x7f0180005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.681+0000 7f0189ffb640 1 -- 172.21.15.7:0/1110115946 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f018409bfe0 con 0x7f018c103260 2024-03-26T08:42:11.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:11 smithi016 ceph-mon[35851]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T08:42:11.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2796682460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:11.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3875161971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:11.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2248032274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:11.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/288791824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.864+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f018c0622a0 con 0x7f018c103260 2024-03-26T08:42:11.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.864+0000 7f0189ffb640 1 -- 172.21.15.7:0/1110115946 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f018409b8e0 con 0x7f018c103260 2024-03-26T08:42:11.857 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f017c0a2880 msgr2=0x7f017c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f017c0a2880 0x7f017c0a4d40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0180000c00 tx=0x7f0180005fd0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c103260 msgr2=0x7f018c19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c103260 0x7f018c19efb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f018400eb10 tx=0x7f018400c2d0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 shutdown_connections 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f017c0a2880 0x7f017c0a4d40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f018c103260 0x7f018c19efb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f018c1028a0 0x7f018c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 --2- 172.21.15.7:0/1110115946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f018c1016b0 0x7f018c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 >> 172.21.15.7:0/1110115946 conn(0x7f018c0fcc20 msgr2=0x7f018c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 shutdown_connections 2024-03-26T08:42:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.866+0000 7f0192fba640 1 -- 172.21.15.7:0/1110115946 wait complete. 2024-03-26T08:42:11.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:42:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 -- 172.21.15.7:0/1817797156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1016b0 msgr2=0x7f4a6c101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 --2- 172.21.15.7:0/1817797156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1016b0 0x7f4a6c101ad0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009900 tx=0x7f4a5c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 -- 172.21.15.7:0/1817797156 shutdown_connections 2024-03-26T08:42:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 --2- 172.21.15.7:0/1817797156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c103260 0x7f4a6c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 --2- 172.21.15.7:0/1817797156 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c1028a0 0x7f4a6c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 --2- 172.21.15.7:0/1817797156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1016b0 0x7f4a6c101ad0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 -- 172.21.15.7:0/1817797156 >> 172.21.15.7:0/1817797156 conn(0x7f4a6c0fcc20 msgr2=0x7f4a6c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 -- 172.21.15.7:0/1817797156 shutdown_connections 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 -- 172.21.15.7:0/1817797156 wait complete. 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 Processor -- start 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.983+0000 7f4a72875640 1 -- start start 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a72875640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c1016b0 0x7f4a6c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a72875640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1028a0 0x7f4a6c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a72875640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c103260 0x7f4a6c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a72875640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c10d300 con 0x7f4a6c1028a0 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a72875640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c10d180 con 0x7f4a6c1016b0 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a72875640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c10d480 con 0x7f4a6c103260 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1028a0 0x7f4a6c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1028a0 0x7f4a6c197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53356/0 (socket says 172.21.15.7:53356) 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6b7fe640 1 -- 172.21.15.7:0/977873484 learned_addr learned my addr 172.21.15.7:0/977873484 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a70e12640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c103260 0x7f4a6c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6bfff640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c1016b0 0x7f4a6c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6b7fe640 1 -- 172.21.15.7:0/977873484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c103260 msgr2=0x7f4a6c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6b7fe640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c103260 0x7f4a6c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6b7fe640 1 -- 172.21.15.7:0/977873484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c1016b0 msgr2=0x7f4a6c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6b7fe640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c1016b0 0x7f4a6c197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6b7fe640 1 -- 172.21.15.7:0/977873484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c19f650 con 0x7f4a6c1028a0 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.984+0000 7f4a6b7fe640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1028a0 0x7f4a6c197f70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400ea60 tx=0x7f4a5400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.985+0000 7f4a697fa640 1 -- 172.21.15.7:0/977873484 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400cd20 con 0x7f4a6c1028a0 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.985+0000 7f4a697fa640 1 -- 172.21.15.7:0/977873484 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a540035f0 con 0x7f4a6c1028a0 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.985+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6c19f940 con 0x7f4a6c1028a0 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.985+0000 7f4a697fa640 1 -- 172.21.15.7:0/977873484 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54010690 con 0x7f4a6c1028a0 2024-03-26T08:42:11.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.985+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c19fe90 con 0x7f4a6c1028a0 2024-03-26T08:42:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.986+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c108aa0 con 0x7f4a6c1028a0 2024-03-26T08:42:11.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.987+0000 7f4a697fa640 1 -- 172.21.15.7:0/977873484 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4a540037a0 con 0x7f4a6c1028a0 2024-03-26T08:42:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.987+0000 7f4a697fa640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4a580a2800 0x7f4a580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.987+0000 7f4a6bfff640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4a580a2800 0x7f4a580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.987+0000 7f4a697fa640 1 -- 172.21.15.7:0/977873484 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4a540ced70 con 0x7f4a6c1028a0 2024-03-26T08:42:11.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.988+0000 7f4a6bfff640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4a580a2800 0x7f4a580a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c000c00 tx=0x7f4a5c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:11.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:11.992+0000 7f4a697fa640 1 -- 172.21.15.7:0/977873484 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4a540d3070 con 0x7f4a6c1028a0 2024-03-26T08:42:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.175+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a6c0622a0 con 0x7f4a6c1028a0 2024-03-26T08:42:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.175+0000 7f4a697fa640 1 -- 172.21.15.7:0/977873484 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a5409be10 con 0x7f4a6c1028a0 2024-03-26T08:42:12.168 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-26T08:42:12.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4a580a2800 msgr2=0x7f4a580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4a580a2800 0x7f4a580a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c000c00 tx=0x7f4a5c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1028a0 msgr2=0x7f4a6c197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1028a0 0x7f4a6c197f70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400ea60 tx=0x7f4a5400ee60 comp rx=0 tx=0).stop 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 shutdown_connections 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4a580a2800 0x7f4a580a4cc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4a6c103260 0x7f4a6c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a6c1028a0 0x7f4a6c197f70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 --2- 172.21.15.7:0/977873484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c1016b0 0x7f4a6c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 >> 172.21.15.7:0/977873484 conn(0x7f4a6c0fcc20 msgr2=0x7f4a6c105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 shutdown_connections 2024-03-26T08:42:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.177+0000 7f4a72875640 1 -- 172.21.15.7:0/977873484 wait complete. 2024-03-26T08:42:12.181 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-26T08:42:12.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:42:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 -- 172.21.15.7:0/3499612965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 msgr2=0x7fafb41045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/3499612965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 0x7fafb41045a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009900 tx=0x7fafa802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 -- 172.21.15.7:0/3499612965 shutdown_connections 2024-03-26T08:42:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/3499612965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4107410 0x7fafb4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/3499612965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafb4104ae0 0x7fafb4106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/3499612965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 0x7fafb41045a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 -- 172.21.15.7:0/3499612965 >> 172.21.15.7:0/3499612965 conn(0x7fafb40fbb40 msgr2=0x7fafb40fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 -- 172.21.15.7:0/3499612965 shutdown_connections 2024-03-26T08:42:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 -- 172.21.15.7:0/3499612965 wait complete. 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 Processor -- start 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.296+0000 7fafbbbc6640 1 -- start start 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafbbbc6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 0x7fafb41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafbbbc6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafb4104ae0 0x7fafb4197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafbbbc6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4107410 0x7fafb419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafbbbc6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb410d350 con 0x7fafb41021b0 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafbbbc6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb410d1d0 con 0x7fafb4107410 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafbbbc6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb410d4d0 con 0x7fafb4104ae0 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9962640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 0x7fafb41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9161640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafb4104ae0 0x7fafb4197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9962640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 0x7fafb41979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53378/0 (socket says 172.21.15.7:53378) 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafba163640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4107410 0x7fafb419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9962640 1 -- 172.21.15.7:0/1524551418 learned_addr learned my addr 172.21.15.7:0/1524551418 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9161640 1 -- 172.21.15.7:0/1524551418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4107410 msgr2=0x7fafb419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9161640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4107410 0x7fafb419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9161640 1 -- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 msgr2=0x7fafb41979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9161640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 0x7fafb41979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9161640 1 -- 172.21.15.7:0/1524551418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb419f5f0 con 0x7fafb4104ae0 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9962640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 0x7fafb41979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafba163640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4107410 0x7fafb419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafb9161640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafb4104ae0 0x7fafb4197f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fafb000e9c0 tx=0x7fafb000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafa6ffd640 1 -- 172.21.15.7:0/1524551418 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb000ccc0 con 0x7fafb4104ae0 2024-03-26T08:42:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb419f8e0 con 0x7fafb4104ae0 2024-03-26T08:42:12.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafa6ffd640 1 -- 172.21.15.7:0/1524551418 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafb0010910 con 0x7fafb4104ae0 2024-03-26T08:42:12.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.297+0000 7fafa6ffd640 1 -- 172.21.15.7:0/1524551418 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0018880 con 0x7fafb4104ae0 2024-03-26T08:42:12.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.298+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb419fe80 con 0x7fafb4104ae0 2024-03-26T08:42:12.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.299+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb40fd630 con 0x7fafb4104ae0 2024-03-26T08:42:12.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.299+0000 7fafa6ffd640 1 -- 172.21.15.7:0/1524551418 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fafb0010ac0 con 0x7fafb4104ae0 2024-03-26T08:42:12.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.300+0000 7fafa6ffd640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fafa00a2890 0x7fafa00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.300+0000 7fafb9962640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fafa00a2890 0x7fafa00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.300+0000 7fafa6ffd640 1 -- 172.21.15.7:0/1524551418 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fafb00cef90 con 0x7fafb4104ae0 2024-03-26T08:42:12.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.300+0000 7fafb9962640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fafa00a2890 0x7fafa00a4d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000c00 tx=0x7fafa802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.306+0000 7fafa6ffd640 1 -- 172.21.15.7:0/1524551418 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fafb009b840 con 0x7fafb4104ae0 2024-03-26T08:42:12.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.529+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafb402d220 con 0x7fafb4104ae0 2024-03-26T08:42:12.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.530+0000 7fafa6ffd640 1 -- 172.21.15.7:0/1524551418 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fafb009f8e0 con 0x7fafb4104ae0 2024-03-26T08:42:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:12.522 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:42:08.844026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24478,24502]},"id":1}]} 2024-03-26T08:42:12.523 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fafa00a2890 msgr2=0x7fafa00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fafa00a2890 0x7fafa00a4d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000c00 tx=0x7fafa802c880 comp rx=0 tx=0).stop 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafb4104ae0 msgr2=0x7fafb4197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafb4104ae0 0x7fafb4197f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fafb000e9c0 tx=0x7fafb000ee80 comp rx=0 tx=0).stop 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 shutdown_connections 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fafa00a2890 0x7fafa00a4d50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb4107410 0x7fafb419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fafb4104ae0 0x7fafb4197f10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 --2- 172.21.15.7:0/1524551418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafb41021b0 0x7fafb41979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 >> 172.21.15.7:0/1524551418 conn(0x7fafb40fbb40 msgr2=0x7fafb4102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 shutdown_connections 2024-03-26T08:42:12.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.532+0000 7fafbbbc6640 1 -- 172.21.15.7:0/1524551418 wait complete. 2024-03-26T08:42:12.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:12 smithi007 ceph-mon[32462]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-26T08:42:12.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:12 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/461184687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:12.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:12 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1110115946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:12.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:12 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/977873484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:12.535 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-26T08:42:12.535 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-26T08:42:12.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T08:42:12.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:12 smithi044 ceph-mon[36461]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-26T08:42:12.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:12 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/461184687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:12.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:12 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1110115946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:12.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:12 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/977873484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 -- 172.21.15.7:0/549467326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 msgr2=0x7f6020101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 --2- 172.21.15.7:0/549467326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 0x7f6020101cb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6008009900 tx=0x7f600802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 -- 172.21.15.7:0/549467326 shutdown_connections 2024-03-26T08:42:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 --2- 172.21.15.7:0/549467326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60201021f0 0x7f6020108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 --2- 172.21.15.7:0/549467326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 0x7f6020101cb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 --2- 172.21.15.7:0/549467326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020100640 0x7f6020100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 -- 172.21.15.7:0/549467326 >> 172.21.15.7:0/549467326 conn(0x7f60200fbbd0 msgr2=0x7f60200fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 -- 172.21.15.7:0/549467326 shutdown_connections 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 -- 172.21.15.7:0/549467326 wait complete. 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.646+0000 7f6027036640 1 Processor -- start 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f6027036640 1 -- start start 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f6027036640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020100640 0x7f60201979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f6027036640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 0x7f6020197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f6027036640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60201021f0 0x7f602019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f6027036640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602010c250 con 0x7f6020101830 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f6027036640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602010c0d0 con 0x7f6020100640 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f6027036640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602010c3d0 con 0x7f60201021f0 2024-03-26T08:42:12.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 0x7f6020197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f6024dd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020100640 0x7f60201979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 0x7f6020197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53392/0 (socket says 172.21.15.7:53392) 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601ffff640 1 -- 172.21.15.7:0/209219583 learned_addr learned my addr 172.21.15.7:0/209219583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f60255d3640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60201021f0 0x7f602019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601ffff640 1 -- 172.21.15.7:0/209219583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60201021f0 msgr2=0x7f602019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601ffff640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60201021f0 0x7f602019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601ffff640 1 -- 172.21.15.7:0/209219583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020100640 msgr2=0x7f60201979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601ffff640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020100640 0x7f60201979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601ffff640 1 -- 172.21.15.7:0/209219583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602019f5d0 con 0x7f6020101830 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f6024dd2640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020100640 0x7f60201979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f60255d3640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60201021f0 0x7f602019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601ffff640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 0x7f6020197ef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6008009230 tx=0x7f600802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.647+0000 7f601dffb640 1 -- 172.21.15.7:0/209219583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008041070 con 0x7f6020101830 2024-03-26T08:42:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.648+0000 7f601dffb640 1 -- 172.21.15.7:0/209219583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6008004450 con 0x7f6020101830 2024-03-26T08:42:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.648+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602019f860 con 0x7f6020101830 2024-03-26T08:42:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.648+0000 7f601dffb640 1 -- 172.21.15.7:0/209219583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008003e20 con 0x7f6020101830 2024-03-26T08:42:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.648+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602019fd50 con 0x7f6020101830 2024-03-26T08:42:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.649+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60201a0070 con 0x7f6020101830 2024-03-26T08:42:12.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.650+0000 7f601dffb640 1 -- 172.21.15.7:0/209219583 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f6008044070 con 0x7f6020101830 2024-03-26T08:42:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.650+0000 7f601dffb640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f600c0a2910 0x7f600c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.650+0000 7f601dffb640 1 -- 172.21.15.7:0/209219583 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f60080f0a50 con 0x7f6020101830 2024-03-26T08:42:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.650+0000 7f6024dd2640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f600c0a2910 0x7f600c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.651+0000 7f6024dd2640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f600c0a2910 0x7f600c0a4dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6020101690 tx=0x7f6010005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:12.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.654+0000 7f601dffb640 1 -- 172.21.15.7:0/209219583 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60080bd410 con 0x7f6020101830 2024-03-26T08:42:12.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:12 smithi016 ceph-mon[35851]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-26T08:42:12.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:12 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/461184687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:12.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:12 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1110115946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:12.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:12 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/977873484' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.832+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60200fdd10 con 0x7f600c0a2910 2024-03-26T08:42:12.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.840+0000 7f601dffb640 1 -- 172.21.15.7:0/209219583 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259090 (secure 0 0 0) 0x7f60200fdd10 con 0x7f600c0a2910 2024-03-26T08:42:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:12.837 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-26T08:42:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f600c0a2910 msgr2=0x7f600c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f600c0a2910 0x7f600c0a4dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6020101690 tx=0x7f6010005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 msgr2=0x7f6020197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 0x7f6020197ef0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6008009230 tx=0x7f600802c650 comp rx=0 tx=0).stop 2024-03-26T08:42:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 shutdown_connections 2024-03-26T08:42:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f600c0a2910 0x7f600c0a4dd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60201021f0 0x7f602019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6020101830 0x7f6020197ef0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 --2- 172.21.15.7:0/209219583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6020100640 0x7f60201979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 >> 172.21.15.7:0/209219583 conn(0x7f60200fbbd0 msgr2=0x7f60200fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 shutdown_connections 2024-03-26T08:42:12.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.843+0000 7f6027036640 1 -- 172.21.15.7:0/209219583 wait complete. 2024-03-26T08:42:12.846 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-03-26T08:42:12.329724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666166,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":855,"num_read_kb":553,"num_write":354,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373236,"kb_used_data":11604,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568588,"statfs":{"total":1151940427776,"available":1151558234112,"internally_reserved":0,"allocated":11882496,"data_stored":7471167,"data_compressed":177645,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6771,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":54,"num_write":8,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002952"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353784+0000","last_change":"2024-03-26T08:40:09.939078+0000","last_active":"2024-03-26T08:41:28.353784+0000","last_peered":"2024-03-26T08:41:28.353784+0000","last_clean":"2024-03-26T08:41:28.353784+0000","last_became_active":"2024-03-26T08:40:09.937626+0000","last_became_peered":"2024-03-26T08:40:09.937626+0000","last_unstale":"2024-03-26T08:41:28.353784+0000","last_undegraded":"2024-03-26T08:41:28.353784+0000","last_fullsized":"2024-03-26T08:41:28.353784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:52:59.254181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345392+0000","last_change":"2024-03-26T08:40:09.924207+0000","last_active":"2024-03-26T08:41:28.345392+0000","last_peered":"2024-03-26T08:41:28.345392+0000","last_clean":"2024-03-26T08:41:28.345392+0000","last_became_active":"2024-03-26T08:40:09.923881+0000","last_became_peered":"2024-03-26T08:40:09.923881+0000","last_unstale":"2024-03-26T08:41:28.345392+0000","last_undegraded":"2024-03-26T08:41:28.345392+0000","last_fullsized":"2024-03-26T08:41:28.345392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:16:45.265521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349461+0000","last_change":"2024-03-26T08:40:09.929154+0000","last_active":"2024-03-26T08:41:28.349461+0000","last_peered":"2024-03-26T08:41:28.349461+0000","last_clean":"2024-03-26T08:41:28.349461+0000","last_became_active":"2024-03-26T08:40:09.928833+0000","last_became_peered":"2024-03-26T08:40:09.928833+0000","last_unstale":"2024-03-26T08:41:28.349461+0000","last_undegraded":"2024-03-26T08:41:28.349461+0000","last_fullsized":"2024-03-26T08:41:28.349461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:22.972210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.341510+0000","last_change":"2024-03-26T08:40:09.930713+0000","last_active":"2024-03-26T08:41:28.341510+0000","last_peered":"2024-03-26T08:41:28.341510+0000","last_clean":"2024-03-26T08:41:28.341510+0000","last_became_active":"2024-03-26T08:40:09.930420+0000","last_became_peered":"2024-03-26T08:40:09.930420+0000","last_unstale":"2024-03-26T08:41:28.341510+0000","last_undegraded":"2024-03-26T08:41:28.341510+0000","last_fullsized":"2024-03-26T08:41:28.341510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:17:03.015703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.336520+0000","last_change":"2024-03-26T08:40:09.923935+0000","last_active":"2024-03-26T08:41:28.336520+0000","last_peered":"2024-03-26T08:41:28.336520+0000","last_clean":"2024-03-26T08:41:28.336520+0000","last_became_active":"2024-03-26T08:40:09.923587+0000","last_became_peered":"2024-03-26T08:40:09.923587+0000","last_unstale":"2024-03-26T08:41:28.336520+0000","last_undegraded":"2024-03-26T08:41:28.336520+0000","last_fullsized":"2024-03-26T08:41:28.336520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:19:38.831448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344419+0000","last_change":"2024-03-26T08:40:09.922361+0000","last_active":"2024-03-26T08:41:28.344419+0000","last_peered":"2024-03-26T08:41:28.344419+0000","last_clean":"2024-03-26T08:41:28.344419+0000","last_became_active":"2024-03-26T08:40:09.921643+0000","last_became_peered":"2024-03-26T08:40:09.921643+0000","last_unstale":"2024-03-26T08:41:28.344419+0000","last_undegraded":"2024-03-26T08:41:28.344419+0000","last_fullsized":"2024-03-26T08:41:28.344419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:37:23.048778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.341431+0000","last_change":"2024-03-26T08:40:09.932428+0000","last_active":"2024-03-26T08:41:28.341431+0000","last_peered":"2024-03-26T08:41:28.341431+0000","last_clean":"2024-03-26T08:41:28.341431+0000","last_became_active":"2024-03-26T08:40:09.932120+0000","last_became_peered":"2024-03-26T08:40:09.932120+0000","last_unstale":"2024-03-26T08:41:28.341431+0000","last_undegraded":"2024-03-26T08:41:28.341431+0000","last_fullsized":"2024-03-26T08:41:28.341431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:24.362568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351911+0000","last_change":"2024-03-26T08:40:09.928251+0000","last_active":"2024-03-26T08:41:28.351911+0000","last_peered":"2024-03-26T08:41:28.351911+0000","last_clean":"2024-03-26T08:41:28.351911+0000","last_became_active":"2024-03-26T08:40:09.927961+0000","last_became_peered":"2024-03-26T08:40:09.927961+0000","last_unstale":"2024-03-26T08:41:28.351911+0000","last_undegraded":"2024-03-26T08:41:28.351911+0000","last_fullsized":"2024-03-26T08:41:28.351911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:25:09.348033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351308+0000","last_change":"2024-03-26T08:40:09.930888+0000","last_active":"2024-03-26T08:41:28.351308+0000","last_peered":"2024-03-26T08:41:28.351308+0000","last_clean":"2024-03-26T08:41:28.351308+0000","last_became_active":"2024-03-26T08:40:09.930513+0000","last_became_peered":"2024-03-26T08:40:09.930513+0000","last_unstale":"2024-03-26T08:41:28.351308+0000","last_undegraded":"2024-03-26T08:41:28.351308+0000","last_fullsized":"2024-03-26T08:41:28.351308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:21.023569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345352+0000","last_change":"2024-03-26T08:40:09.935534+0000","last_active":"2024-03-26T08:41:28.345352+0000","last_peered":"2024-03-26T08:41:28.345352+0000","last_clean":"2024-03-26T08:41:28.345352+0000","last_became_active":"2024-03-26T08:40:09.934959+0000","last_became_peered":"2024-03-26T08:40:09.934959+0000","last_unstale":"2024-03-26T08:41:28.345352+0000","last_undegraded":"2024-03-26T08:41:28.345352+0000","last_fullsized":"2024-03-26T08:41:28.345352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:18:59.311180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351875+0000","last_change":"2024-03-26T08:40:09.930486+0000","last_active":"2024-03-26T08:41:28.351875+0000","last_peered":"2024-03-26T08:41:28.351875+0000","last_clean":"2024-03-26T08:41:28.351875+0000","last_became_active":"2024-03-26T08:40:09.930184+0000","last_became_peered":"2024-03-26T08:40:09.930184+0000","last_unstale":"2024-03-26T08:41:28.351875+0000","last_undegraded":"2024-03-26T08:41:28.351875+0000","last_fullsized":"2024-03-26T08:41:28.351875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:07.246878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345052+0000","last_change":"2024-03-26T08:40:09.924305+0000","last_active":"2024-03-26T08:41:28.345052+0000","last_peered":"2024-03-26T08:41:28.345052+0000","last_clean":"2024-03-26T08:41:28.345052+0000","last_became_active":"2024-03-26T08:40:09.923985+0000","last_became_peered":"2024-03-26T08:40:09.923985+0000","last_unstale":"2024-03-26T08:41:28.345052+0000","last_undegraded":"2024-03-26T08:41:28.345052+0000","last_fullsized":"2024-03-26T08:41:28.345052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:42:39.894245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350807+0000","last_change":"2024-03-26T08:40:09.916637+0000","last_active":"2024-03-26T08:41:28.350807+0000","last_peered":"2024-03-26T08:41:28.350807+0000","last_clean":"2024-03-26T08:41:28.350807+0000","last_became_active":"2024-03-26T08:40:09.916314+0000","last_became_peered":"2024-03-26T08:40:09.916314+0000","last_unstale":"2024-03-26T08:41:28.350807+0000","last_undegraded":"2024-03-26T08:41:28.350807+0000","last_fullsized":"2024-03-26T08:41:28.350807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:45:10.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345246+0000","last_change":"2024-03-26T08:40:09.923753+0000","last_active":"2024-03-26T08:41:28.345246+0000","last_peered":"2024-03-26T08:41:28.345246+0000","last_clean":"2024-03-26T08:41:28.345246+0000","last_became_active":"2024-03-26T08:40:09.923427+0000","last_became_peered":"2024-03-26T08:40:09.923427+0000","last_unstale":"2024-03-26T08:41:28.345246+0000","last_undegraded":"2024-03-26T08:41:28.345246+0000","last_fullsized":"2024-03-26T08:41:28.345246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:13:16.711556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338474+0000","last_change":"2024-03-26T08:40:09.931879+0000","last_active":"2024-03-26T08:41:28.338474+0000","last_peered":"2024-03-26T08:41:28.338474+0000","last_clean":"2024-03-26T08:41:28.338474+0000","last_became_active":"2024-03-26T08:40:09.931589+0000","last_became_peered":"2024-03-26T08:40:09.931589+0000","last_unstale":"2024-03-26T08:41:28.338474+0000","last_undegraded":"2024-03-26T08:41:28.338474+0000","last_fullsized":"2024-03-26T08:41:28.338474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:22:01.038745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353522+0000","last_change":"2024-03-26T08:40:09.930500+0000","last_active":"2024-03-26T08:41:28.353522+0000","last_peered":"2024-03-26T08:41:28.353522+0000","last_clean":"2024-03-26T08:41:28.353522+0000","last_became_active":"2024-03-26T08:40:09.930101+0000","last_became_peered":"2024-03-26T08:40:09.930101+0000","last_unstale":"2024-03-26T08:41:28.353522+0000","last_undegraded":"2024-03-26T08:41:28.353522+0000","last_fullsized":"2024-03-26T08:41:28.353522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:54.229004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.354043+0000","last_change":"2024-03-26T08:40:09.939177+0000","last_active":"2024-03-26T08:41:28.354043+0000","last_peered":"2024-03-26T08:41:28.354043+0000","last_clean":"2024-03-26T08:41:28.354043+0000","last_became_active":"2024-03-26T08:40:09.937727+0000","last_became_peered":"2024-03-26T08:40:09.937727+0000","last_unstale":"2024-03-26T08:41:28.354043+0000","last_undegraded":"2024-03-26T08:41:28.354043+0000","last_fullsized":"2024-03-26T08:41:28.354043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:40:44.040015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350313+0000","last_change":"2024-03-26T08:40:09.928429+0000","last_active":"2024-03-26T08:41:28.350313+0000","last_peered":"2024-03-26T08:41:28.350313+0000","last_clean":"2024-03-26T08:41:28.350313+0000","last_became_active":"2024-03-26T08:40:09.928025+0000","last_became_peered":"2024-03-26T08:40:09.928025+0000","last_unstale":"2024-03-26T08:41:28.350313+0000","last_undegraded":"2024-03-26T08:41:28.350313+0000","last_fullsized":"2024-03-26T08:41:28.350313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:52:55.450291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345617+0000","last_change":"2024-03-26T08:40:09.930856+0000","last_active":"2024-03-26T08:41:28.345617+0000","last_peered":"2024-03-26T08:41:28.345617+0000","last_clean":"2024-03-26T08:41:28.345617+0000","last_became_active":"2024-03-26T08:40:09.930552+0000","last_became_peered":"2024-03-26T08:40:09.930552+0000","last_unstale":"2024-03-26T08:41:28.345617+0000","last_undegraded":"2024-03-26T08:41:28.345617+0000","last_fullsized":"2024-03-26T08:41:28.345617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:52.851287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421132+0000","last_change":"2024-03-26T08:40:09.230609+0000","last_active":"2024-03-26T08:40:59.421132+0000","last_peered":"2024-03-26T08:40:59.421132+0000","last_clean":"2024-03-26T08:40:59.421132+0000","last_became_active":"2024-03-26T08:40:09.230284+0000","last_became_peered":"2024-03-26T08:40:09.230284+0000","last_unstale":"2024-03-26T08:40:59.421132+0000","last_undegraded":"2024-03-26T08:40:59.421132+0000","last_fullsized":"2024-03-26T08:40:59.421132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:06.681725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.337071+0000","last_change":"2024-03-26T08:40:59.425449+0000","last_active":"2024-03-26T08:41:28.337071+0000","last_peered":"2024-03-26T08:41:28.337071+0000","last_clean":"2024-03-26T08:41:28.337071+0000","last_became_active":"2024-03-26T08:40:59.425164+0000","last_became_peered":"2024-03-26T08:40:59.425164+0000","last_unstale":"2024-03-26T08:41:28.337071+0000","last_undegraded":"2024-03-26T08:41:28.337071+0000","last_fullsized":"2024-03-26T08:41:28.337071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:09.846252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419667+0000","last_change":"2024-03-26T08:40:09.235451+0000","last_active":"2024-03-26T08:40:59.419667+0000","last_peered":"2024-03-26T08:40:59.419667+0000","last_clean":"2024-03-26T08:40:59.419667+0000","last_became_active":"2024-03-26T08:40:09.235154+0000","last_became_peered":"2024-03-26T08:40:09.235154+0000","last_unstale":"2024-03-26T08:40:59.419667+0000","last_undegraded":"2024-03-26T08:40:59.419667+0000","last_fullsized":"2024-03-26T08:40:59.419667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:37:38.418888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349428+0000","last_change":"2024-03-26T08:40:09.931728+0000","last_active":"2024-03-26T08:41:28.349428+0000","last_peered":"2024-03-26T08:41:28.349428+0000","last_clean":"2024-03-26T08:41:28.349428+0000","last_became_active":"2024-03-26T08:40:09.931401+0000","last_became_peered":"2024-03-26T08:40:09.931401+0000","last_unstale":"2024-03-26T08:41:28.349428+0000","last_undegraded":"2024-03-26T08:41:28.349428+0000","last_fullsized":"2024-03-26T08:41:28.349428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:02.229763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417066+0000","last_change":"2024-03-26T08:40:09.231247+0000","last_active":"2024-03-26T08:40:59.417066+0000","last_peered":"2024-03-26T08:40:59.417066+0000","last_clean":"2024-03-26T08:40:59.417066+0000","last_became_active":"2024-03-26T08:40:09.230914+0000","last_became_peered":"2024-03-26T08:40:09.230914+0000","last_unstale":"2024-03-26T08:40:59.417066+0000","last_undegraded":"2024-03-26T08:40:59.417066+0000","last_fullsized":"2024-03-26T08:40:59.417066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:03.956444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353346+0000","last_change":"2024-03-26T08:40:09.925174+0000","last_active":"2024-03-26T08:41:28.353346+0000","last_peered":"2024-03-26T08:41:28.353346+0000","last_clean":"2024-03-26T08:41:28.353346+0000","last_became_active":"2024-03-26T08:40:09.924840+0000","last_became_peered":"2024-03-26T08:40:09.924840+0000","last_unstale":"2024-03-26T08:41:28.353346+0000","last_undegraded":"2024-03-26T08:41:28.353346+0000","last_fullsized":"2024-03-26T08:41:28.353346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:57:19.041245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416818+0000","last_change":"2024-03-26T08:40:09.229504+0000","last_active":"2024-03-26T08:40:59.416818+0000","last_peered":"2024-03-26T08:40:59.416818+0000","last_clean":"2024-03-26T08:40:59.416818+0000","last_became_active":"2024-03-26T08:40:09.226449+0000","last_became_peered":"2024-03-26T08:40:09.226449+0000","last_unstale":"2024-03-26T08:40:59.416818+0000","last_undegraded":"2024-03-26T08:40:59.416818+0000","last_fullsized":"2024-03-26T08:40:59.416818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:38.047424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351371+0000","last_change":"2024-03-26T08:40:09.930184+0000","last_active":"2024-03-26T08:41:28.351371+0000","last_peered":"2024-03-26T08:41:28.351371+0000","last_clean":"2024-03-26T08:41:28.351371+0000","last_became_active":"2024-03-26T08:40:09.927679+0000","last_became_peered":"2024-03-26T08:40:09.927679+0000","last_unstale":"2024-03-26T08:41:28.351371+0000","last_undegraded":"2024-03-26T08:41:28.351371+0000","last_fullsized":"2024-03-26T08:41:28.351371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:05:52.949814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'9","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.404325+0000","last_change":"2024-03-26T08:40:09.233025+0000","last_active":"2024-03-26T08:42:10.404325+0000","last_peered":"2024-03-26T08:42:10.404325+0000","last_clean":"2024-03-26T08:42:10.404325+0000","last_became_active":"2024-03-26T08:40:09.232654+0000","last_became_peered":"2024-03-26T08:40:09.232654+0000","last_unstale":"2024-03-26T08:42:10.404325+0000","last_undegraded":"2024-03-26T08:42:10.404325+0000","last_fullsized":"2024-03-26T08:42:10.404325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:01.745583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.333641+0000","last_change":"2024-03-26T08:40:09.924187+0000","last_active":"2024-03-26T08:41:28.333641+0000","last_peered":"2024-03-26T08:41:28.333641+0000","last_clean":"2024-03-26T08:41:28.333641+0000","last_became_active":"2024-03-26T08:40:09.923861+0000","last_became_peered":"2024-03-26T08:40:09.923861+0000","last_unstale":"2024-03-26T08:41:28.333641+0000","last_undegraded":"2024-03-26T08:41:28.333641+0000","last_fullsized":"2024-03-26T08:41:28.333641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:12:17.284186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502123+0000","last_change":"2024-03-26T08:40:09.231368+0000","last_active":"2024-03-26T08:40:59.502123+0000","last_peered":"2024-03-26T08:40:59.502123+0000","last_clean":"2024-03-26T08:40:59.502123+0000","last_became_active":"2024-03-26T08:40:09.231040+0000","last_became_peered":"2024-03-26T08:40:09.231040+0000","last_unstale":"2024-03-26T08:40:59.502123+0000","last_undegraded":"2024-03-26T08:40:59.502123+0000","last_fullsized":"2024-03-26T08:40:59.502123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:14:20.681225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.348766+0000","last_change":"2024-03-26T08:40:09.928125+0000","last_active":"2024-03-26T08:41:28.348766+0000","last_peered":"2024-03-26T08:41:28.348766+0000","last_clean":"2024-03-26T08:41:28.348766+0000","last_became_active":"2024-03-26T08:40:09.927616+0000","last_became_peered":"2024-03-26T08:40:09.927616+0000","last_unstale":"2024-03-26T08:41:28.348766+0000","last_undegraded":"2024-03-26T08:41:28.348766+0000","last_fullsized":"2024-03-26T08:41:28.348766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:12.764054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.462999+0000","last_change":"2024-03-26T08:40:09.229976+0000","last_active":"2024-03-26T08:40:59.462999+0000","last_peered":"2024-03-26T08:40:59.462999+0000","last_clean":"2024-03-26T08:40:59.462999+0000","last_became_active":"2024-03-26T08:40:09.229594+0000","last_became_peered":"2024-03-26T08:40:09.229594+0000","last_unstale":"2024-03-26T08:40:59.462999+0000","last_undegraded":"2024-03-26T08:40:59.462999+0000","last_fullsized":"2024-03-26T08:40:59.462999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:31:24.204853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350174+0000","last_change":"2024-03-26T08:40:09.931290+0000","last_active":"2024-03-26T08:41:28.350174+0000","last_peered":"2024-03-26T08:41:28.350174+0000","last_clean":"2024-03-26T08:41:28.350174+0000","last_became_active":"2024-03-26T08:40:09.930961+0000","last_became_peered":"2024-03-26T08:40:09.930961+0000","last_unstale":"2024-03-26T08:41:28.350174+0000","last_undegraded":"2024-03-26T08:41:28.350174+0000","last_fullsized":"2024-03-26T08:41:28.350174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:36.460807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419243+0000","last_change":"2024-03-26T08:40:09.232166+0000","last_active":"2024-03-26T08:40:59.419243+0000","last_peered":"2024-03-26T08:40:59.419243+0000","last_clean":"2024-03-26T08:40:59.419243+0000","last_became_active":"2024-03-26T08:40:09.231840+0000","last_became_peered":"2024-03-26T08:40:09.231840+0000","last_unstale":"2024-03-26T08:40:59.419243+0000","last_undegraded":"2024-03-26T08:40:59.419243+0000","last_fullsized":"2024-03-26T08:40:59.419243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:51.636322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352211+0000","last_change":"2024-03-26T08:40:09.924680+0000","last_active":"2024-03-26T08:41:28.352211+0000","last_peered":"2024-03-26T08:41:28.352211+0000","last_clean":"2024-03-26T08:41:28.352211+0000","last_became_active":"2024-03-26T08:40:09.924314+0000","last_became_peered":"2024-03-26T08:40:09.924314+0000","last_unstale":"2024-03-26T08:41:28.352211+0000","last_undegraded":"2024-03-26T08:41:28.352211+0000","last_fullsized":"2024-03-26T08:41:28.352211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:49:20.780950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:09.402862+0000","last_change":"2024-03-26T08:40:09.235997+0000","last_active":"2024-03-26T08:42:09.402862+0000","last_peered":"2024-03-26T08:42:09.402862+0000","last_clean":"2024-03-26T08:42:09.402862+0000","last_became_active":"2024-03-26T08:40:09.235729+0000","last_became_peered":"2024-03-26T08:40:09.235729+0000","last_unstale":"2024-03-26T08:42:09.402862+0000","last_undegraded":"2024-03-26T08:42:09.402862+0000","last_fullsized":"2024-03-26T08:42:09.402862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:46:12.322059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351871+0000","last_change":"2024-03-26T08:40:09.926371+0000","last_active":"2024-03-26T08:41:28.351871+0000","last_peered":"2024-03-26T08:41:28.351871+0000","last_clean":"2024-03-26T08:41:28.351871+0000","last_became_active":"2024-03-26T08:40:09.925536+0000","last_became_peered":"2024-03-26T08:40:09.925536+0000","last_unstale":"2024-03-26T08:41:28.351871+0000","last_undegraded":"2024-03-26T08:41:28.351871+0000","last_fullsized":"2024-03-26T08:41:28.351871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:50:39.101684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417249+0000","last_change":"2024-03-26T08:40:09.229018+0000","last_active":"2024-03-26T08:40:59.417249+0000","last_peered":"2024-03-26T08:40:59.417249+0000","last_clean":"2024-03-26T08:40:59.417249+0000","last_became_active":"2024-03-26T08:40:09.228680+0000","last_became_peered":"2024-03-26T08:40:09.228680+0000","last_unstale":"2024-03-26T08:40:59.417249+0000","last_undegraded":"2024-03-26T08:40:59.417249+0000","last_fullsized":"2024-03-26T08:40:59.417249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:31:10.065127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338799+0000","last_change":"2024-03-26T08:40:09.926152+0000","last_active":"2024-03-26T08:41:28.338799+0000","last_peered":"2024-03-26T08:41:28.338799+0000","last_clean":"2024-03-26T08:41:28.338799+0000","last_became_active":"2024-03-26T08:40:09.925825+0000","last_became_peered":"2024-03-26T08:40:09.925825+0000","last_unstale":"2024-03-26T08:41:28.338799+0000","last_undegraded":"2024-03-26T08:41:28.338799+0000","last_fullsized":"2024-03-26T08:41:28.338799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:28:24.287706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419184+0000","last_change":"2024-03-26T08:40:09.234699+0000","last_active":"2024-03-26T08:40:59.419184+0000","last_peered":"2024-03-26T08:40:59.419184+0000","last_clean":"2024-03-26T08:40:59.419184+0000","last_became_active":"2024-03-26T08:40:09.234451+0000","last_became_peered":"2024-03-26T08:40:09.234451+0000","last_unstale":"2024-03-26T08:40:59.419184+0000","last_undegraded":"2024-03-26T08:40:59.419184+0000","last_fullsized":"2024-03-26T08:40:59.419184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:53.337418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352387+0000","last_change":"2024-03-26T08:40:09.927023+0000","last_active":"2024-03-26T08:41:28.352387+0000","last_peered":"2024-03-26T08:41:28.352387+0000","last_clean":"2024-03-26T08:41:28.352387+0000","last_became_active":"2024-03-26T08:40:09.926734+0000","last_became_peered":"2024-03-26T08:40:09.926734+0000","last_unstale":"2024-03-26T08:41:28.352387+0000","last_undegraded":"2024-03-26T08:41:28.352387+0000","last_fullsized":"2024-03-26T08:41:28.352387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:19:44.137513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":297,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.403406+0000","last_change":"2024-03-26T08:40:09.230059+0000","last_active":"2024-03-26T08:42:10.403406+0000","last_peered":"2024-03-26T08:42:10.403406+0000","last_clean":"2024-03-26T08:42:10.403406+0000","last_became_active":"2024-03-26T08:40:09.229776+0000","last_became_peered":"2024-03-26T08:40:09.229776+0000","last_unstale":"2024-03-26T08:42:10.403406+0000","last_undegraded":"2024-03-26T08:42:10.403406+0000","last_fullsized":"2024-03-26T08:42:10.403406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:00.917694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70723,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":242,"num_read_kb":190,"num_write":37,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.334293+0000","last_change":"2024-03-26T08:40:09.930702+0000","last_active":"2024-03-26T08:41:28.334293+0000","last_peered":"2024-03-26T08:41:28.334293+0000","last_clean":"2024-03-26T08:41:28.334293+0000","last_became_active":"2024-03-26T08:40:09.930400+0000","last_became_peered":"2024-03-26T08:40:09.930400+0000","last_unstale":"2024-03-26T08:41:28.334293+0000","last_undegraded":"2024-03-26T08:41:28.334293+0000","last_fullsized":"2024-03-26T08:41:28.334293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:03:12.897812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422276+0000","last_change":"2024-03-26T08:40:09.225972+0000","last_active":"2024-03-26T08:40:59.422276+0000","last_peered":"2024-03-26T08:40:59.422276+0000","last_clean":"2024-03-26T08:40:59.422276+0000","last_became_active":"2024-03-26T08:40:09.225524+0000","last_became_peered":"2024-03-26T08:40:09.225524+0000","last_unstale":"2024-03-26T08:40:59.422276+0000","last_undegraded":"2024-03-26T08:40:59.422276+0000","last_fullsized":"2024-03-26T08:40:59.422276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:08:17.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.348903+0000","last_change":"2024-03-26T08:40:09.920628+0000","last_active":"2024-03-26T08:41:28.348903+0000","last_peered":"2024-03-26T08:41:28.348903+0000","last_clean":"2024-03-26T08:41:28.348903+0000","last_became_active":"2024-03-26T08:40:09.920300+0000","last_became_peered":"2024-03-26T08:40:09.920300+0000","last_unstale":"2024-03-26T08:41:28.348903+0000","last_undegraded":"2024-03-26T08:41:28.348903+0000","last_fullsized":"2024-03-26T08:41:28.348903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:02:16.503881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419684+0000","last_change":"2024-03-26T08:40:09.237802+0000","last_active":"2024-03-26T08:40:59.419684+0000","last_peered":"2024-03-26T08:40:59.419684+0000","last_clean":"2024-03-26T08:40:59.419684+0000","last_became_active":"2024-03-26T08:40:09.237425+0000","last_became_peered":"2024-03-26T08:40:09.237425+0000","last_unstale":"2024-03-26T08:40:59.419684+0000","last_undegraded":"2024-03-26T08:40:59.419684+0000","last_fullsized":"2024-03-26T08:40:59.419684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:05.953882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345665+0000","last_change":"2024-03-26T08:40:09.923221+0000","last_active":"2024-03-26T08:41:28.345665+0000","last_peered":"2024-03-26T08:41:28.345665+0000","last_clean":"2024-03-26T08:41:28.345665+0000","last_became_active":"2024-03-26T08:40:09.922894+0000","last_became_peered":"2024-03-26T08:40:09.922894+0000","last_unstale":"2024-03-26T08:41:28.345665+0000","last_undegraded":"2024-03-26T08:41:28.345665+0000","last_fullsized":"2024-03-26T08:41:28.345665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:24.887625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":141,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.408450+0000","last_change":"2024-03-26T08:40:09.225179+0000","last_active":"2024-03-26T08:42:10.408450+0000","last_peered":"2024-03-26T08:42:10.408450+0000","last_clean":"2024-03-26T08:42:10.408450+0000","last_became_active":"2024-03-26T08:40:09.224861+0000","last_became_peered":"2024-03-26T08:40:09.224861+0000","last_unstale":"2024-03-26T08:42:10.408450+0000","last_undegraded":"2024-03-26T08:42:10.408450+0000","last_fullsized":"2024-03-26T08:42:10.408450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:37:54.843284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.334405+0000","last_change":"2024-03-26T08:40:59.503355+0000","last_active":"2024-03-26T08:41:28.334405+0000","last_peered":"2024-03-26T08:41:28.334405+0000","last_clean":"2024-03-26T08:41:28.334405+0000","last_became_active":"2024-03-26T08:40:59.503009+0000","last_became_peered":"2024-03-26T08:40:59.503009+0000","last_unstale":"2024-03-26T08:41:28.334405+0000","last_undegraded":"2024-03-26T08:41:28.334405+0000","last_fullsized":"2024-03-26T08:41:28.334405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:08:20.694763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416469+0000","last_change":"2024-03-26T08:40:09.232024+0000","last_active":"2024-03-26T08:40:59.416469+0000","last_peered":"2024-03-26T08:40:59.416469+0000","last_clean":"2024-03-26T08:40:59.416469+0000","last_became_active":"2024-03-26T08:40:09.231715+0000","last_became_peered":"2024-03-26T08:40:09.231715+0000","last_unstale":"2024-03-26T08:40:59.416469+0000","last_undegraded":"2024-03-26T08:40:59.416469+0000","last_fullsized":"2024-03-26T08:40:59.416469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:02.229042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351134+0000","last_change":"2024-03-26T08:40:09.926893+0000","last_active":"2024-03-26T08:41:28.351134+0000","last_peered":"2024-03-26T08:41:28.351134+0000","last_clean":"2024-03-26T08:41:28.351134+0000","last_became_active":"2024-03-26T08:40:09.926529+0000","last_became_peered":"2024-03-26T08:40:09.926529+0000","last_unstale":"2024-03-26T08:41:28.351134+0000","last_undegraded":"2024-03-26T08:41:28.351134+0000","last_fullsized":"2024-03-26T08:41:28.351134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:10.189407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.428905+0000","last_change":"2024-03-26T08:40:59.428905+0000","last_active":"2024-03-26T08:40:59.428905+0000","last_peered":"2024-03-26T08:40:59.428905+0000","last_clean":"2024-03-26T08:40:59.428905+0000","last_became_active":"2024-03-26T08:40:59.423345+0000","last_became_peered":"2024-03-26T08:40:59.423345+0000","last_unstale":"2024-03-26T08:40:59.428905+0000","last_undegraded":"2024-03-26T08:40:59.428905+0000","last_fullsized":"2024-03-26T08:40:59.428905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:49.069421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.346046+0000","last_change":"2024-03-26T08:40:59.464333+0000","last_active":"2024-03-26T08:41:28.346046+0000","last_peered":"2024-03-26T08:41:28.346046+0000","last_clean":"2024-03-26T08:41:28.346046+0000","last_became_active":"2024-03-26T08:40:59.464016+0000","last_became_peered":"2024-03-26T08:40:59.464016+0000","last_unstale":"2024-03-26T08:41:28.346046+0000","last_undegraded":"2024-03-26T08:41:28.346046+0000","last_fullsized":"2024-03-26T08:41:28.346046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:50:05.770568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420974+0000","last_change":"2024-03-26T08:40:09.224385+0000","last_active":"2024-03-26T08:40:59.420974+0000","last_peered":"2024-03-26T08:40:59.420974+0000","last_clean":"2024-03-26T08:40:59.420974+0000","last_became_active":"2024-03-26T08:40:09.223991+0000","last_became_peered":"2024-03-26T08:40:09.223991+0000","last_unstale":"2024-03-26T08:40:59.420974+0000","last_undegraded":"2024-03-26T08:40:59.420974+0000","last_fullsized":"2024-03-26T08:40:59.420974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:55.808922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349692+0000","last_change":"2024-03-26T08:40:09.921072+0000","last_active":"2024-03-26T08:41:28.349692+0000","last_peered":"2024-03-26T08:41:28.349692+0000","last_clean":"2024-03-26T08:41:28.349692+0000","last_became_active":"2024-03-26T08:40:09.920747+0000","last_became_peered":"2024-03-26T08:40:09.920747+0000","last_unstale":"2024-03-26T08:41:28.349692+0000","last_undegraded":"2024-03-26T08:41:28.349692+0000","last_fullsized":"2024-03-26T08:41:28.349692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:20.862765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416822+0000","last_change":"2024-03-26T08:40:09.231344+0000","last_active":"2024-03-26T08:40:59.416822+0000","last_peered":"2024-03-26T08:40:59.416822+0000","last_clean":"2024-03-26T08:40:59.416822+0000","last_became_active":"2024-03-26T08:40:09.229313+0000","last_became_peered":"2024-03-26T08:40:09.229313+0000","last_unstale":"2024-03-26T08:40:59.416822+0000","last_undegraded":"2024-03-26T08:40:59.416822+0000","last_fullsized":"2024-03-26T08:40:59.416822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:37.130884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.343324+0000","last_change":"2024-03-26T08:40:09.926355+0000","last_active":"2024-03-26T08:41:28.343324+0000","last_peered":"2024-03-26T08:41:28.343324+0000","last_clean":"2024-03-26T08:41:28.343324+0000","last_became_active":"2024-03-26T08:40:09.922889+0000","last_became_peered":"2024-03-26T08:40:09.922889+0000","last_unstale":"2024-03-26T08:41:28.343324+0000","last_undegraded":"2024-03-26T08:41:28.343324+0000","last_fullsized":"2024-03-26T08:41:28.343324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:42:22.784811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344672+0000","last_change":"2024-03-26T08:40:09.919577+0000","last_active":"2024-03-26T08:41:28.344672+0000","last_peered":"2024-03-26T08:41:28.344672+0000","last_clean":"2024-03-26T08:41:28.344672+0000","last_became_active":"2024-03-26T08:40:09.919234+0000","last_became_peered":"2024-03-26T08:40:09.919234+0000","last_unstale":"2024-03-26T08:41:28.344672+0000","last_undegraded":"2024-03-26T08:41:28.344672+0000","last_fullsized":"2024-03-26T08:41:28.344672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:18.894052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.408078+0000","last_change":"2024-03-26T08:40:09.237190+0000","last_active":"2024-03-26T08:42:10.408078+0000","last_peered":"2024-03-26T08:42:10.408078+0000","last_clean":"2024-03-26T08:42:10.408078+0000","last_became_active":"2024-03-26T08:40:09.236862+0000","last_became_peered":"2024-03-26T08:40:09.236862+0000","last_unstale":"2024-03-26T08:42:10.408078+0000","last_undegraded":"2024-03-26T08:42:10.408078+0000","last_fullsized":"2024-03-26T08:42:10.408078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:29.900711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351087+0000","last_change":"2024-03-26T08:40:09.927444+0000","last_active":"2024-03-26T08:41:28.351087+0000","last_peered":"2024-03-26T08:41:28.351087+0000","last_clean":"2024-03-26T08:41:28.351087+0000","last_became_active":"2024-03-26T08:40:09.927152+0000","last_became_peered":"2024-03-26T08:40:09.927152+0000","last_unstale":"2024-03-26T08:41:28.351087+0000","last_undegraded":"2024-03-26T08:41:28.351087+0000","last_fullsized":"2024-03-26T08:41:28.351087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:44.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420563+0000","last_change":"2024-03-26T08:40:09.224831+0000","last_active":"2024-03-26T08:40:59.420563+0000","last_peered":"2024-03-26T08:40:59.420563+0000","last_clean":"2024-03-26T08:40:59.420563+0000","last_became_active":"2024-03-26T08:40:09.224447+0000","last_became_peered":"2024-03-26T08:40:09.224447+0000","last_unstale":"2024-03-26T08:40:59.420563+0000","last_undegraded":"2024-03-26T08:40:59.420563+0000","last_fullsized":"2024-03-26T08:40:59.420563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:25:50.935342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416753+0000","last_change":"2024-03-26T08:38:54.696342+0000","last_active":"2024-03-26T08:40:59.416753+0000","last_peered":"2024-03-26T08:40:59.416753+0000","last_clean":"2024-03-26T08:40:59.416753+0000","last_became_active":"2024-03-26T08:38:54.695839+0000","last_became_peered":"2024-03-26T08:38:54.695839+0000","last_unstale":"2024-03-26T08:40:59.416753+0000","last_undegraded":"2024-03-26T08:40:59.416753+0000","last_fullsized":"2024-03-26T08:40:59.416753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_clean_scrub_stamp":"2024-03-26T08:35:27.189917+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:13.219778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.343262+0000","last_change":"2024-03-26T08:40:09.930683+0000","last_active":"2024-03-26T08:41:28.343262+0000","last_peered":"2024-03-26T08:41:28.343262+0000","last_clean":"2024-03-26T08:41:28.343262+0000","last_became_active":"2024-03-26T08:40:09.930341+0000","last_became_peered":"2024-03-26T08:40:09.930341+0000","last_unstale":"2024-03-26T08:41:28.343262+0000","last_undegraded":"2024-03-26T08:41:28.343262+0000","last_fullsized":"2024-03-26T08:41:28.343262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:39:20.232007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416894+0000","last_change":"2024-03-26T08:40:09.228548+0000","last_active":"2024-03-26T08:40:59.416894+0000","last_peered":"2024-03-26T08:40:59.416894+0000","last_clean":"2024-03-26T08:40:59.416894+0000","last_became_active":"2024-03-26T08:40:09.228076+0000","last_became_peered":"2024-03-26T08:40:09.228076+0000","last_unstale":"2024-03-26T08:40:59.416894+0000","last_undegraded":"2024-03-26T08:40:59.416894+0000","last_fullsized":"2024-03-26T08:40:59.416894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:52:51.641128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416941+0000","last_change":"2024-03-26T08:40:09.235632+0000","last_active":"2024-03-26T08:40:59.416941+0000","last_peered":"2024-03-26T08:40:59.416941+0000","last_clean":"2024-03-26T08:40:59.416941+0000","last_became_active":"2024-03-26T08:40:09.235334+0000","last_became_peered":"2024-03-26T08:40:09.235334+0000","last_unstale":"2024-03-26T08:40:59.416941+0000","last_undegraded":"2024-03-26T08:40:59.416941+0000","last_fullsized":"2024-03-26T08:40:59.416941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:38:27.064431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.341593+0000","last_change":"2024-03-26T08:40:09.929137+0000","last_active":"2024-03-26T08:41:28.341593+0000","last_peered":"2024-03-26T08:41:28.341593+0000","last_clean":"2024-03-26T08:41:28.341593+0000","last_became_active":"2024-03-26T08:40:09.928831+0000","last_became_peered":"2024-03-26T08:40:09.928831+0000","last_unstale":"2024-03-26T08:41:28.341593+0000","last_undegraded":"2024-03-26T08:41:28.341593+0000","last_fullsized":"2024-03-26T08:41:28.341593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:19.235168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:29.257611+0000","last_change":"2024-03-26T08:40:09.227153+0000","last_active":"2024-03-26T08:41:29.257611+0000","last_peered":"2024-03-26T08:41:29.257611+0000","last_clean":"2024-03-26T08:41:29.257611+0000","last_became_active":"2024-03-26T08:40:09.226772+0000","last_became_peered":"2024-03-26T08:40:09.226772+0000","last_unstale":"2024-03-26T08:41:29.257611+0000","last_undegraded":"2024-03-26T08:41:29.257611+0000","last_fullsized":"2024-03-26T08:41:29.257611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:33.159538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350141+0000","last_change":"2024-03-26T08:40:09.934589+0000","last_active":"2024-03-26T08:41:28.350141+0000","last_peered":"2024-03-26T08:41:28.350141+0000","last_clean":"2024-03-26T08:41:28.350141+0000","last_became_active":"2024-03-26T08:40:09.934312+0000","last_became_peered":"2024-03-26T08:40:09.934312+0000","last_unstale":"2024-03-26T08:41:28.350141+0000","last_undegraded":"2024-03-26T08:41:28.350141+0000","last_fullsized":"2024-03-26T08:41:28.350141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:19:27.737506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420385+0000","last_change":"2024-03-26T08:40:09.230014+0000","last_active":"2024-03-26T08:40:59.420385+0000","last_peered":"2024-03-26T08:40:59.420385+0000","last_clean":"2024-03-26T08:40:59.420385+0000","last_became_active":"2024-03-26T08:40:09.229682+0000","last_became_peered":"2024-03-26T08:40:09.229682+0000","last_unstale":"2024-03-26T08:40:59.420385+0000","last_undegraded":"2024-03-26T08:40:59.420385+0000","last_fullsized":"2024-03-26T08:40:59.420385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:52.725810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.337290+0000","last_change":"2024-03-26T08:40:59.464419+0000","last_active":"2024-03-26T08:41:28.337290+0000","last_peered":"2024-03-26T08:41:28.337290+0000","last_clean":"2024-03-26T08:41:28.337290+0000","last_became_active":"2024-03-26T08:40:59.464051+0000","last_became_peered":"2024-03-26T08:40:59.464051+0000","last_unstale":"2024-03-26T08:41:28.337290+0000","last_undegraded":"2024-03-26T08:41:28.337290+0000","last_fullsized":"2024-03-26T08:41:28.337290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:01:47.975380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.501822+0000","last_change":"2024-03-26T08:40:09.226634+0000","last_active":"2024-03-26T08:40:59.501822+0000","last_peered":"2024-03-26T08:40:59.501822+0000","last_clean":"2024-03-26T08:40:59.501822+0000","last_became_active":"2024-03-26T08:40:09.226248+0000","last_became_peered":"2024-03-26T08:40:09.226248+0000","last_unstale":"2024-03-26T08:40:59.501822+0000","last_undegraded":"2024-03-26T08:40:59.501822+0000","last_fullsized":"2024-03-26T08:40:59.501822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:24:00.390573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.303390+0000","last_change":"2024-03-26T08:40:09.934848+0000","last_active":"2024-03-26T08:41:28.303390+0000","last_peered":"2024-03-26T08:41:28.303390+0000","last_clean":"2024-03-26T08:41:28.303390+0000","last_became_active":"2024-03-26T08:40:09.934521+0000","last_became_peered":"2024-03-26T08:40:09.934521+0000","last_unstale":"2024-03-26T08:41:28.303390+0000","last_undegraded":"2024-03-26T08:41:28.303390+0000","last_fullsized":"2024-03-26T08:41:28.303390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:24:25.994961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419371+0000","last_change":"2024-03-26T08:40:09.222945+0000","last_active":"2024-03-26T08:40:59.419371+0000","last_peered":"2024-03-26T08:40:59.419371+0000","last_clean":"2024-03-26T08:40:59.419371+0000","last_became_active":"2024-03-26T08:40:09.222564+0000","last_became_peered":"2024-03-26T08:40:09.222564+0000","last_unstale":"2024-03-26T08:40:59.419371+0000","last_undegraded":"2024-03-26T08:40:59.419371+0000","last_fullsized":"2024-03-26T08:40:59.419371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:35:11.181812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.347366+0000","last_change":"2024-03-26T08:40:09.932674+0000","last_active":"2024-03-26T08:41:28.347366+0000","last_peered":"2024-03-26T08:41:28.347366+0000","last_clean":"2024-03-26T08:41:28.347366+0000","last_became_active":"2024-03-26T08:40:09.932346+0000","last_became_peered":"2024-03-26T08:40:09.932346+0000","last_unstale":"2024-03-26T08:41:28.347366+0000","last_undegraded":"2024-03-26T08:41:28.347366+0000","last_fullsized":"2024-03-26T08:41:28.347366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:12:45.871467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.431219+0000","last_change":"2024-03-26T08:40:59.431219+0000","last_active":"2024-03-26T08:40:59.431219+0000","last_peered":"2024-03-26T08:40:59.431219+0000","last_clean":"2024-03-26T08:40:59.431219+0000","last_became_active":"2024-03-26T08:40:59.424868+0000","last_became_peered":"2024-03-26T08:40:59.424868+0000","last_unstale":"2024-03-26T08:40:59.431219+0000","last_undegraded":"2024-03-26T08:40:59.431219+0000","last_fullsized":"2024-03-26T08:40:59.431219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:02.837090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350683+0000","last_change":"2024-03-26T08:40:09.922766+0000","last_active":"2024-03-26T08:41:28.350683+0000","last_peered":"2024-03-26T08:41:28.350683+0000","last_clean":"2024-03-26T08:41:28.350683+0000","last_became_active":"2024-03-26T08:40:09.922453+0000","last_became_peered":"2024-03-26T08:40:09.922453+0000","last_unstale":"2024-03-26T08:41:28.350683+0000","last_undegraded":"2024-03-26T08:41:28.350683+0000","last_fullsized":"2024-03-26T08:41:28.350683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:13:28.740291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416510+0000","last_change":"2024-03-26T08:40:09.228114+0000","last_active":"2024-03-26T08:40:59.416510+0000","last_peered":"2024-03-26T08:40:59.416510+0000","last_clean":"2024-03-26T08:40:59.416510+0000","last_became_active":"2024-03-26T08:40:09.227803+0000","last_became_peered":"2024-03-26T08:40:09.227803+0000","last_unstale":"2024-03-26T08:40:59.416510+0000","last_undegraded":"2024-03-26T08:40:59.416510+0000","last_fullsized":"2024-03-26T08:40:59.416510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:31.717981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350700+0000","last_change":"2024-03-26T08:40:09.929671+0000","last_active":"2024-03-26T08:41:28.350700+0000","last_peered":"2024-03-26T08:41:28.350700+0000","last_clean":"2024-03-26T08:41:28.350700+0000","last_became_active":"2024-03-26T08:40:09.929222+0000","last_became_peered":"2024-03-26T08:40:09.929222+0000","last_unstale":"2024-03-26T08:41:28.350700+0000","last_undegraded":"2024-03-26T08:41:28.350700+0000","last_fullsized":"2024-03-26T08:41:28.350700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:00:50.579569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422136+0000","last_change":"2024-03-26T08:40:09.228994+0000","last_active":"2024-03-26T08:40:59.422136+0000","last_peered":"2024-03-26T08:40:59.422136+0000","last_clean":"2024-03-26T08:40:59.422136+0000","last_became_active":"2024-03-26T08:40:09.227138+0000","last_became_peered":"2024-03-26T08:40:09.227138+0000","last_unstale":"2024-03-26T08:40:59.422136+0000","last_undegraded":"2024-03-26T08:40:59.422136+0000","last_fullsized":"2024-03-26T08:40:59.422136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:06.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352041+0000","last_change":"2024-03-26T08:40:09.931752+0000","last_active":"2024-03-26T08:41:28.352041+0000","last_peered":"2024-03-26T08:41:28.352041+0000","last_clean":"2024-03-26T08:41:28.352041+0000","last_became_active":"2024-03-26T08:40:09.931436+0000","last_became_peered":"2024-03-26T08:40:09.931436+0000","last_unstale":"2024-03-26T08:41:28.352041+0000","last_undegraded":"2024-03-26T08:41:28.352041+0000","last_fullsized":"2024-03-26T08:41:28.352041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:40:54.864614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.418961+0000","last_change":"2024-03-26T08:40:09.238356+0000","last_active":"2024-03-26T08:40:59.418961+0000","last_peered":"2024-03-26T08:40:59.418961+0000","last_clean":"2024-03-26T08:40:59.418961+0000","last_became_active":"2024-03-26T08:40:09.238066+0000","last_became_peered":"2024-03-26T08:40:09.238066+0000","last_unstale":"2024-03-26T08:40:59.418961+0000","last_undegraded":"2024-03-26T08:40:59.418961+0000","last_fullsized":"2024-03-26T08:40:59.418961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:13.851490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349384+0000","last_change":"2024-03-26T08:40:59.427153+0000","last_active":"2024-03-26T08:41:28.349384+0000","last_peered":"2024-03-26T08:41:28.349384+0000","last_clean":"2024-03-26T08:41:28.349384+0000","last_became_active":"2024-03-26T08:40:59.423044+0000","last_became_peered":"2024-03-26T08:40:59.423044+0000","last_unstale":"2024-03-26T08:41:28.349384+0000","last_undegraded":"2024-03-26T08:41:28.349384+0000","last_fullsized":"2024-03-26T08:41:28.349384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:20.181241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463434+0000","last_change":"2024-03-26T08:40:09.238009+0000","last_active":"2024-03-26T08:40:59.463434+0000","last_peered":"2024-03-26T08:40:59.463434+0000","last_clean":"2024-03-26T08:40:59.463434+0000","last_became_active":"2024-03-26T08:40:09.237697+0000","last_became_peered":"2024-03-26T08:40:09.237697+0000","last_unstale":"2024-03-26T08:40:59.463434+0000","last_undegraded":"2024-03-26T08:40:59.463434+0000","last_fullsized":"2024-03-26T08:40:59.463434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:17:33.179285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.340759+0000","last_change":"2024-03-26T08:40:09.934672+0000","last_active":"2024-03-26T08:41:28.340759+0000","last_peered":"2024-03-26T08:41:28.340759+0000","last_clean":"2024-03-26T08:41:28.340759+0000","last_became_active":"2024-03-26T08:40:09.934365+0000","last_became_peered":"2024-03-26T08:40:09.934365+0000","last_unstale":"2024-03-26T08:41:28.340759+0000","last_undegraded":"2024-03-26T08:41:28.340759+0000","last_fullsized":"2024-03-26T08:41:28.340759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:41:22.318481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.346142+0000","last_change":"2024-03-26T08:40:09.933910+0000","last_active":"2024-03-26T08:41:28.346142+0000","last_peered":"2024-03-26T08:41:28.346142+0000","last_clean":"2024-03-26T08:41:28.346142+0000","last_became_active":"2024-03-26T08:40:09.933598+0000","last_became_peered":"2024-03-26T08:40:09.933598+0000","last_unstale":"2024-03-26T08:41:28.346142+0000","last_undegraded":"2024-03-26T08:41:28.346142+0000","last_fullsized":"2024-03-26T08:41:28.346142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:19.877984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338094+0000","last_change":"2024-03-26T08:40:09.935809+0000","last_active":"2024-03-26T08:41:28.338094+0000","last_peered":"2024-03-26T08:41:28.338094+0000","last_clean":"2024-03-26T08:41:28.338094+0000","last_became_active":"2024-03-26T08:40:09.935500+0000","last_became_peered":"2024-03-26T08:40:09.935500+0000","last_unstale":"2024-03-26T08:41:28.338094+0000","last_undegraded":"2024-03-26T08:41:28.338094+0000","last_fullsized":"2024-03-26T08:41:28.338094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:23.717845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345125+0000","last_change":"2024-03-26T08:40:09.928093+0000","last_active":"2024-03-26T08:41:28.345125+0000","last_peered":"2024-03-26T08:41:28.345125+0000","last_clean":"2024-03-26T08:41:28.345125+0000","last_became_active":"2024-03-26T08:40:09.927732+0000","last_became_peered":"2024-03-26T08:40:09.927732+0000","last_unstale":"2024-03-26T08:41:28.345125+0000","last_undegraded":"2024-03-26T08:41:28.345125+0000","last_fullsized":"2024-03-26T08:41:28.345125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:35:34.972435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338020+0000","last_change":"2024-03-26T08:40:59.463368+0000","last_active":"2024-03-26T08:41:28.338020+0000","last_peered":"2024-03-26T08:41:28.338020+0000","last_clean":"2024-03-26T08:41:28.338020+0000","last_became_active":"2024-03-26T08:40:59.462979+0000","last_became_peered":"2024-03-26T08:40:59.462979+0000","last_unstale":"2024-03-26T08:41:28.338020+0000","last_undegraded":"2024-03-26T08:41:28.338020+0000","last_fullsized":"2024-03-26T08:41:28.338020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:13:01.413240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.342567+0000","last_change":"2024-03-26T08:40:09.919466+0000","last_active":"2024-03-26T08:41:28.342567+0000","last_peered":"2024-03-26T08:41:28.342567+0000","last_clean":"2024-03-26T08:41:28.342567+0000","last_became_active":"2024-03-26T08:40:09.919152+0000","last_became_peered":"2024-03-26T08:40:09.919152+0000","last_unstale":"2024-03-26T08:41:28.342567+0000","last_undegraded":"2024-03-26T08:41:28.342567+0000","last_fullsized":"2024-03-26T08:41:28.342567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:37.928485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345579+0000","last_change":"2024-03-26T08:40:09.927140+0000","last_active":"2024-03-26T08:41:28.345579+0000","last_peered":"2024-03-26T08:41:28.345579+0000","last_clean":"2024-03-26T08:41:28.345579+0000","last_became_active":"2024-03-26T08:40:09.926823+0000","last_became_peered":"2024-03-26T08:40:09.926823+0000","last_unstale":"2024-03-26T08:41:28.345579+0000","last_undegraded":"2024-03-26T08:41:28.345579+0000","last_fullsized":"2024-03-26T08:41:28.345579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:29:38.477123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353887+0000","last_change":"2024-03-26T08:40:09.934202+0000","last_active":"2024-03-26T08:41:28.353887+0000","last_peered":"2024-03-26T08:41:28.353887+0000","last_clean":"2024-03-26T08:41:28.353887+0000","last_became_active":"2024-03-26T08:40:09.933878+0000","last_became_peered":"2024-03-26T08:40:09.933878+0000","last_unstale":"2024-03-26T08:41:28.353887+0000","last_undegraded":"2024-03-26T08:41:28.353887+0000","last_fullsized":"2024-03-26T08:41:28.353887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:44:26.576140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345991+0000","last_change":"2024-03-26T08:40:09.929019+0000","last_active":"2024-03-26T08:41:28.345991+0000","last_peered":"2024-03-26T08:41:28.345991+0000","last_clean":"2024-03-26T08:41:28.345991+0000","last_became_active":"2024-03-26T08:40:09.928711+0000","last_became_peered":"2024-03-26T08:40:09.928711+0000","last_unstale":"2024-03-26T08:41:28.345991+0000","last_undegraded":"2024-03-26T08:41:28.345991+0000","last_fullsized":"2024-03-26T08:41:28.345991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:47.120826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.347133+0000","last_change":"2024-03-26T08:40:09.927606+0000","last_active":"2024-03-26T08:41:28.347133+0000","last_peered":"2024-03-26T08:41:28.347133+0000","last_clean":"2024-03-26T08:41:28.347133+0000","last_became_active":"2024-03-26T08:40:09.926717+0000","last_became_peered":"2024-03-26T08:40:09.926717+0000","last_unstale":"2024-03-26T08:41:28.347133+0000","last_undegraded":"2024-03-26T08:41:28.347133+0000","last_fullsized":"2024-03-26T08:41:28.347133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:51:52.567252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350770+0000","last_change":"2024-03-26T08:40:09.934659+0000","last_active":"2024-03-26T08:41:28.350770+0000","last_peered":"2024-03-26T08:41:28.350770+0000","last_clean":"2024-03-26T08:41:28.350770+0000","last_became_active":"2024-03-26T08:40:09.934346+0000","last_became_peered":"2024-03-26T08:40:09.934346+0000","last_unstale":"2024-03-26T08:41:28.350770+0000","last_undegraded":"2024-03-26T08:41:28.350770+0000","last_fullsized":"2024-03-26T08:41:28.350770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:35.009264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.348295+0000","last_change":"2024-03-26T08:40:09.925007+0000","last_active":"2024-03-26T08:41:28.348295+0000","last_peered":"2024-03-26T08:41:28.348295+0000","last_clean":"2024-03-26T08:41:28.348295+0000","last_became_active":"2024-03-26T08:40:09.923439+0000","last_became_peered":"2024-03-26T08:40:09.923439+0000","last_unstale":"2024-03-26T08:41:28.348295+0000","last_undegraded":"2024-03-26T08:41:28.348295+0000","last_fullsized":"2024-03-26T08:41:28.348295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:18.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.340590+0000","last_change":"2024-03-26T08:40:09.920795+0000","last_active":"2024-03-26T08:41:28.340590+0000","last_peered":"2024-03-26T08:41:28.340590+0000","last_clean":"2024-03-26T08:41:28.340590+0000","last_became_active":"2024-03-26T08:40:09.920378+0000","last_became_peered":"2024-03-26T08:40:09.920378+0000","last_unstale":"2024-03-26T08:41:28.340590+0000","last_undegraded":"2024-03-26T08:41:28.340590+0000","last_fullsized":"2024-03-26T08:41:28.340590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:19:04.027727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344142+0000","last_change":"2024-03-26T08:40:59.425704+0000","last_active":"2024-03-26T08:41:28.344142+0000","last_peered":"2024-03-26T08:41:28.344142+0000","last_clean":"2024-03-26T08:41:28.344142+0000","last_became_active":"2024-03-26T08:40:59.425014+0000","last_became_peered":"2024-03-26T08:40:59.425014+0000","last_unstale":"2024-03-26T08:41:28.344142+0000","last_undegraded":"2024-03-26T08:41:28.344142+0000","last_fullsized":"2024-03-26T08:41:28.344142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:08:14.847536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282339,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75564,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":729,"num_read_kb":444,"num_write":171,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":231033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":456298,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":1118480,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":456512,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456527,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1048503,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":455901,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":528937,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":457172,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":531518,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456212,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":456338,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1048769,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":73243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":75824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:42:12.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T08:42:12.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 -- 172.21.15.7:0/2372117798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 msgr2=0x7f3974101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 --2- 172.21.15.7:0/2372117798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 0x7f3974101c40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f395c009900 tx=0x7f395c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:12.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 -- 172.21.15.7:0/2372117798 shutdown_connections 2024-03-26T08:42:12.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 --2- 172.21.15.7:0/2372117798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3974102180 0x7f3974108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 --2- 172.21.15.7:0/2372117798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 0x7f3974101c40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 --2- 172.21.15.7:0/2372117798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39741005d0 0x7f39741009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 -- 172.21.15.7:0/2372117798 >> 172.21.15.7:0/2372117798 conn(0x7f39740fbb60 msgr2=0x7f39740fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:12.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 -- 172.21.15.7:0/2372117798 shutdown_connections 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 -- 172.21.15.7:0/2372117798 wait complete. 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 Processor -- start 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.998+0000 7f39789c4640 1 -- start start 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f39789c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39741005d0 0x7f39741968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f39789c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 0x7f3974196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f39789c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3974102180 0x7f397419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f39789c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397410c230 con 0x7f39741017c0 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f39789c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397410c0b0 con 0x7f3974102180 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f39789c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397410c3b0 con 0x7f39741005d0 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 0x7f3974196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 0x7f3974196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53408/0 (socket says 172.21.15.7:53408) 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396bfff640 1 -- 172.21.15.7:0/3812957676 learned_addr learned my addr 172.21.15.7:0/3812957676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f397259c640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39741005d0 0x7f39741968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f3972d9d640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3974102180 0x7f397419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396bfff640 1 -- 172.21.15.7:0/3812957676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39741005d0 msgr2=0x7f39741968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396bfff640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39741005d0 0x7f39741968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396bfff640 1 -- 172.21.15.7:0/3812957676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3974102180 msgr2=0x7f397419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396bfff640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3974102180 0x7f397419de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396bfff640 1 -- 172.21.15.7:0/3812957676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397419e4f0 con 0x7f39741017c0 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396bfff640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 0x7f3974196e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f395c009230 tx=0x7f395c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396b7fe640 1 -- 172.21.15.7:0/3812957676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c041070 con 0x7f39741017c0 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397419e780 con 0x7f39741017c0 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396b7fe640 1 -- 172.21.15.7:0/3812957676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f395c004450 con 0x7f39741017c0 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f396b7fe640 1 -- 172.21.15.7:0/3812957676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c003db0 con 0x7f39741017c0 2024-03-26T08:42:12.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:12.999+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397419ec70 con 0x7f39741017c0 2024-03-26T08:42:12.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.000+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3974107970 con 0x7f39741017c0 2024-03-26T08:42:12.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.001+0000 7f396b7fe640 1 -- 172.21.15.7:0/3812957676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f395c044070 con 0x7f39741017c0 2024-03-26T08:42:12.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.002+0000 7f396b7fe640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f39600a2890 0x7f39600a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:12.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.002+0000 7f397259c640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f39600a2890 0x7f39600a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:12.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.002+0000 7f396b7fe640 1 -- 172.21.15.7:0/3812957676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f395c0f0690 con 0x7f39741017c0 2024-03-26T08:42:12.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.003+0000 7f397259c640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f39600a2890 0x7f39600a4d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39640095f0 tx=0x7f3964005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:12.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.006+0000 7f396b7fe640 1 -- 172.21.15.7:0/3812957676 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f395c0bd6e0 con 0x7f39741017c0 2024-03-26T08:42:13.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.186+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3974061ae0 con 0x7f39600a2890 2024-03-26T08:42:13.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.193+0000 7f396b7fe640 1 -- 172.21.15.7:0/3812957676 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259090 (secure 0 0 0) 0x7f3974061ae0 con 0x7f39600a2890 2024-03-26T08:42:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:13.190 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f39600a2890 msgr2=0x7f39600a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f39600a2890 0x7f39600a4d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39640095f0 tx=0x7f3964005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 msgr2=0x7f3974196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 0x7f3974196e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f395c009230 tx=0x7f395c02c650 comp rx=0 tx=0).stop 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 shutdown_connections 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f39600a2890 0x7f39600a4d50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3974102180 0x7f397419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39741017c0 0x7f3974196e10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 --2- 172.21.15.7:0/3812957676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f39741005d0 0x7f39741968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 >> 172.21.15.7:0/3812957676 conn(0x7f39740fbb60 msgr2=0x7f3974104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 shutdown_connections 2024-03-26T08:42:13.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.196+0000 7f39789c4640 1 -- 172.21.15.7:0/3812957676 wait complete. 2024-03-26T08:42:13.200 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-03-26T08:42:12.329724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666166,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":855,"num_read_kb":553,"num_write":354,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373236,"kb_used_data":11604,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568588,"statfs":{"total":1151940427776,"available":1151558234112,"internally_reserved":0,"allocated":11882496,"data_stored":7471167,"data_compressed":177645,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6771,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":54,"num_write":8,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002952"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353784+0000","last_change":"2024-03-26T08:40:09.939078+0000","last_active":"2024-03-26T08:41:28.353784+0000","last_peered":"2024-03-26T08:41:28.353784+0000","last_clean":"2024-03-26T08:41:28.353784+0000","last_became_active":"2024-03-26T08:40:09.937626+0000","last_became_peered":"2024-03-26T08:40:09.937626+0000","last_unstale":"2024-03-26T08:41:28.353784+0000","last_undegraded":"2024-03-26T08:41:28.353784+0000","last_fullsized":"2024-03-26T08:41:28.353784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:52:59.254181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345392+0000","last_change":"2024-03-26T08:40:09.924207+0000","last_active":"2024-03-26T08:41:28.345392+0000","last_peered":"2024-03-26T08:41:28.345392+0000","last_clean":"2024-03-26T08:41:28.345392+0000","last_became_active":"2024-03-26T08:40:09.923881+0000","last_became_peered":"2024-03-26T08:40:09.923881+0000","last_unstale":"2024-03-26T08:41:28.345392+0000","last_undegraded":"2024-03-26T08:41:28.345392+0000","last_fullsized":"2024-03-26T08:41:28.345392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:16:45.265521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349461+0000","last_change":"2024-03-26T08:40:09.929154+0000","last_active":"2024-03-26T08:41:28.349461+0000","last_peered":"2024-03-26T08:41:28.349461+0000","last_clean":"2024-03-26T08:41:28.349461+0000","last_became_active":"2024-03-26T08:40:09.928833+0000","last_became_peered":"2024-03-26T08:40:09.928833+0000","last_unstale":"2024-03-26T08:41:28.349461+0000","last_undegraded":"2024-03-26T08:41:28.349461+0000","last_fullsized":"2024-03-26T08:41:28.349461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:22.972210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.341510+0000","last_change":"2024-03-26T08:40:09.930713+0000","last_active":"2024-03-26T08:41:28.341510+0000","last_peered":"2024-03-26T08:41:28.341510+0000","last_clean":"2024-03-26T08:41:28.341510+0000","last_became_active":"2024-03-26T08:40:09.930420+0000","last_became_peered":"2024-03-26T08:40:09.930420+0000","last_unstale":"2024-03-26T08:41:28.341510+0000","last_undegraded":"2024-03-26T08:41:28.341510+0000","last_fullsized":"2024-03-26T08:41:28.341510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:17:03.015703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.336520+0000","last_change":"2024-03-26T08:40:09.923935+0000","last_active":"2024-03-26T08:41:28.336520+0000","last_peered":"2024-03-26T08:41:28.336520+0000","last_clean":"2024-03-26T08:41:28.336520+0000","last_became_active":"2024-03-26T08:40:09.923587+0000","last_became_peered":"2024-03-26T08:40:09.923587+0000","last_unstale":"2024-03-26T08:41:28.336520+0000","last_undegraded":"2024-03-26T08:41:28.336520+0000","last_fullsized":"2024-03-26T08:41:28.336520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:19:38.831448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344419+0000","last_change":"2024-03-26T08:40:09.922361+0000","last_active":"2024-03-26T08:41:28.344419+0000","last_peered":"2024-03-26T08:41:28.344419+0000","last_clean":"2024-03-26T08:41:28.344419+0000","last_became_active":"2024-03-26T08:40:09.921643+0000","last_became_peered":"2024-03-26T08:40:09.921643+0000","last_unstale":"2024-03-26T08:41:28.344419+0000","last_undegraded":"2024-03-26T08:41:28.344419+0000","last_fullsized":"2024-03-26T08:41:28.344419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:37:23.048778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.341431+0000","last_change":"2024-03-26T08:40:09.932428+0000","last_active":"2024-03-26T08:41:28.341431+0000","last_peered":"2024-03-26T08:41:28.341431+0000","last_clean":"2024-03-26T08:41:28.341431+0000","last_became_active":"2024-03-26T08:40:09.932120+0000","last_became_peered":"2024-03-26T08:40:09.932120+0000","last_unstale":"2024-03-26T08:41:28.341431+0000","last_undegraded":"2024-03-26T08:41:28.341431+0000","last_fullsized":"2024-03-26T08:41:28.341431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:24.362568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351911+0000","last_change":"2024-03-26T08:40:09.928251+0000","last_active":"2024-03-26T08:41:28.351911+0000","last_peered":"2024-03-26T08:41:28.351911+0000","last_clean":"2024-03-26T08:41:28.351911+0000","last_became_active":"2024-03-26T08:40:09.927961+0000","last_became_peered":"2024-03-26T08:40:09.927961+0000","last_unstale":"2024-03-26T08:41:28.351911+0000","last_undegraded":"2024-03-26T08:41:28.351911+0000","last_fullsized":"2024-03-26T08:41:28.351911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:25:09.348033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351308+0000","last_change":"2024-03-26T08:40:09.930888+0000","last_active":"2024-03-26T08:41:28.351308+0000","last_peered":"2024-03-26T08:41:28.351308+0000","last_clean":"2024-03-26T08:41:28.351308+0000","last_became_active":"2024-03-26T08:40:09.930513+0000","last_became_peered":"2024-03-26T08:40:09.930513+0000","last_unstale":"2024-03-26T08:41:28.351308+0000","last_undegraded":"2024-03-26T08:41:28.351308+0000","last_fullsized":"2024-03-26T08:41:28.351308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:21.023569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345352+0000","last_change":"2024-03-26T08:40:09.935534+0000","last_active":"2024-03-26T08:41:28.345352+0000","last_peered":"2024-03-26T08:41:28.345352+0000","last_clean":"2024-03-26T08:41:28.345352+0000","last_became_active":"2024-03-26T08:40:09.934959+0000","last_became_peered":"2024-03-26T08:40:09.934959+0000","last_unstale":"2024-03-26T08:41:28.345352+0000","last_undegraded":"2024-03-26T08:41:28.345352+0000","last_fullsized":"2024-03-26T08:41:28.345352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:18:59.311180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351875+0000","last_change":"2024-03-26T08:40:09.930486+0000","last_active":"2024-03-26T08:41:28.351875+0000","last_peered":"2024-03-26T08:41:28.351875+0000","last_clean":"2024-03-26T08:41:28.351875+0000","last_became_active":"2024-03-26T08:40:09.930184+0000","last_became_peered":"2024-03-26T08:40:09.930184+0000","last_unstale":"2024-03-26T08:41:28.351875+0000","last_undegraded":"2024-03-26T08:41:28.351875+0000","last_fullsized":"2024-03-26T08:41:28.351875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:07.246878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345052+0000","last_change":"2024-03-26T08:40:09.924305+0000","last_active":"2024-03-26T08:41:28.345052+0000","last_peered":"2024-03-26T08:41:28.345052+0000","last_clean":"2024-03-26T08:41:28.345052+0000","last_became_active":"2024-03-26T08:40:09.923985+0000","last_became_peered":"2024-03-26T08:40:09.923985+0000","last_unstale":"2024-03-26T08:41:28.345052+0000","last_undegraded":"2024-03-26T08:41:28.345052+0000","last_fullsized":"2024-03-26T08:41:28.345052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:42:39.894245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350807+0000","last_change":"2024-03-26T08:40:09.916637+0000","last_active":"2024-03-26T08:41:28.350807+0000","last_peered":"2024-03-26T08:41:28.350807+0000","last_clean":"2024-03-26T08:41:28.350807+0000","last_became_active":"2024-03-26T08:40:09.916314+0000","last_became_peered":"2024-03-26T08:40:09.916314+0000","last_unstale":"2024-03-26T08:41:28.350807+0000","last_undegraded":"2024-03-26T08:41:28.350807+0000","last_fullsized":"2024-03-26T08:41:28.350807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:45:10.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345246+0000","last_change":"2024-03-26T08:40:09.923753+0000","last_active":"2024-03-26T08:41:28.345246+0000","last_peered":"2024-03-26T08:41:28.345246+0000","last_clean":"2024-03-26T08:41:28.345246+0000","last_became_active":"2024-03-26T08:40:09.923427+0000","last_became_peered":"2024-03-26T08:40:09.923427+0000","last_unstale":"2024-03-26T08:41:28.345246+0000","last_undegraded":"2024-03-26T08:41:28.345246+0000","last_fullsized":"2024-03-26T08:41:28.345246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:13:16.711556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338474+0000","last_change":"2024-03-26T08:40:09.931879+0000","last_active":"2024-03-26T08:41:28.338474+0000","last_peered":"2024-03-26T08:41:28.338474+0000","last_clean":"2024-03-26T08:41:28.338474+0000","last_became_active":"2024-03-26T08:40:09.931589+0000","last_became_peered":"2024-03-26T08:40:09.931589+0000","last_unstale":"2024-03-26T08:41:28.338474+0000","last_undegraded":"2024-03-26T08:41:28.338474+0000","last_fullsized":"2024-03-26T08:41:28.338474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:22:01.038745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353522+0000","last_change":"2024-03-26T08:40:09.930500+0000","last_active":"2024-03-26T08:41:28.353522+0000","last_peered":"2024-03-26T08:41:28.353522+0000","last_clean":"2024-03-26T08:41:28.353522+0000","last_became_active":"2024-03-26T08:40:09.930101+0000","last_became_peered":"2024-03-26T08:40:09.930101+0000","last_unstale":"2024-03-26T08:41:28.353522+0000","last_undegraded":"2024-03-26T08:41:28.353522+0000","last_fullsized":"2024-03-26T08:41:28.353522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:54.229004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.354043+0000","last_change":"2024-03-26T08:40:09.939177+0000","last_active":"2024-03-26T08:41:28.354043+0000","last_peered":"2024-03-26T08:41:28.354043+0000","last_clean":"2024-03-26T08:41:28.354043+0000","last_became_active":"2024-03-26T08:40:09.937727+0000","last_became_peered":"2024-03-26T08:40:09.937727+0000","last_unstale":"2024-03-26T08:41:28.354043+0000","last_undegraded":"2024-03-26T08:41:28.354043+0000","last_fullsized":"2024-03-26T08:41:28.354043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:40:44.040015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350313+0000","last_change":"2024-03-26T08:40:09.928429+0000","last_active":"2024-03-26T08:41:28.350313+0000","last_peered":"2024-03-26T08:41:28.350313+0000","last_clean":"2024-03-26T08:41:28.350313+0000","last_became_active":"2024-03-26T08:40:09.928025+0000","last_became_peered":"2024-03-26T08:40:09.928025+0000","last_unstale":"2024-03-26T08:41:28.350313+0000","last_undegraded":"2024-03-26T08:41:28.350313+0000","last_fullsized":"2024-03-26T08:41:28.350313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:52:55.450291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345617+0000","last_change":"2024-03-26T08:40:09.930856+0000","last_active":"2024-03-26T08:41:28.345617+0000","last_peered":"2024-03-26T08:41:28.345617+0000","last_clean":"2024-03-26T08:41:28.345617+0000","last_became_active":"2024-03-26T08:40:09.930552+0000","last_became_peered":"2024-03-26T08:40:09.930552+0000","last_unstale":"2024-03-26T08:41:28.345617+0000","last_undegraded":"2024-03-26T08:41:28.345617+0000","last_fullsized":"2024-03-26T08:41:28.345617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:52.851287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421132+0000","last_change":"2024-03-26T08:40:09.230609+0000","last_active":"2024-03-26T08:40:59.421132+0000","last_peered":"2024-03-26T08:40:59.421132+0000","last_clean":"2024-03-26T08:40:59.421132+0000","last_became_active":"2024-03-26T08:40:09.230284+0000","last_became_peered":"2024-03-26T08:40:09.230284+0000","last_unstale":"2024-03-26T08:40:59.421132+0000","last_undegraded":"2024-03-26T08:40:59.421132+0000","last_fullsized":"2024-03-26T08:40:59.421132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:06.681725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.337071+0000","last_change":"2024-03-26T08:40:59.425449+0000","last_active":"2024-03-26T08:41:28.337071+0000","last_peered":"2024-03-26T08:41:28.337071+0000","last_clean":"2024-03-26T08:41:28.337071+0000","last_became_active":"2024-03-26T08:40:59.425164+0000","last_became_peered":"2024-03-26T08:40:59.425164+0000","last_unstale":"2024-03-26T08:41:28.337071+0000","last_undegraded":"2024-03-26T08:41:28.337071+0000","last_fullsized":"2024-03-26T08:41:28.337071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:09.846252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419667+0000","last_change":"2024-03-26T08:40:09.235451+0000","last_active":"2024-03-26T08:40:59.419667+0000","last_peered":"2024-03-26T08:40:59.419667+0000","last_clean":"2024-03-26T08:40:59.419667+0000","last_became_active":"2024-03-26T08:40:09.235154+0000","last_became_peered":"2024-03-26T08:40:09.235154+0000","last_unstale":"2024-03-26T08:40:59.419667+0000","last_undegraded":"2024-03-26T08:40:59.419667+0000","last_fullsized":"2024-03-26T08:40:59.419667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:37:38.418888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349428+0000","last_change":"2024-03-26T08:40:09.931728+0000","last_active":"2024-03-26T08:41:28.349428+0000","last_peered":"2024-03-26T08:41:28.349428+0000","last_clean":"2024-03-26T08:41:28.349428+0000","last_became_active":"2024-03-26T08:40:09.931401+0000","last_became_peered":"2024-03-26T08:40:09.931401+0000","last_unstale":"2024-03-26T08:41:28.349428+0000","last_undegraded":"2024-03-26T08:41:28.349428+0000","last_fullsized":"2024-03-26T08:41:28.349428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:02.229763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417066+0000","last_change":"2024-03-26T08:40:09.231247+0000","last_active":"2024-03-26T08:40:59.417066+0000","last_peered":"2024-03-26T08:40:59.417066+0000","last_clean":"2024-03-26T08:40:59.417066+0000","last_became_active":"2024-03-26T08:40:09.230914+0000","last_became_peered":"2024-03-26T08:40:09.230914+0000","last_unstale":"2024-03-26T08:40:59.417066+0000","last_undegraded":"2024-03-26T08:40:59.417066+0000","last_fullsized":"2024-03-26T08:40:59.417066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:03.956444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353346+0000","last_change":"2024-03-26T08:40:09.925174+0000","last_active":"2024-03-26T08:41:28.353346+0000","last_peered":"2024-03-26T08:41:28.353346+0000","last_clean":"2024-03-26T08:41:28.353346+0000","last_became_active":"2024-03-26T08:40:09.924840+0000","last_became_peered":"2024-03-26T08:40:09.924840+0000","last_unstale":"2024-03-26T08:41:28.353346+0000","last_undegraded":"2024-03-26T08:41:28.353346+0000","last_fullsized":"2024-03-26T08:41:28.353346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:57:19.041245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416818+0000","last_change":"2024-03-26T08:40:09.229504+0000","last_active":"2024-03-26T08:40:59.416818+0000","last_peered":"2024-03-26T08:40:59.416818+0000","last_clean":"2024-03-26T08:40:59.416818+0000","last_became_active":"2024-03-26T08:40:09.226449+0000","last_became_peered":"2024-03-26T08:40:09.226449+0000","last_unstale":"2024-03-26T08:40:59.416818+0000","last_undegraded":"2024-03-26T08:40:59.416818+0000","last_fullsized":"2024-03-26T08:40:59.416818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:38.047424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351371+0000","last_change":"2024-03-26T08:40:09.930184+0000","last_active":"2024-03-26T08:41:28.351371+0000","last_peered":"2024-03-26T08:41:28.351371+0000","last_clean":"2024-03-26T08:41:28.351371+0000","last_became_active":"2024-03-26T08:40:09.927679+0000","last_became_peered":"2024-03-26T08:40:09.927679+0000","last_unstale":"2024-03-26T08:41:28.351371+0000","last_undegraded":"2024-03-26T08:41:28.351371+0000","last_fullsized":"2024-03-26T08:41:28.351371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:05:52.949814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'9","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.404325+0000","last_change":"2024-03-26T08:40:09.233025+0000","last_active":"2024-03-26T08:42:10.404325+0000","last_peered":"2024-03-26T08:42:10.404325+0000","last_clean":"2024-03-26T08:42:10.404325+0000","last_became_active":"2024-03-26T08:40:09.232654+0000","last_became_peered":"2024-03-26T08:40:09.232654+0000","last_unstale":"2024-03-26T08:42:10.404325+0000","last_undegraded":"2024-03-26T08:42:10.404325+0000","last_fullsized":"2024-03-26T08:42:10.404325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:01.745583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.333641+0000","last_change":"2024-03-26T08:40:09.924187+0000","last_active":"2024-03-26T08:41:28.333641+0000","last_peered":"2024-03-26T08:41:28.333641+0000","last_clean":"2024-03-26T08:41:28.333641+0000","last_became_active":"2024-03-26T08:40:09.923861+0000","last_became_peered":"2024-03-26T08:40:09.923861+0000","last_unstale":"2024-03-26T08:41:28.333641+0000","last_undegraded":"2024-03-26T08:41:28.333641+0000","last_fullsized":"2024-03-26T08:41:28.333641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:12:17.284186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502123+0000","last_change":"2024-03-26T08:40:09.231368+0000","last_active":"2024-03-26T08:40:59.502123+0000","last_peered":"2024-03-26T08:40:59.502123+0000","last_clean":"2024-03-26T08:40:59.502123+0000","last_became_active":"2024-03-26T08:40:09.231040+0000","last_became_peered":"2024-03-26T08:40:09.231040+0000","last_unstale":"2024-03-26T08:40:59.502123+0000","last_undegraded":"2024-03-26T08:40:59.502123+0000","last_fullsized":"2024-03-26T08:40:59.502123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:14:20.681225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.348766+0000","last_change":"2024-03-26T08:40:09.928125+0000","last_active":"2024-03-26T08:41:28.348766+0000","last_peered":"2024-03-26T08:41:28.348766+0000","last_clean":"2024-03-26T08:41:28.348766+0000","last_became_active":"2024-03-26T08:40:09.927616+0000","last_became_peered":"2024-03-26T08:40:09.927616+0000","last_unstale":"2024-03-26T08:41:28.348766+0000","last_undegraded":"2024-03-26T08:41:28.348766+0000","last_fullsized":"2024-03-26T08:41:28.348766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:12.764054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.462999+0000","last_change":"2024-03-26T08:40:09.229976+0000","last_active":"2024-03-26T08:40:59.462999+0000","last_peered":"2024-03-26T08:40:59.462999+0000","last_clean":"2024-03-26T08:40:59.462999+0000","last_became_active":"2024-03-26T08:40:09.229594+0000","last_became_peered":"2024-03-26T08:40:09.229594+0000","last_unstale":"2024-03-26T08:40:59.462999+0000","last_undegraded":"2024-03-26T08:40:59.462999+0000","last_fullsized":"2024-03-26T08:40:59.462999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:31:24.204853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350174+0000","last_change":"2024-03-26T08:40:09.931290+0000","last_active":"2024-03-26T08:41:28.350174+0000","last_peered":"2024-03-26T08:41:28.350174+0000","last_clean":"2024-03-26T08:41:28.350174+0000","last_became_active":"2024-03-26T08:40:09.930961+0000","last_became_peered":"2024-03-26T08:40:09.930961+0000","last_unstale":"2024-03-26T08:41:28.350174+0000","last_undegraded":"2024-03-26T08:41:28.350174+0000","last_fullsized":"2024-03-26T08:41:28.350174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:36.460807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419243+0000","last_change":"2024-03-26T08:40:09.232166+0000","last_active":"2024-03-26T08:40:59.419243+0000","last_peered":"2024-03-26T08:40:59.419243+0000","last_clean":"2024-03-26T08:40:59.419243+0000","last_became_active":"2024-03-26T08:40:09.231840+0000","last_became_peered":"2024-03-26T08:40:09.231840+0000","last_unstale":"2024-03-26T08:40:59.419243+0000","last_undegraded":"2024-03-26T08:40:59.419243+0000","last_fullsized":"2024-03-26T08:40:59.419243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:51.636322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352211+0000","last_change":"2024-03-26T08:40:09.924680+0000","last_active":"2024-03-26T08:41:28.352211+0000","last_peered":"2024-03-26T08:41:28.352211+0000","last_clean":"2024-03-26T08:41:28.352211+0000","last_became_active":"2024-03-26T08:40:09.924314+0000","last_became_peered":"2024-03-26T08:40:09.924314+0000","last_unstale":"2024-03-26T08:41:28.352211+0000","last_undegraded":"2024-03-26T08:41:28.352211+0000","last_fullsized":"2024-03-26T08:41:28.352211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:49:20.780950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:09.402862+0000","last_change":"2024-03-26T08:40:09.235997+0000","last_active":"2024-03-26T08:42:09.402862+0000","last_peered":"2024-03-26T08:42:09.402862+0000","last_clean":"2024-03-26T08:42:09.402862+0000","last_became_active":"2024-03-26T08:40:09.235729+0000","last_became_peered":"2024-03-26T08:40:09.235729+0000","last_unstale":"2024-03-26T08:42:09.402862+0000","last_undegraded":"2024-03-26T08:42:09.402862+0000","last_fullsized":"2024-03-26T08:42:09.402862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:46:12.322059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351871+0000","last_change":"2024-03-26T08:40:09.926371+0000","last_active":"2024-03-26T08:41:28.351871+0000","last_peered":"2024-03-26T08:41:28.351871+0000","last_clean":"2024-03-26T08:41:28.351871+0000","last_became_active":"2024-03-26T08:40:09.925536+0000","last_became_peered":"2024-03-26T08:40:09.925536+0000","last_unstale":"2024-03-26T08:41:28.351871+0000","last_undegraded":"2024-03-26T08:41:28.351871+0000","last_fullsized":"2024-03-26T08:41:28.351871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:50:39.101684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417249+0000","last_change":"2024-03-26T08:40:09.229018+0000","last_active":"2024-03-26T08:40:59.417249+0000","last_peered":"2024-03-26T08:40:59.417249+0000","last_clean":"2024-03-26T08:40:59.417249+0000","last_became_active":"2024-03-26T08:40:09.228680+0000","last_became_peered":"2024-03-26T08:40:09.228680+0000","last_unstale":"2024-03-26T08:40:59.417249+0000","last_undegraded":"2024-03-26T08:40:59.417249+0000","last_fullsized":"2024-03-26T08:40:59.417249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:31:10.065127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338799+0000","last_change":"2024-03-26T08:40:09.926152+0000","last_active":"2024-03-26T08:41:28.338799+0000","last_peered":"2024-03-26T08:41:28.338799+0000","last_clean":"2024-03-26T08:41:28.338799+0000","last_became_active":"2024-03-26T08:40:09.925825+0000","last_became_peered":"2024-03-26T08:40:09.925825+0000","last_unstale":"2024-03-26T08:41:28.338799+0000","last_undegraded":"2024-03-26T08:41:28.338799+0000","last_fullsized":"2024-03-26T08:41:28.338799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:28:24.287706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419184+0000","last_change":"2024-03-26T08:40:09.234699+0000","last_active":"2024-03-26T08:40:59.419184+0000","last_peered":"2024-03-26T08:40:59.419184+0000","last_clean":"2024-03-26T08:40:59.419184+0000","last_became_active":"2024-03-26T08:40:09.234451+0000","last_became_peered":"2024-03-26T08:40:09.234451+0000","last_unstale":"2024-03-26T08:40:59.419184+0000","last_undegraded":"2024-03-26T08:40:59.419184+0000","last_fullsized":"2024-03-26T08:40:59.419184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:53.337418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352387+0000","last_change":"2024-03-26T08:40:09.927023+0000","last_active":"2024-03-26T08:41:28.352387+0000","last_peered":"2024-03-26T08:41:28.352387+0000","last_clean":"2024-03-26T08:41:28.352387+0000","last_became_active":"2024-03-26T08:40:09.926734+0000","last_became_peered":"2024-03-26T08:40:09.926734+0000","last_unstale":"2024-03-26T08:41:28.352387+0000","last_undegraded":"2024-03-26T08:41:28.352387+0000","last_fullsized":"2024-03-26T08:41:28.352387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:19:44.137513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":297,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.403406+0000","last_change":"2024-03-26T08:40:09.230059+0000","last_active":"2024-03-26T08:42:10.403406+0000","last_peered":"2024-03-26T08:42:10.403406+0000","last_clean":"2024-03-26T08:42:10.403406+0000","last_became_active":"2024-03-26T08:40:09.229776+0000","last_became_peered":"2024-03-26T08:40:09.229776+0000","last_unstale":"2024-03-26T08:42:10.403406+0000","last_undegraded":"2024-03-26T08:42:10.403406+0000","last_fullsized":"2024-03-26T08:42:10.403406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:00.917694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70723,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":242,"num_read_kb":190,"num_write":37,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.334293+0000","last_change":"2024-03-26T08:40:09.930702+0000","last_active":"2024-03-26T08:41:28.334293+0000","last_peered":"2024-03-26T08:41:28.334293+0000","last_clean":"2024-03-26T08:41:28.334293+0000","last_became_active":"2024-03-26T08:40:09.930400+0000","last_became_peered":"2024-03-26T08:40:09.930400+0000","last_unstale":"2024-03-26T08:41:28.334293+0000","last_undegraded":"2024-03-26T08:41:28.334293+0000","last_fullsized":"2024-03-26T08:41:28.334293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:03:12.897812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422276+0000","last_change":"2024-03-26T08:40:09.225972+0000","last_active":"2024-03-26T08:40:59.422276+0000","last_peered":"2024-03-26T08:40:59.422276+0000","last_clean":"2024-03-26T08:40:59.422276+0000","last_became_active":"2024-03-26T08:40:09.225524+0000","last_became_peered":"2024-03-26T08:40:09.225524+0000","last_unstale":"2024-03-26T08:40:59.422276+0000","last_undegraded":"2024-03-26T08:40:59.422276+0000","last_fullsized":"2024-03-26T08:40:59.422276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:08:17.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.348903+0000","last_change":"2024-03-26T08:40:09.920628+0000","last_active":"2024-03-26T08:41:28.348903+0000","last_peered":"2024-03-26T08:41:28.348903+0000","last_clean":"2024-03-26T08:41:28.348903+0000","last_became_active":"2024-03-26T08:40:09.920300+0000","last_became_peered":"2024-03-26T08:40:09.920300+0000","last_unstale":"2024-03-26T08:41:28.348903+0000","last_undegraded":"2024-03-26T08:41:28.348903+0000","last_fullsized":"2024-03-26T08:41:28.348903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:02:16.503881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419684+0000","last_change":"2024-03-26T08:40:09.237802+0000","last_active":"2024-03-26T08:40:59.419684+0000","last_peered":"2024-03-26T08:40:59.419684+0000","last_clean":"2024-03-26T08:40:59.419684+0000","last_became_active":"2024-03-26T08:40:09.237425+0000","last_became_peered":"2024-03-26T08:40:09.237425+0000","last_unstale":"2024-03-26T08:40:59.419684+0000","last_undegraded":"2024-03-26T08:40:59.419684+0000","last_fullsized":"2024-03-26T08:40:59.419684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:05.953882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345665+0000","last_change":"2024-03-26T08:40:09.923221+0000","last_active":"2024-03-26T08:41:28.345665+0000","last_peered":"2024-03-26T08:41:28.345665+0000","last_clean":"2024-03-26T08:41:28.345665+0000","last_became_active":"2024-03-26T08:40:09.922894+0000","last_became_peered":"2024-03-26T08:40:09.922894+0000","last_unstale":"2024-03-26T08:41:28.345665+0000","last_undegraded":"2024-03-26T08:41:28.345665+0000","last_fullsized":"2024-03-26T08:41:28.345665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:24.887625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":141,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.408450+0000","last_change":"2024-03-26T08:40:09.225179+0000","last_active":"2024-03-26T08:42:10.408450+0000","last_peered":"2024-03-26T08:42:10.408450+0000","last_clean":"2024-03-26T08:42:10.408450+0000","last_became_active":"2024-03-26T08:40:09.224861+0000","last_became_peered":"2024-03-26T08:40:09.224861+0000","last_unstale":"2024-03-26T08:42:10.408450+0000","last_undegraded":"2024-03-26T08:42:10.408450+0000","last_fullsized":"2024-03-26T08:42:10.408450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:37:54.843284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.334405+0000","last_change":"2024-03-26T08:40:59.503355+0000","last_active":"2024-03-26T08:41:28.334405+0000","last_peered":"2024-03-26T08:41:28.334405+0000","last_clean":"2024-03-26T08:41:28.334405+0000","last_became_active":"2024-03-26T08:40:59.503009+0000","last_became_peered":"2024-03-26T08:40:59.503009+0000","last_unstale":"2024-03-26T08:41:28.334405+0000","last_undegraded":"2024-03-26T08:41:28.334405+0000","last_fullsized":"2024-03-26T08:41:28.334405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:08:20.694763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416469+0000","last_change":"2024-03-26T08:40:09.232024+0000","last_active":"2024-03-26T08:40:59.416469+0000","last_peered":"2024-03-26T08:40:59.416469+0000","last_clean":"2024-03-26T08:40:59.416469+0000","last_became_active":"2024-03-26T08:40:09.231715+0000","last_became_peered":"2024-03-26T08:40:09.231715+0000","last_unstale":"2024-03-26T08:40:59.416469+0000","last_undegraded":"2024-03-26T08:40:59.416469+0000","last_fullsized":"2024-03-26T08:40:59.416469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:02.229042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351134+0000","last_change":"2024-03-26T08:40:09.926893+0000","last_active":"2024-03-26T08:41:28.351134+0000","last_peered":"2024-03-26T08:41:28.351134+0000","last_clean":"2024-03-26T08:41:28.351134+0000","last_became_active":"2024-03-26T08:40:09.926529+0000","last_became_peered":"2024-03-26T08:40:09.926529+0000","last_unstale":"2024-03-26T08:41:28.351134+0000","last_undegraded":"2024-03-26T08:41:28.351134+0000","last_fullsized":"2024-03-26T08:41:28.351134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:10.189407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.428905+0000","last_change":"2024-03-26T08:40:59.428905+0000","last_active":"2024-03-26T08:40:59.428905+0000","last_peered":"2024-03-26T08:40:59.428905+0000","last_clean":"2024-03-26T08:40:59.428905+0000","last_became_active":"2024-03-26T08:40:59.423345+0000","last_became_peered":"2024-03-26T08:40:59.423345+0000","last_unstale":"2024-03-26T08:40:59.428905+0000","last_undegraded":"2024-03-26T08:40:59.428905+0000","last_fullsized":"2024-03-26T08:40:59.428905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:49.069421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.346046+0000","last_change":"2024-03-26T08:40:59.464333+0000","last_active":"2024-03-26T08:41:28.346046+0000","last_peered":"2024-03-26T08:41:28.346046+0000","last_clean":"2024-03-26T08:41:28.346046+0000","last_became_active":"2024-03-26T08:40:59.464016+0000","last_became_peered":"2024-03-26T08:40:59.464016+0000","last_unstale":"2024-03-26T08:41:28.346046+0000","last_undegraded":"2024-03-26T08:41:28.346046+0000","last_fullsized":"2024-03-26T08:41:28.346046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:50:05.770568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420974+0000","last_change":"2024-03-26T08:40:09.224385+0000","last_active":"2024-03-26T08:40:59.420974+0000","last_peered":"2024-03-26T08:40:59.420974+0000","last_clean":"2024-03-26T08:40:59.420974+0000","last_became_active":"2024-03-26T08:40:09.223991+0000","last_became_peered":"2024-03-26T08:40:09.223991+0000","last_unstale":"2024-03-26T08:40:59.420974+0000","last_undegraded":"2024-03-26T08:40:59.420974+0000","last_fullsized":"2024-03-26T08:40:59.420974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:55.808922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349692+0000","last_change":"2024-03-26T08:40:09.921072+0000","last_active":"2024-03-26T08:41:28.349692+0000","last_peered":"2024-03-26T08:41:28.349692+0000","last_clean":"2024-03-26T08:41:28.349692+0000","last_became_active":"2024-03-26T08:40:09.920747+0000","last_became_peered":"2024-03-26T08:40:09.920747+0000","last_unstale":"2024-03-26T08:41:28.349692+0000","last_undegraded":"2024-03-26T08:41:28.349692+0000","last_fullsized":"2024-03-26T08:41:28.349692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:20.862765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416822+0000","last_change":"2024-03-26T08:40:09.231344+0000","last_active":"2024-03-26T08:40:59.416822+0000","last_peered":"2024-03-26T08:40:59.416822+0000","last_clean":"2024-03-26T08:40:59.416822+0000","last_became_active":"2024-03-26T08:40:09.229313+0000","last_became_peered":"2024-03-26T08:40:09.229313+0000","last_unstale":"2024-03-26T08:40:59.416822+0000","last_undegraded":"2024-03-26T08:40:59.416822+0000","last_fullsized":"2024-03-26T08:40:59.416822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:37.130884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.343324+0000","last_change":"2024-03-26T08:40:09.926355+0000","last_active":"2024-03-26T08:41:28.343324+0000","last_peered":"2024-03-26T08:41:28.343324+0000","last_clean":"2024-03-26T08:41:28.343324+0000","last_became_active":"2024-03-26T08:40:09.922889+0000","last_became_peered":"2024-03-26T08:40:09.922889+0000","last_unstale":"2024-03-26T08:41:28.343324+0000","last_undegraded":"2024-03-26T08:41:28.343324+0000","last_fullsized":"2024-03-26T08:41:28.343324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:42:22.784811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344672+0000","last_change":"2024-03-26T08:40:09.919577+0000","last_active":"2024-03-26T08:41:28.344672+0000","last_peered":"2024-03-26T08:41:28.344672+0000","last_clean":"2024-03-26T08:41:28.344672+0000","last_became_active":"2024-03-26T08:40:09.919234+0000","last_became_peered":"2024-03-26T08:40:09.919234+0000","last_unstale":"2024-03-26T08:41:28.344672+0000","last_undegraded":"2024-03-26T08:41:28.344672+0000","last_fullsized":"2024-03-26T08:41:28.344672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:18.894052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.408078+0000","last_change":"2024-03-26T08:40:09.237190+0000","last_active":"2024-03-26T08:42:10.408078+0000","last_peered":"2024-03-26T08:42:10.408078+0000","last_clean":"2024-03-26T08:42:10.408078+0000","last_became_active":"2024-03-26T08:40:09.236862+0000","last_became_peered":"2024-03-26T08:40:09.236862+0000","last_unstale":"2024-03-26T08:42:10.408078+0000","last_undegraded":"2024-03-26T08:42:10.408078+0000","last_fullsized":"2024-03-26T08:42:10.408078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:29.900711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351087+0000","last_change":"2024-03-26T08:40:09.927444+0000","last_active":"2024-03-26T08:41:28.351087+0000","last_peered":"2024-03-26T08:41:28.351087+0000","last_clean":"2024-03-26T08:41:28.351087+0000","last_became_active":"2024-03-26T08:40:09.927152+0000","last_became_peered":"2024-03-26T08:40:09.927152+0000","last_unstale":"2024-03-26T08:41:28.351087+0000","last_undegraded":"2024-03-26T08:41:28.351087+0000","last_fullsized":"2024-03-26T08:41:28.351087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:44.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420563+0000","last_change":"2024-03-26T08:40:09.224831+0000","last_active":"2024-03-26T08:40:59.420563+0000","last_peered":"2024-03-26T08:40:59.420563+0000","last_clean":"2024-03-26T08:40:59.420563+0000","last_became_active":"2024-03-26T08:40:09.224447+0000","last_became_peered":"2024-03-26T08:40:09.224447+0000","last_unstale":"2024-03-26T08:40:59.420563+0000","last_undegraded":"2024-03-26T08:40:59.420563+0000","last_fullsized":"2024-03-26T08:40:59.420563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:25:50.935342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416753+0000","last_change":"2024-03-26T08:38:54.696342+0000","last_active":"2024-03-26T08:40:59.416753+0000","last_peered":"2024-03-26T08:40:59.416753+0000","last_clean":"2024-03-26T08:40:59.416753+0000","last_became_active":"2024-03-26T08:38:54.695839+0000","last_became_peered":"2024-03-26T08:38:54.695839+0000","last_unstale":"2024-03-26T08:40:59.416753+0000","last_undegraded":"2024-03-26T08:40:59.416753+0000","last_fullsized":"2024-03-26T08:40:59.416753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_clean_scrub_stamp":"2024-03-26T08:35:27.189917+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:13.219778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.343262+0000","last_change":"2024-03-26T08:40:09.930683+0000","last_active":"2024-03-26T08:41:28.343262+0000","last_peered":"2024-03-26T08:41:28.343262+0000","last_clean":"2024-03-26T08:41:28.343262+0000","last_became_active":"2024-03-26T08:40:09.930341+0000","last_became_peered":"2024-03-26T08:40:09.930341+0000","last_unstale":"2024-03-26T08:41:28.343262+0000","last_undegraded":"2024-03-26T08:41:28.343262+0000","last_fullsized":"2024-03-26T08:41:28.343262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:39:20.232007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416894+0000","last_change":"2024-03-26T08:40:09.228548+0000","last_active":"2024-03-26T08:40:59.416894+0000","last_peered":"2024-03-26T08:40:59.416894+0000","last_clean":"2024-03-26T08:40:59.416894+0000","last_became_active":"2024-03-26T08:40:09.228076+0000","last_became_peered":"2024-03-26T08:40:09.228076+0000","last_unstale":"2024-03-26T08:40:59.416894+0000","last_undegraded":"2024-03-26T08:40:59.416894+0000","last_fullsized":"2024-03-26T08:40:59.416894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:52:51.641128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416941+0000","last_change":"2024-03-26T08:40:09.235632+0000","last_active":"2024-03-26T08:40:59.416941+0000","last_peered":"2024-03-26T08:40:59.416941+0000","last_clean":"2024-03-26T08:40:59.416941+0000","last_became_active":"2024-03-26T08:40:09.235334+0000","last_became_peered":"2024-03-26T08:40:09.235334+0000","last_unstale":"2024-03-26T08:40:59.416941+0000","last_undegraded":"2024-03-26T08:40:59.416941+0000","last_fullsized":"2024-03-26T08:40:59.416941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:38:27.064431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.341593+0000","last_change":"2024-03-26T08:40:09.929137+0000","last_active":"2024-03-26T08:41:28.341593+0000","last_peered":"2024-03-26T08:41:28.341593+0000","last_clean":"2024-03-26T08:41:28.341593+0000","last_became_active":"2024-03-26T08:40:09.928831+0000","last_became_peered":"2024-03-26T08:40:09.928831+0000","last_unstale":"2024-03-26T08:41:28.341593+0000","last_undegraded":"2024-03-26T08:41:28.341593+0000","last_fullsized":"2024-03-26T08:41:28.341593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:19.235168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:29.257611+0000","last_change":"2024-03-26T08:40:09.227153+0000","last_active":"2024-03-26T08:41:29.257611+0000","last_peered":"2024-03-26T08:41:29.257611+0000","last_clean":"2024-03-26T08:41:29.257611+0000","last_became_active":"2024-03-26T08:40:09.226772+0000","last_became_peered":"2024-03-26T08:40:09.226772+0000","last_unstale":"2024-03-26T08:41:29.257611+0000","last_undegraded":"2024-03-26T08:41:29.257611+0000","last_fullsized":"2024-03-26T08:41:29.257611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:33.159538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350141+0000","last_change":"2024-03-26T08:40:09.934589+0000","last_active":"2024-03-26T08:41:28.350141+0000","last_peered":"2024-03-26T08:41:28.350141+0000","last_clean":"2024-03-26T08:41:28.350141+0000","last_became_active":"2024-03-26T08:40:09.934312+0000","last_became_peered":"2024-03-26T08:40:09.934312+0000","last_unstale":"2024-03-26T08:41:28.350141+0000","last_undegraded":"2024-03-26T08:41:28.350141+0000","last_fullsized":"2024-03-26T08:41:28.350141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:19:27.737506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420385+0000","last_change":"2024-03-26T08:40:09.230014+0000","last_active":"2024-03-26T08:40:59.420385+0000","last_peered":"2024-03-26T08:40:59.420385+0000","last_clean":"2024-03-26T08:40:59.420385+0000","last_became_active":"2024-03-26T08:40:09.229682+0000","last_became_peered":"2024-03-26T08:40:09.229682+0000","last_unstale":"2024-03-26T08:40:59.420385+0000","last_undegraded":"2024-03-26T08:40:59.420385+0000","last_fullsized":"2024-03-26T08:40:59.420385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:52.725810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.337290+0000","last_change":"2024-03-26T08:40:59.464419+0000","last_active":"2024-03-26T08:41:28.337290+0000","last_peered":"2024-03-26T08:41:28.337290+0000","last_clean":"2024-03-26T08:41:28.337290+0000","last_became_active":"2024-03-26T08:40:59.464051+0000","last_became_peered":"2024-03-26T08:40:59.464051+0000","last_unstale":"2024-03-26T08:41:28.337290+0000","last_undegraded":"2024-03-26T08:41:28.337290+0000","last_fullsized":"2024-03-26T08:41:28.337290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:01:47.975380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.501822+0000","last_change":"2024-03-26T08:40:09.226634+0000","last_active":"2024-03-26T08:40:59.501822+0000","last_peered":"2024-03-26T08:40:59.501822+0000","last_clean":"2024-03-26T08:40:59.501822+0000","last_became_active":"2024-03-26T08:40:09.226248+0000","last_became_peered":"2024-03-26T08:40:09.226248+0000","last_unstale":"2024-03-26T08:40:59.501822+0000","last_undegraded":"2024-03-26T08:40:59.501822+0000","last_fullsized":"2024-03-26T08:40:59.501822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:24:00.390573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.303390+0000","last_change":"2024-03-26T08:40:09.934848+0000","last_active":"2024-03-26T08:41:28.303390+0000","last_peered":"2024-03-26T08:41:28.303390+0000","last_clean":"2024-03-26T08:41:28.303390+0000","last_became_active":"2024-03-26T08:40:09.934521+0000","last_became_peered":"2024-03-26T08:40:09.934521+0000","last_unstale":"2024-03-26T08:41:28.303390+0000","last_undegraded":"2024-03-26T08:41:28.303390+0000","last_fullsized":"2024-03-26T08:41:28.303390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:24:25.994961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419371+0000","last_change":"2024-03-26T08:40:09.222945+0000","last_active":"2024-03-26T08:40:59.419371+0000","last_peered":"2024-03-26T08:40:59.419371+0000","last_clean":"2024-03-26T08:40:59.419371+0000","last_became_active":"2024-03-26T08:40:09.222564+0000","last_became_peered":"2024-03-26T08:40:09.222564+0000","last_unstale":"2024-03-26T08:40:59.419371+0000","last_undegraded":"2024-03-26T08:40:59.419371+0000","last_fullsized":"2024-03-26T08:40:59.419371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:35:11.181812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.347366+0000","last_change":"2024-03-26T08:40:09.932674+0000","last_active":"2024-03-26T08:41:28.347366+0000","last_peered":"2024-03-26T08:41:28.347366+0000","last_clean":"2024-03-26T08:41:28.347366+0000","last_became_active":"2024-03-26T08:40:09.932346+0000","last_became_peered":"2024-03-26T08:40:09.932346+0000","last_unstale":"2024-03-26T08:41:28.347366+0000","last_undegraded":"2024-03-26T08:41:28.347366+0000","last_fullsized":"2024-03-26T08:41:28.347366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:12:45.871467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.431219+0000","last_change":"2024-03-26T08:40:59.431219+0000","last_active":"2024-03-26T08:40:59.431219+0000","last_peered":"2024-03-26T08:40:59.431219+0000","last_clean":"2024-03-26T08:40:59.431219+0000","last_became_active":"2024-03-26T08:40:59.424868+0000","last_became_peered":"2024-03-26T08:40:59.424868+0000","last_unstale":"2024-03-26T08:40:59.431219+0000","last_undegraded":"2024-03-26T08:40:59.431219+0000","last_fullsized":"2024-03-26T08:40:59.431219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:02.837090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350683+0000","last_change":"2024-03-26T08:40:09.922766+0000","last_active":"2024-03-26T08:41:28.350683+0000","last_peered":"2024-03-26T08:41:28.350683+0000","last_clean":"2024-03-26T08:41:28.350683+0000","last_became_active":"2024-03-26T08:40:09.922453+0000","last_became_peered":"2024-03-26T08:40:09.922453+0000","last_unstale":"2024-03-26T08:41:28.350683+0000","last_undegraded":"2024-03-26T08:41:28.350683+0000","last_fullsized":"2024-03-26T08:41:28.350683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:13:28.740291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416510+0000","last_change":"2024-03-26T08:40:09.228114+0000","last_active":"2024-03-26T08:40:59.416510+0000","last_peered":"2024-03-26T08:40:59.416510+0000","last_clean":"2024-03-26T08:40:59.416510+0000","last_became_active":"2024-03-26T08:40:09.227803+0000","last_became_peered":"2024-03-26T08:40:09.227803+0000","last_unstale":"2024-03-26T08:40:59.416510+0000","last_undegraded":"2024-03-26T08:40:59.416510+0000","last_fullsized":"2024-03-26T08:40:59.416510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:31.717981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350700+0000","last_change":"2024-03-26T08:40:09.929671+0000","last_active":"2024-03-26T08:41:28.350700+0000","last_peered":"2024-03-26T08:41:28.350700+0000","last_clean":"2024-03-26T08:41:28.350700+0000","last_became_active":"2024-03-26T08:40:09.929222+0000","last_became_peered":"2024-03-26T08:40:09.929222+0000","last_unstale":"2024-03-26T08:41:28.350700+0000","last_undegraded":"2024-03-26T08:41:28.350700+0000","last_fullsized":"2024-03-26T08:41:28.350700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:00:50.579569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422136+0000","last_change":"2024-03-26T08:40:09.228994+0000","last_active":"2024-03-26T08:40:59.422136+0000","last_peered":"2024-03-26T08:40:59.422136+0000","last_clean":"2024-03-26T08:40:59.422136+0000","last_became_active":"2024-03-26T08:40:09.227138+0000","last_became_peered":"2024-03-26T08:40:09.227138+0000","last_unstale":"2024-03-26T08:40:59.422136+0000","last_undegraded":"2024-03-26T08:40:59.422136+0000","last_fullsized":"2024-03-26T08:40:59.422136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:06.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352041+0000","last_change":"2024-03-26T08:40:09.931752+0000","last_active":"2024-03-26T08:41:28.352041+0000","last_peered":"2024-03-26T08:41:28.352041+0000","last_clean":"2024-03-26T08:41:28.352041+0000","last_became_active":"2024-03-26T08:40:09.931436+0000","last_became_peered":"2024-03-26T08:40:09.931436+0000","last_unstale":"2024-03-26T08:41:28.352041+0000","last_undegraded":"2024-03-26T08:41:28.352041+0000","last_fullsized":"2024-03-26T08:41:28.352041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:40:54.864614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.418961+0000","last_change":"2024-03-26T08:40:09.238356+0000","last_active":"2024-03-26T08:40:59.418961+0000","last_peered":"2024-03-26T08:40:59.418961+0000","last_clean":"2024-03-26T08:40:59.418961+0000","last_became_active":"2024-03-26T08:40:09.238066+0000","last_became_peered":"2024-03-26T08:40:09.238066+0000","last_unstale":"2024-03-26T08:40:59.418961+0000","last_undegraded":"2024-03-26T08:40:59.418961+0000","last_fullsized":"2024-03-26T08:40:59.418961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:13.851490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349384+0000","last_change":"2024-03-26T08:40:59.427153+0000","last_active":"2024-03-26T08:41:28.349384+0000","last_peered":"2024-03-26T08:41:28.349384+0000","last_clean":"2024-03-26T08:41:28.349384+0000","last_became_active":"2024-03-26T08:40:59.423044+0000","last_became_peered":"2024-03-26T08:40:59.423044+0000","last_unstale":"2024-03-26T08:41:28.349384+0000","last_undegraded":"2024-03-26T08:41:28.349384+0000","last_fullsized":"2024-03-26T08:41:28.349384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:20.181241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463434+0000","last_change":"2024-03-26T08:40:09.238009+0000","last_active":"2024-03-26T08:40:59.463434+0000","last_peered":"2024-03-26T08:40:59.463434+0000","last_clean":"2024-03-26T08:40:59.463434+0000","last_became_active":"2024-03-26T08:40:09.237697+0000","last_became_peered":"2024-03-26T08:40:09.237697+0000","last_unstale":"2024-03-26T08:40:59.463434+0000","last_undegraded":"2024-03-26T08:40:59.463434+0000","last_fullsized":"2024-03-26T08:40:59.463434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:17:33.179285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.340759+0000","last_change":"2024-03-26T08:40:09.934672+0000","last_active":"2024-03-26T08:41:28.340759+0000","last_peered":"2024-03-26T08:41:28.340759+0000","last_clean":"2024-03-26T08:41:28.340759+0000","last_became_active":"2024-03-26T08:40:09.934365+0000","last_became_peered":"2024-03-26T08:40:09.934365+0000","last_unstale":"2024-03-26T08:41:28.340759+0000","last_undegraded":"2024-03-26T08:41:28.340759+0000","last_fullsized":"2024-03-26T08:41:28.340759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:41:22.318481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.346142+0000","last_change":"2024-03-26T08:40:09.933910+0000","last_active":"2024-03-26T08:41:28.346142+0000","last_peered":"2024-03-26T08:41:28.346142+0000","last_clean":"2024-03-26T08:41:28.346142+0000","last_became_active":"2024-03-26T08:40:09.933598+0000","last_became_peered":"2024-03-26T08:40:09.933598+0000","last_unstale":"2024-03-26T08:41:28.346142+0000","last_undegraded":"2024-03-26T08:41:28.346142+0000","last_fullsized":"2024-03-26T08:41:28.346142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:19.877984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338094+0000","last_change":"2024-03-26T08:40:09.935809+0000","last_active":"2024-03-26T08:41:28.338094+0000","last_peered":"2024-03-26T08:41:28.338094+0000","last_clean":"2024-03-26T08:41:28.338094+0000","last_became_active":"2024-03-26T08:40:09.935500+0000","last_became_peered":"2024-03-26T08:40:09.935500+0000","last_unstale":"2024-03-26T08:41:28.338094+0000","last_undegraded":"2024-03-26T08:41:28.338094+0000","last_fullsized":"2024-03-26T08:41:28.338094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:23.717845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345125+0000","last_change":"2024-03-26T08:40:09.928093+0000","last_active":"2024-03-26T08:41:28.345125+0000","last_peered":"2024-03-26T08:41:28.345125+0000","last_clean":"2024-03-26T08:41:28.345125+0000","last_became_active":"2024-03-26T08:40:09.927732+0000","last_became_peered":"2024-03-26T08:40:09.927732+0000","last_unstale":"2024-03-26T08:41:28.345125+0000","last_undegraded":"2024-03-26T08:41:28.345125+0000","last_fullsized":"2024-03-26T08:41:28.345125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:35:34.972435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338020+0000","last_change":"2024-03-26T08:40:59.463368+0000","last_active":"2024-03-26T08:41:28.338020+0000","last_peered":"2024-03-26T08:41:28.338020+0000","last_clean":"2024-03-26T08:41:28.338020+0000","last_became_active":"2024-03-26T08:40:59.462979+0000","last_became_peered":"2024-03-26T08:40:59.462979+0000","last_unstale":"2024-03-26T08:41:28.338020+0000","last_undegraded":"2024-03-26T08:41:28.338020+0000","last_fullsized":"2024-03-26T08:41:28.338020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:13:01.413240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.342567+0000","last_change":"2024-03-26T08:40:09.919466+0000","last_active":"2024-03-26T08:41:28.342567+0000","last_peered":"2024-03-26T08:41:28.342567+0000","last_clean":"2024-03-26T08:41:28.342567+0000","last_became_active":"2024-03-26T08:40:09.919152+0000","last_became_peered":"2024-03-26T08:40:09.919152+0000","last_unstale":"2024-03-26T08:41:28.342567+0000","last_undegraded":"2024-03-26T08:41:28.342567+0000","last_fullsized":"2024-03-26T08:41:28.342567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:37.928485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345579+0000","last_change":"2024-03-26T08:40:09.927140+0000","last_active":"2024-03-26T08:41:28.345579+0000","last_peered":"2024-03-26T08:41:28.345579+0000","last_clean":"2024-03-26T08:41:28.345579+0000","last_became_active":"2024-03-26T08:40:09.926823+0000","last_became_peered":"2024-03-26T08:40:09.926823+0000","last_unstale":"2024-03-26T08:41:28.345579+0000","last_undegraded":"2024-03-26T08:41:28.345579+0000","last_fullsized":"2024-03-26T08:41:28.345579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:29:38.477123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353887+0000","last_change":"2024-03-26T08:40:09.934202+0000","last_active":"2024-03-26T08:41:28.353887+0000","last_peered":"2024-03-26T08:41:28.353887+0000","last_clean":"2024-03-26T08:41:28.353887+0000","last_became_active":"2024-03-26T08:40:09.933878+0000","last_became_peered":"2024-03-26T08:40:09.933878+0000","last_unstale":"2024-03-26T08:41:28.353887+0000","last_undegraded":"2024-03-26T08:41:28.353887+0000","last_fullsized":"2024-03-26T08:41:28.353887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:44:26.576140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345991+0000","last_change":"2024-03-26T08:40:09.929019+0000","last_active":"2024-03-26T08:41:28.345991+0000","last_peered":"2024-03-26T08:41:28.345991+0000","last_clean":"2024-03-26T08:41:28.345991+0000","last_became_active":"2024-03-26T08:40:09.928711+0000","last_became_peered":"2024-03-26T08:40:09.928711+0000","last_unstale":"2024-03-26T08:41:28.345991+0000","last_undegraded":"2024-03-26T08:41:28.345991+0000","last_fullsized":"2024-03-26T08:41:28.345991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:47.120826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.347133+0000","last_change":"2024-03-26T08:40:09.927606+0000","last_active":"2024-03-26T08:41:28.347133+0000","last_peered":"2024-03-26T08:41:28.347133+0000","last_clean":"2024-03-26T08:41:28.347133+0000","last_became_active":"2024-03-26T08:40:09.926717+0000","last_became_peered":"2024-03-26T08:40:09.926717+0000","last_unstale":"2024-03-26T08:41:28.347133+0000","last_undegraded":"2024-03-26T08:41:28.347133+0000","last_fullsized":"2024-03-26T08:41:28.347133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:51:52.567252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350770+0000","last_change":"2024-03-26T08:40:09.934659+0000","last_active":"2024-03-26T08:41:28.350770+0000","last_peered":"2024-03-26T08:41:28.350770+0000","last_clean":"2024-03-26T08:41:28.350770+0000","last_became_active":"2024-03-26T08:40:09.934346+0000","last_became_peered":"2024-03-26T08:40:09.934346+0000","last_unstale":"2024-03-26T08:41:28.350770+0000","last_undegraded":"2024-03-26T08:41:28.350770+0000","last_fullsized":"2024-03-26T08:41:28.350770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:35.009264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.348295+0000","last_change":"2024-03-26T08:40:09.925007+0000","last_active":"2024-03-26T08:41:28.348295+0000","last_peered":"2024-03-26T08:41:28.348295+0000","last_clean":"2024-03-26T08:41:28.348295+0000","last_became_active":"2024-03-26T08:40:09.923439+0000","last_became_peered":"2024-03-26T08:40:09.923439+0000","last_unstale":"2024-03-26T08:41:28.348295+0000","last_undegraded":"2024-03-26T08:41:28.348295+0000","last_fullsized":"2024-03-26T08:41:28.348295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:18.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.340590+0000","last_change":"2024-03-26T08:40:09.920795+0000","last_active":"2024-03-26T08:41:28.340590+0000","last_peered":"2024-03-26T08:41:28.340590+0000","last_clean":"2024-03-26T08:41:28.340590+0000","last_became_active":"2024-03-26T08:40:09.920378+0000","last_became_peered":"2024-03-26T08:40:09.920378+0000","last_unstale":"2024-03-26T08:41:28.340590+0000","last_undegraded":"2024-03-26T08:41:28.340590+0000","last_fullsized":"2024-03-26T08:41:28.340590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:19:04.027727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344142+0000","last_change":"2024-03-26T08:40:59.425704+0000","last_active":"2024-03-26T08:41:28.344142+0000","last_peered":"2024-03-26T08:41:28.344142+0000","last_clean":"2024-03-26T08:41:28.344142+0000","last_became_active":"2024-03-26T08:40:59.425014+0000","last_became_peered":"2024-03-26T08:40:59.425014+0000","last_unstale":"2024-03-26T08:41:28.344142+0000","last_undegraded":"2024-03-26T08:41:28.344142+0000","last_fullsized":"2024-03-26T08:41:28.344142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:08:14.847536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282339,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75564,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":729,"num_read_kb":444,"num_write":171,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":231033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":456298,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":1118480,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":456512,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456527,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1048503,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":455901,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":528937,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":457172,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":531518,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456212,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":456338,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1048769,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":73243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":75824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:42:13.204 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-26T08:42:13.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T08:42:13.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.351+0000 7f9314c7e640 1 -- 172.21.15.7:0/421904893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 msgr2=0x7f9310101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.351+0000 7f9314c7e640 1 --2- 172.21.15.7:0/421904893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 0x7f9310101c40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009900 tx=0x7f92f802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 -- 172.21.15.7:0/421904893 shutdown_connections 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 --2- 172.21.15.7:0/421904893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9310102180 0x7f9310108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 --2- 172.21.15.7:0/421904893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 0x7f9310101c40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 --2- 172.21.15.7:0/421904893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93101005d0 0x7f93101009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 -- 172.21.15.7:0/421904893 >> 172.21.15.7:0/421904893 conn(0x7f93100fbb60 msgr2=0x7f93100fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 -- 172.21.15.7:0/421904893 shutdown_connections 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 -- 172.21.15.7:0/421904893 wait complete. 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 Processor -- start 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 -- start start 2024-03-26T08:42:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.352+0000 7f9314c7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93101005d0 0x7f9310199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9314c7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 0x7f931019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9314c7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9310102180 0x7f93101a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9314c7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931010c250 con 0x7f93101017c0 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9314c7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931010c0d0 con 0x7f93101005d0 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9314c7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931010c3d0 con 0x7f9310102180 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9307fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 0x7f931019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9307fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 0x7f931019a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53428/0 (socket says 172.21.15.7:53428) 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9307fff640 1 -- 172.21.15.7:0/2838305012 learned_addr learned my addr 172.21.15.7:0/2838305012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f930e59c640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93101005d0 0x7f9310199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f930ed9d640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9310102180 0x7f93101a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9307fff640 1 -- 172.21.15.7:0/2838305012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9310102180 msgr2=0x7f93101a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9307fff640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9310102180 0x7f93101a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9307fff640 1 -- 172.21.15.7:0/2838305012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93101005d0 msgr2=0x7f9310199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9307fff640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93101005d0 0x7f9310199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9307fff640 1 -- 172.21.15.7:0/2838305012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93101a1800 con 0x7f93101017c0 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f930e59c640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93101005d0 0x7f9310199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9307fff640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 0x7f931019a120 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009230 tx=0x7f92f802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f93077fe640 1 -- 172.21.15.7:0/2838305012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8041070 con 0x7f93101017c0 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f93077fe640 1 -- 172.21.15.7:0/2838305012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92f8004450 con 0x7f93101017c0 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93101a1a90 con 0x7f93101017c0 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f93077fe640 1 -- 172.21.15.7:0/2838305012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8003e20 con 0x7f93101017c0 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.353+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93101a1f80 con 0x7f93101017c0 2024-03-26T08:42:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.354+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93101a2210 con 0x7f93101017c0 2024-03-26T08:42:13.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.355+0000 7f93077fe640 1 -- 172.21.15.7:0/2838305012 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f92f8044070 con 0x7f93101017c0 2024-03-26T08:42:13.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.356+0000 7f93077fe640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92fc0a2890 0x7f92fc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:13.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.356+0000 7f930e59c640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92fc0a2890 0x7f92fc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:13.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.356+0000 7f93077fe640 1 -- 172.21.15.7:0/2838305012 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f92f80f06d0 con 0x7f93101017c0 2024-03-26T08:42:13.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.356+0000 7f930e59c640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92fc0a2890 0x7f92fc0a4d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9310101620 tx=0x7f9300005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:13.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.360+0000 7f93077fe640 1 -- 172.21.15.7:0/2838305012 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f92f80bd090 con 0x7f93101017c0 2024-03-26T08:42:13.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.540+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 --> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93100fdcd0 con 0x7f92fc0a2890 2024-03-26T08:42:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.548+0000 7f93077fe640 1 -- 172.21.15.7:0/2838305012 <== mgr.14152 v2:172.21.15.7:6800/1233671832 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259090 (secure 0 0 0) 0x7f93100fdcd0 con 0x7f92fc0a2890 2024-03-26T08:42:13.543 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:13.544 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-03-26T08:42:13.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:13 smithi007 ceph-mon[32462]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-26T08:42:13.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:13 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1524551418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92fc0a2890 msgr2=0x7f92fc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92fc0a2890 0x7f92fc0a4d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9310101620 tx=0x7f9300005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 msgr2=0x7f931019a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 0x7f931019a120 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009230 tx=0x7f92f802c650 comp rx=0 tx=0).stop 2024-03-26T08:42:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 shutdown_connections 2024-03-26T08:42:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92fc0a2890 0x7f92fc0a4d50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9310102180 0x7f93101a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93101017c0 0x7f931019a120 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 --2- 172.21.15.7:0/2838305012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93101005d0 0x7f9310199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 >> 172.21.15.7:0/2838305012 conn(0x7f93100fbb60 msgr2=0x7f93100fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:13.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 shutdown_connections 2024-03-26T08:42:13.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.551+0000 7f9314c7e640 1 -- 172.21.15.7:0/2838305012 wait complete. 2024-03-26T08:42:13.554 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-03-26T08:42:12.329724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666166,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":855,"num_read_kb":553,"num_write":354,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180875,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373236,"kb_used_data":11604,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568588,"statfs":{"total":1151940427776,"available":1151558234112,"internally_reserved":0,"allocated":11882496,"data_stored":7471167,"data_compressed":177645,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6771,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":54,"num_write":8,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002952"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353784+0000","last_change":"2024-03-26T08:40:09.939078+0000","last_active":"2024-03-26T08:41:28.353784+0000","last_peered":"2024-03-26T08:41:28.353784+0000","last_clean":"2024-03-26T08:41:28.353784+0000","last_became_active":"2024-03-26T08:40:09.937626+0000","last_became_peered":"2024-03-26T08:40:09.937626+0000","last_unstale":"2024-03-26T08:41:28.353784+0000","last_undegraded":"2024-03-26T08:41:28.353784+0000","last_fullsized":"2024-03-26T08:41:28.353784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:52:59.254181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345392+0000","last_change":"2024-03-26T08:40:09.924207+0000","last_active":"2024-03-26T08:41:28.345392+0000","last_peered":"2024-03-26T08:41:28.345392+0000","last_clean":"2024-03-26T08:41:28.345392+0000","last_became_active":"2024-03-26T08:40:09.923881+0000","last_became_peered":"2024-03-26T08:40:09.923881+0000","last_unstale":"2024-03-26T08:41:28.345392+0000","last_undegraded":"2024-03-26T08:41:28.345392+0000","last_fullsized":"2024-03-26T08:41:28.345392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:16:45.265521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349461+0000","last_change":"2024-03-26T08:40:09.929154+0000","last_active":"2024-03-26T08:41:28.349461+0000","last_peered":"2024-03-26T08:41:28.349461+0000","last_clean":"2024-03-26T08:41:28.349461+0000","last_became_active":"2024-03-26T08:40:09.928833+0000","last_became_peered":"2024-03-26T08:40:09.928833+0000","last_unstale":"2024-03-26T08:41:28.349461+0000","last_undegraded":"2024-03-26T08:41:28.349461+0000","last_fullsized":"2024-03-26T08:41:28.349461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:17:22.972210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.341510+0000","last_change":"2024-03-26T08:40:09.930713+0000","last_active":"2024-03-26T08:41:28.341510+0000","last_peered":"2024-03-26T08:41:28.341510+0000","last_clean":"2024-03-26T08:41:28.341510+0000","last_became_active":"2024-03-26T08:40:09.930420+0000","last_became_peered":"2024-03-26T08:40:09.930420+0000","last_unstale":"2024-03-26T08:41:28.341510+0000","last_undegraded":"2024-03-26T08:41:28.341510+0000","last_fullsized":"2024-03-26T08:41:28.341510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:17:03.015703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.336520+0000","last_change":"2024-03-26T08:40:09.923935+0000","last_active":"2024-03-26T08:41:28.336520+0000","last_peered":"2024-03-26T08:41:28.336520+0000","last_clean":"2024-03-26T08:41:28.336520+0000","last_became_active":"2024-03-26T08:40:09.923587+0000","last_became_peered":"2024-03-26T08:40:09.923587+0000","last_unstale":"2024-03-26T08:41:28.336520+0000","last_undegraded":"2024-03-26T08:41:28.336520+0000","last_fullsized":"2024-03-26T08:41:28.336520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:19:38.831448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344419+0000","last_change":"2024-03-26T08:40:09.922361+0000","last_active":"2024-03-26T08:41:28.344419+0000","last_peered":"2024-03-26T08:41:28.344419+0000","last_clean":"2024-03-26T08:41:28.344419+0000","last_became_active":"2024-03-26T08:40:09.921643+0000","last_became_peered":"2024-03-26T08:40:09.921643+0000","last_unstale":"2024-03-26T08:41:28.344419+0000","last_undegraded":"2024-03-26T08:41:28.344419+0000","last_fullsized":"2024-03-26T08:41:28.344419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:37:23.048778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.341431+0000","last_change":"2024-03-26T08:40:09.932428+0000","last_active":"2024-03-26T08:41:28.341431+0000","last_peered":"2024-03-26T08:41:28.341431+0000","last_clean":"2024-03-26T08:41:28.341431+0000","last_became_active":"2024-03-26T08:40:09.932120+0000","last_became_peered":"2024-03-26T08:40:09.932120+0000","last_unstale":"2024-03-26T08:41:28.341431+0000","last_undegraded":"2024-03-26T08:41:28.341431+0000","last_fullsized":"2024-03-26T08:41:28.341431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:24.362568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351911+0000","last_change":"2024-03-26T08:40:09.928251+0000","last_active":"2024-03-26T08:41:28.351911+0000","last_peered":"2024-03-26T08:41:28.351911+0000","last_clean":"2024-03-26T08:41:28.351911+0000","last_became_active":"2024-03-26T08:40:09.927961+0000","last_became_peered":"2024-03-26T08:40:09.927961+0000","last_unstale":"2024-03-26T08:41:28.351911+0000","last_undegraded":"2024-03-26T08:41:28.351911+0000","last_fullsized":"2024-03-26T08:41:28.351911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:25:09.348033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351308+0000","last_change":"2024-03-26T08:40:09.930888+0000","last_active":"2024-03-26T08:41:28.351308+0000","last_peered":"2024-03-26T08:41:28.351308+0000","last_clean":"2024-03-26T08:41:28.351308+0000","last_became_active":"2024-03-26T08:40:09.930513+0000","last_became_peered":"2024-03-26T08:40:09.930513+0000","last_unstale":"2024-03-26T08:41:28.351308+0000","last_undegraded":"2024-03-26T08:41:28.351308+0000","last_fullsized":"2024-03-26T08:41:28.351308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:23:21.023569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345352+0000","last_change":"2024-03-26T08:40:09.935534+0000","last_active":"2024-03-26T08:41:28.345352+0000","last_peered":"2024-03-26T08:41:28.345352+0000","last_clean":"2024-03-26T08:41:28.345352+0000","last_became_active":"2024-03-26T08:40:09.934959+0000","last_became_peered":"2024-03-26T08:40:09.934959+0000","last_unstale":"2024-03-26T08:41:28.345352+0000","last_undegraded":"2024-03-26T08:41:28.345352+0000","last_fullsized":"2024-03-26T08:41:28.345352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:18:59.311180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351875+0000","last_change":"2024-03-26T08:40:09.930486+0000","last_active":"2024-03-26T08:41:28.351875+0000","last_peered":"2024-03-26T08:41:28.351875+0000","last_clean":"2024-03-26T08:41:28.351875+0000","last_became_active":"2024-03-26T08:40:09.930184+0000","last_became_peered":"2024-03-26T08:40:09.930184+0000","last_unstale":"2024-03-26T08:41:28.351875+0000","last_undegraded":"2024-03-26T08:41:28.351875+0000","last_fullsized":"2024-03-26T08:41:28.351875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:59:07.246878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345052+0000","last_change":"2024-03-26T08:40:09.924305+0000","last_active":"2024-03-26T08:41:28.345052+0000","last_peered":"2024-03-26T08:41:28.345052+0000","last_clean":"2024-03-26T08:41:28.345052+0000","last_became_active":"2024-03-26T08:40:09.923985+0000","last_became_peered":"2024-03-26T08:40:09.923985+0000","last_unstale":"2024-03-26T08:41:28.345052+0000","last_undegraded":"2024-03-26T08:41:28.345052+0000","last_fullsized":"2024-03-26T08:41:28.345052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:42:39.894245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350807+0000","last_change":"2024-03-26T08:40:09.916637+0000","last_active":"2024-03-26T08:41:28.350807+0000","last_peered":"2024-03-26T08:41:28.350807+0000","last_clean":"2024-03-26T08:41:28.350807+0000","last_became_active":"2024-03-26T08:40:09.916314+0000","last_became_peered":"2024-03-26T08:40:09.916314+0000","last_unstale":"2024-03-26T08:41:28.350807+0000","last_undegraded":"2024-03-26T08:41:28.350807+0000","last_fullsized":"2024-03-26T08:41:28.350807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:45:10.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345246+0000","last_change":"2024-03-26T08:40:09.923753+0000","last_active":"2024-03-26T08:41:28.345246+0000","last_peered":"2024-03-26T08:41:28.345246+0000","last_clean":"2024-03-26T08:41:28.345246+0000","last_became_active":"2024-03-26T08:40:09.923427+0000","last_became_peered":"2024-03-26T08:40:09.923427+0000","last_unstale":"2024-03-26T08:41:28.345246+0000","last_undegraded":"2024-03-26T08:41:28.345246+0000","last_fullsized":"2024-03-26T08:41:28.345246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:13:16.711556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338474+0000","last_change":"2024-03-26T08:40:09.931879+0000","last_active":"2024-03-26T08:41:28.338474+0000","last_peered":"2024-03-26T08:41:28.338474+0000","last_clean":"2024-03-26T08:41:28.338474+0000","last_became_active":"2024-03-26T08:40:09.931589+0000","last_became_peered":"2024-03-26T08:40:09.931589+0000","last_unstale":"2024-03-26T08:41:28.338474+0000","last_undegraded":"2024-03-26T08:41:28.338474+0000","last_fullsized":"2024-03-26T08:41:28.338474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:22:01.038745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353522+0000","last_change":"2024-03-26T08:40:09.930500+0000","last_active":"2024-03-26T08:41:28.353522+0000","last_peered":"2024-03-26T08:41:28.353522+0000","last_clean":"2024-03-26T08:41:28.353522+0000","last_became_active":"2024-03-26T08:40:09.930101+0000","last_became_peered":"2024-03-26T08:40:09.930101+0000","last_unstale":"2024-03-26T08:41:28.353522+0000","last_undegraded":"2024-03-26T08:41:28.353522+0000","last_fullsized":"2024-03-26T08:41:28.353522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:16:54.229004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.354043+0000","last_change":"2024-03-26T08:40:09.939177+0000","last_active":"2024-03-26T08:41:28.354043+0000","last_peered":"2024-03-26T08:41:28.354043+0000","last_clean":"2024-03-26T08:41:28.354043+0000","last_became_active":"2024-03-26T08:40:09.937727+0000","last_became_peered":"2024-03-26T08:40:09.937727+0000","last_unstale":"2024-03-26T08:41:28.354043+0000","last_undegraded":"2024-03-26T08:41:28.354043+0000","last_fullsized":"2024-03-26T08:41:28.354043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:40:44.040015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350313+0000","last_change":"2024-03-26T08:40:09.928429+0000","last_active":"2024-03-26T08:41:28.350313+0000","last_peered":"2024-03-26T08:41:28.350313+0000","last_clean":"2024-03-26T08:41:28.350313+0000","last_became_active":"2024-03-26T08:40:09.928025+0000","last_became_peered":"2024-03-26T08:40:09.928025+0000","last_unstale":"2024-03-26T08:41:28.350313+0000","last_undegraded":"2024-03-26T08:41:28.350313+0000","last_fullsized":"2024-03-26T08:41:28.350313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:52:55.450291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345617+0000","last_change":"2024-03-26T08:40:09.930856+0000","last_active":"2024-03-26T08:41:28.345617+0000","last_peered":"2024-03-26T08:41:28.345617+0000","last_clean":"2024-03-26T08:41:28.345617+0000","last_became_active":"2024-03-26T08:40:09.930552+0000","last_became_peered":"2024-03-26T08:40:09.930552+0000","last_unstale":"2024-03-26T08:41:28.345617+0000","last_undegraded":"2024-03-26T08:41:28.345617+0000","last_fullsized":"2024-03-26T08:41:28.345617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:52.851287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.421132+0000","last_change":"2024-03-26T08:40:09.230609+0000","last_active":"2024-03-26T08:40:59.421132+0000","last_peered":"2024-03-26T08:40:59.421132+0000","last_clean":"2024-03-26T08:40:59.421132+0000","last_became_active":"2024-03-26T08:40:09.230284+0000","last_became_peered":"2024-03-26T08:40:09.230284+0000","last_unstale":"2024-03-26T08:40:59.421132+0000","last_undegraded":"2024-03-26T08:40:59.421132+0000","last_fullsized":"2024-03-26T08:40:59.421132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:06.681725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.337071+0000","last_change":"2024-03-26T08:40:59.425449+0000","last_active":"2024-03-26T08:41:28.337071+0000","last_peered":"2024-03-26T08:41:28.337071+0000","last_clean":"2024-03-26T08:41:28.337071+0000","last_became_active":"2024-03-26T08:40:59.425164+0000","last_became_peered":"2024-03-26T08:40:59.425164+0000","last_unstale":"2024-03-26T08:41:28.337071+0000","last_undegraded":"2024-03-26T08:41:28.337071+0000","last_fullsized":"2024-03-26T08:41:28.337071+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:09.846252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419667+0000","last_change":"2024-03-26T08:40:09.235451+0000","last_active":"2024-03-26T08:40:59.419667+0000","last_peered":"2024-03-26T08:40:59.419667+0000","last_clean":"2024-03-26T08:40:59.419667+0000","last_became_active":"2024-03-26T08:40:09.235154+0000","last_became_peered":"2024-03-26T08:40:09.235154+0000","last_unstale":"2024-03-26T08:40:59.419667+0000","last_undegraded":"2024-03-26T08:40:59.419667+0000","last_fullsized":"2024-03-26T08:40:59.419667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:37:38.418888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349428+0000","last_change":"2024-03-26T08:40:09.931728+0000","last_active":"2024-03-26T08:41:28.349428+0000","last_peered":"2024-03-26T08:41:28.349428+0000","last_clean":"2024-03-26T08:41:28.349428+0000","last_became_active":"2024-03-26T08:40:09.931401+0000","last_became_peered":"2024-03-26T08:40:09.931401+0000","last_unstale":"2024-03-26T08:41:28.349428+0000","last_undegraded":"2024-03-26T08:41:28.349428+0000","last_fullsized":"2024-03-26T08:41:28.349428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:56:02.229763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417066+0000","last_change":"2024-03-26T08:40:09.231247+0000","last_active":"2024-03-26T08:40:59.417066+0000","last_peered":"2024-03-26T08:40:59.417066+0000","last_clean":"2024-03-26T08:40:59.417066+0000","last_became_active":"2024-03-26T08:40:09.230914+0000","last_became_peered":"2024-03-26T08:40:09.230914+0000","last_unstale":"2024-03-26T08:40:59.417066+0000","last_undegraded":"2024-03-26T08:40:59.417066+0000","last_fullsized":"2024-03-26T08:40:59.417066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:03.956444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353346+0000","last_change":"2024-03-26T08:40:09.925174+0000","last_active":"2024-03-26T08:41:28.353346+0000","last_peered":"2024-03-26T08:41:28.353346+0000","last_clean":"2024-03-26T08:41:28.353346+0000","last_became_active":"2024-03-26T08:40:09.924840+0000","last_became_peered":"2024-03-26T08:40:09.924840+0000","last_unstale":"2024-03-26T08:41:28.353346+0000","last_undegraded":"2024-03-26T08:41:28.353346+0000","last_fullsized":"2024-03-26T08:41:28.353346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:57:19.041245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416818+0000","last_change":"2024-03-26T08:40:09.229504+0000","last_active":"2024-03-26T08:40:59.416818+0000","last_peered":"2024-03-26T08:40:59.416818+0000","last_clean":"2024-03-26T08:40:59.416818+0000","last_became_active":"2024-03-26T08:40:09.226449+0000","last_became_peered":"2024-03-26T08:40:09.226449+0000","last_unstale":"2024-03-26T08:40:59.416818+0000","last_undegraded":"2024-03-26T08:40:59.416818+0000","last_fullsized":"2024-03-26T08:40:59.416818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:38.047424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351371+0000","last_change":"2024-03-26T08:40:09.930184+0000","last_active":"2024-03-26T08:41:28.351371+0000","last_peered":"2024-03-26T08:41:28.351371+0000","last_clean":"2024-03-26T08:41:28.351371+0000","last_became_active":"2024-03-26T08:40:09.927679+0000","last_became_peered":"2024-03-26T08:40:09.927679+0000","last_unstale":"2024-03-26T08:41:28.351371+0000","last_undegraded":"2024-03-26T08:41:28.351371+0000","last_fullsized":"2024-03-26T08:41:28.351371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:05:52.949814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'9","reported_seq":150,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.404325+0000","last_change":"2024-03-26T08:40:09.233025+0000","last_active":"2024-03-26T08:42:10.404325+0000","last_peered":"2024-03-26T08:42:10.404325+0000","last_clean":"2024-03-26T08:42:10.404325+0000","last_became_active":"2024-03-26T08:40:09.232654+0000","last_became_peered":"2024-03-26T08:40:09.232654+0000","last_unstale":"2024-03-26T08:42:10.404325+0000","last_undegraded":"2024-03-26T08:42:10.404325+0000","last_fullsized":"2024-03-26T08:42:10.404325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:01.745583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.333641+0000","last_change":"2024-03-26T08:40:09.924187+0000","last_active":"2024-03-26T08:41:28.333641+0000","last_peered":"2024-03-26T08:41:28.333641+0000","last_clean":"2024-03-26T08:41:28.333641+0000","last_became_active":"2024-03-26T08:40:09.923861+0000","last_became_peered":"2024-03-26T08:40:09.923861+0000","last_unstale":"2024-03-26T08:41:28.333641+0000","last_undegraded":"2024-03-26T08:41:28.333641+0000","last_fullsized":"2024-03-26T08:41:28.333641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:12:17.284186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.502123+0000","last_change":"2024-03-26T08:40:09.231368+0000","last_active":"2024-03-26T08:40:59.502123+0000","last_peered":"2024-03-26T08:40:59.502123+0000","last_clean":"2024-03-26T08:40:59.502123+0000","last_became_active":"2024-03-26T08:40:09.231040+0000","last_became_peered":"2024-03-26T08:40:09.231040+0000","last_unstale":"2024-03-26T08:40:59.502123+0000","last_undegraded":"2024-03-26T08:40:59.502123+0000","last_fullsized":"2024-03-26T08:40:59.502123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:14:20.681225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.348766+0000","last_change":"2024-03-26T08:40:09.928125+0000","last_active":"2024-03-26T08:41:28.348766+0000","last_peered":"2024-03-26T08:41:28.348766+0000","last_clean":"2024-03-26T08:41:28.348766+0000","last_became_active":"2024-03-26T08:40:09.927616+0000","last_became_peered":"2024-03-26T08:40:09.927616+0000","last_unstale":"2024-03-26T08:41:28.348766+0000","last_undegraded":"2024-03-26T08:41:28.348766+0000","last_fullsized":"2024-03-26T08:41:28.348766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:55:12.764054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.462999+0000","last_change":"2024-03-26T08:40:09.229976+0000","last_active":"2024-03-26T08:40:59.462999+0000","last_peered":"2024-03-26T08:40:59.462999+0000","last_clean":"2024-03-26T08:40:59.462999+0000","last_became_active":"2024-03-26T08:40:09.229594+0000","last_became_peered":"2024-03-26T08:40:09.229594+0000","last_unstale":"2024-03-26T08:40:59.462999+0000","last_undegraded":"2024-03-26T08:40:59.462999+0000","last_fullsized":"2024-03-26T08:40:59.462999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:31:24.204853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350174+0000","last_change":"2024-03-26T08:40:09.931290+0000","last_active":"2024-03-26T08:41:28.350174+0000","last_peered":"2024-03-26T08:41:28.350174+0000","last_clean":"2024-03-26T08:41:28.350174+0000","last_became_active":"2024-03-26T08:40:09.930961+0000","last_became_peered":"2024-03-26T08:40:09.930961+0000","last_unstale":"2024-03-26T08:41:28.350174+0000","last_undegraded":"2024-03-26T08:41:28.350174+0000","last_fullsized":"2024-03-26T08:41:28.350174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:38:36.460807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419243+0000","last_change":"2024-03-26T08:40:09.232166+0000","last_active":"2024-03-26T08:40:59.419243+0000","last_peered":"2024-03-26T08:40:59.419243+0000","last_clean":"2024-03-26T08:40:59.419243+0000","last_became_active":"2024-03-26T08:40:09.231840+0000","last_became_peered":"2024-03-26T08:40:09.231840+0000","last_unstale":"2024-03-26T08:40:59.419243+0000","last_undegraded":"2024-03-26T08:40:59.419243+0000","last_fullsized":"2024-03-26T08:40:59.419243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:22:51.636322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352211+0000","last_change":"2024-03-26T08:40:09.924680+0000","last_active":"2024-03-26T08:41:28.352211+0000","last_peered":"2024-03-26T08:41:28.352211+0000","last_clean":"2024-03-26T08:41:28.352211+0000","last_became_active":"2024-03-26T08:40:09.924314+0000","last_became_peered":"2024-03-26T08:40:09.924314+0000","last_unstale":"2024-03-26T08:41:28.352211+0000","last_undegraded":"2024-03-26T08:41:28.352211+0000","last_fullsized":"2024-03-26T08:41:28.352211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:49:20.780950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:09.402862+0000","last_change":"2024-03-26T08:40:09.235997+0000","last_active":"2024-03-26T08:42:09.402862+0000","last_peered":"2024-03-26T08:42:09.402862+0000","last_clean":"2024-03-26T08:42:09.402862+0000","last_became_active":"2024-03-26T08:40:09.235729+0000","last_became_peered":"2024-03-26T08:40:09.235729+0000","last_unstale":"2024-03-26T08:42:09.402862+0000","last_undegraded":"2024-03-26T08:42:09.402862+0000","last_fullsized":"2024-03-26T08:42:09.402862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:46:12.322059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351871+0000","last_change":"2024-03-26T08:40:09.926371+0000","last_active":"2024-03-26T08:41:28.351871+0000","last_peered":"2024-03-26T08:41:28.351871+0000","last_clean":"2024-03-26T08:41:28.351871+0000","last_became_active":"2024-03-26T08:40:09.925536+0000","last_became_peered":"2024-03-26T08:40:09.925536+0000","last_unstale":"2024-03-26T08:41:28.351871+0000","last_undegraded":"2024-03-26T08:41:28.351871+0000","last_fullsized":"2024-03-26T08:41:28.351871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:50:39.101684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.417249+0000","last_change":"2024-03-26T08:40:09.229018+0000","last_active":"2024-03-26T08:40:59.417249+0000","last_peered":"2024-03-26T08:40:59.417249+0000","last_clean":"2024-03-26T08:40:59.417249+0000","last_became_active":"2024-03-26T08:40:09.228680+0000","last_became_peered":"2024-03-26T08:40:09.228680+0000","last_unstale":"2024-03-26T08:40:59.417249+0000","last_undegraded":"2024-03-26T08:40:59.417249+0000","last_fullsized":"2024-03-26T08:40:59.417249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:31:10.065127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338799+0000","last_change":"2024-03-26T08:40:09.926152+0000","last_active":"2024-03-26T08:41:28.338799+0000","last_peered":"2024-03-26T08:41:28.338799+0000","last_clean":"2024-03-26T08:41:28.338799+0000","last_became_active":"2024-03-26T08:40:09.925825+0000","last_became_peered":"2024-03-26T08:40:09.925825+0000","last_unstale":"2024-03-26T08:41:28.338799+0000","last_undegraded":"2024-03-26T08:41:28.338799+0000","last_fullsized":"2024-03-26T08:41:28.338799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:28:24.287706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419184+0000","last_change":"2024-03-26T08:40:09.234699+0000","last_active":"2024-03-26T08:40:59.419184+0000","last_peered":"2024-03-26T08:40:59.419184+0000","last_clean":"2024-03-26T08:40:59.419184+0000","last_became_active":"2024-03-26T08:40:09.234451+0000","last_became_peered":"2024-03-26T08:40:09.234451+0000","last_unstale":"2024-03-26T08:40:59.419184+0000","last_undegraded":"2024-03-26T08:40:59.419184+0000","last_fullsized":"2024-03-26T08:40:59.419184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:41:53.337418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352387+0000","last_change":"2024-03-26T08:40:09.927023+0000","last_active":"2024-03-26T08:41:28.352387+0000","last_peered":"2024-03-26T08:41:28.352387+0000","last_clean":"2024-03-26T08:41:28.352387+0000","last_became_active":"2024-03-26T08:40:09.926734+0000","last_became_peered":"2024-03-26T08:40:09.926734+0000","last_unstale":"2024-03-26T08:41:28.352387+0000","last_undegraded":"2024-03-26T08:41:28.352387+0000","last_fullsized":"2024-03-26T08:41:28.352387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:19:44.137513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":297,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.403406+0000","last_change":"2024-03-26T08:40:09.230059+0000","last_active":"2024-03-26T08:42:10.403406+0000","last_peered":"2024-03-26T08:42:10.403406+0000","last_clean":"2024-03-26T08:42:10.403406+0000","last_became_active":"2024-03-26T08:40:09.229776+0000","last_became_peered":"2024-03-26T08:40:09.229776+0000","last_unstale":"2024-03-26T08:42:10.403406+0000","last_undegraded":"2024-03-26T08:42:10.403406+0000","last_fullsized":"2024-03-26T08:42:10.403406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:34:00.917694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70723,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":242,"num_read_kb":190,"num_write":37,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.334293+0000","last_change":"2024-03-26T08:40:09.930702+0000","last_active":"2024-03-26T08:41:28.334293+0000","last_peered":"2024-03-26T08:41:28.334293+0000","last_clean":"2024-03-26T08:41:28.334293+0000","last_became_active":"2024-03-26T08:40:09.930400+0000","last_became_peered":"2024-03-26T08:40:09.930400+0000","last_unstale":"2024-03-26T08:41:28.334293+0000","last_undegraded":"2024-03-26T08:41:28.334293+0000","last_fullsized":"2024-03-26T08:41:28.334293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:03:12.897812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422276+0000","last_change":"2024-03-26T08:40:09.225972+0000","last_active":"2024-03-26T08:40:59.422276+0000","last_peered":"2024-03-26T08:40:59.422276+0000","last_clean":"2024-03-26T08:40:59.422276+0000","last_became_active":"2024-03-26T08:40:09.225524+0000","last_became_peered":"2024-03-26T08:40:09.225524+0000","last_unstale":"2024-03-26T08:40:59.422276+0000","last_undegraded":"2024-03-26T08:40:59.422276+0000","last_fullsized":"2024-03-26T08:40:59.422276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:08:17.262967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.348903+0000","last_change":"2024-03-26T08:40:09.920628+0000","last_active":"2024-03-26T08:41:28.348903+0000","last_peered":"2024-03-26T08:41:28.348903+0000","last_clean":"2024-03-26T08:41:28.348903+0000","last_became_active":"2024-03-26T08:40:09.920300+0000","last_became_peered":"2024-03-26T08:40:09.920300+0000","last_unstale":"2024-03-26T08:41:28.348903+0000","last_undegraded":"2024-03-26T08:41:28.348903+0000","last_fullsized":"2024-03-26T08:41:28.348903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:02:16.503881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419684+0000","last_change":"2024-03-26T08:40:09.237802+0000","last_active":"2024-03-26T08:40:59.419684+0000","last_peered":"2024-03-26T08:40:59.419684+0000","last_clean":"2024-03-26T08:40:59.419684+0000","last_became_active":"2024-03-26T08:40:09.237425+0000","last_became_peered":"2024-03-26T08:40:09.237425+0000","last_unstale":"2024-03-26T08:40:59.419684+0000","last_undegraded":"2024-03-26T08:40:59.419684+0000","last_fullsized":"2024-03-26T08:40:59.419684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:06:05.953882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345665+0000","last_change":"2024-03-26T08:40:09.923221+0000","last_active":"2024-03-26T08:41:28.345665+0000","last_peered":"2024-03-26T08:41:28.345665+0000","last_clean":"2024-03-26T08:41:28.345665+0000","last_became_active":"2024-03-26T08:40:09.922894+0000","last_became_peered":"2024-03-26T08:40:09.922894+0000","last_unstale":"2024-03-26T08:41:28.345665+0000","last_undegraded":"2024-03-26T08:41:28.345665+0000","last_fullsized":"2024-03-26T08:41:28.345665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:24.887625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":141,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.408450+0000","last_change":"2024-03-26T08:40:09.225179+0000","last_active":"2024-03-26T08:42:10.408450+0000","last_peered":"2024-03-26T08:42:10.408450+0000","last_clean":"2024-03-26T08:42:10.408450+0000","last_became_active":"2024-03-26T08:40:09.224861+0000","last_became_peered":"2024-03-26T08:40:09.224861+0000","last_unstale":"2024-03-26T08:42:10.408450+0000","last_undegraded":"2024-03-26T08:42:10.408450+0000","last_fullsized":"2024-03-26T08:42:10.408450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:37:54.843284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.334405+0000","last_change":"2024-03-26T08:40:59.503355+0000","last_active":"2024-03-26T08:41:28.334405+0000","last_peered":"2024-03-26T08:41:28.334405+0000","last_clean":"2024-03-26T08:41:28.334405+0000","last_became_active":"2024-03-26T08:40:59.503009+0000","last_became_peered":"2024-03-26T08:40:59.503009+0000","last_unstale":"2024-03-26T08:41:28.334405+0000","last_undegraded":"2024-03-26T08:41:28.334405+0000","last_fullsized":"2024-03-26T08:41:28.334405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:08:20.694763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416469+0000","last_change":"2024-03-26T08:40:09.232024+0000","last_active":"2024-03-26T08:40:59.416469+0000","last_peered":"2024-03-26T08:40:59.416469+0000","last_clean":"2024-03-26T08:40:59.416469+0000","last_became_active":"2024-03-26T08:40:09.231715+0000","last_became_peered":"2024-03-26T08:40:09.231715+0000","last_unstale":"2024-03-26T08:40:59.416469+0000","last_undegraded":"2024-03-26T08:40:59.416469+0000","last_fullsized":"2024-03-26T08:40:59.416469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:02.229042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351134+0000","last_change":"2024-03-26T08:40:09.926893+0000","last_active":"2024-03-26T08:41:28.351134+0000","last_peered":"2024-03-26T08:41:28.351134+0000","last_clean":"2024-03-26T08:41:28.351134+0000","last_became_active":"2024-03-26T08:40:09.926529+0000","last_became_peered":"2024-03-26T08:40:09.926529+0000","last_unstale":"2024-03-26T08:41:28.351134+0000","last_undegraded":"2024-03-26T08:41:28.351134+0000","last_fullsized":"2024-03-26T08:41:28.351134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:36:10.189407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.428905+0000","last_change":"2024-03-26T08:40:59.428905+0000","last_active":"2024-03-26T08:40:59.428905+0000","last_peered":"2024-03-26T08:40:59.428905+0000","last_clean":"2024-03-26T08:40:59.428905+0000","last_became_active":"2024-03-26T08:40:59.423345+0000","last_became_peered":"2024-03-26T08:40:59.423345+0000","last_unstale":"2024-03-26T08:40:59.428905+0000","last_undegraded":"2024-03-26T08:40:59.428905+0000","last_fullsized":"2024-03-26T08:40:59.428905+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:49.069421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.346046+0000","last_change":"2024-03-26T08:40:59.464333+0000","last_active":"2024-03-26T08:41:28.346046+0000","last_peered":"2024-03-26T08:41:28.346046+0000","last_clean":"2024-03-26T08:41:28.346046+0000","last_became_active":"2024-03-26T08:40:59.464016+0000","last_became_peered":"2024-03-26T08:40:59.464016+0000","last_unstale":"2024-03-26T08:41:28.346046+0000","last_undegraded":"2024-03-26T08:41:28.346046+0000","last_fullsized":"2024-03-26T08:41:28.346046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:50:05.770568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420974+0000","last_change":"2024-03-26T08:40:09.224385+0000","last_active":"2024-03-26T08:40:59.420974+0000","last_peered":"2024-03-26T08:40:59.420974+0000","last_clean":"2024-03-26T08:40:59.420974+0000","last_became_active":"2024-03-26T08:40:09.223991+0000","last_became_peered":"2024-03-26T08:40:09.223991+0000","last_unstale":"2024-03-26T08:40:59.420974+0000","last_undegraded":"2024-03-26T08:40:59.420974+0000","last_fullsized":"2024-03-26T08:40:59.420974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:07:55.808922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349692+0000","last_change":"2024-03-26T08:40:09.921072+0000","last_active":"2024-03-26T08:41:28.349692+0000","last_peered":"2024-03-26T08:41:28.349692+0000","last_clean":"2024-03-26T08:41:28.349692+0000","last_became_active":"2024-03-26T08:40:09.920747+0000","last_became_peered":"2024-03-26T08:40:09.920747+0000","last_unstale":"2024-03-26T08:41:28.349692+0000","last_undegraded":"2024-03-26T08:41:28.349692+0000","last_fullsized":"2024-03-26T08:41:28.349692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:57:20.862765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416822+0000","last_change":"2024-03-26T08:40:09.231344+0000","last_active":"2024-03-26T08:40:59.416822+0000","last_peered":"2024-03-26T08:40:59.416822+0000","last_clean":"2024-03-26T08:40:59.416822+0000","last_became_active":"2024-03-26T08:40:09.229313+0000","last_became_peered":"2024-03-26T08:40:09.229313+0000","last_unstale":"2024-03-26T08:40:59.416822+0000","last_undegraded":"2024-03-26T08:40:59.416822+0000","last_fullsized":"2024-03-26T08:40:59.416822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:37.130884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.343324+0000","last_change":"2024-03-26T08:40:09.926355+0000","last_active":"2024-03-26T08:41:28.343324+0000","last_peered":"2024-03-26T08:41:28.343324+0000","last_clean":"2024-03-26T08:41:28.343324+0000","last_became_active":"2024-03-26T08:40:09.922889+0000","last_became_peered":"2024-03-26T08:40:09.922889+0000","last_unstale":"2024-03-26T08:41:28.343324+0000","last_undegraded":"2024-03-26T08:41:28.343324+0000","last_fullsized":"2024-03-26T08:41:28.343324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:42:22.784811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344672+0000","last_change":"2024-03-26T08:40:09.919577+0000","last_active":"2024-03-26T08:41:28.344672+0000","last_peered":"2024-03-26T08:41:28.344672+0000","last_clean":"2024-03-26T08:41:28.344672+0000","last_became_active":"2024-03-26T08:40:09.919234+0000","last_became_peered":"2024-03-26T08:40:09.919234+0000","last_unstale":"2024-03-26T08:41:28.344672+0000","last_undegraded":"2024-03-26T08:41:28.344672+0000","last_fullsized":"2024-03-26T08:41:28.344672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:18.894052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":140,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:42:10.408078+0000","last_change":"2024-03-26T08:40:09.237190+0000","last_active":"2024-03-26T08:42:10.408078+0000","last_peered":"2024-03-26T08:42:10.408078+0000","last_clean":"2024-03-26T08:42:10.408078+0000","last_became_active":"2024-03-26T08:40:09.236862+0000","last_became_peered":"2024-03-26T08:40:09.236862+0000","last_unstale":"2024-03-26T08:42:10.408078+0000","last_undegraded":"2024-03-26T08:42:10.408078+0000","last_fullsized":"2024-03-26T08:42:10.408078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:29.900711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.351087+0000","last_change":"2024-03-26T08:40:09.927444+0000","last_active":"2024-03-26T08:41:28.351087+0000","last_peered":"2024-03-26T08:41:28.351087+0000","last_clean":"2024-03-26T08:41:28.351087+0000","last_became_active":"2024-03-26T08:40:09.927152+0000","last_became_peered":"2024-03-26T08:40:09.927152+0000","last_unstale":"2024-03-26T08:41:28.351087+0000","last_undegraded":"2024-03-26T08:41:28.351087+0000","last_fullsized":"2024-03-26T08:41:28.351087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:44.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420563+0000","last_change":"2024-03-26T08:40:09.224831+0000","last_active":"2024-03-26T08:40:59.420563+0000","last_peered":"2024-03-26T08:40:59.420563+0000","last_clean":"2024-03-26T08:40:59.420563+0000","last_became_active":"2024-03-26T08:40:09.224447+0000","last_became_peered":"2024-03-26T08:40:09.224447+0000","last_unstale":"2024-03-26T08:40:59.420563+0000","last_undegraded":"2024-03-26T08:40:59.420563+0000","last_fullsized":"2024-03-26T08:40:59.420563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:25:50.935342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416753+0000","last_change":"2024-03-26T08:38:54.696342+0000","last_active":"2024-03-26T08:40:59.416753+0000","last_peered":"2024-03-26T08:40:59.416753+0000","last_clean":"2024-03-26T08:40:59.416753+0000","last_became_active":"2024-03-26T08:38:54.695839+0000","last_became_peered":"2024-03-26T08:38:54.695839+0000","last_unstale":"2024-03-26T08:40:59.416753+0000","last_undegraded":"2024-03-26T08:40:59.416753+0000","last_fullsized":"2024-03-26T08:40:59.416753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:35:27.189917+0000","last_clean_scrub_stamp":"2024-03-26T08:35:27.189917+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:13.219778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.343262+0000","last_change":"2024-03-26T08:40:09.930683+0000","last_active":"2024-03-26T08:41:28.343262+0000","last_peered":"2024-03-26T08:41:28.343262+0000","last_clean":"2024-03-26T08:41:28.343262+0000","last_became_active":"2024-03-26T08:40:09.930341+0000","last_became_peered":"2024-03-26T08:40:09.930341+0000","last_unstale":"2024-03-26T08:41:28.343262+0000","last_undegraded":"2024-03-26T08:41:28.343262+0000","last_fullsized":"2024-03-26T08:41:28.343262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:39:20.232007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416894+0000","last_change":"2024-03-26T08:40:09.228548+0000","last_active":"2024-03-26T08:40:59.416894+0000","last_peered":"2024-03-26T08:40:59.416894+0000","last_clean":"2024-03-26T08:40:59.416894+0000","last_became_active":"2024-03-26T08:40:09.228076+0000","last_became_peered":"2024-03-26T08:40:09.228076+0000","last_unstale":"2024-03-26T08:40:59.416894+0000","last_undegraded":"2024-03-26T08:40:59.416894+0000","last_fullsized":"2024-03-26T08:40:59.416894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T12:52:51.641128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416941+0000","last_change":"2024-03-26T08:40:09.235632+0000","last_active":"2024-03-26T08:40:59.416941+0000","last_peered":"2024-03-26T08:40:59.416941+0000","last_clean":"2024-03-26T08:40:59.416941+0000","last_became_active":"2024-03-26T08:40:09.235334+0000","last_became_peered":"2024-03-26T08:40:09.235334+0000","last_unstale":"2024-03-26T08:40:59.416941+0000","last_undegraded":"2024-03-26T08:40:59.416941+0000","last_fullsized":"2024-03-26T08:40:59.416941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:38:27.064431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.341593+0000","last_change":"2024-03-26T08:40:09.929137+0000","last_active":"2024-03-26T08:41:28.341593+0000","last_peered":"2024-03-26T08:41:28.341593+0000","last_clean":"2024-03-26T08:41:28.341593+0000","last_became_active":"2024-03-26T08:40:09.928831+0000","last_became_peered":"2024-03-26T08:40:09.928831+0000","last_unstale":"2024-03-26T08:41:28.341593+0000","last_undegraded":"2024-03-26T08:41:28.341593+0000","last_fullsized":"2024-03-26T08:41:28.341593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:12:19.235168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:29.257611+0000","last_change":"2024-03-26T08:40:09.227153+0000","last_active":"2024-03-26T08:41:29.257611+0000","last_peered":"2024-03-26T08:41:29.257611+0000","last_clean":"2024-03-26T08:41:29.257611+0000","last_became_active":"2024-03-26T08:40:09.226772+0000","last_became_peered":"2024-03-26T08:40:09.226772+0000","last_unstale":"2024-03-26T08:41:29.257611+0000","last_undegraded":"2024-03-26T08:41:29.257611+0000","last_fullsized":"2024-03-26T08:41:29.257611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:23:33.159538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350141+0000","last_change":"2024-03-26T08:40:09.934589+0000","last_active":"2024-03-26T08:41:28.350141+0000","last_peered":"2024-03-26T08:41:28.350141+0000","last_clean":"2024-03-26T08:41:28.350141+0000","last_became_active":"2024-03-26T08:40:09.934312+0000","last_became_peered":"2024-03-26T08:40:09.934312+0000","last_unstale":"2024-03-26T08:41:28.350141+0000","last_undegraded":"2024-03-26T08:41:28.350141+0000","last_fullsized":"2024-03-26T08:41:28.350141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:19:27.737506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.420385+0000","last_change":"2024-03-26T08:40:09.230014+0000","last_active":"2024-03-26T08:40:59.420385+0000","last_peered":"2024-03-26T08:40:59.420385+0000","last_clean":"2024-03-26T08:40:59.420385+0000","last_became_active":"2024-03-26T08:40:09.229682+0000","last_became_peered":"2024-03-26T08:40:09.229682+0000","last_unstale":"2024-03-26T08:40:59.420385+0000","last_undegraded":"2024-03-26T08:40:59.420385+0000","last_fullsized":"2024-03-26T08:40:59.420385+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:15:52.725810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.337290+0000","last_change":"2024-03-26T08:40:59.464419+0000","last_active":"2024-03-26T08:41:28.337290+0000","last_peered":"2024-03-26T08:41:28.337290+0000","last_clean":"2024-03-26T08:41:28.337290+0000","last_became_active":"2024-03-26T08:40:59.464051+0000","last_became_peered":"2024-03-26T08:40:59.464051+0000","last_unstale":"2024-03-26T08:41:28.337290+0000","last_undegraded":"2024-03-26T08:41:28.337290+0000","last_fullsized":"2024-03-26T08:41:28.337290+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:01:47.975380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.501822+0000","last_change":"2024-03-26T08:40:09.226634+0000","last_active":"2024-03-26T08:40:59.501822+0000","last_peered":"2024-03-26T08:40:59.501822+0000","last_clean":"2024-03-26T08:40:59.501822+0000","last_became_active":"2024-03-26T08:40:09.226248+0000","last_became_peered":"2024-03-26T08:40:09.226248+0000","last_unstale":"2024-03-26T08:40:59.501822+0000","last_undegraded":"2024-03-26T08:40:59.501822+0000","last_fullsized":"2024-03-26T08:40:59.501822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:24:00.390573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.303390+0000","last_change":"2024-03-26T08:40:09.934848+0000","last_active":"2024-03-26T08:41:28.303390+0000","last_peered":"2024-03-26T08:41:28.303390+0000","last_clean":"2024-03-26T08:41:28.303390+0000","last_became_active":"2024-03-26T08:40:09.934521+0000","last_became_peered":"2024-03-26T08:40:09.934521+0000","last_unstale":"2024-03-26T08:41:28.303390+0000","last_undegraded":"2024-03-26T08:41:28.303390+0000","last_fullsized":"2024-03-26T08:41:28.303390+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:24:25.994961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.419371+0000","last_change":"2024-03-26T08:40:09.222945+0000","last_active":"2024-03-26T08:40:59.419371+0000","last_peered":"2024-03-26T08:40:59.419371+0000","last_clean":"2024-03-26T08:40:59.419371+0000","last_became_active":"2024-03-26T08:40:09.222564+0000","last_became_peered":"2024-03-26T08:40:09.222564+0000","last_unstale":"2024-03-26T08:40:59.419371+0000","last_undegraded":"2024-03-26T08:40:59.419371+0000","last_fullsized":"2024-03-26T08:40:59.419371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:35:11.181812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.347366+0000","last_change":"2024-03-26T08:40:09.932674+0000","last_active":"2024-03-26T08:41:28.347366+0000","last_peered":"2024-03-26T08:41:28.347366+0000","last_clean":"2024-03-26T08:41:28.347366+0000","last_became_active":"2024-03-26T08:40:09.932346+0000","last_became_peered":"2024-03-26T08:40:09.932346+0000","last_unstale":"2024-03-26T08:41:28.347366+0000","last_undegraded":"2024-03-26T08:41:28.347366+0000","last_fullsized":"2024-03-26T08:41:28.347366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T13:12:45.871467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.431219+0000","last_change":"2024-03-26T08:40:59.431219+0000","last_active":"2024-03-26T08:40:59.431219+0000","last_peered":"2024-03-26T08:40:59.431219+0000","last_clean":"2024-03-26T08:40:59.431219+0000","last_became_active":"2024-03-26T08:40:59.424868+0000","last_became_peered":"2024-03-26T08:40:59.424868+0000","last_unstale":"2024-03-26T08:40:59.431219+0000","last_undegraded":"2024-03-26T08:40:59.431219+0000","last_fullsized":"2024-03-26T08:40:59.431219+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:05:02.837090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350683+0000","last_change":"2024-03-26T08:40:09.922766+0000","last_active":"2024-03-26T08:41:28.350683+0000","last_peered":"2024-03-26T08:41:28.350683+0000","last_clean":"2024-03-26T08:41:28.350683+0000","last_became_active":"2024-03-26T08:40:09.922453+0000","last_became_peered":"2024-03-26T08:40:09.922453+0000","last_unstale":"2024-03-26T08:41:28.350683+0000","last_undegraded":"2024-03-26T08:41:28.350683+0000","last_fullsized":"2024-03-26T08:41:28.350683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:13:28.740291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.416510+0000","last_change":"2024-03-26T08:40:09.228114+0000","last_active":"2024-03-26T08:40:59.416510+0000","last_peered":"2024-03-26T08:40:59.416510+0000","last_clean":"2024-03-26T08:40:59.416510+0000","last_became_active":"2024-03-26T08:40:09.227803+0000","last_became_peered":"2024-03-26T08:40:09.227803+0000","last_unstale":"2024-03-26T08:40:59.416510+0000","last_undegraded":"2024-03-26T08:40:59.416510+0000","last_fullsized":"2024-03-26T08:40:59.416510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:31.717981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350700+0000","last_change":"2024-03-26T08:40:09.929671+0000","last_active":"2024-03-26T08:41:28.350700+0000","last_peered":"2024-03-26T08:41:28.350700+0000","last_clean":"2024-03-26T08:41:28.350700+0000","last_became_active":"2024-03-26T08:40:09.929222+0000","last_became_peered":"2024-03-26T08:40:09.929222+0000","last_unstale":"2024-03-26T08:41:28.350700+0000","last_undegraded":"2024-03-26T08:41:28.350700+0000","last_fullsized":"2024-03-26T08:41:28.350700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:00:50.579569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.422136+0000","last_change":"2024-03-26T08:40:09.228994+0000","last_active":"2024-03-26T08:40:59.422136+0000","last_peered":"2024-03-26T08:40:59.422136+0000","last_clean":"2024-03-26T08:40:59.422136+0000","last_became_active":"2024-03-26T08:40:09.227138+0000","last_became_peered":"2024-03-26T08:40:09.227138+0000","last_unstale":"2024-03-26T08:40:59.422136+0000","last_undegraded":"2024-03-26T08:40:59.422136+0000","last_fullsized":"2024-03-26T08:40:59.422136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:28:06.533651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.352041+0000","last_change":"2024-03-26T08:40:09.931752+0000","last_active":"2024-03-26T08:41:28.352041+0000","last_peered":"2024-03-26T08:41:28.352041+0000","last_clean":"2024-03-26T08:41:28.352041+0000","last_became_active":"2024-03-26T08:40:09.931436+0000","last_became_peered":"2024-03-26T08:40:09.931436+0000","last_unstale":"2024-03-26T08:41:28.352041+0000","last_undegraded":"2024-03-26T08:41:28.352041+0000","last_fullsized":"2024-03-26T08:41:28.352041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:40:54.864614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.418961+0000","last_change":"2024-03-26T08:40:09.238356+0000","last_active":"2024-03-26T08:40:59.418961+0000","last_peered":"2024-03-26T08:40:59.418961+0000","last_clean":"2024-03-26T08:40:59.418961+0000","last_became_active":"2024-03-26T08:40:09.238066+0000","last_became_peered":"2024-03-26T08:40:09.238066+0000","last_unstale":"2024-03-26T08:40:59.418961+0000","last_undegraded":"2024-03-26T08:40:59.418961+0000","last_fullsized":"2024-03-26T08:40:59.418961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:27:13.851490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.349384+0000","last_change":"2024-03-26T08:40:59.427153+0000","last_active":"2024-03-26T08:41:28.349384+0000","last_peered":"2024-03-26T08:41:28.349384+0000","last_clean":"2024-03-26T08:41:28.349384+0000","last_became_active":"2024-03-26T08:40:59.423044+0000","last_became_peered":"2024-03-26T08:40:59.423044+0000","last_unstale":"2024-03-26T08:41:28.349384+0000","last_undegraded":"2024-03-26T08:41:28.349384+0000","last_fullsized":"2024-03-26T08:41:28.349384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:20.181241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:40:59.463434+0000","last_change":"2024-03-26T08:40:09.238009+0000","last_active":"2024-03-26T08:40:59.463434+0000","last_peered":"2024-03-26T08:40:59.463434+0000","last_clean":"2024-03-26T08:40:59.463434+0000","last_became_active":"2024-03-26T08:40:09.237697+0000","last_became_peered":"2024-03-26T08:40:09.237697+0000","last_unstale":"2024-03-26T08:40:59.463434+0000","last_undegraded":"2024-03-26T08:40:59.463434+0000","last_fullsized":"2024-03-26T08:40:59.463434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:08.207737+0000","last_clean_scrub_stamp":"2024-03-26T08:40:08.207737+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:17:33.179285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.340759+0000","last_change":"2024-03-26T08:40:09.934672+0000","last_active":"2024-03-26T08:41:28.340759+0000","last_peered":"2024-03-26T08:41:28.340759+0000","last_clean":"2024-03-26T08:41:28.340759+0000","last_became_active":"2024-03-26T08:40:09.934365+0000","last_became_peered":"2024-03-26T08:40:09.934365+0000","last_unstale":"2024-03-26T08:41:28.340759+0000","last_undegraded":"2024-03-26T08:41:28.340759+0000","last_fullsized":"2024-03-26T08:41:28.340759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:41:22.318481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.346142+0000","last_change":"2024-03-26T08:40:09.933910+0000","last_active":"2024-03-26T08:41:28.346142+0000","last_peered":"2024-03-26T08:41:28.346142+0000","last_clean":"2024-03-26T08:41:28.346142+0000","last_became_active":"2024-03-26T08:40:09.933598+0000","last_became_peered":"2024-03-26T08:40:09.933598+0000","last_unstale":"2024-03-26T08:41:28.346142+0000","last_undegraded":"2024-03-26T08:41:28.346142+0000","last_fullsized":"2024-03-26T08:41:28.346142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:56:19.877984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338094+0000","last_change":"2024-03-26T08:40:09.935809+0000","last_active":"2024-03-26T08:41:28.338094+0000","last_peered":"2024-03-26T08:41:28.338094+0000","last_clean":"2024-03-26T08:41:28.338094+0000","last_became_active":"2024-03-26T08:40:09.935500+0000","last_became_peered":"2024-03-26T08:40:09.935500+0000","last_unstale":"2024-03-26T08:41:28.338094+0000","last_undegraded":"2024-03-26T08:41:28.338094+0000","last_fullsized":"2024-03-26T08:41:28.338094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:23.717845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345125+0000","last_change":"2024-03-26T08:40:09.928093+0000","last_active":"2024-03-26T08:41:28.345125+0000","last_peered":"2024-03-26T08:41:28.345125+0000","last_clean":"2024-03-26T08:41:28.345125+0000","last_became_active":"2024-03-26T08:40:09.927732+0000","last_became_peered":"2024-03-26T08:40:09.927732+0000","last_unstale":"2024-03-26T08:41:28.345125+0000","last_undegraded":"2024-03-26T08:41:28.345125+0000","last_fullsized":"2024-03-26T08:41:28.345125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:35:34.972435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.338020+0000","last_change":"2024-03-26T08:40:59.463368+0000","last_active":"2024-03-26T08:41:28.338020+0000","last_peered":"2024-03-26T08:41:28.338020+0000","last_clean":"2024-03-26T08:41:28.338020+0000","last_became_active":"2024-03-26T08:40:59.462979+0000","last_became_peered":"2024-03-26T08:40:59.462979+0000","last_unstale":"2024-03-26T08:41:28.338020+0000","last_undegraded":"2024-03-26T08:41:28.338020+0000","last_fullsized":"2024-03-26T08:41:28.338020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:13:01.413240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.342567+0000","last_change":"2024-03-26T08:40:09.919466+0000","last_active":"2024-03-26T08:41:28.342567+0000","last_peered":"2024-03-26T08:41:28.342567+0000","last_clean":"2024-03-26T08:41:28.342567+0000","last_became_active":"2024-03-26T08:40:09.919152+0000","last_became_peered":"2024-03-26T08:40:09.919152+0000","last_unstale":"2024-03-26T08:41:28.342567+0000","last_undegraded":"2024-03-26T08:41:28.342567+0000","last_fullsized":"2024-03-26T08:41:28.342567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:37.928485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345579+0000","last_change":"2024-03-26T08:40:09.927140+0000","last_active":"2024-03-26T08:41:28.345579+0000","last_peered":"2024-03-26T08:41:28.345579+0000","last_clean":"2024-03-26T08:41:28.345579+0000","last_became_active":"2024-03-26T08:40:09.926823+0000","last_became_peered":"2024-03-26T08:40:09.926823+0000","last_unstale":"2024-03-26T08:41:28.345579+0000","last_undegraded":"2024-03-26T08:41:28.345579+0000","last_fullsized":"2024-03-26T08:41:28.345579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:29:38.477123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.353887+0000","last_change":"2024-03-26T08:40:09.934202+0000","last_active":"2024-03-26T08:41:28.353887+0000","last_peered":"2024-03-26T08:41:28.353887+0000","last_clean":"2024-03-26T08:41:28.353887+0000","last_became_active":"2024-03-26T08:40:09.933878+0000","last_became_peered":"2024-03-26T08:40:09.933878+0000","last_unstale":"2024-03-26T08:41:28.353887+0000","last_undegraded":"2024-03-26T08:41:28.353887+0000","last_fullsized":"2024-03-26T08:41:28.353887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T10:44:26.576140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.345991+0000","last_change":"2024-03-26T08:40:09.929019+0000","last_active":"2024-03-26T08:41:28.345991+0000","last_peered":"2024-03-26T08:41:28.345991+0000","last_clean":"2024-03-26T08:41:28.345991+0000","last_became_active":"2024-03-26T08:40:09.928711+0000","last_became_peered":"2024-03-26T08:40:09.928711+0000","last_unstale":"2024-03-26T08:41:28.345991+0000","last_undegraded":"2024-03-26T08:41:28.345991+0000","last_fullsized":"2024-03-26T08:41:28.345991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T08:40:47.120826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.347133+0000","last_change":"2024-03-26T08:40:09.927606+0000","last_active":"2024-03-26T08:41:28.347133+0000","last_peered":"2024-03-26T08:41:28.347133+0000","last_clean":"2024-03-26T08:41:28.347133+0000","last_became_active":"2024-03-26T08:40:09.926717+0000","last_became_peered":"2024-03-26T08:40:09.926717+0000","last_unstale":"2024-03-26T08:41:28.347133+0000","last_undegraded":"2024-03-26T08:41:28.347133+0000","last_fullsized":"2024-03-26T08:41:28.347133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T11:51:52.567252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.350770+0000","last_change":"2024-03-26T08:40:09.934659+0000","last_active":"2024-03-26T08:41:28.350770+0000","last_peered":"2024-03-26T08:41:28.350770+0000","last_clean":"2024-03-26T08:41:28.350770+0000","last_became_active":"2024-03-26T08:40:09.934346+0000","last_became_peered":"2024-03-26T08:40:09.934346+0000","last_unstale":"2024-03-26T08:41:28.350770+0000","last_undegraded":"2024-03-26T08:41:28.350770+0000","last_fullsized":"2024-03-26T08:41:28.350770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:35.009264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.348295+0000","last_change":"2024-03-26T08:40:09.925007+0000","last_active":"2024-03-26T08:41:28.348295+0000","last_peered":"2024-03-26T08:41:28.348295+0000","last_clean":"2024-03-26T08:41:28.348295+0000","last_became_active":"2024-03-26T08:40:09.923439+0000","last_became_peered":"2024-03-26T08:40:09.923439+0000","last_unstale":"2024-03-26T08:41:28.348295+0000","last_undegraded":"2024-03-26T08:41:28.348295+0000","last_fullsized":"2024-03-26T08:41:28.348295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:48:18.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.340590+0000","last_change":"2024-03-26T08:40:09.920795+0000","last_active":"2024-03-26T08:41:28.340590+0000","last_peered":"2024-03-26T08:41:28.340590+0000","last_clean":"2024-03-26T08:41:28.340590+0000","last_became_active":"2024-03-26T08:40:09.920378+0000","last_became_peered":"2024-03-26T08:40:09.920378+0000","last_unstale":"2024-03-26T08:41:28.340590+0000","last_undegraded":"2024-03-26T08:41:28.340590+0000","last_fullsized":"2024-03-26T08:41:28.340590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:19:04.027727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-03-26T08:41:28.344142+0000","last_change":"2024-03-26T08:40:59.425704+0000","last_active":"2024-03-26T08:41:28.344142+0000","last_peered":"2024-03-26T08:41:28.344142+0000","last_clean":"2024-03-26T08:41:28.344142+0000","last_became_active":"2024-03-26T08:40:59.425014+0000","last_became_peered":"2024-03-26T08:40:59.425014+0000","last_unstale":"2024-03-26T08:41:28.344142+0000","last_undegraded":"2024-03-26T08:41:28.344142+0000","last_fullsized":"2024-03-26T08:41:28.344142+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T08:40:09.209283+0000","last_clean_scrub_stamp":"2024-03-26T08:40:09.209283+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T09:08:14.847536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282339,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75564,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":729,"num_read_kb":444,"num_write":171,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":231033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":456298,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1105920,"data_stored":1118480,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":456512,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":456527,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332154,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":1048503,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":455901,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":528937,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":457172,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":531518,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456212,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":456338,"data_compressed":5827,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1048769,"data_compressed":41734,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34268,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35907,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":73243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":75824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T08:42:13.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:42:13.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:13 smithi044 ceph-mon[36461]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-26T08:42:13.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:13 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1524551418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.706+0000 7f21829f3640 1 -- 172.21.15.7:0/4169008469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c1032e0 msgr2=0x7f217c109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.706+0000 7f21829f3640 1 --2- 172.21.15.7:0/4169008469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c1032e0 0x7f217c109b70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2174009ca0 tx=0x7f217402c100 comp rx=0 tx=0).stop 2024-03-26T08:42:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.706+0000 7f21829f3640 1 -- 172.21.15.7:0/4169008469 shutdown_connections 2024-03-26T08:42:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.706+0000 7f21829f3640 1 --2- 172.21.15.7:0/4169008469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c1032e0 0x7f217c109b70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.706+0000 7f21829f3640 1 --2- 172.21.15.7:0/4169008469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c102920 0x7f217c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.706+0000 7f21829f3640 1 --2- 172.21.15.7:0/4169008469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f217c101730 0x7f217c101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.706+0000 7f21829f3640 1 -- 172.21.15.7:0/4169008469 >> 172.21.15.7:0/4169008469 conn(0x7f217c0fccc0 msgr2=0x7f217c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.706+0000 7f21829f3640 1 -- 172.21.15.7:0/4169008469 shutdown_connections 2024-03-26T08:42:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.706+0000 7f21829f3640 1 -- 172.21.15.7:0/4169008469 wait complete. 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f21829f3640 1 Processor -- start 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f21829f3640 1 -- start start 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f21829f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c101730 0x7f217c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f21829f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c102920 0x7f217c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f21829f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f217c1032e0 0x7f217c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f21829f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c10d320 con 0x7f217c101730 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f21829f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c10d1a0 con 0x7f217c1032e0 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f21829f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c10d4a0 con 0x7f217c102920 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c101730 0x7f217c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c101730 0x7f217c197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53450/0 (socket says 172.21.15.7:53450) 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217bfff640 1 -- 172.21.15.7:0/779283254 learned_addr learned my addr 172.21.15.7:0/779283254 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217b7fe640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c102920 0x7f217c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f2180f90640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f217c1032e0 0x7f217c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217bfff640 1 -- 172.21.15.7:0/779283254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c102920 msgr2=0x7f217c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217bfff640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c102920 0x7f217c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217bfff640 1 -- 172.21.15.7:0/779283254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f217c1032e0 msgr2=0x7f217c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217bfff640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f217c1032e0 0x7f217c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217bfff640 1 -- 172.21.15.7:0/779283254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c19f550 con 0x7f217c101730 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f2180f90640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f217c1032e0 0x7f217c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.707+0000 7f217b7fe640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c102920 0x7f217c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.708+0000 7f217bfff640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c101730 0x7f217c197930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f217000e9c0 tx=0x7f217000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.708+0000 7f21797fa640 1 -- 172.21.15.7:0/779283254 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000cde0 con 0x7f217c101730 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.708+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c19f840 con 0x7f217c101730 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.708+0000 7f21797fa640 1 -- 172.21.15.7:0/779283254 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2170010910 con 0x7f217c101730 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.708+0000 7f21797fa640 1 -- 172.21.15.7:0/779283254 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170018a10 con 0x7f217c101730 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.708+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c19fd90 con 0x7f217c101730 2024-03-26T08:42:13.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.709+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c1089f0 con 0x7f217c101730 2024-03-26T08:42:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.710+0000 7f21797fa640 1 -- 172.21.15.7:0/779283254 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f217001f020 con 0x7f217c101730 2024-03-26T08:42:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.710+0000 7f21797fa640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21640a2890 0x7f21640a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.710+0000 7f217b7fe640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21640a2890 0x7f21640a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.710+0000 7f21797fa640 1 -- 172.21.15.7:0/779283254 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f21700cedb0 con 0x7f217c101730 2024-03-26T08:42:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.711+0000 7f217b7fe640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21640a2890 0x7f21640a4d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f217c198e40 tx=0x7f216c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:13.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.715+0000 7f21797fa640 1 -- 172.21.15.7:0/779283254 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f217009bcf0 con 0x7f217c101730 2024-03-26T08:42:13.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:13 smithi016 ceph-mon[35851]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-26T08:42:13.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:13 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1524551418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:13.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.903+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f217c02d2d0 con 0x7f217c101730 2024-03-26T08:42:13.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.905+0000 7f21797fa640 1 -- 172.21.15.7:0/779283254 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7f217009b5f0 con 0x7f217c101730 2024-03-26T08:42:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:42:13.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21640a2890 msgr2=0x7f21640a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21640a2890 0x7f21640a4d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f217c198e40 tx=0x7f216c005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c101730 msgr2=0x7f217c197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c101730 0x7f217c197930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f217000e9c0 tx=0x7f217000ee80 comp rx=0 tx=0).stop 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 shutdown_connections 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21640a2890 0x7f21640a4d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f217c1032e0 0x7f217c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f217c102920 0x7f217c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 --2- 172.21.15.7:0/779283254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f217c101730 0x7f217c197930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 >> 172.21.15.7:0/779283254 conn(0x7f217c0fccc0 msgr2=0x7f217c105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 shutdown_connections 2024-03-26T08:42:13.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:13.907+0000 7f21829f3640 1 -- 172.21.15.7:0/779283254 wait complete. 2024-03-26T08:42:13.910 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:40:08.100906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:40:08.784972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:42:13.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:42:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.024+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/2566657748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c88104a70 msgr2=0x7f2c88106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.024+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/2566657748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c88104a70 0x7f2c88106e60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009900 tx=0x7f2c7c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.024+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/2566657748 shutdown_connections 2024-03-26T08:42:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.024+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/2566657748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c881073a0 0x7f2c881097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.024+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/2566657748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c88104a70 0x7f2c88106e60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.024+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/2566657748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c88102140 0x7f2c88104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.024+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/2566657748 >> 172.21.15.7:0/2566657748 conn(0x7f2c880fbb30 msgr2=0x7f2c880fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.024+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/2566657748 shutdown_connections 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.024+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/2566657748 wait complete. 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c8e0b2640 1 Processor -- start 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c8e0b2640 1 -- start start 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c8e0b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c88102140 0x7f2c88197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c8e0b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c88104a70 0x7f2c88197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c8e0b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c881073a0 0x7f2c8819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c8e0b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8810d330 con 0x7f2c881073a0 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c8e0b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8810d1b0 con 0x7f2c88104a70 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c8e0b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8810d4b0 con 0x7f2c88102140 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c881073a0 0x7f2c8819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c881073a0 0x7f2c8819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53480/0 (socket says 172.21.15.7:53480) 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c87fff640 1 -- 172.21.15.7:0/1702935293 learned_addr learned my addr 172.21.15.7:0/1702935293 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c877fe640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c88102140 0x7f2c88197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c86ffd640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c88104a70 0x7f2c88197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c87fff640 1 -- 172.21.15.7:0/1702935293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c88102140 msgr2=0x7f2c88197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c87fff640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c88102140 0x7f2c88197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c87fff640 1 -- 172.21.15.7:0/1702935293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c88104a70 msgr2=0x7f2c88197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c87fff640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c88104a70 0x7f2c88197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c87fff640 1 -- 172.21.15.7:0/1702935293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8819f620 con 0x7f2c881073a0 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.025+0000 7f2c877fe640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c88102140 0x7f2c88197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.026+0000 7f2c87fff640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c881073a0 0x7f2c8819efb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000eb10 tx=0x7f2c8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.026+0000 7f2c84ff9640 1 -- 172.21.15.7:0/1702935293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80004b20 con 0x7f2c881073a0 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.026+0000 7f2c84ff9640 1 -- 172.21.15.7:0/1702935293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c80004cc0 con 0x7f2c881073a0 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.026+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8819f910 con 0x7f2c881073a0 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.026+0000 7f2c84ff9640 1 -- 172.21.15.7:0/1702935293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80010760 con 0x7f2c881073a0 2024-03-26T08:42:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.026+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8819fde0 con 0x7f2c881073a0 2024-03-26T08:42:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.027+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c880fd620 con 0x7f2c881073a0 2024-03-26T08:42:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.028+0000 7f2c84ff9640 1 -- 172.21.15.7:0/1702935293 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f2c80005020 con 0x7f2c881073a0 2024-03-26T08:42:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.028+0000 7f2c84ff9640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2c700a2890 0x7f2c700a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.028+0000 7f2c877fe640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2c700a2890 0x7f2c700a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.028+0000 7f2c84ff9640 1 -- 172.21.15.7:0/1702935293 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2c800cf160 con 0x7f2c881073a0 2024-03-26T08:42:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.029+0000 7f2c877fe640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2c700a2890 0x7f2c700a4d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009fc0 tx=0x7f2c78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:14.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.033+0000 7f2c84ff9640 1 -- 172.21.15.7:0/1702935293 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c8009c0a0 con 0x7f2c881073a0 2024-03-26T08:42:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.216+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c8802d2c0 con 0x7f2c881073a0 2024-03-26T08:42:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.216+0000 7f2c84ff9640 1 -- 172.21.15.7:0/1702935293 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c8009b9a0 con 0x7f2c881073a0 2024-03-26T08:42:14.209 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:42:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2c700a2890 msgr2=0x7f2c700a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2c700a2890 0x7f2c700a4d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009fc0 tx=0x7f2c78005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c881073a0 msgr2=0x7f2c8819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c881073a0 0x7f2c8819efb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000eb10 tx=0x7f2c8000c2d0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 shutdown_connections 2024-03-26T08:42:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2c700a2890 0x7f2c700a4d50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c881073a0 0x7f2c8819efb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c88104a70 0x7f2c88197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 --2- 172.21.15.7:0/1702935293 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2c88102140 0x7f2c88197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 >> 172.21.15.7:0/1702935293 conn(0x7f2c880fbb30 msgr2=0x7f2c88102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.218+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 shutdown_connections 2024-03-26T08:42:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.219+0000 7f2c8e0b2640 1 -- 172.21.15.7:0/1702935293 wait complete. 2024-03-26T08:42:14.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/36120347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c4102170 msgr2=0x7ff7c4108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/36120347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c4102170 0x7ff7c4108a00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009ca0 tx=0x7ff7bc02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/36120347 shutdown_connections 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/36120347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c4102170 0x7ff7c4108a00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/36120347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c41017b0 0x7ff7c4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/36120347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7c41005c0 0x7ff7c41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/36120347 >> 172.21.15.7:0/36120347 conn(0x7ff7c40fbb30 msgr2=0x7ff7c40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/36120347 shutdown_connections 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/36120347 wait complete. 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.336+0000 7ff7cb4ff640 1 Processor -- start 2024-03-26T08:42:14.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7cb4ff640 1 -- start start 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7cb4ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c41005c0 0x7ff7c41979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7cb4ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7c41017b0 0x7ff7c4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7cb4ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4102170 0x7ff7c419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7cb4ff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c410c220 con 0x7ff7c41005c0 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7cb4ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c410c0a0 con 0x7ff7c41017b0 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7cb4ff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c410c3a0 con 0x7ff7c4102170 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c929b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c41005c0 0x7ff7c41979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c929b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c41005c0 0x7ff7c41979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53510/0 (socket says 172.21.15.7:53510) 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c929b640 1 -- 172.21.15.7:0/3553451494 learned_addr learned my addr 172.21.15.7:0/3553451494 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c8a9a640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7c41017b0 0x7ff7c4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c9a9c640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4102170 0x7ff7c419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c929b640 1 -- 172.21.15.7:0/3553451494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4102170 msgr2=0x7ff7c419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c929b640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4102170 0x7ff7c419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c929b640 1 -- 172.21.15.7:0/3553451494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7c41017b0 msgr2=0x7ff7c4197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c929b640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7c41017b0 0x7ff7c4197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c929b640 1 -- 172.21.15.7:0/3553451494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c419f5d0 con 0x7ff7c41005c0 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.337+0000 7ff7c929b640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c41005c0 0x7ff7c41979b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9c0 tx=0x7ff7c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.338+0000 7ff7ba7fc640 1 -- 172.21.15.7:0/3553451494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000cd80 con 0x7ff7c41005c0 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.338+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c419f8c0 con 0x7ff7c41005c0 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.338+0000 7ff7ba7fc640 1 -- 172.21.15.7:0/3553451494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7c00035f0 con 0x7ff7c41005c0 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.338+0000 7ff7ba7fc640 1 -- 172.21.15.7:0/3553451494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0010690 con 0x7ff7c41005c0 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.338+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c419fe10 con 0x7ff7c41005c0 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.339+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c4068970 con 0x7ff7c41005c0 2024-03-26T08:42:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.340+0000 7ff7ba7fc640 1 -- 172.21.15.7:0/3553451494 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7ff7c00037a0 con 0x7ff7c41005c0 2024-03-26T08:42:14.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.340+0000 7ff7ba7fc640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff7b00a2890 0x7ff7b00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.340+0000 7ff7ba7fc640 1 -- 172.21.15.7:0/3553451494 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff7c00cefa0 con 0x7ff7c41005c0 2024-03-26T08:42:14.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.340+0000 7ff7c8a9a640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff7b00a2890 0x7ff7b00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.341+0000 7ff7c8a9a640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff7b00a2890 0x7ff7b00a4d50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4198ec0 tx=0x7ff7b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:14.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.345+0000 7ff7ba7fc640 1 -- 172.21.15.7:0/3553451494 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff7c009bee0 con 0x7ff7c41005c0 2024-03-26T08:42:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.525+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7c41a0330 con 0x7ff7c41005c0 2024-03-26T08:42:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.526+0000 7ff7ba7fc640 1 -- 172.21.15.7:0/3553451494 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7c009b7e0 con 0x7ff7c41005c0 2024-03-26T08:42:14.518 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-26T08:42:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff7b00a2890 msgr2=0x7ff7b00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff7b00a2890 0x7ff7b00a4d50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4198ec0 tx=0x7ff7b400b410 comp rx=0 tx=0).stop 2024-03-26T08:42:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c41005c0 msgr2=0x7ff7c41979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c41005c0 0x7ff7c41979b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000e9c0 tx=0x7ff7c000ee80 comp rx=0 tx=0).stop 2024-03-26T08:42:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 shutdown_connections 2024-03-26T08:42:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff7b00a2890 0x7ff7b00a4d50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff7c4102170 0x7ff7c419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7c41017b0 0x7ff7c4197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 --2- 172.21.15.7:0/3553451494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7c41005c0 0x7ff7c41979b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 >> 172.21.15.7:0/3553451494 conn(0x7ff7c40fbb30 msgr2=0x7ff7c40fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 shutdown_connections 2024-03-26T08:42:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.528+0000 7ff7cb4ff640 1 -- 172.21.15.7:0/3553451494 wait complete. 2024-03-26T08:42:14.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:14 smithi007 ceph-mon[32462]: pgmap v354: 97 pgs: 97 active+clean; 651 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 8 op/s 2024-03-26T08:42:14.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:14 smithi007 ceph-mon[32462]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:42:14.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:14 smithi007 ceph-mon[32462]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:42:14.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:14 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/779283254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:14.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:14 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1702935293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:14.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:42:14.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:14 smithi044 ceph-mon[36461]: pgmap v354: 97 pgs: 97 active+clean; 651 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 8 op/s 2024-03-26T08:42:14.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:14 smithi044 ceph-mon[36461]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:42:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:14 smithi044 ceph-mon[36461]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:42:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:14 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/779283254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:14.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:14 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1702935293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 -- 172.21.15.7:0/689198057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61681016b0 msgr2=0x7f6168101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 --2- 172.21.15.7:0/689198057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61681016b0 0x7f6168101ad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6164009860 tx=0x7f616402bb70 comp rx=0 tx=0).stop 2024-03-26T08:42:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 -- 172.21.15.7:0/689198057 shutdown_connections 2024-03-26T08:42:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 --2- 172.21.15.7:0/689198057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6168103260 0x7f6168109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 --2- 172.21.15.7:0/689198057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f61681028a0 0x7f6168102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 --2- 172.21.15.7:0/689198057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61681016b0 0x7f6168101ad0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 -- 172.21.15.7:0/689198057 >> 172.21.15.7:0/689198057 conn(0x7f61680fcc20 msgr2=0x7f61680ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 -- 172.21.15.7:0/689198057 shutdown_connections 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 -- 172.21.15.7:0/689198057 wait complete. 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 Processor -- start 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.647+0000 7f61701d4640 1 -- start start 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f61701d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61681016b0 0x7f6168197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f61701d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61681028a0 0x7f6168197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f61701d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168103260 0x7f616819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f61701d4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616810d300 con 0x7f61681016b0 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f61701d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616810d180 con 0x7f61681028a0 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f61701d4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616810d480 con 0x7f6168103260 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616df70640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61681016b0 0x7f6168197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616df70640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61681016b0 0x7f6168197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53538/0 (socket says 172.21.15.7:53538) 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616df70640 1 -- 172.21.15.7:0/2919767420 learned_addr learned my addr 172.21.15.7:0/2919767420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616d76f640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61681028a0 0x7f6168197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616e771640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168103260 0x7f616819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616df70640 1 -- 172.21.15.7:0/2919767420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168103260 msgr2=0x7f616819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616df70640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168103260 0x7f616819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616df70640 1 -- 172.21.15.7:0/2919767420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61681028a0 msgr2=0x7f6168197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616df70640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61681028a0 0x7f6168197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616df70640 1 -- 172.21.15.7:0/2919767420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616819f640 con 0x7f61681016b0 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f616df70640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61681016b0 0x7f6168197a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f616402c0b0 tx=0x7f6164032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f615effd640 1 -- 172.21.15.7:0/2919767420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164041020 con 0x7f61681016b0 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f615effd640 1 -- 172.21.15.7:0/2919767420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f616402c9f0 con 0x7f61681016b0 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616819f8d0 con 0x7f61681016b0 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.648+0000 7f615effd640 1 -- 172.21.15.7:0/2919767420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164039040 con 0x7f61681016b0 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.649+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616819fdc0 con 0x7f61681016b0 2024-03-26T08:42:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.649+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6168108af0 con 0x7f61681016b0 2024-03-26T08:42:14.645 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:14 smithi016 ceph-mon[35851]: pgmap v354: 97 pgs: 97 active+clean; 651 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 8 op/s 2024-03-26T08:42:14.645 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:14 smithi016 ceph-mon[35851]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:42:14.645 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:14 smithi016 ceph-mon[35851]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:42:14.645 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:14 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/779283254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:14.645 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:14 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1702935293' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:42:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.651+0000 7f615effd640 1 -- 172.21.15.7:0/2919767420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f6164044020 con 0x7f61681016b0 2024-03-26T08:42:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.651+0000 7f615effd640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f61540a2890 0x7f61540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.651+0000 7f616d76f640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f61540a2890 0x7f61540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.651+0000 7f615effd640 1 -- 172.21.15.7:0/2919767420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f61640f0780 con 0x7f61681016b0 2024-03-26T08:42:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.652+0000 7f616d76f640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f61540a2890 0x7f61540a4d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6168198f30 tx=0x7f6158005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.655+0000 7f615effd640 1 -- 172.21.15.7:0/2919767420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f616404b020 con 0x7f61681016b0 2024-03-26T08:42:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.837+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f616802d070 con 0x7f61681016b0 2024-03-26T08:42:14.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.837+0000 7f615effd640 1 -- 172.21.15.7:0/2919767420 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61640bd7d0 con 0x7f61681016b0 2024-03-26T08:42:14.830 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f61540a2890 msgr2=0x7f61540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f61540a2890 0x7f61540a4d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6168198f30 tx=0x7f6158005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61681016b0 msgr2=0x7f6168197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61681016b0 0x7f6168197a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f616402c0b0 tx=0x7f6164032430 comp rx=0 tx=0).stop 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 shutdown_connections 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f61540a2890 0x7f61540a4d50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6168103260 0x7f616819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61681028a0 0x7f6168197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 --2- 172.21.15.7:0/2919767420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61681016b0 0x7f6168197a20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 >> 172.21.15.7:0/2919767420 conn(0x7f61680fcc20 msgr2=0x7f6168105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:14.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.839+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 shutdown_connections 2024-03-26T08:42:14.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.840+0000 7f61701d4640 1 -- 172.21.15.7:0/2919767420 wait complete. 2024-03-26T08:42:14.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:42:14.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 -- 172.21.15.7:0/1218103867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2c80ffd60 msgr2=0x7fb2c8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1218103867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2c80ffd60 0x7fb2c8100180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009900 tx=0x7fb2b002bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:14.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 -- 172.21.15.7:0/1218103867 shutdown_connections 2024-03-26T08:42:14.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1218103867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c8101910 0x7fb2c81081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1218103867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2c8100f50 0x7fb2c81013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1218103867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2c80ffd60 0x7fb2c8100180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 -- 172.21.15.7:0/1218103867 >> 172.21.15.7:0/1218103867 conn(0x7fb2c80fb2d0 msgr2=0x7fb2c80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 -- 172.21.15.7:0/1218103867 shutdown_connections 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 -- 172.21.15.7:0/1218103867 wait complete. 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 Processor -- start 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.954+0000 7fb2cd574640 1 -- start start 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2cd574640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2c80ffd60 0x7fb2c8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2cd574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2c8100f50 0x7fb2c8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2cd574640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c8101910 0x7fb2c819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2cd574640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c810c250 con 0x7fb2c8101910 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2cd574640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c810c0d0 con 0x7fb2c8100f50 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2cd574640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c810c3d0 con 0x7fb2c80ffd60 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c8101910 0x7fb2c819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2c80ffd60 0x7fb2c8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2c8100f50 0x7fb2c8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2c80ffd60 0x7fb2c8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33346/0 (socket says 172.21.15.7:33346) 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c6ffd640 1 -- 172.21.15.7:0/1664295602 learned_addr learned my addr 172.21.15.7:0/1664295602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c77fe640 1 -- 172.21.15.7:0/1664295602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2c80ffd60 msgr2=0x7fb2c8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c77fe640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2c80ffd60 0x7fb2c8197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c77fe640 1 -- 172.21.15.7:0/1664295602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2c8100f50 msgr2=0x7fb2c8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c77fe640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2c8100f50 0x7fb2c8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c77fe640 1 -- 172.21.15.7:0/1664295602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c819f670 con 0x7fb2c8101910 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c6ffd640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2c80ffd60 0x7fb2c8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2c77fe640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c8101910 0x7fb2c819f000 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009fd0 tx=0x7fb2c000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2bffff640 1 -- 172.21.15.7:0/1664295602 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c000ce20 con 0x7fb2c8101910 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c819f960 con 0x7fb2c8101910 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2bffff640 1 -- 172.21.15.7:0/1664295602 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb2c0005020 con 0x7fb2c8101910 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2bffff640 1 -- 172.21.15.7:0/1664295602 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0010720 con 0x7fb2c8101910 2024-03-26T08:42:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.955+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c819feb0 con 0x7fb2c8101910 2024-03-26T08:42:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.956+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c81a01a0 con 0x7fb2c8101910 2024-03-26T08:42:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.958+0000 7fb2bffff640 1 -- 172.21.15.7:0/1664295602 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fb2c0005540 con 0x7fb2c8101910 2024-03-26T08:42:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.958+0000 7fb2bffff640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2b40a2890 0x7fb2b40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.958+0000 7fb2c6ffd640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2b40a2890 0x7fb2b40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.958+0000 7fb2bffff640 1 -- 172.21.15.7:0/1664295602 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb2c00cf260 con 0x7fb2c8101910 2024-03-26T08:42:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.959+0000 7fb2c6ffd640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2b40a2890 0x7fb2b40a4d50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb2b002c8e0 tx=0x7fb2b0005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:14.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:14.962+0000 7fb2bffff640 1 -- 172.21.15.7:0/1664295602 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb2c009c1a0 con 0x7fb2c8101910 2024-03-26T08:42:15.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.189+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2c81a0370 con 0x7fb2c8101910 2024-03-26T08:42:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.190+0000 7fb2bffff640 1 -- 172.21.15.7:0/1664295602 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fb2c009baa0 con 0x7fb2c8101910 2024-03-26T08:42:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:15.182 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:42:08.844026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,24478,24502]},"id":1}]} 2024-03-26T08:42:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2b40a2890 msgr2=0x7fb2b40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2b40a2890 0x7fb2b40a4d50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb2b002c8e0 tx=0x7fb2b0005b50 comp rx=0 tx=0).stop 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c8101910 msgr2=0x7fb2c819f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c8101910 0x7fb2c819f000 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009fd0 tx=0x7fb2c000ef10 comp rx=0 tx=0).stop 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 shutdown_connections 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb2b40a2890 0x7fb2b40a4d50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2c8101910 0x7fb2c819f000 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2c8100f50 0x7fb2c8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 --2- 172.21.15.7:0/1664295602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb2c80ffd60 0x7fb2c8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 >> 172.21.15.7:0/1664295602 conn(0x7fb2c80fb2d0 msgr2=0x7fb2c8108ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 shutdown_connections 2024-03-26T08:42:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.192+0000 7fb2cd574640 1 -- 172.21.15.7:0/1664295602 wait complete. 2024-03-26T08:42:15.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:42:15.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.306+0000 7f041eaa1640 1 -- 172.21.15.7:0/4273029125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04181029d0 msgr2=0x7f0418102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.306+0000 7f041eaa1640 1 --2- 172.21.15.7:0/4273029125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04181029d0 0x7f0418102e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0404009890 tx=0x7f040402bb70 comp rx=0 tx=0).stop 2024-03-26T08:42:15.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 -- 172.21.15.7:0/4273029125 shutdown_connections 2024-03-26T08:42:15.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 --2- 172.21.15.7:0/4273029125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418103390 0x7f0418109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 --2- 172.21.15.7:0/4273029125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04181029d0 0x7f0418102e50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 --2- 172.21.15.7:0/4273029125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04180ffa00 0x7f04180ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 -- 172.21.15.7:0/4273029125 >> 172.21.15.7:0/4273029125 conn(0x7f04180fb310 msgr2=0x7f04180fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 -- 172.21.15.7:0/4273029125 shutdown_connections 2024-03-26T08:42:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 -- 172.21.15.7:0/4273029125 wait complete. 2024-03-26T08:42:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 Processor -- start 2024-03-26T08:42:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 -- start start 2024-03-26T08:42:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04180ffa00 0x7f04181022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04181029d0 0x7f0418100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0418103390 0x7f0418100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041810d350 con 0x7f04181029d0 2024-03-26T08:42:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041810d1d0 con 0x7f04180ffa00 2024-03-26T08:42:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041eaa1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041810d4d0 con 0x7f0418103390 2024-03-26T08:42:15.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f0417fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04181029d0 0x7f0418100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f0417fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04181029d0 0x7f0418100980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53580/0 (socket says 172.21.15.7:53580) 2024-03-26T08:42:15.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f0417fff640 1 -- 172.21.15.7:0/3697890873 learned_addr learned my addr 172.21.15.7:0/3697890873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041d03e640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0418103390 0x7f0418100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.307+0000 7f041c83d640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04180ffa00 0x7f04181022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f0417fff640 1 -- 172.21.15.7:0/3697890873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0418103390 msgr2=0x7f0418100ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f0417fff640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0418103390 0x7f0418100ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f0417fff640 1 -- 172.21.15.7:0/3697890873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04180ffa00 msgr2=0x7f04181022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f0417fff640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04180ffa00 0x7f04181022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f0417fff640 1 -- 172.21.15.7:0/3697890873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0418101780 con 0x7f04181029d0 2024-03-26T08:42:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f0417fff640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04181029d0 0x7f0418100980 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f040402c0b0 tx=0x7f0404032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f0415ffb640 1 -- 172.21.15.7:0/3697890873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404041020 con 0x7f04181029d0 2024-03-26T08:42:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f0415ffb640 1 -- 172.21.15.7:0/3697890873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f040402c9f0 con 0x7f04181029d0 2024-03-26T08:42:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f0415ffb640 1 -- 172.21.15.7:0/3697890873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404039040 con 0x7f04181029d0 2024-03-26T08:42:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04181a1b00 con 0x7f04181029d0 2024-03-26T08:42:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.308+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04181a1f70 con 0x7f04181029d0 2024-03-26T08:42:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.309+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0418108b00 con 0x7f04181029d0 2024-03-26T08:42:15.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.310+0000 7f0415ffb640 1 -- 172.21.15.7:0/3697890873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f0404044020 con 0x7f04181029d0 2024-03-26T08:42:15.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.311+0000 7f0415ffb640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f04080a2890 0x7f04080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.311+0000 7f0415ffb640 1 -- 172.21.15.7:0/3697890873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f04040f0390 con 0x7f04181029d0 2024-03-26T08:42:15.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.311+0000 7f041c83d640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f04080a2890 0x7f04080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.311+0000 7f041c83d640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f04080a2890 0x7f04080a4d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0418069c20 tx=0x7f040c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.315+0000 7f0415ffb640 1 -- 172.21.15.7:0/3697890873 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f040404b020 con 0x7f04181029d0 2024-03-26T08:42:15.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.500+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f041802d290 con 0x7f04181029d0 2024-03-26T08:42:15.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.501+0000 7f0415ffb640 1 -- 172.21.15.7:0/3697890873 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7f04040bd3a0 con 0x7f04181029d0 2024-03-26T08:42:15.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:15.495 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:42:15.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f04080a2890 msgr2=0x7f04080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f04080a2890 0x7f04080a4d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0418069c20 tx=0x7f040c005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:15.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04181029d0 msgr2=0x7f0418100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04181029d0 0x7f0418100980 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f040402c0b0 tx=0x7f0404032430 comp rx=0 tx=0).stop 2024-03-26T08:42:15.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 shutdown_connections 2024-03-26T08:42:15.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f04080a2890 0x7f04080a4d50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0418103390 0x7f0418100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04181029d0 0x7f0418100980 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 --2- 172.21.15.7:0/3697890873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04180ffa00 0x7f04181022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 >> 172.21.15.7:0/3697890873 conn(0x7f04180fb310 msgr2=0x7f0418105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.503+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 shutdown_connections 2024-03-26T08:42:15.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.504+0000 7f041eaa1640 1 -- 172.21.15.7:0/3697890873 wait complete. 2024-03-26T08:42:15.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:15 smithi007 ceph-mon[32462]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:42:15.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:15 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3553451494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:15.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:15 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2919767420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:15.506 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:15 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1664295602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:15.508 DEBUG:tasks.fwd_scrub:Yielding 2024-03-26T08:42:15.509 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-26T08:42:15.527 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-03-26T08:42:15.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-26T08:42:15.528 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-26T08:42:15.529 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:42:15.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:15.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:42:15.532 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T08:42:15.547 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T08:42:15.547 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi007.front.sepia.ceph.com 2024-03-26T08:42:15.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-03-26T08:42:15.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:15 smithi044 ceph-mon[36461]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:42:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:15 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3553451494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:15 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2919767420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:15.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:15 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1664295602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:15.587 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:15.603 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem Size Used Avail Use% Mounted on 2024-03-26T08:42:15.603 INFO:teuthology.orchestra.run.smithi007.stdout:0@2c753f3e-eb4b-11ee-95d0-87774f69a715.cephfs=/volumes/qa/sv_1/e0c5a2ed-a3a8-4daf-9d6d-978951487586 340G 0 340G 0% /tmp/cephfs 2024-03-26T08:42:15.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-03-26T08:42:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.646+0000 7f1d526b2640 1 -- 172.21.15.7:0/3110026828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c100890 msgr2=0x7f1d4c100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.646+0000 7f1d526b2640 1 --2- 172.21.15.7:0/3110026828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c100890 0x7f1d4c100cf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1d40009900 tx=0x7f1d4002bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.646+0000 7f1d526b2640 1 -- 172.21.15.7:0/3110026828 shutdown_connections 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.646+0000 7f1d526b2640 1 --2- 172.21.15.7:0/3110026828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d4c101230 0x7f1d4c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.646+0000 7f1d526b2640 1 --2- 172.21.15.7:0/3110026828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c100890 0x7f1d4c100cf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.646+0000 7f1d526b2640 1 --2- 172.21.15.7:0/3110026828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d4c0fff30 0x7f1d4c100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.646+0000 7f1d526b2640 1 -- 172.21.15.7:0/3110026828 >> 172.21.15.7:0/3110026828 conn(0x7f1d4c0fbb30 msgr2=0x7f1d4c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.646+0000 7f1d526b2640 1 -- 172.21.15.7:0/3110026828 shutdown_connections 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.646+0000 7f1d526b2640 1 -- 172.21.15.7:0/3110026828 wait complete. 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d526b2640 1 Processor -- start 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d526b2640 1 -- start start 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d526b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0fff30 0x7f1d4c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d526b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d4c100890 0x7f1d4c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d526b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d4c101230 0x7f1d4c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d526b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c10f520 con 0x7f1d4c0fff30 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d526b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c10f3a0 con 0x7f1d4c100890 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d526b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c10f6a0 con 0x7f1d4c101230 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0fff30 0x7f1d4c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0fff30 0x7f1d4c199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53606/0 (socket says 172.21.15.7:53606) 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d4bfff640 1 -- 172.21.15.7:0/202891349 learned_addr learned my addr 172.21.15.7:0/202891349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d4c100890 0x7f1d4c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.647+0000 7f1d50c4f640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d4c101230 0x7f1d4c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d4bfff640 1 -- 172.21.15.7:0/202891349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d4c101230 msgr2=0x7f1d4c1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d4c101230 0x7f1d4c1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d4bfff640 1 -- 172.21.15.7:0/202891349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d4c100890 msgr2=0x7f1d4c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d4c100890 0x7f1d4c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d4bfff640 1 -- 172.21.15.7:0/202891349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c1a17e0 con 0x7f1d4c0fff30 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d50c4f640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d4c101230 0x7f1d4c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d4c100890 0x7f1d4c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0fff30 0x7f1d4c199bc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00e9f0 tx=0x7f1d3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d497fa640 1 -- 172.21.15.7:0/202891349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c00cd80 con 0x7f1d4c0fff30 2024-03-26T08:42:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d497fa640 1 -- 172.21.15.7:0/202891349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c005500 con 0x7f1d4c0fff30 2024-03-26T08:42:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d497fa640 1 -- 172.21.15.7:0/202891349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c010690 con 0x7f1d4c0fff30 2024-03-26T08:42:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d526b2640 1 -- 172.21.15.7:0/202891349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4c1a1ad0 con 0x7f1d4c0fff30 2024-03-26T08:42:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.648+0000 7f1d526b2640 1 -- 172.21.15.7:0/202891349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4c1a1ff0 con 0x7f1d4c0fff30 2024-03-26T08:42:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.650+0000 7f1d497fa640 1 -- 172.21.15.7:0/202891349 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f1d3c0046c0 con 0x7f1d4c0fff30 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d5537b740 1 Processor -- start 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d5537b740 1 -- start start 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbca960 0x56220dbcad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbcfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbd04c0 0x56220dbd2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d5537b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220db0ac90 con 0x56220dbd04c0 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d5537b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220db30070 con 0x56220dbcb350 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d5537b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220dad8c80 con 0x56220dbca960 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d4b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbd04c0 0x56220dbd2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbcfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d50c4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbca960 0x56220dbcad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d50c4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbca960 0x56220dbcad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33386/0 (socket says 172.21.15.7:33386) 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d50c4f640 1 -- 172.21.15.7:0/2038133632 learned_addr learned my addr 172.21.15.7:0/2038133632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d497fa640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d340a2910 0x7f1d340a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d4bfff640 1 -- 172.21.15.7:0/2038133632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbca960 msgr2=0x56220dbcad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/2038133632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbca960 0x56220dbcad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d4bfff640 1 -- 172.21.15.7:0/2038133632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbd04c0 msgr2=0x56220dbd2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/2038133632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbd04c0 0x56220dbd2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d4bfff640 1 -- 172.21.15.7:0/2038133632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56220dad0d10 con 0x56220dbcb350 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d50c4f640 1 --2- 172.21.15.7:0/2038133632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbca960 0x56220dbcad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/2038133632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbd04c0 0x56220dbd2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.651+0000 7f1d50c4f640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d340a2910 0x7f1d340a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.652+0000 7f1d497fa640 1 -- 172.21.15.7:0/202891349 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d3c0cf0f0 con 0x7f1d4c0fff30 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.652+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/2038133632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbcfe50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0cca30 tx=0x7f1d3c021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb182cf1f427e3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.652+0000 7f1d327fc640 1 -- 172.21.15.7:0/2038133632 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c09b6c0 con 0x56220dbcb350 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.652+0000 7f1d50c4f640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d340a2910 0x7f1d340a4dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x56220dbcbbb0 tx=0x7f1d44004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.652+0000 7f1d327fc640 1 -- 172.21.15.7:0/2038133632 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c09bff0 con 0x56220dbcb350 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.652+0000 7f1d327fc640 1 -- 172.21.15.7:0/2038133632 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c09b250 con 0x56220dbcb350 2024-03-26T08:42:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 -- 172.21.15.7:0/2038133632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 msgr2=0x56220dbcfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2038133632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbcfe50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0cca30 tx=0x7f1d3c021000 comp rx=0 tx=0).stop 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 -- 172.21.15.7:0/2038133632 shutdown_connections 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2038133632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbd04c0 0x56220dbd2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2038133632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbcfe50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2038133632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbca960 0x56220dbcad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 -- 172.21.15.7:0/2038133632 >> 172.21.15.7:0/2038133632 conn(0x56220db931a0 msgr2=0x56220dbc62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 -- 172.21.15.7:0/2038133632 shutdown_connections 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 -- 172.21.15.7:0/2038133632 wait complete. 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 -- start start 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 Processor -- start 2024-03-26T08:42:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbcb350 0x56220dbd5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbd04c0 0x56220dbdc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220daec0d0 con 0x56220dbca960 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220db2c110 con 0x56220dbd04c0 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.653+0000 7f1d5537b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220dbd3410 con 0x56220dbcb350 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d5537b740 10 client.0 _ll_register_callbacks cb 0x56220db27e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbcb350 0x56220dbd5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbcb350 0x56220dbd5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33400/0 (socket says 172.21.15.7:33400) 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d50c4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbd04c0 0x56220dbdc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4b7fe640 1 -- 172.21.15.7:0/2898702326 learned_addr learned my addr 172.21.15.7:0/2898702326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4bfff640 1 -- 172.21.15.7:0/2898702326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbcb350 msgr2=0x56220dbd5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbcb350 0x56220dbd5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4bfff640 1 -- 172.21.15.7:0/2898702326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbd04c0 msgr2=0x56220dbdc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbd04c0 0x56220dbdc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4bfff640 1 -- 172.21.15.7:0/2898702326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56220d90c7e0 con 0x56220dbca960 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d50c4f640 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbd04c0 0x56220dbdc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbcb350 0x56220dbd5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd4c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c09bab0 tx=0x7f1d3c09b250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778020121b743e06 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/2898702326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c09b430 con 0x56220dbca960 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/2898702326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d3c016df0 con 0x56220dbca960 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d5537b740 1 -- 172.21.15.7:0/2898702326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d3c09bff0 con 0x56220dbca960 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/2898702326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d3c09b8f0 con 0x56220dbca960 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d5537b740 10 client.15444 fetch_fsmap learned FSMap version 23 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.654+0000 7f1d5537b740 1 -- 172.21.15.7:0/2898702326 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56220d96cf70 con 0x56220dbca960 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/2898702326 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c09bff0 con 0x56220dbca960 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/2898702326 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1d3c0a3c00 con 0x56220dbca960 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d5537b740 10 client.15444 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T08:42:15.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d5537b740 10 client.15444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d5537b740 20 client.15444 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56220dbe0680 0x56220dbe2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d5537b740 4 client.15444 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d5537b740 1 -- 172.21.15.7:0/2898702326 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56220db3d5b0 con 0x56220dbe0680 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56220dbe0680 0x56220dbe2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d2b7fe640 10 client.15444 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.655+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56220dbe0680 0x56220dbe2a70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f04bd12c44877b server_cookie=79cac9a279ce35a5 in_seq=0 out_seq=0 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/2898702326 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56220db3d5b0 con 0x56220dbe0680 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d2b7fe640 10 client.15444 handle_command_reply: tid=0 2024-03-26T08:42:15.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 client.15444 shutdown 2024-03-26T08:42:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 -- 172.21.15.7:0/2898702326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 msgr2=0x56220dbd4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd4c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c09bab0 tx=0x7f1d3c09b250 comp rx=0 tx=0).stop 2024-03-26T08:42:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 -- 172.21.15.7:0/2898702326 shutdown_connections 2024-03-26T08:42:15.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56220dbe0680 0x56220dbe2a70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbd04c0 0x56220dbdc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbcb350 0x56220dbd5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 --2- 172.21.15.7:0/2898702326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd4c10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 -- 172.21.15.7:0/2898702326 >> 172.21.15.7:0/2898702326 conn(0x56220db931a0 msgr2=0x56220dbc5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d2b7fe640 0 client.15444 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.656+0000 7f1d5537b740 1 -- 172.21.15.7:0/2898702326 shutdown_connections 2024-03-26T08:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.657+0000 7f1d5537b740 1 -- 172.21.15.7:0/2898702326 wait complete. 2024-03-26T08:42:15.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.657+0000 7f1d5537b740 20 client.15444 trim_cache size 0 max 16384 2024-03-26T08:42:15.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d5537b740 1 Processor -- start 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d5537b740 1 -- start start 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbca960 0x56220dbd3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbcb350 0x56220dbdc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbd4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d5537b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1ad100 con 0x56220dbcb350 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d5537b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1aac00 con 0x56220dbca960 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d5537b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1a8c80 con 0x56220dbd04c0 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbcb350 0x56220dbdc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d50c4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbd4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbca960 0x56220dbd3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbcb350 0x56220dbdc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53640/0 (socket says 172.21.15.7:53640) 2024-03-26T08:42:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d50c4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbd4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33416/0 (socket says 172.21.15.7:33416) 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4b7fe640 1 -- 172.21.15.7:0/872410363 learned_addr learned my addr 172.21.15.7:0/872410363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbca960 0x56220dbd3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36466/0 (socket says 172.21.15.7:36466) 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4bfff640 1 -- 172.21.15.7:0/872410363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 msgr2=0x56220dbd4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/872410363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbd4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4bfff640 1 -- 172.21.15.7:0/872410363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbcb350 msgr2=0x56220dbdc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/872410363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbcb350 0x56220dbdc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4bfff640 1 -- 172.21.15.7:0/872410363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56220d804170 con 0x56220dbca960 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/872410363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbcb350 0x56220dbdc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.673+0000 7f1d50c4f640 1 --2- 172.21.15.7:0/872410363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbd4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.674+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/872410363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbca960 0x56220dbd3ca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c097990 tx=0x7f1d3c099e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=900ff2464d987b32 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.674+0000 7f1d31ffb640 1 -- 172.21.15.7:0/872410363 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c09b430 con 0x56220dbca960 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.674+0000 7f1d31ffb640 1 -- 172.21.15.7:0/872410363 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c09bff0 con 0x56220dbca960 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.674+0000 7f1d31ffb640 1 -- 172.21.15.7:0/872410363 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c016df0 con 0x56220dbca960 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.674+0000 7f1d5537b740 1 -- 172.21.15.7:0/872410363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbca960 msgr2=0x56220dbd3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.674+0000 7f1d5537b740 1 --2- 172.21.15.7:0/872410363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbca960 0x56220dbd3ca0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c097990 tx=0x7f1d3c099e00 comp rx=0 tx=0).stop 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 -- 172.21.15.7:0/872410363 shutdown_connections 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 --2- 172.21.15.7:0/872410363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbd4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 --2- 172.21.15.7:0/872410363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbcb350 0x56220dbdc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 --2- 172.21.15.7:0/872410363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbca960 0x56220dbd3ca0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 -- 172.21.15.7:0/872410363 >> 172.21.15.7:0/872410363 conn(0x56220db931a0 msgr2=0x56220dbd4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 -- 172.21.15.7:0/872410363 shutdown_connections 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 -- 172.21.15.7:0/872410363 wait complete. 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 -- start start 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 Processor -- start 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbd97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbda800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1a9280 con 0x56220dbca960 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1aa5a0 con 0x56220dbcb350 2024-03-26T08:42:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4c1a98e0 con 0x56220dbd04c0 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d5537b740 10 client.0 _ll_register_callbacks cb 0x7f1d440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbd97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d50c4f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbda800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbd97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36482/0 (socket says 172.21.15.7:36482) 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53646/0 (socket says 172.21.15.7:53646) 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.675+0000 7f1d4b7fe640 1 -- 172.21.15.7:0/1380562025 learned_addr learned my addr 172.21.15.7:0/1380562025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d4bfff640 1 -- 172.21.15.7:0/1380562025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 msgr2=0x56220dbda800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbda800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d4bfff640 1 -- 172.21.15.7:0/1380562025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 msgr2=0x56220dbd97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbd97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d4bfff640 1 -- 172.21.15.7:0/1380562025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56220dbdadf0 con 0x56220dbca960 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbd97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d50c4f640 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbda800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d4bfff640 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd92b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c07ce50 tx=0x7f1d3c0a4100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a474c4074cd73e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/1380562025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c09b430 con 0x56220dbca960 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/1380562025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d3c016df0 con 0x56220dbca960 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d5537b740 1 -- 172.21.15.7:0/1380562025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d3c09bff0 con 0x56220dbca960 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/1380562025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d3c09b250 con 0x56220dbca960 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d5537b740 10 client.15450 fetch_fsmap learned FSMap version 23 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d5537b740 1 -- 172.21.15.7:0/1380562025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56220dbdadf0 con 0x56220dbca960 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.676+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/1380562025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c09bff0 con 0x56220dbca960 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/1380562025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1d3c0860e0 con 0x56220dbca960 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d5537b740 10 client.15450 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d5537b740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d5537b740 20 client.15450 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d5537b740 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56220dc3be90 0x56220dc3e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d5537b740 4 client.15450 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:42:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d5537b740 1 -- 172.21.15.7:0/1380562025 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56220db37900 con 0x56220dc3be90 2024-03-26T08:42:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56220dc3be90 0x56220dc3e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d2b7fe640 10 client.15450 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.677+0000 7f1d4b7fe640 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56220dc3be90 0x56220dc3e280 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa87b181db4f6f6 server_cookie=1d59a47bdb155f9a in_seq=0 out_seq=0 2024-03-26T08:42:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.678+0000 7f1d2b7fe640 1 -- 172.21.15.7:0/1380562025 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56220db37900 con 0x56220dc3be90 2024-03-26T08:42:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.678+0000 7f1d2b7fe640 10 client.15450 handle_command_reply: tid=0 2024-03-26T08:42:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.678+0000 7f1d5537b740 1 client.15450 shutdown 2024-03-26T08:42:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.678+0000 7f1d5537b740 1 -- 172.21.15.7:0/1380562025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 msgr2=0x56220dbd92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.678+0000 7f1d5537b740 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd92b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c07ce50 tx=0x7f1d3c0a4100 comp rx=0 tx=0).stop 2024-03-26T08:42:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d5537b740 1 -- 172.21.15.7:0/1380562025 shutdown_connections 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d5537b740 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56220dc3be90 0x56220dc3e280 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d5537b740 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56220dbd04c0 0x56220dbda800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d5537b740 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56220dbcb350 0x56220dbd97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d5537b740 1 --2- 172.21.15.7:0/1380562025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56220dbca960 0x56220dbd92b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d5537b740 1 -- 172.21.15.7:0/1380562025 >> 172.21.15.7:0/1380562025 conn(0x56220db931a0 msgr2=0x56220dbd5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d2b7fe640 0 client.15450 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d5537b740 1 -- 172.21.15.7:0/1380562025 shutdown_connections 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d5537b740 1 -- 172.21.15.7:0/1380562025 wait complete. 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d5537b740 20 client.15450 trim_cache size 0 max 16384 2024-03-26T08:42:15.677 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:42:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:42:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "c5dd5958-69eb-472f-8b93-3c7eb95b5edb", 2024-03-26T08:42:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:42:15.678 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:42:15.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 -- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d340a2910 msgr2=0x7f1d340a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d340a2910 0x7f1d340a4dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x56220dbcbbb0 tx=0x7f1d44004a40 comp rx=0 tx=0).stop 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 -- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0fff30 msgr2=0x7f1d4c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0fff30 0x7f1d4c199bc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00e9f0 tx=0x7f1d3c00edf0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 -- 172.21.15.7:0/202891349 shutdown_connections 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d340a2910 0x7f1d340a4dd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1d4c101230 0x7f1d4c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d4c100890 0x7f1d4c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 --2- 172.21.15.7:0/202891349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d4c0fff30 0x7f1d4c199bc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.679+0000 7f1d526b2640 1 -- 172.21.15.7:0/202891349 >> 172.21.15.7:0/202891349 conn(0x7f1d4c0fbb30 msgr2=0x7f1d4c0fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.680+0000 7f1d526b2640 1 -- 172.21.15.7:0/202891349 shutdown_connections 2024-03-26T08:42:15.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.680+0000 7f1d526b2640 1 -- 172.21.15.7:0/202891349 wait complete. 2024-03-26T08:42:15.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:42:15.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:42:15.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-03-26T08:42:15.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:15 smithi016 ceph-mon[35851]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T08:42:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:15 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3553451494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:42:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:15 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2919767420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:42:15.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:15 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1664295602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:15.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-03-26T08:42:15.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.850+0000 7f3df2d90640 1 -- 172.21.15.7:0/3813548958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1016d0 msgr2=0x7f3dec101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.850+0000 7f3df2d90640 1 --2- 172.21.15.7:0/3813548958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1016d0 0x7f3dec101af0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de002bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.850+0000 7f3df2d90640 1 -- 172.21.15.7:0/3813548958 shutdown_connections 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.850+0000 7f3df2d90640 1 --2- 172.21.15.7:0/3813548958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dec103280 0x7f3dec109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.850+0000 7f3df2d90640 1 --2- 172.21.15.7:0/3813548958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec1028c0 0x7f3dec102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.850+0000 7f3df2d90640 1 --2- 172.21.15.7:0/3813548958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1016d0 0x7f3dec101af0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.850+0000 7f3df2d90640 1 -- 172.21.15.7:0/3813548958 >> 172.21.15.7:0/3813548958 conn(0x7f3dec0fcc60 msgr2=0x7f3dec0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.850+0000 7f3df2d90640 1 -- 172.21.15.7:0/3813548958 shutdown_connections 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.850+0000 7f3df2d90640 1 -- 172.21.15.7:0/3813548958 wait complete. 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df2d90640 1 Processor -- start 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df2d90640 1 -- start start 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df2d90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec1016d0 0x7f3dec075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df2d90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1028c0 0x7f3dec073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df2d90640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dec103280 0x7f3dec073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df2d90640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec10d350 con 0x7f3dec1028c0 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df2d90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec10d1d0 con 0x7f3dec1016d0 2024-03-26T08:42:15.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df2d90640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec10d4d0 con 0x7f3dec103280 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3debfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1028c0 0x7f3dec073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3debfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1028c0 0x7f3dec073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53660/0 (socket says 172.21.15.7:53660) 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3debfff640 1 -- 172.21.15.7:0/201223405 learned_addr learned my addr 172.21.15.7:0/201223405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df132d640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dec103280 0x7f3dec073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec1016d0 0x7f3dec075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3debfff640 1 -- 172.21.15.7:0/201223405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dec103280 msgr2=0x7f3dec073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3debfff640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dec103280 0x7f3dec073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3debfff640 1 -- 172.21.15.7:0/201223405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec1016d0 msgr2=0x7f3dec075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3debfff640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec1016d0 0x7f3dec075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.851+0000 7f3debfff640 1 -- 172.21.15.7:0/201223405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec074860 con 0x7f3dec1028c0 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.852+0000 7f3debfff640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1028c0 0x7f3dec073a90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800e9f0 tx=0x7f3dd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.852+0000 7f3de9ffb640 1 -- 172.21.15.7:0/201223405 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800cd80 con 0x7f3dec1028c0 2024-03-26T08:42:15.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.852+0000 7f3df2d90640 1 -- 172.21.15.7:0/201223405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dec1a1b30 con 0x7f3dec1028c0 2024-03-26T08:42:15.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.852+0000 7f3de9ffb640 1 -- 172.21.15.7:0/201223405 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3dd8010910 con 0x7f3dec1028c0 2024-03-26T08:42:15.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.852+0000 7f3de9ffb640 1 -- 172.21.15.7:0/201223405 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8018950 con 0x7f3dec1028c0 2024-03-26T08:42:15.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.852+0000 7f3df2d90640 1 -- 172.21.15.7:0/201223405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dec1a20d0 con 0x7f3dec1028c0 2024-03-26T08:42:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.854+0000 7f3df5980740 1 Processor -- start 2024-03-26T08:42:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df5980740 1 -- start start 2024-03-26T08:42:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3de9ffb640 1 -- 172.21.15.7:0/201223405 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f3dd8010ac0 con 0x7f3dec1028c0 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df5980740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df5980740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d0790 0x55ad9f4d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df5980740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4d7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df5980740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9f4353d0 con 0x55ad9f4d5900 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df5980740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9f3ddad0 con 0x55ad9f4d0790 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df5980740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9f431470 con 0x55ad9f4cfda0 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3debfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4d7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df0b2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d0790 0x55ad9f4d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df132d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df0b2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d0790 0x55ad9f4d5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36498/0 (socket says 172.21.15.7:36498) 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df132d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33462/0 (socket says 172.21.15.7:33462) 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3debfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4d7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53670/0 (socket says 172.21.15.7:53670) 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df0b2c640 1 -- 172.21.15.7:0/3226341571 learned_addr learned my addr 172.21.15.7:0/3226341571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df0b2c640 1 -- 172.21.15.7:0/3226341571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4cfda0 msgr2=0x55ad9f4d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/3226341571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df0b2c640 1 -- 172.21.15.7:0/3226341571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 msgr2=0x55ad9f4d7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/3226341571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4d7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df0b2c640 1 -- 172.21.15.7:0/3226341571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad9f2137d0 con 0x55ad9f4d0790 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3df132d640 1 --2- 172.21.15.7:0/3226341571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.855+0000 7f3de9ffb640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc0a2910 0x7f3ddc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/3226341571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d0790 0x55ad9f4d5290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de0009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc7abf074b3fe3c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df132d640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc0a2910 0x7f3ddc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3de9ffb640 1 -- 172.21.15.7:0/201223405 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3dd80ceef0 con 0x7f3dec1028c0 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3dd6ffd640 1 -- 172.21.15.7:0/3226341571 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de002cb80 con 0x55ad9f4d0790 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3dd6ffd640 1 -- 172.21.15.7:0/3226341571 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de0005040 con 0x55ad9f4d0790 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3dd6ffd640 1 -- 172.21.15.7:0/3226341571 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de00048c0 con 0x55ad9f4d0790 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df5980740 1 -- 172.21.15.7:0/3226341571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d0790 msgr2=0x55ad9f4d5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df5980740 1 --2- 172.21.15.7:0/3226341571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d0790 0x55ad9f4d5290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de0009560 comp rx=0 tx=0).stop 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df132d640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc0a2910 0x7f3ddc0a4dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55ad9f4d0ff0 tx=0x7f3de4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df5980740 1 -- 172.21.15.7:0/3226341571 shutdown_connections 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df5980740 1 --2- 172.21.15.7:0/3226341571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4d7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df5980740 1 --2- 172.21.15.7:0/3226341571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d0790 0x55ad9f4d5290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df5980740 1 --2- 172.21.15.7:0/3226341571 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.856+0000 7f3df5980740 1 -- 172.21.15.7:0/3226341571 >> 172.21.15.7:0/3226341571 conn(0x55ad9f4cb310 msgr2=0x55ad9f4cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 -- 172.21.15.7:0/3226341571 shutdown_connections 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 -- 172.21.15.7:0/3226341571 wait complete. 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 -- start start 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 Processor -- start 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4da730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4e1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9f4d8850 con 0x55ad9f4d5900 2024-03-26T08:42:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9f3f1060 con 0x55ad9f4cfda0 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad9f4d89d0 con 0x55ad9f4d0790 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df5980740 10 client.0 _ll_register_callbacks cb 0x55ad9f3d36a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df0b2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3debfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4da730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df0b2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4da1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36508/0 (socket says 172.21.15.7:36508) 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.857+0000 7f3df0b2c640 1 -- 172.21.15.7:0/2099052354 learned_addr learned my addr 172.21.15.7:0/2099052354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3df132d640 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4e1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3debfff640 1 -- 172.21.15.7:0/2099052354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 msgr2=0x55ad9f4da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3debfff640 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3debfff640 1 -- 172.21.15.7:0/2099052354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 msgr2=0x55ad9f4e1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3debfff640 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4e1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3debfff640 1 -- 172.21.15.7:0/2099052354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad9f1493f0 con 0x55ad9f4d0790 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3debfff640 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4da730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80cc8a0 tx=0x7f3dd80d2610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8385a29f697842 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3dcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3dcbfff640 1 -- 172.21.15.7:0/2099052354 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd809b5b0 con 0x55ad9f4d0790 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3dcbfff640 1 -- 172.21.15.7:0/2099052354 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3dd80a3a40 con 0x55ad9f4d0790 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3df5980740 1 -- 172.21.15.7:0/2099052354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3de0005040 con 0x55ad9f4d0790 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3dcbfff640 1 -- 172.21.15.7:0/2099052354 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd809b140 con 0x55ad9f4d0790 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3df5980740 10 client.25291 fetch_fsmap learned FSMap version 23 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.858+0000 7f3df5980740 1 -- 172.21.15.7:0/2099052354 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ad9f1090f0 con 0x55ad9f4d0790 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3dcbfff640 1 -- 172.21.15.7:0/2099052354 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de0005040 con 0x55ad9f4d0790 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3dcbfff640 1 -- 172.21.15.7:0/2099052354 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 23) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3dd809ac30 con 0x55ad9f4d0790 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3df5980740 10 client.25291 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3df5980740 10 client.25291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3df5980740 20 client.25291 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3df5980740 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ad9f4e5ba0 0x55ad9f4e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3df5980740 4 client.25291 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3df5980740 1 -- 172.21.15.7:0/2099052354 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad9f442910 con 0x55ad9f4e5ba0 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ad9f4e5ba0 0x55ad9f4e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ad9f4e5ba0 0x55ad9f4e8040 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242cb33ed4a573f8 server_cookie=59920c7d209d3741 in_seq=0 out_seq=0 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.859+0000 7f3dcbfff640 10 client.25291 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3dcbfff640 1 -- 172.21.15.7:0/2099052354 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ad9f442910 con 0x55ad9f4e5ba0 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3dcbfff640 10 client.25291 handle_command_reply: tid=0 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 client.25291 shutdown 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 -- 172.21.15.7:0/2099052354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 msgr2=0x55ad9f4da730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4da730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80cc8a0 tx=0x7f3dd80d2610 comp rx=0 tx=0).stop 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 -- 172.21.15.7:0/2099052354 shutdown_connections 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ad9f4e5ba0 0x55ad9f4e8040 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4e1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4da730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3dcbfff640 0 client.25291 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 --2- 172.21.15.7:0/2099052354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 -- 172.21.15.7:0/2099052354 >> 172.21.15.7:0/2099052354 conn(0x55ad9f4cb310 msgr2=0x55ad9f4ca530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 -- 172.21.15.7:0/2099052354 shutdown_connections 2024-03-26T08:42:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 1 -- 172.21.15.7:0/2099052354 wait complete. 2024-03-26T08:42:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.860+0000 7f3df5980740 20 client.25291 trim_cache size 0 max 16384 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.876+0000 7f3df5980740 1 Processor -- start 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df5980740 1 -- start start 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df5980740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df5980740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4e1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df5980740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d5900 0x55ad9f4ec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df5980740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec1ad1e0 con 0x55ad9f4cfda0 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df5980740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec1aace0 con 0x55ad9f4d5900 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df5980740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec1a8d60 con 0x55ad9f4d0790 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df0b2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3debfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df0b2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53690/0 (socket says 172.21.15.7:53690) 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df0b2c640 1 -- 172.21.15.7:0/3586165482 learned_addr learned my addr 172.21.15.7:0/3586165482 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df132d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d5900 0x55ad9f4ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3debfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4e1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33480/0 (socket says 172.21.15.7:33480) 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df0b2c640 1 -- 172.21.15.7:0/3586165482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 msgr2=0x55ad9f4e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/3586165482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df0b2c640 1 -- 172.21.15.7:0/3586165482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d5900 msgr2=0x55ad9f4ec900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/3586165482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d5900 0x55ad9f4ec900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df0b2c640 1 -- 172.21.15.7:0/3586165482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad9f4d0b70 con 0x55ad9f4cfda0 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/3586165482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4cec80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3de0044000 tx=0x7f3de002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f8d3bc9c47f33b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3dd67fc640 1 -- 172.21.15.7:0/3586165482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de004a040 con 0x55ad9f4cfda0 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3dd67fc640 1 -- 172.21.15.7:0/3586165482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de002cc90 con 0x55ad9f4cfda0 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.877+0000 7f3dd67fc640 1 -- 172.21.15.7:0/3586165482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de004f4b0 con 0x55ad9f4cfda0 2024-03-26T08:42:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 -- 172.21.15.7:0/3586165482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4cfda0 msgr2=0x55ad9f4cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 --2- 172.21.15.7:0/3586165482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4cec80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3de0044000 tx=0x7f3de002c660 comp rx=0 tx=0).stop 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 -- 172.21.15.7:0/3586165482 shutdown_connections 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 --2- 172.21.15.7:0/3586165482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4d5900 0x55ad9f4ec900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 --2- 172.21.15.7:0/3586165482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 --2- 172.21.15.7:0/3586165482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4cec80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 -- 172.21.15.7:0/3586165482 >> 172.21.15.7:0/3586165482 conn(0x55ad9f4cb310 msgr2=0x55ad9f4ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 -- 172.21.15.7:0/3586165482 shutdown_connections 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 -- 172.21.15.7:0/3586165482 wait complete. 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.878+0000 7f3df5980740 1 -- start start 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df5980740 1 Processor -- start 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df5980740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4df710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df5980740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df5980740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4e0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df5980740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec1a9360 con 0x55ad9f4d5900 2024-03-26T08:42:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df5980740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec1aa680 con 0x55ad9f4cfda0 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df5980740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec1a99c0 con 0x55ad9f4d0790 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df5980740 10 client.0 _ll_register_callbacks cb 0x7f3dd800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df132d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4e0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df0b2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4df710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3debfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df132d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4e0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53692/0 (socket says 172.21.15.7:53692) 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3df132d640 1 -- 172.21.15.7:0/915846071 learned_addr learned my addr 172.21.15.7:0/915846071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3debfff640 1 -- 172.21.15.7:0/915846071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 msgr2=0x55ad9f4df710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3debfff640 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4df710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3debfff640 1 -- 172.21.15.7:0/915846071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 msgr2=0x55ad9f4e0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3debfff640 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4e0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.879+0000 7f3debfff640 1 -- 172.21.15.7:0/915846071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad9f4e1200 con 0x55ad9f4d0790 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3debfff640 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4dfc50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8097c40 tx=0x7f3dd800c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f187793ca436cadf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3dcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3dcbfff640 1 -- 172.21.15.7:0/915846071 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd809b320 con 0x55ad9f4d0790 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3dcbfff640 1 -- 172.21.15.7:0/915846071 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3dd8010430 con 0x55ad9f4d0790 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df5980740 1 -- 172.21.15.7:0/915846071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3de002cc90 con 0x55ad9f4d0790 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3dcbfff640 1 -- 172.21.15.7:0/915846071 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd809b5b0 con 0x55ad9f4d0790 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df5980740 10 client.25297 fetch_fsmap learned FSMap version 23 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df5980740 1 -- 172.21.15.7:0/915846071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ad9f541df0 con 0x55ad9f4d0790 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3dcbfff640 1 -- 172.21.15.7:0/915846071 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3de002cc90 con 0x55ad9f4d0790 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3dcbfff640 1 -- 172.21.15.7:0/915846071 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 23) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3dd8097530 con 0x55ad9f4d0790 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df5980740 10 client.25297 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df5980740 10 client.25297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df5980740 20 client.25297 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df5980740 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ad9f542680 0x55ad9f544b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df5980740 4 client.25297 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:42:15.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df5980740 1 -- 172.21.15.7:0/915846071 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ad9f43cc60 con 0x55ad9f542680 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.880+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ad9f542680 0x55ad9f544b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3dcbfff640 10 client.25297 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df0b2c640 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ad9f542680 0x55ad9f544b00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6b40232cd41c36 server_cookie=9004d33119797e6f in_seq=0 out_seq=0 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3dcbfff640 1 -- 172.21.15.7:0/915846071 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ad9f43cc60 con 0x55ad9f542680 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3dcbfff640 10 client.25297 handle_command_reply: tid=0 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 client.25297 shutdown 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 -- 172.21.15.7:0/915846071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 msgr2=0x55ad9f4dfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4dfc50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8097c40 tx=0x7f3dd800c840 comp rx=0 tx=0).stop 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 -- 172.21.15.7:0/915846071 shutdown_connections 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ad9f542680 0x55ad9f544b00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad9f4d5900 0x55ad9f4e0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3dcbfff640 0 client.25297 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ad9f4d0790 0x55ad9f4dfc50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 --2- 172.21.15.7:0/915846071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ad9f4cfda0 0x55ad9f4df710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 -- 172.21.15.7:0/915846071 >> 172.21.15.7:0/915846071 conn(0x55ad9f4cb310 msgr2=0x55ad9f4cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 -- 172.21.15.7:0/915846071 shutdown_connections 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 1 -- 172.21.15.7:0/915846071 wait complete. 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.881+0000 7f3df5980740 20 client.25297 trim_cache size 0 max 16384 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "c5dd5958-69eb-472f-8b93-3c7eb95b5edb": { 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "c5dd5958-69eb-472f-8b93-3c7eb95b5edb", 2024-03-26T08:42:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:42:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:42:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:42:15.881 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:42:15.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 -- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc0a2910 msgr2=0x7f3ddc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc0a2910 0x7f3ddc0a4dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55ad9f4d0ff0 tx=0x7f3de4009500 comp rx=0 tx=0).stop 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 -- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1028c0 msgr2=0x7f3dec073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1028c0 0x7f3dec073a90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800e9f0 tx=0x7f3dd800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 -- 172.21.15.7:0/201223405 shutdown_connections 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3ddc0a2910 0x7f3ddc0a4dd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dec103280 0x7f3dec073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec1028c0 0x7f3dec073a90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 --2- 172.21.15.7:0/201223405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dec1016d0 0x7f3dec075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 -- 172.21.15.7:0/201223405 >> 172.21.15.7:0/201223405 conn(0x7f3dec0fcc60 msgr2=0x7f3dec105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 -- 172.21.15.7:0/201223405 shutdown_connections 2024-03-26T08:42:15.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:15.882+0000 7f3df2d90640 1 -- 172.21.15.7:0/201223405 wait complete. 2024-03-26T08:42:15.885 INFO:tasks.cephfs.filesystem:scrub status for tag:c5dd5958-69eb-472f-8b93-3c7eb95b5edb - {'path': '/', 'tag': 'c5dd5958-69eb-472f-8b93-3c7eb95b5edb', 'options': 'recursive,force'} 2024-03-26T08:42:16.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:16 smithi007 ceph-mon[32462]: pgmap v355: 97 pgs: 97 active+clean; 651 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-26T08:42:16.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:16 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3697890873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:16.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:16 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:42:16.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:16 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:16.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:16 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:42:16.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:16 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:16 smithi044 ceph-mon[36461]: pgmap v355: 97 pgs: 97 active+clean; 651 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-26T08:42:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:16 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3697890873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:16 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:42:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:16 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:16 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:42:16.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:16 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:16.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:16 smithi016 ceph-mon[35851]: pgmap v355: 97 pgs: 97 active+clean; 651 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-26T08:42:16.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:16 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3697890873' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:42:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:16 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:42:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:16 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:16 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:42:16.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:16 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:17.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-03-26T08:42:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-03-26T08:42:17.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-03-26T08:42:17.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-26T08:42:17.649 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/: 2024-03-26T08:42:17.649 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:17.649 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:42:17.649 INFO:teuthology.orchestra.run.smithi007.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 .. 2024-03-26T08:42:17.649 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 data 2024-03-26T08:42:17.649 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:17.649 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data: 2024-03-26T08:42:17.649 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:17.650 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:17.650 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 .. 2024-03-26T08:42:17.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-03-26T08:42:17.743 INFO:teuthology.orchestra.run.smithi007.stdout:[Service] 2024-03-26T08:42:17.743 INFO:teuthology.orchestra.run.smithi007.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-03-26T08:42:17.743 INFO:teuthology.orchestra.run.smithi007.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-03-26T08:42:17.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-03-26T08:42:17.825 INFO:teuthology.orchestra.run.smithi007.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-03-26T08:42:18.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:18 smithi007 ceph-mon[32462]: pgmap v356: 97 pgs: 97 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-26T08:42:18.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:18 smithi044 ceph-mon[36461]: pgmap v356: 97 pgs: 97 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-26T08:42:18.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:18 smithi016 ceph-mon[35851]: pgmap v356: 97 pgs: 97 active+clean; 658 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-03-26T08:42:20.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:20 smithi007 ceph-mon[32462]: pgmap v357: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-26T08:42:20.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:20 smithi044 ceph-mon[36461]: pgmap v357: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-26T08:42:20.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:20 smithi016 ceph-mon[35851]: pgmap v357: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-03-26T08:42:22.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:22 smithi007 ceph-mon[32462]: pgmap v358: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-03-26T08:42:22.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:22 smithi007 ceph-mon[32462]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:42:22.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:22 smithi044 ceph-mon[36461]: pgmap v358: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-03-26T08:42:22.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:22 smithi044 ceph-mon[36461]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:42:22.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:22 smithi016 ceph-mon[35851]: pgmap v358: 97 pgs: 97 active+clean; 659 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-03-26T08:42:22.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:22 smithi016 ceph-mon[35851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:42:24.214 INFO:teuthology.orchestra.run.smithi007.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-03-26T08:42:24.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/: 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 .. 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 data 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data: 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:total 35 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 . 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 .. 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 base 2024-03-26T08:42:24.259 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 26 08:42 global 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 log 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_commit_ts 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_dynshmem 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 26 08:42 pg_hba.conf 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 26 08:42 pg_ident.conf 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 pg_logical 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 pg_multixact 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_notify 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_replslot 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_serial 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_snapshots 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_stat 2024-03-26T08:42:24.260 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_stat_tmp 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 pg_subtrans 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_tblspc 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_twophase 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 PG_VERSION 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 pg_wal 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 pg_xact 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 26 08:42 postgresql.auto.conf 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 26 08:42 postgresql.conf 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base: 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 . 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 1 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 13493 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 13494 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 . 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 .. 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 112 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 113 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 1247 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1247_fsm 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1247_vm 2024-03-26T08:42:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 08:42 1249 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1249_fsm 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1249_vm 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 08:42 1255 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1255_fsm 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1255_vm 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 08:42 1259 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1259_fsm 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1259_vm 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 08:42 13344 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13344_fsm 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13344_vm 2024-03-26T08:42:24.263 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13346 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13348 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13349_fsm 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349_vm 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13351 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13353 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13354_fsm 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354_vm 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13356 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13358 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359 2024-03-26T08:42:24.264 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13359_fsm 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359_vm 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13361 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13363 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1417 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1418 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 174 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 175 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2187 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2224 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2328 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2336 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2337 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2579 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2600 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2600_fsm 2024-03-26T08:42:24.265 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2600_vm 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2601_fsm 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601_vm 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2602 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2602_fsm 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2602_vm 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2603 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2603_fsm 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2603_vm 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2604 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2605 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2605_fsm 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2605_vm 2024-03-26T08:42:24.266 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2606_fsm 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606_vm 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2607 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2607_fsm 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2607_vm 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 08:42 2608 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2608_fsm 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2608_vm 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 08:42 2609 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2609_fsm 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2609_vm 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2610 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2610_fsm 2024-03-26T08:42:24.267 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2610_vm 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2611 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2612_fsm 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612_vm 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2613 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2615_fsm 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615_vm 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2616 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2616_fsm 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2616_vm 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 08:42 2617 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2617_fsm 2024-03-26T08:42:24.268 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2617_vm 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 08:42 2618 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2618_fsm 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2618_vm 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 08:42 2619 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2619_fsm 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2619_vm 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2620 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2650 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2651 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2652 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2653 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2654 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2655 2024-03-26T08:42:24.269 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2656 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2657 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 08:42 2658 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 2659 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2660 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2661 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2662 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2663 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2664 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2665 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2666 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2667 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2668 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2669 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2670 2024-03-26T08:42:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 08:42 2673 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 08:42 2674 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 08:42 2675 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2678 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2679 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2680 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2681 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2682 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2683 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2684 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2685 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2686 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2687 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2688 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2689 2024-03-26T08:42:24.271 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 08:42 2690 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 08:42 2691 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2692 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2693 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2696 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2699 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2701 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2702 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2703 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2704 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2753 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2753_fsm 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2753_vm 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2754 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2755 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2756 2024-03-26T08:42:24.272 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2757 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2830 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2831 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2832 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2833 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2834 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2835 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2836 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2837 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 08:42 2838 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2838_fsm 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2838_vm 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2839 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840 2024-03-26T08:42:24.273 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840_fsm 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2840_vm 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2841 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2995 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2996 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3079_fsm 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079_vm 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3080 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3081 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3085 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3118 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3119 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 08:42 3164 2024-03-26T08:42:24.274 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3256 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3257 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3258 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3350 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3351 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3379 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3380 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3381 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3394 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3394_fsm 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3394_vm 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3395 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3429 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3430 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3431 2024-03-26T08:42:24.275 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3433 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3439 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3440 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3455 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 08:42 3456 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3456_fsm 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3456_vm 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3466 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3467 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3468 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3501 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3502 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3503 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3534 2024-03-26T08:42:24.276 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3541_fsm 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541_vm 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3542 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3574 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3575 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3576 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3596 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3597 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3598 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3599 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3600_fsm 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600_vm 2024-03-26T08:42:24.277 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3601_fsm 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601_vm 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3602_fsm 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602_vm 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603_fsm 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3603_vm 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3604 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3605 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3606 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3607 2024-03-26T08:42:24.278 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3608 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 3609 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3712 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3764_fsm 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764_vm 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3766 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3767 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3997 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4143 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4144 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4145 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4146 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4147 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4148 2024-03-26T08:42:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4149 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4150 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4151 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4152 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4153 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4154 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4155 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4156 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4157 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4158 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4159 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4160 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4161 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4162 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4163 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4164 2024-03-26T08:42:24.280 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4165 2024-03-26T08:42:24.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4166 2024-03-26T08:42:24.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4167 2024-03-26T08:42:24.283 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4168 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4169 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4170 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4171 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4172 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4173 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4174 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 5002 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 548 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 549 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6102 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6104 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6106 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6110 2024-03-26T08:42:24.284 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6111 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6112 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6113 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6117 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 826 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 827 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 828 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 08:42 pg_filenode.map 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 PG_VERSION 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 . 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 .. 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 112 2024-03-26T08:42:24.285 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 113 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 1247 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1247_fsm 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1247_vm 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 08:42 1249 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1249_fsm 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1249_vm 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 08:42 1255 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1255_fsm 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1255_vm 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 08:42 1259 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1259_fsm 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1259_vm 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 08:42 13344 2024-03-26T08:42:24.286 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13344_fsm 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13344_vm 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13346 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13348 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13349_fsm 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349_vm 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13351 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13353 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13354_fsm 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354_vm 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13356 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13358 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359 2024-03-26T08:42:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13359_fsm 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359_vm 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13361 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13363 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1417 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1418 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 174 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 175 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2187 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2224 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2328 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2336 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2337 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2579 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2600 2024-03-26T08:42:24.288 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2600_fsm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2600_vm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2601_fsm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601_vm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2602 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2602_fsm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2602_vm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2603 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2603_fsm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2603_vm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2604 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2605 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2605_fsm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2605_vm 2024-03-26T08:42:24.289 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2606_fsm 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606_vm 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2607 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2607_fsm 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2607_vm 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 08:42 2608 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2608_fsm 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2608_vm 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 08:42 2609 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2609_fsm 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2609_vm 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2610 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2610_fsm 2024-03-26T08:42:24.290 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2610_vm 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2611 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2612_fsm 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612_vm 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2613 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2615_fsm 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615_vm 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2616 2024-03-26T08:42:24.291 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2616_fsm 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2616_vm 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 08:42 2617 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2617_fsm 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2617_vm 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 08:42 2618 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2618_fsm 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2618_vm 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 08:42 2619 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2619_fsm 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2619_vm 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2620 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2650 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2651 2024-03-26T08:42:24.292 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2652 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2653 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2654 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2655 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2656 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2657 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 08:42 2658 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 2659 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2660 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2661 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2662 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2663 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2664 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2665 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2666 2024-03-26T08:42:24.293 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2667 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2668 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2669 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2670 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 08:42 2673 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 08:42 2674 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 08:42 2675 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2678 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2679 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2680 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2681 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2682 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2683 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2684 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2685 2024-03-26T08:42:24.294 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2686 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2687 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2688 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2689 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 08:42 2690 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 08:42 2691 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2692 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2693 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2696 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2699 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2701 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2702 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2703 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2704 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2753 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2753_fsm 2024-03-26T08:42:24.295 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2753_vm 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2754 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2755 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2756 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2757 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2830 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2831 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2832 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2833 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2834 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2835 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2836 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2837 2024-03-26T08:42:24.296 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 08:42 2838 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2838_fsm 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2838_vm 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2839 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840_fsm 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2840_vm 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2841 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2995 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2996 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3079_fsm 2024-03-26T08:42:24.297 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079_vm 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3080 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3081 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3085 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3118 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3119 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 08:42 3164 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3256 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3257 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3258 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3350 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3351 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3379 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3380 2024-03-26T08:42:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3381 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3394 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3394_fsm 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3394_vm 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3395 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3429 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3430 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3431 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3433 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3439 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3440 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3455 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 08:42 3456 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3456_fsm 2024-03-26T08:42:24.299 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3456_vm 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3466 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3467 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3468 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3501 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3502 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3503 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3534 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3541_fsm 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541_vm 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3542 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3574 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3575 2024-03-26T08:42:24.300 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3576 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3596 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3597 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3598 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3599 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3600_fsm 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600_vm 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3601_fsm 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601_vm 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3602_fsm 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602_vm 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603 2024-03-26T08:42:24.301 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603_fsm 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3603_vm 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3604 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3605 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3606 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3607 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3608 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 3609 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3712 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3764_fsm 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764_vm 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3766 2024-03-26T08:42:24.302 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3767 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3997 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4143 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4144 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4145 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4146 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4147 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4148 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4149 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4150 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4151 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4152 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4153 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4154 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4155 2024-03-26T08:42:24.303 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4156 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4157 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4158 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4159 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4160 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4161 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4162 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4163 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4164 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4165 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4166 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4167 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4168 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4169 2024-03-26T08:42:24.304 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4170 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4171 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4172 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4173 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4174 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 5002 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 548 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 549 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6102 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6104 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6106 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6110 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6111 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6112 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6113 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6117 2024-03-26T08:42:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 826 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 827 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 828 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 08:42 pg_filenode.map 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 PG_VERSION 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 . 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 .. 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 112 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 113 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 1247 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1247_fsm 2024-03-26T08:42:24.306 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1247_vm 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 08:42 1249 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1249_fsm 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1249_vm 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 08:42 1255 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1255_fsm 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1255_vm 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 08:42 1259 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1259_fsm 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1259_vm 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 08:42 13344 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13344_fsm 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13344_vm 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13346 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13348 2024-03-26T08:42:24.307 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13349_fsm 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349_vm 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13351 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13353 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13354_fsm 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354_vm 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13356 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13358 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13359_fsm 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359_vm 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13361 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13363 2024-03-26T08:42:24.308 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1417 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1418 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 174 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 175 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2187 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2224 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2328 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2336 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2337 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2579 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2600 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2600_fsm 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2600_vm 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601 2024-03-26T08:42:24.309 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2601_fsm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601_vm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2602 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2602_fsm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2602_vm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2603 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2603_fsm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2603_vm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2604 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2605 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2605_fsm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2605_vm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2606_fsm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606_vm 2024-03-26T08:42:24.310 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2607 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2607_fsm 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2607_vm 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 08:42 2608 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2608_fsm 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2608_vm 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 08:42 2609 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2609_fsm 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2609_vm 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2610 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2610_fsm 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2610_vm 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2611 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2612_fsm 2024-03-26T08:42:24.311 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612_vm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2613 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2615_fsm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615_vm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2616 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2616_fsm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2616_vm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 08:42 2617 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2617_fsm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2617_vm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 08:42 2618 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2618_fsm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2618_vm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 08:42 2619 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2619_fsm 2024-03-26T08:42:24.312 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2619_vm 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2620 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2650 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2651 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2652 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2653 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2654 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2655 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2656 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2657 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 08:42 2658 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 2659 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2660 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2661 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2662 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2663 2024-03-26T08:42:24.313 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2664 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2665 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2666 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2667 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2668 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2669 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2670 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 08:42 2673 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 08:42 2674 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 08:42 2675 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2678 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2679 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2680 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2681 2024-03-26T08:42:24.314 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2682 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2683 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2684 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2685 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2686 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2687 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2688 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2689 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 08:42 2690 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 08:42 2691 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2692 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2693 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2696 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2699 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2701 2024-03-26T08:42:24.315 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2702 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2703 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2704 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2753 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2753_fsm 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2753_vm 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2754 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2755 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2756 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2757 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2830 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2831 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2832 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2833 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2834 2024-03-26T08:42:24.316 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2835 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2836 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2837 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 08:42 2838 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2838_fsm 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2838_vm 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2839 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840_fsm 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2840_vm 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2841 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2995 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2996 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3079_fsm 2024-03-26T08:42:24.317 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079_vm 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3080 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3081 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3085 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3118 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3119 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 08:42 3164 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3256 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3257 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3258 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3350 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3351 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3379 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3380 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3381 2024-03-26T08:42:24.318 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3394 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3394_fsm 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3394_vm 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3395 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3429 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3430 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3431 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3433 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3439 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3440 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3455 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 08:42 3456 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3456_fsm 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3456_vm 2024-03-26T08:42:24.319 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3466 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3467 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3468 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3501 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3502 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3503 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3534 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3541_fsm 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541_vm 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3542 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3574 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3575 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3576 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3596 2024-03-26T08:42:24.320 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3597 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3598 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3599 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3600_fsm 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600_vm 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3601_fsm 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601_vm 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3602_fsm 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602_vm 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603_fsm 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3603_vm 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3604 2024-03-26T08:42:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3605 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3606 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3607 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3608 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 3609 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3712 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3764_fsm 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764_vm 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3766 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3767 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3997 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4143 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4144 2024-03-26T08:42:24.322 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4145 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4146 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4147 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4148 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4149 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4150 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4151 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4152 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4153 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4154 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4155 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4156 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4157 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4158 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4159 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4160 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4161 2024-03-26T08:42:24.323 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4162 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4163 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4164 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4165 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4166 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4167 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4168 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4169 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4170 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4171 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4172 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4173 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4174 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 5002 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 548 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 549 2024-03-26T08:42:24.324 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6102 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6104 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6106 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6110 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6111 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6112 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6113 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6117 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 826 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 827 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 828 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 08:42 pg_filenode.map 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 PG_VERSION 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/global: 2024-03-26T08:42:24.325 INFO:teuthology.orchestra.run.smithi007.stdout:total 537 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Mar 26 08:42 . 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1213 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1213_fsm 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1213_vm 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1214 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1214_fsm 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1214_vm 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 1232 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 1233 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1260 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1260_fsm 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1260_vm 2024-03-26T08:42:24.326 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1261 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1261_fsm 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1261_vm 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1262 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1262_fsm 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1262_vm 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2396 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2396_fsm 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2396_vm 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2397 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2671 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2672 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2676 2024-03-26T08:42:24.327 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2677 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2694 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2695 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2697 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2698 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2846 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2847 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2964 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2965 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2966 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2967 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3592 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3593 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4060 2024-03-26T08:42:24.328 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4061 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4175 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4176 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4177 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4178 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4181 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4182 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4183 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4184 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4185 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4186 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6000 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6001 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6002 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6100 2024-03-26T08:42:24.329 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6114 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6115 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 pg_control 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 08:42 pg_filenode.map 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/log: 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout:total 1 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 . 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.331 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 mappings 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 26 08:42 replorigin_checkpoint 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 snapshots 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 .. 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 .. 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-26T08:42:24.332 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 . 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 members 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 offsets 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 .. 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 0000 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-03-26T08:42:24.333 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 .. 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 0000 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-26T08:42:24.334 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.335 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 0000 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:total 16384 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 . 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:42 000000010000000000000001 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 archive_status 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:42:24.337 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:42:24.338 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 .. 2024-03-26T08:42:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:24.338 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-26T08:42:24.338 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-03-26T08:42:24.338 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:42:24.338 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Mar 26 08:42 .. 2024-03-26T08:42:24.338 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 0000 2024-03-26T08:42:24.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-03-26T08:42:24.486 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:24 smithi007 ceph-mon[32462]: pgmap v359: 97 pgs: 97 active+clean; 17 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-03-26T08:42:24.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-03-26T08:42:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:dropping old tables... 2024-03-26T08:42:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-03-26T08:42:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-03-26T08:42:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-03-26T08:42:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-03-26T08:42:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:creating tables... 2024-03-26T08:42:24.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:24 smithi044 ceph-mon[36461]: pgmap v359: 97 pgs: 97 active+clean; 17 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-03-26T08:42:24.581 INFO:teuthology.orchestra.run.smithi007.stderr:generating data (client-side)... 2024-03-26T08:42:24.669 INFO:teuthology.orchestra.run.smithi007.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.75 s) 2024-03-26T08:42:24.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:24 smithi016 ceph-mon[35851]: pgmap v359: 97 pgs: 97 active+clean; 17 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-03-26T08:42:24.800 INFO:teuthology.orchestra.run.smithi007.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.19 s, remaining 2.81 s) 2024-03-26T08:42:24.969 INFO:teuthology.orchestra.run.smithi007.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.36 s, remaining 3.44 s) 2024-03-26T08:42:25.023 INFO:teuthology.orchestra.run.smithi007.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.41 s, remaining 2.87 s) 2024-03-26T08:42:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.57 s, remaining 3.09 s) 2024-03-26T08:42:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.72 s, remaining 3.13 s) 2024-03-26T08:42:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.80 s, remaining 2.87 s) 2024-03-26T08:42:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.96 s, remaining 2.89 s) 2024-03-26T08:42:25.666 INFO:teuthology.orchestra.run.smithi007.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.05 s, remaining 2.69 s) 2024-03-26T08:42:25.814 INFO:teuthology.orchestra.run.smithi007.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.20 s, remaining 2.64 s) 2024-03-26T08:42:25.971 INFO:teuthology.orchestra.run.smithi007.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.36 s, remaining 2.59 s) 2024-03-26T08:42:26.067 INFO:teuthology.orchestra.run.smithi007.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.45 s, remaining 2.42 s) 2024-03-26T08:42:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.58 s, remaining 2.31 s) 2024-03-26T08:42:26.372 INFO:teuthology.orchestra.run.smithi007.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.76 s, remaining 2.26 s) 2024-03-26T08:42:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.85 s, remaining 2.10 s) 2024-03-26T08:42:26.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:26 smithi007 ceph-mon[32462]: pgmap v360: 97 pgs: 97 active+clean; 26 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.2 MiB/s wr, 85 op/s 2024-03-26T08:42:26.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:26 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:42:26.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:26 smithi044 ceph-mon[36461]: pgmap v360: 97 pgs: 97 active+clean; 26 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.2 MiB/s wr, 85 op/s 2024-03-26T08:42:26.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:26 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:42:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:1600000 of 3200000 tuples (50%) done (elapsed 1.98 s, remaining 1.98 s) 2024-03-26T08:42:26.686 INFO:teuthology.orchestra.run.smithi007.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.07 s, remaining 1.83 s) 2024-03-26T08:42:26.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:26 smithi016 ceph-mon[35851]: pgmap v360: 97 pgs: 97 active+clean; 26 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.2 MiB/s wr, 85 op/s 2024-03-26T08:42:26.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:26 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:42:26.843 INFO:teuthology.orchestra.run.smithi007.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.23 s, remaining 1.73 s) 2024-03-26T08:42:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.36 s, remaining 1.62 s) 2024-03-26T08:42:27.066 INFO:teuthology.orchestra.run.smithi007.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.45 s, remaining 1.47 s) 2024-03-26T08:42:27.210 INFO:teuthology.orchestra.run.smithi007.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.60 s, remaining 1.36 s) 2024-03-26T08:42:27.389 INFO:teuthology.orchestra.run.smithi007.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.78 s, remaining 1.26 s) 2024-03-26T08:42:27.458 INFO:teuthology.orchestra.run.smithi007.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.85 s, remaining 1.11 s) 2024-03-26T08:42:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.00 s, remaining 1.00 s) 2024-03-26T08:42:27.770 INFO:teuthology.orchestra.run.smithi007.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.16 s, remaining 0.88 s) 2024-03-26T08:42:27.847 INFO:teuthology.orchestra.run.smithi007.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.23 s, remaining 0.75 s) 2024-03-26T08:42:27.996 INFO:teuthology.orchestra.run.smithi007.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.38 s, remaining 0.63 s) 2024-03-26T08:42:28.084 INFO:teuthology.orchestra.run.smithi007.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.47 s, remaining 0.50 s) 2024-03-26T08:42:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.63 s, remaining 0.38 s) 2024-03-26T08:42:28.387 INFO:teuthology.orchestra.run.smithi007.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.77 s, remaining 0.25 s) 2024-03-26T08:42:28.479 INFO:teuthology.orchestra.run.smithi007.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.87 s, remaining 0.12 s) 2024-03-26T08:42:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:28 smithi007 ceph-mon[32462]: pgmap v361: 97 pgs: 97 active+clean; 62 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 7.6 MiB/s wr, 231 op/s 2024-03-26T08:42:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:28 smithi044 ceph-mon[36461]: pgmap v361: 97 pgs: 97 active+clean; 62 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 7.6 MiB/s wr, 231 op/s 2024-03-26T08:42:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.01 s, remaining 0.00 s) 2024-03-26T08:42:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:vacuuming... 2024-03-26T08:42:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 -- 172.21.15.7:0/572298170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c1021d0 msgr2=0x7fa48c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 --2- 172.21.15.7:0/572298170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c1021d0 0x7fa48c1045c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009900 tx=0x7fa47c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 -- 172.21.15.7:0/572298170 shutdown_connections 2024-03-26T08:42:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 --2- 172.21.15.7:0/572298170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48c107430 0x7fa48c109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 --2- 172.21.15.7:0/572298170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48c104b00 0x7fa48c106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 --2- 172.21.15.7:0/572298170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c1021d0 0x7fa48c1045c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 -- 172.21.15.7:0/572298170 >> 172.21.15.7:0/572298170 conn(0x7fa48c0fbb40 msgr2=0x7fa48c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 -- 172.21.15.7:0/572298170 shutdown_connections 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 -- 172.21.15.7:0/572298170 wait complete. 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.678+0000 7fa490e36640 1 Processor -- start 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa490e36640 1 -- start start 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa490e36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48c1021d0 0x7fa48c101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa490e36640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48c104b00 0x7fa48c100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa490e36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c107430 0x7fa48c1006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa490e36640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c10d370 con 0x7fa48c107430 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa490e36640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c10d1f0 con 0x7fa48c1021d0 2024-03-26T08:42:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa490e36640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c10d4f0 con 0x7fa48c104b00 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c107430 0x7fa48c1006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48c1021d0 0x7fa48c101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa481d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48c104b00 0x7fa48c100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c107430 0x7fa48c1006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47622/0 (socket says 172.21.15.7:47622) 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa481d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48c104b00 0x7fa48c100180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44908/0 (socket says 172.21.15.7:44908) 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48ad9d640 1 -- 172.21.15.7:0/2912956961 learned_addr learned my addr 172.21.15.7:0/2912956961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48c1021d0 0x7fa48c101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53236/0 (socket says 172.21.15.7:53236) 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48ad9d640 1 -- 172.21.15.7:0/2912956961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48c104b00 msgr2=0x7fa48c100180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48ad9d640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48c104b00 0x7fa48c100180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48ad9d640 1 -- 172.21.15.7:0/2912956961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48c1021d0 msgr2=0x7fa48c101b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48ad9d640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48c1021d0 0x7fa48c101b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48ad9d640 1 -- 172.21.15.7:0/2912956961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c100f50 con 0x7fa48c107430 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48a59c640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48c1021d0 0x7fa48c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.679+0000 7fa48ad9d640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c107430 0x7fa48c1006c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa48400eb10 tx=0x7fa48400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.680+0000 7fa483fff640 1 -- 172.21.15.7:0/2912956961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484004b20 con 0x7fa48c107430 2024-03-26T08:42:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.680+0000 7fa483fff640 1 -- 172.21.15.7:0/2912956961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa484004cc0 con 0x7fa48c107430 2024-03-26T08:42:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.680+0000 7fa490e36640 1 -- 172.21.15.7:0/2912956961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c1a1b30 con 0x7fa48c107430 2024-03-26T08:42:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.680+0000 7fa483fff640 1 -- 172.21.15.7:0/2912956961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa484010760 con 0x7fa48c107430 2024-03-26T08:42:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.680+0000 7fa490e36640 1 -- 172.21.15.7:0/2912956961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c1a2080 con 0x7fa48c107430 2024-03-26T08:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.682+0000 7fa483fff640 1 -- 172.21.15.7:0/2912956961 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fa484005020 con 0x7fa48c107430 2024-03-26T08:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.682+0000 7fa493a1d740 1 Processor -- start 2024-03-26T08:42:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.682+0000 7fa493a1d740 1 -- start start 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e43d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e44310 0x558185e48e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e49480 0x558185e4b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa493a1d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558185da8b10 con 0x558185e44310 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa493a1d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558185d51900 con 0x558185e49480 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa493a1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558185da4bb0 con 0x558185e43920 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa48a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e44310 0x558185e48e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa483fff640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4780a2880 0x7fa4780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa481d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e43d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa48a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e44310 0x558185e48e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47626/0 (socket says 172.21.15.7:47626) 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa48a59c640 1 -- 172.21.15.7:0/2970771938 learned_addr learned my addr 172.21.15.7:0/2970771938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa481d9b640 1 --2- 172.21.15.7:0/2970771938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e49480 0x558185e4b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa48ad9d640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4780a2880 0x7fa4780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa481d9b640 1 -- 172.21.15.7:0/2970771938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e49480 msgr2=0x558185e4b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa481d9b640 1 --2- 172.21.15.7:0/2970771938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e49480 0x558185e4b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa481d9b640 1 -- 172.21.15.7:0/2970771938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e44310 msgr2=0x558185e48e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa481d9b640 1 --2- 172.21.15.7:0/2970771938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e44310 0x558185e48e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa481d9b640 1 -- 172.21.15.7:0/2970771938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558185abd210 con 0x558185e43920 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa48a59c640 1 --2- 172.21.15.7:0/2970771938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e44310 0x558185e48e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa483fff640 1 -- 172.21.15.7:0/2912956961 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa4840cf330 con 0x7fa48c107430 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa481d9b640 1 --2- 172.21.15.7:0/2970771938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e43d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4740097e0 tx=0x7fa474009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d80facfa7a16074 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa48ad9d640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4780a2880 0x7fa4780a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa484021000 tx=0x7fa484009aa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.683+0000 7fa480d99640 1 -- 172.21.15.7:0/2970771938 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474004030 con 0x558185e43920 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa480d99640 1 -- 172.21.15.7:0/2970771938 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa474005500 con 0x558185e43920 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa480d99640 1 -- 172.21.15.7:0/2970771938 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474004ba0 con 0x558185e43920 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa493a1d740 1 -- 172.21.15.7:0/2970771938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 msgr2=0x558185e43d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa493a1d740 1 --2- 172.21.15.7:0/2970771938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e43d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4740097e0 tx=0x7fa474009770 comp rx=0 tx=0).stop 2024-03-26T08:42:28.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa493a1d740 1 -- 172.21.15.7:0/2970771938 shutdown_connections 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa493a1d740 1 --2- 172.21.15.7:0/2970771938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e49480 0x558185e4b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa493a1d740 1 --2- 172.21.15.7:0/2970771938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e44310 0x558185e48e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa493a1d740 1 --2- 172.21.15.7:0/2970771938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e43d40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa493a1d740 1 -- 172.21.15.7:0/2970771938 >> 172.21.15.7:0/2970771938 conn(0x558185e3ee90 msgr2=0x558185e3f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa493a1d740 1 -- 172.21.15.7:0/2970771938 shutdown_connections 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.684+0000 7fa493a1d740 1 -- 172.21.15.7:0/2970771938 wait complete. 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa493a1d740 1 -- start start 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa493a1d740 1 Processor -- start 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e43920 0x558185e4dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e4e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e49480 0x558185e551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa493a1d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558185e4c430 con 0x558185e43920 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa493a1d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558185d64d50 con 0x558185e44310 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa493a1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558185e4c5b0 con 0x558185e49480 2024-03-26T08:42:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa493a1d740 10 client.0 _ll_register_callbacks cb 0x558185d64720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa481d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e43920 0x558185e4dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa48a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e4e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa48a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e4e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53252/0 (socket says 172.21.15.7:53252) 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa48a59c640 1 -- 172.21.15.7:0/147735559 learned_addr learned my addr 172.21.15.7:0/147735559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa481d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e43920 0x558185e4dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47638/0 (socket says 172.21.15.7:47638) 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa481d9b640 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e49480 0x558185e551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa48a59c640 1 -- 172.21.15.7:0/147735559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e49480 msgr2=0x558185e551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa48a59c640 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e49480 0x558185e551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa48a59c640 1 -- 172.21.15.7:0/147735559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e43920 msgr2=0x558185e4dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa48a59c640 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e43920 0x558185e4dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa48a59c640 1 -- 172.21.15.7:0/147735559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558185b89620 con 0x558185e44310 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa48a59c640 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e4e1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa47c02c7b0 tx=0x7fa47c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc29e921966d3c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.685+0000 7fa471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:42:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa471ffb640 1 -- 172.21.15.7:0/147735559 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c005d50 con 0x558185e44310 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa493a1d740 1 -- 172.21.15.7:0/147735559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa474005500 con 0x558185e44310 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa471ffb640 1 -- 172.21.15.7:0/147735559 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa47c0046c0 con 0x558185e44310 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa471ffb640 1 -- 172.21.15.7:0/147735559 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa47c0048c0 con 0x558185e44310 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa493a1d740 10 client.25292 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa493a1d740 1 -- 172.21.15.7:0/147735559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558185e585c0 con 0x558185e44310 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa471ffb640 1 -- 172.21.15.7:0/147735559 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa474005500 con 0x558185e44310 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa471ffb640 1 -- 172.21.15.7:0/147735559 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa47c003b60 con 0x558185e44310 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa493a1d740 10 client.25292 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa493a1d740 10 client.25292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa493a1d740 20 client.25292 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa493a1d740 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558185e59bc0 0x558185e5bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa493a1d740 4 client.25292 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa493a1d740 1 -- 172.21.15.7:0/147735559 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558185db6050 con 0x558185e59bc0 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.686+0000 7fa481d9b640 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558185e59bc0 0x558185e5bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.687+0000 7fa471ffb640 10 client.25292 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.687+0000 7fa481d9b640 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558185e59bc0 0x558185e5bff0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3326938986feb0 server_cookie=69581546065aac89 in_seq=0 out_seq=0 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa471ffb640 1 -- 172.21.15.7:0/147735559 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558185db6050 con 0x558185e59bc0 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa471ffb640 10 client.25292 handle_command_reply: tid=0 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 client.25292 shutdown 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 -- 172.21.15.7:0/147735559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 msgr2=0x558185e4e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e4e1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa47c02c7b0 tx=0x7fa47c005270 comp rx=0 tx=0).stop 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 -- 172.21.15.7:0/147735559 shutdown_connections 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558185e59bc0 0x558185e5bff0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e49480 0x558185e551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e4e1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa471ffb640 0 client.25292 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 --2- 172.21.15.7:0/147735559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e43920 0x558185e4dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 -- 172.21.15.7:0/147735559 >> 172.21.15.7:0/147735559 conn(0x558185e3ee90 msgr2=0x558185e3e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 -- 172.21.15.7:0/147735559 shutdown_connections 2024-03-26T08:42:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 1 -- 172.21.15.7:0/147735559 wait complete. 2024-03-26T08:42:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.688+0000 7fa493a1d740 20 client.25292 trim_cache size 0 max 16384 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.702+0000 7fa493a1d740 1 Processor -- start 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.702+0000 7fa493a1d740 1 -- start start 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.702+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.702+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.702+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e42630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.702+0000 7fa493a1d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1ad330 con 0x558185e3ee90 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.702+0000 7fa493a1d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1aae30 con 0x558185e44310 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.702+0000 7fa493a1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a8eb0 con 0x558185e43920 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa481d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa48a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa481d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47640/0 (socket says 172.21.15.7:47640) 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa481d9b640 1 -- 172.21.15.7:0/2369115342 learned_addr learned my addr 172.21.15.7:0/2369115342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa48a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44932/0 (socket says 172.21.15.7:44932) 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa481d9b640 1 --2- 172.21.15.7:0/2369115342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e42630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa48a59c640 1 -- 172.21.15.7:0/2369115342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 msgr2=0x558185e42630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa48a59c640 1 --2- 172.21.15.7:0/2369115342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e42630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa48a59c640 1 -- 172.21.15.7:0/2369115342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 msgr2=0x558185e420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa48a59c640 1 --2- 172.21.15.7:0/2369115342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa48a59c640 1 -- 172.21.15.7:0/2369115342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558185e488e0 con 0x558185e43920 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa481d9b640 1 --2- 172.21.15.7:0/2369115342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa48a59c640 1 --2- 172.21.15.7:0/2369115342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e551a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a20d082f82ffd61 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa473fff640 1 -- 172.21.15.7:0/2369115342 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c005d50 con 0x558185e43920 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa473fff640 1 -- 172.21.15.7:0/2369115342 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa47c055020 con 0x558185e43920 2024-03-26T08:42:28.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.703+0000 7fa473fff640 1 -- 172.21.15.7:0/2369115342 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c046920 con 0x558185e43920 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 -- 172.21.15.7:0/2369115342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 msgr2=0x558185e551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 --2- 172.21.15.7:0/2369115342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e551a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 -- 172.21.15.7:0/2369115342 shutdown_connections 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 --2- 172.21.15.7:0/2369115342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e42630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 --2- 172.21.15.7:0/2369115342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e551a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 --2- 172.21.15.7:0/2369115342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 -- 172.21.15.7:0/2369115342 >> 172.21.15.7:0/2369115342 conn(0x558185e44f10 msgr2=0x558185e3f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 -- 172.21.15.7:0/2369115342 shutdown_connections 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 -- 172.21.15.7:0/2369115342 wait complete. 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 -- start start 2024-03-26T08:42:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.704+0000 7fa493a1d740 1 Processor -- start 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e53ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e53ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa493a1d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e54530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa493a1d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a94b0 con 0x558185e3ee90 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa493a1d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1aa7d0 con 0x558185e44310 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa493a1d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a9b10 con 0x558185e43920 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa493a1d740 10 client.0 _ll_register_callbacks cb 0x7fa47c004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e53ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e53ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47646/0 (socket says 172.21.15.7:47646) 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 -- 172.21.15.7:0/3308100013 learned_addr learned my addr 172.21.15.7:0/3308100013 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa48a59c640 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e53ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e54530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 -- 172.21.15.7:0/3308100013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 msgr2=0x558185e53ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e53ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 -- 172.21.15.7:0/3308100013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 msgr2=0x558185e54530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e54530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 -- 172.21.15.7:0/3308100013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558185eb55c0 con 0x558185e3ee90 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa481d9b640 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e53ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa4740059f0 tx=0x7fa474021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3765e0e6a589f3e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.705+0000 7fa471ffb640 1 -- 172.21.15.7:0/3308100013 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474011050 con 0x558185e3ee90 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa493a1d740 1 -- 172.21.15.7:0/3308100013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa47c055020 con 0x558185e3ee90 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa471ffb640 1 -- 172.21.15.7:0/3308100013 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa474028040 con 0x558185e3ee90 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa471ffb640 1 -- 172.21.15.7:0/3308100013 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa474043050 con 0x558185e3ee90 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa493a1d740 10 client.15486 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa493a1d740 1 -- 172.21.15.7:0/3308100013 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558185eb6930 con 0x558185e3ee90 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa471ffb640 1 -- 172.21.15.7:0/3308100013 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa47c055020 con 0x558185e3ee90 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa471ffb640 1 -- 172.21.15.7:0/3308100013 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa474005370 con 0x558185e3ee90 2024-03-26T08:42:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa493a1d740 10 client.15486 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa493a1d740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa493a1d740 20 client.15486 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa493a1d740 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558185e49480 0x558185eb9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa493a1d740 4 client.15486 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa493a1d740 1 -- 172.21.15.7:0/3308100013 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558185db03a0 con 0x558185e49480 2024-03-26T08:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.706+0000 7fa48a59c640 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558185e49480 0x558185eb9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.707+0000 7fa48a59c640 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558185e49480 0x558185eb9160 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f784be40f9be11d1 server_cookie=30637a99647eb1ce in_seq=0 out_seq=0 2024-03-26T08:42:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.707+0000 7fa471ffb640 10 client.15486 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:28.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:28 smithi016 ceph-mon[35851]: pgmap v361: 97 pgs: 97 active+clean; 62 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 518 KiB/s rd, 7.6 MiB/s wr, 231 op/s 2024-03-26T08:42:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa471ffb640 1 -- 172.21.15.7:0/3308100013 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558185db03a0 con 0x558185e49480 2024-03-26T08:42:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa471ffb640 10 client.15486 handle_command_reply: tid=0 2024-03-26T08:42:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 client.15486 shutdown 2024-03-26T08:42:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 -- 172.21.15.7:0/3308100013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 msgr2=0x558185e53ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e53ab0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa4740059f0 tx=0x7fa474021040 comp rx=0 tx=0).stop 2024-03-26T08:42:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 -- 172.21.15.7:0/3308100013 shutdown_connections 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558185e49480 0x558185eb9160 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558185e44310 0x558185e54530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558185e43920 0x558185e53ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa471ffb640 0 client.15486 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 --2- 172.21.15.7:0/3308100013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558185e3ee90 0x558185e53ab0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 -- 172.21.15.7:0/3308100013 >> 172.21.15.7:0/3308100013 conn(0x558185e44f10 msgr2=0x558185e420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 -- 172.21.15.7:0/3308100013 shutdown_connections 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 1 -- 172.21.15.7:0/3308100013 wait complete. 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.945+0000 7fa493a1d740 20 client.15486 trim_cache size 0 max 16384 2024-03-26T08:42:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:42:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:42:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:42:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:42:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 -- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4780a2880 msgr2=0x7fa4780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4780a2880 0x7fa4780a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa484021000 tx=0x7fa484009aa0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 -- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c107430 msgr2=0x7fa48c1006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c107430 0x7fa48c1006c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa48400eb10 tx=0x7fa48400c2d0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 -- 172.21.15.7:0/2912956961 shutdown_connections 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4780a2880 0x7fa4780a4d40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c107430 0x7fa48c1006c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa48c104b00 0x7fa48c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 --2- 172.21.15.7:0/2912956961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa48c1021d0 0x7fa48c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 -- 172.21.15.7:0/2912956961 >> 172.21.15.7:0/2912956961 conn(0x7fa48c0fbb40 msgr2=0x7fa48c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 -- 172.21.15.7:0/2912956961 shutdown_connections 2024-03-26T08:42:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:28.946+0000 7fa490e36640 1 -- 172.21.15.7:0/2912956961 wait complete. 2024-03-26T08:42:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:creating primary keys... 2024-03-26T08:42:29.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:29 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:42:29.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:29 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:42:29.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:29 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:42:30.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:30 smithi016 ceph-mon[35851]: pgmap v362: 97 pgs: 97 active+clean; 132 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 19 MiB/s wr, 250 op/s 2024-03-26T08:42:30.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:30 smithi044 ceph-mon[36461]: pgmap v362: 97 pgs: 97 active+clean; 132 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 19 MiB/s wr, 250 op/s 2024-03-26T08:42:30.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:30 smithi007 ceph-mon[32462]: pgmap v362: 97 pgs: 97 active+clean; 132 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 19 MiB/s wr, 250 op/s 2024-03-26T08:42:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:done in 6.35 s (drop tables 0.00 s, create tables 0.03 s, client-side generate 4.09 s, vacuum 0.87 s, primary keys 1.36 s). 2024-03-26T08:42:30.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-03-26T08:42:31.070 INFO:teuthology.orchestra.run.smithi007.stderr:starting vacuum...end. 2024-03-26T08:42:32.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:31 smithi007 ceph-mon[32462]: pgmap v363: 97 pgs: 97 active+clean; 278 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 43 MiB/s wr, 300 op/s 2024-03-26T08:42:32.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:31 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.065 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:31 smithi044 ceph-mon[36461]: pgmap v363: 97 pgs: 97 active+clean; 278 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 43 MiB/s wr, 300 op/s 2024-03-26T08:42:32.065 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.065 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:31 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:31 smithi016 ceph-mon[35851]: pgmap v363: 97 pgs: 97 active+clean; 278 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 43 MiB/s wr, 300 op/s 2024-03-26T08:42:32.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:32.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:31 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:33.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:42:33.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:42:33.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:33.065 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:42:33.065 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:42:33.065 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:33.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:42:33.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:42:33.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:42:34.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:33 smithi016 ceph-mon[35851]: pgmap v364: 97 pgs: 97 active+clean; 439 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 67 MiB/s wr, 433 op/s 2024-03-26T08:42:34.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:33 smithi007 ceph-mon[32462]: pgmap v364: 97 pgs: 97 active+clean; 439 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 67 MiB/s wr, 433 op/s 2024-03-26T08:42:34.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:33 smithi044 ceph-mon[36461]: pgmap v364: 97 pgs: 97 active+clean; 439 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 67 MiB/s wr, 433 op/s 2024-03-26T08:42:36.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 5.0 s, 4588.2 tps, lat 21.438 ms stddev 24.930 2024-03-26T08:42:36.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:35 smithi016 ceph-mon[35851]: pgmap v365: 97 pgs: 97 active+clean; 564 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 74 MiB/s wr, 436 op/s 2024-03-26T08:42:36.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:35 smithi007 ceph-mon[32462]: pgmap v365: 97 pgs: 97 active+clean; 564 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 74 MiB/s wr, 436 op/s 2024-03-26T08:42:36.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:35 smithi044 ceph-mon[36461]: pgmap v365: 97 pgs: 97 active+clean; 564 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 74 MiB/s wr, 436 op/s 2024-03-26T08:42:38.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:37 smithi016 ceph-mon[35851]: pgmap v366: 97 pgs: 97 active+clean; 709 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 75 MiB/s wr, 499 op/s 2024-03-26T08:42:38.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:37 smithi007 ceph-mon[32462]: pgmap v366: 97 pgs: 97 active+clean; 709 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 75 MiB/s wr, 499 op/s 2024-03-26T08:42:38.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:37 smithi044 ceph-mon[36461]: pgmap v366: 97 pgs: 97 active+clean; 709 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 75 MiB/s wr, 499 op/s 2024-03-26T08:42:40.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:39 smithi016 ceph-mon[35851]: pgmap v367: 97 pgs: 97 active+clean; 862 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 72 MiB/s wr, 544 op/s 2024-03-26T08:42:40.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:39 smithi007 ceph-mon[32462]: pgmap v367: 97 pgs: 97 active+clean; 862 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 72 MiB/s wr, 544 op/s 2024-03-26T08:42:40.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:39 smithi044 ceph-mon[36461]: pgmap v367: 97 pgs: 97 active+clean; 862 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 72 MiB/s wr, 544 op/s 2024-03-26T08:42:41.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 10.0 s, 4417.8 tps, lat 22.611 ms stddev 22.636 2024-03-26T08:42:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.072+0000 7f50cd148640 1 -- 172.21.15.7:0/1726539272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81017c0 msgr2=0x7f50c8101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.072+0000 7f50cd148640 1 --2- 172.21.15.7:0/1726539272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81017c0 0x7f50c8101c40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009900 tx=0x7f50b802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.072+0000 7f50cd148640 1 -- 172.21.15.7:0/1726539272 shutdown_connections 2024-03-26T08:42:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.072+0000 7f50cd148640 1 --2- 172.21.15.7:0/1726539272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50c8102180 0x7f50c8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.072+0000 7f50cd148640 1 --2- 172.21.15.7:0/1726539272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81017c0 0x7f50c8101c40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.072+0000 7f50cd148640 1 --2- 172.21.15.7:0/1726539272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c81005d0 0x7f50c81009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.072+0000 7f50cd148640 1 -- 172.21.15.7:0/1726539272 >> 172.21.15.7:0/1726539272 conn(0x7f50c80fbb40 msgr2=0x7f50c80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.072+0000 7f50cd148640 1 -- 172.21.15.7:0/1726539272 shutdown_connections 2024-03-26T08:42:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.072+0000 7f50cd148640 1 -- 172.21.15.7:0/1726539272 wait complete. 2024-03-26T08:42:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.073+0000 7f50cd148640 1 Processor -- start 2024-03-26T08:42:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.073+0000 7f50cd148640 1 -- start start 2024-03-26T08:42:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.073+0000 7f50cd148640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81005d0 0x7f50c8197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.073+0000 7f50cd148640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50c81017c0 0x7f50c8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.073+0000 7f50cd148640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c8102180 0x7f50c819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.073+0000 7f50cd148640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c810c2b0 con 0x7f50c81005d0 2024-03-26T08:42:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.073+0000 7f50cd148640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c810c130 con 0x7f50c8102180 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.073+0000 7f50cd148640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c810c430 con 0x7f50c81017c0 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81005d0 0x7f50c8197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c8102180 0x7f50c819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50c81017c0 0x7f50c8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c8102180 0x7f50c819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51472/0 (socket says 172.21.15.7:51472) 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c759e640 1 -- 172.21.15.7:0/2365432225 learned_addr learned my addr 172.21.15.7:0/2365432225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81005d0 0x7f50c8197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44806/0 (socket says 172.21.15.7:44806) 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c759e640 1 -- 172.21.15.7:0/2365432225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50c81017c0 msgr2=0x7f50c8197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c759e640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50c81017c0 0x7f50c8197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c759e640 1 -- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81005d0 msgr2=0x7f50c8197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c759e640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81005d0 0x7f50c8197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c759e640 1 -- 172.21.15.7:0/2365432225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c819f630 con 0x7f50c8102180 2024-03-26T08:42:42.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81005d0 0x7f50c8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50c759e640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c8102180 0x7f50c819efc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50c000eb10 tx=0x7f50c000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50affff640 1 -- 172.21.15.7:0/2365432225 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0004b20 con 0x7f50c8102180 2024-03-26T08:42:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50cd148640 1 -- 172.21.15.7:0/2365432225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c819f920 con 0x7f50c8102180 2024-03-26T08:42:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50affff640 1 -- 172.21.15.7:0/2365432225 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50c0004cc0 con 0x7f50c8102180 2024-03-26T08:42:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50affff640 1 -- 172.21.15.7:0/2365432225 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0010760 con 0x7f50c8102180 2024-03-26T08:42:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.074+0000 7f50cd148640 1 -- 172.21.15.7:0/2365432225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c819fe40 con 0x7f50c8102180 2024-03-26T08:42:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50cfe0b740 1 Processor -- start 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50affff640 1 -- 172.21.15.7:0/2365432225 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f50c0005020 con 0x7f50c8102180 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50cfe0b740 1 -- start start 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 0x55f22c04d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c052220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c052890 0x55f22c054d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22bfadb10 con 0x55f22c04d720 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22bf56900 con 0x55f22c04cd30 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22bfa9bb0 con 0x55f22c052890 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50c6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c052220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 0x55f22c04d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50c6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c052220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44812/0 (socket says 172.21.15.7:44812) 2024-03-26T08:42:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50c6d9d640 1 -- 172.21.15.7:0/1817021161 learned_addr learned my addr 172.21.15.7:0/1817021161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.078+0000 7f50c659c640 1 --2- 172.21.15.7:0/1817021161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c052890 0x55f22c054d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50affff640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f50b00a2880 0x7f50b00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50c6d9d640 1 -- 172.21.15.7:0/1817021161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c052890 msgr2=0x55f22c054d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/1817021161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c052890 0x55f22c054d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50c6d9d640 1 -- 172.21.15.7:0/1817021161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 msgr2=0x55f22c04d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/1817021161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 0x55f22c04d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50c6d9d640 1 -- 172.21.15.7:0/1817021161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f22bcc2210 con 0x55f22c04d720 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50affff640 1 -- 172.21.15.7:0/2365432225 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f50c00cf2d0 con 0x7f50c8102180 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50c759e640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f50b00a2880 0x7f50b00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/1817021161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c052220 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0114c0 tx=0x7f50bc011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdc6a7440343bc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50ad7fa640 1 -- 172.21.15.7:0/1817021161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc004020 con 0x55f22c04d720 2024-03-26T08:42:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50ad7fa640 1 -- 172.21.15.7:0/1817021161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50bc0041c0 con 0x55f22c04d720 2024-03-26T08:42:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50ad7fa640 1 -- 172.21.15.7:0/1817021161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00f940 con 0x55f22c04d720 2024-03-26T08:42:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.079+0000 7f50c759e640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f50b00a2880 0x7f50b00a4d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f50c009b810 tx=0x7f50c006d790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.080+0000 7f50cfe0b740 1 -- 172.21.15.7:0/1817021161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 msgr2=0x55f22c052220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.080+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/1817021161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c052220 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0114c0 tx=0x7f50bc011980 comp rx=0 tx=0).stop 2024-03-26T08:42:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.081+0000 7f50cfe0b740 1 -- 172.21.15.7:0/1817021161 shutdown_connections 2024-03-26T08:42:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.081+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/1817021161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c052890 0x55f22c054d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.081+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/1817021161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c052220 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.081+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/1817021161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 0x55f22c04d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.081+0000 7f50cfe0b740 1 -- 172.21.15.7:0/1817021161 >> 172.21.15.7:0/1817021161 conn(0x55f22c043e90 msgr2=0x55f22c044290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.082+0000 7f50cfe0b740 1 -- 172.21.15.7:0/1817021161 shutdown_connections 2024-03-26T08:42:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.082+0000 7f50cfe0b740 1 -- 172.21.15.7:0/1817021161 wait complete. 2024-03-26T08:42:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.082+0000 7f50cfe0b740 1 -- start start 2024-03-26T08:42:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.082+0000 7f50cfe0b740 1 Processor -- start 2024-03-26T08:42:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c057010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c057550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c05e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22c055840 con 0x55f22c04d720 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22bf69d50 con 0x55f22c052890 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22c0559c0 con 0x55f22c04cd30 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50cfe0b740 10 client.0 _ll_register_callbacks cb 0x55f22bf69720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c05e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c057550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c05e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51490/0 (socket says 172.21.15.7:51490) 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c659c640 1 -- 172.21.15.7:0/459827438 learned_addr learned my addr 172.21.15.7:0/459827438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c057550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44822/0 (socket says 172.21.15.7:44822) 2024-03-26T08:42:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c659c640 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c057010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c659c640 1 -- 172.21.15.7:0/459827438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 msgr2=0x55f22c057010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c659c640 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c057010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c659c640 1 -- 172.21.15.7:0/459827438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 msgr2=0x55f22c057550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c659c640 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c057550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.083+0000 7f50c659c640 1 -- 172.21.15.7:0/459827438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f22bd8e620 con 0x55f22c052890 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.084+0000 7f50c659c640 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c05e590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50b8000c00 tx=0x7f50b802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8272cf16752dc53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.084+0000 7f50aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.084+0000 7f50aaff5640 1 -- 172.21.15.7:0/459827438 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80048d0 con 0x55f22c052890 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.084+0000 7f50aaff5640 1 -- 172.21.15.7:0/459827438 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f50b8004a70 con 0x55f22c052890 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.084+0000 7f50cfe0b740 1 -- 172.21.15.7:0/459827438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50bc0041c0 con 0x55f22c052890 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.084+0000 7f50aaff5640 1 -- 172.21.15.7:0/459827438 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b8004c70 con 0x55f22c052890 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.084+0000 7f50cfe0b740 10 client.25310 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.084+0000 7f50cfe0b740 1 -- 172.21.15.7:0/459827438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f22c061970 con 0x55f22c052890 2024-03-26T08:42:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.085+0000 7f50aaff5640 1 -- 172.21.15.7:0/459827438 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50bc0041c0 con 0x55f22c052890 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.085+0000 7f50aaff5640 1 -- 172.21.15.7:0/459827438 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f50b80036f0 con 0x55f22c052890 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.085+0000 7f50cfe0b740 10 client.25310 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.085+0000 7f50cfe0b740 10 client.25310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.085+0000 7f50cfe0b740 20 client.25310 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.085+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f22c062fe0 0x55f22c0653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.085+0000 7f50cfe0b740 4 client.25310 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.085+0000 7f50cfe0b740 1 -- 172.21.15.7:0/459827438 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f22bfbb050 con 0x55f22c062fe0 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.085+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f22c062fe0 0x55f22c0653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.086+0000 7f50aaff5640 10 client.25310 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.086+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f22c062fe0 0x55f22c0653d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c0fe402be5ef1e server_cookie=13b4467af22da66b in_seq=0 out_seq=0 2024-03-26T08:42:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.087+0000 7f50aaff5640 1 -- 172.21.15.7:0/459827438 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f22bfbb050 con 0x55f22c062fe0 2024-03-26T08:42:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.087+0000 7f50aaff5640 10 client.25310 handle_command_reply: tid=0 2024-03-26T08:42:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.087+0000 7f50cfe0b740 1 client.25310 shutdown 2024-03-26T08:42:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.087+0000 7f50cfe0b740 1 -- 172.21.15.7:0/459827438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 msgr2=0x55f22c05e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c05e590 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f50b8000c00 tx=0x7f50b802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 1 -- 172.21.15.7:0/459827438 shutdown_connections 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f22c062fe0 0x55f22c0653d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c05e590 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50aaff5640 0 client.25310 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c057550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/459827438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c057010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 1 -- 172.21.15.7:0/459827438 >> 172.21.15.7:0/459827438 conn(0x55f22c043e90 msgr2=0x55f22c043020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 1 -- 172.21.15.7:0/459827438 shutdown_connections 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 1 -- 172.21.15.7:0/459827438 wait complete. 2024-03-26T08:42:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.088+0000 7f50cfe0b740 20 client.25310 trim_cache size 0 max 16384 2024-03-26T08:42:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.110+0000 7f50cfe0b740 1 Processor -- start 2024-03-26T08:42:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.110+0000 7f50cfe0b740 1 -- start start 2024-03-26T08:42:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c044270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c05e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c0470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c81a8ec0 con 0x55f22c04d720 2024-03-26T08:42:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c81a69c0 con 0x55f22c052890 2024-03-26T08:42:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c81a4a40 con 0x55f22c04cd30 2024-03-26T08:42:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c044270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c044270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36584/0 (socket says 172.21.15.7:36584) 2024-03-26T08:42:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c659c640 1 -- 172.21.15.7:0/4054563901 learned_addr learned my addr 172.21.15.7:0/4054563901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c659c640 1 --2- 172.21.15.7:0/4054563901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c0470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/4054563901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c05e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c659c640 1 -- 172.21.15.7:0/4054563901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 msgr2=0x55f22c0470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c659c640 1 --2- 172.21.15.7:0/4054563901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c0470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c659c640 1 -- 172.21.15.7:0/4054563901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 msgr2=0x55f22c05e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c659c640 1 --2- 172.21.15.7:0/4054563901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c05e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.111+0000 7f50c659c640 1 -- 172.21.15.7:0/4054563901 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f22c058190 con 0x55f22c04cd30 2024-03-26T08:42:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.112+0000 7f50c659c640 1 --2- 172.21.15.7:0/4054563901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c044270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f50b8005e70 tx=0x7f50b802c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bdecf53f282454c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.112+0000 7f50acff9640 1 -- 172.21.15.7:0/4054563901 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b802cd20 con 0x55f22c04cd30 2024-03-26T08:42:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.112+0000 7f50acff9640 1 -- 172.21.15.7:0/4054563901 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50b8061070 con 0x55f22c04cd30 2024-03-26T08:42:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.112+0000 7f50acff9640 1 -- 172.21.15.7:0/4054563901 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80055f0 con 0x55f22c04cd30 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 -- 172.21.15.7:0/4054563901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 msgr2=0x55f22c044270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/4054563901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c044270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f50b8005e70 tx=0x7f50b802c880 comp rx=0 tx=0).stop 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 -- 172.21.15.7:0/4054563901 shutdown_connections 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/4054563901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c052890 0x55f22c0470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/4054563901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c04d720 0x55f22c05e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/4054563901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04cd30 0x55f22c044270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 -- 172.21.15.7:0/4054563901 >> 172.21.15.7:0/4054563901 conn(0x55f22c043e90 msgr2=0x55f22c03eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 -- 172.21.15.7:0/4054563901 shutdown_connections 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 -- 172.21.15.7:0/4054563901 wait complete. 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 -- start start 2024-03-26T08:42:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 Processor -- start 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 0x55f22c0586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04d720 0x55f22c058c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c052890 0x55f22c05d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c81a5040 con 0x55f22c052890 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c81a6360 con 0x55f22c04cd30 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c81a56a0 con 0x55f22c04d720 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50cfe0b740 10 client.0 _ll_register_callbacks cb 0x7f50b8004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c052890 0x55f22c05d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c052890 0x55f22c05d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44842/0 (socket says 172.21.15.7:44842) 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50c659c640 1 -- 172.21.15.7:0/3688318882 learned_addr learned my addr 172.21.15.7:0/3688318882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04d720 0x55f22c058c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.113+0000 7f50c659c640 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 0x55f22c0586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.114+0000 7f50c6d9d640 1 -- 172.21.15.7:0/3688318882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 msgr2=0x55f22c0586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.114+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 0x55f22c0586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.114+0000 7f50c6d9d640 1 -- 172.21.15.7:0/3688318882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c052890 msgr2=0x55f22c05d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.114+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c052890 0x55f22c05d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.114+0000 7f50c6d9d640 1 -- 172.21.15.7:0/3688318882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f22c05dca0 con 0x55f22c04d720 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.114+0000 7f50c6d9d640 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04d720 0x55f22c058c10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55f22c044e00 tx=0x7f50bc005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e921ef8341630623 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.114+0000 7f50aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.115+0000 7f50aaff5640 1 -- 172.21.15.7:0/3688318882 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc037070 con 0x55f22c04d720 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.115+0000 7f50aaff5640 1 -- 172.21.15.7:0/3688318882 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f50bc02a160 con 0x55f22c04d720 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.115+0000 7f50cfe0b740 1 -- 172.21.15.7:0/3688318882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50b8061070 con 0x55f22c04d720 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.115+0000 7f50aaff5640 1 -- 172.21.15.7:0/3688318882 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50bc01d040 con 0x55f22c04d720 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.115+0000 7f50cfe0b740 10 client.25333 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50cfe0b740 1 -- 172.21.15.7:0/3688318882 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f22c0bad40 con 0x55f22c04d720 2024-03-26T08:42:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50aaff5640 1 -- 172.21.15.7:0/3688318882 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50b8061070 con 0x55f22c04d720 2024-03-26T08:42:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50aaff5640 1 -- 172.21.15.7:0/3688318882 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f50bc024040 con 0x55f22c04d720 2024-03-26T08:42:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50cfe0b740 10 client.25333 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50cfe0b740 10 client.25333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50cfe0b740 20 client.25333 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f22c0bb500 0x55f22c0bd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50cfe0b740 4 client.25333 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:42:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50cfe0b740 1 -- 172.21.15.7:0/3688318882 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f22bfb53a0 con 0x55f22c0bb500 2024-03-26T08:42:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.116+0000 7f50c659c640 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f22c0bb500 0x55f22c0bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.117+0000 7f50c659c640 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f22c0bb500 0x55f22c0bd920 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3468dc194ef164ee server_cookie=1456b3e3bf1e026c in_seq=0 out_seq=0 2024-03-26T08:42:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.117+0000 7f50aaff5640 10 client.25333 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.144+0000 7f50aaff5640 1 -- 172.21.15.7:0/3688318882 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f22bfb53a0 con 0x55f22c0bb500 2024-03-26T08:42:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50aaff5640 10 client.25333 handle_command_reply: tid=0 2024-03-26T08:42:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 client.25333 shutdown 2024-03-26T08:42:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 -- 172.21.15.7:0/3688318882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04d720 msgr2=0x55f22c058c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04d720 0x55f22c058c10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55f22c044e00 tx=0x7f50bc005db0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 -- 172.21.15.7:0/3688318882 shutdown_connections 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f22c0bb500 0x55f22c0bd920 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f22c052890 0x55f22c05d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f22c04d720 0x55f22c058c10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 --2- 172.21.15.7:0/3688318882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f22c04cd30 0x55f22c0586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 -- 172.21.15.7:0/3688318882 >> 172.21.15.7:0/3688318882 conn(0x55f22c043e90 msgr2=0x55f22c044270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50aaff5640 0 client.25333 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 -- 172.21.15.7:0/3688318882 shutdown_connections 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 1 -- 172.21.15.7:0/3688318882 wait complete. 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.145+0000 7f50cfe0b740 20 client.25333 trim_cache size 0 max 16384 2024-03-26T08:42:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:42:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:42:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:42:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 -- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f50b00a2880 msgr2=0x7f50b00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f50b00a2880 0x7f50b00a4d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f50c009b810 tx=0x7f50c006d790 comp rx=0 tx=0).stop 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 -- 172.21.15.7:0/2365432225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c8102180 msgr2=0x7f50c819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c8102180 0x7f50c819efc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50c000eb10 tx=0x7f50c000c2d0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 -- 172.21.15.7:0/2365432225 shutdown_connections 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f50b00a2880 0x7f50b00a4d40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c8102180 0x7f50c819efc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50c81017c0 0x7f50c8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 --2- 172.21.15.7:0/2365432225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c81005d0 0x7f50c8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 -- 172.21.15.7:0/2365432225 >> 172.21.15.7:0/2365432225 conn(0x7f50c80fbb40 msgr2=0x7f50c80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 -- 172.21.15.7:0/2365432225 shutdown_connections 2024-03-26T08:42:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:42.146+0000 7f50cd148640 1 -- 172.21.15.7:0/2365432225 wait complete. 2024-03-26T08:42:42.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:41 smithi016 ceph-mon[35851]: pgmap v368: 97 pgs: 97 active+clean; 902 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 63 MiB/s wr, 617 op/s 2024-03-26T08:42:42.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:41 smithi007 ceph-mon[32462]: pgmap v368: 97 pgs: 97 active+clean; 902 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 63 MiB/s wr, 617 op/s 2024-03-26T08:42:42.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:41 smithi044 ceph-mon[36461]: pgmap v368: 97 pgs: 97 active+clean; 902 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 63 MiB/s wr, 617 op/s 2024-03-26T08:42:44.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:43 smithi016 ceph-mon[35851]: pgmap v369: 97 pgs: 97 active+clean; 922 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 40 MiB/s wr, 713 op/s 2024-03-26T08:42:44.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:43 smithi007 ceph-mon[32462]: pgmap v369: 97 pgs: 97 active+clean; 922 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 40 MiB/s wr, 713 op/s 2024-03-26T08:42:44.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:43 smithi044 ceph-mon[36461]: pgmap v369: 97 pgs: 97 active+clean; 922 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 40 MiB/s wr, 713 op/s 2024-03-26T08:42:45.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:42:45.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:42:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.055+0000 7fa50b33b640 1 -- 172.21.15.7:0/3874205514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5041017b0 msgr2=0x7fa504101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.056+0000 7fa50b33b640 1 --2- 172.21.15.7:0/3874205514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5041017b0 0x7fa504101c30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009900 tx=0x7fa4f402bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.056+0000 7fa50b33b640 1 -- 172.21.15.7:0/3874205514 shutdown_connections 2024-03-26T08:42:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.056+0000 7fa50b33b640 1 --2- 172.21.15.7:0/3874205514 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504102170 0x7fa504108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.056+0000 7fa50b33b640 1 --2- 172.21.15.7:0/3874205514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5041017b0 0x7fa504101c30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.056+0000 7fa50b33b640 1 --2- 172.21.15.7:0/3874205514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5041005c0 0x7fa5041009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.056+0000 7fa50b33b640 1 -- 172.21.15.7:0/3874205514 >> 172.21.15.7:0/3874205514 conn(0x7fa5040fbb30 msgr2=0x7fa5040fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.056+0000 7fa50b33b640 1 -- 172.21.15.7:0/3874205514 shutdown_connections 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.057+0000 7fa50b33b640 1 -- 172.21.15.7:0/3874205514 wait complete. 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.057+0000 7fa50b33b640 1 Processor -- start 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa50b33b640 1 -- start start 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa50b33b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5041005c0 0x7fa504110ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa50b33b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5041017b0 0x7fa504111010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa50b33b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504102170 0x7fa5041ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa50b33b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50410b170 con 0x7fa5041005c0 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa50b33b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50410aff0 con 0x7fa5041017b0 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa50b33b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50410b2f0 con 0x7fa504102170 2024-03-26T08:42:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa5098d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504102170 0x7fa5041ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5041017b0 0x7fa504111010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5041017b0 0x7fa504111010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51526/0 (socket says 172.21.15.7:51526) 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa5088d6640 1 -- 172.21.15.7:0/787067917 learned_addr learned my addr 172.21.15.7:0/787067917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.058+0000 7fa5090d7640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5041005c0 0x7fa504110ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa5098d8640 1 -- 172.21.15.7:0/787067917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5041017b0 msgr2=0x7fa504111010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa5098d8640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5041017b0 0x7fa504111010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa5098d8640 1 -- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5041005c0 msgr2=0x7fa504110ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa5098d8640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5041005c0 0x7fa504110ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa5098d8640 1 -- 172.21.15.7:0/787067917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5041181b0 con 0x7fa504102170 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa5098d8640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504102170 0x7fa5041ae2c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009fd0 tx=0x7fa4fc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa4fa7fc640 1 -- 172.21.15.7:0/787067917 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00ce20 con 0x7fa504102170 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa50b33b640 1 -- 172.21.15.7:0/787067917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5041ae800 con 0x7fa504102170 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa4fa7fc640 1 -- 172.21.15.7:0/787067917 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4fc005020 con 0x7fa504102170 2024-03-26T08:42:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa4fa7fc640 1 -- 172.21.15.7:0/787067917 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc010720 con 0x7fa504102170 2024-03-26T08:42:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.059+0000 7fa50b33b640 1 -- 172.21.15.7:0/787067917 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5041aecc0 con 0x7fa504102170 2024-03-26T08:42:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.063+0000 7fa50e000740 1 Processor -- start 2024-03-26T08:42:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.064+0000 7fa4fa7fc640 1 -- 172.21.15.7:0/787067917 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fa4fc005540 con 0x7fa504102170 2024-03-26T08:42:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.064+0000 7fa50e000740 1 -- start start 2024-03-26T08:42:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.064+0000 7fa50e000740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd123da0 0x55e3bd1241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.064+0000 7fa50e000740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 0x55e3bd129290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.064+0000 7fa50e000740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd12bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.064+0000 7fa50e000740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bd0893d0 con 0x55e3bd124790 2024-03-26T08:42:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa50e000740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bd031ad0 con 0x55e3bd129900 2024-03-26T08:42:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa50e000740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bd085470 con 0x55e3bd123da0 2024-03-26T08:42:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa4fa7fc640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4f00a2950 0x7fa4f00a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd12bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd12bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51540/0 (socket says 172.21.15.7:51540) 2024-03-26T08:42:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 -- 172.21.15.7:0/1932057540 learned_addr learned my addr 172.21.15.7:0/1932057540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 --2- 172.21.15.7:0/1932057540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd123da0 0x55e3bd1241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5090d7640 1 --2- 172.21.15.7:0/1932057540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 0x55e3bd129290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa4fa7fc640 1 -- 172.21.15.7:0/787067917 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa4fc0cf240 con 0x7fa504102170 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 -- 172.21.15.7:0/1932057540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd123da0 msgr2=0x55e3bd1241c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 --2- 172.21.15.7:0/1932057540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd123da0 0x55e3bd1241c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 -- 172.21.15.7:0/1932057540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 msgr2=0x55e3bd129290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 --2- 172.21.15.7:0/1932057540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 0x55e3bd129290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 -- 172.21.15.7:0/1932057540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3bce677d0 con 0x55e3bd129900 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5098d8640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4f00a2950 0x7fa4f00a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.065+0000 7fa5088d6640 1 --2- 172.21.15.7:0/1932057540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd12bd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4f402c1c0 tx=0x7fa4f402c150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92964b9c482552a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.066+0000 7fa4ef7fe640 1 -- 172.21.15.7:0/1932057540 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f404a070 con 0x55e3bd129900 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.066+0000 7fa4ef7fe640 1 -- 172.21.15.7:0/1932057540 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4005050 con 0x55e3bd129900 2024-03-26T08:42:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.066+0000 7fa4ef7fe640 1 -- 172.21.15.7:0/1932057540 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40048e0 con 0x55e3bd129900 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa50e000740 1 -- 172.21.15.7:0/1932057540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 msgr2=0x55e3bd12bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa50e000740 1 --2- 172.21.15.7:0/1932057540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd12bd90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4f402c1c0 tx=0x7fa4f402c150 comp rx=0 tx=0).stop 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa5098d8640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4f00a2950 0x7fa4f00a4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc084fd0 tx=0x7fa4fc005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa50e000740 1 -- 172.21.15.7:0/1932057540 shutdown_connections 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa50e000740 1 --2- 172.21.15.7:0/1932057540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd12bd90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa50e000740 1 --2- 172.21.15.7:0/1932057540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 0x55e3bd129290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa50e000740 1 --2- 172.21.15.7:0/1932057540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd123da0 0x55e3bd1241c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa50e000740 1 -- 172.21.15.7:0/1932057540 >> 172.21.15.7:0/1932057540 conn(0x55e3bd11f310 msgr2=0x55e3bd11f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa50e000740 1 -- 172.21.15.7:0/1932057540 shutdown_connections 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.067+0000 7fa50e000740 1 -- 172.21.15.7:0/1932057540 wait complete. 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.068+0000 7fa50e000740 1 -- start start 2024-03-26T08:42:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.068+0000 7fa50e000740 1 Processor -- start 2024-03-26T08:42:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa50e000740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd123da0 0x55e3bd12e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa50e000740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 0x55e3bd12e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa50e000740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd135750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa50e000740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bd12c850 con 0x55e3bd124790 2024-03-26T08:42:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa50e000740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bd045060 con 0x55e3bd123da0 2024-03-26T08:42:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa50e000740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3bd12c9d0 con 0x55e3bd129900 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa50e000740 10 client.0 _ll_register_callbacks cb 0x55e3bd0276a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd135750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd123da0 0x55e3bd12e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd123da0 0x55e3bd12e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51548/0 (socket says 172.21.15.7:51548) 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.069+0000 7fa5088d6640 1 -- 172.21.15.7:0/3745699677 learned_addr learned my addr 172.21.15.7:0/3745699677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.070+0000 7fa5090d7640 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 0x55e3bd12e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.070+0000 7fa5088d6640 1 -- 172.21.15.7:0/3745699677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 msgr2=0x55e3bd135750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.070+0000 7fa5088d6640 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd135750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.070+0000 7fa5088d6640 1 -- 172.21.15.7:0/3745699677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 msgr2=0x55e3bd12e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.070+0000 7fa5088d6640 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 0x55e3bd12e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.070+0000 7fa5088d6640 1 -- 172.21.15.7:0/3745699677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3bcd9d3f0 con 0x55e3bd123da0 2024-03-26T08:42:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.070+0000 7fa5090d7640 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 0x55e3bd12e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:42:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.071+0000 7fa5088d6640 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd123da0 0x55e3bd12e1d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4006010 tx=0x7fa4f402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21aba522911f094f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.071+0000 7fa4ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:42:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.071+0000 7fa4ecff9640 1 -- 172.21.15.7:0/3745699677 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f404a070 con 0x55e3bd123da0 2024-03-26T08:42:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.071+0000 7fa4ecff9640 1 -- 172.21.15.7:0/3745699677 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa4f40048c0 con 0x55e3bd123da0 2024-03-26T08:42:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.071+0000 7fa50e000740 1 -- 172.21.15.7:0/3745699677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4f4005050 con 0x55e3bd123da0 2024-03-26T08:42:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.072+0000 7fa4ecff9640 1 -- 172.21.15.7:0/3745699677 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4f4004ac0 con 0x55e3bd123da0 2024-03-26T08:42:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.072+0000 7fa50e000740 10 client.25328 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.072+0000 7fa50e000740 1 -- 172.21.15.7:0/3745699677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e3bcd5d0f0 con 0x55e3bd123da0 2024-03-26T08:42:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.072+0000 7fa4ecff9640 1 -- 172.21.15.7:0/3745699677 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4005050 con 0x55e3bd123da0 2024-03-26T08:42:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.072+0000 7fa4ecff9640 1 -- 172.21.15.7:0/3745699677 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa4f4050040 con 0x55e3bd123da0 2024-03-26T08:42:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.072+0000 7fa50e000740 10 client.25328 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.072+0000 7fa50e000740 10 client.25328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.073+0000 7fa50e000740 20 client.25328 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.073+0000 7fa50e000740 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3bd1398b0 0x55e3bd13bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.073+0000 7fa50e000740 4 client.25328 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.073+0000 7fa50e000740 1 -- 172.21.15.7:0/3745699677 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3bd096910 con 0x55e3bd1398b0 2024-03-26T08:42:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.073+0000 7fa5090d7640 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3bd1398b0 0x55e3bd13bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.073+0000 7fa4ecff9640 10 client.25328 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.073+0000 7fa5090d7640 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3bd1398b0 0x55e3bd13bca0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8f676ca36318a5 server_cookie=ba32276e7cafbd9f in_seq=0 out_seq=0 2024-03-26T08:42:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.074+0000 7fa4ecff9640 1 -- 172.21.15.7:0/3745699677 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e3bd096910 con 0x55e3bd1398b0 2024-03-26T08:42:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa4ecff9640 10 client.25328 handle_command_reply: tid=0 2024-03-26T08:42:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa50e000740 1 client.25328 shutdown 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa50e000740 1 -- 172.21.15.7:0/3745699677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd123da0 msgr2=0x55e3bd12e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa50e000740 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd123da0 0x55e3bd12e1d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4006010 tx=0x7fa4f402bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa50e000740 1 -- 172.21.15.7:0/3745699677 shutdown_connections 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa50e000740 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3bd1398b0 0x55e3bd13bca0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa50e000740 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd135750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa50e000740 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd124790 0x55e3bd12e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa50e000740 1 --2- 172.21.15.7:0/3745699677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd123da0 0x55e3bd12e1d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.075+0000 7fa50e000740 1 -- 172.21.15.7:0/3745699677 >> 172.21.15.7:0/3745699677 conn(0x55e3bd11f310 msgr2=0x55e3bd11e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.076+0000 7fa4ecff9640 0 client.25328 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.076+0000 7fa50e000740 1 -- 172.21.15.7:0/3745699677 shutdown_connections 2024-03-26T08:42:46.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.076+0000 7fa50e000740 1 -- 172.21.15.7:0/3745699677 wait complete. 2024-03-26T08:42:46.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.076+0000 7fa50e000740 20 client.25328 trim_cache size 0 max 16384 2024-03-26T08:42:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 15.0 s, 4393.2 tps, lat 22.752 ms stddev 22.940 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.100+0000 7fa50e000740 1 Processor -- start 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.100+0000 7fa50e000740 1 -- start start 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa50e000740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd12e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa50e000740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd124790 0x55e3bd1356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa50e000740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd122c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa50e000740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041b9ef0 con 0x55e3bd123da0 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa50e000740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041b79f0 con 0x55e3bd129900 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa50e000740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041b5a70 con 0x55e3bd124790 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd12e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd12e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44874/0 (socket says 172.21.15.7:44874) 2024-03-26T08:42:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 -- 172.21.15.7:0/516629407 learned_addr learned my addr 172.21.15.7:0/516629407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5090d7640 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd124790 0x55e3bd1356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd122c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 -- 172.21.15.7:0/516629407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd124790 msgr2=0x55e3bd1356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd124790 0x55e3bd1356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 -- 172.21.15.7:0/516629407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 msgr2=0x55e3bd122c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd122c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 -- 172.21.15.7:0/516629407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3bd11fd80 con 0x55e3bd123da0 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5090d7640 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd124790 0x55e3bd1356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.101+0000 7fa5088d6640 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd12e670 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40045c0 tx=0x7fa4f4043740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cf88021499c46f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa4eeffd640 1 -- 172.21.15.7:0/516629407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f404a070 con 0x55e3bd123da0 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa4eeffd640 1 -- 172.21.15.7:0/516629407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4f404b040 con 0x55e3bd123da0 2024-03-26T08:42:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa4eeffd640 1 -- 172.21.15.7:0/516629407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4043c50 con 0x55e3bd123da0 2024-03-26T08:42:46.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa50e000740 1 -- 172.21.15.7:0/516629407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 msgr2=0x55e3bd12e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa50e000740 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd12e670 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40045c0 tx=0x7fa4f4043740 comp rx=0 tx=0).stop 2024-03-26T08:42:46.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa50e000740 1 -- 172.21.15.7:0/516629407 shutdown_connections 2024-03-26T08:42:46.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa50e000740 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd129900 0x55e3bd122c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa50e000740 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd124790 0x55e3bd1356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa50e000740 1 --2- 172.21.15.7:0/516629407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd12e670 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa50e000740 1 -- 172.21.15.7:0/516629407 >> 172.21.15.7:0/516629407 conn(0x55e3bd11f310 msgr2=0x55e3bd122570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa50e000740 1 -- 172.21.15.7:0/516629407 shutdown_connections 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.102+0000 7fa50e000740 1 -- 172.21.15.7:0/516629407 wait complete. 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.103+0000 7fa50e000740 1 -- start start 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.103+0000 7fa50e000740 1 Processor -- start 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa50e000740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd133080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa50e000740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd124790 0x55e3bd1335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa50e000740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd134580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa50e000740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041b6070 con 0x55e3bd123da0 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa50e000740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041b7390 con 0x55e3bd124790 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa50e000740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041b66d0 con 0x55e3bd129900 2024-03-26T08:42:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa50e000740 10 client.0 _ll_register_callbacks cb 0x7fa4f4048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd134580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa5088d6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd134580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36660/0 (socket says 172.21.15.7:36660) 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.104+0000 7fa5088d6640 1 -- 172.21.15.7:0/578620181 learned_addr learned my addr 172.21.15.7:0/578620181 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa5088d6640 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa5088d6640 1 -- 172.21.15.7:0/578620181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd124790 msgr2=0x55e3bd1335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa5088d6640 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd124790 0x55e3bd1335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa5088d6640 1 -- 172.21.15.7:0/578620181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 msgr2=0x55e3bd133080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa5088d6640 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd133080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa5088d6640 1 -- 172.21.15.7:0/578620181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3bd134b70 con 0x55e3bd129900 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa5088d6640 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd134580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4f402c240 tx=0x7fa4f4034e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37632fd14424c32c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa4ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa4ecff9640 1 -- 172.21.15.7:0/578620181 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f404a070 con 0x55e3bd129900 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa50e000740 1 -- 172.21.15.7:0/578620181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4f404b040 con 0x55e3bd129900 2024-03-26T08:42:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.105+0000 7fa4ecff9640 1 -- 172.21.15.7:0/578620181 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa4f4034440 con 0x55e3bd129900 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa4ecff9640 1 -- 172.21.15.7:0/578620181 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4f40439a0 con 0x55e3bd129900 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa50e000740 10 client.25351 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa50e000740 1 -- 172.21.15.7:0/578620181 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e3bd1951d0 con 0x55e3bd129900 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa4ecff9640 1 -- 172.21.15.7:0/578620181 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4f404b040 con 0x55e3bd129900 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa4ecff9640 1 -- 172.21.15.7:0/578620181 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa4f4050040 con 0x55e3bd129900 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa50e000740 10 client.25351 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa50e000740 10 client.25351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa50e000740 20 client.25351 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa50e000740 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3bd195960 0x55e3bd197d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa50e000740 4 client.25351 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa50e000740 1 -- 172.21.15.7:0/578620181 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e3bd090c60 con 0x55e3bd195960 2024-03-26T08:42:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.106+0000 7fa5090d7640 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3bd195960 0x55e3bd197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa4ecff9640 10 client.25351 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa5090d7640 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3bd195960 0x55e3bd197d80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd694646df7b967 server_cookie=b68eaac346a22f76 in_seq=0 out_seq=0 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa4ecff9640 1 -- 172.21.15.7:0/578620181 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e3bd090c60 con 0x55e3bd195960 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa4ecff9640 10 client.25351 handle_command_reply: tid=0 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 client.25351 shutdown 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 -- 172.21.15.7:0/578620181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 msgr2=0x55e3bd134580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd134580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4f402c240 tx=0x7fa4f4034e60 comp rx=0 tx=0).stop 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 -- 172.21.15.7:0/578620181 shutdown_connections 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3bd195960 0x55e3bd197d80 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3bd129900 0x55e3bd134580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3bd124790 0x55e3bd1335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 --2- 172.21.15.7:0/578620181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3bd123da0 0x55e3bd133080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 -- 172.21.15.7:0/578620181 >> 172.21.15.7:0/578620181 conn(0x55e3bd11f310 msgr2=0x55e3bd122e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa4ecff9640 0 client.25351 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:46.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 -- 172.21.15.7:0/578620181 shutdown_connections 2024-03-26T08:42:46.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 1 -- 172.21.15.7:0/578620181 wait complete. 2024-03-26T08:42:46.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.107+0000 7fa50e000740 20 client.25351 trim_cache size 0 max 16384 2024-03-26T08:42:46.103 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:42:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:42:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:42:46.103 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:42:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.108+0000 7fa50b33b640 1 -- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4f00a2950 msgr2=0x7fa4f00a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.108+0000 7fa50b33b640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4f00a2950 0x7fa4f00a4e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc084fd0 tx=0x7fa4fc005b40 comp rx=0 tx=0).stop 2024-03-26T08:42:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.108+0000 7fa50b33b640 1 -- 172.21.15.7:0/787067917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504102170 msgr2=0x7fa5041ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.108+0000 7fa50b33b640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504102170 0x7fa5041ae2c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009fd0 tx=0x7fa4fc00ef10 comp rx=0 tx=0).stop 2024-03-26T08:42:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.109+0000 7fa50b33b640 1 -- 172.21.15.7:0/787067917 shutdown_connections 2024-03-26T08:42:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.109+0000 7fa50b33b640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa4f00a2950 0x7fa4f00a4e10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.109+0000 7fa50b33b640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa504102170 0x7fa5041ae2c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.109+0000 7fa50b33b640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5041017b0 0x7fa504111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.109+0000 7fa50b33b640 1 --2- 172.21.15.7:0/787067917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5041005c0 0x7fa504110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.109+0000 7fa50b33b640 1 -- 172.21.15.7:0/787067917 >> 172.21.15.7:0/787067917 conn(0x7fa5040fbb30 msgr2=0x7fa5041049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.109+0000 7fa50b33b640 1 -- 172.21.15.7:0/787067917 shutdown_connections 2024-03-26T08:42:46.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.109+0000 7fa50b33b640 1 -- 172.21.15.7:0/787067917 wait complete. 2024-03-26T08:42:46.116 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:42:46.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:42:46.154 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:45 smithi007 ceph-mon[32462]: pgmap v370: 97 pgs: 97 active+clean; 930 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 18 MiB/s wr, 730 op/s 2024-03-26T08:42:46.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:45 smithi016 ceph-mon[35851]: pgmap v370: 97 pgs: 97 active+clean; 930 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 18 MiB/s wr, 730 op/s 2024-03-26T08:42:46.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.285+0000 7f755719e640 1 -- 172.21.15.7:0/623153665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7550101910 msgr2=0x7f75501081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.285+0000 7f755719e640 1 --2- 172.21.15.7:0/623153665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7550101910 0x7f75501081a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7548009ca0 tx=0x7f754802c120 comp rx=0 tx=0).stop 2024-03-26T08:42:46.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.285+0000 7f755719e640 1 -- 172.21.15.7:0/623153665 shutdown_connections 2024-03-26T08:42:46.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.285+0000 7f755719e640 1 --2- 172.21.15.7:0/623153665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7550101910 0x7f75501081a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.285+0000 7f755719e640 1 --2- 172.21.15.7:0/623153665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7550100f50 0x7f75501013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.285+0000 7f755719e640 1 --2- 172.21.15.7:0/623153665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75500ffd60 0x7f7550100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.285+0000 7f755719e640 1 -- 172.21.15.7:0/623153665 >> 172.21.15.7:0/623153665 conn(0x7f75500fb2d0 msgr2=0x7f75500fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.285+0000 7f755719e640 1 -- 172.21.15.7:0/623153665 shutdown_connections 2024-03-26T08:42:46.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.285+0000 7f755719e640 1 -- 172.21.15.7:0/623153665 wait complete. 2024-03-26T08:42:46.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.286+0000 7f755719e640 1 Processor -- start 2024-03-26T08:42:46.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.286+0000 7f755719e640 1 -- start start 2024-03-26T08:42:46.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.286+0000 7f755719e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75500ffd60 0x7f75501957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.286+0000 7f755719e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7550100f50 0x7f7550195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.286+0000 7f755719e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7550101910 0x7f755019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f7554f3a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75500ffd60 0x7f75501957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f755573b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7550101910 0x7f755019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f7554f3a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75500ffd60 0x7f75501957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36682/0 (socket says 172.21.15.7:36682) 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f7554f3a640 1 -- 172.21.15.7:0/2761038774 learned_addr learned my addr 172.21.15.7:0/2761038774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755010b1b0 con 0x7f7550101910 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755010b030 con 0x7f7550100f50 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755010b330 con 0x7f75500ffd60 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f754ffff640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7550100f50 0x7f7550195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f755573b640 1 -- 172.21.15.7:0/2761038774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75500ffd60 msgr2=0x7f75501957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f755573b640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75500ffd60 0x7f75501957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f755573b640 1 -- 172.21.15.7:0/2761038774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7550100f50 msgr2=0x7f7550195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f755573b640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7550100f50 0x7f7550195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.287+0000 7f755573b640 1 -- 172.21.15.7:0/2761038774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755019d410 con 0x7f7550101910 2024-03-26T08:42:46.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.288+0000 7f755573b640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7550101910 0x7f755019cda0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7548037040 tx=0x7f7548035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.288+0000 7f754dffb640 1 -- 172.21.15.7:0/2761038774 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548039040 con 0x7f7550101910 2024-03-26T08:42:46.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.288+0000 7f754dffb640 1 -- 172.21.15.7:0/2761038774 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7548044070 con 0x7f7550101910 2024-03-26T08:42:46.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.288+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755019d6a0 con 0x7f7550101910 2024-03-26T08:42:46.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.288+0000 7f754dffb640 1 -- 172.21.15.7:0/2761038774 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548004470 con 0x7f7550101910 2024-03-26T08:42:46.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.288+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755019dbe0 con 0x7f7550101910 2024-03-26T08:42:46.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.290+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7550107080 con 0x7f7550101910 2024-03-26T08:42:46.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.292+0000 7f754dffb640 1 -- 172.21.15.7:0/2761038774 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f7548004610 con 0x7f7550101910 2024-03-26T08:42:46.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.292+0000 7f754dffb640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f75400a4a80 0x7f75400a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.292+0000 7f7554f3a640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f75400a4a80 0x7f75400a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.292+0000 7f754dffb640 1 -- 172.21.15.7:0/2761038774 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f75480f1930 con 0x7f7550101910 2024-03-26T08:42:46.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.293+0000 7f7554f3a640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f75400a4a80 0x7f75400a6f40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f754400bc80 tx=0x7f754400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.298+0000 7f754dffb640 1 -- 172.21.15.7:0/2761038774 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f754804e020 con 0x7f7550101910 2024-03-26T08:42:46.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:45 smithi044 ceph-mon[36461]: pgmap v370: 97 pgs: 97 active+clean; 930 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 18 MiB/s wr, 730 op/s 2024-03-26T08:42:46.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.693+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f755002d230 con 0x7f7550101910 2024-03-26T08:42:46.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.695+0000 7f754dffb640 1 -- 172.21.15.7:0/2761038774 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f755002d230 con 0x7f7550101910 2024-03-26T08:42:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:42:46.686 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:42:21.061940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24502,24512,24478]},"id":1}]} 2024-03-26T08:42:46.687 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f75400a4a80 msgr2=0x7f75400a6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f75400a4a80 0x7f75400a6f40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f754400bc80 tx=0x7f754400b410 comp rx=0 tx=0).stop 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7550101910 msgr2=0x7f755019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7550101910 0x7f755019cda0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7548037040 tx=0x7f7548035d10 comp rx=0 tx=0).stop 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 shutdown_connections 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f75400a4a80 0x7f75400a6f40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7550101910 0x7f755019cda0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7550100f50 0x7f7550195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 --2- 172.21.15.7:0/2761038774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f75500ffd60 0x7f75501957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 >> 172.21.15.7:0/2761038774 conn(0x7f75500fb2d0 msgr2=0x7f7550104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 shutdown_connections 2024-03-26T08:42:46.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.698+0000 7f755719e640 1 -- 172.21.15.7:0/2761038774 wait complete. 2024-03-26T08:42:46.704 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:42:46.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:42:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.920+0000 7fe6c8b77640 1 -- 172.21.15.7:0/2054850083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 msgr2=0x7fe6c4108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.920+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/2054850083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 0x7fe6c4108100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009860 tx=0x7fe6b402bb70 comp rx=0 tx=0).stop 2024-03-26T08:42:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.920+0000 7fe6c8b77640 1 -- 172.21.15.7:0/2054850083 shutdown_connections 2024-03-26T08:42:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.920+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/2054850083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4109890 0x7fe6c4110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.921+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/2054850083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4108ed0 0x7fe6c4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.921+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/2054850083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 0x7fe6c4108100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.921+0000 7fe6c8b77640 1 -- 172.21.15.7:0/2054850083 >> 172.21.15.7:0/2054850083 conn(0x7fe6c4076c90 msgr2=0x7fe6c40790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.921+0000 7fe6c8b77640 1 -- 172.21.15.7:0/2054850083 shutdown_connections 2024-03-26T08:42:46.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.921+0000 7fe6c8b77640 1 -- 172.21.15.7:0/2054850083 wait complete. 2024-03-26T08:42:46.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c8b77640 1 Processor -- start 2024-03-26T08:42:46.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c8b77640 1 -- start start 2024-03-26T08:42:46.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c8b77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 0x7fe6c419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c8b77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4108ed0 0x7fe6c419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c8b77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4109890 0x7fe6c41a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c8b77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41139b0 con 0x7fe6c4108ed0 2024-03-26T08:42:46.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c8b77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4113830 con 0x7fe6c4109890 2024-03-26T08:42:46.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c8b77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4113b30 con 0x7fe6c4107ce0 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 0x7fe6c419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 0x7fe6c419e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49878/0 (socket says 172.21.15.7:49878) 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c259c640 1 -- 172.21.15.7:0/1633998413 learned_addr learned my addr 172.21.15.7:0/1633998413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4108ed0 0x7fe6c419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4109890 0x7fe6c41a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c259c640 1 -- 172.21.15.7:0/1633998413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4109890 msgr2=0x7fe6c41a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c259c640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4109890 0x7fe6c41a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c259c640 1 -- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4108ed0 msgr2=0x7fe6c419e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c259c640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4108ed0 0x7fe6c419e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.922+0000 7fe6c259c640 1 -- 172.21.15.7:0/1633998413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c41a5ca0 con 0x7fe6c4107ce0 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.923+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4108ed0 0x7fe6c419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.923+0000 7fe6c259c640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 0x7fe6c419e080 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009780 tx=0x7fe6b4032440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.924+0000 7fe6b37fe640 1 -- 172.21.15.7:0/1633998413 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4039040 con 0x7fe6c4107ce0 2024-03-26T08:42:46.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.924+0000 7fe6b37fe640 1 -- 172.21.15.7:0/1633998413 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6b4053070 con 0x7fe6c4107ce0 2024-03-26T08:42:46.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.924+0000 7fe6b37fe640 1 -- 172.21.15.7:0/1633998413 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4004920 con 0x7fe6c4107ce0 2024-03-26T08:42:46.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.924+0000 7fe6c8b77640 1 -- 172.21.15.7:0/1633998413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c4079ef0 con 0x7fe6c4107ce0 2024-03-26T08:42:46.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.924+0000 7fe6c8b77640 1 -- 172.21.15.7:0/1633998413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c407a3e0 con 0x7fe6c4107ce0 2024-03-26T08:42:46.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.927+0000 7fe6cb839740 1 Processor -- start 2024-03-26T08:42:46.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.927+0000 7fe6cb839740 1 -- start start 2024-03-26T08:42:46.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.927+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afafab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.927+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afb4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.927+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afb7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.927+0000 7fe6cb839740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b60af72610 con 0x55b60afaf690 2024-03-26T08:42:46.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.927+0000 7fe6cb839740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b60af153c0 con 0x55b60afb0080 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.927+0000 7fe6cb839740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b60aebdd60 con 0x55b60afb51f0 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afafab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afafab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59752/0 (socket says 172.21.15.7:59752) 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/1599994980 learned_addr learned my addr 172.21.15.7:0/1599994980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afb7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6b37fe640 1 -- 172.21.15.7:0/1633998413 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fe6b40050e0 con 0x7fe6c4107ce0 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c259c640 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afb4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/1599994980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 msgr2=0x55b60afb7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afb7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/1599994980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 msgr2=0x55b60afb4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afb4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/1599994980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b60ac293f0 con 0x55b60afaf690 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afb7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.928+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afafab0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009f10 tx=0x7fe6bc00f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba120db7063f67f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.929+0000 7fe6b0ff9640 1 -- 172.21.15.7:0/1599994980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc003970 con 0x55b60afaf690 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.929+0000 7fe6b37fe640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6ac0a2a90 0x7fe6ac0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.929+0000 7fe6b0ff9640 1 -- 172.21.15.7:0/1599994980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc004030 con 0x55b60afaf690 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.929+0000 7fe6b0ff9640 1 -- 172.21.15.7:0/1599994980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc00c9c0 con 0x55b60afaf690 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.929+0000 7fe6b37fe640 1 -- 172.21.15.7:0/1633998413 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6b4101570 con 0x7fe6c4107ce0 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.929+0000 7fe6c259c640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6ac0a2a90 0x7fe6ac0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6cb839740 1 -- 172.21.15.7:0/1599994980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 msgr2=0x55b60afafab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afafab0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009f10 tx=0x7fe6bc00f7d0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6cb839740 1 -- 172.21.15.7:0/1599994980 shutdown_connections 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afb7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afb4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1599994980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afafab0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6cb839740 1 -- 172.21.15.7:0/1599994980 >> 172.21.15.7:0/1599994980 conn(0x55b60afaac00 msgr2=0x55b60afab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6c259c640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6ac0a2a90 0x7fe6ac0a4f50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55b60afb0480 tx=0x7fe6b402bb70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6cb839740 1 -- 172.21.15.7:0/1599994980 shutdown_connections 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.930+0000 7fe6cb839740 1 -- 172.21.15.7:0/1599994980 wait complete. 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6cb839740 1 -- start start 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6cb839740 1 Processor -- start 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afb98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afb9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afc0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6cb839740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b60aed1240 con 0x55b60afaf690 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6cb839740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b60af11460 con 0x55b60afb0080 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6cb839740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b60afb81a0 con 0x55b60afb51f0 2024-03-26T08:42:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6cb839740 10 client.0 _ll_register_callbacks cb 0x55b60aed0c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afb98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6c1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afb9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afb98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59756/0 (socket says 172.21.15.7:59756) 2024-03-26T08:42:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/1706951512 learned_addr learned my addr 172.21.15.7:0/1706951512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.931+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afc0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6c1d9b640 1 -- 172.21.15.7:0/1706951512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 msgr2=0x55b60afc0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afc0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6c1d9b640 1 -- 172.21.15.7:0/1706951512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 msgr2=0x55b60afb98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afb98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6c1d9b640 1 -- 172.21.15.7:0/1706951512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b60acf37d0 con 0x55b60afb0080 2024-03-26T08:42:46.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afc0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:46.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afb9df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55b60afabb70 tx=0x7fe6b8012bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe71fe57621e22ed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:42:46.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/1706951512 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8004320 con 0x55b60afb0080 2024-03-26T08:42:46.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/1706951512 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6b8011970 con 0x55b60afb0080 2024-03-26T08:42:46.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6cb839740 1 -- 172.21.15.7:0/1706951512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6bc004030 con 0x55b60afb0080 2024-03-26T08:42:46.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/1706951512 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b80044c0 con 0x55b60afb0080 2024-03-26T08:42:46.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6cb839740 10 client.25349 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6cb839740 1 -- 172.21.15.7:0/1706951512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b60afc4210 con 0x55b60afb0080 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/1706951512 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc004030 con 0x55b60afb0080 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/1706951512 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fe6b800f440 con 0x55b60afb0080 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6cb839740 10 client.25349 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6cb839740 10 client.25349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6cb839740 20 client.25349 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.932+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b60afc5870 0x55b60afc7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.933+0000 7fe6cb839740 4 client.25349 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.933+0000 7fe6cb839740 1 -- 172.21.15.7:0/1706951512 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b60af22900 con 0x55b60afc5870 2024-03-26T08:42:46.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.933+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b60afc5870 0x55b60afc7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.933+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b60afc5870 0x55b60afc7c90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6cebe1f830d2f1a8 server_cookie=d5210b5c0460c7b3 in_seq=0 out_seq=0 2024-03-26T08:42:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.934+0000 7fe6a9ffb640 10 client.25349 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.934+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/1706951512 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b60af22900 con 0x55b60afc5870 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6a9ffb640 10 client.25349 handle_command_reply: tid=0 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 client.25349 shutdown 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 -- 172.21.15.7:0/1706951512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 msgr2=0x55b60afb9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afb9df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55b60afabb70 tx=0x7fe6b8012bd0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 -- 172.21.15.7:0/1706951512 shutdown_connections 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b60afc5870 0x55b60afc7c90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb51f0 0x55b60afc0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afb9df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1706951512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afb98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 -- 172.21.15.7:0/1706951512 >> 172.21.15.7:0/1706951512 conn(0x55b60afaac00 msgr2=0x55b60afa9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6a9ffb640 0 client.25349 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 -- 172.21.15.7:0/1706951512 shutdown_connections 2024-03-26T08:42:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 1 -- 172.21.15.7:0/1706951512 wait complete. 2024-03-26T08:42:46.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.935+0000 7fe6cb839740 20 client.25349 trim_cache size 0 max 16384 2024-03-26T08:42:46.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.956+0000 7fe6cb839740 1 Processor -- start 2024-03-26T08:42:46.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.956+0000 7fe6cb839740 1 -- start start 2024-03-26T08:42:46.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.956+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afaf690 0x55b60afb8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afc0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afb51f0 0x55b60afae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afaf690 0x55b60afb8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afaf690 0x55b60afb8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49908/0 (socket says 172.21.15.7:49908) 2024-03-26T08:42:46.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/1254829696 learned_addr learned my addr 172.21.15.7:0/1254829696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/1254829696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afc0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6cb839740 1 -- 172.21.15.7:0/1254829696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41a8e10 con 0x55b60afb51f0 2024-03-26T08:42:46.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6cb839740 1 -- 172.21.15.7:0/1254829696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41a6910 con 0x55b60afb0080 2024-03-26T08:42:46.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6cb839740 1 -- 172.21.15.7:0/1254829696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4081220 con 0x55b60afaf690 2024-03-26T08:42:46.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1254829696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afb51f0 0x55b60afae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/1254829696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 msgr2=0x55b60afc0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1254829696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afc0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/1254829696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afb51f0 msgr2=0x55b60afae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1254829696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afb51f0 0x55b60afae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.957+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/1254829696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b60afa6480 con 0x55b60afaf690 2024-03-26T08:42:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.958+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/1254829696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afaf690 0x55b60afb8dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00fcd0 tx=0x7fe6bc005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42076fa064466e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.958+0000 7fe6abfff640 1 -- 172.21.15.7:0/1254829696 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc021040 con 0x55b60afaf690 2024-03-26T08:42:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.958+0000 7fe6abfff640 1 -- 172.21.15.7:0/1254829696 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc004e00 con 0x55b60afaf690 2024-03-26T08:42:46.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.958+0000 7fe6abfff640 1 -- 172.21.15.7:0/1254829696 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc003970 con 0x55b60afaf690 2024-03-26T08:42:46.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.959+0000 7fe6cb839740 1 -- 172.21.15.7:0/1254829696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afaf690 msgr2=0x55b60afb8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.959+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1254829696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afaf690 0x55b60afb8dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00fcd0 tx=0x7fe6bc005fd0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.960+0000 7fe6cb839740 1 -- 172.21.15.7:0/1254829696 shutdown_connections 2024-03-26T08:42:46.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.960+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1254829696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afb51f0 0x55b60afae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.960+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1254829696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb0080 0x55b60afc0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.960+0000 7fe6cb839740 1 --2- 172.21.15.7:0/1254829696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afaf690 0x55b60afb8dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.960+0000 7fe6cb839740 1 -- 172.21.15.7:0/1254829696 >> 172.21.15.7:0/1254829696 conn(0x55b60afaac00 msgr2=0x55b60afade60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.960+0000 7fe6cb839740 1 -- 172.21.15.7:0/1254829696 shutdown_connections 2024-03-26T08:42:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.960+0000 7fe6cb839740 1 -- 172.21.15.7:0/1254829696 wait complete. 2024-03-26T08:42:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6cb839740 1 -- start start 2024-03-26T08:42:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6cb839740 1 Processor -- start 2024-03-26T08:42:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afbe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb0080 0x55b60afbeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6cb839740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb51f0 0x55b60afbfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6cb839740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4081820 con 0x55b60afaf690 2024-03-26T08:42:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6cb839740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41a62b0 con 0x55b60afb51f0 2024-03-26T08:42:46.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6cb839740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c4081e80 con 0x55b60afb0080 2024-03-26T08:42:46.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6cb839740 10 client.0 _ll_register_callbacks cb 0x7fe6b8009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:46.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afbe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6c2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afbe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59774/0 (socket says 172.21.15.7:59774) 2024-03-26T08:42:46.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6c2d9d640 1 -- 172.21.15.7:0/3080172431 learned_addr learned my addr 172.21.15.7:0/3080172431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:46.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.961+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb51f0 0x55b60afbfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.962+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb0080 0x55b60afbeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.962+0000 7fe6c1d9b640 1 -- 172.21.15.7:0/3080172431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb51f0 msgr2=0x55b60afbfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.962+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb51f0 0x55b60afbfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.962+0000 7fe6c1d9b640 1 -- 172.21.15.7:0/3080172431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 msgr2=0x55b60afbe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.962+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afbe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.962+0000 7fe6c1d9b640 1 -- 172.21.15.7:0/3080172431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b60afc0420 con 0x55b60afb0080 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.963+0000 7fe6c1d9b640 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb0080 0x55b60afbeec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800f830 tx=0x7fe6b800fcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2a7a693ceb75bc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.963+0000 7fe6a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.963+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/3080172431 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b802c070 con 0x55b60afb0080 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.963+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/3080172431 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6b8011b80 con 0x55b60afb0080 2024-03-26T08:42:46.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.963+0000 7fe6cb839740 1 -- 172.21.15.7:0/3080172431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6bc004e00 con 0x55b60afb0080 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.963+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/3080172431 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b8009730 con 0x55b60afb0080 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.963+0000 7fe6cb839740 10 client.25375 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.963+0000 7fe6cb839740 1 -- 172.21.15.7:0/3080172431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b60b0215c0 con 0x55b60afb0080 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.964+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/3080172431 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6b802d040 con 0x55b60afb0080 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.964+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/3080172431 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fe6b8039030 con 0x55b60afb0080 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.964+0000 7fe6cb839740 10 client.25375 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.964+0000 7fe6cb839740 10 client.25375 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.964+0000 7fe6cb839740 20 client.25375 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.964+0000 7fe6cb839740 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b60b021f30 0x55b60b024350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.964+0000 7fe6cb839740 4 client.25375 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.964+0000 7fe6cb839740 1 -- 172.21.15.7:0/3080172431 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe6bc004e00 con 0x55b60b021f30 2024-03-26T08:42:46.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.964+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b60b021f30 0x55b60b024350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:46.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.966+0000 7fe6c2d9d640 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b60b021f30 0x55b60b024350 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11f0f8e9ff0805b2 server_cookie=40096eccd66df4e5 in_seq=0 out_seq=0 2024-03-26T08:42:46.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.966+0000 7fe6a9ffb640 10 client.25375 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:42:46.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.966+0000 7fe6a9ffb640 1 -- 172.21.15.7:0/3080172431 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe6bc004e00 con 0x55b60b021f30 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.966+0000 7fe6a9ffb640 10 client.25375 handle_command_reply: tid=0 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 client.25375 shutdown 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 -- 172.21.15.7:0/3080172431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb0080 msgr2=0x55b60afbeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb0080 0x55b60afbeec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800f830 tx=0x7fe6b800fcd0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 -- 172.21.15.7:0/3080172431 shutdown_connections 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b60b021f30 0x55b60b024350 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60afb51f0 0x55b60afbfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b60afb0080 0x55b60afbeec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 --2- 172.21.15.7:0/3080172431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b60afaf690 0x55b60afbe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6a9ffb640 0 client.25375 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 -- 172.21.15.7:0/3080172431 >> 172.21.15.7:0/3080172431 conn(0x55b60afaac00 msgr2=0x55b60afae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 -- 172.21.15.7:0/3080172431 shutdown_connections 2024-03-26T08:42:46.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.967+0000 7fe6cb839740 1 -- 172.21.15.7:0/3080172431 wait complete. 2024-03-26T08:42:46.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.968+0000 7fe6cb839740 20 client.25375 trim_cache size 0 max 16384 2024-03-26T08:42:46.960 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:42:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 -- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6ac0a2a90 msgr2=0x7fe6ac0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6ac0a2a90 0x7fe6ac0a4f50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55b60afb0480 tx=0x7fe6b402bb70 comp rx=0 tx=0).stop 2024-03-26T08:42:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 -- 172.21.15.7:0/1633998413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 msgr2=0x7fe6c419e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 0x7fe6c419e080 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009780 tx=0x7fe6b4032440 comp rx=0 tx=0).stop 2024-03-26T08:42:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 -- 172.21.15.7:0/1633998413 shutdown_connections 2024-03-26T08:42:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6ac0a2a90 0x7fe6ac0a4f50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4109890 0x7fe6c41a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c4108ed0 0x7fe6c419e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 --2- 172.21.15.7:0/1633998413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c4107ce0 0x7fe6c419e080 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 -- 172.21.15.7:0/1633998413 >> 172.21.15.7:0/1633998413 conn(0x7fe6c4076c90 msgr2=0x7fe6c410c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 -- 172.21.15.7:0/1633998413 shutdown_connections 2024-03-26T08:42:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:46.969+0000 7fe6c8b77640 1 -- 172.21.15.7:0/1633998413 wait complete. 2024-03-26T08:42:46.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:46 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2761038774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:46.973 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:42:46.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:42:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.129+0000 7f880e71b640 1 -- 172.21.15.7:0/3527276851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081005e0 msgr2=0x7f8808100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.129+0000 7f880e71b640 1 --2- 172.21.15.7:0/3527276851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081005e0 0x7f8808100a00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009900 tx=0x7f87f802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.130+0000 7f880e71b640 1 -- 172.21.15.7:0/3527276851 shutdown_connections 2024-03-26T08:42:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.130+0000 7f880e71b640 1 --2- 172.21.15.7:0/3527276851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8808102190 0x7f8808108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.130+0000 7f880e71b640 1 --2- 172.21.15.7:0/3527276851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88081017d0 0x7f8808101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.130+0000 7f880e71b640 1 --2- 172.21.15.7:0/3527276851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88081005e0 0x7f8808100a00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.130+0000 7f880e71b640 1 -- 172.21.15.7:0/3527276851 >> 172.21.15.7:0/3527276851 conn(0x7f88080fbb70 msgr2=0x7f88080fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.130+0000 7f880e71b640 1 -- 172.21.15.7:0/3527276851 shutdown_connections 2024-03-26T08:42:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.130+0000 7f880e71b640 1 -- 172.21.15.7:0/3527276851 wait complete. 2024-03-26T08:42:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.131+0000 7f880e71b640 1 Processor -- start 2024-03-26T08:42:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.131+0000 7f880e71b640 1 -- start start 2024-03-26T08:42:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.131+0000 7f880e71b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88081005e0 0x7f8808199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.131+0000 7f880e71b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88081017d0 0x7f880819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.131+0000 7f880e71b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8808102190 0x7f88081a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f880e71b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880810b1b0 con 0x7f8808102190 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f880e71b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880810b030 con 0x7f88081017d0 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f880e71b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880810b330 con 0x7f88081005e0 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f88077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88081017d0 0x7f880819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f8807fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88081005e0 0x7f8808199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f88077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88081017d0 0x7f880819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:32962/0 (socket says 172.21.15.7:32962) 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f8807fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88081005e0 0x7f8808199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49926/0 (socket says 172.21.15.7:49926) 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f88077fe640 1 -- 172.21.15.7:0/4208480763 learned_addr learned my addr 172.21.15.7:0/4208480763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f880ccb8640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8808102190 0x7f88081a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f88077fe640 1 -- 172.21.15.7:0/4208480763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88081005e0 msgr2=0x7f8808199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f88077fe640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88081005e0 0x7f8808199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f88077fe640 1 -- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8808102190 msgr2=0x7f88081a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f88077fe640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8808102190 0x7f88081a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f88077fe640 1 -- 172.21.15.7:0/4208480763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88081a1850 con 0x7f88081017d0 2024-03-26T08:42:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.132+0000 7f8807fff640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88081005e0 0x7f8808199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.133+0000 7f88077fe640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88081017d0 0x7f880819a180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0097d0 tx=0x7f87fc009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.133+0000 7f88057fa640 1 -- 172.21.15.7:0/4208480763 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00eab0 con 0x7f88081017d0 2024-03-26T08:42:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.134+0000 7f88057fa640 1 -- 172.21.15.7:0/4208480763 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87fc019070 con 0x7f88081017d0 2024-03-26T08:42:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.134+0000 7f880e71b640 1 -- 172.21.15.7:0/4208480763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88081a1b40 con 0x7f88081017d0 2024-03-26T08:42:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.134+0000 7f88057fa640 1 -- 172.21.15.7:0/4208480763 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc013360 con 0x7f88081017d0 2024-03-26T08:42:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.134+0000 7f880e71b640 1 -- 172.21.15.7:0/4208480763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88081a2060 con 0x7f88081017d0 2024-03-26T08:42:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.137+0000 7f88113e9740 1 Processor -- start 2024-03-26T08:42:47.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f88113e9740 1 -- start start 2024-03-26T08:42:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f88113e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954bd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f88057fa640 1 -- 172.21.15.7:0/4208480763 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f87fc013070 con 0x7f88081017d0 2024-03-26T08:42:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f88113e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954c2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f88113e9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f88113e9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e195477610 con 0x55e1954c2a00 2024-03-26T08:42:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f88113e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19541a3c0 con 0x55e1954bd890 2024-03-26T08:42:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f88113e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1953c2d60 con 0x55e1954bcea0 2024-03-26T08:42:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f880ccb8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954bd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f880ccb8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954bd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49942/0 (socket says 172.21.15.7:49942) 2024-03-26T08:42:47.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.138+0000 7f880ccb8640 1 -- 172.21.15.7:0/3301790713 learned_addr learned my addr 172.21.15.7:0/3301790713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f8807fff640 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f880ccb8640 1 -- 172.21.15.7:0/3301790713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 msgr2=0x55e1954c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f880ccb8640 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f880ccb8640 1 -- 172.21.15.7:0/3301790713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 msgr2=0x55e1954c4e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:42:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f8807fff640 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:42:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f88077fe640 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f88057fa640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87f00a29e0 0x7f87f00a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f880ccb8640 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f880ccb8640 1 -- 172.21.15.7:0/3301790713 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e19512e3f0 con 0x55e1954bcea0 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f88077fe640 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f88057fa640 1 -- 172.21.15.7:0/4208480763 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f87fc0ce5e0 con 0x7f88081017d0 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.139+0000 7f880ccb8640 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954bd2c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88000039f0 tx=0x7f8800003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a93ea7c9fc1851d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.140+0000 7f8807fff640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87f00a29e0 0x7f87f00a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.141+0000 7f87ee7fc640 1 -- 172.21.15.7:0/3301790713 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880001a470 con 0x55e1954bcea0 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.141+0000 7f87ee7fc640 1 -- 172.21.15.7:0/3301790713 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8800015070 con 0x55e1954bcea0 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.141+0000 7f87ee7fc640 1 -- 172.21.15.7:0/3301790713 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8800010900 con 0x55e1954bcea0 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f88113e9740 1 -- 172.21.15.7:0/3301790713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 msgr2=0x55e1954bd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f8807fff640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87f00a29e0 0x7f87f00a4ea0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f87f802c170 tx=0x7f87f802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f88113e9740 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954bd2c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88000039f0 tx=0x7f8800003eb0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f88113e9740 1 -- 172.21.15.7:0/3301790713 shutdown_connections 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f88113e9740 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f88113e9740 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f88113e9740 1 --2- 172.21.15.7:0/3301790713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954bd2c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f88113e9740 1 -- 172.21.15.7:0/3301790713 >> 172.21.15.7:0/3301790713 conn(0x55e1954afc00 msgr2=0x55e1954b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f88113e9740 1 -- 172.21.15.7:0/3301790713 shutdown_connections 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.142+0000 7f88113e9740 1 -- 172.21.15.7:0/3301790713 wait complete. 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.143+0000 7f88113e9740 1 -- start start 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.143+0000 7f88113e9740 1 Processor -- start 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.143+0000 7f88113e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.143+0000 7f88113e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.143+0000 7f88113e9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.143+0000 7f88113e9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1953d6240 con 0x55e1954c2a00 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.143+0000 7f88113e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e195416460 con 0x55e1954bd890 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.143+0000 7f88113e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1954c5a00 con 0x55e1954bcea0 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f88113e9740 10 client.0 _ll_register_callbacks cb 0x55e1953d5c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f8807fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f8807fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954c72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49946/0 (socket says 172.21.15.7:49946) 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f8807fff640 1 -- 172.21.15.7:0/965658162 learned_addr learned my addr 172.21.15.7:0/965658162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f880ccb8640 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f88077fe640 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f8807fff640 1 -- 172.21.15.7:0/965658162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 msgr2=0x55e1954c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f8807fff640 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f8807fff640 1 -- 172.21.15.7:0/965658162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 msgr2=0x55e1954ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f8807fff640 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.144+0000 7f8807fff640 1 -- 172.21.15.7:0/965658162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1951f87d0 con 0x55e1954bcea0 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.145+0000 7f880ccb8640 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.145+0000 7f8807fff640 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954c72b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87f8000c00 tx=0x7f87f802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f5cfd69a68ee077 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.145+0000 7f87e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.145+0000 7f87e77fe640 1 -- 172.21.15.7:0/965658162 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f804f070 con 0x55e1954bcea0 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.145+0000 7f87e77fe640 1 -- 172.21.15.7:0/965658162 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f87f8004e70 con 0x55e1954bcea0 2024-03-26T08:42:47.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.145+0000 7f88113e9740 1 -- 172.21.15.7:0/965658162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8800015070 con 0x55e1954bcea0 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.145+0000 7f87e77fe640 1 -- 172.21.15.7:0/965658162 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f8005070 con 0x55e1954bcea0 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f88113e9740 10 client.25387 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f88113e9740 1 -- 172.21.15.7:0/965658162 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e1954d1bd0 con 0x55e1954bcea0 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f87e77fe640 1 -- 172.21.15.7:0/965658162 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8800015070 con 0x55e1954bcea0 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f87e77fe640 1 -- 172.21.15.7:0/965658162 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f87f8003c70 con 0x55e1954bcea0 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f88113e9740 10 client.25387 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f88113e9740 10 client.25387 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f88113e9740 20 client.25387 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f88113e9740 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55e1954d36e0 0x55e1954d5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f88113e9740 4 client.25387 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:47.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f88113e9740 1 -- 172.21.15.7:0/965658162 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e195427900 con 0x55e1954d36e0 2024-03-26T08:42:47.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.146+0000 7f88077fe640 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55e1954d36e0 0x55e1954d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.147+0000 7f87e77fe640 10 client.25387 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.147+0000 7f88077fe640 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55e1954d36e0 0x55e1954d5b30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=390d74e1e0b42d26 server_cookie=6c4cb95791071120 in_seq=0 out_seq=0 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.148+0000 7f87e77fe640 1 -- 172.21.15.7:0/965658162 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e195427900 con 0x55e1954d36e0 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.148+0000 7f87e77fe640 10 client.25387 handle_command_reply: tid=0 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.148+0000 7f88113e9740 1 client.25387 shutdown 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 -- 172.21.15.7:0/965658162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 msgr2=0x55e1954c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954c72b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87f8000c00 tx=0x7f87f802bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 -- 172.21.15.7:0/965658162 shutdown_connections 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55e1954d36e0 0x55e1954d5b30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 --2- 172.21.15.7:0/965658162 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954c72b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 -- 172.21.15.7:0/965658162 >> 172.21.15.7:0/965658162 conn(0x55e1954afc00 msgr2=0x55e1954aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f87e77fe640 0 client.25387 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:42:47.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 -- 172.21.15.7:0/965658162 shutdown_connections 2024-03-26T08:42:47.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 1 -- 172.21.15.7:0/965658162 wait complete. 2024-03-26T08:42:47.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.149+0000 7f88113e9740 20 client.25387 trim_cache size 0 max 16384 2024-03-26T08:42:47.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.167+0000 7f88113e9740 1 Processor -- start 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.167+0000 7f88113e9740 1 -- start start 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.167+0000 7f88113e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.167+0000 7f88113e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.167+0000 7f88113e9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.167+0000 7f88113e9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081a8d60 con 0x55e1954c2a00 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.167+0000 7f88113e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081a6860 con 0x55e1954bd890 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.167+0000 7f88113e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081a48e0 con 0x55e1954bcea0 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f880ccb8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f88077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f8807fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f88077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:32990/0 (socket says 172.21.15.7:32990) 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f88077fe640 1 -- 172.21.15.7:0/843560270 learned_addr learned my addr 172.21.15.7:0/843560270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f880ccb8640 1 -- 172.21.15.7:0/843560270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 msgr2=0x55e1954aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f880ccb8640 1 --2- 172.21.15.7:0/843560270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f880ccb8640 1 -- 172.21.15.7:0/843560270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 msgr2=0x55e1954ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f880ccb8640 1 --2- 172.21.15.7:0/843560270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f880ccb8640 1 -- 172.21.15.7:0/843560270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1954ab480 con 0x55e1954c2a00 2024-03-26T08:42:47.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f88077fe640 1 --2- 172.21.15.7:0/843560270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:47.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f880ccb8640 1 --2- 172.21.15.7:0/843560270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954da430 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8800000f80 tx=0x7f8800009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98863301c2e72c93 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f87edffb640 1 -- 172.21.15.7:0/843560270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880002b070 con 0x55e1954c2a00 2024-03-26T08:42:47.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f87edffb640 1 -- 172.21.15.7:0/843560270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8800015040 con 0x55e1954c2a00 2024-03-26T08:42:47.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.168+0000 7f87edffb640 1 -- 172.21.15.7:0/843560270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880001a7d0 con 0x55e1954c2a00 2024-03-26T08:42:47.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.169+0000 7f88113e9740 1 -- 172.21.15.7:0/843560270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 msgr2=0x55e1954da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.169+0000 7f88113e9740 1 --2- 172.21.15.7:0/843560270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954da430 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8800000f80 tx=0x7f8800009da0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.170+0000 7f88113e9740 1 -- 172.21.15.7:0/843560270 shutdown_connections 2024-03-26T08:42:47.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.170+0000 7f88113e9740 1 --2- 172.21.15.7:0/843560270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954c2a00 0x55e1954da430 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.170+0000 7f88113e9740 1 --2- 172.21.15.7:0/843560270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bd890 0x55e1954ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.170+0000 7f88113e9740 1 --2- 172.21.15.7:0/843560270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954bcea0 0x55e1954aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.170+0000 7f88113e9740 1 -- 172.21.15.7:0/843560270 >> 172.21.15.7:0/843560270 conn(0x55e1954afc00 msgr2=0x55e1954ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.170+0000 7f88113e9740 1 -- 172.21.15.7:0/843560270 shutdown_connections 2024-03-26T08:42:47.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.170+0000 7f88113e9740 1 -- 172.21.15.7:0/843560270 wait complete. 2024-03-26T08:42:47.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.170+0000 7f88113e9740 1 -- start start 2024-03-26T08:42:47.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.171+0000 7f88113e9740 1 Processor -- start 2024-03-26T08:42:47.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f88113e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bcea0 0x55e1954cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f88113e9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954bd890 0x55e1954cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f88113e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954c2a00 0x55e19552cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f8807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bcea0 0x55e1954cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f88077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954bd890 0x55e1954cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f8807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bcea0 0x55e1954cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:32998/0 (socket says 172.21.15.7:32998) 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f88077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954bd890 0x55e1954cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59818/0 (socket says 172.21.15.7:59818) 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f8807fff640 1 -- 172.21.15.7:0/2727732423 learned_addr learned my addr 172.21.15.7:0/2727732423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081a4ee0 con 0x55e1954bd890 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081a6200 con 0x55e1954bcea0 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88081a5540 con 0x55e1954c2a00 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f88113e9740 10 client.0 _ll_register_callbacks cb 0x7f87f8004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f880ccb8640 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954c2a00 0x55e19552cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f8807fff640 1 -- 172.21.15.7:0/2727732423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954c2a00 msgr2=0x55e19552cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f8807fff640 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954c2a00 0x55e19552cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f8807fff640 1 -- 172.21.15.7:0/2727732423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954bd890 msgr2=0x55e1954cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f8807fff640 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954bd890 0x55e1954cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.172+0000 7f8807fff640 1 -- 172.21.15.7:0/2727732423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1954ce910 con 0x55e1954bcea0 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f8807fff640 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bcea0 0x55e1954cb370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009900 tx=0x7f87f802c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f04270b972e74ab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f87e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f87e77fe640 1 -- 172.21.15.7:0/2727732423 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8066070 con 0x55e1954bcea0 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f87e77fe640 1 -- 172.21.15.7:0/2727732423 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f87f8049640 con 0x55e1954bcea0 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8800015040 con 0x55e1954bcea0 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f87e77fe640 1 -- 172.21.15.7:0/2727732423 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f804f040 con 0x55e1954bcea0 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f88113e9740 10 client.25370 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e19552e410 con 0x55e1954bcea0 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f87e77fe640 1 -- 172.21.15.7:0/2727732423 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8800015040 con 0x55e1954bcea0 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.173+0000 7f87e77fe640 1 -- 172.21.15.7:0/2727732423 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f87f8003c70 con 0x55e1954bcea0 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.174+0000 7f88113e9740 10 client.25370 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.174+0000 7f88113e9740 10 client.25370 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.174+0000 7f88113e9740 20 client.25370 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.174+0000 7f88113e9740 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55e19552ed00 0x55e1955310f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.174+0000 7f88113e9740 4 client.25370 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.174+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e195421c50 con 0x55e19552ed00 2024-03-26T08:42:47.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.174+0000 7f88077fe640 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55e19552ed00 0x55e1955310f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.174+0000 7f88077fe640 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55e19552ed00 0x55e1955310f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=335b38e6867d5fd server_cookie=92adfa226d4c5779 in_seq=0 out_seq=0 2024-03-26T08:42:47.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.174+0000 7f87e77fe640 10 client.25370 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:42:47.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.175+0000 7f87e77fe640 1 -- 172.21.15.7:0/2727732423 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e195421c50 con 0x55e19552ed00 2024-03-26T08:42:47.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.175+0000 7f87e77fe640 10 client.25370 handle_command_reply: tid=0 2024-03-26T08:42:47.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.175+0000 7f88113e9740 1 client.25370 shutdown 2024-03-26T08:42:47.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.175+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bcea0 msgr2=0x55e1954cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.175+0000 7f88113e9740 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bcea0 0x55e1954cb370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009900 tx=0x7f87f802c3d0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.175+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 shutdown_connections 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.175+0000 7f88113e9740 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55e19552ed00 0x55e1955310f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f88113e9740 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e1954c2a00 0x55e19552cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f88113e9740 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1954bd890 0x55e1954cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f87e77fe640 0 client.25370 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f88113e9740 1 --2- 172.21.15.7:0/2727732423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e1954bcea0 0x55e1954cb370 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 >> 172.21.15.7:0/2727732423 conn(0x55e1954afc00 msgr2=0x55e1954bbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 shutdown_connections 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f88113e9740 1 -- 172.21.15.7:0/2727732423 wait complete. 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f88113e9740 20 client.25370 trim_cache size 0 max 16384 2024-03-26T08:42:47.170 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:42:47.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 -- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87f00a29e0 msgr2=0x7f87f00a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87f00a29e0 0x7f87f00a4ea0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f87f802c170 tx=0x7f87f802df90 comp rx=0 tx=0).stop 2024-03-26T08:42:47.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 -- 172.21.15.7:0/4208480763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88081017d0 msgr2=0x7f880819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88081017d0 0x7f880819a180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0097d0 tx=0x7f87fc009c90 comp rx=0 tx=0).stop 2024-03-26T08:42:47.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 -- 172.21.15.7:0/4208480763 shutdown_connections 2024-03-26T08:42:47.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f87f00a29e0 0x7f87f00a4ea0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8808102190 0x7f88081a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88081017d0 0x7f880819a180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 --2- 172.21.15.7:0/4208480763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88081005e0 0x7f8808199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 -- 172.21.15.7:0/4208480763 >> 172.21.15.7:0/4208480763 conn(0x7f88080fbb70 msgr2=0x7f8808190850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 -- 172.21.15.7:0/4208480763 shutdown_connections 2024-03-26T08:42:47.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.176+0000 7f880e71b640 1 -- 172.21.15.7:0/4208480763 wait complete. 2024-03-26T08:42:47.181 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:42:47.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:42:47.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:46 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2761038774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:47.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:46 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2761038774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:42:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.345+0000 7fc144fa9640 1 -- 172.21.15.7:0/841406694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1401086a0 msgr2=0x7fc14010aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.345+0000 7fc144fa9640 1 --2- 172.21.15.7:0/841406694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1401086a0 0x7fc14010aad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc130009860 tx=0x7fc13002bb70 comp rx=0 tx=0).stop 2024-03-26T08:42:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.346+0000 7fc144fa9640 1 -- 172.21.15.7:0/841406694 shutdown_connections 2024-03-26T08:42:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.346+0000 7fc144fa9640 1 --2- 172.21.15.7:0/841406694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc14010d9c0 0x7fc14010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.346+0000 7fc144fa9640 1 --2- 172.21.15.7:0/841406694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc14010b010 0x7fc14010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.346+0000 7fc144fa9640 1 --2- 172.21.15.7:0/841406694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1401086a0 0x7fc14010aad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.346+0000 7fc144fa9640 1 -- 172.21.15.7:0/841406694 >> 172.21.15.7:0/841406694 conn(0x7fc14006e640 msgr2=0x7fc140070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.346+0000 7fc144fa9640 1 -- 172.21.15.7:0/841406694 shutdown_connections 2024-03-26T08:42:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.347+0000 7fc144fa9640 1 -- 172.21.15.7:0/841406694 wait complete. 2024-03-26T08:42:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc144fa9640 1 Processor -- start 2024-03-26T08:42:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc144fa9640 1 -- start start 2024-03-26T08:42:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc144fa9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1401086a0 0x7fc14019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc144fa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc14010b010 0x7fc14019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc144fa9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc14010d9c0 0x7fc1401a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc144fa9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140111990 con 0x7fc14010d9c0 2024-03-26T08:42:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc144fa9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140111810 con 0x7fc14010b010 2024-03-26T08:42:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc144fa9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140111b10 con 0x7fc1401086a0 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc13ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc14010d9c0 0x7fc1401a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1401086a0 0x7fc14019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc13ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc14010d9c0 0x7fc1401a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59824/0 (socket says 172.21.15.7:59824) 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc13ed9d640 1 -- 172.21.15.7:0/3448251192 learned_addr learned my addr 172.21.15.7:0/3448251192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc13dd9b640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc14010b010 0x7fc14019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc13ed9d640 1 -- 172.21.15.7:0/3448251192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1401086a0 msgr2=0x7fc14019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc13ed9d640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1401086a0 0x7fc14019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.348+0000 7fc13ed9d640 1 -- 172.21.15.7:0/3448251192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc14010b010 msgr2=0x7fc14019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.349+0000 7fc13ed9d640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc14010b010 0x7fc14019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.349+0000 7fc13ed9d640 1 -- 172.21.15.7:0/3448251192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1401a5c70 con 0x7fc14010d9c0 2024-03-26T08:42:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.349+0000 7fc13ed9d640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc14010d9c0 0x7fc1401a5600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc13401e920 tx=0x7fc13401ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.350+0000 7fc12f7fe640 1 -- 172.21.15.7:0/3448251192 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134028070 con 0x7fc14010d9c0 2024-03-26T08:42:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.350+0000 7fc12f7fe640 1 -- 172.21.15.7:0/3448251192 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc134005020 con 0x7fc14010d9c0 2024-03-26T08:42:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.350+0000 7fc12f7fe640 1 -- 172.21.15.7:0/3448251192 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134004d80 con 0x7fc14010d9c0 2024-03-26T08:42:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.350+0000 7fc144fa9640 1 -- 172.21.15.7:0/3448251192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1401a5f60 con 0x7fc14010d9c0 2024-03-26T08:42:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.350+0000 7fc144fa9640 1 -- 172.21.15.7:0/3448251192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1401a6450 con 0x7fc14010d9c0 2024-03-26T08:42:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.353+0000 7fc12f7fe640 1 -- 172.21.15.7:0/3448251192 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fc1340051d0 con 0x7fc14010d9c0 2024-03-26T08:42:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.353+0000 7fc147b8d740 1 Processor -- start 2024-03-26T08:42:47.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc147b8d740 1 -- start start 2024-03-26T08:42:47.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc12f7fe640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1280a2a10 0x7fc1280a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bd1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bd4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc147b8d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e64b8f610 con 0x564e64bd21f0 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc147b8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e64b323c0 con 0x564e64bcc690 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc147b8d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e64adad60 con 0x564e64bcd080 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc12f7fe640 1 -- 172.21.15.7:0/3448251192 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc1340df420 con 0x7fc14010d9c0 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc13ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bd4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bd1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc13ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bd4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59838/0 (socket says 172.21.15.7:59838) 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33044/0 (socket says 172.21.15.7:33044) 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc13dd9b640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1280a2a10 0x7fc1280a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.354+0000 7fc13e59c640 1 -- 172.21.15.7:0/576042197 learned_addr learned my addr 172.21.15.7:0/576042197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc13e59c640 1 -- 172.21.15.7:0/576042197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 msgr2=0x564e64bd1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc13e59c640 1 --2- 172.21.15.7:0/576042197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bd1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc13e59c640 1 -- 172.21.15.7:0/576042197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 msgr2=0x564e64bd4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc13e59c640 1 --2- 172.21.15.7:0/576042197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bd4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc13e59c640 1 -- 172.21.15.7:0/576042197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e648463f0 con 0x564e64bcc690 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc13ed9d640 1 --2- 172.21.15.7:0/576042197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bd4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc13e59c640 1 --2- 172.21.15.7:0/576042197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bccab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc130037040 tx=0x7fc130032480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=197262cf63a11a3c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc13dd9b640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1280a2a10 0x7fc1280a4ed0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x564e64bcd480 tx=0x7fc1380096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc12cff9640 1 -- 172.21.15.7:0/576042197 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130043020 con 0x564e64bcc690 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc12cff9640 1 -- 172.21.15.7:0/576042197 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc130004040 con 0x564e64bcc690 2024-03-26T08:42:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.355+0000 7fc12cff9640 1 -- 172.21.15.7:0/576042197 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130039040 con 0x564e64bcc690 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 -- 172.21.15.7:0/576042197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 msgr2=0x564e64bccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 --2- 172.21.15.7:0/576042197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bccab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc130037040 tx=0x7fc130032480 comp rx=0 tx=0).stop 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 -- 172.21.15.7:0/576042197 shutdown_connections 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 --2- 172.21.15.7:0/576042197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bd4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 --2- 172.21.15.7:0/576042197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bd1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 --2- 172.21.15.7:0/576042197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bccab0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 -- 172.21.15.7:0/576042197 >> 172.21.15.7:0/576042197 conn(0x564e64bc7c00 msgr2=0x564e64bc8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 -- 172.21.15.7:0/576042197 shutdown_connections 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 -- 172.21.15.7:0/576042197 wait complete. 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 -- start start 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.356+0000 7fc147b8d740 1 Processor -- start 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcc690 0x564e64bd6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcd080 0x564e64bd6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bddef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc147b8d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e64aee240 con 0x564e64bd21f0 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc147b8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e64b2e460 con 0x564e64bcd080 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc147b8d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e64bd5140 con 0x564e64bcc690 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc147b8d740 10 client.0 _ll_register_callbacks cb 0x564e64aedc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcc690 0x564e64bd6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcd080 0x564e64bd6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcc690 0x564e64bd6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50006/0 (socket says 172.21.15.7:50006) 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13e59c640 1 -- 172.21.15.7:0/3281440858 learned_addr learned my addr 172.21.15.7:0/3281440858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13ed9d640 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bddef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13dd9b640 1 -- 172.21.15.7:0/3281440858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcc690 msgr2=0x564e64bd6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13dd9b640 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcc690 0x564e64bd6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13dd9b640 1 -- 172.21.15.7:0/3281440858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 msgr2=0x564e64bddef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13dd9b640 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bddef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13dd9b640 1 -- 172.21.15.7:0/3281440858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e649107d0 con 0x564e64bcd080 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.357+0000 7fc13ed9d640 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bddef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:47.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.358+0000 7fc13dd9b640 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcd080 0x564e64bd6eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc138009980 tx=0x7fc138004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ddcf89b75183234 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.358+0000 7fc125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.358+0000 7fc125ffb640 1 -- 172.21.15.7:0/3281440858 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138014040 con 0x564e64bcd080 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.359+0000 7fc125ffb640 1 -- 172.21.15.7:0/3281440858 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc13801d460 con 0x564e64bcd080 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.359+0000 7fc147b8d740 1 -- 172.21.15.7:0/3281440858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc130004040 con 0x564e64bcd080 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.359+0000 7fc125ffb640 1 -- 172.21.15.7:0/3281440858 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc13800ae20 con 0x564e64bcd080 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.359+0000 7fc147b8d740 10 client.25382 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.359+0000 7fc147b8d740 1 -- 172.21.15.7:0/3281440858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564e64be1320 con 0x564e64bcd080 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.359+0000 7fc125ffb640 1 -- 172.21.15.7:0/3281440858 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc130004040 con 0x564e64bcd080 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc125ffb640 1 -- 172.21.15.7:0/3281440858 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc138024070 con 0x564e64bcd080 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc147b8d740 10 client.25382 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc147b8d740 10 client.25382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc147b8d740 20 client.25382 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564e64be2970 0x564e64be4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc147b8d740 4 client.25382 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc147b8d740 1 -- 172.21.15.7:0/3281440858 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e64b3f900 con 0x564e64be2970 2024-03-26T08:42:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc13e59c640 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564e64be2970 0x564e64be4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc13e59c640 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564e64be2970 0x564e64be4d80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85fbebd2a4813753 server_cookie=83a43af317a67ad in_seq=0 out_seq=0 2024-03-26T08:42:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.360+0000 7fc125ffb640 10 client.25382 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.361+0000 7fc125ffb640 1 -- 172.21.15.7:0/3281440858 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564e64b3f900 con 0x564e64be2970 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.361+0000 7fc125ffb640 10 client.25382 handle_command_reply: tid=0 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 client.25382 shutdown 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 -- 172.21.15.7:0/3281440858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcd080 msgr2=0x564e64bd6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcd080 0x564e64bd6eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc138009980 tx=0x7fc138004a40 comp rx=0 tx=0).stop 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 -- 172.21.15.7:0/3281440858 shutdown_connections 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564e64be2970 0x564e64be4d80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bd21f0 0x564e64bddef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcd080 0x564e64bd6eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3281440858 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcc690 0x564e64bd6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 -- 172.21.15.7:0/3281440858 >> 172.21.15.7:0/3281440858 conn(0x564e64bc7c00 msgr2=0x564e64bc6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc125ffb640 0 client.25382 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 -- 172.21.15.7:0/3281440858 shutdown_connections 2024-03-26T08:42:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 1 -- 172.21.15.7:0/3281440858 wait complete. 2024-03-26T08:42:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.362+0000 7fc147b8d740 20 client.25382 trim_cache size 0 max 16384 2024-03-26T08:42:47.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.380+0000 7fc147b8d740 1 Processor -- start 2024-03-26T08:42:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.380+0000 7fc147b8d740 1 -- start start 2024-03-26T08:42:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.380+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bd6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.380+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bcd080 0x564e64bd6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bd6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bd6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33054/0 (socket says 172.21.15.7:33054) 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bcd080 0x564e64bd6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13e59c640 1 -- 172.21.15.7:0/990801141 learned_addr learned my addr 172.21.15.7:0/990801141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc147b8d740 1 --2- 172.21.15.7:0/990801141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bd21f0 0x564e64bddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc147b8d740 1 -- 172.21.15.7:0/990801141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401b1710 con 0x564e64bcd080 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc147b8d740 1 -- 172.21.15.7:0/990801141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401af210 con 0x564e64bcc690 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc147b8d740 1 -- 172.21.15.7:0/990801141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401ad290 con 0x564e64bd21f0 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13ed9d640 1 --2- 172.21.15.7:0/990801141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bd21f0 0x564e64bddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13e59c640 1 -- 172.21.15.7:0/990801141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bd21f0 msgr2=0x564e64bddeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13e59c640 1 --2- 172.21.15.7:0/990801141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bd21f0 0x564e64bddeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13e59c640 1 -- 172.21.15.7:0/990801141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bcd080 msgr2=0x564e64bd6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13e59c640 1 --2- 172.21.15.7:0/990801141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bcd080 0x564e64bd6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13e59c640 1 -- 172.21.15.7:0/990801141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e64bc7550 con 0x564e64bcc690 2024-03-26T08:42:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.381+0000 7fc13e59c640 1 --2- 172.21.15.7:0/990801141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bd6970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc130000c00 tx=0x7fc130034ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bffec392ddbb939f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc127fff640 1 -- 172.21.15.7:0/990801141 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130058070 con 0x564e64bcc690 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc127fff640 1 -- 172.21.15.7:0/990801141 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc130034640 con 0x564e64bcc690 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc127fff640 1 -- 172.21.15.7:0/990801141 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130039040 con 0x564e64bcc690 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc147b8d740 1 -- 172.21.15.7:0/990801141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 msgr2=0x564e64bd6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc147b8d740 1 --2- 172.21.15.7:0/990801141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bd6970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc130000c00 tx=0x7fc130034ab0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc147b8d740 1 -- 172.21.15.7:0/990801141 shutdown_connections 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc147b8d740 1 --2- 172.21.15.7:0/990801141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bd21f0 0x564e64bddeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc147b8d740 1 --2- 172.21.15.7:0/990801141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bcd080 0x564e64bd6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc147b8d740 1 --2- 172.21.15.7:0/990801141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bcc690 0x564e64bd6970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc147b8d740 1 -- 172.21.15.7:0/990801141 >> 172.21.15.7:0/990801141 conn(0x564e64bc7c00 msgr2=0x564e64bce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.382+0000 7fc147b8d740 1 -- 172.21.15.7:0/990801141 shutdown_connections 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 1 -- 172.21.15.7:0/990801141 wait complete. 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 1 -- start start 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 1 Processor -- start 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bcc690 0x564e64bdb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bdb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bd21f0 0x564e64bdc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401ad890 con 0x564e64bcc690 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401aebb0 con 0x564e64bd21f0 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401adef0 con 0x564e64bcd080 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.383+0000 7fc147b8d740 10 client.0 _ll_register_callbacks cb 0x7fc13800b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.384+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bdb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.384+0000 7fc13dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bdb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50038/0 (socket says 172.21.15.7:50038) 2024-03-26T08:42:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.384+0000 7fc13dd9b640 1 -- 172.21.15.7:0/3957553835 learned_addr learned my addr 172.21.15.7:0/3957553835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:47.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.384+0000 7fc13dd9b640 1 -- 172.21.15.7:0/3957553835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bd21f0 msgr2=0x564e64bdc7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:42:47.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.384+0000 7fc13dd9b640 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bd21f0 0x564e64bdc7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.384+0000 7fc13dd9b640 1 -- 172.21.15.7:0/3957553835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bcc690 msgr2=0x564e64bdb280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.384+0000 7fc13dd9b640 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bcc690 0x564e64bdb280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.384+0000 7fc13dd9b640 1 -- 172.21.15.7:0/3957553835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e64bdcdc0 con 0x564e64bcd080 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.385+0000 7fc13dd9b640 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bdb7c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x564e64bd7e50 tx=0x7fc138009390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0d4bd4a9a6ab78 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.385+0000 7fc125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.385+0000 7fc125ffb640 1 -- 172.21.15.7:0/3957553835 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13801d460 con 0x564e64bcd080 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.385+0000 7fc147b8d740 1 -- 172.21.15.7:0/3957553835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc130034640 con 0x564e64bcd080 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.385+0000 7fc125ffb640 1 -- 172.21.15.7:0/3957553835 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc13801d600 con 0x564e64bcd080 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.385+0000 7fc125ffb640 1 -- 172.21.15.7:0/3957553835 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc13801d800 con 0x564e64bcd080 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.385+0000 7fc147b8d740 10 client.25411 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.385+0000 7fc147b8d740 1 -- 172.21.15.7:0/3957553835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564e64bde160 con 0x564e64bcd080 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.386+0000 7fc125ffb640 1 -- 172.21.15.7:0/3957553835 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc130034640 con 0x564e64bcd080 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.386+0000 7fc125ffb640 1 -- 172.21.15.7:0/3957553835 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc138024040 con 0x564e64bcd080 2024-03-26T08:42:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.386+0000 7fc147b8d740 10 client.25411 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.386+0000 7fc147b8d740 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.386+0000 7fc147b8d740 20 client.25411 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.386+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564e64c3e870 0x564e64c40c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.386+0000 7fc147b8d740 4 client.25411 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.386+0000 7fc147b8d740 1 -- 172.21.15.7:0/3957553835 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e64b39c50 con 0x564e64c3e870 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.387+0000 7fc13e59c640 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564e64c3e870 0x564e64c40c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc125ffb640 10 client.25411 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc13e59c640 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564e64c3e870 0x564e64c40c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c60d32f600f0b0d server_cookie=72850bb22f73352a in_seq=0 out_seq=0 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc125ffb640 1 -- 172.21.15.7:0/3957553835 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e64b39c50 con 0x564e64c3e870 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc125ffb640 10 client.25411 handle_command_reply: tid=0 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 client.25411 shutdown 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 -- 172.21.15.7:0/3957553835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 msgr2=0x564e64bdb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bdb7c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x564e64bd7e50 tx=0x7fc138009390 comp rx=0 tx=0).stop 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 -- 172.21.15.7:0/3957553835 shutdown_connections 2024-03-26T08:42:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564e64c3e870 0x564e64c40c90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e64bd21f0 0x564e64bdc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e64bcd080 0x564e64bdb7c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 --2- 172.21.15.7:0/3957553835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e64bcc690 0x564e64bdb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 -- 172.21.15.7:0/3957553835 >> 172.21.15.7:0/3957553835 conn(0x564e64bc7c00 msgr2=0x564e64bd6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc125ffb640 0 client.25411 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 -- 172.21.15.7:0/3957553835 shutdown_connections 2024-03-26T08:42:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 1 -- 172.21.15.7:0/3957553835 wait complete. 2024-03-26T08:42:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.388+0000 7fc147b8d740 20 client.25411 trim_cache size 0 max 16384 2024-03-26T08:42:47.382 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:42:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 -- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1280a2a10 msgr2=0x7fc1280a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1280a2a10 0x7fc1280a4ed0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x564e64bcd480 tx=0x7fc1380096d0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 -- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc14010d9c0 msgr2=0x7fc1401a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc14010d9c0 0x7fc1401a5600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc13401e920 tx=0x7fc13401ede0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 -- 172.21.15.7:0/3448251192 shutdown_connections 2024-03-26T08:42:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc1280a2a10 0x7fc1280a4ed0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc14010d9c0 0x7fc1401a5600 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc14010b010 0x7fc14019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 --2- 172.21.15.7:0/3448251192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1401086a0 0x7fc14019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.389+0000 7fc144fa9640 1 -- 172.21.15.7:0/3448251192 >> 172.21.15.7:0/3448251192 conn(0x7fc14006e640 msgr2=0x7fc140108e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.391+0000 7fc144fa9640 1 -- 172.21.15.7:0/3448251192 shutdown_connections 2024-03-26T08:42:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:47.391+0000 7fc144fa9640 1 -- 172.21.15.7:0/3448251192 wait complete. 2024-03-26T08:42:48.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:47 smithi016 ceph-mon[35851]: pgmap v371: 97 pgs: 97 active+clean; 940 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-03-26T08:42:48.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:47 smithi007 ceph-mon[32462]: pgmap v371: 97 pgs: 97 active+clean; 940 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-03-26T08:42:48.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:47 smithi044 ceph-mon[36461]: pgmap v371: 97 pgs: 97 active+clean; 940 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 11 MiB/s wr, 754 op/s 2024-03-26T08:42:48.398 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:42:48.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:42:48.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:42:48.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.552+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3904253406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538108eb0 msgr2=0x7fd538109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.552+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3904253406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538108eb0 0x7fd538109330 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd528009ca0 tx=0x7fd52802c100 comp rx=0 tx=0).stop 2024-03-26T08:42:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.553+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3904253406 shutdown_connections 2024-03-26T08:42:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.553+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3904253406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd538109870 0x7fd538110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.553+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3904253406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538108eb0 0x7fd538109330 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.553+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3904253406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538107cc0 0x7fd5381080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.553+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3904253406 >> 172.21.15.7:0/3904253406 conn(0x7fd538076c60 msgr2=0x7fd5380790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.554+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3904253406 shutdown_connections 2024-03-26T08:42:48.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.554+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3904253406 wait complete. 2024-03-26T08:42:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.555+0000 7fd53c9cd640 1 Processor -- start 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd53c9cd640 1 -- start start 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd53c9cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd538107cc0 0x7fd53819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd53c9cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538108eb0 0x7fd53819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd53c9cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538109870 0x7fd5381a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd53c9cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd538113920 con 0x7fd538109870 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd53c9cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381137a0 con 0x7fd538108eb0 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd53c9cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd538113aa0 con 0x7fd538107cc0 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd536d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538109870 0x7fd5381a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd53659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd538107cc0 0x7fd53819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd536d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538109870 0x7fd5381a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59866/0 (socket says 172.21.15.7:59866) 2024-03-26T08:42:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd53659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd538107cc0 0x7fd53819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50054/0 (socket says 172.21.15.7:50054) 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd535d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538108eb0 0x7fd53819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.556+0000 7fd536d9d640 1 -- 172.21.15.7:0/3231683518 learned_addr learned my addr 172.21.15.7:0/3231683518 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd535d9b640 1 -- 172.21.15.7:0/3231683518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd538107cc0 msgr2=0x7fd53819e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd535d9b640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd538107cc0 0x7fd53819e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd535d9b640 1 -- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538109870 msgr2=0x7fd5381a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd535d9b640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538109870 0x7fd5381a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd535d9b640 1 -- 172.21.15.7:0/3231683518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5381a5c40 con 0x7fd538108eb0 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd536d9d640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538109870 0x7fd5381a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd53659c640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd538107cc0 0x7fd53819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd535d9b640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538108eb0 0x7fd53819e560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd528009bf0 tx=0x7fd528035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd5277fe640 1 -- 172.21.15.7:0/3231683518 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528039040 con 0x7fd538108eb0 2024-03-26T08:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd5277fe640 1 -- 172.21.15.7:0/3231683518 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd528003970 con 0x7fd538108eb0 2024-03-26T08:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.557+0000 7fd5277fe640 1 -- 172.21.15.7:0/3231683518 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528039040 con 0x7fd538108eb0 2024-03-26T08:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.558+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3231683518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5381a5ed0 con 0x7fd538108eb0 2024-03-26T08:42:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.558+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3231683518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5381a62e0 con 0x7fd538108eb0 2024-03-26T08:42:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.561+0000 7fd5277fe640 1 -- 172.21.15.7:0/3231683518 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fd528046030 con 0x7fd538108eb0 2024-03-26T08:42:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.561+0000 7fd5277fe640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd5200a2d10 0x7fd5200a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.562+0000 7fd53659c640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd5200a2d10 0x7fd5200a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.562+0000 7fd5277fe640 1 -- 172.21.15.7:0/3231683518 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd528044070 con 0x7fd538108eb0 2024-03-26T08:42:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.562+0000 7fd53f5bc740 1 Processor -- start 2024-03-26T08:42:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.562+0000 7fd53f5bc740 1 -- start start 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcb4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcb5350 0x56093bcb9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd53f5bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093bbf4c90 con 0x56093bcba4c0 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd53f5bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093bc1a070 con 0x56093bcb5350 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd53f5bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093bbc2c80 con 0x56093bcb4960 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd536d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd535d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcb5350 0x56093bcb9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd535d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcb5350 0x56093bcb9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33092/0 (socket says 172.21.15.7:33092) 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd536d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59872/0 (socket says 172.21.15.7:59872) 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd535d9b640 1 -- 172.21.15.7:0/694116740 learned_addr learned my addr 172.21.15.7:0/694116740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd53659c640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd5200a2d10 0x7fd5200a51d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd538108d10 tx=0x7fd52c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd535d9b640 1 -- 172.21.15.7:0/694116740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 msgr2=0x56093bcb4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd535d9b640 1 --2- 172.21.15.7:0/694116740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcb4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd535d9b640 1 -- 172.21.15.7:0/694116740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 msgr2=0x56093bcbc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd535d9b640 1 --2- 172.21.15.7:0/694116740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd535d9b640 1 -- 172.21.15.7:0/694116740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56093bbbad10 con 0x56093bcb5350 2024-03-26T08:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.563+0000 7fd536d9d640 1 --2- 172.21.15.7:0/694116740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.564+0000 7fd535d9b640 1 --2- 172.21.15.7:0/694116740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcb5350 0x56093bcb9e50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5280c4e00 tx=0x7fd528035f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a811e251bfbb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.565+0000 7fd524ff9640 1 -- 172.21.15.7:0/694116740 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5280bcc20 con 0x56093bcb5350 2024-03-26T08:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.565+0000 7fd524ff9640 1 -- 172.21.15.7:0/694116740 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5280c0d70 con 0x56093bcb5350 2024-03-26T08:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.565+0000 7fd524ff9640 1 -- 172.21.15.7:0/694116740 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5280ee000 con 0x56093bcb5350 2024-03-26T08:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.566+0000 7fd53f5bc740 1 -- 172.21.15.7:0/694116740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcb5350 msgr2=0x56093bcb9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.566+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/694116740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcb5350 0x56093bcb9e50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5280c4e00 tx=0x7fd528035f00 comp rx=0 tx=0).stop 2024-03-26T08:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.566+0000 7fd53f5bc740 1 -- 172.21.15.7:0/694116740 shutdown_connections 2024-03-26T08:42:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.566+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/694116740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.566+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/694116740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcb5350 0x56093bcb9e50 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5280c4e00 tx=0x7fd528035f00 comp rx=0 tx=0).stop 2024-03-26T08:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.566+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/694116740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcb4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.566+0000 7fd53f5bc740 1 -- 172.21.15.7:0/694116740 >> 172.21.15.7:0/694116740 conn(0x56093bc7d1a0 msgr2=0x56093bcb02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.567+0000 7fd53f5bc740 1 -- 172.21.15.7:0/694116740 shutdown_connections 2024-03-26T08:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.567+0000 7fd53f5bc740 1 -- 172.21.15.7:0/694116740 wait complete. 2024-03-26T08:42:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.567+0000 7fd53f5bc740 1 -- start start 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.567+0000 7fd53f5bc740 1 Processor -- start 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcc4200 0x56093bcc65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53f5bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093bbd60d0 con 0x56093bcba4c0 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53f5bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093bc16110 con 0x56093bcc4200 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53f5bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093bcbd230 con 0x56093bcb4960 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53f5bc740 10 client.0 _ll_register_callbacks cb 0x56093bc11e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50062/0 (socket says 172.21.15.7:50062) 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd53659c640 1 -- 172.21.15.7:0/361245856 learned_addr learned my addr 172.21.15.7:0/361245856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd536d9d640 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcc4200 0x56093bcc65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.568+0000 7fd535d9b640 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd536d9d640 1 -- 172.21.15.7:0/361245856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 msgr2=0x56093bcbec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd536d9d640 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd536d9d640 1 -- 172.21.15.7:0/361245856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 msgr2=0x56093bcbf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd536d9d640 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd536d9d640 1 -- 172.21.15.7:0/361245856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56093b9f67e0 con 0x56093bcc4200 2024-03-26T08:42:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd535d9b640 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd53659c640 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd536d9d640 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcc4200 0x56093bcc65f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd530012cf0 tx=0x7fd53000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bebf885b2637b90 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd51dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd51dffb640 1 -- 172.21.15.7:0/361245856 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5300120d0 con 0x56093bcc4200 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd53f5bc740 1 -- 172.21.15.7:0/361245856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5280c0d70 con 0x56093bcc4200 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.569+0000 7fd51dffb640 1 -- 172.21.15.7:0/361245856 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd530005020 con 0x56093bcc4200 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd51dffb640 1 -- 172.21.15.7:0/361245856 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd530012270 con 0x56093bcc4200 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd53f5bc740 10 client.25397 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd53f5bc740 1 -- 172.21.15.7:0/361245856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56093ba56f70 con 0x56093bcc4200 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd51dffb640 1 -- 172.21.15.7:0/361245856 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5280c0d70 con 0x56093bcc4200 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd51dffb640 1 -- 172.21.15.7:0/361245856 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd53001c470 con 0x56093bcc4200 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd53f5bc740 10 client.25397 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd53f5bc740 10 client.25397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd53f5bc740 20 client.25397 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56093bccae60 0x56093bccd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd53f5bc740 4 client.25397 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.570+0000 7fd53f5bc740 1 -- 172.21.15.7:0/361245856 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56093bc275b0 con 0x56093bccae60 2024-03-26T08:42:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.571+0000 7fd53659c640 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56093bccae60 0x56093bccd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.571+0000 7fd53659c640 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56093bccae60 0x56093bccd280 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c171e91c353a0d02 server_cookie=5b33cfa1603b0d55 in_seq=0 out_seq=0 2024-03-26T08:42:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.571+0000 7fd51dffb640 10 client.25397 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.572+0000 7fd51dffb640 1 -- 172.21.15.7:0/361245856 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56093bc275b0 con 0x56093bccae60 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.572+0000 7fd51dffb640 10 client.25397 handle_command_reply: tid=0 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.572+0000 7fd53f5bc740 1 client.25397 shutdown 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.572+0000 7fd53f5bc740 1 -- 172.21.15.7:0/361245856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcc4200 msgr2=0x56093bcc65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.572+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcc4200 0x56093bcc65f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd530012cf0 tx=0x7fd53000f790 comp rx=0 tx=0).stop 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd53f5bc740 1 -- 172.21.15.7:0/361245856 shutdown_connections 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56093bccae60 0x56093bccd280 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcc4200 0x56093bcc65f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcba4c0 0x56093bcbf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/361245856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd53f5bc740 1 -- 172.21.15.7:0/361245856 >> 172.21.15.7:0/361245856 conn(0x56093bc7d1a0 msgr2=0x56093bcaf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd51dffb640 0 client.25397 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:48.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd53f5bc740 1 -- 172.21.15.7:0/361245856 shutdown_connections 2024-03-26T08:42:48.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd53f5bc740 1 -- 172.21.15.7:0/361245856 wait complete. 2024-03-26T08:42:48.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.573+0000 7fd53f5bc740 20 client.25397 trim_cache size 0 max 16384 2024-03-26T08:42:48.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.605+0000 7fd53f5bc740 1 Processor -- start 2024-03-26T08:42:48.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.606+0000 7fd53f5bc740 1 -- start start 2024-03-26T08:42:48.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.606+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.606+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 0x56093bcb3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.606+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcc4200 0x56093bcb3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.606+0000 7fd53f5bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381b1650 con 0x56093bcc4200 2024-03-26T08:42:48.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.606+0000 7fd53f5bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381af150 con 0x56093bcba4c0 2024-03-26T08:42:48.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.606+0000 7fd53f5bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381ad1d0 con 0x56093bcb4960 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd53659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd53659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50078/0 (socket says 172.21.15.7:50078) 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd53659c640 1 -- 172.21.15.7:0/3514301084 learned_addr learned my addr 172.21.15.7:0/3514301084 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd535d9b640 1 --2- 172.21.15.7:0/3514301084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 0x56093bcb3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd53659c640 1 -- 172.21.15.7:0/3514301084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 msgr2=0x56093bcb3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd53659c640 1 --2- 172.21.15.7:0/3514301084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 0x56093bcb3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd53659c640 1 -- 172.21.15.7:0/3514301084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcc4200 msgr2=0x56093bcb3a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd53659c640 1 --2- 172.21.15.7:0/3514301084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcc4200 0x56093bcb3a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd53659c640 1 -- 172.21.15.7:0/3514301084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56093b8ee170 con 0x56093bcb4960 2024-03-26T08:42:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd53659c640 1 --2- 172.21.15.7:0/3514301084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbdc70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd52c004600 tx=0x7fd52c013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbdb6a16d4b9e1a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.607+0000 7fd51ffff640 1 -- 172.21.15.7:0/3514301084 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c029230 con 0x56093bcb4960 2024-03-26T08:42:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.608+0000 7fd51ffff640 1 -- 172.21.15.7:0/3514301084 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd52c02d030 con 0x56093bcb4960 2024-03-26T08:42:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.608+0000 7fd51ffff640 1 -- 172.21.15.7:0/3514301084 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0035f0 con 0x56093bcb4960 2024-03-26T08:42:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.609+0000 7fd53f5bc740 1 -- 172.21.15.7:0/3514301084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 msgr2=0x56093bcbdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.609+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/3514301084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbdc70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd52c004600 tx=0x7fd52c013040 comp rx=0 tx=0).stop 2024-03-26T08:42:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.609+0000 7fd53f5bc740 1 -- 172.21.15.7:0/3514301084 shutdown_connections 2024-03-26T08:42:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.609+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/3514301084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcc4200 0x56093bcb3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.609+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/3514301084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 0x56093bcb3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.609+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/3514301084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcbdc70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.609+0000 7fd53f5bc740 1 -- 172.21.15.7:0/3514301084 >> 172.21.15.7:0/3514301084 conn(0x56093bc7d1a0 msgr2=0x56093bcbd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.609+0000 7fd53f5bc740 1 -- 172.21.15.7:0/3514301084 shutdown_connections 2024-03-26T08:42:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.609+0000 7fd53f5bc740 1 -- 172.21.15.7:0/3514301084 wait complete. 2024-03-26T08:42:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.610+0000 7fd53f5bc740 1 -- start start 2024-03-26T08:42:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.610+0000 7fd53f5bc740 1 Processor -- start 2024-03-26T08:42:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcdbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd53f5bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 0x56093bcdc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd53659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcdbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd53659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcdbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50082/0 (socket says 172.21.15.7:50082) 2024-03-26T08:42:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd53659c640 1 -- 172.21.15.7:0/1287112715 learned_addr learned my addr 172.21.15.7:0/1287112715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcc4200 0x56093bcdc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381ad7d0 con 0x56093bcc4200 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd535d9b640 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 0x56093bcdc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381aeaf0 con 0x56093bcba4c0 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.611+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381ade30 con 0x56093bcb4960 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.612+0000 7fd536d9d640 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcc4200 0x56093bcdc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.612+0000 7fd53659c640 1 -- 172.21.15.7:0/1287112715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 msgr2=0x56093bcdc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.612+0000 7fd53659c640 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 0x56093bcdc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.612+0000 7fd53f5bc740 10 client.0 _ll_register_callbacks cb 0x7fd530009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.612+0000 7fd53659c640 1 -- 172.21.15.7:0/1287112715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcc4200 msgr2=0x56093bcdc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.612+0000 7fd53659c640 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcc4200 0x56093bcdc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.612+0000 7fd53659c640 1 -- 172.21.15.7:0/1287112715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56093bd25d80 con 0x56093bcb4960 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.612+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd52c02d030 con 0x56093bcb4960 2024-03-26T08:42:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.612+0000 7fd53659c640 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcdbdc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009e80 tx=0x7fd52c029930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a170f95d7a49fc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.613+0000 7fd51dffb640 10 client.25426 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.613+0000 7fd51dffb640 1 -- 172.21.15.7:0/1287112715 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c040050 con 0x56093bcb4960 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.613+0000 7fd51dffb640 1 -- 172.21.15.7:0/1287112715 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd52c029200 con 0x56093bcb4960 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.613+0000 7fd51dffb640 1 -- 172.21.15.7:0/1287112715 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd52c036030 con 0x56093bcb4960 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.613+0000 7fd53f5bc740 10 client.25426 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.613+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x56093bd27060 con 0x56093bcb4960 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.614+0000 7fd51dffb640 1 -- 172.21.15.7:0/1287112715 <== mon.1 v2:172.21.15.44:3300/0 4 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd52c005040 con 0x56093bcb4960 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.614+0000 7fd53f5bc740 10 client.25426 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.614+0000 7fd53f5bc740 10 client.25426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:48.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.614+0000 7fd53f5bc740 20 client.25426 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.614+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56093bd277b0 0x56093bd29bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.614+0000 7fd53f5bc740 4 client.25426 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:42:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.614+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56093bc21900 con 0x56093bd277b0 2024-03-26T08:42:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.615+0000 7fd535d9b640 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56093bd277b0 0x56093bd29bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.616+0000 7fd51dffb640 10 client.25426 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:48.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.616+0000 7fd535d9b640 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56093bd277b0 0x56093bd29bd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0618e2b67fcc81 server_cookie=e1371686ca770f87 in_seq=0 out_seq=0 2024-03-26T08:42:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.617+0000 7fd51dffb640 1 -- 172.21.15.7:0/1287112715 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56093bc21900 con 0x56093bd277b0 2024-03-26T08:42:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.617+0000 7fd51dffb640 10 client.25426 handle_command_reply: tid=0 2024-03-26T08:42:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.618+0000 7fd53f5bc740 1 client.25426 shutdown 2024-03-26T08:42:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.618+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 msgr2=0x56093bcdbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.618+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcdbdc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd52c009e80 tx=0x7fd52c029930 comp rx=0 tx=0).stop 2024-03-26T08:42:48.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.618+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 shutdown_connections 2024-03-26T08:42:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.618+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56093bd277b0 0x56093bd29bd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.618+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56093bcc4200 0x56093bcdc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.618+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56093bcba4c0 0x56093bcdc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.618+0000 7fd53f5bc740 1 --2- 172.21.15.7:0/1287112715 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56093bcb4960 0x56093bcdbdc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.619+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 >> 172.21.15.7:0/1287112715 conn(0x56093bc7d1a0 msgr2=0x56093bcb3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.619+0000 7fd51dffb640 0 client.25426 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.619+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 shutdown_connections 2024-03-26T08:42:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.620+0000 7fd53f5bc740 1 -- 172.21.15.7:0/1287112715 wait complete. 2024-03-26T08:42:48.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.620+0000 7fd53f5bc740 20 client.25426 trim_cache size 0 max 16384 2024-03-26T08:42:48.612 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:42:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:42:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "5c5c842d-c6c4-4692-b1dc-a0930ba01d5d", 2024-03-26T08:42:48.612 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:42:48.612 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd5200a2d10 msgr2=0x7fd5200a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd5200a2d10 0x7fd5200a51d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd538108d10 tx=0x7fd52c005f90 comp rx=0 tx=0).stop 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3231683518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538108eb0 msgr2=0x7fd53819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538108eb0 0x7fd53819e560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd528009bf0 tx=0x7fd528035d10 comp rx=0 tx=0).stop 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3231683518 shutdown_connections 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd5200a2d10 0x7fd5200a51d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd538109870 0x7fd5381a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd538108eb0 0x7fd53819e560 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 --2- 172.21.15.7:0/3231683518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd538107cc0 0x7fd53819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3231683518 >> 172.21.15.7:0/3231683518 conn(0x7fd538076c60 msgr2=0x7fd53810c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3231683518 shutdown_connections 2024-03-26T08:42:48.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.621+0000 7fd53c9cd640 1 -- 172.21.15.7:0/3231683518 wait complete. 2024-03-26T08:42:48.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:42:48.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:42:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.818+0000 7f44c9c85640 1 -- 172.21.15.7:0/3979405712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c40fff30 msgr2=0x7f44c4102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.818+0000 7f44c9c85640 1 --2- 172.21.15.7:0/3979405712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c40fff30 0x7f44c4102360 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b402bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.818+0000 7f44c9c85640 1 -- 172.21.15.7:0/3979405712 shutdown_connections 2024-03-26T08:42:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.818+0000 7f44c9c85640 1 --2- 172.21.15.7:0/3979405712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c4105250 0x7f44c41056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.819+0000 7f44c9c85640 1 --2- 172.21.15.7:0/3979405712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c41028a0 0x7f44c4104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.819+0000 7f44c9c85640 1 --2- 172.21.15.7:0/3979405712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c40fff30 0x7f44c4102360 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.819+0000 7f44c9c85640 1 -- 172.21.15.7:0/3979405712 >> 172.21.15.7:0/3979405712 conn(0x7f44c40fbb30 msgr2=0x7f44c40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.819+0000 7f44c9c85640 1 -- 172.21.15.7:0/3979405712 shutdown_connections 2024-03-26T08:42:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.819+0000 7f44c9c85640 1 -- 172.21.15.7:0/3979405712 wait complete. 2024-03-26T08:42:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.820+0000 7f44c9c85640 1 Processor -- start 2024-03-26T08:42:48.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.820+0000 7f44c9c85640 1 -- start start 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.820+0000 7f44c9c85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c40fff30 0x7f44c419e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.820+0000 7f44c9c85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c41028a0 0x7f44c419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.820+0000 7f44c9c85640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c4105250 0x7f44c41a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.820+0000 7f44c9c85640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c4113980 con 0x7f44c40fff30 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.820+0000 7f44c9c85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c4113800 con 0x7f44c41028a0 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.820+0000 7f44c9c85640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c4113b00 con 0x7f44c4105250 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c41028a0 0x7f44c419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c4105250 0x7f44c41a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c4105250 0x7f44c41a55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50088/0 (socket says 172.21.15.7:50088) 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c3fff640 1 -- 172.21.15.7:0/248405938 learned_addr learned my addr 172.21.15.7:0/248405938 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c41028a0 0x7f44c419e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33146/0 (socket says 172.21.15.7:33146) 2024-03-26T08:42:48.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c37fe640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c40fff30 0x7f44c419e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c3fff640 1 -- 172.21.15.7:0/248405938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c41028a0 msgr2=0x7f44c419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c3fff640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c41028a0 0x7f44c419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c3fff640 1 -- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c40fff30 msgr2=0x7f44c419e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c3fff640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c40fff30 0x7f44c419e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c3fff640 1 -- 172.21.15.7:0/248405938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c41a5c20 con 0x7f44c4105250 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c41028a0 0x7f44c419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.821+0000 7f44c3fff640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c4105250 0x7f44c41a55b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00eb10 tx=0x7f44bc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.822+0000 7f44c0ff9640 1 -- 172.21.15.7:0/248405938 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc004b20 con 0x7f44c4105250 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.822+0000 7f44c9c85640 1 -- 172.21.15.7:0/248405938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c41a5f10 con 0x7f44c4105250 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.822+0000 7f44c9c85640 1 -- 172.21.15.7:0/248405938 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c41a63d0 con 0x7f44c4105250 2024-03-26T08:42:48.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.822+0000 7f44c0ff9640 1 -- 172.21.15.7:0/248405938 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44bc004cc0 con 0x7f44c4105250 2024-03-26T08:42:48.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.822+0000 7f44c0ff9640 1 -- 172.21.15.7:0/248405938 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc010760 con 0x7f44c4105250 2024-03-26T08:42:48.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.825+0000 7f44c0ff9640 1 -- 172.21.15.7:0/248405938 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f44bc010900 con 0x7f44c4105250 2024-03-26T08:42:48.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.826+0000 7f44cc875740 1 Processor -- start 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.826+0000 7f44cc875740 1 -- start start 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44cc875740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa3071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44cc875740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa30c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44cc875740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 0x5614aa30ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44cc875740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aa26c3d0 con 0x5614aa307790 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44cc875740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aa214ad0 con 0x5614aa30c900 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44cc875740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aa268470 con 0x5614aa306da0 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa3071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa3071c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50092/0 (socket says 172.21.15.7:50092) 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c2ffd640 1 -- 172.21.15.7:0/1436378494 learned_addr learned my addr 172.21.15.7:0/1436378494 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c37fe640 1 --2- 172.21.15.7:0/1436378494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa30c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c0ff9640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44ac0a2950 0x7f44ac0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/1436378494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 0x5614aa30ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c2ffd640 1 -- 172.21.15.7:0/1436378494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 msgr2=0x5614aa30ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/1436378494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 0x5614aa30ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c2ffd640 1 -- 172.21.15.7:0/1436378494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 msgr2=0x5614aa30c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/1436378494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa30c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c2ffd640 1 -- 172.21.15.7:0/1436378494 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614aa04a7d0 con 0x5614aa306da0 2024-03-26T08:42:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.827+0000 7f44c0ff9640 1 -- 172.21.15.7:0/248405938 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44bc0cf150 con 0x7f44c4105250 2024-03-26T08:42:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.828+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/1436378494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa3071c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f44b8013c10 tx=0x7f44b8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3504f2b65ba245f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.828+0000 7f44c3fff640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44ac0a2950 0x7f44ac0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.828+0000 7f44a9ffb640 1 -- 172.21.15.7:0/1436378494 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8013d00 con 0x5614aa306da0 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.828+0000 7f44a9ffb640 1 -- 172.21.15.7:0/1436378494 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44b8013120 con 0x5614aa306da0 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.829+0000 7f44a9ffb640 1 -- 172.21.15.7:0/1436378494 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b8003910 con 0x5614aa306da0 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.829+0000 7f44cc875740 1 -- 172.21.15.7:0/1436378494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 msgr2=0x5614aa3071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.829+0000 7f44cc875740 1 --2- 172.21.15.7:0/1436378494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa3071c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f44b8013c10 tx=0x7f44b8009b30 comp rx=0 tx=0).stop 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.829+0000 7f44c3fff640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44ac0a2950 0x7f44ac0a4e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005ba0 tx=0x7f44bc015000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.829+0000 7f44cc875740 1 -- 172.21.15.7:0/1436378494 shutdown_connections 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.829+0000 7f44cc875740 1 --2- 172.21.15.7:0/1436378494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 0x5614aa30ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.829+0000 7f44cc875740 1 --2- 172.21.15.7:0/1436378494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa30c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.829+0000 7f44cc875740 1 --2- 172.21.15.7:0/1436378494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa3071c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.829+0000 7f44cc875740 1 -- 172.21.15.7:0/1436378494 >> 172.21.15.7:0/1436378494 conn(0x5614aa302310 msgr2=0x5614aa302710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.830+0000 7f44cc875740 1 -- 172.21.15.7:0/1436378494 shutdown_connections 2024-03-26T08:42:48.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.830+0000 7f44cc875740 1 -- 172.21.15.7:0/1436378494 wait complete. 2024-03-26T08:42:48.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.830+0000 7f44cc875740 1 -- start start 2024-03-26T08:42:48.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.830+0000 7f44cc875740 1 Processor -- start 2024-03-26T08:42:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44cc875740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa306da0 0x5614aa3111d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44cc875740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa307790 0x5614aa311710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44cc875740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 0x5614aa318750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44cc875740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aa30f8a0 con 0x5614aa306da0 2024-03-26T08:42:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44cc875740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aa228060 con 0x5614aa30c900 2024-03-26T08:42:48.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44cc875740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614aa30fa20 con 0x5614aa307790 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44cc875740 10 client.0 _ll_register_callbacks cb 0x5614aa20a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa307790 0x5614aa311710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa306da0 0x5614aa3111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa307790 0x5614aa311710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50096/0 (socket says 172.21.15.7:50096) 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c37fe640 1 -- 172.21.15.7:0/470845828 learned_addr learned my addr 172.21.15.7:0/470845828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 0x5614aa318750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c37fe640 1 -- 172.21.15.7:0/470845828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 msgr2=0x5614aa318750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c37fe640 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 0x5614aa318750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c37fe640 1 -- 172.21.15.7:0/470845828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa306da0 msgr2=0x5614aa3111d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c37fe640 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa306da0 0x5614aa3111d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c37fe640 1 -- 172.21.15.7:0/470845828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614a9f803f0 con 0x5614aa307790 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.832+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa306da0 0x5614aa3111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.833+0000 7f44c37fe640 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa307790 0x5614aa311710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=224c1b8d5c19c43c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.833+0000 7f44a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:42:48.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.833+0000 7f44a2ffd640 1 -- 172.21.15.7:0/470845828 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4005230 con 0x5614aa307790 2024-03-26T08:42:48.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.833+0000 7f44a2ffd640 1 -- 172.21.15.7:0/470845828 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44b40053d0 con 0x5614aa307790 2024-03-26T08:42:48.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.833+0000 7f44cc875740 1 -- 172.21.15.7:0/470845828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44b8013120 con 0x5614aa307790 2024-03-26T08:42:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.834+0000 7f44a2ffd640 1 -- 172.21.15.7:0/470845828 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44b40055d0 con 0x5614aa307790 2024-03-26T08:42:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.834+0000 7f44cc875740 10 client.25438 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.834+0000 7f44cc875740 1 -- 172.21.15.7:0/470845828 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5614a9f400f0 con 0x5614aa307790 2024-03-26T08:42:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.834+0000 7f44a2ffd640 1 -- 172.21.15.7:0/470845828 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44b8013120 con 0x5614aa307790 2024-03-26T08:42:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.834+0000 7f44a2ffd640 1 -- 172.21.15.7:0/470845828 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f44b4003680 con 0x5614aa307790 2024-03-26T08:42:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.834+0000 7f44cc875740 10 client.25438 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.834+0000 7f44cc875740 10 client.25438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.834+0000 7f44cc875740 20 client.25438 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.835+0000 7f44cc875740 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5614aa31cd20 0x5614aa31f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.835+0000 7f44cc875740 4 client.25438 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.835+0000 7f44cc875740 1 -- 172.21.15.7:0/470845828 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614aa279910 con 0x5614aa31cd20 2024-03-26T08:42:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.835+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5614aa31cd20 0x5614aa31f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.835+0000 7f44a2ffd640 10 client.25438 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.835+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5614aa31cd20 0x5614aa31f160 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60dc48aab91bfb5 server_cookie=77e04507ea28cf85 in_seq=0 out_seq=0 2024-03-26T08:42:48.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.836+0000 7f44a2ffd640 1 -- 172.21.15.7:0/470845828 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5614aa279910 con 0x5614aa31cd20 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.836+0000 7f44a2ffd640 10 client.25438 handle_command_reply: tid=0 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44cc875740 1 client.25438 shutdown 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44cc875740 1 -- 172.21.15.7:0/470845828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa307790 msgr2=0x5614aa311710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44cc875740 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa307790 0x5614aa311710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b4004270 comp rx=0 tx=0).stop 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44cc875740 1 -- 172.21.15.7:0/470845828 shutdown_connections 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44cc875740 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5614aa31cd20 0x5614aa31f160 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44cc875740 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa30c900 0x5614aa318750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44cc875740 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa307790 0x5614aa311710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44cc875740 1 --2- 172.21.15.7:0/470845828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa306da0 0x5614aa3111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44cc875740 1 -- 172.21.15.7:0/470845828 >> 172.21.15.7:0/470845828 conn(0x5614aa302310 msgr2=0x5614aa301530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.837+0000 7f44a2ffd640 0 client.25438 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.838+0000 7f44cc875740 1 -- 172.21.15.7:0/470845828 shutdown_connections 2024-03-26T08:42:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.839+0000 7f44cc875740 1 -- 172.21.15.7:0/470845828 wait complete. 2024-03-26T08:42:48.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.839+0000 7f44cc875740 20 client.25438 trim_cache size 0 max 16384 2024-03-26T08:42:48.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.866+0000 7f44cc875740 1 Processor -- start 2024-03-26T08:42:48.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.866+0000 7f44cc875740 1 -- start start 2024-03-26T08:42:48.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44cc875740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa306da0 0x5614aa305c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44cc875740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa318710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44cc875740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa30c900 0x5614aa323a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44cc875740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b00108b0 con 0x5614aa307790 2024-03-26T08:42:48.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44cc875740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b000e3b0 con 0x5614aa306da0 2024-03-26T08:42:48.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44cc875740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b000c430 con 0x5614aa30c900 2024-03-26T08:42:48.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa318710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa318710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59910/0 (socket says 172.21.15.7:59910) 2024-03-26T08:42:48.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44c37fe640 1 -- 172.21.15.7:0/4277041579 learned_addr learned my addr 172.21.15.7:0/4277041579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/4277041579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa306da0 0x5614aa305c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.867+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/4277041579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa30c900 0x5614aa323a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.868+0000 7f44c37fe640 1 -- 172.21.15.7:0/4277041579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa30c900 msgr2=0x5614aa323a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.868+0000 7f44c37fe640 1 --2- 172.21.15.7:0/4277041579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa30c900 0x5614aa323a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.868+0000 7f44c37fe640 1 -- 172.21.15.7:0/4277041579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa306da0 msgr2=0x5614aa305c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.868+0000 7f44c37fe640 1 --2- 172.21.15.7:0/4277041579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa306da0 0x5614aa305c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.868+0000 7f44c37fe640 1 -- 172.21.15.7:0/4277041579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614aa312070 con 0x5614aa307790 2024-03-26T08:42:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.868+0000 7f44c37fe640 1 --2- 172.21.15.7:0/4277041579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa318710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f44b4003b90 tx=0x7f44b4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c9d63d34f9ceb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.869+0000 7f44a97fa640 1 -- 172.21.15.7:0/4277041579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4009520 con 0x5614aa307790 2024-03-26T08:42:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.869+0000 7f44a97fa640 1 -- 172.21.15.7:0/4277041579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44b4057070 con 0x5614aa307790 2024-03-26T08:42:48.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.869+0000 7f44a97fa640 1 -- 172.21.15.7:0/4277041579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4034a10 con 0x5614aa307790 2024-03-26T08:42:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.870+0000 7f44cc875740 1 -- 172.21.15.7:0/4277041579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 msgr2=0x5614aa318710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.870+0000 7f44cc875740 1 --2- 172.21.15.7:0/4277041579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa318710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f44b4003b90 tx=0x7f44b4009800 comp rx=0 tx=0).stop 2024-03-26T08:42:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.870+0000 7f44cc875740 1 -- 172.21.15.7:0/4277041579 shutdown_connections 2024-03-26T08:42:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.870+0000 7f44cc875740 1 --2- 172.21.15.7:0/4277041579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa30c900 0x5614aa323a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.870+0000 7f44cc875740 1 --2- 172.21.15.7:0/4277041579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa307790 0x5614aa318710 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.870+0000 7f44cc875740 1 --2- 172.21.15.7:0/4277041579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa306da0 0x5614aa305c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.870+0000 7f44cc875740 1 -- 172.21.15.7:0/4277041579 >> 172.21.15.7:0/4277041579 conn(0x5614aa302310 msgr2=0x5614aa305570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.871+0000 7f44cc875740 1 -- 172.21.15.7:0/4277041579 shutdown_connections 2024-03-26T08:42:48.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.871+0000 7f44cc875740 1 -- 172.21.15.7:0/4277041579 wait complete. 2024-03-26T08:42:48.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.871+0000 7f44cc875740 1 -- start start 2024-03-26T08:42:48.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.871+0000 7f44cc875740 1 Processor -- start 2024-03-26T08:42:48.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.872+0000 7f44cc875740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa316750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.872+0000 7f44cc875740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa307790 0x5614aa316c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.872+0000 7f44cc875740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa30c900 0x5614aa317ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.872+0000 7f44cc875740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b000ca30 con 0x5614aa30c900 2024-03-26T08:42:48.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.872+0000 7f44cc875740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b000dd50 con 0x5614aa307790 2024-03-26T08:42:48.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.872+0000 7f44cc875740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b000d090 con 0x5614aa306da0 2024-03-26T08:42:48.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.872+0000 7f44cc875740 10 client.0 _ll_register_callbacks cb 0x7f44b8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:48.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.873+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa307790 0x5614aa316c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.873+0000 7f44c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa307790 0x5614aa316c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33192/0 (socket says 172.21.15.7:33192) 2024-03-26T08:42:48.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.873+0000 7f44c37fe640 1 -- 172.21.15.7:0/2833448719 learned_addr learned my addr 172.21.15.7:0/2833448719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:48.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.873+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa316750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.873+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa30c900 0x5614aa317ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44c37fe640 1 -- 172.21.15.7:0/2833448719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 msgr2=0x5614aa316750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44c37fe640 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa316750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44c37fe640 1 -- 172.21.15.7:0/2833448719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa30c900 msgr2=0x5614aa317ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:48.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44c37fe640 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa30c900 0x5614aa317ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44c37fe640 1 -- 172.21.15.7:0/2833448719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614aa301e50 con 0x5614aa307790 2024-03-26T08:42:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa316750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44c37fe640 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa307790 0x5614aa316c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005ab0 tx=0x7f44b402c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bda595908d71f6b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:42:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44a2ffd640 1 -- 172.21.15.7:0/2833448719 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4061020 con 0x5614aa307790 2024-03-26T08:42:48.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44a2ffd640 1 -- 172.21.15.7:0/2833448719 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44b40346e0 con 0x5614aa307790 2024-03-26T08:42:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.874+0000 7f44cc875740 1 -- 172.21.15.7:0/2833448719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44b4057070 con 0x5614aa307790 2024-03-26T08:42:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44a2ffd640 1 -- 172.21.15.7:0/2833448719 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44b4009520 con 0x5614aa307790 2024-03-26T08:42:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44cc875740 10 client.25421 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44cc875740 1 -- 172.21.15.7:0/2833448719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5614aa378cc0 con 0x5614aa307790 2024-03-26T08:42:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44a2ffd640 1 -- 172.21.15.7:0/2833448719 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44b4057070 con 0x5614aa307790 2024-03-26T08:42:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44a2ffd640 1 -- 172.21.15.7:0/2833448719 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f44b4053910 con 0x5614aa307790 2024-03-26T08:42:48.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44cc875740 10 client.25421 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44cc875740 10 client.25421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44cc875740 20 client.25421 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44cc875740 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5614aa3794e0 0x5614aa37b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44cc875740 4 client.25421 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:42:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.875+0000 7f44cc875740 1 -- 172.21.15.7:0/2833448719 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614aa273c60 con 0x5614aa3794e0 2024-03-26T08:42:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.876+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5614aa3794e0 0x5614aa37b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.876+0000 7f44a2ffd640 10 client.25421 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.876+0000 7f44c2ffd640 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5614aa3794e0 0x5614aa37b900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914e7872a853c930 server_cookie=70934c2eea91ce4e in_seq=0 out_seq=0 2024-03-26T08:42:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.876+0000 7f44a2ffd640 1 -- 172.21.15.7:0/2833448719 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5614aa273c60 con 0x5614aa3794e0 2024-03-26T08:42:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.876+0000 7f44a2ffd640 10 client.25421 handle_command_reply: tid=0 2024-03-26T08:42:48.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.876+0000 7f44cc875740 1 client.25421 shutdown 2024-03-26T08:42:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.877+0000 7f44cc875740 1 -- 172.21.15.7:0/2833448719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa307790 msgr2=0x5614aa316c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.877+0000 7f44cc875740 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa307790 0x5614aa316c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005ab0 tx=0x7f44b402c880 comp rx=0 tx=0).stop 2024-03-26T08:42:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44cc875740 1 -- 172.21.15.7:0/2833448719 shutdown_connections 2024-03-26T08:42:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44cc875740 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5614aa3794e0 0x5614aa37b900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44cc875740 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614aa30c900 0x5614aa317ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44a2ffd640 0 client.25421 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:48.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44cc875740 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614aa307790 0x5614aa316c90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44cc875740 1 --2- 172.21.15.7:0/2833448719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614aa306da0 0x5614aa316750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44cc875740 1 -- 172.21.15.7:0/2833448719 >> 172.21.15.7:0/2833448719 conn(0x5614aa302310 msgr2=0x5614aa305e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44cc875740 1 -- 172.21.15.7:0/2833448719 shutdown_connections 2024-03-26T08:42:48.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44cc875740 1 -- 172.21.15.7:0/2833448719 wait complete. 2024-03-26T08:42:48.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.878+0000 7f44cc875740 20 client.25421 trim_cache size 0 max 16384 2024-03-26T08:42:48.882 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:42:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:42:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:42:48.882 INFO:teuthology.orchestra.run.smithi007.stdout: "5c5c842d-c6c4-4692-b1dc-a0930ba01d5d": { 2024-03-26T08:42:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:42:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "5c5c842d-c6c4-4692-b1dc-a0930ba01d5d", 2024-03-26T08:42:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:42:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:42:48.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:42:48.883 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:42:48.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 -- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44ac0a2950 msgr2=0x7f44ac0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44ac0a2950 0x7f44ac0a4e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f44bc005ba0 tx=0x7f44bc015000 comp rx=0 tx=0).stop 2024-03-26T08:42:48.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 -- 172.21.15.7:0/248405938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c4105250 msgr2=0x7f44c41a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:48.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c4105250 0x7f44c41a55b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00eb10 tx=0x7f44bc00c2d0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 -- 172.21.15.7:0/248405938 shutdown_connections 2024-03-26T08:42:48.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44ac0a2950 0x7f44ac0a4e10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44c4105250 0x7f44c41a55b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c41028a0 0x7f44c419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 --2- 172.21.15.7:0/248405938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44c40fff30 0x7f44c419e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:48.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.880+0000 7f44c9c85640 1 -- 172.21.15.7:0/248405938 >> 172.21.15.7:0/248405938 conn(0x7f44c40fbb30 msgr2=0x7f44c4103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:48.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.881+0000 7f44c9c85640 1 -- 172.21.15.7:0/248405938 shutdown_connections 2024-03-26T08:42:48.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:42:48.881+0000 7f44c9c85640 1 -- 172.21.15.7:0/248405938 wait complete. 2024-03-26T08:42:48.889 INFO:tasks.cephfs.filesystem:scrub status for tag:5c5c842d-c6c4-4692-b1dc-a0930ba01d5d - {'path': '/', 'tag': '5c5c842d-c6c4-4692-b1dc-a0930ba01d5d', 'options': 'recursive,force'} 2024-03-26T08:42:50.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:49 smithi016 ceph-mon[35851]: pgmap v372: 97 pgs: 97 active+clean; 942 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 779 op/s 2024-03-26T08:42:50.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:49 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:42:50.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:49 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:50.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:49 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:42:50.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:49 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:42:50.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:49 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:50.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:49 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:50.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:49 smithi007 ceph-mon[32462]: pgmap v372: 97 pgs: 97 active+clean; 942 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 779 op/s 2024-03-26T08:42:50.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:49 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:42:50.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:49 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:50.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:49 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:42:50.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:49 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:42:50.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:49 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:50.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:49 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:50.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:49 smithi044 ceph-mon[36461]: pgmap v372: 97 pgs: 97 active+clean; 942 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 779 op/s 2024-03-26T08:42:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:49 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:42:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:49 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:49 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:42:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:49 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:42:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:49 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:50.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:49 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:42:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 20.0 s, 4557.4 tps, lat 21.979 ms stddev 22.985 2024-03-26T08:42:52.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:51 smithi016 ceph-mon[35851]: pgmap v373: 97 pgs: 97 active+clean; 951 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 799 op/s 2024-03-26T08:42:52.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:51 smithi007 ceph-mon[32462]: pgmap v373: 97 pgs: 97 active+clean; 951 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 799 op/s 2024-03-26T08:42:52.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:51 smithi044 ceph-mon[36461]: pgmap v373: 97 pgs: 97 active+clean; 951 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 11 MiB/s wr, 799 op/s 2024-03-26T08:42:54.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:53 smithi016 ceph-mon[35851]: pgmap v374: 97 pgs: 97 active+clean; 952 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 932 op/s 2024-03-26T08:42:54.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:53 smithi007 ceph-mon[32462]: pgmap v374: 97 pgs: 97 active+clean; 952 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 932 op/s 2024-03-26T08:42:54.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:53 smithi044 ceph-mon[36461]: pgmap v374: 97 pgs: 97 active+clean; 952 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 932 op/s 2024-03-26T08:42:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.274+0000 7f65f0cd1640 1 -- 172.21.15.7:0/370004490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec1028e0 msgr2=0x7f65ec104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.274+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/370004490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec1028e0 0x7f65ec104d50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009900 tx=0x7f65e002bc30 comp rx=0 tx=0).stop 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.274+0000 7f65f0cd1640 1 -- 172.21.15.7:0/370004490 shutdown_connections 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.274+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/370004490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ec105290 0x7f65ec105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.274+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/370004490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec1028e0 0x7f65ec104d50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.274+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/370004490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65ec0fff70 0x7f65ec1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.274+0000 7f65f0cd1640 1 -- 172.21.15.7:0/370004490 >> 172.21.15.7:0/370004490 conn(0x7f65ec0fbb70 msgr2=0x7f65ec0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.274+0000 7f65f0cd1640 1 -- 172.21.15.7:0/370004490 shutdown_connections 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.274+0000 7f65f0cd1640 1 -- 172.21.15.7:0/370004490 wait complete. 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65f0cd1640 1 Processor -- start 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65f0cd1640 1 -- start start 2024-03-26T08:42:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65f0cd1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ec0fff70 0x7f65ec19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65f0cd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65ec1028e0 0x7f65ec19e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65f0cd1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec105290 0x7f65ec1a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65f0cd1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec113a10 con 0x7f65ec1028e0 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65f0cd1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec113890 con 0x7f65ec105290 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65f0cd1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec113b90 con 0x7f65ec0fff70 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65e9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65ec1028e0 0x7f65ec19e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65ea59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ec0fff70 0x7f65ec19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec105290 0x7f65ec1a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65ea59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ec0fff70 0x7f65ec19e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50122/0 (socket says 172.21.15.7:50122) 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65e9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65ec1028e0 0x7f65ec19e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59936/0 (socket says 172.21.15.7:59936) 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.275+0000 7f65ea59c640 1 -- 172.21.15.7:0/3239816825 learned_addr learned my addr 172.21.15.7:0/3239816825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.276+0000 7f65ea59c640 1 -- 172.21.15.7:0/3239816825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec105290 msgr2=0x7f65ec1a55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.276+0000 7f65ea59c640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec105290 0x7f65ec1a55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.276+0000 7f65ea59c640 1 -- 172.21.15.7:0/3239816825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65ec1028e0 msgr2=0x7f65ec19e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.276+0000 7f65ea59c640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65ec1028e0 0x7f65ec19e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.276+0000 7f65ea59c640 1 -- 172.21.15.7:0/3239816825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec1a5c20 con 0x7f65ec0fff70 2024-03-26T08:42:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.276+0000 7f65ead9d640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec105290 0x7f65ec1a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.276+0000 7f65ea59c640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ec0fff70 0x7f65ec19e000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00b4a0 tx=0x7f65dc00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.277+0000 7f65db7fe640 1 -- 172.21.15.7:0/3239816825 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc014070 con 0x7f65ec0fff70 2024-03-26T08:42:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.277+0000 7f65db7fe640 1 -- 172.21.15.7:0/3239816825 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65dc010af0 con 0x7f65ec0fff70 2024-03-26T08:42:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.277+0000 7f65f0cd1640 1 -- 172.21.15.7:0/3239816825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec1a5f10 con 0x7f65ec0fff70 2024-03-26T08:42:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.277+0000 7f65db7fe640 1 -- 172.21.15.7:0/3239816825 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc017cc0 con 0x7f65ec0fff70 2024-03-26T08:42:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.277+0000 7f65f0cd1640 1 -- 172.21.15.7:0/3239816825 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec1a6310 con 0x7f65ec0fff70 2024-03-26T08:42:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.281+0000 7f65db7fe640 1 -- 172.21.15.7:0/3239816825 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f65dc017070 con 0x7f65ec0fff70 2024-03-26T08:42:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.281+0000 7f65f399a740 1 Processor -- start 2024-03-26T08:42:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.281+0000 7f65f399a740 1 -- start start 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.281+0000 7f65f399a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec6dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.281+0000 7f65f399a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6e310 0x55e02ec72e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.281+0000 7f65f399a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 0x55e02ec75910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.281+0000 7f65f399a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02ebd2b10 con 0x55e02ec6d920 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.281+0000 7f65f399a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02eb7b900 con 0x55e02ec6e310 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.281+0000 7f65f399a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02ebcebb0 con 0x55e02ec73480 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.282+0000 7f65ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6e310 0x55e02ec72e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.282+0000 7f65ead9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec6dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.282+0000 7f65ead9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec6dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59946/0 (socket says 172.21.15.7:59946) 2024-03-26T08:42:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.282+0000 7f65ead9d640 1 -- 172.21.15.7:0/2487569763 learned_addr learned my addr 172.21.15.7:0/2487569763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.282+0000 7f65db7fe640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d40a2a50 0x7f65d40a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.283+0000 7f65ead9d640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d40a2a50 0x7f65d40a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.283+0000 7f65ead9d640 1 -- 172.21.15.7:0/2487569763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 msgr2=0x55e02ec75910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:42:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.283+0000 7f65ead9d640 1 --2- 172.21.15.7:0/2487569763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 0x55e02ec75910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.283+0000 7f65ead9d640 1 -- 172.21.15.7:0/2487569763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6e310 msgr2=0x55e02ec72e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.283+0000 7f65ead9d640 1 --2- 172.21.15.7:0/2487569763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6e310 0x55e02ec72e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.283+0000 7f65ead9d640 1 -- 172.21.15.7:0/2487569763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e02e8e7210 con 0x55e02ec6d920 2024-03-26T08:42:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.283+0000 7f65db7fe640 1 -- 172.21.15.7:0/3239816825 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f65dc0cddf0 con 0x7f65ec0fff70 2024-03-26T08:42:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.283+0000 7f65ead9d640 1 --2- 172.21.15.7:0/2487569763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec6dd40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009fd0 tx=0x7f65e4013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06f5bc67c29dd0c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.284+0000 7f65d8ff9640 1 -- 172.21.15.7:0/2487569763 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004110 con 0x55e02ec6d920 2024-03-26T08:42:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.284+0000 7f65d8ff9640 1 -- 172.21.15.7:0/2487569763 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65e4004790 con 0x55e02ec6d920 2024-03-26T08:42:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.284+0000 7f65d8ff9640 1 -- 172.21.15.7:0/2487569763 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e401ed20 con 0x55e02ec6d920 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 -- 172.21.15.7:0/2487569763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 msgr2=0x55e02ec6dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65ead9d640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d40a2a50 0x7f65d40a4f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f65e4023000 tx=0x7f65e4020030 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 --2- 172.21.15.7:0/2487569763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec6dd40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009fd0 tx=0x7f65e4013830 comp rx=0 tx=0).stop 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 -- 172.21.15.7:0/2487569763 shutdown_connections 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 --2- 172.21.15.7:0/2487569763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 0x55e02ec75910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 --2- 172.21.15.7:0/2487569763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6e310 0x55e02ec72e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 --2- 172.21.15.7:0/2487569763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec6dd40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 -- 172.21.15.7:0/2487569763 >> 172.21.15.7:0/2487569763 conn(0x55e02ec68e90 msgr2=0x55e02ec69290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 -- 172.21.15.7:0/2487569763 shutdown_connections 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 -- 172.21.15.7:0/2487569763 wait complete. 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 -- start start 2024-03-26T08:42:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.285+0000 7f65f399a740 1 Processor -- start 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65f399a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6d920 0x55e02ec77c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65f399a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec78190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65f399a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 0x55e02ec7f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65f399a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02ec76430 con 0x55e02ec6e310 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65f399a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02eb8ed50 con 0x55e02ec6d920 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65f399a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e02ec765b0 con 0x55e02ec73480 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65f399a740 10 client.0 _ll_register_callbacks cb 0x55e02eb8e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6d920 0x55e02ec77c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65ead9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 0x55e02ec7f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65ea59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6d920 0x55e02ec77c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33246/0 (socket says 172.21.15.7:33246) 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65ead9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 0x55e02ec7f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50126/0 (socket says 172.21.15.7:50126) 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65ea59c640 1 -- 172.21.15.7:0/4267955135 learned_addr learned my addr 172.21.15.7:0/4267955135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.286+0000 7f65e9d9b640 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec78190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65ea59c640 1 -- 172.21.15.7:0/4267955135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 msgr2=0x55e02ec7f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65ea59c640 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 0x55e02ec7f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65ea59c640 1 -- 172.21.15.7:0/4267955135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 msgr2=0x55e02ec78190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65ea59c640 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec78190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65ea59c640 1 -- 172.21.15.7:0/4267955135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e02e9b3620 con 0x55e02ec6d920 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65e9d9b640 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec78190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65ea59c640 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6d920 0x55e02ec77c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65dc09a400 tx=0x7f65dc017560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=726e6e5796677d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65d1ffb640 1 -- 172.21.15.7:0/4267955135 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc01e070 con 0x55e02ec6d920 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65f399a740 1 -- 172.21.15.7:0/4267955135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65e4004790 con 0x55e02ec6d920 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65d1ffb640 1 -- 172.21.15.7:0/4267955135 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f65dc096d70 con 0x55e02ec6d920 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.287+0000 7f65d1ffb640 1 -- 172.21.15.7:0/4267955135 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65dc017360 con 0x55e02ec6d920 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65f399a740 10 client.25433 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65f399a740 1 -- 172.21.15.7:0/4267955135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e02ec82600 con 0x55e02ec6d920 2024-03-26T08:42:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65d1ffb640 1 -- 172.21.15.7:0/4267955135 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65e4004790 con 0x55e02ec6d920 2024-03-26T08:42:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65d1ffb640 1 -- 172.21.15.7:0/4267955135 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f65dc099b50 con 0x55e02ec6d920 2024-03-26T08:42:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65f399a740 10 client.25433 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65f399a740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65f399a740 20 client.25433 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65f399a740 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e02ec840f0 0x55e02ec86570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65f399a740 4 client.25433 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:42:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65f399a740 1 -- 172.21.15.7:0/4267955135 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e02ebe0050 con 0x55e02ec840f0 2024-03-26T08:42:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.288+0000 7f65e9d9b640 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e02ec840f0 0x55e02ec86570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.289+0000 7f65d1ffb640 10 client.25433 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.290+0000 7f65e9d9b640 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e02ec840f0 0x55e02ec86570 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c81e15aabdcbe08b server_cookie=c07f5bb0d0bf2876 in_seq=0 out_seq=0 2024-03-26T08:42:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.290+0000 7f65d1ffb640 1 -- 172.21.15.7:0/4267955135 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e02ebe0050 con 0x55e02ec840f0 2024-03-26T08:42:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.290+0000 7f65d1ffb640 10 client.25433 handle_command_reply: tid=0 2024-03-26T08:42:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.290+0000 7f65f399a740 1 client.25433 shutdown 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 -- 172.21.15.7:0/4267955135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6d920 msgr2=0x55e02ec77c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6d920 0x55e02ec77c50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65dc09a400 tx=0x7f65dc017560 comp rx=0 tx=0).stop 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 -- 172.21.15.7:0/4267955135 shutdown_connections 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e02ec840f0 0x55e02ec86570 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec73480 0x55e02ec7f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec78190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 --2- 172.21.15.7:0/4267955135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec6d920 0x55e02ec77c50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 -- 172.21.15.7:0/4267955135 >> 172.21.15.7:0/4267955135 conn(0x55e02ec68e90 msgr2=0x55e02ec68020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65d1ffb640 0 client.25433 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 -- 172.21.15.7:0/4267955135 shutdown_connections 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 1 -- 172.21.15.7:0/4267955135 wait complete. 2024-03-26T08:42:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.291+0000 7f65f399a740 20 client.25433 trim_cache size 0 max 16384 2024-03-26T08:42:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.313+0000 7f65f399a740 1 Processor -- start 2024-03-26T08:42:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.313+0000 7f65f399a740 1 -- start start 2024-03-26T08:42:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.313+0000 7f65f399a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.313+0000 7f65f399a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6e310 0x55e02ec7f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.313+0000 7f65f399a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec6eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.313+0000 7f65f399a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1b15d0 con 0x55e02ec6d920 2024-03-26T08:42:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65f399a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1af0d0 con 0x55e02ec73480 2024-03-26T08:42:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65f399a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1ad150 con 0x55e02ec6e310 2024-03-26T08:42:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec6eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ea59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec864e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ead9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec6eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33260/0 (socket says 172.21.15.7:33260) 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ead9d640 1 -- 172.21.15.7:0/3892511954 learned_addr learned my addr 172.21.15.7:0/3892511954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65e9d9b640 1 --2- 172.21.15.7:0/3892511954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6e310 0x55e02ec7f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ead9d640 1 -- 172.21.15.7:0/3892511954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6e310 msgr2=0x55e02ec7f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ead9d640 1 --2- 172.21.15.7:0/3892511954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6e310 0x55e02ec7f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ead9d640 1 -- 172.21.15.7:0/3892511954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 msgr2=0x55e02ec864e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ead9d640 1 --2- 172.21.15.7:0/3892511954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec864e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ead9d640 1 -- 172.21.15.7:0/3892511954 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e02ec6d710 con 0x55e02ec73480 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ea59c640 1 --2- 172.21.15.7:0/3892511954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec864e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65ead9d640 1 --2- 172.21.15.7:0/3892511954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec6eba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e02ec6eb70 tx=0x7f65e4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f10b9b649dfdcd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65d3fff640 1 -- 172.21.15.7:0/3892511954 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4039070 con 0x55e02ec73480 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65d3fff640 1 -- 172.21.15.7:0/3892511954 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65e400f870 con 0x55e02ec73480 2024-03-26T08:42:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.314+0000 7f65d3fff640 1 -- 172.21.15.7:0/3892511954 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004770 con 0x55e02ec73480 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.315+0000 7f65f399a740 1 -- 172.21.15.7:0/3892511954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 msgr2=0x55e02ec6eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.315+0000 7f65f399a740 1 --2- 172.21.15.7:0/3892511954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec6eba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55e02ec6eb70 tx=0x7f65e4011000 comp rx=0 tx=0).stop 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.315+0000 7f65f399a740 1 -- 172.21.15.7:0/3892511954 shutdown_connections 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.315+0000 7f65f399a740 1 --2- 172.21.15.7:0/3892511954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec6eba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.315+0000 7f65f399a740 1 --2- 172.21.15.7:0/3892511954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6e310 0x55e02ec7f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.315+0000 7f65f399a740 1 --2- 172.21.15.7:0/3892511954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6d920 0x55e02ec864e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.315+0000 7f65f399a740 1 -- 172.21.15.7:0/3892511954 >> 172.21.15.7:0/3892511954 conn(0x55e02ec68e90 msgr2=0x55e02ec72c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 -- 172.21.15.7:0/3892511954 shutdown_connections 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 -- 172.21.15.7:0/3892511954 wait complete. 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 -- start start 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 Processor -- start 2024-03-26T08:42:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6d920 0x55e02ec7c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec7ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec7dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65e9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec7ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65e9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec7ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59960/0 (socket says 172.21.15.7:59960) 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65e9d9b640 1 -- 172.21.15.7:0/1848878424 learned_addr learned my addr 172.21.15.7:0/1848878424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65ea59c640 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6d920 0x55e02ec7c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1ad750 con 0x55e02ec6e310 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1aea70 con 0x55e02ec73480 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1addb0 con 0x55e02ec6d920 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65f399a740 10 client.0 _ll_register_callbacks cb 0x7f65dc017600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.316+0000 7f65ead9d640 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec7dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65e9d9b640 1 -- 172.21.15.7:0/1848878424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6d920 msgr2=0x55e02ec7c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65e9d9b640 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6d920 0x55e02ec7c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65e9d9b640 1 -- 172.21.15.7:0/1848878424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 msgr2=0x55e02ec7dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65e9d9b640 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec7dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65e9d9b640 1 -- 172.21.15.7:0/1848878424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e02ec7e340 con 0x55e02ec6e310 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65ead9d640 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec7dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65e9d9b640 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec7ce00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f65e002c7b0 tx=0x7f65e002cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b010ec398dc876e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65d1ffb640 1 -- 172.21.15.7:0/1848878424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e004e070 con 0x55e02ec6e310 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65d1ffb640 1 -- 172.21.15.7:0/1848878424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f65e0004240 con 0x55e02ec6e310 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65e400f870 con 0x55e02ec6e310 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65d1ffb640 1 -- 172.21.15.7:0/1848878424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65e005a050 con 0x55e02ec6e310 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65f399a740 10 client.15615 fetch_fsmap learned FSMap version 24 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.317+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e02ecdfa40 con 0x55e02ec6e310 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.318+0000 7f65d1ffb640 1 -- 172.21.15.7:0/1848878424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65e0004440 con 0x55e02ec6e310 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.318+0000 7f65d1ffb640 1 -- 172.21.15.7:0/1848878424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f65e0004b10 con 0x55e02ec6e310 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.318+0000 7f65f399a740 10 client.15615 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:42:55.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.318+0000 7f65f399a740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:42:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.318+0000 7f65f399a740 20 client.15615 populate_metadata read hostname 'smithi007' 2024-03-26T08:42:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.318+0000 7f65f399a740 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e02ece0420 0x55e02ece2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:42:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.318+0000 7f65f399a740 4 client.15615 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:42:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.318+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f65e400f870 con 0x55e02ece0420 2024-03-26T08:42:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.318+0000 7f65ea59c640 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e02ece0420 0x55e02ece2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:42:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.319+0000 7f65ea59c640 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e02ece0420 0x55e02ece2840 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8b3d024e77fe458 server_cookie=edc4eda2008a5cdb in_seq=0 out_seq=0 2024-03-26T08:42:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.319+0000 7f65d1ffb640 10 client.15615 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.338+0000 7f65d1ffb640 1 -- 172.21.15.7:0/1848878424 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f65e400f870 con 0x55e02ece0420 2024-03-26T08:42:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.338+0000 7f65d1ffb640 10 client.15615 handle_command_reply: tid=0 2024-03-26T08:42:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.338+0000 7f65f399a740 1 client.15615 shutdown 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 msgr2=0x55e02ec7ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec7ce00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f65e002c7b0 tx=0x7f65e002cc40 comp rx=0 tx=0).stop 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 shutdown_connections 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e02ece0420 0x55e02ece2840 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e02ec73480 0x55e02ec7dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e02ec6e310 0x55e02ec7ce00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65d1ffb640 0 client.15615 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 --2- 172.21.15.7:0/1848878424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e02ec6d920 0x55e02ec7c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 >> 172.21.15.7:0/1848878424 conn(0x55e02ec68e90 msgr2=0x55e02ec72c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 shutdown_connections 2024-03-26T08:42:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 1 -- 172.21.15.7:0/1848878424 wait complete. 2024-03-26T08:42:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.339+0000 7f65f399a740 20 client.15615 trim_cache size 0 max 16384 2024-03-26T08:42:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:42:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:42:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:42:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 -- 172.21.15.7:0/3239816825 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d40a2a50 msgr2=0x7f65d40a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d40a2a50 0x7f65d40a4f10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f65e4023000 tx=0x7f65e4020030 comp rx=0 tx=0).stop 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 -- 172.21.15.7:0/3239816825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ec0fff70 msgr2=0x7f65ec19e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ec0fff70 0x7f65ec19e000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00b4a0 tx=0x7f65dc00b960 comp rx=0 tx=0).stop 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 -- 172.21.15.7:0/3239816825 shutdown_connections 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d40a2a50 0x7f65d40a4f10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec105290 0x7f65ec1a55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65ec1028e0 0x7f65ec19e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 --2- 172.21.15.7:0/3239816825 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65ec0fff70 0x7f65ec19e000 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 -- 172.21.15.7:0/3239816825 >> 172.21.15.7:0/3239816825 conn(0x7f65ec0fbb70 msgr2=0x7f65ec1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 -- 172.21.15.7:0/3239816825 shutdown_connections 2024-03-26T08:42:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:42:55.340+0000 7f65f0cd1640 1 -- 172.21.15.7:0/3239816825 wait complete. 2024-03-26T08:42:56.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 25.0 s, 4685.3 tps, lat 21.257 ms stddev 22.141 2024-03-26T08:42:56.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:55 smithi016 ceph-mon[35851]: pgmap v375: 97 pgs: 97 active+clean; 960 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.8 MiB/s wr, 886 op/s 2024-03-26T08:42:56.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:55 smithi007 ceph-mon[32462]: pgmap v375: 97 pgs: 97 active+clean; 960 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.8 MiB/s wr, 886 op/s 2024-03-26T08:42:56.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:55 smithi044 ceph-mon[36461]: pgmap v375: 97 pgs: 97 active+clean; 960 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 9.8 MiB/s wr, 886 op/s 2024-03-26T08:42:57.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:56 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:42:57.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:56 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:42:57.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:56 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:42:57.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:56 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:42:57.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:56 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:42:57.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:56 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:42:58.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:57 smithi016 ceph-mon[35851]: pgmap v376: 97 pgs: 97 active+clean; 968 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 916 op/s 2024-03-26T08:42:58.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:57 smithi007 ceph-mon[32462]: pgmap v376: 97 pgs: 97 active+clean; 968 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 916 op/s 2024-03-26T08:42:58.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:57 smithi044 ceph-mon[36461]: pgmap v376: 97 pgs: 97 active+clean; 968 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 916 op/s 2024-03-26T08:43:00.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:42:59 smithi016 ceph-mon[35851]: pgmap v377: 97 pgs: 97 active+clean; 974 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 974 op/s 2024-03-26T08:43:00.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:42:59 smithi007 ceph-mon[32462]: pgmap v377: 97 pgs: 97 active+clean; 974 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 974 op/s 2024-03-26T08:43:00.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:42:59 smithi044 ceph-mon[36461]: pgmap v377: 97 pgs: 97 active+clean; 974 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 974 op/s 2024-03-26T08:43:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 30.0 s, 4008.7 tps, lat 25.018 ms stddev 26.502 2024-03-26T08:43:02.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:01 smithi016 ceph-mon[35851]: pgmap v378: 97 pgs: 97 active+clean; 975 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 969 op/s 2024-03-26T08:43:02.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:01 smithi007 ceph-mon[32462]: pgmap v378: 97 pgs: 97 active+clean; 975 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 969 op/s 2024-03-26T08:43:02.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:01 smithi044 ceph-mon[36461]: pgmap v378: 97 pgs: 97 active+clean; 975 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 12 MiB/s wr, 969 op/s 2024-03-26T08:43:04.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:03 smithi016 ceph-mon[35851]: pgmap v379: 97 pgs: 97 active+clean; 993 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 982 op/s 2024-03-26T08:43:04.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:03 smithi007 ceph-mon[32462]: pgmap v379: 97 pgs: 97 active+clean; 993 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 982 op/s 2024-03-26T08:43:04.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:03 smithi044 ceph-mon[36461]: pgmap v379: 97 pgs: 97 active+clean; 993 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 982 op/s 2024-03-26T08:43:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 35.0 s, 3979.0 tps, lat 25.044 ms stddev 27.968 2024-03-26T08:43:06.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:05 smithi016 ceph-mon[35851]: pgmap v380: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 843 op/s 2024-03-26T08:43:06.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:05 smithi007 ceph-mon[32462]: pgmap v380: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 843 op/s 2024-03-26T08:43:06.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:05 smithi044 ceph-mon[36461]: pgmap v380: 97 pgs: 97 active+clean; 1.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 15 MiB/s wr, 843 op/s 2024-03-26T08:43:08.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:07 smithi016 ceph-mon[35851]: pgmap v381: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 872 op/s 2024-03-26T08:43:08.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:07 smithi007 ceph-mon[32462]: pgmap v381: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 872 op/s 2024-03-26T08:43:08.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:07 smithi044 ceph-mon[36461]: pgmap v381: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 MiB/s wr, 872 op/s 2024-03-26T08:43:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.477+0000 7f3197644640 1 -- 172.21.15.7:0/3822444021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3190102180 msgr2=0x7f3190108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.477+0000 7f3197644640 1 --2- 172.21.15.7:0/3822444021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3190102180 0x7f3190108a10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3188009ca0 tx=0x7f318802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.477+0000 7f3197644640 1 -- 172.21.15.7:0/3822444021 shutdown_connections 2024-03-26T08:43:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.477+0000 7f3197644640 1 --2- 172.21.15.7:0/3822444021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3190102180 0x7f3190108a10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.477+0000 7f3197644640 1 --2- 172.21.15.7:0/3822444021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31901017c0 0x7f3190101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.477+0000 7f3197644640 1 --2- 172.21.15.7:0/3822444021 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31901005d0 0x7f31901009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.477+0000 7f3197644640 1 -- 172.21.15.7:0/3822444021 >> 172.21.15.7:0/3822444021 conn(0x7f31900fbb40 msgr2=0x7f31900fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.478+0000 7f3197644640 1 -- 172.21.15.7:0/3822444021 shutdown_connections 2024-03-26T08:43:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.479+0000 7f3197644640 1 -- 172.21.15.7:0/3822444021 wait complete. 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.479+0000 7f3197644640 1 Processor -- start 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.479+0000 7f3197644640 1 -- start start 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3197644640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31901005d0 0x7f319010fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3197644640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31901017c0 0x7f319010ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3197644640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3190102180 0x7f3190110e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3197644640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319010a070 con 0x7f3190102180 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3197644640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3190109ef0 con 0x7f31901005d0 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3197644640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319010a1f0 con 0x7f31901017c0 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3194bdf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31901017c0 0x7f319010ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3194bdf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31901017c0 0x7f319010ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:52726/0 (socket says 172.21.15.7:52726) 2024-03-26T08:43:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3194bdf640 1 -- 172.21.15.7:0/32944079 learned_addr learned my addr 172.21.15.7:0/32944079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f31953e0640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31901005d0 0x7f319010fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3195be1640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3190102180 0x7f3190110e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3194bdf640 1 -- 172.21.15.7:0/32944079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31901005d0 msgr2=0x7f319010fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3194bdf640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31901005d0 0x7f319010fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3194bdf640 1 -- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3190102180 msgr2=0x7f3190110e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3194bdf640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3190102180 0x7f3190110e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.480+0000 7f3194bdf640 1 -- 172.21.15.7:0/32944079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31901115b0 con 0x7f31901017c0 2024-03-26T08:43:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.482+0000 7f3194bdf640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31901017c0 0x7f319010ff80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9f0 tx=0x7f318000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.482+0000 7f31867fc640 1 -- 172.21.15.7:0/32944079 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000cd80 con 0x7f31901017c0 2024-03-26T08:43:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.482+0000 7f3197644640 1 -- 172.21.15.7:0/32944079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31901118a0 con 0x7f31901017c0 2024-03-26T08:43:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.482+0000 7f31867fc640 1 -- 172.21.15.7:0/32944079 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3180010910 con 0x7f31901017c0 2024-03-26T08:43:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.482+0000 7f3197644640 1 -- 172.21.15.7:0/32944079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31901aeca0 con 0x7f31901017c0 2024-03-26T08:43:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.482+0000 7f31867fc640 1 -- 172.21.15.7:0/32944079 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800188f0 con 0x7f31901017c0 2024-03-26T08:43:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.486+0000 7f31867fc640 1 -- 172.21.15.7:0/32944079 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f3180018bb0 con 0x7f31901017c0 2024-03-26T08:43:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.486+0000 7f319a22e740 1 Processor -- start 2024-03-26T08:43:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.487+0000 7f319a22e740 1 -- start start 2024-03-26T08:43:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.487+0000 7f319a22e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.487+0000 7f319a22e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 0x55d4de481e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.487+0000 7f319a22e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de484910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.487+0000 7f319a22e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4de3e1b10 con 0x55d4de482480 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.487+0000 7f319a22e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4de38a900 con 0x55d4de47d310 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.487+0000 7f319a22e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4de3ddbb0 con 0x55d4de47c920 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f31867fc640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f317c0a2900 0x7f317c0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f3195be1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f31953e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 0x55d4de481e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f3195be1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:52742/0 (socket says 172.21.15.7:52742) 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f3195be1640 1 -- 172.21.15.7:0/1753833270 learned_addr learned my addr 172.21.15.7:0/1753833270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f31867fc640 1 -- 172.21.15.7:0/32944079 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f31800cee10 con 0x7f31901017c0 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f3194bdf640 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de484910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f31953e0640 1 -- 172.21.15.7:0/1753833270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 msgr2=0x55d4de47cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f31953e0640 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f31953e0640 1 -- 172.21.15.7:0/1753833270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 msgr2=0x55d4de484910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f31953e0640 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de484910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f31953e0640 1 -- 172.21.15.7:0/1753833270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4de0f6210 con 0x55d4de47d310 2024-03-26T08:43:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f3195be1640 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f3195be1640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f317c0a2900 0x7f317c0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.488+0000 7f3194bdf640 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de484910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.489+0000 7f31953e0640 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 0x55d4de481e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3190101620 tx=0x7f318c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0eb0c170b249a06 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.489+0000 7f317b7fe640 1 -- 172.21.15.7:0/1753833270 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c009e50 con 0x55d4de47d310 2024-03-26T08:43:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.489+0000 7f317b7fe640 1 -- 172.21.15.7:0/1753833270 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f318c0035f0 con 0x55d4de47d310 2024-03-26T08:43:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.489+0000 7f317b7fe640 1 -- 172.21.15.7:0/1753833270 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c0186e0 con 0x55d4de47d310 2024-03-26T08:43:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.490+0000 7f3195be1640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f317c0a2900 0x7f317c0a4dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55d4de47db70 tx=0x7f318802c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.490+0000 7f319a22e740 1 -- 172.21.15.7:0/1753833270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 msgr2=0x55d4de481e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.490+0000 7f319a22e740 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 0x55d4de481e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3190101620 tx=0x7f318c005f90 comp rx=0 tx=0).stop 2024-03-26T08:43:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.490+0000 7f319a22e740 1 -- 172.21.15.7:0/1753833270 shutdown_connections 2024-03-26T08:43:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.491+0000 7f319a22e740 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de484910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.491+0000 7f319a22e740 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 0x55d4de481e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.491+0000 7f319a22e740 1 --2- 172.21.15.7:0/1753833270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.491+0000 7f319a22e740 1 -- 172.21.15.7:0/1753833270 >> 172.21.15.7:0/1753833270 conn(0x55d4de477e90 msgr2=0x55d4de478290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.491+0000 7f319a22e740 1 -- 172.21.15.7:0/1753833270 shutdown_connections 2024-03-26T08:43:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.492+0000 7f319a22e740 1 -- 172.21.15.7:0/1753833270 wait complete. 2024-03-26T08:43:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.492+0000 7f319a22e740 1 -- start start 2024-03-26T08:43:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.492+0000 7f319a22e740 1 Processor -- start 2024-03-26T08:43:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.493+0000 7f319a22e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de486ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.493+0000 7f319a22e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 0x55d4de4871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.493+0000 7f31953e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de486ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.494+0000 7f31953e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de486ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:52750/0 (socket says 172.21.15.7:52750) 2024-03-26T08:43:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.494+0000 7f319a22e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de48e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.494+0000 7f31953e0640 1 -- 172.21.15.7:0/3695043850 learned_addr learned my addr 172.21.15.7:0/3695043850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.494+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4de485430 con 0x55d4de482480 2024-03-26T08:43:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.494+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4de39dd50 con 0x55d4de47d310 2024-03-26T08:43:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.494+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4de4855b0 con 0x55d4de47c920 2024-03-26T08:43:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.494+0000 7f319a22e740 10 client.0 _ll_register_callbacks cb 0x55d4de39d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.494+0000 7f3195be1640 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de48e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.494+0000 7f3194bdf640 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 0x55d4de4871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.495+0000 7f3195be1640 1 -- 172.21.15.7:0/3695043850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 msgr2=0x55d4de486ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.495+0000 7f3195be1640 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de486ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.495+0000 7f3195be1640 1 -- 172.21.15.7:0/3695043850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 msgr2=0x55d4de4871e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.495+0000 7f3195be1640 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 0x55d4de4871e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.495+0000 7f3195be1640 1 -- 172.21.15.7:0/3695043850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4de1c2620 con 0x55d4de482480 2024-03-26T08:43:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.495+0000 7f31953e0640 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de486ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.495+0000 7f3195be1640 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de48e220 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3188009430 tx=0x7f31880095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dc7627ce33653f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.495+0000 7f3178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:43:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.496+0000 7f3178ff9640 1 -- 172.21.15.7:0/3695043850 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188051060 con 0x55d4de482480 2024-03-26T08:43:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.496+0000 7f3178ff9640 1 -- 172.21.15.7:0/3695043850 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3188004450 con 0x55d4de482480 2024-03-26T08:43:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.497+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f318c0035f0 con 0x55d4de482480 2024-03-26T08:43:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.499+0000 7f3178ff9640 1 -- 172.21.15.7:0/3695043850 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3188039040 con 0x55d4de482480 2024-03-26T08:43:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.499+0000 7f319a22e740 10 client.15627 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.499+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d4de4913f0 con 0x55d4de482480 2024-03-26T08:43:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.502+0000 7f3178ff9640 1 -- 172.21.15.7:0/3695043850 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3188047070 con 0x55d4de482480 2024-03-26T08:43:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.502+0000 7f3178ff9640 1 -- 172.21.15.7:0/3695043850 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f318803f070 con 0x55d4de482480 2024-03-26T08:43:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.502+0000 7f319a22e740 10 client.15627 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.502+0000 7f319a22e740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.502+0000 7f319a22e740 20 client.15627 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.502+0000 7f319a22e740 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d4de492ac0 0x55d4de494ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.502+0000 7f319a22e740 4 client.15627 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.502+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f318c0035f0 con 0x55d4de492ac0 2024-03-26T08:43:08.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.503+0000 7f31953e0640 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d4de492ac0 0x55d4de494ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.508+0000 7f31953e0640 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d4de492ac0 0x55d4de494ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2d70d439df6143c server_cookie=bd9fe40f21eeaaae in_seq=0 out_seq=0 2024-03-26T08:43:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.509+0000 7f3178ff9640 10 client.15627 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.511+0000 7f3178ff9640 1 -- 172.21.15.7:0/3695043850 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f318c0035f0 con 0x55d4de492ac0 2024-03-26T08:43:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.511+0000 7f3178ff9640 10 client.15627 handle_command_reply: tid=0 2024-03-26T08:43:08.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.515+0000 7f319a22e740 1 client.15627 shutdown 2024-03-26T08:43:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.518+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 msgr2=0x55d4de48e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.518+0000 7f319a22e740 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de48e220 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3188009430 tx=0x7f31880095c0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.518+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 shutdown_connections 2024-03-26T08:43:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.518+0000 7f319a22e740 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d4de492ac0 0x55d4de494ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.518+0000 7f319a22e740 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de482480 0x55d4de48e220 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.518+0000 7f319a22e740 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de47d310 0x55d4de4871e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.518+0000 7f319a22e740 1 --2- 172.21.15.7:0/3695043850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de486ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.518+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 >> 172.21.15.7:0/3695043850 conn(0x55d4de477e90 msgr2=0x55d4de477020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.519+0000 7f3178ff9640 0 client.15627 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.519+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 shutdown_connections 2024-03-26T08:43:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.520+0000 7f319a22e740 1 -- 172.21.15.7:0/3695043850 wait complete. 2024-03-26T08:43:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.520+0000 7f319a22e740 20 client.15627 trim_cache size 0 max 16384 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f319a22e740 1 Processor -- start 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f319a22e740 1 -- start start 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f319a22e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f319a22e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f319a22e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 0x55d4de47eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f319a22e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3174010980 con 0x55d4de47d310 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f319a22e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317400e480 con 0x55d4de482480 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f319a22e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317400c500 con 0x55d4de47c920 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f3194bdf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f3194bdf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52168/0 (socket says 172.21.15.7:52168) 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f3194bdf640 1 -- 172.21.15.7:0/3156685415 learned_addr learned my addr 172.21.15.7:0/3156685415 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f31953e0640 1 --2- 172.21.15.7:0/3156685415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.547+0000 7f3195be1640 1 --2- 172.21.15.7:0/3156685415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 0x55d4de47eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.548+0000 7f31953e0640 1 -- 172.21.15.7:0/3156685415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 msgr2=0x55d4de47eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.548+0000 7f31953e0640 1 --2- 172.21.15.7:0/3156685415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 0x55d4de47eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.548+0000 7f31953e0640 1 -- 172.21.15.7:0/3156685415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 msgr2=0x55d4de48e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.548+0000 7f31953e0640 1 --2- 172.21.15.7:0/3156685415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.548+0000 7f31953e0640 1 -- 172.21.15.7:0/3156685415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4de47d6f0 con 0x55d4de47c920 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.548+0000 7f31953e0640 1 --2- 172.21.15.7:0/3156685415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47e630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f318c00bfd0 tx=0x7f318c004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea6062c5126ef74 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.548+0000 7f317affd640 1 -- 172.21.15.7:0/3156685415 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c0043c0 con 0x55d4de47c920 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.548+0000 7f317affd640 1 -- 172.21.15.7:0/3156685415 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f318c005520 con 0x55d4de47c920 2024-03-26T08:43:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.548+0000 7f317affd640 1 -- 172.21.15.7:0/3156685415 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c003850 con 0x55d4de47c920 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.549+0000 7f319a22e740 1 -- 172.21.15.7:0/3156685415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 msgr2=0x55d4de47e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.549+0000 7f319a22e740 1 --2- 172.21.15.7:0/3156685415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47e630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f318c00bfd0 tx=0x7f318c004580 comp rx=0 tx=0).stop 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.549+0000 7f319a22e740 1 -- 172.21.15.7:0/3156685415 shutdown_connections 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.549+0000 7f319a22e740 1 --2- 172.21.15.7:0/3156685415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 0x55d4de47eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.549+0000 7f319a22e740 1 --2- 172.21.15.7:0/3156685415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.549+0000 7f319a22e740 1 --2- 172.21.15.7:0/3156685415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de47e630 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.549+0000 7f319a22e740 1 -- 172.21.15.7:0/3156685415 >> 172.21.15.7:0/3156685415 conn(0x55d4de477e90 msgr2=0x55d4de478270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.549+0000 7f319a22e740 1 -- 172.21.15.7:0/3156685415 shutdown_connections 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.549+0000 7f319a22e740 1 -- 172.21.15.7:0/3156685415 wait complete. 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.550+0000 7f319a22e740 1 -- start start 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.550+0000 7f319a22e740 1 Processor -- start 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.550+0000 7f319a22e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de48b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f319a22e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f319a22e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 0x55d4de48ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f319a22e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317400cb00 con 0x55d4de47d310 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f319a22e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317400de20 con 0x55d4de482480 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f319a22e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317400d160 con 0x55d4de47c920 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f3194bdf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f3194bdf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52178/0 (socket says 172.21.15.7:52178) 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f3194bdf640 1 -- 172.21.15.7:0/3346265998 learned_addr learned my addr 172.21.15.7:0/3346265998 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f3195be1640 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 0x55d4de48ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f319a22e740 10 client.0 _ll_register_callbacks cb 0x7f31880329e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f3195be1640 1 -- 172.21.15.7:0/3346265998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 msgr2=0x55d4de48b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f3195be1640 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de48b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f3195be1640 1 -- 172.21.15.7:0/3346265998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 msgr2=0x55d4de48ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f3195be1640 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.551+0000 7f3195be1640 1 -- 172.21.15.7:0/3346265998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4de48d070 con 0x55d4de482480 2024-03-26T08:43:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.552+0000 7f3195be1640 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 0x55d4de48ca80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3188005e30 tx=0x7f3188032c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa56d7d44d8ff3d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.552+0000 7f3178ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.552+0000 7f3178ff9640 1 -- 172.21.15.7:0/3346265998 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3188064060 con 0x55d4de482480 2024-03-26T08:43:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.552+0000 7f3178ff9640 1 -- 172.21.15.7:0/3346265998 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3188042b50 con 0x55d4de482480 2024-03-26T08:43:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.552+0000 7f319a22e740 1 -- 172.21.15.7:0/3346265998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f318c005520 con 0x55d4de482480 2024-03-26T08:43:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.552+0000 7f3178ff9640 1 -- 172.21.15.7:0/3346265998 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3188051050 con 0x55d4de482480 2024-03-26T08:43:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.552+0000 7f319a22e740 10 client.25457 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.552+0000 7f319a22e740 1 -- 172.21.15.7:0/3346265998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d4de48e3d0 con 0x55d4de482480 2024-03-26T08:43:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.553+0000 7f3178ff9640 1 -- 172.21.15.7:0/3346265998 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f318c005520 con 0x55d4de482480 2024-03-26T08:43:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.553+0000 7f3178ff9640 1 -- 172.21.15.7:0/3346265998 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f3188069070 con 0x55d4de482480 2024-03-26T08:43:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.553+0000 7f319a22e740 10 client.25457 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.553+0000 7f319a22e740 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.553+0000 7f319a22e740 20 client.25457 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.553+0000 7f319a22e740 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d4de4ee910 0x55d4de4f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.553+0000 7f319a22e740 4 client.25457 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:43:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.553+0000 7f319a22e740 1 -- 172.21.15.7:0/3346265998 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4de3ef050 con 0x55d4de4ee910 2024-03-26T08:43:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.553+0000 7f31953e0640 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d4de4ee910 0x55d4de4f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.554+0000 7f31953e0640 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d4de4ee910 0x55d4de4f0d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1aa68998ecc92e5 server_cookie=91fdda9e1a426cf7 in_seq=0 out_seq=0 2024-03-26T08:43:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.554+0000 7f3178ff9640 10 client.25457 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.585+0000 7f3178ff9640 1 -- 172.21.15.7:0/3346265998 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4de3ef050 con 0x55d4de4ee910 2024-03-26T08:43:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.585+0000 7f3178ff9640 10 client.25457 handle_command_reply: tid=0 2024-03-26T08:43:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.585+0000 7f319a22e740 1 client.25457 shutdown 2024-03-26T08:43:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f319a22e740 1 -- 172.21.15.7:0/3346265998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 msgr2=0x55d4de48ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f319a22e740 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 0x55d4de48ca80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3188005e30 tx=0x7f3188032c90 comp rx=0 tx=0).stop 2024-03-26T08:43:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f319a22e740 1 -- 172.21.15.7:0/3346265998 shutdown_connections 2024-03-26T08:43:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f319a22e740 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d4de4ee910 0x55d4de4f0d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f3178ff9640 0 client.25457 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f319a22e740 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4de482480 0x55d4de48ca80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f319a22e740 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4de47d310 0x55d4de48ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f319a22e740 1 --2- 172.21.15.7:0/3346265998 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4de47c920 0x55d4de48b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f319a22e740 1 -- 172.21.15.7:0/3346265998 >> 172.21.15.7:0/3346265998 conn(0x55d4de477e90 msgr2=0x55d4de481830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.586+0000 7f319a22e740 1 -- 172.21.15.7:0/3346265998 shutdown_connections 2024-03-26T08:43:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.587+0000 7f319a22e740 1 -- 172.21.15.7:0/3346265998 wait complete. 2024-03-26T08:43:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.587+0000 7f319a22e740 20 client.25457 trim_cache size 0 max 16384 2024-03-26T08:43:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:43:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:43:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:43:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 -- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f317c0a2900 msgr2=0x7f317c0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f317c0a2900 0x7f317c0a4dc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55d4de47db70 tx=0x7f318802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 -- 172.21.15.7:0/32944079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31901017c0 msgr2=0x7f319010ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31901017c0 0x7f319010ff80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f318000e9f0 tx=0x7f318000eeb0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 -- 172.21.15.7:0/32944079 shutdown_connections 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f317c0a2900 0x7f317c0a4dc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3190102180 0x7f3190110e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31901017c0 0x7f319010ff80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 --2- 172.21.15.7:0/32944079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31901005d0 0x7f319010fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 -- 172.21.15.7:0/32944079 >> 172.21.15.7:0/32944079 conn(0x7f31900fbb40 msgr2=0x7f3190104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 -- 172.21.15.7:0/32944079 shutdown_connections 2024-03-26T08:43:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:08.588+0000 7f3197644640 1 -- 172.21.15.7:0/32944079 wait complete. 2024-03-26T08:43:10.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:09 smithi016 ceph-mon[35851]: pgmap v382: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 852 op/s 2024-03-26T08:43:10.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:09 smithi007 ceph-mon[32462]: pgmap v382: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 852 op/s 2024-03-26T08:43:10.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:09 smithi044 ceph-mon[36461]: pgmap v382: 97 pgs: 97 active+clean; 1.0 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 MiB/s wr, 852 op/s 2024-03-26T08:43:11.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 40.0 s, 4220.6 tps, lat 23.770 ms stddev 24.646 2024-03-26T08:43:12.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:11 smithi016 ceph-mon[35851]: pgmap v383: 97 pgs: 97 active+clean; 1021 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 764 op/s 2024-03-26T08:43:12.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:11 smithi007 ceph-mon[32462]: pgmap v383: 97 pgs: 97 active+clean; 1021 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 764 op/s 2024-03-26T08:43:12.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:11 smithi044 ceph-mon[36461]: pgmap v383: 97 pgs: 97 active+clean; 1021 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 764 op/s 2024-03-26T08:43:14.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:13 smithi016 ceph-mon[35851]: pgmap v384: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 892 op/s 2024-03-26T08:43:14.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:13 smithi007 ceph-mon[32462]: pgmap v384: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 892 op/s 2024-03-26T08:43:14.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:13 smithi044 ceph-mon[36461]: pgmap v384: 97 pgs: 97 active+clean; 1014 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 892 op/s 2024-03-26T08:43:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 45.0 s, 4649.0 tps, lat 21.500 ms stddev 22.157 2024-03-26T08:43:16.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:15 smithi016 ceph-mon[35851]: pgmap v385: 97 pgs: 97 active+clean; 1015 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 807 op/s 2024-03-26T08:43:16.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:15 smithi007 ceph-mon[32462]: pgmap v385: 97 pgs: 97 active+clean; 1015 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 807 op/s 2024-03-26T08:43:16.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:15 smithi044 ceph-mon[36461]: pgmap v385: 97 pgs: 97 active+clean; 1015 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 807 op/s 2024-03-26T08:43:18.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:17 smithi016 ceph-mon[35851]: pgmap v386: 97 pgs: 97 active+clean; 1015 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 795 op/s 2024-03-26T08:43:18.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:17 smithi007 ceph-mon[32462]: pgmap v386: 97 pgs: 97 active+clean; 1015 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 795 op/s 2024-03-26T08:43:18.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:17 smithi044 ceph-mon[36461]: pgmap v386: 97 pgs: 97 active+clean; 1015 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 795 op/s 2024-03-26T08:43:18.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:43:18.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:43:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.057+0000 7f2a836f8640 1 -- 172.21.15.7:0/2967805340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7c074100 msgr2=0x7f2a7c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.057+0000 7f2a836f8640 1 --2- 172.21.15.7:0/2967805340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7c074100 0x7f2a7c074580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009930 tx=0x7f2a6c02bc60 comp rx=0 tx=0).stop 2024-03-26T08:43:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.057+0000 7f2a836f8640 1 -- 172.21.15.7:0/2967805340 shutdown_connections 2024-03-26T08:43:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.057+0000 7f2a836f8640 1 --2- 172.21.15.7:0/2967805340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c10dc80 0x7f2a7c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.057+0000 7f2a836f8640 1 --2- 172.21.15.7:0/2967805340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7c074100 0x7f2a7c074580 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.057+0000 7f2a836f8640 1 --2- 172.21.15.7:0/2967805340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c0756e0 0x7f2a7c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.057+0000 7f2a836f8640 1 -- 172.21.15.7:0/2967805340 >> 172.21.15.7:0/2967805340 conn(0x7f2a7c0fcc20 msgr2=0x7f2a7c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.058+0000 7f2a836f8640 1 -- 172.21.15.7:0/2967805340 shutdown_connections 2024-03-26T08:43:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.058+0000 7f2a836f8640 1 -- 172.21.15.7:0/2967805340 wait complete. 2024-03-26T08:43:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.059+0000 7f2a836f8640 1 Processor -- start 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.060+0000 7f2a836f8640 1 -- start start 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.060+0000 7f2a836f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c074100 0x7f2a7c10a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.060+0000 7f2a836f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7c0756e0 0x7f2a7c10a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a836f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c10dc80 0x7f2a7c1ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a836f8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c113a90 con 0x7f2a7c074100 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a836f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c113910 con 0x7f2a7c0756e0 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a836f8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c113c10 con 0x7f2a7c10dc80 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81494640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c074100 0x7f2a7c10a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81c95640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c10dc80 0x7f2a7c1ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a80c93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7c0756e0 0x7f2a7c10a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81c95640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c10dc80 0x7f2a7c1ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58480/0 (socket says 172.21.15.7:58480) 2024-03-26T08:43:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81c95640 1 -- 172.21.15.7:0/1356414004 learned_addr learned my addr 172.21.15.7:0/1356414004 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81494640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c074100 0x7f2a7c10a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53502/0 (socket says 172.21.15.7:53502) 2024-03-26T08:43:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81c95640 1 -- 172.21.15.7:0/1356414004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7c0756e0 msgr2=0x7f2a7c10a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81c95640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7c0756e0 0x7f2a7c10a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81c95640 1 -- 172.21.15.7:0/1356414004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c074100 msgr2=0x7f2a7c10a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81c95640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c074100 0x7f2a7c10a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81c95640 1 -- 172.21.15.7:0/1356414004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c1ae6e0 con 0x7f2a7c10dc80 2024-03-26T08:43:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.061+0000 7f2a81c95640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c10dc80 0x7f2a7c1ae1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400b970 tx=0x7f2a7400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.062+0000 7f2a727fc640 1 -- 172.21.15.7:0/1356414004 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74019070 con 0x7f2a7c10dc80 2024-03-26T08:43:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.062+0000 7f2a836f8640 1 -- 172.21.15.7:0/1356414004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c1ae8b0 con 0x7f2a7c10dc80 2024-03-26T08:43:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.062+0000 7f2a836f8640 1 -- 172.21.15.7:0/1356414004 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c1aed70 con 0x7f2a7c10dc80 2024-03-26T08:43:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.062+0000 7f2a727fc640 1 -- 172.21.15.7:0/1356414004 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a7400cb40 con 0x7f2a7c10dc80 2024-03-26T08:43:19.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.062+0000 7f2a727fc640 1 -- 172.21.15.7:0/1356414004 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740076e0 con 0x7f2a7c10dc80 2024-03-26T08:43:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.066+0000 7f2a727fc640 1 -- 172.21.15.7:0/1356414004 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f2a7401c070 con 0x7f2a7c10dc80 2024-03-26T08:43:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.067+0000 7f2a727fc640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a680a2a60 0x7f2a680a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.067+0000 7f2a727fc640 1 -- 172.21.15.7:0/1356414004 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a740cdf70 con 0x7f2a7c10dc80 2024-03-26T08:43:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.067+0000 7f2a81494640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a680a2a60 0x7f2a680a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.068+0000 7f2a863c1740 1 Processor -- start 2024-03-26T08:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.068+0000 7f2a81494640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a680a2a60 0x7f2a680a4f20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006fd0 tx=0x7f2a780090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.068+0000 7f2a863c1740 1 -- start start 2024-03-26T08:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.068+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55580730bda0 0x55580730c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.068+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730c790 0x555807311290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.069+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555807311900 0x555807313d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.069+0000 7f2a81494640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55580730bda0 0x55580730c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.069+0000 7f2a81494640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55580730bda0 0x55580730c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53510/0 (socket says 172.21.15.7:53510) 2024-03-26T08:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.069+0000 7f2a80c93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730c790 0x555807311290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.069+0000 7f2a80c93640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730c790 0x555807311290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35784/0 (socket says 172.21.15.7:35784) 2024-03-26T08:43:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.069+0000 7f2a81494640 1 -- 172.21.15.7:0/195262250 learned_addr learned my addr 172.21.15.7:0/195262250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.069+0000 7f2a863c1740 1 -- 172.21.15.7:0/195262250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558072713d0 con 0x55580730bda0 2024-03-26T08:43:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.069+0000 7f2a863c1740 1 -- 172.21.15.7:0/195262250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555807219ad0 con 0x55580730c790 2024-03-26T08:43:19.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.069+0000 7f2a863c1740 1 -- 172.21.15.7:0/195262250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55580726d470 con 0x555807311900 2024-03-26T08:43:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.070+0000 7f2a80c93640 1 -- 172.21.15.7:0/195262250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555807311900 msgr2=0x555807313d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.070+0000 7f2a80c93640 1 --2- 172.21.15.7:0/195262250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555807311900 0x555807313d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.070+0000 7f2a80c93640 1 -- 172.21.15.7:0/195262250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55580730bda0 msgr2=0x55580730c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.070+0000 7f2a80c93640 1 --2- 172.21.15.7:0/195262250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55580730bda0 0x55580730c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.070+0000 7f2a80c93640 1 -- 172.21.15.7:0/195262250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55580704f7d0 con 0x55580730c790 2024-03-26T08:43:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.070+0000 7f2a80c93640 1 --2- 172.21.15.7:0/195262250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730c790 0x555807311290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c036040 tx=0x7f2a6c004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0d5649ed0696bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.070+0000 7f2a677fe640 1 -- 172.21.15.7:0/195262250 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c042070 con 0x55580730c790 2024-03-26T08:43:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.071+0000 7f2a677fe640 1 -- 172.21.15.7:0/195262250 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a6c02ccf0 con 0x55580730c790 2024-03-26T08:43:19.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.071+0000 7f2a677fe640 1 -- 172.21.15.7:0/195262250 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c005260 con 0x55580730c790 2024-03-26T08:43:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 -- 172.21.15.7:0/195262250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730c790 msgr2=0x555807311290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 --2- 172.21.15.7:0/195262250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730c790 0x555807311290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c036040 tx=0x7f2a6c004f10 comp rx=0 tx=0).stop 2024-03-26T08:43:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 -- 172.21.15.7:0/195262250 shutdown_connections 2024-03-26T08:43:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 --2- 172.21.15.7:0/195262250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555807311900 0x555807313d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 --2- 172.21.15.7:0/195262250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730c790 0x555807311290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 --2- 172.21.15.7:0/195262250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55580730bda0 0x55580730c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 -- 172.21.15.7:0/195262250 >> 172.21.15.7:0/195262250 conn(0x555807307310 msgr2=0x555807307710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 -- 172.21.15.7:0/195262250 shutdown_connections 2024-03-26T08:43:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 -- 172.21.15.7:0/195262250 wait complete. 2024-03-26T08:43:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 -- start start 2024-03-26T08:43:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.072+0000 7f2a863c1740 1 Processor -- start 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.073+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730bda0 0x555807316110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.073+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.073+0000 7f2a81494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730bda0 0x555807316110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.073+0000 7f2a81494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730bda0 0x555807316110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35790/0 (socket says 172.21.15.7:35790) 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a863c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558073148a0 con 0x555807311900 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a863c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55580722d060 con 0x55580730bda0 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a863c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555807314a20 con 0x55580730c790 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a80c93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a80c93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58504/0 (socket says 172.21.15.7:58504) 2024-03-26T08:43:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a863c1740 10 client.0 _ll_register_callbacks cb 0x55580720f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a80c93640 1 -- 172.21.15.7:0/3223175265 learned_addr learned my addr 172.21.15.7:0/3223175265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a81494640 1 -- 172.21.15.7:0/3223175265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 msgr2=0x555807316650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a81494640 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a81494640 1 -- 172.21.15.7:0/3223175265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 msgr2=0x55580731d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a81494640 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.074+0000 7f2a81494640 1 -- 172.21.15.7:0/3223175265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555806f853f0 con 0x55580730bda0 2024-03-26T08:43:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.075+0000 7f2a81494640 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730bda0 0x555807316110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006d90 tx=0x7f2a78004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=963ab76f78d97a5f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.075+0000 7f2a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.075+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3223175265 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78025070 con 0x55580730bda0 2024-03-26T08:43:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.075+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3223175265 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a780137e0 con 0x55580730bda0 2024-03-26T08:43:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.075+0000 7f2a863c1740 1 -- 172.21.15.7:0/3223175265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a6c02ccf0 con 0x55580730bda0 2024-03-26T08:43:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3223175265 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a7802f050 con 0x55580730bda0 2024-03-26T08:43:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a863c1740 10 client.25469 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a863c1740 1 -- 172.21.15.7:0/3223175265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555806f450f0 con 0x55580730bda0 2024-03-26T08:43:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3223175265 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a6c02ccf0 con 0x55580730bda0 2024-03-26T08:43:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3223175265 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2a7802acb0 con 0x55580730bda0 2024-03-26T08:43:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a863c1740 10 client.25469 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a863c1740 10 client.25469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a863c1740 20 client.25469 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555807321c70 0x5558073240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a863c1740 4 client.25469 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.076+0000 7f2a863c1740 1 -- 172.21.15.7:0/3223175265 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55580727e910 con 0x555807321c70 2024-03-26T08:43:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.077+0000 7f2a80c93640 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555807321c70 0x5558073240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.078+0000 7f2a80c93640 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555807321c70 0x5558073240b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1fd933b95b7604 server_cookie=96d536edb4c0110f in_seq=0 out_seq=0 2024-03-26T08:43:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.078+0000 7f2a64ff9640 10 client.25469 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.079+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3223175265 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55580727e910 con 0x555807321c70 2024-03-26T08:43:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.079+0000 7f2a64ff9640 10 client.25469 handle_command_reply: tid=0 2024-03-26T08:43:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.079+0000 7f2a863c1740 1 client.25469 shutdown 2024-03-26T08:43:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.080+0000 7f2a863c1740 1 -- 172.21.15.7:0/3223175265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730bda0 msgr2=0x555807316110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.080+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730bda0 0x555807316110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006d90 tx=0x7f2a78004750 comp rx=0 tx=0).stop 2024-03-26T08:43:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.080+0000 7f2a863c1740 1 -- 172.21.15.7:0/3223175265 shutdown_connections 2024-03-26T08:43:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.080+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555807321c70 0x5558073240b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.080+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.080+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.080+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3223175265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55580730bda0 0x555807316110 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006d90 tx=0x7f2a78004750 comp rx=0 tx=0).stop 2024-03-26T08:43:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.080+0000 7f2a863c1740 1 -- 172.21.15.7:0/3223175265 >> 172.21.15.7:0/3223175265 conn(0x555807307310 msgr2=0x555807306530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.081+0000 7f2a64ff9640 0 client.25469 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.081+0000 7f2a863c1740 1 -- 172.21.15.7:0/3223175265 shutdown_connections 2024-03-26T08:43:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.081+0000 7f2a863c1740 1 -- 172.21.15.7:0/3223175265 wait complete. 2024-03-26T08:43:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.081+0000 7f2a863c1740 20 client.25469 trim_cache size 0 max 16384 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.120+0000 7f2a863c1740 1 Processor -- start 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.120+0000 7f2a863c1740 1 -- start start 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x555807306120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a863c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600109e0 con 0x555807311900 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a863c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6000e4e0 con 0x555807305ca0 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a863c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6000c560 con 0x55580730c790 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a80c93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a80c93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53522/0 (socket says 172.21.15.7:53522) 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a80c93640 1 -- 172.21.15.7:0/2654460719 learned_addr learned my addr 172.21.15.7:0/2654460719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a81494640 1 --2- 172.21.15.7:0/2654460719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.121+0000 7f2a81c95640 1 --2- 172.21.15.7:0/2654460719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x555807306120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.122+0000 7f2a81494640 1 -- 172.21.15.7:0/2654460719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 msgr2=0x555807306120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.122+0000 7f2a81494640 1 --2- 172.21.15.7:0/2654460719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x555807306120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.122+0000 7f2a81494640 1 -- 172.21.15.7:0/2654460719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 msgr2=0x55580731d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.122+0000 7f2a81494640 1 --2- 172.21.15.7:0/2654460719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.122+0000 7f2a81494640 1 -- 172.21.15.7:0/2654460719 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55580730cb70 con 0x55580730c790 2024-03-26T08:43:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.122+0000 7f2a81494640 1 --2- 172.21.15.7:0/2654460719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a78021000 tx=0x7f2a78009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8090531e0442b017 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.122+0000 7f2a66ffd640 1 -- 172.21.15.7:0/2654460719 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78025040 con 0x55580730c790 2024-03-26T08:43:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.122+0000 7f2a66ffd640 1 -- 172.21.15.7:0/2654460719 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a780137e0 con 0x55580730c790 2024-03-26T08:43:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.122+0000 7f2a66ffd640 1 -- 172.21.15.7:0/2654460719 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7801b040 con 0x55580730c790 2024-03-26T08:43:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.123+0000 7f2a863c1740 1 -- 172.21.15.7:0/2654460719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 msgr2=0x555807316430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.123+0000 7f2a863c1740 1 --2- 172.21.15.7:0/2654460719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a78021000 tx=0x7f2a78009f50 comp rx=0 tx=0).stop 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.123+0000 7f2a863c1740 1 -- 172.21.15.7:0/2654460719 shutdown_connections 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.123+0000 7f2a863c1740 1 --2- 172.21.15.7:0/2654460719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x555807306120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.123+0000 7f2a863c1740 1 --2- 172.21.15.7:0/2654460719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.123+0000 7f2a863c1740 1 --2- 172.21.15.7:0/2654460719 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580730c790 0x555807316430 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2a78021000 tx=0x7f2a78009f50 comp rx=0 tx=0).stop 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.123+0000 7f2a863c1740 1 -- 172.21.15.7:0/2654460719 >> 172.21.15.7:0/2654460719 conn(0x555807307310 msgr2=0x555807315d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.124+0000 7f2a863c1740 1 -- 172.21.15.7:0/2654460719 shutdown_connections 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.124+0000 7f2a863c1740 1 -- 172.21.15.7:0/2654460719 wait complete. 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.124+0000 7f2a863c1740 1 -- start start 2024-03-26T08:43:19.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.125+0000 7f2a863c1740 1 Processor -- start 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.125+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x55580731b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.125+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.125+0000 7f2a863c1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580731c800 0x55580731cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.125+0000 7f2a863c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6000cb60 con 0x555807311900 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.125+0000 7f2a863c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6000de80 con 0x555807305ca0 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.125+0000 7f2a863c1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6000d1c0 con 0x55580731c800 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.125+0000 7f2a863c1740 10 client.0 _ll_register_callbacks cb 0x7f2a7800e930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a81494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x55580731b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a81494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x55580731b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35820/0 (socket says 172.21.15.7:35820) 2024-03-26T08:43:19.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a81494640 1 -- 172.21.15.7:0/3317469900 learned_addr learned my addr 172.21.15.7:0/3317469900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a81c95640 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580731c800 0x55580731cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a80c93640 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a81494640 1 -- 172.21.15.7:0/3317469900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580731c800 msgr2=0x55580731cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a81494640 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580731c800 0x55580731cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a81494640 1 -- 172.21.15.7:0/3317469900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 msgr2=0x55580731b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a81494640 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.126+0000 7f2a81494640 1 -- 172.21.15.7:0/3317469900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55580731d1c0 con 0x555807305ca0 2024-03-26T08:43:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.127+0000 7f2a81494640 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x55580731b2b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x555807316ec0 tx=0x7f2a78021d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=107aedddab416d15 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.127+0000 7f2a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.127+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3317469900 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78044060 con 0x555807305ca0 2024-03-26T08:43:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.127+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3317469900 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a78013da0 con 0x555807305ca0 2024-03-26T08:43:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.127+0000 7f2a863c1740 1 -- 172.21.15.7:0/3317469900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a780137e0 con 0x555807305ca0 2024-03-26T08:43:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3317469900 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a78038040 con 0x555807305ca0 2024-03-26T08:43:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a863c1740 10 client.25475 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a863c1740 1 -- 172.21.15.7:0/3317469900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55580737df10 con 0x555807305ca0 2024-03-26T08:43:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3317469900 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a780137e0 con 0x555807305ca0 2024-03-26T08:43:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3317469900 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2a78034440 con 0x555807305ca0 2024-03-26T08:43:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a863c1740 10 client.25475 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a863c1740 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a863c1740 20 client.25475 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55580737e6b0 0x555807380af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a863c1740 4 client.25475 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:43:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.128+0000 7f2a863c1740 1 -- 172.21.15.7:0/3317469900 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555807278c60 con 0x55580737e6b0 2024-03-26T08:43:19.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.129+0000 7f2a80c93640 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55580737e6b0 0x555807380af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.130+0000 7f2a80c93640 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55580737e6b0 0x555807380af0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50a293b680cd920 server_cookie=e440ae6713f4db0a in_seq=0 out_seq=0 2024-03-26T08:43:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.130+0000 7f2a64ff9640 10 client.25475 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.130+0000 7f2a64ff9640 1 -- 172.21.15.7:0/3317469900 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555807278c60 con 0x55580737e6b0 2024-03-26T08:43:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.130+0000 7f2a64ff9640 10 client.25475 handle_command_reply: tid=0 2024-03-26T08:43:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.130+0000 7f2a863c1740 1 client.25475 shutdown 2024-03-26T08:43:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.130+0000 7f2a863c1740 1 -- 172.21.15.7:0/3317469900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 msgr2=0x55580731b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.131+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x55580731b2b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x555807316ec0 tx=0x7f2a78021d80 comp rx=0 tx=0).stop 2024-03-26T08:43:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.132+0000 7f2a863c1740 1 -- 172.21.15.7:0/3317469900 shutdown_connections 2024-03-26T08:43:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.132+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55580737e6b0 0x555807380af0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.133+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55580731c800 0x55580731cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.133+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555807311900 0x55580731b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.133+0000 7f2a863c1740 1 --2- 172.21.15.7:0/3317469900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555807305ca0 0x55580731b2b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.133+0000 7f2a64ff9640 0 client.25475 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:19.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.133+0000 7f2a863c1740 1 -- 172.21.15.7:0/3317469900 >> 172.21.15.7:0/3317469900 conn(0x555807307310 msgr2=0x555807315cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.133+0000 7f2a863c1740 1 -- 172.21.15.7:0/3317469900 shutdown_connections 2024-03-26T08:43:19.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.133+0000 7f2a863c1740 1 -- 172.21.15.7:0/3317469900 wait complete. 2024-03-26T08:43:19.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.133+0000 7f2a863c1740 20 client.25475 trim_cache size 0 max 16384 2024-03-26T08:43:19.126 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:43:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:43:19.126 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:43:19.126 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.134+0000 7f2a836f8640 1 -- 172.21.15.7:0/1356414004 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a680a2a60 msgr2=0x7f2a680a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.134+0000 7f2a836f8640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a680a2a60 0x7f2a680a4f20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006fd0 tx=0x7f2a780090e0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.134+0000 7f2a836f8640 1 -- 172.21.15.7:0/1356414004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c10dc80 msgr2=0x7f2a7c1ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.134+0000 7f2a836f8640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c10dc80 0x7f2a7c1ae1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400b970 tx=0x7f2a7400be30 comp rx=0 tx=0).stop 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.135+0000 7f2a836f8640 1 -- 172.21.15.7:0/1356414004 shutdown_connections 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.135+0000 7f2a836f8640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a680a2a60 0x7f2a680a4f20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.135+0000 7f2a836f8640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a7c10dc80 0x7f2a7c1ae1a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.135+0000 7f2a836f8640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a7c0756e0 0x7f2a7c10a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.135+0000 7f2a836f8640 1 --2- 172.21.15.7:0/1356414004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a7c074100 0x7f2a7c10a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.135+0000 7f2a836f8640 1 -- 172.21.15.7:0/1356414004 >> 172.21.15.7:0/1356414004 conn(0x7f2a7c0fcc20 msgr2=0x7f2a7c101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.135+0000 7f2a836f8640 1 -- 172.21.15.7:0/1356414004 shutdown_connections 2024-03-26T08:43:19.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.135+0000 7f2a836f8640 1 -- 172.21.15.7:0/1356414004 wait complete. 2024-03-26T08:43:19.146 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:43:19.147 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:43:19.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.326+0000 7fa0731f7640 1 -- 172.21.15.7:0/3422245751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109770 msgr2=0x7fa06c110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.326+0000 7fa0731f7640 1 --2- 172.21.15.7:0/3422245751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109770 0x7fa06c110000 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa064009be0 tx=0x7fa06402bef0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.327+0000 7fa0731f7640 1 -- 172.21.15.7:0/3422245751 shutdown_connections 2024-03-26T08:43:19.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.327+0000 7fa0731f7640 1 --2- 172.21.15.7:0/3422245751 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109770 0x7fa06c110000 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.327+0000 7fa0731f7640 1 --2- 172.21.15.7:0/3422245751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa06c108db0 0x7fa06c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.327+0000 7fa0731f7640 1 --2- 172.21.15.7:0/3422245751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa06c107bc0 0x7fa06c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.327+0000 7fa0731f7640 1 -- 172.21.15.7:0/3422245751 >> 172.21.15.7:0/3422245751 conn(0x7fa06c076b60 msgr2=0x7fa06c078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.327+0000 7fa0731f7640 1 -- 172.21.15.7:0/3422245751 shutdown_connections 2024-03-26T08:43:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.327+0000 7fa0731f7640 1 -- 172.21.15.7:0/3422245751 wait complete. 2024-03-26T08:43:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.327+0000 7fa0731f7640 1 Processor -- start 2024-03-26T08:43:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.327+0000 7fa0731f7640 1 -- start start 2024-03-26T08:43:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.328+0000 7fa0731f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa06c107bc0 0x7fa06c19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.328+0000 7fa0731f7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa06c108db0 0x7fa06c19e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.328+0000 7fa0731f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109770 0x7fa06c1a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.328+0000 7fa0721f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa06c107bc0 0x7fa06c19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.328+0000 7fa0721f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa06c107bc0 0x7fa06c19dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35846/0 (socket says 172.21.15.7:35846) 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.328+0000 7fa0729f6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109770 0x7fa06c1a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.328+0000 7fa0729f6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109770 0x7fa06c1a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58550/0 (socket says 172.21.15.7:58550) 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.328+0000 7fa0731f7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c113890 con 0x7fa06c108db0 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.329+0000 7fa0731f7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c113710 con 0x7fa06c107bc0 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.329+0000 7fa0731f7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c113a10 con 0x7fa06c109770 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.329+0000 7fa0721f5640 1 -- 172.21.15.7:0/2727510138 learned_addr learned my addr 172.21.15.7:0/2727510138 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.329+0000 7fa0721f5640 1 -- 172.21.15.7:0/2727510138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109770 msgr2=0x7fa06c1a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.329+0000 7fa0721f5640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109770 0x7fa06c1a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.329+0000 7fa0721f5640 1 -- 172.21.15.7:0/2727510138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa06c108db0 msgr2=0x7fa06c19e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.329+0000 7fa0721f5640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa06c108db0 0x7fa06c19e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.329+0000 7fa0721f5640 1 -- 172.21.15.7:0/2727510138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c1a5890 con 0x7fa06c107bc0 2024-03-26T08:43:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.330+0000 7fa0721f5640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa06c107bc0 0x7fa06c19dc80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00e9c0 tx=0x7fa05c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.330+0000 7fa0637fe640 1 -- 172.21.15.7:0/2727510138 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00cd80 con 0x7fa06c107bc0 2024-03-26T08:43:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.330+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c1a5b80 con 0x7fa06c107bc0 2024-03-26T08:43:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.330+0000 7fa0637fe640 1 -- 172.21.15.7:0/2727510138 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa05c0035f0 con 0x7fa06c107bc0 2024-03-26T08:43:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.330+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c1a6080 con 0x7fa06c107bc0 2024-03-26T08:43:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.330+0000 7fa0637fe640 1 -- 172.21.15.7:0/2727510138 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c010690 con 0x7fa06c107bc0 2024-03-26T08:43:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.331+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c10eb40 con 0x7fa06c107bc0 2024-03-26T08:43:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.333+0000 7fa0637fe640 1 -- 172.21.15.7:0/2727510138 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fa05c0037a0 con 0x7fa06c107bc0 2024-03-26T08:43:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.333+0000 7fa0637fe640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0580a2880 0x7fa0580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.334+0000 7fa0637fe640 1 -- 172.21.15.7:0/2727510138 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa05c0cefd0 con 0x7fa06c107bc0 2024-03-26T08:43:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.334+0000 7fa0719f4640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0580a2880 0x7fa0580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.336+0000 7fa0719f4640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0580a2880 0x7fa0580a4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa0680058c0 tx=0x7fa068005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.338+0000 7fa0637fe640 1 -- 172.21.15.7:0/2727510138 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa05c09bfd0 con 0x7fa06c107bc0 2024-03-26T08:43:19.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.755+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa06c02d070 con 0x7fa06c107bc0 2024-03-26T08:43:19.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.757+0000 7fa0637fe640 1 -- 172.21.15.7:0/2727510138 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fa05c09b8d0 con 0x7fa06c107bc0 2024-03-26T08:43:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:43:19.749 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:42:21.061940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24502,24512,24478]},"id":1}]} 2024-03-26T08:43:19.750 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0580a2880 msgr2=0x7fa0580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0580a2880 0x7fa0580a4d40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa0680058c0 tx=0x7fa068005850 comp rx=0 tx=0).stop 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa06c107bc0 msgr2=0x7fa06c19dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa06c107bc0 0x7fa06c19dc80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00e9c0 tx=0x7fa05c00ee80 comp rx=0 tx=0).stop 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 shutdown_connections 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0580a2880 0x7fa0580a4d40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa06c109770 0x7fa06c1a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa06c108db0 0x7fa06c19e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 --2- 172.21.15.7:0/2727510138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa06c107bc0 0x7fa06c19dc80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 >> 172.21.15.7:0/2727510138 conn(0x7fa06c076b60 msgr2=0x7fa06c078400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 shutdown_connections 2024-03-26T08:43:19.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.760+0000 7fa0731f7640 1 -- 172.21.15.7:0/2727510138 wait complete. 2024-03-26T08:43:19.769 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:43:19.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:43:19.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.953+0000 7fd3c5777640 1 -- 172.21.15.7:0/3420223807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3c00fffd0 msgr2=0x7fd3c0100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.953+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3420223807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3c00fffd0 0x7fd3c0100ad0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009860 tx=0x7fd3b402bb70 comp rx=0 tx=0).stop 2024-03-26T08:43:19.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.953+0000 7fd3c5777640 1 -- 172.21.15.7:0/3420223807 shutdown_connections 2024-03-26T08:43:19.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.953+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3420223807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c0101010 0x7fd3c0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.953+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3420223807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3c00fffd0 0x7fd3c0100ad0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.953+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3420223807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c00ff670 0x7fd3c00ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.953+0000 7fd3c5777640 1 -- 172.21.15.7:0/3420223807 >> 172.21.15.7:0/3420223807 conn(0x7fd3c00fae50 msgr2=0x7fd3c00fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.954+0000 7fd3c5777640 1 -- 172.21.15.7:0/3420223807 shutdown_connections 2024-03-26T08:43:19.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.954+0000 7fd3c5777640 1 -- 172.21.15.7:0/3420223807 wait complete. 2024-03-26T08:43:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.954+0000 7fd3c5777640 1 Processor -- start 2024-03-26T08:43:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3c5777640 1 -- start start 2024-03-26T08:43:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3c5777640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c00ff670 0x7fd3c0199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3c5777640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c00fffd0 0x7fd3c019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3c5777640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3c0101010 0x7fd3c01a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3c5777640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c010f5d0 con 0x7fd3c0101010 2024-03-26T08:43:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3c5777640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c010f450 con 0x7fd3c00fffd0 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3c5777640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c010f750 con 0x7fd3c00ff670 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3c0101010 0x7fd3c01a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3be7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c00fffd0 0x7fd3c019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3be7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c00fffd0 0x7fd3c019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35862/0 (socket says 172.21.15.7:35862) 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3be7fc640 1 -- 172.21.15.7:0/3973860269 learned_addr learned my addr 172.21.15.7:0/3973860269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.955+0000 7fd3beffd640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c00ff670 0x7fd3c0199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.956+0000 7fd3be7fc640 1 -- 172.21.15.7:0/3973860269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c00ff670 msgr2=0x7fd3c0199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.956+0000 7fd3be7fc640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c00ff670 0x7fd3c0199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.956+0000 7fd3be7fc640 1 -- 172.21.15.7:0/3973860269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3c0101010 msgr2=0x7fd3c01a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.956+0000 7fd3be7fc640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3c0101010 0x7fd3c01a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.956+0000 7fd3be7fc640 1 -- 172.21.15.7:0/3973860269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c01a1850 con 0x7fd3c00fffd0 2024-03-26T08:43:19.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.956+0000 7fd3be7fc640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c00fffd0 0x7fd3c019a180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4000c00 tx=0x7fd3b4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.957+0000 7fd3a7fff640 1 -- 172.21.15.7:0/3973860269 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4039040 con 0x7fd3c00fffd0 2024-03-26T08:43:19.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.957+0000 7fd3a7fff640 1 -- 172.21.15.7:0/3973860269 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3b4005050 con 0x7fd3c00fffd0 2024-03-26T08:43:19.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.957+0000 7fd3a7fff640 1 -- 172.21.15.7:0/3973860269 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4004bb0 con 0x7fd3c00fffd0 2024-03-26T08:43:19.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.958+0000 7fd3c5777640 1 -- 172.21.15.7:0/3973860269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c01a1ae0 con 0x7fd3c00fffd0 2024-03-26T08:43:19.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.958+0000 7fd3c5777640 1 -- 172.21.15.7:0/3973860269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c01a1f80 con 0x7fd3c00fffd0 2024-03-26T08:43:19.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.963+0000 7fd3c8364740 1 Processor -- start 2024-03-26T08:43:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3a7fff640 1 -- 172.21.15.7:0/3973860269 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fd3b4005200 con 0x7fd3c00fffd0 2024-03-26T08:43:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3c8364740 1 -- start start 2024-03-26T08:43:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 0x55ea7e8d8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d9080 0x55ea7e8ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8e0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3c8364740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e89b610 con 0x55ea7e8de1f0 2024-03-26T08:43:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3c8364740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e83e3c0 con 0x55ea7e8d8690 2024-03-26T08:43:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3c8364740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e7e6d60 con 0x55ea7e8d9080 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d9080 0x55ea7e8ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3beffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d9080 0x55ea7e8ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58566/0 (socket says 172.21.15.7:58566) 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3beffd640 1 -- 172.21.15.7:0/1058697790 learned_addr learned my addr 172.21.15.7:0/1058697790 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.964+0000 7fd3be7fc640 1 --2- 172.21.15.7:0/1058697790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.965+0000 7fd3beffd640 1 -- 172.21.15.7:0/1058697790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 msgr2=0x55ea7e8d8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.965+0000 7fd3beffd640 1 --2- 172.21.15.7:0/1058697790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 0x55ea7e8d8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.965+0000 7fd3beffd640 1 -- 172.21.15.7:0/1058697790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8de1f0 msgr2=0x55ea7e8e0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.965+0000 7fd3beffd640 1 --2- 172.21.15.7:0/1058697790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8e0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.965+0000 7fd3beffd640 1 -- 172.21.15.7:0/1058697790 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea7e5523f0 con 0x55ea7e8d9080 2024-03-26T08:43:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.965+0000 7fd3a7fff640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3a80a2a10 0x7fd3a80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.966+0000 7fd3bf7fe640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3a80a2a10 0x7fd3a80a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.966+0000 7fd3a7fff640 1 -- 172.21.15.7:0/3973860269 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd3b40efd30 con 0x7fd3c00fffd0 2024-03-26T08:43:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.966+0000 7fd3beffd640 1 --2- 172.21.15.7:0/1058697790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d9080 0x55ea7e8ddb80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009c60 tx=0x7fd3b0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d407eadfd77deebb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.966+0000 7fd3bf7fe640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3a80a2a10 0x7fd3a80a4ed0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009ba0 tx=0x7fd3b8010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.967+0000 7fd3a57fa640 1 -- 172.21.15.7:0/1058697790 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0004220 con 0x55ea7e8d9080 2024-03-26T08:43:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.967+0000 7fd3a57fa640 1 -- 172.21.15.7:0/1058697790 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3b00043c0 con 0x55ea7e8d9080 2024-03-26T08:43:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.967+0000 7fd3a57fa640 1 -- 172.21.15.7:0/1058697790 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0004e10 con 0x55ea7e8d9080 2024-03-26T08:43:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.968+0000 7fd3c8364740 1 -- 172.21.15.7:0/1058697790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d9080 msgr2=0x55ea7e8ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.968+0000 7fd3c8364740 1 --2- 172.21.15.7:0/1058697790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d9080 0x55ea7e8ddb80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009c60 tx=0x7fd3b0011810 comp rx=0 tx=0).stop 2024-03-26T08:43:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.968+0000 7fd3c8364740 1 -- 172.21.15.7:0/1058697790 shutdown_connections 2024-03-26T08:43:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.968+0000 7fd3c8364740 1 --2- 172.21.15.7:0/1058697790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.968+0000 7fd3c8364740 1 --2- 172.21.15.7:0/1058697790 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d9080 0x55ea7e8ddb80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.968+0000 7fd3c8364740 1 --2- 172.21.15.7:0/1058697790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 0x55ea7e8d8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.968+0000 7fd3c8364740 1 -- 172.21.15.7:0/1058697790 >> 172.21.15.7:0/1058697790 conn(0x55ea7e8d3c00 msgr2=0x55ea7e8d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.969+0000 7fd3c8364740 1 -- 172.21.15.7:0/1058697790 shutdown_connections 2024-03-26T08:43:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.969+0000 7fd3c8364740 1 -- 172.21.15.7:0/1058697790 wait complete. 2024-03-26T08:43:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3c8364740 1 -- start start 2024-03-26T08:43:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3c8364740 1 Processor -- start 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 0x55ea7e8eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d9080 0x55ea7e8eb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3c8364740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e7fa240 con 0x55ea7e8d9080 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3c8364740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e83a460 con 0x55ea7e8d8690 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3c8364740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e8e11a0 con 0x55ea7e8de1f0 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3c8364740 10 client.0 _ll_register_callbacks cb 0x55ea7e7f9c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 0x55ea7e8eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 0x55ea7e8eb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35874/0 (socket says 172.21.15.7:35874) 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3beffd640 1 -- 172.21.15.7:0/3072796499 learned_addr learned my addr 172.21.15.7:0/3072796499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.971+0000 7fd3bf7fe640 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.972+0000 7fd3bf7fe640 1 -- 172.21.15.7:0/3072796499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 msgr2=0x55ea7e8eb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.972+0000 7fd3bf7fe640 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 0x55ea7e8eb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.972+0000 7fd3bf7fe640 1 -- 172.21.15.7:0/3072796499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d9080 msgr2=0x55ea7e8eb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.972+0000 7fd3bf7fe640 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d9080 0x55ea7e8eb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.972+0000 7fd3bf7fe640 1 -- 172.21.15.7:0/3072796499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea7e61c7d0 con 0x55ea7e8de1f0 2024-03-26T08:43:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.972+0000 7fd3bf7fe640 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f2870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8010010 tx=0x7fd3b80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7299036f6d34217 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.972+0000 7fd3a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:43:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.972+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/3072796499 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8003b70 con 0x55ea7e8de1f0 2024-03-26T08:43:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/3072796499 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd3b8003d10 con 0x55ea7e8de1f0 2024-03-26T08:43:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3c8364740 1 -- 172.21.15.7:0/3072796499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3b00043c0 con 0x55ea7e8de1f0 2024-03-26T08:43:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/3072796499 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3b8004020 con 0x55ea7e8de1f0 2024-03-26T08:43:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3c8364740 10 client.25501 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3c8364740 1 -- 172.21.15.7:0/3072796499 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ea7e8e56b0 con 0x55ea7e8de1f0 2024-03-26T08:43:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/3072796499 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3b00043c0 con 0x55ea7e8de1f0 2024-03-26T08:43:19.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/3072796499 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd3b8023070 con 0x55ea7e8de1f0 2024-03-26T08:43:19.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3c8364740 10 client.25501 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:19.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3c8364740 10 client.25501 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:43:19.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3c8364740 20 client.25501 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:19.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3c8364740 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ea7e8e6e40 0x55ea7e8e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3c8364740 4 client.25501 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:19.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.973+0000 7fd3c8364740 1 -- 172.21.15.7:0/3072796499 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea7e84b900 con 0x55ea7e8e6e40 2024-03-26T08:43:19.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.974+0000 7fd3beffd640 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ea7e8e6e40 0x55ea7e8e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.974+0000 7fd3beffd640 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ea7e8e6e40 0x55ea7e8e92c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60019b0f12d12a0f server_cookie=f4a4449e469cbe65 in_seq=0 out_seq=0 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.974+0000 7fd3a2ff5640 10 client.25501 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/3072796499 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ea7e84b900 con 0x55ea7e8e6e40 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3a2ff5640 10 client.25501 handle_command_reply: tid=0 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3c8364740 1 client.25501 shutdown 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3c8364740 1 -- 172.21.15.7:0/3072796499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 msgr2=0x55ea7e8f2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3c8364740 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f2870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8010010 tx=0x7fd3b80095c0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3c8364740 1 -- 172.21.15.7:0/3072796499 shutdown_connections 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3c8364740 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ea7e8e6e40 0x55ea7e8e92c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3c8364740 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f2870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3c8364740 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d9080 0x55ea7e8eb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3c8364740 1 --2- 172.21.15.7:0/3072796499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d8690 0x55ea7e8eb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3c8364740 1 -- 172.21.15.7:0/3072796499 >> 172.21.15.7:0/3072796499 conn(0x55ea7e8d3c00 msgr2=0x55ea7e8d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.975+0000 7fd3a2ff5640 0 client.25501 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:43:19.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.976+0000 7fd3c8364740 1 -- 172.21.15.7:0/3072796499 shutdown_connections 2024-03-26T08:43:19.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.976+0000 7fd3c8364740 1 -- 172.21.15.7:0/3072796499 wait complete. 2024-03-26T08:43:19.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:19.976+0000 7fd3c8364740 20 client.25501 trim_cache size 0 max 16384 2024-03-26T08:43:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3c8364740 1 Processor -- start 2024-03-26T08:43:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3c8364740 1 -- start start 2024-03-26T08:43:19.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d8690 0x55ea7e8d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d9080 0x55ea7e8f2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3c8364740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01a8e20 con 0x55ea7e8d9080 2024-03-26T08:43:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3c8364740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01a6920 con 0x55ea7e8de1f0 2024-03-26T08:43:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3c8364740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01a49a0 con 0x55ea7e8d8690 2024-03-26T08:43:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35878/0 (socket says 172.21.15.7:35878) 2024-03-26T08:43:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3bf7fe640 1 -- 172.21.15.7:0/285686555 learned_addr learned my addr 172.21.15.7:0/285686555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.000+0000 7fd3beffd640 1 --2- 172.21.15.7:0/285686555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d8690 0x55ea7e8d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3beffd640 1 -- 172.21.15.7:0/285686555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8de1f0 msgr2=0x55ea7e8d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3beffd640 1 --2- 172.21.15.7:0/285686555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3beffd640 1 -- 172.21.15.7:0/285686555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d9080 msgr2=0x55ea7e8f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3beffd640 1 --2- 172.21.15.7:0/285686555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d9080 0x55ea7e8f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3beffd640 1 -- 172.21.15.7:0/285686555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea7e8ec470 con 0x55ea7e8d8690 2024-03-26T08:43:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3bf7fe640 1 --2- 172.21.15.7:0/285686555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3beffd640 1 --2- 172.21.15.7:0/285686555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d8690 0x55ea7e8d2590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0011d10 tx=0x7fd3b0012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0bc8abdb1826977 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3a4ff9640 1 -- 172.21.15.7:0/285686555 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b001d040 con 0x55ea7e8d8690 2024-03-26T08:43:19.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3a4ff9640 1 -- 172.21.15.7:0/285686555 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3b002c2d0 con 0x55ea7e8d8690 2024-03-26T08:43:19.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.001+0000 7fd3a4ff9640 1 -- 172.21.15.7:0/285686555 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0003c50 con 0x55ea7e8d8690 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.002+0000 7fd3c8364740 1 -- 172.21.15.7:0/285686555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d8690 msgr2=0x55ea7e8d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.002+0000 7fd3c8364740 1 --2- 172.21.15.7:0/285686555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d8690 0x55ea7e8d2590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0011d10 tx=0x7fd3b0012f90 comp rx=0 tx=0).stop 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.002+0000 7fd3c8364740 1 -- 172.21.15.7:0/285686555 shutdown_connections 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.002+0000 7fd3c8364740 1 --2- 172.21.15.7:0/285686555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.002+0000 7fd3c8364740 1 --2- 172.21.15.7:0/285686555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d9080 0x55ea7e8f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.002+0000 7fd3c8364740 1 --2- 172.21.15.7:0/285686555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8d8690 0x55ea7e8d2590 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.002+0000 7fd3c8364740 1 -- 172.21.15.7:0/285686555 >> 172.21.15.7:0/285686555 conn(0x55ea7e8d3c00 msgr2=0x55ea7e8da5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.003+0000 7fd3c8364740 1 -- 172.21.15.7:0/285686555 shutdown_connections 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.003+0000 7fd3c8364740 1 -- 172.21.15.7:0/285686555 wait complete. 2024-03-26T08:43:19.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.004+0000 7fd3c8364740 1 -- start start 2024-03-26T08:43:19.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.004+0000 7fd3c8364740 1 Processor -- start 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.004+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d8690 0x55ea7e8ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.004+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d9080 0x55ea7e8eee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.004+0000 7fd3c8364740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.004+0000 7fd3c8364740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01a4fa0 con 0x55ea7e8d8690 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.004+0000 7fd3c8364740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01a62c0 con 0x55ea7e8d9080 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.004+0000 7fd3c8364740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01a5600 con 0x55ea7e8de1f0 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.004+0000 7fd3c8364740 10 client.0 _ll_register_callbacks cb 0x7fd3b8019ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.005+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.005+0000 7fd3bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58604/0 (socket says 172.21.15.7:58604) 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.005+0000 7fd3bf7fe640 1 -- 172.21.15.7:0/2607922217 learned_addr learned my addr 172.21.15.7:0/2607922217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.005+0000 7fd3be7fc640 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d9080 0x55ea7e8eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.005+0000 7fd3be7fc640 1 -- 172.21.15.7:0/2607922217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 msgr2=0x55ea7e8f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.005+0000 7fd3be7fc640 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.005+0000 7fd3be7fc640 1 -- 172.21.15.7:0/2607922217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d8690 msgr2=0x55ea7e8ee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.005+0000 7fd3be7fc640 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d8690 0x55ea7e8ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.005+0000 7fd3be7fc640 1 -- 172.21.15.7:0/2607922217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea7e8f2410 con 0x55ea7e8d9080 2024-03-26T08:43:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.006+0000 7fd3be7fc640 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d9080 0x55ea7e8eee00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40ed7f0 tx=0x7fd3b40f3450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb85997f92671dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.006+0000 7fd3a2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.006+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/2607922217 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40b8940 con 0x55ea7e8d9080 2024-03-26T08:43:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.006+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/2607922217 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd3b40c4b90 con 0x55ea7e8d9080 2024-03-26T08:43:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.006+0000 7fd3c8364740 1 -- 172.21.15.7:0/2607922217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3b002c2d0 con 0x55ea7e8d9080 2024-03-26T08:43:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.007+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/2607922217 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3b40ba870 con 0x55ea7e8d9080 2024-03-26T08:43:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.008+0000 7fd3c8364740 10 client.25496 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.008+0000 7fd3c8364740 1 -- 172.21.15.7:0/2607922217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ea7e95b3b0 con 0x55ea7e8d9080 2024-03-26T08:43:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.009+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/2607922217 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3b002c2d0 con 0x55ea7e8d9080 2024-03-26T08:43:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.009+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/2607922217 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd3b403c070 con 0x55ea7e8d9080 2024-03-26T08:43:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.009+0000 7fd3c8364740 10 client.25496 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.009+0000 7fd3c8364740 10 client.25496 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:43:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.009+0000 7fd3c8364740 20 client.25496 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.009+0000 7fd3c8364740 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ea7e95bb20 0x55ea7e95df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.009+0000 7fd3c8364740 4 client.25496 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:43:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.009+0000 7fd3c8364740 1 -- 172.21.15.7:0/2607922217 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea7e845c50 con 0x55ea7e95bb20 2024-03-26T08:43:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.010+0000 7fd3beffd640 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ea7e95bb20 0x55ea7e95df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3a2ff5640 10 client.25496 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:43:20.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3beffd640 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ea7e95bb20 0x55ea7e95df40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffb7c6bc75d5c06d server_cookie=7729fb5cedc5441 in_seq=0 out_seq=0 2024-03-26T08:43:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3a2ff5640 1 -- 172.21.15.7:0/2607922217 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ea7e845c50 con 0x55ea7e95bb20 2024-03-26T08:43:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3a2ff5640 10 client.25496 handle_command_reply: tid=0 2024-03-26T08:43:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3c8364740 1 client.25496 shutdown 2024-03-26T08:43:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3c8364740 1 -- 172.21.15.7:0/2607922217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d9080 msgr2=0x55ea7e8eee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3c8364740 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d9080 0x55ea7e8eee00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3b40ed7f0 tx=0x7fd3b40f3450 comp rx=0 tx=0).stop 2024-03-26T08:43:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3c8364740 1 -- 172.21.15.7:0/2607922217 shutdown_connections 2024-03-26T08:43:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3c8364740 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ea7e95bb20 0x55ea7e95df40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3c8364740 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ea7e8de1f0 0x55ea7e8f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3c8364740 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea7e8d9080 0x55ea7e8eee00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3c8364740 1 --2- 172.21.15.7:0/2607922217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ea7e8d8690 0x55ea7e8ee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3c8364740 1 -- 172.21.15.7:0/2607922217 >> 172.21.15.7:0/2607922217 conn(0x55ea7e8d3c00 msgr2=0x55ea7e8d6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.011+0000 7fd3a2ff5640 0 client.25496 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:43:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.012+0000 7fd3c8364740 1 -- 172.21.15.7:0/2607922217 shutdown_connections 2024-03-26T08:43:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.012+0000 7fd3c8364740 1 -- 172.21.15.7:0/2607922217 wait complete. 2024-03-26T08:43:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.013+0000 7fd3c8364740 20 client.25496 trim_cache size 0 max 16384 2024-03-26T08:43:20.006 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:43:20.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.013+0000 7fd3c5777640 1 -- 172.21.15.7:0/3973860269 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3a80a2a10 msgr2=0x7fd3a80a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.013+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3a80a2a10 0x7fd3a80a4ed0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009ba0 tx=0x7fd3b8010040 comp rx=0 tx=0).stop 2024-03-26T08:43:20.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.013+0000 7fd3c5777640 1 -- 172.21.15.7:0/3973860269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c00fffd0 msgr2=0x7fd3c019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.013+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c00fffd0 0x7fd3c019a180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4000c00 tx=0x7fd3b4004270 comp rx=0 tx=0).stop 2024-03-26T08:43:20.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.015+0000 7fd3c5777640 1 -- 172.21.15.7:0/3973860269 shutdown_connections 2024-03-26T08:43:20.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.015+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd3a80a2a10 0x7fd3a80a4ed0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.015+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3c0101010 0x7fd3c01a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.015+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c00fffd0 0x7fd3c019a180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.015+0000 7fd3c5777640 1 --2- 172.21.15.7:0/3973860269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c00ff670 0x7fd3c0199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.015+0000 7fd3c5777640 1 -- 172.21.15.7:0/3973860269 >> 172.21.15.7:0/3973860269 conn(0x7fd3c00fae50 msgr2=0x7fd3c00fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.015+0000 7fd3c5777640 1 -- 172.21.15.7:0/3973860269 shutdown_connections 2024-03-26T08:43:20.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.015+0000 7fd3c5777640 1 -- 172.21.15.7:0/3973860269 wait complete. 2024-03-26T08:43:20.022 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:19 smithi007 ceph-mon[32462]: pgmap v387: 97 pgs: 97 active+clean; 1020 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 804 op/s 2024-03-26T08:43:20.023 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:19 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2727510138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:43:20.024 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:43:20.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:43:20.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.209+0000 7f8316cbf640 1 -- 172.21.15.7:0/1094935616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310109890 msgr2=0x7f8310110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.209+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1094935616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310109890 0x7f8310110120 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8308009860 tx=0x7f830802bb70 comp rx=0 tx=0).stop 2024-03-26T08:43:20.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.209+0000 7f8316cbf640 1 -- 172.21.15.7:0/1094935616 shutdown_connections 2024-03-26T08:43:20.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.209+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1094935616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310109890 0x7f8310110120 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.209+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1094935616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8310108ed0 0x7f8310109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.209+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1094935616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8310107ce0 0x7f8310108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.209+0000 7f8316cbf640 1 -- 172.21.15.7:0/1094935616 >> 172.21.15.7:0/1094935616 conn(0x7f8310076c90 msgr2=0x7f83100790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.209+0000 7f8316cbf640 1 -- 172.21.15.7:0/1094935616 shutdown_connections 2024-03-26T08:43:20.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.210+0000 7f8316cbf640 1 -- 172.21.15.7:0/1094935616 wait complete. 2024-03-26T08:43:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f8316cbf640 1 Processor -- start 2024-03-26T08:43:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f8316cbf640 1 -- start start 2024-03-26T08:43:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f8316cbf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8310107ce0 0x7f831019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f8316cbf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310108ed0 0x7f831019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f8316cbf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8310109890 0x7f83101a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f8316cbf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8310113940 con 0x7f8310107ce0 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f8316cbf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83101137c0 con 0x7f8310108ed0 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f8316cbf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8310113ac0 con 0x7f8310109890 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f830ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310108ed0 0x7f831019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f830ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310108ed0 0x7f831019e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35904/0 (socket says 172.21.15.7:35904) 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f830ffff640 1 -- 172.21.15.7:0/1382294880 learned_addr learned my addr 172.21.15.7:0/1382294880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f831525c640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8310109890 0x7f83101a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.211+0000 7f8314a5b640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8310107ce0 0x7f831019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f830ffff640 1 -- 172.21.15.7:0/1382294880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8310109890 msgr2=0x7f83101a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f830ffff640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8310109890 0x7f83101a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f830ffff640 1 -- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8310107ce0 msgr2=0x7f831019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f830ffff640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8310107ce0 0x7f831019dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f830ffff640 1 -- 172.21.15.7:0/1382294880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83101a5990 con 0x7f8310108ed0 2024-03-26T08:43:20.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f8314a5b640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8310107ce0 0x7f831019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:43:20.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f830ffff640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310108ed0 0x7f831019e2b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f82f800e9f0 tx=0x7f82f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f830dffb640 1 -- 172.21.15.7:0/1382294880 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800cd80 con 0x7f8310108ed0 2024-03-26T08:43:20.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f830dffb640 1 -- 172.21.15.7:0/1382294880 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82f8010910 con 0x7f8310108ed0 2024-03-26T08:43:20.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f830dffb640 1 -- 172.21.15.7:0/1382294880 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80188f0 con 0x7f8310108ed0 2024-03-26T08:43:20.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f8316cbf640 1 -- 172.21.15.7:0/1382294880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83101a5c80 con 0x7f8310108ed0 2024-03-26T08:43:20.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.213+0000 7f8316cbf640 1 -- 172.21.15.7:0/1382294880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83101a6180 con 0x7f8310108ed0 2024-03-26T08:43:20.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.217+0000 7f830dffb640 1 -- 172.21.15.7:0/1382294880 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f82f8010ac0 con 0x7f8310108ed0 2024-03-26T08:43:20.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.218+0000 7f830dffb640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82fc0a2a60 0x7f82fc0a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.218+0000 7f830dffb640 1 -- 172.21.15.7:0/1382294880 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f82f80cef20 con 0x7f8310108ed0 2024-03-26T08:43:20.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.219+0000 7f8314a5b640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82fc0a2a60 0x7f82fc0a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.219+0000 7f8319988740 1 Processor -- start 2024-03-26T08:43:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.220+0000 7f8314a5b640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82fc0a2a60 0x7f82fc0a4f20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8310108d30 tx=0x7f83000095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.220+0000 7f8319988740 1 -- start start 2024-03-26T08:43:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.220+0000 7f8319988740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 0x5584b08adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.220+0000 7f8319988740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 0x5584b08b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.220+0000 7f8319988740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 0x5584b08b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.220+0000 7f8319988740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b0870610 con 0x5584b08ae080 2024-03-26T08:43:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.220+0000 7f8319988740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b08133c0 con 0x5584b08ad690 2024-03-26T08:43:20.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.220+0000 7f8319988740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b07bbd60 con 0x5584b08b31f0 2024-03-26T08:43:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.220+0000 7f831525c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 0x5584b08b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.221+0000 7f831525c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 0x5584b08b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58634/0 (socket says 172.21.15.7:58634) 2024-03-26T08:43:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.221+0000 7f831525c640 1 -- 172.21.15.7:0/572778173 learned_addr learned my addr 172.21.15.7:0/572778173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.221+0000 7f8314a5b640 1 --2- 172.21.15.7:0/572778173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 0x5584b08adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.221+0000 7f8314a5b640 1 -- 172.21.15.7:0/572778173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 msgr2=0x5584b08b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.221+0000 7f8314a5b640 1 --2- 172.21.15.7:0/572778173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 0x5584b08b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.221+0000 7f8314a5b640 1 -- 172.21.15.7:0/572778173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 msgr2=0x5584b08b2b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:43:20.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.221+0000 7f8314a5b640 1 --2- 172.21.15.7:0/572778173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 0x5584b08b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.221+0000 7f8314a5b640 1 -- 172.21.15.7:0/572778173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584b05273f0 con 0x5584b08ad690 2024-03-26T08:43:20.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.222+0000 7f8314a5b640 1 --2- 172.21.15.7:0/572778173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 0x5584b08adab0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8300005fd0 tx=0x7f8300014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af738e40d94e448 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.222+0000 7f8306ffd640 1 -- 172.21.15.7:0/572778173 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830002b070 con 0x5584b08ad690 2024-03-26T08:43:20.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.222+0000 7f8306ffd640 1 -- 172.21.15.7:0/572778173 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f830001be00 con 0x5584b08ad690 2024-03-26T08:43:20.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.222+0000 7f8306ffd640 1 -- 172.21.15.7:0/572778173 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830000c700 con 0x5584b08ad690 2024-03-26T08:43:20.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.223+0000 7f8319988740 1 -- 172.21.15.7:0/572778173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 msgr2=0x5584b08adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.223+0000 7f8319988740 1 --2- 172.21.15.7:0/572778173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 0x5584b08adab0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8300005fd0 tx=0x7f8300014000 comp rx=0 tx=0).stop 2024-03-26T08:43:20.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.223+0000 7f8319988740 1 -- 172.21.15.7:0/572778173 shutdown_connections 2024-03-26T08:43:20.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.223+0000 7f8319988740 1 --2- 172.21.15.7:0/572778173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 0x5584b08b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.223+0000 7f8319988740 1 --2- 172.21.15.7:0/572778173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 0x5584b08b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.223+0000 7f8319988740 1 --2- 172.21.15.7:0/572778173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 0x5584b08adab0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.223+0000 7f8319988740 1 -- 172.21.15.7:0/572778173 >> 172.21.15.7:0/572778173 conn(0x5584b08a8c00 msgr2=0x5584b08a9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.223+0000 7f8319988740 1 -- 172.21.15.7:0/572778173 shutdown_connections 2024-03-26T08:43:20.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.224+0000 7f8319988740 1 -- 172.21.15.7:0/572778173 wait complete. 2024-03-26T08:43:20.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.224+0000 7f8319988740 1 -- start start 2024-03-26T08:43:20.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.224+0000 7f8319988740 1 Processor -- start 2024-03-26T08:43:20.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.225+0000 7f8319988740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 0x5584b08c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.225+0000 7f8319988740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 0x5584b08c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.225+0000 7f8319988740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 0x5584b08d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.225+0000 7f8319988740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b07cf240 con 0x5584b08ae080 2024-03-26T08:43:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.225+0000 7f8319988740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b080f460 con 0x5584b08ad690 2024-03-26T08:43:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.225+0000 7f8319988740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584b08c71b0 con 0x5584b08b31f0 2024-03-26T08:43:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.225+0000 7f8319988740 10 client.0 _ll_register_callbacks cb 0x5584b07cec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.225+0000 7f830ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 0x5584b08c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.226+0000 7f830ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 0x5584b08c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53640/0 (socket says 172.21.15.7:53640) 2024-03-26T08:43:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.226+0000 7f830ffff640 1 -- 172.21.15.7:0/3482934255 learned_addr learned my addr 172.21.15.7:0/3482934255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.226+0000 7f830ffff640 1 -- 172.21.15.7:0/3482934255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 msgr2=0x5584b08d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.226+0000 7f830ffff640 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 0x5584b08d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.226+0000 7f830ffff640 1 -- 172.21.15.7:0/3482934255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 msgr2=0x5584b08c8af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:43:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.226+0000 7f830ffff640 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 0x5584b08c8af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.226+0000 7f830ffff640 1 -- 172.21.15.7:0/3482934255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584b05f17d0 con 0x5584b08ae080 2024-03-26T08:43:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.227+0000 7f830ffff640 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 0x5584b08c9030 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c6a0 tx=0x7f82f809bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7dde6ac5368f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.227+0000 7f82effff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:43:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.227+0000 7f82effff640 1 -- 172.21.15.7:0/3482934255 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80b4060 con 0x5584b08ae080 2024-03-26T08:43:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.227+0000 7f82effff640 1 -- 172.21.15.7:0/3482934255 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f82f809ab70 con 0x5584b08ae080 2024-03-26T08:43:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.227+0000 7f8319988740 1 -- 172.21.15.7:0/3482934255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f830001be00 con 0x5584b08ae080 2024-03-26T08:43:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.227+0000 7f82effff640 1 -- 172.21.15.7:0/3482934255 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f80b9050 con 0x5584b08ae080 2024-03-26T08:43:20.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.227+0000 7f8319988740 10 client.15666 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.227+0000 7f8319988740 1 -- 172.21.15.7:0/3482934255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5584b08d34b0 con 0x5584b08ae080 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.228+0000 7f82effff640 1 -- 172.21.15.7:0/3482934255 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f830001be00 con 0x5584b08ae080 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.228+0000 7f82effff640 1 -- 172.21.15.7:0/3482934255 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f82f8097500 con 0x5584b08ae080 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.228+0000 7f8319988740 10 client.15666 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.228+0000 7f8319988740 10 client.15666 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.228+0000 7f8319988740 20 client.15666 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.228+0000 7f8319988740 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5584b08d4b60 0x5584b08d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.228+0000 7f8319988740 4 client.15666 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.228+0000 7f8319988740 1 -- 172.21.15.7:0/3482934255 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584b0820900 con 0x5584b08d4b60 2024-03-26T08:43:20.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.228+0000 7f8314a5b640 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5584b08d4b60 0x5584b08d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.229+0000 7f82effff640 10 client.15666 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:43:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.229+0000 7f8314a5b640 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5584b08d4b60 0x5584b08d6fc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f79778da14b4f85 server_cookie=7989889e02ac1c95 in_seq=0 out_seq=0 2024-03-26T08:43:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.229+0000 7f82effff640 1 -- 172.21.15.7:0/3482934255 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5584b0820900 con 0x5584b08d4b60 2024-03-26T08:43:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.229+0000 7f82effff640 10 client.15666 handle_command_reply: tid=0 2024-03-26T08:43:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 client.15666 shutdown 2024-03-26T08:43:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 -- 172.21.15.7:0/3482934255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 msgr2=0x5584b08c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 0x5584b08c9030 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f82f800c6a0 tx=0x7f82f809bee0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 -- 172.21.15.7:0/3482934255 shutdown_connections 2024-03-26T08:43:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5584b08d4b60 0x5584b08d6fc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08b31f0 0x5584b08d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ae080 0x5584b08c9030 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 --2- 172.21.15.7:0/3482934255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ad690 0x5584b08c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 -- 172.21.15.7:0/3482934255 >> 172.21.15.7:0/3482934255 conn(0x5584b08a8c00 msgr2=0x5584b08a7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f82effff640 0 client.15666 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:43:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 -- 172.21.15.7:0/3482934255 shutdown_connections 2024-03-26T08:43:20.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.230+0000 7f8319988740 1 -- 172.21.15.7:0/3482934255 wait complete. 2024-03-26T08:43:20.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.231+0000 7f8319988740 20 client.15666 trim_cache size 0 max 16384 2024-03-26T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:19 smithi016 ceph-mon[35851]: pgmap v387: 97 pgs: 97 active+clean; 1020 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 804 op/s 2024-03-26T08:43:20.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:19 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2727510138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:43:20.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8319988740 1 Processor -- start 2024-03-26T08:43:20.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8319988740 1 -- start start 2024-03-26T08:43:20.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8319988740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ad690 0x5584b08a7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8319988740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ae080 0x5584b08d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8314a5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ad690 0x5584b08a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8314a5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ad690 0x5584b08a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53652/0 (socket says 172.21.15.7:53652) 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8319988740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08b31f0 0x5584b08a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8314a5b640 1 -- 172.21.15.7:0/3054113400 learned_addr learned my addr 172.21.15.7:0/3054113400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8319988740 1 -- 172.21.15.7:0/3054113400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83101a8d00 con 0x5584b08ad690 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8319988740 1 -- 172.21.15.7:0/3054113400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83101a6800 con 0x5584b08b31f0 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f8319988740 1 -- 172.21.15.7:0/3054113400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8310071e00 con 0x5584b08ae080 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f831525c640 1 --2- 172.21.15.7:0/3054113400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08b31f0 0x5584b08a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f830ffff640 1 --2- 172.21.15.7:0/3054113400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ae080 0x5584b08d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f830ffff640 1 -- 172.21.15.7:0/3054113400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08b31f0 msgr2=0x5584b08a7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f830ffff640 1 --2- 172.21.15.7:0/3054113400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08b31f0 0x5584b08a7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f830ffff640 1 -- 172.21.15.7:0/3054113400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ad690 msgr2=0x5584b08a7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f830ffff640 1 --2- 172.21.15.7:0/3054113400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ad690 0x5584b08a7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f830ffff640 1 -- 172.21.15.7:0/3054113400 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584b08a4480 con 0x5584b08ae080 2024-03-26T08:43:20.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.255+0000 7f830ffff640 1 --2- 172.21.15.7:0/3054113400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ae080 0x5584b08d0030 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f82f809bfd0 tx=0x7f82f809f9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c70603619b9cd35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.256+0000 7f83067fc640 1 -- 172.21.15.7:0/3054113400 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f809a6d0 con 0x5584b08ae080 2024-03-26T08:43:20.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.256+0000 7f83067fc640 1 -- 172.21.15.7:0/3054113400 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82f80a3ec0 con 0x5584b08ae080 2024-03-26T08:43:20.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.256+0000 7f83067fc640 1 -- 172.21.15.7:0/3054113400 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f80979c0 con 0x5584b08ae080 2024-03-26T08:43:20.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 -- 172.21.15.7:0/3054113400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ae080 msgr2=0x5584b08d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 --2- 172.21.15.7:0/3054113400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ae080 0x5584b08d0030 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f82f809bfd0 tx=0x7f82f809f9f0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 -- 172.21.15.7:0/3054113400 shutdown_connections 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 --2- 172.21.15.7:0/3054113400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08b31f0 0x5584b08a7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 --2- 172.21.15.7:0/3054113400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ae080 0x5584b08d0030 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 --2- 172.21.15.7:0/3054113400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08ad690 0x5584b08a7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 -- 172.21.15.7:0/3054113400 >> 172.21.15.7:0/3054113400 conn(0x5584b08a8c00 msgr2=0x5584b08c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 -- 172.21.15.7:0/3054113400 shutdown_connections 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 -- 172.21.15.7:0/3054113400 wait complete. 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 -- start start 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.257+0000 7f8319988740 1 Processor -- start 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.258+0000 7f8319988740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ad690 0x5584b08cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.258+0000 7f8319988740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ae080 0x5584b08cde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.258+0000 7f8319988740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08b31f0 0x5584b08cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8314a5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ad690 0x5584b08cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8314a5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ad690 0x5584b08cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58656/0 (socket says 172.21.15.7:58656) 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8314a5b640 1 -- 172.21.15.7:0/2300202328 learned_addr learned my addr 172.21.15.7:0/2300202328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f830ffff640 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ae080 0x5584b08cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8310072400 con 0x5584b08b31f0 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8310073720 con 0x5584b08ae080 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8310072a60 con 0x5584b08ad690 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8314a5b640 1 -- 172.21.15.7:0/2300202328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ae080 msgr2=0x5584b08cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8314a5b640 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ae080 0x5584b08cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8314a5b640 1 -- 172.21.15.7:0/2300202328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08b31f0 msgr2=0x5584b08cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8314a5b640 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08b31f0 0x5584b08cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.259+0000 7f8314a5b640 1 -- 172.21.15.7:0/2300202328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584b08cf460 con 0x5584b08ad690 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.260+0000 7f8319988740 10 client.0 _ll_register_callbacks cb 0x7f82f809b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.260+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82f80a3ec0 con 0x5584b08ad690 2024-03-26T08:43:20.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.260+0000 7f8314a5b640 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ad690 0x5584b08cd920 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8300005fd0 tx=0x7f8300009a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6bb8932c995084 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.260+0000 7f82effff640 10 client.25522 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:43:20.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.260+0000 7f82effff640 1 -- 172.21.15.7:0/2300202328 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300041050 con 0x5584b08ad690 2024-03-26T08:43:20.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.260+0000 7f82effff640 1 -- 172.21.15.7:0/2300202328 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8300021040 con 0x5584b08ad690 2024-03-26T08:43:20.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.260+0000 7f82effff640 1 -- 172.21.15.7:0/2300202328 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8300037070 con 0x5584b08ad690 2024-03-26T08:43:20.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.260+0000 7f8319988740 10 client.25522 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:20.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.260+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x5584b0930580 con 0x5584b08ad690 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.261+0000 7f82effff640 1 -- 172.21.15.7:0/2300202328 <== mon.1 v2:172.21.15.44:3300/0 4 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f830000b040 con 0x5584b08ad690 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.261+0000 7f8319988740 10 client.25522 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.261+0000 7f8319988740 10 client.25522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.261+0000 7f8319988740 20 client.25522 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.261+0000 7f8319988740 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5584b0930ac0 0x5584b0932eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.261+0000 7f8319988740 4 client.25522 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.261+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584b081ac50 con 0x5584b0930ac0 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.261+0000 7f830ffff640 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5584b0930ac0 0x5584b0932eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f82effff640 10 client.25522 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f830ffff640 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5584b0930ac0 0x5584b0932eb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93d53e435a82ed32 server_cookie=a3f1b340929a8780 in_seq=0 out_seq=0 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f82effff640 1 -- 172.21.15.7:0/2300202328 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584b081ac50 con 0x5584b0930ac0 2024-03-26T08:43:20.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f82effff640 10 client.25522 handle_command_reply: tid=0 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f8319988740 1 client.25522 shutdown 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ad690 msgr2=0x5584b08cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f8319988740 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ad690 0x5584b08cd920 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8300005fd0 tx=0x7f8300009a80 comp rx=0 tx=0).stop 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 shutdown_connections 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f8319988740 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584b08b31f0 0x5584b08cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f8319988740 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584b08ae080 0x5584b08cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f8319988740 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5584b0930ac0 0x5584b0932eb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f82effff640 0 client.25522 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f8319988740 1 --2- 172.21.15.7:0/2300202328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5584b08ad690 0x5584b08cd920 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.262+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 >> 172.21.15.7:0/2300202328 conn(0x5584b08a8c00 msgr2=0x5584b08abe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.263+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 shutdown_connections 2024-03-26T08:43:20.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.263+0000 7f8319988740 1 -- 172.21.15.7:0/2300202328 wait complete. 2024-03-26T08:43:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.263+0000 7f8319988740 20 client.25522 trim_cache size 0 max 16384 2024-03-26T08:43:20.259 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:43:20.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 -- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82fc0a2a60 msgr2=0x7f82fc0a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82fc0a2a60 0x7f82fc0a4f20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8310108d30 tx=0x7f83000095c0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 -- 172.21.15.7:0/1382294880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310108ed0 msgr2=0x7f831019e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310108ed0 0x7f831019e2b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f82f800e9f0 tx=0x7f82f800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 -- 172.21.15.7:0/1382294880 shutdown_connections 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f82fc0a2a60 0x7f82fc0a4f20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8310109890 0x7f83101a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8310108ed0 0x7f831019e2b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 --2- 172.21.15.7:0/1382294880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8310107ce0 0x7f831019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 -- 172.21.15.7:0/1382294880 >> 172.21.15.7:0/1382294880 conn(0x7f8310076c90 msgr2=0x7f831010c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 -- 172.21.15.7:0/1382294880 shutdown_connections 2024-03-26T08:43:20.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.265+0000 7f8316cbf640 1 -- 172.21.15.7:0/1382294880 wait complete. 2024-03-26T08:43:20.275 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:43:20.276 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:43:20.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:19 smithi044 ceph-mon[36461]: pgmap v387: 97 pgs: 97 active+clean; 1020 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 804 op/s 2024-03-26T08:43:20.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:19 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2727510138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:43:20.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.459+0000 7f78c5d62640 1 -- 172.21.15.7:0/2059582872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c010d9c0 msgr2=0x7f78c010de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.459+0000 7f78c5d62640 1 --2- 172.21.15.7:0/2059582872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c010d9c0 0x7f78c010de40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009be0 tx=0x7f78b802bef0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.459+0000 7f78c5d62640 1 -- 172.21.15.7:0/2059582872 shutdown_connections 2024-03-26T08:43:20.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.459+0000 7f78c5d62640 1 --2- 172.21.15.7:0/2059582872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c010d9c0 0x7f78c010de40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.459+0000 7f78c5d62640 1 --2- 172.21.15.7:0/2059582872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c010b010 0x7f78c010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.459+0000 7f78c5d62640 1 --2- 172.21.15.7:0/2059582872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c01086a0 0x7f78c010aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.459+0000 7f78c5d62640 1 -- 172.21.15.7:0/2059582872 >> 172.21.15.7:0/2059582872 conn(0x7f78c006e640 msgr2=0x7f78c0070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.459+0000 7f78c5d62640 1 -- 172.21.15.7:0/2059582872 shutdown_connections 2024-03-26T08:43:20.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.459+0000 7f78c5d62640 1 -- 172.21.15.7:0/2059582872 wait complete. 2024-03-26T08:43:20.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.460+0000 7f78c5d62640 1 Processor -- start 2024-03-26T08:43:20.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78c5d62640 1 -- start start 2024-03-26T08:43:20.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78c5d62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c01086a0 0x7f78c0119890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78c5d62640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c010b010 0x7f78c0119dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c01086a0 0x7f78c0119890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78c5d62640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c010d9c0 0x7f78c0117ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78c5d62640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c01107e0 con 0x7f78c010b010 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78c5d62640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c0110660 con 0x7f78c01086a0 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78c5d62640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c0110960 con 0x7f78c010d9c0 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c01086a0 0x7f78c0119890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35940/0 (socket says 172.21.15.7:35940) 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c010b010 0x7f78c0119dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.461+0000 7f78bf7fe640 1 -- 172.21.15.7:0/62016451 learned_addr learned my addr 172.21.15.7:0/62016451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78bffff640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c010d9c0 0x7f78c0117ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78bf7fe640 1 -- 172.21.15.7:0/62016451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c010d9c0 msgr2=0x7f78c0117ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78bf7fe640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c010d9c0 0x7f78c0117ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78bf7fe640 1 -- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c010b010 msgr2=0x7f78c0119dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78bf7fe640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c010b010 0x7f78c0119dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78bf7fe640 1 -- 172.21.15.7:0/62016451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c01185d0 con 0x7f78c01086a0 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78bffff640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c010d9c0 0x7f78c0117ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:20.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78beffd640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c010b010 0x7f78c0119dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:20.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78bf7fe640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c01086a0 0x7f78c0119890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f78b000e9f0 tx=0x7f78b000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.462+0000 7f78bcff9640 1 -- 172.21.15.7:0/62016451 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b000ccc0 con 0x7f78c01086a0 2024-03-26T08:43:20.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.463+0000 7f78bcff9640 1 -- 172.21.15.7:0/62016451 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78b000ce60 con 0x7f78c01086a0 2024-03-26T08:43:20.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.463+0000 7f78bcff9640 1 -- 172.21.15.7:0/62016451 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0010690 con 0x7f78c01086a0 2024-03-26T08:43:20.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.463+0000 7f78c5d62640 1 -- 172.21.15.7:0/62016451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c01188c0 con 0x7f78c01086a0 2024-03-26T08:43:20.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.463+0000 7f78c5d62640 1 -- 172.21.15.7:0/62016451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c01b7460 con 0x7f78c01086a0 2024-03-26T08:43:20.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.466+0000 7f78c8a2b740 1 Processor -- start 2024-03-26T08:43:20.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78c8a2b740 1 -- start start 2024-03-26T08:43:20.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa69ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa71680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147aa2c610 con 0x56147aa6a080 2024-03-26T08:43:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a9cf3c0 con 0x56147aa69690 2024-03-26T08:43:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a977d60 con 0x56147aa6f1f0 2024-03-26T08:43:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78bf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78bf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53694/0 (socket says 172.21.15.7:53694) 2024-03-26T08:43:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78bf7fe640 1 -- 172.21.15.7:0/2591389435 learned_addr learned my addr 172.21.15.7:0/2591389435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.467+0000 7f78bffff640 1 --2- 172.21.15.7:0/2591389435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa69ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.468+0000 7f78bffff640 1 -- 172.21.15.7:0/2591389435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 msgr2=0x56147aa71680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:43:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.468+0000 7f78bffff640 1 --2- 172.21.15.7:0/2591389435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa71680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.468+0000 7f78bffff640 1 -- 172.21.15.7:0/2591389435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 msgr2=0x56147aa6eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.468+0000 7f78bffff640 1 --2- 172.21.15.7:0/2591389435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.468+0000 7f78bffff640 1 -- 172.21.15.7:0/2591389435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56147a6e33f0 con 0x56147aa69690 2024-03-26T08:43:20.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.468+0000 7f78bffff640 1 --2- 172.21.15.7:0/2591389435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa69ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f78b8034fd0 tx=0x7f78b8005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209a2a60d0404622 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.468+0000 7f78bcff9640 1 -- 172.21.15.7:0/62016451 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f78b0010830 con 0x7f78c01086a0 2024-03-26T08:43:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.468+0000 7f78a5ffb640 1 -- 172.21.15.7:0/2591389435 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8043030 con 0x56147aa69690 2024-03-26T08:43:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.468+0000 7f78a5ffb640 1 -- 172.21.15.7:0/2591389435 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78b8003970 con 0x56147aa69690 2024-03-26T08:43:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.469+0000 7f78bcff9640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78a80a2990 0x7f78a80a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.469+0000 7f78c8a2b740 1 -- 172.21.15.7:0/2591389435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 msgr2=0x56147aa69ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.469+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/2591389435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa69ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f78b8034fd0 tx=0x7f78b8005e30 comp rx=0 tx=0).stop 2024-03-26T08:43:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.469+0000 7f78bcff9640 1 -- 172.21.15.7:0/62016451 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78b00ceeb0 con 0x7f78c01086a0 2024-03-26T08:43:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.470+0000 7f78c8a2b740 1 -- 172.21.15.7:0/2591389435 shutdown_connections 2024-03-26T08:43:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.470+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/2591389435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa71680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.470+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/2591389435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.470+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/2591389435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa69ab0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.470+0000 7f78c8a2b740 1 -- 172.21.15.7:0/2591389435 >> 172.21.15.7:0/2591389435 conn(0x56147aa64c00 msgr2=0x56147aa65000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.470+0000 7f78c8a2b740 1 -- 172.21.15.7:0/2591389435 shutdown_connections 2024-03-26T08:43:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.470+0000 7f78bf7fe640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78a80a2990 0x7f78a80a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.470+0000 7f78c8a2b740 1 -- 172.21.15.7:0/2591389435 wait complete. 2024-03-26T08:43:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.470+0000 7f78c8a2b740 1 -- start start 2024-03-26T08:43:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78c8a2b740 1 Processor -- start 2024-03-26T08:43:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa73980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa73ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa7af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a98b240 con 0x56147aa69690 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147a9cb460 con 0x56147aa6a080 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56147aa72190 con 0x56147aa6f1f0 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78c8a2b740 10 client.0 _ll_register_callbacks cb 0x56147a98ac10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78bf7fe640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78a80a2990 0x7f78a80a4e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78b009c6f0 tx=0x7f78b0098b10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa73ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.471+0000 7f78beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa73ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35960/0 (socket says 172.21.15.7:35960) 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.472+0000 7f78beffd640 1 -- 172.21.15.7:0/1901147382 learned_addr learned my addr 172.21.15.7:0/1901147382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.472+0000 7f78bffff640 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa73980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.472+0000 7f78bffff640 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa7af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.472+0000 7f78beffd640 1 -- 172.21.15.7:0/1901147382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 msgr2=0x56147aa7af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.472+0000 7f78beffd640 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa7af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.472+0000 7f78beffd640 1 -- 172.21.15.7:0/1901147382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 msgr2=0x56147aa73980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.472+0000 7f78beffd640 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa73980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.472+0000 7f78beffd640 1 -- 172.21.15.7:0/1901147382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56147a7ad7d0 con 0x56147aa6a080 2024-03-26T08:43:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.472+0000 7f78bffff640 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa7af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:43:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.473+0000 7f78beffd640 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa73ec0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f78b4011970 tx=0x7f78b4011e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ae81cdb6ddf03fa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.473+0000 7f789effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.473+0000 7f789effd640 1 -- 172.21.15.7:0/1901147382 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b400fcb0 con 0x56147aa6a080 2024-03-26T08:43:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.473+0000 7f789effd640 1 -- 172.21.15.7:0/1901147382 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78b4005020 con 0x56147aa6a080 2024-03-26T08:43:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.473+0000 7f78c8a2b740 1 -- 172.21.15.7:0/1901147382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78b8003970 con 0x56147aa6a080 2024-03-26T08:43:20.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.473+0000 7f789effd640 1 -- 172.21.15.7:0/1901147382 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78b400fe50 con 0x56147aa6a080 2024-03-26T08:43:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.474+0000 7f78c8a2b740 10 client.25523 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.474+0000 7f78c8a2b740 1 -- 172.21.15.7:0/1901147382 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56147aa7e310 con 0x56147aa6a080 2024-03-26T08:43:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.474+0000 7f789effd640 1 -- 172.21.15.7:0/1901147382 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78b8003970 con 0x56147aa6a080 2024-03-26T08:43:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.475+0000 7f789effd640 1 -- 172.21.15.7:0/1901147382 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f78b4019430 con 0x56147aa6a080 2024-03-26T08:43:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.475+0000 7f78c8a2b740 10 client.25523 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:20.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.475+0000 7f78c8a2b740 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.475+0000 7f78c8a2b740 20 client.25523 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.475+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56147aa7f930 0x56147aa81d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.475+0000 7f78c8a2b740 4 client.25523 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.475+0000 7f78c8a2b740 1 -- 172.21.15.7:0/1901147382 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56147a9dc900 con 0x56147aa7f930 2024-03-26T08:43:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.475+0000 7f78bffff640 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56147aa7f930 0x56147aa81d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.476+0000 7f78bffff640 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56147aa7f930 0x56147aa81d90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff49b61883989c4 server_cookie=13e7a741cb8d7526 in_seq=0 out_seq=0 2024-03-26T08:43:20.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.476+0000 7f789effd640 10 client.25523 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f789effd640 1 -- 172.21.15.7:0/1901147382 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56147a9dc900 con 0x56147aa7f930 2024-03-26T08:43:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f789effd640 10 client.25523 handle_command_reply: tid=0 2024-03-26T08:43:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 client.25523 shutdown 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 -- 172.21.15.7:0/1901147382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 msgr2=0x56147aa73ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa73ec0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f78b4011970 tx=0x7f78b4011e30 comp rx=0 tx=0).stop 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 -- 172.21.15.7:0/1901147382 shutdown_connections 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56147aa7f930 0x56147aa81d90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa7af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa73ec0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/1901147382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa73980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 -- 172.21.15.7:0/1901147382 >> 172.21.15.7:0/1901147382 conn(0x56147aa64c00 msgr2=0x56147aa63e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f789effd640 0 client.25523 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 -- 172.21.15.7:0/1901147382 shutdown_connections 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.477+0000 7f78c8a2b740 1 -- 172.21.15.7:0/1901147382 wait complete. 2024-03-26T08:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.478+0000 7f78c8a2b740 20 client.25523 trim_cache size 0 max 16384 2024-03-26T08:43:20.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.502+0000 7f78c8a2b740 1 Processor -- start 2024-03-26T08:43:20.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.502+0000 7f78c8a2b740 1 -- start start 2024-03-26T08:43:20.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.502+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa6b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa7aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac010980 con 0x56147aa6a080 2024-03-26T08:43:20.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa6b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00e480 con 0x56147aa69690 2024-03-26T08:43:20.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00c500 con 0x56147aa6f1f0 2024-03-26T08:43:20.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa6b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35968/0 (socket says 172.21.15.7:35968) 2024-03-26T08:43:20.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78bffff640 1 -- 172.21.15.7:0/3398869577 learned_addr learned my addr 172.21.15.7:0/3398869577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78beffd640 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78bffff640 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa7aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78bffff640 1 -- 172.21.15.7:0/3398869577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 msgr2=0x56147aa7aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78bffff640 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa7aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78bffff640 1 -- 172.21.15.7:0/3398869577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 msgr2=0x56147aa6b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78bffff640 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78bffff640 1 -- 172.21.15.7:0/3398869577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56147aa6e6d0 con 0x56147aa69690 2024-03-26T08:43:20.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.503+0000 7f78beffd640 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:43:20.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.504+0000 7f78bffff640 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa6b400 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f78b8005cc0 tx=0x7f78b8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=129d5dfa6894aea8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.504+0000 7f78a57fa640 1 -- 172.21.15.7:0/3398869577 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8043040 con 0x56147aa69690 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.504+0000 7f78a57fa640 1 -- 172.21.15.7:0/3398869577 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78b8052040 con 0x56147aa69690 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.504+0000 7f78a57fa640 1 -- 172.21.15.7:0/3398869577 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8034b60 con 0x56147aa69690 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.505+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3398869577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 msgr2=0x56147aa6b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.505+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa6b400 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f78b8005cc0 tx=0x7f78b8004a40 comp rx=0 tx=0).stop 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.505+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3398869577 shutdown_connections 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.505+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa7aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.505+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa6a080 0x56147aa6b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.505+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3398869577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa69690 0x56147aa6b400 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.505+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3398869577 >> 172.21.15.7:0/3398869577 conn(0x56147aa64c00 msgr2=0x56147aa73980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.505+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3398869577 shutdown_connections 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.506+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3398869577 wait complete. 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.506+0000 7f78c8a2b740 1 -- start start 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.506+0000 7f78c8a2b740 1 Processor -- start 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa78ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78c8a2b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa79a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00cb00 con 0x56147aa69690 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00de20 con 0x56147aa6a080 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78c8a2b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac00d160 con 0x56147aa6f1f0 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78c8a2b740 10 client.0 _ll_register_callbacks cb 0x7f78b4011580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78bffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa78ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78bffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa78580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53724/0 (socket says 172.21.15.7:53724) 2024-03-26T08:43:20.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78bffff640 1 -- 172.21.15.7:0/3304427681 learned_addr learned my addr 172.21.15.7:0/3304427681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.507+0000 7f78bffff640 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa79a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.508+0000 7f78bffff640 1 -- 172.21.15.7:0/3304427681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 msgr2=0x56147aa79a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.508+0000 7f78bffff640 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa79a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.508+0000 7f78bffff640 1 -- 172.21.15.7:0/3304427681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 msgr2=0x56147aa78ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.508+0000 7f78bffff640 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa78ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.508+0000 7f78bffff640 1 -- 172.21.15.7:0/3304427681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56147aa7a070 con 0x56147aa69690 2024-03-26T08:43:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.508+0000 7f78bffff640 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa78580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f78b80334e0 tx=0x7f78b802c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d756c66ea6a39676 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.508+0000 7f789effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:43:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.508+0000 7f789effd640 1 -- 172.21.15.7:0/3304427681 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8043040 con 0x56147aa69690 2024-03-26T08:43:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.508+0000 7f789effd640 1 -- 172.21.15.7:0/3304427681 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78b80348e0 con 0x56147aa69690 2024-03-26T08:43:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3304427681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78b8052040 con 0x56147aa69690 2024-03-26T08:43:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f789effd640 1 -- 172.21.15.7:0/3304427681 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78b8034ae0 con 0x56147aa69690 2024-03-26T08:43:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f78c8a2b740 10 client.15687 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3304427681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56147aadb260 con 0x56147aa69690 2024-03-26T08:43:20.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f789effd640 1 -- 172.21.15.7:0/3304427681 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78b805b020 con 0x56147aa69690 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f789effd640 1 -- 172.21.15.7:0/3304427681 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f78b80043d0 con 0x56147aa69690 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f78c8a2b740 10 client.15687 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f78c8a2b740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f78c8a2b740 20 client.15687 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56147aadb5e0 0x56147aadda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f78c8a2b740 4 client.15687 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.509+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3304427681 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f78b8052040 con 0x56147aadb5e0 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.510+0000 7f78beffd640 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56147aadb5e0 0x56147aadda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78beffd640 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56147aadb5e0 0x56147aadda00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1717fdc373cabe5a server_cookie=1e783c35559128df in_seq=0 out_seq=0 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f789effd640 10 client.15687 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f789effd640 1 -- 172.21.15.7:0/3304427681 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f78b8052040 con 0x56147aadb5e0 2024-03-26T08:43:20.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f789effd640 10 client.15687 handle_command_reply: tid=0 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78c8a2b740 1 client.15687 shutdown 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3304427681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 msgr2=0x56147aa78580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa78580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f78b80334e0 tx=0x7f78b802c570 comp rx=0 tx=0).stop 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3304427681 shutdown_connections 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56147aadb5e0 0x56147aadda00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56147aa6f1f0 0x56147aa79a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147aa6a080 0x56147aa78ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78c8a2b740 1 --2- 172.21.15.7:0/3304427681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56147aa69690 0x56147aa78580 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f789effd640 0 client.15687 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.511+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3304427681 >> 172.21.15.7:0/3304427681 conn(0x56147aa64c00 msgr2=0x56147aa81780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.512+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3304427681 shutdown_connections 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.512+0000 7f78c8a2b740 1 -- 172.21.15.7:0/3304427681 wait complete. 2024-03-26T08:43:20.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.512+0000 7f78c8a2b740 20 client.15687 trim_cache size 0 max 16384 2024-03-26T08:43:20.507 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.513+0000 7f78c5d62640 1 -- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78a80a2990 msgr2=0x7f78a80a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.513+0000 7f78c5d62640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78a80a2990 0x7f78a80a4e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78b009c6f0 tx=0x7f78b0098b10 comp rx=0 tx=0).stop 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.513+0000 7f78c5d62640 1 -- 172.21.15.7:0/62016451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c01086a0 msgr2=0x7f78c0119890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.513+0000 7f78c5d62640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c01086a0 0x7f78c0119890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f78b000e9f0 tx=0x7f78b000edf0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.514+0000 7f78c5d62640 1 -- 172.21.15.7:0/62016451 shutdown_connections 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.514+0000 7f78c5d62640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78a80a2990 0x7f78a80a4e50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.514+0000 7f78c5d62640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78c010d9c0 0x7f78c0117ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.514+0000 7f78c5d62640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c010b010 0x7f78c0119dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.514+0000 7f78c5d62640 1 --2- 172.21.15.7:0/62016451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78c01086a0 0x7f78c0119890 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:20.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.514+0000 7f78c5d62640 1 -- 172.21.15.7:0/62016451 >> 172.21.15.7:0/62016451 conn(0x7f78c006e640 msgr2=0x7f78c010b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:20.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.514+0000 7f78c5d62640 1 -- 172.21.15.7:0/62016451 shutdown_connections 2024-03-26T08:43:20.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:20.514+0000 7f78c5d62640 1 -- 172.21.15.7:0/62016451 wait complete. 2024-03-26T08:43:21.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 50.0 s, 4099.2 tps, lat 24.400 ms stddev 25.523 2024-03-26T08:43:21.525 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:43:21.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:43:21.526 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:43:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.706+0000 7f7fc556b640 1 -- 172.21.15.7:0/4280229819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fb8097f20 msgr2=0x7f7fb8098360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.706+0000 7f7fc556b640 1 --2- 172.21.15.7:0/4280229819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fb8097f20 0x7f7fb8098360 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009900 tx=0x7f7fb402bc30 comp rx=0 tx=0).stop 2024-03-26T08:43:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.706+0000 7f7fc556b640 1 -- 172.21.15.7:0/4280229819 shutdown_connections 2024-03-26T08:43:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.706+0000 7f7fc556b640 1 --2- 172.21.15.7:0/4280229819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb8099ab0 0x7f7fb80a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.706+0000 7f7fc556b640 1 --2- 172.21.15.7:0/4280229819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb80990f0 0x7f7fb8099570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.706+0000 7f7fc556b640 1 --2- 172.21.15.7:0/4280229819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fb8097f20 0x7f7fb8098360 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.706+0000 7f7fc556b640 1 -- 172.21.15.7:0/4280229819 >> 172.21.15.7:0/4280229819 conn(0x7f7fb8093500 msgr2=0x7f7fb8095920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.707+0000 7f7fc556b640 1 -- 172.21.15.7:0/4280229819 shutdown_connections 2024-03-26T08:43:21.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.707+0000 7f7fc556b640 1 -- 172.21.15.7:0/4280229819 wait complete. 2024-03-26T08:43:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.708+0000 7f7fc556b640 1 Processor -- start 2024-03-26T08:43:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.708+0000 7f7fc556b640 1 -- start start 2024-03-26T08:43:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc556b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fb8097f20 0x7f7fb800c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc556b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb80990f0 0x7f7fb800cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc556b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8099ab0 0x7f7fb80093d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc556b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80a3c20 con 0x7f7fb8097f20 2024-03-26T08:43:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc556b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80a3aa0 con 0x7f7fb8099ab0 2024-03-26T08:43:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc556b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb80a3da0 con 0x7f7fb80990f0 2024-03-26T08:43:21.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc4d6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8099ab0 0x7f7fb80093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc4d6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8099ab0 0x7f7fb80093d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35980/0 (socket says 172.21.15.7:35980) 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc4d6a640 1 -- 172.21.15.7:0/879268229 learned_addr learned my addr 172.21.15.7:0/879268229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fbffff640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fb8097f20 0x7f7fb800c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb80990f0 0x7f7fb800cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc4d6a640 1 -- 172.21.15.7:0/879268229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb80990f0 msgr2=0x7f7fb800cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.709+0000 7f7fc4d6a640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb80990f0 0x7f7fb800cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fc4d6a640 1 -- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fb8097f20 msgr2=0x7f7fb800c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fc4d6a640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fb8097f20 0x7f7fb800c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fc4d6a640 1 -- 172.21.15.7:0/879268229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8009c90 con 0x7f7fb8099ab0 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fbffff640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fb8097f20 0x7f7fb800c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fc4d6a640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8099ab0 0x7f7fb80093d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00501b0 tx=0x7f7fc007aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fbd7fa640 1 -- 172.21.15.7:0/879268229 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0061ae0 con 0x7f7fb8099ab0 2024-03-26T08:43:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fbd7fa640 1 -- 172.21.15.7:0/879268229 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0078d60 con 0x7f7fb8099ab0 2024-03-26T08:43:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fbd7fa640 1 -- 172.21.15.7:0/879268229 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0062660 con 0x7f7fb8099ab0 2024-03-26T08:43:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fc556b640 1 -- 172.21.15.7:0/879268229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb813a6c0 con 0x7f7fb8099ab0 2024-03-26T08:43:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.710+0000 7f7fc556b640 1 -- 172.21.15.7:0/879268229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb813aa70 con 0x7f7fb8099ab0 2024-03-26T08:43:21.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.713+0000 7f7fc93bb740 1 Processor -- start 2024-03-26T08:43:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.714+0000 7f7fc93bb740 1 -- start start 2024-03-26T08:43:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.714+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c0b30 0x562e802c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.714+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802c6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.714+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 0x562e802c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.714+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e80200c90 con 0x562e802c1520 2024-03-26T08:43:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.714+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e80226070 con 0x562e802c0b30 2024-03-26T08:43:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.714+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e801cec80 con 0x562e802c6690 2024-03-26T08:43:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.715+0000 7f7fbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.715+0000 7f7fbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c0b30 0x562e802c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.715+0000 7f7fbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c0b30 0x562e802c0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35996/0 (socket says 172.21.15.7:35996) 2024-03-26T08:43:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.715+0000 7f7fbf7fe640 1 -- 172.21.15.7:0/449726471 learned_addr learned my addr 172.21.15.7:0/449726471 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.715+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/449726471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 0x562e802c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.715+0000 7f7fbf7fe640 1 -- 172.21.15.7:0/449726471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 msgr2=0x562e802c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.715+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/449726471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 0x562e802c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.715+0000 7f7fbf7fe640 1 -- 172.21.15.7:0/449726471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 msgr2=0x562e802c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.715+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/449726471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.716+0000 7f7fbf7fe640 1 -- 172.21.15.7:0/449726471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e801c6d10 con 0x562e802c0b30 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.716+0000 7f7fbffff640 1 --2- 172.21.15.7:0/449726471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.716+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/449726471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c0b30 0x562e802c0f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00149b0 tx=0x7f7fb000c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=952bfe1ca1c26e4b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.716+0000 7f7fa67fc640 1 -- 172.21.15.7:0/449726471 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00142d0 con 0x562e802c0b30 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.716+0000 7f7fbd7fa640 1 -- 172.21.15.7:0/879268229 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f7fc0087020 con 0x7f7fb8099ab0 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.716+0000 7f7fa67fc640 1 -- 172.21.15.7:0/449726471 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fb0005500 con 0x562e802c0b30 2024-03-26T08:43:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.716+0000 7f7fa67fc640 1 -- 172.21.15.7:0/449726471 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000f7e0 con 0x562e802c0b30 2024-03-26T08:43:21.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.717+0000 7f7fc93bb740 1 -- 172.21.15.7:0/449726471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c0b30 msgr2=0x562e802c0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.717+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/449726471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c0b30 0x562e802c0f50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00149b0 tx=0x7f7fb000c530 comp rx=0 tx=0).stop 2024-03-26T08:43:21.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.717+0000 7f7fbd7fa640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fa80a2910 0x7f7fa80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.717+0000 7f7fc93bb740 1 -- 172.21.15.7:0/449726471 shutdown_connections 2024-03-26T08:43:21.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.717+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/449726471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 0x562e802c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.717+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/449726471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.717+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/449726471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c0b30 0x562e802c0f50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.717+0000 7f7fc93bb740 1 -- 172.21.15.7:0/449726471 >> 172.21.15.7:0/449726471 conn(0x562e802891a0 msgr2=0x562e802bc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.718+0000 7f7fbd7fa640 1 -- 172.21.15.7:0/879268229 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fc013a330 con 0x7f7fb8099ab0 2024-03-26T08:43:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.718+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fa80a2910 0x7f7fa80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.718+0000 7f7fc93bb740 1 -- 172.21.15.7:0/449726471 shutdown_connections 2024-03-26T08:43:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.718+0000 7f7fc93bb740 1 -- 172.21.15.7:0/449726471 wait complete. 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.719+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fa80a2910 0x7f7fa80a4dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005bf0 tx=0x7f7fb0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.719+0000 7f7fc93bb740 1 -- start start 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.719+0000 7f7fc93bb740 1 Processor -- start 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.719+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.719+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 0x562e802e3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e801e20d0 con 0x562e802c1520 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e80222110 con 0x562e802c6690 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e802da5f0 con 0x562e802c0b30 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802dbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58716/0 (socket says 172.21.15.7:58716) 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fc93bb740 10 client.0 _ll_register_callbacks cb 0x562e8021de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbffff640 1 -- 172.21.15.7:0/291200923 learned_addr learned my addr 172.21.15.7:0/291200923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fc4d6a640 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 0x562e802e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbffff640 1 -- 172.21.15.7:0/291200923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 msgr2=0x562e802e3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbffff640 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 0x562e802e3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbffff640 1 -- 172.21.15.7:0/291200923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 msgr2=0x562e802dc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbffff640 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802dc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbffff640 1 -- 172.21.15.7:0/291200923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e800027e0 con 0x562e802c0b30 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fc4d6a640 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 0x562e802e3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7fbffff640 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802dbdd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4000c00 tx=0x7f7fb4004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ac769a708b5bedd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.720+0000 7f7f9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.721+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/291200923 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4005d50 con 0x562e802c0b30 2024-03-26T08:43:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.721+0000 7f7fc93bb740 1 -- 172.21.15.7:0/291200923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fb0005500 con 0x562e802c0b30 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.721+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/291200923 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fb403a040 con 0x562e802c0b30 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.721+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/291200923 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fb4005d50 con 0x562e802c0b30 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7fc93bb740 10 client.25549 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7fc93bb740 1 -- 172.21.15.7:0/291200923 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562e80062f70 con 0x562e802c0b30 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/291200923 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fb0005500 con 0x562e802c0b30 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/291200923 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f7fb40035f0 con 0x562e802c0b30 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7fc93bb740 10 client.25549 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7fc93bb740 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7fc93bb740 20 client.25549 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562e802e7fb0 0x562e802ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7fc93bb740 4 client.25549 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.722+0000 7f7fc93bb740 1 -- 172.21.15.7:0/291200923 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e802335b0 con 0x562e802e7fb0 2024-03-26T08:43:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.723+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562e802e7fb0 0x562e802ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.723+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562e802e7fb0 0x562e802ea430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d62f65982ab3d3c6 server_cookie=bdff37f4856b1a41 in_seq=0 out_seq=0 2024-03-26T08:43:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.723+0000 7f7f9f7fe640 10 client.25549 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.724+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/291200923 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562e802335b0 con 0x562e802e7fb0 2024-03-26T08:43:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.724+0000 7f7f9f7fe640 10 client.25549 handle_command_reply: tid=0 2024-03-26T08:43:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.724+0000 7f7fc93bb740 1 client.25549 shutdown 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 -- 172.21.15.7:0/291200923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 msgr2=0x562e802dbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802dbdd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4000c00 tx=0x7f7fb4004970 comp rx=0 tx=0).stop 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 -- 172.21.15.7:0/291200923 shutdown_connections 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562e802e7fb0 0x562e802ea430 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7f9f7fe640 0 client.25549 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 0x562e802e3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802dc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/291200923 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802dbdd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 -- 172.21.15.7:0/291200923 >> 172.21.15.7:0/291200923 conn(0x562e802891a0 msgr2=0x562e802bb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 -- 172.21.15.7:0/291200923 shutdown_connections 2024-03-26T08:43:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.725+0000 7f7fc93bb740 1 -- 172.21.15.7:0/291200923 wait complete. 2024-03-26T08:43:21.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.726+0000 7f7fc93bb740 20 client.25549 trim_cache size 0 max 16384 2024-03-26T08:43:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.731+0000 7f35a215b640 1 -- 172.21.15.7:0/1265005215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359409c370 msgr2=0x7f359409c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.731+0000 7f35a215b640 1 --2- 172.21.15.7:0/1265005215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359409c370 0x7f359409c790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3588009900 tx=0x7f358802bc30 comp rx=0 tx=0).stop 2024-03-26T08:43:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.731+0000 7f35a215b640 1 -- 172.21.15.7:0/1265005215 shutdown_connections 2024-03-26T08:43:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.731+0000 7f35a215b640 1 --2- 172.21.15.7:0/1265005215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359409df20 0x7f35940a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.731+0000 7f35a215b640 1 --2- 172.21.15.7:0/1265005215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359409d560 0x7f359409d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.731+0000 7f35a215b640 1 --2- 172.21.15.7:0/1265005215 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359409c370 0x7f359409c790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.731+0000 7f35a215b640 1 -- 172.21.15.7:0/1265005215 >> 172.21.15.7:0/1265005215 conn(0x7f359400b310 msgr2=0x7f359400b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.732+0000 7f35a215b640 1 -- 172.21.15.7:0/1265005215 shutdown_connections 2024-03-26T08:43:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.732+0000 7f35a215b640 1 -- 172.21.15.7:0/1265005215 wait complete. 2024-03-26T08:43:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.732+0000 7f35a215b640 1 Processor -- start 2024-03-26T08:43:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f35a215b640 1 -- start start 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f35a215b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359409c370 0x7f3594132700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f35a215b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359409d560 0x7f3594132c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f35a215b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359409df20 0x7f3594139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f359b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359409c370 0x7f3594132700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f359b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359409c370 0x7f3594132700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36028/0 (socket says 172.21.15.7:36028) 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f359b7fe640 1 -- 172.21.15.7:0/1603770295 learned_addr learned my addr 172.21.15.7:0/1603770295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f35a215b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35940a8050 con 0x7f359409df20 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35940a7ed0 con 0x7f359409c370 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35940a81d0 con 0x7f359409d560 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f359affd640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359409d560 0x7f3594132c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.733+0000 7f359bfff640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359409df20 0x7f3594139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f359b7fe640 1 -- 172.21.15.7:0/1603770295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359409d560 msgr2=0x7f3594132c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f359b7fe640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359409d560 0x7f3594132c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f359b7fe640 1 -- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359409df20 msgr2=0x7f3594139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f359b7fe640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359409df20 0x7f3594139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f359b7fe640 1 -- 172.21.15.7:0/1603770295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359413a2d0 con 0x7f359409c370 2024-03-26T08:43:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f359affd640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359409d560 0x7f3594132c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:43:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f359b7fe640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359409c370 0x7f3594132700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f358802c240 tx=0x7f3588004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f3598ff9640 1 -- 172.21.15.7:0/1603770295 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588043020 con 0x7f359409c370 2024-03-26T08:43:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f3598ff9640 1 -- 172.21.15.7:0/1603770295 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35880054e0 con 0x7f359409c370 2024-03-26T08:43:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.734+0000 7f3598ff9640 1 -- 172.21.15.7:0/1603770295 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3588003c60 con 0x7f359409c370 2024-03-26T08:43:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.735+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359413a560 con 0x7f359409c370 2024-03-26T08:43:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.735+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359413aa60 con 0x7f359409c370 2024-03-26T08:43:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.738+0000 7f35a4e29740 1 Processor -- start 2024-03-26T08:43:21.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.738+0000 7f35a4e29740 1 -- start start 2024-03-26T08:43:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae33b20 0x55ccaae33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae39010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae39680 0x55ccaae3bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f35a4e29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccaad98b10 con 0x55ccaae34510 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f35a4e29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccaad41900 con 0x55ccaae33b20 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f35a4e29740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccaad94bb0 con 0x55ccaae39680 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f359bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae33b20 0x55ccaae33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f359bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae33b20 0x55ccaae33f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36040/0 (socket says 172.21.15.7:36040) 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f359bfff640 1 -- 172.21.15.7:0/1550294506 learned_addr learned my addr 172.21.15.7:0/1550294506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f359b7fe640 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae39010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.739+0000 7f359affd640 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae39680 0x55ccaae3bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.740+0000 7f359bfff640 1 -- 172.21.15.7:0/1550294506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae39680 msgr2=0x55ccaae3bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.740+0000 7f359bfff640 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae39680 0x55ccaae3bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.740+0000 7f3598ff9640 1 -- 172.21.15.7:0/1603770295 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7f3588046020 con 0x7f359409c370 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.740+0000 7f359bfff640 1 -- 172.21.15.7:0/1550294506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 msgr2=0x55ccaae39010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.740+0000 7f359bfff640 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae39010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.740+0000 7f359bfff640 1 -- 172.21.15.7:0/1550294506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccaaaad210 con 0x55ccaae33b20 2024-03-26T08:43:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.740+0000 7f359b7fe640 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae39010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.741+0000 7f3598ff9640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f358c0a29d0 0x7f358c0a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.741+0000 7f359b7fe640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f358c0a29d0 0x7f358c0a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.741+0000 7f3598ff9640 1 -- 172.21.15.7:0/1603770295 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35880f0520 con 0x7f359409c370 2024-03-26T08:43:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.742+0000 7f359bfff640 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae33b20 0x55ccaae33f40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f359c0674f0 tx=0x7f359c073ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=590c5d531155d4ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.742+0000 7f359b7fe640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f358c0a29d0 0x7f358c0a4e90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55ccaae34910 tx=0x7f35880f2040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.742+0000 7f3585ffb640 1 -- 172.21.15.7:0/1550294506 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c07f990 con 0x55ccaae33b20 2024-03-26T08:43:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.742+0000 7f3585ffb640 1 -- 172.21.15.7:0/1550294506 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f359c07fb30 con 0x55ccaae33b20 2024-03-26T08:43:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.742+0000 7f3585ffb640 1 -- 172.21.15.7:0/1550294506 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c07f990 con 0x55ccaae33b20 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.743+0000 7f35a4e29740 1 -- 172.21.15.7:0/1550294506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae33b20 msgr2=0x55ccaae33f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.743+0000 7f35a4e29740 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae33b20 0x55ccaae33f40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f359c0674f0 tx=0x7f359c073ca0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.743+0000 7f35a4e29740 1 -- 172.21.15.7:0/1550294506 shutdown_connections 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.743+0000 7f35a4e29740 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae39680 0x55ccaae3bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.743+0000 7f35a4e29740 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae39010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.743+0000 7f35a4e29740 1 --2- 172.21.15.7:0/1550294506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae33b20 0x55ccaae33f40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.743+0000 7f35a4e29740 1 -- 172.21.15.7:0/1550294506 >> 172.21.15.7:0/1550294506 conn(0x55ccaae2f090 msgr2=0x55ccaae2f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.744+0000 7f35a4e29740 1 -- 172.21.15.7:0/1550294506 shutdown_connections 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.744+0000 7f35a4e29740 1 -- 172.21.15.7:0/1550294506 wait complete. 2024-03-26T08:43:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.744+0000 7f35a4e29740 1 -- start start 2024-03-26T08:43:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.744+0000 7f35a4e29740 1 Processor -- start 2024-03-26T08:43:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 0x55ccaae3ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae34510 0x55ccaae3e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae39680 0x55ccaae45340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f35a4e29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccaae3c630 con 0x55ccaae39680 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f35a4e29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccaad54d50 con 0x55ccaae34510 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f35a4e29740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccaae3c7b0 con 0x55ccaae33b20 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f35a4e29740 10 client.0 _ll_register_callbacks cb 0x55ccaad54720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae34510 0x55ccaae3e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae34510 0x55ccaae3e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36042/0 (socket says 172.21.15.7:36042) 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359affd640 1 -- 172.21.15.7:0/2300704738 learned_addr learned my addr 172.21.15.7:0/2300704738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359bfff640 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 0x55ccaae3ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359bfff640 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae39680 0x55ccaae45340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359affd640 1 -- 172.21.15.7:0/2300704738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 msgr2=0x55ccaae3ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359affd640 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 0x55ccaae3ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359affd640 1 -- 172.21.15.7:0/2300704738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae39680 msgr2=0x55ccaae45340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359affd640 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae39680 0x55ccaae45340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.745+0000 7f359affd640 1 -- 172.21.15.7:0/2300704738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccaab79620 con 0x55ccaae34510 2024-03-26T08:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.746+0000 7f359bfff640 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 0x55ccaae3ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.746+0000 7f359affd640 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae34510 0x55ccaae3e300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3590013790 tx=0x7f3590013c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cefda88a2294bab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.746+0000 7f357affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.746+0000 7f357affd640 1 -- 172.21.15.7:0/2300704738 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590020060 con 0x55ccaae34510 2024-03-26T08:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.746+0000 7f357affd640 1 -- 172.21.15.7:0/2300704738 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3590011dd0 con 0x55ccaae34510 2024-03-26T08:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.746+0000 7f35a4e29740 1 -- 172.21.15.7:0/2300704738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f359c07fb30 con 0x55ccaae34510 2024-03-26T08:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.746+0000 7f357affd640 1 -- 172.21.15.7:0/2300704738 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f359000f450 con 0x55ccaae34510 2024-03-26T08:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.746+0000 7f35a4e29740 10 client.25550 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.746+0000 7f35a4e29740 1 -- 172.21.15.7:0/2300704738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ccaae59760 con 0x55ccaae34510 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.747+0000 7f357affd640 1 -- 172.21.15.7:0/2300704738 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f359c07fb30 con 0x55ccaae34510 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.747+0000 7f357affd640 1 -- 172.21.15.7:0/2300704738 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f3590025070 con 0x55ccaae34510 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.747+0000 7f35a4e29740 10 client.25550 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.747+0000 7f35a4e29740 10 client.25550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.747+0000 7f35a4e29740 20 client.25550 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.747+0000 7f35a4e29740 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ccaae5aed0 0x55ccaae5d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.747+0000 7f35a4e29740 4 client.25550 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.747+0000 7f35a4e29740 1 -- 172.21.15.7:0/2300704738 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccaada6050 con 0x55ccaae5aed0 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.747+0000 7f359bfff640 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ccaae5aed0 0x55ccaae5d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.748+0000 7f357affd640 10 client.25550 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.748+0000 7f359bfff640 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ccaae5aed0 0x55ccaae5d2f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=253c46f91bbe9878 server_cookie=8924f50e8e1799e9 in_seq=0 out_seq=0 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.748+0000 7f357affd640 1 -- 172.21.15.7:0/2300704738 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ccaada6050 con 0x55ccaae5aed0 2024-03-26T08:43:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.748+0000 7f357affd640 10 client.25550 handle_command_reply: tid=0 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.748+0000 7f35a4e29740 1 client.25550 shutdown 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f35a4e29740 1 -- 172.21.15.7:0/2300704738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae34510 msgr2=0x55ccaae3e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f35a4e29740 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae34510 0x55ccaae3e300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3590013790 tx=0x7f3590013c50 comp rx=0 tx=0).stop 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f35a4e29740 1 -- 172.21.15.7:0/2300704738 shutdown_connections 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f35a4e29740 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ccaae5aed0 0x55ccaae5d2f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f35a4e29740 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae39680 0x55ccaae45340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f35a4e29740 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae34510 0x55ccaae3e300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f35a4e29740 1 --2- 172.21.15.7:0/2300704738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 0x55ccaae3ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f357affd640 0 client.25550 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f35a4e29740 1 -- 172.21.15.7:0/2300704738 >> 172.21.15.7:0/2300704738 conn(0x55ccaae2f090 msgr2=0x55ccaae2e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.749+0000 7f35a4e29740 1 -- 172.21.15.7:0/2300704738 shutdown_connections 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.750+0000 7f35a4e29740 1 -- 172.21.15.7:0/2300704738 wait complete. 2024-03-26T08:43:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.750+0000 7f35a4e29740 20 client.25550 trim_cache size 0 max 16384 2024-03-26T08:43:21.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.759+0000 7f7fc93bb740 1 Processor -- start 2024-03-26T08:43:21.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.759+0000 7f7fc93bb740 1 -- start start 2024-03-26T08:43:21.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c0b30 0x562e802eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c1520 0x562e802e3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 0x562e802ef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb813d6e0 con 0x562e802c0b30 2024-03-26T08:43:21.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb813b1e0 con 0x562e802c1520 2024-03-26T08:43:21.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb8134c00 con 0x562e802c6690 2024-03-26T08:43:21.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fc4d6a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 0x562e802ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c1520 0x562e802e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fc4d6a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 0x562e802ef010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58754/0 (socket says 172.21.15.7:58754) 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fc4d6a640 1 -- 172.21.15.7:0/1483595190 learned_addr learned my addr 172.21.15.7:0/1483595190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c0b30 0x562e802eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fbf7fe640 1 -- 172.21.15.7:0/1483595190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 msgr2=0x562e802ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/1483595190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 0x562e802ef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fbf7fe640 1 -- 172.21.15.7:0/1483595190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c0b30 msgr2=0x562e802eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/1483595190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c0b30 0x562e802eead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.760+0000 7f7fbf7fe640 1 -- 172.21.15.7:0/1483595190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e7fefa170 con 0x562e802c1520 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.761+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/1483595190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c1520 0x562e802e3310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00149b0 tx=0x7f7fb0023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc265891826adcbe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.761+0000 7f7fa5ffb640 1 -- 172.21.15.7:0/1483595190 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00362e0 con 0x562e802c1520 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.761+0000 7f7fa5ffb640 1 -- 172.21.15.7:0/1483595190 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fb0025070 con 0x562e802c1520 2024-03-26T08:43:21.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.761+0000 7f7fa5ffb640 1 -- 172.21.15.7:0/1483595190 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0036990 con 0x562e802c1520 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.762+0000 7f7fc93bb740 1 -- 172.21.15.7:0/1483595190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c1520 msgr2=0x562e802e3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.762+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/1483595190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c1520 0x562e802e3310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00149b0 tx=0x7f7fb0023000 comp rx=0 tx=0).stop 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.762+0000 7f7fc93bb740 1 -- 172.21.15.7:0/1483595190 shutdown_connections 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.762+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/1483595190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c6690 0x562e802ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.762+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/1483595190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c1520 0x562e802e3310 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.762+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/1483595190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c0b30 0x562e802eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.762+0000 7f7fc93bb740 1 -- 172.21.15.7:0/1483595190 >> 172.21.15.7:0/1483595190 conn(0x562e802891a0 msgr2=0x562e802bb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.763+0000 7f7fc93bb740 1 -- 172.21.15.7:0/1483595190 shutdown_connections 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.763+0000 7f7fc93bb740 1 -- 172.21.15.7:0/1483595190 wait complete. 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fc93bb740 1 -- start start 2024-03-26T08:43:21.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fc93bb740 1 Processor -- start 2024-03-26T08:43:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fc93bb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 0x562e802e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb8135260 con 0x562e802c1520 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb813aca0 con 0x562e802c6690 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fc93bb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb81358c0 con 0x562e802c0b30 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fc93bb740 10 client.0 _ll_register_callbacks cb 0x7f7fb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fbffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802e0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58768/0 (socket says 172.21.15.7:58768) 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.764+0000 7f7fbffff640 1 -- 172.21.15.7:0/4227272240 learned_addr learned my addr 172.21.15.7:0/4227272240 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.765+0000 7f7fc4d6a640 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 0x562e802e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.765+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.765+0000 7f7fbffff640 1 -- 172.21.15.7:0/4227272240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 msgr2=0x562e802e21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.765+0000 7f7fbffff640 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 0x562e802e21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.765+0000 7f7fbffff640 1 -- 172.21.15.7:0/4227272240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 msgr2=0x562e802e11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.765+0000 7f7fbffff640 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802e11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.765+0000 7f7fbffff640 1 -- 172.21.15.7:0/4227272240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e802e27e0 con 0x562e802c0b30 2024-03-26T08:43:21.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.765+0000 7f7fbffff640 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802e0ca0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4049de0 tx=0x7f7fb4004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0fc26349073a197 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.765+0000 7f7f9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.766+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/4227272240 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4057070 con 0x562e802c0b30 2024-03-26T08:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.766+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/4227272240 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fb403ed70 con 0x562e802c0b30 2024-03-26T08:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.766+0000 7f7fc93bb740 1 -- 172.21.15.7:0/4227272240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fb0025070 con 0x562e802c0b30 2024-03-26T08:43:21.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.766+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/4227272240 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fb4061050 con 0x562e802c0b30 2024-03-26T08:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7fc93bb740 10 client.25564 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7fc93bb740 1 -- 172.21.15.7:0/4227272240 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562e802cf550 con 0x562e802c0b30 2024-03-26T08:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/4227272240 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fb402cd30 con 0x562e802c0b30 2024-03-26T08:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/4227272240 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f7fb40035f0 con 0x562e802c0b30 2024-03-26T08:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7fc93bb740 10 client.25564 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7fc93bb740 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7fc93bb740 20 client.25564 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562e802cfed0 0x562e802d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7fc93bb740 4 client.25564 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.767+0000 7f7fc93bb740 1 -- 172.21.15.7:0/4227272240 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f7fb0025070 con 0x562e802cfed0 2024-03-26T08:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.769+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562e802cfed0 0x562e802d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.769+0000 7f7fbf7fe640 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562e802cfed0 0x562e802d22f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c17fa8ea4265b47 server_cookie=2c7a581776722e37 in_seq=0 out_seq=0 2024-03-26T08:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.769+0000 7f7f9f7fe640 10 client.25564 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.770+0000 7f7f9f7fe640 1 -- 172.21.15.7:0/4227272240 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f7fb0025070 con 0x562e802cfed0 2024-03-26T08:43:21.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.770+0000 7f7f9f7fe640 10 client.25564 handle_command_reply: tid=0 2024-03-26T08:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.770+0000 7f7fc93bb740 1 client.25564 shutdown 2024-03-26T08:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.770+0000 7f7fc93bb740 1 -- 172.21.15.7:0/4227272240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 msgr2=0x562e802e0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.770+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802e0ca0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4049de0 tx=0x7f7fb4004610 comp rx=0 tx=0).stop 2024-03-26T08:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7fc93bb740 1 -- 172.21.15.7:0/4227272240 shutdown_connections 2024-03-26T08:43:21.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562e802cfed0 0x562e802d22f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562e802c6690 0x562e802e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562e802c1520 0x562e802e11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7fc93bb740 1 --2- 172.21.15.7:0/4227272240 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562e802c0b30 0x562e802e0ca0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7fc93bb740 1 -- 172.21.15.7:0/4227272240 >> 172.21.15.7:0/4227272240 conn(0x562e802891a0 msgr2=0x562e802bf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7f9f7fe640 0 client.25564 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7fc93bb740 1 -- 172.21.15.7:0/4227272240 shutdown_connections 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7fc93bb740 1 -- 172.21.15.7:0/4227272240 wait complete. 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.771+0000 7f7fc93bb740 20 client.25564 trim_cache size 0 max 16384 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:43:21.763 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:43:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "7be6ec80-657a-404b-9ff5-8c6799465888", 2024-03-26T08:43:21.764 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:43:21.764 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:43:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 -- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fa80a2910 msgr2=0x7f7fa80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fa80a2910 0x7f7fa80a4dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0005bf0 tx=0x7f7fb0005850 comp rx=0 tx=0).stop 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 -- 172.21.15.7:0/879268229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8099ab0 msgr2=0x7f7fb80093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8099ab0 0x7f7fb80093d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00501b0 tx=0x7f7fc007aee0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 -- 172.21.15.7:0/879268229 shutdown_connections 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fa80a2910 0x7f7fa80a4dd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fb8099ab0 0x7f7fb80093d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fb80990f0 0x7f7fb800cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 --2- 172.21.15.7:0/879268229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fb8097f20 0x7f7fb800c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 -- 172.21.15.7:0/879268229 >> 172.21.15.7:0/879268229 conn(0x7f7fb8093500 msgr2=0x7f7fb809c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 -- 172.21.15.7:0/879268229 shutdown_connections 2024-03-26T08:43:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:21.773+0000 7f7fc556b640 1 -- 172.21.15.7:0/879268229 wait complete. 2024-03-26T08:43:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.781+0000 7f35a4e29740 1 Processor -- start 2024-03-26T08:43:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.781+0000 7f35a4e29740 1 -- start start 2024-03-26T08:43:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.782+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae33b20 0x55ccaae38d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.782+0000 7f359bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae33b20 0x55ccaae38d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.782+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae34510 0x55ccaae45300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.782+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae39680 0x55ccaae322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.782+0000 7f359bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae33b20 0x55ccaae38d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53822/0 (socket says 172.21.15.7:53822) 2024-03-26T08:43:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.782+0000 7f359bfff640 1 -- 172.21.15.7:0/1117702885 learned_addr learned my addr 172.21.15.7:0/1117702885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.782+0000 7f35a4e29740 1 -- 172.21.15.7:0/1117702885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359413d4c0 con 0x55ccaae33b20 2024-03-26T08:43:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.782+0000 7f35a4e29740 1 -- 172.21.15.7:0/1117702885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359413afc0 con 0x55ccaae39680 2024-03-26T08:43:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.782+0000 7f35a4e29740 1 -- 172.21.15.7:0/1117702885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594013e50 con 0x55ccaae34510 2024-03-26T08:43:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.783+0000 7f359affd640 1 --2- 172.21.15.7:0/1117702885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae34510 0x55ccaae45300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.783+0000 7f359bfff640 1 --2- 172.21.15.7:0/1117702885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae39680 0x55ccaae322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.784+0000 7f359affd640 1 -- 172.21.15.7:0/1117702885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae39680 msgr2=0x55ccaae322f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.784+0000 7f359affd640 1 --2- 172.21.15.7:0/1117702885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae39680 0x55ccaae322f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.784+0000 7f359affd640 1 -- 172.21.15.7:0/1117702885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae33b20 msgr2=0x55ccaae38d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.784+0000 7f359affd640 1 --2- 172.21.15.7:0/1117702885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae33b20 0x55ccaae38d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.784+0000 7f359affd640 1 -- 172.21.15.7:0/1117702885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccaae34da0 con 0x55ccaae34510 2024-03-26T08:43:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.784+0000 7f359affd640 1 --2- 172.21.15.7:0/1117702885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae34510 0x55ccaae45300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35900268c0 tx=0x7f35900118e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6208411e92b5dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.784+0000 7f35857fa640 1 -- 172.21.15.7:0/1117702885 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590000e60 con 0x55ccaae34510 2024-03-26T08:43:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.784+0000 7f35857fa640 1 -- 172.21.15.7:0/1117702885 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35900055f0 con 0x55ccaae34510 2024-03-26T08:43:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.784+0000 7f35857fa640 1 -- 172.21.15.7:0/1117702885 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359002b030 con 0x55ccaae34510 2024-03-26T08:43:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.785+0000 7f35a4e29740 1 -- 172.21.15.7:0/1117702885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae34510 msgr2=0x55ccaae45300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.785+0000 7f35a4e29740 1 --2- 172.21.15.7:0/1117702885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae34510 0x55ccaae45300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35900268c0 tx=0x7f35900118e0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.786+0000 7f35a4e29740 1 -- 172.21.15.7:0/1117702885 shutdown_connections 2024-03-26T08:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.786+0000 7f35a4e29740 1 --2- 172.21.15.7:0/1117702885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae39680 0x55ccaae322f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.786+0000 7f35a4e29740 1 --2- 172.21.15.7:0/1117702885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae34510 0x55ccaae45300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.786+0000 7f35a4e29740 1 --2- 172.21.15.7:0/1117702885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae33b20 0x55ccaae38d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.786+0000 7f35a4e29740 1 -- 172.21.15.7:0/1117702885 >> 172.21.15.7:0/1117702885 conn(0x55ccaae2f090 msgr2=0x55ccaae2f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.787+0000 7f35a4e29740 1 -- 172.21.15.7:0/1117702885 shutdown_connections 2024-03-26T08:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.787+0000 7f35a4e29740 1 -- 172.21.15.7:0/1117702885 wait complete. 2024-03-26T08:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.787+0000 7f35a4e29740 1 -- start start 2024-03-26T08:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.787+0000 7f35a4e29740 1 Processor -- start 2024-03-26T08:43:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.788+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 0x55ccaae42830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.788+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae42d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.788+0000 7f35a4e29740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae39680 0x55ccaae43d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.788+0000 7f35a4e29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594014450 con 0x55ccaae34510 2024-03-26T08:43:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.788+0000 7f35a4e29740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594015770 con 0x55ccaae39680 2024-03-26T08:43:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.788+0000 7f35a4e29740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594014ab0 con 0x55ccaae33b20 2024-03-26T08:43:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.788+0000 7f35a4e29740 10 client.0 _ll_register_callbacks cb 0x7f3590009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.789+0000 7f359affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae42d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.789+0000 7f359affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae42d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53830/0 (socket says 172.21.15.7:53830) 2024-03-26T08:43:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.789+0000 7f359affd640 1 -- 172.21.15.7:0/10976309 learned_addr learned my addr 172.21.15.7:0/10976309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.789+0000 7f359affd640 1 -- 172.21.15.7:0/10976309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 msgr2=0x55ccaae42830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:43:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.789+0000 7f359affd640 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 0x55ccaae42830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.789+0000 7f359affd640 1 -- 172.21.15.7:0/10976309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae39680 msgr2=0x55ccaae43d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:43:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.789+0000 7f359affd640 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae39680 0x55ccaae43d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.789+0000 7f359affd640 1 -- 172.21.15.7:0/10976309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccaae44320 con 0x55ccaae34510 2024-03-26T08:43:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.790+0000 7f359affd640 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae42d70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f35900057a0 tx=0x7f3590011820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc822daa5b02e576 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.790+0000 7f357affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:43:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.790+0000 7f357affd640 1 -- 172.21.15.7:0/10976309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359002b040 con 0x55ccaae34510 2024-03-26T08:43:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.790+0000 7f35a4e29740 1 -- 172.21.15.7:0/10976309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35900055f0 con 0x55ccaae34510 2024-03-26T08:43:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.790+0000 7f357affd640 1 -- 172.21.15.7:0/10976309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3590011dd0 con 0x55ccaae34510 2024-03-26T08:43:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.791+0000 7f357affd640 1 -- 172.21.15.7:0/10976309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f359003f050 con 0x55ccaae34510 2024-03-26T08:43:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.791+0000 7f35a4e29740 10 client.15723 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.791+0000 7f35a4e29740 1 -- 172.21.15.7:0/10976309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ccaae45630 con 0x55ccaae34510 2024-03-26T08:43:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.792+0000 7f357affd640 1 -- 172.21.15.7:0/10976309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35900055f0 con 0x55ccaae34510 2024-03-26T08:43:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.792+0000 7f357affd640 1 -- 172.21.15.7:0/10976309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f3590025070 con 0x55ccaae34510 2024-03-26T08:43:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.792+0000 7f35a4e29740 10 client.15723 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.792+0000 7f35a4e29740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.792+0000 7f35a4e29740 20 client.15723 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.792+0000 7f35a4e29740 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ccaaeb6f20 0x55ccaaeb9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.792+0000 7f35a4e29740 4 client.15723 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:43:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.792+0000 7f35a4e29740 1 -- 172.21.15.7:0/10976309 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ccaada03a0 con 0x55ccaaeb6f20 2024-03-26T08:43:21.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:43:21.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:43:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.793+0000 7f359bfff640 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ccaaeb6f20 0x55ccaaeb9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.794+0000 7f357affd640 10 client.15723 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.794+0000 7f359bfff640 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ccaaeb6f20 0x55ccaaeb9340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8467b74858c7262b server_cookie=320437cf9dc80715 in_seq=0 out_seq=0 2024-03-26T08:43:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.819+0000 7f357affd640 1 -- 172.21.15.7:0/10976309 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ccaada03a0 con 0x55ccaaeb6f20 2024-03-26T08:43:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.819+0000 7f357affd640 10 client.15723 handle_command_reply: tid=0 2024-03-26T08:43:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.819+0000 7f35a4e29740 1 client.15723 shutdown 2024-03-26T08:43:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.820+0000 7f35a4e29740 1 -- 172.21.15.7:0/10976309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 msgr2=0x55ccaae42d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.820+0000 7f35a4e29740 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae42d70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f35900057a0 tx=0x7f3590011820 comp rx=0 tx=0).stop 2024-03-26T08:43:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.820+0000 7f35a4e29740 1 -- 172.21.15.7:0/10976309 shutdown_connections 2024-03-26T08:43:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.820+0000 7f35a4e29740 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ccaaeb6f20 0x55ccaaeb9340 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.820+0000 7f35a4e29740 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccaae39680 0x55ccaae43d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.820+0000 7f35a4e29740 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccaae34510 0x55ccaae42d70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.820+0000 7f35a4e29740 1 --2- 172.21.15.7:0/10976309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ccaae33b20 0x55ccaae42830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.820+0000 7f35a4e29740 1 -- 172.21.15.7:0/10976309 >> 172.21.15.7:0/10976309 conn(0x55ccaae2f090 msgr2=0x55ccaae38d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.820+0000 7f357affd640 0 client.15723 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:43:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:43:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:43:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:43:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.821+0000 7f35a4e29740 1 -- 172.21.15.7:0/10976309 shutdown_connections 2024-03-26T08:43:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.821+0000 7f35a4e29740 1 -- 172.21.15.7:0/10976309 wait complete. 2024-03-26T08:43:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.821+0000 7f35a4e29740 20 client.15723 trim_cache size 0 max 16384 2024-03-26T08:43:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f358c0a29d0 msgr2=0x7f358c0a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f358c0a29d0 0x7f358c0a4e90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55ccaae34910 tx=0x7f35880f2040 comp rx=0 tx=0).stop 2024-03-26T08:43:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359409c370 msgr2=0x7f3594132700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359409c370 0x7f3594132700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f358802c240 tx=0x7f3588004e30 comp rx=0 tx=0).stop 2024-03-26T08:43:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 shutdown_connections 2024-03-26T08:43:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f358c0a29d0 0x7f358c0a4e90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f359409df20 0x7f3594139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f359409d560 0x7f3594132c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 --2- 172.21.15.7:0/1603770295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f359409c370 0x7f3594132700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 >> 172.21.15.7:0/1603770295 conn(0x7f359400b310 msgr2=0x7f35940a0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 shutdown_connections 2024-03-26T08:43:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:21.822+0000 7f35a215b640 1 -- 172.21.15.7:0/1603770295 wait complete. 2024-03-26T08:43:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.001+0000 7fa457875640 1 -- 172.21.15.7:0/3523099193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4501011c0 msgr2=0x7fa450108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.001+0000 7fa457875640 1 --2- 172.21.15.7:0/3523099193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4501011c0 0x7fa450108a60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa448009ca0 tx=0x7fa44802c140 comp rx=0 tx=0).stop 2024-03-26T08:43:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.001+0000 7fa457875640 1 -- 172.21.15.7:0/3523099193 shutdown_connections 2024-03-26T08:43:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.001+0000 7fa457875640 1 --2- 172.21.15.7:0/3523099193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4501011c0 0x7fa450108a60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.001+0000 7fa457875640 1 --2- 172.21.15.7:0/3523099193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa450100800 0x7fa450100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.001+0000 7fa457875640 1 --2- 172.21.15.7:0/3523099193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450101c40 0x7fa450102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.001+0000 7fa457875640 1 -- 172.21.15.7:0/3523099193 >> 172.21.15.7:0/3523099193 conn(0x7fa4500fb2b0 msgr2=0x7fa4500fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.002+0000 7fa457875640 1 -- 172.21.15.7:0/3523099193 shutdown_connections 2024-03-26T08:43:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.002+0000 7fa457875640 1 -- 172.21.15.7:0/3523099193 wait complete. 2024-03-26T08:43:21.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.003+0000 7fa457875640 1 Processor -- start 2024-03-26T08:43:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.003+0000 7fa457875640 1 -- start start 2024-03-26T08:43:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.003+0000 7fa457875640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa450100800 0x7fa4501002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.003+0000 7fa457875640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4501011c0 0x7fa45019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.003+0000 7fa457875640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450101c40 0x7fa45019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.003+0000 7fa457875640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45010c2b0 con 0x7fa4501011c0 2024-03-26T08:43:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.003+0000 7fa457875640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45010c130 con 0x7fa450101c40 2024-03-26T08:43:21.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.003+0000 7fa457875640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45010c430 con 0x7fa450100800 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455e12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450101c40 0x7fa45019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455e12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450101c40 0x7fa45019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36092/0 (socket says 172.21.15.7:36092) 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455e12640 1 -- 172.21.15.7:0/1684835547 learned_addr learned my addr 172.21.15.7:0/1684835547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa454e10640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4501011c0 0x7fa45019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455611640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa450100800 0x7fa4501002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455e12640 1 -- 172.21.15.7:0/1684835547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa450100800 msgr2=0x7fa4501002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455e12640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa450100800 0x7fa4501002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455e12640 1 -- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4501011c0 msgr2=0x7fa45019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455e12640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4501011c0 0x7fa45019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455e12640 1 -- 172.21.15.7:0/1684835547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45019b300 con 0x7fa450101c40 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa455e12640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450101c40 0x7fa45019aa40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa448009bf0 tx=0x7fa4480323b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.004+0000 7fa4467fc640 1 -- 172.21.15.7:0/1684835547 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448039040 con 0x7fa450101c40 2024-03-26T08:43:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.005+0000 7fa4467fc640 1 -- 172.21.15.7:0/1684835547 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa448044070 con 0x7fa450101c40 2024-03-26T08:43:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.005+0000 7fa457875640 1 -- 172.21.15.7:0/1684835547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45019b590 con 0x7fa450101c40 2024-03-26T08:43:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.005+0000 7fa4467fc640 1 -- 172.21.15.7:0/1684835547 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44803f440 con 0x7fa450101c40 2024-03-26T08:43:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.005+0000 7fa457875640 1 -- 172.21.15.7:0/1684835547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45019bad0 con 0x7fa450101c40 2024-03-26T08:43:21.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.008+0000 7fa4467fc640 1 -- 172.21.15.7:0/1684835547 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) v1 ==== 137340+0+0 (secure 0 0 0) 0x7fa448004040 con 0x7fa450101c40 2024-03-26T08:43:21.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.008+0000 7fa45a45b740 1 Processor -- start 2024-03-26T08:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.008+0000 7fa45a45b740 1 -- start start 2024-03-26T08:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.008+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 0x561f683e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.008+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683e5790 0x561f683ea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683ea900 0x561f683ecd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa45a45b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f6834a3d0 con 0x561f683e4da0 2024-03-26T08:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa45a45b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f682f2ad0 con 0x561f683e5790 2024-03-26T08:43:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 0x561f683e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa45a45b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f68346470 con 0x561f683ea900 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa455611640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683e5790 0x561f683ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 0x561f683e51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53858/0 (socket says 172.21.15.7:53858) 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 -- 172.21.15.7:0/3621447109 learned_addr learned my addr 172.21.15.7:0/3621447109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 --2- 172.21.15.7:0/3621447109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683ea900 0x561f683ecd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 -- 172.21.15.7:0/3621447109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683ea900 msgr2=0x561f683ecd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 --2- 172.21.15.7:0/3621447109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683ea900 0x561f683ecd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 -- 172.21.15.7:0/3621447109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683e5790 msgr2=0x561f683ea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 --2- 172.21.15.7:0/3621447109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683e5790 0x561f683ea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 -- 172.21.15.7:0/3621447109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f681287d0 con 0x561f683e4da0 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa455611640 1 --2- 172.21.15.7:0/3621447109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683e5790 0x561f683ea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:43:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa4467fc640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa43c0a29c0 0x7fa43c0a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.009+0000 7fa454e10640 1 --2- 172.21.15.7:0/3621447109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 0x561f683e51c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa4400135b0 tx=0x7fa440009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53592076b396772 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.010+0000 7fa455611640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa43c0a29c0 0x7fa43c0a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.010+0000 7fa4467fc640 1 -- 172.21.15.7:0/1684835547 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa4480f0f00 con 0x7fa450101c40 2024-03-26T08:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.010+0000 7fa43b7fe640 1 -- 172.21.15.7:0/3621447109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa440004020 con 0x561f683e4da0 2024-03-26T08:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.010+0000 7fa43b7fe640 1 -- 172.21.15.7:0/3621447109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4400041c0 con 0x561f683e4da0 2024-03-26T08:43:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.010+0000 7fa43b7fe640 1 -- 172.21.15.7:0/3621447109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44000f870 con 0x561f683e4da0 2024-03-26T08:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa455611640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa43c0a29c0 0x7fa43c0a4e80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x561f683e5b90 tx=0x7fa44c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 -- 172.21.15.7:0/3621447109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 msgr2=0x561f683e51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 --2- 172.21.15.7:0/3621447109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 0x561f683e51c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa4400135b0 tx=0x7fa440009830 comp rx=0 tx=0).stop 2024-03-26T08:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 -- 172.21.15.7:0/3621447109 shutdown_connections 2024-03-26T08:43:22.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 --2- 172.21.15.7:0/3621447109 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683ea900 0x561f683ecd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 --2- 172.21.15.7:0/3621447109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683e5790 0x561f683ea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 --2- 172.21.15.7:0/3621447109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 0x561f683e51c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 -- 172.21.15.7:0/3621447109 >> 172.21.15.7:0/3621447109 conn(0x561f683e0310 msgr2=0x561f683e0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 -- 172.21.15.7:0/3621447109 shutdown_connections 2024-03-26T08:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 -- 172.21.15.7:0/3621447109 wait complete. 2024-03-26T08:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 -- start start 2024-03-26T08:43:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.011+0000 7fa45a45b740 1 Processor -- start 2024-03-26T08:43:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 0x561f683ef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 0x561f683ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f683f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa45a45b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f683ed8b0 con 0x561f683e5790 2024-03-26T08:43:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa45a45b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f68306060 con 0x561f683ea900 2024-03-26T08:43:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa45a45b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f683eda30 con 0x561f683e4da0 2024-03-26T08:43:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa455611640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 0x561f683ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa45a45b740 10 client.0 _ll_register_callbacks cb 0x561f682e86a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa455611640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 0x561f683ef290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58818/0 (socket says 172.21.15.7:58818) 2024-03-26T08:43:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa455611640 1 -- 172.21.15.7:0/1113537837 learned_addr learned my addr 172.21.15.7:0/1113537837 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.012+0000 7fa455e12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f683f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.013+0000 7fa455e12640 1 -- 172.21.15.7:0/1113537837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 msgr2=0x561f683ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.013+0000 7fa455e12640 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 0x561f683ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.013+0000 7fa455e12640 1 -- 172.21.15.7:0/1113537837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 msgr2=0x561f683ef7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:43:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.013+0000 7fa455e12640 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 0x561f683ef7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.013+0000 7fa455e12640 1 -- 172.21.15.7:0/1113537837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f6805e3f0 con 0x561f683ea900 2024-03-26T08:43:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.013+0000 7fa455e12640 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f683f6810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4480ee980 tx=0x7fa4480bd6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=384df8f3acef8f96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.013+0000 7fa438ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa438ff9640 1 -- 172.21.15.7:0/1113537837 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480bb950 con 0x561f683ea900 2024-03-26T08:43:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa438ff9640 1 -- 172.21.15.7:0/1113537837 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa4480bda70 con 0x561f683ea900 2024-03-26T08:43:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa45a45b740 1 -- 172.21.15.7:0/1113537837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4400041c0 con 0x561f683ea900 2024-03-26T08:43:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa438ff9640 1 -- 172.21.15.7:0/1113537837 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa448032970 con 0x561f683ea900 2024-03-26T08:43:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa45a45b740 10 client.25571 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa45a45b740 1 -- 172.21.15.7:0/1113537837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561f6801e0f0 con 0x561f683ea900 2024-03-26T08:43:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa438ff9640 1 -- 172.21.15.7:0/1113537837 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4400041c0 con 0x561f683ea900 2024-03-26T08:43:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa438ff9640 1 -- 172.21.15.7:0/1113537837 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa4480b9040 con 0x561f683ea900 2024-03-26T08:43:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa45a45b740 10 client.25571 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa45a45b740 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa45a45b740 20 client.25571 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa45a45b740 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561f683fac20 0x561f683fd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa45a45b740 4 client.25571 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.014+0000 7fa45a45b740 1 -- 172.21.15.7:0/1113537837 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f68357910 con 0x561f683fac20 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.015+0000 7fa455611640 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561f683fac20 0x561f683fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.015+0000 7fa438ff9640 10 client.25571 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa455611640 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561f683fac20 0x561f683fd080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756c2e69cd076955 server_cookie=36e9d52a07acd7c4 in_seq=0 out_seq=0 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa438ff9640 1 -- 172.21.15.7:0/1113537837 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561f68357910 con 0x561f683fac20 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa438ff9640 10 client.25571 handle_command_reply: tid=0 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa45a45b740 1 client.25571 shutdown 2024-03-26T08:43:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa45a45b740 1 -- 172.21.15.7:0/1113537837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 msgr2=0x561f683f6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa45a45b740 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f683f6810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa4480ee980 tx=0x7fa4480bd6a0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa45a45b740 1 -- 172.21.15.7:0/1113537837 shutdown_connections 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa45a45b740 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561f683fac20 0x561f683fd080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa438ff9640 0 client.25571 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa45a45b740 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f683f6810 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa45a45b740 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 0x561f683ef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.016+0000 7fa45a45b740 1 --2- 172.21.15.7:0/1113537837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 0x561f683ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.017+0000 7fa45a45b740 1 -- 172.21.15.7:0/1113537837 >> 172.21.15.7:0/1113537837 conn(0x561f683e0310 msgr2=0x561f683df530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.017+0000 7fa45a45b740 1 -- 172.21.15.7:0/1113537837 shutdown_connections 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.017+0000 7fa45a45b740 1 -- 172.21.15.7:0/1113537837 wait complete. 2024-03-26T08:43:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.018+0000 7fa45a45b740 20 client.25571 trim_cache size 0 max 16384 2024-03-26T08:43:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.051+0000 7fa45a45b740 1 Processor -- start 2024-03-26T08:43:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.051+0000 7fa45a45b740 1 -- start start 2024-03-26T08:43:22.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.052+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 0x561f683e3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.052+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e5790 0x561f683f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.052+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f68401940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.052+0000 7fa45a45b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4501a8c70 con 0x561f683e4da0 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.052+0000 7fa455e12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f68401940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.052+0000 7fa455e12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f68401940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36114/0 (socket says 172.21.15.7:36114) 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.052+0000 7fa455e12640 1 -- 172.21.15.7:0/3894479083 learned_addr learned my addr 172.21.15.7:0/3894479083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.053+0000 7fa45a45b740 1 -- 172.21.15.7:0/3894479083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4501a4760 con 0x561f683ea900 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.053+0000 7fa45a45b740 1 -- 172.21.15.7:0/3894479083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4501a27e0 con 0x561f683e5790 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.053+0000 7fa454e10640 1 --2- 172.21.15.7:0/3894479083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e5790 0x561f683f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.053+0000 7fa455e12640 1 -- 172.21.15.7:0/3894479083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e5790 msgr2=0x561f683f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.053+0000 7fa455e12640 1 --2- 172.21.15.7:0/3894479083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e5790 0x561f683f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.053+0000 7fa455e12640 1 -- 172.21.15.7:0/3894479083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 msgr2=0x561f683e3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.053+0000 7fa455e12640 1 --2- 172.21.15.7:0/3894479083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 0x561f683e3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.053+0000 7fa455e12640 1 -- 172.21.15.7:0/3894479083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f683e1080 con 0x561f683ea900 2024-03-26T08:43:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.054+0000 7fa455e12640 1 --2- 172.21.15.7:0/3894479083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f68401940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4480bd7a0 tx=0x7fa4480bd710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0fd33bb7aad4d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.055+0000 7fa43affd640 1 -- 172.21.15.7:0/3894479083 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480bb950 con 0x561f683ea900 2024-03-26T08:43:22.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.056+0000 7fa43affd640 1 -- 172.21.15.7:0/3894479083 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4480bdf10 con 0x561f683ea900 2024-03-26T08:43:22.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.056+0000 7fa43affd640 1 -- 172.21.15.7:0/3894479083 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480bdbe0 con 0x561f683ea900 2024-03-26T08:43:22.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.057+0000 7fa45a45b740 1 -- 172.21.15.7:0/3894479083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 msgr2=0x561f68401940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.057+0000 7fa45a45b740 1 --2- 172.21.15.7:0/3894479083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f68401940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4480bd7a0 tx=0x7fa4480bd710 comp rx=0 tx=0).stop 2024-03-26T08:43:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.057+0000 7fa45a45b740 1 -- 172.21.15.7:0/3894479083 shutdown_connections 2024-03-26T08:43:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.057+0000 7fa45a45b740 1 --2- 172.21.15.7:0/3894479083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f68401940 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.057+0000 7fa45a45b740 1 --2- 172.21.15.7:0/3894479083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e5790 0x561f683f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.057+0000 7fa45a45b740 1 --2- 172.21.15.7:0/3894479083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e4da0 0x561f683e3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.057+0000 7fa45a45b740 1 -- 172.21.15.7:0/3894479083 >> 172.21.15.7:0/3894479083 conn(0x561f683e0310 msgr2=0x561f683e3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.058+0000 7fa45a45b740 1 -- 172.21.15.7:0/3894479083 shutdown_connections 2024-03-26T08:43:22.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.058+0000 7fa45a45b740 1 -- 172.21.15.7:0/3894479083 wait complete. 2024-03-26T08:43:22.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.059+0000 7fa45a45b740 1 -- start start 2024-03-26T08:43:22.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.059+0000 7fa45a45b740 1 Processor -- start 2024-03-26T08:43:22.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 0x561f683f6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 0x561f683f4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa45a45b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f683f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa45a45b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4501a2de0 con 0x561f683e5790 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa45a45b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4501a4100 con 0x561f683ea900 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa45a45b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4501a3440 con 0x561f683e4da0 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa45a45b740 10 client.0 _ll_register_callbacks cb 0x7fa4480b9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa454e10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 0x561f683f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa454e10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 0x561f683f4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53882/0 (socket says 172.21.15.7:53882) 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.060+0000 7fa454e10640 1 -- 172.21.15.7:0/2582021371 learned_addr learned my addr 172.21.15.7:0/2582021371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.061+0000 7fa455611640 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 0x561f683f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.061+0000 7fa455e12640 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f683f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.061+0000 7fa454e10640 1 -- 172.21.15.7:0/2582021371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 msgr2=0x561f683f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:22.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.061+0000 7fa454e10640 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 0x561f683f6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.061+0000 7fa454e10640 1 -- 172.21.15.7:0/2582021371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 msgr2=0x561f683f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:22.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.061+0000 7fa454e10640 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f683f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.061+0000 7fa454e10640 1 -- 172.21.15.7:0/2582021371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f683f5390 con 0x561f683e5790 2024-03-26T08:43:22.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.062+0000 7fa454e10640 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 0x561f683f4910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa44001c000 tx=0x7fa440009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b20647feaf5ff7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:22.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.063+0000 7fa438ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:43:22.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.064+0000 7fa438ff9640 1 -- 172.21.15.7:0/2582021371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4400035f0 con 0x561f683e5790 2024-03-26T08:43:22.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.064+0000 7fa438ff9640 1 -- 172.21.15.7:0/2582021371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa440003790 con 0x561f683e5790 2024-03-26T08:43:22.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.065+0000 7fa45a45b740 1 -- 172.21.15.7:0/2582021371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4480bdf10 con 0x561f683e5790 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.065+0000 7fa438ff9640 1 -- 172.21.15.7:0/2582021371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa440003990 con 0x561f683e5790 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.065+0000 7fa45a45b740 10 client.15735 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.065+0000 7fa45a45b740 1 -- 172.21.15.7:0/2582021371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561f684571d0 con 0x561f683e5790 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.065+0000 7fa438ff9640 1 -- 172.21.15.7:0/2582021371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa4480bdf10 con 0x561f683e5790 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.065+0000 7fa438ff9640 1 -- 172.21.15.7:0/2582021371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa44000fc60 con 0x561f683e5790 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.066+0000 7fa45a45b740 10 client.15735 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.066+0000 7fa45a45b740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.066+0000 7fa45a45b740 20 client.15735 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.066+0000 7fa45a45b740 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561f683fac20 0x561f68459910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.066+0000 7fa45a45b740 4 client.15735 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:43:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.066+0000 7fa45a45b740 1 -- 172.21.15.7:0/2582021371 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f68351c60 con 0x561f683fac20 2024-03-26T08:43:22.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.066+0000 7fa455611640 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561f683fac20 0x561f68459910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:22.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.066+0000 7fa438ff9640 10 client.15735 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:22.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.066+0000 7fa455611640 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561f683fac20 0x561f68459910 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdac05f67aca4036 server_cookie=52a9c1c6c33c33ea in_seq=0 out_seq=0 2024-03-26T08:43:22.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa438ff9640 1 -- 172.21.15.7:0/2582021371 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561f68351c60 con 0x561f683fac20 2024-03-26T08:43:22.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa438ff9640 10 client.15735 handle_command_reply: tid=0 2024-03-26T08:43:22.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 client.15735 shutdown 2024-03-26T08:43:22.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 -- 172.21.15.7:0/2582021371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 msgr2=0x561f683f4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:22.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 0x561f683f4910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa44001c000 tx=0x7fa440009640 comp rx=0 tx=0).stop 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 -- 172.21.15.7:0/2582021371 shutdown_connections 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561f683fac20 0x561f68459910 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f683ea900 0x561f683f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f683e5790 0x561f683f4910 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 --2- 172.21.15.7:0/2582021371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f683e4da0 0x561f683f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa438ff9640 0 client.15735 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 -- 172.21.15.7:0/2582021371 >> 172.21.15.7:0/2582021371 conn(0x561f683e0310 msgr2=0x561f683e3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 -- 172.21.15.7:0/2582021371 shutdown_connections 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 1 -- 172.21.15.7:0/2582021371 wait complete. 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.067+0000 7fa45a45b740 20 client.15735 trim_cache size 0 max 16384 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:43:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:43:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:43:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "7be6ec80-657a-404b-9ff5-8c6799465888": { 2024-03-26T08:43:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:43:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "7be6ec80-657a-404b-9ff5-8c6799465888", 2024-03-26T08:43:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:43:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:43:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:43:22.061 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:43:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 -- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa43c0a29c0 msgr2=0x7fa43c0a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa43c0a29c0 0x7fa43c0a4e80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x561f683e5b90 tx=0x7fa44c005f90 comp rx=0 tx=0).stop 2024-03-26T08:43:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 -- 172.21.15.7:0/1684835547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450101c40 msgr2=0x7fa45019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450101c40 0x7fa45019aa40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa448009bf0 tx=0x7fa4480323b0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 -- 172.21.15.7:0/1684835547 shutdown_connections 2024-03-26T08:43:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa43c0a29c0 0x7fa43c0a4e80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa450101c40 0x7fa45019aa40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4501011c0 0x7fa45019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 --2- 172.21.15.7:0/1684835547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa450100800 0x7fa4501002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 -- 172.21.15.7:0/1684835547 >> 172.21.15.7:0/1684835547 conn(0x7fa4500fb2b0 msgr2=0x7fa450104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.068+0000 7fa457875640 1 -- 172.21.15.7:0/1684835547 shutdown_connections 2024-03-26T08:43:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:22.069+0000 7fa457875640 1 -- 172.21.15.7:0/1684835547 wait complete. 2024-03-26T08:43:22.075 INFO:tasks.cephfs.filesystem:scrub status for tag:7be6ec80-657a-404b-9ff5-8c6799465888 - {'path': '/', 'tag': '7be6ec80-657a-404b-9ff5-8c6799465888', 'options': 'recursive,force'} 2024-03-26T08:43:22.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:21 smithi016 ceph-mon[35851]: pgmap v388: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 783 op/s 2024-03-26T08:43:22.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:21 smithi007 ceph-mon[32462]: pgmap v388: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 783 op/s 2024-03-26T08:43:22.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:21 smithi044 ceph-mon[36461]: pgmap v388: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 783 op/s 2024-03-26T08:43:23.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:22 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:43:23.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:22 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:23.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:22 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:43:23.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:22 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:43:23.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:22 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:23.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:22 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:23.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:22 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:43:23.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:22 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:23.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:22 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:43:23.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:22 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:43:23.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:22 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:23.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:22 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:23.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:22 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:43:23.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:22 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:23.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:22 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:43:23.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:22 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:43:23.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:22 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:23.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:22 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:24.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:23 smithi016 ceph-mon[35851]: pgmap v389: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 914 op/s 2024-03-26T08:43:24.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:23 smithi007 ceph-mon[32462]: pgmap v389: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 914 op/s 2024-03-26T08:43:24.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:23 smithi044 ceph-mon[36461]: pgmap v389: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 914 op/s 2024-03-26T08:43:26.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 55.0 s, 4057.8 tps, lat 24.579 ms stddev 26.911 2024-03-26T08:43:26.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:25 smithi016 ceph-mon[35851]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.3 MiB/s wr, 843 op/s 2024-03-26T08:43:26.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:25 smithi007 ceph-mon[32462]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.3 MiB/s wr, 843 op/s 2024-03-26T08:43:26.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:25 smithi044 ceph-mon[36461]: pgmap v390: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.3 MiB/s wr, 843 op/s 2024-03-26T08:43:28.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:27 smithi016 ceph-mon[35851]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 9.5 MiB/s wr, 845 op/s 2024-03-26T08:43:28.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:27 smithi007 ceph-mon[32462]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 9.5 MiB/s wr, 845 op/s 2024-03-26T08:43:28.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:27 smithi044 ceph-mon[36461]: pgmap v391: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 9.5 MiB/s wr, 845 op/s 2024-03-26T08:43:30.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:29 smithi016 ceph-mon[35851]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 10 MiB/s wr, 872 op/s 2024-03-26T08:43:30.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:29 smithi007 ceph-mon[32462]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 10 MiB/s wr, 872 op/s 2024-03-26T08:43:30.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:29 smithi044 ceph-mon[36461]: pgmap v392: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 10 MiB/s wr, 872 op/s 2024-03-26T08:43:31.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 60.0 s, 3880.2 tps, lat 25.814 ms stddev 27.809 2024-03-26T08:43:32.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:31 smithi016 ceph-mon[35851]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-03-26T08:43:32.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:31 smithi016 ceph-mon[35851]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-26T08:43:32.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:31 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:43:32.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:31 smithi007 ceph-mon[32462]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-03-26T08:43:32.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:31 smithi007 ceph-mon[32462]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-26T08:43:32.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:31 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:43:32.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:31 smithi044 ceph-mon[36461]: pgmap v393: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-03-26T08:43:32.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:31 smithi044 ceph-mon[36461]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-26T08:43:32.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:31 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:43:33.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:32 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:43:33.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:43:33.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.243 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:32 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:32 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:43:33.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:43:33.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:32 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:32 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:43:33.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:33.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:32 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:34.238 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:33 smithi007 ceph-mon[32462]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 11 MiB/s wr, 942 op/s 2024-03-26T08:43:34.239 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:43:34.239 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:43:34.239 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:33 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:34.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:33 smithi016 ceph-mon[35851]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 11 MiB/s wr, 942 op/s 2024-03-26T08:43:34.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:43:34.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:43:34.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:33 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:34.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:33 smithi044 ceph-mon[36461]: pgmap v394: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 11 MiB/s wr, 942 op/s 2024-03-26T08:43:34.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:43:34.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:43:34.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:33 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:43:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.978+0000 7f65f5762640 1 -- 172.21.15.7:0/998625111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f01086a0 msgr2=0x7f65f010aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.978+0000 7f65f5762640 1 --2- 172.21.15.7:0/998625111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f01086a0 0x7f65f010aad0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009860 tx=0x7f65e802bb90 comp rx=0 tx=0).stop 2024-03-26T08:43:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.978+0000 7f65f5762640 1 -- 172.21.15.7:0/998625111 shutdown_connections 2024-03-26T08:43:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.978+0000 7f65f5762640 1 --2- 172.21.15.7:0/998625111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65f010d9c0 0x7f65f010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.978+0000 7f65f5762640 1 --2- 172.21.15.7:0/998625111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65f010b010 0x7f65f010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.978+0000 7f65f5762640 1 --2- 172.21.15.7:0/998625111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f01086a0 0x7f65f010aad0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.978+0000 7f65f5762640 1 -- 172.21.15.7:0/998625111 >> 172.21.15.7:0/998625111 conn(0x7f65f006e6b0 msgr2=0x7f65f0070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.979+0000 7f65f5762640 1 -- 172.21.15.7:0/998625111 shutdown_connections 2024-03-26T08:43:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.979+0000 7f65f5762640 1 -- 172.21.15.7:0/998625111 wait complete. 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.979+0000 7f65f5762640 1 Processor -- start 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65f5762640 1 -- start start 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65f5762640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65f01086a0 0x7f65f01a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65f5762640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f010b010 0x7f65f01a6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65f5762640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65f010d9c0 0x7f65f01addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65f5762640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f011c1a0 con 0x7f65f01086a0 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65f5762640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f011c020 con 0x7f65f010b010 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65f5762640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f011c320 con 0x7f65f010d9c0 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65f010d9c0 0x7f65f01addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f010b010 0x7f65f01a6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f010b010 0x7f65f01a6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57200/0 (socket says 172.21.15.7:57200) 2024-03-26T08:43:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65f010d9c0 0x7f65f01addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38984/0 (socket says 172.21.15.7:38984) 2024-03-26T08:43:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65ee7fc640 1 -- 172.21.15.7:0/3183407971 learned_addr learned my addr 172.21.15.7:0/3183407971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.980+0000 7f65eeffd640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65f01086a0 0x7f65f01a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.981+0000 7f65eeffd640 1 -- 172.21.15.7:0/3183407971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65f010d9c0 msgr2=0x7f65f01addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.981+0000 7f65eeffd640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65f010d9c0 0x7f65f01addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.981+0000 7f65eeffd640 1 -- 172.21.15.7:0/3183407971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f010b010 msgr2=0x7f65f01a6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.981+0000 7f65eeffd640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f010b010 0x7f65f01a6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.981+0000 7f65eeffd640 1 -- 172.21.15.7:0/3183407971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f01ae430 con 0x7f65f01086a0 2024-03-26T08:43:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.981+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65f010d9c0 0x7f65f01addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.981+0000 7f65eeffd640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65f01086a0 0x7f65f01a6810 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f65e8034980 tx=0x7f65e8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.982+0000 7f65d7fff640 1 -- 172.21.15.7:0/3183407971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8041020 con 0x7f65f01086a0 2024-03-26T08:43:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.982+0000 7f65f5762640 1 -- 172.21.15.7:0/3183407971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f01ae6c0 con 0x7f65f01086a0 2024-03-26T08:43:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.982+0000 7f65d7fff640 1 -- 172.21.15.7:0/3183407971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65e802ca10 con 0x7f65f01086a0 2024-03-26T08:43:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.982+0000 7f65f5762640 1 -- 172.21.15.7:0/3183407971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f01aec00 con 0x7f65f01086a0 2024-03-26T08:43:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.982+0000 7f65d7fff640 1 -- 172.21.15.7:0/3183407971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8041020 con 0x7f65f01086a0 2024-03-26T08:43:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.986+0000 7f65d7fff640 1 -- 172.21.15.7:0/3183407971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f65e8044020 con 0x7f65f01086a0 2024-03-26T08:43:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.986+0000 7f65f842d740 1 Processor -- start 2024-03-26T08:43:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.986+0000 7f65d7fff640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d80a2c30 0x7f65d80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.987+0000 7f65d7fff640 1 -- 172.21.15.7:0/3183407971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f65e8004230 con 0x7f65f01086a0 2024-03-26T08:43:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.987+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d80a2c30 0x7f65d80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.987+0000 7f65f842d740 1 -- start start 2024-03-26T08:43:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d80a2c30 0x7f65d80a50f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f65e4005fd0 tx=0x7f65e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65f842d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a440d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65f842d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 0x562c3a445e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65f842d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a448910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a440d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a440d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57204/0 (socket says 172.21.15.7:57204) 2024-03-26T08:43:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65f842d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3a3a5b10 con 0x562c3a441310 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65f842d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3a34e900 con 0x562c3a440920 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65eeffd640 1 -- 172.21.15.7:0/4043880471 learned_addr learned my addr 172.21.15.7:0/4043880471 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65f842d740 1 -- 172.21.15.7:0/4043880471 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3a3a1bb0 con 0x562c3a446480 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65ee7fc640 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 0x562c3a445e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.988+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a448910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.989+0000 7f65ee7fc640 1 -- 172.21.15.7:0/4043880471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 msgr2=0x562c3a448910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.989+0000 7f65ee7fc640 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a448910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.989+0000 7f65ee7fc640 1 -- 172.21.15.7:0/4043880471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 msgr2=0x562c3a440d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.989+0000 7f65ee7fc640 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a440d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.989+0000 7f65ee7fc640 1 -- 172.21.15.7:0/4043880471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c3a0ba210 con 0x562c3a441310 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.989+0000 7f65eeffd640 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a440d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.989+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a448910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.989+0000 7f65ee7fc640 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 0x562c3a445e10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f65e0011440 tx=0x7f65e0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34e4c3fe5a98f9d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.990+0000 7f65d57fa640 1 -- 172.21.15.7:0/4043880471 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0003c30 con 0x562c3a441310 2024-03-26T08:43:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.990+0000 7f65d57fa640 1 -- 172.21.15.7:0/4043880471 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65e0003dd0 con 0x562c3a441310 2024-03-26T08:43:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.990+0000 7f65d57fa640 1 -- 172.21.15.7:0/4043880471 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e000fb40 con 0x562c3a441310 2024-03-26T08:43:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.991+0000 7f65f842d740 1 -- 172.21.15.7:0/4043880471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 msgr2=0x562c3a445e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.991+0000 7f65f842d740 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 0x562c3a445e10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f65e0011440 tx=0x7f65e0011900 comp rx=0 tx=0).stop 2024-03-26T08:43:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.992+0000 7f65f842d740 1 -- 172.21.15.7:0/4043880471 shutdown_connections 2024-03-26T08:43:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.992+0000 7f65f842d740 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a448910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.992+0000 7f65f842d740 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 0x562c3a445e10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.992+0000 7f65f842d740 1 --2- 172.21.15.7:0/4043880471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a440d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.992+0000 7f65f842d740 1 -- 172.21.15.7:0/4043880471 >> 172.21.15.7:0/4043880471 conn(0x562c3a43be90 msgr2=0x562c3a43c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.992+0000 7f65f842d740 1 -- 172.21.15.7:0/4043880471 shutdown_connections 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.992+0000 7f65f842d740 1 -- 172.21.15.7:0/4043880471 wait complete. 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65f842d740 1 -- start start 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65f842d740 1 Processor -- start 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65f842d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a44ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65f842d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 0x562c3a44b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65f842d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a452130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65f842d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3a449430 con 0x562c3a441310 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65f842d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3a361d50 con 0x562c3a440920 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65f842d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3a4495b0 con 0x562c3a446480 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65f842d740 10 client.0 _ll_register_callbacks cb 0x562c3a361720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a452130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a44ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a44ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57206/0 (socket says 172.21.15.7:57206) 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a452130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39010/0 (socket says 172.21.15.7:39010) 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65eeffd640 1 -- 172.21.15.7:0/2297337538 learned_addr learned my addr 172.21.15.7:0/2297337538 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.993+0000 7f65ee7fc640 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 0x562c3a44b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65ef7fe640 1 -- 172.21.15.7:0/2297337538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 msgr2=0x562c3a44ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a44ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65ef7fe640 1 -- 172.21.15.7:0/2297337538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 msgr2=0x562c3a44b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 0x562c3a44b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65ef7fe640 1 -- 172.21.15.7:0/2297337538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c3a186620 con 0x562c3a446480 2024-03-26T08:43:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65eeffd640 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a44ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a452130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f65e4003890 tx=0x7f65e4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d6467db8044667 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:43:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65d2ff5640 1 -- 172.21.15.7:0/2297337538 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e401eb10 con 0x562c3a446480 2024-03-26T08:43:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.994+0000 7f65d2ff5640 1 -- 172.21.15.7:0/2297337538 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f65e401ecb0 con 0x562c3a446480 2024-03-26T08:43:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.995+0000 7f65f842d740 1 -- 172.21.15.7:0/2297337538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65e0003dd0 con 0x562c3a446480 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.995+0000 7f65d2ff5640 1 -- 172.21.15.7:0/2297337538 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65e4028420 con 0x562c3a446480 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.995+0000 7f65f842d740 10 client.25597 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.995+0000 7f65f842d740 1 -- 172.21.15.7:0/2297337538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562c3a455510 con 0x562c3a446480 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.995+0000 7f65d2ff5640 1 -- 172.21.15.7:0/2297337538 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65e0003dd0 con 0x562c3a446480 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65d2ff5640 1 -- 172.21.15.7:0/2297337538 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f65e401a050 con 0x562c3a446480 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65f842d740 10 client.25597 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65f842d740 10 client.25597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65f842d740 20 client.25597 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65f842d740 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562c3a456c30 0x562c3a459050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65f842d740 4 client.25597 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65f842d740 1 -- 172.21.15.7:0/2297337538 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c3a3b3050 con 0x562c3a456c30 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65eeffd640 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562c3a456c30 0x562c3a459050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65d2ff5640 10 client.25597 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.996+0000 7f65eeffd640 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562c3a456c30 0x562c3a459050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86aef495f5bdcb0e server_cookie=fd8551e58175f306 in_seq=0 out_seq=0 2024-03-26T08:43:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.997+0000 7f65d2ff5640 1 -- 172.21.15.7:0/2297337538 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562c3a3b3050 con 0x562c3a456c30 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.997+0000 7f65d2ff5640 10 client.25597 handle_command_reply: tid=0 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65f842d740 1 client.25597 shutdown 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65f842d740 1 -- 172.21.15.7:0/2297337538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 msgr2=0x562c3a452130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65f842d740 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a452130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f65e4003890 tx=0x7f65e4004730 comp rx=0 tx=0).stop 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65f842d740 1 -- 172.21.15.7:0/2297337538 shutdown_connections 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65f842d740 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562c3a456c30 0x562c3a459050 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65f842d740 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a452130 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65f842d740 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a441310 0x562c3a44b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65d2ff5640 0 client.25597 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65f842d740 1 --2- 172.21.15.7:0/2297337538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a440920 0x562c3a44ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.998+0000 7f65f842d740 1 -- 172.21.15.7:0/2297337538 >> 172.21.15.7:0/2297337538 conn(0x562c3a43be90 msgr2=0x562c3a43b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.999+0000 7f65f842d740 1 -- 172.21.15.7:0/2297337538 shutdown_connections 2024-03-26T08:43:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.999+0000 7f65f842d740 1 -- 172.21.15.7:0/2297337538 wait complete. 2024-03-26T08:43:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:34.999+0000 7f65f842d740 20 client.25597 trim_cache size 0 max 16384 2024-03-26T08:43:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.025+0000 7f65f842d740 1 Processor -- start 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.025+0000 7f65f842d740 1 -- start start 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65f842d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a440920 0x562c3a449ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65f842d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 0x562c3a452050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65f842d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a446480 0x562c3a44a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65eeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a440920 0x562c3a449ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65eeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a440920 0x562c3a449ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39024/0 (socket says 172.21.15.7:39024) 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 0x562c3a452050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65ee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 0x562c3a452050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57212/0 (socket says 172.21.15.7:57212) 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65eeffd640 1 -- 172.21.15.7:0/2699718856 learned_addr learned my addr 172.21.15.7:0/2699718856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65f842d740 1 -- 172.21.15.7:0/2699718856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc010990 con 0x562c3a446480 2024-03-26T08:43:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65f842d740 1 -- 172.21.15.7:0/2699718856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00e490 con 0x562c3a441310 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65f842d740 1 -- 172.21.15.7:0/2699718856 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00c510 con 0x562c3a440920 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/2699718856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a446480 0x562c3a44a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65ee7fc640 1 -- 172.21.15.7:0/2699718856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a440920 msgr2=0x562c3a449ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65ee7fc640 1 --2- 172.21.15.7:0/2699718856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a440920 0x562c3a449ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65ee7fc640 1 -- 172.21.15.7:0/2699718856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a446480 msgr2=0x562c3a44a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65ee7fc640 1 --2- 172.21.15.7:0/2699718856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a446480 0x562c3a44a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.026+0000 7f65ee7fc640 1 -- 172.21.15.7:0/2699718856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c3a449ce0 con 0x562c3a441310 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.027+0000 7f65ee7fc640 1 --2- 172.21.15.7:0/2699718856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 0x562c3a452050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x562c3a43ce00 tx=0x7f65e0009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d7b5dd7e6e43e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.027+0000 7f65d4ff9640 1 -- 172.21.15.7:0/2699718856 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e002a070 con 0x562c3a441310 2024-03-26T08:43:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.027+0000 7f65d4ff9640 1 -- 172.21.15.7:0/2699718856 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f65e0003b70 con 0x562c3a441310 2024-03-26T08:43:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.027+0000 7f65d4ff9640 1 -- 172.21.15.7:0/2699718856 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0005330 con 0x562c3a441310 2024-03-26T08:43:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.029+0000 7f65f842d740 1 -- 172.21.15.7:0/2699718856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 msgr2=0x562c3a452050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.029+0000 7f65f842d740 1 --2- 172.21.15.7:0/2699718856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 0x562c3a452050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x562c3a43ce00 tx=0x7f65e0009fd0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.029+0000 7f65f842d740 1 -- 172.21.15.7:0/2699718856 shutdown_connections 2024-03-26T08:43:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.029+0000 7f65f842d740 1 --2- 172.21.15.7:0/2699718856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a446480 0x562c3a44a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.029+0000 7f65f842d740 1 --2- 172.21.15.7:0/2699718856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 0x562c3a452050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.030+0000 7f65f842d740 1 --2- 172.21.15.7:0/2699718856 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a440920 0x562c3a449ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.030+0000 7f65f842d740 1 -- 172.21.15.7:0/2699718856 >> 172.21.15.7:0/2699718856 conn(0x562c3a43be90 msgr2=0x562c3a43c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.030+0000 7f65f842d740 1 -- 172.21.15.7:0/2699718856 shutdown_connections 2024-03-26T08:43:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.030+0000 7f65f842d740 1 -- 172.21.15.7:0/2699718856 wait complete. 2024-03-26T08:43:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.031+0000 7f65f842d740 1 -- start start 2024-03-26T08:43:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.031+0000 7f65f842d740 1 Processor -- start 2024-03-26T08:43:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.031+0000 7f65f842d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a440920 0x562c3a44fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.031+0000 7f65f842d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 0x562c3a4501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.031+0000 7f65f842d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a4511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a4511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65ef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a4511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39028/0 (socket says 172.21.15.7:39028) 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65ef7fe640 1 -- 172.21.15.7:0/3773280271 learned_addr learned my addr 172.21.15.7:0/3773280271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00cb10 con 0x562c3a440920 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00de30 con 0x562c3a441310 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc00d170 con 0x562c3a446480 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65ef7fe640 1 -- 172.21.15.7:0/3773280271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 msgr2=0x562c3a4501b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 0x562c3a4501b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65ef7fe640 1 -- 172.21.15.7:0/3773280271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a440920 msgr2=0x562c3a44fc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a440920 0x562c3a44fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.032+0000 7f65ef7fe640 1 -- 172.21.15.7:0/3773280271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c3a4517b0 con 0x562c3a446480 2024-03-26T08:43:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.033+0000 7f65ef7fe640 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a4511c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65e4011d40 tx=0x7f65e4032ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a9b21fa5375744 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.033+0000 7f65f842d740 10 client.0 _ll_register_callbacks cb 0x7f65e40035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.033+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65e0003b70 con 0x562c3a446480 2024-03-26T08:43:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.033+0000 7f65d2ff5640 10 client.25603 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:43:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.033+0000 7f65d2ff5640 1 -- 172.21.15.7:0/3773280271 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4032e30 con 0x562c3a446480 2024-03-26T08:43:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.033+0000 7f65d2ff5640 1 -- 172.21.15.7:0/3773280271 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f65e40285c0 con 0x562c3a446480 2024-03-26T08:43:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.033+0000 7f65d2ff5640 1 -- 172.21.15.7:0/3773280271 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65e4028800 con 0x562c3a446480 2024-03-26T08:43:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.033+0000 7f65f842d740 10 client.25603 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.034+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x562c3a4b29a0 con 0x562c3a446480 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.034+0000 7f65d2ff5640 1 -- 172.21.15.7:0/3773280271 <== mon.1 v2:172.21.15.44:3300/0 4 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f65e401a040 con 0x562c3a446480 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.035+0000 7f65f842d740 10 client.25603 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.035+0000 7f65f842d740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.035+0000 7f65f842d740 20 client.25603 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.035+0000 7f65f842d740 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562c3a4b3280 0x562c3a4b5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.035+0000 7f65f842d740 4 client.25603 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.035+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562c3a3ad3a0 con 0x562c3a4b3280 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.035+0000 7f65eeffd640 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562c3a4b3280 0x562c3a4b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.036+0000 7f65eeffd640 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562c3a4b3280 0x562c3a4b5740 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58104ad2cb00faec server_cookie=e7991c91cbe47641 in_seq=0 out_seq=0 2024-03-26T08:43:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.036+0000 7f65d2ff5640 10 client.25603 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65d2ff5640 1 -- 172.21.15.7:0/3773280271 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562c3a3ad3a0 con 0x562c3a4b3280 2024-03-26T08:43:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65d2ff5640 10 client.25603 handle_command_reply: tid=0 2024-03-26T08:43:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65f842d740 1 client.25603 shutdown 2024-03-26T08:43:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 msgr2=0x562c3a4511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65f842d740 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a4511c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f65e4011d40 tx=0x7f65e4032ab0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 shutdown_connections 2024-03-26T08:43:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65f842d740 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562c3a4b3280 0x562c3a4b5740 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65f842d740 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562c3a446480 0x562c3a4511c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65f842d740 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c3a441310 0x562c3a4501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65f842d740 1 --2- 172.21.15.7:0/3773280271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562c3a440920 0x562c3a44fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 >> 172.21.15.7:0/3773280271 conn(0x562c3a43be90 msgr2=0x562c3a449ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.071+0000 7f65d2ff5640 0 client.25603 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.072+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 shutdown_connections 2024-03-26T08:43:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.073+0000 7f65f842d740 1 -- 172.21.15.7:0/3773280271 wait complete. 2024-03-26T08:43:35.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.073+0000 7f65f842d740 20 client.25603 trim_cache size 0 max 16384 2024-03-26T08:43:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:43:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:43:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:43:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:43:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 -- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d80a2c30 msgr2=0x7f65d80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:35.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d80a2c30 0x7f65d80a50f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f65e4005fd0 tx=0x7f65e4009260 comp rx=0 tx=0).stop 2024-03-26T08:43:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 -- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65f01086a0 msgr2=0x7f65f01a6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65f01086a0 0x7f65f01a6810 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f65e8034980 tx=0x7f65e8004040 comp rx=0 tx=0).stop 2024-03-26T08:43:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 -- 172.21.15.7:0/3183407971 shutdown_connections 2024-03-26T08:43:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f65d80a2c30 0x7f65d80a50f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f65f010d9c0 0x7f65f01addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f010b010 0x7f65f01a6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 --2- 172.21.15.7:0/3183407971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65f01086a0 0x7f65f01a6810 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 -- 172.21.15.7:0/3183407971 >> 172.21.15.7:0/3183407971 conn(0x7f65f006e6b0 msgr2=0x7f65f010b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 -- 172.21.15.7:0/3183407971 shutdown_connections 2024-03-26T08:43:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:35.075+0000 7f65f5762640 1 -- 172.21.15.7:0/3183407971 wait complete. 2024-03-26T08:43:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 65.0 s, 4700.4 tps, lat 21.285 ms stddev 22.799 2024-03-26T08:43:36.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:35 smithi016 ceph-mon[35851]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 11 MiB/s wr, 902 op/s 2024-03-26T08:43:36.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:35 smithi007 ceph-mon[32462]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 11 MiB/s wr, 902 op/s 2024-03-26T08:43:36.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:35 smithi044 ceph-mon[36461]: pgmap v395: 97 pgs: 97 active+clean; 1.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 11 MiB/s wr, 902 op/s 2024-03-26T08:43:38.242 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:38 smithi016 ceph-mon[35851]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 12 MiB/s wr, 911 op/s 2024-03-26T08:43:38.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:38 smithi007 ceph-mon[32462]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 12 MiB/s wr, 911 op/s 2024-03-26T08:43:38.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:38 smithi044 ceph-mon[36461]: pgmap v396: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 12 MiB/s wr, 911 op/s 2024-03-26T08:43:40.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:40 smithi007 ceph-mon[32462]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 13 MiB/s wr, 889 op/s 2024-03-26T08:43:40.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:40 smithi044 ceph-mon[36461]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 13 MiB/s wr, 889 op/s 2024-03-26T08:43:40.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:40 smithi016 ceph-mon[35851]: pgmap v397: 97 pgs: 97 active+clean; 1.0 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 13 MiB/s wr, 889 op/s 2024-03-26T08:43:41.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 70.0 s, 4508.0 tps, lat 22.165 ms stddev 22.521 2024-03-26T08:43:42.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:42 smithi007 ceph-mon[32462]: pgmap v398: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 14 MiB/s wr, 916 op/s 2024-03-26T08:43:42.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:42 smithi044 ceph-mon[36461]: pgmap v398: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 14 MiB/s wr, 916 op/s 2024-03-26T08:43:42.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:42 smithi016 ceph-mon[35851]: pgmap v398: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 14 MiB/s wr, 916 op/s 2024-03-26T08:43:44.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:44 smithi007 ceph-mon[32462]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 931 op/s 2024-03-26T08:43:44.293 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:44 smithi016 ceph-mon[35851]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 931 op/s 2024-03-26T08:43:44.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:44 smithi044 ceph-mon[36461]: pgmap v399: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 13 MiB/s wr, 931 op/s 2024-03-26T08:43:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 75.0 s, 3945.6 tps, lat 25.333 ms stddev 26.099 2024-03-26T08:43:46.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:46 smithi007 ceph-mon[32462]: pgmap v400: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 12 MiB/s wr, 848 op/s 2024-03-26T08:43:46.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:46 smithi044 ceph-mon[36461]: pgmap v400: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 12 MiB/s wr, 848 op/s 2024-03-26T08:43:46.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:46 smithi016 ceph-mon[35851]: pgmap v400: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 12 MiB/s wr, 848 op/s 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.193+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2799397489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 msgr2=0x7f6b58102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.193+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2799397489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 0x7f6b58102370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009ca0 tx=0x7f6b5002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.193+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2799397489 shutdown_connections 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.193+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2799397489 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b58105260 0x7f6b581056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.193+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2799397489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b581028b0 0x7f6b58104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.193+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2799397489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 0x7f6b58102370 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.193+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2799397489 >> 172.21.15.7:0/2799397489 conn(0x7f6b580fbb40 msgr2=0x7f6b580fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.193+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2799397489 shutdown_connections 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.193+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2799397489 wait complete. 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.194+0000 7f6b5fa85640 1 Processor -- start 2024-03-26T08:43:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.194+0000 7f6b5fa85640 1 -- start start 2024-03-26T08:43:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.194+0000 7f6b5fa85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 0x7f6b5819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.194+0000 7f6b5fa85640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b581028b0 0x7f6b5819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.194+0000 7f6b5fa85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b58105260 0x7f6b581a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.194+0000 7f6b5d821640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 0x7f6b5819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.194+0000 7f6b5d821640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 0x7f6b5819e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41402/0 (socket says 172.21.15.7:41402) 2024-03-26T08:43:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.194+0000 7f6b5d821640 1 -- 172.21.15.7:0/2301584419 learned_addr learned my addr 172.21.15.7:0/2301584419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58113920 con 0x7f6b58105260 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581137a0 con 0x7f6b580fff40 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58113aa0 con 0x7f6b581028b0 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5d020640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b581028b0 0x7f6b5819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5e022640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b58105260 0x7f6b581a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5d821640 1 -- 172.21.15.7:0/2301584419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b581028b0 msgr2=0x7f6b5819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5d821640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b581028b0 0x7f6b5819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5d821640 1 -- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b58105260 msgr2=0x7f6b581a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5d821640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b58105260 0x7f6b581a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.195+0000 7f6b5d821640 1 -- 172.21.15.7:0/2301584419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b581a5c00 con 0x7f6b580fff40 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.196+0000 7f6b5d821640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 0x7f6b5819e030 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b500325f0 tx=0x7f6b50032ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.196+0000 7f6b4effd640 1 -- 172.21.15.7:0/2301584419 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50041020 con 0x7f6b580fff40 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.196+0000 7f6b4effd640 1 -- 172.21.15.7:0/2301584419 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b50034cd0 con 0x7f6b580fff40 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.196+0000 7f6b4effd640 1 -- 172.21.15.7:0/2301584419 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50039040 con 0x7f6b580fff40 2024-03-26T08:43:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.196+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b581a5e90 con 0x7f6b580fff40 2024-03-26T08:43:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.196+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b581a62f0 con 0x7f6b580fff40 2024-03-26T08:43:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.199+0000 7f6b6267a740 1 Processor -- start 2024-03-26T08:43:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.200+0000 7f6b6267a740 1 -- start start 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.200+0000 7f6b4effd640 1 -- 172.21.15.7:0/2301584419 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6b50044020 con 0x7f6b580fff40 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceeb1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceeb6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 0x5651ceeb9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b6267a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651cee16b10 con 0x5651ceeb1920 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b6267a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651cedbf900 con 0x5651ceeb2310 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b6267a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651cee12bb0 con 0x5651ceeb7480 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5e022640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceeb1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5d020640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 0x5651ceeb9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5d821640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceeb6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5e022640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceeb1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43342/0 (socket says 172.21.15.7:43342) 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5d020640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 0x5651ceeb9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57138/0 (socket says 172.21.15.7:57138) 2024-03-26T08:43:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5e022640 1 -- 172.21.15.7:0/2029140951 learned_addr learned my addr 172.21.15.7:0/2029140951 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5d821640 1 -- 172.21.15.7:0/2029140951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 msgr2=0x5651ceeb9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5d821640 1 --2- 172.21.15.7:0/2029140951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 0x5651ceeb9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5d821640 1 -- 172.21.15.7:0/2029140951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 msgr2=0x5651ceeb1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5d821640 1 --2- 172.21.15.7:0/2029140951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceeb1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b5d821640 1 -- 172.21.15.7:0/2029140951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651ceb2b210 con 0x5651ceeb2310 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.201+0000 7f6b4effd640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6b440a28f0 0x7f6b440a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.202+0000 7f6b4effd640 1 -- 172.21.15.7:0/2301584419 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6b500f07b0 con 0x7f6b580fff40 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.202+0000 7f6b5e022640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6b440a28f0 0x7f6b440a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.202+0000 7f6b5d821640 1 --2- 172.21.15.7:0/2029140951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceeb6e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6b500ee210 tx=0x7f6b500f6b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c50200a8f8ff769 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.202+0000 7f6b43fff640 1 -- 172.21.15.7:0/2029140951 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500b9440 con 0x5651ceeb2310 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.203+0000 7f6b43fff640 1 -- 172.21.15.7:0/2029140951 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b50034920 con 0x5651ceeb2310 2024-03-26T08:43:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.203+0000 7f6b43fff640 1 -- 172.21.15.7:0/2029140951 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500bcb00 con 0x5651ceeb2310 2024-03-26T08:43:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.203+0000 7f6b5e022640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6b440a28f0 0x7f6b440a4db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5651ceeb2b70 tx=0x7f6b54005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.203+0000 7f6b6267a740 1 -- 172.21.15.7:0/2029140951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 msgr2=0x5651ceeb6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.203+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2029140951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceeb6e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6b500ee210 tx=0x7f6b500f6b20 comp rx=0 tx=0).stop 2024-03-26T08:43:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.204+0000 7f6b6267a740 1 -- 172.21.15.7:0/2029140951 shutdown_connections 2024-03-26T08:43:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.204+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2029140951 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 0x5651ceeb9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.204+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2029140951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceeb6e10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.204+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2029140951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceeb1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.204+0000 7f6b6267a740 1 -- 172.21.15.7:0/2029140951 >> 172.21.15.7:0/2029140951 conn(0x5651ceeace90 msgr2=0x5651ceead290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.204+0000 7f6b6267a740 1 -- 172.21.15.7:0/2029140951 shutdown_connections 2024-03-26T08:43:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.204+0000 7f6b6267a740 1 -- 172.21.15.7:0/2029140951 wait complete. 2024-03-26T08:43:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.205+0000 7f6b6267a740 1 -- start start 2024-03-26T08:43:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.205+0000 7f6b6267a740 1 Processor -- start 2024-03-26T08:43:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceebb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceebbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 0x5651ceec2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d821640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceebb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d821640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceebb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43356/0 (socket says 172.21.15.7:43356) 2024-03-26T08:43:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d020640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceebbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d020640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceebbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41424/0 (socket says 172.21.15.7:41424) 2024-03-26T08:43:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d821640 1 -- 172.21.15.7:0/3228608330 learned_addr learned my addr 172.21.15.7:0/3228608330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ceeba1f0 con 0x5651ceeb1920 2024-03-26T08:43:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651cedd2d50 con 0x5651ceeb2310 2024-03-26T08:43:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ceeba370 con 0x5651ceeb7480 2024-03-26T08:43:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5e022640 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 0x5651ceec2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b6267a740 10 client.0 _ll_register_callbacks cb 0x5651cedd2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d821640 1 -- 172.21.15.7:0/3228608330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 msgr2=0x5651ceec2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d821640 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 0x5651ceec2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d821640 1 -- 172.21.15.7:0/3228608330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 msgr2=0x5651ceebbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d821640 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceebbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.206+0000 7f6b5d821640 1 -- 172.21.15.7:0/3228608330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651cebf7620 con 0x5651ceeb1920 2024-03-26T08:43:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.207+0000 7f6b5d821640 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceebb910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6b500ee240 tx=0x7f6b500bb6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e44bf364dde6d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.208+0000 7f6b417fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:43:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.208+0000 7f6b417fa640 1 -- 172.21.15.7:0/3228608330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500bcce0 con 0x5651ceeb1920 2024-03-26T08:43:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.208+0000 7f6b417fa640 1 -- 172.21.15.7:0/3228608330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6b50003a70 con 0x5651ceeb1920 2024-03-26T08:43:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.208+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b50034920 con 0x5651ceeb1920 2024-03-26T08:43:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.208+0000 7f6b417fa640 1 -- 172.21.15.7:0/3228608330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b500bcf70 con 0x5651ceeb1920 2024-03-26T08:43:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.208+0000 7f6b6267a740 10 client.15762 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.208+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5651ceeba1f0 con 0x5651ceeb1920 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.209+0000 7f6b417fa640 1 -- 172.21.15.7:0/3228608330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b50034920 con 0x5651ceeb1920 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.209+0000 7f6b417fa640 1 -- 172.21.15.7:0/3228608330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f6b5003c030 con 0x5651ceeb1920 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.209+0000 7f6b6267a740 10 client.15762 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.209+0000 7f6b6267a740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.209+0000 7f6b6267a740 20 client.15762 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.209+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5651ceec78f0 0x5651ceec9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.209+0000 7f6b6267a740 4 client.15762 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.209+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651cee24050 con 0x5651ceec78f0 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.209+0000 7f6b5d020640 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5651ceec78f0 0x5651ceec9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.210+0000 7f6b5d020640 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5651ceec78f0 0x5651ceec9d70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6d6bfedfe71153 server_cookie=eeeedb0f4b280834 in_seq=0 out_seq=0 2024-03-26T08:43:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.210+0000 7f6b417fa640 10 client.15762 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.211+0000 7f6b417fa640 1 -- 172.21.15.7:0/3228608330 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5651cee24050 con 0x5651ceec78f0 2024-03-26T08:43:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.211+0000 7f6b417fa640 10 client.15762 handle_command_reply: tid=0 2024-03-26T08:43:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.211+0000 7f6b6267a740 1 client.15762 shutdown 2024-03-26T08:43:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.212+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 msgr2=0x5651ceebb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.212+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceebb910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6b500ee240 tx=0x7f6b500bb6b0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 shutdown_connections 2024-03-26T08:43:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5651ceec78f0 0x5651ceec9d70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb7480 0x5651ceec2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb2310 0x5651ceebbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3228608330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb1920 0x5651ceebb910 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b417fa640 0 client.15762 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 >> 172.21.15.7:0/3228608330 conn(0x5651ceeace90 msgr2=0x5651ceeac020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 shutdown_connections 2024-03-26T08:43:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b6267a740 1 -- 172.21.15.7:0/3228608330 wait complete. 2024-03-26T08:43:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.213+0000 7f6b6267a740 20 client.15762 trim_cache size 0 max 16384 2024-03-26T08:43:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.237+0000 7f6b6267a740 1 Processor -- start 2024-03-26T08:43:48.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b6267a740 1 -- start start 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb1920 0x5651ceead270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb2310 0x5651ceec2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb7480 0x5651ceeb3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b6267a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581b1660 con 0x5651ceeb7480 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b6267a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581af160 con 0x5651ceeb1920 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b6267a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581ad1e0 con 0x5651ceeb2310 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b5d020640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb2310 0x5651ceec2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b5d020640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb2310 0x5651ceec2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57156/0 (socket says 172.21.15.7:57156) 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b5e022640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb7480 0x5651ceeb3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b5d020640 1 -- 172.21.15.7:0/3039518392 learned_addr learned my addr 172.21.15.7:0/3039518392 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.238+0000 7f6b5d821640 1 --2- 172.21.15.7:0/3039518392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb1920 0x5651ceead270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.239+0000 7f6b5d020640 1 -- 172.21.15.7:0/3039518392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb1920 msgr2=0x5651ceead270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:48.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.239+0000 7f6b5d020640 1 --2- 172.21.15.7:0/3039518392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb1920 0x5651ceead270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.239+0000 7f6b5d020640 1 -- 172.21.15.7:0/3039518392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb7480 msgr2=0x5651ceeb3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.239+0000 7f6b5d020640 1 --2- 172.21.15.7:0/3039518392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb7480 0x5651ceeb3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.239+0000 7f6b5d020640 1 -- 172.21.15.7:0/3039518392 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651cee21670 con 0x5651ceeb2310 2024-03-26T08:43:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.239+0000 7f6b5d020640 1 --2- 172.21.15.7:0/3039518392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb2310 0x5651ceec2e20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5651ceeade00 tx=0x7f6b48005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d87d056a8dec7ed6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.239+0000 7f6b437fe640 1 -- 172.21.15.7:0/3039518392 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48003d20 con 0x5651ceeb2310 2024-03-26T08:43:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.239+0000 7f6b437fe640 1 -- 172.21.15.7:0/3039518392 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b4801fbe0 con 0x5651ceeb2310 2024-03-26T08:43:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.239+0000 7f6b437fe640 1 -- 172.21.15.7:0/3039518392 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48010b10 con 0x5651ceeb2310 2024-03-26T08:43:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.241+0000 7f6b6267a740 1 -- 172.21.15.7:0/3039518392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb2310 msgr2=0x5651ceec2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.241+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3039518392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb2310 0x5651ceec2e20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5651ceeade00 tx=0x7f6b48005f90 comp rx=0 tx=0).stop 2024-03-26T08:43:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.241+0000 7f6b6267a740 1 -- 172.21.15.7:0/3039518392 shutdown_connections 2024-03-26T08:43:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.241+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3039518392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb7480 0x5651ceeb3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.241+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3039518392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb2310 0x5651ceec2e20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.241+0000 7f6b6267a740 1 --2- 172.21.15.7:0/3039518392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb1920 0x5651ceead270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.241+0000 7f6b6267a740 1 -- 172.21.15.7:0/3039518392 >> 172.21.15.7:0/3039518392 conn(0x5651ceeace90 msgr2=0x5651ceeb6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.241+0000 7f6b6267a740 1 -- 172.21.15.7:0/3039518392 shutdown_connections 2024-03-26T08:43:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.241+0000 7f6b6267a740 1 -- 172.21.15.7:0/3039518392 wait complete. 2024-03-26T08:43:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.242+0000 7f6b6267a740 1 -- start start 2024-03-26T08:43:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.242+0000 7f6b6267a740 1 Processor -- start 2024-03-26T08:43:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.242+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb1920 0x5651ceec00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.242+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb2310 0x5651ceec0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.242+0000 7f6b6267a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb7480 0x5651cef211f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.242+0000 7f6b6267a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581ad7e0 con 0x5651ceeb2310 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.242+0000 7f6b6267a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581aeb00 con 0x5651ceeb7480 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.242+0000 7f6b6267a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581ade40 con 0x5651ceeb1920 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.242+0000 7f6b6267a740 10 client.0 _ll_register_callbacks cb 0x7f6b500f68e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5e022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb7480 0x5651cef211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d821640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb1920 0x5651ceec00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d020640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb2310 0x5651ceec0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d821640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb1920 0x5651ceec00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57166/0 (socket says 172.21.15.7:57166) 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5e022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb7480 0x5651cef211f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41432/0 (socket says 172.21.15.7:41432) 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d020640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb2310 0x5651ceec0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43366/0 (socket says 172.21.15.7:43366) 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d821640 1 -- 172.21.15.7:0/2896603438 learned_addr learned my addr 172.21.15.7:0/2896603438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d020640 1 -- 172.21.15.7:0/2896603438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb1920 msgr2=0x5651ceec00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d020640 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb1920 0x5651ceec00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d020640 1 -- 172.21.15.7:0/2896603438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb7480 msgr2=0x5651cef211f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d020640 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb7480 0x5651cef211f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d020640 1 -- 172.21.15.7:0/2896603438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651cef217b0 con 0x5651ceeb2310 2024-03-26T08:43:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.243+0000 7f6b5d821640 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb1920 0x5651ceec00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:43:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.244+0000 7f6b5d020640 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb2310 0x5651ceec0620 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5651ceea8600 tx=0x7f6b48010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5123292075d6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.244+0000 7f6b417fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.244+0000 7f6b417fa640 1 -- 172.21.15.7:0/2896603438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48003b20 con 0x5651ceeb2310 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.244+0000 7f6b417fa640 1 -- 172.21.15.7:0/2896603438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6b48003cc0 con 0x5651ceeb2310 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.245+0000 7f6b6267a740 1 -- 172.21.15.7:0/2896603438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b4801fbe0 con 0x5651ceeb2310 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.245+0000 7f6b417fa640 1 -- 172.21.15.7:0/2896603438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b48038050 con 0x5651ceeb2310 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.245+0000 7f6b6267a740 10 client.15768 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.245+0000 7f6b6267a740 1 -- 172.21.15.7:0/2896603438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5651cef22ad0 con 0x5651ceeb2310 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.246+0000 7f6b417fa640 1 -- 172.21.15.7:0/2896603438 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b48010d70 con 0x5651ceeb2310 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.246+0000 7f6b417fa640 1 -- 172.21.15.7:0/2896603438 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f6b480047d0 con 0x5651ceeb2310 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.246+0000 7f6b6267a740 10 client.15768 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.246+0000 7f6b6267a740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.246+0000 7f6b6267a740 20 client.15768 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.246+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5651cef23390 0x5651cef257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.246+0000 7f6b6267a740 4 client.15768 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:43:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.246+0000 7f6b6267a740 1 -- 172.21.15.7:0/2896603438 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f6b4801fbe0 con 0x5651cef23390 2024-03-26T08:43:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.246+0000 7f6b5d821640 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5651cef23390 0x5651cef257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.247+0000 7f6b5d821640 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5651cef23390 0x5651cef257b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84cef4404ad1ae45 server_cookie=b436d13c24ecf563 in_seq=0 out_seq=0 2024-03-26T08:43:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.247+0000 7f6b417fa640 10 client.15768 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b417fa640 1 -- 172.21.15.7:0/2896603438 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f6b4801fbe0 con 0x5651cef23390 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b417fa640 10 client.15768 handle_command_reply: tid=0 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 client.15768 shutdown 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 -- 172.21.15.7:0/2896603438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb2310 msgr2=0x5651ceec0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb2310 0x5651ceec0620 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5651ceea8600 tx=0x7f6b48010660 comp rx=0 tx=0).stop 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 -- 172.21.15.7:0/2896603438 shutdown_connections 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5651cef23390 0x5651cef257b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5651ceeb7480 0x5651cef211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5651ceeb2310 0x5651ceec0620 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 --2- 172.21.15.7:0/2896603438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5651ceeb1920 0x5651ceec00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b417fa640 0 client.15768 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 -- 172.21.15.7:0/2896603438 >> 172.21.15.7:0/2896603438 conn(0x5651ceeace90 msgr2=0x5651ceebb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.267+0000 7f6b6267a740 1 -- 172.21.15.7:0/2896603438 shutdown_connections 2024-03-26T08:43:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b6267a740 1 -- 172.21.15.7:0/2896603438 wait complete. 2024-03-26T08:43:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b6267a740 20 client.15768 trim_cache size 0 max 16384 2024-03-26T08:43:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:43:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:43:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:43:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:43:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6b440a28f0 msgr2=0x7f6b440a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6b440a28f0 0x7f6b440a4db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5651ceeb2b70 tx=0x7f6b54005f90 comp rx=0 tx=0).stop 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 msgr2=0x7f6b5819e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 0x7f6b5819e030 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b500325f0 tx=0x7f6b50032ec0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 shutdown_connections 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6b440a28f0 0x7f6b440a4db0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b58105260 0x7f6b581a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b581028b0 0x7f6b5819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 --2- 172.21.15.7:0/2301584419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b580fff40 0x7f6b5819e030 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 >> 172.21.15.7:0/2301584419 conn(0x7f6b580fbb40 msgr2=0x7f6b58103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 shutdown_connections 2024-03-26T08:43:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:43:48.268+0000 7f6b5fa85640 1 -- 172.21.15.7:0/2301584419 wait complete. 2024-03-26T08:43:48.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:48 smithi007 ceph-mon[32462]: pgmap v401: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 885 op/s 2024-03-26T08:43:48.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:48 smithi044 ceph-mon[36461]: pgmap v401: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 885 op/s 2024-03-26T08:43:48.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:48 smithi016 ceph-mon[35851]: pgmap v401: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 MiB/s wr, 885 op/s 2024-03-26T08:43:50.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:50 smithi007 ceph-mon[32462]: pgmap v402: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 MiB/s wr, 832 op/s 2024-03-26T08:43:50.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:50 smithi044 ceph-mon[36461]: pgmap v402: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 MiB/s wr, 832 op/s 2024-03-26T08:43:50.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:50 smithi016 ceph-mon[35851]: pgmap v402: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 MiB/s wr, 832 op/s 2024-03-26T08:43:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 80.0 s, 4373.4 tps, lat 22.873 ms stddev 24.546 2024-03-26T08:43:52.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:43:52.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:43:52.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/4118203661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6a41028a0 msgr2=0x7ff6a4102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/4118203661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6a41028a0 0x7ff6a4102d20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff694009900 tx=0x7ff69402bc30 comp rx=0 tx=0).stop 2024-03-26T08:43:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/4118203661 shutdown_connections 2024-03-26T08:43:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/4118203661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a4103260 0x7ff6a4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/4118203661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6a41028a0 0x7ff6a4102d20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/4118203661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41016b0 0x7ff6a4101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/4118203661 >> 172.21.15.7:0/4118203661 conn(0x7ff6a40fcc20 msgr2=0x7ff6a40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/4118203661 shutdown_connections 2024-03-26T08:43:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/4118203661 wait complete. 2024-03-26T08:43:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.246+0000 7ff6a9ac6640 1 Processor -- start 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a9ac6640 1 -- start start 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a9ac6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41016b0 0x7ff6a4075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a9ac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a41028a0 0x7ff6a4073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a9ac6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6a4103260 0x7ff6a4073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a9ac6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a410d360 con 0x7ff6a4103260 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a9ac6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a410d1e0 con 0x7ff6a41028a0 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a9ac6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a410d4e0 con 0x7ff6a41016b0 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41016b0 0x7ff6a4075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41016b0 0x7ff6a4075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57184/0 (socket says 172.21.15.7:57184) 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a41028a0 0x7ff6a4073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a37fe640 1 -- 172.21.15.7:0/3914888135 learned_addr learned my addr 172.21.15.7:0/3914888135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.247+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6a4103260 0x7ff6a4073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/3914888135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41016b0 msgr2=0x7ff6a4075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41016b0 0x7ff6a4075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6a4103260 msgr2=0x7ff6a4073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6a4103260 0x7ff6a4073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/3914888135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a419b930 con 0x7ff6a41028a0 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a41028a0 0x7ff6a4073a90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff694036040 tx=0x7ff69402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a0ff9640 1 -- 172.21.15.7:0/3914888135 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694043020 con 0x7ff6a41028a0 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a0ff9640 1 -- 172.21.15.7:0/3914888135 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6940054e0 con 0x7ff6a41028a0 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/3914888135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a419bbc0 con 0x7ff6a41028a0 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a0ff9640 1 -- 172.21.15.7:0/3914888135 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694003c10 con 0x7ff6a41028a0 2024-03-26T08:43:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.248+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/3914888135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a419c0e0 con 0x7ff6a41028a0 2024-03-26T08:43:52.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.250+0000 7ff6a0ff9640 1 -- 172.21.15.7:0/3914888135 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff694046020 con 0x7ff6a41028a0 2024-03-26T08:43:52.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.251+0000 7ff6ac78f740 1 Processor -- start 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.251+0000 7ff6ac78f740 1 -- start start 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.251+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 0x56346a03d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.251+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a042290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.251+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 0x56346a044d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.251+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563469fa23d0 con 0x56346a03d790 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.251+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563469f4aad0 con 0x56346a03cda0 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.251+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563469f9e470 con 0x56346a042900 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a042290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.251+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 0x56346a03d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a042290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43386/0 (socket says 172.21.15.7:43386) 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a37fe640 1 -- 172.21.15.7:0/2755948878 learned_addr learned my addr 172.21.15.7:0/2755948878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:52.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/2755948878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 0x56346a044d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a3fff640 1 -- 172.21.15.7:0/2755948878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 msgr2=0x56346a044d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/2755948878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 0x56346a044d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a0ff9640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff68c0a2800 0x7ff68c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a3fff640 1 -- 172.21.15.7:0/2755948878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 msgr2=0x56346a042290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/2755948878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a042290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a3fff640 1 -- 172.21.15.7:0/2755948878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563469d807d0 con 0x56346a03cda0 2024-03-26T08:43:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a0ff9640 1 -- 172.21.15.7:0/3914888135 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff6940f0a20 con 0x7ff6a41028a0 2024-03-26T08:43:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/2755948878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 0x56346a03d1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009f00 tx=0x7ff69c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc9b097afaae10eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.252+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff68c0a2800 0x7ff68c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.253+0000 7ff689ffb640 1 -- 172.21.15.7:0/2755948878 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c003a90 con 0x56346a03cda0 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.253+0000 7ff689ffb640 1 -- 172.21.15.7:0/2755948878 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff69c003c30 con 0x56346a03cda0 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.253+0000 7ff689ffb640 1 -- 172.21.15.7:0/2755948878 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00fc00 con 0x56346a03cda0 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.253+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2755948878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 msgr2=0x56346a03d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.253+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/2755948878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 0x56346a03d1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009f00 tx=0x7ff69c011810 comp rx=0 tx=0).stop 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.253+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff68c0a2800 0x7ff68c0a4cc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0049d0 tx=0x7ff69c012f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2755948878 shutdown_connections 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/2755948878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 0x56346a044d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/2755948878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a042290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/2755948878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 0x56346a03d1c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2755948878 >> 172.21.15.7:0/2755948878 conn(0x56346a038310 msgr2=0x56346a038710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2755948878 shutdown_connections 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2755948878 wait complete. 2024-03-26T08:43:52.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 -- start start 2024-03-26T08:43:52.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 Processor -- start 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 0x56346a0471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a047730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a042900 0x56346a04e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56346a0458b0 con 0x56346a03d790 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563469f5e060 con 0x56346a042900 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56346a045a30 con 0x56346a03cda0 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6ac78f740 10 client.0 _ll_register_callbacks cb 0x563469f406a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 0x56346a0471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.254+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a042900 0x56346a04e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 0x56346a0471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57206/0 (socket says 172.21.15.7:57206) 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a042900 0x56346a04e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41488/0 (socket says 172.21.15.7:41488) 2024-03-26T08:43:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a37fe640 1 -- 172.21.15.7:0/1390311741 learned_addr learned my addr 172.21.15.7:0/1390311741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a047730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a3fff640 1 -- 172.21.15.7:0/1390311741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 msgr2=0x56346a0471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 0x56346a0471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a3fff640 1 -- 172.21.15.7:0/1390311741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 msgr2=0x56346a047730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a047730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a3fff640 1 -- 172.21.15.7:0/1390311741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563469cb63f0 con 0x56346a042900 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.255+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a042900 0x56346a04e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009f00 tx=0x7ff69c018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf8f34b52760eea6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.256+0000 7ff682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.256+0000 7ff682ffd640 1 -- 172.21.15.7:0/1390311741 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c036060 con 0x56346a042900 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.256+0000 7ff6ac78f740 1 -- 172.21.15.7:0/1390311741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff69c003c30 con 0x56346a042900 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.256+0000 7ff682ffd640 1 -- 172.21.15.7:0/1390311741 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff69c004c10 con 0x56346a042900 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.256+0000 7ff682ffd640 1 -- 172.21.15.7:0/1390311741 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff69c004e10 con 0x56346a042900 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.256+0000 7ff6ac78f740 10 client.25622 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.256+0000 7ff6ac78f740 1 -- 172.21.15.7:0/1390311741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563469c760f0 con 0x56346a042900 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff682ffd640 1 -- 172.21.15.7:0/1390311741 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff69c003c30 con 0x56346a042900 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff682ffd640 1 -- 172.21.15.7:0/1390311741 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff69c03b070 con 0x56346a042900 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff6ac78f740 10 client.25622 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff6ac78f740 10 client.25622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff6ac78f740 20 client.25622 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56346a053030 0x56346a055490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff6ac78f740 4 client.25622 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff6ac78f740 1 -- 172.21.15.7:0/1390311741 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563469faf910 con 0x56346a053030 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff6a37fe640 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56346a053030 0x56346a055490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff682ffd640 10 client.25622 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.257+0000 7ff6a37fe640 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56346a053030 0x56346a055490 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d3320b1fa9697a server_cookie=73fa15e7ff81cc45 in_seq=0 out_seq=0 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff682ffd640 1 -- 172.21.15.7:0/1390311741 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563469faf910 con 0x56346a053030 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff682ffd640 10 client.25622 handle_command_reply: tid=0 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 client.25622 shutdown 2024-03-26T08:43:52.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 -- 172.21.15.7:0/1390311741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a042900 msgr2=0x56346a04e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a042900 0x56346a04e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009f00 tx=0x7ff69c018000 comp rx=0 tx=0).stop 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 -- 172.21.15.7:0/1390311741 shutdown_connections 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56346a053030 0x56346a055490 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a042900 0x56346a04e770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a047730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/1390311741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 0x56346a0471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 -- 172.21.15.7:0/1390311741 >> 172.21.15.7:0/1390311741 conn(0x56346a038310 msgr2=0x56346a037530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff682ffd640 0 client.25622 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 -- 172.21.15.7:0/1390311741 shutdown_connections 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.258+0000 7ff6ac78f740 1 -- 172.21.15.7:0/1390311741 wait complete. 2024-03-26T08:43:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.259+0000 7ff6ac78f740 20 client.25622 trim_cache size 0 max 16384 2024-03-26T08:43:52.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6ac78f740 1 Processor -- start 2024-03-26T08:43:52.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6ac78f740 1 -- start start 2024-03-26T08:43:52.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 0x56346a0471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03d790 0x56346a04e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a042900 0x56346a047730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41b9db0 con 0x56346a042900 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41b78b0 con 0x56346a03d790 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41a29f0 con 0x56346a03cda0 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a042900 0x56346a047730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6a2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03d790 0x56346a04e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6a37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 0x56346a0471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a042900 0x56346a047730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43402/0 (socket says 172.21.15.7:43402) 2024-03-26T08:43:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6a2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03d790 0x56346a04e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41490/0 (socket says 172.21.15.7:41490) 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.281+0000 7ff6a3fff640 1 -- 172.21.15.7:0/2645304677 learned_addr learned my addr 172.21.15.7:0/2645304677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.282+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/2645304677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 msgr2=0x56346a0471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.282+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/2645304677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 0x56346a0471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.282+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/2645304677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a042900 msgr2=0x56346a047730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.282+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/2645304677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a042900 0x56346a047730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.282+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/2645304677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56346a039080 con 0x56346a03d790 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.282+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/2645304677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a042900 0x56346a047730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.282+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/2645304677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03d790 0x56346a04e730 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff6940f3fd0 tx=0x7ff6940b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52b5b494e3efcd82 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.282+0000 7ff6897fa640 1 -- 172.21.15.7:0/2645304677 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6940bd190 con 0x56346a03d790 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.282+0000 7ff6897fa640 1 -- 172.21.15.7:0/2645304677 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6940f0e50 con 0x56346a03d790 2024-03-26T08:43:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.283+0000 7ff6897fa640 1 -- 172.21.15.7:0/2645304677 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6940bcd20 con 0x56346a03d790 2024-03-26T08:43:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2645304677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03d790 msgr2=0x56346a04e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/2645304677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03d790 0x56346a04e730 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff6940f3fd0 tx=0x7ff6940b9040 comp rx=0 tx=0).stop 2024-03-26T08:43:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2645304677 shutdown_connections 2024-03-26T08:43:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/2645304677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a042900 0x56346a047730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/2645304677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03d790 0x56346a04e730 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/2645304677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a03cda0 0x56346a0471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2645304677 >> 172.21.15.7:0/2645304677 conn(0x56346a038310 msgr2=0x56346a0386f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2645304677 shutdown_connections 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 -- 172.21.15.7:0/2645304677 wait complete. 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 -- start start 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 Processor -- start 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 0x56346a04b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a04bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.284+0000 7ff6ac78f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 0x56346a0ac950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41a2ff0 con 0x56346a03d790 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41b7250 con 0x56346a03cda0 2024-03-26T08:43:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6ac78f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a41a3630 con 0x56346a042900 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6ac78f740 10 client.0 _ll_register_callbacks cb 0x7ff698009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a04bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a04bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43416/0 (socket says 172.21.15.7:43416) 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/981302885 learned_addr learned my addr 172.21.15.7:0/981302885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a37fe640 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 0x56346a04b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a3fff640 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 0x56346a0ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/981302885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 msgr2=0x56346a0ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 0x56346a0ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/981302885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 msgr2=0x56346a04b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 0x56346a04b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a2ffd640 1 -- 172.21.15.7:0/981302885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56346a0acf10 con 0x56346a03d790 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff6a2ffd640 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a04bd00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff6940b9820 tx=0x7ff6940f6300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0db91aa7b0623fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.285+0000 7ff682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:43:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.286+0000 7ff682ffd640 1 -- 172.21.15.7:0/981302885 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6940d6060 con 0x56346a03d790 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.286+0000 7ff682ffd640 1 -- 172.21.15.7:0/981302885 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff6940d6200 con 0x56346a03d790 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.286+0000 7ff6ac78f740 1 -- 172.21.15.7:0/981302885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6940f0e50 con 0x56346a03d790 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.286+0000 7ff682ffd640 1 -- 172.21.15.7:0/981302885 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6940bcf00 con 0x56346a03d790 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.286+0000 7ff6ac78f740 10 client.15786 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.286+0000 7ff6ac78f740 1 -- 172.21.15.7:0/981302885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56346a0ae220 con 0x56346a03d790 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.286+0000 7ff682ffd640 1 -- 172.21.15.7:0/981302885 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6940f0e50 con 0x56346a03d790 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.287+0000 7ff682ffd640 1 -- 172.21.15.7:0/981302885 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ff6940ee5a0 con 0x56346a03d790 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.287+0000 7ff6ac78f740 10 client.15786 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.287+0000 7ff6ac78f740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.287+0000 7ff6ac78f740 20 client.15786 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.287+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56346a0aeb40 0x56346a0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.287+0000 7ff6ac78f740 4 client.15786 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:43:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.287+0000 7ff6ac78f740 1 -- 172.21.15.7:0/981302885 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563469fa9c60 con 0x56346a0aeb40 2024-03-26T08:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.288+0000 7ff6a37fe640 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56346a0aeb40 0x56346a0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6a37fe640 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56346a0aeb40 0x56346a0b0f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2745bddb8a50df88 server_cookie=87f99e5a8a326cae in_seq=0 out_seq=0 2024-03-26T08:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff682ffd640 10 client.15786 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff682ffd640 1 -- 172.21.15.7:0/981302885 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563469fa9c60 con 0x56346a0aeb40 2024-03-26T08:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff682ffd640 10 client.15786 handle_command_reply: tid=0 2024-03-26T08:43:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 client.15786 shutdown 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 -- 172.21.15.7:0/981302885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 msgr2=0x56346a04bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a04bd00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff6940b9820 tx=0x7ff6940f6300 comp rx=0 tx=0).stop 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 -- 172.21.15.7:0/981302885 shutdown_connections 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56346a0aeb40 0x56346a0b0f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff682ffd640 0 client.15786 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56346a042900 0x56346a0ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56346a03d790 0x56346a04bd00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 --2- 172.21.15.7:0/981302885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56346a03cda0 0x56346a04b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 -- 172.21.15.7:0/981302885 >> 172.21.15.7:0/981302885 conn(0x56346a038310 msgr2=0x56346a03eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 -- 172.21.15.7:0/981302885 shutdown_connections 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 1 -- 172.21.15.7:0/981302885 wait complete. 2024-03-26T08:43:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.289+0000 7ff6ac78f740 20 client.15786 trim_cache size 0 max 16384 2024-03-26T08:43:52.285 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:43:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:43:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:43:52.285 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff68c0a2800 msgr2=0x7ff68c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff68c0a2800 0x7ff68c0a4cc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff69c0049d0 tx=0x7ff69c012f90 comp rx=0 tx=0).stop 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/3914888135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a41028a0 msgr2=0x7ff6a4073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a41028a0 0x7ff6a4073a90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff694036040 tx=0x7ff69402c650 comp rx=0 tx=0).stop 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/3914888135 shutdown_connections 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff68c0a2800 0x7ff68c0a4cc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6a4103260 0x7ff6a4073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6a41028a0 0x7ff6a4073a90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 --2- 172.21.15.7:0/3914888135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6a41016b0 0x7ff6a4075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/3914888135 >> 172.21.15.7:0/3914888135 conn(0x7ff6a40fcc20 msgr2=0x7ff6a4105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/3914888135 shutdown_connections 2024-03-26T08:43:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.291+0000 7ff6a9ac6640 1 -- 172.21.15.7:0/3914888135 wait complete. 2024-03-26T08:43:52.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:52 smithi007 ceph-mon[32462]: pgmap v403: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.0 MiB/s wr, 762 op/s 2024-03-26T08:43:52.295 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:43:52.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:43:52.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:52 smithi044 ceph-mon[36461]: pgmap v403: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.0 MiB/s wr, 762 op/s 2024-03-26T08:43:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 -- 172.21.15.7:0/1224036291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c102200 msgr2=0x7fce9c1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/1224036291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c102200 0x7fce9c1045f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009900 tx=0x7fce8c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:43:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 -- 172.21.15.7:0/1224036291 shutdown_connections 2024-03-26T08:43:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/1224036291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9c107460 0x7fce9c109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/1224036291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce9c104b30 0x7fce9c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/1224036291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c102200 0x7fce9c1045f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 -- 172.21.15.7:0/1224036291 >> 172.21.15.7:0/1224036291 conn(0x7fce9c0fbb30 msgr2=0x7fce9c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 -- 172.21.15.7:0/1224036291 shutdown_connections 2024-03-26T08:43:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 -- 172.21.15.7:0/1224036291 wait complete. 2024-03-26T08:43:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 Processor -- start 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.458+0000 7fcea1ce8640 1 -- start start 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.459+0000 7fcea1ce8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce9c102200 0x7fce9c101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.459+0000 7fcea1ce8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c104b30 0x7fce9c1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.459+0000 7fcea1ce8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9c107460 0x7fce9c1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.459+0000 7fcea1ce8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c10d380 con 0x7fce9c107460 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.459+0000 7fcea1ce8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c10d200 con 0x7fce9c104b30 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.459+0000 7fcea1ce8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c10d500 con 0x7fce9c102200 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.459+0000 7fce9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9c107460 0x7fce9c1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.459+0000 7fce9bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9c107460 0x7fce9c1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43440/0 (socket says 172.21.15.7:43440) 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.459+0000 7fce9bfff640 1 -- 172.21.15.7:0/989976956 learned_addr learned my addr 172.21.15.7:0/989976956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.460+0000 7fce9affd640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c104b30 0x7fce9c1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.460+0000 7fce9bfff640 1 -- 172.21.15.7:0/989976956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce9c102200 msgr2=0x7fce9c101b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:43:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.460+0000 7fce9bfff640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce9c102200 0x7fce9c101b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.460+0000 7fce9bfff640 1 -- 172.21.15.7:0/989976956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c104b30 msgr2=0x7fce9c1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.460+0000 7fce9bfff640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c104b30 0x7fce9c1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.460+0000 7fce9bfff640 1 -- 172.21.15.7:0/989976956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c100f80 con 0x7fce9c107460 2024-03-26T08:43:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.460+0000 7fce9bfff640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9c107460 0x7fce9c1006f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fce9400efd0 tx=0x7fce9400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.460+0000 7fce98ff9640 1 -- 172.21.15.7:0/989976956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94003b70 con 0x7fce9c107460 2024-03-26T08:43:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.460+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9c1a1b70 con 0x7fce9c107460 2024-03-26T08:43:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.461+0000 7fce98ff9640 1 -- 172.21.15.7:0/989976956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce94003d10 con 0x7fce9c107460 2024-03-26T08:43:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.461+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c1a20c0 con 0x7fce9c107460 2024-03-26T08:43:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.461+0000 7fce98ff9640 1 -- 172.21.15.7:0/989976956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940109f0 con 0x7fce9c107460 2024-03-26T08:43:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.462+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce880052c0 con 0x7fce9c107460 2024-03-26T08:43:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.462+0000 7fce98ff9640 1 -- 172.21.15.7:0/989976956 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fce94010b90 con 0x7fce9c107460 2024-03-26T08:43:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.463+0000 7fce98ff9640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fce840a2880 0x7fce840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.463+0000 7fce98ff9640 1 -- 172.21.15.7:0/989976956 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fce940cee70 con 0x7fce9c107460 2024-03-26T08:43:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.464+0000 7fce9b7fe640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fce840a2880 0x7fce840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.465+0000 7fce9b7fe640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fce840a2880 0x7fce840a4d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fce8c005c20 tx=0x7fce8c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.470+0000 7fce98ff9640 1 -- 172.21.15.7:0/989976956 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce9409bf10 con 0x7fce9c107460 2024-03-26T08:43:52.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:52 smithi016 ceph-mon[35851]: pgmap v403: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9.0 MiB/s wr, 762 op/s 2024-03-26T08:43:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.828+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce880055b0 con 0x7fce9c107460 2024-03-26T08:43:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.830+0000 7fce98ff9640 1 -- 172.21.15.7:0/989976956 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11358 (secure 0 0 0) 0x7fce9409b810 con 0x7fce9c107460 2024-03-26T08:43:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:43:52.821 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:42:21.061940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24502,24512,24478]},"id":1}]} 2024-03-26T08:43:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-26T08:43:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fce840a2880 msgr2=0x7fce840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fce840a2880 0x7fce840a4d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fce8c005c20 tx=0x7fce8c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9c107460 msgr2=0x7fce9c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9c107460 0x7fce9c1006f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fce9400efd0 tx=0x7fce9400c6a0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 shutdown_connections 2024-03-26T08:43:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fce840a2880 0x7fce840a4d40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9c107460 0x7fce9c1006f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce9c104b30 0x7fce9c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 --2- 172.21.15.7:0/989976956 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fce9c102200 0x7fce9c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 >> 172.21.15.7:0/989976956 conn(0x7fce9c0fbb30 msgr2=0x7fce9c102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 shutdown_connections 2024-03-26T08:43:52.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:52.832+0000 7fcea1ce8640 1 -- 172.21.15.7:0/989976956 wait complete. 2024-03-26T08:43:52.840 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:43:52.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:43:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.010+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/565390882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e4102170 msgr2=0x7fc7e4108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.010+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/565390882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e4102170 0x7fc7e4108a00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009be0 tx=0x7fc7dc02bef0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.010+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/565390882 shutdown_connections 2024-03-26T08:43:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.010+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/565390882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e4102170 0x7fc7e4108a00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.010+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/565390882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e41017b0 0x7fc7e4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.010+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/565390882 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e41005c0 0x7fc7e41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.010+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/565390882 >> 172.21.15.7:0/565390882 conn(0x7fc7e40fbb30 msgr2=0x7fc7e40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.010+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/565390882 shutdown_connections 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.010+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/565390882 wait complete. 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7eaf6f640 1 Processor -- start 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7eaf6f640 1 -- start start 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7eaf6f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e41005c0 0x7fc7e4197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7eaf6f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e41017b0 0x7fc7e4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7eaf6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e4102170 0x7fc7e419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7eaf6f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e410d3a0 con 0x7fc7e41017b0 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7eaf6f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e410d220 con 0x7fc7e4102170 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7eaf6f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e410d520 con 0x7fc7e41005c0 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e41017b0 0x7fc7e4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7e8d0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e41005c0 0x7fc7e4197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7e950c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e4102170 0x7fc7e419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e41017b0 0x7fc7e4197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43450/0 (socket says 172.21.15.7:43450) 2024-03-26T08:43:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.011+0000 7fc7e3fff640 1 -- 172.21.15.7:0/2203558917 learned_addr learned my addr 172.21.15.7:0/2203558917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.012+0000 7fc7e950c640 1 -- 172.21.15.7:0/2203558917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e41005c0 msgr2=0x7fc7e4197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.012+0000 7fc7e950c640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e41005c0 0x7fc7e4197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.012+0000 7fc7e950c640 1 -- 172.21.15.7:0/2203558917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e41017b0 msgr2=0x7fc7e4197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.012+0000 7fc7e950c640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e41017b0 0x7fc7e4197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.012+0000 7fc7e950c640 1 -- 172.21.15.7:0/2203558917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e419f580 con 0x7fc7e4102170 2024-03-26T08:43:53.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.012+0000 7fc7e950c640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e4102170 0x7fc7e419ef10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc02c430 tx=0x7fc7dc0337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.012+0000 7fc7e1ffb640 1 -- 172.21.15.7:0/2203558917 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc043020 con 0x7fc7e4102170 2024-03-26T08:43:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.013+0000 7fc7e1ffb640 1 -- 172.21.15.7:0/2203558917 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7dc02ca60 con 0x7fc7e4102170 2024-03-26T08:43:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.013+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/2203558917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e419f810 con 0x7fc7e4102170 2024-03-26T08:43:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.013+0000 7fc7e1ffb640 1 -- 172.21.15.7:0/2203558917 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc034690 con 0x7fc7e4102170 2024-03-26T08:43:53.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.013+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/2203558917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e419fd60 con 0x7fc7e4102170 2024-03-26T08:43:53.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.016+0000 7fc7e1ffb640 1 -- 172.21.15.7:0/2203558917 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc7dc046020 con 0x7fc7e4102170 2024-03-26T08:43:53.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7edc30740 1 Processor -- start 2024-03-26T08:43:53.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7edc30740 1 -- start start 2024-03-26T08:43:53.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203befab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 0x55b203bf4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 0x55b203bf7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7edc30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b203bb2610 con 0x55b203bf51f0 2024-03-26T08:43:53.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7edc30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b203b553c0 con 0x55b203bf0080 2024-03-26T08:43:53.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7edc30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b203afdd60 con 0x55b203bef690 2024-03-26T08:43:53.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203befab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203befab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57292/0 (socket says 172.21.15.7:57292) 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7e3fff640 1 -- 172.21.15.7:0/1037926254 learned_addr learned my addr 172.21.15.7:0/1037926254 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 0x55b203bf4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 0x55b203bf7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7e1ffb640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a2950 0x7fc7d40a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.017+0000 7fc7e3fff640 1 -- 172.21.15.7:0/1037926254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 msgr2=0x55b203bf4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 0x55b203bf4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7e3fff640 1 -- 172.21.15.7:0/1037926254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 msgr2=0x55b203bf7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 0x55b203bf7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7e3fff640 1 -- 172.21.15.7:0/1037926254 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2038693f0 con 0x55b203bef690 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 0x55b203bf4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7e950c640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a2950 0x7fc7d40a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203befab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00118a0 tx=0x7fc7d00095d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753d63024d6aa6ba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7e1ffb640 1 -- 172.21.15.7:0/2203558917 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7dc0f0ef0 con 0x7fc7e4102170 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7ceffd640 1 -- 172.21.15.7:0/1037926254 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0004030 con 0x55b203bef690 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7ceffd640 1 -- 172.21.15.7:0/1037926254 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7d0005500 con 0x55b203bef690 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.018+0000 7fc7ceffd640 1 -- 172.21.15.7:0/1037926254 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0004bc0 con 0x55b203bef690 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 -- 172.21.15.7:0/1037926254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 msgr2=0x55b203befab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7e950c640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a2950 0x7fc7d40a4e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc041000 tx=0x7fc7dc031810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203befab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00118a0 tx=0x7fc7d00095d0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 -- 172.21.15.7:0/1037926254 shutdown_connections 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 0x55b203bf7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 0x55b203bf4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1037926254 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203befab0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 -- 172.21.15.7:0/1037926254 >> 172.21.15.7:0/1037926254 conn(0x55b203beac00 msgr2=0x55b203beb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 -- 172.21.15.7:0/1037926254 shutdown_connections 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 -- 172.21.15.7:0/1037926254 wait complete. 2024-03-26T08:43:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 -- start start 2024-03-26T08:43:53.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.019+0000 7fc7edc30740 1 Processor -- start 2024-03-26T08:43:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.020+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203bf9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.020+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 0x55b203bf9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.020+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 0x55b203c01030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.020+0000 7fc7edc30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b203b11240 con 0x55b203bf51f0 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.020+0000 7fc7edc30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b203b51460 con 0x55b203bf0080 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.020+0000 7fc7edc30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b203bf81a0 con 0x55b203bef690 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.020+0000 7fc7edc30740 10 client.0 _ll_register_callbacks cb 0x55b203b10c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.021+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203bf9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.021+0000 7fc7e3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203bf9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57304/0 (socket says 172.21.15.7:57304) 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.021+0000 7fc7e3fff640 1 -- 172.21.15.7:0/3844698576 learned_addr learned my addr 172.21.15.7:0/3844698576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.021+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 0x55b203c01030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.021+0000 7fc7e3fff640 1 -- 172.21.15.7:0/3844698576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 msgr2=0x55b203bf9ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.021+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 0x55b203bf9ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.021+0000 7fc7e3fff640 1 -- 172.21.15.7:0/3844698576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 msgr2=0x55b203c01030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.021+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 0x55b203c01030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.021+0000 7fc7e3fff640 1 -- 172.21.15.7:0/3844698576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2039337d0 con 0x55b203bef690 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.022+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203bf9ab0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0004d30 tx=0x7fc7d0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61152cebe6548e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.022+0000 7fc7c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.022+0000 7fc7c3fff640 1 -- 172.21.15.7:0/3844698576 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0029040 con 0x55b203bef690 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.022+0000 7fc7edc30740 1 -- 172.21.15.7:0/3844698576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7d0005500 con 0x55b203bef690 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.022+0000 7fc7c3fff640 1 -- 172.21.15.7:0/3844698576 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7d000f790 con 0x55b203bef690 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.022+0000 7fc7c3fff640 1 -- 172.21.15.7:0/3844698576 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7d00056b0 con 0x55b203bef690 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.022+0000 7fc7edc30740 10 client.25654 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.022+0000 7fc7edc30740 1 -- 172.21.15.7:0/3844698576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b203c043a0 con 0x55b203bef690 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.022+0000 7fc7c3fff640 1 -- 172.21.15.7:0/3844698576 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7d0005500 con 0x55b203bef690 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.023+0000 7fc7c3fff640 1 -- 172.21.15.7:0/3844698576 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc7d001fae0 con 0x55b203bef690 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.023+0000 7fc7edc30740 10 client.25654 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.023+0000 7fc7edc30740 10 client.25654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.023+0000 7fc7edc30740 20 client.25654 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.023+0000 7fc7edc30740 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b203c05cb0 0x55b203c080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.023+0000 7fc7edc30740 4 client.25654 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.023+0000 7fc7edc30740 1 -- 172.21.15.7:0/3844698576 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b203b62900 con 0x55b203c05cb0 2024-03-26T08:43:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.023+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b203c05cb0 0x55b203c080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.024+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b203c05cb0 0x55b203c080f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e8849ee113cd89a server_cookie=843250f8f5fd261f in_seq=0 out_seq=0 2024-03-26T08:43:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.024+0000 7fc7c3fff640 10 client.25654 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:43:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.024+0000 7fc7c3fff640 1 -- 172.21.15.7:0/3844698576 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b203b62900 con 0x55b203c05cb0 2024-03-26T08:43:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.024+0000 7fc7c3fff640 10 client.25654 handle_command_reply: tid=0 2024-03-26T08:43:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.024+0000 7fc7edc30740 1 client.25654 shutdown 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.025+0000 7fc7edc30740 1 -- 172.21.15.7:0/3844698576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 msgr2=0x55b203bf9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.025+0000 7fc7edc30740 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203bf9ab0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0004d30 tx=0x7fc7d0009840 comp rx=0 tx=0).stop 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7edc30740 1 -- 172.21.15.7:0/3844698576 shutdown_connections 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7edc30740 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b203c05cb0 0x55b203c080f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7edc30740 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bf51f0 0x55b203c01030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7edc30740 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf0080 0x55b203bf9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7edc30740 1 --2- 172.21.15.7:0/3844698576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bef690 0x55b203bf9ab0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7edc30740 1 -- 172.21.15.7:0/3844698576 >> 172.21.15.7:0/3844698576 conn(0x55b203beac00 msgr2=0x55b203be9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7c3fff640 0 client.25654 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7edc30740 1 -- 172.21.15.7:0/3844698576 shutdown_connections 2024-03-26T08:43:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7edc30740 1 -- 172.21.15.7:0/3844698576 wait complete. 2024-03-26T08:43:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.026+0000 7fc7edc30740 20 client.25654 trim_cache size 0 max 16384 2024-03-26T08:43:53.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.048+0000 7fc7edc30740 1 Processor -- start 2024-03-26T08:43:53.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7edc30740 1 -- start start 2024-03-26T08:43:53.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bef690 0x55b203bf9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203c00fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203be9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7edc30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41a8dd0 con 0x55b203bef690 2024-03-26T08:43:53.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7edc30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41a68d0 con 0x55b203bf51f0 2024-03-26T08:43:53.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7edc30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41a4950 con 0x55b203bf0080 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7e8d0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203c00fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7e8d0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203c00fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57316/0 (socket says 172.21.15.7:57316) 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.049+0000 7fc7e8d0b640 1 -- 172.21.15.7:0/4186748944 learned_addr learned my addr 172.21.15.7:0/4186748944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bef690 0x55b203bf9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203be9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7e8d0b640 1 -- 172.21.15.7:0/4186748944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 msgr2=0x55b203be9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203be9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7e8d0b640 1 -- 172.21.15.7:0/4186748944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bef690 msgr2=0x55b203bf9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bef690 0x55b203bf9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7e8d0b640 1 -- 172.21.15.7:0/4186748944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b203bf48b0 con 0x55b203bf0080 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203be9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:43:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203c00fc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55b203bf0af0 tx=0x7fc7d801da50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffd4269083133178 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7ce7fc640 1 -- 172.21.15.7:0/4186748944 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d801d220 con 0x55b203bf0080 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7ce7fc640 1 -- 172.21.15.7:0/4186748944 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7d80035f0 con 0x55b203bf0080 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.050+0000 7fc7ce7fc640 1 -- 172.21.15.7:0/4186748944 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8010690 con 0x55b203bf0080 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.052+0000 7fc7edc30740 1 -- 172.21.15.7:0/4186748944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 msgr2=0x55b203c00fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.052+0000 7fc7edc30740 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203c00fc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55b203bf0af0 tx=0x7fc7d801da50 comp rx=0 tx=0).stop 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.052+0000 7fc7edc30740 1 -- 172.21.15.7:0/4186748944 shutdown_connections 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.052+0000 7fc7edc30740 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203be9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.052+0000 7fc7edc30740 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203c00fc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.052+0000 7fc7edc30740 1 --2- 172.21.15.7:0/4186748944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bef690 0x55b203bf9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.052+0000 7fc7edc30740 1 -- 172.21.15.7:0/4186748944 >> 172.21.15.7:0/4186748944 conn(0x55b203beac00 msgr2=0x55b203bf96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.052+0000 7fc7edc30740 1 -- 172.21.15.7:0/4186748944 shutdown_connections 2024-03-26T08:43:53.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.052+0000 7fc7edc30740 1 -- 172.21.15.7:0/4186748944 wait complete. 2024-03-26T08:43:53.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.053+0000 7fc7edc30740 1 -- start start 2024-03-26T08:43:53.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.053+0000 7fc7edc30740 1 Processor -- start 2024-03-26T08:43:53.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bef690 0x55b203bfdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203bfe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7edc30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203c5f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7edc30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41a4f50 con 0x55b203bef690 2024-03-26T08:43:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7edc30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41a6270 con 0x55b203bf51f0 2024-03-26T08:43:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7edc30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e41a55b0 con 0x55b203bf0080 2024-03-26T08:43:53.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7edc30740 10 client.0 _ll_register_callbacks cb 0x7fc7d00117a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7e8d0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203bfe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7e8d0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203bfe040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57318/0 (socket says 172.21.15.7:57318) 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7e8d0b640 1 -- 172.21.15.7:0/1659483378 learned_addr learned my addr 172.21.15.7:0/1659483378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.054+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203c5f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.055+0000 7fc7e8d0b640 1 -- 172.21.15.7:0/1659483378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 msgr2=0x55b203c5f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.055+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203c5f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.055+0000 7fc7e8d0b640 1 -- 172.21.15.7:0/1659483378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bef690 msgr2=0x55b203bfdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.055+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bef690 0x55b203bfdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.055+0000 7fc7e8d0b640 1 -- 172.21.15.7:0/1659483378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b203c01110 con 0x55b203bf0080 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.055+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203c5f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:53.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.056+0000 7fc7e8d0b640 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203bfe040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8003f40 tx=0x7fc7d8009990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2663425de61300 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.056+0000 7fc7c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:43:53.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.056+0000 7fc7c3fff640 1 -- 172.21.15.7:0/1659483378 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d802e030 con 0x55b203bf0080 2024-03-26T08:43:53.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.056+0000 7fc7c3fff640 1 -- 172.21.15.7:0/1659483378 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7d801d020 con 0x55b203bf0080 2024-03-26T08:43:53.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.056+0000 7fc7edc30740 1 -- 172.21.15.7:0/1659483378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7d80035f0 con 0x55b203bf0080 2024-03-26T08:43:53.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.056+0000 7fc7c3fff640 1 -- 172.21.15.7:0/1659483378 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7d801d220 con 0x55b203bf0080 2024-03-26T08:43:53.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7edc30740 10 client.25660 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:53.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7edc30740 1 -- 172.21.15.7:0/1659483378 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b203c60a30 con 0x55b203bf0080 2024-03-26T08:43:53.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7c3fff640 1 -- 172.21.15.7:0/1659483378 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7d80035f0 con 0x55b203bf0080 2024-03-26T08:43:53.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7c3fff640 1 -- 172.21.15.7:0/1659483378 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc7d80053a0 con 0x55b203bf0080 2024-03-26T08:43:53.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7edc30740 10 client.25660 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:53.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7edc30740 10 client.25660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:43:53.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7edc30740 20 client.25660 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:53.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b203c613b0 0x55b203c637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7edc30740 4 client.25660 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:43:53.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7edc30740 1 -- 172.21.15.7:0/1659483378 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b203b5cc50 con 0x55b203c613b0 2024-03-26T08:43:53.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.057+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b203c613b0 0x55b203c637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.058+0000 7fc7c3fff640 10 client.25660 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:43:53.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.058+0000 7fc7e3fff640 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b203c613b0 0x55b203c637d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27c779c192f7e289 server_cookie=95733b352a9de2cf in_seq=0 out_seq=0 2024-03-26T08:43:53.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.058+0000 7fc7c3fff640 1 -- 172.21.15.7:0/1659483378 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b203b5cc50 con 0x55b203c613b0 2024-03-26T08:43:53.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.058+0000 7fc7c3fff640 10 client.25660 handle_command_reply: tid=0 2024-03-26T08:43:53.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.058+0000 7fc7edc30740 1 client.25660 shutdown 2024-03-26T08:43:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.058+0000 7fc7edc30740 1 -- 172.21.15.7:0/1659483378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 msgr2=0x55b203bfe040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.058+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203bfe040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8003f40 tx=0x7fc7d8009990 comp rx=0 tx=0).stop 2024-03-26T08:43:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.059+0000 7fc7edc30740 1 -- 172.21.15.7:0/1659483378 shutdown_connections 2024-03-26T08:43:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.059+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55b203c613b0 0x55b203c637d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.059+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b203bf51f0 0x55b203c5f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.059+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b203bf0080 0x55b203bfe040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.059+0000 7fc7edc30740 1 --2- 172.21.15.7:0/1659483378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b203bef690 0x55b203bfdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.059+0000 7fc7edc30740 1 -- 172.21.15.7:0/1659483378 >> 172.21.15.7:0/1659483378 conn(0x55b203beac00 msgr2=0x55b203bede60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.059+0000 7fc7c3fff640 0 client.25660 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:43:53.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.060+0000 7fc7edc30740 1 -- 172.21.15.7:0/1659483378 shutdown_connections 2024-03-26T08:43:53.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.061+0000 7fc7edc30740 1 -- 172.21.15.7:0/1659483378 wait complete. 2024-03-26T08:43:53.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.061+0000 7fc7edc30740 20 client.25660 trim_cache size 0 max 16384 2024-03-26T08:43:53.053 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:43:53.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.062+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/2203558917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a2950 msgr2=0x7fc7d40a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.062+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a2950 0x7fc7d40a4e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc041000 tx=0x7fc7dc031810 comp rx=0 tx=0).stop 2024-03-26T08:43:53.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.062+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/2203558917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e4102170 msgr2=0x7fc7e419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.062+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e4102170 0x7fc7e419ef10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc02c430 tx=0x7fc7dc0337b0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.063+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/2203558917 shutdown_connections 2024-03-26T08:43:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.063+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a2950 0x7fc7d40a4e10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.063+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e4102170 0x7fc7e419ef10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.063+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e41017b0 0x7fc7e4197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.063+0000 7fc7eaf6f640 1 --2- 172.21.15.7:0/2203558917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e41005c0 0x7fc7e4197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.063+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/2203558917 >> 172.21.15.7:0/2203558917 conn(0x7fc7e40fbb30 msgr2=0x7fc7e4104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.063+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/2203558917 shutdown_connections 2024-03-26T08:43:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.063+0000 7fc7eaf6f640 1 -- 172.21.15.7:0/2203558917 wait complete. 2024-03-26T08:43:53.068 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:43:53.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:43:53.102 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:53 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/989976956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:43:53.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.241+0000 7fd92f3cd640 1 -- 172.21.15.7:0/457307611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 msgr2=0x7fd928104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.241+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/457307611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 0x7fd928104590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd924009860 tx=0x7fd92402bb70 comp rx=0 tx=0).stop 2024-03-26T08:43:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.241+0000 7fd92f3cd640 1 -- 172.21.15.7:0/457307611 shutdown_connections 2024-03-26T08:43:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.241+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/457307611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd928107400 0x7fd9281097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.241+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/457307611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd928104ad0 0x7fd928106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.241+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/457307611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 0x7fd928104590 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.241+0000 7fd92f3cd640 1 -- 172.21.15.7:0/457307611 >> 172.21.15.7:0/457307611 conn(0x7fd9280fbb30 msgr2=0x7fd9280fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.242+0000 7fd92f3cd640 1 -- 172.21.15.7:0/457307611 shutdown_connections 2024-03-26T08:43:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.242+0000 7fd92f3cd640 1 -- 172.21.15.7:0/457307611 wait complete. 2024-03-26T08:43:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.243+0000 7fd92f3cd640 1 Processor -- start 2024-03-26T08:43:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.243+0000 7fd92f3cd640 1 -- start start 2024-03-26T08:43:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92f3cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 0x7fd92819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92f3cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd928104ad0 0x7fd92819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92f3cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd928107400 0x7fd9281a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92f3cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928101810 con 0x7fd928104ad0 2024-03-26T08:43:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92f3cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928101690 con 0x7fd928107400 2024-03-26T08:43:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92f3cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928101990 con 0x7fd9281021a0 2024-03-26T08:43:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92d169640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 0x7fd92819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92d169640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 0x7fd92819dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57340/0 (socket says 172.21.15.7:57340) 2024-03-26T08:43:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92d169640 1 -- 172.21.15.7:0/3790574490 learned_addr learned my addr 172.21.15.7:0/3790574490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.244+0000 7fd92d96a640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd928107400 0x7fd9281a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.245+0000 7fd92c968640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd928104ad0 0x7fd92819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.245+0000 7fd92d169640 1 -- 172.21.15.7:0/3790574490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd928107400 msgr2=0x7fd9281a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.245+0000 7fd92d169640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd928107400 0x7fd9281a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.245+0000 7fd92d169640 1 -- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd928104ad0 msgr2=0x7fd92819e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.245+0000 7fd92d169640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd928104ad0 0x7fd92819e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.245+0000 7fd92d169640 1 -- 172.21.15.7:0/3790574490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9281a5bf0 con 0x7fd9281021a0 2024-03-26T08:43:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.245+0000 7fd92d96a640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd928107400 0x7fd9281a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.245+0000 7fd92d169640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 0x7fd92819dfd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9240349e0 tx=0x7fd9240042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.246+0000 7fd91e7fc640 1 -- 172.21.15.7:0/3790574490 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924041030 con 0x7fd9281021a0 2024-03-26T08:43:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.246+0000 7fd91e7fc640 1 -- 172.21.15.7:0/3790574490 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9240347b0 con 0x7fd9281021a0 2024-03-26T08:43:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.246+0000 7fd92f3cd640 1 -- 172.21.15.7:0/3790574490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9281a5e80 con 0x7fd9281021a0 2024-03-26T08:43:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.246+0000 7fd91e7fc640 1 -- 172.21.15.7:0/3790574490 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924039040 con 0x7fd9281021a0 2024-03-26T08:43:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.246+0000 7fd92f3cd640 1 -- 172.21.15.7:0/3790574490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9281a6230 con 0x7fd9281021a0 2024-03-26T08:43:53.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.249+0000 7fd91e7fc640 1 -- 172.21.15.7:0/3790574490 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd924044070 con 0x7fd9281021a0 2024-03-26T08:43:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.250+0000 7fd931fbc740 1 Processor -- start 2024-03-26T08:43:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.250+0000 7fd931fbc740 1 -- start start 2024-03-26T08:43:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.250+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb71ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.250+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb76b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.250+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb79680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.250+0000 7fd931fbc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5eb34610 con 0x560e5eb771f0 2024-03-26T08:43:53.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.250+0000 7fd931fbc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5ead73c0 con 0x560e5eb72080 2024-03-26T08:43:53.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.250+0000 7fd931fbc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5ea7fd60 con 0x560e5eb71690 2024-03-26T08:43:53.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd91e7fc640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9140a2900 0x7fd9140a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d96a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb71ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92c968640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb79680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d96a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb71ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57346/0 (socket says 172.21.15.7:57346) 2024-03-26T08:43:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d96a640 1 -- 172.21.15.7:0/575099092 learned_addr learned my addr 172.21.15.7:0/575099092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd91e7fc640 1 -- 172.21.15.7:0/3790574490 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd9240f0100 con 0x7fd9281021a0 2024-03-26T08:43:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d169640 1 --2- 172.21.15.7:0/575099092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb76b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d96a640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9140a2900 0x7fd9140a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d96a640 1 -- 172.21.15.7:0/575099092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 msgr2=0x560e5eb76b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d96a640 1 --2- 172.21.15.7:0/575099092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb76b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d96a640 1 -- 172.21.15.7:0/575099092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 msgr2=0x560e5eb79680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d96a640 1 --2- 172.21.15.7:0/575099092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb79680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.251+0000 7fd92d96a640 1 -- 172.21.15.7:0/575099092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e5e7eb3f0 con 0x560e5eb71690 2024-03-26T08:43:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.252+0000 7fd92d96a640 1 --2- 172.21.15.7:0/575099092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb71ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9200099e0 tx=0x7fd92000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66d77af67e37607d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.252+0000 7fd92d96a640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9140a2900 0x7fd9140a4dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd92001c000 tx=0x7fd92001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.252+0000 7fd9137fe640 1 -- 172.21.15.7:0/575099092 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920003e00 con 0x560e5eb71690 2024-03-26T08:43:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.252+0000 7fd9137fe640 1 -- 172.21.15.7:0/575099092 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd920023070 con 0x560e5eb71690 2024-03-26T08:43:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.252+0000 7fd9137fe640 1 -- 172.21.15.7:0/575099092 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92001ecf0 con 0x560e5eb71690 2024-03-26T08:43:53.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.253+0000 7fd931fbc740 1 -- 172.21.15.7:0/575099092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 msgr2=0x560e5eb71ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.253+0000 7fd931fbc740 1 --2- 172.21.15.7:0/575099092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb71ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9200099e0 tx=0x7fd92000f790 comp rx=0 tx=0).stop 2024-03-26T08:43:53.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.253+0000 7fd931fbc740 1 -- 172.21.15.7:0/575099092 shutdown_connections 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.253+0000 7fd931fbc740 1 --2- 172.21.15.7:0/575099092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb79680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.253+0000 7fd931fbc740 1 --2- 172.21.15.7:0/575099092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb76b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.253+0000 7fd931fbc740 1 --2- 172.21.15.7:0/575099092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb71ab0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.253+0000 7fd931fbc740 1 -- 172.21.15.7:0/575099092 >> 172.21.15.7:0/575099092 conn(0x560e5eb6cc00 msgr2=0x560e5eb6d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.253+0000 7fd931fbc740 1 -- 172.21.15.7:0/575099092 shutdown_connections 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.253+0000 7fd931fbc740 1 -- 172.21.15.7:0/575099092 wait complete. 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.254+0000 7fd931fbc740 1 -- start start 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.254+0000 7fd931fbc740 1 Processor -- start 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.254+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb7bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.254+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb7c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.254+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb830d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92d169640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb7bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92d96a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb830d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92c968640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb7c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92d96a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb830d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43534/0 (socket says 172.21.15.7:43534) 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92d96a640 1 -- 172.21.15.7:0/2418617524 learned_addr learned my addr 172.21.15.7:0/2418617524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92c968640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb7c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41618/0 (socket says 172.21.15.7:41618) 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5ea93240 con 0x560e5eb771f0 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5ead3460 con 0x560e5eb72080 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e5eb7a140 con 0x560e5eb71690 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd931fbc740 10 client.0 _ll_register_callbacks cb 0x560e5ea92c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92c968640 1 -- 172.21.15.7:0/2418617524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 msgr2=0x560e5eb7bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92c968640 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb7bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92c968640 1 -- 172.21.15.7:0/2418617524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 msgr2=0x560e5eb830d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92c968640 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb830d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92c968640 1 -- 172.21.15.7:0/2418617524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e5e8b57d0 con 0x560e5eb72080 2024-03-26T08:43:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.255+0000 7fd92d169640 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb7bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:53.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.256+0000 7fd92c968640 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb7c090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd92819f4e0 tx=0x7fd918012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b29902fcecc51885 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.256+0000 7fd910ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:53.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.257+0000 7fd910ff9640 1 -- 172.21.15.7:0/2418617524 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180035f0 con 0x560e5eb72080 2024-03-26T08:43:53.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.257+0000 7fd910ff9640 1 -- 172.21.15.7:0/2418617524 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd918003790 con 0x560e5eb72080 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.257+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd920023070 con 0x560e5eb72080 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.257+0000 7fd910ff9640 1 -- 172.21.15.7:0/2418617524 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd918003990 con 0x560e5eb72080 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.257+0000 7fd931fbc740 10 client.25661 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.257+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560e5eb86470 con 0x560e5eb72080 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.258+0000 7fd910ff9640 1 -- 172.21.15.7:0/2418617524 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd920023070 con 0x560e5eb72080 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.258+0000 7fd910ff9640 1 -- 172.21.15.7:0/2418617524 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd918019020 con 0x560e5eb72080 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.258+0000 7fd931fbc740 10 client.25661 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.258+0000 7fd931fbc740 10 client.25661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.258+0000 7fd931fbc740 20 client.25661 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.258+0000 7fd931fbc740 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x560e5eb87b70 0x560e5eb89fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.258+0000 7fd931fbc740 4 client.25661 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.258+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e5eae4900 con 0x560e5eb87b70 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.258+0000 7fd92d169640 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x560e5eb87b70 0x560e5eb89fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.259+0000 7fd92d169640 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x560e5eb87b70 0x560e5eb89fd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f021fe40cef8703 server_cookie=6e2b02725ab105d9 in_seq=0 out_seq=0 2024-03-26T08:43:53.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.259+0000 7fd910ff9640 10 client.25661 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:43:53.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd910ff9640 1 -- 172.21.15.7:0/2418617524 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560e5eae4900 con 0x560e5eb87b70 2024-03-26T08:43:53.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd910ff9640 10 client.25661 handle_command_reply: tid=0 2024-03-26T08:43:53.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd931fbc740 1 client.25661 shutdown 2024-03-26T08:43:53.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 msgr2=0x560e5eb7c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd931fbc740 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb7c090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd92819f4e0 tx=0x7fd918012820 comp rx=0 tx=0).stop 2024-03-26T08:43:53.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 shutdown_connections 2024-03-26T08:43:53.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd931fbc740 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb830d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd931fbc740 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb7c090 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd931fbc740 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x560e5eb87b70 0x560e5eb89fd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd931fbc740 1 --2- 172.21.15.7:0/2418617524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb7bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd910ff9640 0 client.25661 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:43:53.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.260+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 >> 172.21.15.7:0/2418617524 conn(0x560e5eb6cc00 msgr2=0x560e5eb6be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.261+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 shutdown_connections 2024-03-26T08:43:53.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.261+0000 7fd931fbc740 1 -- 172.21.15.7:0/2418617524 wait complete. 2024-03-26T08:43:53.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.261+0000 7fd931fbc740 20 client.25661 trim_cache size 0 max 16384 2024-03-26T08:43:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.286+0000 7fd931fbc740 1 Processor -- start 2024-03-26T08:43:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.286+0000 7fd931fbc740 1 -- start start 2024-03-26T08:43:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.286+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb71690 0x560e5eb7abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.286+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb72080 0x560e5eb83090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb771f0 0x560e5eb6b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd92c968640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb72080 0x560e5eb83090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd92c968640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb72080 0x560e5eb83090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57376/0 (socket says 172.21.15.7:57376) 2024-03-26T08:43:53.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd92d169640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb71690 0x560e5eb7abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd92d96a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb771f0 0x560e5eb6b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd92d169640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb71690 0x560e5eb7abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43542/0 (socket says 172.21.15.7:43542) 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd92d96a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb771f0 0x560e5eb6b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41628/0 (socket says 172.21.15.7:41628) 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd92c968640 1 -- 172.21.15.7:0/1627151103 learned_addr learned my addr 172.21.15.7:0/1627151103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd931fbc740 1 -- 172.21.15.7:0/1627151103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9281b16a0 con 0x560e5eb71690 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd931fbc740 1 -- 172.21.15.7:0/1627151103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9281af1a0 con 0x560e5eb771f0 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.287+0000 7fd931fbc740 1 -- 172.21.15.7:0/1627151103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9281ad220 con 0x560e5eb72080 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.288+0000 7fd92c968640 1 -- 172.21.15.7:0/1627151103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb771f0 msgr2=0x560e5eb6b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.288+0000 7fd92c968640 1 --2- 172.21.15.7:0/1627151103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb771f0 0x560e5eb6b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.288+0000 7fd92c968640 1 -- 172.21.15.7:0/1627151103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb71690 msgr2=0x560e5eb7abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.288+0000 7fd92c968640 1 --2- 172.21.15.7:0/1627151103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb71690 0x560e5eb7abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.288+0000 7fd92c968640 1 -- 172.21.15.7:0/1627151103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e5eb7a280 con 0x560e5eb72080 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.288+0000 7fd92d96a640 1 --2- 172.21.15.7:0/1627151103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb771f0 0x560e5eb6b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.288+0000 7fd92c968640 1 --2- 172.21.15.7:0/1627151103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb72080 0x560e5eb83090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd91800dfd0 tx=0x7fd91800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d40f65adad7e96a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.289+0000 7fd912ffd640 1 -- 172.21.15.7:0/1627151103 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918025470 con 0x560e5eb72080 2024-03-26T08:43:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.289+0000 7fd912ffd640 1 -- 172.21.15.7:0/1627151103 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd9180053a0 con 0x560e5eb72080 2024-03-26T08:43:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.289+0000 7fd912ffd640 1 -- 172.21.15.7:0/1627151103 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918025660 con 0x560e5eb72080 2024-03-26T08:43:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.290+0000 7fd931fbc740 1 -- 172.21.15.7:0/1627151103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb72080 msgr2=0x560e5eb83090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.290+0000 7fd931fbc740 1 --2- 172.21.15.7:0/1627151103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb72080 0x560e5eb83090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd91800dfd0 tx=0x7fd91800db40 comp rx=0 tx=0).stop 2024-03-26T08:43:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.290+0000 7fd931fbc740 1 -- 172.21.15.7:0/1627151103 shutdown_connections 2024-03-26T08:43:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.290+0000 7fd931fbc740 1 --2- 172.21.15.7:0/1627151103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb771f0 0x560e5eb6b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.290+0000 7fd931fbc740 1 --2- 172.21.15.7:0/1627151103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb72080 0x560e5eb83090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.290+0000 7fd931fbc740 1 --2- 172.21.15.7:0/1627151103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb71690 0x560e5eb7abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.290+0000 7fd931fbc740 1 -- 172.21.15.7:0/1627151103 >> 172.21.15.7:0/1627151103 conn(0x560e5eb6cc00 msgr2=0x560e5eb7a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.291+0000 7fd931fbc740 1 -- 172.21.15.7:0/1627151103 shutdown_connections 2024-03-26T08:43:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.291+0000 7fd931fbc740 1 -- 172.21.15.7:0/1627151103 wait complete. 2024-03-26T08:43:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.291+0000 7fd931fbc740 1 -- start start 2024-03-26T08:43:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.292+0000 7fd931fbc740 1 Processor -- start 2024-03-26T08:43:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.292+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb806f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.292+0000 7fd931fbc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb80c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.293+0000 7fd92d169640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.293+0000 7fd92d169640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb806f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57382/0 (socket says 172.21.15.7:57382) 2024-03-26T08:43:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.293+0000 7fd92d169640 1 -- 172.21.15.7:0/376889245 learned_addr learned my addr 172.21.15.7:0/376889245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.293+0000 7fd931fbc740 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb81bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.293+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9281ad820 con 0x560e5eb771f0 2024-03-26T08:43:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.293+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9281aeb40 con 0x560e5eb72080 2024-03-26T08:43:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.293+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9281ade80 con 0x560e5eb71690 2024-03-26T08:43:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.293+0000 7fd931fbc740 10 client.0 _ll_register_callbacks cb 0x7fd91800d8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.293+0000 7fd92c968640 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb80c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd92c968640 1 -- 172.21.15.7:0/376889245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 msgr2=0x560e5eb806f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd92c968640 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb806f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd92d96a640 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb81bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd92c968640 1 -- 172.21.15.7:0/376889245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 msgr2=0x560e5eb81bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd92c968640 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb81bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd92c968640 1 -- 172.21.15.7:0/376889245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e5eb821e0 con 0x560e5eb72080 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd92d96a640 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd92c968640 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb80c30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd918006010 tx=0x7fd918003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f3f2de0dcbdc7be server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd910ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.294+0000 7fd910ff9640 1 -- 172.21.15.7:0/376889245 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91803d060 con 0x560e5eb72080 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd910ff9640 1 -- 172.21.15.7:0/376889245 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd918003b30 con 0x560e5eb72080 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9180053a0 con 0x560e5eb72080 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd910ff9640 1 -- 172.21.15.7:0/376889245 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd918003d30 con 0x560e5eb72080 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd931fbc740 10 client.25667 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560e5ebe3460 con 0x560e5eb72080 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd910ff9640 1 -- 172.21.15.7:0/376889245 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd918038960 con 0x560e5eb72080 2024-03-26T08:43:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd910ff9640 1 -- 172.21.15.7:0/376889245 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd918042070 con 0x560e5eb72080 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd931fbc740 10 client.25667 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd931fbc740 10 client.25667 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd931fbc740 20 client.25667 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd931fbc740 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x560e5eb87b70 0x560e5ebe5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd931fbc740 4 client.25667 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fd9180053a0 con 0x560e5eb87b70 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.295+0000 7fd92d169640 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x560e5eb87b70 0x560e5ebe5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.296+0000 7fd910ff9640 10 client.25667 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.296+0000 7fd92d169640 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x560e5eb87b70 0x560e5ebe5c40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1afa0cfac0ec8ee1 server_cookie=bcfd09419ce44220 in_seq=0 out_seq=0 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.296+0000 7fd910ff9640 1 -- 172.21.15.7:0/376889245 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fd9180053a0 con 0x560e5eb87b70 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.296+0000 7fd910ff9640 10 client.25667 handle_command_reply: tid=0 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.296+0000 7fd931fbc740 1 client.25667 shutdown 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.296+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 msgr2=0x560e5eb80c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.296+0000 7fd931fbc740 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb80c30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd918006010 tx=0x7fd918003820 comp rx=0 tx=0).stop 2024-03-26T08:43:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.297+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 shutdown_connections 2024-03-26T08:43:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.297+0000 7fd931fbc740 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5eb771f0 0x560e5eb81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.297+0000 7fd931fbc740 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5eb72080 0x560e5eb80c30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.297+0000 7fd931fbc740 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x560e5eb87b70 0x560e5ebe5c40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.297+0000 7fd910ff9640 0 client.25667 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:43:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.297+0000 7fd931fbc740 1 --2- 172.21.15.7:0/376889245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e5eb71690 0x560e5eb806f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.297+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 >> 172.21.15.7:0/376889245 conn(0x560e5eb6cc00 msgr2=0x560e5eb6fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.297+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 shutdown_connections 2024-03-26T08:43:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.297+0000 7fd931fbc740 1 -- 172.21.15.7:0/376889245 wait complete. 2024-03-26T08:43:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.298+0000 7fd931fbc740 20 client.25667 trim_cache size 0 max 16384 2024-03-26T08:43:53.292 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.299+0000 7fd92f3cd640 1 -- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9140a2900 msgr2=0x7fd9140a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.299+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9140a2900 0x7fd9140a4dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd92001c000 tx=0x7fd92001e000 comp rx=0 tx=0).stop 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.299+0000 7fd92f3cd640 1 -- 172.21.15.7:0/3790574490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 msgr2=0x7fd92819dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.299+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 0x7fd92819dfd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9240349e0 tx=0x7fd9240042f0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.300+0000 7fd92f3cd640 1 -- 172.21.15.7:0/3790574490 shutdown_connections 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.300+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9140a2900 0x7fd9140a4dc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.300+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd928107400 0x7fd9281a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.300+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd928104ad0 0x7fd92819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.300+0000 7fd92f3cd640 1 --2- 172.21.15.7:0/3790574490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd9281021a0 0x7fd92819dfd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.300+0000 7fd92f3cd640 1 -- 172.21.15.7:0/3790574490 >> 172.21.15.7:0/3790574490 conn(0x7fd9280fbb30 msgr2=0x7fd928102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.300+0000 7fd92f3cd640 1 -- 172.21.15.7:0/3790574490 shutdown_connections 2024-03-26T08:43:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.300+0000 7fd92f3cd640 1 -- 172.21.15.7:0/3790574490 wait complete. 2024-03-26T08:43:53.307 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:43:53.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:43:53.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:53 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/989976956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:43:53.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:53 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/989976956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:43:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.522+0000 7fd21af40640 1 -- 172.21.15.7:0/2602639272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd214104ae0 msgr2=0x7fd214104f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.522+0000 7fd21af40640 1 --2- 172.21.15.7:0/2602639272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd214104ae0 0x7fd214104f60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009900 tx=0x7fd1fc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:43:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.524+0000 7fd21af40640 1 -- 172.21.15.7:0/2602639272 shutdown_connections 2024-03-26T08:43:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.524+0000 7fd21af40640 1 --2- 172.21.15.7:0/2602639272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2141054a0 0x7fd21410bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.524+0000 7fd21af40640 1 --2- 172.21.15.7:0/2602639272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd214104ae0 0x7fd214104f60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.524+0000 7fd21af40640 1 --2- 172.21.15.7:0/2602639272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2141038f0 0x7fd214103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.524+0000 7fd21af40640 1 -- 172.21.15.7:0/2602639272 >> 172.21.15.7:0/2602639272 conn(0x7fd2140fee60 msgr2=0x7fd2141012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.524+0000 7fd21af40640 1 -- 172.21.15.7:0/2602639272 shutdown_connections 2024-03-26T08:43:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.524+0000 7fd21af40640 1 -- 172.21.15.7:0/2602639272 wait complete. 2024-03-26T08:43:53.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.525+0000 7fd21af40640 1 Processor -- start 2024-03-26T08:43:53.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.525+0000 7fd21af40640 1 -- start start 2024-03-26T08:43:53.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.525+0000 7fd21af40640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2141038f0 0x7fd21419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.525+0000 7fd21af40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd214104ae0 0x7fd21419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.525+0000 7fd21af40640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2141054a0 0x7fd2141a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.525+0000 7fd21af40640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140778c0 con 0x7fd2141038f0 2024-03-26T08:43:53.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.525+0000 7fd21af40640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd214077740 con 0x7fd214104ae0 2024-03-26T08:43:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.525+0000 7fd21af40640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd214077a40 con 0x7fd2141054a0 2024-03-26T08:43:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.525+0000 7fd213fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd214104ae0 0x7fd21419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.526+0000 7fd213fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd214104ae0 0x7fd21419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41662/0 (socket says 172.21.15.7:41662) 2024-03-26T08:43:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.526+0000 7fd213fff640 1 -- 172.21.15.7:0/699129525 learned_addr learned my addr 172.21.15.7:0/699129525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.526+0000 7fd2194dd640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2141054a0 0x7fd2141a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.526+0000 7fd2194dd640 1 -- 172.21.15.7:0/699129525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd214104ae0 msgr2=0x7fd21419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.526+0000 7fd2194dd640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd214104ae0 0x7fd21419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.526+0000 7fd2194dd640 1 -- 172.21.15.7:0/699129525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2141038f0 msgr2=0x7fd21419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.526+0000 7fd2194dd640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2141038f0 0x7fd21419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.526+0000 7fd2194dd640 1 -- 172.21.15.7:0/699129525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2141a5c60 con 0x7fd2141054a0 2024-03-26T08:43:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.527+0000 7fd2194dd640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2141054a0 0x7fd2141a55f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00e970 tx=0x7fd20c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.527+0000 7fd211ffb640 1 -- 172.21.15.7:0/699129525 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00ccc0 con 0x7fd2141054a0 2024-03-26T08:43:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.527+0000 7fd211ffb640 1 -- 172.21.15.7:0/699129525 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd20c005020 con 0x7fd2141054a0 2024-03-26T08:43:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.527+0000 7fd21af40640 1 -- 172.21.15.7:0/699129525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2141a5f50 con 0x7fd2141054a0 2024-03-26T08:43:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.527+0000 7fd211ffb640 1 -- 172.21.15.7:0/699129525 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00ccc0 con 0x7fd2141054a0 2024-03-26T08:43:53.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.527+0000 7fd21af40640 1 -- 172.21.15.7:0/699129525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2141a63c0 con 0x7fd2141054a0 2024-03-26T08:43:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.531+0000 7fd211ffb640 1 -- 172.21.15.7:0/699129525 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd20c0051d0 con 0x7fd2141054a0 2024-03-26T08:43:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.531+0000 7fd21dbfd740 1 Processor -- start 2024-03-26T08:43:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.532+0000 7fd211ffb640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2000a2ac0 0x7fd2000a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.532+0000 7fd21dbfd740 1 -- start start 2024-03-26T08:43:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.532+0000 7fd211ffb640 1 -- 172.21.15.7:0/699129525 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd20c0ce700 con 0x7fd2141054a0 2024-03-26T08:43:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd213fff640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2000a2ac0 0x7fd2000a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e8080 0x55f0552ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552ed1f0 0x55f0552ef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd21dbfd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0552aa610 con 0x55f0552e7690 2024-03-26T08:43:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd21dbfd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f05524d3c0 con 0x55f0552ed1f0 2024-03-26T08:43:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd21dbfd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0551f5d60 con 0x55f0552e8080 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd218cdc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd218cdc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552e7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43572/0 (socket says 172.21.15.7:43572) 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd218cdc640 1 -- 172.21.15.7:0/1959704862 learned_addr learned my addr 172.21.15.7:0/1959704862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd2194dd640 1 --2- 172.21.15.7:0/1959704862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552ed1f0 0x55f0552ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.533+0000 7fd213fff640 1 --2- 172.21.15.7:0/1959704862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e8080 0x55f0552ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd213fff640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2000a2ac0 0x7fd2000a4f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc000c00 tx=0x7fd1fc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd218cdc640 1 -- 172.21.15.7:0/1959704862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e8080 msgr2=0x55f0552ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd218cdc640 1 --2- 172.21.15.7:0/1959704862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e8080 0x55f0552ecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd218cdc640 1 -- 172.21.15.7:0/1959704862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552ed1f0 msgr2=0x55f0552ef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd218cdc640 1 --2- 172.21.15.7:0/1959704862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552ed1f0 0x55f0552ef680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd218cdc640 1 -- 172.21.15.7:0/1959704862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f054f613f0 con 0x55f0552e7690 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd218cdc640 1 --2- 172.21.15.7:0/1959704862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552e7ab0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd204005c40 tx=0x7fd204009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e82c3839bdb6df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd20affd640 1 -- 172.21.15.7:0/1959704862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204003b70 con 0x55f0552e7690 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd20affd640 1 -- 172.21.15.7:0/1959704862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd204003d10 con 0x55f0552e7690 2024-03-26T08:43:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.534+0000 7fd20affd640 1 -- 172.21.15.7:0/1959704862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204010960 con 0x55f0552e7690 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.535+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1959704862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 msgr2=0x55f0552e7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.535+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/1959704862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552e7ab0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd204005c40 tx=0x7fd204009f90 comp rx=0 tx=0).stop 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.535+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1959704862 shutdown_connections 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.536+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/1959704862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552ed1f0 0x55f0552ef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.536+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/1959704862 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e8080 0x55f0552ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.536+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/1959704862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552e7ab0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.536+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1959704862 >> 172.21.15.7:0/1959704862 conn(0x55f0552e2c00 msgr2=0x55f0552e3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.536+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1959704862 shutdown_connections 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.536+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1959704862 wait complete. 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.537+0000 7fd21dbfd740 1 -- start start 2024-03-26T08:43:53.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.537+0000 7fd21dbfd740 1 Processor -- start 2024-03-26T08:43:53.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.538+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.538+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e8080 0x55f0552f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.538+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 0x55f0552f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.538+0000 7fd213fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e8080 0x55f0552f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.538+0000 7fd213fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e8080 0x55f0552f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41674/0 (socket says 172.21.15.7:41674) 2024-03-26T08:43:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.538+0000 7fd213fff640 1 -- 172.21.15.7:0/745229620 learned_addr learned my addr 172.21.15.7:0/745229620 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.538+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f055209240 con 0x55f0552e7690 2024-03-26T08:43:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.538+0000 7fd2194dd640 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 0x55f0552f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.538+0000 7fd218cdc640 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f055249460 con 0x55f0552e8080 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0552f01a0 con 0x55f0552ed1f0 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd21dbfd740 10 client.0 _ll_register_callbacks cb 0x55f055208c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd213fff640 1 -- 172.21.15.7:0/745229620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 msgr2=0x55f0552f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd213fff640 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 0x55f0552f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd213fff640 1 -- 172.21.15.7:0/745229620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 msgr2=0x55f0552f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd213fff640 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552f19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd213fff640 1 -- 172.21.15.7:0/745229620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f05502b7d0 con 0x55f0552e8080 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd204003d10 con 0x55f0552e8080 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd213fff640 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e8080 0x55f0552f1f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc02c240 tx=0x7fd1fc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25ddc6f76f28110 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd1f3fff640 10 client.25679 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.539+0000 7fd1f3fff640 1 -- 172.21.15.7:0/745229620 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc04e030 con 0x55f0552e8080 2024-03-26T08:43:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd1f3fff640 1 -- 172.21.15.7:0/745229620 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd1fc0046c0 con 0x55f0552e8080 2024-03-26T08:43:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd1f3fff640 1 -- 172.21.15.7:0/745229620 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1fc0048c0 con 0x55f0552e8080 2024-03-26T08:43:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd21dbfd740 10 client.25679 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x55f0552fc280 con 0x55f0552e8080 2024-03-26T08:43:53.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd1f3fff640 1 -- 172.21.15.7:0/745229620 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd1fc049040 con 0x55f0552e8080 2024-03-26T08:43:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd21dbfd740 10 client.25679 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd21dbfd740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd21dbfd740 20 client.25679 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f0552fd850 0x55f0552ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd21dbfd740 4 client.25679 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.540+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f05525a900 con 0x55f0552fd850 2024-03-26T08:43:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.541+0000 7fd218cdc640 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f0552fd850 0x55f0552ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.542+0000 7fd218cdc640 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f0552fd850 0x55f0552ffcf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f09797431f0dcb server_cookie=94eebaeeb6acba66 in_seq=0 out_seq=0 2024-03-26T08:43:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.542+0000 7fd1f3fff640 10 client.25679 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.542+0000 7fd1f3fff640 1 -- 172.21.15.7:0/745229620 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f05525a900 con 0x55f0552fd850 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.542+0000 7fd1f3fff640 10 client.25679 handle_command_reply: tid=0 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.542+0000 7fd21dbfd740 1 client.25679 shutdown 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e8080 msgr2=0x55f0552f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e8080 0x55f0552f1f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc02c240 tx=0x7fd1fc009260 comp rx=0 tx=0).stop 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 shutdown_connections 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f0552fd850 0x55f0552ffcf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 0x55f0552f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e8080 0x55f0552f1f10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/745229620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e7690 0x55f0552f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 >> 172.21.15.7:0/745229620 conn(0x55f0552e2c00 msgr2=0x55f0552e1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd1f3fff640 0 client.25679 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 shutdown_connections 2024-03-26T08:43:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.543+0000 7fd21dbfd740 1 -- 172.21.15.7:0/745229620 wait complete. 2024-03-26T08:43:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.544+0000 7fd21dbfd740 20 client.25679 trim_cache size 0 max 16384 2024-03-26T08:43:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.564+0000 7fd21dbfd740 1 Processor -- start 2024-03-26T08:43:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.564+0000 7fd21dbfd740 1 -- start start 2024-03-26T08:43:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.564+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e7690 0x55f0552f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.565+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 0x55f0552f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.565+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552ed1f0 0x55f0552e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.565+0000 7fd21dbfd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2141b1740 con 0x55f0552e8080 2024-03-26T08:43:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.565+0000 7fd21dbfd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2141af240 con 0x55f0552ed1f0 2024-03-26T08:43:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.565+0000 7fd21dbfd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2141ad2c0 con 0x55f0552e7690 2024-03-26T08:43:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.565+0000 7fd218cdc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e7690 0x55f0552f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.565+0000 7fd218cdc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e7690 0x55f0552f0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57452/0 (socket says 172.21.15.7:57452) 2024-03-26T08:43:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.565+0000 7fd218cdc640 1 -- 172.21.15.7:0/1327513265 learned_addr learned my addr 172.21.15.7:0/1327513265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.566+0000 7fd213fff640 1 --2- 172.21.15.7:0/1327513265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 0x55f0552f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.566+0000 7fd218cdc640 1 -- 172.21.15.7:0/1327513265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552ed1f0 msgr2=0x55f0552e1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.566+0000 7fd218cdc640 1 --2- 172.21.15.7:0/1327513265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552ed1f0 0x55f0552e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.566+0000 7fd218cdc640 1 -- 172.21.15.7:0/1327513265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 msgr2=0x55f0552f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.566+0000 7fd218cdc640 1 --2- 172.21.15.7:0/1327513265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 0x55f0552f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.566+0000 7fd218cdc640 1 -- 172.21.15.7:0/1327513265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0552e2a30 con 0x55f0552e7690 2024-03-26T08:43:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.566+0000 7fd218cdc640 1 --2- 172.21.15.7:0/1327513265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e7690 0x55f0552f0ff0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd20400ebb0 tx=0x7fd204005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bae7de98adfa283 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.566+0000 7fd20a7fc640 1 -- 172.21.15.7:0/1327513265 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20401d040 con 0x55f0552e7690 2024-03-26T08:43:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd20a7fc640 1 -- 172.21.15.7:0/1327513265 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd20402d030 con 0x55f0552e7690 2024-03-26T08:43:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd20a7fc640 1 -- 172.21.15.7:0/1327513265 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204010e60 con 0x55f0552e7690 2024-03-26T08:43:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1327513265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e7690 msgr2=0x55f0552f0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/1327513265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e7690 0x55f0552f0ff0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd20400ebb0 tx=0x7fd204005fd0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1327513265 shutdown_connections 2024-03-26T08:43:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/1327513265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552ed1f0 0x55f0552e1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/1327513265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 0x55f0552f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/1327513265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552e7690 0x55f0552f0ff0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1327513265 >> 172.21.15.7:0/1327513265 conn(0x55f0552e2c00 msgr2=0x55f0552e9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1327513265 shutdown_connections 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.567+0000 7fd21dbfd740 1 -- 172.21.15.7:0/1327513265 wait complete. 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd21dbfd740 1 -- start start 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd21dbfd740 1 Processor -- start 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e7690 0x55f0552f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 0x55f0552f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd21dbfd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 0x55f0552f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd21dbfd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2141ad8c0 con 0x55f0552e8080 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd21dbfd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2141aebe0 con 0x55f0552e7690 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd21dbfd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2141adf20 con 0x55f0552ed1f0 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd21dbfd740 10 client.0 _ll_register_callbacks cb 0x7fd1fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd218cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e7690 0x55f0552f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd2194dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 0x55f0552f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd213fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 0x55f0552f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd218cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e7690 0x55f0552f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41704/0 (socket says 172.21.15.7:41704) 2024-03-26T08:43:53.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd2194dd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 0x55f0552f83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57456/0 (socket says 172.21.15.7:57456) 2024-03-26T08:43:53.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.568+0000 7fd218cdc640 1 -- 172.21.15.7:0/3490639096 learned_addr learned my addr 172.21.15.7:0/3490639096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd218cdc640 1 -- 172.21.15.7:0/3490639096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 msgr2=0x55f0552f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd218cdc640 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 0x55f0552f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd218cdc640 1 -- 172.21.15.7:0/3490639096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 msgr2=0x55f0552f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd218cdc640 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 0x55f0552f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd218cdc640 1 -- 172.21.15.7:0/3490639096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0552de480 con 0x55f0552e7690 2024-03-26T08:43:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd218cdc640 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e7690 0x55f0552f6e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd204009560 tx=0x7fd204003ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e1c6decc07974a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd1f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:53.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd1f3fff640 1 -- 172.21.15.7:0/3490639096 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204010be0 con 0x55f0552e7690 2024-03-26T08:43:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd1f3fff640 1 -- 172.21.15.7:0/3490639096 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd204029040 con 0x55f0552e7690 2024-03-26T08:43:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.569+0000 7fd21dbfd740 1 -- 172.21.15.7:0/3490639096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd20402d030 con 0x55f0552e7690 2024-03-26T08:43:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.570+0000 7fd1f3fff640 1 -- 172.21.15.7:0/3490639096 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd204010d80 con 0x55f0552e7690 2024-03-26T08:43:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.570+0000 7fd21dbfd740 10 client.25682 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.570+0000 7fd21dbfd740 1 -- 172.21.15.7:0/3490639096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f05536a850 con 0x55f0552e7690 2024-03-26T08:43:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.570+0000 7fd1f3fff640 1 -- 172.21.15.7:0/3490639096 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd20402d030 con 0x55f0552e7690 2024-03-26T08:43:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.570+0000 7fd1f3fff640 1 -- 172.21.15.7:0/3490639096 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd204005220 con 0x55f0552e7690 2024-03-26T08:43:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.570+0000 7fd21dbfd740 10 client.25682 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.570+0000 7fd21dbfd740 10 client.25682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd21dbfd740 20 client.25682 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f05536aff0 0x55f05536d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd21dbfd740 4 client.25682 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd21dbfd740 1 -- 172.21.15.7:0/3490639096 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f055254c50 con 0x55f05536aff0 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd213fff640 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f05536aff0 0x55f05536d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd213fff640 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f05536aff0 0x55f05536d3e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5e84f08f5a32e8 server_cookie=67d9a82e65fc1bb6 in_seq=0 out_seq=0 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd1f3fff640 10 client.25682 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd1f3fff640 1 -- 172.21.15.7:0/3490639096 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f055254c50 con 0x55f05536aff0 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd1f3fff640 10 client.25682 handle_command_reply: tid=0 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.571+0000 7fd21dbfd740 1 client.25682 shutdown 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 -- 172.21.15.7:0/3490639096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e7690 msgr2=0x55f0552f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e7690 0x55f0552f6e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd204009560 tx=0x7fd204003ad0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 -- 172.21.15.7:0/3490639096 shutdown_connections 2024-03-26T08:43:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f05536aff0 0x55f05536d3e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f0552ed1f0 0x55f0552f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f0552e8080 0x55f0552f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 --2- 172.21.15.7:0/3490639096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f0552e7690 0x55f0552f6e90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 -- 172.21.15.7:0/3490639096 >> 172.21.15.7:0/3490639096 conn(0x55f0552e2c00 msgr2=0x55f0552e5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd1f3fff640 0 client.25682 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 -- 172.21.15.7:0/3490639096 shutdown_connections 2024-03-26T08:43:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 1 -- 172.21.15.7:0/3490639096 wait complete. 2024-03-26T08:43:53.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21dbfd740 20 client.25682 trim_cache size 0 max 16384 2024-03-26T08:43:53.567 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:43:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 -- 172.21.15.7:0/699129525 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2000a2ac0 msgr2=0x7fd2000a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2000a2ac0 0x7fd2000a4f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc000c00 tx=0x7fd1fc02df90 comp rx=0 tx=0).stop 2024-03-26T08:43:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 -- 172.21.15.7:0/699129525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2141054a0 msgr2=0x7fd2141a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2141054a0 0x7fd2141a55f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00e970 tx=0x7fd20c00ee30 comp rx=0 tx=0).stop 2024-03-26T08:43:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 -- 172.21.15.7:0/699129525 shutdown_connections 2024-03-26T08:43:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2000a2ac0 0x7fd2000a4f80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2141054a0 0x7fd2141a55f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd214104ae0 0x7fd21419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 --2- 172.21.15.7:0/699129525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2141038f0 0x7fd21419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.572+0000 7fd21af40640 1 -- 172.21.15.7:0/699129525 >> 172.21.15.7:0/699129525 conn(0x7fd2140fee60 msgr2=0x7fd2140ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.573+0000 7fd21af40640 1 -- 172.21.15.7:0/699129525 shutdown_connections 2024-03-26T08:43:53.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:53.573+0000 7fd21af40640 1 -- 172.21.15.7:0/699129525 wait complete. 2024-03-26T08:43:54.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:54 smithi007 ceph-mon[32462]: pgmap v404: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 MiB/s wr, 787 op/s 2024-03-26T08:43:54.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:54 smithi044 ceph-mon[36461]: pgmap v404: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 MiB/s wr, 787 op/s 2024-03-26T08:43:54.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:54 smithi016 ceph-mon[35851]: pgmap v404: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.7 MiB/s wr, 787 op/s 2024-03-26T08:43:54.580 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:43:54.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:43:54.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:43:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.720+0000 7f9a78bb9640 1 -- 172.21.15.7:0/3550613993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 msgr2=0x7f9a74109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.720+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/3550613993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 0x7f9a74109af0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009900 tx=0x7f9a6c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.720+0000 7f9a78bb9640 1 -- 172.21.15.7:0/3550613993 shutdown_connections 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.720+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/3550613993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 0x7f9a74109af0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.720+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/3550613993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a741028a0 0x7f9a74102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.720+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/3550613993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a741016b0 0x7f9a74101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.720+0000 7f9a78bb9640 1 -- 172.21.15.7:0/3550613993 >> 172.21.15.7:0/3550613993 conn(0x7f9a740fcc20 msgr2=0x7f9a740ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.720+0000 7f9a78bb9640 1 -- 172.21.15.7:0/3550613993 shutdown_connections 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.720+0000 7f9a78bb9640 1 -- 172.21.15.7:0/3550613993 wait complete. 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a78bb9640 1 Processor -- start 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a78bb9640 1 -- start start 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a78bb9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a741016b0 0x7f9a74197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a78bb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a741028a0 0x7f9a74197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a78bb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 0x7f9a7419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a78bb9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7410d310 con 0x7f9a74103260 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a78bb9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7410d190 con 0x7f9a741028a0 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a78bb9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7410d490 con 0x7f9a741016b0 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a72d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 0x7f9a7419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a72d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 0x7f9a7419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43608/0 (socket says 172.21.15.7:43608) 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a72d9d640 1 -- 172.21.15.7:0/2322016737 learned_addr learned my addr 172.21.15.7:0/2322016737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.721+0000 7f9a7259c640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a741016b0 0x7f9a74197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a741028a0 0x7f9a74197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a72d9d640 1 -- 172.21.15.7:0/2322016737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a741016b0 msgr2=0x7f9a74197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a741016b0 0x7f9a74197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a72d9d640 1 -- 172.21.15.7:0/2322016737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a741028a0 msgr2=0x7f9a74197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a741028a0 0x7f9a74197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a72d9d640 1 -- 172.21.15.7:0/2322016737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7419f570 con 0x7f9a74103260 2024-03-26T08:43:54.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 0x7f9a7419ef00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c02c7b0 tx=0x7f9a6c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a6b7fe640 1 -- 172.21.15.7:0/2322016737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c041070 con 0x7f9a74103260 2024-03-26T08:43:54.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a6b7fe640 1 -- 172.21.15.7:0/2322016737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a6c0054e0 con 0x7f9a74103260 2024-03-26T08:43:54.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a6b7fe640 1 -- 172.21.15.7:0/2322016737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c034440 con 0x7f9a74103260 2024-03-26T08:43:54.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a78bb9640 1 -- 172.21.15.7:0/2322016737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7419f800 con 0x7f9a74103260 2024-03-26T08:43:54.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.722+0000 7f9a78bb9640 1 -- 172.21.15.7:0/2322016737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7419fd40 con 0x7f9a74103260 2024-03-26T08:43:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.724+0000 7f9a6b7fe640 1 -- 172.21.15.7:0/2322016737 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9a6c044070 con 0x7f9a74103260 2024-03-26T08:43:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7b7a4740 1 Processor -- start 2024-03-26T08:43:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7b7a4740 1 -- start start 2024-03-26T08:43:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a6b7fe640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a600a2880 0x7f9a600a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc35190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc35760 0x56224dc3a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 0x56224dc3cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224db70c90 con 0x56224dc35760 2024-03-26T08:43:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224db96070 con 0x56224dc3a8d0 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224db3ec80 con 0x56224dc34d70 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a600a2880 0x7f9a600a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc35190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc35190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57482/0 (socket says 172.21.15.7:57482) 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7259c640 1 -- 172.21.15.7:0/1101866597 learned_addr learned my addr 172.21.15.7:0/1101866597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 0x56224dc3cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a6b7fe640 1 -- 172.21.15.7:0/2322016737 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9a6c0f0680 con 0x7f9a74103260 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc35760 0x56224dc3a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7259c640 1 -- 172.21.15.7:0/1101866597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 msgr2=0x56224dc3cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7259c640 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 0x56224dc3cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7259c640 1 -- 172.21.15.7:0/1101866597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc35760 msgr2=0x56224dc3a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7259c640 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc35760 0x56224dc3a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a7259c640 1 -- 172.21.15.7:0/1101866597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56224db36d10 con 0x56224dc34d70 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.725+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 0x56224dc3cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.726+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a600a2880 0x7f9a600a4d40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9a74198e60 tx=0x7f9a64009e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.726+0000 7f9a7259c640 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc35190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009ca0 tx=0x7f9a5c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa27c885ab205b78 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.726+0000 7f9a68ff9640 1 -- 172.21.15.7:0/1101866597 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c004220 con 0x56224dc34d70 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.726+0000 7f9a68ff9640 1 -- 172.21.15.7:0/1101866597 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a5c0043c0 con 0x56224dc34d70 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.726+0000 7f9a68ff9640 1 -- 172.21.15.7:0/1101866597 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0198d0 con 0x56224dc34d70 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1101866597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 msgr2=0x56224dc35190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc35190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009ca0 tx=0x7f9a5c011810 comp rx=0 tx=0).stop 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1101866597 shutdown_connections 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 0x56224dc3cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc35760 0x56224dc3a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/1101866597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc35190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1101866597 >> 172.21.15.7:0/1101866597 conn(0x56224dbf91a0 msgr2=0x56224dc306e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1101866597 shutdown_connections 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1101866597 wait complete. 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 -- start start 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 Processor -- start 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc34d70 0x56224dc3f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 0x56224dc3f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 0x56224dc46680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224db520d0 con 0x56224dc34d70 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224db92110 con 0x56224dc3a8d0 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56224dc3d880 con 0x56224dc35760 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc34d70 0x56224dc3f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7b7a4740 10 client.0 _ll_register_callbacks cb 0x56224db8de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc34d70 0x56224dc3f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43618/0 (socket says 172.21.15.7:43618) 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.727+0000 7f9a7259c640 1 -- 172.21.15.7:0/3053070338 learned_addr learned my addr 172.21.15.7:0/3053070338 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 0x56224dc3f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 0x56224dc46680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a7259c640 1 -- 172.21.15.7:0/3053070338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 msgr2=0x56224dc3f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a7259c640 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 0x56224dc3f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a7259c640 1 -- 172.21.15.7:0/3053070338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 msgr2=0x56224dc46680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a7259c640 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 0x56224dc46680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a7259c640 1 -- 172.21.15.7:0/3053070338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56224d9727e0 con 0x56224dc34d70 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a7259c640 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc34d70 0x56224dc3f100 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009380 tx=0x7f9a5c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe691434cff5c89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3053070338 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c028070 con 0x56224dc34d70 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3053070338 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9a5c005040 con 0x56224dc34d70 2024-03-26T08:43:54.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3053070338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a5c0043c0 con 0x56224dc34d70 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3053070338 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a5c034050 con 0x56224dc34d70 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a7b7a4740 10 client.15849 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3053070338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56224d9d2f70 con 0x56224dc34d70 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.728+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3053070338 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a5c0043c0 con 0x56224dc34d70 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3053070338 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f9a5c019c80 con 0x56224dc34d70 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a7b7a4740 10 client.15849 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a7b7a4740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a7b7a4740 20 client.15849 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56224dc4ad50 0x56224dc4d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a7b7a4740 4 client.15849 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3053070338 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56224dba35b0 con 0x56224dc4ad50 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56224dc4ad50 0x56224dc4d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a59ffb640 10 client.15849 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:54.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.729+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56224dc4ad50 0x56224dc4d180 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d05ebadab66fa0 server_cookie=2dffc1182bbdcb6b in_seq=0 out_seq=0 2024-03-26T08:43:54.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.733+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3053070338 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56224dba35b0 con 0x56224dc4ad50 2024-03-26T08:43:54.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.733+0000 7f9a59ffb640 10 client.15849 handle_command_reply: tid=0 2024-03-26T08:43:54.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.733+0000 7f9a7b7a4740 1 client.15849 shutdown 2024-03-26T08:43:54.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3053070338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc34d70 msgr2=0x56224dc3f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc34d70 0x56224dc3f100 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009380 tx=0x7f9a5c0038a0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3053070338 shutdown_connections 2024-03-26T08:43:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56224dc4ad50 0x56224dc4d180 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc3a8d0 0x56224dc46680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 0x56224dc3f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3053070338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc34d70 0x56224dc3f100 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3053070338 >> 172.21.15.7:0/3053070338 conn(0x56224dbf91a0 msgr2=0x56224dc2b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a59ffb640 0 client.15849 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3053070338 shutdown_connections 2024-03-26T08:43:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.742+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3053070338 wait complete. 2024-03-26T08:43:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.743+0000 7f9a7b7a4740 20 client.15849 trim_cache size 0 max 16384 2024-03-26T08:43:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a7b7a4740 1 Processor -- start 2024-03-26T08:43:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a7b7a4740 1 -- start start 2024-03-26T08:43:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc34d70 0x56224dc34030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 0x56224dc465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc34570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741a8d50 con 0x56224dc3a8d0 2024-03-26T08:43:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741a6850 con 0x56224dc34d70 2024-03-26T08:43:54.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741a48d0 con 0x56224dc35760 2024-03-26T08:43:54.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a72d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc34570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a71d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 0x56224dc465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a72d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc34570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43624/0 (socket says 172.21.15.7:43624) 2024-03-26T08:43:54.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a7259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc34d70 0x56224dc34030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a72d9d640 1 -- 172.21.15.7:0/1921905674 learned_addr learned my addr 172.21.15.7:0/1921905674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a72d9d640 1 -- 172.21.15.7:0/1921905674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 msgr2=0x56224dc465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/1921905674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 0x56224dc465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a72d9d640 1 -- 172.21.15.7:0/1921905674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc34d70 msgr2=0x56224dc34030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/1921905674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc34d70 0x56224dc34030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.777+0000 7f9a72d9d640 1 -- 172.21.15.7:0/1921905674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56224d86a170 con 0x56224dc3a8d0 2024-03-26T08:43:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.778+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/1921905674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc34570 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c02cec0 tx=0x7f9a6c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc7b16ee1877309 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.778+0000 7f9a5bfff640 1 -- 172.21.15.7:0/1921905674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c04b040 con 0x56224dc3a8d0 2024-03-26T08:43:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.778+0000 7f9a5bfff640 1 -- 172.21.15.7:0/1921905674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a6c004c80 con 0x56224dc3a8d0 2024-03-26T08:43:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.778+0000 7f9a5bfff640 1 -- 172.21.15.7:0/1921905674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0b9700 con 0x56224dc3a8d0 2024-03-26T08:43:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.779+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1921905674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 msgr2=0x56224dc34570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.779+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/1921905674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc34570 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c02cec0 tx=0x7f9a6c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:43:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.780+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1921905674 shutdown_connections 2024-03-26T08:43:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.780+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/1921905674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc34570 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.780+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/1921905674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc35760 0x56224dc465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.780+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/1921905674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc34d70 0x56224dc34030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.780+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1921905674 >> 172.21.15.7:0/1921905674 conn(0x56224dbf91a0 msgr2=0x56224dc33920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.780+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1921905674 shutdown_connections 2024-03-26T08:43:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.780+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/1921905674 wait complete. 2024-03-26T08:43:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.780+0000 7f9a7b7a4740 1 -- start start 2024-03-26T08:43:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.780+0000 7f9a7b7a4740 1 Processor -- start 2024-03-26T08:43:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc44620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc35760 0x56224dc44b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7b7a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc45b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc44620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741a4ed0 con 0x56224dc3a8d0 2024-03-26T08:43:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741a61f0 con 0x56224dc35760 2024-03-26T08:43:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7b7a4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741a5530 con 0x56224dc34d70 2024-03-26T08:43:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc44620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57508/0 (socket says 172.21.15.7:57508) 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7259c640 1 -- 172.21.15.7:0/3644607597 learned_addr learned my addr 172.21.15.7:0/3644607597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7b7a4740 10 client.0 _ll_register_callbacks cb 0x7f9a640127b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc35760 0x56224dc44b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a72d9d640 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc45b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7259c640 1 -- 172.21.15.7:0/3644607597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc35760 msgr2=0x56224dc44b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7259c640 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc35760 0x56224dc44b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7259c640 1 -- 172.21.15.7:0/3644607597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 msgr2=0x56224dc45b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7259c640 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc45b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a7259c640 1 -- 172.21.15.7:0/3644607597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56224dc46160 con 0x56224dc34d70 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.781+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc35760 0x56224dc44b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.782+0000 7f9a7259c640 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc44620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c011850 tx=0x7f9a5c004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=312e5f34f2205a11 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.782+0000 7f9a59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.782+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3644607597 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c03d060 con 0x56224dc34d70 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.782+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3644607597 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9a5c02d640 con 0x56224dc34d70 2024-03-26T08:43:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.782+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3644607597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a6c004c80 con 0x56224dc34d70 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.782+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3644607597 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a5c028040 con 0x56224dc34d70 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.782+0000 7f9a7b7a4740 10 client.25714 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.782+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3644607597 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56224dca6e30 con 0x56224dc34d70 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.782+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3644607597 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9a6c004c80 con 0x56224dc34d70 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.783+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3644607597 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f9a5c042070 con 0x56224dc34d70 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.783+0000 7f9a7b7a4740 10 client.25714 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.783+0000 7f9a7b7a4740 10 client.25714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.783+0000 7f9a7b7a4740 20 client.25714 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.783+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56224dca7780 0x56224dca9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.783+0000 7f9a7b7a4740 4 client.25714 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:43:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.783+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3644607597 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56224db9d900 con 0x56224dca7780 2024-03-26T08:43:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.785+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56224dca7780 0x56224dca9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.786+0000 7f9a71d9b640 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56224dca7780 0x56224dca9ba0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223f0766a7023d8e server_cookie=1f284cc6a0f06823 in_seq=0 out_seq=0 2024-03-26T08:43:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.786+0000 7f9a59ffb640 10 client.25714 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a59ffb640 1 -- 172.21.15.7:0/3644607597 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56224db9d900 con 0x56224dca7780 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a59ffb640 10 client.25714 handle_command_reply: tid=0 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 client.25714 shutdown 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3644607597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 msgr2=0x56224dc44620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc44620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c011850 tx=0x7f9a5c004310 comp rx=0 tx=0).stop 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3644607597 shutdown_connections 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56224dca7780 0x56224dca9ba0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56224dc3a8d0 0x56224dc45b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56224dc35760 0x56224dc44b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 --2- 172.21.15.7:0/3644607597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56224dc34d70 0x56224dc44620 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3644607597 >> 172.21.15.7:0/3644607597 conn(0x56224dbf91a0 msgr2=0x56224dc341b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a59ffb640 0 client.25714 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.787+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3644607597 shutdown_connections 2024-03-26T08:43:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a7b7a4740 1 -- 172.21.15.7:0/3644607597 wait complete. 2024-03-26T08:43:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a7b7a4740 20 client.25714 trim_cache size 0 max 16384 2024-03-26T08:43:54.780 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:43:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:43:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "eac35edf-8015-4b2c-b490-171bc8c43e54", 2024-03-26T08:43:54.780 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:43:54.780 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:43:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 -- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a600a2880 msgr2=0x7f9a600a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a600a2880 0x7f9a600a4d40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9a74198e60 tx=0x7f9a64009e70 comp rx=0 tx=0).stop 2024-03-26T08:43:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 -- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 msgr2=0x7f9a7419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 0x7f9a7419ef00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c02c7b0 tx=0x7f9a6c005dd0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 -- 172.21.15.7:0/2322016737 shutdown_connections 2024-03-26T08:43:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9a600a2880 0x7f9a600a4d40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a74103260 0x7f9a7419ef00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a741028a0 0x7f9a74197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 --2- 172.21.15.7:0/2322016737 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a741016b0 0x7f9a74197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.788+0000 7f9a78bb9640 1 -- 172.21.15.7:0/2322016737 >> 172.21.15.7:0/2322016737 conn(0x7f9a740fcc20 msgr2=0x7f9a74105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:54.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.789+0000 7f9a78bb9640 1 -- 172.21.15.7:0/2322016737 shutdown_connections 2024-03-26T08:43:54.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.789+0000 7f9a78bb9640 1 -- 172.21.15.7:0/2322016737 wait complete. 2024-03-26T08:43:54.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:43:54.796 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:43:54.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.971+0000 7f244011b640 1 -- 172.21.15.7:0/2355308427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438104ab0 msgr2=0x7f2438106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.971+0000 7f244011b640 1 --2- 172.21.15.7:0/2355308427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438104ab0 0x7f2438106ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2428009860 tx=0x7f242802bb70 comp rx=0 tx=0).stop 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.971+0000 7f244011b640 1 -- 172.21.15.7:0/2355308427 shutdown_connections 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.971+0000 7f244011b640 1 --2- 172.21.15.7:0/2355308427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24381073e0 0x7f24381097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.971+0000 7f244011b640 1 --2- 172.21.15.7:0/2355308427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438104ab0 0x7f2438106ea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.971+0000 7f244011b640 1 --2- 172.21.15.7:0/2355308427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2438102180 0x7f2438104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.971+0000 7f244011b640 1 -- 172.21.15.7:0/2355308427 >> 172.21.15.7:0/2355308427 conn(0x7f24380fbb70 msgr2=0x7f24380fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.972+0000 7f244011b640 1 -- 172.21.15.7:0/2355308427 shutdown_connections 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.972+0000 7f244011b640 1 -- 172.21.15.7:0/2355308427 wait complete. 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.972+0000 7f244011b640 1 Processor -- start 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.972+0000 7f244011b640 1 -- start start 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f244011b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438102180 0x7f24381963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f244011b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2438104ab0 0x7f2438196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f244011b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24381073e0 0x7f2438192f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f244011b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243810d370 con 0x7f24381073e0 2024-03-26T08:43:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f244011b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243810d1f0 con 0x7f2438102180 2024-03-26T08:43:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f244011b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243810d4f0 con 0x7f2438104ab0 2024-03-26T08:43:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243d6b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2438104ab0 0x7f2438196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243deb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438102180 0x7f24381963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243d6b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2438104ab0 0x7f2438196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57526/0 (socket says 172.21.15.7:57526) 2024-03-26T08:43:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243d6b6640 1 -- 172.21.15.7:0/1160400000 learned_addr learned my addr 172.21.15.7:0/1160400000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243deb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438102180 0x7f24381963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41770/0 (socket says 172.21.15.7:41770) 2024-03-26T08:43:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243e6b8640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24381073e0 0x7f2438192f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243deb7640 1 -- 172.21.15.7:0/1160400000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2438104ab0 msgr2=0x7f2438196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243deb7640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2438104ab0 0x7f2438196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243deb7640 1 -- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24381073e0 msgr2=0x7f2438192f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243deb7640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24381073e0 0x7f2438192f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.973+0000 7f243deb7640 1 -- 172.21.15.7:0/1160400000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24381937f0 con 0x7f2438102180 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.974+0000 7f243deb7640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438102180 0x7f24381963c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f243400e790 tx=0x7f243400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.974+0000 7f242effd640 1 -- 172.21.15.7:0/1160400000 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243400eea0 con 0x7f2438102180 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.975+0000 7f242effd640 1 -- 172.21.15.7:0/1160400000 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243400ccc0 con 0x7f2438102180 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.975+0000 7f242effd640 1 -- 172.21.15.7:0/1160400000 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434018820 con 0x7f2438102180 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.975+0000 7f244011b640 1 -- 172.21.15.7:0/1160400000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24381a60e0 con 0x7f2438102180 2024-03-26T08:43:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.975+0000 7f244011b640 1 -- 172.21.15.7:0/1160400000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24381a64f0 con 0x7f2438102180 2024-03-26T08:43:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.978+0000 7f2442de1740 1 Processor -- start 2024-03-26T08:43:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f242effd640 1 -- 172.21.15.7:0/1160400000 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f24340189c0 con 0x7f2438102180 2024-03-26T08:43:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f2442de1740 1 -- start start 2024-03-26T08:43:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f2442de1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d1941c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f2442de1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d194790 0x55d00d199290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f2442de1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d199900 0x55d00d19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f2442de1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00d0f93d0 con 0x55d00d193da0 2024-03-26T08:43:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f2442de1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00d0a1ad0 con 0x55d00d194790 2024-03-26T08:43:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f2442de1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00d0f5470 con 0x55d00d199900 2024-03-26T08:43:54.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f243d6b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d199900 0x55d00d19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f243deb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d194790 0x55d00d199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f243e6b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d1941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f243deb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d194790 0x55d00d199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41772/0 (socket says 172.21.15.7:41772) 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f243d6b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d199900 0x55d00d19bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57540/0 (socket says 172.21.15.7:57540) 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.979+0000 7f243deb7640 1 -- 172.21.15.7:0/524977197 learned_addr learned my addr 172.21.15.7:0/524977197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f243deb7640 1 -- 172.21.15.7:0/524977197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d199900 msgr2=0x55d00d19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f243deb7640 1 --2- 172.21.15.7:0/524977197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d199900 0x55d00d19bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f243deb7640 1 -- 172.21.15.7:0/524977197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 msgr2=0x55d00d1941c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f243deb7640 1 --2- 172.21.15.7:0/524977197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d1941c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f243deb7640 1 -- 172.21.15.7:0/524977197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d00ced77d0 con 0x55d00d194790 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f243e6b8640 1 --2- 172.21.15.7:0/524977197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d1941c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f242effd640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24240a2a00 0x7f24240a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f243deb7640 1 --2- 172.21.15.7:0/524977197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d194790 0x55d00d199290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24340d2b90 tx=0x7f24340d7500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a871572ab06170c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f243e6b8640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24240a2a00 0x7f24240a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f2423fff640 1 -- 172.21.15.7:0/524977197 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340977e0 con 0x55d00d194790 2024-03-26T08:43:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f2423fff640 1 -- 172.21.15.7:0/524977197 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243409b540 con 0x55d00d194790 2024-03-26T08:43:54.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f2423fff640 1 -- 172.21.15.7:0/524977197 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434016740 con 0x55d00d194790 2024-03-26T08:43:54.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.980+0000 7f242effd640 1 -- 172.21.15.7:0/1160400000 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f24340ceb30 con 0x7f2438102180 2024-03-26T08:43:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f2442de1740 1 -- 172.21.15.7:0/524977197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d194790 msgr2=0x55d00d199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f243e6b8640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24240a2a00 0x7f24240a4ec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d00d194ff0 tx=0x7f2430009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f2442de1740 1 --2- 172.21.15.7:0/524977197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d194790 0x55d00d199290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24340d2b90 tx=0x7f24340d7500 comp rx=0 tx=0).stop 2024-03-26T08:43:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f2442de1740 1 -- 172.21.15.7:0/524977197 shutdown_connections 2024-03-26T08:43:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f2442de1740 1 --2- 172.21.15.7:0/524977197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d199900 0x55d00d19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f2442de1740 1 --2- 172.21.15.7:0/524977197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d194790 0x55d00d199290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f2442de1740 1 --2- 172.21.15.7:0/524977197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d1941c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f2442de1740 1 -- 172.21.15.7:0/524977197 >> 172.21.15.7:0/524977197 conn(0x55d00d18f310 msgr2=0x55d00d18f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f2442de1740 1 -- 172.21.15.7:0/524977197 shutdown_connections 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.981+0000 7f2442de1740 1 -- 172.21.15.7:0/524977197 wait complete. 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.982+0000 7f2442de1740 1 -- start start 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.982+0000 7f2442de1740 1 Processor -- start 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.982+0000 7f2442de1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d19e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.982+0000 7f2442de1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 0x55d00d19e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.982+0000 7f2442de1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.982+0000 7f2442de1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00d19c8b0 con 0x55d00d193da0 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.982+0000 7f2442de1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00d0b5060 con 0x55d00d199900 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243d6b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 0x55d00d19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243deb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d19e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243e6b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243d6b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 0x55d00d19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57548/0 (socket says 172.21.15.7:57548) 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243deb7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d19e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43674/0 (socket says 172.21.15.7:43674) 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243d6b6640 1 -- 172.21.15.7:0/1230550630 learned_addr learned my addr 172.21.15.7:0/1230550630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f2442de1740 1 -- 172.21.15.7:0/1230550630 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d00d19ca30 con 0x55d00d194790 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f2442de1740 10 client.0 _ll_register_callbacks cb 0x55d00d0976a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243e6b8640 1 -- 172.21.15.7:0/1230550630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 msgr2=0x55d00d19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243e6b8640 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 0x55d00d19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243e6b8640 1 -- 172.21.15.7:0/1230550630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 msgr2=0x55d00d19e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243e6b8640 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d19e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243e6b8640 1 -- 172.21.15.7:0/1230550630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d00ce0d3f0 con 0x55d00d199900 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f243e6b8640 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a5770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2430003b80 tx=0x7f2430004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6c7d8da19c00c49 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f24217fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f24217fa640 1 -- 172.21.15.7:0/1230550630 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004250 con 0x55d00d199900 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f24217fa640 1 -- 172.21.15.7:0/1230550630 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f24300043f0 con 0x55d00d199900 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f2442de1740 1 -- 172.21.15.7:0/1230550630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f243409b540 con 0x55d00d199900 2024-03-26T08:43:54.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f24217fa640 1 -- 172.21.15.7:0/1230550630 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24300045f0 con 0x55d00d199900 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f2442de1740 10 client.25712 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.983+0000 7f2442de1740 1 -- 172.21.15.7:0/1230550630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d00cdcd0f0 con 0x55d00d199900 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.984+0000 7f24217fa640 1 -- 172.21.15.7:0/1230550630 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243409b540 con 0x55d00d199900 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.984+0000 7f24217fa640 1 -- 172.21.15.7:0/1230550630 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2430026070 con 0x55d00d199900 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.985+0000 7f2442de1740 10 client.25712 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.985+0000 7f2442de1740 10 client.25712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.985+0000 7f2442de1740 20 client.25712 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.985+0000 7f2442de1740 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d00d1a9c10 0x55d00d1ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.985+0000 7f2442de1740 4 client.25712 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.985+0000 7f2442de1740 1 -- 172.21.15.7:0/1230550630 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d00d106910 con 0x55d00d1a9c10 2024-03-26T08:43:54.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.985+0000 7f243deb7640 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d00d1a9c10 0x55d00d1ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.986+0000 7f243deb7640 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d00d1a9c10 0x55d00d1ac0d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ccccd9f80826b3 server_cookie=86e68e5d79815572 in_seq=0 out_seq=0 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.986+0000 7f24217fa640 10 client.25712 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.986+0000 7f24217fa640 1 -- 172.21.15.7:0/1230550630 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d00d106910 con 0x55d00d1a9c10 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.986+0000 7f24217fa640 10 client.25712 handle_command_reply: tid=0 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.986+0000 7f2442de1740 1 client.25712 shutdown 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 -- 172.21.15.7:0/1230550630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 msgr2=0x55d00d1a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a5770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2430003b80 tx=0x7f2430004ab0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 -- 172.21.15.7:0/1230550630 shutdown_connections 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d00d1a9c10 0x55d00d1ac0d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a5770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 0x55d00d19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 --2- 172.21.15.7:0/1230550630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d193da0 0x55d00d19e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:54.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 -- 172.21.15.7:0/1230550630 >> 172.21.15.7:0/1230550630 conn(0x55d00d18f310 msgr2=0x55d00d18e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:54.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f24217fa640 0 client.25712 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:54.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 -- 172.21.15.7:0/1230550630 shutdown_connections 2024-03-26T08:43:54.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 1 -- 172.21.15.7:0/1230550630 wait complete. 2024-03-26T08:43:54.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:54.987+0000 7f2442de1740 20 client.25712 trim_cache size 0 max 16384 2024-03-26T08:43:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.015+0000 7f2442de1740 1 Processor -- start 2024-03-26T08:43:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f2442de1740 1 -- start start 2024-03-26T08:43:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f2442de1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d193da0 0x55d00d18dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f2442de1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 0x55d00d1a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f2442de1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d199900 0x55d00d18e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f243deb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d193da0 0x55d00d18dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f243d6b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 0x55d00d1a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f243deb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d193da0 0x55d00d18dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41786/0 (socket says 172.21.15.7:41786) 2024-03-26T08:43:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f2442de1740 1 -- 172.21.15.7:0/983554433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24381a8f90 con 0x55d00d199900 2024-03-26T08:43:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f243deb7640 1 -- 172.21.15.7:0/983554433 learned_addr learned my addr 172.21.15.7:0/983554433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f2442de1740 1 -- 172.21.15.7:0/983554433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24381a6a90 con 0x55d00d193da0 2024-03-26T08:43:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.016+0000 7f2442de1740 1 -- 172.21.15.7:0/983554433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438071c00 con 0x55d00d194790 2024-03-26T08:43:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.017+0000 7f243e6b8640 1 --2- 172.21.15.7:0/983554433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d199900 0x55d00d18e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.017+0000 7f243deb7640 1 -- 172.21.15.7:0/983554433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 msgr2=0x55d00d1a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.017+0000 7f243deb7640 1 --2- 172.21.15.7:0/983554433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 0x55d00d1a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.017+0000 7f243deb7640 1 -- 172.21.15.7:0/983554433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d199900 msgr2=0x55d00d18e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.017+0000 7f243deb7640 1 --2- 172.21.15.7:0/983554433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d199900 0x55d00d18e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.017+0000 7f243deb7640 1 -- 172.21.15.7:0/983554433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d00d198e40 con 0x55d00d193da0 2024-03-26T08:43:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.017+0000 7f243deb7640 1 --2- 172.21.15.7:0/983554433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d193da0 0x55d00d18dca0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f243400c6a0 tx=0x7f24340d24f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=770cf20e099ce468 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f24237fe640 1 -- 172.21.15.7:0/983554433 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340b4070 con 0x55d00d193da0 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f24237fe640 1 -- 172.21.15.7:0/983554433 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243409f5e0 con 0x55d00d193da0 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f24237fe640 1 -- 172.21.15.7:0/983554433 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434026060 con 0x55d00d193da0 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f2442de1740 1 -- 172.21.15.7:0/983554433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d193da0 msgr2=0x55d00d18dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f2442de1740 1 --2- 172.21.15.7:0/983554433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d193da0 0x55d00d18dca0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f243400c6a0 tx=0x7f24340d24f0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f2442de1740 1 -- 172.21.15.7:0/983554433 shutdown_connections 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f2442de1740 1 --2- 172.21.15.7:0/983554433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d199900 0x55d00d18e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f2442de1740 1 --2- 172.21.15.7:0/983554433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d194790 0x55d00d1a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f2442de1740 1 --2- 172.21.15.7:0/983554433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d193da0 0x55d00d18dca0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.018+0000 7f2442de1740 1 -- 172.21.15.7:0/983554433 >> 172.21.15.7:0/983554433 conn(0x55d00d18f310 msgr2=0x55d00d1ab9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:55.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.019+0000 7f2442de1740 1 -- 172.21.15.7:0/983554433 shutdown_connections 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.019+0000 7f2442de1740 1 -- 172.21.15.7:0/983554433 wait complete. 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.019+0000 7f2442de1740 1 -- start start 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.019+0000 7f2442de1740 1 Processor -- start 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.019+0000 7f2442de1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d193da0 0x55d00d1a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.019+0000 7f2442de1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d194790 0x55d00d1a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.019+0000 7f2442de1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f2442de1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438072200 con 0x55d00d194790 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f2442de1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438073520 con 0x55d00d199900 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f2442de1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438072860 con 0x55d00d193da0 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243d6b6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d194790 0x55d00d1a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f2442de1740 10 client.0 _ll_register_callbacks cb 0x7f24300114a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243d6b6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d194790 0x55d00d1a2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43690/0 (socket says 172.21.15.7:43690) 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243deb7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d193da0 0x55d00d1a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243d6b6640 1 -- 172.21.15.7:0/3671523352 learned_addr learned my addr 172.21.15.7:0/3671523352 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243e6b8640 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243e6b8640 1 -- 172.21.15.7:0/3671523352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d193da0 msgr2=0x55d00d1a2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243e6b8640 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d193da0 0x55d00d1a2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243e6b8640 1 -- 172.21.15.7:0/3671523352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d194790 msgr2=0x55d00d1a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243e6b8640 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d194790 0x55d00d1a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243e6b8640 1 -- 172.21.15.7:0/3671523352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d00d1a45c0 con 0x55d00d199900 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243deb7640 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d193da0 0x55d00d1a2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f243e6b8640 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a3fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f243001e540 tx=0x7f243000f7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d5e63c34a574762 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.020+0000 7f24217fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.021+0000 7f24217fa640 1 -- 172.21.15.7:0/3671523352 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004450 con 0x55d00d199900 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.021+0000 7f24217fa640 1 -- 172.21.15.7:0/3671523352 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f243001eac0 con 0x55d00d199900 2024-03-26T08:43:55.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.021+0000 7f2442de1740 1 -- 172.21.15.7:0/3671523352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f243409f5e0 con 0x55d00d199900 2024-03-26T08:43:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.021+0000 7f24217fa640 1 -- 172.21.15.7:0/3671523352 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2430040050 con 0x55d00d199900 2024-03-26T08:43:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f2442de1740 10 client.25718 fetch_fsmap learned FSMap version 24 2024-03-26T08:43:55.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f2442de1740 1 -- 172.21.15.7:0/3671523352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d00d1a5940 con 0x55d00d199900 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f24217fa640 1 -- 172.21.15.7:0/3671523352 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f243001ecc0 con 0x55d00d199900 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f24217fa640 1 -- 172.21.15.7:0/3671523352 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2430026040 con 0x55d00d199900 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f2442de1740 10 client.25718 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f2442de1740 10 client.25718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f2442de1740 20 client.25718 populate_metadata read hostname 'smithi007' 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f2442de1740 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d00d205b30 0x55d00d207f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f2442de1740 4 client.25718 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f2442de1740 1 -- 172.21.15.7:0/3671523352 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f243409f5e0 con 0x55d00d205b30 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.022+0000 7f243deb7640 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d00d205b30 0x55d00d207f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f24217fa640 10 client.25718 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f243deb7640 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d00d205b30 0x55d00d207f50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f005a69145e262e6 server_cookie=8e5d5ac7ec72f363 in_seq=0 out_seq=0 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f24217fa640 1 -- 172.21.15.7:0/3671523352 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f243409f5e0 con 0x55d00d205b30 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f24217fa640 10 client.25718 handle_command_reply: tid=0 2024-03-26T08:43:55.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 client.25718 shutdown 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 -- 172.21.15.7:0/3671523352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 msgr2=0x55d00d1a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a3fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f243001e540 tx=0x7f243000f7e0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 -- 172.21.15.7:0/3671523352 shutdown_connections 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d00d205b30 0x55d00d207f50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d00d199900 0x55d00d1a3fd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f24217fa640 0 client.25718 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d00d194790 0x55d00d1a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 --2- 172.21.15.7:0/3671523352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d00d193da0 0x55d00d1a2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 -- 172.21.15.7:0/3671523352 >> 172.21.15.7:0/3671523352 conn(0x55d00d18f310 msgr2=0x55d00d192570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 -- 172.21.15.7:0/3671523352 shutdown_connections 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.023+0000 7f2442de1740 1 -- 172.21.15.7:0/3671523352 wait complete. 2024-03-26T08:43:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f2442de1740 20 client.25718 trim_cache size 0 max 16384 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout: "eac35edf-8015-4b2c-b490-171bc8c43e54": { 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "eac35edf-8015-4b2c-b490-171bc8c43e54", 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:43:55.019 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:43:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 -- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24240a2a00 msgr2=0x7f24240a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24240a2a00 0x7f24240a4ec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d00d194ff0 tx=0x7f2430009500 comp rx=0 tx=0).stop 2024-03-26T08:43:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 -- 172.21.15.7:0/1160400000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438102180 msgr2=0x7f24381963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:43:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438102180 0x7f24381963c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f243400e790 tx=0x7f243400ec50 comp rx=0 tx=0).stop 2024-03-26T08:43:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 -- 172.21.15.7:0/1160400000 shutdown_connections 2024-03-26T08:43:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f24240a2a00 0x7f24240a4ec0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24381073e0 0x7f2438192f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2438104ab0 0x7f2438196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 --2- 172.21.15.7:0/1160400000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2438102180 0x7f24381963c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:43:55.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 -- 172.21.15.7:0/1160400000 >> 172.21.15.7:0/1160400000 conn(0x7f24380fbb70 msgr2=0x7f24381019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:43:55.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 -- 172.21.15.7:0/1160400000 shutdown_connections 2024-03-26T08:43:55.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:43:55.024+0000 7f244011b640 1 -- 172.21.15.7:0/1160400000 wait complete. 2024-03-26T08:43:55.029 INFO:tasks.cephfs.filesystem:scrub status for tag:eac35edf-8015-4b2c-b490-171bc8c43e54 - {'path': '/', 'tag': 'eac35edf-8015-4b2c-b490-171bc8c43e54', 'options': 'recursive,force'} 2024-03-26T08:43:56.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 85.0 s, 3704.2 tps, lat 27.022 ms stddev 29.680 2024-03-26T08:43:56.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:56 smithi007 ceph-mon[32462]: pgmap v405: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 MiB/s wr, 766 op/s 2024-03-26T08:43:56.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:56 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:43:56.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:56 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:56.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:56 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:43:56.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:56 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:43:56.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:56 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:56.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:56 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:56.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:56 smithi044 ceph-mon[36461]: pgmap v405: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 MiB/s wr, 766 op/s 2024-03-26T08:43:56.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:56 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:43:56.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:56 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:56.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:56 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:43:56.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:56 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:43:56.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:56 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:56.316 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:56 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:56.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:56 smithi016 ceph-mon[35851]: pgmap v405: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 10 MiB/s wr, 766 op/s 2024-03-26T08:43:56.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:56 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:43:56.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:56 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:56.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:56 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:43:56.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:56 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:43:56.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:56 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:56.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:56 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:43:58.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:43:58 smithi007 ceph-mon[32462]: pgmap v406: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.3 MiB/s wr, 783 op/s 2024-03-26T08:43:58.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:43:58 smithi044 ceph-mon[36461]: pgmap v406: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.3 MiB/s wr, 783 op/s 2024-03-26T08:43:58.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:43:58 smithi016 ceph-mon[35851]: pgmap v406: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.3 MiB/s wr, 783 op/s 2024-03-26T08:44:00.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:00 smithi007 ceph-mon[32462]: pgmap v407: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.3 MiB/s wr, 861 op/s 2024-03-26T08:44:00.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:00 smithi044 ceph-mon[36461]: pgmap v407: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.3 MiB/s wr, 861 op/s 2024-03-26T08:44:00.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:00 smithi016 ceph-mon[35851]: pgmap v407: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 9.3 MiB/s wr, 861 op/s 2024-03-26T08:44:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 90.0 s, 4289.6 tps, lat 23.273 ms stddev 26.381 2024-03-26T08:44:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.411+0000 7f06c7745640 1 -- 172.21.15.7:0/3582182753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c0076450 msgr2=0x7f06c00788c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.411+0000 7f06c7745640 1 --2- 172.21.15.7:0/3582182753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c0076450 0x7f06c00788c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06b400dc70 tx=0x7f06b4034100 comp rx=0 tx=0).stop 2024-03-26T08:44:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.412+0000 7f06c7745640 1 -- 172.21.15.7:0/3582182753 shutdown_connections 2024-03-26T08:44:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.412+0000 7f06c7745640 1 --2- 172.21.15.7:0/3582182753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c0078e00 0x7f06c007b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.412+0000 7f06c7745640 1 --2- 172.21.15.7:0/3582182753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c0076450 0x7f06c00788c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.412+0000 7f06c7745640 1 --2- 172.21.15.7:0/3582182753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0073570 0x7f06c0073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.412+0000 7f06c7745640 1 -- 172.21.15.7:0/3582182753 >> 172.21.15.7:0/3582182753 conn(0x7f06c006e6b0 msgr2=0x7f06c0070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.413+0000 7f06c7745640 1 -- 172.21.15.7:0/3582182753 shutdown_connections 2024-03-26T08:44:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.413+0000 7f06c7745640 1 -- 172.21.15.7:0/3582182753 wait complete. 2024-03-26T08:44:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.414+0000 7f06c7745640 1 Processor -- start 2024-03-26T08:44:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.414+0000 7f06c7745640 1 -- start start 2024-03-26T08:44:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.414+0000 7f06c7745640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c0073570 0x7f06c012a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.414+0000 7f06c7745640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0076450 0x7f06c012ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.415+0000 7f06c54e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c0073570 0x7f06c012a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.415+0000 7f06c54e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c0073570 0x7f06c012a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60698/0 (socket says 172.21.15.7:60698) 2024-03-26T08:44:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.415+0000 7f06c54e1640 1 -- 172.21.15.7:0/4097347905 learned_addr learned my addr 172.21.15.7:0/4097347905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.415+0000 7f06c7745640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c0078e00 0x7f06c0131820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.415+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c007eb20 con 0x7f06c0076450 2024-03-26T08:44:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.415+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c007e9a0 con 0x7f06c0073570 2024-03-26T08:44:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.415+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c007eca0 con 0x7f06c0078e00 2024-03-26T08:44:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.415+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0076450 0x7f06c012ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.415+0000 7f06c5ce2640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c0078e00 0x7f06c0131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.416+0000 7f06c5ce2640 1 -- 172.21.15.7:0/4097347905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c0073570 msgr2=0x7f06c012a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.416+0000 7f06c5ce2640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c0073570 0x7f06c012a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.416+0000 7f06c5ce2640 1 -- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0076450 msgr2=0x7f06c012ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.416+0000 7f06c5ce2640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0076450 0x7f06c012ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.416+0000 7f06c5ce2640 1 -- 172.21.15.7:0/4097347905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0131f50 con 0x7f06c0078e00 2024-03-26T08:44:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.416+0000 7f06c5ce2640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c0078e00 0x7f06c0131820 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06b800afd0 tx=0x7f06b800dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.418+0000 7f06c54e1640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c0073570 0x7f06c012a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:44:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.418+0000 7f06be7fc640 1 -- 172.21.15.7:0/4097347905 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8017030 con 0x7f06c0078e00 2024-03-26T08:44:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.418+0000 7f06be7fc640 1 -- 172.21.15.7:0/4097347905 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06b8003e50 con 0x7f06c0078e00 2024-03-26T08:44:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.419+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0076450 0x7f06c012ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:44:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.420+0000 7f06be7fc640 1 -- 172.21.15.7:0/4097347905 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8004420 con 0x7f06c0078e00 2024-03-26T08:44:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.420+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c0132180 con 0x7f06c0078e00 2024-03-26T08:44:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.420+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c0132650 con 0x7f06c0078e00 2024-03-26T08:44:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.424+0000 7f06ca40d740 1 Processor -- start 2024-03-26T08:44:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.425+0000 7f06be7fc640 1 -- 172.21.15.7:0/4097347905 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f06b801a070 con 0x7f06c0078e00 2024-03-26T08:44:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.425+0000 7f06ca40d740 1 -- start start 2024-03-26T08:44:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.425+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee6ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.425+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee6fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.425+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee72910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.425+0000 7f06ca40d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6edcfb10 con 0x562a6ee6b310 2024-03-26T08:44:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.425+0000 7f06ca40d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6ed78900 con 0x562a6ee6a920 2024-03-26T08:44:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.425+0000 7f06ca40d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6edcbbb0 con 0x562a6ee70480 2024-03-26T08:44:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c4ce0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee6ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c4ce0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee6ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60704/0 (socket says 172.21.15.7:60704) 2024-03-26T08:44:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c4ce0640 1 -- 172.21.15.7:0/715813798 learned_addr learned my addr 172.21.15.7:0/715813798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/715813798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee72910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06be7fc640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06a80a2870 0x7f06a80a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c54e1640 1 --2- 172.21.15.7:0/715813798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee6fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06be7fc640 1 -- 172.21.15.7:0/4097347905 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06b80cd790 con 0x7f06c0078e00 2024-03-26T08:44:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c4ce0640 1 -- 172.21.15.7:0/715813798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 msgr2=0x562a6ee72910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/715813798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee72910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c4ce0640 1 -- 172.21.15.7:0/715813798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 msgr2=0x562a6ee6fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/715813798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee6fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c4ce0640 1 -- 172.21.15.7:0/715813798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6eae4210 con 0x562a6ee6a920 2024-03-26T08:44:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.426+0000 7f06c5ce2640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06a80a2870 0x7f06a80a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.427+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/715813798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee6ad40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005b60 tx=0x7f06b4005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd6baf42dbd2c1db server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.427+0000 7f06a77fe640 1 -- 172.21.15.7:0/715813798 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b403c470 con 0x562a6ee6a920 2024-03-26T08:44:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.428+0000 7f06a77fe640 1 -- 172.21.15.7:0/715813798 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06b403c610 con 0x562a6ee6a920 2024-03-26T08:44:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.428+0000 7f06a77fe640 1 -- 172.21.15.7:0/715813798 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b403cc80 con 0x562a6ee6a920 2024-03-26T08:44:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.428+0000 7f06c5ce2640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06a80a2870 0x7f06a80a4d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f06b800a430 tx=0x7f06b8007200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.429+0000 7f06ca40d740 1 -- 172.21.15.7:0/715813798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 msgr2=0x562a6ee6ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.429+0000 7f06ca40d740 1 --2- 172.21.15.7:0/715813798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee6ad40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005b60 tx=0x7f06b4005c90 comp rx=0 tx=0).stop 2024-03-26T08:44:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.430+0000 7f06ca40d740 1 -- 172.21.15.7:0/715813798 shutdown_connections 2024-03-26T08:44:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.430+0000 7f06ca40d740 1 --2- 172.21.15.7:0/715813798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee72910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.430+0000 7f06ca40d740 1 --2- 172.21.15.7:0/715813798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee6fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.430+0000 7f06ca40d740 1 --2- 172.21.15.7:0/715813798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee6ad40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.430+0000 7f06ca40d740 1 -- 172.21.15.7:0/715813798 >> 172.21.15.7:0/715813798 conn(0x562a6ee65e90 msgr2=0x562a6ee66290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.430+0000 7f06ca40d740 1 -- 172.21.15.7:0/715813798 shutdown_connections 2024-03-26T08:44:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.430+0000 7f06ca40d740 1 -- 172.21.15.7:0/715813798 wait complete. 2024-03-26T08:44:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.430+0000 7f06ca40d740 1 -- start start 2024-03-26T08:44:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.430+0000 7f06ca40d740 1 Processor -- start 2024-03-26T08:44:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee74c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee6b310 0x562a6ee75190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee70480 0x562a6ee7c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06ca40d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6ee733d0 con 0x562a6ee6a920 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06ca40d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6ed8bd50 con 0x562a6ee70480 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06ca40d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6ee73550 con 0x562a6ee6b310 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06ca40d740 10 client.0 _ll_register_callbacks cb 0x562a6ed8b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06c4ce0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee74c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06c4ce0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee74c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47172/0 (socket says 172.21.15.7:47172) 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06c4ce0640 1 -- 172.21.15.7:0/1710178001 learned_addr learned my addr 172.21.15.7:0/1710178001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06c54e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee6b310 0x562a6ee75190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.431+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee70480 0x562a6ee7c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06c4ce0640 1 -- 172.21.15.7:0/1710178001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee6b310 msgr2=0x562a6ee75190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee6b310 0x562a6ee75190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06c4ce0640 1 -- 172.21.15.7:0/1710178001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee70480 msgr2=0x562a6ee7c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee70480 0x562a6ee7c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06c4ce0640 1 -- 172.21.15.7:0/1710178001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a6ebb0620 con 0x562a6ee6a920 2024-03-26T08:44:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee74c50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005fd0 tx=0x7f06b403c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c0e3901236ded4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:44:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1710178001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4009090 con 0x562a6ee6a920 2024-03-26T08:44:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1710178001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06b4004020 con 0x562a6ee6a920 2024-03-26T08:44:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.432+0000 7f06ca40d740 1 -- 172.21.15.7:0/1710178001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06b403c610 con 0x562a6ee6a920 2024-03-26T08:44:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.433+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1710178001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b4004220 con 0x562a6ee6a920 2024-03-26T08:44:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06ca40d740 10 client.15882 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06ca40d740 1 -- 172.21.15.7:0/1710178001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562a6ee733d0 con 0x562a6ee6a920 2024-03-26T08:44:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1710178001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06b403c610 con 0x562a6ee6a920 2024-03-26T08:44:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1710178001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f06b4003680 con 0x562a6ee6a920 2024-03-26T08:44:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06ca40d740 10 client.15882 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06ca40d740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06ca40d740 20 client.15882 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562a6ee80570 0x562a6ee82960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06ca40d740 4 client.15882 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.434+0000 7f06ca40d740 1 -- 172.21.15.7:0/1710178001 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a6eddd050 con 0x562a6ee80570 2024-03-26T08:44:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.435+0000 7f06c54e1640 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562a6ee80570 0x562a6ee82960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.435+0000 7f06a4ff9640 10 client.15882 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.435+0000 7f06c54e1640 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562a6ee80570 0x562a6ee82960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc576140e809d24b server_cookie=892d7e4aeec93aef in_seq=0 out_seq=0 2024-03-26T08:44:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.437+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1710178001 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562a6eddd050 con 0x562a6ee80570 2024-03-26T08:44:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.437+0000 7f06a4ff9640 10 client.15882 handle_command_reply: tid=0 2024-03-26T08:44:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.437+0000 7f06ca40d740 1 client.15882 shutdown 2024-03-26T08:44:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 -- 172.21.15.7:0/1710178001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 msgr2=0x562a6ee74c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee74c50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005fd0 tx=0x7f06b403c7c0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 -- 172.21.15.7:0/1710178001 shutdown_connections 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562a6ee80570 0x562a6ee82960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee70480 0x562a6ee7c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06a4ff9640 0 client.15882 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee6b310 0x562a6ee75190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1710178001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee74c50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 -- 172.21.15.7:0/1710178001 >> 172.21.15.7:0/1710178001 conn(0x562a6ee65e90 msgr2=0x562a6ee65020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 -- 172.21.15.7:0/1710178001 shutdown_connections 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 1 -- 172.21.15.7:0/1710178001 wait complete. 2024-03-26T08:44:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.438+0000 7f06ca40d740 20 client.15882 trim_cache size 0 max 16384 2024-03-26T08:44:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.463+0000 7f06ca40d740 1 Processor -- start 2024-03-26T08:44:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.463+0000 7f06ca40d740 1 -- start start 2024-03-26T08:44:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee74f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee7c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee64790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06ca40d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c01e43a0 con 0x562a6ee6b310 2024-03-26T08:44:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06ca40d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c013b170 con 0x562a6ee6a920 2024-03-26T08:44:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06c4ce0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee74f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06c54e1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee7c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06c4ce0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee74f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60714/0 (socket says 172.21.15.7:60714) 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06c4ce0640 1 -- 172.21.15.7:0/1408781397 learned_addr learned my addr 172.21.15.7:0/1408781397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06c54e1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee7c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47184/0 (socket says 172.21.15.7:47184) 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06ca40d740 1 -- 172.21.15.7:0/1408781397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c01391f0 con 0x562a6ee70480 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.464+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1408781397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee64790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06c4ce0640 1 -- 172.21.15.7:0/1408781397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 msgr2=0x562a6ee64790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1408781397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee64790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06c4ce0640 1 -- 172.21.15.7:0/1408781397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 msgr2=0x562a6ee7c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1408781397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee7c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06c4ce0640 1 -- 172.21.15.7:0/1408781397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a6ee6bb70 con 0x562a6ee6a920 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06c54e1640 1 --2- 172.21.15.7:0/1408781397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee7c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1408781397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee74f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f06b4034800 tx=0x7f06b4005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=292cb86fc94f0c43 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06a6ffd640 1 -- 172.21.15.7:0/1408781397 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4003f80 con 0x562a6ee6a920 2024-03-26T08:44:01.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06a6ffd640 1 -- 172.21.15.7:0/1408781397 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06b403ce30 con 0x562a6ee6a920 2024-03-26T08:44:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.465+0000 7f06a6ffd640 1 -- 172.21.15.7:0/1408781397 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b40037f0 con 0x562a6ee6a920 2024-03-26T08:44:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.466+0000 7f06ca40d740 1 -- 172.21.15.7:0/1408781397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 msgr2=0x562a6ee74f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.466+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1408781397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee74f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f06b4034800 tx=0x7f06b4005f10 comp rx=0 tx=0).stop 2024-03-26T08:44:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.466+0000 7f06ca40d740 1 -- 172.21.15.7:0/1408781397 shutdown_connections 2024-03-26T08:44:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.466+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1408781397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.466+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1408781397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6b310 0x562a6ee7c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.466+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1408781397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6a920 0x562a6ee74f70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.466+0000 7f06ca40d740 1 -- 172.21.15.7:0/1408781397 >> 172.21.15.7:0/1408781397 conn(0x562a6ee65e90 msgr2=0x562a6ee74860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.466+0000 7f06ca40d740 1 -- 172.21.15.7:0/1408781397 shutdown_connections 2024-03-26T08:44:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.466+0000 7f06ca40d740 1 -- 172.21.15.7:0/1408781397 wait complete. 2024-03-26T08:44:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.467+0000 7f06ca40d740 1 -- start start 2024-03-26T08:44:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.467+0000 7f06ca40d740 1 Processor -- start 2024-03-26T08:44:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.467+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee7a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6b310 0x562a6ee7a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06ca40d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee7b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06ca40d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c01397f0 con 0x562a6ee6a920 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06ca40d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c013ab10 con 0x562a6ee6b310 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06ca40d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0139e50 con 0x562a6ee70480 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c54e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6b310 0x562a6ee7a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c54e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6b310 0x562a6ee7a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60718/0 (socket says 172.21.15.7:60718) 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c54e1640 1 -- 172.21.15.7:0/1298212722 learned_addr learned my addr 172.21.15.7:0/1298212722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee7a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06ca40d740 10 client.0 _ll_register_callbacks cb 0x7f06b4016270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c54e1640 1 -- 172.21.15.7:0/1298212722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 msgr2=0x562a6ee7b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee7b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c54e1640 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee7b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c54e1640 1 -- 172.21.15.7:0/1298212722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 msgr2=0x562a6ee7a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c54e1640 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee7a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c54e1640 1 -- 172.21.15.7:0/1298212722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a6ee7bb50 con 0x562a6ee6b310 2024-03-26T08:44:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee7b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:44:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06c54e1640 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6b310 0x562a6ee7a610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b800 tx=0x7f06b00039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cfc84ac38e30f29 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.468+0000 7f06a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1298212722 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000b030 con 0x562a6ee6b310 2024-03-26T08:44:01.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1298212722 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06b000b1d0 con 0x562a6ee6b310 2024-03-26T08:44:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06ca40d740 1 -- 172.21.15.7:0/1298212722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06b403ce30 con 0x562a6ee6b310 2024-03-26T08:44:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1298212722 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b001f520 con 0x562a6ee6b310 2024-03-26T08:44:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06ca40d740 10 client.25733 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06ca40d740 1 -- 172.21.15.7:0/1298212722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562a6eedc540 con 0x562a6ee6b310 2024-03-26T08:44:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1298212722 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06b403ce30 con 0x562a6ee6b310 2024-03-26T08:44:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1298212722 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f06b00285e0 con 0x562a6ee6b310 2024-03-26T08:44:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06ca40d740 10 client.25733 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06ca40d740 10 client.25733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06ca40d740 20 client.25733 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562a6eedcf00 0x562a6eedf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06ca40d740 4 client.25733 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:44:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.469+0000 7f06ca40d740 1 -- 172.21.15.7:0/1298212722 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562a6edd73a0 con 0x562a6eedcf00 2024-03-26T08:44:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.470+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562a6eedcf00 0x562a6eedf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.470+0000 7f06a4ff9640 10 client.25733 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.470+0000 7f06c4ce0640 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562a6eedcf00 0x562a6eedf320 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a5c0d35b960dd7d server_cookie=3a31458d08d600bf in_seq=0 out_seq=0 2024-03-26T08:44:01.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.487+0000 7f06a4ff9640 1 -- 172.21.15.7:0/1298212722 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562a6edd73a0 con 0x562a6eedcf00 2024-03-26T08:44:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.487+0000 7f06a4ff9640 10 client.25733 handle_command_reply: tid=0 2024-03-26T08:44:01.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.487+0000 7f06ca40d740 1 client.25733 shutdown 2024-03-26T08:44:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 -- 172.21.15.7:0/1298212722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6b310 msgr2=0x562a6ee7a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6b310 0x562a6ee7a610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b800 tx=0x7f06b00039c0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 -- 172.21.15.7:0/1298212722 shutdown_connections 2024-03-26T08:44:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562a6eedcf00 0x562a6eedf320 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562a6ee70480 0x562a6ee7b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a6ee6b310 0x562a6ee7a610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 --2- 172.21.15.7:0/1298212722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562a6ee6a920 0x562a6ee7a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 -- 172.21.15.7:0/1298212722 >> 172.21.15.7:0/1298212722 conn(0x562a6ee65e90 msgr2=0x562a6ee64790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06a4ff9640 0 client.25733 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 -- 172.21.15.7:0/1298212722 shutdown_connections 2024-03-26T08:44:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 1 -- 172.21.15.7:0/1298212722 wait complete. 2024-03-26T08:44:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.488+0000 7f06ca40d740 20 client.25733 trim_cache size 0 max 16384 2024-03-26T08:44:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:44:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:44:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:44:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:44:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.489+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06a80a2870 msgr2=0x7f06a80a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.489+0000 7f06c7745640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06a80a2870 0x7f06a80a4d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f06b800a430 tx=0x7f06b8007200 comp rx=0 tx=0).stop 2024-03-26T08:44:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.489+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c0078e00 msgr2=0x7f06c0131820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.489+0000 7f06c7745640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c0078e00 0x7f06c0131820 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f06b800afd0 tx=0x7f06b800dd20 comp rx=0 tx=0).stop 2024-03-26T08:44:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.490+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 shutdown_connections 2024-03-26T08:44:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.490+0000 7f06c7745640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06a80a2870 0x7f06a80a4d30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.490+0000 7f06c7745640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06c0078e00 0x7f06c0131820 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.490+0000 7f06c7745640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c0076450 0x7f06c012ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.490+0000 7f06c7745640 1 --2- 172.21.15.7:0/4097347905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c0073570 0x7f06c012a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.490+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 >> 172.21.15.7:0/4097347905 conn(0x7f06c006e6b0 msgr2=0x7f06c007b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.490+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 shutdown_connections 2024-03-26T08:44:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:01.490+0000 7f06c7745640 1 -- 172.21.15.7:0/4097347905 wait complete. 2024-03-26T08:44:02.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:02 smithi007 ceph-mon[32462]: pgmap v408: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.9 MiB/s wr, 819 op/s 2024-03-26T08:44:02.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:02 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:44:02.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:02 smithi044 ceph-mon[36461]: pgmap v408: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.9 MiB/s wr, 819 op/s 2024-03-26T08:44:02.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:02 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:02 smithi016 ceph-mon[35851]: pgmap v408: 97 pgs: 97 active+clean; 1.1 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 8.9 MiB/s wr, 819 op/s 2024-03-26T08:44:02.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:02 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:44:03.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:03 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:44:03.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:03 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:44:03.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:03 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:44:04.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:04 smithi044 ceph-mon[36461]: pgmap v409: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 953 op/s 2024-03-26T08:44:04.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:04 smithi016 ceph-mon[35851]: pgmap v409: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 953 op/s 2024-03-26T08:44:04.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:04 smithi007 ceph-mon[32462]: pgmap v409: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 953 op/s 2024-03-26T08:44:06.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 95.0 s, 4103.4 tps, lat 24.347 ms stddev 23.394 2024-03-26T08:44:06.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:06 smithi044 ceph-mon[36461]: pgmap v410: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 945 op/s 2024-03-26T08:44:06.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:06 smithi016 ceph-mon[35851]: pgmap v410: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 945 op/s 2024-03-26T08:44:06.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:06 smithi007 ceph-mon[32462]: pgmap v410: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 945 op/s 2024-03-26T08:44:08.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:08 smithi044 ceph-mon[36461]: pgmap v411: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 924 op/s 2024-03-26T08:44:08.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:08 smithi016 ceph-mon[35851]: pgmap v411: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 924 op/s 2024-03-26T08:44:08.543 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:08 smithi007 ceph-mon[32462]: pgmap v411: 97 pgs: 97 active+clean; 1.1 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 924 op/s 2024-03-26T08:44:10.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:10 smithi044 ceph-mon[36461]: pgmap v412: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 924 op/s 2024-03-26T08:44:10.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:10 smithi016 ceph-mon[35851]: pgmap v412: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 924 op/s 2024-03-26T08:44:10.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:10 smithi007 ceph-mon[32462]: pgmap v412: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 12 MiB/s wr, 924 op/s 2024-03-26T08:44:11.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 100.0 s, 4119.8 tps, lat 24.269 ms stddev 26.543 2024-03-26T08:44:12.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:12 smithi044 ceph-mon[36461]: pgmap v413: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 769 op/s 2024-03-26T08:44:12.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:12 smithi016 ceph-mon[35851]: pgmap v413: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 769 op/s 2024-03-26T08:44:12.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:12 smithi007 ceph-mon[32462]: pgmap v413: 97 pgs: 97 active+clean; 1.1 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 769 op/s 2024-03-26T08:44:14.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:14 smithi044 ceph-mon[36461]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 852 op/s 2024-03-26T08:44:14.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:14 smithi016 ceph-mon[35851]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 852 op/s 2024-03-26T08:44:14.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:14 smithi007 ceph-mon[32462]: pgmap v414: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 852 op/s 2024-03-26T08:44:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.624+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2183496229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ac102180 msgr2=0x7f28ac108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.624+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2183496229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ac102180 0x7f28ac108a10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009ca0 tx=0x7f28a402c100 comp rx=0 tx=0).stop 2024-03-26T08:44:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.625+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2183496229 shutdown_connections 2024-03-26T08:44:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.625+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2183496229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ac102180 0x7f28ac108a10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.625+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2183496229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac1017c0 0x7f28ac101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.625+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2183496229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28ac1005d0 0x7f28ac1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.625+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2183496229 >> 172.21.15.7:0/2183496229 conn(0x7f28ac0fbb40 msgr2=0x7f28ac0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:14.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.625+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2183496229 shutdown_connections 2024-03-26T08:44:14.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.626+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2183496229 wait complete. 2024-03-26T08:44:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.626+0000 7f28b0ef8640 1 Processor -- start 2024-03-26T08:44:14.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.626+0000 7f28b0ef8640 1 -- start start 2024-03-26T08:44:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.626+0000 7f28b0ef8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28ac1005d0 0x7f28ac19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.626+0000 7f28b0ef8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ac1017c0 0x7f28ac19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.626+0000 7f28b0ef8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac102180 0x7f28ac1a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.626+0000 7f28b0ef8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac10a860 con 0x7f28ac1005d0 2024-03-26T08:44:14.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.626+0000 7f28b0ef8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac10a6e0 con 0x7f28ac102180 2024-03-26T08:44:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.626+0000 7f28b0ef8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac10a9e0 con 0x7f28ac1017c0 2024-03-26T08:44:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28ac1005d0 0x7f28ac19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28a9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ac1017c0 0x7f28ac19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac102180 0x7f28ac1a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28a9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ac1017c0 0x7f28ac19e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46690/0 (socket says 172.21.15.7:46690) 2024-03-26T08:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28ac1005d0 0x7f28ac19e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57702/0 (socket says 172.21.15.7:57702) 2024-03-26T08:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28a9d9b640 1 -- 172.21.15.7:0/2820776707 learned_addr learned my addr 172.21.15.7:0/2820776707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28aad9d640 1 -- 172.21.15.7:0/2820776707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ac1017c0 msgr2=0x7f28ac19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28aad9d640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ac1017c0 0x7f28ac19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28aad9d640 1 -- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28ac1005d0 msgr2=0x7f28ac19e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28aad9d640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28ac1005d0 0x7f28ac19e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28aad9d640 1 -- 172.21.15.7:0/2820776707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac1a5c30 con 0x7f28ac102180 2024-03-26T08:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.627+0000 7f28aad9d640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac102180 0x7f28ac1a55c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28a4037040 tx=0x7f28a4035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.628+0000 7f289b7fe640 1 -- 172.21.15.7:0/2820776707 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4039040 con 0x7f28ac102180 2024-03-26T08:44:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.628+0000 7f289b7fe640 1 -- 172.21.15.7:0/2820776707 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28a4044070 con 0x7f28ac102180 2024-03-26T08:44:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.628+0000 7f289b7fe640 1 -- 172.21.15.7:0/2820776707 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4004470 con 0x7f28ac102180 2024-03-26T08:44:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.628+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2820776707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac1a5ec0 con 0x7f28ac102180 2024-03-26T08:44:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.628+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2820776707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac1a6350 con 0x7f28ac102180 2024-03-26T08:44:14.625 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.628+0000 7f28aa59c640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28ac1005d0 0x7f28ac19e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:14.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.631+0000 7f289b7fe640 1 -- 172.21.15.7:0/2820776707 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f28a4004610 con 0x7f28ac102180 2024-03-26T08:44:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.631+0000 7f28b3bbe740 1 Processor -- start 2024-03-26T08:44:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.632+0000 7f289b7fe640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28940a28f0 0x7f28940a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.632+0000 7f289b7fe640 1 -- 172.21.15.7:0/2820776707 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28a40f0e90 con 0x7f28ac102180 2024-03-26T08:44:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.633+0000 7f28b3bbe740 1 -- start start 2024-03-26T08:44:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.633+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a5920 0x5559fa8a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.633+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 0x5559fa8aae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.633+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8ab480 0x5559fa8ad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.633+0000 7f28b3bbe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fa80ab10 con 0x5559fa8a5920 2024-03-26T08:44:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.633+0000 7f28b3bbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fa7b3900 con 0x5559fa8a6310 2024-03-26T08:44:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.633+0000 7f28b3bbe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fa806bb0 con 0x5559fa8ab480 2024-03-26T08:44:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.633+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8ab480 0x5559fa8ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.633+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8ab480 0x5559fa8ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46696/0 (socket says 172.21.15.7:46696) 2024-03-26T08:44:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a5920 0x5559fa8a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28aad9d640 1 -- 172.21.15.7:0/260849171 learned_addr learned my addr 172.21.15.7:0/260849171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28a9d9b640 1 --2- 172.21.15.7:0/260849171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 0x5559fa8aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28a9d9b640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28940a28f0 0x7f28940a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28aa59c640 1 -- 172.21.15.7:0/260849171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8ab480 msgr2=0x5559fa8ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28aa59c640 1 --2- 172.21.15.7:0/260849171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8ab480 0x5559fa8ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28aa59c640 1 -- 172.21.15.7:0/260849171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 msgr2=0x5559fa8aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28aa59c640 1 --2- 172.21.15.7:0/260849171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 0x5559fa8aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28aa59c640 1 -- 172.21.15.7:0/260849171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559fa51f210 con 0x5559fa8a5920 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28aa59c640 1 --2- 172.21.15.7:0/260849171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a5920 0x5559fa8a5d40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f289c007750 tx=0x7f289c007c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4bcb957518141f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.634+0000 7f28a9d9b640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28940a28f0 0x7f28940a4db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5559fa8a6710 tx=0x7f28a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.635+0000 7f2898ff9640 1 -- 172.21.15.7:0/260849171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c007e60 con 0x5559fa8a5920 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.635+0000 7f2898ff9640 1 -- 172.21.15.7:0/260849171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f289c018070 con 0x5559fa8a5920 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.635+0000 7f2898ff9640 1 -- 172.21.15.7:0/260849171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c013de0 con 0x5559fa8a5920 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.636+0000 7f28b3bbe740 1 -- 172.21.15.7:0/260849171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a5920 msgr2=0x5559fa8a5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.636+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/260849171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a5920 0x5559fa8a5d40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f289c007750 tx=0x7f289c007c10 comp rx=0 tx=0).stop 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.636+0000 7f28b3bbe740 1 -- 172.21.15.7:0/260849171 shutdown_connections 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.636+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/260849171 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8ab480 0x5559fa8ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.636+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/260849171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 0x5559fa8aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.636+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/260849171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a5920 0x5559fa8a5d40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.636+0000 7f28b3bbe740 1 -- 172.21.15.7:0/260849171 >> 172.21.15.7:0/260849171 conn(0x5559fa8a0e90 msgr2=0x5559fa8a1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.636+0000 7f28b3bbe740 1 -- 172.21.15.7:0/260849171 shutdown_connections 2024-03-26T08:44:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.637+0000 7f28b3bbe740 1 -- 172.21.15.7:0/260849171 wait complete. 2024-03-26T08:44:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.638+0000 7f28b3bbe740 1 -- start start 2024-03-26T08:44:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.638+0000 7f28b3bbe740 1 Processor -- start 2024-03-26T08:44:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.639+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.639+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.639+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46706/0 (socket says 172.21.15.7:46706) 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.639+0000 7f28aa59c640 1 -- 172.21.15.7:0/208981134 learned_addr learned my addr 172.21.15.7:0/208981134 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.639+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a6310 0x5559fa8c1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.639+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8ab480 0x5559fa8c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.639+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fa8bf440 con 0x5559fa8a6310 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.639+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fa7c6d50 con 0x5559fa8ab480 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.639+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fa8bf5c0 con 0x5559fa8a5920 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f28b3bbe740 10 client.0 _ll_register_callbacks cb 0x5559fa7c6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f28a9d9b640 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a6310 0x5559fa8c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f28aad9d640 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8ab480 0x5559fa8c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.638 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f28aad9d640 1 -- 172.21.15.7:0/208981134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 msgr2=0x5559fa8c0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f28aad9d640 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f28aad9d640 1 -- 172.21.15.7:0/208981134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a6310 msgr2=0x5559fa8c1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f28aad9d640 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a6310 0x5559fa8c1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f28aad9d640 1 -- 172.21.15.7:0/208981134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559fa5eb620 con 0x5559fa8ab480 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f28aad9d640 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8ab480 0x5559fa8c8190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28a40b9c50 tx=0x7f28a40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319a9941731d879d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.640+0000 7f2891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.641+0000 7f2891ffb640 1 -- 172.21.15.7:0/208981134 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a40bdef0 con 0x5559fa8ab480 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.641+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f289c018070 con 0x5559fa8ab480 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.641+0000 7f2891ffb640 1 -- 172.21.15.7:0/208981134 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28a40a8070 con 0x5559fa8ab480 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.641+0000 7f2891ffb640 1 -- 172.21.15.7:0/208981134 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28a40098b0 con 0x5559fa8ab480 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.641+0000 7f28b3bbe740 10 client.25745 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.641+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5559fa8cb500 con 0x5559fa8ab480 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.642+0000 7f2891ffb640 1 -- 172.21.15.7:0/208981134 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f289c018070 con 0x5559fa8ab480 2024-03-26T08:44:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.642+0000 7f2891ffb640 1 -- 172.21.15.7:0/208981134 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f28a40dd040 con 0x5559fa8ab480 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.642+0000 7f28b3bbe740 10 client.25745 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.642+0000 7f28b3bbe740 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.642+0000 7f28b3bbe740 20 client.25745 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.642+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5559fa8ccc30 0x5559fa8cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.642+0000 7f28b3bbe740 4 client.25745 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.642+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559fa818050 con 0x5559fa8ccc30 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.642+0000 7f28aa59c640 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5559fa8ccc30 0x5559fa8cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.643+0000 7f28aa59c640 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5559fa8ccc30 0x5559fa8cf090 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8509101218d1a6 server_cookie=6cec9b568fe3e27a in_seq=0 out_seq=0 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.643+0000 7f2891ffb640 10 client.25745 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.644+0000 7f2891ffb640 1 -- 172.21.15.7:0/208981134 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5559fa818050 con 0x5559fa8ccc30 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.644+0000 7f2891ffb640 10 client.25745 handle_command_reply: tid=0 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.644+0000 7f28b3bbe740 1 client.25745 shutdown 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.644+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8ab480 msgr2=0x5559fa8c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8ab480 0x5559fa8c8190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28a40b9c50 tx=0x7f28a40095c0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 shutdown_connections 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5559fa8ccc30 0x5559fa8cf090 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8ab480 0x5559fa8c8190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8a6310 0x5559fa8c1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/208981134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 >> 172.21.15.7:0/208981134 conn(0x5559fa8a0e90 msgr2=0x5559fa8a0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f2891ffb640 0 client.25745 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 shutdown_connections 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 1 -- 172.21.15.7:0/208981134 wait complete. 2024-03-26T08:44:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.645+0000 7f28b3bbe740 20 client.25745 trim_cache size 0 max 16384 2024-03-26T08:44:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.672+0000 7f28b3bbe740 1 Processor -- start 2024-03-26T08:44:14.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.672+0000 7f28b3bbe740 1 -- start start 2024-03-26T08:44:14.663 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.672+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a5920 0x5559fa8bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.672+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a6310 0x5559fa8c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.672+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8a7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.672+0000 7f28b3bbe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1a8df0 con 0x5559fa8ab480 2024-03-26T08:44:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.672+0000 7f28b3bbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1a68f0 con 0x5559fa8a5920 2024-03-26T08:44:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.672+0000 7f28b3bbe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac071c90 con 0x5559fa8a6310 2024-03-26T08:44:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.673+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.673+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8a7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57734/0 (socket says 172.21.15.7:57734) 2024-03-26T08:44:14.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.673+0000 7f28aad9d640 1 -- 172.21.15.7:0/2718465674 learned_addr learned my addr 172.21.15.7:0/2718465674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.673+0000 7f28a9d9b640 1 --2- 172.21.15.7:0/2718465674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a6310 0x5559fa8c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.674+0000 7f28aad9d640 1 -- 172.21.15.7:0/2718465674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a6310 msgr2=0x5559fa8c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.674+0000 7f28aad9d640 1 --2- 172.21.15.7:0/2718465674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a6310 0x5559fa8c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.674+0000 7f28aad9d640 1 -- 172.21.15.7:0/2718465674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a5920 msgr2=0x5559fa8bf990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:44:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.674+0000 7f28aad9d640 1 --2- 172.21.15.7:0/2718465674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a5920 0x5559fa8bf990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.674+0000 7f28aad9d640 1 -- 172.21.15.7:0/2718465674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559fa8a5710 con 0x5559fa8ab480 2024-03-26T08:44:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.674+0000 7f28aad9d640 1 --2- 172.21.15.7:0/2718465674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8a7590 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f28a402c710 tx=0x7f28a4004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb5d6801754d95c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.674+0000 7f2893fff640 1 -- 172.21.15.7:0/2718465674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a40d1bd0 con 0x5559fa8ab480 2024-03-26T08:44:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.675+0000 7f2893fff640 1 -- 172.21.15.7:0/2718465674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28a40bdef0 con 0x5559fa8ab480 2024-03-26T08:44:14.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.675+0000 7f2893fff640 1 -- 172.21.15.7:0/2718465674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a40c5b00 con 0x5559fa8ab480 2024-03-26T08:44:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.676+0000 7f28b3bbe740 1 -- 172.21.15.7:0/2718465674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 msgr2=0x5559fa8a7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.676+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/2718465674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8a7590 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f28a402c710 tx=0x7f28a4004990 comp rx=0 tx=0).stop 2024-03-26T08:44:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.676+0000 7f28b3bbe740 1 -- 172.21.15.7:0/2718465674 shutdown_connections 2024-03-26T08:44:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.676+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/2718465674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8a7590 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.676+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/2718465674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a6310 0x5559fa8c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.676+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/2718465674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a5920 0x5559fa8bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.676+0000 7f28b3bbe740 1 -- 172.21.15.7:0/2718465674 >> 172.21.15.7:0/2718465674 conn(0x5559fa8a0e90 msgr2=0x5559fa89beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:14.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.676+0000 7f28b3bbe740 1 -- 172.21.15.7:0/2718465674 shutdown_connections 2024-03-26T08:44:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.676+0000 7f28b3bbe740 1 -- 172.21.15.7:0/2718465674 wait complete. 2024-03-26T08:44:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28b3bbe740 1 -- start start 2024-03-26T08:44:14.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28b3bbe740 1 Processor -- start 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 0x5559fa8c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28b3bbe740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28b3bbe740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac072290 con 0x5559fa8ab480 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28b3bbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0735b0 con 0x5559fa8a6310 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28b3bbe740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0728f0 con 0x5559fa8a5920 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28b3bbe740 10 client.0 _ll_register_callbacks cb 0x7f28a40b9a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28aa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28aad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8c6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57750/0 (socket says 172.21.15.7:57750) 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.677+0000 7f28aad9d640 1 -- 172.21.15.7:0/3303879766 learned_addr learned my addr 172.21.15.7:0/3303879766 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:14.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.678+0000 7f28a9d9b640 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 0x5559fa8c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.683+0000 7f28a9d9b640 1 -- 172.21.15.7:0/3303879766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 msgr2=0x5559fa8c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.683+0000 7f28a9d9b640 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.683+0000 7f28a9d9b640 1 -- 172.21.15.7:0/3303879766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 msgr2=0x5559fa8c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.683+0000 7f28a9d9b640 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.683+0000 7f28a9d9b640 1 -- 172.21.15.7:0/3303879766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559fa8c7470 con 0x5559fa8a6310 2024-03-26T08:44:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.683+0000 7f28aa59c640 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.687+0000 7f28a9d9b640 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 0x5559fa8c5e70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d760 tx=0x7f28a0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ac6ec638878a2f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.687+0000 7f2891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.687+0000 7f2891ffb640 1 -- 172.21.15.7:0/3303879766 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0003a00 con 0x5559fa8a6310 2024-03-26T08:44:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.687+0000 7f28b3bbe740 1 -- 172.21.15.7:0/3303879766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28a40bdef0 con 0x5559fa8a6310 2024-03-26T08:44:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.688+0000 7f2891ffb640 1 -- 172.21.15.7:0/3303879766 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28a0003ba0 con 0x5559fa8a6310 2024-03-26T08:44:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.688+0000 7f2891ffb640 1 -- 172.21.15.7:0/3303879766 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28a0003da0 con 0x5559fa8a6310 2024-03-26T08:44:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.688+0000 7f28b3bbe740 10 client.25748 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.688+0000 7f28b3bbe740 1 -- 172.21.15.7:0/3303879766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5559fa939680 con 0x5559fa8a6310 2024-03-26T08:44:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.689+0000 7f2891ffb640 1 -- 172.21.15.7:0/3303879766 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28a40bdef0 con 0x5559fa8a6310 2024-03-26T08:44:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.689+0000 7f2891ffb640 1 -- 172.21.15.7:0/3303879766 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f28a001c040 con 0x5559fa8a6310 2024-03-26T08:44:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.689+0000 7f28b3bbe740 10 client.25748 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.689+0000 7f28b3bbe740 10 client.25748 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.689+0000 7f28b3bbe740 20 client.25748 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.689+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5559fa93a010 0x5559fa93c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.689+0000 7f28b3bbe740 4 client.25748 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:44:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.689+0000 7f28b3bbe740 1 -- 172.21.15.7:0/3303879766 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5559fa8123a0 con 0x5559fa93a010 2024-03-26T08:44:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.689+0000 7f28aa59c640 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5559fa93a010 0x5559fa93c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.690+0000 7f28aa59c640 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5559fa93a010 0x5559fa93c430 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f515451ebaa208b server_cookie=8bdc3c4ca086ee9e in_seq=0 out_seq=0 2024-03-26T08:44:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.690+0000 7f2891ffb640 10 client.25748 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f2891ffb640 1 -- 172.21.15.7:0/3303879766 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5559fa8123a0 con 0x5559fa93a010 2024-03-26T08:44:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f2891ffb640 10 client.25748 handle_command_reply: tid=0 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 client.25748 shutdown 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 -- 172.21.15.7:0/3303879766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 msgr2=0x5559fa8c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 0x5559fa8c5e70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f28a000d760 tx=0x7f28a0004730 comp rx=0 tx=0).stop 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 -- 172.21.15.7:0/3303879766 shutdown_connections 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5559fa93a010 0x5559fa93c430 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5559fa8ab480 0x5559fa8c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5559fa8a6310 0x5559fa8c5e70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 --2- 172.21.15.7:0/3303879766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5559fa8a5920 0x5559fa8c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 -- 172.21.15.7:0/3303879766 >> 172.21.15.7:0/3303879766 conn(0x5559fa8a0e90 msgr2=0x5559fa8a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f2891ffb640 0 client.25748 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 -- 172.21.15.7:0/3303879766 shutdown_connections 2024-03-26T08:44:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.709+0000 7f28b3bbe740 1 -- 172.21.15.7:0/3303879766 wait complete. 2024-03-26T08:44:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.711+0000 7f28b3bbe740 20 client.25748 trim_cache size 0 max 16384 2024-03-26T08:44:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:44:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:44:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:44:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:44:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.712+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28940a28f0 msgr2=0x7f28940a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.712+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28940a28f0 0x7f28940a4db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5559fa8a6710 tx=0x7f28a0005f90 comp rx=0 tx=0).stop 2024-03-26T08:44:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.712+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2820776707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac102180 msgr2=0x7f28ac1a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.713+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac102180 0x7f28ac1a55c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28a4037040 tx=0x7f28a4035d10 comp rx=0 tx=0).stop 2024-03-26T08:44:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.713+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2820776707 shutdown_connections 2024-03-26T08:44:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.713+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28940a28f0 0x7f28940a4db0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.713+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ac102180 0x7f28ac1a55c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.713+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28ac1017c0 0x7f28ac19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.704 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.713+0000 7f28b0ef8640 1 --2- 172.21.15.7:0/2820776707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28ac1005d0 0x7f28ac19e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.713+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2820776707 >> 172.21.15.7:0/2820776707 conn(0x7f28ac0fbb40 msgr2=0x7f28ac10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.714+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2820776707 shutdown_connections 2024-03-26T08:44:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:14.714+0000 7f28b0ef8640 1 -- 172.21.15.7:0/2820776707 wait complete. 2024-03-26T08:44:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 105.0 s, 4389.0 tps, lat 22.806 ms stddev 22.924 2024-03-26T08:44:16.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:16 smithi044 ceph-mon[36461]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 805 op/s 2024-03-26T08:44:16.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:16 smithi016 ceph-mon[35851]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 805 op/s 2024-03-26T08:44:16.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:16 smithi007 ceph-mon[32462]: pgmap v415: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 805 op/s 2024-03-26T08:44:18.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:18 smithi044 ceph-mon[36461]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-03-26T08:44:18.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:18 smithi016 ceph-mon[35851]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-03-26T08:44:18.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:18 smithi007 ceph-mon[32462]: pgmap v416: 97 pgs: 97 active+clean; 1.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 11 MiB/s wr, 873 op/s 2024-03-26T08:44:20.315 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:20 smithi044 ceph-mon[36461]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 812 op/s 2024-03-26T08:44:20.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:20 smithi016 ceph-mon[35851]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 812 op/s 2024-03-26T08:44:20.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:20 smithi007 ceph-mon[32462]: pgmap v417: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 MiB/s wr, 812 op/s 2024-03-26T08:44:21.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 110.0 s, 4449.2 tps, lat 22.407 ms stddev 22.477 2024-03-26T08:44:22.343 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:22 smithi007 ceph-mon[32462]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 810 op/s 2024-03-26T08:44:22.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:22 smithi016 ceph-mon[35851]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 810 op/s 2024-03-26T08:44:22.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:22 smithi044 ceph-mon[36461]: pgmap v418: 97 pgs: 97 active+clean; 1.1 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 810 op/s 2024-03-26T08:44:24.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:24 smithi016 ceph-mon[35851]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 890 op/s 2024-03-26T08:44:24.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:24 smithi007 ceph-mon[32462]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 890 op/s 2024-03-26T08:44:24.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:24 smithi044 ceph-mon[36461]: pgmap v419: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 890 op/s 2024-03-26T08:44:25.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:44:25.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:44:25.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.184+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/3152050726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281073a0 msgr2=0x7f2f281097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.184+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/3152050726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281073a0 0x7f2f281097f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009ca0 tx=0x7f2f2002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.184+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/3152050726 shutdown_connections 2024-03-26T08:44:25.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.184+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/3152050726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281073a0 0x7f2f281097f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.184+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/3152050726 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f28104a70 0x7f2f28106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.184+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/3152050726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28102140 0x7f2f28104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.184+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/3152050726 >> 172.21.15.7:0/3152050726 conn(0x7f2f280fbb30 msgr2=0x7f2f280fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.184+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/3152050726 shutdown_connections 2024-03-26T08:44:25.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.184+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/3152050726 wait complete. 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2fcf1640 1 Processor -- start 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2fcf1640 1 -- start start 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2fcf1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f28102140 0x7f2f28195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2fcf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28104a70 0x7f2f28195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2d28c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28104a70 0x7f2f28195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2fcf1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f281073a0 0x7f2f2819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2da8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f28102140 0x7f2f28195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2fcf1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2810d340 con 0x7f2f28102140 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2d28c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28104a70 0x7f2f28195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42442/0 (socket says 172.21.15.7:42442) 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2d28c640 1 -- 172.21.15.7:0/2321076576 learned_addr learned my addr 172.21.15.7:0/2321076576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2810d1c0 con 0x7f2f28104a70 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2da8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f28102140 0x7f2f28195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55462/0 (socket says 172.21.15.7:55462) 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.185+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2810d4c0 con 0x7f2f281073a0 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2e28e640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f281073a0 0x7f2f2819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2d28c640 1 -- 172.21.15.7:0/2321076576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f281073a0 msgr2=0x7f2f2819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f281073a0 0x7f2f2819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2d28c640 1 -- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f28102140 msgr2=0x7f2f28195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f28102140 0x7f2f28195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2d28c640 1 -- 172.21.15.7:0/2321076576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2819d3f0 con 0x7f2f28104a70 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f28102140 0x7f2f28195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2e28e640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f281073a0 0x7f2f2819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28104a70 0x7f2f28195d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800e9c0 tx=0x7f2f1800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f1effd640 1 -- 172.21.15.7:0/2321076576 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800cd80 con 0x7f2f28104a70 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2819d6e0 con 0x7f2f28104a70 2024-03-26T08:44:25.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f1effd640 1 -- 172.21.15.7:0/2321076576 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f18005500 con 0x7f2f28104a70 2024-03-26T08:44:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f1effd640 1 -- 172.21.15.7:0/2321076576 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18010690 con 0x7f2f28104a70 2024-03-26T08:44:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.186+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2819dc80 con 0x7f2f28104a70 2024-03-26T08:44:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.189+0000 7f2f1effd640 1 -- 172.21.15.7:0/2321076576 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2f180046c0 con 0x7f2f28104a70 2024-03-26T08:44:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.189+0000 7f2f329b0740 1 Processor -- start 2024-03-26T08:44:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.189+0000 7f2f329b0740 1 -- start start 2024-03-26T08:44:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723883290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 0x557723885d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f329b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577237e33d0 con 0x55772387e790 2024-03-26T08:44:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f329b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772378bad0 con 0x55772387dda0 2024-03-26T08:44:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f329b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577237df470 con 0x557723883900 2024-03-26T08:44:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f2e28e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f2da8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723883290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f2e28e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42450/0 (socket says 172.21.15.7:42450) 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f2d28c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 0x557723885d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.190+0000 7f2f2e28e640 1 -- 172.21.15.7:0/4207153026 learned_addr learned my addr 172.21.15.7:0/4207153026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f2e28e640 1 -- 172.21.15.7:0/4207153026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 msgr2=0x557723885d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f2e28e640 1 --2- 172.21.15.7:0/4207153026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 0x557723885d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f2e28e640 1 -- 172.21.15.7:0/4207153026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 msgr2=0x557723883290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f2e28e640 1 --2- 172.21.15.7:0/4207153026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723883290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f2e28e640 1 -- 172.21.15.7:0/4207153026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577235c17d0 con 0x55772387dda0 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/4207153026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723883290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f1effd640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f140a2800 0x7f2f140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f2e28e640 1 --2- 172.21.15.7:0/4207153026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387e1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009ca0 tx=0x7f2f20009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d374bb6ddbc402c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f1effd640 1 -- 172.21.15.7:0/2321076576 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f180cee70 con 0x7f2f28104a70 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f140a2800 0x7f2f140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f13fff640 1 -- 172.21.15.7:0/4207153026 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20043020 con 0x55772387dda0 2024-03-26T08:44:25.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f13fff640 1 -- 172.21.15.7:0/4207153026 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f20032c90 con 0x55772387dda0 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.191+0000 7f2f13fff640 1 -- 172.21.15.7:0/4207153026 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20039040 con 0x55772387dda0 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f329b0740 1 -- 172.21.15.7:0/4207153026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 msgr2=0x55772387e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f329b0740 1 --2- 172.21.15.7:0/4207153026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387e1c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009ca0 tx=0x7f2f20009800 comp rx=0 tx=0).stop 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f140a2800 0x7f2f140a4cc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009fd0 tx=0x7f2f24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f329b0740 1 -- 172.21.15.7:0/4207153026 shutdown_connections 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f329b0740 1 --2- 172.21.15.7:0/4207153026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 0x557723885d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f329b0740 1 --2- 172.21.15.7:0/4207153026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723883290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f329b0740 1 --2- 172.21.15.7:0/4207153026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387e1c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f329b0740 1 -- 172.21.15.7:0/4207153026 >> 172.21.15.7:0/4207153026 conn(0x557723879310 msgr2=0x557723879710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f329b0740 1 -- 172.21.15.7:0/4207153026 shutdown_connections 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.192+0000 7f2f329b0740 1 -- 172.21.15.7:0/4207153026 wait complete. 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.193+0000 7f2f329b0740 1 -- start start 2024-03-26T08:44:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.193+0000 7f2f329b0740 1 Processor -- start 2024-03-26T08:44:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.193+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387dda0 0x557723888200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387e790 0x557723888740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 0x55772388f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f329b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577238868f0 con 0x55772387dda0 2024-03-26T08:44:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f329b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55772379f060 con 0x557723883900 2024-03-26T08:44:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f329b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557723886a70 con 0x55772387e790 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f329b0740 10 client.0 _ll_register_callbacks cb 0x5577237816a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2da8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387dda0 0x557723888200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2da8d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387dda0 0x557723888200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55472/0 (socket says 172.21.15.7:55472) 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2da8d640 1 -- 172.21.15.7:0/956278230 learned_addr learned my addr 172.21.15.7:0/956278230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387e790 0x557723888740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2e28e640 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 0x55772388f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2da8d640 1 -- 172.21.15.7:0/956278230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387e790 msgr2=0x557723888740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387e790 0x557723888740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2da8d640 1 -- 172.21.15.7:0/956278230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 msgr2=0x55772388f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 0x55772388f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2da8d640 1 -- 172.21.15.7:0/956278230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577234f73f0 con 0x55772387dda0 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387dda0 0x557723888200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f24012df0 tx=0x7f2f24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c5980f53ee38ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.194+0000 7f2f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.195+0000 7f2f117fa640 1 -- 172.21.15.7:0/956278230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24004050 con 0x55772387dda0 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f117fa640 1 -- 172.21.15.7:0/956278230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f240041f0 con 0x55772387dda0 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f329b0740 1 -- 172.21.15.7:0/956278230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f20032c90 con 0x55772387dda0 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f117fa640 1 -- 172.21.15.7:0/956278230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f240043f0 con 0x55772387dda0 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f329b0740 10 client.15915 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f329b0740 1 -- 172.21.15.7:0/956278230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5577234b70f0 con 0x55772387dda0 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f117fa640 1 -- 172.21.15.7:0/956278230 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f240035f0 con 0x55772387dda0 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f117fa640 1 -- 172.21.15.7:0/956278230 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2f24027020 con 0x55772387dda0 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f329b0740 10 client.15915 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f329b0740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f329b0740 20 client.15915 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f329b0740 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557723893e00 0x557723896260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f329b0740 4 client.15915 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.196+0000 7f2f329b0740 1 -- 172.21.15.7:0/956278230 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f20032c90 con 0x557723893e00 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.197+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557723893e00 0x557723896260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.197+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557723893e00 0x557723896260 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d61b0aedbac51a server_cookie=4d12bec55f0b8b3f in_seq=0 out_seq=0 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.197+0000 7f2f117fa640 10 client.15915 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f117fa640 1 -- 172.21.15.7:0/956278230 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f2f20032c90 con 0x557723893e00 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f117fa640 10 client.15915 handle_command_reply: tid=0 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 client.15915 shutdown 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 -- 172.21.15.7:0/956278230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387dda0 msgr2=0x557723888200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387dda0 0x557723888200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f24012df0 tx=0x7f2f24004730 comp rx=0 tx=0).stop 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 -- 172.21.15.7:0/956278230 shutdown_connections 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557723893e00 0x557723896260 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 0x55772388f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387e790 0x557723888740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 --2- 172.21.15.7:0/956278230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387dda0 0x557723888200 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 -- 172.21.15.7:0/956278230 >> 172.21.15.7:0/956278230 conn(0x557723879310 msgr2=0x557723878530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f117fa640 0 client.15915 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.198+0000 7f2f329b0740 1 -- 172.21.15.7:0/956278230 shutdown_connections 2024-03-26T08:44:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.199+0000 7f2f329b0740 1 -- 172.21.15.7:0/956278230 wait complete. 2024-03-26T08:44:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.199+0000 7f2f329b0740 20 client.15915 trim_cache size 0 max 16384 2024-03-26T08:44:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f329b0740 1 Processor -- start 2024-03-26T08:44:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f329b0740 1 -- start start 2024-03-26T08:44:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723880130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 0x55772388f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f329b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281b9fd0 con 0x55772387e790 2024-03-26T08:44:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f329b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281b7ad0 con 0x55772387dda0 2024-03-26T08:44:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f329b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281b5b50 con 0x557723883900 2024-03-26T08:44:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2d28c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723880130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2d28c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723880130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55482/0 (socket says 172.21.15.7:55482) 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2d28c640 1 -- 172.21.15.7:0/2531477583 learned_addr learned my addr 172.21.15.7:0/2531477583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/2531477583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2e28e640 1 --2- 172.21.15.7:0/2531477583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 0x55772388f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2d28c640 1 -- 172.21.15.7:0/2531477583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 msgr2=0x55772388f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/2531477583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 0x55772388f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2d28c640 1 -- 172.21.15.7:0/2531477583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 msgr2=0x55772387fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/2531477583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.220+0000 7f2f2d28c640 1 -- 172.21.15.7:0/2531477583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557723886bb0 con 0x55772387e790 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.221+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/2531477583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723880130 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f18097c70 tx=0x7f2f180cc840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a241a8b180504b6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.221+0000 7f2f137fe640 1 -- 172.21.15.7:0/2531477583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1809b5e0 con 0x55772387e790 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.221+0000 7f2f137fe640 1 -- 172.21.15.7:0/2531477583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f1809bf10 con 0x55772387e790 2024-03-26T08:44:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.221+0000 7f2f137fe640 1 -- 172.21.15.7:0/2531477583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1809b170 con 0x55772387e790 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.221+0000 7f2f329b0740 1 -- 172.21.15.7:0/2531477583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 msgr2=0x557723880130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.221+0000 7f2f329b0740 1 --2- 172.21.15.7:0/2531477583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723880130 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f18097c70 tx=0x7f2f180cc840 comp rx=0 tx=0).stop 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.222+0000 7f2f329b0740 1 -- 172.21.15.7:0/2531477583 shutdown_connections 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.222+0000 7f2f329b0740 1 --2- 172.21.15.7:0/2531477583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557723883900 0x55772388f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.222+0000 7f2f329b0740 1 --2- 172.21.15.7:0/2531477583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x557723880130 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.222+0000 7f2f329b0740 1 --2- 172.21.15.7:0/2531477583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55772387dda0 0x55772387fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.222+0000 7f2f329b0740 1 -- 172.21.15.7:0/2531477583 >> 172.21.15.7:0/2531477583 conn(0x557723879310 msgr2=0x557723895e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.222+0000 7f2f329b0740 1 -- 172.21.15.7:0/2531477583 shutdown_connections 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.222+0000 7f2f329b0740 1 -- 172.21.15.7:0/2531477583 wait complete. 2024-03-26T08:44:25.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.222+0000 7f2f329b0740 1 -- start start 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.222+0000 7f2f329b0740 1 Processor -- start 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387dda0 0x55772388d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x55772388d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f329b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 0x55772388e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f329b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281b6150 con 0x55772387e790 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f329b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281b7470 con 0x557723883900 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f329b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f281b67b0 con 0x55772387dda0 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f329b0740 10 client.0 _ll_register_callbacks cb 0x7f2f24009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f2da8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387dda0 0x55772388d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f2da8d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387dda0 0x55772388d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34590/0 (socket says 172.21.15.7:34590) 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f2da8d640 1 -- 172.21.15.7:0/3050630288 learned_addr learned my addr 172.21.15.7:0/3050630288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x55772388d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f2da8d640 1 -- 172.21.15.7:0/3050630288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 msgr2=0x55772388e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.223+0000 7f2f2e28e640 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 0x55772388e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 0x55772388e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f2da8d640 1 -- 172.21.15.7:0/3050630288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 msgr2=0x55772388d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x55772388d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f2da8d640 1 -- 172.21.15.7:0/3050630288 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55772388ebc0 con 0x55772387dda0 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f2e28e640 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 0x55772388e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x55772388d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f2da8d640 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387dda0 0x55772388d1e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2f24012dc0 tx=0x7f2f24012620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38400ec50a13d0f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f117fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f117fa640 1 -- 172.21.15.7:0/3050630288 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24029a10 con 0x55772387dda0 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f329b0740 1 -- 172.21.15.7:0/3050630288 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f1809bf10 con 0x55772387dda0 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f117fa640 1 -- 172.21.15.7:0/3050630288 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f24029bb0 con 0x55772387dda0 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f117fa640 1 -- 172.21.15.7:0/3050630288 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f24029db0 con 0x55772387dda0 2024-03-26T08:44:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f329b0740 10 client.25783 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.224+0000 7f2f329b0740 1 -- 172.21.15.7:0/3050630288 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5577238ef920 con 0x55772387dda0 2024-03-26T08:44:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.225+0000 7f2f117fa640 1 -- 172.21.15.7:0/3050630288 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f1809bf10 con 0x55772387dda0 2024-03-26T08:44:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.225+0000 7f2f117fa640 1 -- 172.21.15.7:0/3050630288 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f2f24027070 con 0x55772387dda0 2024-03-26T08:44:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.225+0000 7f2f329b0740 10 client.25783 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.225+0000 7f2f329b0740 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.225+0000 7f2f329b0740 20 client.25783 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.225+0000 7f2f329b0740 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5577238f01b0 0x5577238f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.225+0000 7f2f329b0740 4 client.25783 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.225+0000 7f2f329b0740 1 -- 172.21.15.7:0/3050630288 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577237f0910 con 0x5577238f01b0 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5577238f01b0 0x5577238f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f2d28c640 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5577238f01b0 0x5577238f25c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b5a875a9b6fc38d server_cookie=17eae9119f2badbe in_seq=0 out_seq=0 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f117fa640 10 client.25783 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f117fa640 1 -- 172.21.15.7:0/3050630288 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5577237f0910 con 0x5577238f01b0 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f117fa640 10 client.25783 handle_command_reply: tid=0 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 client.25783 shutdown 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 -- 172.21.15.7:0/3050630288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387dda0 msgr2=0x55772388d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387dda0 0x55772388d1e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2f24012dc0 tx=0x7f2f24012620 comp rx=0 tx=0).stop 2024-03-26T08:44:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 -- 172.21.15.7:0/3050630288 shutdown_connections 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5577238f01b0 0x5577238f25c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557723883900 0x55772388e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55772387e790 0x55772388d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 --2- 172.21.15.7:0/3050630288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55772387dda0 0x55772388d1e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f117fa640 0 client.25783 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 -- 172.21.15.7:0/3050630288 >> 172.21.15.7:0/3050630288 conn(0x557723879310 msgr2=0x55772387fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.226+0000 7f2f329b0740 1 -- 172.21.15.7:0/3050630288 shutdown_connections 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f329b0740 1 -- 172.21.15.7:0/3050630288 wait complete. 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f329b0740 20 client.25783 trim_cache size 0 max 16384 2024-03-26T08:44:25.222 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:44:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:44:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:44:25.223 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f140a2800 msgr2=0x7f2f140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f140a2800 0x7f2f140a4cc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009fd0 tx=0x7f2f24005f90 comp rx=0 tx=0).stop 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28104a70 msgr2=0x7f2f28195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28104a70 0x7f2f28195d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800e9c0 tx=0x7f2f1800ee80 comp rx=0 tx=0).stop 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 shutdown_connections 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f140a2800 0x7f2f140a4cc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f281073a0 0x7f2f2819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f28104a70 0x7f2f28195d60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 --2- 172.21.15.7:0/2321076576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f28102140 0x7f2f28195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 >> 172.21.15.7:0/2321076576 conn(0x7f2f280fbb30 msgr2=0x7f2f281018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 shutdown_connections 2024-03-26T08:44:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.227+0000 7f2f2fcf1640 1 -- 172.21.15.7:0/2321076576 wait complete. 2024-03-26T08:44:25.232 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:44:25.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:44:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 -- 172.21.15.7:0/1523190636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d4108ed0 msgr2=0x7f52d4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 --2- 172.21.15.7:0/1523190636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d4108ed0 0x7f52d4109350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009900 tx=0x7f52c402bc30 comp rx=0 tx=0).stop 2024-03-26T08:44:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 -- 172.21.15.7:0/1523190636 shutdown_connections 2024-03-26T08:44:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 --2- 172.21.15.7:0/1523190636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52d4109890 0x7f52d4110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 --2- 172.21.15.7:0/1523190636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d4108ed0 0x7f52d4109350 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 --2- 172.21.15.7:0/1523190636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d4107ce0 0x7f52d4108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 -- 172.21.15.7:0/1523190636 >> 172.21.15.7:0/1523190636 conn(0x7f52d4076c60 msgr2=0x7f52d40790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 -- 172.21.15.7:0/1523190636 shutdown_connections 2024-03-26T08:44:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 -- 172.21.15.7:0/1523190636 wait complete. 2024-03-26T08:44:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.382+0000 7f52d99d4640 1 Processor -- start 2024-03-26T08:44:25.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.383+0000 7f52d99d4640 1 -- start start 2024-03-26T08:44:25.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.383+0000 7f52d99d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d4107ce0 0x7f52d419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.383+0000 7f52d99d4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d4108ed0 0x7f52d419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.383+0000 7f52d99d4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52d4109890 0x7f52d41a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d99d4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41139b0 con 0x7f52d4108ed0 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d99d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4113830 con 0x7f52d4107ce0 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d99d4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4113b30 con 0x7f52d4109890 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d4108ed0 0x7f52d419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d4108ed0 0x7f52d419e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55518/0 (socket says 172.21.15.7:55518) 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d27fc640 1 -- 172.21.15.7:0/4177194455 learned_addr learned my addr 172.21.15.7:0/4177194455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d2ffd640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d4107ce0 0x7f52d419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d37fe640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52d4109890 0x7f52d41a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d2ffd640 1 -- 172.21.15.7:0/4177194455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52d4109890 msgr2=0x7f52d41a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d2ffd640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52d4109890 0x7f52d41a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d2ffd640 1 -- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d4108ed0 msgr2=0x7f52d419e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d2ffd640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d4108ed0 0x7f52d419e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d2ffd640 1 -- 172.21.15.7:0/4177194455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d41a5bf0 con 0x7f52d4107ce0 2024-03-26T08:44:25.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d27fc640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d4108ed0 0x7f52d419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.384+0000 7f52d37fe640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52d4109890 0x7f52d41a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.385+0000 7f52d2ffd640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d4107ce0 0x7f52d419dfd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52c800e9c0 tx=0x7f52c800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.385+0000 7f52d89d2640 1 -- 172.21.15.7:0/4177194455 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c800cd80 con 0x7f52d4107ce0 2024-03-26T08:44:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.385+0000 7f52d89d2640 1 -- 172.21.15.7:0/4177194455 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52c8010910 con 0x7f52d4107ce0 2024-03-26T08:44:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.385+0000 7f52d89d2640 1 -- 172.21.15.7:0/4177194455 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8018980 con 0x7f52d4107ce0 2024-03-26T08:44:25.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.385+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d41a5ee0 con 0x7f52d4107ce0 2024-03-26T08:44:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.385+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d41a62f0 con 0x7f52d4107ce0 2024-03-26T08:44:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.387+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d410f000 con 0x7f52d4107ce0 2024-03-26T08:44:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.388+0000 7f52d89d2640 1 -- 172.21.15.7:0/4177194455 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f52c801f020 con 0x7f52d4107ce0 2024-03-26T08:44:25.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.388+0000 7f52d89d2640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f52bc0a2880 0x7f52bc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.389+0000 7f52d89d2640 1 -- 172.21.15.7:0/4177194455 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f52c80ce9d0 con 0x7f52d4107ce0 2024-03-26T08:44:25.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.389+0000 7f52d27fc640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f52bc0a2880 0x7f52bc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.390+0000 7f52d27fc640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f52bc0a2880 0x7f52bc0a4d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52c402c8e0 tx=0x7f52c402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.393+0000 7f52d89d2640 1 -- 172.21.15.7:0/4177194455 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52c809ba30 con 0x7f52d4107ce0 2024-03-26T08:44:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.750+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52d402d070 con 0x7f52d4107ce0 2024-03-26T08:44:25.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.752+0000 7f52d89d2640 1 -- 172.21.15.7:0/4177194455 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11358 (secure 0 0 0) 0x7f52c809b330 con 0x7f52d4107ce0 2024-03-26T08:44:25.743 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:44:25.743 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:42:21.061940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24502,24512,24478]},"id":1}]} 2024-03-26T08:44:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-03-26T08:44:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f52bc0a2880 msgr2=0x7f52bc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f52bc0a2880 0x7f52bc0a4d40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52c402c8e0 tx=0x7f52c402df90 comp rx=0 tx=0).stop 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d4107ce0 msgr2=0x7f52d419dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d4107ce0 0x7f52d419dfd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52c800e9c0 tx=0x7f52c800ee80 comp rx=0 tx=0).stop 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 shutdown_connections 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f52bc0a2880 0x7f52bc0a4d40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52d4109890 0x7f52d41a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d4108ed0 0x7f52d419e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 --2- 172.21.15.7:0/4177194455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52d4107ce0 0x7f52d419dfd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 >> 172.21.15.7:0/4177194455 conn(0x7f52d4076c60 msgr2=0x7f52d410c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 shutdown_connections 2024-03-26T08:44:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.755+0000 7f52d99d4640 1 -- 172.21.15.7:0/4177194455 wait complete. 2024-03-26T08:44:25.763 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:44:25.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:44:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.940+0000 7f9e51881640 1 -- 172.21.15.7:0/1764477328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e44093a10 msgr2=0x7f9e44093e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.940+0000 7f9e51881640 1 --2- 172.21.15.7:0/1764477328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e44093a10 0x7f9e44093e30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009860 tx=0x7f9e3c02bb70 comp rx=0 tx=0).stop 2024-03-26T08:44:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.940+0000 7f9e51881640 1 -- 172.21.15.7:0/1764477328 shutdown_connections 2024-03-26T08:44:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.940+0000 7f9e51881640 1 --2- 172.21.15.7:0/1764477328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e4409ba90 0x7f9e4409dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.940+0000 7f9e51881640 1 --2- 172.21.15.7:0/1764477328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e44094400 0x7f9e4409b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.940+0000 7f9e51881640 1 --2- 172.21.15.7:0/1764477328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e44093a10 0x7f9e44093e30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.940+0000 7f9e51881640 1 -- 172.21.15.7:0/1764477328 >> 172.21.15.7:0/1764477328 conn(0x7f9e4408f540 msgr2=0x7f9e440919a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.941+0000 7f9e51881640 1 -- 172.21.15.7:0/1764477328 shutdown_connections 2024-03-26T08:44:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.941+0000 7f9e51881640 1 -- 172.21.15.7:0/1764477328 wait complete. 2024-03-26T08:44:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.941+0000 7f9e51881640 1 Processor -- start 2024-03-26T08:44:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e51881640 1 -- start start 2024-03-26T08:44:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e51881640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e44093a10 0x7f9e44096ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e51881640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e44094400 0x7f9e44095500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e51881640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e4409ba90 0x7f9e44095a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e51881640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440a1a00 con 0x7f9e44094400 2024-03-26T08:44:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e51881640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440a1880 con 0x7f9e4409ba90 2024-03-26T08:44:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e51881640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440a1b80 con 0x7f9e44093a10 2024-03-26T08:44:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e44093a10 0x7f9e44096ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e44093a10 0x7f9e44096ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34626/0 (socket says 172.21.15.7:34626) 2024-03-26T08:44:25.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.942+0000 7f9e4affd640 1 -- 172.21.15.7:0/849629130 learned_addr learned my addr 172.21.15.7:0/849629130 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.943+0000 7f9e4b7fe640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e4409ba90 0x7f9e44095a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.943+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e44094400 0x7f9e44095500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.943+0000 7f9e4b7fe640 1 -- 172.21.15.7:0/849629130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e44093a10 msgr2=0x7f9e44096ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.943+0000 7f9e4b7fe640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e44093a10 0x7f9e44096ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.943+0000 7f9e4b7fe640 1 -- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e44094400 msgr2=0x7f9e44095500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.943+0000 7f9e4b7fe640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e44094400 0x7f9e44095500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.943+0000 7f9e4b7fe640 1 -- 172.21.15.7:0/849629130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e44130020 con 0x7f9e4409ba90 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.943+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e44094400 0x7f9e44095500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.943+0000 7f9e4b7fe640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e4409ba90 0x7f9e44095a40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c067bb0 tx=0x7f9e4c07aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.944+0000 7f9e5087f640 1 -- 172.21.15.7:0/849629130 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c084070 con 0x7f9e4409ba90 2024-03-26T08:44:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.944+0000 7f9e51881640 1 -- 172.21.15.7:0/849629130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e441302b0 con 0x7f9e4409ba90 2024-03-26T08:44:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.944+0000 7f9e51881640 1 -- 172.21.15.7:0/849629130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e44130780 con 0x7f9e4409ba90 2024-03-26T08:44:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.944+0000 7f9e5087f640 1 -- 172.21.15.7:0/849629130 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c078d60 con 0x7f9e4409ba90 2024-03-26T08:44:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.944+0000 7f9e5087f640 1 -- 172.21.15.7:0/849629130 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0626d0 con 0x7f9e4409ba90 2024-03-26T08:44:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.947+0000 7f9e5087f640 1 -- 172.21.15.7:0/849629130 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9e4c07e050 con 0x7f9e4409ba90 2024-03-26T08:44:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.947+0000 7f9e5446e740 1 Processor -- start 2024-03-26T08:44:25.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e5087f640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e340a2880 0x7f9e340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e5446e740 1 -- start start 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e5087f640 1 -- 172.21.15.7:0/849629130 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9e4c139990 con 0x7f9e4409ba90 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e340a2880 0x7f9e340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 0x5632107a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e5446e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563210767610 con 0x5632107a5250 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e5446e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56321070a3c0 con 0x5632107aa3c0 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e5446e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632106b2d60 con 0x5632107a4860 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e4a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 0x5632107a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e4a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 0x5632107a9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55554/0 (socket says 172.21.15.7:55554) 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e4a7fc640 1 -- 172.21.15.7:0/22422490 learned_addr learned my addr 172.21.15.7:0/22422490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e4affd640 1 --2- 172.21.15.7:0/22422490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.949+0000 7f9e4b7fe640 1 --2- 172.21.15.7:0/22422490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.950+0000 7f9e4affd640 1 -- 172.21.15.7:0/22422490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 msgr2=0x5632107ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.950+0000 7f9e4affd640 1 --2- 172.21.15.7:0/22422490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.950+0000 7f9e4affd640 1 -- 172.21.15.7:0/22422490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 msgr2=0x5632107a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.950+0000 7f9e4affd640 1 --2- 172.21.15.7:0/22422490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 0x5632107a9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.950+0000 7f9e4affd640 1 -- 172.21.15.7:0/22422490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56321041e3f0 con 0x5632107a4860 2024-03-26T08:44:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.950+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e340a2880 0x7f9e340a4d40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009fd0 tx=0x7f9e400138e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.950+0000 7f9e4affd640 1 --2- 172.21.15.7:0/22422490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107a4c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c00 tx=0x7f9e3c005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72910c93575edae6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.950+0000 7f9e31ffb640 1 -- 172.21.15.7:0/22422490 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c039040 con 0x5632107a4860 2024-03-26T08:44:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.950+0000 7f9e31ffb640 1 -- 172.21.15.7:0/22422490 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e3c043070 con 0x5632107a4860 2024-03-26T08:44:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.951+0000 7f9e31ffb640 1 -- 172.21.15.7:0/22422490 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0052a0 con 0x5632107a4860 2024-03-26T08:44:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.952+0000 7f9e5446e740 1 -- 172.21.15.7:0/22422490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 msgr2=0x5632107a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.952+0000 7f9e5446e740 1 --2- 172.21.15.7:0/22422490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107a4c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c00 tx=0x7f9e3c005da0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.952+0000 7f9e5446e740 1 -- 172.21.15.7:0/22422490 shutdown_connections 2024-03-26T08:44:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.952+0000 7f9e5446e740 1 --2- 172.21.15.7:0/22422490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.952+0000 7f9e5446e740 1 --2- 172.21.15.7:0/22422490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 0x5632107a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.952+0000 7f9e5446e740 1 --2- 172.21.15.7:0/22422490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107a4c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.952+0000 7f9e5446e740 1 -- 172.21.15.7:0/22422490 >> 172.21.15.7:0/22422490 conn(0x56321079fdd0 msgr2=0x5632107a01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.952+0000 7f9e5446e740 1 -- 172.21.15.7:0/22422490 shutdown_connections 2024-03-26T08:44:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.952+0000 7f9e5446e740 1 -- 172.21.15.7:0/22422490 wait complete. 2024-03-26T08:44:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.953+0000 7f9e5446e740 1 -- start start 2024-03-26T08:44:25.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.954+0000 7f9e5446e740 1 Processor -- start 2024-03-26T08:44:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.954+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.954+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 0x5632107af110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.954+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e5446e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632106c6240 con 0x5632107a5250 2024-03-26T08:44:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e5446e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563210706460 con 0x5632107aa3c0 2024-03-26T08:44:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e5446e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632107ad310 con 0x5632107a4860 2024-03-26T08:44:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e5446e740 10 client.0 _ll_register_callbacks cb 0x5632106c5c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34644/0 (socket says 172.21.15.7:34644) 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4affd640 1 -- 172.21.15.7:0/2726561789 learned_addr learned my addr 172.21.15.7:0/2726561789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 0x5632107af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4b7fe640 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4affd640 1 -- 172.21.15.7:0/2726561789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 msgr2=0x5632107b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4affd640 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4affd640 1 -- 172.21.15.7:0/2726561789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 msgr2=0x5632107af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4affd640 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 0x5632107af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.955+0000 7f9e4affd640 1 -- 172.21.15.7:0/2726561789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632104e87d0 con 0x5632107a4860 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.956+0000 7f9e4affd640 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107aebd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c02f020 tx=0x7f9e3c0323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92a36e8b04d17723 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.956+0000 7f9e2affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.956+0000 7f9e2affd640 1 -- 172.21.15.7:0/2726561789 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c04b070 con 0x5632107a4860 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.956+0000 7f9e2affd640 1 -- 172.21.15.7:0/2726561789 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9e3c034640 con 0x5632107a4860 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.956+0000 7f9e5446e740 1 -- 172.21.15.7:0/2726561789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e3c043070 con 0x5632107a4860 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.957+0000 7f9e2affd640 1 -- 172.21.15.7:0/2726561789 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e3c039070 con 0x5632107a4860 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.957+0000 7f9e5446e740 10 client.25801 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.957+0000 7f9e5446e740 1 -- 172.21.15.7:0/2726561789 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5632107b94c0 con 0x5632107a4860 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.957+0000 7f9e2affd640 1 -- 172.21.15.7:0/2726561789 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e3c043070 con 0x5632107a4860 2024-03-26T08:44:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.957+0000 7f9e2affd640 1 -- 172.21.15.7:0/2726561789 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f9e3c02ca10 con 0x5632107a4860 2024-03-26T08:44:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.957+0000 7f9e5446e740 10 client.25801 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.957+0000 7f9e5446e740 10 client.25801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:44:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.957+0000 7f9e5446e740 20 client.25801 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.957+0000 7f9e5446e740 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5632107ba810 0x5632107bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.958+0000 7f9e5446e740 4 client.25801 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.958+0000 7f9e5446e740 1 -- 172.21.15.7:0/2726561789 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563210717900 con 0x5632107ba810 2024-03-26T08:44:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.958+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5632107ba810 0x5632107bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.960+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5632107ba810 0x5632107bcc00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8110b843dd221df1 server_cookie=c12360456e80798b in_seq=0 out_seq=0 2024-03-26T08:44:25.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.960+0000 7f9e2affd640 10 client.25801 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.961+0000 7f9e2affd640 1 -- 172.21.15.7:0/2726561789 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563210717900 con 0x5632107ba810 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.961+0000 7f9e2affd640 10 client.25801 handle_command_reply: tid=0 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.961+0000 7f9e5446e740 1 client.25801 shutdown 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 -- 172.21.15.7:0/2726561789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 msgr2=0x5632107aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107aebd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c02f020 tx=0x7f9e3c0323c0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 -- 172.21.15.7:0/2726561789 shutdown_connections 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5632107ba810 0x5632107bcc00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a5250 0x5632107af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 --2- 172.21.15.7:0/2726561789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107aebd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 -- 172.21.15.7:0/2726561789 >> 172.21.15.7:0/2726561789 conn(0x56321079fdd0 msgr2=0x56321079f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e2affd640 0 client.25801 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:44:25.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 -- 172.21.15.7:0/2726561789 shutdown_connections 2024-03-26T08:44:25.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 1 -- 172.21.15.7:0/2726561789 wait complete. 2024-03-26T08:44:25.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.962+0000 7f9e5446e740 20 client.25801 trim_cache size 0 max 16384 2024-03-26T08:44:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.982+0000 7f9e5446e740 1 Processor -- start 2024-03-26T08:44:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e5446e740 1 -- start start 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a4860 0x5632107a3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a5250 0x5632107b6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e5446e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e38010980 con 0x5632107a4860 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e5446e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3800e480 con 0x5632107aa3c0 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a4860 0x5632107a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107c1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42526/0 (socket says 172.21.15.7:42526) 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e4b7fe640 1 -- 172.21.15.7:0/1563220723 learned_addr learned my addr 172.21.15.7:0/1563220723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/1563220723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a5250 0x5632107b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.983+0000 7f9e5446e740 1 -- 172.21.15.7:0/1563220723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3800c500 con 0x5632107a5250 2024-03-26T08:44:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.984+0000 7f9e4affd640 1 -- 172.21.15.7:0/1563220723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a5250 msgr2=0x5632107b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.984+0000 7f9e4affd640 1 --2- 172.21.15.7:0/1563220723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a5250 0x5632107b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.984+0000 7f9e4affd640 1 -- 172.21.15.7:0/1563220723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 msgr2=0x5632107c1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.984+0000 7f9e4affd640 1 --2- 172.21.15.7:0/1563220723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107c1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.984+0000 7f9e4affd640 1 -- 172.21.15.7:0/1563220723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632107a0820 con 0x5632107a4860 2024-03-26T08:44:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.984+0000 7f9e4affd640 1 --2- 172.21.15.7:0/1563220723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a4860 0x5632107a3740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005ad0 tx=0x7f9e3c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50f65fa97a66701 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.985+0000 7f9e317fa640 1 -- 172.21.15.7:0/1563220723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c04b040 con 0x5632107a4860 2024-03-26T08:44:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.985+0000 7f9e317fa640 1 -- 172.21.15.7:0/1563220723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e3c043070 con 0x5632107a4860 2024-03-26T08:44:25.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.985+0000 7f9e317fa640 1 -- 172.21.15.7:0/1563220723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c02cad0 con 0x5632107a4860 2024-03-26T08:44:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.986+0000 7f9e5446e740 1 -- 172.21.15.7:0/1563220723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a4860 msgr2=0x5632107a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.986+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1563220723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a4860 0x5632107a3740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c005ad0 tx=0x7f9e3c009520 comp rx=0 tx=0).stop 2024-03-26T08:44:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.986+0000 7f9e5446e740 1 -- 172.21.15.7:0/1563220723 shutdown_connections 2024-03-26T08:44:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.986+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1563220723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107aa3c0 0x5632107c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.986+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1563220723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a5250 0x5632107b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.986+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1563220723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107a4860 0x5632107a3740 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.986+0000 7f9e5446e740 1 -- 172.21.15.7:0/1563220723 >> 172.21.15.7:0/1563220723 conn(0x56321079fdd0 msgr2=0x5632107a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.987+0000 7f9e5446e740 1 -- 172.21.15.7:0/1563220723 shutdown_connections 2024-03-26T08:44:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.988+0000 7f9e5446e740 1 -- 172.21.15.7:0/1563220723 wait complete. 2024-03-26T08:44:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.988+0000 7f9e5446e740 1 -- start start 2024-03-26T08:44:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.989+0000 7f9e5446e740 1 Processor -- start 2024-03-26T08:44:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.989+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.989+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107a5250 0x5632107b4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e5446e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107aa3c0 0x5632107b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e4affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34660/0 (socket says 172.21.15.7:34660) 2024-03-26T08:44:25.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e4affd640 1 -- 172.21.15.7:0/1091597228 learned_addr learned my addr 172.21.15.7:0/1091597228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:25.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3800cb00 con 0x5632107aa3c0 2024-03-26T08:44:25.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3800de20 con 0x5632107a5250 2024-03-26T08:44:25.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107a5250 0x5632107b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3800d160 con 0x5632107a4860 2024-03-26T08:44:25.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e4b7fe640 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107aa3c0 0x5632107b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.990+0000 7f9e5446e740 10 client.0 _ll_register_callbacks cb 0x7f9e4c0679e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:25.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.991+0000 7f9e4affd640 1 -- 172.21.15.7:0/1091597228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107a5250 msgr2=0x5632107b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.991+0000 7f9e4affd640 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107a5250 0x5632107b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.991+0000 7f9e4affd640 1 -- 172.21.15.7:0/1091597228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107aa3c0 msgr2=0x5632107b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.991+0000 7f9e4affd640 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107aa3c0 0x5632107b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.991+0000 7f9e4affd640 1 -- 172.21.15.7:0/1091597228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632107b56b0 con 0x5632107a4860 2024-03-26T08:44:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.991+0000 7f9e4b7fe640 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107aa3c0 0x5632107b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.991+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107a5250 0x5632107b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.992+0000 7f9e4affd640 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107b3c10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c004710 tx=0x7f9e3c005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=670b528a14842bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.992+0000 7f9e2affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.992+0000 7f9e2affd640 1 -- 172.21.15.7:0/1091597228 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c05f030 con 0x5632107a4860 2024-03-26T08:44:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.992+0000 7f9e2affd640 1 -- 172.21.15.7:0/1091597228 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9e3c034440 con 0x5632107a4860 2024-03-26T08:44:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.992+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e3c043070 con 0x5632107a4860 2024-03-26T08:44:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.993+0000 7f9e2affd640 1 -- 172.21.15.7:0/1091597228 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e3c04b040 con 0x5632107a4860 2024-03-26T08:44:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.993+0000 7f9e5446e740 10 client.25807 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.993+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563210816220 con 0x5632107a4860 2024-03-26T08:44:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.993+0000 7f9e2affd640 1 -- 172.21.15.7:0/1091597228 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e3c043070 con 0x5632107a4860 2024-03-26T08:44:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.993+0000 7f9e2affd640 1 -- 172.21.15.7:0/1091597228 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f9e3c05a070 con 0x5632107a4860 2024-03-26T08:44:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.993+0000 7f9e5446e740 10 client.25807 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.993+0000 7f9e5446e740 10 client.25807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:44:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.994+0000 7f9e5446e740 20 client.25807 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.994+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563210816920 0x563210818d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.994+0000 7f9e5446e740 4 client.25807 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:44:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.994+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563210711c50 con 0x563210816920 2024-03-26T08:44:25.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.994+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563210816920 0x563210818d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:25.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.995+0000 7f9e2affd640 10 client.25807 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:44:25.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.995+0000 7f9e4a7fc640 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563210816920 0x563210818d10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f646b4c35ef419c4 server_cookie=628404520860e300 in_seq=0 out_seq=0 2024-03-26T08:44:25.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.995+0000 7f9e2affd640 1 -- 172.21.15.7:0/1091597228 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563210711c50 con 0x563210816920 2024-03-26T08:44:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.995+0000 7f9e2affd640 10 client.25807 handle_command_reply: tid=0 2024-03-26T08:44:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.995+0000 7f9e5446e740 1 client.25807 shutdown 2024-03-26T08:44:25.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.998+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 msgr2=0x5632107b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:25.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.998+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107b3c10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c004710 tx=0x7f9e3c005bd0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.999+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 shutdown_connections 2024-03-26T08:44:25.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.999+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563210816920 0x563210818d10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.999+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632107aa3c0 0x5632107b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.999+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632107a5250 0x5632107b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.999+0000 7f9e5446e740 1 --2- 172.21.15.7:0/1091597228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5632107a4860 0x5632107b3c10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:25.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.999+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 >> 172.21.15.7:0/1091597228 conn(0x56321079fdd0 msgr2=0x5632107a38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:25.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:25.999+0000 7f9e2affd640 0 client.25807 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:44:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.003+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 shutdown_connections 2024-03-26T08:44:25.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.003+0000 7f9e5446e740 1 -- 172.21.15.7:0/1091597228 wait complete. 2024-03-26T08:44:25.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.003+0000 7f9e5446e740 20 client.25807 trim_cache size 0 max 16384 2024-03-26T08:44:25.998 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:44:26.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.014+0000 7f9e51881640 1 -- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e340a2880 msgr2=0x7f9e340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.014+0000 7f9e51881640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e340a2880 0x7f9e340a4d40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009fd0 tx=0x7f9e400138e0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.014+0000 7f9e51881640 1 -- 172.21.15.7:0/849629130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e4409ba90 msgr2=0x7f9e44095a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.014+0000 7f9e51881640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e4409ba90 0x7f9e44095a40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c067bb0 tx=0x7f9e4c07aee0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.017+0000 7f9e51881640 1 -- 172.21.15.7:0/849629130 shutdown_connections 2024-03-26T08:44:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.017+0000 7f9e51881640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e340a2880 0x7f9e340a4d40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.017+0000 7f9e51881640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e4409ba90 0x7f9e44095a40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.017+0000 7f9e51881640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e44094400 0x7f9e44095500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.017+0000 7f9e51881640 1 --2- 172.21.15.7:0/849629130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e44093a10 0x7f9e44096ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.017+0000 7f9e51881640 1 -- 172.21.15.7:0/849629130 >> 172.21.15.7:0/849629130 conn(0x7f9e4408f540 msgr2=0x7f9e440901b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.019+0000 7f9e51881640 1 -- 172.21.15.7:0/849629130 shutdown_connections 2024-03-26T08:44:26.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.019+0000 7f9e51881640 1 -- 172.21.15.7:0/849629130 wait complete. 2024-03-26T08:44:26.026 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:44:26.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:44:26.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 115.0 s, 3727.7 tps, lat 26.891 ms stddev 31.440 2024-03-26T08:44:26.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.194+0000 7fa3cf19d640 1 -- 172.21.15.7:0/4098126693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8100d50 msgr2=0x7fa3c81011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.194+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/4098126693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8100d50 0x7fa3c81011d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009900 tx=0x7fa3b802bc30 comp rx=0 tx=0).stop 2024-03-26T08:44:26.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.195+0000 7fa3cf19d640 1 -- 172.21.15.7:0/4098126693 shutdown_connections 2024-03-26T08:44:26.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.195+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/4098126693 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3c8101710 0x7fa3c8107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.195+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/4098126693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8100d50 0x7fa3c81011d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.195+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/4098126693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c80ffb60 0x7fa3c80fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.195+0000 7fa3cf19d640 1 -- 172.21.15.7:0/4098126693 >> 172.21.15.7:0/4098126693 conn(0x7fa3c80fb0d0 msgr2=0x7fa3c80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.195+0000 7fa3cf19d640 1 -- 172.21.15.7:0/4098126693 shutdown_connections 2024-03-26T08:44:26.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.196+0000 7fa3cf19d640 1 -- 172.21.15.7:0/4098126693 wait complete. 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cf19d640 1 Processor -- start 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cf19d640 1 -- start start 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cf19d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3c80ffb60 0x7fa3c8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cf19d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c8100d50 0x7fa3c8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cf19d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8101710 0x7fa3c81a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cf19d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c810afc0 con 0x7fa3c8100d50 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cf19d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c810ae40 con 0x7fa3c8101710 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cf19d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c810b140 con 0x7fa3c80ffb60 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cd99a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c8100d50 0x7fa3c8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cd99a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c8100d50 0x7fa3c8199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55570/0 (socket says 172.21.15.7:55570) 2024-03-26T08:44:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3cd99a640 1 -- 172.21.15.7:0/2287732079 learned_addr learned my addr 172.21.15.7:0/2287732079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3ce19b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3c80ffb60 0x7fa3c8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.197+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8101710 0x7fa3c81a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3ce19b640 1 -- 172.21.15.7:0/2287732079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8101710 msgr2=0x7fa3c81a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8101710 0x7fa3c81a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3ce19b640 1 -- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c8100d50 msgr2=0x7fa3c8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c8100d50 0x7fa3c8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3ce19b640 1 -- 172.21.15.7:0/2287732079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c81a1650 con 0x7fa3c80ffb60 2024-03-26T08:44:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3cd99a640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c8100d50 0x7fa3c8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:26.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3c80ffb60 0x7fa3c8199a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3bf7fe640 1 -- 172.21.15.7:0/2287732079 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4003970 con 0x7fa3c80ffb60 2024-03-26T08:44:26.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3bf7fe640 1 -- 172.21.15.7:0/2287732079 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c4004030 con 0x7fa3c80ffb60 2024-03-26T08:44:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3cf19d640 1 -- 172.21.15.7:0/2287732079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c81a1940 con 0x7fa3c80ffb60 2024-03-26T08:44:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.198+0000 7fa3cf19d640 1 -- 172.21.15.7:0/2287732079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c81a1e50 con 0x7fa3c80ffb60 2024-03-26T08:44:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.199+0000 7fa3bf7fe640 1 -- 172.21.15.7:0/2287732079 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c40107f0 con 0x7fa3c80ffb60 2024-03-26T08:44:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.203+0000 7fa3d2fef740 1 Processor -- start 2024-03-26T08:44:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.203+0000 7fa3d2fef740 1 -- start start 2024-03-26T08:44:26.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.203+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.203+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.204+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089db3a0 0x5588089dd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.204+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558808998610 con 0x5588089db3a0 2024-03-26T08:44:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.204+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880893b3c0 con 0x5588089d6230 2024-03-26T08:44:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.204+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588088e3d60 con 0x5588089d5840 2024-03-26T08:44:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.204+0000 7fa3bf7fe640 1 -- 172.21.15.7:0/2287732079 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa3c4010990 con 0x7fa3c80ffb60 2024-03-26T08:44:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.205+0000 7fa3ce99c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.205+0000 7fa3ce99c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089d5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34698/0 (socket says 172.21.15.7:34698) 2024-03-26T08:44:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.205+0000 7fa3ce99c640 1 -- 172.21.15.7:0/4141640342 learned_addr learned my addr 172.21.15.7:0/4141640342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.205+0000 7fa3bf7fe640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa3b40a2900 0x7fa3b40a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.206+0000 7fa3bf7fe640 1 -- 172.21.15.7:0/2287732079 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3c40cf1b0 con 0x7fa3c80ffb60 2024-03-26T08:44:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.209+0000 7fa3ce99c640 1 -- 172.21.15.7:0/4141640342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 msgr2=0x5588089dad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.209+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/4141640342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.209+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/4141640342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089dad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.209+0000 7fa3ce99c640 1 -- 172.21.15.7:0/4141640342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089db3a0 msgr2=0x5588089dd830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:44:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.209+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/4141640342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089db3a0 0x5588089dd830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.209+0000 7fa3ce99c640 1 -- 172.21.15.7:0/4141640342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55880864f3f0 con 0x5588089d5840 2024-03-26T08:44:26.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.209+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa3b40a2900 0x7fa3b40a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.213+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/4141640342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089d5c60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ae20 tx=0x7fa3c000e820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59743194d0f6f66d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.215+0000 7fa3bcbfe640 1 -- 172.21.15.7:0/4141640342 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0024400 con 0x5588089d5840 2024-03-26T08:44:26.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.215+0000 7fa3bcbfe640 1 -- 172.21.15.7:0/4141640342 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c001a8d0 con 0x5588089d5840 2024-03-26T08:44:26.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.215+0000 7fa3bcbfe640 1 -- 172.21.15.7:0/4141640342 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c001b6e0 con 0x5588089d5840 2024-03-26T08:44:26.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.216+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa3b40a2900 0x7fa3b40a4dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa3c001b000 tx=0x7fa3c001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.218+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4141640342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 msgr2=0x5588089d5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.218+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4141640342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089d5c60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ae20 tx=0x7fa3c000e820 comp rx=0 tx=0).stop 2024-03-26T08:44:26.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.219+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4141640342 shutdown_connections 2024-03-26T08:44:26.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.219+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4141640342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089db3a0 0x5588089dd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.219+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4141640342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089dad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.219+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4141640342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089d5c60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.219+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4141640342 >> 172.21.15.7:0/4141640342 conn(0x5588089d0db0 msgr2=0x5588089d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.223+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4141640342 shutdown_connections 2024-03-26T08:44:26.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.223+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4141640342 wait complete. 2024-03-26T08:44:26.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.224+0000 7fa3d2fef740 1 -- start start 2024-03-26T08:44:26.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.224+0000 7fa3d2fef740 1 Processor -- start 2024-03-26T08:44:26.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.224+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089e8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.224+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.224+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089db3a0 0x5588089ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.224+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588088f7240 con 0x5588089d5840 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.224+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558808937460 con 0x5588089d6230 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.224+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588089e6b60 con 0x5588089db3a0 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.224+0000 7fa3d2fef740 10 client.0 _ll_register_callbacks cb 0x5588088f6c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3ce19b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3ce99c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089db3a0 0x5588089ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3cd99a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3ce19b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089e8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55582/0 (socket says 172.21.15.7:55582) 2024-03-26T08:44:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3ce19b640 1 -- 172.21.15.7:0/4172931073 learned_addr learned my addr 172.21.15.7:0/4172931073 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3cd99a640 1 -- 172.21.15.7:0/4172931073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089db3a0 msgr2=0x5588089ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3cd99a640 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089db3a0 0x5588089ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3cd99a640 1 -- 172.21.15.7:0/4172931073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 msgr2=0x5588089e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3cd99a640 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089e8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3cd99a640 1 -- 172.21.15.7:0/4172931073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588087197d0 con 0x5588089d6230 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089db3a0 0x5588089ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3cd99a640 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089e89a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005ab0 tx=0x7fa3b80048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9235d79dd11e27c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.225+0000 7fa3b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.226+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/4172931073 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8045070 con 0x5588089d6230 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.226+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/4172931073 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3b8004c70 con 0x5588089d6230 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.226+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4172931073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3c001a8d0 con 0x5588089d6230 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.226+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/4172931073 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3b8005d50 con 0x5588089d6230 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.226+0000 7fa3d2fef740 10 client.25796 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.226+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4172931073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558808a03dd0 con 0x5588089d6230 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.226+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/4172931073 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c001a8d0 con 0x5588089d6230 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.226+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/4172931073 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa3b8003680 con 0x5588089d6230 2024-03-26T08:44:26.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.227+0000 7fa3d2fef740 10 client.25796 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.227+0000 7fa3d2fef740 10 client.25796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:44:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.227+0000 7fa3d2fef740 20 client.25796 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.227+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558808a05490 0x558808a078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.227+0000 7fa3d2fef740 4 client.25796 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.227+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4172931073 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558808948900 con 0x558808a05490 2024-03-26T08:44:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.227+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558808a05490 0x558808a078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.227+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558808a05490 0x558808a078b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34b4c45bbcbc8ff5 server_cookie=9d3594445c5e561a in_seq=0 out_seq=0 2024-03-26T08:44:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.227+0000 7fa3b1ffb640 10 client.25796 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:44:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.228+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/4172931073 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558808948900 con 0x558808a05490 2024-03-26T08:44:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.228+0000 7fa3b1ffb640 10 client.25796 handle_command_reply: tid=0 2024-03-26T08:44:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.228+0000 7fa3d2fef740 1 client.25796 shutdown 2024-03-26T08:44:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.229+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4172931073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 msgr2=0x5588089e89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.229+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089e89a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005ab0 tx=0x7fa3b80048f0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.229+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4172931073 shutdown_connections 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.229+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558808a05490 0x558808a078b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.229+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089db3a0 0x5588089ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.229+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089e89a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.229+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/4172931073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.229+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4172931073 >> 172.21.15.7:0/4172931073 conn(0x5588089d0db0 msgr2=0x5588089d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.229+0000 7fa3b1ffb640 0 client.25796 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.230+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4172931073 shutdown_connections 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.230+0000 7fa3d2fef740 1 -- 172.21.15.7:0/4172931073 wait complete. 2024-03-26T08:44:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.230+0000 7fa3d2fef740 20 client.25796 trim_cache size 0 max 16384 2024-03-26T08:44:26.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.250+0000 7fa3d2fef740 1 Processor -- start 2024-03-26T08:44:26.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.250+0000 7fa3d2fef740 1 -- start start 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089db3a0 0x5588089cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac010950 con 0x5588089db3a0 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac00e450 con 0x5588089d6230 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac00c4d0 con 0x5588089d5840 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3ce19b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3cd99a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3ce19b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089e7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34714/0 (socket says 172.21.15.7:34714) 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3ce19b640 1 -- 172.21.15.7:0/1813693521 learned_addr learned my addr 172.21.15.7:0/1813693521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/1813693521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089db3a0 0x5588089cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3cd99a640 1 -- 172.21.15.7:0/1813693521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 msgr2=0x5588089e7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3cd99a640 1 --2- 172.21.15.7:0/1813693521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089e7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3cd99a640 1 -- 172.21.15.7:0/1813693521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089db3a0 msgr2=0x5588089cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3cd99a640 1 --2- 172.21.15.7:0/1813693521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089db3a0 0x5588089cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3cd99a640 1 -- 172.21.15.7:0/1813693521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588089d5480 con 0x5588089d6230 2024-03-26T08:44:26.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.251+0000 7fa3cd99a640 1 --2- 172.21.15.7:0/1813693521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089ef9a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa3b802c8e0 tx=0x7fa3b8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85c48e73bfb89ca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.252+0000 7fa3b3fff640 1 -- 172.21.15.7:0/1813693521 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8000e60 con 0x5588089d6230 2024-03-26T08:44:26.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.252+0000 7fa3b3fff640 1 -- 172.21.15.7:0/1813693521 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3b802cd30 con 0x5588089d6230 2024-03-26T08:44:26.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.252+0000 7fa3b3fff640 1 -- 172.21.15.7:0/1813693521 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8045040 con 0x5588089d6230 2024-03-26T08:44:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.253+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1813693521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 msgr2=0x5588089ef9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.253+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1813693521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089ef9a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa3b802c8e0 tx=0x7fa3b8005fd0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.253+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1813693521 shutdown_connections 2024-03-26T08:44:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.253+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1813693521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089db3a0 0x5588089cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.253+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1813693521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089d6230 0x5588089ef9a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.253+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1813693521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d5840 0x5588089e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.253+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1813693521 >> 172.21.15.7:0/1813693521 conn(0x5588089d0db0 msgr2=0x5588089d77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.253+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1813693521 shutdown_connections 2024-03-26T08:44:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.253+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1813693521 wait complete. 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.254+0000 7fa3d2fef740 1 -- start start 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.254+0000 7fa3d2fef740 1 Processor -- start 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.254+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089ed910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.254+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d6230 0x5588089ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.254+0000 7fa3d2fef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089db3a0 0x5588089eee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.254+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac00cad0 con 0x5588089d5840 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.254+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac00ddf0 con 0x5588089db3a0 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.254+0000 7fa3d2fef740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac00d130 con 0x5588089d6230 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.254+0000 7fa3d2fef740 10 client.0 _ll_register_callbacks cb 0x7fa3b8004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.255+0000 7fa3ce99c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089db3a0 0x5588089eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.255+0000 7fa3ce99c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089db3a0 0x5588089eee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42572/0 (socket says 172.21.15.7:42572) 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.255+0000 7fa3ce99c640 1 -- 172.21.15.7:0/1545704519 learned_addr learned my addr 172.21.15.7:0/1545704519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.255+0000 7fa3cd99a640 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d6230 0x5588089ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.256+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089ed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.256+0000 7fa3ce99c640 1 -- 172.21.15.7:0/1545704519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d6230 msgr2=0x5588089ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.256+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d6230 0x5588089ede50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.256+0000 7fa3ce99c640 1 -- 172.21.15.7:0/1545704519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 msgr2=0x5588089ed910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.256+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089ed910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.256+0000 7fa3ce99c640 1 -- 172.21.15.7:0/1545704519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588089ef400 con 0x5588089db3a0 2024-03-26T08:44:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.256+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089ed910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.256+0000 7fa3ce99c640 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089db3a0 0x5588089eee10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ae20 tx=0x7fa3c0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b380104a6b74f06 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.257+0000 7fa3b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.257+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/1545704519 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c001bd10 con 0x5588089db3a0 2024-03-26T08:44:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.257+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1545704519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3b802cd30 con 0x5588089db3a0 2024-03-26T08:44:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.257+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/1545704519 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3c0030480 con 0x5588089db3a0 2024-03-26T08:44:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.257+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/1545704519 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3c0030680 con 0x5588089db3a0 2024-03-26T08:44:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.257+0000 7fa3d2fef740 10 client.25802 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.257+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1545704519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558808a615b0 con 0x5588089db3a0 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.258+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/1545704519 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3b802cd30 con 0x5588089db3a0 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.258+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/1545704519 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fa3c000b040 con 0x5588089db3a0 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.258+0000 7fa3d2fef740 10 client.25802 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.258+0000 7fa3d2fef740 10 client.25802 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.258+0000 7fa3d2fef740 20 client.25802 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.258+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558808a61d80 0x558808a641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.258+0000 7fa3d2fef740 4 client.25802 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.258+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1545704519 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588089e8250 con 0x558808a61d80 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558808a61d80 0x558808a641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3ce19b640 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558808a61d80 0x558808a641a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae28f8264ed62e77 server_cookie=f84cd71c184d52ec in_seq=0 out_seq=0 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3b1ffb640 10 client.25802 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3b1ffb640 1 -- 172.21.15.7:0/1545704519 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588089e8250 con 0x558808a61d80 2024-03-26T08:44:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3b1ffb640 10 client.25802 handle_command_reply: tid=0 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3d2fef740 1 client.25802 shutdown 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1545704519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089db3a0 msgr2=0x5588089eee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089db3a0 0x5588089eee10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ae20 tx=0x7fa3c0004730 comp rx=0 tx=0).stop 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1545704519 shutdown_connections 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5588089db3a0 0x5588089eee10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558808a61d80 0x558808a641a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3b1ffb640 0 client.25802 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5588089d6230 0x5588089ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3d2fef740 1 --2- 172.21.15.7:0/1545704519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5588089d5840 0x5588089ed910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.259+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1545704519 >> 172.21.15.7:0/1545704519 conn(0x5588089d0db0 msgr2=0x5588089d4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.260+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1545704519 shutdown_connections 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.260+0000 7fa3d2fef740 1 -- 172.21.15.7:0/1545704519 wait complete. 2024-03-26T08:44:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.260+0000 7fa3d2fef740 20 client.25802 trim_cache size 0 max 16384 2024-03-26T08:44:26.255 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:44:26.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 -- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa3b40a2900 msgr2=0x7fa3b40a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa3b40a2900 0x7fa3b40a4dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa3c001b000 tx=0x7fa3c001d000 comp rx=0 tx=0).stop 2024-03-26T08:44:26.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 -- 172.21.15.7:0/2287732079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3c80ffb60 msgr2=0x7fa3c8199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3c80ffb60 0x7fa3c8199a30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4005fd0 tx=0x7fa3c400ef90 comp rx=0 tx=0).stop 2024-03-26T08:44:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 -- 172.21.15.7:0/2287732079 shutdown_connections 2024-03-26T08:44:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa3b40a2900 0x7fa3b40a4dc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3c8101710 0x7fa3c81a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3c8100d50 0x7fa3c8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 --2- 172.21.15.7:0/2287732079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa3c80ffb60 0x7fa3c8199a30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.261+0000 7fa3cf19d640 1 -- 172.21.15.7:0/2287732079 >> 172.21.15.7:0/2287732079 conn(0x7fa3c80fb0d0 msgr2=0x7fa3c8103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.262+0000 7fa3cf19d640 1 -- 172.21.15.7:0/2287732079 shutdown_connections 2024-03-26T08:44:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.262+0000 7fa3cf19d640 1 -- 172.21.15.7:0/2287732079 wait complete. 2024-03-26T08:44:26.270 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:44:26.270 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:44:26.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:26 smithi007 ceph-mon[32462]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 921 op/s 2024-03-26T08:44:26.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:26 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4177194455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:44:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.451+0000 7fcfd77fe640 1 -- 172.21.15.7:0/3945531011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800a2d0 con 0x7fcfd8101490 2024-03-26T08:44:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.452+0000 7fcfdec20640 1 -- 172.21.15.7:0/3945531011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd8101490 msgr2=0x7fcfd8107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.452+0000 7fcfdec20640 1 --2- 172.21.15.7:0/3945531011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd8101490 0x7fcfd8107d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80074e0 tx=0x7fcfc802c8b0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.452+0000 7fcfdec20640 1 -- 172.21.15.7:0/3945531011 shutdown_connections 2024-03-26T08:44:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.452+0000 7fcfdec20640 1 --2- 172.21.15.7:0/3945531011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd8101490 0x7fcfd8107d20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.452+0000 7fcfdec20640 1 --2- 172.21.15.7:0/3945531011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfd8100ad0 0x7fcfd8100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.452+0000 7fcfdec20640 1 --2- 172.21.15.7:0/3945531011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfd80ff8e0 0x7fcfd80ffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.452+0000 7fcfdec20640 1 -- 172.21.15.7:0/3945531011 >> 172.21.15.7:0/3945531011 conn(0x7fcfd80fae50 msgr2=0x7fcfd80fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.452+0000 7fcfdec20640 1 -- 172.21.15.7:0/3945531011 shutdown_connections 2024-03-26T08:44:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.453+0000 7fcfdec20640 1 -- 172.21.15.7:0/3945531011 wait complete. 2024-03-26T08:44:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.453+0000 7fcfdec20640 1 Processor -- start 2024-03-26T08:44:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfdec20640 1 -- start start 2024-03-26T08:44:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfdec20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd80ff8e0 0x7fcfd819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfdec20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfd8100ad0 0x7fcfd819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfdec20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfd8101490 0x7fcfd81a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfdec20640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd810a210 con 0x7fcfd80ff8e0 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfdec20640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd810a090 con 0x7fcfd8101490 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfdec20640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd810a390 con 0x7fcfd8100ad0 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfd8100ad0 0x7fcfd819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfd8100ad0 0x7fcfd819e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34738/0 (socket says 172.21.15.7:34738) 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfd7fff640 1 -- 172.21.15.7:0/1090734727 learned_addr learned my addr 172.21.15.7:0/1090734727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd80ff8e0 0x7fcfd819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfdd1bd640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfd8101490 0x7fcfd81a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfd7fff640 1 -- 172.21.15.7:0/1090734727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfd8101490 msgr2=0x7fcfd81a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfd8101490 0x7fcfd81a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfd7fff640 1 -- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd80ff8e0 msgr2=0x7fcfd819e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd80ff8e0 0x7fcfd819e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.454+0000 7fcfd7fff640 1 -- 172.21.15.7:0/1090734727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd806b1f0 con 0x7fcfd8100ad0 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.455+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd80ff8e0 0x7fcfd819e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.455+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfd8100ad0 0x7fcfd819e990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00c890 tx=0x7fcfcc00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.455+0000 7fcfd5ffb640 1 -- 172.21.15.7:0/1090734727 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc004220 con 0x7fcfd8100ad0 2024-03-26T08:44:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.455+0000 7fcfd5ffb640 1 -- 172.21.15.7:0/1090734727 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc0043c0 con 0x7fcfd8100ad0 2024-03-26T08:44:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.455+0000 7fcfd5ffb640 1 -- 172.21.15.7:0/1090734727 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0049e0 con 0x7fcfd8100ad0 2024-03-26T08:44:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.455+0000 7fcfdec20640 1 -- 172.21.15.7:0/1090734727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd806b4e0 con 0x7fcfd8100ad0 2024-03-26T08:44:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.455+0000 7fcfdec20640 1 -- 172.21.15.7:0/1090734727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd806b9e0 con 0x7fcfd8100ad0 2024-03-26T08:44:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.458+0000 7fcfd5ffb640 1 -- 172.21.15.7:0/1090734727 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcfcc0079b0 con 0x7fcfd8100ad0 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.458+0000 7fcfd5ffb640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfc00a2880 0x7fcfc00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfe180d740 1 Processor -- start 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfe180d740 1 -- start start 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfd5ffb640 1 -- 172.21.15.7:0/1090734727 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcfcc0ceb50 con 0x7fcfd8100ad0 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfc00a2880 0x7fcfc00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb325ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 0x55acdb32ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb32b1f0 0x55acdb32d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfdc9bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb325ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfdc9bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb325ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42588/0 (socket says 172.21.15.7:42588) 2024-03-26T08:44:26.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfdc9bc640 1 -- 172.21.15.7:0/412841402 learned_addr learned my addr 172.21.15.7:0/412841402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/412841402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 0x55acdb32ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfe180d740 1 -- 172.21.15.7:0/412841402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb2e8610 con 0x55acdb326080 2024-03-26T08:44:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfdd1bd640 1 --2- 172.21.15.7:0/412841402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb32b1f0 0x55acdb32d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfe180d740 1 -- 172.21.15.7:0/412841402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb28b3c0 con 0x55acdb325690 2024-03-26T08:44:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.459+0000 7fcfe180d740 1 -- 172.21.15.7:0/412841402 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb233d60 con 0x55acdb32b1f0 2024-03-26T08:44:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.460+0000 7fcfdc9bc640 1 -- 172.21.15.7:0/412841402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb32b1f0 msgr2=0x55acdb32d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.460+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/412841402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb32b1f0 0x55acdb32d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.460+0000 7fcfdc9bc640 1 -- 172.21.15.7:0/412841402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 msgr2=0x55acdb32ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.460+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/412841402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 0x55acdb32ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.460+0000 7fcfdc9bc640 1 -- 172.21.15.7:0/412841402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acdaf9f3f0 con 0x55acdb325690 2024-03-26T08:44:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.460+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfc00a2880 0x7fcfc00a4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000afd0 tx=0x7fcfd0013b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.460+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/412841402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb325ab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0013fd0 tx=0x7fcfd0019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dd7e3f01a0d5be1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.461+0000 7fcfbeffd640 1 -- 172.21.15.7:0/412841402 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0004b20 con 0x55acdb325690 2024-03-26T08:44:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.461+0000 7fcfbeffd640 1 -- 172.21.15.7:0/412841402 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfd0004cc0 con 0x55acdb325690 2024-03-26T08:44:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.461+0000 7fcfbeffd640 1 -- 172.21.15.7:0/412841402 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd001f970 con 0x55acdb325690 2024-03-26T08:44:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.462+0000 7fcfe180d740 1 -- 172.21.15.7:0/412841402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 msgr2=0x55acdb325ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.462+0000 7fcfe180d740 1 --2- 172.21.15.7:0/412841402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb325ab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0013fd0 tx=0x7fcfd0019000 comp rx=0 tx=0).stop 2024-03-26T08:44:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.463+0000 7fcfe180d740 1 -- 172.21.15.7:0/412841402 shutdown_connections 2024-03-26T08:44:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.463+0000 7fcfe180d740 1 --2- 172.21.15.7:0/412841402 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb32b1f0 0x55acdb32d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.463+0000 7fcfe180d740 1 --2- 172.21.15.7:0/412841402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 0x55acdb32ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.463+0000 7fcfe180d740 1 --2- 172.21.15.7:0/412841402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb325ab0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.463+0000 7fcfe180d740 1 -- 172.21.15.7:0/412841402 >> 172.21.15.7:0/412841402 conn(0x55acdb320c00 msgr2=0x55acdb321000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.463+0000 7fcfe180d740 1 -- 172.21.15.7:0/412841402 shutdown_connections 2024-03-26T08:44:26.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.464+0000 7fcfe180d740 1 -- 172.21.15.7:0/412841402 wait complete. 2024-03-26T08:44:26.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.464+0000 7fcfe180d740 1 -- start start 2024-03-26T08:44:26.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.464+0000 7fcfe180d740 1 Processor -- start 2024-03-26T08:44:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb32f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb32fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb336d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfe180d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb247240 con 0x55acdb32b1f0 2024-03-26T08:44:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfe180d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb287460 con 0x55acdb325690 2024-03-26T08:44:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb32fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfdc9bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb32f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb32fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34760/0 (socket says 172.21.15.7:34760) 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfd7fff640 1 -- 172.21.15.7:0/3904241049 learned_addr learned my addr 172.21.15.7:0/3904241049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfdd1bd640 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb336d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.465+0000 7fcfe180d740 1 -- 172.21.15.7:0/3904241049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb32e0e0 con 0x55acdb326080 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfe180d740 10 client.0 _ll_register_callbacks cb 0x55acdb246c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfd7fff640 1 -- 172.21.15.7:0/3904241049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 msgr2=0x55acdb32f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb32f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfd7fff640 1 -- 172.21.15.7:0/3904241049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 msgr2=0x55acdb336d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb336d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfd7fff640 1 -- 172.21.15.7:0/3904241049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acdb0697d0 con 0x55acdb326080 2024-03-26T08:44:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb32fd50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0cc650 tx=0x7fcfcc09b030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709c26cb74ff1514 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfb7fff640 1 -- 172.21.15.7:0/3904241049 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0b4060 con 0x55acdb326080 2024-03-26T08:44:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfb7fff640 1 -- 172.21.15.7:0/3904241049 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcfcc0b4200 con 0x55acdb326080 2024-03-26T08:44:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.466+0000 7fcfe180d740 1 -- 172.21.15.7:0/3904241049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfd0004cc0 con 0x55acdb326080 2024-03-26T08:44:26.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.468+0000 7fcfb7fff640 1 -- 172.21.15.7:0/3904241049 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfcc01d040 con 0x55acdb326080 2024-03-26T08:44:26.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.468+0000 7fcfe180d740 10 client.25834 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.468+0000 7fcfe180d740 1 -- 172.21.15.7:0/3904241049 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55acdb33a140 con 0x55acdb326080 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.469+0000 7fcfb7fff640 1 -- 172.21.15.7:0/3904241049 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfd0004cc0 con 0x55acdb326080 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.469+0000 7fcfb7fff640 1 -- 172.21.15.7:0/3904241049 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcfcc09f1b0 con 0x55acdb326080 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.469+0000 7fcfe180d740 10 client.25834 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.469+0000 7fcfe180d740 10 client.25834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.469+0000 7fcfe180d740 20 client.25834 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.469+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55acdb33b760 0x55acdb33db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.469+0000 7fcfe180d740 4 client.25834 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.469+0000 7fcfe180d740 1 -- 172.21.15.7:0/3904241049 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acdb298900 con 0x55acdb33b760 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.469+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55acdb33b760 0x55acdb33db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.470+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55acdb33b760 0x55acdb33db90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d929d0c92d63f8 server_cookie=43164f2acef6ed32 in_seq=0 out_seq=0 2024-03-26T08:44:26.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.470+0000 7fcfb7fff640 10 client.25834 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfb7fff640 1 -- 172.21.15.7:0/3904241049 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55acdb298900 con 0x55acdb33b760 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfb7fff640 10 client.25834 handle_command_reply: tid=0 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 client.25834 shutdown 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 -- 172.21.15.7:0/3904241049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 msgr2=0x55acdb32fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb32fd50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0cc650 tx=0x7fcfcc09b030 comp rx=0 tx=0).stop 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 -- 172.21.15.7:0/3904241049 shutdown_connections 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55acdb33b760 0x55acdb33db90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb336d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb32fd50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3904241049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb32f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 -- 172.21.15.7:0/3904241049 >> 172.21.15.7:0/3904241049 conn(0x55acdb320c00 msgr2=0x55acdb31fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfb7fff640 0 client.25834 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 -- 172.21.15.7:0/3904241049 shutdown_connections 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 1 -- 172.21.15.7:0/3904241049 wait complete. 2024-03-26T08:44:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.471+0000 7fcfe180d740 20 client.25834 trim_cache size 0 max 16384 2024-03-26T08:44:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.493+0000 7fcfe180d740 1 Processor -- start 2024-03-26T08:44:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.493+0000 7fcfe180d740 1 -- start start 2024-03-26T08:44:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb33d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb336ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb3273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfe180d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81b9d30 con 0x55acdb32b1f0 2024-03-26T08:44:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfe180d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81b7830 con 0x55acdb325690 2024-03-26T08:44:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfe180d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80727c0 con 0x55acdb326080 2024-03-26T08:44:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfdd1bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb3273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfdd1bd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb3273e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55638/0 (socket says 172.21.15.7:55638) 2024-03-26T08:44:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfdd1bd640 1 -- 172.21.15.7:0/3868445108 learned_addr learned my addr 172.21.15.7:0/3868445108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb336ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb33d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfd7fff640 1 -- 172.21.15.7:0/3868445108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 msgr2=0x55acdb33d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb33d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfd7fff640 1 -- 172.21.15.7:0/3868445108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 msgr2=0x55acdb3273e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb3273e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.494+0000 7fcfd7fff640 1 -- 172.21.15.7:0/3868445108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acdb3205a0 con 0x55acdb326080 2024-03-26T08:44:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.495+0000 7fcfdd1bd640 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb3273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.495+0000 7fcfd7fff640 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb336ce0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc097950 tx=0x7fcfcc099a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc85ed31871ab9f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.495+0000 7fcfbe7fc640 1 -- 172.21.15.7:0/3868445108 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0a3c90 con 0x55acdb326080 2024-03-26T08:44:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.495+0000 7fcfbe7fc640 1 -- 172.21.15.7:0/3868445108 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc09b4f0 con 0x55acdb326080 2024-03-26T08:44:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.495+0000 7fcfbe7fc640 1 -- 172.21.15.7:0/3868445108 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0b4060 con 0x55acdb326080 2024-03-26T08:44:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.496+0000 7fcfe180d740 1 -- 172.21.15.7:0/3868445108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 msgr2=0x55acdb336ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.496+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb336ce0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc097950 tx=0x7fcfcc099a00 comp rx=0 tx=0).stop 2024-03-26T08:44:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.496+0000 7fcfe180d740 1 -- 172.21.15.7:0/3868445108 shutdown_connections 2024-03-26T08:44:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.496+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb32b1f0 0x55acdb3273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.496+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb326080 0x55acdb336ce0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.496+0000 7fcfe180d740 1 --2- 172.21.15.7:0/3868445108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb325690 0x55acdb33d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.496+0000 7fcfe180d740 1 -- 172.21.15.7:0/3868445108 >> 172.21.15.7:0/3868445108 conn(0x55acdb320c00 msgr2=0x55acdb320fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.496+0000 7fcfe180d740 1 -- 172.21.15.7:0/3868445108 shutdown_connections 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.496+0000 7fcfe180d740 1 -- 172.21.15.7:0/3868445108 wait complete. 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.497+0000 7fcfe180d740 1 -- start start 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.497+0000 7fcfe180d740 1 Processor -- start 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.497+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb325690 0x55acdb334050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.497+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 0x55acdb334590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.497+0000 7fcfe180d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb32b1f0 0x55acdb335530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.497+0000 7fcfe180d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8072dc0 con 0x55acdb326080 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.497+0000 7fcfe180d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81b71d0 con 0x55acdb32b1f0 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.497+0000 7fcfe180d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8073420 con 0x55acdb325690 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.497+0000 7fcfe180d740 10 client.0 _ll_register_callbacks cb 0x7fcfcc097790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdd1bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb32b1f0 0x55acdb335530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdd1bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb32b1f0 0x55acdb335530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42616/0 (socket says 172.21.15.7:42616) 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 0x55acdb334590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdd1bd640 1 -- 172.21.15.7:0/4203321601 learned_addr learned my addr 172.21.15.7:0/4203321601 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb325690 0x55acdb334050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdd1bd640 1 -- 172.21.15.7:0/4203321601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb325690 msgr2=0x55acdb334050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdd1bd640 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb325690 0x55acdb334050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdd1bd640 1 -- 172.21.15.7:0/4203321601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 msgr2=0x55acdb334590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdd1bd640 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 0x55acdb334590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdd1bd640 1 -- 172.21.15.7:0/4203321601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acdb335b20 con 0x55acdb32b1f0 2024-03-26T08:44:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.498+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb325690 0x55acdb334050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfdd1bd640 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb32b1f0 0x55acdb335530 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8004df0 tx=0x7fcfc8007dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329750610ddf963b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfb7fff640 1 -- 172.21.15.7:0/4203321601 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc800a7e0 con 0x55acdb32b1f0 2024-03-26T08:44:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfb7fff640 1 -- 172.21.15.7:0/4203321601 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcfc800a980 con 0x55acdb32b1f0 2024-03-26T08:44:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfe180d740 1 -- 172.21.15.7:0/4203321601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfcc09b4f0 con 0x55acdb32b1f0 2024-03-26T08:44:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfb7fff640 1 -- 172.21.15.7:0/4203321601 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfc80046c0 con 0x55acdb32b1f0 2024-03-26T08:44:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfe180d740 10 client.25817 fetch_fsmap learned FSMap version 24 2024-03-26T08:44:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfe180d740 1 -- 172.21.15.7:0/4203321601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55acdb336e90 con 0x55acdb32b1f0 2024-03-26T08:44:26.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfb7fff640 1 -- 172.21.15.7:0/4203321601 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc09b4f0 con 0x55acdb32b1f0 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfb7fff640 1 -- 172.21.15.7:0/4203321601 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcfc8050030 con 0x55acdb32b1f0 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfe180d740 10 client.25817 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfe180d740 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.499+0000 7fcfe180d740 20 client.25817 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.500+0000 7fcfe180d740 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55acdb3976a0 0x55acdb399ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.500+0000 7fcfe180d740 4 client.25817 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.500+0000 7fcfe180d740 1 -- 172.21.15.7:0/4203321601 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55acdb292c50 con 0x55acdb3976a0 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.500+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55acdb3976a0 0x55acdb399ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.501+0000 7fcfdc9bc640 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55acdb3976a0 0x55acdb399ac0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976617cc81b2ed74 server_cookie=d4c3f1f845798611 in_seq=0 out_seq=0 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.501+0000 7fcfb7fff640 10 client.25817 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.501+0000 7fcfb7fff640 1 -- 172.21.15.7:0/4203321601 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55acdb292c50 con 0x55acdb3976a0 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.501+0000 7fcfb7fff640 10 client.25817 handle_command_reply: tid=0 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.501+0000 7fcfe180d740 1 client.25817 shutdown 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.501+0000 7fcfe180d740 1 -- 172.21.15.7:0/4203321601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb32b1f0 msgr2=0x55acdb335530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.501+0000 7fcfe180d740 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb32b1f0 0x55acdb335530 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8004df0 tx=0x7fcfc8007dc0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfe180d740 1 -- 172.21.15.7:0/4203321601 shutdown_connections 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfe180d740 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55acdb3976a0 0x55acdb399ac0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfe180d740 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55acdb32b1f0 0x55acdb335530 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfe180d740 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acdb326080 0x55acdb334590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfb7fff640 0 client.25817 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfe180d740 1 --2- 172.21.15.7:0/4203321601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55acdb325690 0x55acdb334050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfe180d740 1 -- 172.21.15.7:0/4203321601 >> 172.21.15.7:0/4203321601 conn(0x55acdb320c00 msgr2=0x55acdb33d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfe180d740 1 -- 172.21.15.7:0/4203321601 shutdown_connections 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfe180d740 1 -- 172.21.15.7:0/4203321601 wait complete. 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.502+0000 7fcfe180d740 20 client.25817 trim_cache size 0 max 16384 2024-03-26T08:44:26.496 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:44:26.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.503+0000 7fcfdec20640 1 -- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfc00a2880 msgr2=0x7fcfc00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.503+0000 7fcfdec20640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfc00a2880 0x7fcfc00a4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000afd0 tx=0x7fcfd0013b40 comp rx=0 tx=0).stop 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.503+0000 7fcfdec20640 1 -- 172.21.15.7:0/1090734727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfd8100ad0 msgr2=0x7fcfd819e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.503+0000 7fcfdec20640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfd8100ad0 0x7fcfd819e990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00c890 tx=0x7fcfcc00cd50 comp rx=0 tx=0).stop 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.504+0000 7fcfdec20640 1 -- 172.21.15.7:0/1090734727 shutdown_connections 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.504+0000 7fcfdec20640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfc00a2880 0x7fcfc00a4d40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.504+0000 7fcfdec20640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfd8101490 0x7fcfd81a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.504+0000 7fcfdec20640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfd8100ad0 0x7fcfd819e990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.504+0000 7fcfdec20640 1 --2- 172.21.15.7:0/1090734727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfd80ff8e0 0x7fcfd819e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.504+0000 7fcfdec20640 1 -- 172.21.15.7:0/1090734727 >> 172.21.15.7:0/1090734727 conn(0x7fcfd80fae50 msgr2=0x7fcfd80fb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.504+0000 7fcfdec20640 1 -- 172.21.15.7:0/1090734727 shutdown_connections 2024-03-26T08:44:26.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:26.504+0000 7fcfdec20640 1 -- 172.21.15.7:0/1090734727 wait complete. 2024-03-26T08:44:26.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:26 smithi016 ceph-mon[35851]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 921 op/s 2024-03-26T08:44:26.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:26 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4177194455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:44:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:26 smithi044 ceph-mon[36461]: pgmap v420: 97 pgs: 97 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 921 op/s 2024-03-26T08:44:26.566 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:26 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4177194455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:44:27.511 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:44:27.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:44:27.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:44:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.700+0000 7f32f3ba9640 1 -- 172.21.15.7:0/470924231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ec0ff8e0 msgr2=0x7f32ec0ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.700+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/470924231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ec0ff8e0 0x7f32ec0ffd00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009900 tx=0x7f32e802cd50 comp rx=0 tx=0).stop 2024-03-26T08:44:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.701+0000 7f32f3ba9640 1 -- 172.21.15.7:0/470924231 shutdown_connections 2024-03-26T08:44:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.701+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/470924231 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec101490 0x7f32ec107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.701+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/470924231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec100ad0 0x7f32ec100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.701+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/470924231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ec0ff8e0 0x7f32ec0ffd00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.701+0000 7f32f3ba9640 1 -- 172.21.15.7:0/470924231 >> 172.21.15.7:0/470924231 conn(0x7f32ec0fae50 msgr2=0x7f32ec0fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.702+0000 7f32f3ba9640 1 -- 172.21.15.7:0/470924231 shutdown_connections 2024-03-26T08:44:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.702+0000 7f32f3ba9640 1 -- 172.21.15.7:0/470924231 wait complete. 2024-03-26T08:44:27.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.702+0000 7f32f3ba9640 1 Processor -- start 2024-03-26T08:44:27.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f3ba9640 1 -- start start 2024-03-26T08:44:27.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f3ba9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec0ff8e0 0x7f32ec103870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f3ba9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ec100ad0 0x7f32ec101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f3ba9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec101490 0x7f32ec1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f3ba9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec10b150 con 0x7f32ec101490 2024-03-26T08:44:27.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f3ba9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec10afd0 con 0x7f32ec100ad0 2024-03-26T08:44:27.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f2146640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec101490 0x7f32ec1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f2146640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec101490 0x7f32ec1023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50446/0 (socket says 172.21.15.7:50446) 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f2146640 1 -- 172.21.15.7:0/1681463841 learned_addr learned my addr 172.21.15.7:0/1681463841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.704+0000 7f32f3ba9640 1 -- 172.21.15.7:0/1681463841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec10b2d0 con 0x7f32ec0ff8e0 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.705+0000 7f32f1144640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ec100ad0 0x7f32ec101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.705+0000 7f32f1945640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec0ff8e0 0x7f32ec103870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.705+0000 7f32f2146640 1 -- 172.21.15.7:0/1681463841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec0ff8e0 msgr2=0x7f32ec103870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.705+0000 7f32f2146640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec0ff8e0 0x7f32ec103870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.705+0000 7f32f2146640 1 -- 172.21.15.7:0/1681463841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ec100ad0 msgr2=0x7f32ec101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.705+0000 7f32f2146640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ec100ad0 0x7f32ec101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.705+0000 7f32f2146640 1 -- 172.21.15.7:0/1681463841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec102c80 con 0x7f32ec101490 2024-03-26T08:44:27.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.705+0000 7f32f1945640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec0ff8e0 0x7f32ec103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:44:27.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.706+0000 7f32f2146640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec101490 0x7f32ec1023f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f32e400eaa0 tx=0x7f32e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.706+0000 7f32e2ffd640 1 -- 172.21.15.7:0/1681463841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e400cdb0 con 0x7f32ec101490 2024-03-26T08:44:27.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.706+0000 7f32e2ffd640 1 -- 172.21.15.7:0/1681463841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32e4005020 con 0x7f32ec101490 2024-03-26T08:44:27.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.707+0000 7f32e2ffd640 1 -- 172.21.15.7:0/1681463841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e40116b0 con 0x7f32ec101490 2024-03-26T08:44:27.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.707+0000 7f32f3ba9640 1 -- 172.21.15.7:0/1681463841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec1a1970 con 0x7f32ec101490 2024-03-26T08:44:27.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.707+0000 7f32f3ba9640 1 -- 172.21.15.7:0/1681463841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec1a1e70 con 0x7f32ec101490 2024-03-26T08:44:27.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.710+0000 7f32f678f740 1 Processor -- start 2024-03-26T08:44:27.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.711+0000 7f32e2ffd640 1 -- 172.21.15.7:0/1681463841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f32e4005560 con 0x7f32ec101490 2024-03-26T08:44:27.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.711+0000 7f32f678f740 1 -- start start 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f678f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a448590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f678f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448b60 0x55970a44d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f678f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 0x55970a450160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f678f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970a37fc90 con 0x55970a448170 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f678f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970a3a5070 con 0x55970a448b60 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f678f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970a34dc80 con 0x55970a44dcd0 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f1144640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a448590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f1144640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a448590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50454/0 (socket says 172.21.15.7:50454) 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f1144640 1 -- 172.21.15.7:0/3160437481 learned_addr learned my addr 172.21.15.7:0/3160437481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f1945640 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448b60 0x55970a44d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32f1144640 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 0x55970a450160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.712+0000 7f32e2ffd640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32d80a28d0 0x7f32d80a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32f1144640 1 -- 172.21.15.7:0/3160437481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448b60 msgr2=0x55970a44d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32f1144640 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448b60 0x55970a44d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32f1144640 1 -- 172.21.15.7:0/3160437481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 msgr2=0x55970a448590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32f1144640 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a448590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32f1144640 1 -- 172.21.15.7:0/3160437481 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55970a345d10 con 0x55970a44dcd0 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32e2ffd640 1 -- 172.21.15.7:0/1681463841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f32e40cecb0 con 0x7f32ec101490 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32f1144640 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 0x55970a450160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55970a4493c0 tx=0x7f32dc00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6604a781df44aa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32f1945640 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448b60 0x55970a44d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:27.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32d7fff640 1 -- 172.21.15.7:0/3160437481 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc009670 con 0x55970a44dcd0 2024-03-26T08:44:27.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32f2146640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32d80a28d0 0x7f32d80a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32d7fff640 1 -- 172.21.15.7:0/3160437481 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32dc009cf0 con 0x55970a44dcd0 2024-03-26T08:44:27.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.713+0000 7f32d7fff640 1 -- 172.21.15.7:0/3160437481 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc010c50 con 0x55970a44dcd0 2024-03-26T08:44:27.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.714+0000 7f32f678f740 1 -- 172.21.15.7:0/3160437481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 msgr2=0x55970a450160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.714+0000 7f32f678f740 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 0x55970a450160 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55970a4493c0 tx=0x7f32dc00b390 comp rx=0 tx=0).stop 2024-03-26T08:44:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.715+0000 7f32f2146640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32d80a28d0 0x7f32d80a4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32e4097ab0 tx=0x7f32e4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.715+0000 7f32f678f740 1 -- 172.21.15.7:0/3160437481 shutdown_connections 2024-03-26T08:44:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.715+0000 7f32f678f740 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 0x55970a450160 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.715+0000 7f32f678f740 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448b60 0x55970a44d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.715+0000 7f32f678f740 1 --2- 172.21.15.7:0/3160437481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a448590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.715+0000 7f32f678f740 1 -- 172.21.15.7:0/3160437481 >> 172.21.15.7:0/3160437481 conn(0x55970a4081a0 msgr2=0x55970a43b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.715+0000 7f32f678f740 1 -- 172.21.15.7:0/3160437481 shutdown_connections 2024-03-26T08:44:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.715+0000 7f32f678f740 1 -- 172.21.15.7:0/3160437481 wait complete. 2024-03-26T08:44:27.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.716+0000 7f32f678f740 1 -- start start 2024-03-26T08:44:27.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.716+0000 7f32f678f740 1 Processor -- start 2024-03-26T08:44:27.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.716+0000 7f32f678f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448170 0x55970a452380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f678f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4528c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f678f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 0x55970a459900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448170 0x55970a452380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448170 0x55970a452380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35202/0 (socket says 172.21.15.7:35202) 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 -- 172.21.15.7:0/3906330631 learned_addr learned my addr 172.21.15.7:0/3906330631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f678f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970a3610d0 con 0x55970a448b60 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970a3a1110 con 0x55970a44dcd0 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970a450c80 con 0x55970a448170 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1945640 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 0x55970a459900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f678f740 10 client.0 _ll_register_callbacks cb 0x55970a39ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 -- 172.21.15.7:0/3906330631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 msgr2=0x55970a459900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 0x55970a459900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 -- 172.21.15.7:0/3906330631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 msgr2=0x55970a4528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4528c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 -- 172.21.15.7:0/3906330631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55970a1817e0 con 0x55970a448170 2024-03-26T08:44:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.717+0000 7f32f1144640 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448170 0x55970a452380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00bcf0 tx=0x7f32dc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=924907cc281c0690 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.718+0000 7f32d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.718+0000 7f32d57fa640 1 -- 172.21.15.7:0/3906330631 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc016050 con 0x55970a448170 2024-03-26T08:44:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.718+0000 7f32d57fa640 1 -- 172.21.15.7:0/3906330631 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f32dc01a040 con 0x55970a448170 2024-03-26T08:44:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.718+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32dc009cf0 con 0x55970a448170 2024-03-26T08:44:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.718+0000 7f32d57fa640 1 -- 172.21.15.7:0/3906330631 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32dc035050 con 0x55970a448170 2024-03-26T08:44:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.719+0000 7f32f678f740 10 client.25849 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.719+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55970a1e1f70 con 0x55970a448170 2024-03-26T08:44:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.719+0000 7f32d57fa640 1 -- 172.21.15.7:0/3906330631 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32dc009cf0 con 0x55970a448170 2024-03-26T08:44:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.719+0000 7f32d57fa640 1 -- 172.21.15.7:0/3906330631 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f32dc0035f0 con 0x55970a448170 2024-03-26T08:44:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.719+0000 7f32f678f740 10 client.25849 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.719+0000 7f32f678f740 10 client.25849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.720+0000 7f32f678f740 20 client.25849 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.720+0000 7f32f678f740 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55970a45e330 0x55970a460770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.720+0000 7f32f678f740 4 client.25849 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.720+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55970a3b25b0 con 0x55970a45e330 2024-03-26T08:44:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.720+0000 7f32f1945640 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55970a45e330 0x55970a460770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.720+0000 7f32d57fa640 10 client.25849 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.720+0000 7f32f1945640 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55970a45e330 0x55970a460770 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e05dd0205ca24187 server_cookie=eda936cd49f8c5d2 in_seq=0 out_seq=0 2024-03-26T08:44:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.721+0000 7f32d57fa640 1 -- 172.21.15.7:0/3906330631 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55970a3b25b0 con 0x55970a45e330 2024-03-26T08:44:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.721+0000 7f32d57fa640 10 client.25849 handle_command_reply: tid=0 2024-03-26T08:44:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.721+0000 7f32f678f740 1 client.25849 shutdown 2024-03-26T08:44:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.722+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448170 msgr2=0x55970a452380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.722+0000 7f32f678f740 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448170 0x55970a452380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00bcf0 tx=0x7f32dc00bc80 comp rx=0 tx=0).stop 2024-03-26T08:44:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.722+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 shutdown_connections 2024-03-26T08:44:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.722+0000 7f32f678f740 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55970a45e330 0x55970a460770 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.723+0000 7f32f678f740 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 0x55970a459900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.723+0000 7f32f678f740 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.723+0000 7f32f678f740 1 --2- 172.21.15.7:0/3906330631 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448170 0x55970a452380 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.723+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 >> 172.21.15.7:0/3906330631 conn(0x55970a4081a0 msgr2=0x55970a43a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.723+0000 7f32d57fa640 0 client.25849 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.723+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 shutdown_connections 2024-03-26T08:44:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.723+0000 7f32f678f740 1 -- 172.21.15.7:0/3906330631 wait complete. 2024-03-26T08:44:27.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.723+0000 7f32f678f740 20 client.25849 trim_cache size 0 max 16384 2024-03-26T08:44:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.750+0000 7f32f678f740 1 Processor -- start 2024-03-26T08:44:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.751+0000 7f32f678f740 1 -- start start 2024-03-26T08:44:27.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.751+0000 7f32f678f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448170 0x55970a452820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.751+0000 7f32f678f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.751+0000 7f32f678f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 0x55970a439ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.752+0000 7f32f678f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1a8c50 con 0x55970a448b60 2024-03-26T08:44:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.752+0000 7f32f678f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1a6750 con 0x55970a448170 2024-03-26T08:44:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.752+0000 7f32f678f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1a4810 con 0x55970a44dcd0 2024-03-26T08:44:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.752+0000 7f32f1945640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.752+0000 7f32f1945640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50470/0 (socket says 172.21.15.7:50470) 2024-03-26T08:44:27.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.752+0000 7f32f1945640 1 -- 172.21.15.7:0/1605985331 learned_addr learned my addr 172.21.15.7:0/1605985331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.753+0000 7f32f1945640 1 -- 172.21.15.7:0/1605985331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 msgr2=0x55970a439ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:44:27.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.753+0000 7f32f1945640 1 --2- 172.21.15.7:0/1605985331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 0x55970a439ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.753+0000 7f32f1945640 1 -- 172.21.15.7:0/1605985331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448170 msgr2=0x55970a452820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.753+0000 7f32f1945640 1 --2- 172.21.15.7:0/1605985331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448170 0x55970a452820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.753+0000 7f32f1945640 1 -- 172.21.15.7:0/1605985331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55970a449970 con 0x55970a448b60 2024-03-26T08:44:27.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.753+0000 7f32f1945640 1 --2- 172.21.15.7:0/1605985331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4598c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f32ec100930 tx=0x7f32e802db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188c15b7c79bca05 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.753+0000 7f32d77fe640 1 -- 172.21.15.7:0/1605985331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8049020 con 0x55970a448b60 2024-03-26T08:44:27.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.753+0000 7f32d77fe640 1 -- 172.21.15.7:0/1605985331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32e802dc00 con 0x55970a448b60 2024-03-26T08:44:27.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.754+0000 7f32d77fe640 1 -- 172.21.15.7:0/1605985331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e80052a0 con 0x55970a448b60 2024-03-26T08:44:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.754+0000 7f32f678f740 1 -- 172.21.15.7:0/1605985331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 msgr2=0x55970a4598c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.754+0000 7f32f678f740 1 --2- 172.21.15.7:0/1605985331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4598c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f32ec100930 tx=0x7f32e802db10 comp rx=0 tx=0).stop 2024-03-26T08:44:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.755+0000 7f32f678f740 1 -- 172.21.15.7:0/1605985331 shutdown_connections 2024-03-26T08:44:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.755+0000 7f32f678f740 1 --2- 172.21.15.7:0/1605985331 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a44dcd0 0x55970a439ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.755+0000 7f32f678f740 1 --2- 172.21.15.7:0/1605985331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448b60 0x55970a4598c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.755+0000 7f32f678f740 1 --2- 172.21.15.7:0/1605985331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a448170 0x55970a452820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.755+0000 7f32f678f740 1 -- 172.21.15.7:0/1605985331 >> 172.21.15.7:0/1605985331 conn(0x55970a4081a0 msgr2=0x55970a43eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.755+0000 7f32f678f740 1 -- 172.21.15.7:0/1605985331 shutdown_connections 2024-03-26T08:44:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.755+0000 7f32f678f740 1 -- 172.21.15.7:0/1605985331 wait complete. 2024-03-26T08:44:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.756+0000 7f32f678f740 1 -- start start 2024-03-26T08:44:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.756+0000 7f32f678f740 1 Processor -- start 2024-03-26T08:44:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.756+0000 7f32f678f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a4575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.756+0000 7f32f678f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448b60 0x55970a457b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.756+0000 7f32f678f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 0x55970a458b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.756+0000 7f32f678f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1a4e10 con 0x55970a448170 2024-03-26T08:44:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.756+0000 7f32f678f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1a60f0 con 0x55970a44dcd0 2024-03-26T08:44:27.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.756+0000 7f32f678f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec1a5430 con 0x55970a448b60 2024-03-26T08:44:27.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.756+0000 7f32f678f740 10 client.0 _ll_register_callbacks cb 0x7f32dc00bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:27.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.757+0000 7f32f1945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448b60 0x55970a457b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1945640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448b60 0x55970a457b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35212/0 (socket says 172.21.15.7:35212) 2024-03-26T08:44:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1945640 1 -- 172.21.15.7:0/4208083204 learned_addr learned my addr 172.21.15.7:0/4208083204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1144640 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 0x55970a458b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1144640 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a4575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1945640 1 -- 172.21.15.7:0/4208083204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 msgr2=0x55970a458b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1945640 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 0x55970a458b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1945640 1 -- 172.21.15.7:0/4208083204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 msgr2=0x55970a4575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1945640 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a4575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1945640 1 -- 172.21.15.7:0/4208083204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55970a079170 con 0x55970a448b60 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1144640 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a4575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32f1945640 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448b60 0x55970a457b10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f32e8037040 tx=0x7f32e8005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da30a671f756f7a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.758+0000 7f32d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.759+0000 7f32d57fa640 1 -- 172.21.15.7:0/4208083204 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8049030 con 0x55970a448b60 2024-03-26T08:44:27.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.759+0000 7f32d57fa640 1 -- 172.21.15.7:0/4208083204 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f32e8004450 con 0x55970a448b60 2024-03-26T08:44:27.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.759+0000 7f32f678f740 1 -- 172.21.15.7:0/4208083204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32e802dc00 con 0x55970a448b60 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32d57fa640 1 -- 172.21.15.7:0/4208083204 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32e805d050 con 0x55970a448b60 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32f678f740 10 client.25852 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32f678f740 1 -- 172.21.15.7:0/4208083204 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55970a43fa20 con 0x55970a448b60 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32d57fa640 1 -- 172.21.15.7:0/4208083204 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32e802dc00 con 0x55970a448b60 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32d57fa640 1 -- 172.21.15.7:0/4208083204 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f32e8004ca0 con 0x55970a448b60 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32f678f740 10 client.25852 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32f678f740 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32f678f740 20 client.25852 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32f678f740 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55970a440420 0x55970a442840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32f678f740 4 client.25852 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32f678f740 1 -- 172.21.15.7:0/4208083204 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55970a3ac900 con 0x55970a440420 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.760+0000 7f32f1144640 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55970a440420 0x55970a442840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.761+0000 7f32d57fa640 10 client.25852 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:27.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.761+0000 7f32f1144640 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55970a440420 0x55970a442840 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650438b05c3c22bc server_cookie=ef1fe0bfef3cfeeb in_seq=0 out_seq=0 2024-03-26T08:44:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32d57fa640 1 -- 172.21.15.7:0/4208083204 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f32dc023080 con 0x55970a440420 2024-03-26T08:44:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32d57fa640 10 client.25852 handle_command_reply: tid=0 2024-03-26T08:44:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32f678f740 1 client.25852 shutdown 2024-03-26T08:44:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32f678f740 1 -- 172.21.15.7:0/4208083204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448b60 msgr2=0x55970a457b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32f678f740 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448b60 0x55970a457b10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f32e8037040 tx=0x7f32e8005bb0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32f678f740 1 -- 172.21.15.7:0/4208083204 shutdown_connections 2024-03-26T08:44:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32f678f740 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55970a440420 0x55970a442840 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32f678f740 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55970a44dcd0 0x55970a458b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32f678f740 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55970a448b60 0x55970a457b10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32f678f740 1 --2- 172.21.15.7:0/4208083204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55970a448170 0x55970a4575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32d57fa640 0 client.25852 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.763+0000 7f32f678f740 1 -- 172.21.15.7:0/4208083204 >> 172.21.15.7:0/4208083204 conn(0x55970a4081a0 msgr2=0x55970a465070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.764+0000 7f32f678f740 1 -- 172.21.15.7:0/4208083204 shutdown_connections 2024-03-26T08:44:27.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.764+0000 7f32f678f740 1 -- 172.21.15.7:0/4208083204 wait complete. 2024-03-26T08:44:27.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.765+0000 7f32f678f740 20 client.25852 trim_cache size 0 max 16384 2024-03-26T08:44:27.756 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:44:27.756 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:44:27.756 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "036bb239-7520-4988-b988-0fa31906f63e", 2024-03-26T08:44:27.756 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:44:27.756 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 -- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32d80a28d0 msgr2=0x7f32d80a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32d80a28d0 0x7f32d80a4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32e4097ab0 tx=0x7f32e4005850 comp rx=0 tx=0).stop 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 -- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec101490 msgr2=0x7f32ec1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec101490 0x7f32ec1023f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f32e400eaa0 tx=0x7f32e400ef60 comp rx=0 tx=0).stop 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 -- 172.21.15.7:0/1681463841 shutdown_connections 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32d80a28d0 0x7f32d80a4d90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32ec101490 0x7f32ec1023f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ec100ad0 0x7f32ec101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 --2- 172.21.15.7:0/1681463841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32ec0ff8e0 0x7f32ec103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 -- 172.21.15.7:0/1681463841 >> 172.21.15.7:0/1681463841 conn(0x7f32ec0fae50 msgr2=0x7f32ec105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 -- 172.21.15.7:0/1681463841 shutdown_connections 2024-03-26T08:44:27.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.766+0000 7f32f3ba9640 1 -- 172.21.15.7:0/1681463841 wait complete. 2024-03-26T08:44:27.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:44:27.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:44:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.861+0000 7fe67d7b3640 1 -- 172.21.15.7:0/3245359458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe678077c40 msgr2=0x7fe6780784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.861+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/3245359458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe678077c40 0x7fe6780784e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe668009900 tx=0x7fe66802bc30 comp rx=0 tx=0).stop 2024-03-26T08:44:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.861+0000 7fe67d7b3640 1 -- 172.21.15.7:0/3245359458 shutdown_connections 2024-03-26T08:44:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.861+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/3245359458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe678077c40 0x7fe6780784e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.861+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/3245359458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6780772a0 0x7fe678077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.861+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/3245359458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe678076060 0x7fe678076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.861+0000 7fe67d7b3640 1 -- 172.21.15.7:0/3245359458 >> 172.21.15.7:0/3245359458 conn(0x7fe6780fecd0 msgr2=0x7fe678101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.862+0000 7fe67d7b3640 1 -- 172.21.15.7:0/3245359458 shutdown_connections 2024-03-26T08:44:27.853 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.862+0000 7fe67d7b3640 1 -- 172.21.15.7:0/3245359458 wait complete. 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.862+0000 7fe67d7b3640 1 Processor -- start 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe67d7b3640 1 -- start start 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe67d7b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe678076060 0x7fe67819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe67d7b3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6780772a0 0x7fe67819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe67d7b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe678077c40 0x7fe6781a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe67d7b3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe678113870 con 0x7fe678077c40 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe67d7b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781136f0 con 0x7fe678076060 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe67d7b3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781139f0 con 0x7fe6780772a0 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe676ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe678076060 0x7fe67819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6780772a0 0x7fe67819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe676ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe678076060 0x7fe67819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58318/0 (socket says 172.21.15.7:58318) 2024-03-26T08:44:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe676ffd640 1 -- 172.21.15.7:0/1361445450 learned_addr learned my addr 172.21.15.7:0/1361445450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6780772a0 0x7fe67819e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35236/0 (socket says 172.21.15.7:35236) 2024-03-26T08:44:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe6777fe640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe678077c40 0x7fe6781a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe676ffd640 1 -- 172.21.15.7:0/1361445450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6780772a0 msgr2=0x7fe67819e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe676ffd640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6780772a0 0x7fe67819e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe676ffd640 1 -- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe678077c40 msgr2=0x7fe6781a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe676ffd640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe678077c40 0x7fe6781a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.863+0000 7fe676ffd640 1 -- 172.21.15.7:0/1361445450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6781a5a20 con 0x7fe678076060 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.864+0000 7fe676ffd640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe678076060 0x7fe67819de00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b4a0 tx=0x7fe66c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.864+0000 7fe65ffff640 1 -- 172.21.15.7:0/1361445450 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c014070 con 0x7fe678076060 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.864+0000 7fe65ffff640 1 -- 172.21.15.7:0/1361445450 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66c010af0 con 0x7fe678076060 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.864+0000 7fe65ffff640 1 -- 172.21.15.7:0/1361445450 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c017d10 con 0x7fe678076060 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.865+0000 7fe67d7b3640 1 -- 172.21.15.7:0/1361445450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6781a5d10 con 0x7fe678076060 2024-03-26T08:44:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.865+0000 7fe67d7b3640 1 -- 172.21.15.7:0/1361445450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6781a61b0 con 0x7fe678076060 2024-03-26T08:44:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.869+0000 7fe65ffff640 1 -- 172.21.15.7:0/1361445450 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe66c017070 con 0x7fe678076060 2024-03-26T08:44:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.872+0000 7fe65ffff640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6600a29b0 0x7fe6600a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.872+0000 7fe65ffff640 1 -- 172.21.15.7:0/1361445450 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe66c0cdfb0 con 0x7fe678076060 2024-03-26T08:44:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.873+0000 7fe6767fc640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6600a29b0 0x7fe6600a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.873+0000 7fe68039a740 1 Processor -- start 2024-03-26T08:44:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.873+0000 7fe6767fc640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6600a29b0 0x7fe6600a4e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe67819f310 tx=0x7fe670009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe68039a740 1 -- start start 2024-03-26T08:44:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe68039a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568ea1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe68039a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568ea7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe68039a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568ea9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe68039a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b568e06b10 con 0x55b568ea2510 2024-03-26T08:44:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe68039a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b568daf900 con 0x55b568ea7680 2024-03-26T08:44:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe68039a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b568e02bb0 con 0x55b568ea1b20 2024-03-26T08:44:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568ea9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568ea9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58322/0 (socket says 172.21.15.7:58322) 2024-03-26T08:44:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe676ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568ea1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe6777fe640 1 -- 172.21.15.7:0/3558018632 learned_addr learned my addr 172.21.15.7:0/3558018632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.874+0000 7fe6767fc640 1 --2- 172.21.15.7:0/3558018632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568ea7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.875+0000 7fe6777fe640 1 -- 172.21.15.7:0/3558018632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 msgr2=0x55b568ea1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.875+0000 7fe6777fe640 1 --2- 172.21.15.7:0/3558018632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568ea1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.875+0000 7fe6777fe640 1 -- 172.21.15.7:0/3558018632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 msgr2=0x55b568ea7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.875+0000 7fe6777fe640 1 --2- 172.21.15.7:0/3558018632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568ea7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.875+0000 7fe6777fe640 1 -- 172.21.15.7:0/3558018632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b568b1b210 con 0x55b568ea7680 2024-03-26T08:44:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.875+0000 7fe6777fe640 1 --2- 172.21.15.7:0/3558018632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568ea9b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe66802c8e0 tx=0x7fe668009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f00d8984515fd38 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.876+0000 7fe65d7fa640 1 -- 172.21.15.7:0/3558018632 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668045070 con 0x55b568ea7680 2024-03-26T08:44:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.876+0000 7fe65d7fa640 1 -- 172.21.15.7:0/3558018632 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66802ca40 con 0x55b568ea7680 2024-03-26T08:44:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.876+0000 7fe65d7fa640 1 -- 172.21.15.7:0/3558018632 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6680046c0 con 0x55b568ea7680 2024-03-26T08:44:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.877+0000 7fe68039a740 1 -- 172.21.15.7:0/3558018632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 msgr2=0x55b568ea9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.877+0000 7fe68039a740 1 --2- 172.21.15.7:0/3558018632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568ea9b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe66802c8e0 tx=0x7fe668009610 comp rx=0 tx=0).stop 2024-03-26T08:44:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.877+0000 7fe68039a740 1 -- 172.21.15.7:0/3558018632 shutdown_connections 2024-03-26T08:44:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.877+0000 7fe68039a740 1 --2- 172.21.15.7:0/3558018632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568ea9b10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.877+0000 7fe68039a740 1 --2- 172.21.15.7:0/3558018632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568ea7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.877+0000 7fe68039a740 1 --2- 172.21.15.7:0/3558018632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568ea1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.877+0000 7fe68039a740 1 -- 172.21.15.7:0/3558018632 >> 172.21.15.7:0/3558018632 conn(0x55b568e9d090 msgr2=0x55b568e9d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.877+0000 7fe68039a740 1 -- 172.21.15.7:0/3558018632 shutdown_connections 2024-03-26T08:44:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.878+0000 7fe68039a740 1 -- 172.21.15.7:0/3558018632 wait complete. 2024-03-26T08:44:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.878+0000 7fe68039a740 1 -- start start 2024-03-26T08:44:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.878+0000 7fe68039a740 1 Processor -- start 2024-03-26T08:44:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.878+0000 7fe68039a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eabe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.878+0000 7fe68039a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568eac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe68039a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568eb3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe68039a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b568eaa670 con 0x55b568ea2510 2024-03-26T08:44:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe68039a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b568dc2d50 con 0x55b568ea7680 2024-03-26T08:44:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe68039a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b568eaa7f0 con 0x55b568ea1b20 2024-03-26T08:44:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe68039a740 10 client.0 _ll_register_callbacks cb 0x55b568dc2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568eb3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568eb3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58338/0 (socket says 172.21.15.7:58338) 2024-03-26T08:44:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe676ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eabe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe6777fe640 1 -- 172.21.15.7:0/1591344431 learned_addr learned my addr 172.21.15.7:0/1591344431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe6767fc640 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568eac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe6777fe640 1 -- 172.21.15.7:0/1591344431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 msgr2=0x55b568eabe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe6777fe640 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eabe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe6777fe640 1 -- 172.21.15.7:0/1591344431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 msgr2=0x55b568eac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe6777fe640 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568eac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.879+0000 7fe6777fe640 1 -- 172.21.15.7:0/1591344431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b568be7620 con 0x55b568ea7680 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.880+0000 7fe6777fe640 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568eb3390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe668009900 tx=0x7fe668005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0834238d100ad3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.880+0000 7fe65aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.880+0000 7fe65aff5640 1 -- 172.21.15.7:0/1591344431 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668045040 con 0x55b568ea7680 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.880+0000 7fe68039a740 1 -- 172.21.15.7:0/1591344431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe66802ca40 con 0x55b568ea7680 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.880+0000 7fe65aff5640 1 -- 172.21.15.7:0/1591344431 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe668004ad0 con 0x55b568ea7680 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.880+0000 7fe65aff5640 1 -- 172.21.15.7:0/1591344431 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66805b050 con 0x55b568ea7680 2024-03-26T08:44:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.880+0000 7fe68039a740 10 client.25844 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.880+0000 7fe68039a740 1 -- 172.21.15.7:0/1591344431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b568eb6700 con 0x55b568ea7680 2024-03-26T08:44:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.881+0000 7fe65aff5640 1 -- 172.21.15.7:0/1591344431 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66802ca40 con 0x55b568ea7680 2024-03-26T08:44:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.881+0000 7fe65aff5640 1 -- 172.21.15.7:0/1591344431 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe66803f440 con 0x55b568ea7680 2024-03-26T08:44:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.881+0000 7fe68039a740 10 client.25844 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.881+0000 7fe68039a740 10 client.25844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.881+0000 7fe68039a740 20 client.25844 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.881+0000 7fe68039a740 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b568eb7f30 0x55b568eba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.881+0000 7fe68039a740 4 client.25844 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.881+0000 7fe68039a740 1 -- 172.21.15.7:0/1591344431 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b568e14050 con 0x55b568eb7f30 2024-03-26T08:44:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.883+0000 7fe676ffd640 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b568eb7f30 0x55b568eba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.883+0000 7fe65aff5640 10 client.25844 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.883+0000 7fe676ffd640 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b568eb7f30 0x55b568eba350 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2509029c512f4d5 server_cookie=cee7c473e8898a41 in_seq=0 out_seq=0 2024-03-26T08:44:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.884+0000 7fe65aff5640 1 -- 172.21.15.7:0/1591344431 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b568e14050 con 0x55b568eb7f30 2024-03-26T08:44:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.884+0000 7fe65aff5640 10 client.25844 handle_command_reply: tid=0 2024-03-26T08:44:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.884+0000 7fe68039a740 1 client.25844 shutdown 2024-03-26T08:44:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 -- 172.21.15.7:0/1591344431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 msgr2=0x55b568eb3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568eb3390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe668009900 tx=0x7fe668005870 comp rx=0 tx=0).stop 2024-03-26T08:44:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 -- 172.21.15.7:0/1591344431 shutdown_connections 2024-03-26T08:44:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b568eb7f30 0x55b568eba350 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568eb3390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568eac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 --2- 172.21.15.7:0/1591344431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eabe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe65aff5640 0 client.25844 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:27.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 -- 172.21.15.7:0/1591344431 >> 172.21.15.7:0/1591344431 conn(0x55b568e9d090 msgr2=0x55b568e9c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 -- 172.21.15.7:0/1591344431 shutdown_connections 2024-03-26T08:44:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 1 -- 172.21.15.7:0/1591344431 wait complete. 2024-03-26T08:44:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.885+0000 7fe68039a740 20 client.25844 trim_cache size 0 max 16384 2024-03-26T08:44:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.918+0000 7fe68039a740 1 Processor -- start 2024-03-26T08:44:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.919+0000 7fe68039a740 1 -- start start 2024-03-26T08:44:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.920+0000 7fe68039a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eabe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.920+0000 7fe68039a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea2510 0x55b568eb3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.920+0000 7fe68039a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea7680 0x55b568eac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.920+0000 7fe68039a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781b1490 con 0x55b568ea7680 2024-03-26T08:44:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.920+0000 7fe68039a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781aef90 con 0x55b568ea2510 2024-03-26T08:44:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.920+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea2510 0x55b568eb3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.920+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea2510 0x55b568eb3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58348/0 (socket says 172.21.15.7:58348) 2024-03-26T08:44:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.920+0000 7fe6767fc640 1 -- 172.21.15.7:0/2094644584 learned_addr learned my addr 172.21.15.7:0/2094644584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.921+0000 7fe68039a740 1 -- 172.21.15.7:0/2094644584 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781ad010 con 0x55b568ea1b20 2024-03-26T08:44:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.921+0000 7fe6777fe640 1 --2- 172.21.15.7:0/2094644584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea7680 0x55b568eac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.921+0000 7fe6767fc640 1 -- 172.21.15.7:0/2094644584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 msgr2=0x55b568eabe80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:44:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.921+0000 7fe6767fc640 1 --2- 172.21.15.7:0/2094644584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eabe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.922+0000 7fe6767fc640 1 -- 172.21.15.7:0/2094644584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea7680 msgr2=0x55b568eac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.922+0000 7fe6767fc640 1 --2- 172.21.15.7:0/2094644584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea7680 0x55b568eac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.922+0000 7fe6767fc640 1 -- 172.21.15.7:0/2094644584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b568eba9b0 con 0x55b568ea2510 2024-03-26T08:44:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.922+0000 7fe6767fc640 1 --2- 172.21.15.7:0/2094644584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea2510 0x55b568eb3350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe670005fd0 tx=0x7fe670014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd242106495b00a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.922+0000 7fe65cff9640 1 -- 172.21.15.7:0/2094644584 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670004740 con 0x55b568ea2510 2024-03-26T08:44:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.922+0000 7fe65cff9640 1 -- 172.21.15.7:0/2094644584 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe67002d070 con 0x55b568ea2510 2024-03-26T08:44:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.922+0000 7fe65cff9640 1 -- 172.21.15.7:0/2094644584 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67001d680 con 0x55b568ea2510 2024-03-26T08:44:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.923+0000 7fe68039a740 1 -- 172.21.15.7:0/2094644584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea2510 msgr2=0x55b568eb3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.923+0000 7fe68039a740 1 --2- 172.21.15.7:0/2094644584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea2510 0x55b568eb3350 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe670005fd0 tx=0x7fe670014000 comp rx=0 tx=0).stop 2024-03-26T08:44:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.923+0000 7fe68039a740 1 -- 172.21.15.7:0/2094644584 shutdown_connections 2024-03-26T08:44:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.923+0000 7fe68039a740 1 --2- 172.21.15.7:0/2094644584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea7680 0x55b568eac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.924+0000 7fe68039a740 1 --2- 172.21.15.7:0/2094644584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea2510 0x55b568eb3350 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.924+0000 7fe68039a740 1 --2- 172.21.15.7:0/2094644584 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eabe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.924+0000 7fe68039a740 1 -- 172.21.15.7:0/2094644584 >> 172.21.15.7:0/2094644584 conn(0x55b568e9d090 msgr2=0x55b568ea37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.924+0000 7fe68039a740 1 -- 172.21.15.7:0/2094644584 shutdown_connections 2024-03-26T08:44:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.924+0000 7fe68039a740 1 -- 172.21.15.7:0/2094644584 wait complete. 2024-03-26T08:44:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe68039a740 1 -- start start 2024-03-26T08:44:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe68039a740 1 Processor -- start 2024-03-26T08:44:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe68039a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eb0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe68039a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568eb1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe676ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eb0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe68039a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568eb2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe676ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eb0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35248/0 (socket says 172.21.15.7:35248) 2024-03-26T08:44:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe68039a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781ad610 con 0x55b568ea2510 2024-03-26T08:44:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe68039a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781ae930 con 0x55b568ea7680 2024-03-26T08:44:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe68039a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6781adc70 con 0x55b568ea1b20 2024-03-26T08:44:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe68039a740 10 client.0 _ll_register_callbacks cb 0x7fe668009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.925+0000 7fe676ffd640 1 -- 172.21.15.7:0/1000175051 learned_addr learned my addr 172.21.15.7:0/1000175051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.926+0000 7fe6767fc640 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568eb1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.926+0000 7fe676ffd640 1 -- 172.21.15.7:0/1000175051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 msgr2=0x55b568eb2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.926+0000 7fe676ffd640 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568eb2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.926+0000 7fe676ffd640 1 -- 172.21.15.7:0/1000175051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 msgr2=0x55b568eb1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.926+0000 7fe676ffd640 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568eb1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.926+0000 7fe676ffd640 1 -- 172.21.15.7:0/1000175051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b568eb2870 con 0x55b568ea1b20 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.926+0000 7fe676ffd640 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eb0d30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe66c096d70 tx=0x7fe66c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94fdf28a04871e43 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.926+0000 7fe65aff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.927+0000 7fe65aff5640 1 -- 172.21.15.7:0/1000175051 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0a30b0 con 0x55b568ea1b20 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.927+0000 7fe65aff5640 1 -- 172.21.15.7:0/1000175051 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe66c017360 con 0x55b568ea1b20 2024-03-26T08:44:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.927+0000 7fe68039a740 1 -- 172.21.15.7:0/1000175051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe67002d070 con 0x55b568ea1b20 2024-03-26T08:44:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.928+0000 7fe65aff5640 1 -- 172.21.15.7:0/1000175051 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe66c096f80 con 0x55b568ea1b20 2024-03-26T08:44:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.929+0000 7fe68039a740 10 client.25867 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.929+0000 7fe68039a740 1 -- 172.21.15.7:0/1000175051 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b568f13ab0 con 0x55b568ea1b20 2024-03-26T08:44:27.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.929+0000 7fe65aff5640 1 -- 172.21.15.7:0/1000175051 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe66c099d60 con 0x55b568ea1b20 2024-03-26T08:44:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.929+0000 7fe65aff5640 1 -- 172.21.15.7:0/1000175051 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe66c096140 con 0x55b568ea1b20 2024-03-26T08:44:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.930+0000 7fe68039a740 10 client.25867 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.930+0000 7fe68039a740 10 client.25867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.930+0000 7fe68039a740 20 client.25867 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.930+0000 7fe68039a740 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b568f14460 0x55b568f16880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.930+0000 7fe68039a740 4 client.25867 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:44:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.930+0000 7fe68039a740 1 -- 172.21.15.7:0/1000175051 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fe67002d070 con 0x55b568f14460 2024-03-26T08:44:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.930+0000 7fe6767fc640 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b568f14460 0x55b568f16880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.931+0000 7fe6767fc640 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b568f14460 0x55b568f16880 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c5d84102d9e1406 server_cookie=c94c90a75e2ae5e4 in_seq=0 out_seq=0 2024-03-26T08:44:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.931+0000 7fe65aff5640 10 client.25867 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.974+0000 7fe65aff5640 1 -- 172.21.15.7:0/1000175051 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fe67002d070 con 0x55b568f14460 2024-03-26T08:44:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.974+0000 7fe65aff5640 10 client.25867 handle_command_reply: tid=0 2024-03-26T08:44:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.974+0000 7fe68039a740 1 client.25867 shutdown 2024-03-26T08:44:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.974+0000 7fe68039a740 1 -- 172.21.15.7:0/1000175051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 msgr2=0x55b568eb0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.965 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.974+0000 7fe68039a740 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eb0d30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe66c096d70 tx=0x7fe66c000bc0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.975+0000 7fe68039a740 1 -- 172.21.15.7:0/1000175051 shutdown_connections 2024-03-26T08:44:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.975+0000 7fe68039a740 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b568f14460 0x55b568f16880 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.975+0000 7fe68039a740 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b568ea7680 0x55b568eb2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.975+0000 7fe68039a740 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b568ea2510 0x55b568eb1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.975+0000 7fe68039a740 1 --2- 172.21.15.7:0/1000175051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b568ea1b20 0x55b568eb0d30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.975+0000 7fe68039a740 1 -- 172.21.15.7:0/1000175051 >> 172.21.15.7:0/1000175051 conn(0x55b568e9d090 msgr2=0x55b568eabe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.975+0000 7fe65aff5640 0 client.25867 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.976+0000 7fe68039a740 1 -- 172.21.15.7:0/1000175051 shutdown_connections 2024-03-26T08:44:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.976+0000 7fe68039a740 1 -- 172.21.15.7:0/1000175051 wait complete. 2024-03-26T08:44:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.977+0000 7fe68039a740 20 client.25867 trim_cache size 0 max 16384 2024-03-26T08:44:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:44:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:44:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:44:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:44:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.978+0000 7fe67d7b3640 1 -- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6600a29b0 msgr2=0x7fe6600a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.978+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6600a29b0 0x7fe6600a4e70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe67819f310 tx=0x7fe670009500 comp rx=0 tx=0).stop 2024-03-26T08:44:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.978+0000 7fe67d7b3640 1 -- 172.21.15.7:0/1361445450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe678076060 msgr2=0x7fe67819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.969 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.978+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe678076060 0x7fe67819de00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00b4a0 tx=0x7fe66c00b960 comp rx=0 tx=0).stop 2024-03-26T08:44:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.980+0000 7fe67d7b3640 1 -- 172.21.15.7:0/1361445450 shutdown_connections 2024-03-26T08:44:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.980+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6600a29b0 0x7fe6600a4e70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.971 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.980+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe678077c40 0x7fe6781a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.980+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6780772a0 0x7fe67819e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.980+0000 7fe67d7b3640 1 --2- 172.21.15.7:0/1361445450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe678076060 0x7fe67819de00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.980+0000 7fe67d7b3640 1 -- 172.21.15.7:0/1361445450 >> 172.21.15.7:0/1361445450 conn(0x7fe6780fecd0 msgr2=0x7fe6781007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.981+0000 7fe67d7b3640 1 -- 172.21.15.7:0/1361445450 shutdown_connections 2024-03-26T08:44:27.972 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:27.981+0000 7fe67d7b3640 1 -- 172.21.15.7:0/1361445450 wait complete. 2024-03-26T08:44:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.989+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/1512284181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb41017b0 msgr2=0x7f4fb4101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.989+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/1512284181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb41017b0 0x7f4fb4101c30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009900 tx=0x7f4fa402bc30 comp rx=0 tx=0).stop 2024-03-26T08:44:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.990+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/1512284181 shutdown_connections 2024-03-26T08:44:27.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.990+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/1512284181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fb4102170 0x7f4fb4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.990+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/1512284181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb41017b0 0x7f4fb4101c30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.990+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/1512284181 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb41005c0 0x7f4fb41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.990+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/1512284181 >> 172.21.15.7:0/1512284181 conn(0x7f4fb40fbb30 msgr2=0x7f4fb40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.990+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/1512284181 shutdown_connections 2024-03-26T08:44:27.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.990+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/1512284181 wait complete. 2024-03-26T08:44:27.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.992+0000 7f4fbb9b3640 1 Processor -- start 2024-03-26T08:44:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fbb9b3640 1 -- start start 2024-03-26T08:44:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fbb9b3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb41005c0 0x7f4fb419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fbb9b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fb41017b0 0x7f4fb419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fbb9b3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb4102170 0x7f4fb41a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fbb9b3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb410b190 con 0x7f4fb4102170 2024-03-26T08:44:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fbb9b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb410b010 con 0x7f4fb41017b0 2024-03-26T08:44:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fbb9b3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb410b310 con 0x7f4fb41005c0 2024-03-26T08:44:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fb974f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb41005c0 0x7f4fb419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fb974f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb41005c0 0x7f4fb419dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35258/0 (socket says 172.21.15.7:35258) 2024-03-26T08:44:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fb974f640 1 -- 172.21.15.7:0/2842723550 learned_addr learned my addr 172.21.15.7:0/2842723550 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.994+0000 7f4fb9f50640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb4102170 0x7f4fb41a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fb41017b0 0x7f4fb419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4fb9f50640 1 -- 172.21.15.7:0/2842723550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb41005c0 msgr2=0x7f4fb419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4fb9f50640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb41005c0 0x7f4fb419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4fb9f50640 1 -- 172.21.15.7:0/2842723550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fb41017b0 msgr2=0x7f4fb419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4fb9f50640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fb41017b0 0x7f4fb419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4fb9f50640 1 -- 172.21.15.7:0/2842723550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb41a5c10 con 0x7f4fb4102170 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4fb9f50640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb4102170 0x7f4fb41a55a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009fd0 tx=0x7f4fac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4faa7fc640 1 -- 172.21.15.7:0/2842723550 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00ce20 con 0x7f4fb4102170 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/2842723550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb41a5f00 con 0x7f4fb4102170 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4faa7fc640 1 -- 172.21.15.7:0/2842723550 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fac010430 con 0x7f4fb4102170 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.995+0000 7f4faa7fc640 1 -- 172.21.15.7:0/2842723550 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00f620 con 0x7f4fb4102170 2024-03-26T08:44:27.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:27.996+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/2842723550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb41a6310 con 0x7f4fb4102170 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.000+0000 7f4fbe596740 1 Processor -- start 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.000+0000 7f4fbe596740 1 -- start start 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.000+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 0x564f233181d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.000+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f233187a0 0x564f2331d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.000+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f2331d910 0x564f2331fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fbe596740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f2326c3d0 con 0x564f2331d910 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fbe596740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f23214ad0 con 0x564f23317db0 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fbe596740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f23268470 con 0x564f233187a0 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fb974f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f233187a0 0x564f2331d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fb974f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f233187a0 0x564f2331d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35274/0 (socket says 172.21.15.7:35274) 2024-03-26T08:44:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fb974f640 1 -- 172.21.15.7:0/866097088 learned_addr learned my addr 172.21.15.7:0/866097088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/866097088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 0x564f233181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fb974f640 1 -- 172.21.15.7:0/866097088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 msgr2=0x564f233181d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fb974f640 1 --2- 172.21.15.7:0/866097088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 0x564f233181d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fb974f640 1 -- 172.21.15.7:0/866097088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f2331d910 msgr2=0x564f2331fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.001+0000 7f4fb974f640 1 --2- 172.21.15.7:0/866097088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f2331d910 0x564f2331fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.002+0000 7f4fb974f640 1 -- 172.21.15.7:0/866097088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f2304a7d0 con 0x564f233187a0 2024-03-26T08:44:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.002+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/866097088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 0x564f233181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:44:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.002+0000 7f4faa7fc640 1 -- 172.21.15.7:0/2842723550 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4fac010950 con 0x7f4fb4102170 2024-03-26T08:44:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.002+0000 7f4fb974f640 1 --2- 172.21.15.7:0/866097088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f233187a0 0x564f2331d2a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000acb0 tx=0x7f4fb00317a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b43104912ba6a7e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:27.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.004+0000 7f4f9f7fe640 1 -- 172.21.15.7:0/866097088 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0005220 con 0x564f233187a0 2024-03-26T08:44:27.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.004+0000 7f4faa7fc640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4fa00a2930 0x7f4fa00a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:27.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.005+0000 7f4f9f7fe640 1 -- 172.21.15.7:0/866097088 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fb0004c80 con 0x564f233187a0 2024-03-26T08:44:27.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.005+0000 7f4faa7fc640 1 -- 172.21.15.7:0/2842723550 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4fac0ced60 con 0x7f4fb4102170 2024-03-26T08:44:27.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.005+0000 7f4f9f7fe640 1 -- 172.21.15.7:0/866097088 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb003a9c0 con 0x564f233187a0 2024-03-26T08:44:27.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.005+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4fa00a2930 0x7f4fa00a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:27.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.006+0000 7f4fbe596740 1 -- 172.21.15.7:0/866097088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f233187a0 msgr2=0x564f2331d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:27.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.006+0000 7f4fbe596740 1 --2- 172.21.15.7:0/866097088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f233187a0 0x564f2331d2a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000acb0 tx=0x7f4fb00317a0 comp rx=0 tx=0).stop 2024-03-26T08:44:27.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.008+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4fa00a2930 0x7f4fa00a4df0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4fa402c240 tx=0x7f4fa4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:28.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.008+0000 7f4fbe596740 1 -- 172.21.15.7:0/866097088 shutdown_connections 2024-03-26T08:44:28.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.008+0000 7f4fbe596740 1 --2- 172.21.15.7:0/866097088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f2331d910 0x564f2331fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.008+0000 7f4fbe596740 1 --2- 172.21.15.7:0/866097088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f233187a0 0x564f2331d2a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.008+0000 7f4fbe596740 1 --2- 172.21.15.7:0/866097088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 0x564f233181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.009+0000 7f4fbe596740 1 -- 172.21.15.7:0/866097088 >> 172.21.15.7:0/866097088 conn(0x564f23302310 msgr2=0x564f23302710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:28.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.009+0000 7f4fbe596740 1 -- 172.21.15.7:0/866097088 shutdown_connections 2024-03-26T08:44:28.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.009+0000 7f4fbe596740 1 -- 172.21.15.7:0/866097088 wait complete. 2024-03-26T08:44:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.010+0000 7f4fbe596740 1 -- start start 2024-03-26T08:44:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.010+0000 7f4fbe596740 1 Processor -- start 2024-03-26T08:44:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.010+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 0x564f23308580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.011+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f23308ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.011+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f2331d910 0x564f2330fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.011+0000 7f4fbe596740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f23306cd0 con 0x564f233187a0 2024-03-26T08:44:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.011+0000 7f4fb8f4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f23308ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.011+0000 7f4fb8f4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f23308ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50548/0 (socket says 172.21.15.7:50548) 2024-03-26T08:44:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.011+0000 7f4fb8f4e640 1 -- 172.21.15.7:0/971507510 learned_addr learned my addr 172.21.15.7:0/971507510 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.011+0000 7f4fb9f50640 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f2331d910 0x564f2330fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.011+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f23228060 con 0x564f23317db0 2024-03-26T08:44:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.011+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f23306e50 con 0x564f2331d910 2024-03-26T08:44:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.012+0000 7f4fbe596740 10 client.0 _ll_register_callbacks cb 0x564f2320a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.012+0000 7f4fb8f4e640 1 -- 172.21.15.7:0/971507510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f2331d910 msgr2=0x564f2330fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.012+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f2331d910 0x564f2330fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.012+0000 7f4fb8f4e640 1 -- 172.21.15.7:0/971507510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 msgr2=0x564f23308580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.012+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 0x564f23308580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.012+0000 7f4fb8f4e640 1 -- 172.21.15.7:0/971507510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f22f803f0 con 0x564f233187a0 2024-03-26T08:44:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.013+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f23308ac0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4fa402c710 tx=0x7f4fa402df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1731dcaab855a3e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.013+0000 7f4f9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:44:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.013+0000 7f4f9cff9640 1 -- 172.21.15.7:0/971507510 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4050020 con 0x564f233187a0 2024-03-26T08:44:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.013+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fb0004c80 con 0x564f233187a0 2024-03-26T08:44:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.014+0000 7f4f9cff9640 1 -- 172.21.15.7:0/971507510 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4fa4005310 con 0x564f233187a0 2024-03-26T08:44:28.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.014+0000 7f4f9cff9640 1 -- 172.21.15.7:0/971507510 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fa4005510 con 0x564f233187a0 2024-03-26T08:44:28.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.014+0000 7f4fbe596740 10 client.16017 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:28.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.014+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564f22f400f0 con 0x564f233187a0 2024-03-26T08:44:28.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.015+0000 7f4f9cff9640 1 -- 172.21.15.7:0/971507510 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fa4004ba0 con 0x564f233187a0 2024-03-26T08:44:28.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.016+0000 7f4f9cff9640 1 -- 172.21.15.7:0/971507510 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4fa40039d0 con 0x564f233187a0 2024-03-26T08:44:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.016+0000 7f4fbe596740 10 client.16017 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.016+0000 7f4fbe596740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.016+0000 7f4fbe596740 20 client.16017 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.016+0000 7f4fbe596740 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564f23313f40 0x564f233163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.016+0000 7f4fbe596740 4 client.16017 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.016+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fb0004c80 con 0x564f23313f40 2024-03-26T08:44:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.017+0000 7f4fb974f640 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564f23313f40 0x564f233163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.017+0000 7f4fb974f640 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564f23313f40 0x564f233163e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab5d901469cdbfc server_cookie=c6844fff97323a79 in_seq=0 out_seq=0 2024-03-26T08:44:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.017+0000 7f4f9cff9640 10 client.16017 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.018+0000 7f4f9cff9640 1 -- 172.21.15.7:0/971507510 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f4fb0004c80 con 0x564f23313f40 2024-03-26T08:44:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.018+0000 7f4f9cff9640 10 client.16017 handle_command_reply: tid=0 2024-03-26T08:44:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.018+0000 7f4fbe596740 1 client.16017 shutdown 2024-03-26T08:44:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.019+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 msgr2=0x564f23308ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:28.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.019+0000 7f4fbe596740 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f23308ac0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4fa402c710 tx=0x7f4fa402df90 comp rx=0 tx=0).stop 2024-03-26T08:44:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.019+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 shutdown_connections 2024-03-26T08:44:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.019+0000 7f4fbe596740 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564f23313f40 0x564f233163e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.019+0000 7f4fbe596740 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f2331d910 0x564f2330fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.019+0000 7f4fbe596740 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f23308ac0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.019+0000 7f4fbe596740 1 --2- 172.21.15.7:0/971507510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f23317db0 0x564f23308580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.020+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 >> 172.21.15.7:0/971507510 conn(0x564f23302310 msgr2=0x564f23301530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.020+0000 7f4f9cff9640 0 client.16017 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.020+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 shutdown_connections 2024-03-26T08:44:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.020+0000 7f4fbe596740 1 -- 172.21.15.7:0/971507510 wait complete. 2024-03-26T08:44:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.021+0000 7f4fbe596740 20 client.16017 trim_cache size 0 max 16384 2024-03-26T08:44:28.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.054+0000 7f4fbe596740 1 Processor -- start 2024-03-26T08:44:28.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.055+0000 7f4fbe596740 1 -- start start 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.055+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f23319b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.055+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f2330fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.055+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f2331d910 0x564f2331a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.055+0000 7f4fbe596740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb41a8cd0 con 0x564f233187a0 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.055+0000 7f4fbe596740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb41a67d0 con 0x564f2331d910 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb974f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f23319b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb8f4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f2330fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb8f4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f2330fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50552/0 (socket says 172.21.15.7:50552) 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb8f4e640 1 -- 172.21.15.7:0/3381884170 learned_addr learned my addr 172.21.15.7:0/3381884170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb974f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f23319b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35300/0 (socket says 172.21.15.7:35300) 2024-03-26T08:44:28.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fbe596740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb4071c40 con 0x564f23317db0 2024-03-26T08:44:28.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb9f50640 1 --2- 172.21.15.7:0/3381884170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f2331d910 0x564f2331a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:28.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb974f640 1 -- 172.21.15.7:0/3381884170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f2331d910 msgr2=0x564f2331a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:28.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb974f640 1 --2- 172.21.15.7:0/3381884170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f2331d910 0x564f2331a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb974f640 1 -- 172.21.15.7:0/3381884170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 msgr2=0x564f2330fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:28.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb974f640 1 --2- 172.21.15.7:0/3381884170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f2330fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb974f640 1 -- 172.21.15.7:0/3381884170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f232fdb60 con 0x564f23317db0 2024-03-26T08:44:28.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.056+0000 7f4fb974f640 1 --2- 172.21.15.7:0/3381884170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f23319b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x564f23303280 tx=0x7f4fb0005680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc2732fb129fcc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:28.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.057+0000 7f4f9effd640 1 -- 172.21.15.7:0/3381884170 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0051070 con 0x564f23317db0 2024-03-26T08:44:28.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.057+0000 7f4f9effd640 1 -- 172.21.15.7:0/3381884170 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fb0000bc0 con 0x564f23317db0 2024-03-26T08:44:28.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.057+0000 7f4f9effd640 1 -- 172.21.15.7:0/3381884170 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0003e20 con 0x564f23317db0 2024-03-26T08:44:28.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.058+0000 7f4fbe596740 1 -- 172.21.15.7:0/3381884170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 msgr2=0x564f23319b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:28.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.058+0000 7f4fbe596740 1 --2- 172.21.15.7:0/3381884170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f23319b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x564f23303280 tx=0x7f4fb0005680 comp rx=0 tx=0).stop 2024-03-26T08:44:28.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.058+0000 7f4fbe596740 1 -- 172.21.15.7:0/3381884170 shutdown_connections 2024-03-26T08:44:28.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.058+0000 7f4fbe596740 1 --2- 172.21.15.7:0/3381884170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f2331d910 0x564f2331a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.058+0000 7f4fbe596740 1 --2- 172.21.15.7:0/3381884170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f233187a0 0x564f2330fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.058+0000 7f4fbe596740 1 --2- 172.21.15.7:0/3381884170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f23319b20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.058+0000 7f4fbe596740 1 -- 172.21.15.7:0/3381884170 >> 172.21.15.7:0/3381884170 conn(0x564f23302310 msgr2=0x564f23308580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:28.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.059+0000 7f4fbe596740 1 -- 172.21.15.7:0/3381884170 shutdown_connections 2024-03-26T08:44:28.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.059+0000 7f4fbe596740 1 -- 172.21.15.7:0/3381884170 wait complete. 2024-03-26T08:44:28.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.060+0000 7f4fbe596740 1 -- start start 2024-03-26T08:44:28.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.060+0000 7f4fbe596740 1 Processor -- start 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f2330c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f233187a0 0x564f2330cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fbe596740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f2331d910 0x564f23377260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fbe596740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40721b0 con 0x564f2331d910 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fbe596740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb4073490 con 0x564f233187a0 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fbe596740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb40727d0 con 0x564f23317db0 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fbe596740 10 client.0 _ll_register_callbacks cb 0x7f4fa4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fb974f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f2330c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fb974f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f2330c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35308/0 (socket says 172.21.15.7:35308) 2024-03-26T08:44:28.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fb974f640 1 -- 172.21.15.7:0/1261590775 learned_addr learned my addr 172.21.15.7:0/1261590775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:28.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f233187a0 0x564f2330cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:28.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fb8f4e640 1 -- 172.21.15.7:0/1261590775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 msgr2=0x564f2330c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f2330c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fb8f4e640 1 -- 172.21.15.7:0/1261590775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f2331d910 msgr2=0x564f23377260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.061+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f2331d910 0x564f23377260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.062+0000 7f4fb8f4e640 1 -- 172.21.15.7:0/1261590775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f2330fc50 con 0x564f233187a0 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.062+0000 7f4fb8f4e640 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f233187a0 0x564f2330cbf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009850 tx=0x7f4fa4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1c4edc668b0758 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.062+0000 7f4f9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.062+0000 7f4f9cff9640 1 -- 172.21.15.7:0/1261590775 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4050040 con 0x564f233187a0 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.062+0000 7f4f9cff9640 1 -- 172.21.15.7:0/1261590775 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4fa403fa60 con 0x564f233187a0 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4fbe596740 1 -- 172.21.15.7:0/1261590775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fb0000bc0 con 0x564f233187a0 2024-03-26T08:44:28.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4f9cff9640 1 -- 172.21.15.7:0/1261590775 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fa403fc60 con 0x564f233187a0 2024-03-26T08:44:28.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4fbe596740 10 client.25859 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:28.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4fbe596740 1 -- 172.21.15.7:0/1261590775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564f23378790 con 0x564f233187a0 2024-03-26T08:44:28.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4f9cff9640 1 -- 172.21.15.7:0/1261590775 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fb0000bc0 con 0x564f233187a0 2024-03-26T08:44:28.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4f9cff9640 1 -- 172.21.15.7:0/1261590775 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4fa405f030 con 0x564f233187a0 2024-03-26T08:44:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4fbe596740 10 client.25859 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4fbe596740 10 client.25859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4fbe596740 20 client.25859 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.063+0000 7f4fbe596740 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564f23378ef0 0x564f2337b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.064+0000 7f4fbe596740 4 client.25859 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:44:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.064+0000 7f4fbe596740 1 -- 172.21.15.7:0/1261590775 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f23279910 con 0x564f23378ef0 2024-03-26T08:44:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.064+0000 7f4fb974f640 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564f23378ef0 0x564f2337b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:28.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.065+0000 7f4fb974f640 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564f23378ef0 0x564f2337b310 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec15364918ecd066 server_cookie=a522730cdcd75db0 in_seq=0 out_seq=0 2024-03-26T08:44:28.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.065+0000 7f4f9cff9640 10 client.25859 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4f9cff9640 1 -- 172.21.15.7:0/1261590775 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564f23279910 con 0x564f23378ef0 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4f9cff9640 10 client.25859 handle_command_reply: tid=0 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4fbe596740 1 client.25859 shutdown 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4fbe596740 1 -- 172.21.15.7:0/1261590775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f233187a0 msgr2=0x564f2330cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4fbe596740 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f233187a0 0x564f2330cbf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009850 tx=0x7f4fa4003b90 comp rx=0 tx=0).stop 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4fbe596740 1 -- 172.21.15.7:0/1261590775 shutdown_connections 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4fbe596740 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564f23378ef0 0x564f2337b310 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4fbe596740 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564f2331d910 0x564f23377260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4fbe596740 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f233187a0 0x564f2330cbf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4fbe596740 1 --2- 172.21.15.7:0/1261590775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564f23317db0 0x564f2330c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4f9cff9640 0 client.25859 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.066+0000 7f4fbe596740 1 -- 172.21.15.7:0/1261590775 >> 172.21.15.7:0/1261590775 conn(0x564f23302310 msgr2=0x564f23308450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.067+0000 7f4fbe596740 1 -- 172.21.15.7:0/1261590775 shutdown_connections 2024-03-26T08:44:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbe596740 1 -- 172.21.15.7:0/1261590775 wait complete. 2024-03-26T08:44:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbe596740 20 client.25859 trim_cache size 0 max 16384 2024-03-26T08:44:28.059 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:44:28.060 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:44:28.060 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:44:28.060 INFO:teuthology.orchestra.run.smithi007.stdout: "036bb239-7520-4988-b988-0fa31906f63e": { 2024-03-26T08:44:28.060 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:44:28.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "036bb239-7520-4988-b988-0fa31906f63e", 2024-03-26T08:44:28.060 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:44:28.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:44:28.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:44:28.060 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4fa00a2930 msgr2=0x7f4fa00a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4fa00a2930 0x7f4fa00a4df0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4fa402c240 tx=0x7f4fa4005fd0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb4102170 msgr2=0x7f4fb41a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb4102170 0x7f4fb41a55a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009fd0 tx=0x7f4fac00ef10 comp rx=0 tx=0).stop 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/2842723550 shutdown_connections 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4fa00a2930 0x7f4fa00a4df0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fb4102170 0x7f4fb41a55a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.068+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fb41017b0 0x7f4fb419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.069+0000 7f4fbb9b3640 1 --2- 172.21.15.7:0/2842723550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4fb41005c0 0x7f4fb419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.069+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/2842723550 >> 172.21.15.7:0/2842723550 conn(0x7f4fb40fbb30 msgr2=0x7f4fb4104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:28.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.070+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/2842723550 shutdown_connections 2024-03-26T08:44:28.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:28.071+0000 7f4fbb9b3640 1 -- 172.21.15.7:0/2842723550 wait complete. 2024-03-26T08:44:28.079 INFO:tasks.cephfs.filesystem:scrub status for tag:036bb239-7520-4988-b988-0fa31906f63e - {'path': '/', 'tag': '036bb239-7520-4988-b988-0fa31906f63e', 'options': 'recursive,force'} 2024-03-26T08:44:28.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:28 smithi016 ceph-mon[35851]: pgmap v421: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 906 op/s 2024-03-26T08:44:28.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:28 smithi016 ceph-mon[35851]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:44:28.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:28 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:44:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:28 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:44:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:28 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:44:28.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:28 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:44:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:28 smithi007 ceph-mon[32462]: pgmap v421: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 906 op/s 2024-03-26T08:44:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:28 smithi007 ceph-mon[32462]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:44:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:28 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:44:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:28 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:44:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:28 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:44:28.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:28 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:44:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:28 smithi044 ceph-mon[36461]: pgmap v421: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 906 op/s 2024-03-26T08:44:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:28 smithi044 ceph-mon[36461]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T08:44:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:28 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:44:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:28 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:44:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:28 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:44:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:28 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:44:29.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:29 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:44:29.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:29 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:44:29.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:29 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:44:29.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:29 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:44:29.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:29 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:44:29.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:29 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:44:30.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:30 smithi016 ceph-mon[35851]: pgmap v422: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 860 op/s 2024-03-26T08:44:30.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:30 smithi007 ceph-mon[32462]: pgmap v422: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 860 op/s 2024-03-26T08:44:30.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:30 smithi044 ceph-mon[36461]: pgmap v422: 97 pgs: 97 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 860 op/s 2024-03-26T08:44:31.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 120.0 s, 4101.7 tps, lat 24.375 ms stddev 25.750 2024-03-26T08:44:32.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:32 smithi016 ceph-mon[35851]: pgmap v423: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 17 MiB/s wr, 899 op/s 2024-03-26T08:44:32.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:32 smithi007 ceph-mon[32462]: pgmap v423: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 17 MiB/s wr, 899 op/s 2024-03-26T08:44:32.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:32 smithi044 ceph-mon[36461]: pgmap v423: 97 pgs: 97 active+clean; 1.2 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 17 MiB/s wr, 899 op/s 2024-03-26T08:44:34.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:34 smithi016 ceph-mon[35851]: pgmap v424: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-03-26T08:44:34.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:34 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:44:34.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:34 smithi007 ceph-mon[32462]: pgmap v424: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-03-26T08:44:34.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:34 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:44:34.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:34 smithi044 ceph-mon[36461]: pgmap v424: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 18 MiB/s wr, 1.07k op/s 2024-03-26T08:44:34.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:34 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:44:35.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:44:35.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:44:35.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:35 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:44:35.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:44:35.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:44:35.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:35 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:44:35.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:44:35.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:44:35.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:35 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:44:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 125.0 s, 3875.4 tps, lat 25.796 ms stddev 31.789 2024-03-26T08:44:36.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:36 smithi016 ceph-mon[35851]: pgmap v425: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 17 MiB/s wr, 985 op/s 2024-03-26T08:44:36.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:36 smithi007 ceph-mon[32462]: pgmap v425: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 17 MiB/s wr, 985 op/s 2024-03-26T08:44:36.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:36 smithi044 ceph-mon[36461]: pgmap v425: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 17 MiB/s wr, 985 op/s 2024-03-26T08:44:37.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:37 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:44:37.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:37 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:44:37.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:37 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:44:37.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:37 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:44:37.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:37 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:44:37.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:37 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:44:38.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:38 smithi016 ceph-mon[35851]: pgmap v426: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 15 MiB/s wr, 899 op/s 2024-03-26T08:44:38.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:38 smithi007 ceph-mon[32462]: pgmap v426: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 15 MiB/s wr, 899 op/s 2024-03-26T08:44:38.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:38 smithi044 ceph-mon[36461]: pgmap v426: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 15 MiB/s wr, 899 op/s 2024-03-26T08:44:40.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:40 smithi016 ceph-mon[35851]: pgmap v427: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 15 MiB/s wr, 973 op/s 2024-03-26T08:44:40.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:40 smithi007 ceph-mon[32462]: pgmap v427: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 15 MiB/s wr, 973 op/s 2024-03-26T08:44:40.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:40 smithi044 ceph-mon[36461]: pgmap v427: 97 pgs: 97 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 15 MiB/s wr, 973 op/s 2024-03-26T08:44:41.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 130.0 s, 3877.0 tps, lat 25.748 ms stddev 27.777 2024-03-26T08:44:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 -- 172.21.15.7:0/2774356566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e401028e0 msgr2=0x7f1e40104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 --2- 172.21.15.7:0/2774356566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e401028e0 0x7f1e40104d50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009860 tx=0x7f1e3002bb70 comp rx=0 tx=0).stop 2024-03-26T08:44:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 -- 172.21.15.7:0/2774356566 shutdown_connections 2024-03-26T08:44:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 --2- 172.21.15.7:0/2774356566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e40105290 0x7f1e40105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 --2- 172.21.15.7:0/2774356566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e401028e0 0x7f1e40104d50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 --2- 172.21.15.7:0/2774356566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e400fff70 0x7f1e401023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 -- 172.21.15.7:0/2774356566 >> 172.21.15.7:0/2774356566 conn(0x7f1e400fbb70 msgr2=0x7f1e400fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 -- 172.21.15.7:0/2774356566 shutdown_connections 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 -- 172.21.15.7:0/2774356566 wait complete. 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 Processor -- start 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 -- start start 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e400fff70 0x7f1e4019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e401028e0 0x7f1e4019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e40105290 0x7f1e401a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40113950 con 0x7f1e40105290 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401137d0 con 0x7f1e400fff70 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.112+0000 7f1e475cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40113ad0 con 0x7f1e401028e0 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45b6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e40105290 0x7f1e401a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45369640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e400fff70 0x7f1e4019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45b6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e40105290 0x7f1e401a5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33280/0 (socket says 172.21.15.7:33280) 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e44b68640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e401028e0 0x7f1e4019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45b6a640 1 -- 172.21.15.7:0/1437783262 learned_addr learned my addr 172.21.15.7:0/1437783262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45369640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e400fff70 0x7f1e4019df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59480/0 (socket says 172.21.15.7:59480) 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45369640 1 -- 172.21.15.7:0/1437783262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e401028e0 msgr2=0x7f1e4019e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45369640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e401028e0 0x7f1e4019e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45369640 1 -- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e40105290 msgr2=0x7f1e401a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45369640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e40105290 0x7f1e401a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45369640 1 -- 172.21.15.7:0/1437783262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e401a5b90 con 0x7f1e400fff70 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e44b68640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e401028e0 0x7f1e4019e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.113+0000 7f1e45b6a640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e40105290 0x7f1e401a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.114+0000 7f1e45369640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e400fff70 0x7f1e4019df70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005fd0 tx=0x7f1e3c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.114+0000 7f1e367fc640 1 -- 172.21.15.7:0/1437783262 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00cd00 con 0x7f1e400fff70 2024-03-26T08:44:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.114+0000 7f1e367fc640 1 -- 172.21.15.7:0/1437783262 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c0035f0 con 0x7f1e400fff70 2024-03-26T08:44:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.114+0000 7f1e475cd640 1 -- 172.21.15.7:0/1437783262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e401a5e80 con 0x7f1e400fff70 2024-03-26T08:44:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.114+0000 7f1e367fc640 1 -- 172.21.15.7:0/1437783262 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c010690 con 0x7f1e400fff70 2024-03-26T08:44:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.114+0000 7f1e475cd640 1 -- 172.21.15.7:0/1437783262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e401a62e0 con 0x7f1e400fff70 2024-03-26T08:44:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.117+0000 7f1e4a1b0740 1 Processor -- start 2024-03-26T08:44:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.117+0000 7f1e367fc640 1 -- 172.21.15.7:0/1437783262 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1e3c004020 con 0x7f1e400fff70 2024-03-26T08:44:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.118+0000 7f1e367fc640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1e2c0a29c0 0x7f1e2c0a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.118+0000 7f1e4a1b0740 1 -- start start 2024-03-26T08:44:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e367fc640 1 -- 172.21.15.7:0/1437783262 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1e3c0ceb40 con 0x7f1e400fff70 2024-03-26T08:44:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128aa310 0x563b128aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128af480 0x563b128b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b1280eb10 con 0x563b128af480 2024-03-26T08:44:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b127b7900 con 0x563b128aa310 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b1280abb0 con 0x563b128a9920 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45369640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45b6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128af480 0x563b128b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45369640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128a9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46872/0 (socket says 172.21.15.7:46872) 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45369640 1 -- 172.21.15.7:0/1304299591 learned_addr learned my addr 172.21.15.7:0/1304299591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e44b68640 1 --2- 172.21.15.7:0/1304299591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128aa310 0x563b128aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45b6a640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1e2c0a29c0 0x7f1e2c0a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45369640 1 -- 172.21.15.7:0/1304299591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128aa310 msgr2=0x563b128aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45369640 1 --2- 172.21.15.7:0/1304299591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128aa310 0x563b128aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45369640 1 -- 172.21.15.7:0/1304299591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128af480 msgr2=0x563b128b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45369640 1 --2- 172.21.15.7:0/1304299591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128af480 0x563b128b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.119+0000 7f1e45369640 1 -- 172.21.15.7:0/1304299591 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b12523210 con 0x563b128a9920 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.120+0000 7f1e45369640 1 --2- 172.21.15.7:0/1304299591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128a9d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c09b430 tx=0x7f1e3c09ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97d79f973b010f97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.120+0000 7f1e45b6a640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1e2c0a29c0 0x7f1e2c0a4e80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1e380094f0 tx=0x7f1e38009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.120+0000 7f1e2b7fe640 1 -- 172.21.15.7:0/1304299591 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c09b4a0 con 0x563b128a9920 2024-03-26T08:44:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.120+0000 7f1e2b7fe640 1 -- 172.21.15.7:0/1304299591 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c0b4070 con 0x563b128a9920 2024-03-26T08:44:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.120+0000 7f1e2b7fe640 1 -- 172.21.15.7:0/1304299591 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0b4220 con 0x563b128a9920 2024-03-26T08:44:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1304299591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 msgr2=0x563b128a9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/1304299591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128a9d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c09b430 tx=0x7f1e3c09ae00 comp rx=0 tx=0).stop 2024-03-26T08:44:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1304299591 shutdown_connections 2024-03-26T08:44:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/1304299591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128af480 0x563b128b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/1304299591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128aa310 0x563b128aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/1304299591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128a9d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1304299591 >> 172.21.15.7:0/1304299591 conn(0x563b128a4e90 msgr2=0x563b128a5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1304299591 shutdown_connections 2024-03-26T08:44:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1304299591 wait complete. 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 -- start start 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.121+0000 7f1e4a1b0740 1 Processor -- start 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 0x563b128bb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b128b23d0 con 0x563b128aa310 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b127cad50 con 0x563b128af480 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e44b68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45369640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b128b2550 con 0x563b128a9920 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45369640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46884/0 (socket says 172.21.15.7:46884) 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45b6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 0x563b128bb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e44b68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33296/0 (socket says 172.21.15.7:33296) 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45369640 1 -- 172.21.15.7:0/778087508 learned_addr learned my addr 172.21.15.7:0/778087508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e4a1b0740 10 client.0 _ll_register_callbacks cb 0x563b127ca720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45369640 1 -- 172.21.15.7:0/778087508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 msgr2=0x563b128bb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45369640 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 0x563b128bb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45369640 1 -- 172.21.15.7:0/778087508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 msgr2=0x563b128b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45369640 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45369640 1 -- 172.21.15.7:0/778087508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b125ef620 con 0x563b128a9920 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.122+0000 7f1e45b6a640 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 0x563b128bb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.123+0000 7f1e45369640 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128b3c80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00c6a0 tx=0x7f1e3c097840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6492755d846674ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.123+0000 7f1e28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.123+0000 7f1e28ff9640 1 -- 172.21.15.7:0/778087508 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c09b4a0 con 0x563b128a9920 2024-03-26T08:44:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.123+0000 7f1e28ff9640 1 -- 172.21.15.7:0/778087508 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1e3c097b10 con 0x563b128a9920 2024-03-26T08:44:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.123+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/778087508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e3c0b4070 con 0x563b128a9920 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e28ff9640 1 -- 172.21.15.7:0/778087508 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e3c0ab050 con 0x563b128a9920 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e4a1b0740 10 client.25894 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/778087508 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563b128b2440 con 0x563b128a9920 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e28ff9640 1 -- 172.21.15.7:0/778087508 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c0b4070 con 0x563b128a9920 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e28ff9640 1 -- 172.21.15.7:0/778087508 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1e3c0af390 con 0x563b128a9920 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e4a1b0740 10 client.25894 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e4a1b0740 10 client.25894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e4a1b0740 20 client.25894 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563b128bf740 0x563b128c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e4a1b0740 4 client.25894 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/778087508 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b1281c050 con 0x563b128bf740 2024-03-26T08:44:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.127+0000 7f1e44b68640 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563b128bf740 0x563b128c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.128+0000 7f1e28ff9640 10 client.25894 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.128+0000 7f1e44b68640 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563b128bf740 0x563b128c1b30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44694a05a751c49 server_cookie=4d14eb85fc92fc66 in_seq=0 out_seq=0 2024-03-26T08:44:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e28ff9640 1 -- 172.21.15.7:0/778087508 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563b1281c050 con 0x563b128bf740 2024-03-26T08:44:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e28ff9640 10 client.25894 handle_command_reply: tid=0 2024-03-26T08:44:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 client.25894 shutdown 2024-03-26T08:44:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/778087508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 msgr2=0x563b128b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128b3c80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00c6a0 tx=0x7f1e3c097840 comp rx=0 tx=0).stop 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/778087508 shutdown_connections 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563b128bf740 0x563b128c1b30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 0x563b128bb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/778087508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128a9920 0x563b128b3c80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/778087508 >> 172.21.15.7:0/778087508 conn(0x563b128a4e90 msgr2=0x563b128a4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e28ff9640 0 client.25894 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/778087508 shutdown_connections 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/778087508 wait complete. 2024-03-26T08:44:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.129+0000 7f1e4a1b0740 20 client.25894 trim_cache size 0 max 16384 2024-03-26T08:44:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e4a1b0740 1 Processor -- start 2024-03-26T08:44:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e4a1b0740 1 -- start start 2024-03-26T08:44:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128a9920 0x563b128a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128af480 0x563b128a8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401b1520 con 0x563b128aa310 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401af020 con 0x563b128a9920 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401ad0a0 con 0x563b128af480 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e44b68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e44b68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128bb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33308/0 (socket says 172.21.15.7:33308) 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e44b68640 1 -- 172.21.15.7:0/1909184347 learned_addr learned my addr 172.21.15.7:0/1909184347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e45369640 1 --2- 172.21.15.7:0/1909184347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128a9920 0x563b128a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e45b6a640 1 --2- 172.21.15.7:0/1909184347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128af480 0x563b128a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e44b68640 1 -- 172.21.15.7:0/1909184347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128af480 msgr2=0x563b128a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e44b68640 1 --2- 172.21.15.7:0/1909184347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128af480 0x563b128a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e44b68640 1 -- 172.21.15.7:0/1909184347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128a9920 msgr2=0x563b128a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:41.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e44b68640 1 --2- 172.21.15.7:0/1909184347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128a9920 0x563b128a80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.150+0000 7f1e44b68640 1 -- 172.21.15.7:0/1909184347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b128a8b70 con 0x563b128aa310 2024-03-26T08:44:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.151+0000 7f1e44b68640 1 --2- 172.21.15.7:0/1909184347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128bb150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e300057c0 tx=0x7f1e30005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabaae2f411c1915 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.151+0000 7f1e2affd640 1 -- 172.21.15.7:0/1909184347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30039040 con 0x563b128aa310 2024-03-26T08:44:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.151+0000 7f1e2affd640 1 -- 172.21.15.7:0/1909184347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e3004f070 con 0x563b128aa310 2024-03-26T08:44:41.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.151+0000 7f1e2affd640 1 -- 172.21.15.7:0/1909184347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30032640 con 0x563b128aa310 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.152+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1909184347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 msgr2=0x563b128bb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.152+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/1909184347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128bb150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e300057c0 tx=0x7f1e30005f90 comp rx=0 tx=0).stop 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.153+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1909184347 shutdown_connections 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.153+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/1909184347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128af480 0x563b128a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.153+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/1909184347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128aa310 0x563b128bb150 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.153+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/1909184347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128a9920 0x563b128a80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.153+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1909184347 >> 172.21.15.7:0/1909184347 conn(0x563b128a4e90 msgr2=0x563b128a5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.153+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1909184347 shutdown_connections 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.153+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/1909184347 wait complete. 2024-03-26T08:44:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.153+0000 7f1e4a1b0740 1 -- start start 2024-03-26T08:44:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e4a1b0740 1 Processor -- start 2024-03-26T08:44:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128a9920 0x563b128b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128aa310 0x563b128b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e4a1b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 0x563b128b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401ad6a0 con 0x563b128a9920 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401ae9c0 con 0x563b128af480 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e4a1b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401add00 con 0x563b128aa310 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e45369640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128a9920 0x563b128b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e45369640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128a9920 0x563b128b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33324/0 (socket says 172.21.15.7:33324) 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e4a1b0740 10 client.0 _ll_register_callbacks cb 0x7f1e3c09b320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.154+0000 7f1e45369640 1 -- 172.21.15.7:0/238407595 learned_addr learned my addr 172.21.15.7:0/238407595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.155+0000 7f1e44b68640 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128aa310 0x563b128b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.155+0000 7f1e45b6a640 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 0x563b128b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.155+0000 7f1e45369640 1 -- 172.21.15.7:0/238407595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128aa310 msgr2=0x563b128b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.155+0000 7f1e45369640 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128aa310 0x563b128b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.155+0000 7f1e45369640 1 -- 172.21.15.7:0/238407595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 msgr2=0x563b128b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.155+0000 7f1e45369640 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 0x563b128b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.155+0000 7f1e45369640 1 -- 172.21.15.7:0/238407595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b128ba3d0 con 0x563b128a9920 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.155+0000 7f1e45369640 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128a9920 0x563b128b8910 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0b41f0 tx=0x7f1e3c0999b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e452d223617c6374 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.155+0000 7f1e28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.156+0000 7f1e28ff9640 1 -- 172.21.15.7:0/238407595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c09f3b0 con 0x563b128a9920 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.156+0000 7f1e28ff9640 1 -- 172.21.15.7:0/238407595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1e3c0bb110 con 0x563b128a9920 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.156+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/238407595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e3004f070 con 0x563b128a9920 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.156+0000 7f1e28ff9640 1 -- 172.21.15.7:0/238407595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e3c09b4a0 con 0x563b128a9920 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.156+0000 7f1e4a1b0740 10 client.16038 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.156+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/238407595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563b1291afb0 con 0x563b128a9920 2024-03-26T08:44:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.157+0000 7f1e28ff9640 1 -- 172.21.15.7:0/238407595 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e3c0b4040 con 0x563b128a9920 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.157+0000 7f1e28ff9640 1 -- 172.21.15.7:0/238407595 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1e3c0ccac0 con 0x563b128a9920 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.157+0000 7f1e4a1b0740 10 client.16038 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.157+0000 7f1e4a1b0740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.157+0000 7f1e4a1b0740 20 client.16038 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.157+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563b1291b500 0x563b1291d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.157+0000 7f1e4a1b0740 4 client.16038 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.157+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/238407595 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f1e3004f070 con 0x563b1291b500 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.157+0000 7f1e44b68640 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563b1291b500 0x563b1291d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.158+0000 7f1e44b68640 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563b1291b500 0x563b1291d9a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21cc00b129536c21 server_cookie=b716e0afa1998a39 in_seq=0 out_seq=0 2024-03-26T08:44:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.158+0000 7f1e28ff9640 10 client.16038 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.175+0000 7f1e28ff9640 1 -- 172.21.15.7:0/238407595 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f1e3004f070 con 0x563b1291b500 2024-03-26T08:44:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.175+0000 7f1e28ff9640 10 client.16038 handle_command_reply: tid=0 2024-03-26T08:44:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.175+0000 7f1e4a1b0740 1 client.16038 shutdown 2024-03-26T08:44:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/238407595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128a9920 msgr2=0x563b128b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128a9920 0x563b128b8910 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0b41f0 tx=0x7f1e3c0999b0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/238407595 shutdown_connections 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563b1291b500 0x563b1291d9a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e28ff9640 0 client.16038 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b128af480 0x563b128b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563b128aa310 0x563b128b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 --2- 172.21.15.7:0/238407595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b128a9920 0x563b128b8910 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/238407595 >> 172.21.15.7:0/238407595 conn(0x563b128a4e90 msgr2=0x563b128ab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/238407595 shutdown_connections 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 1 -- 172.21.15.7:0/238407595 wait complete. 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e4a1b0740 20 client.16038 trim_cache size 0 max 16384 2024-03-26T08:44:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:44:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:44:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:44:41.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e475cd640 1 -- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1e2c0a29c0 msgr2=0x7f1e2c0a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e475cd640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1e2c0a29c0 0x7f1e2c0a4e80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1e380094f0 tx=0x7f1e38009680 comp rx=0 tx=0).stop 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e475cd640 1 -- 172.21.15.7:0/1437783262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e400fff70 msgr2=0x7f1e4019df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.176+0000 7f1e475cd640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e400fff70 0x7f1e4019df70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c005fd0 tx=0x7f1e3c00ee50 comp rx=0 tx=0).stop 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.177+0000 7f1e475cd640 1 -- 172.21.15.7:0/1437783262 shutdown_connections 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.177+0000 7f1e475cd640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1e2c0a29c0 0x7f1e2c0a4e80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.177+0000 7f1e475cd640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e40105290 0x7f1e401a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.177+0000 7f1e475cd640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e401028e0 0x7f1e4019e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.177+0000 7f1e475cd640 1 --2- 172.21.15.7:0/1437783262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e400fff70 0x7f1e4019df70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.177+0000 7f1e475cd640 1 -- 172.21.15.7:0/1437783262 >> 172.21.15.7:0/1437783262 conn(0x7f1e400fbb70 msgr2=0x7f1e40100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.177+0000 7f1e475cd640 1 -- 172.21.15.7:0/1437783262 shutdown_connections 2024-03-26T08:44:41.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:41.177+0000 7f1e475cd640 1 -- 172.21.15.7:0/1437783262 wait complete. 2024-03-26T08:44:42.367 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:42 smithi044 ceph-mon[36461]: pgmap v428: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 11 MiB/s wr, 911 op/s 2024-03-26T08:44:42.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:42 smithi016 ceph-mon[35851]: pgmap v428: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 11 MiB/s wr, 911 op/s 2024-03-26T08:44:42.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:42 smithi007 ceph-mon[32462]: pgmap v428: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 11 MiB/s wr, 911 op/s 2024-03-26T08:44:44.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:44 smithi016 ceph-mon[35851]: pgmap v429: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 11 MiB/s wr, 969 op/s 2024-03-26T08:44:44.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:44 smithi007 ceph-mon[32462]: pgmap v429: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 11 MiB/s wr, 969 op/s 2024-03-26T08:44:44.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:44 smithi044 ceph-mon[36461]: pgmap v429: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 11 MiB/s wr, 969 op/s 2024-03-26T08:44:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 135.0 s, 3691.6 tps, lat 27.133 ms stddev 29.700 2024-03-26T08:44:46.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:46 smithi016 ceph-mon[35851]: pgmap v430: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 872 op/s 2024-03-26T08:44:46.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:46 smithi007 ceph-mon[32462]: pgmap v430: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 872 op/s 2024-03-26T08:44:46.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:46 smithi044 ceph-mon[36461]: pgmap v430: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 872 op/s 2024-03-26T08:44:48.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:48 smithi016 ceph-mon[35851]: pgmap v431: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 848 op/s 2024-03-26T08:44:48.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:48 smithi007 ceph-mon[32462]: pgmap v431: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 848 op/s 2024-03-26T08:44:48.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:48 smithi044 ceph-mon[36461]: pgmap v431: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 848 op/s 2024-03-26T08:44:50.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:50 smithi016 ceph-mon[35851]: pgmap v432: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 819 op/s 2024-03-26T08:44:50.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:50 smithi007 ceph-mon[32462]: pgmap v432: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 819 op/s 2024-03-26T08:44:50.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:50 smithi044 ceph-mon[36461]: pgmap v432: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 819 op/s 2024-03-26T08:44:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 140.0 s, 3906.4 tps, lat 25.591 ms stddev 25.607 2024-03-26T08:44:52.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:52 smithi016 ceph-mon[35851]: pgmap v433: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 787 op/s 2024-03-26T08:44:52.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:52 smithi007 ceph-mon[32462]: pgmap v433: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 787 op/s 2024-03-26T08:44:52.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:52 smithi044 ceph-mon[36461]: pgmap v433: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.2 MiB/s wr, 787 op/s 2024-03-26T08:44:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.317+0000 7f35c5d32640 1 -- 172.21.15.7:0/2396586139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c010b010 msgr2=0x7f35c010d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.317+0000 7f35c5d32640 1 --2- 172.21.15.7:0/2396586139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c010b010 0x7f35c010d480 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009900 tx=0x7f35b002bc60 comp rx=0 tx=0).stop 2024-03-26T08:44:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.317+0000 7f35c5d32640 1 -- 172.21.15.7:0/2396586139 shutdown_connections 2024-03-26T08:44:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.317+0000 7f35c5d32640 1 --2- 172.21.15.7:0/2396586139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c010d9c0 0x7f35c010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.317+0000 7f35c5d32640 1 --2- 172.21.15.7:0/2396586139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c010b010 0x7f35c010d480 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.317+0000 7f35c5d32640 1 --2- 172.21.15.7:0/2396586139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35c01086a0 0x7f35c010aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.317+0000 7f35c5d32640 1 -- 172.21.15.7:0/2396586139 >> 172.21.15.7:0/2396586139 conn(0x7f35c006e6b0 msgr2=0x7f35c0070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.318+0000 7f35c5d32640 1 -- 172.21.15.7:0/2396586139 shutdown_connections 2024-03-26T08:44:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.318+0000 7f35c5d32640 1 -- 172.21.15.7:0/2396586139 wait complete. 2024-03-26T08:44:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.319+0000 7f35c5d32640 1 Processor -- start 2024-03-26T08:44:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.319+0000 7f35c5d32640 1 -- start start 2024-03-26T08:44:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.319+0000 7f35c5d32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c01086a0 0x7f35c0114d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35c5d32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35c010b010 0x7f35c0115270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35c5d32640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c010d9c0 0x7f35c01b6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c01086a0 0x7f35c0114d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c01086a0 0x7f35c0114d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58850/0 (socket says 172.21.15.7:58850) 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35bf7fe640 1 -- 172.21.15.7:0/844317248 learned_addr learned my addr 172.21.15.7:0/844317248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c011c280 con 0x7f35c010d9c0 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c011c100 con 0x7f35c010b010 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35beffd640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35c010b010 0x7f35c0115270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c011c400 con 0x7f35c01086a0 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.320+0000 7f35bffff640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c010d9c0 0x7f35c01b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.321+0000 7f35beffd640 1 -- 172.21.15.7:0/844317248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c01086a0 msgr2=0x7f35c0114d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.321+0000 7f35beffd640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c01086a0 0x7f35c0114d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.321+0000 7f35beffd640 1 -- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c010d9c0 msgr2=0x7f35c01b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.321+0000 7f35beffd640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c010d9c0 0x7f35c01b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.321+0000 7f35beffd640 1 -- 172.21.15.7:0/844317248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c01b6ed0 con 0x7f35c010b010 2024-03-26T08:44:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.321+0000 7f35bffff640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c010d9c0 0x7f35c01b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.321+0000 7f35beffd640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35c010b010 0x7f35c0115270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f35b0036040 tx=0x7f35b002c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.321+0000 7f35bcff9640 1 -- 172.21.15.7:0/844317248 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0041070 con 0x7f35c010b010 2024-03-26T08:44:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.322+0000 7f35bcff9640 1 -- 172.21.15.7:0/844317248 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b00054e0 con 0x7f35c010b010 2024-03-26T08:44:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.322+0000 7f35bcff9640 1 -- 172.21.15.7:0/844317248 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0034440 con 0x7f35c010b010 2024-03-26T08:44:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.324+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c01b70a0 con 0x7f35c010b010 2024-03-26T08:44:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.324+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c01b7450 con 0x7f35c010b010 2024-03-26T08:44:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.328+0000 7f35c89f3740 1 Processor -- start 2024-03-26T08:44:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.328+0000 7f35c89f3740 1 -- start start 2024-03-26T08:44:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.328+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c08d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.328+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 0x561b55c0de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.328+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c10910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.328+0000 7f35c89f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b55b6db10 con 0x561b55c08920 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.328+0000 7f35c89f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b55b16900 con 0x561b55c0e480 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.328+0000 7f35c89f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b55b69bb0 con 0x561b55c09310 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.328+0000 7f35bcff9640 1 -- 172.21.15.7:0/844317248 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f35b0044070 con 0x7f35c010b010 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35bffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c08d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 0x561b55c0de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c10910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 0x561b55c0de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58864/0 (socket says 172.21.15.7:58864) 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35bf7fe640 1 -- 172.21.15.7:0/3146646878 learned_addr learned my addr 172.21.15.7:0/3146646878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35beffd640 1 -- 172.21.15.7:0/3146646878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 msgr2=0x561b55c0de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35beffd640 1 --2- 172.21.15.7:0/3146646878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 0x561b55c0de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35beffd640 1 -- 172.21.15.7:0/3146646878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 msgr2=0x561b55c08d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35beffd640 1 --2- 172.21.15.7:0/3146646878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c08d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35beffd640 1 -- 172.21.15.7:0/3146646878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b55882210 con 0x561b55c0e480 2024-03-26T08:44:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35bcff9640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35ac0a2980 0x7f35ac0a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35bf7fe640 1 --2- 172.21.15.7:0/3146646878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 0x561b55c0de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.329+0000 7f35bcff9640 1 -- 172.21.15.7:0/844317248 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35b00f05e0 con 0x7f35c010b010 2024-03-26T08:44:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.330+0000 7f35beffd640 1 --2- 172.21.15.7:0/3146646878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c10910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35b00ee080 tx=0x7f35b00f6620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786e2c87b0a43b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.330+0000 7f35bffff640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35ac0a2980 0x7f35ac0a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.331+0000 7f35b5ffb640 1 -- 172.21.15.7:0/3146646878 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00bcde0 con 0x561b55c0e480 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.331+0000 7f35b5ffb640 1 -- 172.21.15.7:0/3146646878 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b0004c80 con 0x561b55c0e480 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.331+0000 7f35bffff640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35ac0a2980 0x7f35ac0a4e40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x561b55c09b70 tx=0x7f35a8009060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.331+0000 7f35b5ffb640 1 -- 172.21.15.7:0/3146646878 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00bc970 con 0x561b55c0e480 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 -- 172.21.15.7:0/3146646878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 msgr2=0x561b55c10910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 --2- 172.21.15.7:0/3146646878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c10910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35b00ee080 tx=0x7f35b00f6620 comp rx=0 tx=0).stop 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 -- 172.21.15.7:0/3146646878 shutdown_connections 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 --2- 172.21.15.7:0/3146646878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c10910 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 --2- 172.21.15.7:0/3146646878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 0x561b55c0de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 --2- 172.21.15.7:0/3146646878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c08d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 -- 172.21.15.7:0/3146646878 >> 172.21.15.7:0/3146646878 conn(0x561b55c03e90 msgr2=0x561b55c04290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 -- 172.21.15.7:0/3146646878 shutdown_connections 2024-03-26T08:44:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 -- 172.21.15.7:0/3146646878 wait complete. 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.332+0000 7f35c89f3740 1 -- start start 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35c89f3740 1 Processor -- start 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 0x561b55c12f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c129d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43804/0 (socket says 172.21.15.7:43804) 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35bf7fe640 1 -- 172.21.15.7:0/1184452081 learned_addr learned my addr 172.21.15.7:0/1184452081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c19f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b55c111f0 con 0x561b55c08920 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b55b29d50 con 0x561b55c0e480 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.333+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b55c11370 con 0x561b55c09310 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.334+0000 7f35c89f3740 10 client.0 _ll_register_callbacks cb 0x561b55b29720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.334+0000 7f35bffff640 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c19f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.334+0000 7f35bffff640 1 -- 172.21.15.7:0/1184452081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 msgr2=0x561b55c12f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.334+0000 7f35bffff640 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 0x561b55c12f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.334+0000 7f35bffff640 1 -- 172.21.15.7:0/1184452081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 msgr2=0x561b55c129d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.334+0000 7f35bffff640 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c129d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.334+0000 7f35bffff640 1 -- 172.21.15.7:0/1184452081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b5594e620 con 0x561b55c0e480 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.334+0000 7f35bf7fe640 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.335+0000 7f35bffff640 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c19f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f35a8006fd0 tx=0x7f35a8004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3760ebaa9ad6cdd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.335+0000 7f359effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.335+0000 7f359effd640 1 -- 172.21.15.7:0/1184452081 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8025070 con 0x561b55c0e480 2024-03-26T08:44:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.335+0000 7f359effd640 1 -- 172.21.15.7:0/1184452081 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35a8013920 con 0x561b55c0e480 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.335+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35b0004c80 con 0x561b55c0e480 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.335+0000 7f359effd640 1 -- 172.21.15.7:0/1184452081 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a8031050 con 0x561b55c0e480 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.335+0000 7f35c89f3740 10 client.25886 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.335+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561b55c1d330 con 0x561b55c0e480 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f359effd640 1 -- 172.21.15.7:0/1184452081 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b0004c80 con 0x561b55c0e480 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f359effd640 1 -- 172.21.15.7:0/1184452081 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f35a8020070 con 0x561b55c0e480 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f35c89f3740 10 client.25886 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f35c89f3740 10 client.25886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f35c89f3740 20 client.25886 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b55c1ee30 0x561b55c21270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f35c89f3740 4 client.25886 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b55b7b050 con 0x561b55c1ee30 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f35bf7fe640 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b55c1ee30 0x561b55c21270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f35bf7fe640 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b55c1ee30 0x561b55c21270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8bf8491c836876f server_cookie=cc05749b44426791 in_seq=0 out_seq=0 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.336+0000 7f359effd640 10 client.25886 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.337+0000 7f359effd640 1 -- 172.21.15.7:0/1184452081 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561b55b7b050 con 0x561b55c1ee30 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.337+0000 7f359effd640 10 client.25886 handle_command_reply: tid=0 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.337+0000 7f35c89f3740 1 client.25886 shutdown 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.337+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 msgr2=0x561b55c19f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.337+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c19f50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f35a8006fd0 tx=0x7f35a8004700 comp rx=0 tx=0).stop 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.338+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 shutdown_connections 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.338+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b55c1ee30 0x561b55c21270 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.338+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c19f50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.338+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c09310 0x561b55c12f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.338+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1184452081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c08920 0x561b55c129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.338+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 >> 172.21.15.7:0/1184452081 conn(0x561b55c03e90 msgr2=0x561b55c03020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.338+0000 7f359effd640 0 client.25886 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.339+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 shutdown_connections 2024-03-26T08:44:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.339+0000 7f35c89f3740 1 -- 172.21.15.7:0/1184452081 wait complete. 2024-03-26T08:44:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.339+0000 7f35c89f3740 20 client.25886 trim_cache size 0 max 16384 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.363+0000 7f35c89f3740 1 Processor -- start 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.363+0000 7f35c89f3740 1 -- start start 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.363+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c08920 0x561b55bfeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.363+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c19f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.363+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35c89f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01c2540 con 0x561b55c09310 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35c89f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01c0040 con 0x561b55c0e480 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35c89f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01be0c0 con 0x561b55c08920 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c08920 0x561b55bfeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c19f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c08920 0x561b55bfeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58886/0 (socket says 172.21.15.7:58886) 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52460/0 (socket says 172.21.15.7:52460) 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c19f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43808/0 (socket says 172.21.15.7:43808) 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35bf7fe640 1 -- 172.21.15.7:0/1621244212 learned_addr learned my addr 172.21.15.7:0/1621244212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35beffd640 1 -- 172.21.15.7:0/1621244212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c08920 msgr2=0x561b55bfeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35beffd640 1 --2- 172.21.15.7:0/1621244212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c08920 0x561b55bfeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35beffd640 1 -- 172.21.15.7:0/1621244212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 msgr2=0x561b55c070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35beffd640 1 --2- 172.21.15.7:0/1621244212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c070f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35beffd640 1 -- 172.21.15.7:0/1621244212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b55c218f0 con 0x561b55c09310 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35bf7fe640 1 --2- 172.21.15.7:0/1621244212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c08920 0x561b55bfeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.364+0000 7f35bffff640 1 --2- 172.21.15.7:0/1621244212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.365+0000 7f35beffd640 1 --2- 172.21.15.7:0/1621244212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c19f10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f35b00ee080 tx=0x7f35b002c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d87145688768138 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.366+0000 7f35b57fa640 1 -- 172.21.15.7:0/1621244212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00bcb50 con 0x561b55c09310 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.366+0000 7f35b57fa640 1 -- 172.21.15.7:0/1621244212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b0004c80 con 0x561b55c09310 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.366+0000 7f35b57fa640 1 -- 172.21.15.7:0/1621244212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b00c5720 con 0x561b55c09310 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.367+0000 7f35c89f3740 1 -- 172.21.15.7:0/1621244212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 msgr2=0x561b55c19f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.367+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1621244212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c19f10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f35b00ee080 tx=0x7f35b002c180 comp rx=0 tx=0).stop 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.367+0000 7f35c89f3740 1 -- 172.21.15.7:0/1621244212 shutdown_connections 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.367+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1621244212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c0e480 0x561b55c070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.367+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1621244212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c19f10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.367+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1621244212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c08920 0x561b55bfeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.367+0000 7f35c89f3740 1 -- 172.21.15.7:0/1621244212 >> 172.21.15.7:0/1621244212 conn(0x561b55c03e90 msgr2=0x561b55c04270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.367+0000 7f35c89f3740 1 -- 172.21.15.7:0/1621244212 shutdown_connections 2024-03-26T08:44:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.368+0000 7f35c89f3740 1 -- 172.21.15.7:0/1621244212 wait complete. 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.368+0000 7f35c89f3740 1 -- start start 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.369+0000 7f35c89f3740 1 Processor -- start 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.369+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c08920 0x561b55c16c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.369+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c17180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.369+0000 7f35c89f3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c0e480 0x561b55c78730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.369+0000 7f35c89f3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01be6c0 con 0x561b55c09310 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.369+0000 7f35c89f3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01bf9e0 con 0x561b55c08920 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.369+0000 7f35c89f3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c01bed20 con 0x561b55c0e480 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c08920 0x561b55c16c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35c89f3740 10 client.0 _ll_register_callbacks cb 0x7f35a8014ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35beffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c08920 0x561b55c16c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52472/0 (socket says 172.21.15.7:52472) 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bf7fe640 1 -- 172.21.15.7:0/1259753092 learned_addr learned my addr 172.21.15.7:0/1259753092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bffff640 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c0e480 0x561b55c78730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bf7fe640 1 -- 172.21.15.7:0/1259753092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c0e480 msgr2=0x561b55c78730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bf7fe640 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c0e480 0x561b55c78730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bf7fe640 1 -- 172.21.15.7:0/1259753092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 msgr2=0x561b55c17180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bf7fe640 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c17180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bf7fe640 1 -- 172.21.15.7:0/1259753092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b55c1a200 con 0x561b55c08920 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bffff640 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c0e480 0x561b55c78730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35beffd640 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c17180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f35bf7fe640 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c08920 0x561b55c16c40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x561b55c21af0 tx=0x7f35b8007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e74b1d5a4de34d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f359effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f359effd640 1 -- 172.21.15.7:0/1259753092 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8017aa0 con 0x561b55c08920 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.370+0000 7f359effd640 1 -- 172.21.15.7:0/1259753092 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35b8017c40 con 0x561b55c08920 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.371+0000 7f35c89f3740 1 -- 172.21.15.7:0/1259753092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35b0004c80 con 0x561b55c08920 2024-03-26T08:44:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.371+0000 7f359effd640 1 -- 172.21.15.7:0/1259753092 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35b8017e40 con 0x561b55c08920 2024-03-26T08:44:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.371+0000 7f35c89f3740 10 client.25892 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.371+0000 7f35c89f3740 1 -- 172.21.15.7:0/1259753092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561b55c79ce0 con 0x561b55c08920 2024-03-26T08:44:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.372+0000 7f359effd640 1 -- 172.21.15.7:0/1259753092 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b0004c80 con 0x561b55c08920 2024-03-26T08:44:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.372+0000 7f359effd640 1 -- 172.21.15.7:0/1259753092 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f35b8031660 con 0x561b55c08920 2024-03-26T08:44:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.372+0000 7f35c89f3740 10 client.25892 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.372+0000 7f35c89f3740 10 client.25892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.372+0000 7f35c89f3740 20 client.25892 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.372+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b55c7a670 0x561b55c7ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.372+0000 7f35c89f3740 4 client.25892 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:44:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.372+0000 7f35c89f3740 1 -- 172.21.15.7:0/1259753092 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561b55b753a0 con 0x561b55c7a670 2024-03-26T08:44:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.372+0000 7f35beffd640 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b55c7a670 0x561b55c7ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.373+0000 7f35beffd640 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b55c7a670 0x561b55c7ca90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f5aaff97847839 server_cookie=938436624c0e82d5 in_seq=0 out_seq=0 2024-03-26T08:44:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.373+0000 7f359effd640 10 client.25892 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f359effd640 1 -- 172.21.15.7:0/1259753092 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561b55b753a0 con 0x561b55c7a670 2024-03-26T08:44:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f359effd640 10 client.25892 handle_command_reply: tid=0 2024-03-26T08:44:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 client.25892 shutdown 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 -- 172.21.15.7:0/1259753092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c08920 msgr2=0x561b55c16c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c08920 0x561b55c16c40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x561b55c21af0 tx=0x7f35b8007ec0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 -- 172.21.15.7:0/1259753092 shutdown_connections 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b55c7a670 0x561b55c7ca90 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b55c0e480 0x561b55c78730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b55c09310 0x561b55c17180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 --2- 172.21.15.7:0/1259753092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b55c08920 0x561b55c16c40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 -- 172.21.15.7:0/1259753092 >> 172.21.15.7:0/1259753092 conn(0x561b55c03e90 msgr2=0x561b55bfeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f359effd640 0 client.25892 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 -- 172.21.15.7:0/1259753092 shutdown_connections 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.395+0000 7f35c89f3740 1 -- 172.21.15.7:0/1259753092 wait complete. 2024-03-26T08:44:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.396+0000 7f35c89f3740 20 client.25892 trim_cache size 0 max 16384 2024-03-26T08:44:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:44:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:44:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:44:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:44:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35ac0a2980 msgr2=0x7f35ac0a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35ac0a2980 0x7f35ac0a4e40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x561b55c09b70 tx=0x7f35a8009060 comp rx=0 tx=0).stop 2024-03-26T08:44:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35c010b010 msgr2=0x7f35c0115270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35c010b010 0x7f35c0115270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f35b0036040 tx=0x7f35b002c670 comp rx=0 tx=0).stop 2024-03-26T08:44:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 shutdown_connections 2024-03-26T08:44:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f35ac0a2980 0x7f35ac0a4e40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35c010d9c0 0x7f35c01b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35c010b010 0x7f35c0115270 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 --2- 172.21.15.7:0/844317248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f35c01086a0 0x7f35c0114d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 >> 172.21.15.7:0/844317248 conn(0x7f35c006e6b0 msgr2=0x7f35c010a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 shutdown_connections 2024-03-26T08:44:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:44:54.398+0000 7f35c5d32640 1 -- 172.21.15.7:0/844317248 wait complete. 2024-03-26T08:44:54.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:54 smithi016 ceph-mon[35851]: pgmap v434: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.4 MiB/s wr, 828 op/s 2024-03-26T08:44:54.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:54 smithi007 ceph-mon[32462]: pgmap v434: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.4 MiB/s wr, 828 op/s 2024-03-26T08:44:54.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:54 smithi044 ceph-mon[36461]: pgmap v434: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.4 MiB/s wr, 828 op/s 2024-03-26T08:44:56.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 145.0 s, 3431.8 tps, lat 29.146 ms stddev 34.070 2024-03-26T08:44:56.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:56 smithi016 ceph-mon[35851]: pgmap v435: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 796 op/s 2024-03-26T08:44:56.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:56 smithi007 ceph-mon[32462]: pgmap v435: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 796 op/s 2024-03-26T08:44:56.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:56 smithi044 ceph-mon[36461]: pgmap v435: 97 pgs: 97 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 796 op/s 2024-03-26T08:44:58.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:44:58.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:44:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.263+0000 7f8585fb3640 1 -- 172.21.15.7:0/2755673724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801032a0 msgr2=0x7f8580109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.263+0000 7f8585fb3640 1 --2- 172.21.15.7:0/2755673724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801032a0 0x7f8580109b30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8578009ca0 tx=0x7f857802c100 comp rx=0 tx=0).stop 2024-03-26T08:44:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.263+0000 7f8585fb3640 1 -- 172.21.15.7:0/2755673724 shutdown_connections 2024-03-26T08:44:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.263+0000 7f8585fb3640 1 --2- 172.21.15.7:0/2755673724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801032a0 0x7f8580109b30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.263+0000 7f8585fb3640 1 --2- 172.21.15.7:0/2755673724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85801028e0 0x7f8580102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.263+0000 7f8585fb3640 1 --2- 172.21.15.7:0/2755673724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85801016f0 0x7f8580101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.263+0000 7f8585fb3640 1 -- 172.21.15.7:0/2755673724 >> 172.21.15.7:0/2755673724 conn(0x7f85800fcc60 msgr2=0x7f85800ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.263+0000 7f8585fb3640 1 -- 172.21.15.7:0/2755673724 shutdown_connections 2024-03-26T08:44:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.263+0000 7f8585fb3640 1 -- 172.21.15.7:0/2755673724 wait complete. 2024-03-26T08:44:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.264+0000 7f8585fb3640 1 Processor -- start 2024-03-26T08:44:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.265+0000 7f8585fb3640 1 -- start start 2024-03-26T08:44:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.265+0000 7f8585fb3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85801016f0 0x7f8580199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.265+0000 7f8585fb3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801028e0 0x7f858019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.265+0000 7f8585fb3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85801032a0 0x7f85801a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f8585fb3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858010f580 con 0x7f85801032a0 2024-03-26T08:44:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f8585fb3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858010f400 con 0x7f85801016f0 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801028e0 0x7f858019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801028e0 0x7f858019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33758/0 (socket says 172.21.15.7:33758) 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857effd640 1 -- 172.21.15.7:0/3336706704 learned_addr learned my addr 172.21.15.7:0/3336706704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857ffff640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85801032a0 0x7f85801a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85801016f0 0x7f8580199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f8585fb3640 1 -- 172.21.15.7:0/3336706704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858010f700 con 0x7f85801028e0 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857effd640 1 -- 172.21.15.7:0/3336706704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85801016f0 msgr2=0x7f8580199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857effd640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85801016f0 0x7f8580199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857effd640 1 -- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85801032a0 msgr2=0x7f85801a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857effd640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85801032a0 0x7f85801a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.266+0000 7f857effd640 1 -- 172.21.15.7:0/3336706704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85801a1780 con 0x7f85801028e0 2024-03-26T08:44:58.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.267+0000 7f857effd640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801028e0 0x7f858019a0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f857400d950 tx=0x7f857400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:58.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.267+0000 7f857cff9640 1 -- 172.21.15.7:0/3336706704 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574017070 con 0x7f85801028e0 2024-03-26T08:44:58.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.267+0000 7f857cff9640 1 -- 172.21.15.7:0/3336706704 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8574009c30 con 0x7f85801028e0 2024-03-26T08:44:58.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.267+0000 7f857cff9640 1 -- 172.21.15.7:0/3336706704 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857400f690 con 0x7f85801028e0 2024-03-26T08:44:58.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.267+0000 7f8585fb3640 1 -- 172.21.15.7:0/3336706704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85801a1a70 con 0x7f85801028e0 2024-03-26T08:44:58.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.267+0000 7f8585fb3640 1 -- 172.21.15.7:0/3336706704 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85801a1eb0 con 0x7f85801028e0 2024-03-26T08:44:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.271+0000 7f857cff9640 1 -- 172.21.15.7:0/3336706704 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f85740109d0 con 0x7f85801028e0 2024-03-26T08:44:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f8588b99740 1 Processor -- start 2024-03-26T08:44:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f8588b99740 1 -- start start 2024-03-26T08:44:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f8588b99740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e934500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f8588b99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e934ad0 0x56360e9395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f8588b99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e939c40 0x56360e952240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f8588b99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360e89e3d0 con 0x56360e939c40 2024-03-26T08:44:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f8588b99740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360e846ad0 con 0x56360e9340e0 2024-03-26T08:44:58.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f8588b99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360e89a470 con 0x56360e934ad0 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f857ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e934500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f857ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e934500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34908/0 (socket says 172.21.15.7:34908) 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.272+0000 7f857ffff640 1 -- 172.21.15.7:0/2991637277 learned_addr learned my addr 172.21.15.7:0/2991637277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.273+0000 7f857effd640 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e939c40 0x56360e952240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.273+0000 7f857f7fe640 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e934ad0 0x56360e9395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.273+0000 7f857ffff640 1 -- 172.21.15.7:0/2991637277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e934ad0 msgr2=0x56360e9395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.273+0000 7f857ffff640 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e934ad0 0x56360e9395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.273+0000 7f857ffff640 1 -- 172.21.15.7:0/2991637277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e939c40 msgr2=0x56360e952240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.273+0000 7f857ffff640 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e939c40 0x56360e952240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.273+0000 7f857ffff640 1 -- 172.21.15.7:0/2991637277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56360e67c7d0 con 0x56360e9340e0 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.273+0000 7f857effd640 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e939c40 0x56360e952240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.274+0000 7f857cff9640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85680a2880 0x7f85680a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.274+0000 7f857cff9640 1 -- 172.21.15.7:0/3336706704 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85740ce650 con 0x7f85801028e0 2024-03-26T08:44:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.274+0000 7f857ffff640 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e934500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f85780324c0 tx=0x7f8578032340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47019a63d878dab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.275+0000 7f8565ffb640 1 -- 172.21.15.7:0/2991637277 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578043020 con 0x56360e9340e0 2024-03-26T08:44:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.275+0000 7f8565ffb640 1 -- 172.21.15.7:0/2991637277 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8578003970 con 0x56360e9340e0 2024-03-26T08:44:58.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.275+0000 7f8565ffb640 1 -- 172.21.15.7:0/2991637277 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578039040 con 0x56360e9340e0 2024-03-26T08:44:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.276+0000 7f8588b99740 1 -- 172.21.15.7:0/2991637277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 msgr2=0x56360e934500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.276+0000 7f8588b99740 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e934500 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f85780324c0 tx=0x7f8578032340 comp rx=0 tx=0).stop 2024-03-26T08:44:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.276+0000 7f8588b99740 1 -- 172.21.15.7:0/2991637277 shutdown_connections 2024-03-26T08:44:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.276+0000 7f8588b99740 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e939c40 0x56360e952240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.276+0000 7f8588b99740 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e934ad0 0x56360e9395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.276+0000 7f8588b99740 1 --2- 172.21.15.7:0/2991637277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e934500 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.276+0000 7f8588b99740 1 -- 172.21.15.7:0/2991637277 >> 172.21.15.7:0/2991637277 conn(0x56360e93cb20 msgr2=0x56360e93cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:58.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.279+0000 7f8588b99740 1 -- 172.21.15.7:0/2991637277 shutdown_connections 2024-03-26T08:44:58.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.279+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85680a2880 0x7f85680a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.280+0000 7f8588b99740 1 -- 172.21.15.7:0/2991637277 wait complete. 2024-03-26T08:44:58.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.280+0000 7f8588b99740 1 -- start start 2024-03-26T08:44:58.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.280+0000 7f8588b99740 1 Processor -- start 2024-03-26T08:44:58.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.280+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85680a2880 0x7f85680a4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x56360e93da90 tx=0x7f8570005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:58.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f8588b99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e9340e0 0x56360e954240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f8588b99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 0x56360e954780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f8588b99740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e939c40 0x56360e95b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f8588b99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360e9528d0 con 0x56360e934ad0 2024-03-26T08:44:58.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f8588b99740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360e85a060 con 0x56360e939c40 2024-03-26T08:44:58.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f8588b99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360e952a50 con 0x56360e9340e0 2024-03-26T08:44:58.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f8588b99740 10 client.0 _ll_register_callbacks cb 0x56360e83c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:58.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f857effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 0x56360e954780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f857effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 0x56360e954780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50292/0 (socket says 172.21.15.7:50292) 2024-03-26T08:44:58.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f857effd640 1 -- 172.21.15.7:0/3196683411 learned_addr learned my addr 172.21.15.7:0/3196683411 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:58.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.282+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e9340e0 0x56360e954240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.283+0000 7f857effd640 1 -- 172.21.15.7:0/3196683411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e9340e0 msgr2=0x56360e954240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:58.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.283+0000 7f857effd640 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e9340e0 0x56360e954240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.283+0000 7f857effd640 1 -- 172.21.15.7:0/3196683411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e939c40 msgr2=0x56360e95b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:58.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.283+0000 7f857effd640 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e939c40 0x56360e95b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.283+0000 7f857effd640 1 -- 172.21.15.7:0/3196683411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56360e5b23f0 con 0x56360e934ad0 2024-03-26T08:44:58.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.284+0000 7f857effd640 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 0x56360e954780 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8574000c00 tx=0x7f857400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3839d266d76243bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:58.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.284+0000 7f855effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:44:58.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.284+0000 7f855effd640 1 -- 172.21.15.7:0/3196683411 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574020030 con 0x56360e934ad0 2024-03-26T08:44:58.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.284+0000 7f855effd640 1 -- 172.21.15.7:0/3196683411 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8574095b60 con 0x56360e934ad0 2024-03-26T08:44:58.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.286+0000 7f8588b99740 1 -- 172.21.15.7:0/3196683411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8578003970 con 0x56360e934ad0 2024-03-26T08:44:58.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.286+0000 7f855effd640 1 -- 172.21.15.7:0/3196683411 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8574009de0 con 0x56360e934ad0 2024-03-26T08:44:58.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.287+0000 7f8588b99740 10 client.16068 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:58.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.287+0000 7f8588b99740 1 -- 172.21.15.7:0/3196683411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56360e5720f0 con 0x56360e934ad0 2024-03-26T08:44:58.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.289+0000 7f855effd640 1 -- 172.21.15.7:0/3196683411 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8578003970 con 0x56360e934ad0 2024-03-26T08:44:58.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.289+0000 7f855effd640 1 -- 172.21.15.7:0/3196683411 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f857409a440 con 0x56360e934ad0 2024-03-26T08:44:58.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.289+0000 7f8588b99740 10 client.16068 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:58.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.289+0000 7f8588b99740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:58.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.289+0000 7f8588b99740 20 client.16068 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:58.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.289+0000 7f8588b99740 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56360e960100 0x56360e962580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.289+0000 7f8588b99740 4 client.16068 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:58.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.289+0000 7f8588b99740 1 -- 172.21.15.7:0/3196683411 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56360e8ab910 con 0x56360e960100 2024-03-26T08:44:58.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.294+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56360e960100 0x56360e962580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.296+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56360e960100 0x56360e962580 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b7c902c0fdab34 server_cookie=31ff7b7bb00a9ac8 in_seq=0 out_seq=0 2024-03-26T08:44:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.296+0000 7f855effd640 10 client.16068 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.299+0000 7f855effd640 1 -- 172.21.15.7:0/3196683411 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56360e8ab910 con 0x56360e960100 2024-03-26T08:44:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.299+0000 7f855effd640 10 client.16068 handle_command_reply: tid=0 2024-03-26T08:44:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.299+0000 7f8588b99740 1 client.16068 shutdown 2024-03-26T08:44:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.300+0000 7f8588b99740 1 -- 172.21.15.7:0/3196683411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 msgr2=0x56360e954780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.300+0000 7f8588b99740 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 0x56360e954780 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8574000c00 tx=0x7f857400bf60 comp rx=0 tx=0).stop 2024-03-26T08:44:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.300+0000 7f8588b99740 1 -- 172.21.15.7:0/3196683411 shutdown_connections 2024-03-26T08:44:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.300+0000 7f8588b99740 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56360e960100 0x56360e962580 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.300+0000 7f8588b99740 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e939c40 0x56360e95b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.300+0000 7f8588b99740 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 0x56360e954780 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.300+0000 7f8588b99740 1 --2- 172.21.15.7:0/3196683411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e9340e0 0x56360e954240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.300+0000 7f8588b99740 1 -- 172.21.15.7:0/3196683411 >> 172.21.15.7:0/3196683411 conn(0x56360e93cb20 msgr2=0x56360e933530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.300+0000 7f855effd640 0 client.16068 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.301+0000 7f8588b99740 1 -- 172.21.15.7:0/3196683411 shutdown_connections 2024-03-26T08:44:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.302+0000 7f8588b99740 1 -- 172.21.15.7:0/3196683411 wait complete. 2024-03-26T08:44:58.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.302+0000 7f8588b99740 20 client.16068 trim_cache size 0 max 16384 2024-03-26T08:44:58.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.330+0000 7f8588b99740 1 Processor -- start 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.330+0000 7f8588b99740 1 -- start start 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f8588b99740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e936070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f8588b99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 0x56360e9365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f8588b99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 0x56360e95b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f8588b99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85801a8f70 con 0x56360e934ad0 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f8588b99740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85801a6a70 con 0x56360e9340e0 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f8588b99740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85801a4b10 con 0x56360e939c40 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e936070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e936070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34924/0 (socket says 172.21.15.7:34924) 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857f7fe640 1 -- 172.21.15.7:0/3129650026 learned_addr learned my addr 172.21.15.7:0/3129650026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857ffff640 1 --2- 172.21.15.7:0/3129650026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 0x56360e95b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857f7fe640 1 -- 172.21.15.7:0/3129650026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 msgr2=0x56360e95b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3129650026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 0x56360e95b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857f7fe640 1 -- 172.21.15.7:0/3129650026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 msgr2=0x56360e9365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3129650026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 0x56360e9365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857f7fe640 1 -- 172.21.15.7:0/3129650026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56360e9525b0 con 0x56360e9340e0 2024-03-26T08:44:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.331+0000 7f857ffff640 1 --2- 172.21.15.7:0/3129650026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 0x56360e95b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.332+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3129650026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e936070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8570012f80 tx=0x7f8570017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13098bb7e341e05e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.332+0000 7f85657fa640 1 -- 172.21.15.7:0/3129650026 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570031070 con 0x56360e9340e0 2024-03-26T08:44:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.332+0000 7f85657fa640 1 -- 172.21.15.7:0/3129650026 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f857002b0d0 con 0x56360e9340e0 2024-03-26T08:44:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.332+0000 7f85657fa640 1 -- 172.21.15.7:0/3129650026 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857001f420 con 0x56360e9340e0 2024-03-26T08:44:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.333+0000 7f8588b99740 1 -- 172.21.15.7:0/3129650026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 msgr2=0x56360e936070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.333+0000 7f8588b99740 1 --2- 172.21.15.7:0/3129650026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e936070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8570012f80 tx=0x7f8570017000 comp rx=0 tx=0).stop 2024-03-26T08:44:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.333+0000 7f8588b99740 1 -- 172.21.15.7:0/3129650026 shutdown_connections 2024-03-26T08:44:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.333+0000 7f8588b99740 1 --2- 172.21.15.7:0/3129650026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 0x56360e95b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.333+0000 7f8588b99740 1 --2- 172.21.15.7:0/3129650026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e934ad0 0x56360e9365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.333+0000 7f8588b99740 1 --2- 172.21.15.7:0/3129650026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e9340e0 0x56360e936070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.333+0000 7f8588b99740 1 -- 172.21.15.7:0/3129650026 >> 172.21.15.7:0/3129650026 conn(0x56360e93cb20 msgr2=0x56360e93cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.333+0000 7f8588b99740 1 -- 172.21.15.7:0/3129650026 shutdown_connections 2024-03-26T08:44:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.334+0000 7f8588b99740 1 -- 172.21.15.7:0/3129650026 wait complete. 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.334+0000 7f8588b99740 1 -- start start 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.334+0000 7f8588b99740 1 Processor -- start 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.334+0000 7f8588b99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e9340e0 0x56360e956750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.334+0000 7f8588b99740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e934ad0 0x56360e956c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.334+0000 7f8588b99740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 0x56360e9caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.334+0000 7f8588b99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85801a5110 con 0x56360e9340e0 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e9340e0 0x56360e956750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e9340e0 0x56360e956750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50298/0 (socket says 172.21.15.7:50298) 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857f7fe640 1 -- 172.21.15.7:0/3628330214 learned_addr learned my addr 172.21.15.7:0/3628330214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f8588b99740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85801a6410 con 0x56360e934ad0 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f8588b99740 1 -- 172.21.15.7:0/3628330214 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85801a5750 con 0x56360e939c40 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857effd640 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e934ad0 0x56360e956c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857ffff640 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 0x56360e9caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f8588b99740 10 client.0 _ll_register_callbacks cb 0x7f857400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857effd640 1 -- 172.21.15.7:0/3628330214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 msgr2=0x56360e9caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857effd640 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 0x56360e9caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857effd640 1 -- 172.21.15.7:0/3628330214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e9340e0 msgr2=0x56360e956750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857effd640 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e9340e0 0x56360e956750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857effd640 1 -- 172.21.15.7:0/3628330214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56360e9caf90 con 0x56360e934ad0 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f857effd640 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e934ad0 0x56360e956c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85740058c0 tx=0x7f857400b530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5189a108ff654ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f855effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f855effd640 1 -- 172.21.15.7:0/3628330214 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8574020070 con 0x56360e934ad0 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.335+0000 7f855effd640 1 -- 172.21.15.7:0/3628330214 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85740b9af0 con 0x56360e934ad0 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.336+0000 7f8588b99740 1 -- 172.21.15.7:0/3628330214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f857002b0d0 con 0x56360e934ad0 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.336+0000 7f855effd640 1 -- 172.21.15.7:0/3628330214 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8574097290 con 0x56360e934ad0 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.336+0000 7f8588b99740 10 client.25907 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.336+0000 7f8588b99740 1 -- 172.21.15.7:0/3628330214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56360e9cc1f0 con 0x56360e934ad0 2024-03-26T08:44:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.337+0000 7f855effd640 1 -- 172.21.15.7:0/3628330214 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f857002b0d0 con 0x56360e934ad0 2024-03-26T08:44:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.337+0000 7f855effd640 1 -- 172.21.15.7:0/3628330214 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8574010460 con 0x56360e934ad0 2024-03-26T08:44:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.337+0000 7f8588b99740 10 client.25907 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.337+0000 7f8588b99740 10 client.25907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.337+0000 7f8588b99740 20 client.25907 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.337+0000 7f8588b99740 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56360e9ccb10 0x56360e9cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.337+0000 7f8588b99740 4 client.25907 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:44:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.338+0000 7f8588b99740 1 -- 172.21.15.7:0/3628330214 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56360e8a5c60 con 0x56360e9ccb10 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.338+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56360e9ccb10 0x56360e9cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.338+0000 7f857f7fe640 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56360e9ccb10 0x56360e9cef30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ee60af70627eed server_cookie=147a59ce0752b634 in_seq=0 out_seq=0 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f855effd640 10 client.25907 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f855effd640 1 -- 172.21.15.7:0/3628330214 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56360e8a5c60 con 0x56360e9ccb10 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f855effd640 10 client.25907 handle_command_reply: tid=0 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f8588b99740 1 client.25907 shutdown 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f8588b99740 1 -- 172.21.15.7:0/3628330214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e934ad0 msgr2=0x56360e956c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f8588b99740 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e934ad0 0x56360e956c90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85740058c0 tx=0x7f857400b530 comp rx=0 tx=0).stop 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f8588b99740 1 -- 172.21.15.7:0/3628330214 shutdown_connections 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f8588b99740 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56360e9ccb10 0x56360e9cef30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f855effd640 0 client.25907 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f8588b99740 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56360e939c40 0x56360e9caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f8588b99740 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56360e934ad0 0x56360e956c90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.339+0000 7f8588b99740 1 --2- 172.21.15.7:0/3628330214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56360e9340e0 0x56360e956750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.340+0000 7f8588b99740 1 -- 172.21.15.7:0/3628330214 >> 172.21.15.7:0/3628330214 conn(0x56360e93cb20 msgr2=0x56360e936070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.340+0000 7f8588b99740 1 -- 172.21.15.7:0/3628330214 shutdown_connections 2024-03-26T08:44:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.340+0000 7f8588b99740 1 -- 172.21.15.7:0/3628330214 wait complete. 2024-03-26T08:44:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.340+0000 7f8588b99740 20 client.25907 trim_cache size 0 max 16384 2024-03-26T08:44:58.334 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:44:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:44:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:44:58.334 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:44:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 -- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85680a2880 msgr2=0x7f85680a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85680a2880 0x7f85680a4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x56360e93da90 tx=0x7f8570005f90 comp rx=0 tx=0).stop 2024-03-26T08:44:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 -- 172.21.15.7:0/3336706704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801028e0 msgr2=0x7f858019a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801028e0 0x7f858019a0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f857400d950 tx=0x7f857400de10 comp rx=0 tx=0).stop 2024-03-26T08:44:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 -- 172.21.15.7:0/3336706704 shutdown_connections 2024-03-26T08:44:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85680a2880 0x7f85680a4d40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85801032a0 0x7f85801a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85801028e0 0x7f858019a0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 --2- 172.21.15.7:0/3336706704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85801016f0 0x7f8580199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 -- 172.21.15.7:0/3336706704 >> 172.21.15.7:0/3336706704 conn(0x7f85800fcc60 msgr2=0x7f8580105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 -- 172.21.15.7:0/3336706704 shutdown_connections 2024-03-26T08:44:58.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.342+0000 7f8585fb3640 1 -- 172.21.15.7:0/3336706704 wait complete. 2024-03-26T08:44:58.351 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:44:58.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:44:58.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:58 smithi007 ceph-mon[32462]: pgmap v436: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 20 MiB/s wr, 795 op/s 2024-03-26T08:44:58.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:58 smithi016 ceph-mon[35851]: pgmap v436: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 20 MiB/s wr, 795 op/s 2024-03-26T08:44:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.523+0000 7f36b8a2a640 1 -- 172.21.15.7:0/1253866512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4109770 msgr2=0x7f36b4110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.523+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/1253866512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4109770 0x7f36b4110000 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009900 tx=0x7f36a402bc40 comp rx=0 tx=0).stop 2024-03-26T08:44:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.523+0000 7f36b8a2a640 1 -- 172.21.15.7:0/1253866512 shutdown_connections 2024-03-26T08:44:58.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.523+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/1253866512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4109770 0x7f36b4110000 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.523+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/1253866512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b4108db0 0x7f36b4109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.523+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/1253866512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b4107bc0 0x7f36b4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.523+0000 7f36b8a2a640 1 -- 172.21.15.7:0/1253866512 >> 172.21.15.7:0/1253866512 conn(0x7f36b4076b60 msgr2=0x7f36b4078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:58.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.525+0000 7f36b8a2a640 1 -- 172.21.15.7:0/1253866512 shutdown_connections 2024-03-26T08:44:58.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.525+0000 7f36b8a2a640 1 -- 172.21.15.7:0/1253866512 wait complete. 2024-03-26T08:44:58.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.526+0000 7f36b8a2a640 1 Processor -- start 2024-03-26T08:44:58.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.526+0000 7f36b8a2a640 1 -- start start 2024-03-26T08:44:58.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b8a2a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b4107bc0 0x7f36b41a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b8a2a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4108db0 0x7f36b41ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b8a2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b4109770 0x7f36b41add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b8a2a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b407c140 con 0x7f36b4108db0 2024-03-26T08:44:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b8a2a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b407bfc0 con 0x7f36b4109770 2024-03-26T08:44:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b8a2a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b407c2c0 con 0x7f36b4107bc0 2024-03-26T08:44:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b4109770 0x7f36b41add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b4109770 0x7f36b41add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34952/0 (socket says 172.21.15.7:34952) 2024-03-26T08:44:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b3fff640 1 -- 172.21.15.7:0/2600276250 learned_addr learned my addr 172.21.15.7:0/2600276250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:58.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.527+0000 7f36b37fe640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b4107bc0 0x7f36b41a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.528+0000 7f36b2ffd640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4108db0 0x7f36b41ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.528+0000 7f36b37fe640 1 -- 172.21.15.7:0/2600276250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b4109770 msgr2=0x7f36b41add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.528+0000 7f36b37fe640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b4109770 0x7f36b41add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.528+0000 7f36b37fe640 1 -- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4108db0 msgr2=0x7f36b41ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.528+0000 7f36b37fe640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4108db0 0x7f36b41ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.528+0000 7f36b37fe640 1 -- 172.21.15.7:0/2600276250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b41ae2d0 con 0x7f36b4107bc0 2024-03-26T08:44:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.529+0000 7f36b37fe640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b4107bc0 0x7f36b41a92f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36a800d6a0 tx=0x7f36a800db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.530+0000 7f36b0ff9640 1 -- 172.21.15.7:0/2600276250 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8017070 con 0x7f36b4107bc0 2024-03-26T08:44:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.530+0000 7f36b0ff9640 1 -- 172.21.15.7:0/2600276250 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36a8009d00 con 0x7f36b4107bc0 2024-03-26T08:44:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.530+0000 7f36b0ff9640 1 -- 172.21.15.7:0/2600276250 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8003c00 con 0x7f36b4107bc0 2024-03-26T08:44:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.531+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b41ae5c0 con 0x7f36b4107bc0 2024-03-26T08:44:58.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.531+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b41ae9d0 con 0x7f36b4107bc0 2024-03-26T08:44:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.534+0000 7f36b0ff9640 1 -- 172.21.15.7:0/2600276250 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f36a8009820 con 0x7f36b4107bc0 2024-03-26T08:44:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.535+0000 7f36b0ff9640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f369c0a2a00 0x7f369c0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.535+0000 7f36b0ff9640 1 -- 172.21.15.7:0/2600276250 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f36a80ce320 con 0x7f36b4107bc0 2024-03-26T08:44:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.536+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b404f470 con 0x7f36b4107bc0 2024-03-26T08:44:58.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.536+0000 7f36b2ffd640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f369c0a2a00 0x7f369c0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:58.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.537+0000 7f36b2ffd640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f369c0a2a00 0x7f369c0a4ec0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f36b410fdd0 tx=0x7f36ac009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:58.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.542+0000 7f36b0ff9640 1 -- 172.21.15.7:0/2600276250 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f36a809b380 con 0x7f36b4107bc0 2024-03-26T08:44:58.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:58 smithi044 ceph-mon[36461]: pgmap v436: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 20 MiB/s wr, 795 op/s 2024-03-26T08:44:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.919+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36b40819c0 con 0x7f36b4107bc0 2024-03-26T08:44:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.922+0000 7f36b0ff9640 1 -- 172.21.15.7:0/2600276250 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f36a809ac80 con 0x7f36b4107bc0 2024-03-26T08:44:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:44:58.913 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:44:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:44:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.925+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f369c0a2a00 msgr2=0x7f369c0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.925+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f369c0a2a00 0x7f369c0a4ec0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f36b410fdd0 tx=0x7f36ac009870 comp rx=0 tx=0).stop 2024-03-26T08:44:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.925+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b4107bc0 msgr2=0x7f36b41a92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.926+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b4107bc0 0x7f36b41a92f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36a800d6a0 tx=0x7f36a800db60 comp rx=0 tx=0).stop 2024-03-26T08:44:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.926+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 shutdown_connections 2024-03-26T08:44:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.926+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f369c0a2a00 0x7f369c0a4ec0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.926+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36b4109770 0x7f36b41add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.926+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36b4108db0 0x7f36b41ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.926+0000 7f36b8a2a640 1 --2- 172.21.15.7:0/2600276250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f36b4107bc0 0x7f36b41a92f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.926+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 >> 172.21.15.7:0/2600276250 conn(0x7f36b4076b60 msgr2=0x7f36b407eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.926+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 shutdown_connections 2024-03-26T08:44:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:58.927+0000 7f36b8a2a640 1 -- 172.21.15.7:0/2600276250 wait complete. 2024-03-26T08:44:58.939 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:44:58.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:44:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.151+0000 7f13d0b09640 1 -- 172.21.15.7:0/1387314558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 msgr2=0x7f13cc101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.151+0000 7f13d0b09640 1 --2- 172.21.15.7:0/1387314558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 0x7f13cc101c50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009900 tx=0x7f13c002bc30 comp rx=0 tx=0).stop 2024-03-26T08:44:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.151+0000 7f13d0b09640 1 -- 172.21.15.7:0/1387314558 shutdown_connections 2024-03-26T08:44:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.151+0000 7f13d0b09640 1 --2- 172.21.15.7:0/1387314558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13cc102190 0x7f13cc108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.152+0000 7f13d0b09640 1 --2- 172.21.15.7:0/1387314558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 0x7f13cc101c50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.152+0000 7f13d0b09640 1 --2- 172.21.15.7:0/1387314558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc1005e0 0x7f13cc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.152+0000 7f13d0b09640 1 -- 172.21.15.7:0/1387314558 >> 172.21.15.7:0/1387314558 conn(0x7f13cc0fbb70 msgr2=0x7f13cc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.152+0000 7f13d0b09640 1 -- 172.21.15.7:0/1387314558 shutdown_connections 2024-03-26T08:44:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.152+0000 7f13d0b09640 1 -- 172.21.15.7:0/1387314558 wait complete. 2024-03-26T08:44:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.153+0000 7f13d0b09640 1 Processor -- start 2024-03-26T08:44:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.153+0000 7f13d0b09640 1 -- start start 2024-03-26T08:44:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13d0b09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc1005e0 0x7f13cc199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13d0b09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 0x7f13cc19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc1005e0 0x7f13cc199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc1005e0 0x7f13cc199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34966/0 (socket says 172.21.15.7:34966) 2024-03-26T08:44:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13c9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 0x7f13cc19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13d0b09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13cc102190 0x7f13cc1a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13c9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 0x7f13cc19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50332/0 (socket says 172.21.15.7:50332) 2024-03-26T08:44:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13d0b09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc10b160 con 0x7f13cc1017d0 2024-03-26T08:44:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13d0b09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc10afe0 con 0x7f13cc1005e0 2024-03-26T08:44:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13d0b09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc10b2e0 con 0x7f13cc102190 2024-03-26T08:44:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13ca59c640 1 -- 172.21.15.7:0/2158168682 learned_addr learned my addr 172.21.15.7:0/2158168682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.154+0000 7f13cad9d640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13cc102190 0x7f13cc1a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.155+0000 7f13c9d9b640 1 -- 172.21.15.7:0/2158168682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13cc102190 msgr2=0x7f13cc1a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.155+0000 7f13c9d9b640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13cc102190 0x7f13cc1a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.155+0000 7f13c9d9b640 1 -- 172.21.15.7:0/2158168682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc1005e0 msgr2=0x7f13cc199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.155+0000 7f13c9d9b640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc1005e0 0x7f13cc199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.155+0000 7f13c9d9b640 1 -- 172.21.15.7:0/2158168682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc1a17a0 con 0x7f13cc1017d0 2024-03-26T08:44:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.155+0000 7f13ca59c640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc1005e0 0x7f13cc199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.155+0000 7f13cad9d640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13cc102190 0x7f13cc1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.155+0000 7f13c9d9b640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 0x7f13cc19a0c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009230 tx=0x7f13c002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.158+0000 7f13bb7fe640 1 -- 172.21.15.7:0/2158168682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0041070 con 0x7f13cc1017d0 2024-03-26T08:44:59.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.158+0000 7f13bb7fe640 1 -- 172.21.15.7:0/2158168682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c0004450 con 0x7f13cc1017d0 2024-03-26T08:44:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.158+0000 7f13bb7fe640 1 -- 172.21.15.7:0/2158168682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c0003df0 con 0x7f13cc1017d0 2024-03-26T08:44:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.158+0000 7f13d0b09640 1 -- 172.21.15.7:0/2158168682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13cc1a1a30 con 0x7f13cc1017d0 2024-03-26T08:44:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.159+0000 7f13d0b09640 1 -- 172.21.15.7:0/2158168682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13cc1a1ed0 con 0x7f13cc1017d0 2024-03-26T08:44:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.162+0000 7f13bb7fe640 1 -- 172.21.15.7:0/2158168682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f13c0044070 con 0x7f13cc1017d0 2024-03-26T08:44:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.162+0000 7f13d37d3740 1 Processor -- start 2024-03-26T08:44:59.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.163+0000 7f13d37d3740 1 -- start start 2024-03-26T08:44:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.164+0000 7f13bb7fe640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13b40a2950 0x7f13b40a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.164+0000 7f13bb7fe640 1 -- 172.21.15.7:0/2158168682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f13c00f03d0 con 0x7f13cc1017d0 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.164+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405feea0 0x557f405ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.164+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 0x557f40604390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.164+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f40606e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.164+0000 7f13d37d3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f405b9610 con 0x557f405ff890 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.164+0000 7f13d37d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4055c3c0 con 0x557f405feea0 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.164+0000 7f13d37d3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f40504d60 con 0x557f40604a00 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405feea0 0x557f405ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405feea0 0x557f405ff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34978/0 (socket says 172.21.15.7:34978) 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13ca59c640 1 -- 172.21.15.7:0/823291885 learned_addr learned my addr 172.21.15.7:0/823291885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13c9d9b640 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 0x557f40604390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13cad9d640 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f40606e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13cad9d640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13b40a2950 0x7f13b40a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13ca59c640 1 -- 172.21.15.7:0/823291885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 msgr2=0x557f40606e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13ca59c640 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f40606e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13ca59c640 1 -- 172.21.15.7:0/823291885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 msgr2=0x557f40604390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13ca59c640 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 0x557f40604390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13ca59c640 1 -- 172.21.15.7:0/823291885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f402703f0 con 0x557f405feea0 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13cad9d640 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f40606e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.165+0000 7f13ca59c640 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405feea0 0x557f405ff2c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0115c0 tx=0x7f13bc011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=902d06fdb79a1d35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.166+0000 7f13cad9d640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13b40a2950 0x7f13b40a4e10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f13c400b480 tx=0x7f13c400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.166+0000 7f13b8ff9640 1 -- 172.21.15.7:0/823291885 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc005020 con 0x557f405feea0 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.166+0000 7f13b8ff9640 1 -- 172.21.15.7:0/823291885 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13bc0047a0 con 0x557f405feea0 2024-03-26T08:44:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.166+0000 7f13b8ff9640 1 -- 172.21.15.7:0/823291885 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc00fa00 con 0x557f405feea0 2024-03-26T08:44:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.167+0000 7f13d37d3740 1 -- 172.21.15.7:0/823291885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405feea0 msgr2=0x557f405ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.167+0000 7f13d37d3740 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405feea0 0x557f405ff2c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0115c0 tx=0x7f13bc011a80 comp rx=0 tx=0).stop 2024-03-26T08:44:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.167+0000 7f13d37d3740 1 -- 172.21.15.7:0/823291885 shutdown_connections 2024-03-26T08:44:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.167+0000 7f13d37d3740 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f40606e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.167+0000 7f13d37d3740 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 0x557f40604390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.167+0000 7f13d37d3740 1 --2- 172.21.15.7:0/823291885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405feea0 0x557f405ff2c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.167+0000 7f13d37d3740 1 -- 172.21.15.7:0/823291885 >> 172.21.15.7:0/823291885 conn(0x557f405f1c00 msgr2=0x557f405f2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.168+0000 7f13d37d3740 1 -- 172.21.15.7:0/823291885 shutdown_connections 2024-03-26T08:44:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.168+0000 7f13d37d3740 1 -- 172.21.15.7:0/823291885 wait complete. 2024-03-26T08:44:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.168+0000 7f13d37d3740 1 -- start start 2024-03-26T08:44:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.168+0000 7f13d37d3740 1 Processor -- start 2024-03-26T08:44:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 0x557f40608fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f40609510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f40610550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13d37d3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f40518240 con 0x557f405feea0 2024-03-26T08:44:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13d37d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f40558460 con 0x557f405ff890 2024-03-26T08:44:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13d37d3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f406079b0 con 0x557f40604a00 2024-03-26T08:44:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13d37d3740 10 client.0 _ll_register_callbacks cb 0x557f40517c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f40609510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13cad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f40610550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 0x557f40608fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f40609510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34986/0 (socket says 172.21.15.7:34986) 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13c9d9b640 1 -- 172.21.15.7:0/1009162755 learned_addr learned my addr 172.21.15.7:0/1009162755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13c9d9b640 1 -- 172.21.15.7:0/1009162755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 msgr2=0x557f40610550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13c9d9b640 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f40610550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13c9d9b640 1 -- 172.21.15.7:0/1009162755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 msgr2=0x557f40608fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13c9d9b640 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 0x557f40608fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.169+0000 7f13c9d9b640 1 -- 172.21.15.7:0/1009162755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f4033a7d0 con 0x557f405ff890 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.170+0000 7f13c9d9b640 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f40609510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f13c0006010 tx=0x7f13c00bc740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0985a84e27a5bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.170+0000 7f13b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.170+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1009162755 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00da060 con 0x557f405ff890 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.170+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1009162755 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f13c00c5040 con 0x557f405ff890 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.170+0000 7f13d37d3740 1 -- 172.21.15.7:0/1009162755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13bc0047a0 con 0x557f405ff890 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.170+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1009162755 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c004b040 con 0x557f405ff890 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.170+0000 7f13d37d3740 10 client.25925 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.170+0000 7f13d37d3740 1 -- 172.21.15.7:0/1009162755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557f406138c0 con 0x557f405ff890 2024-03-26T08:44:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.170+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1009162755 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c00c0e80 con 0x557f405ff890 2024-03-26T08:44:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.171+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1009162755 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f13c00bbca0 con 0x557f405ff890 2024-03-26T08:44:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.171+0000 7f13d37d3740 10 client.25925 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.171+0000 7f13d37d3740 10 client.25925 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:44:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.171+0000 7f13d37d3740 20 client.25925 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.171+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557f40614ea0 0x557f40617320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.171+0000 7f13d37d3740 4 client.25925 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.171+0000 7f13d37d3740 1 -- 172.21.15.7:0/1009162755 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13bc0047a0 con 0x557f40614ea0 2024-03-26T08:44:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.171+0000 7f13ca59c640 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557f40614ea0 0x557f40617320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.172+0000 7f13b1ffb640 10 client.25925 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:44:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.172+0000 7f13ca59c640 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557f40614ea0 0x557f40617320 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60bbe4d79c6c1843 server_cookie=87a852a2d8f3612a in_seq=0 out_seq=0 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.172+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1009162755 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f13bc0047a0 con 0x557f40614ea0 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.172+0000 7f13b1ffb640 10 client.25925 handle_command_reply: tid=0 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 client.25925 shutdown 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 -- 172.21.15.7:0/1009162755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 msgr2=0x557f40609510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f40609510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f13c0006010 tx=0x7f13c00bc740 comp rx=0 tx=0).stop 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 -- 172.21.15.7:0/1009162755 shutdown_connections 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557f40614ea0 0x557f40617320 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f40610550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f40609510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1009162755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 0x557f40608fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 -- 172.21.15.7:0/1009162755 >> 172.21.15.7:0/1009162755 conn(0x557f405f1c00 msgr2=0x557f405f0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13b1ffb640 0 client.25925 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 -- 172.21.15.7:0/1009162755 shutdown_connections 2024-03-26T08:44:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 1 -- 172.21.15.7:0/1009162755 wait complete. 2024-03-26T08:44:59.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.173+0000 7f13d37d3740 20 client.25925 trim_cache size 0 max 16384 2024-03-26T08:44:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.196+0000 7f13d37d3740 1 Processor -- start 2024-03-26T08:44:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.196+0000 7f13d37d3740 1 -- start start 2024-03-26T08:44:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.196+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f405feea0 0x557f40600b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.196+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 0x557f40610510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.196+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f40604a00 0x557f40601090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.196+0000 7f13d37d3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc071de0 con 0x557f405ff890 2024-03-26T08:44:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.196+0000 7f13d37d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc06f8e0 con 0x557f40604a00 2024-03-26T08:44:59.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.196+0000 7f13d37d3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc06d980 con 0x557f405feea0 2024-03-26T08:44:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f40604a00 0x557f40601090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f405feea0 0x557f40600b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f40604a00 0x557f40601090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34992/0 (socket says 172.21.15.7:34992) 2024-03-26T08:44:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f405feea0 0x557f40600b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33866/0 (socket says 172.21.15.7:33866) 2024-03-26T08:44:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13cad9d640 1 -- 172.21.15.7:0/2277466266 learned_addr learned my addr 172.21.15.7:0/2277466266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13cad9d640 1 -- 172.21.15.7:0/2277466266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f405feea0 msgr2=0x557f40600b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13cad9d640 1 --2- 172.21.15.7:0/2277466266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f405feea0 0x557f40600b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13c9d9b640 1 --2- 172.21.15.7:0/2277466266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 0x557f40610510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13cad9d640 1 -- 172.21.15.7:0/2277466266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 msgr2=0x557f40610510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13cad9d640 1 --2- 172.21.15.7:0/2277466266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 0x557f40610510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13cad9d640 1 -- 172.21.15.7:0/2277466266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f405ffc70 con 0x557f40604a00 2024-03-26T08:44:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.197+0000 7f13c9d9b640 1 --2- 172.21.15.7:0/2277466266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 0x557f40610510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:44:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.198+0000 7f13cad9d640 1 --2- 172.21.15.7:0/2277466266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f40604a00 0x557f40601090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13c4004280 tx=0x7f13c400b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362299035a4f8bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.198+0000 7f13b3fff640 1 -- 172.21.15.7:0/2277466266 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4004800 con 0x557f40604a00 2024-03-26T08:44:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.198+0000 7f13b3fff640 1 -- 172.21.15.7:0/2277466266 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c402b070 con 0x557f40604a00 2024-03-26T08:44:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.198+0000 7f13b3fff640 1 -- 172.21.15.7:0/2277466266 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4026bb0 con 0x557f40604a00 2024-03-26T08:44:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.199+0000 7f13d37d3740 1 -- 172.21.15.7:0/2277466266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f40604a00 msgr2=0x557f40601090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.199+0000 7f13d37d3740 1 --2- 172.21.15.7:0/2277466266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f40604a00 0x557f40601090 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13c4004280 tx=0x7f13c400b710 comp rx=0 tx=0).stop 2024-03-26T08:44:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.199+0000 7f13d37d3740 1 -- 172.21.15.7:0/2277466266 shutdown_connections 2024-03-26T08:44:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.199+0000 7f13d37d3740 1 --2- 172.21.15.7:0/2277466266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f40604a00 0x557f40601090 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.199+0000 7f13d37d3740 1 --2- 172.21.15.7:0/2277466266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405ff890 0x557f40610510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.199+0000 7f13d37d3740 1 --2- 172.21.15.7:0/2277466266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f405feea0 0x557f40600b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.199+0000 7f13d37d3740 1 -- 172.21.15.7:0/2277466266 >> 172.21.15.7:0/2277466266 conn(0x557f405f1c00 msgr2=0x557f405f1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.199+0000 7f13d37d3740 1 -- 172.21.15.7:0/2277466266 shutdown_connections 2024-03-26T08:44:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.200+0000 7f13d37d3740 1 -- 172.21.15.7:0/2277466266 wait complete. 2024-03-26T08:44:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.200+0000 7f13d37d3740 1 -- start start 2024-03-26T08:44:59.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.200+0000 7f13d37d3740 1 Processor -- start 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 0x557f406102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f4060e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13d37d3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f4060edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13d37d3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc06df80 con 0x557f405feea0 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13d37d3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc06f280 con 0x557f405ff890 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13d37d3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc06e5c0 con 0x557f40604a00 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13d37d3740 10 client.0 _ll_register_callbacks cb 0x7f13c00bcbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 0x557f406102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f4060e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 0x557f406102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50356/0 (socket says 172.21.15.7:50356) 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f4060e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35002/0 (socket says 172.21.15.7:35002) 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13ca59c640 1 -- 172.21.15.7:0/1378653118 learned_addr learned my addr 172.21.15.7:0/1378653118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13cad9d640 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f4060edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13cad9d640 1 -- 172.21.15.7:0/1378653118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 msgr2=0x557f4060e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13cad9d640 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f4060e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13cad9d640 1 -- 172.21.15.7:0/1378653118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 msgr2=0x557f406102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13cad9d640 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 0x557f406102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.201+0000 7f13cad9d640 1 -- 172.21.15.7:0/1378653118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f4060f460 con 0x557f40604a00 2024-03-26T08:44:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.203+0000 7f13cad9d640 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f4060edf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13c401a7f0 tx=0x7f13c4026d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a69d3ada9e944fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.203+0000 7f13b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.203+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1378653118 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4035020 con 0x557f40604a00 2024-03-26T08:44:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.203+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1378653118 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f13c4014040 con 0x557f40604a00 2024-03-26T08:44:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.203+0000 7f13d37d3740 1 -- 172.21.15.7:0/1378653118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13c402b070 con 0x557f40604a00 2024-03-26T08:44:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1378653118 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13c4004800 con 0x557f40604a00 2024-03-26T08:44:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13d37d3740 10 client.25954 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13d37d3740 1 -- 172.21.15.7:0/1378653118 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557f40671610 con 0x557f40604a00 2024-03-26T08:44:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1378653118 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c402b070 con 0x557f40604a00 2024-03-26T08:44:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1378653118 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f13c4031940 con 0x557f40604a00 2024-03-26T08:44:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13d37d3740 10 client.25954 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13d37d3740 10 client.25954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:44:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13d37d3740 20 client.25954 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557f40671f40 0x557f40674360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13d37d3740 4 client.25954 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:44:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.204+0000 7f13d37d3740 1 -- 172.21.15.7:0/1378653118 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f40569900 con 0x557f40671f40 2024-03-26T08:44:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.205+0000 7f13ca59c640 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557f40671f40 0x557f40674360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.206+0000 7f13b1ffb640 10 client.25954 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.206+0000 7f13ca59c640 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557f40671f40 0x557f40674360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d4f13c84fffbb3 server_cookie=90607e851ca50115 in_seq=0 out_seq=0 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.206+0000 7f13b1ffb640 1 -- 172.21.15.7:0/1378653118 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f40569900 con 0x557f40671f40 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.206+0000 7f13b1ffb640 10 client.25954 handle_command_reply: tid=0 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.206+0000 7f13d37d3740 1 client.25954 shutdown 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 -- 172.21.15.7:0/1378653118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 msgr2=0x557f4060edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f4060edf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13c401a7f0 tx=0x7f13c4026d80 comp rx=0 tx=0).stop 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 -- 172.21.15.7:0/1378653118 shutdown_connections 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557f40671f40 0x557f40674360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557f40604a00 0x557f4060edf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f405ff890 0x557f4060e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 --2- 172.21.15.7:0/1378653118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f405feea0 0x557f406102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 -- 172.21.15.7:0/1378653118 >> 172.21.15.7:0/1378653118 conn(0x557f405f1c00 msgr2=0x557f40608ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13b1ffb640 0 client.25954 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:44:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 -- 172.21.15.7:0/1378653118 shutdown_connections 2024-03-26T08:44:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 1 -- 172.21.15.7:0/1378653118 wait complete. 2024-03-26T08:44:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d37d3740 20 client.25954 trim_cache size 0 max 16384 2024-03-26T08:44:59.199 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d0b09640 1 -- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13b40a2950 msgr2=0x7f13b40a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d0b09640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13b40a2950 0x7f13b40a4e10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f13c400b480 tx=0x7f13c400d040 comp rx=0 tx=0).stop 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d0b09640 1 -- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 msgr2=0x7f13cc19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.207+0000 7f13d0b09640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 0x7f13cc19a0c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009230 tx=0x7f13c002c650 comp rx=0 tx=0).stop 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.208+0000 7f13d0b09640 1 -- 172.21.15.7:0/2158168682 shutdown_connections 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.208+0000 7f13d0b09640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13b40a2950 0x7f13b40a4e10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.208+0000 7f13d0b09640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13cc102190 0x7f13cc1a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.208+0000 7f13d0b09640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13cc1017d0 0x7f13cc19a0c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.208+0000 7f13d0b09640 1 --2- 172.21.15.7:0/2158168682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc1005e0 0x7f13cc199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.208+0000 7f13d0b09640 1 -- 172.21.15.7:0/2158168682 >> 172.21.15.7:0/2158168682 conn(0x7f13cc0fbb70 msgr2=0x7f13cc104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.208+0000 7f13d0b09640 1 -- 172.21.15.7:0/2158168682 shutdown_connections 2024-03-26T08:44:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.208+0000 7f13d0b09640 1 -- 172.21.15.7:0/2158168682 wait complete. 2024-03-26T08:44:59.213 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:44:59 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2600276250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:44:59.215 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:44:59.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:44:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.390+0000 7f78aae3f640 1 -- 172.21.15.7:0/2943925955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a4102190 msgr2=0x7f78a4108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.390+0000 7f78aae3f640 1 --2- 172.21.15.7:0/2943925955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a4102190 0x7f78a4108a20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f789c009ca0 tx=0x7f789c02c100 comp rx=0 tx=0).stop 2024-03-26T08:44:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.390+0000 7f78aae3f640 1 -- 172.21.15.7:0/2943925955 shutdown_connections 2024-03-26T08:44:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.390+0000 7f78aae3f640 1 --2- 172.21.15.7:0/2943925955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a4102190 0x7f78a4108a20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.390+0000 7f78aae3f640 1 --2- 172.21.15.7:0/2943925955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a41017d0 0x7f78a4101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.390+0000 7f78aae3f640 1 --2- 172.21.15.7:0/2943925955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a41005e0 0x7f78a4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.390+0000 7f78aae3f640 1 -- 172.21.15.7:0/2943925955 >> 172.21.15.7:0/2943925955 conn(0x7f78a40fbb70 msgr2=0x7f78a40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.391+0000 7f78aae3f640 1 -- 172.21.15.7:0/2943925955 shutdown_connections 2024-03-26T08:44:59.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.391+0000 7f78aae3f640 1 -- 172.21.15.7:0/2943925955 wait complete. 2024-03-26T08:44:59.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.391+0000 7f78aae3f640 1 Processor -- start 2024-03-26T08:44:59.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.391+0000 7f78aae3f640 1 -- start start 2024-03-26T08:44:59.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78aae3f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a41005e0 0x7f78a4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78aae3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a41017d0 0x7f78a4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78aae3f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a4102190 0x7f78a419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78aae3f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a410d3e0 con 0x7f78a41005e0 2024-03-26T08:44:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78aae3f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a410d260 con 0x7f78a41017d0 2024-03-26T08:44:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78aae3f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a410d560 con 0x7f78a4102190 2024-03-26T08:44:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78a93dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a4102190 0x7f78a419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78a8bdb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a41005e0 0x7f78a4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78a93dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a4102190 0x7f78a419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33912/0 (socket says 172.21.15.7:33912) 2024-03-26T08:44:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78a93dc640 1 -- 172.21.15.7:0/3557114981 learned_addr learned my addr 172.21.15.7:0/3557114981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.392+0000 7f78a3fff640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a41017d0 0x7f78a4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.393+0000 7f78a8bdb640 1 -- 172.21.15.7:0/3557114981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a4102190 msgr2=0x7f78a419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.393+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a4102190 0x7f78a419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.393+0000 7f78a8bdb640 1 -- 172.21.15.7:0/3557114981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a41017d0 msgr2=0x7f78a4197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.393+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a41017d0 0x7f78a4197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.393+0000 7f78a8bdb640 1 -- 172.21.15.7:0/3557114981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a419f620 con 0x7f78a41005e0 2024-03-26T08:44:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.393+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a41005e0 0x7f78a4197a00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f789400b3c0 tx=0x7f789400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.394+0000 7f78a1ffb640 1 -- 172.21.15.7:0/3557114981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894011830 con 0x7f78a41005e0 2024-03-26T08:44:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.394+0000 7f78aae3f640 1 -- 172.21.15.7:0/3557114981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a406b120 con 0x7f78a41005e0 2024-03-26T08:44:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.394+0000 7f78aae3f640 1 -- 172.21.15.7:0/3557114981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a406b6f0 con 0x7f78a41005e0 2024-03-26T08:44:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.394+0000 7f78a1ffb640 1 -- 172.21.15.7:0/3557114981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7894016070 con 0x7f78a41005e0 2024-03-26T08:44:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.395+0000 7f78a1ffb640 1 -- 172.21.15.7:0/3557114981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789400f2e0 con 0x7f78a41005e0 2024-03-26T08:44:59.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.397+0000 7f78a1ffb640 1 -- 172.21.15.7:0/3557114981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f789400f480 con 0x7f78a41005e0 2024-03-26T08:44:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78ada20740 1 Processor -- start 2024-03-26T08:44:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78ada20740 1 -- start start 2024-03-26T08:44:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78ada20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78ada20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c6080 0x55c3da0cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78ada20740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78ada20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3da088610 con 0x55c3da0c5690 2024-03-26T08:44:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78ada20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3da02b3c0 con 0x55c3da0c6080 2024-03-26T08:44:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78ada20740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d9fd3d60 con 0x55c3da0cb1f0 2024-03-26T08:44:59.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a8bdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c6080 0x55c3da0cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a93dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a1ffb640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f788c0a2950 0x7f788c0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a93dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0c5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50374/0 (socket says 172.21.15.7:50374) 2024-03-26T08:44:59.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a93dc640 1 -- 172.21.15.7:0/3037005845 learned_addr learned my addr 172.21.15.7:0/3037005845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a3fff640 1 -- 172.21.15.7:0/3037005845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c6080 msgr2=0x55c3da0cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a3fff640 1 --2- 172.21.15.7:0/3037005845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c6080 0x55c3da0cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a3fff640 1 -- 172.21.15.7:0/3037005845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 msgr2=0x55c3da0c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a3fff640 1 --2- 172.21.15.7:0/3037005845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a3fff640 1 -- 172.21.15.7:0/3037005845 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d9d3f3f0 con 0x55c3da0cb1f0 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.399+0000 7f78a93dc640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f788c0a2950 0x7f788c0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.399+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/3037005845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c6080 0x55c3da0cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.398+0000 7f78a1ffb640 1 -- 172.21.15.7:0/3557114981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78940cde00 con 0x7f78a41005e0 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.400+0000 7f78a93dc640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f788c0a2950 0x7f788c0a4e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c3da0c68e0 tx=0x7f789c02c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.400+0000 7f78a3fff640 1 --2- 172.21.15.7:0/3037005845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0cd680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7898009bf0 tx=0x7f7898011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5afac2349f190c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.400+0000 7f788affd640 1 -- 172.21.15.7:0/3037005845 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898009da0 con 0x55c3da0cb1f0 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.401+0000 7f788affd640 1 -- 172.21.15.7:0/3037005845 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7898005020 con 0x55c3da0cb1f0 2024-03-26T08:44:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.401+0000 7f788affd640 1 -- 172.21.15.7:0/3037005845 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898004bc0 con 0x55c3da0cb1f0 2024-03-26T08:44:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.401+0000 7f78ada20740 1 -- 172.21.15.7:0/3037005845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 msgr2=0x55c3da0cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.401+0000 7f78ada20740 1 --2- 172.21.15.7:0/3037005845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0cd680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7898009bf0 tx=0x7f7898011810 comp rx=0 tx=0).stop 2024-03-26T08:44:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.401+0000 7f78ada20740 1 -- 172.21.15.7:0/3037005845 shutdown_connections 2024-03-26T08:44:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.401+0000 7f78ada20740 1 --2- 172.21.15.7:0/3037005845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0cd680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.401+0000 7f78ada20740 1 --2- 172.21.15.7:0/3037005845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c6080 0x55c3da0cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.401+0000 7f78ada20740 1 --2- 172.21.15.7:0/3037005845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.401+0000 7f78ada20740 1 -- 172.21.15.7:0/3037005845 >> 172.21.15.7:0/3037005845 conn(0x55c3da0c0c00 msgr2=0x55c3da0c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.402+0000 7f78ada20740 1 -- 172.21.15.7:0/3037005845 shutdown_connections 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.402+0000 7f78ada20740 1 -- 172.21.15.7:0/3037005845 wait complete. 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.402+0000 7f78ada20740 1 -- start start 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.402+0000 7f78ada20740 1 Processor -- start 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.402+0000 7f78ada20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.402+0000 7f78ada20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 0x55c3da0cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.402+0000 7f78ada20740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.402+0000 7f78ada20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d9fe7240 con 0x55c3da0c6080 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.402+0000 7f78ada20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3da027460 con 0x55c3da0c5690 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a8bdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78ada20740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3da0ce1a0 con 0x55c3da0cb1f0 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a8bdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35042/0 (socket says 172.21.15.7:35042) 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a8bdb640 1 -- 172.21.15.7:0/1568064411 learned_addr learned my addr 172.21.15.7:0/1568064411 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a3fff640 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 0x55c3da0cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a93dc640 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78ada20740 10 client.0 _ll_register_callbacks cb 0x55c3d9fe6c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a93dc640 1 -- 172.21.15.7:0/1568064411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 msgr2=0x55c3da0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a93dc640 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a93dc640 1 -- 172.21.15.7:0/1568064411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 msgr2=0x55c3da0cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a93dc640 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 0x55c3da0cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.403+0000 7f78a93dc640 1 -- 172.21.15.7:0/1568064411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3d9e097d0 con 0x55c3da0cb1f0 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.404+0000 7f78a93dc640 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d6fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f789c005e30 tx=0x7f789c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=940526a76c47464b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.404+0000 7f7883fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.404+0000 7f7883fff640 1 -- 172.21.15.7:0/1568064411 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c064060 con 0x55c3da0cb1f0 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.404+0000 7f7883fff640 1 -- 172.21.15.7:0/1568064411 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f789c004450 con 0x55c3da0cb1f0 2024-03-26T08:44:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.404+0000 7f78ada20740 1 -- 172.21.15.7:0/1568064411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7898005020 con 0x55c3da0cb1f0 2024-03-26T08:44:59.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f7883fff640 1 -- 172.21.15.7:0/1568064411 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f789c058070 con 0x55c3da0cb1f0 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f78ada20740 10 client.25966 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f78ada20740 1 -- 172.21.15.7:0/1568064411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c3da0da360 con 0x55c3da0cb1f0 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f7883fff640 1 -- 172.21.15.7:0/1568064411 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7898005020 con 0x55c3da0cb1f0 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f7883fff640 1 -- 172.21.15.7:0/1568064411 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f789c069070 con 0x55c3da0cb1f0 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f78ada20740 10 client.25966 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f78ada20740 10 client.25966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f78ada20740 20 client.25966 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f78ada20740 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c3da0dba30 0x55c3da0ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f78ada20740 4 client.25966 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f78ada20740 1 -- 172.21.15.7:0/1568064411 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3da038900 con 0x55c3da0dba30 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.405+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c3da0dba30 0x55c3da0ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.408+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c3da0dba30 0x55c3da0ddea0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7c1b323bcad04ae server_cookie=687ffb6f32aa2e85 in_seq=0 out_seq=0 2024-03-26T08:44:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.408+0000 7f7883fff640 10 client.25966 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.408+0000 7f7883fff640 1 -- 172.21.15.7:0/1568064411 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c3da038900 con 0x55c3da0dba30 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.408+0000 7f7883fff640 10 client.25966 handle_command_reply: tid=0 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.408+0000 7f78ada20740 1 client.25966 shutdown 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f78ada20740 1 -- 172.21.15.7:0/1568064411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 msgr2=0x55c3da0d6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f78ada20740 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d6fa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f789c005e30 tx=0x7f789c0095c0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f78ada20740 1 -- 172.21.15.7:0/1568064411 shutdown_connections 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f78ada20740 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c3da0dba30 0x55c3da0ddea0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f78ada20740 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d6fa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f78ada20740 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 0x55c3da0cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f78ada20740 1 --2- 172.21.15.7:0/1568064411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f78ada20740 1 -- 172.21.15.7:0/1568064411 >> 172.21.15.7:0/1568064411 conn(0x55c3da0c0c00 msgr2=0x55c3da0bfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f7883fff640 0 client.25966 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:44:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.409+0000 7f78ada20740 1 -- 172.21.15.7:0/1568064411 shutdown_connections 2024-03-26T08:44:59.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.410+0000 7f78ada20740 1 -- 172.21.15.7:0/1568064411 wait complete. 2024-03-26T08:44:59.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.410+0000 7f78ada20740 20 client.25966 trim_cache size 0 max 16384 2024-03-26T08:44:59.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.430+0000 7f78ada20740 1 Processor -- start 2024-03-26T08:44:59.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.430+0000 7f78ada20740 1 -- start start 2024-03-26T08:44:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78ada20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0dda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78ada20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 0x55c3da0d6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78ada20740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78ada20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7890010980 con 0x55c3da0c6080 2024-03-26T08:44:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78ada20740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789000e480 con 0x55c3da0c5690 2024-03-26T08:44:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78ada20740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789000c500 con 0x55c3da0cb1f0 2024-03-26T08:44:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78a8bdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78a8bdb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0dda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35048/0 (socket says 172.21.15.7:35048) 2024-03-26T08:44:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78a8bdb640 1 -- 172.21.15.7:0/2495677025 learned_addr learned my addr 172.21.15.7:0/2495677025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78a3fff640 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 0x55c3da0d6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78a93dc640 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78a8bdb640 1 -- 172.21.15.7:0/2495677025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 msgr2=0x55c3da0c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78a8bdb640 1 -- 172.21.15.7:0/2495677025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 msgr2=0x55c3da0d6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.431+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 0x55c3da0d6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.432+0000 7f78a8bdb640 1 -- 172.21.15.7:0/2495677025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3da0de210 con 0x55c3da0c5690 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.432+0000 7f78a3fff640 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 0x55c3da0d6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.432+0000 7f78a93dc640 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.432+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0dda40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f78940cb8d0 tx=0x7f7894013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b51df3252e174050 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.432+0000 7f788a7fc640 1 -- 172.21.15.7:0/2495677025 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894009180 con 0x55c3da0c5690 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.432+0000 7f788a7fc640 1 -- 172.21.15.7:0/2495677025 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f789409e8f0 con 0x55c3da0c5690 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f788a7fc640 1 -- 172.21.15.7:0/2495677025 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894096e10 con 0x55c3da0c5690 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 -- 172.21.15.7:0/2495677025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 msgr2=0x55c3da0dda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0dda40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f78940cb8d0 tx=0x7f7894013040 comp rx=0 tx=0).stop 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 -- 172.21.15.7:0/2495677025 shutdown_connections 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0cb1f0 0x55c3da0c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c6080 0x55c3da0d6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 --2- 172.21.15.7:0/2495677025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0c5690 0x55c3da0dda40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 -- 172.21.15.7:0/2495677025 >> 172.21.15.7:0/2495677025 conn(0x55c3da0c0c00 msgr2=0x55c3da0c3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 -- 172.21.15.7:0/2495677025 shutdown_connections 2024-03-26T08:44:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 -- 172.21.15.7:0/2495677025 wait complete. 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.433+0000 7f78ada20740 1 -- start start 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.434+0000 7f78ada20740 1 Processor -- start 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.434+0000 7f78ada20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.434+0000 7f78ada20740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0c6080 0x55c3da0d4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.434+0000 7f78ada20740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.434+0000 7f78a8bdb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.434+0000 7f78a8bdb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0d47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50396/0 (socket says 172.21.15.7:50396) 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.434+0000 7f78a3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0c6080 0x55c3da0d4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0c6080 0x55c3da0d4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33940/0 (socket says 172.21.15.7:33940) 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a8bdb640 1 -- 172.21.15.7:0/554498433 learned_addr learned my addr 172.21.15.7:0/554498433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a93dc640 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789000cb00 con 0x55c3da0c5690 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789000de20 con 0x55c3da0cb1f0 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789000d160 con 0x55c3da0c6080 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78ada20740 10 client.0 _ll_register_callbacks cb 0x7f789c032c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a8bdb640 1 -- 172.21.15.7:0/554498433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0c6080 msgr2=0x55c3da0d4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0c6080 0x55c3da0d4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a8bdb640 1 -- 172.21.15.7:0/554498433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0cb1f0 msgr2=0x55c3da0d5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a8bdb640 1 -- 172.21.15.7:0/554498433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3da0d6300 con 0x55c3da0c5690 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a93dc640 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a3fff640 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0c6080 0x55c3da0d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78a8bdb640 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0d47c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f78940cb8a0 tx=0x7f78940a2f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1576dbf00723fe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f7883fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f7883fff640 1 -- 172.21.15.7:0/554498433 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78940b8060 con 0x55c3da0c5690 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.435+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f789409e8f0 con 0x55c3da0c5690 2024-03-26T08:44:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.436+0000 7f7883fff640 1 -- 172.21.15.7:0/554498433 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7894010640 con 0x55c3da0c5690 2024-03-26T08:44:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.436+0000 7f7883fff640 1 -- 172.21.15.7:0/554498433 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f789409a7a0 con 0x55c3da0c5690 2024-03-26T08:44:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.436+0000 7f78ada20740 10 client.16107 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.436+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c3da137480 con 0x55c3da0c5690 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.436+0000 7f7883fff640 1 -- 172.21.15.7:0/554498433 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f789409e8f0 con 0x55c3da0c5690 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.439+0000 7f7883fff640 1 -- 172.21.15.7:0/554498433 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7894099bf0 con 0x55c3da0c5690 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.439+0000 7f78ada20740 10 client.16107 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.439+0000 7f78ada20740 10 client.16107 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.439+0000 7f78ada20740 20 client.16107 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.439+0000 7f78ada20740 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c3da137db0 0x55c3da13a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.439+0000 7f78ada20740 4 client.16107 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.439+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3da032c50 con 0x55c3da137db0 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.439+0000 7f78a3fff640 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c3da137db0 0x55c3da13a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78a3fff640 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c3da137db0 0x55c3da13a1d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b1f9b6be3ac84e0 server_cookie=4c21e6845ac3c47e in_seq=0 out_seq=0 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f7883fff640 10 client.16107 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f7883fff640 1 -- 172.21.15.7:0/554498433 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3da032c50 con 0x55c3da137db0 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f7883fff640 10 client.16107 handle_command_reply: tid=0 2024-03-26T08:44:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 client.16107 shutdown 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 msgr2=0x55c3da0d47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0d47c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f78940cb8a0 tx=0x7f78940a2f40 comp rx=0 tx=0).stop 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 shutdown_connections 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c3da0cb1f0 0x55c3da0d5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55c3da137db0 0x55c3da13a1d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c3da0c6080 0x55c3da0d4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 --2- 172.21.15.7:0/554498433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c3da0c5690 0x55c3da0d47c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 >> 172.21.15.7:0/554498433 conn(0x55c3da0c0c00 msgr2=0x55c3da0c46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f7883fff640 0 client.16107 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 shutdown_connections 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.440+0000 7f78ada20740 1 -- 172.21.15.7:0/554498433 wait complete. 2024-03-26T08:44:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78ada20740 20 client.16107 trim_cache size 0 max 16384 2024-03-26T08:44:59.434 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:44:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 -- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f788c0a2950 msgr2=0x7f788c0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f788c0a2950 0x7f788c0a4e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55c3da0c68e0 tx=0x7f789c02c100 comp rx=0 tx=0).stop 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 -- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a41005e0 msgr2=0x7f78a4197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a41005e0 0x7f78a4197a00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f789400b3c0 tx=0x7f789400b880 comp rx=0 tx=0).stop 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 -- 172.21.15.7:0/3557114981 shutdown_connections 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f788c0a2950 0x7f788c0a4e10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f78a4102190 0x7f78a419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a41017d0 0x7f78a4197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 --2- 172.21.15.7:0/3557114981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78a41005e0 0x7f78a4197a00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 -- 172.21.15.7:0/3557114981 >> 172.21.15.7:0/3557114981 conn(0x7f78a40fbb70 msgr2=0x7f78a4104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.441+0000 7f78aae3f640 1 -- 172.21.15.7:0/3557114981 shutdown_connections 2024-03-26T08:44:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.442+0000 7f78aae3f640 1 -- 172.21.15.7:0/3557114981 wait complete. 2024-03-26T08:44:59.449 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:44:59.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:44:59.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:44:59 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2600276250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:44:59.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:44:59 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2600276250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:44:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.600+0000 7facc5387640 1 -- 172.21.15.7:0/2543615855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc00ffd50 msgr2=0x7facc0100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.600+0000 7facc5387640 1 --2- 172.21.15.7:0/2543615855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc00ffd50 0x7facc0100170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7facb000d960 tx=0x7facb002db10 comp rx=0 tx=0).stop 2024-03-26T08:44:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.600+0000 7facc5387640 1 -- 172.21.15.7:0/2543615855 shutdown_connections 2024-03-26T08:44:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.600+0000 7facc5387640 1 --2- 172.21.15.7:0/2543615855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc0101900 0x7facc0108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.600+0000 7facc5387640 1 --2- 172.21.15.7:0/2543615855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc0100f40 0x7facc01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.600+0000 7facc5387640 1 --2- 172.21.15.7:0/2543615855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc00ffd50 0x7facc0100170 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7facb000d960 tx=0x7facb002db10 comp rx=0 tx=0).stop 2024-03-26T08:44:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.600+0000 7facc5387640 1 -- 172.21.15.7:0/2543615855 >> 172.21.15.7:0/2543615855 conn(0x7facc00fb2c0 msgr2=0x7facc00fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.601+0000 7facc5387640 1 -- 172.21.15.7:0/2543615855 shutdown_connections 2024-03-26T08:44:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.601+0000 7facc5387640 1 -- 172.21.15.7:0/2543615855 wait complete. 2024-03-26T08:44:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.602+0000 7facc5387640 1 Processor -- start 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.602+0000 7facc5387640 1 -- start start 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.602+0000 7facc5387640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc0100f40 0x7facc0199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.602+0000 7facc5387640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc0101900 0x7facc019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.602+0000 7facc5387640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc019f3b0 0x7facc01a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.602+0000 7facc5387640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc010b110 con 0x7facc019f3b0 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.602+0000 7facc5387640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc010af90 con 0x7facc0101900 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.602+0000 7facc5387640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc010b290 con 0x7facc0100f40 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc0101900 0x7facc019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc0101900 0x7facc019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35070/0 (socket says 172.21.15.7:35070) 2024-03-26T08:44:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbe7fc640 1 -- 172.21.15.7:0/839352485 learned_addr learned my addr 172.21.15.7:0/839352485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc019f3b0 0x7facc01a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbeffd640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc0100f40 0x7facc0199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbe7fc640 1 -- 172.21.15.7:0/839352485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc0100f40 msgr2=0x7facc0199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbe7fc640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc0100f40 0x7facc0199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbe7fc640 1 -- 172.21.15.7:0/839352485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc019f3b0 msgr2=0x7facc01a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbe7fc640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc019f3b0 0x7facc01a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.603+0000 7facbe7fc640 1 -- 172.21.15.7:0/839352485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc01a1e10 con 0x7facc0101900 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.604+0000 7facbe7fc640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc0101900 0x7facc019a3a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7facb400e9f0 tx=0x7facb400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.604+0000 7faca7fff640 1 -- 172.21.15.7:0/839352485 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb400cd80 con 0x7facc0101900 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.604+0000 7faca7fff640 1 -- 172.21.15.7:0/839352485 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facb4010910 con 0x7facc0101900 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.604+0000 7faca7fff640 1 -- 172.21.15.7:0/839352485 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4018990 con 0x7facc0101900 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.604+0000 7facc5387640 1 -- 172.21.15.7:0/839352485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc01a2100 con 0x7facc0101900 2024-03-26T08:44:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.604+0000 7facc5387640 1 -- 172.21.15.7:0/839352485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc01a2650 con 0x7facc0101900 2024-03-26T08:44:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.607+0000 7faca7fff640 1 -- 172.21.15.7:0/839352485 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7facb4018b30 con 0x7facc0101900 2024-03-26T08:44:59.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.607+0000 7faca7fff640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faca80a2800 0x7faca80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.607+0000 7facc7f72740 1 Processor -- start 2024-03-26T08:44:59.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.607+0000 7facc7f72740 1 -- start start 2024-03-26T08:44:59.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facc7f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d779ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbeffd640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faca80a2800 0x7faca80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facc7f72740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d77eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7faca7fff640 1 -- 172.21.15.7:0/839352485 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7facb40cef10 con 0x7facc0101900 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facc7f72740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 0x56036d781680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facc7f72740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036d73c610 con 0x56036d77a080 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facc7f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036d6df3c0 con 0x56036d779690 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facc7f72740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036d687d60 con 0x56036d77f1f0 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d779ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d77eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d77eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50430/0 (socket says 172.21.15.7:50430) 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d779ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35074/0 (socket says 172.21.15.7:35074) 2024-03-26T08:44:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbe7fc640 1 -- 172.21.15.7:0/3825010621 learned_addr learned my addr 172.21.15.7:0/3825010621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbf7fe640 1 --2- 172.21.15.7:0/3825010621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 0x56036d781680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbeffd640 1 -- 172.21.15.7:0/3825010621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 msgr2=0x56036d781680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbeffd640 1 --2- 172.21.15.7:0/3825010621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 0x56036d781680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbeffd640 1 -- 172.21.15.7:0/3825010621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 msgr2=0x56036d77eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbeffd640 1 --2- 172.21.15.7:0/3825010621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d77eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbeffd640 1 -- 172.21.15.7:0/3825010621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56036d3f33f0 con 0x56036d779690 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.608+0000 7facbeffd640 1 --2- 172.21.15.7:0/3825010621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d779ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7facc00ff890 tx=0x7facb00376e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ecdfea2b10842a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.609+0000 7facbeffd640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faca80a2800 0x7faca80a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7facb0037ac0 tx=0x7facb003c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.609+0000 7faca57fa640 1 -- 172.21.15.7:0/3825010621 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0053070 con 0x56036d779690 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.609+0000 7faca57fa640 1 -- 172.21.15.7:0/3825010621 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facb004a030 con 0x56036d779690 2024-03-26T08:44:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.609+0000 7faca57fa640 1 -- 172.21.15.7:0/3825010621 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0004450 con 0x56036d779690 2024-03-26T08:44:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 -- 172.21.15.7:0/3825010621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 msgr2=0x56036d779ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 --2- 172.21.15.7:0/3825010621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d779ab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7facc00ff890 tx=0x7facb00376e0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 -- 172.21.15.7:0/3825010621 shutdown_connections 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 --2- 172.21.15.7:0/3825010621 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 0x56036d781680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 --2- 172.21.15.7:0/3825010621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d77eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 --2- 172.21.15.7:0/3825010621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d779ab0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 -- 172.21.15.7:0/3825010621 >> 172.21.15.7:0/3825010621 conn(0x56036d774c00 msgr2=0x56036d775000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 -- 172.21.15.7:0/3825010621 shutdown_connections 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 -- 172.21.15.7:0/3825010621 wait complete. 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.610+0000 7facc7f72740 1 -- start start 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facc7f72740 1 Processor -- start 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facc7f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d78c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facc7f72740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d78c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facc7f72740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 0x56036d7937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facc7f72740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036d69b240 con 0x56036d77a080 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facc7f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036d6db460 con 0x56036d779690 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facc7f72740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036d782190 con 0x56036d77f1f0 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facc7f72740 10 client.0 _ll_register_callbacks cb 0x56036d69ac10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d78c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d78c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50442/0 (socket says 172.21.15.7:50442) 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facbf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 0x56036d7937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facbe7fc640 1 -- 172.21.15.7:0/914339271 learned_addr learned my addr 172.21.15.7:0/914339271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.611+0000 7facbeffd640 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d78c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7facbeffd640 1 -- 172.21.15.7:0/914339271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 msgr2=0x56036d7937f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7facbeffd640 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 0x56036d7937f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7facbeffd640 1 -- 172.21.15.7:0/914339271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 msgr2=0x56036d78c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7facbeffd640 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d78c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7facbeffd640 1 -- 172.21.15.7:0/914339271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56036d4bd7d0 con 0x56036d779690 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7facbeffd640 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d78c270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7facb0037bf0 tx=0x7facb0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbc81dc8af1a4b81 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7faca2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7faca2ff5640 1 -- 172.21.15.7:0/914339271 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb005e060 con 0x56036d779690 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7faca2ff5640 1 -- 172.21.15.7:0/914339271 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7facb00054d0 con 0x56036d779690 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.612+0000 7facc7f72740 1 -- 172.21.15.7:0/914339271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facb004a030 con 0x56036d779690 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7faca2ff5640 1 -- 172.21.15.7:0/914339271 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb0053040 con 0x56036d779690 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7facc7f72740 10 client.25961 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7facc7f72740 1 -- 172.21.15.7:0/914339271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56036d796b60 con 0x56036d779690 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7faca2ff5640 1 -- 172.21.15.7:0/914339271 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facb004a030 con 0x56036d779690 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7faca2ff5640 1 -- 172.21.15.7:0/914339271 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7facb002ea10 con 0x56036d779690 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7facc7f72740 10 client.25961 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7facc7f72740 10 client.25961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7facc7f72740 20 client.25961 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:59.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7facc7f72740 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56036d798380 0x56036d79a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7facc7f72740 4 client.25961 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:44:59.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.613+0000 7facc7f72740 1 -- 172.21.15.7:0/914339271 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56036d6ec900 con 0x56036d798380 2024-03-26T08:44:59.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.614+0000 7facbe7fc640 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56036d798380 0x56036d79a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.614+0000 7facbe7fc640 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56036d798380 0x56036d79a800 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977a0845bb849d28 server_cookie=f1c3ac3d83b70c67 in_seq=0 out_seq=0 2024-03-26T08:44:59.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.614+0000 7faca2ff5640 10 client.25961 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:59.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.615+0000 7faca2ff5640 1 -- 172.21.15.7:0/914339271 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56036d6ec900 con 0x56036d798380 2024-03-26T08:44:59.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.615+0000 7faca2ff5640 10 client.25961 handle_command_reply: tid=0 2024-03-26T08:44:59.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.615+0000 7facc7f72740 1 client.25961 shutdown 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 -- 172.21.15.7:0/914339271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 msgr2=0x56036d78c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d78c270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7facb0037bf0 tx=0x7facb0005f90 comp rx=0 tx=0).stop 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 -- 172.21.15.7:0/914339271 shutdown_connections 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56036d798380 0x56036d79a800 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d77f1f0 0x56036d7937f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77a080 0x56036d78c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 --2- 172.21.15.7:0/914339271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d779690 0x56036d78c270 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7facb0037bf0 tx=0x7facb0005f90 comp rx=0 tx=0).stop 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 -- 172.21.15.7:0/914339271 >> 172.21.15.7:0/914339271 conn(0x56036d774c00 msgr2=0x56036d773e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7faca2ff5640 0 client.25961 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 -- 172.21.15.7:0/914339271 shutdown_connections 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 1 -- 172.21.15.7:0/914339271 wait complete. 2024-03-26T08:44:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.616+0000 7facc7f72740 20 client.25961 trim_cache size 0 max 16384 2024-03-26T08:44:59.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.634+0000 7facc7f72740 1 Processor -- start 2024-03-26T08:44:59.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facc7f72740 1 -- start start 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facc7f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 0x56036d77b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facc7f72740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d77bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facc7f72740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d7937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facc7f72740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01a9370 con 0x56036d77f1f0 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facc7f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01a6e70 con 0x56036d77a080 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facc7f72740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01a4f10 con 0x56036d782c40 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d77bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 0x56036d77b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d77bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50450/0 (socket says 172.21.15.7:50450) 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facbe7fc640 1 -- 172.21.15.7:0/365694016 learned_addr learned my addr 172.21.15.7:0/365694016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.635+0000 7facbf7fe640 1 --2- 172.21.15.7:0/365694016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d7937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.636+0000 7facbe7fc640 1 -- 172.21.15.7:0/365694016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 msgr2=0x56036d7937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.636+0000 7facbe7fc640 1 --2- 172.21.15.7:0/365694016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d7937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.636+0000 7facbe7fc640 1 -- 172.21.15.7:0/365694016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 msgr2=0x56036d77b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.636+0000 7facbe7fc640 1 --2- 172.21.15.7:0/365694016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 0x56036d77b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.636+0000 7facbe7fc640 1 -- 172.21.15.7:0/365694016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56036d775970 con 0x56036d77f1f0 2024-03-26T08:44:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.636+0000 7facbe7fc640 1 --2- 172.21.15.7:0/365694016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d77bb60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7facb409b7b0 tx=0x7facb401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f48deda551ba2ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.636+0000 7faca4ff9640 1 -- 172.21.15.7:0/365694016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4099910 con 0x56036d77f1f0 2024-03-26T08:44:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.637+0000 7faca4ff9640 1 -- 172.21.15.7:0/365694016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facb4099b70 con 0x56036d77f1f0 2024-03-26T08:44:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.637+0000 7faca4ff9640 1 -- 172.21.15.7:0/365694016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4097ee0 con 0x56036d77f1f0 2024-03-26T08:44:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.637+0000 7facc7f72740 1 -- 172.21.15.7:0/365694016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 msgr2=0x56036d77bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.637+0000 7facc7f72740 1 --2- 172.21.15.7:0/365694016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d77bb60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7facb409b7b0 tx=0x7facb401a040 comp rx=0 tx=0).stop 2024-03-26T08:44:59.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.637+0000 7facc7f72740 1 -- 172.21.15.7:0/365694016 shutdown_connections 2024-03-26T08:44:59.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.637+0000 7facc7f72740 1 --2- 172.21.15.7:0/365694016 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d7937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.637+0000 7facc7f72740 1 --2- 172.21.15.7:0/365694016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d77bb60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.637+0000 7facc7f72740 1 --2- 172.21.15.7:0/365694016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 0x56036d77b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.637+0000 7facc7f72740 1 -- 172.21.15.7:0/365694016 >> 172.21.15.7:0/365694016 conn(0x56036d774c00 msgr2=0x56036d782530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.638+0000 7facc7f72740 1 -- 172.21.15.7:0/365694016 shutdown_connections 2024-03-26T08:44:59.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.638+0000 7facc7f72740 1 -- 172.21.15.7:0/365694016 wait complete. 2024-03-26T08:44:59.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.638+0000 7facc7f72740 1 -- start start 2024-03-26T08:44:59.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.638+0000 7facc7f72740 1 Processor -- start 2024-03-26T08:44:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facc7f72740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 0x56036d790b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facc7f72740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d7910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facc7f72740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d792050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facc7f72740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01a5510 con 0x56036d77f1f0 2024-03-26T08:44:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facc7f72740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01a6810 con 0x56036d77a080 2024-03-26T08:44:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facc7f72740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01a5b50 con 0x56036d782c40 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facc7f72740 10 client.0 _ll_register_callbacks cb 0x7facb8009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d792050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d792050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34010/0 (socket says 172.21.15.7:34010) 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbf7fe640 1 -- 172.21.15.7:0/2734315750 learned_addr learned my addr 172.21.15.7:0/2734315750 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbeffd640 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 0x56036d790b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbe7fc640 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d7910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbeffd640 1 -- 172.21.15.7:0/2734315750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 msgr2=0x56036d792050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbeffd640 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d792050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbeffd640 1 -- 172.21.15.7:0/2734315750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 msgr2=0x56036d7910b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:44:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbeffd640 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d7910b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbeffd640 1 -- 172.21.15.7:0/2734315750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56036d792640 con 0x56036d77a080 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbe7fc640 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d7910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.639+0000 7facbf7fe640 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d792050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facbeffd640 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 0x56036d790b70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7facb002edb0 tx=0x7facb0004d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87525c8f432e44af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7faca2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7faca2ff5640 1 -- 172.21.15.7:0/2734315750 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0068040 con 0x56036d77a080 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facc7f72740 1 -- 172.21.15.7:0/2734315750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facb4099b70 con 0x56036d77a080 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7faca2ff5640 1 -- 172.21.15.7:0/2734315750 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7facb002e5f0 con 0x56036d77a080 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7faca2ff5640 1 -- 172.21.15.7:0/2734315750 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb005e040 con 0x56036d77a080 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facc7f72740 10 client.25967 fetch_fsmap learned FSMap version 25 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facc7f72740 1 -- 172.21.15.7:0/2734315750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56036d793940 con 0x56036d77a080 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7faca2ff5640 1 -- 172.21.15.7:0/2734315750 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7facb4099b70 con 0x56036d77a080 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7faca2ff5640 1 -- 172.21.15.7:0/2734315750 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7facb002f4c0 con 0x56036d77a080 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facc7f72740 10 client.25967 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:44:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facc7f72740 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facc7f72740 20 client.25967 populate_metadata read hostname 'smithi007' 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facc7f72740 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56036d7f40e0 0x56036d7f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facc7f72740 4 client.25967 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.640+0000 7facc7f72740 1 -- 172.21.15.7:0/2734315750 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56036d6e6c50 con 0x56036d7f40e0 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.641+0000 7facbe7fc640 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56036d7f40e0 0x56036d7f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7facbe7fc640 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56036d7f40e0 0x56036d7f6500 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eabdec448ad11c39 server_cookie=41552e0af92ff825 in_seq=0 out_seq=0 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7faca2ff5640 10 client.25967 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7faca2ff5640 1 -- 172.21.15.7:0/2734315750 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56036d6e6c50 con 0x56036d7f40e0 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7faca2ff5640 10 client.25967 handle_command_reply: tid=0 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7facc7f72740 1 client.25967 shutdown 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7facc7f72740 1 -- 172.21.15.7:0/2734315750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 msgr2=0x56036d790b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7facc7f72740 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 0x56036d790b70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7facb002edb0 tx=0x7facb0004d40 comp rx=0 tx=0).stop 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7facc7f72740 1 -- 172.21.15.7:0/2734315750 shutdown_connections 2024-03-26T08:44:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7facc7f72740 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56036d7f40e0 0x56036d7f6500 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7facc7f72740 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56036d782c40 0x56036d792050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.642+0000 7faca2ff5640 0 client.25967 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.643+0000 7facc7f72740 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56036d77f1f0 0x56036d7910b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.643+0000 7facc7f72740 1 --2- 172.21.15.7:0/2734315750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56036d77a080 0x56036d790b70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.643+0000 7facc7f72740 1 -- 172.21.15.7:0/2734315750 >> 172.21.15.7:0/2734315750 conn(0x56036d774c00 msgr2=0x56036d77b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.643+0000 7facc7f72740 1 -- 172.21.15.7:0/2734315750 shutdown_connections 2024-03-26T08:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.643+0000 7facc7f72740 1 -- 172.21.15.7:0/2734315750 wait complete. 2024-03-26T08:44:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.643+0000 7facc7f72740 20 client.25967 trim_cache size 0 max 16384 2024-03-26T08:44:59.639 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 -- 172.21.15.7:0/839352485 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faca80a2800 msgr2=0x7faca80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faca80a2800 0x7faca80a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7facb0037ac0 tx=0x7facb003c000 comp rx=0 tx=0).stop 2024-03-26T08:44:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 -- 172.21.15.7:0/839352485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc0101900 msgr2=0x7facc019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc0101900 0x7facc019a3a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7facb400e9f0 tx=0x7facb400eeb0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 -- 172.21.15.7:0/839352485 shutdown_connections 2024-03-26T08:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faca80a2800 0x7faca80a4cc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc019f3b0 0x7facc01a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc0101900 0x7facc019a3a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 --2- 172.21.15.7:0/839352485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7facc0100f40 0x7facc0199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 -- 172.21.15.7:0/839352485 >> 172.21.15.7:0/839352485 conn(0x7facc00fb2c0 msgr2=0x7facc0104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 -- 172.21.15.7:0/839352485 shutdown_connections 2024-03-26T08:44:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:44:59.644+0000 7facc5387640 1 -- 172.21.15.7:0/839352485 wait complete. 2024-03-26T08:45:00.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:00 smithi016 ceph-mon[35851]: pgmap v437: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 24 MiB/s wr, 856 op/s 2024-03-26T08:45:00.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:00 smithi007 ceph-mon[32462]: pgmap v437: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 24 MiB/s wr, 856 op/s 2024-03-26T08:45:00.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:00 smithi044 ceph-mon[36461]: pgmap v437: 97 pgs: 97 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 24 MiB/s wr, 856 op/s 2024-03-26T08:45:00.650 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:45:00.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:45:00.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:45:00.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.833+0000 7fcedb885640 1 -- 172.21.15.7:0/878356285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fced4108ed0 msgr2=0x7fced4109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.833+0000 7fcedb885640 1 --2- 172.21.15.7:0/878356285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fced4108ed0 0x7fced4109350 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009900 tx=0x7fcec402bc30 comp rx=0 tx=0).stop 2024-03-26T08:45:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.833+0000 7fcedb885640 1 -- 172.21.15.7:0/878356285 shutdown_connections 2024-03-26T08:45:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.833+0000 7fcedb885640 1 --2- 172.21.15.7:0/878356285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4109890 0x7fced4110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.833+0000 7fcedb885640 1 --2- 172.21.15.7:0/878356285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fced4108ed0 0x7fced4109350 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.833+0000 7fcedb885640 1 --2- 172.21.15.7:0/878356285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fced4107ce0 0x7fced4108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.833+0000 7fcedb885640 1 -- 172.21.15.7:0/878356285 >> 172.21.15.7:0/878356285 conn(0x7fced4076c90 msgr2=0x7fced40790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.834+0000 7fcedb885640 1 -- 172.21.15.7:0/878356285 shutdown_connections 2024-03-26T08:45:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.834+0000 7fcedb885640 1 -- 172.21.15.7:0/878356285 wait complete. 2024-03-26T08:45:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.835+0000 7fcedb885640 1 Processor -- start 2024-03-26T08:45:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.835+0000 7fcedb885640 1 -- start start 2024-03-26T08:45:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.835+0000 7fcedb885640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fced4107ce0 0x7fced419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.835+0000 7fcedb885640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fced4108ed0 0x7fced419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.835+0000 7fcedb885640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4109890 0x7fced41a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.835+0000 7fcedb885640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41139b0 con 0x7fced4107ce0 2024-03-26T08:45:00.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.835+0000 7fcedb885640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced4113830 con 0x7fced4109890 2024-03-26T08:45:00.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.836+0000 7fcedb885640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced4113b30 con 0x7fced4108ed0 2024-03-26T08:45:00.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.836+0000 7fced8e20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fced4108ed0 0x7fced419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.836+0000 7fced9e22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4109890 0x7fced41a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.836+0000 7fced9e22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4109890 0x7fced41a5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35112/0 (socket says 172.21.15.7:35112) 2024-03-26T08:45:00.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.836+0000 7fced9e22640 1 -- 172.21.15.7:0/3777917203 learned_addr learned my addr 172.21.15.7:0/3777917203 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.837+0000 7fced9e22640 1 -- 172.21.15.7:0/3777917203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fced4108ed0 msgr2=0x7fced419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.837+0000 7fced9e22640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fced4108ed0 0x7fced419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.837+0000 7fced9e22640 1 -- 172.21.15.7:0/3777917203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fced4107ce0 msgr2=0x7fced419ddd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.837+0000 7fced9e22640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fced4107ce0 0x7fced419ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.837+0000 7fced9e22640 1 -- 172.21.15.7:0/3777917203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced41a59f0 con 0x7fced4109890 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.838+0000 7fced9e22640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4109890 0x7fced41a5380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00eb10 tx=0x7fcecc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.838+0000 7fceca7fc640 1 -- 172.21.15.7:0/3777917203 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc004b20 con 0x7fced4109890 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.838+0000 7fceca7fc640 1 -- 172.21.15.7:0/3777917203 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcecc004cc0 con 0x7fced4109890 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.838+0000 7fceca7fc640 1 -- 172.21.15.7:0/3777917203 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc010760 con 0x7fced4109890 2024-03-26T08:45:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.838+0000 7fcedb885640 1 -- 172.21.15.7:0/3777917203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced41a5ce0 con 0x7fced4109890 2024-03-26T08:45:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.839+0000 7fcedb885640 1 -- 172.21.15.7:0/3777917203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced41a61b0 con 0x7fced4109890 2024-03-26T08:45:00.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.841+0000 7fceca7fc640 1 -- 172.21.15.7:0/3777917203 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcecc005020 con 0x7fced4109890 2024-03-26T08:45:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fceca7fc640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcec00a2950 0x7fcec00a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fcede478740 1 Processor -- start 2024-03-26T08:45:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fcede478740 1 -- start start 2024-03-26T08:45:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fceca7fc640 1 -- 172.21.15.7:0/3777917203 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcecc0cefc0 con 0x7fced4109890 2024-03-26T08:45:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fcede478740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46e960 0x55684b46ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fcede478740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46f350 0x55684b473e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fcede478740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b476950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fcede478740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684b3aec90 con 0x55684b4744c0 2024-03-26T08:45:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fcede478740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684b3d4070 con 0x55684b46f350 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fcede478740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684b37cc80 con 0x55684b46e960 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fced9621640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcec00a2950 0x7fcec00a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fced8e20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46f350 0x55684b473e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fced9e22640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b476950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fced8e20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46f350 0x55684b473e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35128/0 (socket says 172.21.15.7:35128) 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fced8e20640 1 -- 172.21.15.7:0/1946891848 learned_addr learned my addr 172.21.15.7:0/1946891848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.843+0000 7fced9621640 1 --2- 172.21.15.7:0/1946891848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46e960 0x55684b46ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.844+0000 7fced8e20640 1 -- 172.21.15.7:0/1946891848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46e960 msgr2=0x55684b46ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.844+0000 7fced8e20640 1 --2- 172.21.15.7:0/1946891848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46e960 0x55684b46ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.844+0000 7fced8e20640 1 -- 172.21.15.7:0/1946891848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 msgr2=0x55684b476950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.844+0000 7fced8e20640 1 --2- 172.21.15.7:0/1946891848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b476950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.844+0000 7fced8e20640 1 -- 172.21.15.7:0/1946891848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55684b374d10 con 0x55684b46f350 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.844+0000 7fced9e22640 1 --2- 172.21.15.7:0/1946891848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b476950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.844+0000 7fced9621640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcec00a2950 0x7fcec00a4e10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fced0009460 tx=0x7fced00099c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.844+0000 7fced8e20640 1 --2- 172.21.15.7:0/1946891848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46f350 0x55684b473e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009900 tx=0x7fcec4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1169d5a6311e090 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.844+0000 7fcebf7fe640 1 -- 172.21.15.7:0/1946891848 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4045070 con 0x55684b46f350 2024-03-26T08:45:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.845+0000 7fcebf7fe640 1 -- 172.21.15.7:0/1946891848 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcec402cb00 con 0x55684b46f350 2024-03-26T08:45:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.845+0000 7fcebf7fe640 1 -- 172.21.15.7:0/1946891848 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4005120 con 0x55684b46f350 2024-03-26T08:45:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 -- 172.21.15.7:0/1946891848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46f350 msgr2=0x55684b473e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 --2- 172.21.15.7:0/1946891848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46f350 0x55684b473e50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009900 tx=0x7fcec4000bc0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 -- 172.21.15.7:0/1946891848 shutdown_connections 2024-03-26T08:45:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 --2- 172.21.15.7:0/1946891848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b476950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 --2- 172.21.15.7:0/1946891848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46f350 0x55684b473e50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 --2- 172.21.15.7:0/1946891848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46e960 0x55684b46ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 -- 172.21.15.7:0/1946891848 >> 172.21.15.7:0/1946891848 conn(0x55684b4371a0 msgr2=0x55684b46a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 -- 172.21.15.7:0/1946891848 shutdown_connections 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 -- 172.21.15.7:0/1946891848 wait complete. 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 -- start start 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.846+0000 7fcede478740 1 Processor -- start 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fcede478740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b478cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fcede478740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b479230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fcede478740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b480270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fcede478740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684b3900d0 con 0x55684b4744c0 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fcede478740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684b3d0110 con 0x55684b46e960 2024-03-26T08:45:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fcede478740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55684b477460 con 0x55684b46f350 2024-03-26T08:45:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fcede478740 10 client.0 _ll_register_callbacks cb 0x55684b3cbe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fced8e20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b479230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fced8e20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b479230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34048/0 (socket says 172.21.15.7:34048) 2024-03-26T08:45:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fced8e20640 1 -- 172.21.15.7:0/3764203457 learned_addr learned my addr 172.21.15.7:0/3764203457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.847+0000 7fced9e22640 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b480270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fced9621640 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b478cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fced9e22640 1 -- 172.21.15.7:0/3764203457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 msgr2=0x55684b479230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fced9e22640 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b479230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fced9e22640 1 -- 172.21.15.7:0/3764203457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 msgr2=0x55684b478cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fced9e22640 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b478cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fced9e22640 1 -- 172.21.15.7:0/3764203457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55684b1b07e0 con 0x55684b4744c0 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fced9e22640 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b480270 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0cca80 tx=0x7fcecc09c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117fe5410148db64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fcebcff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fcebcff9640 1 -- 172.21.15.7:0/3764203457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc096020 con 0x55684b4744c0 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.848+0000 7fcebcff9640 1 -- 172.21.15.7:0/3764203457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcecc0a3c30 con 0x55684b4744c0 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.849+0000 7fcede478740 1 -- 172.21.15.7:0/3764203457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcec402cb00 con 0x55684b4744c0 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.849+0000 7fcebcff9640 1 -- 172.21.15.7:0/3764203457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcecc0a40c0 con 0x55684b4744c0 2024-03-26T08:45:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.849+0000 7fcede478740 10 client.16134 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.849+0000 7fcede478740 1 -- 172.21.15.7:0/3764203457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55684b210f70 con 0x55684b4744c0 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.849+0000 7fcebcff9640 1 -- 172.21.15.7:0/3764203457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcec402cb00 con 0x55684b4744c0 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.849+0000 7fcebcff9640 1 -- 172.21.15.7:0/3764203457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcecc09ad70 con 0x55684b4744c0 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.850+0000 7fcede478740 10 client.16134 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.850+0000 7fcede478740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.850+0000 7fcede478740 20 client.16134 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.850+0000 7fcede478740 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55684b485060 0x55684b4874e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.850+0000 7fcede478740 4 client.16134 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.850+0000 7fcede478740 1 -- 172.21.15.7:0/3764203457 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55684b3e15b0 con 0x55684b485060 2024-03-26T08:45:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.850+0000 7fced9621640 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55684b485060 0x55684b4874e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.851+0000 7fcebcff9640 10 client.16134 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.851+0000 7fced9621640 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55684b485060 0x55684b4874e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff704c8acda51955 server_cookie=a871911e4e1906a4 in_seq=0 out_seq=0 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.852+0000 7fcebcff9640 1 -- 172.21.15.7:0/3764203457 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55684b3e15b0 con 0x55684b485060 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.852+0000 7fcebcff9640 10 client.16134 handle_command_reply: tid=0 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcede478740 1 client.16134 shutdown 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcede478740 1 -- 172.21.15.7:0/3764203457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 msgr2=0x55684b480270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcede478740 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b480270 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0cca80 tx=0x7fcecc09c0e0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcede478740 1 -- 172.21.15.7:0/3764203457 shutdown_connections 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcede478740 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55684b485060 0x55684b4874e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcede478740 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b480270 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcede478740 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b479230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcede478740 1 --2- 172.21.15.7:0/3764203457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b478cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcede478740 1 -- 172.21.15.7:0/3764203457 >> 172.21.15.7:0/3764203457 conn(0x55684b4371a0 msgr2=0x55684b469530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:00.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.853+0000 7fcebcff9640 0 client.16134 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:00.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.854+0000 7fcede478740 1 -- 172.21.15.7:0/3764203457 shutdown_connections 2024-03-26T08:45:00.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.854+0000 7fcede478740 1 -- 172.21.15.7:0/3764203457 wait complete. 2024-03-26T08:45:00.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.854+0000 7fcede478740 20 client.16134 trim_cache size 0 max 16384 2024-03-26T08:45:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.891+0000 7fcede478740 1 Processor -- start 2024-03-26T08:45:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.891+0000 7fcede478740 1 -- start start 2024-03-26T08:45:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.891+0000 7fcede478740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b4786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.892+0000 7fcede478740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b46f350 0x55684b480230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.892+0000 7fcede478740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b4744c0 0x55684b46d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.892+0000 7fcede478740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41b12e0 con 0x55684b46f350 2024-03-26T08:45:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.892+0000 7fcede478740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41aede0 con 0x55684b46e960 2024-03-26T08:45:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.892+0000 7fcede478740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41ace60 con 0x55684b4744c0 2024-03-26T08:45:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.892+0000 7fced9621640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b4786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.892+0000 7fced8e20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b46f350 0x55684b480230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.892+0000 7fced8e20640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b46f350 0x55684b480230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50484/0 (socket says 172.21.15.7:50484) 2024-03-26T08:45:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.892+0000 7fced8e20640 1 -- 172.21.15.7:0/1336774982 learned_addr learned my addr 172.21.15.7:0/1336774982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fced9e22640 1 --2- 172.21.15.7:0/1336774982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b4744c0 0x55684b46d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fced8e20640 1 -- 172.21.15.7:0/1336774982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b4744c0 msgr2=0x55684b46d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fced8e20640 1 --2- 172.21.15.7:0/1336774982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b4744c0 0x55684b46d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fced8e20640 1 -- 172.21.15.7:0/1336774982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 msgr2=0x55684b4786f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fced8e20640 1 --2- 172.21.15.7:0/1336774982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b4786f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fced8e20640 1 -- 172.21.15.7:0/1336774982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55684b0a8170 con 0x55684b46f350 2024-03-26T08:45:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fced9e22640 1 --2- 172.21.15.7:0/1336774982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b4744c0 0x55684b46d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:45:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fced8e20640 1 --2- 172.21.15.7:0/1336774982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b46f350 0x55684b480230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcec4047000 tx=0x7fcec4005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13082e86b02c9b8d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fcebeffd640 1 -- 172.21.15.7:0/1336774982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec404e070 con 0x55684b46f350 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fcebeffd640 1 -- 172.21.15.7:0/1336774982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcec402cd00 con 0x55684b46f350 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.893+0000 7fcebeffd640 1 -- 172.21.15.7:0/1336774982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4045040 con 0x55684b46f350 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.894+0000 7fcede478740 1 -- 172.21.15.7:0/1336774982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b46f350 msgr2=0x55684b480230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.894+0000 7fcede478740 1 --2- 172.21.15.7:0/1336774982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b46f350 0x55684b480230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcec4047000 tx=0x7fcec4005e20 comp rx=0 tx=0).stop 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.894+0000 7fcede478740 1 -- 172.21.15.7:0/1336774982 shutdown_connections 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.895+0000 7fcede478740 1 --2- 172.21.15.7:0/1336774982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b4744c0 0x55684b46d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.895+0000 7fcede478740 1 --2- 172.21.15.7:0/1336774982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b46f350 0x55684b480230 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.895+0000 7fcede478740 1 --2- 172.21.15.7:0/1336774982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b4786f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.895+0000 7fcede478740 1 -- 172.21.15.7:0/1336774982 >> 172.21.15.7:0/1336774982 conn(0x55684b4371a0 msgr2=0x55684b477fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.895+0000 7fcede478740 1 -- 172.21.15.7:0/1336774982 shutdown_connections 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.895+0000 7fcede478740 1 -- 172.21.15.7:0/1336774982 wait complete. 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.895+0000 7fcede478740 1 -- start start 2024-03-26T08:45:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.895+0000 7fcede478740 1 Processor -- start 2024-03-26T08:45:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fcede478740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b47fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fcede478740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b47e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fcede478740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b47eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fcede478740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41ad460 con 0x55684b4744c0 2024-03-26T08:45:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced8e20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b47e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced8e20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b47e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34064/0 (socket says 172.21.15.7:34064) 2024-03-26T08:45:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fcede478740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41ae780 con 0x55684b46e960 2024-03-26T08:45:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced8e20640 1 -- 172.21.15.7:0/3671238785 learned_addr learned my addr 172.21.15.7:0/3671238785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fcede478740 1 -- 172.21.15.7:0/3671238785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41adac0 con 0x55684b46f350 2024-03-26T08:45:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced9e22640 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b47eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fcede478740 10 client.0 _ll_register_callbacks cb 0x7fcecc09b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced8e20640 1 -- 172.21.15.7:0/3671238785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 msgr2=0x55684b47fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced9621640 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b47fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced8e20640 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b47fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced8e20640 1 -- 172.21.15.7:0/3671238785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 msgr2=0x55684b47eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced8e20640 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b47eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced8e20640 1 -- 172.21.15.7:0/3671238785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55684b47f0c0 con 0x55684b46f350 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced9621640 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b47fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.896+0000 7fced9e22640 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b47eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.897+0000 7fced8e20640 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b47e5f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009850 tx=0x7fcec40053a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75581e8c031c3914 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.897+0000 7fcebcff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.897+0000 7fcebcff9640 1 -- 172.21.15.7:0/3671238785 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4058060 con 0x55684b46f350 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.897+0000 7fcede478740 1 -- 172.21.15.7:0/3671238785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcec402cd00 con 0x55684b46f350 2024-03-26T08:45:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.897+0000 7fcebcff9640 1 -- 172.21.15.7:0/3671238785 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcec4003a00 con 0x55684b46f350 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.897+0000 7fcebcff9640 1 -- 172.21.15.7:0/3671238785 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcec4045030 con 0x55684b46f350 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.897+0000 7fcede478740 10 client.26002 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.897+0000 7fcede478740 1 -- 172.21.15.7:0/3671238785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55684b4f26f0 con 0x55684b46f350 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.897+0000 7fcebcff9640 1 -- 172.21.15.7:0/3671238785 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcec4005600 con 0x55684b46f350 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.898+0000 7fcebcff9640 1 -- 172.21.15.7:0/3671238785 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcec405d070 con 0x55684b46f350 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.898+0000 7fcede478740 10 client.26002 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.898+0000 7fcede478740 10 client.26002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.898+0000 7fcede478740 20 client.26002 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.898+0000 7fcede478740 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55684b4f2b90 0x55684b4f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.898+0000 7fcede478740 4 client.26002 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.898+0000 7fcede478740 1 -- 172.21.15.7:0/3671238785 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7fcec402cd00 con 0x55684b4f2b90 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.898+0000 7fced9621640 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55684b4f2b90 0x55684b4f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.899+0000 7fcebcff9640 10 client.26002 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.899+0000 7fced9621640 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55684b4f2b90 0x55684b4f4f80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42401c7ff51ec5e8 server_cookie=545bc07d560739e8 in_seq=0 out_seq=0 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcebcff9640 1 -- 172.21.15.7:0/3671238785 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7fcec402cd00 con 0x55684b4f2b90 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcebcff9640 10 client.26002 handle_command_reply: tid=0 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 client.26002 shutdown 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 -- 172.21.15.7:0/3671238785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 msgr2=0x55684b47e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b47e5f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009850 tx=0x7fcec40053a0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 -- 172.21.15.7:0/3671238785 shutdown_connections 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55684b4f2b90 0x55684b4f4f80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55684b4744c0 0x55684b47eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55684b46f350 0x55684b47e5f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcebcff9640 0 client.26002 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 --2- 172.21.15.7:0/3671238785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55684b46e960 0x55684b47fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 -- 172.21.15.7:0/3671238785 >> 172.21.15.7:0/3671238785 conn(0x55684b4371a0 msgr2=0x55684b46d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 -- 172.21.15.7:0/3671238785 shutdown_connections 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 1 -- 172.21.15.7:0/3671238785 wait complete. 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.901+0000 7fcede478740 20 client.26002 trim_cache size 0 max 16384 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "59430051-0344-4069-bc15-e81d214dfce6", 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:45:00.896 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:45:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 -- 172.21.15.7:0/3777917203 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcec00a2950 msgr2=0x7fcec00a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcec00a2950 0x7fcec00a4e10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fced0009460 tx=0x7fced00099c0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 -- 172.21.15.7:0/3777917203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4109890 msgr2=0x7fced41a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4109890 0x7fced41a5380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00eb10 tx=0x7fcecc00c2d0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 -- 172.21.15.7:0/3777917203 shutdown_connections 2024-03-26T08:45:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcec00a2950 0x7fcec00a4e10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fced4109890 0x7fced41a5380 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fced4108ed0 0x7fced419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 --2- 172.21.15.7:0/3777917203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fced4107ce0 0x7fced419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 -- 172.21.15.7:0/3777917203 >> 172.21.15.7:0/3777917203 conn(0x7fced4076c90 msgr2=0x7fced410c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 -- 172.21.15.7:0/3777917203 shutdown_connections 2024-03-26T08:45:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:00.902+0000 7fcedb885640 1 -- 172.21.15.7:0/3777917203 wait complete. 2024-03-26T08:45:00.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:45:00.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:45:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 150.0 s, 3867.8 tps, lat 25.840 ms stddev 31.283 2024-03-26T08:45:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.092+0000 7f860c8d0640 1 -- 172.21.15.7:0/2679241977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810e830 msgr2=0x7f8608110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.092+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2679241977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810e830 0x7f8608110cf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8600009ca0 tx=0x7f860002c100 comp rx=0 tx=0).stop 2024-03-26T08:45:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.093+0000 7f860c8d0640 1 -- 172.21.15.7:0/2679241977 shutdown_connections 2024-03-26T08:45:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.093+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2679241977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810e830 0x7f8608110cf0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.093+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2679241977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f860810bd90 0x7f860810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.093+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2679241977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8608073490 0x7f860810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.093+0000 7f860c8d0640 1 -- 172.21.15.7:0/2679241977 >> 172.21.15.7:0/2679241977 conn(0x7f860806e6a0 msgr2=0x7f8608070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.093+0000 7f860c8d0640 1 -- 172.21.15.7:0/2679241977 shutdown_connections 2024-03-26T08:45:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.093+0000 7f860c8d0640 1 -- 172.21.15.7:0/2679241977 wait complete. 2024-03-26T08:45:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.094+0000 7f860c8d0640 1 Processor -- start 2024-03-26T08:45:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.094+0000 7f860c8d0640 1 -- start start 2024-03-26T08:45:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f860c8d0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8608073490 0x7f86081a0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f860c8d0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810bd90 0x7f86081a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f860c8d0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860810e830 0x7f86081a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f860c8d0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081148b0 con 0x7f860810bd90 2024-03-26T08:45:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f860c8d0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8608114730 con 0x7f860810e830 2024-03-26T08:45:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f860c8d0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8608114a30 con 0x7f8608073490 2024-03-26T08:45:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f860659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810bd90 0x7f86081a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f8606d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8608073490 0x7f86081a0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f860759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860810e830 0x7f86081a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.095+0000 7f860659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810bd90 0x7f86081a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50522/0 (socket says 172.21.15.7:50522) 2024-03-26T08:45:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.096+0000 7f860759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860810e830 0x7f86081a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35170/0 (socket says 172.21.15.7:35170) 2024-03-26T08:45:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.096+0000 7f860659c640 1 -- 172.21.15.7:0/2153339491 learned_addr learned my addr 172.21.15.7:0/2153339491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.096+0000 7f860759e640 1 -- 172.21.15.7:0/2153339491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8608073490 msgr2=0x7f86081a0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.096+0000 7f860759e640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8608073490 0x7f86081a0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.096+0000 7f860759e640 1 -- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810bd90 msgr2=0x7f86081a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.096+0000 7f860759e640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810bd90 0x7f86081a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.096+0000 7f860759e640 1 -- 172.21.15.7:0/2153339491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86081a7c30 con 0x7f860810e830 2024-03-26T08:45:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.096+0000 7f860659c640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810bd90 0x7f86081a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.096+0000 7f860759e640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860810e830 0x7f86081a75c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8600037040 tx=0x7f8600035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.097+0000 7f85effff640 1 -- 172.21.15.7:0/2153339491 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600039040 con 0x7f860810e830 2024-03-26T08:45:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.097+0000 7f85effff640 1 -- 172.21.15.7:0/2153339491 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8600044070 con 0x7f860810e830 2024-03-26T08:45:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.097+0000 7f85effff640 1 -- 172.21.15.7:0/2153339491 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86000043c0 con 0x7f860810e830 2024-03-26T08:45:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.099+0000 7f860c8d0640 1 -- 172.21.15.7:0/2153339491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86081a7ec0 con 0x7f860810e830 2024-03-26T08:45:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.099+0000 7f860c8d0640 1 -- 172.21.15.7:0/2153339491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86081a8350 con 0x7f860810e830 2024-03-26T08:45:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.102+0000 7f860fd97740 1 Processor -- start 2024-03-26T08:45:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.102+0000 7f860fd97740 1 -- start start 2024-03-26T08:45:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860fd97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860fd97740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b952c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860fd97740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b952ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860fd97740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b948c3d0 con 0x5612b952cae0 2024-03-26T08:45:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860fd97740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b9434ad0 con 0x5612b9527970 2024-03-26T08:45:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860fd97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b9488470 con 0x5612b9526f80 2024-03-26T08:45:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f8606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b952c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95273a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34096/0 (socket says 172.21.15.7:34096) 2024-03-26T08:45:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f8606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b952c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35174/0 (socket says 172.21.15.7:35174) 2024-03-26T08:45:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860659c640 1 -- 172.21.15.7:0/2008034288 learned_addr learned my addr 172.21.15.7:0/2008034288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860659c640 1 --2- 172.21.15.7:0/2008034288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b952ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860659c640 1 -- 172.21.15.7:0/2008034288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 msgr2=0x5612b952c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860659c640 1 --2- 172.21.15.7:0/2008034288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b952c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860659c640 1 -- 172.21.15.7:0/2008034288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 msgr2=0x5612b952ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860659c640 1 --2- 172.21.15.7:0/2008034288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b952ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.103+0000 7f860659c640 1 -- 172.21.15.7:0/2008034288 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612b926a7d0 con 0x5612b9526f80 2024-03-26T08:45:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.104+0000 7f8606d9d640 1 --2- 172.21.15.7:0/2008034288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b952c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.104+0000 7f860659c640 1 --2- 172.21.15.7:0/2008034288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95273a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005c50 tx=0x7f85fc00bcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398987ac822f3bae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.106+0000 7f85effff640 1 -- 172.21.15.7:0/2153339491 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8600004560 con 0x7f860810e830 2024-03-26T08:45:01.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.106+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2008034288 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc013020 con 0x5612b9526f80 2024-03-26T08:45:01.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.106+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2008034288 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85fc00de70 con 0x5612b9526f80 2024-03-26T08:45:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f85effff640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85f00a29d0 0x7f85f00a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f85ed7fa640 1 -- 172.21.15.7:0/2008034288 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc00e350 con 0x5612b9526f80 2024-03-26T08:45:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f85effff640 1 -- 172.21.15.7:0/2153339491 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f86000f0aa0 con 0x7f860810e830 2024-03-26T08:45:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f8606d9d640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85f00a29d0 0x7f85f00a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f860fd97740 1 -- 172.21.15.7:0/2008034288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 msgr2=0x5612b95273a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f860fd97740 1 --2- 172.21.15.7:0/2008034288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95273a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f85fc005c50 tx=0x7f85fc00bcc0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f860fd97740 1 -- 172.21.15.7:0/2008034288 shutdown_connections 2024-03-26T08:45:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f860fd97740 1 --2- 172.21.15.7:0/2008034288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b952ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f860fd97740 1 --2- 172.21.15.7:0/2008034288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b952c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f860fd97740 1 --2- 172.21.15.7:0/2008034288 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95273a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.107+0000 7f860fd97740 1 -- 172.21.15.7:0/2008034288 >> 172.21.15.7:0/2008034288 conn(0x5612b95224f0 msgr2=0x5612b95228f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:01.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.108+0000 7f8606d9d640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85f00a29d0 0x7f85f00a4e90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5612b9527d70 tx=0x7f85f8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:01.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.109+0000 7f860fd97740 1 -- 172.21.15.7:0/2008034288 shutdown_connections 2024-03-26T08:45:01.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.109+0000 7f860fd97740 1 -- 172.21.15.7:0/2008034288 wait complete. 2024-03-26T08:45:01.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.109+0000 7f860fd97740 1 -- start start 2024-03-26T08:45:01.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.109+0000 7f860fd97740 1 Processor -- start 2024-03-26T08:45:01.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860fd97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860fd97740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b9531a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860fd97740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b9538a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860fd97740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b952fae0 con 0x5612b952cae0 2024-03-26T08:45:01.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860fd97740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b9448060 con 0x5612b9527970 2024-03-26T08:45:01.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860fd97740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612b952fc60 con 0x5612b9526f80 2024-03-26T08:45:01.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860fd97740 10 client.0 _ll_register_callbacks cb 0x5612b942a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:01.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f8606d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b9531a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b9538a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b9531a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35186/0 (socket says 172.21.15.7:35186) 2024-03-26T08:45:01.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f8606d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95314d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34106/0 (socket says 172.21.15.7:34106) 2024-03-26T08:45:01.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860659c640 1 -- 172.21.15.7:0/3792164667 learned_addr learned my addr 172.21.15.7:0/3792164667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860759e640 1 -- 172.21.15.7:0/3792164667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 msgr2=0x5612b95314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860759e640 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860759e640 1 -- 172.21.15.7:0/3792164667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 msgr2=0x5612b9531a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860759e640 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b9531a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.110+0000 7f860759e640 1 -- 172.21.15.7:0/3792164667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612b91a03f0 con 0x5612b952cae0 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.111+0000 7f8606d9d640 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.111+0000 7f860659c640 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b9531a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.111+0000 7f860759e640 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b9538a50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8600032bc0 tx=0x7f86000bd290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3614660a1588d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.111+0000 7f85eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:45:01.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.111+0000 7f85eaff5640 1 -- 172.21.15.7:0/3792164667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86000c1320 con 0x5612b952cae0 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.111+0000 7f860fd97740 1 -- 172.21.15.7:0/3792164667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85fc00de70 con 0x5612b952cae0 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.111+0000 7f85eaff5640 1 -- 172.21.15.7:0/3792164667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f86000d6190 con 0x5612b952cae0 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.112+0000 7f85eaff5640 1 -- 172.21.15.7:0/3792164667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86000bdb00 con 0x5612b952cae0 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.112+0000 7f860fd97740 10 client.16152 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.112+0000 7f860fd97740 1 -- 172.21.15.7:0/3792164667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5612b91600f0 con 0x5612b952cae0 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.112+0000 7f85eaff5640 1 -- 172.21.15.7:0/3792164667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85fc00de70 con 0x5612b952cae0 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.112+0000 7f85eaff5640 1 -- 172.21.15.7:0/3792164667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f860003c030 con 0x5612b952cae0 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.113+0000 7f860fd97740 10 client.16152 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.113+0000 7f860fd97740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.113+0000 7f860fd97740 20 client.16152 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.113+0000 7f860fd97740 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5612b953cfa0 0x5612b953f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.113+0000 7f860fd97740 4 client.16152 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.113+0000 7f860fd97740 1 -- 172.21.15.7:0/3792164667 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612b9499910 con 0x5612b953cfa0 2024-03-26T08:45:01.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.113+0000 7f8606d9d640 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5612b953cfa0 0x5612b953f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.113+0000 7f85eaff5640 10 client.16152 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.114+0000 7f8606d9d640 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5612b953cfa0 0x5612b953f3c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d1c870a839713 server_cookie=91fe338647db19ad in_seq=0 out_seq=0 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f85eaff5640 1 -- 172.21.15.7:0/3792164667 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5612b9499910 con 0x5612b953cfa0 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f85eaff5640 10 client.16152 handle_command_reply: tid=0 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 client.16152 shutdown 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 -- 172.21.15.7:0/3792164667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 msgr2=0x5612b9538a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b9538a50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8600032bc0 tx=0x7f86000bd290 comp rx=0 tx=0).stop 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 -- 172.21.15.7:0/3792164667 shutdown_connections 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5612b953cfa0 0x5612b953f3c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b9538a50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9527970 0x5612b9531a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 --2- 172.21.15.7:0/3792164667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 -- 172.21.15.7:0/3792164667 >> 172.21.15.7:0/3792164667 conn(0x5612b95224f0 msgr2=0x5612b95217b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:01.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f85eaff5640 0 client.16152 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:01.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 -- 172.21.15.7:0/3792164667 shutdown_connections 2024-03-26T08:45:01.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 1 -- 172.21.15.7:0/3792164667 wait complete. 2024-03-26T08:45:01.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.115+0000 7f860fd97740 20 client.16152 trim_cache size 0 max 16384 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.135+0000 7f860fd97740 1 Processor -- start 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.135+0000 7f860fd97740 1 -- start start 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.135+0000 7f860fd97740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9526f80 0x5612b9530080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.135+0000 7f860fd97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9527970 0x5612b9538a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.135+0000 7f860fd97740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b95314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f8606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9526f80 0x5612b9530080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9527970 0x5612b9538a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f8606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9526f80 0x5612b9530080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35196/0 (socket says 172.21.15.7:35196) 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f8606d9d640 1 -- 172.21.15.7:0/1532223890 learned_addr learned my addr 172.21.15.7:0/1532223890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860fd97740 1 -- 172.21.15.7:0/1532223890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081b1460 con 0x5612b952cae0 2024-03-26T08:45:01.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860fd97740 1 -- 172.21.15.7:0/1532223890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081aef60 con 0x5612b9526f80 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860fd97740 1 -- 172.21.15.7:0/1532223890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081acfe0 con 0x5612b9527970 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860759e640 1 --2- 172.21.15.7:0/1532223890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b95314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860659c640 1 -- 172.21.15.7:0/1532223890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9526f80 msgr2=0x5612b9530080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860659c640 1 --2- 172.21.15.7:0/1532223890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9526f80 0x5612b9530080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860659c640 1 -- 172.21.15.7:0/1532223890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 msgr2=0x5612b95314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860659c640 1 --2- 172.21.15.7:0/1532223890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b95314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860659c640 1 -- 172.21.15.7:0/1532223890 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612b9523260 con 0x5612b9527970 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f8606d9d640 1 --2- 172.21.15.7:0/1532223890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9526f80 0x5612b9530080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f860659c640 1 --2- 172.21.15.7:0/1532223890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9527970 0x5612b9538a10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00db40 tx=0x7f85fc00bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f397a07980f65b9f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f85ecff9640 1 -- 172.21.15.7:0/1532223890 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc013030 con 0x5612b9527970 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f85ecff9640 1 -- 172.21.15.7:0/1532223890 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85fc00dc70 con 0x5612b9527970 2024-03-26T08:45:01.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.136+0000 7f85ecff9640 1 -- 172.21.15.7:0/1532223890 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0052a0 con 0x5612b9527970 2024-03-26T08:45:01.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.137+0000 7f860fd97740 1 -- 172.21.15.7:0/1532223890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9527970 msgr2=0x5612b9538a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:01.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.137+0000 7f860fd97740 1 --2- 172.21.15.7:0/1532223890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9527970 0x5612b9538a10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00db40 tx=0x7f85fc00bf00 comp rx=0 tx=0).stop 2024-03-26T08:45:01.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.137+0000 7f860fd97740 1 -- 172.21.15.7:0/1532223890 shutdown_connections 2024-03-26T08:45:01.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.137+0000 7f860fd97740 1 --2- 172.21.15.7:0/1532223890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b952cae0 0x5612b95314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.137+0000 7f860fd97740 1 --2- 172.21.15.7:0/1532223890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9527970 0x5612b9538a10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.137+0000 7f860fd97740 1 --2- 172.21.15.7:0/1532223890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b9526f80 0x5612b9530080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.137+0000 7f860fd97740 1 -- 172.21.15.7:0/1532223890 >> 172.21.15.7:0/1532223890 conn(0x5612b95224f0 msgr2=0x5612b9530af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.137+0000 7f860fd97740 1 -- 172.21.15.7:0/1532223890 shutdown_connections 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.137+0000 7f860fd97740 1 -- 172.21.15.7:0/1532223890 wait complete. 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860fd97740 1 -- start start 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860fd97740 1 Processor -- start 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860fd97740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860fd97740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b9527970 0x5612b9535c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860fd97740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b952cae0 0x5612b9596840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860fd97740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081ad5e0 con 0x5612b9527970 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860fd97740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081ae900 con 0x5612b952cae0 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f8606d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f8606d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34126/0 (socket says 172.21.15.7:34126) 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f8606d9d640 1 -- 172.21.15.7:0/1073532749 learned_addr learned my addr 172.21.15.7:0/1073532749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:01.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860659c640 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b9527970 0x5612b9535c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860fd97740 1 -- 172.21.15.7:0/1073532749 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081adc40 con 0x5612b9526f80 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860759e640 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b952cae0 0x5612b9596840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.138+0000 7f860fd97740 10 client.0 _ll_register_callbacks cb 0x7f8600004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.139+0000 7f860659c640 1 -- 172.21.15.7:0/1073532749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 msgr2=0x5612b95356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.139+0000 7f860659c640 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.139+0000 7f860659c640 1 -- 172.21.15.7:0/1073532749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b952cae0 msgr2=0x5612b9596840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.139+0000 7f860659c640 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b952cae0 0x5612b9596840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.139+0000 7f860659c640 1 -- 172.21.15.7:0/1073532749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612b9536230 con 0x5612b9527970 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.139+0000 7f860759e640 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b952cae0 0x5612b9596840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.140+0000 7f860659c640 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b9527970 0x5612b9535c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5612b95170e0 tx=0x7f85fc024a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4971bfc7f00619bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.140+0000 7f85eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.141+0000 7f85eaff5640 1 -- 172.21.15.7:0/1073532749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc013040 con 0x5612b9527970 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.141+0000 7f85eaff5640 1 -- 172.21.15.7:0/1073532749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85fc014430 con 0x5612b9527970 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.141+0000 7f860fd97740 1 -- 172.21.15.7:0/1073532749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85fc00dc70 con 0x5612b9527970 2024-03-26T08:45:01.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.141+0000 7f85eaff5640 1 -- 172.21.15.7:0/1073532749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85fc0050e0 con 0x5612b9527970 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.141+0000 7f860fd97740 10 client.16158 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.141+0000 7f860fd97740 1 -- 172.21.15.7:0/1073532749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5612b9597de0 con 0x5612b9527970 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.141+0000 7f85eaff5640 1 -- 172.21.15.7:0/1073532749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85fc00dc70 con 0x5612b9527970 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.142+0000 7f85eaff5640 1 -- 172.21.15.7:0/1073532749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f85fc02d070 con 0x5612b9527970 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.142+0000 7f860fd97740 10 client.16158 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.142+0000 7f860fd97740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.142+0000 7f860fd97740 20 client.16158 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.142+0000 7f860fd97740 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5612b9598470 0x5612b959a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.142+0000 7f860fd97740 4 client.16158 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.142+0000 7f860fd97740 1 -- 172.21.15.7:0/1073532749 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5612b9493c60 con 0x5612b9598470 2024-03-26T08:45:01.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.142+0000 7f8606d9d640 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5612b9598470 0x5612b959a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:01.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.145+0000 7f8606d9d640 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5612b9598470 0x5612b959a890 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe446fb2052051f server_cookie=51108d32670b01d9 in_seq=0 out_seq=0 2024-03-26T08:45:01.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.145+0000 7f85eaff5640 10 client.16158 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:01.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.146+0000 7f85eaff5640 1 -- 172.21.15.7:0/1073532749 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5612b9493c60 con 0x5612b9598470 2024-03-26T08:45:01.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.146+0000 7f85eaff5640 10 client.16158 handle_command_reply: tid=0 2024-03-26T08:45:01.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.146+0000 7f860fd97740 1 client.16158 shutdown 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.146+0000 7f860fd97740 1 -- 172.21.15.7:0/1073532749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b9527970 msgr2=0x5612b9535c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b9527970 0x5612b9535c20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5612b95170e0 tx=0x7f85fc024a60 comp rx=0 tx=0).stop 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 1 -- 172.21.15.7:0/1073532749 shutdown_connections 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5612b9598470 0x5612b959a890 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5612b952cae0 0x5612b9596840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f85eaff5640 0 client.16158 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5612b9527970 0x5612b9535c20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 1 --2- 172.21.15.7:0/1073532749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5612b9526f80 0x5612b95356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 1 -- 172.21.15.7:0/1073532749 >> 172.21.15.7:0/1073532749 conn(0x5612b95224f0 msgr2=0x5612b95314d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 1 -- 172.21.15.7:0/1073532749 shutdown_connections 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 1 -- 172.21.15.7:0/1073532749 wait complete. 2024-03-26T08:45:01.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860fd97740 20 client.16158 trim_cache size 0 max 16384 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "59430051-0344-4069-bc15-e81d214dfce6": { 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "59430051-0344-4069-bc15-e81d214dfce6", 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:45:01.140 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:45:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 -- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85f00a29d0 msgr2=0x7f85f00a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85f00a29d0 0x7f85f00a4e90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5612b9527d70 tx=0x7f85f8009110 comp rx=0 tx=0).stop 2024-03-26T08:45:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 -- 172.21.15.7:0/2153339491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860810e830 msgr2=0x7f86081a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860810e830 0x7f86081a75c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8600037040 tx=0x7f8600035d10 comp rx=0 tx=0).stop 2024-03-26T08:45:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 -- 172.21.15.7:0/2153339491 shutdown_connections 2024-03-26T08:45:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85f00a29d0 0x7f85f00a4e90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860810e830 0x7f86081a75c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f860810bd90 0x7f86081a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 --2- 172.21.15.7:0/2153339491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8608073490 0x7f86081a0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.147+0000 7f860c8d0640 1 -- 172.21.15.7:0/2153339491 >> 172.21.15.7:0/2153339491 conn(0x7f860806e6a0 msgr2=0x7f8608108bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.148+0000 7f860c8d0640 1 -- 172.21.15.7:0/2153339491 shutdown_connections 2024-03-26T08:45:01.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:01.148+0000 7f860c8d0640 1 -- 172.21.15.7:0/2153339491 wait complete. 2024-03-26T08:45:01.150 INFO:tasks.cephfs.filesystem:scrub status for tag:59430051-0344-4069-bc15-e81d214dfce6 - {'path': '/', 'tag': '59430051-0344-4069-bc15-e81d214dfce6', 'options': 'recursive,force'} 2024-03-26T08:45:01.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:01 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:45:01.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:01 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:01.292 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:01 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:45:01.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:01 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:45:01.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:01 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:01.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:01 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:45:01.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:01 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:45:01.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:01 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:01.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:01 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:45:02.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:02 smithi016 ceph-mon[35851]: pgmap v438: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 912 op/s 2024-03-26T08:45:02.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:02 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:45:02.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:02 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:02.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:02 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:02.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:02 smithi007 ceph-mon[32462]: pgmap v438: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 912 op/s 2024-03-26T08:45:02.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:02 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:45:02.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:02 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:02.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:02 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:02.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:02 smithi044 ceph-mon[36461]: pgmap v438: 97 pgs: 97 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 33 MiB/s wr, 912 op/s 2024-03-26T08:45:02.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:02 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:45:02.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:02 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:02.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:02 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:04.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:04 smithi016 ceph-mon[35851]: pgmap v439: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 40 MiB/s wr, 957 op/s 2024-03-26T08:45:04.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:04 smithi007 ceph-mon[32462]: pgmap v439: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 40 MiB/s wr, 957 op/s 2024-03-26T08:45:04.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:04 smithi044 ceph-mon[36461]: pgmap v439: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 40 MiB/s wr, 957 op/s 2024-03-26T08:45:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 155.0 s, 4196.0 tps, lat 23.835 ms stddev 28.952 2024-03-26T08:45:06.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:06 smithi016 ceph-mon[35851]: pgmap v440: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 51 MiB/s wr, 1.04k op/s 2024-03-26T08:45:06.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:06 smithi007 ceph-mon[32462]: pgmap v440: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 51 MiB/s wr, 1.04k op/s 2024-03-26T08:45:06.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:06 smithi044 ceph-mon[36461]: pgmap v440: 97 pgs: 97 active+clean; 1.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 51 MiB/s wr, 1.04k op/s 2024-03-26T08:45:07.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:07 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:45:07.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:07 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:45:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.535+0000 7f7ba5df4640 1 -- 172.21.15.7:0/1364767035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba00772a0 msgr2=0x7f7ba0077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.535+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/1364767035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba00772a0 0x7f7ba0077700 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b88009900 tx=0x7f7b8802bc30 comp rx=0 tx=0).stop 2024-03-26T08:45:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.535+0000 7f7ba5df4640 1 -- 172.21.15.7:0/1364767035 shutdown_connections 2024-03-26T08:45:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.535+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/1364767035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba0077c40 0x7f7ba00784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.535+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/1364767035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba00772a0 0x7f7ba0077700 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.535+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/1364767035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba0076060 0x7f7ba0076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.535+0000 7f7ba5df4640 1 -- 172.21.15.7:0/1364767035 >> 172.21.15.7:0/1364767035 conn(0x7f7ba00fee90 msgr2=0x7f7ba01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.535+0000 7f7ba5df4640 1 -- 172.21.15.7:0/1364767035 shutdown_connections 2024-03-26T08:45:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.535+0000 7f7ba5df4640 1 -- 172.21.15.7:0/1364767035 wait complete. 2024-03-26T08:45:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.536+0000 7f7ba5df4640 1 Processor -- start 2024-03-26T08:45:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.536+0000 7f7ba5df4640 1 -- start start 2024-03-26T08:45:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.536+0000 7f7ba5df4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0076060 0x7f7ba019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.536+0000 7f7ba5df4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba00772a0 0x7f7ba019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.536+0000 7f7ba5df4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba0077c40 0x7f7ba01a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.536+0000 7f7ba5df4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0113a30 con 0x7f7ba00772a0 2024-03-26T08:45:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.536+0000 7f7ba5df4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba01138b0 con 0x7f7ba0076060 2024-03-26T08:45:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.536+0000 7f7ba5df4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0113bb0 con 0x7f7ba0077c40 2024-03-26T08:45:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba00772a0 0x7f7ba019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0076060 0x7f7ba019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba0077c40 0x7f7ba01a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0076060 0x7f7ba019e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40850/0 (socket says 172.21.15.7:40850) 2024-03-26T08:45:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9f7fe640 1 -- 172.21.15.7:0/3631662317 learned_addr learned my addr 172.21.15.7:0/3631662317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba0077c40 0x7f7ba01a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:52900/0 (socket says 172.21.15.7:52900) 2024-03-26T08:45:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9effd640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba00772a0 0x7f7ba019e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48938/0 (socket says 172.21.15.7:48938) 2024-03-26T08:45:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9effd640 1 -- 172.21.15.7:0/3631662317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba0077c40 msgr2=0x7f7ba01a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9effd640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba0077c40 0x7f7ba01a55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.537+0000 7f7b9effd640 1 -- 172.21.15.7:0/3631662317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0076060 msgr2=0x7f7ba019e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.538+0000 7f7b9effd640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0076060 0x7f7ba019e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.538+0000 7f7b9effd640 1 -- 172.21.15.7:0/3631662317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba01a5c60 con 0x7f7ba00772a0 2024-03-26T08:45:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.538+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0076060 0x7f7ba019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.538+0000 7f7b9effd640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba00772a0 0x7f7ba019e580 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7b88036040 tx=0x7f7b88005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.539+0000 7f7b9cff9640 1 -- 172.21.15.7:0/3631662317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88043020 con 0x7f7ba00772a0 2024-03-26T08:45:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.539+0000 7f7ba5df4640 1 -- 172.21.15.7:0/3631662317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba01a5ef0 con 0x7f7ba00772a0 2024-03-26T08:45:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.539+0000 7f7ba5df4640 1 -- 172.21.15.7:0/3631662317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba01a6400 con 0x7f7ba00772a0 2024-03-26T08:45:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.539+0000 7f7b9cff9640 1 -- 172.21.15.7:0/3631662317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b88004410 con 0x7f7ba00772a0 2024-03-26T08:45:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.539+0000 7f7b9cff9640 1 -- 172.21.15.7:0/3631662317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88043020 con 0x7f7ba00772a0 2024-03-26T08:45:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.542+0000 7f7ba89e4740 1 Processor -- start 2024-03-26T08:45:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.542+0000 7f7ba89e4740 1 -- start start 2024-03-26T08:45:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024bce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7ba89e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30241cb10 con 0x55b3024b7920 2024-03-26T08:45:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7ba89e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3023c5900 con 0x55b3024b8310 2024-03-26T08:45:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7ba89e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b302418bb0 con 0x55b3024bd480 2024-03-26T08:45:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9cff9640 1 -- 172.21.15.7:0/3631662317 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7b88046020 con 0x7f7ba00772a0 2024-03-26T08:45:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024b7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48950/0 (socket says 172.21.15.7:48950) 2024-03-26T08:45:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9ffff640 1 -- 172.21.15.7:0/2935087718 learned_addr learned my addr 172.21.15.7:0/2935087718 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9f7fe640 1 -- 172.21.15.7:0/2935087718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 msgr2=0x55b3024bf910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:45:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/2935087718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024bf910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9f7fe640 1 -- 172.21.15.7:0/2935087718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 msgr2=0x55b3024b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/2935087718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9f7fe640 1 -- 172.21.15.7:0/2935087718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b302131210 con 0x55b3024b8310 2024-03-26T08:45:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/2935087718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024bce10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b980117b0 tx=0x7f7b98011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8bafff19741426 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.543+0000 7f7b9cff9640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b8c0a29f0 0x7f7b8c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.544+0000 7f7b95ffb640 1 -- 172.21.15.7:0/2935087718 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98003b70 con 0x55b3024b8310 2024-03-26T08:45:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.544+0000 7f7b95ffb640 1 -- 172.21.15.7:0/2935087718 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b9800fe20 con 0x55b3024b8310 2024-03-26T08:45:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.544+0000 7f7b95ffb640 1 -- 172.21.15.7:0/2935087718 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98004820 con 0x55b3024b8310 2024-03-26T08:45:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.544+0000 7f7b9ffff640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b8c0a29f0 0x7f7b8c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.544+0000 7f7b9cff9640 1 -- 172.21.15.7:0/3631662317 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7b880f06a0 con 0x7f7ba00772a0 2024-03-26T08:45:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.544+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2935087718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 msgr2=0x55b3024bce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.545+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2935087718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024bce10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b980117b0 tx=0x7f7b98011c70 comp rx=0 tx=0).stop 2024-03-26T08:45:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.545+0000 7f7b9ffff640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b8c0a29f0 0x7f7b8c0a4eb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55b3024b8b70 tx=0x7f7b900090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.545+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2935087718 shutdown_connections 2024-03-26T08:45:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.545+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2935087718 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024bf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.545+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2935087718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024bce10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.545+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2935087718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.545+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2935087718 >> 172.21.15.7:0/2935087718 conn(0x55b3024b2e90 msgr2=0x55b3024b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.545+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2935087718 shutdown_connections 2024-03-26T08:45:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.546+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2935087718 wait complete. 2024-03-26T08:45:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.546+0000 7f7ba89e4740 1 -- start start 2024-03-26T08:45:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.546+0000 7f7ba89e4740 1 Processor -- start 2024-03-26T08:45:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7ba89e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3024c0430 con 0x55b3024b7920 2024-03-26T08:45:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7b9f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7ba89e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3023d8d50 con 0x55b3024b8310 2024-03-26T08:45:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7ba89e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3024c05b0 con 0x55b3024bd480 2024-03-26T08:45:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7b9f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024c1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48952/0 (socket says 172.21.15.7:48952) 2024-03-26T08:45:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7b9f7fe640 1 -- 172.21.15.7:0/3881906422 learned_addr learned my addr 172.21.15.7:0/3881906422 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7ba89e4740 10 client.0 _ll_register_callbacks cb 0x55b3023d8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7b9effd640 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7b9ffff640 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7b9effd640 1 -- 172.21.15.7:0/3881906422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 msgr2=0x55b3024c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7b9effd640 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7b9effd640 1 -- 172.21.15.7:0/3881906422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 msgr2=0x55b3024c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.547+0000 7f7b9effd640 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024c1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7b9effd640 1 -- 172.21.15.7:0/3881906422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3021fd620 con 0x55b3024b8310 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7b9effd640 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c21b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7b880ee1a0 tx=0x7f7b880bc9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb7630c42b2cbc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7b7effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7b7effd640 1 -- 172.21.15.7:0/3881906422 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b880c1190 con 0x55b3024b8310 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7b7effd640 1 -- 172.21.15.7:0/3881906422 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7b88003be0 con 0x55b3024b8310 2024-03-26T08:45:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7ba89e4740 1 -- 172.21.15.7:0/3881906422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b9800fe20 con 0x55b3024b8310 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7b7effd640 1 -- 172.21.15.7:0/3881906422 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b880c57e0 con 0x55b3024b8310 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7ba89e4740 10 client.26009 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7ba89e4740 1 -- 172.21.15.7:0/3881906422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b3024cc560 con 0x55b3024b8310 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7b7effd640 1 -- 172.21.15.7:0/3881906422 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b9800fe20 con 0x55b3024b8310 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7b7effd640 1 -- 172.21.15.7:0/3881906422 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7b880b7bb0 con 0x55b3024b8310 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7ba89e4740 10 client.26009 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7ba89e4740 10 client.26009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.548+0000 7f7ba89e4740 20 client.26009 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.549+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b3024cdd70 0x55b3024d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.549+0000 7f7ba89e4740 4 client.26009 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.549+0000 7f7ba89e4740 1 -- 172.21.15.7:0/3881906422 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b30242a050 con 0x55b3024cdd70 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.549+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b3024cdd70 0x55b3024d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.550+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b3024cdd70 0x55b3024d0190 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c491289119bb4708 server_cookie=f3af34829887c151 in_seq=0 out_seq=0 2024-03-26T08:45:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.550+0000 7f7b7effd640 10 client.26009 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:07.556 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:07 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:45:07.556 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:07 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.550+0000 7f7b7effd640 1 -- 172.21.15.7:0/3881906422 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b30242a050 con 0x55b3024cdd70 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.550+0000 7f7b7effd640 10 client.26009 handle_command_reply: tid=0 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 client.26009 shutdown 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 -- 172.21.15.7:0/3881906422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 msgr2=0x55b3024c21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c21b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7b880ee1a0 tx=0x7f7b880bc9a0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 -- 172.21.15.7:0/3881906422 shutdown_connections 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b3024cdd70 0x55b3024d0190 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7b7effd640 0 client.26009 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c21b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/3881906422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b7920 0x55b3024c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 -- 172.21.15.7:0/3881906422 >> 172.21.15.7:0/3881906422 conn(0x55b3024b2e90 msgr2=0x55b3024b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 -- 172.21.15.7:0/3881906422 shutdown_connections 2024-03-26T08:45:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 1 -- 172.21.15.7:0/3881906422 wait complete. 2024-03-26T08:45:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.551+0000 7f7ba89e4740 20 client.26009 trim_cache size 0 max 16384 2024-03-26T08:45:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7ba89e4740 1 Processor -- start 2024-03-26T08:45:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7ba89e4740 1 -- start start 2024-03-26T08:45:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b7920 0x55b3024c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b8310 0x55b3024c9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7ba89e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80010870 con 0x55b3024b8310 2024-03-26T08:45:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7ba89e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8000e370 con 0x55b3024b7920 2024-03-26T08:45:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7ba89e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8000c3f0 con 0x55b3024bd480 2024-03-26T08:45:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7b9effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b8310 0x55b3024c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7b9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b7920 0x55b3024c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7b9ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7b9effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b8310 0x55b3024c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48958/0 (socket says 172.21.15.7:48958) 2024-03-26T08:45:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.572+0000 7f7b9effd640 1 -- 172.21.15.7:0/4183924159 learned_addr learned my addr 172.21.15.7:0/4183924159 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.573+0000 7f7b9ffff640 1 -- 172.21.15.7:0/4183924159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b7920 msgr2=0x55b3024c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.573+0000 7f7b9ffff640 1 --2- 172.21.15.7:0/4183924159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b7920 0x55b3024c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.573+0000 7f7b9ffff640 1 -- 172.21.15.7:0/4183924159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b8310 msgr2=0x55b3024c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.573+0000 7f7b9ffff640 1 --2- 172.21.15.7:0/4183924159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b8310 0x55b3024c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.573+0000 7f7b9ffff640 1 -- 172.21.15.7:0/4183924159 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3024b3c00 con 0x55b3024bd480 2024-03-26T08:45:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.574+0000 7f7b9ffff640 1 --2- 172.21.15.7:0/4183924159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024b9630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b90006a30 tx=0x7f7b90014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f45e4a76e3682b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:07.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:07 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:45:07.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:07 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:45:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.574+0000 7f7b957fa640 1 -- 172.21.15.7:0/4183924159 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90027030 con 0x55b3024bd480 2024-03-26T08:45:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.574+0000 7f7b957fa640 1 -- 172.21.15.7:0/4183924159 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b9001b070 con 0x55b3024bd480 2024-03-26T08:45:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.574+0000 7f7b957fa640 1 -- 172.21.15.7:0/4183924159 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900223e0 con 0x55b3024bd480 2024-03-26T08:45:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.575+0000 7f7ba89e4740 1 -- 172.21.15.7:0/4183924159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 msgr2=0x55b3024b9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.575+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/4183924159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024b9630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b90006a30 tx=0x7f7b90014000 comp rx=0 tx=0).stop 2024-03-26T08:45:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.575+0000 7f7ba89e4740 1 -- 172.21.15.7:0/4183924159 shutdown_connections 2024-03-26T08:45:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.575+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/4183924159 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024bd480 0x55b3024b9630 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.575+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/4183924159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024b8310 0x55b3024c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.575+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/4183924159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b7920 0x55b3024c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.575+0000 7f7ba89e4740 1 -- 172.21.15.7:0/4183924159 >> 172.21.15.7:0/4183924159 conn(0x55b3024b2e90 msgr2=0x55b3024b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.575+0000 7f7ba89e4740 1 -- 172.21.15.7:0/4183924159 shutdown_connections 2024-03-26T08:45:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.575+0000 7f7ba89e4740 1 -- 172.21.15.7:0/4183924159 wait complete. 2024-03-26T08:45:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.576+0000 7f7ba89e4740 1 -- start start 2024-03-26T08:45:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.576+0000 7f7ba89e4740 1 Processor -- start 2024-03-26T08:45:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.577+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024b7920 0x55b3024c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.577+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.577+0000 7f7ba89e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024bd480 0x55b3024c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.577+0000 7f7b9f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024b7920 0x55b3024c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.577+0000 7f7b9f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024b7920 0x55b3024c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:52928/0 (socket says 172.21.15.7:52928) 2024-03-26T08:45:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.577+0000 7f7b9effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.577+0000 7f7b9effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40880/0 (socket says 172.21.15.7:40880) 2024-03-26T08:45:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7b9f7fe640 1 -- 172.21.15.7:0/2435593771 learned_addr learned my addr 172.21.15.7:0/2435593771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8000c9f0 con 0x55b3024bd480 2024-03-26T08:45:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8000dd10 con 0x55b3024b8310 2024-03-26T08:45:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8000d050 con 0x55b3024b7920 2024-03-26T08:45:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7ba89e4740 10 client.0 _ll_register_callbacks cb 0x7f7b88009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7b9ffff640 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024bd480 0x55b3024c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7b9f7fe640 1 -- 172.21.15.7:0/2435593771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 msgr2=0x55b3024c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7b9f7fe640 1 -- 172.21.15.7:0/2435593771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024bd480 msgr2=0x55b3024c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024bd480 0x55b3024c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7b9f7fe640 1 -- 172.21.15.7:0/2435593771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3024c8020 con 0x55b3024b7920 2024-03-26T08:45:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7b9f7fe640 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024b7920 0x55b3024c64e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7b980044f0 tx=0x7f7b98004130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=885a24aa8ad43031 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.578+0000 7f7b7effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:45:07.583 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.579+0000 7f7b7effd640 1 -- 172.21.15.7:0/2435593771 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98031060 con 0x55b3024b7920 2024-03-26T08:45:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.579+0000 7f7b7effd640 1 -- 172.21.15.7:0/2435593771 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7b9802b1d0 con 0x55b3024b7920 2024-03-26T08:45:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.579+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b9001b070 con 0x55b3024b7920 2024-03-26T08:45:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.580+0000 7f7b7effd640 1 -- 172.21.15.7:0/2435593771 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b9801f040 con 0x55b3024b7920 2024-03-26T08:45:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.580+0000 7f7ba89e4740 10 client.26035 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.580+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b3024c9330 con 0x55b3024b7920 2024-03-26T08:45:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.581+0000 7f7b7effd640 1 -- 172.21.15.7:0/2435593771 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b9800f940 con 0x55b3024b7920 2024-03-26T08:45:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.581+0000 7f7b7effd640 1 -- 172.21.15.7:0/2435593771 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7b98036070 con 0x55b3024b7920 2024-03-26T08:45:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.581+0000 7f7ba89e4740 10 client.26035 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.581+0000 7f7ba89e4740 10 client.26035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.581+0000 7f7ba89e4740 20 client.26035 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.581+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b3024cdd70 0x55b30252b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.581+0000 7f7ba89e4740 4 client.26035 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:45:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.581+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f7b9001b070 con 0x55b3024cdd70 2024-03-26T08:45:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.581+0000 7f7b9effd640 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b3024cdd70 0x55b30252b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.582+0000 7f7b9effd640 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b3024cdd70 0x55b30252b960 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=affff8f8dc6824b3 server_cookie=8c27fc3879a22d24 in_seq=0 out_seq=0 2024-03-26T08:45:07.586 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.582+0000 7f7b7effd640 10 client.26035 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.607+0000 7f7b7effd640 1 -- 172.21.15.7:0/2435593771 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f7b9001b070 con 0x55b3024cdd70 2024-03-26T08:45:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.608+0000 7f7b7effd640 10 client.26035 handle_command_reply: tid=0 2024-03-26T08:45:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.608+0000 7f7ba89e4740 1 client.26035 shutdown 2024-03-26T08:45:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.608+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024b7920 msgr2=0x55b3024c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:07.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.608+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024b7920 0x55b3024c64e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7b980044f0 tx=0x7f7b98004130 comp rx=0 tx=0).stop 2024-03-26T08:45:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 shutdown_connections 2024-03-26T08:45:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b3024cdd70 0x55b30252b960 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3024bd480 0x55b3024c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3024b8310 0x55b3024c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7ba89e4740 1 --2- 172.21.15.7:0/2435593771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3024b7920 0x55b3024c64e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 >> 172.21.15.7:0/2435593771 conn(0x55b3024b2e90 msgr2=0x55b3024b9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:07.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7b7effd640 0 client.26035 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 shutdown_connections 2024-03-26T08:45:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7ba89e4740 1 -- 172.21.15.7:0/2435593771 wait complete. 2024-03-26T08:45:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.609+0000 7f7ba89e4740 20 client.26035 trim_cache size 0 max 16384 2024-03-26T08:45:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:45:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:45:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:45:07.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 -- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b8c0a29f0 msgr2=0x7f7b8c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b8c0a29f0 0x7f7b8c0a4eb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55b3024b8b70 tx=0x7f7b900090e0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 -- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba00772a0 msgr2=0x7f7ba019e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba00772a0 0x7f7ba019e580 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7b88036040 tx=0x7f7b88005dd0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 -- 172.21.15.7:0/3631662317 shutdown_connections 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7b8c0a29f0 0x7f7b8c0a4eb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ba0077c40 0x7f7ba01a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ba00772a0 0x7f7ba019e580 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 --2- 172.21.15.7:0/3631662317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba0076060 0x7f7ba019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.610+0000 7f7ba5df4640 1 -- 172.21.15.7:0/3631662317 >> 172.21.15.7:0/3631662317 conn(0x7f7ba00fee90 msgr2=0x7f7ba0100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.611+0000 7f7ba5df4640 1 -- 172.21.15.7:0/3631662317 shutdown_connections 2024-03-26T08:45:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:07.612+0000 7f7ba5df4640 1 -- 172.21.15.7:0/3631662317 wait complete. 2024-03-26T08:45:08.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:08 smithi016 ceph-mon[35851]: pgmap v441: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 54 MiB/s wr, 1.08k op/s 2024-03-26T08:45:08.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:08 smithi007 ceph-mon[32462]: pgmap v441: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 54 MiB/s wr, 1.08k op/s 2024-03-26T08:45:08.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:08 smithi044 ceph-mon[36461]: pgmap v441: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 54 MiB/s wr, 1.08k op/s 2024-03-26T08:45:10.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:10 smithi016 ceph-mon[35851]: pgmap v442: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 49 MiB/s wr, 1.07k op/s 2024-03-26T08:45:10.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:10 smithi007 ceph-mon[32462]: pgmap v442: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 49 MiB/s wr, 1.07k op/s 2024-03-26T08:45:10.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:10 smithi044 ceph-mon[36461]: pgmap v442: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 49 MiB/s wr, 1.07k op/s 2024-03-26T08:45:11.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 160.0 s, 3673.2 tps, lat 27.278 ms stddev 48.648 2024-03-26T08:45:12.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:12 smithi016 ceph-mon[35851]: pgmap v443: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 47 MiB/s wr, 1.05k op/s 2024-03-26T08:45:12.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:12 smithi007 ceph-mon[32462]: pgmap v443: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 47 MiB/s wr, 1.05k op/s 2024-03-26T08:45:12.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:12 smithi044 ceph-mon[36461]: pgmap v443: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 47 MiB/s wr, 1.05k op/s 2024-03-26T08:45:14.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:14 smithi016 ceph-mon[35851]: pgmap v444: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 61 MiB/s wr, 1.44k op/s 2024-03-26T08:45:14.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:14 smithi007 ceph-mon[32462]: pgmap v444: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 61 MiB/s wr, 1.44k op/s 2024-03-26T08:45:14.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:14 smithi044 ceph-mon[36461]: pgmap v444: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 61 MiB/s wr, 1.44k op/s 2024-03-26T08:45:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 165.0 s, 4585.2 tps, lat 21.757 ms stddev 22.601 2024-03-26T08:45:16.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:16 smithi016 ceph-mon[35851]: pgmap v445: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 69 MiB/s wr, 1.69k op/s 2024-03-26T08:45:16.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:16 smithi007 ceph-mon[32462]: pgmap v445: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 69 MiB/s wr, 1.69k op/s 2024-03-26T08:45:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:16 smithi044 ceph-mon[36461]: pgmap v445: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 69 MiB/s wr, 1.69k op/s 2024-03-26T08:45:18.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:18 smithi016 ceph-mon[35851]: pgmap v446: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 59 MiB/s wr, 1.58k op/s 2024-03-26T08:45:18.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:18 smithi007 ceph-mon[32462]: pgmap v446: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 59 MiB/s wr, 1.58k op/s 2024-03-26T08:45:18.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:18 smithi044 ceph-mon[36461]: pgmap v446: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 59 MiB/s wr, 1.58k op/s 2024-03-26T08:45:20.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:20 smithi016 ceph-mon[35851]: pgmap v447: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 56 MiB/s wr, 1.57k op/s 2024-03-26T08:45:20.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:20 smithi007 ceph-mon[32462]: pgmap v447: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 56 MiB/s wr, 1.57k op/s 2024-03-26T08:45:20.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:20 smithi044 ceph-mon[36461]: pgmap v447: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 56 MiB/s wr, 1.57k op/s 2024-03-26T08:45:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.742+0000 7fcf347df640 1 -- 172.21.15.7:0/1442474476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c10ff20 msgr2=0x7fcf2c112310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.742+0000 7fcf347df640 1 --2- 172.21.15.7:0/1442474476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c10ff20 0x7fcf2c112310 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009ca0 tx=0x7fcf2402c100 comp rx=0 tx=0).stop 2024-03-26T08:45:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.742+0000 7fcf347df640 1 -- 172.21.15.7:0/1442474476 shutdown_connections 2024-03-26T08:45:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.742+0000 7fcf347df640 1 --2- 172.21.15.7:0/1442474476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c10ff20 0x7fcf2c112310 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.742+0000 7fcf347df640 1 --2- 172.21.15.7:0/1442474476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf2c109000 0x7fcf2c10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.742+0000 7fcf347df640 1 --2- 172.21.15.7:0/1442474476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf2c1086a0 0x7fcf2c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.742+0000 7fcf347df640 1 -- 172.21.15.7:0/1442474476 >> 172.21.15.7:0/1442474476 conn(0x7fcf2c06e6b0 msgr2=0x7fcf2c070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.742+0000 7fcf347df640 1 -- 172.21.15.7:0/1442474476 shutdown_connections 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.742+0000 7fcf347df640 1 -- 172.21.15.7:0/1442474476 wait complete. 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf347df640 1 Processor -- start 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf347df640 1 -- start start 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf347df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf2c1086a0 0x7fcf2c1a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf347df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c109000 0x7fcf2c1a2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf347df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf2c10ff20 0x7fcf2c1a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf347df640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c115b20 con 0x7fcf2c1086a0 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf347df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c1159a0 con 0x7fcf2c109000 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf347df640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c115ca0 con 0x7fcf2c10ff20 2024-03-26T08:45:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf32d7c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf2c10ff20 0x7fcf2c1a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf3257b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf2c1086a0 0x7fcf2c1a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf32d7c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf2c10ff20 0x7fcf2c1a9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48558/0 (socket says 172.21.15.7:48558) 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf32d7c640 1 -- 172.21.15.7:0/1374472218 learned_addr learned my addr 172.21.15.7:0/1374472218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.743+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c109000 0x7fcf2c1a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.744+0000 7fcf32d7c640 1 -- 172.21.15.7:0/1374472218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c109000 msgr2=0x7fcf2c1a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.744+0000 7fcf32d7c640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c109000 0x7fcf2c1a2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.744+0000 7fcf32d7c640 1 -- 172.21.15.7:0/1374472218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf2c1086a0 msgr2=0x7fcf2c1a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.744+0000 7fcf32d7c640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf2c1086a0 0x7fcf2c1a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.744+0000 7fcf32d7c640 1 -- 172.21.15.7:0/1374472218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c1a9ee0 con 0x7fcf2c10ff20 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.744+0000 7fcf32d7c640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf2c10ff20 0x7fcf2c1a9870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf2402c640 tx=0x7fcf24035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:20.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.744+0000 7fcf237fe640 1 -- 172.21.15.7:0/1374472218 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24039040 con 0x7fcf2c10ff20 2024-03-26T08:45:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.745+0000 7fcf237fe640 1 -- 172.21.15.7:0/1374472218 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf24044070 con 0x7fcf2c10ff20 2024-03-26T08:45:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.745+0000 7fcf237fe640 1 -- 172.21.15.7:0/1374472218 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24004350 con 0x7fcf2c10ff20 2024-03-26T08:45:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.745+0000 7fcf347df640 1 -- 172.21.15.7:0/1374472218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c1aa170 con 0x7fcf2c10ff20 2024-03-26T08:45:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.745+0000 7fcf347df640 1 -- 172.21.15.7:0/1374472218 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c1aa610 con 0x7fcf2c10ff20 2024-03-26T08:45:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.748+0000 7fcf237fe640 1 -- 172.21.15.7:0/1374472218 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcf240044f0 con 0x7fcf2c10ff20 2024-03-26T08:45:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf373c7740 1 Processor -- start 2024-03-26T08:45:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf373c7740 1 -- start start 2024-03-26T08:45:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242efd920 0x55c242efdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f02e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 0x55c242f05910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf373c7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c242e62b10 con 0x55c242f03480 2024-03-26T08:45:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf373c7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c242e0b900 con 0x55c242efe310 2024-03-26T08:45:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf373c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c242e5ebb0 con 0x55c242efd920 2024-03-26T08:45:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf237fe640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcf180a2870 0x7fcf180a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf31d7a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242efd920 0x55c242efdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf3257b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f02e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf31d7a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242efd920 0x55c242efdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48564/0 (socket says 172.21.15.7:48564) 2024-03-26T08:45:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf3257b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f02e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59640/0 (socket says 172.21.15.7:59640) 2024-03-26T08:45:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf31d7a640 1 -- 172.21.15.7:0/376695054 learned_addr learned my addr 172.21.15.7:0/376695054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.749+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/376695054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 0x55c242f05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf237fe640 1 -- 172.21.15.7:0/1374472218 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcf240f0990 con 0x7fcf2c10ff20 2024-03-26T08:45:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf32d7c640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcf180a2870 0x7fcf180a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf31d7a640 1 -- 172.21.15.7:0/376695054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 msgr2=0x55c242f02e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/376695054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f02e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf31d7a640 1 -- 172.21.15.7:0/376695054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 msgr2=0x55c242f05910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/376695054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 0x55c242f05910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf31d7a640 1 -- 172.21.15.7:0/376695054 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c242b77210 con 0x55c242efd920 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/376695054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242efd920 0x55c242efdd40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0117e0 tx=0x7fcf1c00a760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1333f95b259ffaa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf32d7c640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcf180a2870 0x7fcf180a4d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf240c55a0 tx=0x7fcf240093d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.750+0000 7fcf20ff9640 1 -- 172.21.15.7:0/376695054 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c011e20 con 0x55c242efd920 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.751+0000 7fcf20ff9640 1 -- 172.21.15.7:0/376695054 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c005020 con 0x55c242efd920 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.751+0000 7fcf20ff9640 1 -- 172.21.15.7:0/376695054 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c004a00 con 0x55c242efd920 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.752+0000 7fcf373c7740 1 -- 172.21.15.7:0/376695054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242efd920 msgr2=0x55c242efdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.752+0000 7fcf373c7740 1 --2- 172.21.15.7:0/376695054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242efd920 0x55c242efdd40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0117e0 tx=0x7fcf1c00a760 comp rx=0 tx=0).stop 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.752+0000 7fcf373c7740 1 -- 172.21.15.7:0/376695054 shutdown_connections 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.752+0000 7fcf373c7740 1 --2- 172.21.15.7:0/376695054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 0x55c242f05910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.752+0000 7fcf373c7740 1 --2- 172.21.15.7:0/376695054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f02e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.752+0000 7fcf373c7740 1 --2- 172.21.15.7:0/376695054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242efd920 0x55c242efdd40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.752+0000 7fcf373c7740 1 -- 172.21.15.7:0/376695054 >> 172.21.15.7:0/376695054 conn(0x55c242ef8e90 msgr2=0x55c242ef9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.752+0000 7fcf373c7740 1 -- 172.21.15.7:0/376695054 shutdown_connections 2024-03-26T08:45:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.753+0000 7fcf373c7740 1 -- 172.21.15.7:0/376695054 wait complete. 2024-03-26T08:45:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.753+0000 7fcf373c7740 1 -- start start 2024-03-26T08:45:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.753+0000 7fcf373c7740 1 Processor -- start 2024-03-26T08:45:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efd920 0x55c242f104e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242efe310 0x55c242f10a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 0x55c242f17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf373c7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c242f0ebb0 con 0x55c242efe310 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf373c7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c242e1ed50 con 0x55c242efd920 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf373c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c242f0ed30 con 0x55c242f03480 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf373c7740 10 client.0 _ll_register_callbacks cb 0x55c242e1e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf3257b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242efe310 0x55c242f10a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf3257b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242efe310 0x55c242f10a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37762/0 (socket says 172.21.15.7:37762) 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf3257b640 1 -- 172.21.15.7:0/1865988987 learned_addr learned my addr 172.21.15.7:0/1865988987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 0x55c242f17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.754+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efd920 0x55c242f104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf31d7a640 1 -- 172.21.15.7:0/1865988987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 msgr2=0x55c242f17a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 0x55c242f17a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf31d7a640 1 -- 172.21.15.7:0/1865988987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242efe310 msgr2=0x55c242f10a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242efe310 0x55c242f10a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf31d7a640 1 -- 172.21.15.7:0/1865988987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c242c43620 con 0x55c242efd920 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efd920 0x55c242f104e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00a650 tx=0x7fcf1c00ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fba4ab95016aac54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1865988987 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c004660 con 0x55c242efd920 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1865988987 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcf1c004800 con 0x55c242efd920 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf373c7740 1 -- 172.21.15.7:0/1865988987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf1c005020 con 0x55c242efd920 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1865988987 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf1c004a00 con 0x55c242efd920 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.755+0000 7fcf373c7740 10 client.26027 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf373c7740 1 -- 172.21.15.7:0/1865988987 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c242f1add0 con 0x55c242efd920 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1865988987 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c005020 con 0x55c242efd920 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1865988987 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcf1c02a020 con 0x55c242efd920 2024-03-26T08:45:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf373c7740 10 client.26027 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf373c7740 10 client.26027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf373c7740 20 client.26027 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c242f1c1c0 0x55c242f1e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf373c7740 4 client.26027 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf373c7740 1 -- 172.21.15.7:0/1865988987 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c242e70050 con 0x55c242f1c1c0 2024-03-26T08:45:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.756+0000 7fcf3257b640 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c242f1c1c0 0x55c242f1e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.757+0000 7fcf3257b640 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c242f1c1c0 0x55c242f1e5b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9021b2bee871fdd server_cookie=e4283a70898c3230 in_seq=0 out_seq=0 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.757+0000 7fcf15ffb640 10 client.26027 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.758+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1865988987 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c242e70050 con 0x55c242f1c1c0 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.758+0000 7fcf15ffb640 10 client.26027 handle_command_reply: tid=0 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.758+0000 7fcf373c7740 1 client.26027 shutdown 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.758+0000 7fcf373c7740 1 -- 172.21.15.7:0/1865988987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efd920 msgr2=0x55c242f104e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.758+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efd920 0x55c242f104e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00a650 tx=0x7fcf1c00ffd0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.758+0000 7fcf373c7740 1 -- 172.21.15.7:0/1865988987 shutdown_connections 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.758+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c242f1c1c0 0x55c242f1e5b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.758+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 0x55c242f17a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.758+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242efe310 0x55c242f10a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.759+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1865988987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efd920 0x55c242f104e0 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00a650 tx=0x7fcf1c00ffd0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.759+0000 7fcf373c7740 1 -- 172.21.15.7:0/1865988987 >> 172.21.15.7:0/1865988987 conn(0x55c242ef8e90 msgr2=0x55c242ef8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.759+0000 7fcf15ffb640 0 client.26027 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.759+0000 7fcf373c7740 1 -- 172.21.15.7:0/1865988987 shutdown_connections 2024-03-26T08:45:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.759+0000 7fcf373c7740 1 -- 172.21.15.7:0/1865988987 wait complete. 2024-03-26T08:45:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.759+0000 7fcf373c7740 20 client.26027 trim_cache size 0 max 16384 2024-03-26T08:45:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf373c7740 1 Processor -- start 2024-03-26T08:45:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf373c7740 1 -- start start 2024-03-26T08:45:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 0x55c242f17a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f10630 0x55c242ef7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf373c7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100108c0 con 0x55c242f03480 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf373c7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000e3c0 con 0x55c242efe310 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf373c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000c440 con 0x55c242f10630 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf31d7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf3257b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 0x55c242f17a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf31d7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f100f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59652/0 (socket says 172.21.15.7:59652) 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf31d7a640 1 -- 172.21.15.7:0/3055568859 learned_addr learned my addr 172.21.15.7:0/3055568859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/3055568859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f10630 0x55c242ef7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf31d7a640 1 -- 172.21.15.7:0/3055568859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f10630 msgr2=0x55c242ef7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/3055568859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f10630 0x55c242ef7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf31d7a640 1 -- 172.21.15.7:0/3055568859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 msgr2=0x55c242f17a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/3055568859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 0x55c242f17a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.783+0000 7fcf31d7a640 1 -- 172.21.15.7:0/3055568859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c242f1ec30 con 0x55c242efe310 2024-03-26T08:45:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.784+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/3055568859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f100f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c01ed70 tx=0x7fcf1c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3281c429011dd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.784+0000 7fcf17fff640 1 -- 172.21.15.7:0/3055568859 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00fd20 con 0x55c242efe310 2024-03-26T08:45:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.784+0000 7fcf17fff640 1 -- 172.21.15.7:0/3055568859 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c004660 con 0x55c242efe310 2024-03-26T08:45:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.784+0000 7fcf17fff640 1 -- 172.21.15.7:0/3055568859 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c004ba0 con 0x55c242efe310 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 -- 172.21.15.7:0/3055568859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 msgr2=0x55c242f100f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 --2- 172.21.15.7:0/3055568859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f100f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c01ed70 tx=0x7fcf1c000bc0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 -- 172.21.15.7:0/3055568859 shutdown_connections 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 --2- 172.21.15.7:0/3055568859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f10630 0x55c242ef7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 --2- 172.21.15.7:0/3055568859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f03480 0x55c242f17a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 --2- 172.21.15.7:0/3055568859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f100f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 -- 172.21.15.7:0/3055568859 >> 172.21.15.7:0/3055568859 conn(0x55c242ef8e90 msgr2=0x55c242ef9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 -- 172.21.15.7:0/3055568859 shutdown_connections 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 -- 172.21.15.7:0/3055568859 wait complete. 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 -- start start 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.785+0000 7fcf373c7740 1 Processor -- start 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f15520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 0x55c242f15a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf373c7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f10630 0x55c242f16a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf373c7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000ca40 con 0x55c242f10630 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf373c7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000dd60 con 0x55c242efe310 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf373c7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000d0a0 con 0x55c242f03480 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf31d7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f15520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf31d7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f15520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59662/0 (socket says 172.21.15.7:59662) 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf31d7a640 1 -- 172.21.15.7:0/1894395209 learned_addr learned my addr 172.21.15.7:0/1894395209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf373c7740 10 client.0 _ll_register_callbacks cb 0x7fcf1c005570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf3257b640 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 0x55c242f15a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f10630 0x55c242f16a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf31d7a640 1 -- 172.21.15.7:0/1894395209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 msgr2=0x55c242f15a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 0x55c242f15a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf31d7a640 1 -- 172.21.15.7:0/1894395209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f10630 msgr2=0x55c242f16a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f10630 0x55c242f16a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.786+0000 7fcf31d7a640 1 -- 172.21.15.7:0/1894395209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c242f17060 con 0x55c242efe310 2024-03-26T08:45:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.788+0000 7fcf31d7a640 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f15520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0056a0 tx=0x7fcf1c005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7577cea045fbe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.788+0000 7fcf15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.789+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1894395209 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00a420 con 0x55c242efe310 2024-03-26T08:45:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.789+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1894395209 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcf1c011250 con 0x55c242efe310 2024-03-26T08:45:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.789+0000 7fcf373c7740 1 -- 172.21.15.7:0/1894395209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf1c004660 con 0x55c242efe310 2024-03-26T08:45:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.789+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1894395209 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf1c004810 con 0x55c242efe310 2024-03-26T08:45:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.789+0000 7fcf373c7740 10 client.26033 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.789+0000 7fcf373c7740 1 -- 172.21.15.7:0/1894395209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c242f77da0 con 0x55c242efe310 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.789+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1894395209 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c004660 con 0x55c242efe310 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1894395209 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcf1c02a040 con 0x55c242efe310 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf373c7740 10 client.26033 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf373c7740 10 client.26033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf373c7740 20 client.26033 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c242f78670 0x55c242f7aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf373c7740 4 client.26033 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf373c7740 1 -- 172.21.15.7:0/1894395209 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c242e6a3a0 con 0x55c242f78670 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf3257b640 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c242f78670 0x55c242f7aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf3257b640 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c242f78670 0x55c242f7aa90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4fb55f52da6e46c server_cookie=2420677b065282e9 in_seq=0 out_seq=0 2024-03-26T08:45:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.790+0000 7fcf15ffb640 10 client.26033 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.872+0000 7fcf15ffb640 1 -- 172.21.15.7:0/1894395209 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c242e6a3a0 con 0x55c242f78670 2024-03-26T08:45:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.872+0000 7fcf15ffb640 10 client.26033 handle_command_reply: tid=0 2024-03-26T08:45:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.872+0000 7fcf373c7740 1 client.26033 shutdown 2024-03-26T08:45:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 -- 172.21.15.7:0/1894395209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 msgr2=0x55c242f15520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f15520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0056a0 tx=0x7fcf1c005e60 comp rx=0 tx=0).stop 2024-03-26T08:45:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 -- 172.21.15.7:0/1894395209 shutdown_connections 2024-03-26T08:45:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c242f78670 0x55c242f7aa90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c242f10630 0x55c242f16a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c242f03480 0x55c242f15a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 --2- 172.21.15.7:0/1894395209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c242efe310 0x55c242f15520 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 -- 172.21.15.7:0/1894395209 >> 172.21.15.7:0/1894395209 conn(0x55c242ef8e90 msgr2=0x55c242ef7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf15ffb640 0 client.26033 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 -- 172.21.15.7:0/1894395209 shutdown_connections 2024-03-26T08:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 1 -- 172.21.15.7:0/1894395209 wait complete. 2024-03-26T08:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.873+0000 7fcf373c7740 20 client.26033 trim_cache size 0 max 16384 2024-03-26T08:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:45:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:45:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 -- 172.21.15.7:0/1374472218 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcf180a2870 msgr2=0x7fcf180a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcf180a2870 0x7fcf180a4d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf240c55a0 tx=0x7fcf240093d0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 -- 172.21.15.7:0/1374472218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf2c10ff20 msgr2=0x7fcf2c1a9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf2c10ff20 0x7fcf2c1a9870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf2402c640 tx=0x7fcf24035d10 comp rx=0 tx=0).stop 2024-03-26T08:45:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 -- 172.21.15.7:0/1374472218 shutdown_connections 2024-03-26T08:45:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcf180a2870 0x7fcf180a4d30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcf2c10ff20 0x7fcf2c1a9870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c109000 0x7fcf2c1a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 --2- 172.21.15.7:0/1374472218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf2c1086a0 0x7fcf2c1a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 -- 172.21.15.7:0/1374472218 >> 172.21.15.7:0/1374472218 conn(0x7fcf2c06e6b0 msgr2=0x7fcf2c070240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 -- 172.21.15.7:0/1374472218 shutdown_connections 2024-03-26T08:45:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:20.874+0000 7fcf347df640 1 -- 172.21.15.7:0/1374472218 wait complete. 2024-03-26T08:45:21.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 170.0 s, 3806.0 tps, lat 26.256 ms stddev 28.542 2024-03-26T08:45:22.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:22 smithi016 ceph-mon[35851]: pgmap v448: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 51 MiB/s wr, 1.48k op/s 2024-03-26T08:45:22.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:22 smithi007 ceph-mon[32462]: pgmap v448: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 51 MiB/s wr, 1.48k op/s 2024-03-26T08:45:22.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:22 smithi044 ceph-mon[36461]: pgmap v448: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 51 MiB/s wr, 1.48k op/s 2024-03-26T08:45:24.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:24 smithi016 ceph-mon[35851]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 50 MiB/s wr, 1.52k op/s 2024-03-26T08:45:24.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:24 smithi007 ceph-mon[32462]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 50 MiB/s wr, 1.52k op/s 2024-03-26T08:45:24.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:24 smithi044 ceph-mon[36461]: pgmap v449: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 50 MiB/s wr, 1.52k op/s 2024-03-26T08:45:26.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 175.0 s, 3938.0 tps, lat 25.433 ms stddev 26.855 2024-03-26T08:45:26.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:26 smithi016 ceph-mon[35851]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 29 MiB/s wr, 1.14k op/s 2024-03-26T08:45:26.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:26 smithi007 ceph-mon[32462]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 29 MiB/s wr, 1.14k op/s 2024-03-26T08:45:26.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:26 smithi044 ceph-mon[36461]: pgmap v450: 97 pgs: 97 active+clean; 1.5 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 29 MiB/s wr, 1.14k op/s 2024-03-26T08:45:28.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:28 smithi016 ceph-mon[35851]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 812 op/s 2024-03-26T08:45:28.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:28 smithi007 ceph-mon[32462]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 812 op/s 2024-03-26T08:45:28.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:28 smithi044 ceph-mon[36461]: pgmap v451: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 812 op/s 2024-03-26T08:45:30.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:30 smithi016 ceph-mon[35851]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 825 op/s 2024-03-26T08:45:30.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:30 smithi007 ceph-mon[32462]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 825 op/s 2024-03-26T08:45:30.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:30 smithi044 ceph-mon[36461]: pgmap v452: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 MiB/s wr, 825 op/s 2024-03-26T08:45:31.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 180.0 s, 3534.2 tps, lat 28.192 ms stddev 29.517 2024-03-26T08:45:31.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:45:31.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:45:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.338+0000 7fcff4b2e640 1 -- 172.21.15.7:0/1813052756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80a6710 msgr2=0x7fcfe80aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.338+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/1813052756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80a6710 0x7fcfe80aafd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcff00674e0 tx=0x7fcff008f370 comp rx=0 tx=0).stop 2024-03-26T08:45:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.339+0000 7fcff4b2e640 1 -- 172.21.15.7:0/1813052756 shutdown_connections 2024-03-26T08:45:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.339+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/1813052756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80a6710 0x7fcfe80aafd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.339+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/1813052756 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5d70 0x7fcfe80a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.339+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/1813052756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe80a4c40 0x7fcfe80a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.339+0000 7fcff4b2e640 1 -- 172.21.15.7:0/1813052756 >> 172.21.15.7:0/1813052756 conn(0x7fcfe80a00d0 msgr2=0x7fcfe80a2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.342+0000 7fcff4b2e640 1 -- 172.21.15.7:0/1813052756 shutdown_connections 2024-03-26T08:45:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.342+0000 7fcff4b2e640 1 -- 172.21.15.7:0/1813052756 wait complete. 2024-03-26T08:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.343+0000 7fcff4b2e640 1 Processor -- start 2024-03-26T08:45:31.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.343+0000 7fcff4b2e640 1 -- start start 2024-03-26T08:45:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.344+0000 7fcff4b2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80a4c40 0x7fcfe8143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.345+0000 7fcff4b2e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5d70 0x7fcfe8143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.345+0000 7fcff4b2e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe80a6710 0x7fcfe814abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.345+0000 7fcfeeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5d70 0x7fcfe8143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.345+0000 7fcfeeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5d70 0x7fcfe8143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38832/0 (socket says 172.21.15.7:38832) 2024-03-26T08:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.345+0000 7fcfeeffd640 1 -- 172.21.15.7:0/2373251091 learned_addr learned my addr 172.21.15.7:0/2373251091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.345+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe80a6710 0x7fcfe814abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.345+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80b8ff0 con 0x7fcfe80a6710 2024-03-26T08:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.345+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80b8e70 con 0x7fcfe80a4c40 2024-03-26T08:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.346+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80b9170 con 0x7fcfe80a5d70 2024-03-26T08:45:31.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.346+0000 7fcfeffff640 1 -- 172.21.15.7:0/2373251091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5d70 msgr2=0x7fcfe8143b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.346+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5d70 0x7fcfe8143b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.346+0000 7fcfeffff640 1 -- 172.21.15.7:0/2373251091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80a4c40 msgr2=0x7fcfe8143610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:45:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.346+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80a4c40 0x7fcfe8143610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.346+0000 7fcfeffff640 1 -- 172.21.15.7:0/2373251091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe814b230 con 0x7fcfe80a6710 2024-03-26T08:45:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.346+0000 7fcfeeffd640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5d70 0x7fcfe8143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.346+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe80a6710 0x7fcfe814abc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcff002d600 tx=0x7fcff00963d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.347+0000 7fcfecff9640 1 -- 172.21.15.7:0/2373251091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0069130 con 0x7fcfe80a6710 2024-03-26T08:45:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.347+0000 7fcfecff9640 1 -- 172.21.15.7:0/2373251091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcff009d040 con 0x7fcfe80a6710 2024-03-26T08:45:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.347+0000 7fcfecff9640 1 -- 172.21.15.7:0/2373251091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0097be0 con 0x7fcfe80a6710 2024-03-26T08:45:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.347+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe814b4c0 con 0x7fcfe80a6710 2024-03-26T08:45:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.348+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe814b9b0 con 0x7fcfe80a6710 2024-03-26T08:45:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.351+0000 7fcfecff9640 1 -- 172.21.15.7:0/2373251091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcff00a9030 con 0x7fcfe80a6710 2024-03-26T08:45:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.352+0000 7fcfecff9640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfe00a2a00 0x7fcfe00a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.352+0000 7fcfecff9640 1 -- 172.21.15.7:0/2373251091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcff01536f0 con 0x7fcfe80a6710 2024-03-26T08:45:31.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.352+0000 7fcfef7fe640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfe00a2a00 0x7fcfe00a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.354+0000 7fcff897e740 1 Processor -- start 2024-03-26T08:45:31.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.354+0000 7fcff897e740 1 -- start start 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcff897e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9243f80 0x5616a92443a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcff897e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9244970 0x5616a9249470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcff897e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcff897e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a91a93d0 con 0x5616a9244970 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcff897e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a9151ad0 con 0x5616a9243f80 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcff897e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a91a5470 con 0x5616a9249ae0 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfeeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9244970 0x5616a9249470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9243f80 0x5616a92443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfeeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9244970 0x5616a9249470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54322/0 (socket says 172.21.15.7:54322) 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfeeffd640 1 -- 172.21.15.7:0/4129086273 learned_addr learned my addr 172.21.15.7:0/4129086273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfeffff640 1 --2- 172.21.15.7:0/4129086273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfeffff640 1 -- 172.21.15.7:0/4129086273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9243f80 msgr2=0x5616a92443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfeffff640 1 --2- 172.21.15.7:0/4129086273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9243f80 0x5616a92443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfeffff640 1 -- 172.21.15.7:0/4129086273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9244970 msgr2=0x5616a9249470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfeffff640 1 --2- 172.21.15.7:0/4129086273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9244970 0x5616a9249470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfeffff640 1 -- 172.21.15.7:0/4129086273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616a8f877d0 con 0x5616a9249ae0 2024-03-26T08:45:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.355+0000 7fcfef7fe640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfe00a2a00 0x7fcfe00a4ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009670 tx=0x7fcfdc006bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.356+0000 7fcfeffff640 1 --2- 172.21.15.7:0/4129086273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924bf70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcff0067350 tx=0x7fcff008f370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a97c7b5fb1e577 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:31.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.357+0000 7fcfd9ffb640 1 -- 172.21.15.7:0/4129086273 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0120050 con 0x5616a9249ae0 2024-03-26T08:45:31.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.357+0000 7fcfd9ffb640 1 -- 172.21.15.7:0/4129086273 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcff01241a0 con 0x5616a9249ae0 2024-03-26T08:45:31.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.357+0000 7fcfd9ffb640 1 -- 172.21.15.7:0/4129086273 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0097550 con 0x5616a9249ae0 2024-03-26T08:45:31.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.358+0000 7fcff897e740 1 -- 172.21.15.7:0/4129086273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 msgr2=0x5616a924bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.358+0000 7fcff897e740 1 --2- 172.21.15.7:0/4129086273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924bf70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcff0067350 tx=0x7fcff008f370 comp rx=0 tx=0).stop 2024-03-26T08:45:31.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.358+0000 7fcff897e740 1 -- 172.21.15.7:0/4129086273 shutdown_connections 2024-03-26T08:45:31.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.358+0000 7fcff897e740 1 --2- 172.21.15.7:0/4129086273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924bf70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.358+0000 7fcff897e740 1 --2- 172.21.15.7:0/4129086273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9244970 0x5616a9249470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.358+0000 7fcff897e740 1 --2- 172.21.15.7:0/4129086273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9243f80 0x5616a92443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.358+0000 7fcff897e740 1 -- 172.21.15.7:0/4129086273 >> 172.21.15.7:0/4129086273 conn(0x5616a923f4f0 msgr2=0x5616a923f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.359+0000 7fcff897e740 1 -- 172.21.15.7:0/4129086273 shutdown_connections 2024-03-26T08:45:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.359+0000 7fcff897e740 1 -- 172.21.15.7:0/4129086273 wait complete. 2024-03-26T08:45:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.360+0000 7fcff897e740 1 -- start start 2024-03-26T08:45:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.360+0000 7fcff897e740 1 Processor -- start 2024-03-26T08:45:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.360+0000 7fcff897e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.361+0000 7fcff897e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9244970 0x5616a924e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.361+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.361+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54326/0 (socket says 172.21.15.7:54326) 2024-03-26T08:45:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.361+0000 7fcfef7fe640 1 -- 172.21.15.7:0/2512648831 learned_addr learned my addr 172.21.15.7:0/2512648831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.361+0000 7fcff897e740 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9249ae0 0x5616a9255410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.361+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a91a5470 con 0x5616a9243f80 2024-03-26T08:45:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.361+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a9165060 con 0x5616a9249ae0 2024-03-26T08:45:31.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.361+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616a924c850 con 0x5616a9244970 2024-03-26T08:45:31.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.361+0000 7fcff897e740 10 client.0 _ll_register_callbacks cb 0x5616a91476a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.362+0000 7fcfef7fe640 1 -- 172.21.15.7:0/2512648831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9244970 msgr2=0x5616a924e3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.362+0000 7fcfef7fe640 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9244970 0x5616a924e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.362+0000 7fcfef7fe640 1 -- 172.21.15.7:0/2512648831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9249ae0 msgr2=0x5616a9255410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.362+0000 7fcfef7fe640 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9249ae0 0x5616a9255410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.362+0000 7fcfef7fe640 1 -- 172.21.15.7:0/2512648831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616a8ebd3f0 con 0x5616a9243f80 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.363+0000 7fcfef7fe640 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924de90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc006a50 tx=0x7fcfdc004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ca1bf3cebafc0b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.363+0000 7fcfceffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.363+0000 7fcfceffd640 1 -- 172.21.15.7:0/2512648831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc022020 con 0x5616a9243f80 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.363+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcff01241a0 con 0x5616a9243f80 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.363+0000 7fcfceffd640 1 -- 172.21.15.7:0/2512648831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcfdc01e810 con 0x5616a9243f80 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.363+0000 7fcfceffd640 1 -- 172.21.15.7:0/2512648831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfdc019050 con 0x5616a9243f80 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.363+0000 7fcff897e740 10 client.16197 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.363+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5616a8e7d0f0 con 0x5616a9243f80 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.363+0000 7fcfceffd640 1 -- 172.21.15.7:0/2512648831 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcff01241a0 con 0x5616a9243f80 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcfceffd640 1 -- 172.21.15.7:0/2512648831 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcfdc012070 con 0x5616a9243f80 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcff897e740 10 client.16197 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcff897e740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcff897e740 20 client.16197 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcff897e740 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5616a9259a70 0x5616a925bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcff897e740 4 client.16197 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616a91b6910 con 0x5616a9259a70 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcfeeffd640 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5616a9259a70 0x5616a925bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcfeeffd640 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5616a9259a70 0x5616a925bf30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb15891ab96591f6 server_cookie=b9ef2cd0231471af in_seq=0 out_seq=0 2024-03-26T08:45:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.364+0000 7fcfceffd640 10 client.16197 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.365+0000 7fcfceffd640 1 -- 172.21.15.7:0/2512648831 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5616a91b6910 con 0x5616a9259a70 2024-03-26T08:45:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.365+0000 7fcfceffd640 10 client.16197 handle_command_reply: tid=0 2024-03-26T08:45:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.365+0000 7fcff897e740 1 client.16197 shutdown 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.366+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 msgr2=0x5616a924de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.366+0000 7fcff897e740 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924de90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc006a50 tx=0x7fcfdc004700 comp rx=0 tx=0).stop 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.366+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 shutdown_connections 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.366+0000 7fcff897e740 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5616a9259a70 0x5616a925bf30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.366+0000 7fcff897e740 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9249ae0 0x5616a9255410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.366+0000 7fcfceffd640 0 client.16197 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.366+0000 7fcff897e740 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9244970 0x5616a924e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.366+0000 7fcff897e740 1 --2- 172.21.15.7:0/2512648831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924de90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.366+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 >> 172.21.15.7:0/2512648831 conn(0x5616a923f4f0 msgr2=0x5616a923e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.367+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 shutdown_connections 2024-03-26T08:45:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.367+0000 7fcff897e740 1 -- 172.21.15.7:0/2512648831 wait complete. 2024-03-26T08:45:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.367+0000 7fcff897e740 20 client.16197 trim_cache size 0 max 16384 2024-03-26T08:45:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcff897e740 1 Processor -- start 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcff897e740 1 -- start start 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcff897e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcff897e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a92553d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcff897e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcff897e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8156c30 con 0x5616a9243f80 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcff897e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8154730 con 0x5616a9244970 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcff897e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe81527b0 con 0x5616a9249ae0 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcfeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a92553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcfeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a92553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59380/0 (socket says 172.21.15.7:59380) 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.399+0000 7fcfeeffd640 1 -- 172.21.15.7:0/2764408238 learned_addr learned my addr 172.21.15.7:0/2764408238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2764408238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfef7fe640 1 --2- 172.21.15.7:0/2764408238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfeffff640 1 -- 172.21.15.7:0/2764408238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 msgr2=0x5616a92553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2764408238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a92553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfeffff640 1 -- 172.21.15.7:0/2764408238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 msgr2=0x5616a924d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2764408238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfeffff640 1 -- 172.21.15.7:0/2764408238 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616a923a640 con 0x5616a9249ae0 2024-03-26T08:45:31.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2764408238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924daa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcff008f980 tx=0x7fcff00502f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a4cead3d668750a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:31.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfd97fa640 1 -- 172.21.15.7:0/2764408238 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0138070 con 0x5616a9249ae0 2024-03-26T08:45:31.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.400+0000 7fcfd97fa640 1 -- 172.21.15.7:0/2764408238 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcff01241a0 con 0x5616a9249ae0 2024-03-26T08:45:31.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.401+0000 7fcfd97fa640 1 -- 172.21.15.7:0/2764408238 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff013eb50 con 0x5616a9249ae0 2024-03-26T08:45:31.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.402+0000 7fcff897e740 1 -- 172.21.15.7:0/2764408238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 msgr2=0x5616a924daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.402+0000 7fcff897e740 1 --2- 172.21.15.7:0/2764408238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924daa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcff008f980 tx=0x7fcff00502f0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.403+0000 7fcff897e740 1 -- 172.21.15.7:0/2764408238 shutdown_connections 2024-03-26T08:45:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.403+0000 7fcff897e740 1 --2- 172.21.15.7:0/2764408238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9249ae0 0x5616a924daa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.403+0000 7fcff897e740 1 --2- 172.21.15.7:0/2764408238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a92553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.403+0000 7fcff897e740 1 --2- 172.21.15.7:0/2764408238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9243f80 0x5616a924d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.403+0000 7fcff897e740 1 -- 172.21.15.7:0/2764408238 >> 172.21.15.7:0/2764408238 conn(0x5616a923f4f0 msgr2=0x5616a924cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.403+0000 7fcff897e740 1 -- 172.21.15.7:0/2764408238 shutdown_connections 2024-03-26T08:45:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.403+0000 7fcff897e740 1 -- 172.21.15.7:0/2764408238 wait complete. 2024-03-26T08:45:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.404+0000 7fcff897e740 1 -- start start 2024-03-26T08:45:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.404+0000 7fcff897e740 1 Processor -- start 2024-03-26T08:45:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcff897e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9243f80 0x5616a92522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcff897e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a9252800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcff897e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9249ae0 0x5616a92b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcfeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a9252800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9243f80 0x5616a92522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcfeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a9252800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59382/0 (socket says 172.21.15.7:59382) 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9243f80 0x5616a92522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38856/0 (socket says 172.21.15.7:38856) 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcfeeffd640 1 -- 172.21.15.7:0/2283541433 learned_addr learned my addr 172.21.15.7:0/2283541433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8152db0 con 0x5616a9249ae0 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe81540d0 con 0x5616a9244970 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8153410 con 0x5616a9243f80 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.405+0000 7fcff897e740 10 client.0 _ll_register_callbacks cb 0x7fcfdc016040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.406+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9249ae0 0x5616a92b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.406+0000 7fcfeffff640 1 -- 172.21.15.7:0/2283541433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9243f80 msgr2=0x5616a92522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.406+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9243f80 0x5616a92522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.406+0000 7fcfeffff640 1 -- 172.21.15.7:0/2283541433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 msgr2=0x5616a9252800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.406+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a9252800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.406+0000 7fcfeffff640 1 -- 172.21.15.7:0/2283541433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616a92b38f0 con 0x5616a9249ae0 2024-03-26T08:45:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.406+0000 7fcfeffff640 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9249ae0 0x5616a92b33b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcff004fcc0 tx=0x7fcff008f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f64c951c628e236 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.406+0000 7fcfceffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:45:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.407+0000 7fcfceffd640 1 -- 172.21.15.7:0/2283541433 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0138070 con 0x5616a9249ae0 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.407+0000 7fcfceffd640 1 -- 172.21.15.7:0/2283541433 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcff011add0 con 0x5616a9249ae0 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.407+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcff01241a0 con 0x5616a9249ae0 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.407+0000 7fcfceffd640 1 -- 172.21.15.7:0/2283541433 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcff012d050 con 0x5616a9249ae0 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.407+0000 7fcff897e740 10 client.16203 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.407+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5616a92b4b20 con 0x5616a9249ae0 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.408+0000 7fcfceffd640 1 -- 172.21.15.7:0/2283541433 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcff01241a0 con 0x5616a9249ae0 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.408+0000 7fcfceffd640 1 -- 172.21.15.7:0/2283541433 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcff011f4a0 con 0x5616a9249ae0 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.408+0000 7fcff897e740 10 client.16203 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.408+0000 7fcff897e740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.408+0000 7fcff897e740 20 client.16203 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.408+0000 7fcff897e740 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5616a92b5050 0x5616a92b7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.408+0000 7fcff897e740 4 client.16203 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:45:31.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.408+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5616a91b0c60 con 0x5616a92b5050 2024-03-26T08:45:31.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.408+0000 7fcfef7fe640 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5616a92b5050 0x5616a92b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.409+0000 7fcfef7fe640 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5616a92b5050 0x5616a92b7440 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca5fa08b6f15b1 server_cookie=564176c4eeb48305 in_seq=0 out_seq=0 2024-03-26T08:45:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.409+0000 7fcfceffd640 10 client.16203 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcfceffd640 1 -- 172.21.15.7:0/2283541433 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5616a91b0c60 con 0x5616a92b5050 2024-03-26T08:45:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcfceffd640 10 client.16203 handle_command_reply: tid=0 2024-03-26T08:45:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcff897e740 1 client.16203 shutdown 2024-03-26T08:45:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9249ae0 msgr2=0x5616a92b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcff897e740 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9249ae0 0x5616a92b33b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcff004fcc0 tx=0x7fcff008f890 comp rx=0 tx=0).stop 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 shutdown_connections 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcff897e740 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5616a92b5050 0x5616a92b7440 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcff897e740 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616a9249ae0 0x5616a92b33b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcff897e740 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5616a9244970 0x5616a9252800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcff897e740 1 --2- 172.21.15.7:0/2283541433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5616a9243f80 0x5616a92522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 >> 172.21.15.7:0/2283541433 conn(0x5616a923f4f0 msgr2=0x5616a9242fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.410+0000 7fcfceffd640 0 client.16203 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 shutdown_connections 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff897e740 1 -- 172.21.15.7:0/2283541433 wait complete. 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff897e740 20 client.16203 trim_cache size 0 max 16384 2024-03-26T08:45:31.403 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:45:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:45:31.404 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:45:31.404 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:45:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfe00a2a00 msgr2=0x7fcfe00a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfe00a2a00 0x7fcfe00a4ec0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009670 tx=0x7fcfdc006bd0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe80a6710 msgr2=0x7fcfe814abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe80a6710 0x7fcfe814abc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fcff002d600 tx=0x7fcff00963d0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 shutdown_connections 2024-03-26T08:45:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcfe00a2a00 0x7fcfe00a4ec0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfe80a6710 0x7fcfe814abc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfe80a5d70 0x7fcfe8143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 --2- 172.21.15.7:0/2373251091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80a4c40 0x7fcfe8143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.411+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 >> 172.21.15.7:0/2373251091 conn(0x7fcfe80a00d0 msgr2=0x7fcfe80a6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.412+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 shutdown_connections 2024-03-26T08:45:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.412+0000 7fcff4b2e640 1 -- 172.21.15.7:0/2373251091 wait complete. 2024-03-26T08:45:31.417 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:45:31.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:45:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.591+0000 7f4c99259640 1 -- 172.21.15.7:0/1090839282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94103280 msgr2=0x7f4c94109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.591+0000 7f4c99259640 1 --2- 172.21.15.7:0/1090839282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94103280 0x7f4c94109b10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009ca0 tx=0x7f4c8c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.591+0000 7f4c99259640 1 -- 172.21.15.7:0/1090839282 shutdown_connections 2024-03-26T08:45:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.591+0000 7f4c99259640 1 --2- 172.21.15.7:0/1090839282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94103280 0x7f4c94109b10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.591+0000 7f4c99259640 1 --2- 172.21.15.7:0/1090839282 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c941028c0 0x7f4c94102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.591+0000 7f4c99259640 1 --2- 172.21.15.7:0/1090839282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c941016d0 0x7f4c94101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.591+0000 7f4c99259640 1 -- 172.21.15.7:0/1090839282 >> 172.21.15.7:0/1090839282 conn(0x7f4c940fcc40 msgr2=0x7f4c940ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.591+0000 7f4c99259640 1 -- 172.21.15.7:0/1090839282 shutdown_connections 2024-03-26T08:45:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.592+0000 7f4c99259640 1 -- 172.21.15.7:0/1090839282 wait complete. 2024-03-26T08:45:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.593+0000 7f4c99259640 1 Processor -- start 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.593+0000 7f4c99259640 1 -- start start 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.593+0000 7f4c99259640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c941016d0 0x7f4c941979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.593+0000 7f4c99259640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c941028c0 0x7f4c94197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.593+0000 7f4c99259640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94103280 0x7f4c9419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.593+0000 7f4c99259640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9410d350 con 0x7f4c94103280 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.593+0000 7f4c99259640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9410d1d0 con 0x7f4c941028c0 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.593+0000 7f4c99259640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9410d4d0 con 0x7f4c941016d0 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c92d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c941016d0 0x7f4c941979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c92d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c941016d0 0x7f4c941979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38878/0 (socket says 172.21.15.7:38878) 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c92d9d640 1 -- 172.21.15.7:0/3971494962 learned_addr learned my addr 172.21.15.7:0/3971494962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:31.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c8bfff640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c941028c0 0x7f4c94197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c9359e640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94103280 0x7f4c9419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c92d9d640 1 -- 172.21.15.7:0/3971494962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c941028c0 msgr2=0x7f4c94197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c92d9d640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c941028c0 0x7f4c94197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c92d9d640 1 -- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94103280 msgr2=0x7f4c9419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c92d9d640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94103280 0x7f4c9419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c92d9d640 1 -- 172.21.15.7:0/3971494962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9406b1f0 con 0x7f4c941016d0 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c8bfff640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c941028c0 0x7f4c94197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.594+0000 7f4c92d9d640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c941016d0 0x7f4c941979b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400bea0 tx=0x7f4c8400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.595+0000 7f4c90d99640 1 -- 172.21.15.7:0/3971494962 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84018070 con 0x7f4c941016d0 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.595+0000 7f4c90d99640 1 -- 172.21.15.7:0/3971494962 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c84016070 con 0x7f4c941016d0 2024-03-26T08:45:31.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.595+0000 7f4c90d99640 1 -- 172.21.15.7:0/3971494962 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8401d7e0 con 0x7f4c941016d0 2024-03-26T08:45:31.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.595+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9406b4e0 con 0x7f4c941016d0 2024-03-26T08:45:31.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.595+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9406b9e0 con 0x7f4c941016d0 2024-03-26T08:45:31.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.597+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c94108990 con 0x7f4c941016d0 2024-03-26T08:45:31.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.599+0000 7f4c90d99640 1 -- 172.21.15.7:0/3971494962 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4c8401d9e0 con 0x7f4c941016d0 2024-03-26T08:45:31.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.599+0000 7f4c90d99640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4c800a2900 0x7f4c800a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:31.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.599+0000 7f4c90d99640 1 -- 172.21.15.7:0/3971494962 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4c840d29c0 con 0x7f4c941016d0 2024-03-26T08:45:31.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.600+0000 7f4c8bfff640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4c800a2900 0x7f4c800a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:31.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.601+0000 7f4c8bfff640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4c800a2900 0x7f4c800a4dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c005fd0 tx=0x7f4c7c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:31.609+0000 7f4c90d99640 1 -- 172.21.15.7:0/3971494962 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c8409fa20 con 0x7f4c941016d0 2024-03-26T08:45:32.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.016+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c94062340 con 0x7f4c941016d0 2024-03-26T08:45:32.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.016+0000 7f4c90d99640 1 -- 172.21.15.7:0/3971494962 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f4c840bde20 con 0x7f4c941016d0 2024-03-26T08:45:32.008 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:45:32.008 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:45:32.008 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:45:32.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4c800a2900 msgr2=0x7f4c800a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4c800a2900 0x7f4c800a4dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c005fd0 tx=0x7f4c7c00b390 comp rx=0 tx=0).stop 2024-03-26T08:45:32.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c941016d0 msgr2=0x7f4c941979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c941016d0 0x7f4c941979b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400bea0 tx=0x7f4c8400bf90 comp rx=0 tx=0).stop 2024-03-26T08:45:32.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 shutdown_connections 2024-03-26T08:45:32.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4c800a2900 0x7f4c800a4dc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c94103280 0x7f4c9419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c941028c0 0x7f4c94197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 --2- 172.21.15.7:0/3971494962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4c941016d0 0x7f4c941979b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 >> 172.21.15.7:0/3971494962 conn(0x7f4c940fcc40 msgr2=0x7f4c94105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 shutdown_connections 2024-03-26T08:45:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.019+0000 7f4c99259640 1 -- 172.21.15.7:0/3971494962 wait complete. 2024-03-26T08:45:32.024 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:45:32.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:45:32.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.192+0000 7fd141bdf640 1 -- 172.21.15.7:0/4290623332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c105250 msgr2=0x7fd13c1056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.192+0000 7fd141bdf640 1 --2- 172.21.15.7:0/4290623332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c105250 0x7fd13c1056d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd134009ca0 tx=0x7fd13402c140 comp rx=0 tx=0).stop 2024-03-26T08:45:32.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.192+0000 7fd141bdf640 1 -- 172.21.15.7:0/4290623332 shutdown_connections 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.192+0000 7fd141bdf640 1 --2- 172.21.15.7:0/4290623332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c105250 0x7fd13c1056d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.192+0000 7fd141bdf640 1 --2- 172.21.15.7:0/4290623332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1028a0 0x7fd13c104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.192+0000 7fd141bdf640 1 --2- 172.21.15.7:0/4290623332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd13c0fff30 0x7fd13c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.192+0000 7fd141bdf640 1 -- 172.21.15.7:0/4290623332 >> 172.21.15.7:0/4290623332 conn(0x7fd13c0fbb30 msgr2=0x7fd13c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.192+0000 7fd141bdf640 1 -- 172.21.15.7:0/4290623332 shutdown_connections 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.192+0000 7fd141bdf640 1 -- 172.21.15.7:0/4290623332 wait complete. 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd141bdf640 1 Processor -- start 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd141bdf640 1 -- start start 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd141bdf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c0fff30 0x7fd13c19e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd141bdf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1028a0 0x7fd13c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd141bdf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd13c105250 0x7fd13c1a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd141bdf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c113a40 con 0x7fd13c105250 2024-03-26T08:45:32.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd141bdf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c1138c0 con 0x7fd13c1028a0 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd141bdf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c113bc0 con 0x7fd13c0fff30 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd13c105250 0x7fd13c1a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c0fff30 0x7fd13c19e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd13c105250 0x7fd13c1a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54368/0 (socket says 172.21.15.7:54368) 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd13bfff640 1 -- 172.21.15.7:0/1807237976 learned_addr learned my addr 172.21.15.7:0/1807237976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c0fff30 0x7fd13c19e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38906/0 (socket says 172.21.15.7:38906) 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.193+0000 7fd132ffd640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1028a0 0x7fd13c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd13bfff640 1 -- 172.21.15.7:0/1807237976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c0fff30 msgr2=0x7fd13c19e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd13bfff640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c0fff30 0x7fd13c19e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd13bfff640 1 -- 172.21.15.7:0/1807237976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1028a0 msgr2=0x7fd13c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd13bfff640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1028a0 0x7fd13c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd13bfff640 1 -- 172.21.15.7:0/1807237976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c1a5d40 con 0x7fd13c105250 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd13bfff640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd13c105250 0x7fd13c1a56d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd134037040 tx=0x7fd134035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd1397fa640 1 -- 172.21.15.7:0/1807237976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd134039040 con 0x7fd13c105250 2024-03-26T08:45:32.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd141bdf640 1 -- 172.21.15.7:0/1807237976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c1a5fd0 con 0x7fd13c105250 2024-03-26T08:45:32.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd1397fa640 1 -- 172.21.15.7:0/1807237976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd134044070 con 0x7fd13c105250 2024-03-26T08:45:32.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd1397fa640 1 -- 172.21.15.7:0/1807237976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd134004710 con 0x7fd13c105250 2024-03-26T08:45:32.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.194+0000 7fd141bdf640 1 -- 172.21.15.7:0/1807237976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c1a6510 con 0x7fd13c105250 2024-03-26T08:45:32.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.197+0000 7fd1397fa640 1 -- 172.21.15.7:0/1807237976 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd134003970 con 0x7fd13c105250 2024-03-26T08:45:32.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.198+0000 7fd1447c6740 1 Processor -- start 2024-03-26T08:45:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.198+0000 7fd1447c6740 1 -- start start 2024-03-26T08:45:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.198+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 0x558cd9b84ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.198+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b85080 0x558cd9b89b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.198+0000 7fd1397fa640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd1280a2880 0x7fd1280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.198+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b8a1f0 0x558cd9b8c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.198+0000 7fd1447c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9b47610 con 0x558cd9b85080 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.198+0000 7fd1447c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9aea3c0 con 0x558cd9b8a1f0 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.198+0000 7fd1447c6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9a92d60 con 0x558cd9b84690 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b8a1f0 0x558cd9b8c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 0x558cd9b84ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b8a1f0 0x558cd9b8c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59434/0 (socket says 172.21.15.7:59434) 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13bfff640 1 -- 172.21.15.7:0/2398287614 learned_addr learned my addr 172.21.15.7:0/2398287614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd1397fa640 1 -- 172.21.15.7:0/1807237976 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd1340f1340 con 0x7fd13c105250 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd132ffd640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd1280a2880 0x7fd1280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13b7fe640 1 --2- 172.21.15.7:0/2398287614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b85080 0x558cd9b89b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13bfff640 1 -- 172.21.15.7:0/2398287614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 msgr2=0x558cd9b84ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13bfff640 1 --2- 172.21.15.7:0/2398287614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 0x558cd9b84ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13bfff640 1 -- 172.21.15.7:0/2398287614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b85080 msgr2=0x558cd9b89b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13bfff640 1 --2- 172.21.15.7:0/2398287614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b85080 0x558cd9b89b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13bfff640 1 -- 172.21.15.7:0/2398287614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd97fe3f0 con 0x558cd9b8a1f0 2024-03-26T08:45:32.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13b7fe640 1 --2- 172.21.15.7:0/2398287614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b85080 0x558cd9b89b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:45:32.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd13bfff640 1 --2- 172.21.15.7:0/2398287614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b8a1f0 0x558cd9b8c680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd134048000 tx=0x7fd13402c140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5666f3f9289c91d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd132ffd640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd1280a2880 0x7fd1280a4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x558cd9b858e0 tx=0x7fd1240090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd131ffb640 1 -- 172.21.15.7:0/2398287614 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340bb330 con 0x558cd9b8a1f0 2024-03-26T08:45:32.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd131ffb640 1 -- 172.21.15.7:0/2398287614 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1340098d0 con 0x558cd9b8a1f0 2024-03-26T08:45:32.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.199+0000 7fd131ffb640 1 -- 172.21.15.7:0/2398287614 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340c4a70 con 0x558cd9b8a1f0 2024-03-26T08:45:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 -- 172.21.15.7:0/2398287614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b8a1f0 msgr2=0x558cd9b8c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2398287614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b8a1f0 0x558cd9b8c680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd134048000 tx=0x7fd13402c140 comp rx=0 tx=0).stop 2024-03-26T08:45:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 -- 172.21.15.7:0/2398287614 shutdown_connections 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2398287614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b8a1f0 0x558cd9b8c680 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2398287614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b85080 0x558cd9b89b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2398287614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 0x558cd9b84ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 -- 172.21.15.7:0/2398287614 >> 172.21.15.7:0/2398287614 conn(0x558cd9b7fc00 msgr2=0x558cd9b80000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 -- 172.21.15.7:0/2398287614 shutdown_connections 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 -- 172.21.15.7:0/2398287614 wait complete. 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.200+0000 7fd1447c6740 1 -- start start 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.201+0000 7fd1447c6740 1 Processor -- start 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.201+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b84690 0x558cd9b9f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.201+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b85080 0x558cd9b9fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.201+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd1447c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9aa6240 con 0x558cd9b8a1f0 2024-03-26T08:45:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd1447c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9ae6460 con 0x558cd9b84690 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd1447c6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd9b9df70 con 0x558cd9b85080 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b84690 0x558cd9b9f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd1447c6740 10 client.0 _ll_register_callbacks cb 0x558cd9aa5c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b85080 0x558cd9b9fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54388/0 (socket says 172.21.15.7:54388) 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b84690 0x558cd9b9f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59444/0 (socket says 172.21.15.7:59444) 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13bfff640 1 -- 172.21.15.7:0/3681544984 learned_addr learned my addr 172.21.15.7:0/3681544984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13bfff640 1 -- 172.21.15.7:0/3681544984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b85080 msgr2=0x558cd9b9fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13bfff640 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b85080 0x558cd9b9fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13bfff640 1 -- 172.21.15.7:0/3681544984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b84690 msgr2=0x558cd9b9f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13bfff640 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b84690 0x558cd9b9f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13bfff640 1 -- 172.21.15.7:0/3681544984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cd98c87d0 con 0x558cd9b8a1f0 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd132ffd640 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b85080 0x558cd9b9fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13b7fe640 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b84690 0x558cd9b9f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd13bfff640 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba6cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd1340c5f50 tx=0x7fd1340eab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f902ed557a4f2cc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd11affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:45:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.202+0000 7fd11affd640 1 -- 172.21.15.7:0/3681544984 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1340bb330 con 0x558cd9b8a1f0 2024-03-26T08:45:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.203+0000 7fd11affd640 1 -- 172.21.15.7:0/3681544984 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd1340ba350 con 0x558cd9b8a1f0 2024-03-26T08:45:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.203+0000 7fd1447c6740 1 -- 172.21.15.7:0/3681544984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cd9af7900 con 0x558cd9b8a1f0 2024-03-26T08:45:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.204+0000 7fd11affd640 1 -- 172.21.15.7:0/3681544984 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1340ec4b0 con 0x558cd9b8a1f0 2024-03-26T08:45:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.204+0000 7fd1447c6740 10 client.16218 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.204+0000 7fd1447c6740 1 -- 172.21.15.7:0/3681544984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558cd9baa040 con 0x558cd9b8a1f0 2024-03-26T08:45:32.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.204+0000 7fd11affd640 1 -- 172.21.15.7:0/3681544984 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1340bdce0 con 0x558cd9b8a1f0 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.205+0000 7fd11affd640 1 -- 172.21.15.7:0/3681544984 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd13403c030 con 0x558cd9b8a1f0 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.205+0000 7fd1447c6740 10 client.16218 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.205+0000 7fd1447c6740 10 client.16218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.205+0000 7fd1447c6740 20 client.16218 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.205+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558cd9bab4c0 0x558cd9bad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.205+0000 7fd1447c6740 4 client.16218 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.205+0000 7fd1447c6740 1 -- 172.21.15.7:0/3681544984 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cd9af7900 con 0x558cd9bab4c0 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.205+0000 7fd13b7fe640 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558cd9bab4c0 0x558cd9bad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.206+0000 7fd13b7fe640 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558cd9bab4c0 0x558cd9bad8b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d14c9ca1344f10b7 server_cookie=c77a0380991321bf in_seq=0 out_seq=0 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.206+0000 7fd11affd640 10 client.16218 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.207+0000 7fd11affd640 1 -- 172.21.15.7:0/3681544984 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558cd9af7900 con 0x558cd9bab4c0 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.207+0000 7fd11affd640 10 client.16218 handle_command_reply: tid=0 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.207+0000 7fd1447c6740 1 client.16218 shutdown 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.207+0000 7fd1447c6740 1 -- 172.21.15.7:0/3681544984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 msgr2=0x558cd9ba6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.207+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba6cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd1340c5f50 tx=0x7fd1340eab50 comp rx=0 tx=0).stop 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd1447c6740 1 -- 172.21.15.7:0/3681544984 shutdown_connections 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558cd9bab4c0 0x558cd9bad8b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba6cd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b85080 0x558cd9b9fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3681544984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b84690 0x558cd9b9f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd11affd640 0 client.16218 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd1447c6740 1 -- 172.21.15.7:0/3681544984 >> 172.21.15.7:0/3681544984 conn(0x558cd9b7fc00 msgr2=0x558cd9b7ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd1447c6740 1 -- 172.21.15.7:0/3681544984 shutdown_connections 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd1447c6740 1 -- 172.21.15.7:0/3681544984 wait complete. 2024-03-26T08:45:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.208+0000 7fd1447c6740 20 client.16218 trim_cache size 0 max 16384 2024-03-26T08:45:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.225+0000 7fd1447c6740 1 Processor -- start 2024-03-26T08:45:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.225+0000 7fd1447c6740 1 -- start start 2024-03-26T08:45:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.225+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b84690 0x558cd9b83570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.225+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.225+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b8a1f0 0x558cd9bb2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.225+0000 7fd1447c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c1a8ec0 con 0x558cd9b84690 2024-03-26T08:45:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.225+0000 7fd1447c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c1a69c0 con 0x558cd9b85080 2024-03-26T08:45:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.225+0000 7fd1447c6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c10cf90 con 0x558cd9b8a1f0 2024-03-26T08:45:32.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b84690 0x558cd9b83570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd13b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b84690 0x558cd9b83570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54404/0 (socket says 172.21.15.7:54404) 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b8a1f0 0x558cd9bb2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd13b7fe640 1 -- 172.21.15.7:0/3313138158 learned_addr learned my addr 172.21.15.7:0/3313138158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd132ffd640 1 -- 172.21.15.7:0/3313138158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b8a1f0 msgr2=0x558cd9bb2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd132ffd640 1 --2- 172.21.15.7:0/3313138158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b8a1f0 0x558cd9bb2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd132ffd640 1 -- 172.21.15.7:0/3313138158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b84690 msgr2=0x558cd9b83570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd132ffd640 1 --2- 172.21.15.7:0/3313138158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b84690 0x558cd9b83570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd132ffd640 1 -- 172.21.15.7:0/3313138158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd9b7b480 con 0x558cd9b85080 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd13b7fe640 1 --2- 172.21.15.7:0/3313138158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b84690 0x558cd9b83570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:32.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd132ffd640 1 --2- 172.21.15.7:0/3313138158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba6c90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd124009fd0 tx=0x7fd124009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=995b0bff6b779f34 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.226+0000 7fd1317fa640 1 -- 172.21.15.7:0/3313138158 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1240187e0 con 0x558cd9b85080 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.227+0000 7fd1317fa640 1 -- 172.21.15.7:0/3313138158 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd12401c040 con 0x558cd9b85080 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.227+0000 7fd1317fa640 1 -- 172.21.15.7:0/3313138158 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124022e00 con 0x558cd9b85080 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.227+0000 7fd1447c6740 1 -- 172.21.15.7:0/3313138158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 msgr2=0x558cd9ba6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.227+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3313138158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba6c90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd124009fd0 tx=0x7fd124009480 comp rx=0 tx=0).stop 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.228+0000 7fd1447c6740 1 -- 172.21.15.7:0/3313138158 shutdown_connections 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.228+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3313138158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b8a1f0 0x558cd9bb2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.228+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3313138158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba6c90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.228+0000 7fd1447c6740 1 --2- 172.21.15.7:0/3313138158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b84690 0x558cd9b83570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.228+0000 7fd1447c6740 1 -- 172.21.15.7:0/3313138158 >> 172.21.15.7:0/3313138158 conn(0x558cd9b7fc00 msgr2=0x558cd9b82e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.228+0000 7fd1447c6740 1 -- 172.21.15.7:0/3313138158 shutdown_connections 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.228+0000 7fd1447c6740 1 -- 172.21.15.7:0/3313138158 wait complete. 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.228+0000 7fd1447c6740 1 -- start start 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.228+0000 7fd1447c6740 1 Processor -- start 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 0x558cd9ba47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd1447c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd1447c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c10d590 con 0x558cd9b8a1f0 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd1447c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c10e8b0 con 0x558cd9b85080 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd1447c6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c10dbf0 con 0x558cd9b84690 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59458/0 (socket says 172.21.15.7:59458) 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd132ffd640 1 -- 172.21.15.7:0/2186501430 learned_addr learned my addr 172.21.15.7:0/2186501430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd1447c6740 10 client.0 _ll_register_callbacks cb 0x7fd1340bd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd13b7fe640 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 0x558cd9ba47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.229+0000 7fd13bfff640 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.230+0000 7fd132ffd640 1 -- 172.21.15.7:0/2186501430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 msgr2=0x558cd9ba47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.230+0000 7fd132ffd640 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 0x558cd9ba47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.230+0000 7fd132ffd640 1 -- 172.21.15.7:0/2186501430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 msgr2=0x558cd9ba5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.230+0000 7fd132ffd640 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.230+0000 7fd132ffd640 1 -- 172.21.15.7:0/2186501430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cd9ba6320 con 0x558cd9b85080 2024-03-26T08:45:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.231+0000 7fd132ffd640 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba4d20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd124015fd0 tx=0x7fd124005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95021157a17b2e2e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.231+0000 7fd11affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.231+0000 7fd11affd640 1 -- 172.21.15.7:0/2186501430 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124030070 con 0x558cd9b85080 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.231+0000 7fd11affd640 1 -- 172.21.15.7:0/2186501430 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd124018bf0 con 0x558cd9b85080 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.231+0000 7fd1447c6740 1 -- 172.21.15.7:0/2186501430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd12401c040 con 0x558cd9b85080 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.231+0000 7fd11affd640 1 -- 172.21.15.7:0/2186501430 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd124005e60 con 0x558cd9b85080 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.231+0000 7fd1447c6740 10 client.26066 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.231+0000 7fd1447c6740 1 -- 172.21.15.7:0/2186501430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558cd9c071c0 con 0x558cd9b85080 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd11affd640 1 -- 172.21.15.7:0/2186501430 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd12401c040 con 0x558cd9b85080 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd11affd640 1 -- 172.21.15.7:0/2186501430 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd124021040 con 0x558cd9b85080 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd1447c6740 10 client.26066 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd1447c6740 10 client.26066 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd1447c6740 20 client.26066 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558cd9c07980 0x558cd9c09da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd1447c6740 4 client.26066 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd1447c6740 1 -- 172.21.15.7:0/2186501430 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558cd9af1c50 con 0x558cd9c07980 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd13b7fe640 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558cd9c07980 0x558cd9c09da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd11affd640 10 client.26066 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.232+0000 7fd13b7fe640 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558cd9c07980 0x558cd9c09da0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b8e9768eeb3e3dd server_cookie=7bf2ee9a2453cf8c in_seq=0 out_seq=0 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd11affd640 1 -- 172.21.15.7:0/2186501430 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558cd9af1c50 con 0x558cd9c07980 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd11affd640 10 client.26066 handle_command_reply: tid=0 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 client.26066 shutdown 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 -- 172.21.15.7:0/2186501430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 msgr2=0x558cd9ba4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba4d20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd124015fd0 tx=0x7fd124005c40 comp rx=0 tx=0).stop 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 -- 172.21.15.7:0/2186501430 shutdown_connections 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558cd9c07980 0x558cd9c09da0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558cd9b8a1f0 0x558cd9ba5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558cd9b85080 0x558cd9ba4d20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd11affd640 0 client.26066 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:45:32.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 --2- 172.21.15.7:0/2186501430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558cd9b84690 0x558cd9ba47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 -- 172.21.15.7:0/2186501430 >> 172.21.15.7:0/2186501430 conn(0x558cd9b7fc00 msgr2=0x558cd9b836f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 -- 172.21.15.7:0/2186501430 shutdown_connections 2024-03-26T08:45:32.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 1 -- 172.21.15.7:0/2186501430 wait complete. 2024-03-26T08:45:32.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.233+0000 7fd1447c6740 20 client.26066 trim_cache size 0 max 16384 2024-03-26T08:45:32.229 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:45:32.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 -- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd1280a2880 msgr2=0x7fd1280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd1280a2880 0x7fd1280a4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x558cd9b858e0 tx=0x7fd1240090e0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 -- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd13c105250 msgr2=0x7fd13c1a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd13c105250 0x7fd13c1a56d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd134037040 tx=0x7fd134035d90 comp rx=0 tx=0).stop 2024-03-26T08:45:32.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 -- 172.21.15.7:0/1807237976 shutdown_connections 2024-03-26T08:45:32.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd1280a2880 0x7fd1280a4d40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd13c105250 0x7fd13c1a56d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd13c1028a0 0x7fd13c19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 --2- 172.21.15.7:0/1807237976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd13c0fff30 0x7fd13c19e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 -- 172.21.15.7:0/1807237976 >> 172.21.15.7:0/1807237976 conn(0x7fd13c0fbb30 msgr2=0x7fd13c1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 -- 172.21.15.7:0/1807237976 shutdown_connections 2024-03-26T08:45:32.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.234+0000 7fd141bdf640 1 -- 172.21.15.7:0/1807237976 wait complete. 2024-03-26T08:45:32.239 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:45:32.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:45:32.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:32 smithi007 ceph-mon[32462]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 722 op/s 2024-03-26T08:45:32.291 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:32 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3971494962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:45:32.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.398+0000 7fc263a0b640 1 -- 172.21.15.7:0/3432627137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 msgr2=0x7fc25c108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.398+0000 7fc263a0b640 1 --2- 172.21.15.7:0/3432627137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 0x7fc25c108ad0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc25400bda0 tx=0x7fc25402dc50 comp rx=0 tx=0).stop 2024-03-26T08:45:32.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.398+0000 7fc263a0b640 1 -- 172.21.15.7:0/3432627137 shutdown_connections 2024-03-26T08:45:32.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.398+0000 7fc263a0b640 1 --2- 172.21.15.7:0/3432627137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 0x7fc25c108ad0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.398+0000 7fc263a0b640 1 --2- 172.21.15.7:0/3432627137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc25c100870 0x7fc25c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.398+0000 7fc263a0b640 1 --2- 172.21.15.7:0/3432627137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c101bd0 0x7fc25c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.398+0000 7fc263a0b640 1 -- 172.21.15.7:0/3432627137 >> 172.21.15.7:0/3432627137 conn(0x7fc25c0fb2c0 msgr2=0x7fc25c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.398+0000 7fc263a0b640 1 -- 172.21.15.7:0/3432627137 shutdown_connections 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.398+0000 7fc263a0b640 1 -- 172.21.15.7:0/3432627137 wait complete. 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc263a0b640 1 Processor -- start 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc263a0b640 1 -- start start 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc263a0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c100870 0x7fc25c195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc263a0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 0x7fc25c195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc263a0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc25c101bd0 0x7fc25c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc263a0b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c10b160 con 0x7fc25c101230 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc263a0b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c10afe0 con 0x7fc25c100870 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc263a0b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c10b2e0 con 0x7fc25c101bd0 2024-03-26T08:45:32.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc260fa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 0x7fc25c195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc260fa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 0x7fc25c195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54434/0 (socket says 172.21.15.7:54434) 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc260fa6640 1 -- 172.21.15.7:0/2828426710 learned_addr learned my addr 172.21.15.7:0/2828426710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc261fa8640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc25c101bd0 0x7fc25c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc2617a7640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c100870 0x7fc25c195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc260fa6640 1 -- 172.21.15.7:0/2828426710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc25c101bd0 msgr2=0x7fc25c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc260fa6640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc25c101bd0 0x7fc25c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc260fa6640 1 -- 172.21.15.7:0/2828426710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c100870 msgr2=0x7fc25c195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc260fa6640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c100870 0x7fc25c195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc260fa6640 1 -- 172.21.15.7:0/2828426710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c19d380 con 0x7fc25c101230 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc2617a7640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c100870 0x7fc25c195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:32.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.399+0000 7fc260fa6640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 0x7fc25c195ca0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00e9c0 tx=0x7fc24c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.400+0000 7fc2527fc640 1 -- 172.21.15.7:0/2828426710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c00ccc0 con 0x7fc25c101230 2024-03-26T08:45:32.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.400+0000 7fc263a0b640 1 -- 172.21.15.7:0/2828426710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c19d670 con 0x7fc25c101230 2024-03-26T08:45:32.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.400+0000 7fc2527fc640 1 -- 172.21.15.7:0/2828426710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc24c005500 con 0x7fc25c101230 2024-03-26T08:45:32.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.400+0000 7fc2527fc640 1 -- 172.21.15.7:0/2828426710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c010430 con 0x7fc25c101230 2024-03-26T08:45:32.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.400+0000 7fc263a0b640 1 -- 172.21.15.7:0/2828426710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c19dbc0 con 0x7fc25c101230 2024-03-26T08:45:32.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.403+0000 7fc2665fa740 1 Processor -- start 2024-03-26T08:45:32.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc2527fc640 1 -- 172.21.15.7:0/2828426710 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc24c005020 con 0x7fc25c101230 2024-03-26T08:45:32.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc2665fa740 1 -- start start 2024-03-26T08:45:32.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576ae78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576aec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 0x558576aeeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc2665fa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558576aa1610 con 0x558576aeca80 2024-03-26T08:45:32.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc2665fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558576a443c0 con 0x558576ae7e80 2024-03-26T08:45:32.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc2665fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585769ecd60 con 0x558576ae7490 2024-03-26T08:45:32.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc261fa8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576ae78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc2617a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576aec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc260fa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 0x558576aeeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.404+0000 7fc261fa8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576ae78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38952/0 (socket says 172.21.15.7:38952) 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc261fa8640 1 -- 172.21.15.7:0/881737660 learned_addr learned my addr 172.21.15.7:0/881737660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2527fc640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2480a2880 0x7fc2480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2617a7640 1 -- 172.21.15.7:0/881737660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 msgr2=0x558576ae78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2617a7640 1 --2- 172.21.15.7:0/881737660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576ae78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2617a7640 1 -- 172.21.15.7:0/881737660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 msgr2=0x558576aeeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2617a7640 1 --2- 172.21.15.7:0/881737660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 0x558576aeeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2617a7640 1 -- 172.21.15.7:0/881737660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585767583f0 con 0x558576ae7e80 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc261fa8640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2480a2880 0x7fc2480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2527fc640 1 -- 172.21.15.7:0/2828426710 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc24c0ced70 con 0x7fc25c101230 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2617a7640 1 --2- 172.21.15.7:0/881737660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576aec410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc258009fa0 tx=0x7fc2580118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383d418e1a415a08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2477fe640 1 -- 172.21.15.7:0/881737660 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258004310 con 0x558576ae7e80 2024-03-26T08:45:32.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2477fe640 1 -- 172.21.15.7:0/881737660 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2580044b0 con 0x558576ae7e80 2024-03-26T08:45:32.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.405+0000 7fc2477fe640 1 -- 172.21.15.7:0/881737660 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2580037f0 con 0x558576ae7e80 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 -- 172.21.15.7:0/881737660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 msgr2=0x558576aec410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc261fa8640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2480a2880 0x7fc2480a4d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x558576add840 tx=0x7fc25400b710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 --2- 172.21.15.7:0/881737660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576aec410 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc258009fa0 tx=0x7fc2580118d0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 -- 172.21.15.7:0/881737660 shutdown_connections 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 --2- 172.21.15.7:0/881737660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 0x558576aeeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 --2- 172.21.15.7:0/881737660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576aec410 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 --2- 172.21.15.7:0/881737660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576ae78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 -- 172.21.15.7:0/881737660 >> 172.21.15.7:0/881737660 conn(0x558576ad9b80 msgr2=0x558576ad9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 -- 172.21.15.7:0/881737660 shutdown_connections 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 -- 172.21.15.7:0/881737660 wait complete. 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.406+0000 7fc2665fa740 1 -- start start 2024-03-26T08:45:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.407+0000 7fc2665fa740 1 Processor -- start 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.407+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.407+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576af17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 0x558576af8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc2665fa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558576a00240 con 0x558576aeca80 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc2665fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558576a40460 con 0x558576ae7e80 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc2665fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558576aefa10 con 0x558576ae7490 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc2665fa740 10 client.0 _ll_register_callbacks cb 0x5585769ffc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc260fa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576af17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc2617a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc260fa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576af17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59482/0 (socket says 172.21.15.7:59482) 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc260fa6640 1 -- 172.21.15.7:0/353441499 learned_addr learned my addr 172.21.15.7:0/353441499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc261fa8640 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 0x558576af8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc260fa6640 1 -- 172.21.15.7:0/353441499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 msgr2=0x558576af1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc260fa6640 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc260fa6640 1 -- 172.21.15.7:0/353441499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 msgr2=0x558576af8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc260fa6640 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 0x558576af8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc260fa6640 1 -- 172.21.15.7:0/353441499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585768227d0 con 0x558576ae7e80 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc260fa6640 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576af17d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc24c09b660 tx=0x7fc24c0972b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26ce2da6b9278ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc244ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc244ff9640 1 -- 172.21.15.7:0/353441499 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c09b030 con 0x558576ae7e80 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc2665fa740 1 -- 172.21.15.7:0/353441499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2580044b0 con 0x558576ae7e80 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.408+0000 7fc244ff9640 1 -- 172.21.15.7:0/353441499 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc24c0999d0 con 0x558576ae7e80 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc244ff9640 1 -- 172.21.15.7:0/353441499 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc24c0056b0 con 0x558576ae7e80 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc2665fa740 10 client.26078 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc2665fa740 1 -- 172.21.15.7:0/353441499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558576afbc40 con 0x558576ae7e80 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc244ff9640 1 -- 172.21.15.7:0/353441499 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2580044b0 con 0x558576ae7e80 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc244ff9640 1 -- 172.21.15.7:0/353441499 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc24c0d6040 con 0x558576ae7e80 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc2665fa740 10 client.26078 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc2665fa740 10 client.26078 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc2665fa740 20 client.26078 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc2665fa740 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558576afd700 0x558576affb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc2665fa740 4 client.26078 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc2665fa740 1 -- 172.21.15.7:0/353441499 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558576a51900 con 0x558576afd700 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.409+0000 7fc2617a7640 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558576afd700 0x558576affb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2617a7640 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558576afd700 0x558576affb40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=492ddce71f4d616f server_cookie=34af22dcd978eae in_seq=0 out_seq=0 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc244ff9640 10 client.26078 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc244ff9640 1 -- 172.21.15.7:0/353441499 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558576a51900 con 0x558576afd700 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc244ff9640 10 client.26078 handle_command_reply: tid=0 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 client.26078 shutdown 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 -- 172.21.15.7:0/353441499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 msgr2=0x558576af17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576af17d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc24c09b660 tx=0x7fc24c0972b0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 -- 172.21.15.7:0/353441499 shutdown_connections 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576aeca80 0x558576af8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576af17d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558576afd700 0x558576affb40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 --2- 172.21.15.7:0/353441499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 -- 172.21.15.7:0/353441499 >> 172.21.15.7:0/353441499 conn(0x558576ad9b80 msgr2=0x558576ae6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc244ff9640 0 client.26078 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:45:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.410+0000 7fc2665fa740 1 -- 172.21.15.7:0/353441499 shutdown_connections 2024-03-26T08:45:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.411+0000 7fc2665fa740 1 -- 172.21.15.7:0/353441499 wait complete. 2024-03-26T08:45:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.411+0000 7fc2665fa740 20 client.26078 trim_cache size 0 max 16384 2024-03-26T08:45:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2665fa740 1 Processor -- start 2024-03-26T08:45:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2665fa740 1 -- start start 2024-03-26T08:45:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7490 0x558576ae8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576ae93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576aeca80 0x558576af87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2665fa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1a8e80 con 0x558576ae7490 2024-03-26T08:45:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2665fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1a6980 con 0x558576ae7e80 2024-03-26T08:45:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2665fa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1a4a00 con 0x558576aeca80 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2617a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7490 0x558576ae8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc260fa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576ae93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc261fa8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576aeca80 0x558576af87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc260fa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576ae93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59488/0 (socket says 172.21.15.7:59488) 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2617a7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7490 0x558576ae8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54466/0 (socket says 172.21.15.7:54466) 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2617a7640 1 -- 172.21.15.7:0/323230665 learned_addr learned my addr 172.21.15.7:0/323230665 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc261fa8640 1 -- 172.21.15.7:0/323230665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 msgr2=0x558576ae93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc261fa8640 1 --2- 172.21.15.7:0/323230665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576ae93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc261fa8640 1 -- 172.21.15.7:0/323230665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7490 msgr2=0x558576ae8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc261fa8640 1 --2- 172.21.15.7:0/323230665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7490 0x558576ae8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc261fa8640 1 -- 172.21.15.7:0/323230665 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558576ada8f0 con 0x558576aeca80 2024-03-26T08:45:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc2617a7640 1 --2- 172.21.15.7:0/323230665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7490 0x558576ae8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:32.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.429+0000 7fc260fa6640 1 --2- 172.21.15.7:0/323230665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576ae93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:32.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.430+0000 7fc261fa8640 1 --2- 172.21.15.7:0/323230665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576aeca80 0x558576af87d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc254036c20 tx=0x7fc254042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc3b5c1369863d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.430+0000 7fc246ffd640 1 -- 172.21.15.7:0/323230665 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254054070 con 0x558576aeca80 2024-03-26T08:45:32.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.430+0000 7fc246ffd640 1 -- 172.21.15.7:0/323230665 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc254003e50 con 0x558576aeca80 2024-03-26T08:45:32.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.430+0000 7fc246ffd640 1 -- 172.21.15.7:0/323230665 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254042450 con 0x558576aeca80 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 -- 172.21.15.7:0/323230665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576aeca80 msgr2=0x558576af87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 --2- 172.21.15.7:0/323230665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576aeca80 0x558576af87d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc254036c20 tx=0x7fc254042000 comp rx=0 tx=0).stop 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 -- 172.21.15.7:0/323230665 shutdown_connections 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 --2- 172.21.15.7:0/323230665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576aeca80 0x558576af87d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 --2- 172.21.15.7:0/323230665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576ae7e80 0x558576ae93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 --2- 172.21.15.7:0/323230665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7490 0x558576ae8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 -- 172.21.15.7:0/323230665 >> 172.21.15.7:0/323230665 conn(0x558576ad9b80 msgr2=0x558576aefd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 -- 172.21.15.7:0/323230665 shutdown_connections 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 -- 172.21.15.7:0/323230665 wait complete. 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 -- start start 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.431+0000 7fc2665fa740 1 Processor -- start 2024-03-26T08:45:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7e80 0x558576af6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2665fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576aeca80 0x558576af75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2617a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2617a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38972/0 (socket says 172.21.15.7:38972) 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc260fa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7e80 0x558576af6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc260fa6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7e80 0x558576af6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54480/0 (socket says 172.21.15.7:54480) 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2617a7640 1 -- 172.21.15.7:0/2083471353 learned_addr learned my addr 172.21.15.7:0/2083471353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc261fa8640 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576aeca80 0x558576af75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1a5000 con 0x558576ae7e80 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1a6320 con 0x558576aeca80 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1a5660 con 0x558576ae7490 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.432+0000 7fc2665fa740 10 client.0 _ll_register_callbacks cb 0x7fc24c0d2420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc2617a7640 1 -- 172.21.15.7:0/2083471353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576aeca80 msgr2=0x558576af75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc2617a7640 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576aeca80 0x558576af75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc2617a7640 1 -- 172.21.15.7:0/2083471353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7e80 msgr2=0x558576af6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc2617a7640 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7e80 0x558576af6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc2617a7640 1 -- 172.21.15.7:0/2083471353 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558576af7bc0 con 0x558576ae7490 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc2617a7640 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af60d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc258005fd0 tx=0x7fc258009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b70707cb8113b67 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc244ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc244ff9640 1 -- 172.21.15.7:0/2083471353 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25801d040 con 0x558576ae7490 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc244ff9640 1 -- 172.21.15.7:0/2083471353 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc25802b1a0 con 0x558576ae7490 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.433+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc254003e50 con 0x558576ae7490 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.434+0000 7fc244ff9640 1 -- 172.21.15.7:0/2083471353 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc258039050 con 0x558576ae7490 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.434+0000 7fc2665fa740 10 client.26107 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.434+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558576b591b0 con 0x558576ae7490 2024-03-26T08:45:32.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.435+0000 7fc244ff9640 1 -- 172.21.15.7:0/2083471353 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc25802f070 con 0x558576ae7490 2024-03-26T08:45:32.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.435+0000 7fc244ff9640 1 -- 172.21.15.7:0/2083471353 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc25802b520 con 0x558576ae7490 2024-03-26T08:45:32.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.435+0000 7fc2665fa740 10 client.26107 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:32.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.435+0000 7fc2665fa740 10 client.26107 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:45:32.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.435+0000 7fc2665fa740 20 client.26107 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:32.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.435+0000 7fc2665fa740 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558576b59980 0x558576b5bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.435+0000 7fc2665fa740 4 client.26107 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:45:32.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.435+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fc254003e50 con 0x558576b59980 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.435+0000 7fc260fa6640 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558576b59980 0x558576b5bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.436+0000 7fc244ff9640 10 client.26107 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.436+0000 7fc260fa6640 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558576b59980 0x558576b5bda0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7e5dc47282cbaef server_cookie=c69d2032d0e88250 in_seq=0 out_seq=0 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.436+0000 7fc244ff9640 1 -- 172.21.15.7:0/2083471353 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fc254003e50 con 0x558576b59980 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.436+0000 7fc244ff9640 10 client.26107 handle_command_reply: tid=0 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.436+0000 7fc2665fa740 1 client.26107 shutdown 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.437+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 msgr2=0x558576af60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.437+0000 7fc2665fa740 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af60d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc258005fd0 tx=0x7fc258009310 comp rx=0 tx=0).stop 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.437+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 shutdown_connections 2024-03-26T08:45:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.437+0000 7fc2665fa740 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558576aeca80 0x558576af75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.437+0000 7fc2665fa740 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558576ae7e80 0x558576af6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.437+0000 7fc2665fa740 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x558576b59980 0x558576b5bda0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.437+0000 7fc2665fa740 1 --2- 172.21.15.7:0/2083471353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558576ae7490 0x558576af60d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.437+0000 7fc244ff9640 0 client.26107 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:45:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.437+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 >> 172.21.15.7:0/2083471353 conn(0x558576ad9b80 msgr2=0x558576af1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 shutdown_connections 2024-03-26T08:45:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc2665fa740 1 -- 172.21.15.7:0/2083471353 wait complete. 2024-03-26T08:45:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc2665fa740 20 client.26107 trim_cache size 0 max 16384 2024-03-26T08:45:32.432 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 -- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2480a2880 msgr2=0x7fc2480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2480a2880 0x7fc2480a4d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x558576add840 tx=0x7fc25400b710 comp rx=0 tx=0).stop 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 -- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 msgr2=0x7fc25c195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 0x7fc25c195ca0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00e9c0 tx=0x7fc24c00ee80 comp rx=0 tx=0).stop 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 -- 172.21.15.7:0/2828426710 shutdown_connections 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2480a2880 0x7fc2480a4d40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc25c101bd0 0x7fc25c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc25c101230 0x7fc25c195ca0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 --2- 172.21.15.7:0/2828426710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc25c100870 0x7fc25c195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.438+0000 7fc263a0b640 1 -- 172.21.15.7:0/2828426710 >> 172.21.15.7:0/2828426710 conn(0x7fc25c0fb2c0 msgr2=0x7fc25c104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.440+0000 7fc263a0b640 1 -- 172.21.15.7:0/2828426710 shutdown_connections 2024-03-26T08:45:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.440+0000 7fc263a0b640 1 -- 172.21.15.7:0/2828426710 wait complete. 2024-03-26T08:45:32.445 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:45:32.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:45:32.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:32 smithi016 ceph-mon[35851]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 722 op/s 2024-03-26T08:45:32.494 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:32 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3971494962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:45:32.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:32 smithi044 ceph-mon[36461]: pgmap v453: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 10 MiB/s wr, 722 op/s 2024-03-26T08:45:32.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:32 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3971494962' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:45:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.617+0000 7efc19cdc640 1 -- 172.21.15.7:0/2088446482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc141028c0 msgr2=0x7efc14102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.617+0000 7efc19cdc640 1 --2- 172.21.15.7:0/2088446482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc141028c0 0x7efc14102d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc04009900 tx=0x7efc0402bc40 comp rx=0 tx=0).stop 2024-03-26T08:45:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.617+0000 7efc19cdc640 1 -- 172.21.15.7:0/2088446482 shutdown_connections 2024-03-26T08:45:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.617+0000 7efc19cdc640 1 --2- 172.21.15.7:0/2088446482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc14103280 0x7efc14109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.617+0000 7efc19cdc640 1 --2- 172.21.15.7:0/2088446482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc141028c0 0x7efc14102d40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.617+0000 7efc19cdc640 1 --2- 172.21.15.7:0/2088446482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc141016d0 0x7efc14101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.617+0000 7efc19cdc640 1 -- 172.21.15.7:0/2088446482 >> 172.21.15.7:0/2088446482 conn(0x7efc140fcc60 msgr2=0x7efc140ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.617+0000 7efc19cdc640 1 -- 172.21.15.7:0/2088446482 shutdown_connections 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.617+0000 7efc19cdc640 1 -- 172.21.15.7:0/2088446482 wait complete. 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.618+0000 7efc19cdc640 1 Processor -- start 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.618+0000 7efc19cdc640 1 -- start start 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.618+0000 7efc19cdc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc141016d0 0x7efc141957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.618+0000 7efc19cdc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc141028c0 0x7efc14195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.618+0000 7efc19cdc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc14103280 0x7efc1419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.618+0000 7efc19cdc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1410b170 con 0x7efc14103280 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.618+0000 7efc19cdc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1410aff0 con 0x7efc141028c0 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.618+0000 7efc19cdc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1410b2f0 con 0x7efc141016d0 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc137fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc141016d0 0x7efc141957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc13fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc14103280 0x7efc1419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc141028c0 0x7efc14195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc141028c0 0x7efc14195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59530/0 (socket says 172.21.15.7:59530) 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc137fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc141016d0 0x7efc141957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38998/0 (socket says 172.21.15.7:38998) 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc12ffd640 1 -- 172.21.15.7:0/3779007612 learned_addr learned my addr 172.21.15.7:0/3779007612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc12ffd640 1 -- 172.21.15.7:0/3779007612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc141016d0 msgr2=0x7efc141957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc12ffd640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc141016d0 0x7efc141957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc12ffd640 1 -- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc14103280 msgr2=0x7efc1419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc12ffd640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc14103280 0x7efc1419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc12ffd640 1 -- 172.21.15.7:0/3779007612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1419d410 con 0x7efc141028c0 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc13fff640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc14103280 0x7efc1419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc12ffd640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc141028c0 0x7efc14195d30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc04009230 tx=0x7efc0402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc10ff9640 1 -- 172.21.15.7:0/3779007612 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04041070 con 0x7efc141028c0 2024-03-26T08:45:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.619+0000 7efc10ff9640 1 -- 172.21.15.7:0/3779007612 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc04004450 con 0x7efc141028c0 2024-03-26T08:45:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.620+0000 7efc10ff9640 1 -- 172.21.15.7:0/3779007612 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc04003d90 con 0x7efc141028c0 2024-03-26T08:45:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.620+0000 7efc19cdc640 1 -- 172.21.15.7:0/3779007612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1419d6a0 con 0x7efc141028c0 2024-03-26T08:45:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.620+0000 7efc19cdc640 1 -- 172.21.15.7:0/3779007612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1419db90 con 0x7efc141028c0 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.623+0000 7efc1c9a3740 1 Processor -- start 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.623+0000 7efc1c9a3740 1 -- start start 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc10ff9640 1 -- 172.21.15.7:0/3779007612 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7efc04044070 con 0x7efc141028c0 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b6690 0x55ae464b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b7080 0x55ae464bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bc1f0 0x55ae464be680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc1c9a3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae46479610 con 0x55ae464bc1f0 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc1c9a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae4641c3c0 con 0x55ae464b7080 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc1c9a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae463c4d60 con 0x55ae464b6690 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b7080 0x55ae464bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b7080 0x55ae464bbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59538/0 (socket says 172.21.15.7:59538) 2024-03-26T08:45:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc137fe640 1 -- 172.21.15.7:0/2958781417 learned_addr learned my addr 172.21.15.7:0/2958781417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc137fe640 1 -- 172.21.15.7:0/2958781417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b6690 msgr2=0x55ae464b6ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc137fe640 1 --2- 172.21.15.7:0/2958781417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b6690 0x55ae464b6ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc137fe640 1 -- 172.21.15.7:0/2958781417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bc1f0 msgr2=0x55ae464be680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc137fe640 1 --2- 172.21.15.7:0/2958781417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bc1f0 0x55ae464be680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc10ff9640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efbfc0a2950 0x7efbfc0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.624+0000 7efc137fe640 1 -- 172.21.15.7:0/2958781417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae461303f0 con 0x55ae464b7080 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.625+0000 7efc10ff9640 1 -- 172.21.15.7:0/3779007612 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efc040f0590 con 0x7efc141028c0 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.625+0000 7efc137fe640 1 --2- 172.21.15.7:0/2958781417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b7080 0x55ae464bbb80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc0c011470 tx=0x7efc0c011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f35b983e3af29c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.625+0000 7efc13fff640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efbfc0a2950 0x7efbfc0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.625+0000 7efbf9ffb640 1 -- 172.21.15.7:0/2958781417 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c003970 con 0x55ae464b7080 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.625+0000 7efbf9ffb640 1 -- 172.21.15.7:0/2958781417 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc0c004020 con 0x55ae464b7080 2024-03-26T08:45:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.625+0000 7efbf9ffb640 1 -- 172.21.15.7:0/2958781417 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c00f900 con 0x55ae464b7080 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.626+0000 7efc1c9a3740 1 -- 172.21.15.7:0/2958781417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b7080 msgr2=0x55ae464bbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.626+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/2958781417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b7080 0x55ae464bbb80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc0c011470 tx=0x7efc0c011930 comp rx=0 tx=0).stop 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.626+0000 7efc1c9a3740 1 -- 172.21.15.7:0/2958781417 shutdown_connections 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.626+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/2958781417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bc1f0 0x55ae464be680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.626+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/2958781417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b7080 0x55ae464bbb80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.626+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/2958781417 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b6690 0x55ae464b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.626+0000 7efc1c9a3740 1 -- 172.21.15.7:0/2958781417 >> 172.21.15.7:0/2958781417 conn(0x55ae464b1c00 msgr2=0x55ae464b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.626+0000 7efc1c9a3740 1 -- 172.21.15.7:0/2958781417 shutdown_connections 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.627+0000 7efc13fff640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efbfc0a2950 0x7efbfc0a4e10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efc0800da00 tx=0x7efc08018040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.627+0000 7efc1c9a3740 1 -- 172.21.15.7:0/2958781417 wait complete. 2024-03-26T08:45:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.627+0000 7efc1c9a3740 1 -- start start 2024-03-26T08:45:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.627+0000 7efc1c9a3740 1 Processor -- start 2024-03-26T08:45:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b6690 0x55ae464c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464b7080 0x55ae464c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464bc1f0 0x55ae464c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc12ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464b7080 0x55ae464c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b6690 0x55ae464c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc12ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464b7080 0x55ae464c0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54510/0 (socket says 172.21.15.7:54510) 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc12ffd640 1 -- 172.21.15.7:0/3452562532 learned_addr learned my addr 172.21.15.7:0/3452562532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae463d8240 con 0x55ae464b7080 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae46418460 con 0x55ae464b6690 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae464bf1a0 con 0x55ae464bc1f0 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc1c9a3740 10 client.0 _ll_register_callbacks cb 0x55ae463d7c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.628+0000 7efc137fe640 1 -- 172.21.15.7:0/3452562532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464bc1f0 msgr2=0x55ae464c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efc13fff640 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464bc1f0 0x55ae464c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efc137fe640 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464bc1f0 0x55ae464c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efc137fe640 1 -- 172.21.15.7:0/3452562532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464b7080 msgr2=0x55ae464c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efc137fe640 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464b7080 0x55ae464c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efc137fe640 1 -- 172.21.15.7:0/3452562532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae461fa7d0 con 0x55ae464b6690 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efc12ffd640 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464b7080 0x55ae464c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:45:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efc13fff640 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464bc1f0 0x55ae464c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc0c004020 con 0x55ae464b6690 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efc137fe640 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b6690 0x55ae464c0940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ae464b2b70 tx=0x7efc0c009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd8a5a1d1639236 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efbf2ffd640 10 client.26096 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efbf2ffd640 1 -- 172.21.15.7:0/3452562532 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c028020 con 0x55ae464b6690 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.629+0000 7efbf2ffd640 1 -- 172.21.15.7:0/3452562532 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efc0c016040 con 0x55ae464b6690 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efbf2ffd640 1 -- 172.21.15.7:0/3452562532 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc0c003c00 con 0x55ae464b6690 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efc1c9a3740 10 client.26096 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55ae464cb130 con 0x55ae464b6690 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efbf2ffd640 1 -- 172.21.15.7:0/3452562532 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7efc0c00f600 con 0x55ae464b6690 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efc1c9a3740 10 client.26096 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efc1c9a3740 10 client.26096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efc1c9a3740 20 client.26096 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ae464cc480 0x55ae464ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efc1c9a3740 4 client.26096 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.630+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae46429900 con 0x55ae464cc480 2024-03-26T08:45:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.631+0000 7efc12ffd640 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ae464cc480 0x55ae464ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.631+0000 7efc12ffd640 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ae464cc480 0x55ae464ce870 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a88a9c0da5f9421 server_cookie=62b941d3d63def25 in_seq=0 out_seq=0 2024-03-26T08:45:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.631+0000 7efbf2ffd640 10 client.26096 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.632+0000 7efbf2ffd640 1 -- 172.21.15.7:0/3452562532 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ae46429900 con 0x55ae464cc480 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.632+0000 7efbf2ffd640 10 client.26096 handle_command_reply: tid=0 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.632+0000 7efc1c9a3740 1 client.26096 shutdown 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.632+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b6690 msgr2=0x55ae464c0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.632+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b6690 0x55ae464c0940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ae464b2b70 tx=0x7efc0c009320 comp rx=0 tx=0).stop 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.632+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 shutdown_connections 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.633+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ae464cc480 0x55ae464ce870 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.633+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464bc1f0 0x55ae464c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.633+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464b7080 0x55ae464c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.633+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/3452562532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464b6690 0x55ae464c0940 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.633+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 >> 172.21.15.7:0/3452562532 conn(0x55ae464b1c00 msgr2=0x55ae464b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.633+0000 7efbf2ffd640 0 client.26096 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.633+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 shutdown_connections 2024-03-26T08:45:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.633+0000 7efc1c9a3740 1 -- 172.21.15.7:0/3452562532 wait complete. 2024-03-26T08:45:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.633+0000 7efc1c9a3740 20 client.26096 trim_cache size 0 max 16384 2024-03-26T08:45:32.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.654+0000 7efc1c9a3740 1 Processor -- start 2024-03-26T08:45:32.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.654+0000 7efc1c9a3740 1 -- start start 2024-03-26T08:45:32.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.654+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 0x55ae464b5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.654+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.654+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 0x55ae464c03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc1c9a3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141a8ef0 con 0x55ae464bff90 2024-03-26T08:45:32.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc1c9a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141a69f0 con 0x55ae464bc1f0 2024-03-26T08:45:32.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc1c9a3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141a4a70 con 0x55ae464b7080 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59558/0 (socket says 172.21.15.7:59558) 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc12ffd640 1 -- 172.21.15.7:0/375585305 learned_addr learned my addr 172.21.15.7:0/375585305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc13fff640 1 --2- 172.21.15.7:0/375585305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 0x55ae464c03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc137fe640 1 --2- 172.21.15.7:0/375585305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 0x55ae464b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc12ffd640 1 -- 172.21.15.7:0/375585305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 msgr2=0x55ae464b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc12ffd640 1 --2- 172.21.15.7:0/375585305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 0x55ae464b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc12ffd640 1 -- 172.21.15.7:0/375585305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 msgr2=0x55ae464c03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc12ffd640 1 --2- 172.21.15.7:0/375585305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 0x55ae464c03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.655+0000 7efc12ffd640 1 -- 172.21.15.7:0/375585305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae464b2970 con 0x55ae464bc1f0 2024-03-26T08:45:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.656+0000 7efc12ffd640 1 --2- 172.21.15.7:0/375585305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c7d70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc04003fd0 tx=0x7efc040f0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ab0db23b7442808 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.656+0000 7efbf97fa640 1 -- 172.21.15.7:0/375585305 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0404b040 con 0x55ae464bc1f0 2024-03-26T08:45:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.656+0000 7efbf97fa640 1 -- 172.21.15.7:0/375585305 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc040da070 con 0x55ae464bc1f0 2024-03-26T08:45:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.656+0000 7efbf97fa640 1 -- 172.21.15.7:0/375585305 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc040bb280 con 0x55ae464bc1f0 2024-03-26T08:45:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.657+0000 7efc1c9a3740 1 -- 172.21.15.7:0/375585305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 msgr2=0x55ae464c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.657+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/375585305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c7d70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc04003fd0 tx=0x7efc040f0040 comp rx=0 tx=0).stop 2024-03-26T08:45:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.657+0000 7efc1c9a3740 1 -- 172.21.15.7:0/375585305 shutdown_connections 2024-03-26T08:45:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.657+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/375585305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 0x55ae464c03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.657+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/375585305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c7d70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.657+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/375585305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 0x55ae464b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.657+0000 7efc1c9a3740 1 -- 172.21.15.7:0/375585305 >> 172.21.15.7:0/375585305 conn(0x55ae464b1c00 msgr2=0x55ae464b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.657+0000 7efc1c9a3740 1 -- 172.21.15.7:0/375585305 shutdown_connections 2024-03-26T08:45:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.658+0000 7efc1c9a3740 1 -- 172.21.15.7:0/375585305 wait complete. 2024-03-26T08:45:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.659+0000 7efc1c9a3740 1 -- start start 2024-03-26T08:45:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.659+0000 7efc1c9a3740 1 Processor -- start 2024-03-26T08:45:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.659+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 0x55ae464c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.659+0000 7efc1c9a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.659+0000 7efc137fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 0x55ae464c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.659+0000 7efc137fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 0x55ae464c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39026/0 (socket says 172.21.15.7:39026) 2024-03-26T08:45:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.659+0000 7efc137fe640 1 -- 172.21.15.7:0/926602487 learned_addr learned my addr 172.21.15.7:0/926602487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.659+0000 7efc12ffd640 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 0x55ae464c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141a5070 con 0x55ae464bff90 2024-03-26T08:45:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141a6390 con 0x55ae464bc1f0 2024-03-26T08:45:32.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc141a56d0 con 0x55ae464b7080 2024-03-26T08:45:32.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc1c9a3740 10 client.0 _ll_register_callbacks cb 0x7efc0c004c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:32.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc13fff640 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 0x55ae464c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc12ffd640 1 -- 172.21.15.7:0/926602487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 msgr2=0x55ae464c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc12ffd640 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 0x55ae464c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc12ffd640 1 -- 172.21.15.7:0/926602487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 msgr2=0x55ae464c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:32.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc12ffd640 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 0x55ae464c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.660+0000 7efc12ffd640 1 -- 172.21.15.7:0/926602487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae464c7920 con 0x55ae464bc1f0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.661+0000 7efc12ffd640 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c6320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efc04003fd0 tx=0x7efc04005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbce06aff716926 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.661+0000 7efbf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.661+0000 7efbf2ffd640 1 -- 172.21.15.7:0/926602487 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc040c5690 con 0x55ae464bc1f0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.661+0000 7efbf2ffd640 1 -- 172.21.15.7:0/926602487 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efc040035f0 con 0x55ae464bc1f0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.661+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc040da070 con 0x55ae464bc1f0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.661+0000 7efbf2ffd640 1 -- 172.21.15.7:0/926602487 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc040a0050 con 0x55ae464bc1f0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.661+0000 7efc1c9a3740 10 client.26102 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.661+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae46528500 con 0x55ae464bc1f0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.662+0000 7efbf2ffd640 1 -- 172.21.15.7:0/926602487 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc040da070 con 0x55ae464bc1f0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.662+0000 7efbf2ffd640 1 -- 172.21.15.7:0/926602487 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7efc040f00b0 con 0x55ae464bc1f0 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.662+0000 7efc1c9a3740 10 client.26102 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.662+0000 7efc1c9a3740 10 client.26102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:32.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.662+0000 7efc1c9a3740 20 client.26102 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:32.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.662+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ae46528d20 0x55ae4652b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:32.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.662+0000 7efc1c9a3740 4 client.26102 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:45:32.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.662+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae46423c50 con 0x55ae46528d20 2024-03-26T08:45:32.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.662+0000 7efc137fe640 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ae46528d20 0x55ae4652b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:32.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.663+0000 7efbf2ffd640 10 client.26102 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.663+0000 7efc137fe640 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ae46528d20 0x55ae4652b140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a686b56f149775 server_cookie=81d731ef26c40cd0 in_seq=0 out_seq=0 2024-03-26T08:45:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.663+0000 7efbf2ffd640 1 -- 172.21.15.7:0/926602487 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae46423c50 con 0x55ae46528d20 2024-03-26T08:45:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.663+0000 7efbf2ffd640 10 client.26102 handle_command_reply: tid=0 2024-03-26T08:45:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.663+0000 7efc1c9a3740 1 client.26102 shutdown 2024-03-26T08:45:32.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 msgr2=0x55ae464c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c6320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efc04003fd0 tx=0x7efc04005e50 comp rx=0 tx=0).stop 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 shutdown_connections 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ae46528d20 0x55ae4652b140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae464bff90 0x55ae464c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae464bc1f0 0x55ae464c6320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 --2- 172.21.15.7:0/926602487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ae464b7080 0x55ae464c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efbf2ffd640 0 client.26102 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 >> 172.21.15.7:0/926602487 conn(0x55ae464b1c00 msgr2=0x55ae464b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 shutdown_connections 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 1 -- 172.21.15.7:0/926602487 wait complete. 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.664+0000 7efc1c9a3740 20 client.26102 trim_cache size 0 max 16384 2024-03-26T08:45:32.657 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 -- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efbfc0a2950 msgr2=0x7efbfc0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efbfc0a2950 0x7efbfc0a4e10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efc0800da00 tx=0x7efc08018040 comp rx=0 tx=0).stop 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 -- 172.21.15.7:0/3779007612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc141028c0 msgr2=0x7efc14195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc141028c0 0x7efc14195d30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efc04009230 tx=0x7efc0402c660 comp rx=0 tx=0).stop 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 -- 172.21.15.7:0/3779007612 shutdown_connections 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efbfc0a2950 0x7efbfc0a4e10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc14103280 0x7efc1419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc141028c0 0x7efc14195d30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 --2- 172.21.15.7:0/3779007612 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc141016d0 0x7efc141957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.665+0000 7efc19cdc640 1 -- 172.21.15.7:0/3779007612 >> 172.21.15.7:0/3779007612 conn(0x7efc140fcc60 msgr2=0x7efc14105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.666+0000 7efc19cdc640 1 -- 172.21.15.7:0/3779007612 shutdown_connections 2024-03-26T08:45:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:32.667+0000 7efc19cdc640 1 -- 172.21.15.7:0/3779007612 wait complete. 2024-03-26T08:45:33.673 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:45:33.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:45:33.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:45:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 -- 172.21.15.7:0/433022394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 msgr2=0x7f0e801045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 --2- 172.21.15.7:0/433022394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 0x7f0e801045a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009900 tx=0x7f0e7c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:45:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 -- 172.21.15.7:0/433022394 shutdown_connections 2024-03-26T08:45:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 --2- 172.21.15.7:0/433022394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e80107410 0x7f0e80109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 --2- 172.21.15.7:0/433022394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e80104ae0 0x7f0e80106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 --2- 172.21.15.7:0/433022394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 0x7f0e801045a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 -- 172.21.15.7:0/433022394 >> 172.21.15.7:0/433022394 conn(0x7f0e800fbb00 msgr2=0x7f0e800fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 -- 172.21.15.7:0/433022394 shutdown_connections 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 -- 172.21.15.7:0/433022394 wait complete. 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 Processor -- start 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.844+0000 7f0e876dc640 1 -- start start 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e876dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 0x7f0e80197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e876dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e80104ae0 0x7f0e80197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e876dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e80107410 0x7f0e8019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e876dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8010d350 con 0x7f0e801021b0 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e876dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8010d1d0 con 0x7f0e80107410 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e876dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8010d4d0 con 0x7f0e80104ae0 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85478640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 0x7f0e80197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85478640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 0x7f0e80197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54542/0 (socket says 172.21.15.7:54542) 2024-03-26T08:45:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85478640 1 -- 172.21.15.7:0/3823442812 learned_addr learned my addr 172.21.15.7:0/3823442812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e84c77640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e80104ae0 0x7f0e80197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85c79640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e80107410 0x7f0e8019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85478640 1 -- 172.21.15.7:0/3823442812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e80104ae0 msgr2=0x7f0e80197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85478640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e80104ae0 0x7f0e80197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85478640 1 -- 172.21.15.7:0/3823442812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e80107410 msgr2=0x7f0e8019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85478640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e80107410 0x7f0e8019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85478640 1 -- 172.21.15.7:0/3823442812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8019f640 con 0x7f0e801021b0 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e85478640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 0x7f0e80197a20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009230 tx=0x7f0e7c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e767fc640 1 -- 172.21.15.7:0/3823442812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c041070 con 0x7f0e801021b0 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e876dc640 1 -- 172.21.15.7:0/3823442812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8019f8d0 con 0x7f0e801021b0 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.845+0000 7f0e767fc640 1 -- 172.21.15.7:0/3823442812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c004450 con 0x7f0e801021b0 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.846+0000 7f0e767fc640 1 -- 172.21.15.7:0/3823442812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c003db0 con 0x7f0e801021b0 2024-03-26T08:45:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.846+0000 7f0e876dc640 1 -- 172.21.15.7:0/3823442812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8019fdc0 con 0x7f0e801021b0 2024-03-26T08:45:33.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.849+0000 7f0e8a3a1740 1 Processor -- start 2024-03-26T08:45:33.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.849+0000 7f0e767fc640 1 -- 172.21.15.7:0/3823442812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0e7c044070 con 0x7f0e801021b0 2024-03-26T08:45:33.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e8a3a1740 1 -- start start 2024-03-26T08:45:33.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12dd5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12dd8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12ddb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c12d0ac90 con 0x564c12dd8ce0 2024-03-26T08:45:33.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c12d30070 con 0x564c12dd6280 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c12cd8c80 con 0x564c12dd3950 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e84c77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12ddb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85478640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12dd8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85478640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12dd8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59592/0 (socket says 172.21.15.7:59592) 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85478640 1 -- 172.21.15.7:0/975146735 learned_addr learned my addr 172.21.15.7:0/975146735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85c79640 1 --2- 172.21.15.7:0/975146735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12dd5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85478640 1 -- 172.21.15.7:0/975146735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 msgr2=0x564c12dd5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e767fc640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e6c0a2880 0x7f0e6c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85478640 1 --2- 172.21.15.7:0/975146735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12dd5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85478640 1 -- 172.21.15.7:0/975146735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 msgr2=0x564c12ddb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85478640 1 --2- 172.21.15.7:0/975146735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12ddb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85478640 1 -- 172.21.15.7:0/975146735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c12cd0d10 con 0x564c12dd6280 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.850+0000 7f0e85c79640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e6c0a2880 0x7f0e6c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.851+0000 7f0e767fc640 1 -- 172.21.15.7:0/3823442812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0e7c0f0620 con 0x7f0e801021b0 2024-03-26T08:45:33.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.851+0000 7f0e85478640 1 --2- 172.21.15.7:0/975146735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12dd8670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0b9950 tx=0x7f0e7c0b9050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c47ba00f8ea78d03 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:33.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.851+0000 7f0e6b7fe640 1 -- 172.21.15.7:0/975146735 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c04b030 con 0x564c12dd6280 2024-03-26T08:45:33.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.851+0000 7f0e6b7fe640 1 -- 172.21.15.7:0/975146735 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c0bd680 con 0x564c12dd6280 2024-03-26T08:45:33.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.851+0000 7f0e6b7fe640 1 -- 172.21.15.7:0/975146735 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0bcd50 con 0x564c12dd6280 2024-03-26T08:45:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/975146735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 msgr2=0x564c12dd8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/975146735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12dd8670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0b9950 tx=0x7f0e7c0b9050 comp rx=0 tx=0).stop 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e85c79640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e6c0a2880 0x7f0e6c0a4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x564c12dc9b10 tx=0x7f0e78004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/975146735 shutdown_connections 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/975146735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12ddb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/975146735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12dd8670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/975146735 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12dd5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/975146735 >> 172.21.15.7:0/975146735 conn(0x564c12d931a0 msgr2=0x564c12dc6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/975146735 shutdown_connections 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.852+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/975146735 wait complete. 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.853+0000 7f0e8a3a1740 1 -- start start 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.853+0000 7f0e8a3a1740 1 Processor -- start 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.853+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12ddd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.853+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12ddd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.853+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12de47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.853+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c12cec0d0 con 0x564c12dd8ce0 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.853+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c12d2c110 con 0x564c12dd6280 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.853+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c12ddba30 con 0x564c12dd3950 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e8a3a1740 10 client.0 _ll_register_callbacks cb 0x564c12d27e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e85478640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12ddd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e85478640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12ddd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39068/0 (socket says 172.21.15.7:39068) 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e85478640 1 -- 172.21.15.7:0/4039080113 learned_addr learned my addr 172.21.15.7:0/4039080113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e84c77640 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12ddd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e85478640 1 -- 172.21.15.7:0/4039080113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 msgr2=0x564c12ddd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e85478640 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12ddd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e85478640 1 -- 172.21.15.7:0/4039080113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 msgr2=0x564c12de47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e85478640 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12de47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e85478640 1 -- 172.21.15.7:0/4039080113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c12b0c7e0 con 0x564c12dd3950 2024-03-26T08:45:33.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e85478640 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12ddd2a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02cec0 tx=0x7f0e7c0f08e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b06715a4bde2312f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.854+0000 7f0e68ff9640 1 -- 172.21.15.7:0/4039080113 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0bc8e0 con 0x564c12dd3950 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/4039080113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e7c0bd680 con 0x564c12dd3950 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e68ff9640 1 -- 172.21.15.7:0/4039080113 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0e7c0035f0 con 0x564c12dd3950 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e68ff9640 1 -- 172.21.15.7:0/4039080113 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e7c0bcac0 con 0x564c12dd3950 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e8a3a1740 10 client.26131 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/4039080113 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564c12b6cf70 con 0x564c12dd3950 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e68ff9640 1 -- 172.21.15.7:0/4039080113 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c0bd680 con 0x564c12dd3950 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e68ff9640 1 -- 172.21.15.7:0/4039080113 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0e7c0b8cd0 con 0x564c12dd3950 2024-03-26T08:45:33.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e8a3a1740 10 client.26131 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e8a3a1740 10 client.26131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e8a3a1740 20 client.26131 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564c12de8fe0 0x564c12deb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e8a3a1740 4 client.26131 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/4039080113 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c12d3d5b0 con 0x564c12de8fe0 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.855+0000 7f0e84c77640 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564c12de8fe0 0x564c12deb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e84c77640 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564c12de8fe0 0x564c12deb460 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbb4b20659827ad server_cookie=8292f7b0f33d3bff in_seq=0 out_seq=0 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e68ff9640 10 client.26131 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e68ff9640 1 -- 172.21.15.7:0/4039080113 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564c12d3d5b0 con 0x564c12de8fe0 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e68ff9640 10 client.26131 handle_command_reply: tid=0 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e8a3a1740 1 client.26131 shutdown 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/4039080113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 msgr2=0x564c12ddd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12ddd2a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02cec0 tx=0x7f0e7c0f08e0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/4039080113 shutdown_connections 2024-03-26T08:45:33.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564c12de8fe0 0x564c12deb460 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12de47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd6280 0x564c12ddd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e68ff9640 0 client.26131 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:33.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/4039080113 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd3950 0x564c12ddd2a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.856+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/4039080113 >> 172.21.15.7:0/4039080113 conn(0x564c12d931a0 msgr2=0x564c12dd31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:33.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.857+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/4039080113 shutdown_connections 2024-03-26T08:45:33.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.857+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/4039080113 wait complete. 2024-03-26T08:45:33.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.857+0000 7f0e8a3a1740 20 client.26131 trim_cache size 0 max 16384 2024-03-26T08:45:33.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.877+0000 7f0e8a3a1740 1 Processor -- start 2024-03-26T08:45:33.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.877+0000 7f0e8a3a1740 1 -- start start 2024-03-26T08:45:33.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12dd3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12defd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801a8e00 con 0x564c12dd8ce0 2024-03-26T08:45:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801a6900 con 0x564c12dd3950 2024-03-26T08:45:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801a4980 con 0x564c12dd6280 2024-03-26T08:45:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e85478640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12dd3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e84c77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e85478640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12dd3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59618/0 (socket says 172.21.15.7:59618) 2024-03-26T08:45:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e84c77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39078/0 (socket says 172.21.15.7:39078) 2024-03-26T08:45:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e85478640 1 -- 172.21.15.7:0/2354147675 learned_addr learned my addr 172.21.15.7:0/2354147675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:33.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e85c79640 1 --2- 172.21.15.7:0/2354147675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12defd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e84c77640 1 -- 172.21.15.7:0/2354147675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 msgr2=0x564c12dd3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:33.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e84c77640 1 --2- 172.21.15.7:0/2354147675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12dd3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e84c77640 1 -- 172.21.15.7:0/2354147675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 msgr2=0x564c12defd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:33.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e84c77640 1 --2- 172.21.15.7:0/2354147675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12defd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e84c77640 1 -- 172.21.15.7:0/2354147675 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c12a04170 con 0x564c12dd6280 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.878+0000 7f0e85c79640 1 --2- 172.21.15.7:0/2354147675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12defd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.879+0000 7f0e84c77640 1 --2- 172.21.15.7:0/2354147675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de4740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x564c12debca0 tx=0x7f0e7001fb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da7ddd4b10a2c949 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.879+0000 7f0e6affd640 1 -- 172.21.15.7:0/2354147675 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e700120d0 con 0x564c12dd6280 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.879+0000 7f0e6affd640 1 -- 172.21.15.7:0/2354147675 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e70012270 con 0x564c12dd6280 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.879+0000 7f0e6affd640 1 -- 172.21.15.7:0/2354147675 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70010790 con 0x564c12dd6280 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2354147675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 msgr2=0x564c12de4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2354147675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de4740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x564c12debca0 tx=0x7f0e7001fb50 comp rx=0 tx=0).stop 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2354147675 shutdown_connections 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2354147675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12defd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2354147675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de4740 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2354147675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12dd3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2354147675 >> 172.21.15.7:0/2354147675 conn(0x564c12d931a0 msgr2=0x564c12dd2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2354147675 shutdown_connections 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2354147675 wait complete. 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.880+0000 7f0e8a3a1740 1 -- start start 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e8a3a1740 1 Processor -- start 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12de4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e8a3a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12de3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e8a3a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801a4f80 con 0x564c12dd8ce0 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e85478640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12de4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e85478640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12de4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59624/0 (socket says 172.21.15.7:59624) 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e85478640 1 -- 172.21.15.7:0/2520772385 learned_addr learned my addr 172.21.15.7:0/2520772385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e84c77640 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801a62a0 con 0x564c12dd3950 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801a55e0 con 0x564c12dd6280 2024-03-26T08:45:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e85c79640 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12de3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.881+0000 7f0e8a3a1740 10 client.0 _ll_register_callbacks cb 0x7f0e70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e85478640 1 -- 172.21.15.7:0/2520772385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 msgr2=0x564c12de2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e85478640 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e85478640 1 -- 172.21.15.7:0/2520772385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 msgr2=0x564c12de3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e85478640 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12de3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e85478640 1 -- 172.21.15.7:0/2520772385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c12de3880 con 0x564c12dd3950 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e85c79640 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12de3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e85478640 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12de4570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0b9900 tx=0x7f0e7c005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79ba81f9206e03bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e68ff9640 1 -- 172.21.15.7:0/2520772385 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c04b070 con 0x564c12dd3950 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e68ff9640 1 -- 172.21.15.7:0/2520772385 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0e7c0c5290 con 0x564c12dd3950 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e70012270 con 0x564c12dd3950 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e68ff9640 1 -- 172.21.15.7:0/2520772385 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e7c0bc8e0 con 0x564c12dd3950 2024-03-26T08:45:33.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e8a3a1740 10 client.26117 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564c12de3880 con 0x564c12dd3950 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e68ff9640 1 -- 172.21.15.7:0/2520772385 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e70012270 con 0x564c12dd3950 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e68ff9640 1 -- 172.21.15.7:0/2520772385 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0e7c0bc3d0 con 0x564c12dd3950 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e8a3a1740 10 client.26117 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e8a3a1740 10 client.26117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e8a3a1740 20 client.26117 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564c12e45d80 0x564c12e481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e8a3a1740 4 client.26117 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.882+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564c12d37900 con 0x564c12e45d80 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.883+0000 7f0e84c77640 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564c12e45d80 0x564c12e481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.883+0000 7f0e84c77640 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564c12e45d80 0x564c12e481a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28cec4819a8df55 server_cookie=a1567fc7e666b8d9 in_seq=0 out_seq=0 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.883+0000 7f0e68ff9640 10 client.26117 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.884+0000 7f0e68ff9640 1 -- 172.21.15.7:0/2520772385 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564c12d37900 con 0x564c12e45d80 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.884+0000 7f0e68ff9640 10 client.26117 handle_command_reply: tid=0 2024-03-26T08:45:33.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.884+0000 7f0e8a3a1740 1 client.26117 shutdown 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 msgr2=0x564c12de4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12de4570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0b9900 tx=0x7f0e7c005e50 comp rx=0 tx=0).stop 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 shutdown_connections 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564c12e45d80 0x564c12e481a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c12dd8ce0 0x564c12de3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564c12dd6280 0x564c12de2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 --2- 172.21.15.7:0/2520772385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c12dd3950 0x564c12de4570 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 >> 172.21.15.7:0/2520772385 conn(0x564c12d931a0 msgr2=0x564c12dd31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e68ff9640 0 client.26117 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 shutdown_connections 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 1 -- 172.21.15.7:0/2520772385 wait complete. 2024-03-26T08:45:33.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.885+0000 7f0e8a3a1740 20 client.26117 trim_cache size 0 max 16384 2024-03-26T08:45:33.883 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:45:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:45:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "8712b789-8524-4d97-a500-2f49b8bfa3ed", 2024-03-26T08:45:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:45:33.883 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:45:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 -- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e6c0a2880 msgr2=0x7f0e6c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:33.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e6c0a2880 0x7f0e6c0a4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x564c12dc9b10 tx=0x7f0e78004a40 comp rx=0 tx=0).stop 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 -- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 msgr2=0x7f0e80197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 0x7f0e80197a20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009230 tx=0x7f0e7c02c650 comp rx=0 tx=0).stop 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 -- 172.21.15.7:0/3823442812 shutdown_connections 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e6c0a2880 0x7f0e6c0a4d40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e80107410 0x7f0e8019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e80104ae0 0x7f0e80197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 --2- 172.21.15.7:0/3823442812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e801021b0 0x7f0e80197a20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.886+0000 7f0e876dc640 1 -- 172.21.15.7:0/3823442812 >> 172.21.15.7:0/3823442812 conn(0x7f0e800fbb00 msgr2=0x7f0e80102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.887+0000 7f0e876dc640 1 -- 172.21.15.7:0/3823442812 shutdown_connections 2024-03-26T08:45:33.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:33.887+0000 7f0e876dc640 1 -- 172.21.15.7:0/3823442812 wait complete. 2024-03-26T08:45:33.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:45:33.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.048+0000 7fbdcf044640 1 -- 172.21.15.7:0/841940034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc810b010 msgr2=0x7fbdc810d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.048+0000 7fbdcf044640 1 --2- 172.21.15.7:0/841940034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc810b010 0x7fbdc810d480 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc009900 tx=0x7fbdbc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.048+0000 7fbdcf044640 1 -- 172.21.15.7:0/841940034 shutdown_connections 2024-03-26T08:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.048+0000 7fbdcf044640 1 --2- 172.21.15.7:0/841940034 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc810d9c0 0x7fbdc810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.048+0000 7fbdcf044640 1 --2- 172.21.15.7:0/841940034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc810b010 0x7fbdc810d480 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.048+0000 7fbdcf044640 1 --2- 172.21.15.7:0/841940034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc81086a0 0x7fbdc810aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.048+0000 7fbdcf044640 1 -- 172.21.15.7:0/841940034 >> 172.21.15.7:0/841940034 conn(0x7fbdc806e6b0 msgr2=0x7fbdc8070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.048+0000 7fbdcf044640 1 -- 172.21.15.7:0/841940034 shutdown_connections 2024-03-26T08:45:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.048+0000 7fbdcf044640 1 -- 172.21.15.7:0/841940034 wait complete. 2024-03-26T08:45:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.049+0000 7fbdcf044640 1 Processor -- start 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.049+0000 7fbdcf044640 1 -- start start 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.049+0000 7fbdcf044640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81086a0 0x7fbdc8114d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.049+0000 7fbdcf044640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc810b010 0x7fbdc81152b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.049+0000 7fbdcf044640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc810d9c0 0x7fbdc81b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.049+0000 7fbdcf044640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc811c260 con 0x7fbdc810b010 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.049+0000 7fbdcf044640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc811c0e0 con 0x7fbdc81086a0 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.049+0000 7fbdcf044640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc811c3e0 con 0x7fbdc810d9c0 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.050+0000 7fbdcd5e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc810d9c0 0x7fbdc81b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.050+0000 7fbdccde0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81086a0 0x7fbdc8114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.050+0000 7fbdcd5e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc810d9c0 0x7fbdc81b6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39090/0 (socket says 172.21.15.7:39090) 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.050+0000 7fbdcd5e1640 1 -- 172.21.15.7:0/2369563553 learned_addr learned my addr 172.21.15.7:0/2369563553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.050+0000 7fbdcd5e1640 1 -- 172.21.15.7:0/2369563553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81086a0 msgr2=0x7fbdc8114d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.050+0000 7fbdcd5e1640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81086a0 0x7fbdc8114d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.050+0000 7fbdcd5e1640 1 -- 172.21.15.7:0/2369563553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc810b010 msgr2=0x7fbdc81152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.050+0000 7fbdcd5e1640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc810b010 0x7fbdc81152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.050+0000 7fbdcd5e1640 1 -- 172.21.15.7:0/2369563553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc81b6eb0 con 0x7fbdc810d9c0 2024-03-26T08:45:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.051+0000 7fbdcd5e1640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc810d9c0 0x7fbdc81b6970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000b970 tx=0x7fbdc000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.051+0000 7fbdc5ffb640 1 -- 172.21.15.7:0/2369563553 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0019070 con 0x7fbdc810d9c0 2024-03-26T08:45:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.051+0000 7fbdc5ffb640 1 -- 172.21.15.7:0/2369563553 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdc000cb40 con 0x7fbdc810d9c0 2024-03-26T08:45:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.051+0000 7fbdc5ffb640 1 -- 172.21.15.7:0/2369563553 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00076e0 con 0x7fbdc810d9c0 2024-03-26T08:45:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.051+0000 7fbdcf044640 1 -- 172.21.15.7:0/2369563553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc81b7080 con 0x7fbdc810d9c0 2024-03-26T08:45:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.052+0000 7fbdcf044640 1 -- 172.21.15.7:0/2369563553 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc81b74f0 con 0x7fbdc810d9c0 2024-03-26T08:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.057+0000 7fbdd1c2f740 1 Processor -- start 2024-03-26T08:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.057+0000 7fbdc5ffb640 1 -- 172.21.15.7:0/2369563553 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fbdc001c070 con 0x7fbdc810d9c0 2024-03-26T08:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdd1c2f740 1 -- start start 2024-03-26T08:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc64d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 0x55cfbcc69e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 0x55cfbcc6c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdd1c2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbcbc9b10 con 0x55cfbcc64920 2024-03-26T08:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdd1c2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbcb72900 con 0x55cfbcc6a480 2024-03-26T08:45:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdd1c2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbcbc5bb0 con 0x55cfbcc65310 2024-03-26T08:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdccde0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 0x55cfbcc69e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 0x55cfbcc6c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc64d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc64d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54614/0 (socket says 172.21.15.7:54614) 2024-03-26T08:45:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdc7fff640 1 -- 172.21.15.7:0/3883088099 learned_addr learned my addr 172.21.15.7:0/3883088099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.058+0000 7fbdc5ffb640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb00a2a00 0x7fbdb00a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.059+0000 7fbdc5ffb640 1 -- 172.21.15.7:0/2369563553 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbdc00cdfc0 con 0x7fbdc810d9c0 2024-03-26T08:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.059+0000 7fbdcd5e1640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb00a2a00 0x7fbdb00a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.060+0000 7fbdccde0640 1 -- 172.21.15.7:0/3883088099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 msgr2=0x55cfbcc6c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.060+0000 7fbdccde0640 1 --2- 172.21.15.7:0/3883088099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 0x55cfbcc6c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.060+0000 7fbdccde0640 1 -- 172.21.15.7:0/3883088099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 msgr2=0x55cfbcc64d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.060+0000 7fbdccde0640 1 --2- 172.21.15.7:0/3883088099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc64d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.060+0000 7fbdccde0640 1 -- 172.21.15.7:0/3883088099 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfbc8de210 con 0x55cfbcc65310 2024-03-26T08:45:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.060+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/3883088099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 0x55cfbcc6c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:45:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.060+0000 7fbdcd5e1640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb00a2a00 0x7fbdb00a4ec0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00972f0 tx=0x7fbdc0004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.061+0000 7fbdccde0640 1 --2- 172.21.15.7:0/3883088099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 0x55cfbcc69e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800a830 tx=0x7fbdb800acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a4620421ae91f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.061+0000 7fbdaeffd640 1 -- 172.21.15.7:0/3883088099 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8015070 con 0x55cfbcc65310 2024-03-26T08:45:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.061+0000 7fbdaeffd640 1 -- 172.21.15.7:0/3883088099 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdb80113f0 con 0x55cfbcc65310 2024-03-26T08:45:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.061+0000 7fbdaeffd640 1 -- 172.21.15.7:0/3883088099 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80185c0 con 0x55cfbcc65310 2024-03-26T08:45:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.062+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/3883088099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 msgr2=0x55cfbcc69e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.062+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/3883088099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 0x55cfbcc69e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800a830 tx=0x7fbdb800acf0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.062+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/3883088099 shutdown_connections 2024-03-26T08:45:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.062+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/3883088099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 0x55cfbcc6c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.062+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/3883088099 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 0x55cfbcc69e10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.062+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/3883088099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc64d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.062+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/3883088099 >> 172.21.15.7:0/3883088099 conn(0x55cfbcc5fe90 msgr2=0x55cfbcc60290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.063+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/3883088099 shutdown_connections 2024-03-26T08:45:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.063+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/3883088099 wait complete. 2024-03-26T08:45:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.063+0000 7fbdd1c2f740 1 -- start start 2024-03-26T08:45:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.063+0000 7fbdd1c2f740 1 Processor -- start 2024-03-26T08:45:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc64920 0x55cfbcc6ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc65310 0x55cfbcc6f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc6a480 0x55cfbcc76150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc64920 0x55cfbcc6ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc64920 0x55cfbcc6ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59650/0 (socket says 172.21.15.7:59650) 2024-03-26T08:45:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdc7fff640 1 -- 172.21.15.7:0/143685827 learned_addr learned my addr 172.21.15.7:0/143685827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbcc6d430 con 0x55cfbcc65310 2024-03-26T08:45:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbcb85d50 con 0x55cfbcc64920 2024-03-26T08:45:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfbcc6d5b0 con 0x55cfbcc6a480 2024-03-26T08:45:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.064+0000 7fbdccde0640 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc65310 0x55cfbcc6f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.065+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc6a480 0x55cfbcc76150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.065+0000 7fbdd1c2f740 10 client.0 _ll_register_callbacks cb 0x55cfbcb85720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.065+0000 7fbdc7fff640 1 -- 172.21.15.7:0/143685827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc6a480 msgr2=0x55cfbcc76150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.065+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc6a480 0x55cfbcc76150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.065+0000 7fbdc7fff640 1 -- 172.21.15.7:0/143685827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc65310 msgr2=0x55cfbcc6f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.065+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc65310 0x55cfbcc6f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.065+0000 7fbdc7fff640 1 -- 172.21.15.7:0/143685827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfbc9aa620 con 0x55cfbcc64920 2024-03-26T08:45:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.065+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdb80113f0 con 0x55cfbcc64920 2024-03-26T08:45:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.065+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc64920 0x55cfbcc6ebd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc02c240 tx=0x7fbdbc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2941c30013cc0b9e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.066+0000 7fbda7fff640 10 client.26123 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.066+0000 7fbda7fff640 1 -- 172.21.15.7:0/143685827 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0042f0 con 0x55cfbcc64920 2024-03-26T08:45:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.066+0000 7fbda7fff640 1 -- 172.21.15.7:0/143685827 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbdbc0046c0 con 0x55cfbcc64920 2024-03-26T08:45:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.066+0000 7fbda7fff640 1 -- 172.21.15.7:0/143685827 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdbc0048e0 con 0x55cfbcc64920 2024-03-26T08:45:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.066+0000 7fbdd1c2f740 10 client.26123 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.066+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55cfbcc794c0 con 0x55cfbcc64920 2024-03-26T08:45:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.067+0000 7fbda7fff640 1 -- 172.21.15.7:0/143685827 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbdbc0035f0 con 0x55cfbcc64920 2024-03-26T08:45:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.067+0000 7fbdd1c2f740 10 client.26123 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.067+0000 7fbdd1c2f740 10 client.26123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.067+0000 7fbdd1c2f740 20 client.26123 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.067+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cfbcc7ae30 0x55cfbcc7d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.067+0000 7fbdd1c2f740 4 client.26123 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.067+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfbcbd7050 con 0x55cfbcc7ae30 2024-03-26T08:45:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.067+0000 7fbdccde0640 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cfbcc7ae30 0x55cfbcc7d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.068+0000 7fbdccde0640 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cfbcc7ae30 0x55cfbcc7d260 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3fad364c0b03df1 server_cookie=848ff0dcf9df0a67 in_seq=0 out_seq=0 2024-03-26T08:45:34.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.068+0000 7fbda7fff640 10 client.26123 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.070+0000 7fbda7fff640 1 -- 172.21.15.7:0/143685827 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cfbcbd7050 con 0x55cfbcc7ae30 2024-03-26T08:45:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.070+0000 7fbda7fff640 10 client.26123 handle_command_reply: tid=0 2024-03-26T08:45:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.070+0000 7fbdd1c2f740 1 client.26123 shutdown 2024-03-26T08:45:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc64920 msgr2=0x55cfbcc6ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc64920 0x55cfbcc6ebd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc02c240 tx=0x7fbdbc005fd0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 shutdown_connections 2024-03-26T08:45:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cfbcc7ae30 0x55cfbcc7d260 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc6a480 0x55cfbcc76150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc65310 0x55cfbcc6f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/143685827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc64920 0x55cfbcc6ebd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 >> 172.21.15.7:0/143685827 conn(0x55cfbcc5fe90 msgr2=0x55cfbcc5f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbda7fff640 0 client.26123 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 shutdown_connections 2024-03-26T08:45:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/143685827 wait complete. 2024-03-26T08:45:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.071+0000 7fbdd1c2f740 20 client.26123 trim_cache size 0 max 16384 2024-03-26T08:45:34.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.096+0000 7f4770a25640 1 -- 172.21.15.7:0/2083482378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476c103700 msgr2=0x7f476c103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.096+0000 7f4770a25640 1 --2- 172.21.15.7:0/2083482378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476c103700 0x7f476c103b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f475c009860 tx=0x7f475c02bb90 comp rx=0 tx=0).stop 2024-03-26T08:45:34.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.097+0000 7f4770a25640 1 -- 172.21.15.7:0/2083482378 shutdown_connections 2024-03-26T08:45:34.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.097+0000 7f4770a25640 1 --2- 172.21.15.7:0/2083482378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f476c1052b0 0x7f476c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.097+0000 7f4770a25640 1 --2- 172.21.15.7:0/2083482378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1048f0 0x7f476c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.097+0000 7f4770a25640 1 --2- 172.21.15.7:0/2083482378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476c103700 0x7f476c103b20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.097+0000 7f4770a25640 1 -- 172.21.15.7:0/2083482378 >> 172.21.15.7:0/2083482378 conn(0x7f476c0fec70 msgr2=0x7f476c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.097+0000 7f4770a25640 1 -- 172.21.15.7:0/2083482378 shutdown_connections 2024-03-26T08:45:34.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.097+0000 7f4770a25640 1 -- 172.21.15.7:0/2083482378 wait complete. 2024-03-26T08:45:34.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.098+0000 7f4770a25640 1 Processor -- start 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.098+0000 7f4770a25640 1 -- start start 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.098+0000 7f4770a25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f476c103700 0x7f476c071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.098+0000 7f4770a25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476c1048f0 0x7f476c071750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.098+0000 7f4770a25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1052b0 0x7f476c0727d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.098+0000 7f476ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f476c103700 0x7f476c071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.098+0000 7f476ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f476c103700 0x7f476c071210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39124/0 (socket says 172.21.15.7:39124) 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.098+0000 7f476ad9d640 1 -- 172.21.15.7:0/391452290 learned_addr learned my addr 172.21.15.7:0/391452290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c06bb10 con 0x7f476c1052b0 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c06b990 con 0x7f476c1048f0 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f476b59e640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1052b0 0x7f476c0727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c06bc90 con 0x7f476c103700 2024-03-26T08:45:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f476ad9d640 1 -- 172.21.15.7:0/391452290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476c1048f0 msgr2=0x7f476c071750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:45:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f476ad9d640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476c1048f0 0x7f476c071750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f476ad9d640 1 -- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1052b0 msgr2=0x7f476c0727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f476ad9d640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1052b0 0x7f476c0727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f476ad9d640 1 -- 172.21.15.7:0/391452290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c072e70 con 0x7f476c103700 2024-03-26T08:45:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.099+0000 7f476ad9d640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f476c103700 0x7f476c071210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f475c02c180 tx=0x7f475c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.101+0000 7fbdd1c2f740 1 Processor -- start 2024-03-26T08:45:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.101+0000 7fbdd1c2f740 1 -- start start 2024-03-26T08:45:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.101+0000 7f4753fff640 1 -- 172.21.15.7:0/391452290 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c041030 con 0x7f476c103700 2024-03-26T08:45:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.101+0000 7f4753fff640 1 -- 172.21.15.7:0/391452290 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475c02c9f0 con 0x7f476c103700 2024-03-26T08:45:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.101+0000 7f4753fff640 1 -- 172.21.15.7:0/391452290 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c039040 con 0x7f476c103700 2024-03-26T08:45:34.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.101+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc5e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.101+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c073100 con 0x7f476c103700 2024-03-26T08:45:34.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.101+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c0735a0 con 0x7f476c103700 2024-03-26T08:45:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.101+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 0x55cfbcc76110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.101+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 0x55cfbcc5ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.101+0000 7fbdd1c2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81c2720 con 0x55cfbcc64920 2024-03-26T08:45:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.101+0000 7fbdd1c2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81c0220 con 0x55cfbcc6a480 2024-03-26T08:45:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.101+0000 7fbdd1c2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81be2a0 con 0x55cfbcc65310 2024-03-26T08:45:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.102+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc5e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.102+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc5e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54636/0 (socket says 172.21.15.7:54636) 2024-03-26T08:45:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.102+0000 7fbdc7fff640 1 -- 172.21.15.7:0/2349313105 learned_addr learned my addr 172.21.15.7:0/2349313105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.102+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/2349313105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 0x55cfbcc5ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.103+0000 7fbdc7fff640 1 -- 172.21.15.7:0/2349313105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 msgr2=0x55cfbcc76110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:45:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.103+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/2349313105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 0x55cfbcc76110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.103+0000 7fbdc7fff640 1 -- 172.21.15.7:0/2349313105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 msgr2=0x55cfbcc5ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.103+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/2349313105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 0x55cfbcc5ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.103+0000 7fbdc7fff640 1 -- 172.21.15.7:0/2349313105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfbcc60c00 con 0x55cfbcc64920 2024-03-26T08:45:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.103+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/2349313105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc5e790 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc001eb0 tx=0x7fbdbc043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafcc956c81c3315 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.103+0000 7fbdae7fc640 1 -- 172.21.15.7:0/2349313105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc003e00 con 0x55cfbcc64920 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.103+0000 7fbdae7fc640 1 -- 172.21.15.7:0/2349313105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdb80113f0 con 0x55cfbcc64920 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.104+0000 7fbdae7fc640 1 -- 172.21.15.7:0/2349313105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc04e6b0 con 0x55cfbcc64920 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.104+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2349313105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 msgr2=0x55cfbcc5e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.104+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2349313105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc5e790 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc001eb0 tx=0x7fbdbc043000 comp rx=0 tx=0).stop 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.105+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2349313105 shutdown_connections 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.105+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2349313105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc6a480 0x55cfbcc5ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.105+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2349313105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc65310 0x55cfbcc76110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.105+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2349313105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc5e790 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.105+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2349313105 >> 172.21.15.7:0/2349313105 conn(0x55cfbcc5fe90 msgr2=0x55cfbcc60270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.105+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2349313105 shutdown_connections 2024-03-26T08:45:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.105+0000 7f4773e0c740 1 Processor -- start 2024-03-26T08:45:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.105+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2349313105 wait complete. 2024-03-26T08:45:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.105+0000 7fbdd1c2f740 1 -- start start 2024-03-26T08:45:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.105+0000 7fbdd1c2f740 1 Processor -- start 2024-03-26T08:45:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.105+0000 7f4773e0c740 1 -- start start 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.105+0000 7f4753fff640 1 -- 172.21.15.7:0/391452290 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f475c044070 con 0x7f476c103700 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.105+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be2fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 0x55741be34ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f4773e0c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741bd8c3d0 con 0x55741be35310 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f4773e0c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741bd34ad0 con 0x55741be2f7b0 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f4773e0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741bd88470 con 0x55741be301a0 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f476a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f476b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be2fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f476a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be377a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54644/0 (socket says 172.21.15.7:54644) 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f476a59c640 1 -- 172.21.15.7:0/681418495 learned_addr learned my addr 172.21.15.7:0/681418495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f476ad9d640 1 --2- 172.21.15.7:0/681418495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 0x55741be34ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc65310 0x55cfbcc73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdd1c2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc6a480 0x55cfbcc74a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdd1c2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81be8a0 con 0x55cfbcc64920 2024-03-26T08:45:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdd1c2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81bfbc0 con 0x55cfbcc65310 2024-03-26T08:45:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdd1c2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc81bef00 con 0x55cfbcc6a480 2024-03-26T08:45:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdd1c2f740 10 client.0 _ll_register_callbacks cb 0x7fbdbc005090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdccde0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc65310 0x55cfbcc73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdccde0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc65310 0x55cfbcc73a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59684/0 (socket says 172.21.15.7:59684) 2024-03-26T08:45:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.106+0000 7fbdccde0640 1 -- 172.21.15.7:0/2853748902 learned_addr learned my addr 172.21.15.7:0/2853748902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.106+0000 7f4753fff640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47540a2940 0x7f47540a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.107+0000 7f476b59e640 1 -- 172.21.15.7:0/681418495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 msgr2=0x55741be34ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.107+0000 7f476b59e640 1 --2- 172.21.15.7:0/681418495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 0x55741be34ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.107+0000 7f476b59e640 1 -- 172.21.15.7:0/681418495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 msgr2=0x55741be377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.107+0000 7f476b59e640 1 --2- 172.21.15.7:0/681418495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be377a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.107+0000 7f476b59e640 1 -- 172.21.15.7:0/681418495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741bb6a7d0 con 0x55741be2f7b0 2024-03-26T08:45:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.107+0000 7f476b59e640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47540a2940 0x7f47540a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.107+0000 7f476b59e640 1 --2- 172.21.15.7:0/681418495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be2fbd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4764009fd0 tx=0x7f4764011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49109ef72027a9f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.107+0000 7f476a59c640 1 --2- 172.21.15.7:0/681418495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.107+0000 7fbdccde0640 1 -- 172.21.15.7:0/2853748902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc6a480 msgr2=0x55cfbcc74a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.107+0000 7fbdccde0640 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc6a480 0x55cfbcc74a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.107+0000 7fbdccde0640 1 -- 172.21.15.7:0/2853748902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 msgr2=0x55cfbcc734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.107+0000 7fbdccde0640 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.107+0000 7fbdccde0640 1 -- 172.21.15.7:0/2853748902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfbcc75010 con 0x55cfbcc65310 2024-03-26T08:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.107+0000 7fbdccde0640 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc65310 0x55cfbcc73a10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800a150 tx=0x7fbdb80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba56cce6c7f43c1a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.107+0000 7fbda7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.108+0000 7fbda7fff640 1 -- 172.21.15.7:0/2853748902 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb802c060 con 0x55cfbcc65310 2024-03-26T08:45:34.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.108+0000 7fbda7fff640 1 -- 172.21.15.7:0/2853748902 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbdb8011620 con 0x55cfbcc65310 2024-03-26T08:45:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.108+0000 7f47517fa640 1 -- 172.21.15.7:0/681418495 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764017070 con 0x55741be2f7b0 2024-03-26T08:45:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.108+0000 7f47517fa640 1 -- 172.21.15.7:0/681418495 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4764003df0 con 0x55741be2f7b0 2024-03-26T08:45:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.108+0000 7f47517fa640 1 -- 172.21.15.7:0/681418495 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476400fd90 con 0x55741be2f7b0 2024-03-26T08:45:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.108+0000 7f4753fff640 1 -- 172.21.15.7:0/391452290 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f475c0ef700 con 0x7f476c103700 2024-03-26T08:45:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.109+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2853748902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdb80113f0 con 0x55cfbcc65310 2024-03-26T08:45:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.109+0000 7fbda7fff640 1 -- 172.21.15.7:0/2853748902 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdb8015040 con 0x55cfbcc65310 2024-03-26T08:45:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.109+0000 7f476b59e640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47540a2940 0x7f47540a4e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f476400ffd0 tx=0x7f4764009230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.109+0000 7f4773e0c740 1 -- 172.21.15.7:0/681418495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 msgr2=0x55741be2fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.109+0000 7f4773e0c740 1 --2- 172.21.15.7:0/681418495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be2fbd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4764009fd0 tx=0x7f4764011920 comp rx=0 tx=0).stop 2024-03-26T08:45:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.109+0000 7fbdd1c2f740 10 client.26135 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.109+0000 7f4773e0c740 1 -- 172.21.15.7:0/681418495 shutdown_connections 2024-03-26T08:45:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.109+0000 7f4773e0c740 1 --2- 172.21.15.7:0/681418495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.109+0000 7f4773e0c740 1 --2- 172.21.15.7:0/681418495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 0x55741be34ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.109+0000 7f4773e0c740 1 --2- 172.21.15.7:0/681418495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be2fbd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.109+0000 7f4773e0c740 1 -- 172.21.15.7:0/681418495 >> 172.21.15.7:0/681418495 conn(0x55741be2ad20 msgr2=0x55741be2b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.109+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2853748902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cfbcc76320 con 0x55cfbcc65310 2024-03-26T08:45:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.109+0000 7f4773e0c740 1 -- 172.21.15.7:0/681418495 shutdown_connections 2024-03-26T08:45:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.110+0000 7fbda7fff640 1 -- 172.21.15.7:0/2853748902 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbdb80113f0 con 0x55cfbcc65310 2024-03-26T08:45:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.110+0000 7fbda7fff640 1 -- 172.21.15.7:0/2853748902 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbdb801a4e0 con 0x55cfbcc65310 2024-03-26T08:45:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.110+0000 7fbdd1c2f740 10 client.26135 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.110+0000 7fbdd1c2f740 10 client.26135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.110+0000 7fbdd1c2f740 20 client.26135 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.110+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cfbccd68d0 0x55cfbccd8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.110+0000 7fbdd1c2f740 4 client.26135 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:45:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.110+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2853748902 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cfbcbd13a0 con 0x55cfbccd68d0 2024-03-26T08:45:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.110+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cfbccd68d0 0x55cfbccd8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.111+0000 7f4773e0c740 1 -- 172.21.15.7:0/681418495 wait complete. 2024-03-26T08:45:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.111+0000 7f4773e0c740 1 -- start start 2024-03-26T08:45:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.111+0000 7fbda7fff640 10 client.26135 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.111+0000 7f4773e0c740 1 Processor -- start 2024-03-26T08:45:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.111+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be39cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 0x55741be3a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.112+0000 7fbdc7fff640 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cfbccd68d0 0x55cfbccd8cc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6873ef0584d49b7a server_cookie=987fb1bd6cdfc13f in_seq=0 out_seq=0 2024-03-26T08:45:34.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 0x55741be411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f4773e0c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741be382c0 con 0x55741be301a0 2024-03-26T08:45:34.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f4773e0c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741bd48060 con 0x55741be35310 2024-03-26T08:45:34.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f4773e0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55741be38440 con 0x55741be2f7b0 2024-03-26T08:45:34.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be39cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be39cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39156/0 (socket says 172.21.15.7:39156) 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476ad9d640 1 -- 172.21.15.7:0/4145712264 learned_addr learned my addr 172.21.15.7:0/4145712264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476b59e640 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 0x55741be411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476a59c640 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 0x55741be3a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f4773e0c740 10 client.0 _ll_register_callbacks cb 0x55741bd2a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476ad9d640 1 -- 172.21.15.7:0/4145712264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 msgr2=0x55741be411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476ad9d640 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 0x55741be411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476ad9d640 1 -- 172.21.15.7:0/4145712264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 msgr2=0x55741be3a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476ad9d640 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 0x55741be3a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476ad9d640 1 -- 172.21.15.7:0/4145712264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55741baa03f0 con 0x55741be2f7b0 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f476ad9d640 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be39cb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f475c009830 tx=0x7f475c005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b84892de683e7d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.112+0000 7f474eff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.113+0000 7f474eff5640 1 -- 172.21.15.7:0/4145712264 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0bba80 con 0x55741be2f7b0 2024-03-26T08:45:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.113+0000 7f4773e0c740 1 -- 172.21.15.7:0/4145712264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4764003df0 con 0x55741be2f7b0 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.113+0000 7f474eff5640 1 -- 172.21.15.7:0/4145712264 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f475c0c4430 con 0x55741be2f7b0 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.113+0000 7f474eff5640 1 -- 172.21.15.7:0/4145712264 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475c0bc820 con 0x55741be2f7b0 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.114+0000 7f4773e0c740 10 client.26158 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.114+0000 7f4773e0c740 1 -- 172.21.15.7:0/4145712264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55741ba600f0 con 0x55741be2f7b0 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.114+0000 7f474eff5640 1 -- 172.21.15.7:0/4145712264 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475c0bc120 con 0x55741be2f7b0 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.114+0000 7f474eff5640 1 -- 172.21.15.7:0/4145712264 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f475c03c030 con 0x55741be2f7b0 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.114+0000 7f4773e0c740 10 client.26158 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.114+0000 7f4773e0c740 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.114+0000 7f4773e0c740 20 client.26158 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.115+0000 7f4773e0c740 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55741be45920 0x55741be47d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.115+0000 7f4773e0c740 4 client.26158 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.115+0000 7f4773e0c740 1 -- 172.21.15.7:0/4145712264 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4764003df0 con 0x55741be45920 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.115+0000 7f476a59c640 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55741be45920 0x55741be47d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.116+0000 7f476a59c640 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55741be45920 0x55741be47d30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c125003fc76fce server_cookie=853d3f075af7213c in_seq=0 out_seq=0 2024-03-26T08:45:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.116+0000 7f474eff5640 10 client.26158 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:34.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.135+0000 7fbda7fff640 1 -- 172.21.15.7:0/2853748902 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cfbcbd13a0 con 0x55cfbccd68d0 2024-03-26T08:45:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.135+0000 7fbda7fff640 10 client.26135 handle_command_reply: tid=0 2024-03-26T08:45:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.135+0000 7fbdd1c2f740 1 client.26135 shutdown 2024-03-26T08:45:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.135+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2853748902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc65310 msgr2=0x55cfbcc73a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.135+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc65310 0x55cfbcc73a10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800a150 tx=0x7fbdb80046c0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f474eff5640 1 -- 172.21.15.7:0/4145712264 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f4764003df0 con 0x55741be45920 2024-03-26T08:45:34.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f474eff5640 10 client.26158 handle_command_reply: tid=0 2024-03-26T08:45:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2853748902 shutdown_connections 2024-03-26T08:45:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cfbccd68d0 0x55cfbccd8cc0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cfbcc6a480 0x55cfbcc74a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfbcc65310 0x55cfbcc73a10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbdd1c2f740 1 --2- 172.21.15.7:0/2853748902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cfbcc64920 0x55cfbcc734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2853748902 >> 172.21.15.7:0/2853748902 conn(0x55cfbcc5fe90 msgr2=0x55cfbcc5e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbda7fff640 0 client.26135 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:34.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f4773e0c740 1 client.26158 shutdown 2024-03-26T08:45:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2853748902 shutdown_connections 2024-03-26T08:45:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbdd1c2f740 1 -- 172.21.15.7:0/2853748902 wait complete. 2024-03-26T08:45:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.136+0000 7fbdd1c2f740 20 client.26135 trim_cache size 0 max 16384 2024-03-26T08:45:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f4773e0c740 1 -- 172.21.15.7:0/4145712264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 msgr2=0x55741be39cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f4773e0c740 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be39cb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f475c009830 tx=0x7f475c005d60 comp rx=0 tx=0).stop 2024-03-26T08:45:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f4773e0c740 1 -- 172.21.15.7:0/4145712264 shutdown_connections 2024-03-26T08:45:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f4773e0c740 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55741be45920 0x55741be47d30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f4773e0c740 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 0x55741be411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f4773e0c740 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 0x55741be3a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f4773e0c740 1 --2- 172.21.15.7:0/4145712264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be39cb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.136+0000 7f474eff5640 0 client.26158 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.137+0000 7f4773e0c740 1 -- 172.21.15.7:0/4145712264 >> 172.21.15.7:0/4145712264 conn(0x55741be2ad20 msgr2=0x55741be217b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:45:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:45:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:45:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:45:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.137+0000 7f4773e0c740 1 -- 172.21.15.7:0/4145712264 shutdown_connections 2024-03-26T08:45:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.137+0000 7fbdcf044640 1 -- 172.21.15.7:0/2369563553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb00a2a00 msgr2=0x7fbdb00a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.137+0000 7fbdcf044640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb00a2a00 0x7fbdb00a4ec0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00972f0 tx=0x7fbdc0004b50 comp rx=0 tx=0).stop 2024-03-26T08:45:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.138+0000 7fbdcf044640 1 -- 172.21.15.7:0/2369563553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc810d9c0 msgr2=0x7fbdc81b6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.138+0000 7fbdcf044640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc810d9c0 0x7fbdc81b6970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000b970 tx=0x7fbdc000be30 comp rx=0 tx=0).stop 2024-03-26T08:45:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.138+0000 7f4773e0c740 1 -- 172.21.15.7:0/4145712264 wait complete. 2024-03-26T08:45:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.138+0000 7fbdcf044640 1 -- 172.21.15.7:0/2369563553 shutdown_connections 2024-03-26T08:45:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.138+0000 7fbdcf044640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbdb00a2a00 0x7fbdb00a4ec0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.138+0000 7fbdcf044640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbdc810d9c0 0x7fbdc81b6970 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.138+0000 7fbdcf044640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdc810b010 0x7fbdc81152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.138+0000 7fbdcf044640 1 --2- 172.21.15.7:0/2369563553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdc81086a0 0x7fbdc8114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.138+0000 7fbdcf044640 1 -- 172.21.15.7:0/2369563553 >> 172.21.15.7:0/2369563553 conn(0x7fbdc806e6b0 msgr2=0x7fbdc810a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.138+0000 7f4773e0c740 20 client.26158 trim_cache size 0 max 16384 2024-03-26T08:45:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.139+0000 7fbdcf044640 1 -- 172.21.15.7:0/2369563553 shutdown_connections 2024-03-26T08:45:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:34.139+0000 7fbdcf044640 1 -- 172.21.15.7:0/2369563553 wait complete. 2024-03-26T08:45:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f4773e0c740 1 Processor -- start 2024-03-26T08:45:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f4773e0c740 1 -- start start 2024-03-26T08:45:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be3a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 0x55741be41150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 0x55741be2e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f4773e0c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c11b590 con 0x55741be301a0 2024-03-26T08:45:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f4773e0c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c119090 con 0x55741be35310 2024-03-26T08:45:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f4773e0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c117110 con 0x55741be2f7b0 2024-03-26T08:45:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f476ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be3a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f476b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 0x55741be2e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f476ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be3a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39160/0 (socket says 172.21.15.7:39160) 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f476ad9d640 1 -- 172.21.15.7:0/3444183056 learned_addr learned my addr 172.21.15.7:0/3444183056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.167+0000 7f476a59c640 1 --2- 172.21.15.7:0/3444183056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 0x55741be41150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.168+0000 7f476ad9d640 1 -- 172.21.15.7:0/3444183056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 msgr2=0x55741be2e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.168+0000 7f476ad9d640 1 --2- 172.21.15.7:0/3444183056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 0x55741be2e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.168+0000 7f476ad9d640 1 -- 172.21.15.7:0/3444183056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 msgr2=0x55741be41150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.168+0000 7f476ad9d640 1 --2- 172.21.15.7:0/3444183056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 0x55741be41150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.168+0000 7f476ad9d640 1 -- 172.21.15.7:0/3444183056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55741be2ba90 con 0x55741be2f7b0 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.168+0000 7f476ad9d640 1 --2- 172.21.15.7:0/3444183056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be3a150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b8580 tx=0x7f475c0ba630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b2c24b011a382a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.168+0000 7f4750ff9640 1 -- 172.21.15.7:0/3444183056 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0bba80 con 0x55741be2f7b0 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.168+0000 7f4750ff9640 1 -- 172.21.15.7:0/3444183056 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475c0c0270 con 0x55741be2f7b0 2024-03-26T08:45:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.168+0000 7f4750ff9640 1 -- 172.21.15.7:0/3444183056 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0ed3e0 con 0x55741be2f7b0 2024-03-26T08:45:34.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.169+0000 7f4773e0c740 1 -- 172.21.15.7:0/3444183056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 msgr2=0x55741be3a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.169+0000 7f4773e0c740 1 --2- 172.21.15.7:0/3444183056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be3a150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b8580 tx=0x7f475c0ba630 comp rx=0 tx=0).stop 2024-03-26T08:45:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.171+0000 7f4773e0c740 1 -- 172.21.15.7:0/3444183056 shutdown_connections 2024-03-26T08:45:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.171+0000 7f4773e0c740 1 --2- 172.21.15.7:0/3444183056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be35310 0x55741be2e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.171+0000 7f4773e0c740 1 --2- 172.21.15.7:0/3444183056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be301a0 0x55741be41150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.171+0000 7f4773e0c740 1 --2- 172.21.15.7:0/3444183056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be2f7b0 0x55741be3a150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.172+0000 7f4773e0c740 1 -- 172.21.15.7:0/3444183056 >> 172.21.15.7:0/3444183056 conn(0x55741be2ad20 msgr2=0x55741be2df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.172+0000 7f4773e0c740 1 -- 172.21.15.7:0/3444183056 shutdown_connections 2024-03-26T08:45:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.172+0000 7f4773e0c740 1 -- 172.21.15.7:0/3444183056 wait complete. 2024-03-26T08:45:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.173+0000 7f4773e0c740 1 -- start start 2024-03-26T08:45:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.173+0000 7f4773e0c740 1 Processor -- start 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be3ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 0x55741be3f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f4773e0c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be3ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f4773e0c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c117710 con 0x55741be35310 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f4773e0c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c118a30 con 0x55741be2f7b0 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f4773e0c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c117d70 con 0x55741be301a0 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be3ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59708/0 (socket says 172.21.15.7:59708) 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476ad9d640 1 -- 172.21.15.7:0/2515779127 learned_addr learned my addr 172.21.15.7:0/2515779127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476a59c640 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 0x55741be3f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f4773e0c740 10 client.0 _ll_register_callbacks cb 0x7f475c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476b59e640 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476a59c640 1 -- 172.21.15.7:0/2515779127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 msgr2=0x55741be3ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476a59c640 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be3ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476a59c640 1 -- 172.21.15.7:0/2515779127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 msgr2=0x55741be404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476a59c640 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476a59c640 1 -- 172.21.15.7:0/2515779127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55741be40ad0 con 0x55741be301a0 2024-03-26T08:45:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.174+0000 7f476ad9d640 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be3ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.175+0000 7f476a59c640 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 0x55741be3f4d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55741be485e0 tx=0x7f476001eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3252445ca4f132f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.175+0000 7f474eff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:45:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.175+0000 7f474eff5640 1 -- 172.21.15.7:0/2515779127 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760014890 con 0x55741be301a0 2024-03-26T08:45:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.175+0000 7f474eff5640 1 -- 172.21.15.7:0/2515779127 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4760014a30 con 0x55741be301a0 2024-03-26T08:45:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.175+0000 7f4773e0c740 1 -- 172.21.15.7:0/2515779127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f475c0c0270 con 0x55741be301a0 2024-03-26T08:45:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.175+0000 7f474eff5640 1 -- 172.21.15.7:0/2515779127 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4760014c30 con 0x55741be301a0 2024-03-26T08:45:34.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.175+0000 7f4773e0c740 10 client.26164 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.175+0000 7f4773e0c740 1 -- 172.21.15.7:0/2515779127 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55741bea1900 con 0x55741be301a0 2024-03-26T08:45:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.176+0000 7f474eff5640 1 -- 172.21.15.7:0/2515779127 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f475c0c0270 con 0x55741be301a0 2024-03-26T08:45:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.176+0000 7f474eff5640 1 -- 172.21.15.7:0/2515779127 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4760025070 con 0x55741be301a0 2024-03-26T08:45:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.176+0000 7f4773e0c740 10 client.26164 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.176+0000 7f4773e0c740 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.176+0000 7f4773e0c740 20 client.26164 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.176+0000 7f4773e0c740 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55741be45920 0x55741bea42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.176+0000 7f4773e0c740 4 client.26164 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:45:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.176+0000 7f4773e0c740 1 -- 172.21.15.7:0/2515779127 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55741bd99910 con 0x55741be45920 2024-03-26T08:45:34.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.178+0000 7f476ad9d640 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55741be45920 0x55741bea42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.178+0000 7f476ad9d640 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55741be45920 0x55741bea42f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5185b872fe0b8476 server_cookie=d4194cedc1b170d9 in_seq=0 out_seq=0 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.178+0000 7f474eff5640 10 client.26164 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f474eff5640 1 -- 172.21.15.7:0/2515779127 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55741bd99910 con 0x55741be45920 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f474eff5640 10 client.26164 handle_command_reply: tid=0 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f4773e0c740 1 client.26164 shutdown 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f4773e0c740 1 -- 172.21.15.7:0/2515779127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 msgr2=0x55741be3f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f4773e0c740 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 0x55741be3f4d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55741be485e0 tx=0x7f476001eac0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f4773e0c740 1 -- 172.21.15.7:0/2515779127 shutdown_connections 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f4773e0c740 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55741be45920 0x55741bea42f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f4773e0c740 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55741be35310 0x55741be404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f4773e0c740 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55741be301a0 0x55741be3f4d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f4773e0c740 1 --2- 172.21.15.7:0/2515779127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55741be2f7b0 0x55741be3ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f474eff5640 0 client.26164 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.179+0000 7f4773e0c740 1 -- 172.21.15.7:0/2515779127 >> 172.21.15.7:0/2515779127 conn(0x55741be2ad20 msgr2=0x55741be39f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4773e0c740 1 -- 172.21.15.7:0/2515779127 shutdown_connections 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4773e0c740 1 -- 172.21.15.7:0/2515779127 wait complete. 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4773e0c740 20 client.26164 trim_cache size 0 max 16384 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "8712b789-8524-4d97-a500-2f49b8bfa3ed": { 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:45:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "8712b789-8524-4d97-a500-2f49b8bfa3ed", 2024-03-26T08:45:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:45:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:45:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:45:34.173 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:45:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47540a2940 msgr2=0x7f47540a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47540a2940 0x7f47540a4e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f476400ffd0 tx=0x7f4764009230 comp rx=0 tx=0).stop 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f476c103700 msgr2=0x7f476c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f476c103700 0x7f476c071210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f475c02c180 tx=0x7f475c0040c0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 shutdown_connections 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f47540a2940 0x7f47540a4e00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f476c1052b0 0x7f476c0727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476c1048f0 0x7f476c071750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 --2- 172.21.15.7:0/391452290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f476c103700 0x7f476c071210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 >> 172.21.15.7:0/391452290 conn(0x7f476c0fec70 msgr2=0x7f476c107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.180+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 shutdown_connections 2024-03-26T08:45:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:45:34.181+0000 7f4770a25640 1 -- 172.21.15.7:0/391452290 wait complete. 2024-03-26T08:45:34.189 INFO:tasks.cephfs.filesystem:scrub status for tag:8712b789-8524-4d97-a500-2f49b8bfa3ed - {'path': '/', 'tag': '8712b789-8524-4d97-a500-2f49b8bfa3ed', 'options': 'recursive,force'} 2024-03-26T08:45:34.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:34 smithi016 ceph-mon[35851]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 868 op/s 2024-03-26T08:45:34.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:34 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:45:34.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:34 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:34.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:34 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:45:34.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:34 smithi007 ceph-mon[32462]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 868 op/s 2024-03-26T08:45:34.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:34 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:45:34.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:34 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:34.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:34 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:45:34.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:34 smithi044 ceph-mon[36461]: pgmap v454: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 868 op/s 2024-03-26T08:45:34.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:34 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:45:34.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:34 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:34.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:34 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:45:35.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:35 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:45:35.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:35 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:35.493 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:35 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:35.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:35 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:45:35.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:35 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:35.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:35 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:35.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:35 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:45:35.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:35 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:35.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:35 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:45:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 185.0 s, 4163.4 tps, lat 24.108 ms stddev 27.101 2024-03-26T08:45:36.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:36 smithi016 ceph-mon[35851]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 12 MiB/s wr, 832 op/s 2024-03-26T08:45:36.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:36 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:45:36.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:36 smithi007 ceph-mon[32462]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 12 MiB/s wr, 832 op/s 2024-03-26T08:45:36.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:36 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:45:36.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:36 smithi044 ceph-mon[36461]: pgmap v455: 97 pgs: 97 active+clean; 1.5 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 12 MiB/s wr, 832 op/s 2024-03-26T08:45:36.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:36 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:45:37.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:37 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:45:37.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:37 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:45:37.492 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:37 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:45:37.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:45:37.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:45:37.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:45:37.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:37 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:45:37.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:37 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:45:37.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:37 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:45:38.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:38 smithi007 ceph-mon[32462]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 885 op/s 2024-03-26T08:45:38.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:38 smithi044 ceph-mon[36461]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 885 op/s 2024-03-26T08:45:38.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:38 smithi016 ceph-mon[35851]: pgmap v456: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 12 MiB/s wr, 885 op/s 2024-03-26T08:45:40.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:40 smithi007 ceph-mon[32462]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 11 MiB/s wr, 925 op/s 2024-03-26T08:45:40.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:40 smithi044 ceph-mon[36461]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 11 MiB/s wr, 925 op/s 2024-03-26T08:45:40.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:40 smithi016 ceph-mon[35851]: pgmap v457: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 11 MiB/s wr, 925 op/s 2024-03-26T08:45:41.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 190.0 s, 3524.4 tps, lat 28.333 ms stddev 29.383 2024-03-26T08:45:42.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:42 smithi007 ceph-mon[32462]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 11 MiB/s wr, 912 op/s 2024-03-26T08:45:42.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:42 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:45:42.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:42 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:45:42.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:42 smithi044 ceph-mon[36461]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 11 MiB/s wr, 912 op/s 2024-03-26T08:45:42.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:42 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:45:42.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:42 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:45:42.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:42 smithi016 ceph-mon[35851]: pgmap v458: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 11 MiB/s wr, 912 op/s 2024-03-26T08:45:42.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:42 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:45:42.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:42 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:45:44.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:44 smithi007 ceph-mon[32462]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-03-26T08:45:44.543 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:44 smithi016 ceph-mon[35851]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-03-26T08:45:44.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:44 smithi044 ceph-mon[36461]: pgmap v459: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 11 MiB/s wr, 1.01k op/s 2024-03-26T08:45:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 195.0 s, 3932.1 tps, lat 25.476 ms stddev 27.757 2024-03-26T08:45:46.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:46 smithi007 ceph-mon[32462]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 10 MiB/s wr, 978 op/s 2024-03-26T08:45:46.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:46 smithi044 ceph-mon[36461]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 10 MiB/s wr, 978 op/s 2024-03-26T08:45:46.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:46 smithi016 ceph-mon[35851]: pgmap v460: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 10 MiB/s wr, 978 op/s 2024-03-26T08:45:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.275+0000 7f0cffabf640 1 -- 172.21.15.7:0/1704842741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf8103920 msgr2=0x7f0cf8103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.275+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1704842741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf8103920 0x7f0cf8103d40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009900 tx=0x7f0cf002bc30 comp rx=0 tx=0).stop 2024-03-26T08:45:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.275+0000 7f0cffabf640 1 -- 172.21.15.7:0/1704842741 shutdown_connections 2024-03-26T08:45:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.275+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1704842741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf81054d0 0x7f0cf810bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.275+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1704842741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf8104b10 0x7f0cf8104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.276+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1704842741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf8103920 0x7f0cf8103d40 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009900 tx=0x7f0cf002bc30 comp rx=0 tx=0).stop 2024-03-26T08:45:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.276+0000 7f0cffabf640 1 -- 172.21.15.7:0/1704842741 >> 172.21.15.7:0/1704842741 conn(0x7f0cf80fee90 msgr2=0x7f0cf81012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.276+0000 7f0cffabf640 1 -- 172.21.15.7:0/1704842741 shutdown_connections 2024-03-26T08:45:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.276+0000 7f0cffabf640 1 -- 172.21.15.7:0/1704842741 wait complete. 2024-03-26T08:45:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.277+0000 7f0cffabf640 1 Processor -- start 2024-03-26T08:45:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.277+0000 7f0cffabf640 1 -- start start 2024-03-26T08:45:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.277+0000 7f0cffabf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf8104b10 0x7f0cf819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.277+0000 7f0cffabf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf81054d0 0x7f0cf819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.277+0000 7f0cffabf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf81a3800 0x7f0cf81a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.277+0000 7f0cffabf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf810f5f0 con 0x7f0cf81a3800 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.277+0000 7f0cffabf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf810f470 con 0x7f0cf8104b10 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.277+0000 7f0cffabf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf810f770 con 0x7f0cf81054d0 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfd05a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf81054d0 0x7f0cf819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfd05a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf81054d0 0x7f0cf819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44756/0 (socket says 172.21.15.7:44756) 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfd05a640 1 -- 172.21.15.7:0/1603007669 learned_addr learned my addr 172.21.15.7:0/1603007669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfe05c640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf81a3800 0x7f0cf81a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfd85b640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf8104b10 0x7f0cf819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfd05a640 1 -- 172.21.15.7:0/1603007669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf8104b10 msgr2=0x7f0cf819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf8104b10 0x7f0cf819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfd05a640 1 -- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf81a3800 msgr2=0x7f0cf81a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf81a3800 0x7f0cf81a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.278+0000 7f0cfd05a640 1 -- 172.21.15.7:0/1603007669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf81a6260 con 0x7f0cf81054d0 2024-03-26T08:45:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.279+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf81054d0 0x7f0cf819e7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800e9f0 tx=0x7f0ce800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.279+0000 7f0ceeffd640 1 -- 172.21.15.7:0/1603007669 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800cd80 con 0x7f0cf81054d0 2024-03-26T08:45:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.279+0000 7f0ceeffd640 1 -- 172.21.15.7:0/1603007669 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ce8010910 con 0x7f0cf81054d0 2024-03-26T08:45:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.279+0000 7f0ceeffd640 1 -- 172.21.15.7:0/1603007669 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8018990 con 0x7f0cf81054d0 2024-03-26T08:45:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.279+0000 7f0cffabf640 1 -- 172.21.15.7:0/1603007669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf806b120 con 0x7f0cf81054d0 2024-03-26T08:45:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.279+0000 7f0cffabf640 1 -- 172.21.15.7:0/1603007669 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf806b620 con 0x7f0cf81054d0 2024-03-26T08:45:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.282+0000 7f0ceeffd640 1 -- 172.21.15.7:0/1603007669 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0ce8010ac0 con 0x7f0cf81054d0 2024-03-26T08:45:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.283+0000 7f0d02781740 1 Processor -- start 2024-03-26T08:45:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.284+0000 7f0ceeffd640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0ce40a29c0 0x7f0ce40a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.284+0000 7f0ceeffd640 1 -- 172.21.15.7:0/1603007669 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0ce80cecf0 con 0x7f0cf81054d0 2024-03-26T08:45:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.284+0000 7f0cfe05c640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0ce40a29c0 0x7f0ce40a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0d02781740 1 -- start start 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0d02781740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a42920 0x55de11a42d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0d02781740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a47e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0d02781740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11a4a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0d02781740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de119a7b10 con 0x55de11a48480 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0d02781740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11950900 con 0x55de11a43310 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0d02781740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de119a3bb0 con 0x55de11a42920 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfd85b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a42920 0x55de11a42d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfe05c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11a4a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfd85b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a42920 0x55de11a42d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44764/0 (socket says 172.21.15.7:44764) 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfe05c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11a4a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46592/0 (socket says 172.21.15.7:46592) 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfd85b640 1 -- 172.21.15.7:0/1170306271 learned_addr learned my addr 172.21.15.7:0/1170306271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/1170306271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a47e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfe05c640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0ce40a29c0 0x7f0ce40a4e80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4005fd0 tx=0x7f0cf4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfd05a640 1 -- 172.21.15.7:0/1170306271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a42920 msgr2=0x55de11a42d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/1170306271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a42920 0x55de11a42d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfd05a640 1 -- 172.21.15.7:0/1170306271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 msgr2=0x55de11a4a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/1170306271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11a4a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.285+0000 7f0cfd05a640 1 -- 172.21.15.7:0/1170306271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de116bc210 con 0x55de11a43310 2024-03-26T08:45:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.286+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/1170306271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a47e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800e9c0 tx=0x7f0ce809b150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99948eeb6a16bf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.286+0000 7f0ce3fff640 1 -- 172.21.15.7:0/1170306271 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80cc8c0 con 0x55de11a43310 2024-03-26T08:45:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.287+0000 7f0ce3fff640 1 -- 172.21.15.7:0/1170306271 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ce8097d70 con 0x55de11a43310 2024-03-26T08:45:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.287+0000 7f0ce3fff640 1 -- 172.21.15.7:0/1170306271 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce809adb0 con 0x55de11a43310 2024-03-26T08:45:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 -- 172.21.15.7:0/1170306271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 msgr2=0x55de11a47e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 --2- 172.21.15.7:0/1170306271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a47e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800e9c0 tx=0x7f0ce809b150 comp rx=0 tx=0).stop 2024-03-26T08:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 -- 172.21.15.7:0/1170306271 shutdown_connections 2024-03-26T08:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 --2- 172.21.15.7:0/1170306271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11a4a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 --2- 172.21.15.7:0/1170306271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a47e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 --2- 172.21.15.7:0/1170306271 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a42920 0x55de11a42d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 -- 172.21.15.7:0/1170306271 >> 172.21.15.7:0/1170306271 conn(0x55de11a3de90 msgr2=0x55de11a3e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 -- 172.21.15.7:0/1170306271 shutdown_connections 2024-03-26T08:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 -- 172.21.15.7:0/1170306271 wait complete. 2024-03-26T08:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.288+0000 7f0d02781740 1 -- start start 2024-03-26T08:45:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0d02781740 1 Processor -- start 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0d02781740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a42920 0x55de11a4cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0d02781740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a4d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0d02781740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 0x55de11a54260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0d02781740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11a4b430 con 0x55de11a42920 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0d02781740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11963d50 con 0x55de11a43310 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0d02781740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de11a4b5b0 con 0x55de11a48480 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0d02781740 10 client.0 _ll_register_callbacks cb 0x55de11963720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0cfd85b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a42920 0x55de11a4cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0cfd85b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a42920 0x55de11a4cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46604/0 (socket says 172.21.15.7:46604) 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.289+0000 7f0cfd85b640 1 -- 172.21.15.7:0/3123755976 learned_addr learned my addr 172.21.15.7:0/3123755976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a4d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0cfe05c640 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 0x55de11a54260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0cfd05a640 1 -- 172.21.15.7:0/3123755976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 msgr2=0x55de11a54260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 0x55de11a54260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0cfd05a640 1 -- 172.21.15.7:0/3123755976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a42920 msgr2=0x55de11a4cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a42920 0x55de11a4cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0cfd05a640 1 -- 172.21.15.7:0/3123755976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de11788620 con 0x55de11a43310 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0cfe05c640 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 0x55de11a54260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a4d220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55de11a43d80 tx=0x7f0ce8097ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3908c3cbf8f3a8c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.290+0000 7f0ce17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3123755976 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8018b30 con 0x55de11a43310 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3123755976 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0ce80b4470 con 0x55de11a43310 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0d02781740 1 -- 172.21.15.7:0/3123755976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ce8097d70 con 0x55de11a43310 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3123755976 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce8010430 con 0x55de11a43310 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0d02781740 10 client.26156 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0d02781740 1 -- 172.21.15.7:0/3123755976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de11a4b430 con 0x55de11a43310 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3123755976 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ce8097d70 con 0x55de11a43310 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3123755976 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0ce80a06c0 con 0x55de11a43310 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0d02781740 10 client.26156 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0d02781740 10 client.26156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0d02781740 20 client.26156 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0d02781740 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de11a58b80 0x55de11a5afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0d02781740 4 client.26156 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:45:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.291+0000 7f0d02781740 1 -- 172.21.15.7:0/3123755976 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de119b5050 con 0x55de11a58b80 2024-03-26T08:45:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.292+0000 7f0cfd85b640 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de11a58b80 0x55de11a5afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.292+0000 7f0cfd85b640 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de11a58b80 0x55de11a5afa0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af73d1c4fbfaa3d server_cookie=b1dd2477939ac39 in_seq=0 out_seq=0 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.293+0000 7f0ce17fa640 10 client.26156 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.293+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3123755976 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55de119b5050 con 0x55de11a58b80 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.293+0000 7f0ce17fa640 10 client.26156 handle_command_reply: tid=0 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 client.26156 shutdown 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 -- 172.21.15.7:0/3123755976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 msgr2=0x55de11a4d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a4d220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55de11a43d80 tx=0x7f0ce8097ad0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 -- 172.21.15.7:0/3123755976 shutdown_connections 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de11a58b80 0x55de11a5afa0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 0x55de11a54260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a43310 0x55de11a4d220 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 --2- 172.21.15.7:0/3123755976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a42920 0x55de11a4cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 -- 172.21.15.7:0/3123755976 >> 172.21.15.7:0/3123755976 conn(0x55de11a3de90 msgr2=0x55de11a3d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0ce17fa640 0 client.26156 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 -- 172.21.15.7:0/3123755976 shutdown_connections 2024-03-26T08:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 1 -- 172.21.15.7:0/3123755976 wait complete. 2024-03-26T08:45:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.294+0000 7f0d02781740 20 client.26156 trim_cache size 0 max 16384 2024-03-26T08:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.317+0000 7f0d02781740 1 Processor -- start 2024-03-26T08:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.317+0000 7f0d02781740 1 -- start start 2024-03-26T08:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0d02781740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a4cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0d02781740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a43310 0x55de11a4d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0d02781740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 0x55de11a54220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0d02781740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81a9470 con 0x55de11a43310 2024-03-26T08:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0d02781740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81a6f70 con 0x55de11a42920 2024-03-26T08:45:47.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0d02781740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8072490 con 0x55de11a48480 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfd85b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a4cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfd05a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a43310 0x55de11a4d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfd85b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a4cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51630/0 (socket says 172.21.15.7:51630) 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfd85b640 1 -- 172.21.15.7:0/890997898 learned_addr learned my addr 172.21.15.7:0/890997898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfe05c640 1 --2- 172.21.15.7:0/890997898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 0x55de11a54220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfd85b640 1 -- 172.21.15.7:0/890997898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 msgr2=0x55de11a54220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfd85b640 1 --2- 172.21.15.7:0/890997898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 0x55de11a54220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfd85b640 1 -- 172.21.15.7:0/890997898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a43310 msgr2=0x55de11a4d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfd85b640 1 --2- 172.21.15.7:0/890997898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a43310 0x55de11a4d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.318+0000 7f0cfd85b640 1 -- 172.21.15.7:0/890997898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de11a556e0 con 0x55de11a42920 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.319+0000 7f0cfd85b640 1 --2- 172.21.15.7:0/890997898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a4cce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0036040 tx=0x7f0cf002df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89482d3a1e041920 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.319+0000 7f0ce37fe640 1 -- 172.21.15.7:0/890997898 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf002cac0 con 0x55de11a42920 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.319+0000 7f0ce37fe640 1 -- 172.21.15.7:0/890997898 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf002cc60 con 0x55de11a42920 2024-03-26T08:45:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.319+0000 7f0ce37fe640 1 -- 172.21.15.7:0/890997898 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00046c0 con 0x55de11a42920 2024-03-26T08:45:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.320+0000 7f0d02781740 1 -- 172.21.15.7:0/890997898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 msgr2=0x55de11a4cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.320+0000 7f0d02781740 1 --2- 172.21.15.7:0/890997898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a4cce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0036040 tx=0x7f0cf002df90 comp rx=0 tx=0).stop 2024-03-26T08:45:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.321+0000 7f0d02781740 1 -- 172.21.15.7:0/890997898 shutdown_connections 2024-03-26T08:45:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.321+0000 7f0d02781740 1 --2- 172.21.15.7:0/890997898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a48480 0x55de11a54220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.321+0000 7f0d02781740 1 --2- 172.21.15.7:0/890997898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a43310 0x55de11a4d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.321+0000 7f0d02781740 1 --2- 172.21.15.7:0/890997898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a4cce0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.321+0000 7f0d02781740 1 -- 172.21.15.7:0/890997898 >> 172.21.15.7:0/890997898 conn(0x55de11a3de90 msgr2=0x55de11a3e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.321+0000 7f0d02781740 1 -- 172.21.15.7:0/890997898 shutdown_connections 2024-03-26T08:45:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.321+0000 7f0d02781740 1 -- 172.21.15.7:0/890997898 wait complete. 2024-03-26T08:45:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.322+0000 7f0d02781740 1 -- start start 2024-03-26T08:45:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.322+0000 7f0d02781740 1 Processor -- start 2024-03-26T08:45:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.322+0000 7f0d02781740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a51100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.322+0000 7f0d02781740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a43310 0x55de11a51640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.322+0000 7f0d02781740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11ab2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.322+0000 7f0d02781740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf8072a90 con 0x55de11a48480 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.322+0000 7f0d02781740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81a6910 con 0x55de11a42920 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.322+0000 7f0d02781740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80730f0 con 0x55de11a43310 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0d02781740 10 client.0 _ll_register_callbacks cb 0x7f0ce809b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfd85b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a51100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfd85b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a51100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51634/0 (socket says 172.21.15.7:51634) 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfd85b640 1 -- 172.21.15.7:0/3137272423 learned_addr learned my addr 172.21.15.7:0/3137272423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfe05c640 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11ab2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfd85b640 1 -- 172.21.15.7:0/3137272423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a43310 msgr2=0x55de11a51640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfd85b640 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a43310 0x55de11a51640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfd85b640 1 -- 172.21.15.7:0/3137272423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 msgr2=0x55de11ab2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:45:47.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfd85b640 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11ab2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfd85b640 1 -- 172.21.15.7:0/3137272423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de11a51ce0 con 0x55de11a42920 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfe05c640 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11ab2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0cfd85b640 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a51100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0cf005d040 tx=0x7f0cf0005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e69daf675a6d632 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.323+0000 7f0ce17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3137272423 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0069030 con 0x55de11a42920 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3137272423 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0cf004a040 con 0x55de11a42920 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0d02781740 1 -- 172.21.15.7:0/3137272423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cf002cc60 con 0x55de11a42920 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3137272423 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf0004250 con 0x55de11a42920 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0d02781740 10 client.26162 fetch_fsmap learned FSMap version 25 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0d02781740 1 -- 172.21.15.7:0/3137272423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de11ab3ac0 con 0x55de11a42920 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3137272423 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0cf002cc60 con 0x55de11a42920 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3137272423 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0cf00036f0 con 0x55de11a42920 2024-03-26T08:45:47.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0d02781740 10 client.26162 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0d02781740 10 client.26162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0d02781740 20 client.26162 populate_metadata read hostname 'smithi007' 2024-03-26T08:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0d02781740 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de11ab4400 0x55de11ab6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0d02781740 4 client.26162 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.324+0000 7f0d02781740 1 -- 172.21.15.7:0/3137272423 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de119af3a0 con 0x55de11ab4400 2024-03-26T08:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.325+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de11ab4400 0x55de11ab6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.326+0000 7f0ce17fa640 10 client.26162 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.326+0000 7f0cfd05a640 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de11ab4400 0x55de11ab6820 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdfcc171421638a server_cookie=f6df06337ca7718 in_seq=0 out_seq=0 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.341+0000 7f0ce17fa640 1 -- 172.21.15.7:0/3137272423 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de119af3a0 con 0x55de11ab4400 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.341+0000 7f0ce17fa640 10 client.26162 handle_command_reply: tid=0 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.341+0000 7f0d02781740 1 client.26162 shutdown 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.341+0000 7f0d02781740 1 -- 172.21.15.7:0/3137272423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 msgr2=0x55de11a51100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.341+0000 7f0d02781740 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a51100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0cf005d040 tx=0x7f0cf0005fd0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0d02781740 1 -- 172.21.15.7:0/3137272423 shutdown_connections 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0d02781740 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de11ab4400 0x55de11ab6820 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0d02781740 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de11a48480 0x55de11ab2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0d02781740 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de11a43310 0x55de11a51640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0d02781740 1 --2- 172.21.15.7:0/3137272423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de11a42920 0x55de11a51100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0ce17fa640 0 client.26162 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:45:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0d02781740 1 -- 172.21.15.7:0/3137272423 >> 172.21.15.7:0/3137272423 conn(0x55de11a3de90 msgr2=0x55de11a3e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0d02781740 1 -- 172.21.15.7:0/3137272423 shutdown_connections 2024-03-26T08:45:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0d02781740 1 -- 172.21.15.7:0/3137272423 wait complete. 2024-03-26T08:45:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0d02781740 20 client.26162 trim_cache size 0 max 16384 2024-03-26T08:45:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:45:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:45:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:45:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0cffabf640 1 -- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0ce40a29c0 msgr2=0x7f0ce40a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0ce40a29c0 0x7f0ce40a4e80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4005fd0 tx=0x7f0cf4009500 comp rx=0 tx=0).stop 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0cffabf640 1 -- 172.21.15.7:0/1603007669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf81054d0 msgr2=0x7f0cf819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.342+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf81054d0 0x7f0cf819e7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800e9f0 tx=0x7f0ce800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.343+0000 7f0cffabf640 1 -- 172.21.15.7:0/1603007669 shutdown_connections 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.343+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0ce40a29c0 0x7f0ce40a4e80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.343+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf81a3800 0x7f0cf81a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.343+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0cf81054d0 0x7f0cf819e7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.343+0000 7f0cffabf640 1 --2- 172.21.15.7:0/1603007669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf8104b10 0x7f0cf819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:45:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.343+0000 7f0cffabf640 1 -- 172.21.15.7:0/1603007669 >> 172.21.15.7:0/1603007669 conn(0x7f0cf80fee90 msgr2=0x7f0cf8077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:45:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.344+0000 7f0cffabf640 1 -- 172.21.15.7:0/1603007669 shutdown_connections 2024-03-26T08:45:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:45:47.344+0000 7f0cffabf640 1 -- 172.21.15.7:0/1603007669 wait complete. 2024-03-26T08:45:48.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:48 smithi007 ceph-mon[32462]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.7 MiB/s wr, 924 op/s 2024-03-26T08:45:48.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:48 smithi044 ceph-mon[36461]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.7 MiB/s wr, 924 op/s 2024-03-26T08:45:48.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:48 smithi016 ceph-mon[35851]: pgmap v461: 97 pgs: 97 active+clean; 1.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.7 MiB/s wr, 924 op/s 2024-03-26T08:45:50.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:50 smithi007 ceph-mon[32462]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.1 MiB/s wr, 849 op/s 2024-03-26T08:45:50.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:50 smithi044 ceph-mon[36461]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.1 MiB/s wr, 849 op/s 2024-03-26T08:45:50.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:50 smithi016 ceph-mon[35851]: pgmap v462: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.1 MiB/s wr, 849 op/s 2024-03-26T08:45:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 200.0 s, 3282.4 tps, lat 30.344 ms stddev 35.183 2024-03-26T08:45:52.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:52 smithi007 ceph-mon[32462]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.2 MiB/s wr, 810 op/s 2024-03-26T08:45:52.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:52 smithi044 ceph-mon[36461]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.2 MiB/s wr, 810 op/s 2024-03-26T08:45:52.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:52 smithi016 ceph-mon[35851]: pgmap v463: 97 pgs: 97 active+clean; 1.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 9.2 MiB/s wr, 810 op/s 2024-03-26T08:45:54.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:54 smithi007 ceph-mon[32462]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 MiB/s wr, 843 op/s 2024-03-26T08:45:54.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:54 smithi044 ceph-mon[36461]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 MiB/s wr, 843 op/s 2024-03-26T08:45:54.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:54 smithi016 ceph-mon[35851]: pgmap v464: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 MiB/s wr, 843 op/s 2024-03-26T08:45:56.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 205.0 s, 3695.6 tps, lat 26.926 ms stddev 29.134 2024-03-26T08:45:56.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:56 smithi007 ceph-mon[32462]: pgmap v465: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 786 op/s 2024-03-26T08:45:56.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:56 smithi044 ceph-mon[36461]: pgmap v465: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 786 op/s 2024-03-26T08:45:56.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:56 smithi016 ceph-mon[35851]: pgmap v465: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 MiB/s wr, 786 op/s 2024-03-26T08:45:58.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:45:58 smithi007 ceph-mon[32462]: pgmap v466: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 720 op/s 2024-03-26T08:45:58.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:45:58 smithi044 ceph-mon[36461]: pgmap v466: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 720 op/s 2024-03-26T08:45:58.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:45:58 smithi016 ceph-mon[35851]: pgmap v466: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 MiB/s wr, 720 op/s 2024-03-26T08:46:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.460+0000 7f13cace8640 1 -- 172.21.15.7:0/2274624789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c410ff20 msgr2=0x7f13c4112310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.460+0000 7f13cace8640 1 --2- 172.21.15.7:0/2274624789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c410ff20 0x7f13c4112310 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009cd0 tx=0x7f13bc02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.460+0000 7f13cace8640 1 -- 172.21.15.7:0/2274624789 shutdown_connections 2024-03-26T08:46:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.460+0000 7f13cace8640 1 --2- 172.21.15.7:0/2274624789 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c410ff20 0x7f13c4112310 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.460+0000 7f13cace8640 1 --2- 172.21.15.7:0/2274624789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13c4109000 0x7f13c410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.460+0000 7f13cace8640 1 --2- 172.21.15.7:0/2274624789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c41086a0 0x7f13c4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.460+0000 7f13cace8640 1 -- 172.21.15.7:0/2274624789 >> 172.21.15.7:0/2274624789 conn(0x7f13c406eff0 msgr2=0x7f13c4071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.460+0000 7f13cace8640 1 -- 172.21.15.7:0/2274624789 shutdown_connections 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.460+0000 7f13cace8640 1 -- 172.21.15.7:0/2274624789 wait complete. 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13cace8640 1 Processor -- start 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13cace8640 1 -- start start 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13cace8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c41086a0 0x7f13c41a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13cace8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c4109000 0x7f13c41a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13cace8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13c410ff20 0x7f13c41a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13cace8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4115b90 con 0x7f13c410ff20 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13cace8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4115a10 con 0x7f13c41086a0 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13cace8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4115d10 con 0x7f13c4109000 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13c8a84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c41086a0 0x7f13c41a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13c9285640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13c410ff20 0x7f13c41a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13c3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c4109000 0x7f13c41a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13c8a84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c41086a0 0x7f13c41a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55222/0 (socket says 172.21.15.7:55222) 2024-03-26T08:46:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13c8a84640 1 -- 172.21.15.7:0/592527710 learned_addr learned my addr 172.21.15.7:0/592527710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13c3fff640 1 -- 172.21.15.7:0/592527710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c41086a0 msgr2=0x7f13c41a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.461+0000 7f13c3fff640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c41086a0 0x7f13c41a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.462+0000 7f13c3fff640 1 -- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13c410ff20 msgr2=0x7f13c41a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.462+0000 7f13c3fff640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13c410ff20 0x7f13c41a77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.462+0000 7f13c3fff640 1 -- 172.21.15.7:0/592527710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c41a7e50 con 0x7f13c4109000 2024-03-26T08:46:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.462+0000 7f13c8a84640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c41086a0 0x7f13c41a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.462+0000 7f13c9285640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13c410ff20 0x7f13c41a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.463+0000 7f13c3fff640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c4109000 0x7f13c41a0770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f13b800e9f0 tx=0x7f13b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.463+0000 7f13c1ffb640 1 -- 172.21.15.7:0/592527710 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b800ccc0 con 0x7f13c4109000 2024-03-26T08:46:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.463+0000 7f13cace8640 1 -- 172.21.15.7:0/592527710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c41a8140 con 0x7f13c4109000 2024-03-26T08:46:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.463+0000 7f13c1ffb640 1 -- 172.21.15.7:0/592527710 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b8010910 con 0x7f13c4109000 2024-03-26T08:46:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.463+0000 7f13c1ffb640 1 -- 172.21.15.7:0/592527710 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8018960 con 0x7f13c4109000 2024-03-26T08:46:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.463+0000 7f13cace8640 1 -- 172.21.15.7:0/592527710 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c41a86e0 con 0x7f13c4109000 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13c1ffb640 1 -- 172.21.15.7:0/592527710 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f13b8010ac0 con 0x7f13c4109000 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13cd9ae740 1 Processor -- start 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13cd9ae740 1 -- start start 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c88f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 0x55a1c88fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88fe480 0x55a1c8900910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c885db10 con 0x55a1c88f8920 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c8806900 con 0x55a1c88f9310 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c8859bb0 con 0x55a1c88fe480 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13c9285640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c88f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.466+0000 7f13c9285640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c88f8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51114/0 (socket says 172.21.15.7:51114) 2024-03-26T08:46:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c9285640 1 -- 172.21.15.7:0/461411332 learned_addr learned my addr 172.21.15.7:0/461411332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c8a84640 1 --2- 172.21.15.7:0/461411332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 0x55a1c88fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c1ffb640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13ac0a2870 0x7f13ac0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c3fff640 1 --2- 172.21.15.7:0/461411332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88fe480 0x55a1c8900910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c8a84640 1 -- 172.21.15.7:0/461411332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88fe480 msgr2=0x55a1c8900910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c8a84640 1 --2- 172.21.15.7:0/461411332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88fe480 0x55a1c8900910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c8a84640 1 -- 172.21.15.7:0/461411332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 msgr2=0x55a1c88f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c8a84640 1 --2- 172.21.15.7:0/461411332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c88f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c8a84640 1 -- 172.21.15.7:0/461411332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1c8572210 con 0x55a1c88f9310 2024-03-26T08:46:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.467+0000 7f13c1ffb640 1 -- 172.21.15.7:0/592527710 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f13b80cedb0 con 0x7f13c4109000 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.468+0000 7f13c9285640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13ac0a2870 0x7f13ac0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.468+0000 7f13c8a84640 1 --2- 172.21.15.7:0/461411332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 0x55a1c88fde10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f13b40058c0 tx=0x7f13b4011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b63d2b1f7ac25dba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.468+0000 7f13aaffd640 1 -- 172.21.15.7:0/461411332 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b401e070 con 0x55a1c88f9310 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.468+0000 7f13aaffd640 1 -- 172.21.15.7:0/461411332 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b40035f0 con 0x55a1c88f9310 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.468+0000 7f13aaffd640 1 -- 172.21.15.7:0/461411332 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b400f7e0 con 0x55a1c88f9310 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 -- 172.21.15.7:0/461411332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 msgr2=0x55a1c88fde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13c9285640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13ac0a2870 0x7f13ac0a4d30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f13bc005e30 tx=0x7f13bc02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/461411332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 0x55a1c88fde10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f13b40058c0 tx=0x7f13b4011950 comp rx=0 tx=0).stop 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 -- 172.21.15.7:0/461411332 shutdown_connections 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/461411332 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88fe480 0x55a1c8900910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/461411332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 0x55a1c88fde10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/461411332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c88f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 -- 172.21.15.7:0/461411332 >> 172.21.15.7:0/461411332 conn(0x55a1c88f3e90 msgr2=0x55a1c88f4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 -- 172.21.15.7:0/461411332 shutdown_connections 2024-03-26T08:46:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 -- 172.21.15.7:0/461411332 wait complete. 2024-03-26T08:46:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 -- start start 2024-03-26T08:46:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.469+0000 7f13cd9ae740 1 Processor -- start 2024-03-26T08:46:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c89070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88f9310 0x55a1c8907600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88fe480 0x55a1c890e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c8901430 con 0x55a1c88f8920 2024-03-26T08:46:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c8819d50 con 0x55a1c88fe480 2024-03-26T08:46:00.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1c89015b0 con 0x55a1c88f9310 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13cd9ae740 10 client.0 _ll_register_callbacks cb 0x55a1c8819720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c8a84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c89070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c9285640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88fe480 0x55a1c890e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c8a84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c89070c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51128/0 (socket says 172.21.15.7:51128) 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c9285640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88fe480 0x55a1c890e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55238/0 (socket says 172.21.15.7:55238) 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c8a84640 1 -- 172.21.15.7:0/4061025139 learned_addr learned my addr 172.21.15.7:0/4061025139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c8a84640 1 -- 172.21.15.7:0/4061025139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88f9310 msgr2=0x55a1c8907600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c8a84640 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88f9310 0x55a1c8907600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c8a84640 1 -- 172.21.15.7:0/4061025139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88fe480 msgr2=0x55a1c890e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c8a84640 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88fe480 0x55a1c890e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.471+0000 7f13c8a84640 1 -- 172.21.15.7:0/4061025139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1c863e620 con 0x55a1c88f8920 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13c8a84640 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c89070c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009600 tx=0x7f13b4004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43421284c0947363 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13a3fff640 1 -- 172.21.15.7:0/4061025139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b401e040 con 0x55a1c88f8920 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13a3fff640 1 -- 172.21.15.7:0/4061025139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f13b4005040 con 0x55a1c88f8920 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13cd9ae740 1 -- 172.21.15.7:0/4061025139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13b40035f0 con 0x55a1c88f8920 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13a3fff640 1 -- 172.21.15.7:0/4061025139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b4009cb0 con 0x55a1c88f8920 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13cd9ae740 10 client.16329 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13cd9ae740 1 -- 172.21.15.7:0/4061025139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a1c89119b0 con 0x55a1c88f8920 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13a3fff640 1 -- 172.21.15.7:0/4061025139 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b40035f0 con 0x55a1c88f8920 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13a3fff640 1 -- 172.21.15.7:0/4061025139 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f13b400f540 con 0x55a1c88f8920 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.472+0000 7f13cd9ae740 10 client.16329 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.473+0000 7f13cd9ae740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.473+0000 7f13cd9ae740 20 client.16329 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.473+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1c8913400 0x55a1c8915880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.473+0000 7f13cd9ae740 4 client.16329 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:00.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.473+0000 7f13cd9ae740 1 -- 172.21.15.7:0/4061025139 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1c886b050 con 0x55a1c8913400 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.473+0000 7f13c3fff640 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1c8913400 0x55a1c8915880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.473+0000 7f13a3fff640 10 client.16329 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.473+0000 7f13c3fff640 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1c8913400 0x55a1c8915880 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7be6f6ad01363b8 server_cookie=a47cb6a0dd83a1ce in_seq=0 out_seq=0 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13a3fff640 1 -- 172.21.15.7:0/4061025139 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a1c886b050 con 0x55a1c8913400 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13a3fff640 10 client.16329 handle_command_reply: tid=0 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 client.16329 shutdown 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 -- 172.21.15.7:0/4061025139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 msgr2=0x55a1c89070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c89070c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009600 tx=0x7f13b4004230 comp rx=0 tx=0).stop 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 -- 172.21.15.7:0/4061025139 shutdown_connections 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1c8913400 0x55a1c8915880 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88fe480 0x55a1c890e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88f9310 0x55a1c8907600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/4061025139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f8920 0x55a1c89070c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 -- 172.21.15.7:0/4061025139 >> 172.21.15.7:0/4061025139 conn(0x55a1c88f3e90 msgr2=0x55a1c88f3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13a3fff640 0 client.16329 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.474+0000 7f13cd9ae740 1 -- 172.21.15.7:0/4061025139 shutdown_connections 2024-03-26T08:46:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.475+0000 7f13cd9ae740 1 -- 172.21.15.7:0/4061025139 wait complete. 2024-03-26T08:46:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.475+0000 7f13cd9ae740 20 client.16329 trim_cache size 0 max 16384 2024-03-26T08:46:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.497+0000 7f13cd9ae740 1 Processor -- start 2024-03-26T08:46:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.497+0000 7f13cd9ae740 1 -- start start 2024-03-26T08:46:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.497+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f8920 0x55a1c88fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.497+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f9310 0x55a1c88f2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.497+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88fe480 0x55a1c88f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.497+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41b16a0 con 0x55a1c88f9310 2024-03-26T08:46:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.497+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41af1a0 con 0x55a1c88f8920 2024-03-26T08:46:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.497+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41ad220 con 0x55a1c88fe480 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.500+0000 7f13c3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f9310 0x55a1c88f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.500+0000 7f13c3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f9310 0x55a1c88f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51136/0 (socket says 172.21.15.7:51136) 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.500+0000 7f13c3fff640 1 -- 172.21.15.7:0/611100945 learned_addr learned my addr 172.21.15.7:0/611100945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.500+0000 7f13c3fff640 1 -- 172.21.15.7:0/611100945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88fe480 msgr2=0x55a1c88f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.500+0000 7f13c3fff640 1 --2- 172.21.15.7:0/611100945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88fe480 0x55a1c88f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.500+0000 7f13c3fff640 1 -- 172.21.15.7:0/611100945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f8920 msgr2=0x55a1c88fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.500+0000 7f13c3fff640 1 --2- 172.21.15.7:0/611100945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f8920 0x55a1c88fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.500+0000 7f13c3fff640 1 -- 172.21.15.7:0/611100945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1c8901b80 con 0x55a1c88f9310 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.501+0000 7f13c3fff640 1 --2- 172.21.15.7:0/611100945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f9310 0x55a1c88f2790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f13b8097c20 tx=0x7f13b80cc740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22837f8a44472297 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.502+0000 7f13aa7fc640 1 -- 172.21.15.7:0/611100945 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b80a3eb0 con 0x55a1c88f9310 2024-03-26T08:46:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.502+0000 7f13aa7fc640 1 -- 172.21.15.7:0/611100945 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13b8099ac0 con 0x55a1c88f9310 2024-03-26T08:46:00.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.502+0000 7f13aa7fc640 1 -- 172.21.15.7:0/611100945 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b809ae70 con 0x55a1c88f9310 2024-03-26T08:46:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.503+0000 7f13cd9ae740 1 -- 172.21.15.7:0/611100945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f9310 msgr2=0x55a1c88f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.503+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/611100945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f9310 0x55a1c88f2790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f13b8097c20 tx=0x7f13b80cc740 comp rx=0 tx=0).stop 2024-03-26T08:46:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.505+0000 7f13cd9ae740 1 -- 172.21.15.7:0/611100945 shutdown_connections 2024-03-26T08:46:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.505+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/611100945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88fe480 0x55a1c88f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.505+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/611100945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88f9310 0x55a1c88f2790 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.505+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/611100945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f8920 0x55a1c88fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.505+0000 7f13cd9ae740 1 -- 172.21.15.7:0/611100945 >> 172.21.15.7:0/611100945 conn(0x55a1c88f3e90 msgr2=0x55a1c88f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.506+0000 7f13cd9ae740 1 -- 172.21.15.7:0/611100945 shutdown_connections 2024-03-26T08:46:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.506+0000 7f13cd9ae740 1 -- 172.21.15.7:0/611100945 wait complete. 2024-03-26T08:46:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.506+0000 7f13cd9ae740 1 -- start start 2024-03-26T08:46:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.507+0000 7f13cd9ae740 1 Processor -- start 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.507+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88f8920 0x55a1c88fd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.507+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 0x55a1c88fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.507+0000 7f13cd9ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88fe480 0x55a1c8968d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.508+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41ad820 con 0x55a1c88fe480 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.508+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41aeb40 con 0x55a1c88f9310 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.508+0000 7f13cd9ae740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41ade80 con 0x55a1c88f8920 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.508+0000 7f13c9285640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88fe480 0x55a1c8968d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.508+0000 7f13c9285640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88fe480 0x55a1c8968d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51148/0 (socket says 172.21.15.7:51148) 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.508+0000 7f13c9285640 1 -- 172.21.15.7:0/3618099688 learned_addr learned my addr 172.21.15.7:0/3618099688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.508+0000 7f13cd9ae740 10 client.0 _ll_register_callbacks cb 0x7f13b4009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.509+0000 7f13c9285640 1 -- 172.21.15.7:0/3618099688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88f8920 msgr2=0x55a1c88fd2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:46:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.509+0000 7f13c9285640 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88f8920 0x55a1c88fd2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.509+0000 7f13c9285640 1 -- 172.21.15.7:0/3618099688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 msgr2=0x55a1c88fd7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:46:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.509+0000 7f13c9285640 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 0x55a1c88fd7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.509+0000 7f13c9285640 1 -- 172.21.15.7:0/3618099688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1c8969260 con 0x55a1c88fe480 2024-03-26T08:46:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.509+0000 7f13c9285640 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88fe480 0x55a1c8968d20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f13bc037040 tx=0x7f13bc004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d524a4a36393b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.510+0000 7f13a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.510+0000 7f13a3fff640 1 -- 172.21.15.7:0/3618099688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc051020 con 0x55a1c88fe480 2024-03-26T08:46:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.510+0000 7f13a3fff640 1 -- 172.21.15.7:0/3618099688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f13bc0348b0 con 0x55a1c88fe480 2024-03-26T08:46:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.510+0000 7f13cd9ae740 1 -- 172.21.15.7:0/3618099688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13b8099ac0 con 0x55a1c88fe480 2024-03-26T08:46:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.510+0000 7f13a3fff640 1 -- 172.21.15.7:0/3618099688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13bc039040 con 0x55a1c88fe480 2024-03-26T08:46:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.510+0000 7f13cd9ae740 10 client.16335 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.510+0000 7f13cd9ae740 1 -- 172.21.15.7:0/3618099688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a1c896a5a0 con 0x55a1c88fe480 2024-03-26T08:46:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.511+0000 7f13a3fff640 1 -- 172.21.15.7:0/3618099688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13bc046040 con 0x55a1c88fe480 2024-03-26T08:46:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.511+0000 7f13a3fff640 1 -- 172.21.15.7:0/3618099688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f13bc03f040 con 0x55a1c88fe480 2024-03-26T08:46:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.511+0000 7f13cd9ae740 10 client.16335 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.511+0000 7f13cd9ae740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.511+0000 7f13cd9ae740 20 client.16335 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.511+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1c896af00 0x55a1c896d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.511+0000 7f13cd9ae740 4 client.16335 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:46:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.511+0000 7f13cd9ae740 1 -- 172.21.15.7:0/3618099688 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f13b8099ac0 con 0x55a1c896af00 2024-03-26T08:46:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.512+0000 7f13c8a84640 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1c896af00 0x55a1c896d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.513+0000 7f13a3fff640 10 client.16335 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.513+0000 7f13c8a84640 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1c896af00 0x55a1c896d320 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5f4759d50f02f4 server_cookie=b44e68d2be510c22 in_seq=0 out_seq=0 2024-03-26T08:46:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13a3fff640 1 -- 172.21.15.7:0/3618099688 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f13b8099ac0 con 0x55a1c896af00 2024-03-26T08:46:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13a3fff640 10 client.16335 handle_command_reply: tid=0 2024-03-26T08:46:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 client.16335 shutdown 2024-03-26T08:46:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 -- 172.21.15.7:0/3618099688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88fe480 msgr2=0x55a1c8968d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88fe480 0x55a1c8968d20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f13bc037040 tx=0x7f13bc004a80 comp rx=0 tx=0).stop 2024-03-26T08:46:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 -- 172.21.15.7:0/3618099688 shutdown_connections 2024-03-26T08:46:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1c896af00 0x55a1c896d320 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1c88fe480 0x55a1c8968d20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1c88f9310 0x55a1c88fd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 --2- 172.21.15.7:0/3618099688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1c88f8920 0x55a1c88fd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 -- 172.21.15.7:0/3618099688 >> 172.21.15.7:0/3618099688 conn(0x55a1c88f3e90 msgr2=0x55a1c88f2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13a3fff640 0 client.16335 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 -- 172.21.15.7:0/3618099688 shutdown_connections 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 1 -- 172.21.15.7:0/3618099688 wait complete. 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.536+0000 7f13cd9ae740 20 client.16335 trim_cache size 0 max 16384 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:46:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:46:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 -- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13ac0a2870 msgr2=0x7f13ac0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13ac0a2870 0x7f13ac0a4d30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f13bc005e30 tx=0x7f13bc02e040 comp rx=0 tx=0).stop 2024-03-26T08:46:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 -- 172.21.15.7:0/592527710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c4109000 msgr2=0x7f13c41a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c4109000 0x7f13c41a0770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f13b800e9f0 tx=0x7f13b800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 -- 172.21.15.7:0/592527710 shutdown_connections 2024-03-26T08:46:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13ac0a2870 0x7f13ac0a4d30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13c410ff20 0x7f13c41a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c4109000 0x7f13c41a0770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 --2- 172.21.15.7:0/592527710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13c41086a0 0x7f13c41a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 -- 172.21.15.7:0/592527710 >> 172.21.15.7:0/592527710 conn(0x7f13c406eff0 msgr2=0x7f13c4070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 -- 172.21.15.7:0/592527710 shutdown_connections 2024-03-26T08:46:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:00.537+0000 7f13cace8640 1 -- 172.21.15.7:0/592527710 wait complete. 2024-03-26T08:46:00.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:00 smithi007 ceph-mon[32462]: pgmap v467: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 MiB/s wr, 722 op/s 2024-03-26T08:46:00.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:00 smithi044 ceph-mon[36461]: pgmap v467: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 MiB/s wr, 722 op/s 2024-03-26T08:46:00.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:00 smithi016 ceph-mon[35851]: pgmap v467: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.9 MiB/s wr, 722 op/s 2024-03-26T08:46:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 210.0 s, 3754.8 tps, lat 26.771 ms stddev 28.432 2024-03-26T08:46:02.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:02 smithi007 ceph-mon[32462]: pgmap v468: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 798 op/s 2024-03-26T08:46:02.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:02 smithi044 ceph-mon[36461]: pgmap v468: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 798 op/s 2024-03-26T08:46:02.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:02 smithi016 ceph-mon[35851]: pgmap v468: 97 pgs: 97 active+clean; 1.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 798 op/s 2024-03-26T08:46:04.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:46:04.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:46:04.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.359+0000 7f03831b9640 1 -- 172.21.15.7:0/2035146431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c1005c0 msgr2=0x7f037c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.359+0000 7f03831b9640 1 --2- 172.21.15.7:0/2035146431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c1005c0 0x7f037c1009e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f036c009900 tx=0x7f036c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:46:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.360+0000 7f03831b9640 1 -- 172.21.15.7:0/2035146431 shutdown_connections 2024-03-26T08:46:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.360+0000 7f03831b9640 1 --2- 172.21.15.7:0/2035146431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f037c102170 0x7f037c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.360+0000 7f03831b9640 1 --2- 172.21.15.7:0/2035146431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1017b0 0x7f037c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.360+0000 7f03831b9640 1 --2- 172.21.15.7:0/2035146431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c1005c0 0x7f037c1009e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.360+0000 7f03831b9640 1 -- 172.21.15.7:0/2035146431 >> 172.21.15.7:0/2035146431 conn(0x7f037c0fbb30 msgr2=0x7f037c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.360+0000 7f03831b9640 1 -- 172.21.15.7:0/2035146431 shutdown_connections 2024-03-26T08:46:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.360+0000 7f03831b9640 1 -- 172.21.15.7:0/2035146431 wait complete. 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.361+0000 7f03831b9640 1 Processor -- start 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.361+0000 7f03831b9640 1 -- start start 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.361+0000 7f03831b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c1005c0 0x7f037c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.361+0000 7f03831b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1017b0 0x7f037c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.361+0000 7f03831b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f037c102170 0x7f037c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f03831b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c10c280 con 0x7f037c1017b0 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f03831b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c10c100 con 0x7f037c102170 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f03831b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c10c400 con 0x7f037c1005c0 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0380f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c1005c0 0x7f037c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0381756640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f037c102170 0x7f037c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f037bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1017b0 0x7f037c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0380f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c1005c0 0x7f037c197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38944/0 (socket says 172.21.15.7:38944) 2024-03-26T08:46:04.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0381756640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f037c102170 0x7f037c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55266/0 (socket says 172.21.15.7:55266) 2024-03-26T08:46:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0380f55640 1 -- 172.21.15.7:0/330226597 learned_addr learned my addr 172.21.15.7:0/330226597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0381756640 1 -- 172.21.15.7:0/330226597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c1005c0 msgr2=0x7f037c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0381756640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c1005c0 0x7f037c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0381756640 1 -- 172.21.15.7:0/330226597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1017b0 msgr2=0x7f037c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0381756640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1017b0 0x7f037c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.362+0000 7f0381756640 1 -- 172.21.15.7:0/330226597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c19f630 con 0x7f037c102170 2024-03-26T08:46:04.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.363+0000 7f0381756640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f037c102170 0x7f037c19efc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f037000e970 tx=0x7f037000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.363+0000 7f0379ffb640 1 -- 172.21.15.7:0/330226597 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037000ccc0 con 0x7f037c102170 2024-03-26T08:46:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.363+0000 7f0379ffb640 1 -- 172.21.15.7:0/330226597 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0370005020 con 0x7f037c102170 2024-03-26T08:46:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.363+0000 7f0379ffb640 1 -- 172.21.15.7:0/330226597 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370004a00 con 0x7f037c102170 2024-03-26T08:46:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.363+0000 7f03831b9640 1 -- 172.21.15.7:0/330226597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037c19f920 con 0x7f037c102170 2024-03-26T08:46:04.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.363+0000 7f03831b9640 1 -- 172.21.15.7:0/330226597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037c19fdc0 con 0x7f037c102170 2024-03-26T08:46:04.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.366+0000 7f0379ffb640 1 -- 172.21.15.7:0/330226597 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f03700051d0 con 0x7f037c102170 2024-03-26T08:46:04.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0385da4740 1 Processor -- start 2024-03-26T08:46:04.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0385da4740 1 -- start start 2024-03-26T08:46:04.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0379ffb640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f03640a29d0 0x7f03640a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0385da4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f635411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0385da4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f63546290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0385da4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 0x556f63548d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0385da4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f634a63d0 con 0x556f63541790 2024-03-26T08:46:04.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0385da4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6344ead0 con 0x556f63546900 2024-03-26T08:46:04.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0385da4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f634a2470 con 0x556f63540da0 2024-03-26T08:46:04.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0379ffb640 1 -- 172.21.15.7:0/330226597 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f03700ce930 con 0x7f037c102170 2024-03-26T08:46:04.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0380f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f635411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0380f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f635411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38952/0 (socket says 172.21.15.7:38952) 2024-03-26T08:46:04.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0380f55640 1 -- 172.21.15.7:0/242890767 learned_addr learned my addr 172.21.15.7:0/242890767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:04.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.367+0000 7f0381756640 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 0x556f63548d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.368+0000 7f0381756640 1 -- 172.21.15.7:0/242890767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 msgr2=0x556f635411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:04.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.368+0000 7f037bfff640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f03640a29d0 0x7f03640a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.368+0000 7f037bfff640 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f63546290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.368+0000 7f0381756640 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f635411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.368+0000 7f0381756640 1 -- 172.21.15.7:0/242890767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 msgr2=0x556f63546290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:04.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.368+0000 7f0381756640 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f63546290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.368+0000 7f0381756640 1 -- 172.21.15.7:0/242890767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f632847d0 con 0x556f63546900 2024-03-26T08:46:04.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.368+0000 7f037bfff640 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f63546290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:46:04.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.369+0000 7f0381756640 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 0x556f63548d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f03700cc3f0 tx=0x7f037009b290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=575e8ff525019daf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.369+0000 7f0362ffd640 1 -- 172.21.15.7:0/242890767 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700b4070 con 0x556f63546900 2024-03-26T08:46:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.369+0000 7f0362ffd640 1 -- 172.21.15.7:0/242890767 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f037009f170 con 0x556f63546900 2024-03-26T08:46:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.369+0000 7f0362ffd640 1 -- 172.21.15.7:0/242890767 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037001c070 con 0x556f63546900 2024-03-26T08:46:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f0385da4740 1 -- 172.21.15.7:0/242890767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 msgr2=0x556f63548d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f0385da4740 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 0x556f63548d90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f03700cc3f0 tx=0x7f037009b290 comp rx=0 tx=0).stop 2024-03-26T08:46:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f037bfff640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f03640a29d0 0x7f03640a4e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x556f63541b90 tx=0x7f0374009bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f0385da4740 1 -- 172.21.15.7:0/242890767 shutdown_connections 2024-03-26T08:46:04.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f0385da4740 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 0x556f63548d90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f0385da4740 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f63546290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f0385da4740 1 --2- 172.21.15.7:0/242890767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f635411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f0385da4740 1 -- 172.21.15.7:0/242890767 >> 172.21.15.7:0/242890767 conn(0x556f6353c310 msgr2=0x556f6353c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f0385da4740 1 -- 172.21.15.7:0/242890767 shutdown_connections 2024-03-26T08:46:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.370+0000 7f0385da4740 1 -- 172.21.15.7:0/242890767 wait complete. 2024-03-26T08:46:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0385da4740 1 -- start start 2024-03-26T08:46:04.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0385da4740 1 Processor -- start 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0385da4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f6355c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0385da4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f6355c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0385da4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 0x556f635638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0385da4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6355a910 con 0x556f63541790 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0385da4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f63462060 con 0x556f63546900 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0385da4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6355aa90 con 0x556f63540da0 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0385da4740 10 client.0 _ll_register_callbacks cb 0x556f634446a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0380f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f6355c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f037bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f6355c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0380f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f6355c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38954/0 (socket says 172.21.15.7:38954) 2024-03-26T08:46:04.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0380f55640 1 -- 172.21.15.7:0/3224722495 learned_addr learned my addr 172.21.15.7:0/3224722495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.371+0000 7f0381756640 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 0x556f635638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f0380f55640 1 -- 172.21.15.7:0/3224722495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 msgr2=0x556f635638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f0380f55640 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 0x556f635638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f0380f55640 1 -- 172.21.15.7:0/3224722495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 msgr2=0x556f6355c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f0380f55640 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f6355c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f0380f55640 1 -- 172.21.15.7:0/3224722495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f631ba3f0 con 0x556f63540da0 2024-03-26T08:46:04.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f0380f55640 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f6355c400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f036c005ab0 tx=0x7f036c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d144723e04422711 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f035bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f035bfff640 1 -- 172.21.15.7:0/3224722495 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c005d50 con 0x556f63540da0 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f035bfff640 1 -- 172.21.15.7:0/3224722495 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f036c0046c0 con 0x556f63540da0 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.372+0000 7f0385da4740 1 -- 172.21.15.7:0/3224722495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f037009f170 con 0x556f63540da0 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.373+0000 7f035bfff640 1 -- 172.21.15.7:0/3224722495 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f036c0048c0 con 0x556f63540da0 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.373+0000 7f0385da4740 10 client.26197 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.373+0000 7f0385da4740 1 -- 172.21.15.7:0/3224722495 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f6317a0f0 con 0x556f63540da0 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.373+0000 7f035bfff640 1 -- 172.21.15.7:0/3224722495 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f036c004c70 con 0x556f63540da0 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.373+0000 7f035bfff640 1 -- 172.21.15.7:0/3224722495 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f036c0038d0 con 0x556f63540da0 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.374+0000 7f0385da4740 10 client.26197 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.374+0000 7f0385da4740 10 client.26197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.374+0000 7f0385da4740 20 client.26197 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.374+0000 7f0385da4740 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f63567cd0 0x556f6356a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.374+0000 7f0385da4740 4 client.26197 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:04.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.374+0000 7f0385da4740 1 -- 172.21.15.7:0/3224722495 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f037009f170 con 0x556f63567cd0 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.374+0000 7f037bfff640 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f63567cd0 0x556f6356a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f035bfff640 10 client.26197 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f037bfff640 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f63567cd0 0x556f6356a110 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7340eb17894d31a server_cookie=ded5dc6f59a23bcb in_seq=0 out_seq=0 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f035bfff640 1 -- 172.21.15.7:0/3224722495 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f037009f170 con 0x556f63567cd0 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f035bfff640 10 client.26197 handle_command_reply: tid=0 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f0385da4740 1 client.26197 shutdown 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f0385da4740 1 -- 172.21.15.7:0/3224722495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 msgr2=0x556f6355c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f0385da4740 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f6355c400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f036c005ab0 tx=0x7f036c004270 comp rx=0 tx=0).stop 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f0385da4740 1 -- 172.21.15.7:0/3224722495 shutdown_connections 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f0385da4740 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f63567cd0 0x556f6356a110 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f0385da4740 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63546900 0x556f635638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f0385da4740 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f6355c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f0385da4740 1 --2- 172.21.15.7:0/3224722495 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f6355c400 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.375+0000 7f0385da4740 1 -- 172.21.15.7:0/3224722495 >> 172.21.15.7:0/3224722495 conn(0x556f6353c310 msgr2=0x556f6353b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:04.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.376+0000 7f035bfff640 0 client.26197 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:04.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.376+0000 7f0385da4740 1 -- 172.21.15.7:0/3224722495 shutdown_connections 2024-03-26T08:46:04.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.376+0000 7f0385da4740 1 -- 172.21.15.7:0/3224722495 wait complete. 2024-03-26T08:46:04.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.376+0000 7f0385da4740 20 client.26197 trim_cache size 0 max 16384 2024-03-26T08:46:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0385da4740 1 Processor -- start 2024-03-26T08:46:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0385da4740 1 -- start start 2024-03-26T08:46:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0385da4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63540da0 0x556f6353aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0385da4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f635637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0385da4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63546900 0x556f6353b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0385da4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c1a0730 con 0x556f63541790 2024-03-26T08:46:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0385da4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c071a70 con 0x556f63540da0 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0385da4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c06faf0 con 0x556f63546900 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f037bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f635637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0380f55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63540da0 0x556f6353aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f037bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f635637c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51196/0 (socket says 172.21.15.7:51196) 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f037bfff640 1 -- 172.21.15.7:0/3199678906 learned_addr learned my addr 172.21.15.7:0/3199678906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f0381756640 1 --2- 172.21.15.7:0/3199678906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63546900 0x556f6353b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f037bfff640 1 -- 172.21.15.7:0/3199678906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63546900 msgr2=0x556f6353b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f037bfff640 1 --2- 172.21.15.7:0/3199678906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63546900 0x556f6353b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f037bfff640 1 -- 172.21.15.7:0/3199678906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63540da0 msgr2=0x556f6353aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f037bfff640 1 --2- 172.21.15.7:0/3199678906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63540da0 0x556f6353aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.403+0000 7f037bfff640 1 -- 172.21.15.7:0/3199678906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f6353d280 con 0x556f63541790 2024-03-26T08:46:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.405+0000 7f037bfff640 1 --2- 172.21.15.7:0/3199678906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f635637c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x556f63542200 tx=0x7f0374013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d19ce949e6710 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.405+0000 7f03627fc640 1 -- 172.21.15.7:0/3199678906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0374033070 con 0x556f63541790 2024-03-26T08:46:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.405+0000 7f03627fc640 1 -- 172.21.15.7:0/3199678906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0374018d40 con 0x556f63541790 2024-03-26T08:46:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.405+0000 7f03627fc640 1 -- 172.21.15.7:0/3199678906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037400fb40 con 0x556f63541790 2024-03-26T08:46:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.406+0000 7f0385da4740 1 -- 172.21.15.7:0/3199678906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 msgr2=0x556f635637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.406+0000 7f0385da4740 1 --2- 172.21.15.7:0/3199678906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f635637c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x556f63542200 tx=0x7f0374013040 comp rx=0 tx=0).stop 2024-03-26T08:46:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.406+0000 7f0385da4740 1 -- 172.21.15.7:0/3199678906 shutdown_connections 2024-03-26T08:46:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.406+0000 7f0385da4740 1 --2- 172.21.15.7:0/3199678906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63546900 0x556f6353b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.406+0000 7f0385da4740 1 --2- 172.21.15.7:0/3199678906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63541790 0x556f635637c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.406+0000 7f0385da4740 1 --2- 172.21.15.7:0/3199678906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63540da0 0x556f6353aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.406+0000 7f0385da4740 1 -- 172.21.15.7:0/3199678906 >> 172.21.15.7:0/3199678906 conn(0x556f6353c310 msgr2=0x556f63542d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.406+0000 7f0385da4740 1 -- 172.21.15.7:0/3199678906 shutdown_connections 2024-03-26T08:46:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.406+0000 7f0385da4740 1 -- 172.21.15.7:0/3199678906 wait complete. 2024-03-26T08:46:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.407+0000 7f0385da4740 1 -- start start 2024-03-26T08:46:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.407+0000 7f0385da4740 1 Processor -- start 2024-03-26T08:46:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.408+0000 7f0385da4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f63560c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.408+0000 7f0385da4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63541790 0x556f63561140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.408+0000 7f0385da4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63546900 0x556f635621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.408+0000 7f0385da4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c0700f0 con 0x556f63546900 2024-03-26T08:46:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.408+0000 7f0385da4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c071410 con 0x556f63541790 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0385da4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c070750 con 0x556f63540da0 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0381756640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63546900 0x556f635621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0381756640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63546900 0x556f635621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51206/0 (socket says 172.21.15.7:51206) 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0381756640 1 -- 172.21.15.7:0/2909199544 learned_addr learned my addr 172.21.15.7:0/2909199544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0385da4740 10 client.0 _ll_register_callbacks cb 0x7f0370005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f037bfff640 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63541790 0x556f63561140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0380f55640 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f63560c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0381756640 1 -- 172.21.15.7:0/2909199544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 msgr2=0x556f63560c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0381756640 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f63560c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0381756640 1 -- 172.21.15.7:0/2909199544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63541790 msgr2=0x556f63561140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0381756640 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63541790 0x556f63561140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.409+0000 7f0381756640 1 -- 172.21.15.7:0/2909199544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f635627a0 con 0x556f63546900 2024-03-26T08:46:04.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.410+0000 7f0381756640 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63546900 0x556f635621b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0370097c60 tx=0x7f037009b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4675e2e6b15791f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:04.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.410+0000 7f035bfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:04.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.410+0000 7f035bfff640 1 -- 172.21.15.7:0/2909199544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700b9020 con 0x556f63546900 2024-03-26T08:46:04.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.410+0000 7f0385da4740 1 -- 172.21.15.7:0/2909199544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0374018d40 con 0x556f63546900 2024-03-26T08:46:04.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.410+0000 7f035bfff640 1 -- 172.21.15.7:0/2909199544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f03700cc500 con 0x556f63546900 2024-03-26T08:46:04.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f035bfff640 1 -- 172.21.15.7:0/2909199544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f037001c040 con 0x556f63546900 2024-03-26T08:46:04.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f0385da4740 10 client.16350 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:04.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f0385da4740 1 -- 172.21.15.7:0/2909199544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f63563ab0 con 0x556f63546900 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f035bfff640 1 -- 172.21.15.7:0/2909199544 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0374018d40 con 0x556f63546900 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f035bfff640 1 -- 172.21.15.7:0/2909199544 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f03700ac050 con 0x556f63546900 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f0385da4740 10 client.16350 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f0385da4740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f0385da4740 20 client.16350 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f0385da4740 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f635c3c90 0x556f635c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f0385da4740 4 client.16350 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.411+0000 7f0385da4740 1 -- 172.21.15.7:0/2909199544 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556f634b3910 con 0x556f635c3c90 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.412+0000 7f0380f55640 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f635c3c90 0x556f635c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.412+0000 7f035bfff640 10 client.16350 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.412+0000 7f0380f55640 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f635c3c90 0x556f635c40a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362a803a67e7bb45 server_cookie=c30dccbd71c53e32 in_seq=0 out_seq=0 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.412+0000 7f035bfff640 1 -- 172.21.15.7:0/2909199544 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556f634b3910 con 0x556f635c3c90 2024-03-26T08:46:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.412+0000 7f035bfff640 10 client.16350 handle_command_reply: tid=0 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.412+0000 7f0385da4740 1 client.16350 shutdown 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.412+0000 7f0385da4740 1 -- 172.21.15.7:0/2909199544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63546900 msgr2=0x556f635621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.412+0000 7f0385da4740 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63546900 0x556f635621b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0370097c60 tx=0x7f037009b060 comp rx=0 tx=0).stop 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f0385da4740 1 -- 172.21.15.7:0/2909199544 shutdown_connections 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f0385da4740 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f635c3c90 0x556f635c40a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f0385da4740 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f63546900 0x556f635621b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f0385da4740 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f63541790 0x556f63561140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f0385da4740 1 --2- 172.21.15.7:0/2909199544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f63540da0 0x556f63560c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f0385da4740 1 -- 172.21.15.7:0/2909199544 >> 172.21.15.7:0/2909199544 conn(0x556f6353c310 msgr2=0x556f6353f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f035bfff640 0 client.16350 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f0385da4740 1 -- 172.21.15.7:0/2909199544 shutdown_connections 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f0385da4740 1 -- 172.21.15.7:0/2909199544 wait complete. 2024-03-26T08:46:04.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f0385da4740 20 client.16350 trim_cache size 0 max 16384 2024-03-26T08:46:04.408 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:46:04.408 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:46:04.408 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:46:04.408 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f03831b9640 1 -- 172.21.15.7:0/330226597 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f03640a29d0 msgr2=0x7f03640a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f03831b9640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f03640a29d0 0x7f03640a4e90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x556f63541b90 tx=0x7f0374009bf0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f03831b9640 1 -- 172.21.15.7:0/330226597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f037c102170 msgr2=0x7f037c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.413+0000 7f03831b9640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f037c102170 0x7f037c19efc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f037000e970 tx=0x7f037000ee30 comp rx=0 tx=0).stop 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.414+0000 7f03831b9640 1 -- 172.21.15.7:0/330226597 shutdown_connections 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.414+0000 7f03831b9640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f03640a29d0 0x7f03640a4e90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.414+0000 7f03831b9640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f037c102170 0x7f037c19efc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.414+0000 7f03831b9640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037c1017b0 0x7f037c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.414+0000 7f03831b9640 1 --2- 172.21.15.7:0/330226597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f037c1005c0 0x7f037c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.414+0000 7f03831b9640 1 -- 172.21.15.7:0/330226597 >> 172.21.15.7:0/330226597 conn(0x7f037c0fbb30 msgr2=0x7f037c0fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.414+0000 7f03831b9640 1 -- 172.21.15.7:0/330226597 shutdown_connections 2024-03-26T08:46:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.414+0000 7f03831b9640 1 -- 172.21.15.7:0/330226597 wait complete. 2024-03-26T08:46:04.418 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:46:04.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:46:04.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:04 smithi007 ceph-mon[32462]: pgmap v469: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 806 op/s 2024-03-26T08:46:04.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:04 smithi044 ceph-mon[36461]: pgmap v469: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 806 op/s 2024-03-26T08:46:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.597+0000 7fb919548640 1 -- 172.21.15.7:0/21102389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9141038e0 msgr2=0x7fb914103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.597+0000 7fb919548640 1 --2- 172.21.15.7:0/21102389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9141038e0 0x7fb914103d00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb904009900 tx=0x7fb90402bc30 comp rx=0 tx=0).stop 2024-03-26T08:46:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.597+0000 7fb919548640 1 -- 172.21.15.7:0/21102389 shutdown_connections 2024-03-26T08:46:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.597+0000 7fb919548640 1 --2- 172.21.15.7:0/21102389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb914105490 0x7fb91410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.597+0000 7fb919548640 1 --2- 172.21.15.7:0/21102389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb914104ad0 0x7fb914104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.597+0000 7fb919548640 1 --2- 172.21.15.7:0/21102389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9141038e0 0x7fb914103d00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.597+0000 7fb919548640 1 -- 172.21.15.7:0/21102389 >> 172.21.15.7:0/21102389 conn(0x7fb9140fee50 msgr2=0x7fb914101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.597+0000 7fb919548640 1 -- 172.21.15.7:0/21102389 shutdown_connections 2024-03-26T08:46:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.597+0000 7fb919548640 1 -- 172.21.15.7:0/21102389 wait complete. 2024-03-26T08:46:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.598+0000 7fb919548640 1 Processor -- start 2024-03-26T08:46:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.599+0000 7fb919548640 1 -- start start 2024-03-26T08:46:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.599+0000 7fb919548640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9141038e0 0x7fb914195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.599+0000 7fb919548640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb914104ad0 0x7fb914195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.599+0000 7fb919548640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb914105490 0x7fb91419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.599+0000 7fb9127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb914104ad0 0x7fb914195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb912ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9141038e0 0x7fb914195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb9127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb914104ad0 0x7fb914195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51226/0 (socket says 172.21.15.7:51226) 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb912ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9141038e0 0x7fb914195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38994/0 (socket says 172.21.15.7:38994) 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb919548640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91410f580 con 0x7fb914104ad0 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb919548640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91410f400 con 0x7fb914105490 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb919548640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91410f700 con 0x7fb9141038e0 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb9127fc640 1 -- 172.21.15.7:0/2496781699 learned_addr learned my addr 172.21.15.7:0/2496781699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb9137fe640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb914105490 0x7fb91419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb9127fc640 1 -- 172.21.15.7:0/2496781699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9141038e0 msgr2=0x7fb914195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb9127fc640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9141038e0 0x7fb914195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb9127fc640 1 -- 172.21.15.7:0/2496781699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb914105490 msgr2=0x7fb91419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb9127fc640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb914105490 0x7fb91419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb9127fc640 1 -- 172.21.15.7:0/2496781699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9141ae480 con 0x7fb914104ad0 2024-03-26T08:46:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb912ffd640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9141038e0 0x7fb914195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.600+0000 7fb9127fc640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb914104ad0 0x7fb914195d60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00ebf0 tx=0x7fb90c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.601+0000 7fb8fbfff640 1 -- 172.21.15.7:0/2496781699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c010630 con 0x7fb914104ad0 2024-03-26T08:46:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.601+0000 7fb8fbfff640 1 -- 172.21.15.7:0/2496781699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb90c010cb0 con 0x7fb914104ad0 2024-03-26T08:46:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.601+0000 7fb8fbfff640 1 -- 172.21.15.7:0/2496781699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c018bf0 con 0x7fb914104ad0 2024-03-26T08:46:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.601+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9141ae770 con 0x7fb914104ad0 2024-03-26T08:46:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.601+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9141aed10 con 0x7fb914104ad0 2024-03-26T08:46:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.602+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9141aefc0 con 0x7fb914104ad0 2024-03-26T08:46:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.604+0000 7fb8fbfff640 1 -- 172.21.15.7:0/2496781699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb90c0107d0 con 0x7fb914104ad0 2024-03-26T08:46:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.605+0000 7fb8fbfff640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8fc0a2950 0x7fb8fc0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.605+0000 7fb912ffd640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8fc0a2950 0x7fb8fc0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.605+0000 7fb8fbfff640 1 -- 172.21.15.7:0/2496781699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb90c0cf200 con 0x7fb914104ad0 2024-03-26T08:46:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.606+0000 7fb912ffd640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8fc0a2950 0x7fb8fc0a4e10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb904009850 tx=0x7fb90402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:04.612+0000 7fb8fbfff640 1 -- 172.21.15.7:0/2496781699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb90c053740 con 0x7fb914104ad0 2024-03-26T08:46:04.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:04 smithi016 ceph-mon[35851]: pgmap v469: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.8 MiB/s wr, 806 op/s 2024-03-26T08:46:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.004+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9141af230 con 0x7fb914104ad0 2024-03-26T08:46:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.005+0000 7fb8fbfff640 1 -- 172.21.15.7:0/2496781699 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fb90c09c260 con 0x7fb914104ad0 2024-03-26T08:46:04.997 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:46:04.997 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:46:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:46:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8fc0a2950 msgr2=0x7fb8fc0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8fc0a2950 0x7fb8fc0a4e10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb904009850 tx=0x7fb90402df90 comp rx=0 tx=0).stop 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb914104ad0 msgr2=0x7fb914195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb914104ad0 0x7fb914195d60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00ebf0 tx=0x7fb90c00c6a0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 shutdown_connections 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb8fc0a2950 0x7fb8fc0a4e10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb914105490 0x7fb91419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb914104ad0 0x7fb914195d60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 --2- 172.21.15.7:0/2496781699 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb9141038e0 0x7fb914195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.007+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 >> 172.21.15.7:0/2496781699 conn(0x7fb9140fee50 msgr2=0x7fb914077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.008+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 shutdown_connections 2024-03-26T08:46:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.008+0000 7fb919548640 1 -- 172.21.15.7:0/2496781699 wait complete. 2024-03-26T08:46:05.014 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:46:05.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:46:05.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.190+0000 7fd29660b640 1 -- 172.21.15.7:0/1960681498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29010d9c0 msgr2=0x7fd29010de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.190+0000 7fd29660b640 1 --2- 172.21.15.7:0/1960681498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29010d9c0 0x7fd29010de40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd288009ca0 tx=0x7fd28802c120 comp rx=0 tx=0).stop 2024-03-26T08:46:05.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.191+0000 7fd29660b640 1 -- 172.21.15.7:0/1960681498 shutdown_connections 2024-03-26T08:46:05.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.191+0000 7fd29660b640 1 --2- 172.21.15.7:0/1960681498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29010d9c0 0x7fd29010de40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.191+0000 7fd29660b640 1 --2- 172.21.15.7:0/1960681498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29010b010 0x7fd29010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.191+0000 7fd29660b640 1 --2- 172.21.15.7:0/1960681498 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2901086a0 0x7fd29010aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.191+0000 7fd29660b640 1 -- 172.21.15.7:0/1960681498 >> 172.21.15.7:0/1960681498 conn(0x7fd29006e640 msgr2=0x7fd290070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.192+0000 7fd29660b640 1 -- 172.21.15.7:0/1960681498 shutdown_connections 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.193+0000 7fd29660b640 1 -- 172.21.15.7:0/1960681498 wait complete. 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.193+0000 7fd29660b640 1 Processor -- start 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd29660b640 1 -- start start 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd29660b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2901086a0 0x7fd2901a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd29660b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29010b010 0x7fd2901a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd29660b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29010d9c0 0x7fd2901adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd29660b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29011c200 con 0x7fd29010d9c0 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd29660b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29011c080 con 0x7fd29010b010 2024-03-26T08:46:05.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd29660b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29011c380 con 0x7fd2901086a0 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd294ba8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29010d9c0 0x7fd2901adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd294ba8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29010d9c0 0x7fd2901adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51240/0 (socket says 172.21.15.7:51240) 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd294ba8640 1 -- 172.21.15.7:0/3187936184 learned_addr learned my addr 172.21.15.7:0/3187936184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.194+0000 7fd28ffff640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2901086a0 0x7fd2901a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.195+0000 7fd28ffff640 1 -- 172.21.15.7:0/3187936184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29010b010 msgr2=0x7fd2901a6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.195+0000 7fd28ffff640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29010b010 0x7fd2901a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.195+0000 7fd28ffff640 1 -- 172.21.15.7:0/3187936184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29010d9c0 msgr2=0x7fd2901adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.195+0000 7fd28ffff640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29010d9c0 0x7fd2901adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.195+0000 7fd28ffff640 1 -- 172.21.15.7:0/3187936184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2901ae570 con 0x7fd2901086a0 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.195+0000 7fd28ffff640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2901086a0 0x7fd2901a69a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd280009fd0 tx=0x7fd28000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.196+0000 7fd28d7fa640 1 -- 172.21.15.7:0/3187936184 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280015020 con 0x7fd2901086a0 2024-03-26T08:46:05.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.196+0000 7fd29660b640 1 -- 172.21.15.7:0/3187936184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2901ae860 con 0x7fd2901086a0 2024-03-26T08:46:05.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.196+0000 7fd28d7fa640 1 -- 172.21.15.7:0/3187936184 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd280011d10 con 0x7fd2901086a0 2024-03-26T08:46:05.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.196+0000 7fd29660b640 1 -- 172.21.15.7:0/3187936184 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2901aee80 con 0x7fd2901086a0 2024-03-26T08:46:05.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.196+0000 7fd28d7fa640 1 -- 172.21.15.7:0/3187936184 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28000f2e0 con 0x7fd2901086a0 2024-03-26T08:46:05.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.199+0000 7fd2991f4740 1 Processor -- start 2024-03-26T08:46:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.199+0000 7fd2991f4740 1 -- start start 2024-03-26T08:46:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd28d7fa640 1 -- 172.21.15.7:0/3187936184 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd280018020 con 0x7fd2901086a0 2024-03-26T08:46:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0043690 0x55d1d0043ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d0048b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d004b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd2991f4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d0006610 con 0x55d1d0043690 2024-03-26T08:46:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd2991f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cffa93c0 con 0x55d1d0044080 2024-03-26T08:46:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd2991f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cff51d60 con 0x55d1d00491f0 2024-03-26T08:46:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d004b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d004b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39018/0 (socket says 172.21.15.7:39018) 2024-03-26T08:46:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d0048b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d0048b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55336/0 (socket says 172.21.15.7:55336) 2024-03-26T08:46:05.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.200+0000 7fd28ffff640 1 -- 172.21.15.7:0/2660137439 learned_addr learned my addr 172.21.15.7:0/2660137439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.201+0000 7fd28d7fa640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2780a28d0 0x7fd2780a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.201+0000 7fd294ba8640 1 --2- 172.21.15.7:0/2660137439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0043690 0x55d1d0043ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.201+0000 7fd28d7fa640 1 -- 172.21.15.7:0/3187936184 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2800ce1a0 con 0x7fd2901086a0 2024-03-26T08:46:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.201+0000 7fd28f7fe640 1 -- 172.21.15.7:0/2660137439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 msgr2=0x55d1d0048b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.201+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/2660137439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d0048b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.201+0000 7fd28f7fe640 1 -- 172.21.15.7:0/2660137439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0043690 msgr2=0x55d1d0043ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.201+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/2660137439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0043690 0x55d1d0043ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.201+0000 7fd28f7fe640 1 -- 172.21.15.7:0/2660137439 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1cfcbd3f0 con 0x55d1d00491f0 2024-03-26T08:46:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.201+0000 7fd294ba8640 1 --2- 172.21.15.7:0/2660137439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0043690 0x55d1d0043ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:46:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.202+0000 7fd294ba8640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2780a28d0 0x7fd2780a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.202+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/2660137439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d004b680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd284009720 tx=0x7fd284009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad21654c22850bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.203+0000 7fd294ba8640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2780a28d0 0x7fd2780a4d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd288032490 tx=0x7fd28802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.203+0000 7fd2767fc640 1 -- 172.21.15.7:0/2660137439 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284015020 con 0x55d1d00491f0 2024-03-26T08:46:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.203+0000 7fd2767fc640 1 -- 172.21.15.7:0/2660137439 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd28400bda0 con 0x55d1d00491f0 2024-03-26T08:46:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.203+0000 7fd2767fc640 1 -- 172.21.15.7:0/2660137439 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284010380 con 0x55d1d00491f0 2024-03-26T08:46:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 -- 172.21.15.7:0/2660137439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 msgr2=0x55d1d004b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 --2- 172.21.15.7:0/2660137439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d004b680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd284009720 tx=0x7fd284009b20 comp rx=0 tx=0).stop 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 -- 172.21.15.7:0/2660137439 shutdown_connections 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 --2- 172.21.15.7:0/2660137439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d004b680 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 --2- 172.21.15.7:0/2660137439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d0048b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 --2- 172.21.15.7:0/2660137439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0043690 0x55d1d0043ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 -- 172.21.15.7:0/2660137439 >> 172.21.15.7:0/2660137439 conn(0x55d1d003ec00 msgr2=0x55d1d003f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 -- 172.21.15.7:0/2660137439 shutdown_connections 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 -- 172.21.15.7:0/2660137439 wait complete. 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.204+0000 7fd2991f4740 1 -- start start 2024-03-26T08:46:05.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.205+0000 7fd2991f4740 1 Processor -- start 2024-03-26T08:46:05.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.205+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0043690 0x55d1d004da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.205+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d004dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.205+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 0x55d1d0055000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.206+0000 7fd2991f4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cff65240 con 0x55d1d00491f0 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.206+0000 7fd2991f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1cffa5460 con 0x55d1d0044080 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.206+0000 7fd294ba8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 0x55d1d0055000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.206+0000 7fd294ba8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 0x55d1d0055000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51254/0 (socket says 172.21.15.7:51254) 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.206+0000 7fd294ba8640 1 -- 172.21.15.7:0/3556757467 learned_addr learned my addr 172.21.15.7:0/3556757467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.206+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0043690 0x55d1d004da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.206+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d004dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.206+0000 7fd2991f4740 1 -- 172.21.15.7:0/3556757467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1d004c1a0 con 0x55d1d0043690 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.206+0000 7fd28ffff640 1 -- 172.21.15.7:0/3556757467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 msgr2=0x55d1d004dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd28ffff640 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d004dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd28ffff640 1 -- 172.21.15.7:0/3556757467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 msgr2=0x55d1d0055000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd28ffff640 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 0x55d1d0055000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd28ffff640 1 -- 172.21.15.7:0/3556757467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1cfd877d0 con 0x55d1d0043690 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd2991f4740 10 client.0 _ll_register_callbacks cb 0x55d1cff64c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d004dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd2991f4740 1 -- 172.21.15.7:0/3556757467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd28400bda0 con 0x55d1d0043690 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd28ffff640 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0043690 0x55d1d004da80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd280010930 tx=0x7fd280010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f662572f8c9fe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd26f7fe640 10 client.26218 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3556757467 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800b3dc0 con 0x55d1d0043690 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3556757467 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd280099fb0 con 0x55d1d0043690 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3556757467 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd28009a1d0 con 0x55d1d0043690 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd2991f4740 10 client.26218 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.207+0000 7fd2991f4740 1 -- 172.21.15.7:0/3556757467 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55d1d0058210 con 0x55d1d0043690 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.208+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3556757467 <== mon.1 v2:172.21.15.44:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd280011830 con 0x55d1d0043690 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.208+0000 7fd2991f4740 10 client.26218 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.208+0000 7fd2991f4740 10 client.26218 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.208+0000 7fd2991f4740 20 client.26218 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.208+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55d1d0059850 0x55d1d005bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.208+0000 7fd2991f4740 4 client.26218 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.208+0000 7fd2991f4740 1 -- 172.21.15.7:0/3556757467 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1cffb6900 con 0x55d1d0059850 2024-03-26T08:46:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.208+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55d1d0059850 0x55d1d005bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.209+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55d1d0059850 0x55d1d005bcf0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4106a65d330ef72e server_cookie=5920ceba8d46936e in_seq=0 out_seq=0 2024-03-26T08:46:05.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.209+0000 7fd26f7fe640 10 client.26218 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.209+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3556757467 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d1cffb6900 con 0x55d1d0059850 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.209+0000 7fd26f7fe640 10 client.26218 handle_command_reply: tid=0 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 client.26218 shutdown 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 -- 172.21.15.7:0/3556757467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0043690 msgr2=0x55d1d004da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0043690 0x55d1d004da80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd280010930 tx=0x7fd280010f60 comp rx=0 tx=0).stop 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 -- 172.21.15.7:0/3556757467 shutdown_connections 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55d1d0059850 0x55d1d005bcf0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 0x55d1d0055000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0044080 0x55d1d004dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3556757467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0043690 0x55d1d004da80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd26f7fe640 0 client.26218 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:46:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 -- 172.21.15.7:0/3556757467 >> 172.21.15.7:0/3556757467 conn(0x55d1d003ec00 msgr2=0x55d1d003de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 -- 172.21.15.7:0/3556757467 shutdown_connections 2024-03-26T08:46:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 1 -- 172.21.15.7:0/3556757467 wait complete. 2024-03-26T08:46:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.210+0000 7fd2991f4740 20 client.26218 trim_cache size 0 max 16384 2024-03-26T08:46:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.230+0000 7fd2991f4740 1 Processor -- start 2024-03-26T08:46:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.230+0000 7fd2991f4740 1 -- start start 2024-03-26T08:46:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 0x55d1d004dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0044080 0x55d1d0054fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d003d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd2991f4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c0108b0 con 0x55d1d0044080 2024-03-26T08:46:05.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd2991f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00e3b0 con 0x55d1d0043690 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd2991f4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00c430 con 0x55d1d00491f0 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd294ba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d003d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd294ba8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d003d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39040/0 (socket says 172.21.15.7:39040) 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 0x55d1d004dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd294ba8640 1 -- 172.21.15.7:0/2767018359 learned_addr learned my addr 172.21.15.7:0/2767018359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/2767018359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0044080 0x55d1d0054fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd28ffff640 1 -- 172.21.15.7:0/2767018359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 msgr2=0x55d1d003d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.231+0000 7fd28ffff640 1 --2- 172.21.15.7:0/2767018359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d003d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.232+0000 7fd28ffff640 1 -- 172.21.15.7:0/2767018359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0044080 msgr2=0x55d1d0054fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.232+0000 7fd28ffff640 1 --2- 172.21.15.7:0/2767018359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0044080 0x55d1d0054fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.232+0000 7fd28ffff640 1 -- 172.21.15.7:0/2767018359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1d0048820 con 0x55d1d0043690 2024-03-26T08:46:05.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.232+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/2767018359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0044080 0x55d1d0054fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:05.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.232+0000 7fd28ffff640 1 --2- 172.21.15.7:0/2767018359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 0x55d1d004dda0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2800a2d70 tx=0x7fd2800cd000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce013f1c2faaf5e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.232+0000 7fd275ffb640 1 -- 172.21.15.7:0/2767018359 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28009b200 con 0x55d1d0043690 2024-03-26T08:46:05.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.232+0000 7fd275ffb640 1 -- 172.21.15.7:0/2767018359 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd28400bda0 con 0x55d1d0043690 2024-03-26T08:46:05.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.232+0000 7fd275ffb640 1 -- 172.21.15.7:0/2767018359 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280010720 con 0x55d1d0043690 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.233+0000 7fd2991f4740 1 -- 172.21.15.7:0/2767018359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 msgr2=0x55d1d004dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.233+0000 7fd2991f4740 1 --2- 172.21.15.7:0/2767018359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 0x55d1d004dda0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd2800a2d70 tx=0x7fd2800cd000 comp rx=0 tx=0).stop 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.234+0000 7fd2991f4740 1 -- 172.21.15.7:0/2767018359 shutdown_connections 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.234+0000 7fd2991f4740 1 --2- 172.21.15.7:0/2767018359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d00491f0 0x55d1d003d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.234+0000 7fd2991f4740 1 --2- 172.21.15.7:0/2767018359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d0044080 0x55d1d0054fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.234+0000 7fd2991f4740 1 --2- 172.21.15.7:0/2767018359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 0x55d1d004dda0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.234+0000 7fd2991f4740 1 -- 172.21.15.7:0/2767018359 >> 172.21.15.7:0/2767018359 conn(0x55d1d003ec00 msgr2=0x55d1d004d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.234+0000 7fd2991f4740 1 -- 172.21.15.7:0/2767018359 shutdown_connections 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.234+0000 7fd2991f4740 1 -- 172.21.15.7:0/2767018359 wait complete. 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.235+0000 7fd2991f4740 1 -- start start 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.235+0000 7fd2991f4740 1 Processor -- start 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.235+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 0x55d1d00527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.235+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0044080 0x55d1d0052d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.235+0000 7fd2991f4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 0x55d1d0053d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.235+0000 7fd28ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 0x55d1d00527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0044080 0x55d1d0052d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0044080 0x55d1d0052d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39048/0 (socket says 172.21.15.7:39048) 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd2991f4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00ca30 con 0x55d1d00491f0 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd28f7fe640 1 -- 172.21.15.7:0/3493793850 learned_addr learned my addr 172.21.15.7:0/3493793850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00dd50 con 0x55d1d0043690 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c00d090 con 0x55d1d0044080 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd2991f4740 10 client.0 _ll_register_callbacks cb 0x7fd28009a4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd294ba8640 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 0x55d1d0053d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd28f7fe640 1 -- 172.21.15.7:0/3493793850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 msgr2=0x55d1d00527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 0x55d1d00527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd28f7fe640 1 -- 172.21.15.7:0/3493793850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 msgr2=0x55d1d0053d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 0x55d1d0053d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.236+0000 7fd28f7fe640 1 -- 172.21.15.7:0/3493793850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1d0054320 con 0x55d1d0044080 2024-03-26T08:46:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd28f7fe640 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0044080 0x55d1d0052d20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd284009670 tx=0x7fd2840047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2d536bd2ca73dd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd26f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3493793850 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28402c060 con 0x55d1d0044080 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd28400bda0 con 0x55d1d0044080 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3493793850 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2840260d0 con 0x55d1d0044080 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3493793850 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd284015040 con 0x55d1d0044080 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd2991f4740 10 client.26224 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d1d00b52e0 con 0x55d1d0044080 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3493793850 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd28400bda0 con 0x55d1d0044080 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3493793850 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd284031070 con 0x55d1d0044080 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd2991f4740 10 client.26224 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd2991f4740 10 client.26224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.237+0000 7fd2991f4740 20 client.26224 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.238+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55d1d0059850 0x55d1d00b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.238+0000 7fd2991f4740 4 client.26224 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.238+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d1cffb0c50 con 0x55d1d0059850 2024-03-26T08:46:05.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.238+0000 7fd28ffff640 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55d1d0059850 0x55d1d00b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.239+0000 7fd28ffff640 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55d1d0059850 0x55d1d00b7b40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1b83a436514c99f server_cookie=ca02475d8f30d93a in_seq=0 out_seq=0 2024-03-26T08:46:05.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.239+0000 7fd26f7fe640 10 client.26224 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:46:05.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.239+0000 7fd26f7fe640 1 -- 172.21.15.7:0/3493793850 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d1cffb0c50 con 0x55d1d0059850 2024-03-26T08:46:05.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.239+0000 7fd26f7fe640 10 client.26224 handle_command_reply: tid=0 2024-03-26T08:46:05.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.239+0000 7fd2991f4740 1 client.26224 shutdown 2024-03-26T08:46:05.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0044080 msgr2=0x55d1d0052d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0044080 0x55d1d0052d20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd284009670 tx=0x7fd2840047d0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 shutdown_connections 2024-03-26T08:46:05.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55d1d0059850 0x55d1d00b7b40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1d00491f0 0x55d1d0053d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d1d0044080 0x55d1d0052d20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd2991f4740 1 --2- 172.21.15.7:0/3493793850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1d0043690 0x55d1d00527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 >> 172.21.15.7:0/3493793850 conn(0x55d1d003ec00 msgr2=0x55d1d004de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd26f7fe640 0 client.26224 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:46:05.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.240+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 shutdown_connections 2024-03-26T08:46:05.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.241+0000 7fd2991f4740 1 -- 172.21.15.7:0/3493793850 wait complete. 2024-03-26T08:46:05.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.241+0000 7fd2991f4740 20 client.26224 trim_cache size 0 max 16384 2024-03-26T08:46:05.234 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:46:05.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.241+0000 7fd29660b640 1 -- 172.21.15.7:0/3187936184 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2780a28d0 msgr2=0x7fd2780a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.241+0000 7fd29660b640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2780a28d0 0x7fd2780a4d90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd288032490 tx=0x7fd28802e040 comp rx=0 tx=0).stop 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.241+0000 7fd29660b640 1 -- 172.21.15.7:0/3187936184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2901086a0 msgr2=0x7fd2901a69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.241+0000 7fd29660b640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2901086a0 0x7fd2901a69a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd280009fd0 tx=0x7fd28000b870 comp rx=0 tx=0).stop 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.242+0000 7fd29660b640 1 -- 172.21.15.7:0/3187936184 shutdown_connections 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.242+0000 7fd29660b640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd2780a28d0 0x7fd2780a4d90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.242+0000 7fd29660b640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd29010d9c0 0x7fd2901adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.242+0000 7fd29660b640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29010b010 0x7fd2901a6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.242+0000 7fd29660b640 1 --2- 172.21.15.7:0/3187936184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2901086a0 0x7fd2901a69a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.242+0000 7fd29660b640 1 -- 172.21.15.7:0/3187936184 >> 172.21.15.7:0/3187936184 conn(0x7fd29006e640 msgr2=0x7fd290108ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.242+0000 7fd29660b640 1 -- 172.21.15.7:0/3187936184 shutdown_connections 2024-03-26T08:46:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.242+0000 7fd29660b640 1 -- 172.21.15.7:0/3187936184 wait complete. 2024-03-26T08:46:05.246 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:46:05.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:46:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.424+0000 7f9be8d3b640 1 -- 172.21.15.7:0/536846894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9be4100f40 msgr2=0x7f9be41013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.424+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/536846894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9be4100f40 0x7f9be41013c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009ca0 tx=0x7f9bdc02c100 comp rx=0 tx=0).stop 2024-03-26T08:46:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.424+0000 7f9be8d3b640 1 -- 172.21.15.7:0/536846894 shutdown_connections 2024-03-26T08:46:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.424+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/536846894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9be4100f40 0x7f9be41013c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.424+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/536846894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4069960 0x7f9be41002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.424+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/536846894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be40694b0 0x7f9be40ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.424+0000 7f9be8d3b640 1 -- 172.21.15.7:0/536846894 >> 172.21.15.7:0/536846894 conn(0x7f9be40fb2c0 msgr2=0x7f9be40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.425+0000 7f9be8d3b640 1 -- 172.21.15.7:0/536846894 shutdown_connections 2024-03-26T08:46:05.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.425+0000 7f9be8d3b640 1 -- 172.21.15.7:0/536846894 wait complete. 2024-03-26T08:46:05.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.425+0000 7f9be8d3b640 1 Processor -- start 2024-03-26T08:46:05.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.425+0000 7f9be8d3b640 1 -- start start 2024-03-26T08:46:05.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be8d3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be40694b0 0x7f9be4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be8d3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4069960 0x7f9be4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be8d3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9be4100f40 0x7f9be419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be8d3b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be410d3f0 con 0x7f9be4069960 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be8d3b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be410d270 con 0x7f9be40694b0 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be8d3b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be410d570 con 0x7f9be4100f40 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4069960 0x7f9be4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4069960 0x7f9be4197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51284/0 (socket says 172.21.15.7:51284) 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9be4100f40 0x7f9be419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be1d9b640 1 -- 172.21.15.7:0/1094840755 learned_addr learned my addr 172.21.15.7:0/1094840755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be259c640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be40694b0 0x7f9be4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be2d9d640 1 -- 172.21.15.7:0/1094840755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be40694b0 msgr2=0x7f9be4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be2d9d640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be40694b0 0x7f9be4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be2d9d640 1 -- 172.21.15.7:0/1094840755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4069960 msgr2=0x7f9be4197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be2d9d640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4069960 0x7f9be4197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.426+0000 7f9be2d9d640 1 -- 172.21.15.7:0/1094840755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be41092a0 con 0x7f9be4100f40 2024-03-26T08:46:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.427+0000 7f9be2d9d640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9be4100f40 0x7f9be419efe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc037040 tx=0x7f9bdc035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.427+0000 7f9bd37fe640 1 -- 172.21.15.7:0/1094840755 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc039040 con 0x7f9be4100f40 2024-03-26T08:46:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.427+0000 7f9be8d3b640 1 -- 172.21.15.7:0/1094840755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be4109530 con 0x7f9be4100f40 2024-03-26T08:46:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.427+0000 7f9bd37fe640 1 -- 172.21.15.7:0/1094840755 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bdc044070 con 0x7f9be4100f40 2024-03-26T08:46:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.427+0000 7f9bd37fe640 1 -- 172.21.15.7:0/1094840755 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc004400 con 0x7f9be4100f40 2024-03-26T08:46:05.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.427+0000 7f9be8d3b640 1 -- 172.21.15.7:0/1094840755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be4109ac0 con 0x7f9be4100f40 2024-03-26T08:46:05.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.430+0000 7f9bd37fe640 1 -- 172.21.15.7:0/1094840755 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9bdc0045a0 con 0x7f9be4100f40 2024-03-26T08:46:05.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.430+0000 7f9beba03740 1 Processor -- start 2024-03-26T08:46:05.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9beba03740 1 -- start start 2024-03-26T08:46:05.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9beba03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 0x55df66980ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9beba03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df66985b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9beba03740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df669861f0 0x55df66988680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9beba03740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df66943610 con 0x55df669861f0 2024-03-26T08:46:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9beba03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df668e63c0 con 0x55df66981080 2024-03-26T08:46:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9beba03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df6688ed60 con 0x55df66980690 2024-03-26T08:46:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9be259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df66985b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9be259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df66985b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55410/0 (socket says 172.21.15.7:55410) 2024-03-26T08:46:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9be259c640 1 -- 172.21.15.7:0/4192253136 learned_addr learned my addr 172.21.15.7:0/4192253136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.431+0000 7f9bd37fe640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bcc0a2880 0x7f9bcc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/4192253136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 0x55df66980ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/4192253136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df669861f0 0x55df66988680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9bd37fe640 1 -- 172.21.15.7:0/1094840755 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9bdc0f1120 con 0x7f9be4100f40 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9be259c640 1 -- 172.21.15.7:0/4192253136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 msgr2=0x55df66980ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9be259c640 1 --2- 172.21.15.7:0/4192253136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 0x55df66980ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9be259c640 1 -- 172.21.15.7:0/4192253136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df669861f0 msgr2=0x55df66988680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9be259c640 1 --2- 172.21.15.7:0/4192253136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df669861f0 0x55df66988680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9be259c640 1 -- 172.21.15.7:0/4192253136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df665fa3f0 con 0x55df66981080 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9be2d9d640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bcc0a2880 0x7f9bcc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9be259c640 1 --2- 172.21.15.7:0/4192253136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df66985b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009a40 tx=0x7f9bd4009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f713ec579db6e75e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.432+0000 7f9bd0ff9640 1 -- 172.21.15.7:0/4192253136 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4017070 con 0x55df66981080 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.433+0000 7f9bd0ff9640 1 -- 172.21.15.7:0/4192253136 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd4013470 con 0x55df66981080 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.433+0000 7f9bd0ff9640 1 -- 172.21.15.7:0/4192253136 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4012610 con 0x55df66981080 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.433+0000 7f9beba03740 1 -- 172.21.15.7:0/4192253136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 msgr2=0x55df66985b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.433+0000 7f9beba03740 1 --2- 172.21.15.7:0/4192253136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df66985b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009a40 tx=0x7f9bd4009f00 comp rx=0 tx=0).stop 2024-03-26T08:46:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.434+0000 7f9be2d9d640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bcc0a2880 0x7f9bcc0a4d40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0bd780 tx=0x7f9bdc0095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.434+0000 7f9beba03740 1 -- 172.21.15.7:0/4192253136 shutdown_connections 2024-03-26T08:46:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.434+0000 7f9beba03740 1 --2- 172.21.15.7:0/4192253136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df669861f0 0x55df66988680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.434+0000 7f9beba03740 1 --2- 172.21.15.7:0/4192253136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df66985b80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.434+0000 7f9beba03740 1 --2- 172.21.15.7:0/4192253136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 0x55df66980ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.434+0000 7f9beba03740 1 -- 172.21.15.7:0/4192253136 >> 172.21.15.7:0/4192253136 conn(0x55df6697bc00 msgr2=0x55df6697c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.434+0000 7f9beba03740 1 -- 172.21.15.7:0/4192253136 shutdown_connections 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 1 -- 172.21.15.7:0/4192253136 wait complete. 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 1 -- start start 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 1 Processor -- start 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 0x55df66993210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 0x55df66993750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df669861f0 0x55df6699a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df668a2240 con 0x55df66980690 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df668e2460 con 0x55df669861f0 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df669919b0 con 0x55df66981080 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9beba03740 10 client.0 _ll_register_callbacks cb 0x55df668a1c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.435+0000 7f9be1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 0x55df66993210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 0x55df66993210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51296/0 (socket says 172.21.15.7:51296) 2024-03-26T08:46:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be1d9b640 1 -- 172.21.15.7:0/2756515346 learned_addr learned my addr 172.21.15.7:0/2756515346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be259c640 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 0x55df66993750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df669861f0 0x55df6699a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be259c640 1 -- 172.21.15.7:0/2756515346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df669861f0 msgr2=0x55df6699a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be259c640 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df669861f0 0x55df6699a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be259c640 1 -- 172.21.15.7:0/2756515346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 msgr2=0x55df66993210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be259c640 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 0x55df66993210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be259c640 1 -- 172.21.15.7:0/2756515346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df666c47d0 con 0x55df66981080 2024-03-26T08:46:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.436+0000 7f9be259c640 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 0x55df66993750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55df6697cb70 tx=0x7f9bd400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1f71948e3995bd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.437+0000 7f9bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.437+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/2756515346 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4017050 con 0x55df66981080 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.437+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/2756515346 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9bd401c450 con 0x55df66981080 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.437+0000 7f9beba03740 1 -- 172.21.15.7:0/2756515346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bd4013470 con 0x55df66981080 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/2756515346 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bd4013a30 con 0x55df66981080 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9beba03740 10 client.26236 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9beba03740 1 -- 172.21.15.7:0/2756515346 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55df6699db50 con 0x55df66981080 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/2756515346 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd4013470 con 0x55df66981080 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/2756515346 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9bd4026710 con 0x55df66981080 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9beba03740 10 client.26236 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9beba03740 10 client.26236 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9beba03740 20 client.26236 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9beba03740 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55df6699ef50 0x55df669a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9beba03740 4 client.26236 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9beba03740 1 -- 172.21.15.7:0/2756515346 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df668f3900 con 0x55df6699ef50 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.438+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55df6699ef50 0x55df669a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.439+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55df6699ef50 0x55df669a1340 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa224271b5ea0503 server_cookie=5a94f20b66eb3d56 in_seq=0 out_seq=0 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.439+0000 7f9bc9ffb640 10 client.26236 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.439+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/2756515346 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55df668f3900 con 0x55df6699ef50 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.439+0000 7f9bc9ffb640 10 client.26236 handle_command_reply: tid=0 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.440+0000 7f9beba03740 1 client.26236 shutdown 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.440+0000 7f9beba03740 1 -- 172.21.15.7:0/2756515346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 msgr2=0x55df66993750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.440+0000 7f9beba03740 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 0x55df66993750 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55df6697cb70 tx=0x7f9bd400ef90 comp rx=0 tx=0).stop 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.441+0000 7f9beba03740 1 -- 172.21.15.7:0/2756515346 shutdown_connections 2024-03-26T08:46:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.441+0000 7f9beba03740 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df669861f0 0x55df6699a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.441+0000 7f9beba03740 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55df6699ef50 0x55df669a1340 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.441+0000 7f9beba03740 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 0x55df66993750 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.441+0000 7f9beba03740 1 --2- 172.21.15.7:0/2756515346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 0x55df66993210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.441+0000 7f9beba03740 1 -- 172.21.15.7:0/2756515346 >> 172.21.15.7:0/2756515346 conn(0x55df6697bc00 msgr2=0x55df6697ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.441+0000 7f9bc9ffb640 0 client.26236 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:46:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.441+0000 7f9beba03740 1 -- 172.21.15.7:0/2756515346 shutdown_connections 2024-03-26T08:46:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.441+0000 7f9beba03740 1 -- 172.21.15.7:0/2756515346 wait complete. 2024-03-26T08:46:05.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.442+0000 7f9beba03740 20 client.26236 trim_cache size 0 max 16384 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.462+0000 7f9beba03740 1 Processor -- start 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9beba03740 1 -- start start 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9beba03740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df6697bc00 0x55df66991d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9beba03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 0x55df6699a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9beba03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df6697f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9beba03740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a8ec0 con 0x55df6697bc00 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9beba03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a69c0 con 0x55df66981080 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9beba03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a4a40 con 0x55df66980690 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df6697bc00 0x55df66991d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df6697bc00 0x55df66991d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51302/0 (socket says 172.21.15.7:51302) 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 0x55df6699a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be1d9b640 1 -- 172.21.15.7:0/4043856736 learned_addr learned my addr 172.21.15.7:0/4043856736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/4043856736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df6697f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be259c640 1 -- 172.21.15.7:0/4043856736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 msgr2=0x55df6697f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be259c640 1 --2- 172.21.15.7:0/4043856736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df6697f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be259c640 1 -- 172.21.15.7:0/4043856736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df6697bc00 msgr2=0x55df66991d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be259c640 1 --2- 172.21.15.7:0/4043856736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df6697bc00 0x55df66991d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be259c640 1 -- 172.21.15.7:0/4043856736 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df6697ba30 con 0x55df66980690 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.463+0000 7f9be259c640 1 --2- 172.21.15.7:0/4043856736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 0x55df6699a750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009150 tx=0x7f9bd40046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c655c8333a477b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.464+0000 7f9bcbfff640 1 -- 172.21.15.7:0/4043856736 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4017070 con 0x55df66980690 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.464+0000 7f9bcbfff640 1 -- 172.21.15.7:0/4043856736 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd4031520 con 0x55df66980690 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.464+0000 7f9bcbfff640 1 -- 172.21.15.7:0/4043856736 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd401c690 con 0x55df66980690 2024-03-26T08:46:05.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.465+0000 7f9beba03740 1 -- 172.21.15.7:0/4043856736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 msgr2=0x55df6699a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.465+0000 7f9beba03740 1 --2- 172.21.15.7:0/4043856736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 0x55df6699a750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009150 tx=0x7f9bd40046b0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.465+0000 7f9beba03740 1 -- 172.21.15.7:0/4043856736 shutdown_connections 2024-03-26T08:46:05.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.465+0000 7f9beba03740 1 --2- 172.21.15.7:0/4043856736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df66981080 0x55df6697f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.465+0000 7f9beba03740 1 --2- 172.21.15.7:0/4043856736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66980690 0x55df6699a750 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.465+0000 7f9beba03740 1 --2- 172.21.15.7:0/4043856736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df6697bc00 0x55df66991d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.465+0000 7f9beba03740 1 -- 172.21.15.7:0/4043856736 >> 172.21.15.7:0/4043856736 conn(0x55df66981e60 msgr2=0x55df6697ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 -- 172.21.15.7:0/4043856736 shutdown_connections 2024-03-26T08:46:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 -- 172.21.15.7:0/4043856736 wait complete. 2024-03-26T08:46:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 -- start start 2024-03-26T08:46:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 Processor -- start 2024-03-26T08:46:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6697bc00 0x55df6699a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 0x55df6699a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 0x55df66998860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a5040 con 0x55df66980690 2024-03-26T08:46:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a6360 con 0x55df6697bc00 2024-03-26T08:46:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41a56a0 con 0x55df66981080 2024-03-26T08:46:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.466+0000 7f9beba03740 10 client.0 _ll_register_callbacks cb 0x7f9bd4009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6697bc00 0x55df6699a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6697bc00 0x55df6699a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55444/0 (socket says 172.21.15.7:55444) 2024-03-26T08:46:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 -- 172.21.15.7:0/1297459979 learned_addr learned my addr 172.21.15.7:0/1297459979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be259c640 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 0x55df6699a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 0x55df66998860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 -- 172.21.15.7:0/1297459979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 msgr2=0x55df66998860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 0x55df66998860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 -- 172.21.15.7:0/1297459979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 msgr2=0x55df6699a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 0x55df6699a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 -- 172.21.15.7:0/1297459979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df66999090 con 0x55df6697bc00 2024-03-26T08:46:05.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9be1d9b640 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6697bc00 0x55df6699a180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8004650 tx=0x7f9bd80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efc4c180fde8ba5e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.467+0000 7f9bc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:05.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/1297459979 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8003d50 con 0x55df6697bc00 2024-03-26T08:46:05.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/1297459979 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9bd8012500 con 0x55df6697bc00 2024-03-26T08:46:05.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9beba03740 1 -- 172.21.15.7:0/1297459979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bd4031520 con 0x55df6697bc00 2024-03-26T08:46:05.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/1297459979 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bd8012700 con 0x55df6697bc00 2024-03-26T08:46:05.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9beba03740 10 client.26228 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:05.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9beba03740 1 -- 172.21.15.7:0/1297459979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55df669fb2e0 con 0x55df6697bc00 2024-03-26T08:46:05.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/1297459979 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd8012d80 con 0x55df6697bc00 2024-03-26T08:46:05.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/1297459979 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9bd8033a00 con 0x55df6697bc00 2024-03-26T08:46:05.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9beba03740 10 client.26228 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9beba03740 10 client.26228 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:46:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.468+0000 7f9beba03740 20 client.26228 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.469+0000 7f9beba03740 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55df669861f0 0x55df669fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.469+0000 7f9beba03740 4 client.26228 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:46:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.469+0000 7f9beba03740 1 -- 172.21.15.7:0/1297459979 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f9bd4031520 con 0x55df669861f0 2024-03-26T08:46:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.469+0000 7f9be259c640 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55df669861f0 0x55df669fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.469+0000 7f9be259c640 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55df669861f0 0x55df669fdbe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4c93bfa1bdefa7c server_cookie=36fa703e8bd52942 in_seq=0 out_seq=0 2024-03-26T08:46:05.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.469+0000 7f9bc9ffb640 10 client.26228 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:46:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9bc9ffb640 1 -- 172.21.15.7:0/1297459979 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f9bd4031520 con 0x55df669861f0 2024-03-26T08:46:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9bc9ffb640 10 client.26228 handle_command_reply: tid=0 2024-03-26T08:46:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9beba03740 1 client.26228 shutdown 2024-03-26T08:46:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9beba03740 1 -- 172.21.15.7:0/1297459979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6697bc00 msgr2=0x55df6699a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9beba03740 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6697bc00 0x55df6699a180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8004650 tx=0x7f9bd80046c0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9beba03740 1 -- 172.21.15.7:0/1297459979 shutdown_connections 2024-03-26T08:46:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9beba03740 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55df669861f0 0x55df669fdbe0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9beba03740 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55df66981080 0x55df66998860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9beba03740 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df66980690 0x55df6699a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9beba03740 1 --2- 172.21.15.7:0/1297459979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6697bc00 0x55df6699a180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9beba03740 1 -- 172.21.15.7:0/1297459979 >> 172.21.15.7:0/1297459979 conn(0x55df66981e60 msgr2=0x55df6697f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.470+0000 7f9bc9ffb640 0 client.26228 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:46:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.471+0000 7f9beba03740 1 -- 172.21.15.7:0/1297459979 shutdown_connections 2024-03-26T08:46:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.471+0000 7f9beba03740 1 -- 172.21.15.7:0/1297459979 wait complete. 2024-03-26T08:46:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.471+0000 7f9beba03740 20 client.26228 trim_cache size 0 max 16384 2024-03-26T08:46:05.469 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 -- 172.21.15.7:0/1094840755 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bcc0a2880 msgr2=0x7f9bcc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bcc0a2880 0x7f9bcc0a4d40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0bd780 tx=0x7f9bdc0095c0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 -- 172.21.15.7:0/1094840755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9be4100f40 msgr2=0x7f9be419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9be4100f40 0x7f9be419efe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc037040 tx=0x7f9bdc035d10 comp rx=0 tx=0).stop 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 -- 172.21.15.7:0/1094840755 shutdown_connections 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bcc0a2880 0x7f9bcc0a4d40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9be4100f40 0x7f9be419efe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4069960 0x7f9be4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 --2- 172.21.15.7:0/1094840755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9be40694b0 0x7f9be4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.472+0000 7f9be8d3b640 1 -- 172.21.15.7:0/1094840755 >> 172.21.15.7:0/1094840755 conn(0x7f9be40fb2c0 msgr2=0x7f9be40fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.473+0000 7f9be8d3b640 1 -- 172.21.15.7:0/1094840755 shutdown_connections 2024-03-26T08:46:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.473+0000 7f9be8d3b640 1 -- 172.21.15.7:0/1094840755 wait complete. 2024-03-26T08:46:05.480 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:46:05.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:46:05.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:05 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2496781699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:46:05.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:05 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2496781699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:46:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.658+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1833807895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81086a0 msgr2=0x7fa0c810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.658+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1833807895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81086a0 0x7fa0c810aad0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009860 tx=0x7fa0bc02bb70 comp rx=0 tx=0).stop 2024-03-26T08:46:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.658+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1833807895 shutdown_connections 2024-03-26T08:46:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.658+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1833807895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c8111de0 0x7fa0c81141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.658+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1833807895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c810b010 0x7fa0c810b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.658+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1833807895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c81086a0 0x7fa0c810aad0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.658+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1833807895 >> 172.21.15.7:0/1833807895 conn(0x7fa0c806e640 msgr2=0x7fa0c8070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.658+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1833807895 shutdown_connections 2024-03-26T08:46:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.659+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1833807895 wait complete. 2024-03-26T08:46:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.659+0000 7fa0cdd09640 1 Processor -- start 2024-03-26T08:46:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0cdd09640 1 -- start start 2024-03-26T08:46:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0cdd09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c81086a0 0x7fa0c81a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0cdd09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c810b010 0x7fa0c81a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0cdd09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c8111de0 0x7fa0c81add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0cdd09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8117d30 con 0x7fa0c8111de0 2024-03-26T08:46:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0cdd09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8117bb0 con 0x7fa0c81086a0 2024-03-26T08:46:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0cdd09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8117eb0 con 0x7fa0c810b010 2024-03-26T08:46:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c810b010 0x7fa0c81a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c810b010 0x7fa0c81a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39132/0 (socket says 172.21.15.7:39132) 2024-03-26T08:46:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/1742980912 learned_addr learned my addr 172.21.15.7:0/1742980912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.660+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c8111de0 0x7fa0c81add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c81086a0 0x7fa0c81a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/1742980912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c81086a0 msgr2=0x7fa0c81a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c81086a0 0x7fa0c81a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c8111de0 msgr2=0x7fa0c81add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c8111de0 0x7fa0c81add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/1742980912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c81ae400 con 0x7fa0c810b010 2024-03-26T08:46:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c8111de0 0x7fa0c81add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c81086a0 0x7fa0c81a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:46:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c810b010 0x7fa0c81a6d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800e9c0 tx=0x7fa0b800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.661+0000 7fa0c4ff9640 1 -- 172.21.15.7:0/1742980912 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800ccc0 con 0x7fa0c810b010 2024-03-26T08:46:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.662+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1742980912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c81ae6f0 con 0x7fa0c810b010 2024-03-26T08:46:05.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.662+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1742980912 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c81aec90 con 0x7fa0c810b010 2024-03-26T08:46:05.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.662+0000 7fa0c4ff9640 1 -- 172.21.15.7:0/1742980912 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0b8005500 con 0x7fa0c810b010 2024-03-26T08:46:05.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.662+0000 7fa0c4ff9640 1 -- 172.21.15.7:0/1742980912 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8010430 con 0x7fa0c810b010 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0d09d2740 1 Processor -- start 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0d09d2740 1 -- start start 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4a690 0x56128ba4aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4b080 0x56128ba4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba52680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0d09d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ba0d610 con 0x56128ba501f0 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0d09d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b9b03c0 con 0x56128ba4b080 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0d09d2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b958d60 con 0x56128ba4a690 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4b080 0x56128ba4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4b080 0x56128ba4fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55464/0 (socket says 172.21.15.7:55464) 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0c7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4a690 0x56128ba4aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.665+0000 7fa0c77fe640 1 -- 172.21.15.7:0/1743048249 learned_addr learned my addr 172.21.15.7:0/1743048249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.666+0000 7fa0c77fe640 1 -- 172.21.15.7:0/1743048249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4a690 msgr2=0x56128ba4aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.666+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/1743048249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4a690 0x56128ba4aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.666+0000 7fa0c77fe640 1 -- 172.21.15.7:0/1743048249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 msgr2=0x56128ba52680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.666+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/1743048249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba52680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.666+0000 7fa0c77fe640 1 -- 172.21.15.7:0/1743048249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56128b6c43f0 con 0x56128ba4b080 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.666+0000 7fa0c4ff9640 1 -- 172.21.15.7:0/1742980912 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa0b8005020 con 0x7fa0c810b010 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.667+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/1743048249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4b080 0x56128ba4fb80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc000c00 tx=0x7fa0bc032340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9475cd6eb22f92ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.667+0000 7fa0adffb640 1 -- 172.21.15.7:0/1743048249 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc039040 con 0x56128ba4b080 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.667+0000 7fa0adffb640 1 -- 172.21.15.7:0/1743048249 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0bc043070 con 0x56128ba4b080 2024-03-26T08:46:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.667+0000 7fa0adffb640 1 -- 172.21.15.7:0/1743048249 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0053a0 con 0x56128ba4b080 2024-03-26T08:46:05.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.667+0000 7fa0c4ff9640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0b00a2900 0x7fa0b00a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.667+0000 7fa0c4ff9640 1 -- 172.21.15.7:0/1742980912 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa0b80cec60 con 0x7fa0c810b010 2024-03-26T08:46:05.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0b00a2900 0x7fa0b00a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 -- 172.21.15.7:0/1743048249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4b080 msgr2=0x56128ba4fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/1743048249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4b080 0x56128ba4fb80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc000c00 tx=0x7fa0bc032340 comp rx=0 tx=0).stop 2024-03-26T08:46:05.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0b00a2900 0x7fa0b00a4dc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x56128ba4b8e0 tx=0x7fa0c0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 -- 172.21.15.7:0/1743048249 shutdown_connections 2024-03-26T08:46:05.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/1743048249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba52680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/1743048249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4b080 0x56128ba4fb80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/1743048249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4a690 0x56128ba4aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 -- 172.21.15.7:0/1743048249 >> 172.21.15.7:0/1743048249 conn(0x56128ba45c00 msgr2=0x56128ba46000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 -- 172.21.15.7:0/1743048249 shutdown_connections 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 -- 172.21.15.7:0/1743048249 wait complete. 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.668+0000 7fa0d09d2740 1 -- start start 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.669+0000 7fa0d09d2740 1 Processor -- start 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.669+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.669+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 0x56128ba55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.669+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba5c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.669+0000 7fa0d09d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b96c240 con 0x56128ba501f0 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.669+0000 7fa0d09d2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128b9ac460 con 0x56128ba4a690 2024-03-26T08:46:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.669+0000 7fa0d09d2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ba53140 con 0x56128ba4b080 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.669+0000 7fa0d09d2740 10 client.0 _ll_register_callbacks cb 0x56128b96bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.670+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.670+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55474/0 (socket says 172.21.15.7:55474) 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.670+0000 7fa0c77fe640 1 -- 172.21.15.7:0/2643724783 learned_addr learned my addr 172.21.15.7:0/2643724783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.670+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba5c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.670+0000 7fa0c7fff640 1 -- 172.21.15.7:0/2643724783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 msgr2=0x56128ba55000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.670+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 0x56128ba55000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.670+0000 7fa0c7fff640 1 -- 172.21.15.7:0/2643724783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 msgr2=0x56128ba54ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.670+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.670+0000 7fa0c7fff640 1 -- 172.21.15.7:0/2643724783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56128b78e7d0 con 0x56128ba501f0 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.671+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba5c040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0003f20 tx=0x7fa0c0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85429f0a1ca66b20 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.671+0000 7fa0a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.671+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/2643724783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00045c0 con 0x56128ba501f0 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.671+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/2643724783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa0c0004760 con 0x56128ba501f0 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.671+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2643724783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0bc043070 con 0x56128ba501f0 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.671+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/2643724783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0c0029440 con 0x56128ba501f0 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.671+0000 7fa0d09d2740 10 client.16398 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.671+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2643724783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56128ba5f4b0 con 0x56128ba501f0 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.671+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/2643724783 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0bc043070 con 0x56128ba501f0 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.672+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/2643724783 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa0c0027070 con 0x56128ba501f0 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.672+0000 7fa0d09d2740 10 client.16398 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.672+0000 7fa0d09d2740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.672+0000 7fa0d09d2740 20 client.16398 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.672+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56128ba61010 0x56128ba634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.672+0000 7fa0d09d2740 4 client.16398 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.672+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2643724783 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56128b9bd900 con 0x56128ba61010 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.672+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56128ba61010 0x56128ba634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.673+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56128ba61010 0x56128ba634b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=842cbaa71873bc65 server_cookie=8e55272037a50cca in_seq=0 out_seq=0 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.673+0000 7fa0a6ffd640 10 client.16398 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/2643724783 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56128b9bd900 con 0x56128ba61010 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0a6ffd640 10 client.16398 handle_command_reply: tid=0 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 client.16398 shutdown 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2643724783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 msgr2=0x56128ba5c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba5c040 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0003f20 tx=0x7fa0c0004ab0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2643724783 shutdown_connections 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56128ba61010 0x56128ba634b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba5c040 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 0x56128ba55000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2643724783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2643724783 >> 172.21.15.7:0/2643724783 conn(0x56128ba45c00 msgr2=0x56128ba44e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0a6ffd640 0 client.16398 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2643724783 shutdown_connections 2024-03-26T08:46:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2643724783 wait complete. 2024-03-26T08:46:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.674+0000 7fa0d09d2740 20 client.16398 trim_cache size 0 max 16384 2024-03-26T08:46:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0d09d2740 1 Processor -- start 2024-03-26T08:46:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0d09d2740 1 -- start start 2024-03-26T08:46:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba4b080 0x56128ba55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba501f0 0x56128ba5c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0d09d2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40108b0 con 0x56128ba4b080 2024-03-26T08:46:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55480/0 (socket says 172.21.15.7:55480) 2024-03-26T08:46:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0c77fe640 1 -- 172.21.15.7:0/2452535341 learned_addr learned my addr 172.21.15.7:0/2452535341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2452535341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b400e3b0 con 0x56128ba4a690 2024-03-26T08:46:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba4b080 0x56128ba55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.695+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2452535341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b400c430 con 0x56128ba501f0 2024-03-26T08:46:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.696+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba501f0 0x56128ba5c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.696+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/2452535341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba501f0 msgr2=0x56128ba5c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.696+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba501f0 0x56128ba5c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.696+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/2452535341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 msgr2=0x56128ba54ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.696+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.696+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/2452535341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56128ba4b460 con 0x56128ba4b080 2024-03-26T08:46:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.696+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:05.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.696+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba4b080 0x56128ba55000 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x56128ba55fa0 tx=0x7fa0b809b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9327281b93018ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.697+0000 7fa0ad7fa640 1 -- 172.21.15.7:0/2452535341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80b3020 con 0x56128ba4b080 2024-03-26T08:46:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.697+0000 7fa0ad7fa640 1 -- 172.21.15.7:0/2452535341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0b809b5c0 con 0x56128ba4b080 2024-03-26T08:46:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.697+0000 7fa0ad7fa640 1 -- 172.21.15.7:0/2452535341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80b4260 con 0x56128ba4b080 2024-03-26T08:46:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2452535341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba4b080 msgr2=0x56128ba55000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba4b080 0x56128ba55000 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x56128ba55fa0 tx=0x7fa0b809b450 comp rx=0 tx=0).stop 2024-03-26T08:46:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2452535341 shutdown_connections 2024-03-26T08:46:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba501f0 0x56128ba5c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba4b080 0x56128ba55000 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/2452535341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba54ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2452535341 >> 172.21.15.7:0/2452535341 conn(0x56128ba45c00 msgr2=0x56128ba63020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2452535341 shutdown_connections 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 -- 172.21.15.7:0/2452535341 wait complete. 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 -- start start 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.698+0000 7fa0d09d2740 1 Processor -- start 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba59130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 0x56128ba59670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0d09d2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba5a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 0x56128ba59670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0c6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 0x56128ba59670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39158/0 (socket says 172.21.15.7:39158) 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/4193039519 learned_addr learned my addr 172.21.15.7:0/4193039519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b400ca30 con 0x56128ba501f0 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b400dd50 con 0x56128ba4a690 2024-03-26T08:46:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b400d090 con 0x56128ba4b080 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0d09d2740 10 client.0 _ll_register_callbacks cb 0x7fa0c0003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.699+0000 7fa0c7fff640 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba5a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.700+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/4193039519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 msgr2=0x56128ba59130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.700+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba59130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.700+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba59130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.700+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/4193039519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 msgr2=0x56128ba5a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.700+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba5a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.700+0000 7fa0c6ffd640 1 -- 172.21.15.7:0/4193039519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56128ba5ac70 con 0x56128ba4b080 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.700+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba59130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.700+0000 7fa0c6ffd640 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 0x56128ba59670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0b809b2c0 tx=0x7fa0b8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a35c78588a52c7d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.700+0000 7fa0a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.701+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/4193039519 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b80b3070 con 0x56128ba4b080 2024-03-26T08:46:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.701+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/4193039519 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa0b80ccde0 con 0x56128ba4b080 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.701+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0b809b5c0 con 0x56128ba4b080 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.701+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/4193039519 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0b80b4470 con 0x56128ba4b080 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.701+0000 7fa0d09d2740 10 client.26257 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.701+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56128ba5bf70 con 0x56128ba4b080 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.701+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/4193039519 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0b809b5c0 con 0x56128ba4b080 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.702+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/4193039519 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa0b8084c70 con 0x56128ba4b080 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.702+0000 7fa0d09d2740 10 client.26257 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.702+0000 7fa0d09d2740 10 client.26257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.702+0000 7fa0d09d2740 20 client.26257 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.702+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56128babcbb0 0x56128babefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.702+0000 7fa0d09d2740 4 client.26257 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.702+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56128b9b7c50 con 0x56128babcbb0 2024-03-26T08:46:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.702+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56128babcbb0 0x56128babefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.703+0000 7fa0a6ffd640 10 client.26257 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.703+0000 7fa0c77fe640 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56128babcbb0 0x56128babefd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd6e3078c4d458f server_cookie=1266a263d6c4716f in_seq=0 out_seq=0 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.703+0000 7fa0a6ffd640 1 -- 172.21.15.7:0/4193039519 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56128b9b7c50 con 0x56128babcbb0 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.703+0000 7fa0a6ffd640 10 client.26257 handle_command_reply: tid=0 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 client.26257 shutdown 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 msgr2=0x56128ba59670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 0x56128ba59670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0b809b2c0 tx=0x7fa0b8005880 comp rx=0 tx=0).stop 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 shutdown_connections 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56128babcbb0 0x56128babefd0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56128ba501f0 0x56128ba5a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56128ba4b080 0x56128ba59670 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 --2- 172.21.15.7:0/4193039519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56128ba4a690 0x56128ba59130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0a6ffd640 0 client.26257 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 >> 172.21.15.7:0/4193039519 conn(0x56128ba45c00 msgr2=0x56128ba4c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 shutdown_connections 2024-03-26T08:46:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 1 -- 172.21.15.7:0/4193039519 wait complete. 2024-03-26T08:46:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.704+0000 7fa0d09d2740 20 client.26257 trim_cache size 0 max 16384 2024-03-26T08:46:05.699 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:46:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0b00a2900 msgr2=0x7fa0b00a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0b00a2900 0x7fa0b00a4dc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x56128ba4b8e0 tx=0x7fa0c0009500 comp rx=0 tx=0).stop 2024-03-26T08:46:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1742980912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c810b010 msgr2=0x7fa0c81a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c810b010 0x7fa0c81a6d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800e9c0 tx=0x7fa0b800ee80 comp rx=0 tx=0).stop 2024-03-26T08:46:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1742980912 shutdown_connections 2024-03-26T08:46:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa0b00a2900 0x7fa0b00a4dc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0c8111de0 0x7fa0c81add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa0c810b010 0x7fa0c81a6d20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 --2- 172.21.15.7:0/1742980912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0c81086a0 0x7fa0c81a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.705+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1742980912 >> 172.21.15.7:0/1742980912 conn(0x7fa0c806e640 msgr2=0x7fa0c8108a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.706+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1742980912 shutdown_connections 2024-03-26T08:46:05.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:05.706+0000 7fa0cdd09640 1 -- 172.21.15.7:0/1742980912 wait complete. 2024-03-26T08:46:05.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:05 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2496781699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:46:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 215.0 s, 3493.6 tps, lat 28.536 ms stddev 30.933 2024-03-26T08:46:06.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:06 smithi007 ceph-mon[32462]: pgmap v470: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 769 op/s 2024-03-26T08:46:06.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:06 smithi044 ceph-mon[36461]: pgmap v470: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 769 op/s 2024-03-26T08:46:06.713 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:46:06.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:46:06.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:46:06.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:06 smithi016 ceph-mon[35851]: pgmap v470: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 769 op/s 2024-03-26T08:46:06.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.877+0000 7f8d8f29a640 1 -- 172.21.15.7:0/2507249633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d881005c0 msgr2=0x7f8d881009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.877+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/2507249633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d881005c0 0x7f8d881009e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009be0 tx=0x7f8d8002bef0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.877+0000 7f8d8f29a640 1 -- 172.21.15.7:0/2507249633 shutdown_connections 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.877+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/2507249633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d88102170 0x7f8d88108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.877+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/2507249633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d881017b0 0x7f8d88101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.877+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/2507249633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d881005c0 0x7f8d881009e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.877+0000 7f8d8f29a640 1 -- 172.21.15.7:0/2507249633 >> 172.21.15.7:0/2507249633 conn(0x7f8d880fbb30 msgr2=0x7f8d880fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.877+0000 7f8d8f29a640 1 -- 172.21.15.7:0/2507249633 shutdown_connections 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.877+0000 7f8d8f29a640 1 -- 172.21.15.7:0/2507249633 wait complete. 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.878+0000 7f8d8f29a640 1 Processor -- start 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.878+0000 7f8d8f29a640 1 -- start start 2024-03-26T08:46:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.878+0000 7f8d8f29a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d881005c0 0x7f8d8819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.878+0000 7f8d8f29a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d881017b0 0x7f8d8819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.878+0000 7f8d8f29a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d88102170 0x7f8d881a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.878+0000 7f8d8f29a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8810b160 con 0x7f8d881017b0 2024-03-26T08:46:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.878+0000 7f8d8f29a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8810afe0 con 0x7f8d881005c0 2024-03-26T08:46:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.879+0000 7f8d8f29a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8810b2e0 con 0x7f8d88102170 2024-03-26T08:46:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.879+0000 7f8d8c835640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d881017b0 0x7f8d8819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.879+0000 7f8d8c835640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d881017b0 0x7f8d8819e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44900/0 (socket says 172.21.15.7:44900) 2024-03-26T08:46:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.879+0000 7f8d8c835640 1 -- 172.21.15.7:0/1040620302 learned_addr learned my addr 172.21.15.7:0/1040620302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.879+0000 7f8d8d837640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d88102170 0x7f8d881a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.880+0000 7f8d8c835640 1 -- 172.21.15.7:0/1040620302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d88102170 msgr2=0x7f8d881a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:06.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.880+0000 7f8d8c835640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d88102170 0x7f8d881a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.880+0000 7f8d8c835640 1 -- 172.21.15.7:0/1040620302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d881005c0 msgr2=0x7f8d8819e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:06.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.880+0000 7f8d8c835640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d881005c0 0x7f8d8819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.880+0000 7f8d8c835640 1 -- 172.21.15.7:0/1040620302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8806b1f0 con 0x7f8d881017b0 2024-03-26T08:46:06.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.880+0000 7f8d8c835640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d881017b0 0x7f8d8819e560 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800e9f0 tx=0x7f8d7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.885+0000 7f8d7e7fc640 1 -- 172.21.15.7:0/1040620302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7800cd80 con 0x7f8d881017b0 2024-03-26T08:46:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.885+0000 7f8d7e7fc640 1 -- 172.21.15.7:0/1040620302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d78010910 con 0x7f8d881017b0 2024-03-26T08:46:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.885+0000 7f8d7e7fc640 1 -- 172.21.15.7:0/1040620302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78018990 con 0x7f8d881017b0 2024-03-26T08:46:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.886+0000 7f8d8f29a640 1 -- 172.21.15.7:0/1040620302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8806b4e0 con 0x7f8d881017b0 2024-03-26T08:46:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.886+0000 7f8d8f29a640 1 -- 172.21.15.7:0/1040620302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8806b9e0 con 0x7f8d881017b0 2024-03-26T08:46:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.891+0000 7f8d91f63740 1 Processor -- start 2024-03-26T08:46:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.894+0000 7f8d91f63740 1 -- start start 2024-03-26T08:46:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.895+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e3960 0x5636314e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.895+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 0x5636314e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.895+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e94c0 0x5636314eb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.895+0000 7f8d91f63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563631423c90 con 0x5636314e3960 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.895+0000 7f8d91f63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563631449070 con 0x5636314e94c0 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.895+0000 7f8d91f63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636313f1c80 con 0x5636314e4350 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.895+0000 7f8d8d837640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e3960 0x5636314e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.895+0000 7f8d8d837640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e3960 0x5636314e3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44908/0 (socket says 172.21.15.7:44908) 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.895+0000 7f8d8d837640 1 -- 172.21.15.7:0/3616907262 learned_addr learned my addr 172.21.15.7:0/3616907262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.897+0000 7f8d8c835640 1 --2- 172.21.15.7:0/3616907262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e94c0 0x5636314eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.897+0000 7f8d7e7fc640 1 -- 172.21.15.7:0/1040620302 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8d78010ac0 con 0x7f8d881017b0 2024-03-26T08:46:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.898+0000 7f8d7e7fc640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8d740a2a10 0x7f8d740a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.898+0000 7f8d7e7fc640 1 -- 172.21.15.7:0/1040620302 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8d780cee50 con 0x7f8d881017b0 2024-03-26T08:46:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.899+0000 7f8d8c835640 1 -- 172.21.15.7:0/3616907262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 msgr2=0x5636314e8e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:46:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.899+0000 7f8d8c835640 1 --2- 172.21.15.7:0/3616907262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 0x5636314e8e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.899+0000 7f8d8c835640 1 -- 172.21.15.7:0/3616907262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e3960 msgr2=0x5636314e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.899+0000 7f8d8c835640 1 --2- 172.21.15.7:0/3616907262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e3960 0x5636314e3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.899+0000 7f8d8c835640 1 -- 172.21.15.7:0/3616907262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636313e9d10 con 0x5636314e94c0 2024-03-26T08:46:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.901+0000 7f8d8d837640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8d740a2a10 0x7f8d740a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.903+0000 7f8d8d837640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8d740a2a10 0x7f8d740a4ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5636314e4bb0 tx=0x7f8d840046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:06.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.905+0000 7f8d8c835640 1 --2- 172.21.15.7:0/3616907262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e94c0 0x5636314eb950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d78098180 tx=0x7f8d7809b110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94b143ffb10c0c11 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:06.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.906+0000 7f8d737fe640 1 -- 172.21.15.7:0/3616907262 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7809b7b0 con 0x5636314e94c0 2024-03-26T08:46:06.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.906+0000 7f8d737fe640 1 -- 172.21.15.7:0/3616907262 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d7809f900 con 0x5636314e94c0 2024-03-26T08:46:06.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.906+0000 7f8d737fe640 1 -- 172.21.15.7:0/3616907262 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78018b30 con 0x5636314e94c0 2024-03-26T08:46:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.907+0000 7f8d91f63740 1 -- 172.21.15.7:0/3616907262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e94c0 msgr2=0x5636314eb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.907+0000 7f8d91f63740 1 --2- 172.21.15.7:0/3616907262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e94c0 0x5636314eb950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d78098180 tx=0x7f8d7809b110 comp rx=0 tx=0).stop 2024-03-26T08:46:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.910+0000 7f8d91f63740 1 -- 172.21.15.7:0/3616907262 shutdown_connections 2024-03-26T08:46:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.910+0000 7f8d91f63740 1 --2- 172.21.15.7:0/3616907262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e94c0 0x5636314eb950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.910+0000 7f8d91f63740 1 --2- 172.21.15.7:0/3616907262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 0x5636314e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.910+0000 7f8d91f63740 1 --2- 172.21.15.7:0/3616907262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e3960 0x5636314e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.910+0000 7f8d91f63740 1 -- 172.21.15.7:0/3616907262 >> 172.21.15.7:0/3616907262 conn(0x5636314ac1a0 msgr2=0x5636314df2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.911+0000 7f8d91f63740 1 -- 172.21.15.7:0/3616907262 shutdown_connections 2024-03-26T08:46:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.911+0000 7f8d91f63740 1 -- 172.21.15.7:0/3616907262 wait complete. 2024-03-26T08:46:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.911+0000 7f8d91f63740 1 -- start start 2024-03-26T08:46:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.912+0000 7f8d91f63740 1 Processor -- start 2024-03-26T08:46:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.912+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.912+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e4350 0x5636314ee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.912+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.912+0000 7f8d91f63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636314050d0 con 0x5636314e4350 2024-03-26T08:46:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.912+0000 7f8d91f63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563631445110 con 0x5636314e3960 2024-03-26T08:46:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.912+0000 7f8d91f63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636314ec410 con 0x5636314e94c0 2024-03-26T08:46:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.912+0000 7f8d91f63740 10 client.0 _ll_register_callbacks cb 0x563631440e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.912+0000 7f8d8c835640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e4350 0x5636314ee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.913+0000 7f8d8d036640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.913+0000 7f8d8d036640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314edd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35802/0 (socket says 172.21.15.7:35802) 2024-03-26T08:46:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.913+0000 7f8d8d036640 1 -- 172.21.15.7:0/2906514384 learned_addr learned my addr 172.21.15.7:0/2906514384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.913+0000 7f8d8d837640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.916+0000 7f8d8d036640 1 -- 172.21.15.7:0/2906514384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 msgr2=0x5636314f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.916+0000 7f8d8d036640 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.916+0000 7f8d8d036640 1 -- 172.21.15.7:0/2906514384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e4350 msgr2=0x5636314ee290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.916+0000 7f8d8d036640 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e4350 0x5636314ee290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.916+0000 7f8d8d036640 1 -- 172.21.15.7:0/2906514384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636312257e0 con 0x5636314e3960 2024-03-26T08:46:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.916+0000 7f8d8d036640 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314edd50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009b30 tx=0x7f8d80033d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc1658024d818658 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.916+0000 7f8d70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.917+0000 7f8d70ff9640 1 -- 172.21.15.7:0/2906514384 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80042020 con 0x5636314e3960 2024-03-26T08:46:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.917+0000 7f8d70ff9640 1 -- 172.21.15.7:0/2906514384 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8d80003b80 con 0x5636314e3960 2024-03-26T08:46:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.917+0000 7f8d91f63740 1 -- 172.21.15.7:0/2906514384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d7809f900 con 0x5636314e3960 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.917+0000 7f8d70ff9640 1 -- 172.21.15.7:0/2906514384 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d80003d80 con 0x5636314e3960 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.917+0000 7f8d91f63740 10 client.26252 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.917+0000 7f8d91f63740 1 -- 172.21.15.7:0/2906514384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563631285f70 con 0x5636314e3960 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.918+0000 7f8d70ff9640 1 -- 172.21.15.7:0/2906514384 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d7809f900 con 0x5636314e3960 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.918+0000 7f8d70ff9640 1 -- 172.21.15.7:0/2906514384 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8d800041e0 con 0x5636314e3960 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.918+0000 7f8d91f63740 10 client.26252 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.918+0000 7f8d91f63740 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.918+0000 7f8d91f63740 20 client.26252 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.918+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5636314f9ad0 0x5636314fbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.918+0000 7f8d91f63740 4 client.26252 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.918+0000 7f8d91f63740 1 -- 172.21.15.7:0/2906514384 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636314565b0 con 0x5636314f9ad0 2024-03-26T08:46:06.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.919+0000 7f8d8c835640 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5636314f9ad0 0x5636314fbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.920+0000 7f8d70ff9640 10 client.26252 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.920+0000 7f8d8c835640 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5636314f9ad0 0x5636314fbf40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cfd23428a3a391 server_cookie=a1da0e8c5e7c8bad in_seq=0 out_seq=0 2024-03-26T08:46:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.921+0000 7f8d70ff9640 1 -- 172.21.15.7:0/2906514384 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5636314565b0 con 0x5636314f9ad0 2024-03-26T08:46:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.921+0000 7f8d70ff9640 10 client.26252 handle_command_reply: tid=0 2024-03-26T08:46:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.921+0000 7f8d91f63740 1 client.26252 shutdown 2024-03-26T08:46:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.923+0000 7f8d91f63740 1 -- 172.21.15.7:0/2906514384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 msgr2=0x5636314edd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.923+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314edd50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009b30 tx=0x7f8d80033d70 comp rx=0 tx=0).stop 2024-03-26T08:46:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.923+0000 7f8d91f63740 1 -- 172.21.15.7:0/2906514384 shutdown_connections 2024-03-26T08:46:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.923+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5636314f9ad0 0x5636314fbf40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.923+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.923+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e4350 0x5636314ee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.923+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2906514384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314edd50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.923+0000 7f8d91f63740 1 -- 172.21.15.7:0/2906514384 >> 172.21.15.7:0/2906514384 conn(0x5636314ac1a0 msgr2=0x5636314de530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.923+0000 7f8d70ff9640 0 client.26252 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.924+0000 7f8d91f63740 1 -- 172.21.15.7:0/2906514384 shutdown_connections 2024-03-26T08:46:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.924+0000 7f8d91f63740 1 -- 172.21.15.7:0/2906514384 wait complete. 2024-03-26T08:46:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.924+0000 7f8d91f63740 20 client.26252 trim_cache size 0 max 16384 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.952+0000 7f8d91f63740 1 Processor -- start 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d91f63740 1 -- start start 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314e2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 0x5636314f5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e94c0 0x5636314e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d91f63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d881b9f60 con 0x5636314e94c0 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d91f63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d881b7a60 con 0x5636314e3960 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8d837640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e94c0 0x5636314e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8d837640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e94c0 0x5636314e3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44934/0 (socket says 172.21.15.7:44934) 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8d837640 1 -- 172.21.15.7:0/2834683921 learned_addr learned my addr 172.21.15.7:0/2834683921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d91f63740 1 -- 172.21.15.7:0/2834683921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88072910 con 0x5636314e4350 2024-03-26T08:46:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8d036640 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314e2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8c835640 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 0x5636314f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8c835640 1 -- 172.21.15.7:0/2834683921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 msgr2=0x5636314e2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8c835640 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314e2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8c835640 1 -- 172.21.15.7:0/2834683921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e94c0 msgr2=0x5636314e3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8c835640 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e94c0 0x5636314e3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.953+0000 7f8d8c835640 1 -- 172.21.15.7:0/2834683921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56363111d170 con 0x5636314e4350 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.954+0000 7f8d8d837640 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e94c0 0x5636314e3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.954+0000 7f8d8c835640 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 0x5636314f5290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d78097c10 tx=0x7f8d78000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8ddb91ae5ea320 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.954+0000 7f8d72ffd640 1 -- 172.21.15.7:0/2834683921 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d780b4070 con 0x5636314e4350 2024-03-26T08:46:06.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.954+0000 7f8d72ffd640 1 -- 172.21.15.7:0/2834683921 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d7809f900 con 0x5636314e4350 2024-03-26T08:46:06.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.954+0000 7f8d72ffd640 1 -- 172.21.15.7:0/2834683921 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d780afb90 con 0x5636314e4350 2024-03-26T08:46:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 -- 172.21.15.7:0/2834683921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 msgr2=0x5636314f5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:06.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 0x5636314f5290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d78097c10 tx=0x7f8d78000bc0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 -- 172.21.15.7:0/2834683921 shutdown_connections 2024-03-26T08:46:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e94c0 0x5636314e3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e4350 0x5636314f5290 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 --2- 172.21.15.7:0/2834683921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314e2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 -- 172.21.15.7:0/2834683921 >> 172.21.15.7:0/2834683921 conn(0x5636314ac1a0 msgr2=0x5636314e2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:06.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 -- 172.21.15.7:0/2834683921 shutdown_connections 2024-03-26T08:46:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 -- 172.21.15.7:0/2834683921 wait complete. 2024-03-26T08:46:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 -- start start 2024-03-26T08:46:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.955+0000 7f8d91f63740 1 Processor -- start 2024-03-26T08:46:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e4350 0x5636314f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d91f63740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d91f63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88072ee0 con 0x5636314e4350 2024-03-26T08:46:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d91f63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d881b7400 con 0x5636314e3960 2024-03-26T08:46:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d91f63740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88073540 con 0x5636314e94c0 2024-03-26T08:46:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d8d837640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d8d837640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53656/0 (socket says 172.21.15.7:53656) 2024-03-26T08:46:06.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d8d837640 1 -- 172.21.15.7:0/1964202758 learned_addr learned my addr 172.21.15.7:0/1964202758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.956+0000 7f8d91f63740 10 client.0 _ll_register_callbacks cb 0x7f8d84005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.957+0000 7f8d8d837640 1 -- 172.21.15.7:0/1964202758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 msgr2=0x5636314f31f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:46:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.957+0000 7f8d8d837640 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314f31f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.957+0000 7f8d8d837640 1 -- 172.21.15.7:0/1964202758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e4350 msgr2=0x5636314f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.957+0000 7f8d8d837640 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e4350 0x5636314f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.957+0000 7f8d8d837640 1 -- 172.21.15.7:0/1964202758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636314f4ce0 con 0x5636314e94c0 2024-03-26T08:46:06.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.957+0000 7f8d8d837640 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f46f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8d840128a0 tx=0x7f8d84004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f23a57a9e040ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.957+0000 7f8d70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.957+0000 7f8d70ff9640 1 -- 172.21.15.7:0/1964202758 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84011b50 con 0x5636314e94c0 2024-03-26T08:46:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.957+0000 7f8d70ff9640 1 -- 172.21.15.7:0/1964202758 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8d84011cf0 con 0x5636314e94c0 2024-03-26T08:46:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d91f63740 1 -- 172.21.15.7:0/1964202758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d7809f900 con 0x5636314e94c0 2024-03-26T08:46:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d70ff9640 1 -- 172.21.15.7:0/1964202758 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d84005020 con 0x5636314e94c0 2024-03-26T08:46:06.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d91f63740 10 client.26272 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d91f63740 1 -- 172.21.15.7:0/1964202758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563631555c70 con 0x5636314e94c0 2024-03-26T08:46:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d70ff9640 1 -- 172.21.15.7:0/1964202758 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d7809f900 con 0x5636314e94c0 2024-03-26T08:46:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d70ff9640 1 -- 172.21.15.7:0/1964202758 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8d8401f440 con 0x5636314e94c0 2024-03-26T08:46:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d91f63740 10 client.26272 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d91f63740 10 client.26272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d91f63740 20 client.26272 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d91f63740 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563631556600 0x563631558a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d91f63740 4 client.26272 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:46:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.958+0000 7f8d91f63740 1 -- 172.21.15.7:0/1964202758 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563631450900 con 0x563631556600 2024-03-26T08:46:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.959+0000 7f8d8d036640 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563631556600 0x563631558a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.960+0000 7f8d70ff9640 10 client.26272 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.960+0000 7f8d8d036640 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563631556600 0x563631558a40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45905cbbfb30504c server_cookie=16dae34c9d9632b5 in_seq=0 out_seq=0 2024-03-26T08:46:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.961+0000 7f8d70ff9640 1 -- 172.21.15.7:0/1964202758 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563631450900 con 0x563631556600 2024-03-26T08:46:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.961+0000 7f8d70ff9640 10 client.26272 handle_command_reply: tid=0 2024-03-26T08:46:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.961+0000 7f8d91f63740 1 client.26272 shutdown 2024-03-26T08:46:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.961+0000 7f8d91f63740 1 -- 172.21.15.7:0/1964202758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 msgr2=0x5636314f46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.961+0000 7f8d91f63740 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f46f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8d840128a0 tx=0x7f8d84004730 comp rx=0 tx=0).stop 2024-03-26T08:46:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.962+0000 7f8d91f63740 1 -- 172.21.15.7:0/1964202758 shutdown_connections 2024-03-26T08:46:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.962+0000 7f8d91f63740 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563631556600 0x563631558a40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.962+0000 7f8d91f63740 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5636314e94c0 0x5636314f46f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.962+0000 7f8d91f63740 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636314e4350 0x5636314f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.962+0000 7f8d91f63740 1 --2- 172.21.15.7:0/1964202758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5636314e3960 0x5636314f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.962+0000 7f8d91f63740 1 -- 172.21.15.7:0/1964202758 >> 172.21.15.7:0/1964202758 conn(0x5636314ac1a0 msgr2=0x5636314e2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.962+0000 7f8d70ff9640 0 client.26272 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.963+0000 7f8d91f63740 1 -- 172.21.15.7:0/1964202758 shutdown_connections 2024-03-26T08:46:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.963+0000 7f8d91f63740 1 -- 172.21.15.7:0/1964202758 wait complete. 2024-03-26T08:46:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.963+0000 7f8d91f63740 20 client.26272 trim_cache size 0 max 16384 2024-03-26T08:46:06.965 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:46:06.965 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:46:06.965 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "6a90323c-2a33-4da1-a149-79d32455ba28", 2024-03-26T08:46:06.965 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:46:06.966 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:46:06.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 -- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8d740a2a10 msgr2=0x7f8d740a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:06.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8d740a2a10 0x7f8d740a4ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x5636314e4bb0 tx=0x7f8d840046c0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 -- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d881017b0 msgr2=0x7f8d8819e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d881017b0 0x7f8d8819e560 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800e9f0 tx=0x7f8d7800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 -- 172.21.15.7:0/1040620302 shutdown_connections 2024-03-26T08:46:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8d740a2a10 0x7f8d740a4ed0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d88102170 0x7f8d881a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d881017b0 0x7f8d8819e560 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 --2- 172.21.15.7:0/1040620302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d881005c0 0x7f8d8819e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.964+0000 7f8d8f29a640 1 -- 172.21.15.7:0/1040620302 >> 172.21.15.7:0/1040620302 conn(0x7f8d880fbb30 msgr2=0x7f8d8810dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.965+0000 7f8d8f29a640 1 -- 172.21.15.7:0/1040620302 shutdown_connections 2024-03-26T08:46:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:06.965+0000 7f8d8f29a640 1 -- 172.21.15.7:0/1040620302 wait complete. 2024-03-26T08:46:06.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:46:06.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:46:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.147+0000 7f85109df640 1 -- 172.21.15.7:0/1200321328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c1086a0 msgr2=0x7f850c10aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.147+0000 7f85109df640 1 --2- 172.21.15.7:0/1200321328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c1086a0 0x7f850c10aad0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009860 tx=0x7f84f402bb70 comp rx=0 tx=0).stop 2024-03-26T08:46:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.148+0000 7f85109df640 1 -- 172.21.15.7:0/1200321328 shutdown_connections 2024-03-26T08:46:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.148+0000 7f85109df640 1 --2- 172.21.15.7:0/1200321328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c10d9c0 0x7f850c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.148+0000 7f85109df640 1 --2- 172.21.15.7:0/1200321328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c10b010 0x7f850c10d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.148+0000 7f85109df640 1 --2- 172.21.15.7:0/1200321328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c1086a0 0x7f850c10aad0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.148+0000 7f85109df640 1 -- 172.21.15.7:0/1200321328 >> 172.21.15.7:0/1200321328 conn(0x7f850c06e640 msgr2=0x7f850c070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.148+0000 7f85109df640 1 -- 172.21.15.7:0/1200321328 shutdown_connections 2024-03-26T08:46:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.148+0000 7f85109df640 1 -- 172.21.15.7:0/1200321328 wait complete. 2024-03-26T08:46:07.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f85109df640 1 Processor -- start 2024-03-26T08:46:07.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f85109df640 1 -- start start 2024-03-26T08:46:07.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f85109df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c1086a0 0x7f850c1a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f85109df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c10b010 0x7f850c1a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f85109df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c10d9c0 0x7f850c1add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f85109df640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c11c120 con 0x7f850c10b010 2024-03-26T08:46:07.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f85109df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c11bfa0 con 0x7f850c1086a0 2024-03-26T08:46:07.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f85109df640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c11c2a0 con 0x7f850c10d9c0 2024-03-26T08:46:07.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f850a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c1086a0 0x7f850c1a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f850ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c10d9c0 0x7f850c1add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f850ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c10d9c0 0x7f850c1add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53672/0 (socket says 172.21.15.7:53672) 2024-03-26T08:46:07.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.149+0000 7f850ad9d640 1 -- 172.21.15.7:0/3431236210 learned_addr learned my addr 172.21.15.7:0/3431236210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:07.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.150+0000 7f850a59c640 1 -- 172.21.15.7:0/3431236210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c10d9c0 msgr2=0x7f850c1add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:07.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.150+0000 7f850a59c640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c10d9c0 0x7f850c1add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.151+0000 7f850a59c640 1 -- 172.21.15.7:0/3431236210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c10b010 msgr2=0x7f850c1a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.151+0000 7f850a59c640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c10b010 0x7f850c1a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.151+0000 7f850a59c640 1 -- 172.21.15.7:0/3431236210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c1ae3b0 con 0x7f850c1086a0 2024-03-26T08:46:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.151+0000 7f850a59c640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c1086a0 0x7f850c1a67e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84f402c0b0 tx=0x7f84f40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.151+0000 7f85037fe640 1 -- 172.21.15.7:0/3431236210 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4041020 con 0x7f850c1086a0 2024-03-26T08:46:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.151+0000 7f85037fe640 1 -- 172.21.15.7:0/3431236210 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84f40323b0 con 0x7f850c1086a0 2024-03-26T08:46:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.151+0000 7f85037fe640 1 -- 172.21.15.7:0/3431236210 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4039040 con 0x7f850c1086a0 2024-03-26T08:46:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.152+0000 7f85109df640 1 -- 172.21.15.7:0/3431236210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c1ae640 con 0x7f850c1086a0 2024-03-26T08:46:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.152+0000 7f85109df640 1 -- 172.21.15.7:0/3431236210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c1aeab0 con 0x7f850c1086a0 2024-03-26T08:46:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.155+0000 7f85135c6740 1 Processor -- start 2024-03-26T08:46:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.155+0000 7f85135c6740 1 -- start start 2024-03-26T08:46:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.155+0000 7f85037fe640 1 -- 172.21.15.7:0/3431236210 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f84f4044020 con 0x7f850c1086a0 2024-03-26T08:46:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.155+0000 7f85135c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450b9da0 0x559e450ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f85135c6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f85135c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450bf900 0x559e450c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f85135c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4501f3d0 con 0x559e450b9da0 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f85135c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e44fc7ad0 con 0x559e450bf900 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f85135c6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4501b470 con 0x559e450ba790 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450b9da0 0x559e450ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450bf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53684/0 (socket says 172.21.15.7:53684) 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850a59c640 1 -- 172.21.15.7:0/178668564 learned_addr learned my addr 172.21.15.7:0/178668564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450b9da0 0x559e450ba1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44964/0 (socket says 172.21.15.7:44964) 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f8509d9b640 1 --2- 172.21.15.7:0/178668564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450bf900 0x559e450c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f85037fe640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84f80a2880 0x7f84f80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850a59c640 1 -- 172.21.15.7:0/178668564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450bf900 msgr2=0x559e450c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850a59c640 1 --2- 172.21.15.7:0/178668564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450bf900 0x559e450c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850a59c640 1 -- 172.21.15.7:0/178668564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450b9da0 msgr2=0x559e450ba1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850a59c640 1 --2- 172.21.15.7:0/178668564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450b9da0 0x559e450ba1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850a59c640 1 -- 172.21.15.7:0/178668564 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e44dfd7d0 con 0x559e450ba790 2024-03-26T08:46:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f8509d9b640 1 --2- 172.21.15.7:0/178668564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450bf900 0x559e450c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.156+0000 7f850ad9d640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84f80a2880 0x7f84f80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.157+0000 7f85037fe640 1 -- 172.21.15.7:0/3431236210 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84f40f02f0 con 0x7f850c1086a0 2024-03-26T08:46:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.157+0000 7f850ad9d640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84f80a2880 0x7f84f80a4d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x559e450baff0 tx=0x7f85040046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.159+0000 7f850a59c640 1 --2- 172.21.15.7:0/178668564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450bf290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f84f40de8d0 tx=0x7f84f40b89e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=890e482e6d130112 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.159+0000 7f8500ff9640 1 -- 172.21.15.7:0/178668564 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f40bca60 con 0x559e450ba790 2024-03-26T08:46:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.159+0000 7f8500ff9640 1 -- 172.21.15.7:0/178668564 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84f40de4a0 con 0x559e450ba790 2024-03-26T08:46:07.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.159+0000 7f8500ff9640 1 -- 172.21.15.7:0/178668564 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f40347d0 con 0x559e450ba790 2024-03-26T08:46:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.160+0000 7f85135c6740 1 -- 172.21.15.7:0/178668564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 msgr2=0x559e450bf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.160+0000 7f85135c6740 1 --2- 172.21.15.7:0/178668564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450bf290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f84f40de8d0 tx=0x7f84f40b89e0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.160+0000 7f85135c6740 1 -- 172.21.15.7:0/178668564 shutdown_connections 2024-03-26T08:46:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.160+0000 7f85135c6740 1 --2- 172.21.15.7:0/178668564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450bf900 0x559e450c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.160+0000 7f85135c6740 1 --2- 172.21.15.7:0/178668564 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450bf290 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.161+0000 7f85135c6740 1 --2- 172.21.15.7:0/178668564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450b9da0 0x559e450ba1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.161+0000 7f85135c6740 1 -- 172.21.15.7:0/178668564 >> 172.21.15.7:0/178668564 conn(0x559e450b5310 msgr2=0x559e450b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.161+0000 7f85135c6740 1 -- 172.21.15.7:0/178668564 shutdown_connections 2024-03-26T08:46:07.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.161+0000 7f85135c6740 1 -- 172.21.15.7:0/178668564 wait complete. 2024-03-26T08:46:07.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.161+0000 7f85135c6740 1 -- start start 2024-03-26T08:46:07.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.161+0000 7f85135c6740 1 Processor -- start 2024-03-26T08:46:07.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f85135c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450b9da0 0x559e450c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f85135c6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f85135c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f85135c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e450c2670 con 0x559e450bf900 2024-03-26T08:46:07.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f850a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450b9da0 0x559e450c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f850a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450b9da0 0x559e450c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35840/0 (socket says 172.21.15.7:35840) 2024-03-26T08:46:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f850a59c640 1 -- 172.21.15.7:0/143658111 learned_addr learned my addr 172.21.15.7:0/143658111 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e44fdb060 con 0x559e450b9da0 2024-03-26T08:46:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e450c27f0 con 0x559e450ba790 2024-03-26T08:46:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f8509d9b640 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f850ad9d640 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.162+0000 7f85135c6740 10 client.0 _ll_register_callbacks cb 0x559e44fbd6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:07.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f850a59c640 1 -- 172.21.15.7:0/143658111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 msgr2=0x559e450c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:07.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f850a59c640 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f850a59c640 1 -- 172.21.15.7:0/143658111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 msgr2=0x559e450cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:07.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f850a59c640 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f850a59c640 1 -- 172.21.15.7:0/143658111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e44d333f0 con 0x559e450b9da0 2024-03-26T08:46:07.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f850a59c640 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450b9da0 0x559e450c3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84f40c1ec0 tx=0x7f84f40f0570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d171c2c7c4c312 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:07.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f84f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:07.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f84f1ffb640 1 -- 172.21.15.7:0/143658111 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f40d4020 con 0x559e450b9da0 2024-03-26T08:46:07.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f84f1ffb640 1 -- 172.21.15.7:0/143658111 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84f40baf90 con 0x559e450b9da0 2024-03-26T08:46:07.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.163+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e4502c910 con 0x559e450b9da0 2024-03-26T08:46:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f84f1ffb640 1 -- 172.21.15.7:0/143658111 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84f40c1d00 con 0x559e450b9da0 2024-03-26T08:46:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f85135c6740 10 client.26267 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559e44cf30f0 con 0x559e450b9da0 2024-03-26T08:46:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f84f1ffb640 1 -- 172.21.15.7:0/143658111 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84f40bd390 con 0x559e450b9da0 2024-03-26T08:46:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f84f1ffb640 1 -- 172.21.15.7:0/143658111 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f84f403c030 con 0x559e450b9da0 2024-03-26T08:46:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f85135c6740 10 client.26267 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f85135c6740 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f85135c6740 20 client.26267 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f85135c6740 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559e450cf730 0x559e450d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f85135c6740 4 client.26267 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.164+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e4502c910 con 0x559e450cf730 2024-03-26T08:46:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.165+0000 7f8509d9b640 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559e450cf730 0x559e450d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.166+0000 7f8509d9b640 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559e450cf730 0x559e450d1bb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5ead593213557c server_cookie=dd925e85f2b15880 in_seq=0 out_seq=0 2024-03-26T08:46:07.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.166+0000 7f84f1ffb640 10 client.26267 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:07.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.167+0000 7f84f1ffb640 1 -- 172.21.15.7:0/143658111 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559e4502c910 con 0x559e450cf730 2024-03-26T08:46:07.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.167+0000 7f84f1ffb640 10 client.26267 handle_command_reply: tid=0 2024-03-26T08:46:07.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.167+0000 7f85135c6740 1 client.26267 shutdown 2024-03-26T08:46:07.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450b9da0 msgr2=0x559e450c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:07.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450b9da0 0x559e450c3ef0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84f40c1ec0 tx=0x7f84f40f0570 comp rx=0 tx=0).stop 2024-03-26T08:46:07.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 shutdown_connections 2024-03-26T08:46:07.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559e450cf730 0x559e450d1bb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f84f1ffb640 0 client.26267 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:07.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450ba790 0x559e450c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 --2- 172.21.15.7:0/143658111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450b9da0 0x559e450c3ef0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 >> 172.21.15.7:0/143658111 conn(0x559e450b5310 msgr2=0x559e450b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:07.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 shutdown_connections 2024-03-26T08:46:07.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 1 -- 172.21.15.7:0/143658111 wait complete. 2024-03-26T08:46:07.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.168+0000 7f85135c6740 20 client.26267 trim_cache size 0 max 16384 2024-03-26T08:46:07.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.194+0000 7f85135c6740 1 Processor -- start 2024-03-26T08:46:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f85135c6740 1 -- start start 2024-03-26T08:46:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f85135c6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 0x559e450c3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f85135c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f85135c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f85135c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec010860 con 0x559e450bf900 2024-03-26T08:46:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f85135c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec00e360 con 0x559e450ba790 2024-03-26T08:46:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f85135c6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec00c3e0 con 0x559e450b9da0 2024-03-26T08:46:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f850ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f850ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450b8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44982/0 (socket says 172.21.15.7:44982) 2024-03-26T08:46:07.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f850ad9d640 1 -- 172.21.15.7:0/2154670574 learned_addr learned my addr 172.21.15.7:0/2154670574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f850a59c640 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 0x559e450c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f8509d9b640 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f850ad9d640 1 -- 172.21.15.7:0/2154670574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 msgr2=0x559e450c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f850ad9d640 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 0x559e450c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f850ad9d640 1 -- 172.21.15.7:0/2154670574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 msgr2=0x559e450cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f850ad9d640 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.195+0000 7f850ad9d640 1 -- 172.21.15.7:0/2154670574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e450b0da0 con 0x559e450bf900 2024-03-26T08:46:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.196+0000 7f8509d9b640 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.196+0000 7f850ad9d640 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450b8c80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f85040143f0 tx=0x7f8504017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55df09fbf4963586 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.196+0000 7f84f3fff640 1 -- 172.21.15.7:0/2154670574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504005020 con 0x559e450bf900 2024-03-26T08:46:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.196+0000 7f84f3fff640 1 -- 172.21.15.7:0/2154670574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85040047a0 con 0x559e450bf900 2024-03-26T08:46:07.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.196+0000 7f84f3fff640 1 -- 172.21.15.7:0/2154670574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85040116a0 con 0x559e450bf900 2024-03-26T08:46:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 -- 172.21.15.7:0/2154670574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 msgr2=0x559e450b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450b8c80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f85040143f0 tx=0x7f8504017000 comp rx=0 tx=0).stop 2024-03-26T08:46:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 -- 172.21.15.7:0/2154670574 shutdown_connections 2024-03-26T08:46:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450b8c80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 --2- 172.21.15.7:0/2154670574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 0x559e450c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 -- 172.21.15.7:0/2154670574 >> 172.21.15.7:0/2154670574 conn(0x559e450b5310 msgr2=0x559e450b8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 -- 172.21.15.7:0/2154670574 shutdown_connections 2024-03-26T08:46:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 -- 172.21.15.7:0/2154670574 wait complete. 2024-03-26T08:46:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.197+0000 7f85135c6740 1 -- start start 2024-03-26T08:46:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f85135c6740 1 Processor -- start 2024-03-26T08:46:07.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f85135c6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 0x559e450cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f85135c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f85135c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f85135c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec00c9e0 con 0x559e450bf900 2024-03-26T08:46:07.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f85135c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec00dd00 con 0x559e450ba790 2024-03-26T08:46:07.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f85135c6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec00d040 con 0x559e450b9da0 2024-03-26T08:46:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f8509d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f850a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 0x559e450cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f850ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f8509d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35870/0 (socket says 172.21.15.7:35870) 2024-03-26T08:46:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f850ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44996/0 (socket says 172.21.15.7:44996) 2024-03-26T08:46:07.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f8509d9b640 1 -- 172.21.15.7:0/2200335848 learned_addr learned my addr 172.21.15.7:0/2200335848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:07.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f850a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 0x559e450cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53714/0 (socket says 172.21.15.7:53714) 2024-03-26T08:46:07.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f85135c6740 10 client.0 _ll_register_callbacks cb 0x7f8504009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:07.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f8509d9b640 1 -- 172.21.15.7:0/2200335848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 msgr2=0x559e450cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:07.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f8509d9b640 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 0x559e450cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f8509d9b640 1 -- 172.21.15.7:0/2200335848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 msgr2=0x559e450c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f8509d9b640 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.198+0000 7f8509d9b640 1 -- 172.21.15.7:0/2200335848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e450ca160 con 0x559e450ba790 2024-03-26T08:46:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.199+0000 7f8509d9b640 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450c96e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009bc0 tx=0x7f84fc01cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dd10067919fb64d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.199+0000 7f84f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.199+0000 7f84f1ffb640 1 -- 172.21.15.7:0/2200335848 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc01ce40 con 0x559e450ba790 2024-03-26T08:46:07.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.199+0000 7f84f1ffb640 1 -- 172.21.15.7:0/2200335848 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84fc014a70 con 0x559e450ba790 2024-03-26T08:46:07.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.199+0000 7f85135c6740 1 -- 172.21.15.7:0/2200335848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85040047a0 con 0x559e450ba790 2024-03-26T08:46:07.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.199+0000 7f84f1ffb640 1 -- 172.21.15.7:0/2200335848 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84fc014c70 con 0x559e450ba790 2024-03-26T08:46:07.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.199+0000 7f85135c6740 10 client.26273 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:07.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.200+0000 7f85135c6740 1 -- 172.21.15.7:0/2200335848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559e4512be80 con 0x559e450ba790 2024-03-26T08:46:07.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.200+0000 7f84f1ffb640 1 -- 172.21.15.7:0/2200335848 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85040047a0 con 0x559e450ba790 2024-03-26T08:46:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.200+0000 7f84f1ffb640 1 -- 172.21.15.7:0/2200335848 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f84fc023070 con 0x559e450ba790 2024-03-26T08:46:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.200+0000 7f85135c6740 10 client.26273 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.200+0000 7f85135c6740 10 client.26273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.201+0000 7f85135c6740 20 client.26273 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.201+0000 7f85135c6740 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559e4512c7e0 0x559e4512ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.201+0000 7f85135c6740 4 client.26273 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:46:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.201+0000 7f85135c6740 1 -- 172.21.15.7:0/2200335848 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559e45026c60 con 0x559e4512c7e0 2024-03-26T08:46:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.201+0000 7f850a59c640 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559e4512c7e0 0x559e4512ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.201+0000 7f850a59c640 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559e4512c7e0 0x559e4512ec00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21a7c172927c286b server_cookie=45604131cf307a91 in_seq=0 out_seq=0 2024-03-26T08:46:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.201+0000 7f84f1ffb640 10 client.26273 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:07.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f84f1ffb640 1 -- 172.21.15.7:0/2200335848 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559e45026c60 con 0x559e4512c7e0 2024-03-26T08:46:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f84f1ffb640 10 client.26273 handle_command_reply: tid=0 2024-03-26T08:46:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 client.26273 shutdown 2024-03-26T08:46:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 -- 172.21.15.7:0/2200335848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 msgr2=0x559e450c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450c96e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009bc0 tx=0x7f84fc01cb30 comp rx=0 tx=0).stop 2024-03-26T08:46:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 -- 172.21.15.7:0/2200335848 shutdown_connections 2024-03-26T08:46:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559e4512c7e0 0x559e4512ec00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559e450bf900 0x559e450c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f84f1ffb640 0 client.26273 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e450ba790 0x559e450c96e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 --2- 172.21.15.7:0/2200335848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559e450b9da0 0x559e450cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 -- 172.21.15.7:0/2200335848 >> 172.21.15.7:0/2200335848 conn(0x559e450b5310 msgr2=0x559e450b8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 -- 172.21.15.7:0/2200335848 shutdown_connections 2024-03-26T08:46:07.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 1 -- 172.21.15.7:0/2200335848 wait complete. 2024-03-26T08:46:07.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.202+0000 7f85135c6740 20 client.26273 trim_cache size 0 max 16384 2024-03-26T08:46:07.213 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:46:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:46:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:46:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "6a90323c-2a33-4da1-a149-79d32455ba28": { 2024-03-26T08:46:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:46:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "6a90323c-2a33-4da1-a149-79d32455ba28", 2024-03-26T08:46:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:46:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:46:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:46:07.215 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:46:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 -- 172.21.15.7:0/3431236210 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84f80a2880 msgr2=0x7f84f80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84f80a2880 0x7f84f80a4d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x559e450baff0 tx=0x7f85040046c0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 -- 172.21.15.7:0/3431236210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c1086a0 msgr2=0x7f850c1a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c1086a0 0x7f850c1a67e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84f402c0b0 tx=0x7f84f40042f0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 -- 172.21.15.7:0/3431236210 shutdown_connections 2024-03-26T08:46:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f84f80a2880 0x7f84f80a4d40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f850c10d9c0 0x7f850c1add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c10b010 0x7f850c1a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 --2- 172.21.15.7:0/3431236210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f850c1086a0 0x7f850c1a67e0 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84f402c0b0 tx=0x7f84f40042f0 comp rx=0 tx=0).stop 2024-03-26T08:46:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.203+0000 7f85109df640 1 -- 172.21.15.7:0/3431236210 >> 172.21.15.7:0/3431236210 conn(0x7f850c06e640 msgr2=0x7f850c10c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.204+0000 7f85109df640 1 -- 172.21.15.7:0/3431236210 shutdown_connections 2024-03-26T08:46:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:07.204+0000 7f85109df640 1 -- 172.21.15.7:0/3431236210 wait complete. 2024-03-26T08:46:07.220 INFO:tasks.cephfs.filesystem:scrub status for tag:6a90323c-2a33-4da1-a149-79d32455ba28 - {'path': '/', 'tag': '6a90323c-2a33-4da1-a149-79d32455ba28', 'options': 'recursive,force'} 2024-03-26T08:46:07.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:07 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:46:07.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:07 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:07.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:07 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:46:07.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:07 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:46:07.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:07 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:07.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:07 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:46:07.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:07 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:46:07.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:07 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:07.794 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:07 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:46:08.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:08 smithi044 ceph-mon[36461]: pgmap v471: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 788 op/s 2024-03-26T08:46:08.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:08 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:46:08.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:08 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:08.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:08 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:08.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:08 smithi016 ceph-mon[35851]: pgmap v471: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 788 op/s 2024-03-26T08:46:08.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:08 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:46:08.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:08 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:08.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:08 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:08.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:08 smithi007 ceph-mon[32462]: pgmap v471: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 788 op/s 2024-03-26T08:46:08.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:08 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:46:08.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:08 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:08.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:08 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:10.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:10 smithi044 ceph-mon[36461]: pgmap v472: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.7 MiB/s wr, 783 op/s 2024-03-26T08:46:10.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:10 smithi016 ceph-mon[35851]: pgmap v472: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.7 MiB/s wr, 783 op/s 2024-03-26T08:46:10.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:10 smithi007 ceph-mon[32462]: pgmap v472: 97 pgs: 97 active+clean; 1.5 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.7 MiB/s wr, 783 op/s 2024-03-26T08:46:11.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 220.0 s, 3882.8 tps, lat 25.878 ms stddev 28.009 2024-03-26T08:46:12.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:12 smithi044 ceph-mon[36461]: pgmap v473: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 860 op/s 2024-03-26T08:46:12.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:12 smithi016 ceph-mon[35851]: pgmap v473: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 860 op/s 2024-03-26T08:46:12.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:12 smithi007 ceph-mon[32462]: pgmap v473: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 10 MiB/s wr, 860 op/s 2024-03-26T08:46:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.674+0000 7f1db29f3640 1 -- 172.21.15.7:0/378101982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dac1086a0 msgr2=0x7f1dac10aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:13.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.675+0000 7f1db29f3640 1 --2- 172.21.15.7:0/378101982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dac1086a0 0x7f1dac10aad0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009ca0 tx=0x7f1da402c140 comp rx=0 tx=0).stop 2024-03-26T08:46:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.676+0000 7f1db29f3640 1 -- 172.21.15.7:0/378101982 shutdown_connections 2024-03-26T08:46:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.676+0000 7f1db29f3640 1 --2- 172.21.15.7:0/378101982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dac10d9c0 0x7f1dac10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.676+0000 7f1db29f3640 1 --2- 172.21.15.7:0/378101982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac10b010 0x7f1dac10d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.676+0000 7f1db29f3640 1 --2- 172.21.15.7:0/378101982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dac1086a0 0x7f1dac10aad0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.676+0000 7f1db29f3640 1 -- 172.21.15.7:0/378101982 >> 172.21.15.7:0/378101982 conn(0x7f1dac06e6b0 msgr2=0x7f1dac070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.680+0000 7f1db29f3640 1 -- 172.21.15.7:0/378101982 shutdown_connections 2024-03-26T08:46:13.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.683+0000 7f1db29f3640 1 -- 172.21.15.7:0/378101982 wait complete. 2024-03-26T08:46:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.684+0000 7f1db29f3640 1 Processor -- start 2024-03-26T08:46:13.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.684+0000 7f1db29f3640 1 -- start start 2024-03-26T08:46:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.685+0000 7f1db29f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac1086a0 0x7f1dac1a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.685+0000 7f1db29f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dac10b010 0x7f1dac1a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.685+0000 7f1db29f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dac10d9c0 0x7f1dac1adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.685+0000 7f1db29f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac11c2a0 con 0x7f1dac10d9c0 2024-03-26T08:46:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.685+0000 7f1db0f90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dac10d9c0 0x7f1dac1adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.685+0000 7f1db0f90640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dac10d9c0 0x7f1dac1adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45014/0 (socket says 172.21.15.7:45014) 2024-03-26T08:46:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.685+0000 7f1db0f90640 1 -- 172.21.15.7:0/902973946 learned_addr learned my addr 172.21.15.7:0/902973946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.685+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac11c120 con 0x7f1dac1086a0 2024-03-26T08:46:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.685+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac11c420 con 0x7f1dac10b010 2024-03-26T08:46:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.686+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dac10b010 0x7f1dac1a6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.686+0000 7f1dab7fe640 1 -- 172.21.15.7:0/902973946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac1086a0 msgr2=0x7f1dac1a69b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:46:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.686+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac1086a0 0x7f1dac1a69b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.686+0000 7f1dab7fe640 1 -- 172.21.15.7:0/902973946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dac10d9c0 msgr2=0x7f1dac1adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.686+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dac10d9c0 0x7f1dac1adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.686+0000 7f1dab7fe640 1 -- 172.21.15.7:0/902973946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac1ae5d0 con 0x7f1dac10b010 2024-03-26T08:46:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.687+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dac10b010 0x7f1dac1a6ef0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009fd0 tx=0x7f1da000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.687+0000 7f1da97fa640 1 -- 172.21.15.7:0/902973946 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0015020 con 0x7f1dac10b010 2024-03-26T08:46:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.687+0000 7f1da97fa640 1 -- 172.21.15.7:0/902973946 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1da0011d10 con 0x7f1dac10b010 2024-03-26T08:46:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.687+0000 7f1da97fa640 1 -- 172.21.15.7:0/902973946 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da000f2e0 con 0x7f1dac10b010 2024-03-26T08:46:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.687+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac1ae8c0 con 0x7f1dac10b010 2024-03-26T08:46:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.688+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac1aede0 con 0x7f1dac10b010 2024-03-26T08:46:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.692+0000 7f1da97fa640 1 -- 172.21.15.7:0/902973946 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1da0018020 con 0x7f1dac10b010 2024-03-26T08:46:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db55d7740 1 Processor -- start 2024-03-26T08:46:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db55d7740 1 -- start start 2024-03-26T08:46:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 0x556ae19cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db55d7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae192ab10 con 0x556ae19c6310 2024-03-26T08:46:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db55d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae18d3900 con 0x556ae19c5920 2024-03-26T08:46:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db55d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae1926bb0 con 0x556ae19cb480 2024-03-26T08:46:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db0f90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db0f90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19c5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35896/0 (socket says 172.21.15.7:35896) 2024-03-26T08:46:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.694+0000 7f1db0f90640 1 -- 172.21.15.7:0/1709096773 learned_addr learned my addr 172.21.15.7:0/1709096773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/1709096773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1da97fa640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d940a2930 0x7f1d940a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1db0f90640 1 -- 172.21.15.7:0/1709096773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 msgr2=0x556ae19cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1db0f90640 1 --2- 172.21.15.7:0/1709096773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1db0f90640 1 -- 172.21.15.7:0/1709096773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 msgr2=0x556ae19cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1db0f90640 1 --2- 172.21.15.7:0/1709096773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 0x556ae19cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1db0f90640 1 -- 172.21.15.7:0/1709096773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae163f210 con 0x556ae19c5920 2024-03-26T08:46:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1db0f90640 1 --2- 172.21.15.7:0/1709096773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19c5d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009a90 tx=0x7f1d9c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c1771e75e122af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1db0f90640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d940a2930 0x7f1d940a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.695+0000 7f1da97fa640 1 -- 172.21.15.7:0/902973946 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1da00ce2f0 con 0x7f1dac10b010 2024-03-26T08:46:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.696+0000 7f1d927fc640 1 -- 172.21.15.7:0/1709096773 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c017070 con 0x556ae19c5920 2024-03-26T08:46:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.696+0000 7f1d927fc640 1 -- 172.21.15.7:0/1709096773 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c00ed40 con 0x556ae19c5920 2024-03-26T08:46:13.688 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.696+0000 7f1d927fc640 1 -- 172.21.15.7:0/1709096773 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c01ca10 con 0x556ae19c5920 2024-03-26T08:46:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.697+0000 7f1db55d7740 1 -- 172.21.15.7:0/1709096773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 msgr2=0x556ae19c5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.697+0000 7f1db55d7740 1 --2- 172.21.15.7:0/1709096773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19c5d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009a90 tx=0x7f1d9c009f50 comp rx=0 tx=0).stop 2024-03-26T08:46:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.697+0000 7f1db0f90640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d940a2930 0x7f1d940a4df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c019000 tx=0x7f1d9c01cc90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.698+0000 7f1db55d7740 1 -- 172.21.15.7:0/1709096773 shutdown_connections 2024-03-26T08:46:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.698+0000 7f1db55d7740 1 --2- 172.21.15.7:0/1709096773 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.698+0000 7f1db55d7740 1 --2- 172.21.15.7:0/1709096773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 0x556ae19cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.698+0000 7f1db55d7740 1 --2- 172.21.15.7:0/1709096773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19c5d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.698+0000 7f1db55d7740 1 -- 172.21.15.7:0/1709096773 >> 172.21.15.7:0/1709096773 conn(0x556ae19c0e90 msgr2=0x556ae19c1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.699+0000 7f1db55d7740 1 -- 172.21.15.7:0/1709096773 shutdown_connections 2024-03-26T08:46:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.699+0000 7f1db55d7740 1 -- 172.21.15.7:0/1709096773 wait complete. 2024-03-26T08:46:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.700+0000 7f1db55d7740 1 -- start start 2024-03-26T08:46:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.700+0000 7f1db55d7740 1 Processor -- start 2024-03-26T08:46:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.700+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.700+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 0x556ae19d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.700+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.700+0000 7f1db55d7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae19ce430 con 0x556ae19c6310 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.700+0000 7f1db55d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae18e6d50 con 0x556ae19c5920 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.700+0000 7f1db55d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae19ce5b0 con 0x556ae19cb480 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db55d7740 10 client.0 _ll_register_callbacks cb 0x556ae18e6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db0f90640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1dab7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 0x556ae19d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db0f90640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53756/0 (socket says 172.21.15.7:53756) 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db0f90640 1 -- 172.21.15.7:0/2725378690 learned_addr learned my addr 172.21.15.7:0/2725378690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1dabfff640 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db0f90640 1 -- 172.21.15.7:0/2725378690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 msgr2=0x556ae19cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db0f90640 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db0f90640 1 -- 172.21.15.7:0/2725378690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 msgr2=0x556ae19d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db0f90640 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 0x556ae19d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db0f90640 1 -- 172.21.15.7:0/2725378690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ae170b620 con 0x556ae19cb480 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 0x556ae19d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1db0f90640 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d7260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0136a0 tx=0x7f1d9c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d40c510f9c60e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.701+0000 7f1d8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2725378690 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c031470 con 0x556ae19cb480 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2725378690 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d9c031610 con 0x556ae19cb480 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1db55d7740 1 -- 172.21.15.7:0/2725378690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d9c00ed40 con 0x556ae19cb480 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2725378690 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d9c005970 con 0x556ae19cb480 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1db55d7740 10 client.26299 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1db55d7740 1 -- 172.21.15.7:0/2725378690 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556ae19da640 con 0x556ae19cb480 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2725378690 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c00ed40 con 0x556ae19cb480 2024-03-26T08:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2725378690 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1d9c006020 con 0x556ae19cb480 2024-03-26T08:46:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1db55d7740 10 client.26299 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1db55d7740 10 client.26299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1db55d7740 20 client.26299 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556ae19dba40 0x556ae19dde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1db55d7740 4 client.26299 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:13.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.702+0000 7f1db55d7740 1 -- 172.21.15.7:0/2725378690 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ae1938050 con 0x556ae19dba40 2024-03-26T08:46:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.704+0000 7f1dabfff640 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556ae19dba40 0x556ae19dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.704+0000 7f1d8b7fe640 10 client.26299 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.704+0000 7f1dabfff640 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556ae19dba40 0x556ae19dde30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af37c274aea29418 server_cookie=9aa21cd5a7a04205 in_seq=0 out_seq=0 2024-03-26T08:46:13.697 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2725378690 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556ae1938050 con 0x556ae19dba40 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1d8b7fe640 10 client.26299 handle_command_reply: tid=0 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 client.26299 shutdown 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 -- 172.21.15.7:0/2725378690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 msgr2=0x556ae19d7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d7260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0136a0 tx=0x7f1d9c017000 comp rx=0 tx=0).stop 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 -- 172.21.15.7:0/2725378690 shutdown_connections 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556ae19dba40 0x556ae19dde30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d7260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1d8b7fe640 0 client.26299 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c6310 0x556ae19d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2725378690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c5920 0x556ae19cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 -- 172.21.15.7:0/2725378690 >> 172.21.15.7:0/2725378690 conn(0x556ae19c0e90 msgr2=0x556ae19c0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 -- 172.21.15.7:0/2725378690 shutdown_connections 2024-03-26T08:46:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 1 -- 172.21.15.7:0/2725378690 wait complete. 2024-03-26T08:46:13.699 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.705+0000 7f1db55d7740 20 client.26299 trim_cache size 0 max 16384 2024-03-26T08:46:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.726+0000 7f1db55d7740 1 Processor -- start 2024-03-26T08:46:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.726+0000 7f1db55d7740 1 -- start start 2024-03-26T08:46:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19c6310 0x556ae19d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19cb480 0x556ae19c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1db55d7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1b9ff0 con 0x556ae19c5920 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1db55d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1b7af0 con 0x556ae19cb480 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1db55d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1b5b70 con 0x556ae19c6310 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1dabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1dabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45056/0 (socket says 172.21.15.7:45056) 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1dabfff640 1 -- 172.21.15.7:0/1250808458 learned_addr learned my addr 172.21.15.7:0/1250808458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1db0f90640 1 --2- 172.21.15.7:0/1250808458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19cb480 0x556ae19c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/1250808458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19c6310 0x556ae19d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1dabfff640 1 -- 172.21.15.7:0/1250808458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19c6310 msgr2=0x556ae19d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1dabfff640 1 --2- 172.21.15.7:0/1250808458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19c6310 0x556ae19d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1dabfff640 1 -- 172.21.15.7:0/1250808458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19cb480 msgr2=0x556ae19c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:13.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1dabfff640 1 --2- 172.21.15.7:0/1250808458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19cb480 0x556ae19c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.727+0000 7f1dabfff640 1 -- 172.21.15.7:0/1250808458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae19de1a0 con 0x556ae19c5920 2024-03-26T08:46:13.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.728+0000 7f1dabfff640 1 --2- 172.21.15.7:0/1250808458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19c76d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1da40096c0 tx=0x7f1da4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25b1fc24f9acbed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.728+0000 7f1d91ffb640 1 -- 172.21.15.7:0/1250808458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4047020 con 0x556ae19c5920 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.728+0000 7f1d91ffb640 1 -- 172.21.15.7:0/1250808458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1da4032cd0 con 0x556ae19c5920 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.728+0000 7f1d91ffb640 1 -- 172.21.15.7:0/1250808458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4039040 con 0x556ae19c5920 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 -- 172.21.15.7:0/1250808458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 msgr2=0x556ae19c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 --2- 172.21.15.7:0/1250808458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19c76d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1da40096c0 tx=0x7f1da4000f40 comp rx=0 tx=0).stop 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 -- 172.21.15.7:0/1250808458 shutdown_connections 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 --2- 172.21.15.7:0/1250808458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19cb480 0x556ae19c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 --2- 172.21.15.7:0/1250808458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19c6310 0x556ae19d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 --2- 172.21.15.7:0/1250808458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19c76d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 -- 172.21.15.7:0/1250808458 >> 172.21.15.7:0/1250808458 conn(0x556ae19c0e90 msgr2=0x556ae19c1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 -- 172.21.15.7:0/1250808458 shutdown_connections 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 -- 172.21.15.7:0/1250808458 wait complete. 2024-03-26T08:46:13.722 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.729+0000 7f1db55d7740 1 -- start start 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1db55d7740 1 Processor -- start 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c6310 0x556ae19d4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1db55d7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1db55d7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1b6170 con 0x556ae19c5920 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1db55d7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1b7490 con 0x556ae19c6310 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1db55d7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1b67d0 con 0x556ae19cb480 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1db55d7740 10 client.0 _ll_register_callbacks cb 0x7f1d9c01ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1dab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c6310 0x556ae19d4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1dab7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c6310 0x556ae19d4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35940/0 (socket says 172.21.15.7:35940) 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.730+0000 7f1dab7fe640 1 -- 172.21.15.7:0/2862355572 learned_addr learned my addr 172.21.15.7:0/2862355572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.731+0000 7f1dabfff640 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.731+0000 7f1db0f90640 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.731+0000 7f1dab7fe640 1 -- 172.21.15.7:0/2862355572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 msgr2=0x556ae19d5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.731+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1dab7fe640 1 -- 172.21.15.7:0/2862355572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 msgr2=0x556ae19d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1dab7fe640 1 -- 172.21.15.7:0/2862355572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ae19d6120 con 0x556ae19c6310 2024-03-26T08:46:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1db0f90640 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1dabfff640 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1dab7fe640 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c6310 0x556ae19d4b20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1da009abe0 tx=0x7f1da009a790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60cc1da76e2eb2ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1d8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2862355572 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da00d1e20 con 0x556ae19c6310 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2862355572 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1da0010510 con 0x556ae19c6310 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1db55d7740 1 -- 172.21.15.7:0/2862355572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1da4032cd0 con 0x556ae19c6310 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2862355572 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1da009ac50 con 0x556ae19c6310 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1db55d7740 10 client.26285 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1db55d7740 1 -- 172.21.15.7:0/2862355572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556ae19d74a0 con 0x556ae19c6310 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.732+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2862355572 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1da4032cd0 con 0x556ae19c6310 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2862355572 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1da00969a0 con 0x556ae19c6310 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1db55d7740 10 client.26285 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1db55d7740 10 client.26285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1db55d7740 20 client.26285 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:13.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556ae1a37910 0x556ae1a39d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1db55d7740 4 client.26285 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:46:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1db55d7740 1 -- 172.21.15.7:0/2862355572 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ae19323a0 con 0x556ae1a37910 2024-03-26T08:46:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1dabfff640 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556ae1a37910 0x556ae1a39d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1dabfff640 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556ae1a37910 0x556ae1a39d30 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=957b85faeb46906e server_cookie=631e9093cad2f30d in_seq=0 out_seq=0 2024-03-26T08:46:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.733+0000 7f1d8b7fe640 10 client.26285 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.749+0000 7f1d8b7fe640 1 -- 172.21.15.7:0/2862355572 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ae19323a0 con 0x556ae1a37910 2024-03-26T08:46:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.749+0000 7f1d8b7fe640 10 client.26285 handle_command_reply: tid=0 2024-03-26T08:46:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 client.26285 shutdown 2024-03-26T08:46:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 -- 172.21.15.7:0/2862355572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c6310 msgr2=0x556ae19d4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c6310 0x556ae19d4b20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1da009abe0 tx=0x7f1da009a790 comp rx=0 tx=0).stop 2024-03-26T08:46:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 -- 172.21.15.7:0/2862355572 shutdown_connections 2024-03-26T08:46:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556ae1a37910 0x556ae1a39d30 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae19cb480 0x556ae19d5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae19c6310 0x556ae19d4b20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 --2- 172.21.15.7:0/2862355572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae19c5920 0x556ae19d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 -- 172.21.15.7:0/2862355572 >> 172.21.15.7:0/2862355572 conn(0x556ae19c0e90 msgr2=0x556ae19cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1d8b7fe640 0 client.26285 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 -- 172.21.15.7:0/2862355572 shutdown_connections 2024-03-26T08:46:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 1 -- 172.21.15.7:0/2862355572 wait complete. 2024-03-26T08:46:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.750+0000 7f1db55d7740 20 client.26285 trim_cache size 0 max 16384 2024-03-26T08:46:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:46:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:46:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:46:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:46:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d940a2930 msgr2=0x7f1d940a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d940a2930 0x7f1d940a4df0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c019000 tx=0x7f1d9c01cc90 comp rx=0 tx=0).stop 2024-03-26T08:46:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dac10b010 msgr2=0x7f1dac1a6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dac10b010 0x7f1dac1a6ef0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009fd0 tx=0x7f1da000b870 comp rx=0 tx=0).stop 2024-03-26T08:46:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 shutdown_connections 2024-03-26T08:46:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1d940a2930 0x7f1d940a4df0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1dac10d9c0 0x7f1dac1adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1dac10b010 0x7f1dac1a6ef0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 --2- 172.21.15.7:0/902973946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dac1086a0 0x7f1dac1a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 >> 172.21.15.7:0/902973946 conn(0x7f1dac06e6b0 msgr2=0x7f1dac10b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 shutdown_connections 2024-03-26T08:46:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:13.751+0000 7f1db29f3640 1 -- 172.21.15.7:0/902973946 wait complete. 2024-03-26T08:46:14.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:14 smithi044 ceph-mon[36461]: pgmap v474: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.1 MiB/s wr, 863 op/s 2024-03-26T08:46:14.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:14 smithi016 ceph-mon[35851]: pgmap v474: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.1 MiB/s wr, 863 op/s 2024-03-26T08:46:14.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:14 smithi007 ceph-mon[32462]: pgmap v474: 97 pgs: 97 active+clean; 1.5 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.1 MiB/s wr, 863 op/s 2024-03-26T08:46:16.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 225.0 s, 3376.6 tps, lat 29.603 ms stddev 32.914 2024-03-26T08:46:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:16 smithi044 ceph-mon[36461]: pgmap v475: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.8 MiB/s wr, 902 op/s 2024-03-26T08:46:16.565 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:16 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:46:16.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:16 smithi016 ceph-mon[35851]: pgmap v475: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.8 MiB/s wr, 902 op/s 2024-03-26T08:46:16.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:16 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:46:16.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:16 smithi007 ceph-mon[32462]: pgmap v475: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.8 MiB/s wr, 902 op/s 2024-03-26T08:46:16.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:16 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:46:17.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:17 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:46:17.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:17 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:46:17.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:17 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:46:18.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:18 smithi016 ceph-mon[35851]: pgmap v476: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.3 MiB/s wr, 944 op/s 2024-03-26T08:46:18.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:18 smithi007 ceph-mon[32462]: pgmap v476: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.3 MiB/s wr, 944 op/s 2024-03-26T08:46:18.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:18 smithi044 ceph-mon[36461]: pgmap v476: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.3 MiB/s wr, 944 op/s 2024-03-26T08:46:20.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:20 smithi016 ceph-mon[35851]: pgmap v477: 97 pgs: 97 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.2 MiB/s wr, 882 op/s 2024-03-26T08:46:20.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:20 smithi007 ceph-mon[32462]: pgmap v477: 97 pgs: 97 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.2 MiB/s wr, 882 op/s 2024-03-26T08:46:20.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:20 smithi044 ceph-mon[36461]: pgmap v477: 97 pgs: 97 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.2 MiB/s wr, 882 op/s 2024-03-26T08:46:21.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 230.0 s, 3653.6 tps, lat 27.239 ms stddev 31.977 2024-03-26T08:46:22.593 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:22 smithi007 ceph-mon[32462]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.7 MiB/s wr, 822 op/s 2024-03-26T08:46:22.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:22 smithi016 ceph-mon[35851]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.7 MiB/s wr, 822 op/s 2024-03-26T08:46:22.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:22 smithi044 ceph-mon[36461]: pgmap v478: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.7 MiB/s wr, 822 op/s 2024-03-26T08:46:24.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:24 smithi016 ceph-mon[35851]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.9 MiB/s wr, 899 op/s 2024-03-26T08:46:24.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:24 smithi007 ceph-mon[32462]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.9 MiB/s wr, 899 op/s 2024-03-26T08:46:24.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:24 smithi044 ceph-mon[36461]: pgmap v479: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.9 MiB/s wr, 899 op/s 2024-03-26T08:46:26.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 235.0 s, 3421.4 tps, lat 29.359 ms stddev 33.027 2024-03-26T08:46:26.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:26 smithi016 ceph-mon[35851]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 856 op/s 2024-03-26T08:46:26.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:26 smithi007 ceph-mon[32462]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 856 op/s 2024-03-26T08:46:26.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:26 smithi044 ceph-mon[36461]: pgmap v480: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 856 op/s 2024-03-26T08:46:26.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.877+0000 7f37b641f640 1 -- 172.21.15.7:0/3938786645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b01017c0 msgr2=0x7f37b0101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:26.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.877+0000 7f37b641f640 1 --2- 172.21.15.7:0/3938786645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b01017c0 0x7f37b0101c40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009900 tx=0x7f37a402bc30 comp rx=0 tx=0).stop 2024-03-26T08:46:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.877+0000 7f37b641f640 1 -- 172.21.15.7:0/3938786645 shutdown_connections 2024-03-26T08:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.877+0000 7f37b641f640 1 --2- 172.21.15.7:0/3938786645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b0102180 0x7f37b0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.877+0000 7f37b641f640 1 --2- 172.21.15.7:0/3938786645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b01017c0 0x7f37b0101c40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.877+0000 7f37b641f640 1 --2- 172.21.15.7:0/3938786645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37b01005d0 0x7f37b01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.877+0000 7f37b641f640 1 -- 172.21.15.7:0/3938786645 >> 172.21.15.7:0/3938786645 conn(0x7f37b00fbb40 msgr2=0x7f37b00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.878+0000 7f37b641f640 1 -- 172.21.15.7:0/3938786645 shutdown_connections 2024-03-26T08:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.878+0000 7f37b641f640 1 -- 172.21.15.7:0/3938786645 wait complete. 2024-03-26T08:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37b641f640 1 Processor -- start 2024-03-26T08:46:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37b641f640 1 -- start start 2024-03-26T08:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37b641f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b01005d0 0x7f37b0195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37b641f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b01017c0 0x7f37b0195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37af7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b01017c0 0x7f37b0195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b01005d0 0x7f37b0195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37b641f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37b0102180 0x7f37b019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37b641f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b010c230 con 0x7f37b01017c0 2024-03-26T08:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b01005d0 0x7f37b0195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48048/0 (socket says 172.21.15.7:48048) 2024-03-26T08:46:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37affff640 1 -- 172.21.15.7:0/4145282696 learned_addr learned my addr 172.21.15.7:0/4145282696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b010c0b0 con 0x7f37b01005d0 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b010c3b0 con 0x7f37b0102180 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.879+0000 7f37b49bc640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37b0102180 0x7f37b019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37affff640 1 -- 172.21.15.7:0/4145282696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37b0102180 msgr2=0x7f37b019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37affff640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37b0102180 0x7f37b019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37affff640 1 -- 172.21.15.7:0/4145282696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b01017c0 msgr2=0x7f37b0195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37affff640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b01017c0 0x7f37b0195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37affff640 1 -- 172.21.15.7:0/4145282696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b019d4a0 con 0x7f37b01005d0 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37affff640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b01005d0 0x7f37b0195880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f37a000e9c0 tx=0x7f37a000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37ad7fa640 1 -- 172.21.15.7:0/4145282696 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a000cd80 con 0x7f37b01005d0 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b019d790 con 0x7f37b01005d0 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b019dd00 con 0x7f37b01005d0 2024-03-26T08:46:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.880+0000 7f37ad7fa640 1 -- 172.21.15.7:0/4145282696 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37a00035f0 con 0x7f37b01005d0 2024-03-26T08:46:26.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.881+0000 7f37ad7fa640 1 -- 172.21.15.7:0/4145282696 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0010690 con 0x7f37b01005d0 2024-03-26T08:46:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.882+0000 7f37ad7fa640 1 -- 172.21.15.7:0/4145282696 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f37a00037a0 con 0x7f37b01005d0 2024-03-26T08:46:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b9005740 1 Processor -- start 2024-03-26T08:46:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b9005740 1 -- start start 2024-03-26T08:46:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b9005740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 0x559c7d44ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b9005740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44b310 0x559c7d44fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b9005740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d450480 0x559c7d452910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b9005740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7d3afb10 con 0x559c7d44b310 2024-03-26T08:46:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b9005740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7d358900 con 0x559c7d450480 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b9005740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7d3abbb0 con 0x559c7d44a920 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b49bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 0x559c7d44ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d450480 0x559c7d452910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b49bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 0x559c7d44ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38692/0 (socket says 172.21.15.7:38692) 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37b49bc640 1 -- 172.21.15.7:0/2054471035 learned_addr learned my addr 172.21.15.7:0/2054471035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37affff640 1 --2- 172.21.15.7:0/2054471035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44b310 0x559c7d44fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.883+0000 7f37ad7fa640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37980a26c0 0x7f37980a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37af7fe640 1 -- 172.21.15.7:0/2054471035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 msgr2=0x559c7d44ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37af7fe640 1 --2- 172.21.15.7:0/2054471035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 0x559c7d44ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37af7fe640 1 -- 172.21.15.7:0/2054471035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44b310 msgr2=0x559c7d44fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37af7fe640 1 --2- 172.21.15.7:0/2054471035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44b310 0x559c7d44fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37af7fe640 1 -- 172.21.15.7:0/2054471035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c7d0c4210 con 0x559c7d450480 2024-03-26T08:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37b49bc640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37980a26c0 0x7f37980a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37ad7fa640 1 -- 172.21.15.7:0/4145282696 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f37a00cee10 con 0x7f37b01005d0 2024-03-26T08:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37af7fe640 1 --2- 172.21.15.7:0/2054471035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d450480 0x559c7d452910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000c00 tx=0x7f37a4009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8450951c7b9b0301 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37967fc640 1 -- 172.21.15.7:0/2054471035 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4042070 con 0x559c7d450480 2024-03-26T08:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37967fc640 1 -- 172.21.15.7:0/2054471035 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37a402c5d0 con 0x559c7d450480 2024-03-26T08:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.884+0000 7f37967fc640 1 -- 172.21.15.7:0/2054471035 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4004ef0 con 0x559c7d450480 2024-03-26T08:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.885+0000 7f37b9005740 1 -- 172.21.15.7:0/2054471035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d450480 msgr2=0x559c7d452910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.885+0000 7f37b9005740 1 --2- 172.21.15.7:0/2054471035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d450480 0x559c7d452910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000c00 tx=0x7f37a4009610 comp rx=0 tx=0).stop 2024-03-26T08:46:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.885+0000 7f37b49bc640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37980a26c0 0x7f37980a4b80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x559c7d44bb70 tx=0x7f37a8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.886+0000 7f37b9005740 1 -- 172.21.15.7:0/2054471035 shutdown_connections 2024-03-26T08:46:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.886+0000 7f37b9005740 1 --2- 172.21.15.7:0/2054471035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d450480 0x559c7d452910 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.886+0000 7f37b9005740 1 --2- 172.21.15.7:0/2054471035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44b310 0x559c7d44fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.886+0000 7f37b9005740 1 --2- 172.21.15.7:0/2054471035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 0x559c7d44ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.886+0000 7f37b9005740 1 -- 172.21.15.7:0/2054471035 >> 172.21.15.7:0/2054471035 conn(0x559c7d445e90 msgr2=0x559c7d446290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.886+0000 7f37b9005740 1 -- 172.21.15.7:0/2054471035 shutdown_connections 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.886+0000 7f37b9005740 1 -- 172.21.15.7:0/2054471035 wait complete. 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.886+0000 7f37b9005740 1 -- start start 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37b9005740 1 Processor -- start 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37b9005740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44a920 0x559c7d465bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37b9005740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d466130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37b9005740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d450480 0x559c7d46d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37b9005740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7d4533d0 con 0x559c7d44a920 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d466130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37af7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d466130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48072/0 (socket says 172.21.15.7:48072) 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37af7fe640 1 -- 172.21.15.7:0/3930598905 learned_addr learned my addr 172.21.15.7:0/3930598905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7d36bd50 con 0x559c7d44b310 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7d453550 con 0x559c7d450480 2024-03-26T08:46:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37affff640 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44a920 0x559c7d465bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.887+0000 7f37b49bc640 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d450480 0x559c7d46d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f37af7fe640 1 -- 172.21.15.7:0/3930598905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d450480 msgr2=0x559c7d46d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f37af7fe640 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d450480 0x559c7d46d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f37af7fe640 1 -- 172.21.15.7:0/3930598905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44a920 msgr2=0x559c7d465bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f37af7fe640 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44a920 0x559c7d465bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f37af7fe640 1 -- 172.21.15.7:0/3930598905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c7d190620 con 0x559c7d44b310 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f37b9005740 10 client.0 _ll_register_callbacks cb 0x559c7d36b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f37af7fe640 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d466130 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37a402cad0 tx=0x7f37a402cb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99aa77892674536a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f378f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37a402c5d0 con 0x559c7d44b310 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f378f7fe640 1 -- 172.21.15.7:0/3930598905 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4042040 con 0x559c7d44b310 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.888+0000 7f378f7fe640 1 -- 172.21.15.7:0/3930598905 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f37a4004450 con 0x559c7d44b310 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f378f7fe640 1 -- 172.21.15.7:0/3930598905 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a40046c0 con 0x559c7d44b310 2024-03-26T08:46:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f37b9005740 10 client.26297 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x559c7d4700e0 con 0x559c7d44b310 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f378f7fe640 1 -- 172.21.15.7:0/3930598905 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f37a4004ef0 con 0x559c7d44b310 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f37b9005740 10 client.26297 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f37b9005740 10 client.26297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f37b9005740 20 client.26297 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f37b9005740 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559c7d471540 0x559c7d473930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f37b9005740 4 client.26297 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c7d3bd050 con 0x559c7d471540 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.889+0000 7f37affff640 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559c7d471540 0x559c7d473930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.890+0000 7f37affff640 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559c7d471540 0x559c7d473930 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0441e32af284a46 server_cookie=342c7f2159eb9d00 in_seq=0 out_seq=0 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.890+0000 7f378f7fe640 10 client.26297 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f378f7fe640 1 -- 172.21.15.7:0/3930598905 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559c7d3bd050 con 0x559c7d471540 2024-03-26T08:46:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f378f7fe640 10 client.26297 handle_command_reply: tid=0 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 client.26297 shutdown 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 msgr2=0x559c7d466130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d466130 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f37a402cad0 tx=0x7f37a402cb00 comp rx=0 tx=0).stop 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 shutdown_connections 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559c7d471540 0x559c7d473930 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d450480 0x559c7d46d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d466130 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 --2- 172.21.15.7:0/3930598905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d44a920 0x559c7d465bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f378f7fe640 0 client.26297 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 >> 172.21.15.7:0/3930598905 conn(0x559c7d445e90 msgr2=0x559c7d445020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 shutdown_connections 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 1 -- 172.21.15.7:0/3930598905 wait complete. 2024-03-26T08:46:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.891+0000 7f37b9005740 20 client.26297 trim_cache size 0 max 16384 2024-03-26T08:46:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.914+0000 7f37b9005740 1 Processor -- start 2024-03-26T08:46:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b9005740 1 -- start start 2024-03-26T08:46:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b9005740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 0x559c7d44bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b9005740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d46d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b9005740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d4490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b9005740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b01b1400 con 0x559c7d450480 2024-03-26T08:46:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b9005740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b0073300 con 0x559c7d44b310 2024-03-26T08:46:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b9005740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b004fc50 con 0x559c7d44a920 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b49bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d4490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b49bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d4490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58306/0 (socket says 172.21.15.7:58306) 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37affff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 0x559c7d44bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.915+0000 7f37b49bc640 1 -- 172.21.15.7:0/3694340292 learned_addr learned my addr 172.21.15.7:0/3694340292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.916+0000 7f37b49bc640 1 -- 172.21.15.7:0/3694340292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 msgr2=0x559c7d44bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.916+0000 7f37b49bc640 1 --2- 172.21.15.7:0/3694340292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 0x559c7d44bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.916+0000 7f37b49bc640 1 -- 172.21.15.7:0/3694340292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 msgr2=0x559c7d46d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.916+0000 7f37af7fe640 1 --2- 172.21.15.7:0/3694340292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d46d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.916+0000 7f37b49bc640 1 --2- 172.21.15.7:0/3694340292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d46d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.916+0000 7f37b49bc640 1 -- 172.21.15.7:0/3694340292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c7d44bba0 con 0x559c7d450480 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.916+0000 7f37af7fe640 1 --2- 172.21.15.7:0/3694340292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d46d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.916+0000 7f37b49bc640 1 --2- 172.21.15.7:0/3694340292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d4490f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f37a8013430 tx=0x7f37a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b5c0e534408f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.916+0000 7f3795ffb640 1 -- 172.21.15.7:0/3694340292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8010ba0 con 0x559c7d450480 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.917+0000 7f3795ffb640 1 -- 172.21.15.7:0/3694340292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37a8010d40 con 0x559c7d450480 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.917+0000 7f3795ffb640 1 -- 172.21.15.7:0/3694340292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80118a0 con 0x559c7d450480 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.917+0000 7f37b9005740 1 -- 172.21.15.7:0/3694340292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 msgr2=0x559c7d4490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.917+0000 7f37b9005740 1 --2- 172.21.15.7:0/3694340292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d4490f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f37a8013430 tx=0x7f37a8016000 comp rx=0 tx=0).stop 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.918+0000 7f37b9005740 1 -- 172.21.15.7:0/3694340292 shutdown_connections 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.918+0000 7f37b9005740 1 --2- 172.21.15.7:0/3694340292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d4490f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.918+0000 7f37b9005740 1 --2- 172.21.15.7:0/3694340292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44b310 0x559c7d46d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.918+0000 7f37b9005740 1 --2- 172.21.15.7:0/3694340292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44a920 0x559c7d44bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.918+0000 7f37b9005740 1 -- 172.21.15.7:0/3694340292 >> 172.21.15.7:0/3694340292 conn(0x559c7d445e90 msgr2=0x559c7d446270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.918+0000 7f37b9005740 1 -- 172.21.15.7:0/3694340292 shutdown_connections 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.918+0000 7f37b9005740 1 -- 172.21.15.7:0/3694340292 wait complete. 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.918+0000 7f37b9005740 1 -- start start 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37b9005740 1 Processor -- start 2024-03-26T08:46:26.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37b9005740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44a920 0x559c7d46ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37b9005740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44b310 0x559c7d46b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37b9005740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d46c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37b9005740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00719a0 con 0x559c7d450480 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37b9005740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b0072ca0 con 0x559c7d44a920 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37b9005740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b0071fe0 con 0x559c7d44b310 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37b9005740 10 client.0 _ll_register_callbacks cb 0x7f37a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44a920 0x559c7d46ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44a920 0x559c7d46ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48098/0 (socket says 172.21.15.7:48098) 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37af7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44b310 0x559c7d46b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37affff640 1 -- 172.21.15.7:0/3462456278 learned_addr learned my addr 172.21.15.7:0/3462456278 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.919+0000 7f37b49bc640 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d46c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37af7fe640 1 -- 172.21.15.7:0/3462456278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44a920 msgr2=0x559c7d46ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37af7fe640 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44a920 0x559c7d46ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37af7fe640 1 -- 172.21.15.7:0/3462456278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 msgr2=0x559c7d46c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:26.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37af7fe640 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d46c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37af7fe640 1 -- 172.21.15.7:0/3462456278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c7d46c680 con 0x559c7d44b310 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37affff640 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44a920 0x559c7d46ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37af7fe640 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44b310 0x559c7d46b080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f37a402c780 tx=0x7f37a4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11eefcf27a8589a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f378f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f378f7fe640 1 -- 172.21.15.7:0/3462456278 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4042070 con 0x559c7d44b310 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f378f7fe640 1 -- 172.21.15.7:0/3462456278 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f37a4004860 con 0x559c7d44b310 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37b9005740 1 -- 172.21.15.7:0/3462456278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37a8010d40 con 0x559c7d44b310 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f378f7fe640 1 -- 172.21.15.7:0/3462456278 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a402cc20 con 0x559c7d44b310 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37b9005740 10 client.26323 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f37b9005740 1 -- 172.21.15.7:0/3462456278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559c7d4cd070 con 0x559c7d44b310 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.920+0000 7f378f7fe640 1 -- 172.21.15.7:0/3462456278 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37a8010d40 con 0x559c7d44b310 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f378f7fe640 1 -- 172.21.15.7:0/3462456278 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f37a4065460 con 0x559c7d44b310 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f37b9005740 10 client.26323 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f37b9005740 10 client.26323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:26.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f37b9005740 20 client.26323 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f37b9005740 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559c7d4cd9f0 0x559c7d4cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f37b9005740 4 client.26323 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:46:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f37b9005740 1 -- 172.21.15.7:0/3462456278 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559c7d3b73a0 con 0x559c7d4cd9f0 2024-03-26T08:46:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f37affff640 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559c7d4cd9f0 0x559c7d4cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f378f7fe640 10 client.26323 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:26.918 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.921+0000 7f37affff640 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559c7d4cd9f0 0x559c7d4cfe10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5acd9a07757d808 server_cookie=d838afa402471faf in_seq=0 out_seq=0 2024-03-26T08:46:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.967+0000 7f378f7fe640 1 -- 172.21.15.7:0/3462456278 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559c7d3b73a0 con 0x559c7d4cd9f0 2024-03-26T08:46:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.967+0000 7f378f7fe640 10 client.26323 handle_command_reply: tid=0 2024-03-26T08:46:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.967+0000 7f37b9005740 1 client.26323 shutdown 2024-03-26T08:46:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.967+0000 7f37b9005740 1 -- 172.21.15.7:0/3462456278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44b310 msgr2=0x559c7d46b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.967+0000 7f37b9005740 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44b310 0x559c7d46b080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f37a402c780 tx=0x7f37a4005fd0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f37b9005740 1 -- 172.21.15.7:0/3462456278 shutdown_connections 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f37b9005740 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559c7d4cd9f0 0x559c7d4cfe10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f37b9005740 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559c7d450480 0x559c7d46c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f378f7fe640 0 client.26323 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f37b9005740 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559c7d44b310 0x559c7d46b080 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f37b9005740 1 --2- 172.21.15.7:0/3462456278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559c7d44a920 0x559c7d46ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f37b9005740 1 -- 172.21.15.7:0/3462456278 >> 172.21.15.7:0/3462456278 conn(0x559c7d445e90 msgr2=0x559c7d446270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f37b9005740 1 -- 172.21.15.7:0/3462456278 shutdown_connections 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f37b9005740 1 -- 172.21.15.7:0/3462456278 wait complete. 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.968+0000 7f37b9005740 20 client.26323 trim_cache size 0 max 16384 2024-03-26T08:46:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:46:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:46:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:46:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37980a26c0 msgr2=0x7f37980a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37980a26c0 0x7f37980a4b80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x559c7d44bb70 tx=0x7f37a8004a40 comp rx=0 tx=0).stop 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b01005d0 msgr2=0x7f37b0195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b01005d0 0x7f37b0195880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f37a000e9c0 tx=0x7f37a000ee80 comp rx=0 tx=0).stop 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 shutdown_connections 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f37980a26c0 0x7f37980a4b80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37b0102180 0x7f37b019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37b01017c0 0x7f37b0195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 --2- 172.21.15.7:0/4145282696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b01005d0 0x7f37b0195880 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 >> 172.21.15.7:0/4145282696 conn(0x7f37b00fbb40 msgr2=0x7f37b00fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 shutdown_connections 2024-03-26T08:46:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:26.969+0000 7f37b641f640 1 -- 172.21.15.7:0/4145282696 wait complete. 2024-03-26T08:46:28.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:28 smithi016 ceph-mon[35851]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 744 op/s 2024-03-26T08:46:28.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:28 smithi007 ceph-mon[32462]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 744 op/s 2024-03-26T08:46:28.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:28 smithi044 ceph-mon[36461]: pgmap v481: 97 pgs: 97 active+clean; 1.6 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 MiB/s wr, 744 op/s 2024-03-26T08:46:30.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:30 smithi016 ceph-mon[35851]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-03-26T08:46:30.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:30 smithi007 ceph-mon[32462]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-03-26T08:46:30.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:30 smithi044 ceph-mon[36461]: pgmap v482: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-03-26T08:46:31.070 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 240.0 s, 2949.0 tps, lat 33.823 ms stddev 40.176 2024-03-26T08:46:32.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:32 smithi016 ceph-mon[35851]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 754 op/s 2024-03-26T08:46:32.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:32 smithi007 ceph-mon[32462]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 754 op/s 2024-03-26T08:46:32.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:32 smithi044 ceph-mon[36461]: pgmap v483: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 754 op/s 2024-03-26T08:46:33.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:33 smithi016 ceph-mon[35851]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 821 op/s 2024-03-26T08:46:33.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:33 smithi007 ceph-mon[32462]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 821 op/s 2024-03-26T08:46:33.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:33 smithi044 ceph-mon[36461]: pgmap v484: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.4 MiB/s wr, 821 op/s 2024-03-26T08:46:35.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:35 smithi016 ceph-mon[35851]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 709 op/s 2024-03-26T08:46:35.791 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:35 smithi007 ceph-mon[32462]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 709 op/s 2024-03-26T08:46:35.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:35 smithi044 ceph-mon[36461]: pgmap v485: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 709 op/s 2024-03-26T08:46:36.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 245.0 s, 2973.8 tps, lat 33.660 ms stddev 36.565 2024-03-26T08:46:36.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:36 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:46:36.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:36 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:46:36.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:36 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:46:37.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:46:37.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:46:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.388+0000 7f5d036c0640 1 -- 172.21.15.7:0/3077194664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1005c0 msgr2=0x7f5cfc1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.388+0000 7f5d036c0640 1 --2- 172.21.15.7:0/3077194664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1005c0 0x7f5cfc1009e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009900 tx=0x7f5cf802bc30 comp rx=0 tx=0).stop 2024-03-26T08:46:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.388+0000 7f5d036c0640 1 -- 172.21.15.7:0/3077194664 shutdown_connections 2024-03-26T08:46:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.388+0000 7f5d036c0640 1 --2- 172.21.15.7:0/3077194664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc102170 0x7f5cfc108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.388+0000 7f5d036c0640 1 --2- 172.21.15.7:0/3077194664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cfc1017b0 0x7f5cfc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.388+0000 7f5d036c0640 1 --2- 172.21.15.7:0/3077194664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1005c0 0x7f5cfc1009e0 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009900 tx=0x7f5cf802bc30 comp rx=0 tx=0).stop 2024-03-26T08:46:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.388+0000 7f5d036c0640 1 -- 172.21.15.7:0/3077194664 >> 172.21.15.7:0/3077194664 conn(0x7f5cfc0fbb30 msgr2=0x7f5cfc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.389+0000 7f5d036c0640 1 -- 172.21.15.7:0/3077194664 shutdown_connections 2024-03-26T08:46:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.389+0000 7f5d036c0640 1 -- 172.21.15.7:0/3077194664 wait complete. 2024-03-26T08:46:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.390+0000 7f5d036c0640 1 Processor -- start 2024-03-26T08:46:37.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.393+0000 7f5d036c0640 1 -- start start 2024-03-26T08:46:37.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.393+0000 7f5d036c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1017b0 0x7f5cfc111350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.393+0000 7f5d036c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc102170 0x7f5cfc111890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.393+0000 7f5d036c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cfc10f990 0x7f5cfc10fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.393+0000 7f5d036c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc10a060 con 0x7f5cfc10f990 2024-03-26T08:46:37.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.393+0000 7f5d036c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc109ee0 con 0x7f5cfc102170 2024-03-26T08:46:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.393+0000 7f5d036c0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc10a1e0 con 0x7f5cfc1017b0 2024-03-26T08:46:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.395+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1017b0 0x7f5cfc111350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.395+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1017b0 0x7f5cfc111350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59348/0 (socket says 172.21.15.7:59348) 2024-03-26T08:46:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.395+0000 7f5d0145c640 1 -- 172.21.15.7:0/4278869314 learned_addr learned my addr 172.21.15.7:0/4278869314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.396+0000 7f5d01c5d640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cfc10f990 0x7f5cfc10fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.398+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc102170 0x7f5cfc111890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.398+0000 7f5d0145c640 1 -- 172.21.15.7:0/4278869314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc102170 msgr2=0x7f5cfc111890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.398+0000 7f5d0145c640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc102170 0x7f5cfc111890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.398+0000 7f5d0145c640 1 -- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cfc10f990 msgr2=0x7f5cfc10fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.398+0000 7f5d0145c640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cfc10f990 0x7f5cfc10fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.398+0000 7f5d0145c640 1 -- 172.21.15.7:0/4278869314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc1106d0 con 0x7f5cfc1017b0 2024-03-26T08:46:37.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.398+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc102170 0x7f5cfc111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:46:37.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.399+0000 7f5d0145c640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1017b0 0x7f5cfc111350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009230 tx=0x7f5cf8004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:37.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.401+0000 7f5cf27fc640 1 -- 172.21.15.7:0/4278869314 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8043020 con 0x7f5cfc1017b0 2024-03-26T08:46:37.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.401+0000 7f5cf27fc640 1 -- 172.21.15.7:0/4278869314 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cf80054e0 con 0x7f5cfc1017b0 2024-03-26T08:46:37.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.401+0000 7f5cf27fc640 1 -- 172.21.15.7:0/4278869314 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8003ba0 con 0x7f5cfc1017b0 2024-03-26T08:46:37.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.404+0000 7f5d036c0640 1 -- 172.21.15.7:0/4278869314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc06b0c0 con 0x7f5cfc1017b0 2024-03-26T08:46:37.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.404+0000 7f5d036c0640 1 -- 172.21.15.7:0/4278869314 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc06b4d0 con 0x7f5cfc1017b0 2024-03-26T08:46:37.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.409+0000 7f5cf27fc640 1 -- 172.21.15.7:0/4278869314 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5cf8046020 con 0x7f5cfc1017b0 2024-03-26T08:46:37.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.410+0000 7f5cf27fc640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ce80a2a00 0x7f5ce80a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.410+0000 7f5cf27fc640 1 -- 172.21.15.7:0/4278869314 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5cf80f04b0 con 0x7f5cfc1017b0 2024-03-26T08:46:37.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.412+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ce80a2a00 0x7f5ce80a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.414+0000 7f5d06385740 1 Processor -- start 2024-03-26T08:46:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.414+0000 7f5d06385740 1 -- start start 2024-03-26T08:46:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.414+0000 7f5d06385740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.414+0000 7f5d06385740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a0790 0x55c9664a5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.414+0000 7f5d06385740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 0x55c9664a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.414+0000 7f5d06385740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9664053d0 con 0x55c96649fda0 2024-03-26T08:46:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.414+0000 7f5d06385740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9663adad0 con 0x55c9664a5900 2024-03-26T08:46:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.414+0000 7f5d06385740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c966401470 con 0x55c9664a0790 2024-03-26T08:46:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.415+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ce80a2a00 0x7f5ce80a4ec0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc1110c0 tx=0x7f5cec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.415+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.416+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55404/0 (socket says 172.21.15.7:55404) 2024-03-26T08:46:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.416+0000 7f5d0145c640 1 -- 172.21.15.7:0/1448300472 learned_addr learned my addr 172.21.15.7:0/1448300472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.416+0000 7f5d01c5d640 1 --2- 172.21.15.7:0/1448300472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 0x55c9664a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.417+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/1448300472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a0790 0x55c9664a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.419+0000 7f5d00c5b640 1 -- 172.21.15.7:0/1448300472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 msgr2=0x55c9664a7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.419+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/1448300472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 0x55c9664a7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.419+0000 7f5d00c5b640 1 -- 172.21.15.7:0/1448300472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 msgr2=0x55c9664a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.419+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/1448300472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.419+0000 7f5d00c5b640 1 -- 172.21.15.7:0/1448300472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9661e37d0 con 0x55c9664a0790 2024-03-26T08:46:37.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.420+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/1448300472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a0790 0x55c9664a5290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5cec010e30 tx=0x7f5cec013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c023435d982e7f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.422+0000 7f5ce77fe640 1 -- 172.21.15.7:0/1448300472 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec012e60 con 0x55c9664a0790 2024-03-26T08:46:37.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.422+0000 7f5ce77fe640 1 -- 172.21.15.7:0/1448300472 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cec010e60 con 0x55c9664a0790 2024-03-26T08:46:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.423+0000 7f5ce77fe640 1 -- 172.21.15.7:0/1448300472 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec01f6b0 con 0x55c9664a0790 2024-03-26T08:46:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.423+0000 7f5d06385740 1 -- 172.21.15.7:0/1448300472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a0790 msgr2=0x55c9664a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.423+0000 7f5d06385740 1 --2- 172.21.15.7:0/1448300472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a0790 0x55c9664a5290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5cec010e30 tx=0x7f5cec013000 comp rx=0 tx=0).stop 2024-03-26T08:46:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.423+0000 7f5d06385740 1 -- 172.21.15.7:0/1448300472 shutdown_connections 2024-03-26T08:46:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.423+0000 7f5d06385740 1 --2- 172.21.15.7:0/1448300472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 0x55c9664a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.423+0000 7f5d06385740 1 --2- 172.21.15.7:0/1448300472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a0790 0x55c9664a5290 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5cec010e30 tx=0x7f5cec013000 comp rx=0 tx=0).stop 2024-03-26T08:46:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.423+0000 7f5d06385740 1 --2- 172.21.15.7:0/1448300472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.423+0000 7f5d06385740 1 -- 172.21.15.7:0/1448300472 >> 172.21.15.7:0/1448300472 conn(0x55c96649b310 msgr2=0x55c96649b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.423+0000 7f5d06385740 1 -- 172.21.15.7:0/1448300472 shutdown_connections 2024-03-26T08:46:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.424+0000 7f5d06385740 1 -- 172.21.15.7:0/1448300472 wait complete. 2024-03-26T08:46:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.424+0000 7f5d06385740 1 -- start start 2024-03-26T08:46:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.424+0000 7f5d06385740 1 Processor -- start 2024-03-26T08:46:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d06385740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d06385740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 0x55c9664aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d06385740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 0x55c9664b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d06385740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9664a8900 con 0x55c96649fda0 2024-03-26T08:46:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55416/0 (socket says 172.21.15.7:55416) 2024-03-26T08:46:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d0145c640 1 -- 172.21.15.7:0/3106762173 learned_addr learned my addr 172.21.15.7:0/3106762173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9663c1060 con 0x55c9664a5900 2024-03-26T08:46:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9664a8a80 con 0x55c9664afa30 2024-03-26T08:46:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d01c5d640 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 0x55c9664b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d06385740 10 client.0 _ll_register_callbacks cb 0x55c9663a36a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.425+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 0x55c9664aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.426+0000 7f5d0145c640 1 -- 172.21.15.7:0/3106762173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 msgr2=0x55c9664b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.426+0000 7f5d0145c640 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 0x55c9664b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.426+0000 7f5d0145c640 1 -- 172.21.15.7:0/3106762173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 msgr2=0x55c9664aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.426+0000 7f5d0145c640 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 0x55c9664aaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.426+0000 7f5d0145c640 1 -- 172.21.15.7:0/3106762173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9661193f0 con 0x55c96649fda0 2024-03-26T08:46:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.426+0000 7f5d01c5d640 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 0x55c9664b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.426+0000 7f5d0145c640 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664aa4c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5cf802ceb0 tx=0x7f5cf802c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52baaf4bf1abeb6d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.426+0000 7f5ce4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.426+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/3106762173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80da060 con 0x55c96649fda0 2024-03-26T08:46:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.427+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/3106762173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5cf8003d40 con 0x55c96649fda0 2024-03-26T08:46:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.427+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cec010e60 con 0x55c96649fda0 2024-03-26T08:46:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.431+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/3106762173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cf80bd510 con 0x55c96649fda0 2024-03-26T08:46:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.432+0000 7f5d06385740 10 client.16479 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.432+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c9660d90f0 con 0x55c96649fda0 2024-03-26T08:46:37.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.434+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/3106762173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cec010e60 con 0x55c96649fda0 2024-03-26T08:46:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.434+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/3106762173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5cf80d4070 con 0x55c96649fda0 2024-03-26T08:46:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.434+0000 7f5d06385740 10 client.16479 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.434+0000 7f5d06385740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.434+0000 7f5d06385740 20 client.16479 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.434+0000 7f5d06385740 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c9664b6210 0x55c9664b8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.434+0000 7f5d06385740 4 client.16479 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.434+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c966412910 con 0x55c9664b6210 2024-03-26T08:46:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.435+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c9664b6210 0x55c9664b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.436+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c9664b6210 0x55c9664b8680 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4c401c90d9c124 server_cookie=f3423aff35599804 in_seq=0 out_seq=0 2024-03-26T08:46:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.437+0000 7f5ce4ff9640 10 client.16479 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.438+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/3106762173 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c966412910 con 0x55c9664b6210 2024-03-26T08:46:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.438+0000 7f5ce4ff9640 10 client.16479 handle_command_reply: tid=0 2024-03-26T08:46:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.438+0000 7f5d06385740 1 client.16479 shutdown 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.438+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 msgr2=0x55c9664aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.438+0000 7f5d06385740 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664aa4c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5cf802ceb0 tx=0x7f5cf802c6d0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 shutdown_connections 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5d06385740 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c9664b6210 0x55c9664b8680 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5d06385740 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 0x55c9664b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5d06385740 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664a5900 0x55c9664aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5d06385740 1 --2- 172.21.15.7:0/3106762173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c96649fda0 0x55c9664aa4c0 secure :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5cf802ceb0 tx=0x7f5cf802c6d0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 >> 172.21.15.7:0/3106762173 conn(0x55c96649b310 msgr2=0x55c96649a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5ce4ff9640 0 client.16479 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 shutdown_connections 2024-03-26T08:46:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5d06385740 1 -- 172.21.15.7:0/3106762173 wait complete. 2024-03-26T08:46:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.439+0000 7f5d06385740 20 client.16479 trim_cache size 0 max 16384 2024-03-26T08:46:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.471+0000 7f5d06385740 1 Processor -- start 2024-03-26T08:46:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.472+0000 7f5d06385740 1 -- start start 2024-03-26T08:46:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.472+0000 7f5d06385740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a5900 0x55c9664a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.472+0000 7f5d06385740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664afa30 0x55c9664a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.472+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a5900 0x55c9664a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.472+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a5900 0x55c9664a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59382/0 (socket says 172.21.15.7:59382) 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.473+0000 7f5d0145c640 1 -- 172.21.15.7:0/3486056174 learned_addr learned my addr 172.21.15.7:0/3486056174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.473+0000 7f5d06385740 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664aa0d0 0x55c9664aa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.473+0000 7f5d06385740 1 -- 172.21.15.7:0/3486056174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc115f60 con 0x55c9664afa30 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.473+0000 7f5d06385740 1 -- 172.21.15.7:0/3486056174 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc113a60 con 0x55c9664aa0d0 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.473+0000 7f5d01c5d640 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664aa0d0 0x55c9664aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.473+0000 7f5d06385740 1 -- 172.21.15.7:0/3486056174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc072420 con 0x55c9664a5900 2024-03-26T08:46:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.473+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664afa30 0x55c9664a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.474+0000 7f5d0145c640 1 -- 172.21.15.7:0/3486056174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664aa0d0 msgr2=0x55c9664aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.474+0000 7f5d0145c640 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664aa0d0 0x55c9664aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.474+0000 7f5d0145c640 1 -- 172.21.15.7:0/3486056174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664afa30 msgr2=0x55c9664a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.474+0000 7f5d0145c640 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664afa30 0x55c9664a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.474+0000 7f5d0145c640 1 -- 172.21.15.7:0/3486056174 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9664a50f0 con 0x55c9664a5900 2024-03-26T08:46:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.474+0000 7f5d0145c640 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a5900 0x55c9664a1e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80edf70 tx=0x7f5cf802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d112a569fa0c7596 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.474+0000 7f5ce6ffd640 1 -- 172.21.15.7:0/3486056174 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80da070 con 0x55c9664a5900 2024-03-26T08:46:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.474+0000 7f5ce6ffd640 1 -- 172.21.15.7:0/3486056174 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cf80bce10 con 0x55c9664a5900 2024-03-26T08:46:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.474+0000 7f5ce6ffd640 1 -- 172.21.15.7:0/3486056174 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80ee080 con 0x55c9664a5900 2024-03-26T08:46:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.475+0000 7f5d06385740 1 -- 172.21.15.7:0/3486056174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a5900 msgr2=0x55c9664a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.475+0000 7f5d06385740 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a5900 0x55c9664a1e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80edf70 tx=0x7f5cf802bc30 comp rx=0 tx=0).stop 2024-03-26T08:46:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.476+0000 7f5d06385740 1 -- 172.21.15.7:0/3486056174 shutdown_connections 2024-03-26T08:46:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.476+0000 7f5d06385740 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664aa0d0 0x55c9664aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.476+0000 7f5d06385740 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664afa30 0x55c9664a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.476+0000 7f5d06385740 1 --2- 172.21.15.7:0/3486056174 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664a5900 0x55c9664a1e70 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80edf70 tx=0x7f5cf802bc30 comp rx=0 tx=0).stop 2024-03-26T08:46:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.476+0000 7f5d06385740 1 -- 172.21.15.7:0/3486056174 >> 172.21.15.7:0/3486056174 conn(0x55c96649b310 msgr2=0x55c966496390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.476+0000 7f5d06385740 1 -- 172.21.15.7:0/3486056174 shutdown_connections 2024-03-26T08:46:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.476+0000 7f5d06385740 1 -- 172.21.15.7:0/3486056174 wait complete. 2024-03-26T08:46:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.477+0000 7f5d06385740 1 -- start start 2024-03-26T08:46:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.477+0000 7f5d06385740 1 Processor -- start 2024-03-26T08:46:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.477+0000 7f5d06385740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664aa0d0 0x55c9664c7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d06385740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 0x55c9664c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664aa0d0 0x55c9664c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d0145c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664aa0d0 0x55c9664c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55436/0 (socket says 172.21.15.7:55436) 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d0145c640 1 -- 172.21.15.7:0/24959959 learned_addr learned my addr 172.21.15.7:0/24959959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d06385740 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664c7ec0 0x55c966511000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc072a20 con 0x55c9664aa0d0 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc113400 con 0x55c9664c7ec0 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d01c5d640 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664c7ec0 0x55c966511000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc073080 con 0x55c9664afa30 2024-03-26T08:46:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.478+0000 7f5d00c5b640 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 0x55c9664c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.479+0000 7f5d06385740 10 client.0 _ll_register_callbacks cb 0x7f5cf8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.479+0000 7f5d01c5d640 1 -- 172.21.15.7:0/24959959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 msgr2=0x55c9664c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.479+0000 7f5d01c5d640 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 0x55c9664c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.479+0000 7f5d01c5d640 1 -- 172.21.15.7:0/24959959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664aa0d0 msgr2=0x55c9664c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.479+0000 7f5d01c5d640 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664aa0d0 0x55c9664c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.479+0000 7f5d01c5d640 1 -- 172.21.15.7:0/24959959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9665116f0 con 0x55c9664c7ec0 2024-03-26T08:46:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.480+0000 7f5d01c5d640 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664c7ec0 0x55c966511000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55c966495b50 tx=0x7f5cf4010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf6ef4025d2aeba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.480+0000 7f5ce4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.480+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/24959959 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4004b20 con 0x55c9664c7ec0 2024-03-26T08:46:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.480+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/24959959 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5cf4004cc0 con 0x55c9664c7ec0 2024-03-26T08:46:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.480+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cf80bce10 con 0x55c9664c7ec0 2024-03-26T08:46:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.481+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/24959959 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cf4005020 con 0x55c9664c7ec0 2024-03-26T08:46:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.481+0000 7f5d06385740 10 client.26312 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.481+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c966512a70 con 0x55c9664c7ec0 2024-03-26T08:46:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.482+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/24959959 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cf40056a0 con 0x55c9664c7ec0 2024-03-26T08:46:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.482+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/24959959 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5cf4026020 con 0x55c9664c7ec0 2024-03-26T08:46:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.482+0000 7f5d06385740 10 client.26312 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.482+0000 7f5d06385740 10 client.26312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.482+0000 7f5d06385740 20 client.26312 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.482+0000 7f5d06385740 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c9665134d0 0x55c9665158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.482+0000 7f5d06385740 4 client.26312 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:46:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.482+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f5cf80bce10 con 0x55c9665134d0 2024-03-26T08:46:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.482+0000 7f5d0145c640 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c9665134d0 0x55c9665158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.483+0000 7f5ce4ff9640 10 client.26312 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.483+0000 7f5d0145c640 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c9665134d0 0x55c9665158f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d30b1f95b40a66 server_cookie=3b2d20ae32c57df1 in_seq=0 out_seq=0 2024-03-26T08:46:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.483+0000 7f5ce4ff9640 1 -- 172.21.15.7:0/24959959 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f5cf80bce10 con 0x55c9665134d0 2024-03-26T08:46:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.483+0000 7f5ce4ff9640 10 client.26312 handle_command_reply: tid=0 2024-03-26T08:46:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.483+0000 7f5d06385740 1 client.26312 shutdown 2024-03-26T08:46:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.484+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664c7ec0 msgr2=0x55c966511000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.484+0000 7f5d06385740 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664c7ec0 0x55c966511000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55c966495b50 tx=0x7f5cf4010800 comp rx=0 tx=0).stop 2024-03-26T08:46:37.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.484+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 shutdown_connections 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.484+0000 7f5d06385740 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c9665134d0 0x55c9665158f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.484+0000 7f5d06385740 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c9664c7ec0 0x55c966511000 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.484+0000 7f5ce4ff9640 0 client.26312 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.484+0000 7f5d06385740 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c9664afa30 0x55c9664c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.484+0000 7f5d06385740 1 --2- 172.21.15.7:0/24959959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9664aa0d0 0x55c9664c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.484+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 >> 172.21.15.7:0/24959959 conn(0x55c96649b310 msgr2=0x55c9664aa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.485+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 shutdown_connections 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.485+0000 7f5d06385740 1 -- 172.21.15.7:0/24959959 wait complete. 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:46:37.477 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:46:37.478 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:46:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.485+0000 7f5d06385740 20 client.26312 trim_cache size 0 max 16384 2024-03-26T08:46:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 -- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ce80a2a00 msgr2=0x7f5ce80a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ce80a2a00 0x7f5ce80a4ec0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc1110c0 tx=0x7f5cec005f90 comp rx=0 tx=0).stop 2024-03-26T08:46:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 -- 172.21.15.7:0/4278869314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1017b0 msgr2=0x7f5cfc111350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1017b0 0x7f5cfc111350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009230 tx=0x7f5cf8004e30 comp rx=0 tx=0).stop 2024-03-26T08:46:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 -- 172.21.15.7:0/4278869314 shutdown_connections 2024-03-26T08:46:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5ce80a2a00 0x7f5ce80a4ec0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5cfc10f990 0x7f5cfc10fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cfc102170 0x7f5cfc111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 --2- 172.21.15.7:0/4278869314 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5cfc1017b0 0x7f5cfc111350 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 -- 172.21.15.7:0/4278869314 >> 172.21.15.7:0/4278869314 conn(0x7f5cfc0fbb30 msgr2=0x7f5cfc104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.486+0000 7f5d036c0640 1 -- 172.21.15.7:0/4278869314 shutdown_connections 2024-03-26T08:46:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.487+0000 7f5d036c0640 1 -- 172.21.15.7:0/4278869314 wait complete. 2024-03-26T08:46:37.496 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:46:37.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:46:37.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:37 smithi007 ceph-mon[32462]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 703 op/s 2024-03-26T08:46:37.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:46:37.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:46:37.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:37 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:46:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.677+0000 7f0e5bcac640 1 -- 172.21.15.7:0/2721728679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541017c0 msgr2=0x7f0e54101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.677+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/2721728679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541017c0 0x7f0e54101c40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009860 tx=0x7f0e4402bb70 comp rx=0 tx=0).stop 2024-03-26T08:46:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.677+0000 7f0e5bcac640 1 -- 172.21.15.7:0/2721728679 shutdown_connections 2024-03-26T08:46:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.677+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/2721728679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e54102180 0x7f0e54108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.677+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/2721728679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541017c0 0x7f0e54101c40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.677+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/2721728679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e541005d0 0x7f0e541009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.677+0000 7f0e5bcac640 1 -- 172.21.15.7:0/2721728679 >> 172.21.15.7:0/2721728679 conn(0x7f0e540fbb40 msgr2=0x7f0e540fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.677+0000 7f0e5bcac640 1 -- 172.21.15.7:0/2721728679 shutdown_connections 2024-03-26T08:46:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.677+0000 7f0e5bcac640 1 -- 172.21.15.7:0/2721728679 wait complete. 2024-03-26T08:46:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.678+0000 7f0e5bcac640 1 Processor -- start 2024-03-26T08:46:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.678+0000 7f0e5bcac640 1 -- start start 2024-03-26T08:46:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e5bcac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541005d0 0x7f0e54195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e5bcac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e541017c0 0x7f0e54195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e5bcac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102180 0x7f0e5419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e5bcac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5410b150 con 0x7f0e541017c0 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e5bcac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5410afd0 con 0x7f0e54102180 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e59a48640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541005d0 0x7f0e54195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e59a48640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541005d0 0x7f0e54195720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59414/0 (socket says 172.21.15.7:59414) 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e59a48640 1 -- 172.21.15.7:0/4106088640 learned_addr learned my addr 172.21.15.7:0/4106088640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e5a249640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102180 0x7f0e5419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.679+0000 7f0e59247640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e541017c0 0x7f0e54195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.680+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5410b2d0 con 0x7f0e541005d0 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.680+0000 7f0e59a48640 1 -- 172.21.15.7:0/4106088640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102180 msgr2=0x7f0e5419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.680+0000 7f0e59a48640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102180 0x7f0e5419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.680+0000 7f0e59a48640 1 -- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e541017c0 msgr2=0x7f0e54195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.680+0000 7f0e59a48640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e541017c0 0x7f0e54195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.680+0000 7f0e59a48640 1 -- 172.21.15.7:0/4106088640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5419d340 con 0x7f0e541005d0 2024-03-26T08:46:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.680+0000 7f0e59a48640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541005d0 0x7f0e54195720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0e50007830 tx=0x7f0e50007cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.680+0000 7f0e4affd640 1 -- 172.21.15.7:0/4106088640 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50019030 con 0x7f0e541005d0 2024-03-26T08:46:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.681+0000 7f0e4affd640 1 -- 172.21.15.7:0/4106088640 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0e5000f920 con 0x7f0e541005d0 2024-03-26T08:46:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.681+0000 7f0e4affd640 1 -- 172.21.15.7:0/4106088640 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50003ac0 con 0x7f0e541005d0 2024-03-26T08:46:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.681+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5419d630 con 0x7f0e541005d0 2024-03-26T08:46:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.681+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5419dba0 con 0x7f0e541005d0 2024-03-26T08:46:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.682+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e541078f0 con 0x7f0e541005d0 2024-03-26T08:46:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.685+0000 7f0e4affd640 1 -- 172.21.15.7:0/4106088640 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0e5001c070 con 0x7f0e541005d0 2024-03-26T08:46:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.686+0000 7f0e4affd640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e400a2900 0x7f0e400a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.686+0000 7f0e4affd640 1 -- 172.21.15.7:0/4106088640 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0e500ce750 con 0x7f0e541005d0 2024-03-26T08:46:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.686+0000 7f0e59247640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e400a2900 0x7f0e400a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.687+0000 7f0e59247640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e400a2900 0x7f0e400a4dc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009830 tx=0x7f0e44034ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:37.688+0000 7f0e4affd640 1 -- 172.21.15.7:0/4106088640 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e50023070 con 0x7f0e541005d0 2024-03-26T08:46:37.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:37 smithi016 ceph-mon[35851]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 703 op/s 2024-03-26T08:46:37.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:37 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:46:37.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:37 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:46:37.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:37 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:46:37.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:37 smithi044 ceph-mon[36461]: pgmap v486: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.1 MiB/s wr, 703 op/s 2024-03-26T08:46:37.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:37 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:46:37.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:37 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:46:37.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:37 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:46:38.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.105+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e5402d070 con 0x7f0e541005d0 2024-03-26T08:46:38.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.105+0000 7f0e4affd640 1 -- 172.21.15.7:0/4106088640 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f0e5009b7b0 con 0x7f0e541005d0 2024-03-26T08:46:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:46:38.097 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:46:38.097 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.108+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e400a2900 msgr2=0x7f0e400a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e400a2900 0x7f0e400a4dc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009830 tx=0x7f0e44034ab0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541005d0 msgr2=0x7f0e54195720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541005d0 0x7f0e54195720 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0e50007830 tx=0x7f0e50007cf0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 shutdown_connections 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0e400a2900 0x7f0e400a4dc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e54102180 0x7f0e5419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e541017c0 0x7f0e54195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 --2- 172.21.15.7:0/4106088640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0e541005d0 0x7f0e54195720 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 >> 172.21.15.7:0/4106088640 conn(0x7f0e540fbb40 msgr2=0x7f0e54104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 shutdown_connections 2024-03-26T08:46:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.109+0000 7f0e5bcac640 1 -- 172.21.15.7:0/4106088640 wait complete. 2024-03-26T08:46:38.117 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:46:38.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:46:38.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.335+0000 7f85b6cea640 1 -- 172.21.15.7:0/3917046345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b01018f0 msgr2=0x7f85b0108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.335+0000 7f85b6cea640 1 --2- 172.21.15.7:0/3917046345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b01018f0 0x7f85b0108180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f85a001a910 tx=0x7f85a003cc70 comp rx=0 tx=0).stop 2024-03-26T08:46:38.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.335+0000 7f85b6cea640 1 -- 172.21.15.7:0/3917046345 shutdown_connections 2024-03-26T08:46:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.335+0000 7f85b6cea640 1 --2- 172.21.15.7:0/3917046345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b01018f0 0x7f85b0108180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.335+0000 7f85b6cea640 1 --2- 172.21.15.7:0/3917046345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b0100f30 0x7f85b01013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.335+0000 7f85b6cea640 1 --2- 172.21.15.7:0/3917046345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b00ffd40 0x7f85b0100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.335+0000 7f85b6cea640 1 -- 172.21.15.7:0/3917046345 >> 172.21.15.7:0/3917046345 conn(0x7f85b00fb2b0 msgr2=0x7f85b00fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.336+0000 7f85b6cea640 1 -- 172.21.15.7:0/3917046345 shutdown_connections 2024-03-26T08:46:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.336+0000 7f85b6cea640 1 -- 172.21.15.7:0/3917046345 wait complete. 2024-03-26T08:46:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b6cea640 1 Processor -- start 2024-03-26T08:46:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b6cea640 1 -- start start 2024-03-26T08:46:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b6cea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b00ffd40 0x7f85b018faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b6cea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b0100f30 0x7f85b0192040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b6cea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b01018f0 0x7f85b0194590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b6cea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b006d490 con 0x7f85b00ffd40 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b6cea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b006d310 con 0x7f85b0100f30 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b6cea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b006d610 con 0x7f85b01018f0 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b4a86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b00ffd40 0x7f85b018faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b0100f30 0x7f85b0192040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b4a86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b00ffd40 0x7f85b018faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55480/0 (socket says 172.21.15.7:55480) 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b4a86640 1 -- 172.21.15.7:0/1672775532 learned_addr learned my addr 172.21.15.7:0/1672775532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.337+0000 7f85b5287640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b01018f0 0x7f85b0194590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85b4a86640 1 -- 172.21.15.7:0/1672775532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b01018f0 msgr2=0x7f85b0194590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85b4a86640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b01018f0 0x7f85b0194590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85b4a86640 1 -- 172.21.15.7:0/1672775532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b0100f30 msgr2=0x7f85b0192040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85b4a86640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b0100f30 0x7f85b0192040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85b4a86640 1 -- 172.21.15.7:0/1672775532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0194c00 con 0x7f85b00ffd40 2024-03-26T08:46:38.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85b4a86640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b00ffd40 0x7f85b018faf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f85a800b9c0 tx=0x7f85a800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85adffb640 1 -- 172.21.15.7:0/1672775532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a801d070 con 0x7f85b00ffd40 2024-03-26T08:46:38.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85adffb640 1 -- 172.21.15.7:0/1672775532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85a8018920 con 0x7f85b00ffd40 2024-03-26T08:46:38.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85b6cea640 1 -- 172.21.15.7:0/1672775532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b0194ef0 con 0x7f85b00ffd40 2024-03-26T08:46:38.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.338+0000 7f85adffb640 1 -- 172.21.15.7:0/1672775532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8016ac0 con 0x7f85b00ffd40 2024-03-26T08:46:38.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.339+0000 7f85b6cea640 1 -- 172.21.15.7:0/1672775532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b0195430 con 0x7f85b00ffd40 2024-03-26T08:46:38.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.343+0000 7f85adffb640 1 -- 172.21.15.7:0/1672775532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f85a8020070 con 0x7f85b00ffd40 2024-03-26T08:46:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.343+0000 7f85adffb640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85980a2860 0x7f85980a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.344+0000 7f85adffb640 1 -- 172.21.15.7:0/1672775532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85a80d2420 con 0x7f85b00ffd40 2024-03-26T08:46:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.344+0000 7f85b99b4740 1 Processor -- start 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.344+0000 7f85b99b4740 1 -- start start 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.344+0000 7f85affff640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85980a2860 0x7f85980a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.344+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83cc2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.344+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc3080 0x55ed83cc7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.344+0000 7f85b4a86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83cc2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.344+0000 7f85b4a86640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83cc2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55492/0 (socket says 172.21.15.7:55492) 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85b4a86640 1 -- 172.21.15.7:0/2633739048 learned_addr learned my addr 172.21.15.7:0/2633739048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85b99b4740 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc81f0 0x55ed83cca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85b99b4740 1 -- 172.21.15.7:0/2633739048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed83c85610 con 0x55ed83cc2690 2024-03-26T08:46:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85affff640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85980a2860 0x7f85980a4d20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f85b01080a0 tx=0x7f85a400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85b99b4740 1 -- 172.21.15.7:0/2633739048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed83c283c0 con 0x55ed83cc81f0 2024-03-26T08:46:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85b99b4740 1 -- 172.21.15.7:0/2633739048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed83bd0d60 con 0x55ed83cc3080 2024-03-26T08:46:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85affff640 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc3080 0x55ed83cc7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85b5287640 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc81f0 0x55ed83cca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85affff640 1 -- 172.21.15.7:0/2633739048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc81f0 msgr2=0x55ed83cca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85affff640 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc81f0 0x55ed83cca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85affff640 1 -- 172.21.15.7:0/2633739048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 msgr2=0x55ed83cc2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85affff640 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83cc2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85affff640 1 -- 172.21.15.7:0/2633739048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed8393c3f0 con 0x55ed83cc3080 2024-03-26T08:46:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.345+0000 7f85affff640 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc3080 0x55ed83cc7b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85a400b9c0 tx=0x7f85a4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85a14fa1ac62fdcc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.346+0000 7f8596ffd640 1 -- 172.21.15.7:0/2633739048 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4005220 con 0x55ed83cc3080 2024-03-26T08:46:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.346+0000 7f8596ffd640 1 -- 172.21.15.7:0/2633739048 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85a4011e50 con 0x55ed83cc3080 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 -- 172.21.15.7:0/2633739048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc3080 msgr2=0x55ed83cc7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc3080 0x55ed83cc7b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85a400b9c0 tx=0x7f85a4013000 comp rx=0 tx=0).stop 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 -- 172.21.15.7:0/2633739048 shutdown_connections 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc81f0 0x55ed83cca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc3080 0x55ed83cc7b80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 --2- 172.21.15.7:0/2633739048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83cc2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 -- 172.21.15.7:0/2633739048 >> 172.21.15.7:0/2633739048 conn(0x55ed83cbdc00 msgr2=0x55ed83cbe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 -- 172.21.15.7:0/2633739048 shutdown_connections 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 -- 172.21.15.7:0/2633739048 wait complete. 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.347+0000 7f85b99b4740 1 -- start start 2024-03-26T08:46:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.348+0000 7f85b99b4740 1 Processor -- start 2024-03-26T08:46:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.349+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83ccc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b99b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed83be4240 con 0x55ed83cc2690 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b99b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed83c24460 con 0x55ed83cc3080 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b99b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed83ccb1f0 con 0x55ed83cc81f0 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b99b4740 10 client.0 _ll_register_callbacks cb 0x55ed83be3c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b5287640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b5287640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59452/0 (socket says 172.21.15.7:59452) 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b5287640 1 -- 172.21.15.7:0/779978036 learned_addr learned my addr 172.21.15.7:0/779978036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85affff640 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.350+0000 7f85b4a86640 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83ccc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f85b5287640 1 -- 172.21.15.7:0/779978036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 msgr2=0x55ed83cccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f85b5287640 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f85b5287640 1 -- 172.21.15.7:0/779978036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 msgr2=0x55ed83ccc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f85b5287640 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83ccc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f85b5287640 1 -- 172.21.15.7:0/779978036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed83a067d0 con 0x55ed83cc81f0 2024-03-26T08:46:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f85affff640 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:46:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f85b5287640 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd3f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f85a001a410 tx=0x7f85a00053f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b8dc2d081ab5faf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f858ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f858ffff640 1 -- 172.21.15.7:0/779978036 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0054070 con 0x55ed83cc81f0 2024-03-26T08:46:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f85b99b4740 1 -- 172.21.15.7:0/779978036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85a4011e50 con 0x55ed83cc81f0 2024-03-26T08:46:38.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.351+0000 7f858ffff640 1 -- 172.21.15.7:0/779978036 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85a0019030 con 0x55ed83cc81f0 2024-03-26T08:46:38.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.352+0000 7f858ffff640 1 -- 172.21.15.7:0/779978036 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85a0005020 con 0x55ed83cc81f0 2024-03-26T08:46:38.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.352+0000 7f85b99b4740 10 client.26359 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.352+0000 7f85b99b4740 1 -- 172.21.15.7:0/779978036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ed83cd7310 con 0x55ed83cc81f0 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.352+0000 7f858ffff640 1 -- 172.21.15.7:0/779978036 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85a4011e50 con 0x55ed83cc81f0 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.352+0000 7f858ffff640 1 -- 172.21.15.7:0/779978036 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f85a00036f0 con 0x55ed83cc81f0 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.352+0000 7f85b99b4740 10 client.26359 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.352+0000 7f85b99b4740 10 client.26359 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.353+0000 7f85b99b4740 20 client.26359 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.353+0000 7f85b99b4740 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ed83cd8a30 0x55ed83cdae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.353+0000 7f85b99b4740 4 client.26359 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.353+0000 7f85b99b4740 1 -- 172.21.15.7:0/779978036 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed83c35900 con 0x55ed83cd8a30 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.353+0000 7f85b4a86640 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ed83cd8a30 0x55ed83cdae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.354+0000 7f858ffff640 10 client.26359 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.354+0000 7f85b4a86640 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ed83cd8a30 0x55ed83cdae20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aee666d6753c3e19 server_cookie=93f1c6107c401591 in_seq=0 out_seq=0 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.354+0000 7f858ffff640 1 -- 172.21.15.7:0/779978036 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ed83c35900 con 0x55ed83cd8a30 2024-03-26T08:46:38.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.354+0000 7f858ffff640 10 client.26359 handle_command_reply: tid=0 2024-03-26T08:46:38.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.354+0000 7f85b99b4740 1 client.26359 shutdown 2024-03-26T08:46:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.356+0000 7f85b99b4740 1 -- 172.21.15.7:0/779978036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 msgr2=0x55ed83cd3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.356+0000 7f85b99b4740 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd3f30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f85a001a410 tx=0x7f85a00053f0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f85b99b4740 1 -- 172.21.15.7:0/779978036 shutdown_connections 2024-03-26T08:46:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f85b99b4740 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ed83cd8a30 0x55ed83cdae20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f85b99b4740 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd3f30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f85b99b4740 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f85b99b4740 1 --2- 172.21.15.7:0/779978036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83ccc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f858ffff640 0 client.26359 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:46:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f85b99b4740 1 -- 172.21.15.7:0/779978036 >> 172.21.15.7:0/779978036 conn(0x55ed83cbdc00 msgr2=0x55ed83cbce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f85b99b4740 1 -- 172.21.15.7:0/779978036 shutdown_connections 2024-03-26T08:46:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f85b99b4740 1 -- 172.21.15.7:0/779978036 wait complete. 2024-03-26T08:46:38.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.357+0000 7f85b99b4740 20 client.26359 trim_cache size 0 max 16384 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.379+0000 7f85b99b4740 1 Processor -- start 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.379+0000 7f85b99b4740 1 -- start start 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83cc44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cd3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cc49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85b99b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b01a06e0 con 0x55ed83cc2690 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85b99b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b019e1e0 con 0x55ed83cc3080 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85b99b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b019c260 con 0x55ed83cc81f0 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cd3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cd3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50756/0 (socket says 172.21.15.7:50756) 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85affff640 1 -- 172.21.15.7:0/1900889577 learned_addr learned my addr 172.21.15.7:0/1900889577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85b5287640 1 --2- 172.21.15.7:0/1900889577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cc49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85affff640 1 -- 172.21.15.7:0/1900889577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 msgr2=0x55ed83cc49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85affff640 1 --2- 172.21.15.7:0/1900889577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cc49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85affff640 1 -- 172.21.15.7:0/1900889577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 msgr2=0x55ed83cc44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85affff640 1 --2- 172.21.15.7:0/1900889577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83cc44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.380+0000 7f85affff640 1 -- 172.21.15.7:0/1900889577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed83cc7700 con 0x55ed83cc3080 2024-03-26T08:46:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.381+0000 7f85affff640 1 --2- 172.21.15.7:0/1900889577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cd3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f85a400b9c0 tx=0x7f85a401e4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af92c4b2db17b96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.381+0000 7f85967fc640 1 -- 172.21.15.7:0/1900889577 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a401eb80 con 0x55ed83cc3080 2024-03-26T08:46:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.381+0000 7f85967fc640 1 -- 172.21.15.7:0/1900889577 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85a401ed20 con 0x55ed83cc3080 2024-03-26T08:46:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.381+0000 7f85967fc640 1 -- 172.21.15.7:0/1900889577 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4004c80 con 0x55ed83cc3080 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.382+0000 7f85b99b4740 1 -- 172.21.15.7:0/1900889577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 msgr2=0x55ed83cd3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.382+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1900889577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cd3ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f85a400b9c0 tx=0x7f85a401e4a0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.382+0000 7f85b99b4740 1 -- 172.21.15.7:0/1900889577 shutdown_connections 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.382+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1900889577 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cc49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.382+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1900889577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc3080 0x55ed83cd3ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.382+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1900889577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc2690 0x55ed83cc44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.382+0000 7f85b99b4740 1 -- 172.21.15.7:0/1900889577 >> 172.21.15.7:0/1900889577 conn(0x55ed83cbdc00 msgr2=0x55ed83cbdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.382+0000 7f85b99b4740 1 -- 172.21.15.7:0/1900889577 shutdown_connections 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.382+0000 7f85b99b4740 1 -- 172.21.15.7:0/1900889577 wait complete. 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.383+0000 7f85b99b4740 1 -- start start 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.383+0000 7f85b99b4740 1 Processor -- start 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc2690 0x55ed83cd19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc3080 0x55ed83cd1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b99b4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b99b4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b019c860 con 0x55ed83cc3080 2024-03-26T08:46:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b99b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b019db80 con 0x55ed83cc2690 2024-03-26T08:46:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b99b4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b019cec0 con 0x55ed83cc81f0 2024-03-26T08:46:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b99b4740 10 client.0 _ll_register_callbacks cb 0x7f85a001a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b5287640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b5287640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59472/0 (socket says 172.21.15.7:59472) 2024-03-26T08:46:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b5287640 1 -- 172.21.15.7:0/1519969412 learned_addr learned my addr 172.21.15.7:0/1519969412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.384+0000 7f85b4a86640 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc2690 0x55ed83cd19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.385+0000 7f85affff640 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc3080 0x55ed83cd1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.385+0000 7f85b5287640 1 -- 172.21.15.7:0/1519969412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc2690 msgr2=0x55ed83cd19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.385+0000 7f85b5287640 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc2690 0x55ed83cd19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.385+0000 7f85b5287640 1 -- 172.21.15.7:0/1519969412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc3080 msgr2=0x55ed83cd1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.385+0000 7f85b5287640 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc3080 0x55ed83cd1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.385+0000 7f85b5287640 1 -- 172.21.15.7:0/1519969412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed83cd3520 con 0x55ed83cc81f0 2024-03-26T08:46:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.385+0000 7f85b5287640 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd2f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85a001a860 tx=0x7f85a0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602a3104e34923af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.385+0000 7f858ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.386+0000 7f858ffff640 1 -- 172.21.15.7:0/1519969412 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0054040 con 0x55ed83cc81f0 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.386+0000 7f85b99b4740 1 -- 172.21.15.7:0/1519969412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85a401ed20 con 0x55ed83cc81f0 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.386+0000 7f858ffff640 1 -- 172.21.15.7:0/1519969412 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85a00037f0 con 0x55ed83cc81f0 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.386+0000 7f858ffff640 1 -- 172.21.15.7:0/1519969412 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85a003d820 con 0x55ed83cc81f0 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.386+0000 7f85b99b4740 10 client.26365 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.386+0000 7f85b99b4740 1 -- 172.21.15.7:0/1519969412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ed83d3ce70 con 0x55ed83cc81f0 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.387+0000 7f858ffff640 1 -- 172.21.15.7:0/1519969412 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f85a401ed20 con 0x55ed83cc81f0 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.387+0000 7f858ffff640 1 -- 172.21.15.7:0/1519969412 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f85a0045440 con 0x55ed83cc81f0 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.387+0000 7f85b99b4740 10 client.26365 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.387+0000 7f85b99b4740 10 client.26365 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.387+0000 7f85b99b4740 20 client.26365 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.387+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ed83cd8a30 0x55ed83d3f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.387+0000 7f85b99b4740 4 client.26365 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:46:38.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.387+0000 7f85b99b4740 1 -- 172.21.15.7:0/1519969412 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed83c2fc50 con 0x55ed83cd8a30 2024-03-26T08:46:38.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.387+0000 7f85b4a86640 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ed83cd8a30 0x55ed83d3f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.388+0000 7f858ffff640 10 client.26365 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:46:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.388+0000 7f85b4a86640 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ed83cd8a30 0x55ed83d3f7d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d012feddde100fa server_cookie=fe04ca0360c60635 in_seq=0 out_seq=0 2024-03-26T08:46:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.388+0000 7f858ffff640 1 -- 172.21.15.7:0/1519969412 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed83c2fc50 con 0x55ed83cd8a30 2024-03-26T08:46:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.388+0000 7f858ffff640 10 client.26365 handle_command_reply: tid=0 2024-03-26T08:46:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.388+0000 7f85b99b4740 1 client.26365 shutdown 2024-03-26T08:46:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.389+0000 7f85b99b4740 1 -- 172.21.15.7:0/1519969412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 msgr2=0x55ed83cd2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.389+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd2f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85a001a860 tx=0x7f85a0005b50 comp rx=0 tx=0).stop 2024-03-26T08:46:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.389+0000 7f85b99b4740 1 -- 172.21.15.7:0/1519969412 shutdown_connections 2024-03-26T08:46:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.389+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55ed83cd8a30 0x55ed83d3f7d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.389+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ed83cc81f0 0x55ed83cd2f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.389+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed83cc3080 0x55ed83cd1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.389+0000 7f85b99b4740 1 --2- 172.21.15.7:0/1519969412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed83cc2690 0x55ed83cd19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.389+0000 7f858ffff640 0 client.26365 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:46:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.389+0000 7f85b99b4740 1 -- 172.21.15.7:0/1519969412 >> 172.21.15.7:0/1519969412 conn(0x55ed83cbdc00 msgr2=0x55ed83cc44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.390+0000 7f85b99b4740 1 -- 172.21.15.7:0/1519969412 shutdown_connections 2024-03-26T08:46:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.390+0000 7f85b99b4740 1 -- 172.21.15.7:0/1519969412 wait complete. 2024-03-26T08:46:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.390+0000 7f85b99b4740 20 client.26365 trim_cache size 0 max 16384 2024-03-26T08:46:38.383 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:46:38.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 -- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85980a2860 msgr2=0x7f85980a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85980a2860 0x7f85980a4d20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f85b01080a0 tx=0x7f85a400b390 comp rx=0 tx=0).stop 2024-03-26T08:46:38.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 -- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b00ffd40 msgr2=0x7f85b018faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b00ffd40 0x7f85b018faf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f85a800b9c0 tx=0x7f85a800be80 comp rx=0 tx=0).stop 2024-03-26T08:46:38.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 -- 172.21.15.7:0/1672775532 shutdown_connections 2024-03-26T08:46:38.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f85980a2860 0x7f85980a4d20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f85b01018f0 0x7f85b0194590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85b0100f30 0x7f85b0192040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 --2- 172.21.15.7:0/1672775532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85b00ffd40 0x7f85b018faf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 -- 172.21.15.7:0/1672775532 >> 172.21.15.7:0/1672775532 conn(0x7f85b00fb2b0 msgr2=0x7f85b006ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 -- 172.21.15.7:0/1672775532 shutdown_connections 2024-03-26T08:46:38.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.391+0000 7f85b6cea640 1 -- 172.21.15.7:0/1672775532 wait complete. 2024-03-26T08:46:38.399 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:46:38.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:46:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.582+0000 7fdf78e24640 1 -- 172.21.15.7:0/2465089539 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70032340 con 0x7fdf74102170 2024-03-26T08:46:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.583+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2465089539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74102170 msgr2=0x7fdf74108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.583+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2465089539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74102170 0x7fdf74108a00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009860 tx=0x7fdf7002bb90 comp rx=0 tx=0).stop 2024-03-26T08:46:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.583+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2465089539 shutdown_connections 2024-03-26T08:46:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.583+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2465089539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf74102170 0x7fdf74108a00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.583+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2465089539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf741017b0 0x7fdf74101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.583+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2465089539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf741005c0 0x7fdf741009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.583+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2465089539 >> 172.21.15.7:0/2465089539 conn(0x7fdf740fbb30 msgr2=0x7fdf740fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.583+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2465089539 shutdown_connections 2024-03-26T08:46:38.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.584+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2465089539 wait complete. 2024-03-26T08:46:38.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.584+0000 7fdf7c08a640 1 Processor -- start 2024-03-26T08:46:38.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.584+0000 7fdf7c08a640 1 -- start start 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf7c08a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf741005c0 0x7fdf740694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf7c08a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf741017b0 0x7fdf7419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf7c08a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf74102170 0x7fdf741a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf7c08a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7410c310 con 0x7fdf741017b0 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf7c08a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7410c190 con 0x7fdf74102170 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf7c08a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7410c490 con 0x7fdf741005c0 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf79625640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf741017b0 0x7fdf7419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf79625640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf741017b0 0x7fdf7419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55546/0 (socket says 172.21.15.7:55546) 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf79625640 1 -- 172.21.15.7:0/2380316117 learned_addr learned my addr 172.21.15.7:0/2380316117 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf79e26640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf741005c0 0x7fdf740694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf7a627640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf74102170 0x7fdf741a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf79625640 1 -- 172.21.15.7:0/2380316117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf741005c0 msgr2=0x7fdf740694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf79625640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf741005c0 0x7fdf740694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf79625640 1 -- 172.21.15.7:0/2380316117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf74102170 msgr2=0x7fdf741a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf79625640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf74102170 0x7fdf741a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.585+0000 7fdf79625640 1 -- 172.21.15.7:0/2380316117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74198650 con 0x7fdf741017b0 2024-03-26T08:46:38.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.586+0000 7fdf79625640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf741017b0 0x7fdf7419ea70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400e9f0 tx=0x7fdf6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.586+0000 7fdf6affd640 1 -- 172.21.15.7:0/2380316117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6400cd80 con 0x7fdf741017b0 2024-03-26T08:46:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.586+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2380316117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf741988e0 con 0x7fdf741017b0 2024-03-26T08:46:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.586+0000 7fdf6affd640 1 -- 172.21.15.7:0/2380316117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf64010910 con 0x7fdf741017b0 2024-03-26T08:46:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.586+0000 7fdf6affd640 1 -- 172.21.15.7:0/2380316117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64018990 con 0x7fdf741017b0 2024-03-26T08:46:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.586+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2380316117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf74198e80 con 0x7fdf741017b0 2024-03-26T08:46:38.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.589+0000 7fdf7ec70740 1 Processor -- start 2024-03-26T08:46:38.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7ec70740 1 -- start start 2024-03-26T08:46:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de27ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de28080 0x56258de2cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de2f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7ec70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258ddea610 con 0x56258de28080 2024-03-26T08:46:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7ec70740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258dd8d3c0 con 0x56258de2d1f0 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7ec70740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258dd35d60 con 0x56258de27690 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7a627640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de27ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7a627640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de27ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59506/0 (socket says 172.21.15.7:59506) 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.590+0000 7fdf7a627640 1 -- 172.21.15.7:0/2246746572 learned_addr learned my addr 172.21.15.7:0/2246746572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.591+0000 7fdf7a627640 1 -- 172.21.15.7:0/2246746572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 msgr2=0x56258de2f680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.591+0000 7fdf7a627640 1 --2- 172.21.15.7:0/2246746572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de2f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.591+0000 7fdf7a627640 1 -- 172.21.15.7:0/2246746572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de28080 msgr2=0x56258de2cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.591+0000 7fdf7a627640 1 --2- 172.21.15.7:0/2246746572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de28080 0x56258de2cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.591+0000 7fdf7a627640 1 -- 172.21.15.7:0/2246746572 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56258daa13f0 con 0x56258de27690 2024-03-26T08:46:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.591+0000 7fdf6affd640 1 -- 172.21.15.7:0/2380316117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fdf64010ac0 con 0x7fdf741017b0 2024-03-26T08:46:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.592+0000 7fdf6affd640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdf600a2930 0x7fdf600a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.592+0000 7fdf7a627640 1 --2- 172.21.15.7:0/2246746572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de27ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf70034b10 tx=0x7fdf70005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d6cf61c8388556 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.592+0000 7fdf6affd640 1 -- 172.21.15.7:0/2380316117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdf640ced10 con 0x7fdf741017b0 2024-03-26T08:46:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.592+0000 7fdf5ffff640 1 -- 172.21.15.7:0/2246746572 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70043020 con 0x56258de27690 2024-03-26T08:46:38.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.593+0000 7fdf5ffff640 1 -- 172.21.15.7:0/2246746572 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf70034db0 con 0x56258de27690 2024-03-26T08:46:38.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.593+0000 7fdf5ffff640 1 -- 172.21.15.7:0/2246746572 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70039040 con 0x56258de27690 2024-03-26T08:46:38.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.593+0000 7fdf79e26640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdf600a2930 0x7fdf600a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.594+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2246746572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 msgr2=0x56258de27ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.594+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2246746572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de27ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf70034b10 tx=0x7fdf70005b20 comp rx=0 tx=0).stop 2024-03-26T08:46:38.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.594+0000 7fdf79e26640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdf600a2930 0x7fdf600a4df0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009bc0 tx=0x7fdf6c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.594+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2246746572 shutdown_connections 2024-03-26T08:46:38.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.594+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2246746572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de2f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.595+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2246746572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de28080 0x56258de2cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.595+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2246746572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de27ab0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.595+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2246746572 >> 172.21.15.7:0/2246746572 conn(0x56258de22c00 msgr2=0x56258de23000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.595+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2246746572 shutdown_connections 2024-03-26T08:46:38.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.595+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2246746572 wait complete. 2024-03-26T08:46:38.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.596+0000 7fdf7ec70740 1 -- start start 2024-03-26T08:46:38.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.596+0000 7fdf7ec70740 1 Processor -- start 2024-03-26T08:46:38.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.597+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de42a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.597+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de28080 0x56258de42fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.597+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de4a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.597+0000 7fdf7ec70740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258dd49240 con 0x56258de27690 2024-03-26T08:46:38.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.597+0000 7fdf7ec70740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258dd89460 con 0x56258de2d1f0 2024-03-26T08:46:38.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.597+0000 7fdf7ec70740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258de41150 con 0x56258de28080 2024-03-26T08:46:38.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.597+0000 7fdf7ec70740 10 client.0 _ll_register_callbacks cb 0x56258dd48c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:38.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.597+0000 7fdf7a627640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de4a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.597+0000 7fdf79e26640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de42a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf79e26640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de42a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55568/0 (socket says 172.21.15.7:55568) 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf7a627640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de4a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50800/0 (socket says 172.21.15.7:50800) 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf79e26640 1 -- 172.21.15.7:0/2398445228 learned_addr learned my addr 172.21.15.7:0/2398445228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf79625640 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de28080 0x56258de42fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf7a627640 1 -- 172.21.15.7:0/2398445228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de28080 msgr2=0x56258de42fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf7a627640 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de28080 0x56258de42fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf7a627640 1 -- 172.21.15.7:0/2398445228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 msgr2=0x56258de42a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf7a627640 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de42a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf7a627640 1 -- 172.21.15.7:0/2398445228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56258db6b7d0 con 0x56258de2d1f0 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf7a627640 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de4a010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf700349e0 tx=0x7fdf700044a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca742a61f763fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.598+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2398445228 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70039040 con 0x56258de2d1f0 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.599+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2398445228 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdf7002c610 con 0x56258de2d1f0 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.599+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2398445228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf70034db0 con 0x56258de2d1f0 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.599+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2398445228 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf7002c810 con 0x56258de2d1f0 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.600+0000 7fdf7ec70740 10 client.26345 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.600+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2398445228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56258de4d380 con 0x56258de2d1f0 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.600+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2398445228 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf70034db0 con 0x56258de2d1f0 2024-03-26T08:46:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.601+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2398445228 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdf700035f0 con 0x56258de2d1f0 2024-03-26T08:46:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.601+0000 7fdf7ec70740 10 client.26345 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.601+0000 7fdf7ec70740 10 client.26345 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:46:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.601+0000 7fdf7ec70740 20 client.26345 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.601+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56258de4eb50 0x56258de50f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.601+0000 7fdf7ec70740 4 client.26345 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.601+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2398445228 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56258dd9a900 con 0x56258de4eb50 2024-03-26T08:46:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.601+0000 7fdf79e26640 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56258de4eb50 0x56258de50f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.602+0000 7fdf79e26640 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56258de4eb50 0x56258de50f60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41592ce885c33e56 server_cookie=54a3084ae4602418 in_seq=0 out_seq=0 2024-03-26T08:46:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.602+0000 7fdf5d7fa640 10 client.26345 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:46:38.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.603+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2398445228 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56258dd9a900 con 0x56258de4eb50 2024-03-26T08:46:38.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.603+0000 7fdf5d7fa640 10 client.26345 handle_command_reply: tid=0 2024-03-26T08:46:38.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.603+0000 7fdf7ec70740 1 client.26345 shutdown 2024-03-26T08:46:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.604+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2398445228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 msgr2=0x56258de4a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.604+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de4a010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf700349e0 tx=0x7fdf700044a0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.604+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2398445228 shutdown_connections 2024-03-26T08:46:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.604+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de4a010 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.604+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56258de4eb50 0x56258de50f60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.604+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de28080 0x56258de42fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.604+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2398445228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de42a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.604+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2398445228 >> 172.21.15.7:0/2398445228 conn(0x56258de22c00 msgr2=0x56258de21e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.605+0000 7fdf5d7fa640 0 client.26345 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:46:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.605+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2398445228 shutdown_connections 2024-03-26T08:46:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.605+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2398445228 wait complete. 2024-03-26T08:46:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.605+0000 7fdf7ec70740 20 client.26345 trim_cache size 0 max 16384 2024-03-26T08:46:38.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.628+0000 7fdf7ec70740 1 Processor -- start 2024-03-26T08:46:38.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.628+0000 7fdf7ec70740 1 -- start start 2024-03-26T08:46:38.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de1dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de28080 0x56258de49fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf79e26640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de1dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf79e26640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de1dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59534/0 (socket says 172.21.15.7:59534) 2024-03-26T08:46:38.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf79e26640 1 -- 172.21.15.7:0/4205677913 learned_addr learned my addr 172.21.15.7:0/4205677913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf7ec70740 1 -- 172.21.15.7:0/4205677913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74072750 con 0x56258de28080 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf7ec70740 1 -- 172.21.15.7:0/4205677913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74070250 con 0x56258de2d1f0 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf79625640 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de28080 0x56258de49fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.629+0000 7fdf7a627640 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf7ec70740 1 -- 172.21.15.7:0/4205677913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7406e2d0 con 0x56258de27690 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf79e26640 1 -- 172.21.15.7:0/4205677913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 msgr2=0x56258de414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf79e26640 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf79e26640 1 -- 172.21.15.7:0/4205677913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de28080 msgr2=0x56258de49fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf79e26640 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de28080 0x56258de49fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf79e26640 1 -- 172.21.15.7:0/4205677913 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56258de1e480 con 0x56258de27690 2024-03-26T08:46:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf79e26640 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de1dcb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00d7f0 tx=0x7fdf6c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5154d0bbe955e45e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf5f7fe640 1 -- 172.21.15.7:0/4205677913 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c00de70 con 0x56258de27690 2024-03-26T08:46:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf5f7fe640 1 -- 172.21.15.7:0/4205677913 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf6c019df0 con 0x56258de27690 2024-03-26T08:46:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.630+0000 7fdf5f7fe640 1 -- 172.21.15.7:0/4205677913 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c027e20 con 0x56258de27690 2024-03-26T08:46:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.631+0000 7fdf7ec70740 1 -- 172.21.15.7:0/4205677913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 msgr2=0x56258de1dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.631+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de1dcb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00d7f0 tx=0x7fdf6c014000 comp rx=0 tx=0).stop 2024-03-26T08:46:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.631+0000 7fdf7ec70740 1 -- 172.21.15.7:0/4205677913 shutdown_connections 2024-03-26T08:46:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.632+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de2d1f0 0x56258de414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.632+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de28080 0x56258de49fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.632+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/4205677913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de27690 0x56258de1dcb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.632+0000 7fdf7ec70740 1 -- 172.21.15.7:0/4205677913 >> 172.21.15.7:0/4205677913 conn(0x56258de22c00 msgr2=0x56258de42a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.632+0000 7fdf7ec70740 1 -- 172.21.15.7:0/4205677913 shutdown_connections 2024-03-26T08:46:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.632+0000 7fdf7ec70740 1 -- 172.21.15.7:0/4205677913 wait complete. 2024-03-26T08:46:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.633+0000 7fdf7ec70740 1 -- start start 2024-03-26T08:46:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.633+0000 7fdf7ec70740 1 Processor -- start 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.634+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.634+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de28080 0x56258de47610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf7ec70740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de2d1f0 0x56258de485b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79e26640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79625640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de28080 0x56258de47610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79e26640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55582/0 (socket says 172.21.15.7:55582) 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79625640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de28080 0x56258de47610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50822/0 (socket says 172.21.15.7:50822) 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79e26640 1 -- 172.21.15.7:0/2505727071 learned_addr learned my addr 172.21.15.7:0/2505727071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf7a627640 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de2d1f0 0x56258de485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7406e8d0 con 0x56258de27690 2024-03-26T08:46:38.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7406fbf0 con 0x56258de28080 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7406ef30 con 0x56258de2d1f0 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf7ec70740 10 client.0 _ll_register_callbacks cb 0x7fdf70005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79625640 1 -- 172.21.15.7:0/2505727071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de2d1f0 msgr2=0x56258de485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79625640 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de2d1f0 0x56258de485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79625640 1 -- 172.21.15.7:0/2505727071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 msgr2=0x56258de470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79625640 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79625640 1 -- 172.21.15.7:0/2505727071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56258de48ba0 con 0x56258de28080 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.635+0000 7fdf79e26640 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.636+0000 7fdf79625640 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de28080 0x56258de47610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56258de43f70 tx=0x7fdf6409b140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bda9dce4bf4628b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.636+0000 7fdf5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.636+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2505727071 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640bb040 con 0x56258de28080 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.636+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2505727071 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdf64018b30 con 0x56258de28080 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.636+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf6c019df0 con 0x56258de28080 2024-03-26T08:46:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.636+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2505727071 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf64018d30 con 0x56258de28080 2024-03-26T08:46:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.636+0000 7fdf7ec70740 10 client.26351 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.636+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56258de49ed0 con 0x56258de28080 2024-03-26T08:46:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.637+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2505727071 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf640b3030 con 0x56258de28080 2024-03-26T08:46:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.637+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2505727071 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdf6409ad00 con 0x56258de28080 2024-03-26T08:46:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.637+0000 7fdf7ec70740 10 client.26351 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.637+0000 7fdf7ec70740 10 client.26351 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.637+0000 7fdf7ec70740 20 client.26351 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.637+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56258deaa6b0 0x56258deacad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.637+0000 7fdf7ec70740 4 client.26351 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.637+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56258dd94c50 con 0x56258deaa6b0 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.638+0000 7fdf79e26640 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56258deaa6b0 0x56258deacad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.638+0000 7fdf79e26640 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56258deaa6b0 0x56258deacad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=247c916e4a6e7fe server_cookie=d6c47ed921b92283 in_seq=0 out_seq=0 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.638+0000 7fdf5d7fa640 10 client.26351 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf5d7fa640 1 -- 172.21.15.7:0/2505727071 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56258dd94c50 con 0x56258deaa6b0 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf5d7fa640 10 client.26351 handle_command_reply: tid=0 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 client.26351 shutdown 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de28080 msgr2=0x56258de47610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de28080 0x56258de47610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56258de43f70 tx=0x7fdf6409b140 comp rx=0 tx=0).stop 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 shutdown_connections 2024-03-26T08:46:38.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56258deaa6b0 0x56258deacad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258de2d1f0 0x56258de485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258de28080 0x56258de47610 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 --2- 172.21.15.7:0/2505727071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258de27690 0x56258de470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf5d7fa640 0 client.26351 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:46:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 >> 172.21.15.7:0/2505727071 conn(0x56258de22c00 msgr2=0x56258de414f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 shutdown_connections 2024-03-26T08:46:38.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.639+0000 7fdf7ec70740 1 -- 172.21.15.7:0/2505727071 wait complete. 2024-03-26T08:46:38.633 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:46:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.640+0000 7fdf7ec70740 20 client.26351 trim_cache size 0 max 16384 2024-03-26T08:46:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.641+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdf600a2930 msgr2=0x7fdf600a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.641+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdf600a2930 0x7fdf600a4df0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009bc0 tx=0x7fdf6c010040 comp rx=0 tx=0).stop 2024-03-26T08:46:38.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.641+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf741017b0 msgr2=0x7fdf7419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.641+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf741017b0 0x7fdf7419ea70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400e9f0 tx=0x7fdf6400eeb0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.642+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2380316117 shutdown_connections 2024-03-26T08:46:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.642+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdf600a2930 0x7fdf600a4df0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.642+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf74102170 0x7fdf741a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.642+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf741017b0 0x7fdf7419ea70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.642+0000 7fdf7c08a640 1 --2- 172.21.15.7:0/2380316117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf741005c0 0x7fdf740694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.642+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2380316117 >> 172.21.15.7:0/2380316117 conn(0x7fdf740fbb30 msgr2=0x7fdf74104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.642+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2380316117 shutdown_connections 2024-03-26T08:46:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.642+0000 7fdf7c08a640 1 -- 172.21.15.7:0/2380316117 wait complete. 2024-03-26T08:46:38.653 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:38 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4106088640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:46:38.654 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:46:38.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:46:38.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:38 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4106088640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:46:38.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:38 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4106088640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:46:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.837+0000 7efff4e34640 1 -- 172.21.15.7:0/3543558086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff0105470 msgr2=0x7efff010bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.837+0000 7efff4e34640 1 --2- 172.21.15.7:0/3543558086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff0105470 0x7efff010bd00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7effe8009ca0 tx=0x7effe802c100 comp rx=0 tx=0).stop 2024-03-26T08:46:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.837+0000 7efff4e34640 1 -- 172.21.15.7:0/3543558086 shutdown_connections 2024-03-26T08:46:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.838+0000 7efff4e34640 1 --2- 172.21.15.7:0/3543558086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff0105470 0x7efff010bd00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.838+0000 7efff4e34640 1 --2- 172.21.15.7:0/3543558086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff0104ab0 0x7efff0104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.838+0000 7efff4e34640 1 --2- 172.21.15.7:0/3543558086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff01038c0 0x7efff0103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.838+0000 7efff4e34640 1 -- 172.21.15.7:0/3543558086 >> 172.21.15.7:0/3543558086 conn(0x7efff00fee50 msgr2=0x7efff0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.838+0000 7efff4e34640 1 -- 172.21.15.7:0/3543558086 shutdown_connections 2024-03-26T08:46:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.838+0000 7efff4e34640 1 -- 172.21.15.7:0/3543558086 wait complete. 2024-03-26T08:46:38.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.839+0000 7efff4e34640 1 Processor -- start 2024-03-26T08:46:38.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.840+0000 7efff4e34640 1 -- start start 2024-03-26T08:46:38.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7efff4e34640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff01038c0 0x7efff019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7efff4e34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff0104ab0 0x7efff019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7efff4e34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff0105470 0x7efff01a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7efff4e34640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff010f540 con 0x7efff0104ab0 2024-03-26T08:46:38.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7efff4e34640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff010f3c0 con 0x7efff0105470 2024-03-26T08:46:38.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7efff4e34640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff010f6c0 con 0x7efff01038c0 2024-03-26T08:46:38.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7effee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff01038c0 0x7efff019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7effeed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff0105470 0x7efff01a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7effee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff01038c0 0x7efff019dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59568/0 (socket says 172.21.15.7:59568) 2024-03-26T08:46:38.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7effee59c640 1 -- 172.21.15.7:0/4197826803 learned_addr learned my addr 172.21.15.7:0/4197826803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7effee59c640 1 -- 172.21.15.7:0/4197826803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff0105470 msgr2=0x7efff01a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7effee59c640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff0105470 0x7efff01a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7effee59c640 1 -- 172.21.15.7:0/4197826803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff0104ab0 msgr2=0x7efff019e4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:46:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7effee59c640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff0104ab0 0x7efff019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.841+0000 7effee59c640 1 -- 172.21.15.7:0/4197826803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff01a5bd0 con 0x7efff01038c0 2024-03-26T08:46:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.842+0000 7effeed9d640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff0105470 0x7efff01a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.842+0000 7effee59c640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff01038c0 0x7efff019dfb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7effe400e9c0 tx=0x7effe400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.842+0000 7effdf7fe640 1 -- 172.21.15.7:0/4197826803 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe400ccc0 con 0x7efff01038c0 2024-03-26T08:46:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.842+0000 7effdf7fe640 1 -- 172.21.15.7:0/4197826803 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effe40035f0 con 0x7efff01038c0 2024-03-26T08:46:38.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.842+0000 7efff4e34640 1 -- 172.21.15.7:0/4197826803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff01a5ec0 con 0x7efff01038c0 2024-03-26T08:46:38.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.842+0000 7effdf7fe640 1 -- 172.21.15.7:0/4197826803 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe400ce60 con 0x7efff01038c0 2024-03-26T08:46:38.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.842+0000 7efff4e34640 1 -- 172.21.15.7:0/4197826803 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff01a6380 con 0x7efff01038c0 2024-03-26T08:46:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.846+0000 7efff7a19740 1 Processor -- start 2024-03-26T08:46:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.846+0000 7effdf7fe640 1 -- 172.21.15.7:0/4197826803 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7effe4010690 con 0x7efff01038c0 2024-03-26T08:46:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.846+0000 7efff7a19740 1 -- start start 2024-03-26T08:46:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.846+0000 7efff7a19740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c596690 0x558b1c596ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.846+0000 7efff7a19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c597080 0x558b1c59bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.846+0000 7efff7a19740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c59e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.846+0000 7efff7a19740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1c559610 con 0x558b1c59c1f0 2024-03-26T08:46:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.846+0000 7efff7a19740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1c4fc3c0 con 0x558b1c596690 2024-03-26T08:46:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.846+0000 7efff7a19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1c4a4d60 con 0x558b1c597080 2024-03-26T08:46:38.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.847+0000 7effee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c597080 0x558b1c59bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.847+0000 7effeed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c596690 0x558b1c596ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.847+0000 7effeed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c596690 0x558b1c596ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50860/0 (socket says 172.21.15.7:50860) 2024-03-26T08:46:38.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.847+0000 7effee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c597080 0x558b1c59bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59580/0 (socket says 172.21.15.7:59580) 2024-03-26T08:46:38.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.847+0000 7effeed9d640 1 -- 172.21.15.7:0/192732165 learned_addr learned my addr 172.21.15.7:0/192732165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.847+0000 7effedd9b640 1 --2- 172.21.15.7:0/192732165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c59e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.847+0000 7effdf7fe640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7effd80a28d0 0x7effd80a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.847+0000 7effdf7fe640 1 -- 172.21.15.7:0/4197826803 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7effe40cedd0 con 0x7efff01038c0 2024-03-26T08:46:38.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.848+0000 7effeed9d640 1 -- 172.21.15.7:0/192732165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c597080 msgr2=0x558b1c59bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.848+0000 7effeed9d640 1 --2- 172.21.15.7:0/192732165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c597080 0x558b1c59bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.848+0000 7effeed9d640 1 -- 172.21.15.7:0/192732165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 msgr2=0x558b1c59e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.848+0000 7effeed9d640 1 --2- 172.21.15.7:0/192732165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c59e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.848+0000 7effeed9d640 1 -- 172.21.15.7:0/192732165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b1c2103f0 con 0x558b1c596690 2024-03-26T08:46:38.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.848+0000 7effeed9d640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7effd80a28d0 0x7effd80a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.848+0000 7effeed9d640 1 --2- 172.21.15.7:0/192732165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c596690 0x558b1c596ab0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7effe8005e30 tx=0x7effe80098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0611fff0f756e69 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.849+0000 7effdcff9640 1 -- 172.21.15.7:0/192732165 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8039040 con 0x558b1c596690 2024-03-26T08:46:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.849+0000 7effdcff9640 1 -- 172.21.15.7:0/192732165 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effe8034cd0 con 0x558b1c596690 2024-03-26T08:46:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.849+0000 7effdcff9640 1 -- 172.21.15.7:0/192732165 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8004510 con 0x558b1c596690 2024-03-26T08:46:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.849+0000 7effeed9d640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7effd80a28d0 0x7effd80a4d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7effe8048000 tx=0x7effe8045000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.850+0000 7efff7a19740 1 -- 172.21.15.7:0/192732165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c596690 msgr2=0x558b1c596ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.850+0000 7efff7a19740 1 --2- 172.21.15.7:0/192732165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c596690 0x558b1c596ab0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7effe8005e30 tx=0x7effe80098d0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.850+0000 7efff7a19740 1 -- 172.21.15.7:0/192732165 shutdown_connections 2024-03-26T08:46:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.850+0000 7efff7a19740 1 --2- 172.21.15.7:0/192732165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c59e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.850+0000 7efff7a19740 1 --2- 172.21.15.7:0/192732165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c597080 0x558b1c59bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.850+0000 7efff7a19740 1 --2- 172.21.15.7:0/192732165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c596690 0x558b1c596ab0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.850+0000 7efff7a19740 1 -- 172.21.15.7:0/192732165 >> 172.21.15.7:0/192732165 conn(0x558b1c591c00 msgr2=0x558b1c592000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.851+0000 7efff7a19740 1 -- 172.21.15.7:0/192732165 shutdown_connections 2024-03-26T08:46:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.851+0000 7efff7a19740 1 -- 172.21.15.7:0/192732165 wait complete. 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.851+0000 7efff7a19740 1 -- start start 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.851+0000 7efff7a19740 1 Processor -- start 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.851+0000 7efff7a19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c596690 0x558b1c5a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7efff7a19740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7efff7a19740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c5a8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effeed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c5a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c596690 0x558b1c5a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effee59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c596690 0x558b1c5a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59582/0 (socket says 172.21.15.7:59582) 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effeed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c5a8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55624/0 (socket says 172.21.15.7:55624) 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effee59c640 1 -- 172.21.15.7:0/1834261771 learned_addr learned my addr 172.21.15.7:0/1834261771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7efff7a19740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1c4b8240 con 0x558b1c59c1f0 2024-03-26T08:46:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1c4f8460 con 0x558b1c597080 2024-03-26T08:46:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b1c59f1a0 con 0x558b1c596690 2024-03-26T08:46:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7efff7a19740 10 client.0 _ll_register_callbacks cb 0x558b1c4b7c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effedd9b640 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effeed9d640 1 -- 172.21.15.7:0/1834261771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c596690 msgr2=0x558b1c5a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effeed9d640 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c596690 0x558b1c5a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effeed9d640 1 -- 172.21.15.7:0/1834261771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 msgr2=0x558b1c5a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effeed9d640 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.852+0000 7effeed9d640 1 -- 172.21.15.7:0/1834261771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b1c2da7d0 con 0x558b1c59c1f0 2024-03-26T08:46:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.853+0000 7effeed9d640 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c5a8030 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7effe8009bf0 tx=0x7effe8032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8536600f550c98d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.853+0000 7effd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.853+0000 7effd5ffb640 1 -- 172.21.15.7:0/1834261771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8059060 con 0x558b1c59c1f0 2024-03-26T08:46:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.853+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effe8034cd0 con 0x558b1c59c1f0 2024-03-26T08:46:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.853+0000 7effd5ffb640 1 -- 172.21.15.7:0/1834261771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7effe80349f0 con 0x558b1c59c1f0 2024-03-26T08:46:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.854+0000 7effd5ffb640 1 -- 172.21.15.7:0/1834261771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7effe8039040 con 0x558b1c59c1f0 2024-03-26T08:46:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.854+0000 7efff7a19740 10 client.16527 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.854+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558b1c5ab3a0 con 0x558b1c59c1f0 2024-03-26T08:46:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.855+0000 7effd5ffb640 1 -- 172.21.15.7:0/1834261771 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effe8034cd0 con 0x558b1c59c1f0 2024-03-26T08:46:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.855+0000 7effd5ffb640 1 -- 172.21.15.7:0/1834261771 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7effe805e070 con 0x558b1c59c1f0 2024-03-26T08:46:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.855+0000 7efff7a19740 10 client.16527 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.855+0000 7efff7a19740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.855+0000 7efff7a19740 20 client.16527 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.855+0000 7efff7a19740 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558b1c5acb70 0x558b1c5aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.855+0000 7efff7a19740 4 client.16527 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.855+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b1c509900 con 0x558b1c5acb70 2024-03-26T08:46:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.855+0000 7effee59c640 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558b1c5acb70 0x558b1c5aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.856+0000 7effee59c640 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558b1c5acb70 0x558b1c5aefa0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=630272f8df8c1322 server_cookie=1561e1b444caf343 in_seq=0 out_seq=0 2024-03-26T08:46:38.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.856+0000 7effd5ffb640 10 client.16527 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:38.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.857+0000 7effd5ffb640 1 -- 172.21.15.7:0/1834261771 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558b1c509900 con 0x558b1c5acb70 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.857+0000 7effd5ffb640 10 client.16527 handle_command_reply: tid=0 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.857+0000 7efff7a19740 1 client.16527 shutdown 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.857+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 msgr2=0x558b1c5a8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.857+0000 7efff7a19740 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c5a8030 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7effe8009bf0 tx=0x7effe8032c90 comp rx=0 tx=0).stop 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.857+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 shutdown_connections 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.857+0000 7efff7a19740 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558b1c5acb70 0x558b1c5aefa0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.858+0000 7effd5ffb640 0 client.16527 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.857+0000 7efff7a19740 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c59c1f0 0x558b1c5a8030 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.858+0000 7efff7a19740 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.858+0000 7efff7a19740 1 --2- 172.21.15.7:0/1834261771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c596690 0x558b1c5a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.858+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 >> 172.21.15.7:0/1834261771 conn(0x558b1c591c00 msgr2=0x558b1c590e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.858+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 shutdown_connections 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.858+0000 7efff7a19740 1 -- 172.21.15.7:0/1834261771 wait complete. 2024-03-26T08:46:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.858+0000 7efff7a19740 20 client.16527 trim_cache size 0 max 16384 2024-03-26T08:46:38.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.886+0000 7efff7a19740 1 Processor -- start 2024-03-26T08:46:38.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.887+0000 7efff7a19740 1 -- start start 2024-03-26T08:46:38.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.888+0000 7efff7a19740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c597a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.888+0000 7efff7a19740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.888+0000 7efff7a19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 0x558b1c591fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.888+0000 7efff7a19740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff01b1600 con 0x558b1c596690 2024-03-26T08:46:38.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.888+0000 7efff7a19740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff01af100 con 0x558b1c597080 2024-03-26T08:46:38.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.888+0000 7efff7a19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff01ad180 con 0x558b1c59c1f0 2024-03-26T08:46:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.892+0000 7effedd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.892+0000 7effeed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 0x558b1c591fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.892+0000 7effee59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c597a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.892+0000 7effee59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c597a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55628/0 (socket says 172.21.15.7:55628) 2024-03-26T08:46:38.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.892+0000 7effee59c640 1 -- 172.21.15.7:0/2003291559 learned_addr learned my addr 172.21.15.7:0/2003291559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.894+0000 7effee59c640 1 -- 172.21.15.7:0/2003291559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 msgr2=0x558b1c591fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.894+0000 7effee59c640 1 --2- 172.21.15.7:0/2003291559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 0x558b1c591fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.894+0000 7effee59c640 1 -- 172.21.15.7:0/2003291559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 msgr2=0x558b1c5a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.894+0000 7effee59c640 1 --2- 172.21.15.7:0/2003291559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.894+0000 7effee59c640 1 -- 172.21.15.7:0/2003291559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b1c598ea0 con 0x558b1c596690 2024-03-26T08:46:38.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.895+0000 7effee59c640 1 --2- 172.21.15.7:0/2003291559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c597a80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7effe4097b90 tx=0x7effe4086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb63688fe941ad36 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.898+0000 7effd7fff640 1 -- 172.21.15.7:0/2003291559 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe409f880 con 0x558b1c596690 2024-03-26T08:46:38.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.898+0000 7effd7fff640 1 -- 172.21.15.7:0/2003291559 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effe40a3ed0 con 0x558b1c596690 2024-03-26T08:46:38.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.898+0000 7effd7fff640 1 -- 172.21.15.7:0/2003291559 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4097da0 con 0x558b1c596690 2024-03-26T08:46:38.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.900+0000 7efff7a19740 1 -- 172.21.15.7:0/2003291559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 msgr2=0x558b1c597a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.900+0000 7efff7a19740 1 --2- 172.21.15.7:0/2003291559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c597a80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7effe4097b90 tx=0x7effe4086040 comp rx=0 tx=0).stop 2024-03-26T08:46:38.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.905+0000 7efff7a19740 1 -- 172.21.15.7:0/2003291559 shutdown_connections 2024-03-26T08:46:38.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.905+0000 7efff7a19740 1 --2- 172.21.15.7:0/2003291559 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 0x558b1c591fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.905+0000 7efff7a19740 1 --2- 172.21.15.7:0/2003291559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.905+0000 7efff7a19740 1 --2- 172.21.15.7:0/2003291559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c597a80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.905+0000 7efff7a19740 1 -- 172.21.15.7:0/2003291559 >> 172.21.15.7:0/2003291559 conn(0x558b1c591c00 msgr2=0x558b1c59f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.907+0000 7efff7a19740 1 -- 172.21.15.7:0/2003291559 shutdown_connections 2024-03-26T08:46:38.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.911+0000 7efff7a19740 1 -- 172.21.15.7:0/2003291559 wait complete. 2024-03-26T08:46:38.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.911+0000 7efff7a19740 1 -- start start 2024-03-26T08:46:38.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.911+0000 7efff7a19740 1 Processor -- start 2024-03-26T08:46:38.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.913+0000 7efff7a19740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c5a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.913+0000 7efff7a19740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.913+0000 7efff7a19740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 0x558b1c5a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.913+0000 7efff7a19740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff01ad780 con 0x558b1c596690 2024-03-26T08:46:38.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.914+0000 7efff7a19740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff01aeaa0 con 0x558b1c597080 2024-03-26T08:46:38.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.914+0000 7efff7a19740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff01adde0 con 0x558b1c59c1f0 2024-03-26T08:46:38.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.914+0000 7efff7a19740 10 client.0 _ll_register_callbacks cb 0x7effe8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:38.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effedd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effedd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50890/0 (socket says 172.21.15.7:50890) 2024-03-26T08:46:38.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effedd9b640 1 -- 172.21.15.7:0/1239370529 learned_addr learned my addr 172.21.15.7:0/1239370529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effee59c640 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c5a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effeed9d640 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 0x558b1c5a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effee59c640 1 -- 172.21.15.7:0/1239370529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 msgr2=0x558b1c5a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effee59c640 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 0x558b1c5a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effee59c640 1 -- 172.21.15.7:0/1239370529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 msgr2=0x558b1c5a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effee59c640 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.915+0000 7effee59c640 1 -- 172.21.15.7:0/1239370529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b1c5a7180 con 0x558b1c596690 2024-03-26T08:46:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.916+0000 7effee59c640 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c5a5640 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x558b1c5acaf0 tx=0x7effe409b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc45da876b1a293 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.916+0000 7effd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.916+0000 7effd5ffb640 1 -- 172.21.15.7:0/1239370529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe40b8020 con 0x558b1c596690 2024-03-26T08:46:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.916+0000 7efff7a19740 1 -- 172.21.15.7:0/1239370529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effe40a3ed0 con 0x558b1c596690 2024-03-26T08:46:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.916+0000 7effd5ffb640 1 -- 172.21.15.7:0/1239370529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7effe4099a30 con 0x558b1c596690 2024-03-26T08:46:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.917+0000 7effd5ffb640 1 -- 172.21.15.7:0/1239370529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7effe4097da0 con 0x558b1c596690 2024-03-26T08:46:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.918+0000 7efff7a19740 10 client.16533 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.918+0000 7efff7a19740 1 -- 172.21.15.7:0/1239370529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558b1c619480 con 0x558b1c596690 2024-03-26T08:46:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.919+0000 7effd5ffb640 1 -- 172.21.15.7:0/1239370529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7effe40037a0 con 0x558b1c596690 2024-03-26T08:46:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.919+0000 7effd5ffb640 1 -- 172.21.15.7:0/1239370529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7effe409ac90 con 0x558b1c596690 2024-03-26T08:46:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.919+0000 7efff7a19740 10 client.16533 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.919+0000 7efff7a19740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.919+0000 7efff7a19740 20 client.16533 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.919+0000 7efff7a19740 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558b1c5acb70 0x558b1c61bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.919+0000 7efff7a19740 4 client.16533 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:46:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.919+0000 7efff7a19740 1 -- 172.21.15.7:0/1239370529 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7effe40a3ed0 con 0x558b1c5acb70 2024-03-26T08:46:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.920+0000 7effedd9b640 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558b1c5acb70 0x558b1c61bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.921+0000 7effedd9b640 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558b1c5acb70 0x558b1c61bd80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e292c4c0991f1bb0 server_cookie=6c29068242321ec4 in_seq=0 out_seq=0 2024-03-26T08:46:38.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.923+0000 7effd5ffb640 10 client.16533 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:38.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.923+0000 7effd5ffb640 1 -- 172.21.15.7:0/1239370529 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7effe40a3ed0 con 0x558b1c5acb70 2024-03-26T08:46:38.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.923+0000 7effd5ffb640 10 client.16533 handle_command_reply: tid=0 2024-03-26T08:46:38.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.923+0000 7efff7a19740 1 client.16533 shutdown 2024-03-26T08:46:38.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.924+0000 7efff7a19740 1 -- 172.21.15.7:0/1239370529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 msgr2=0x558b1c5a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.924+0000 7efff7a19740 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c5a5640 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x558b1c5acaf0 tx=0x7effe409b860 comp rx=0 tx=0).stop 2024-03-26T08:46:38.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.924+0000 7efff7a19740 1 -- 172.21.15.7:0/1239370529 shutdown_connections 2024-03-26T08:46:38.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.924+0000 7efff7a19740 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558b1c5acb70 0x558b1c61bd80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.924+0000 7efff7a19740 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558b1c59c1f0 0x558b1c5a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.925+0000 7efff7a19740 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558b1c597080 0x558b1c5a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.925+0000 7effd5ffb640 0 client.16533 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:38.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.925+0000 7efff7a19740 1 --2- 172.21.15.7:0/1239370529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558b1c596690 0x558b1c5a5640 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.925+0000 7efff7a19740 1 -- 172.21.15.7:0/1239370529 >> 172.21.15.7:0/1239370529 conn(0x558b1c591c00 msgr2=0x558b1c597a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.925+0000 7efff7a19740 1 -- 172.21.15.7:0/1239370529 shutdown_connections 2024-03-26T08:46:38.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.925+0000 7efff7a19740 1 -- 172.21.15.7:0/1239370529 wait complete. 2024-03-26T08:46:38.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.925+0000 7efff7a19740 20 client.16533 trim_cache size 0 max 16384 2024-03-26T08:46:38.917 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.926+0000 7efff4e34640 1 -- 172.21.15.7:0/4197826803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7effd80a28d0 msgr2=0x7effd80a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.926+0000 7efff4e34640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7effd80a28d0 0x7effd80a4d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7effe8048000 tx=0x7effe8045000 comp rx=0 tx=0).stop 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.926+0000 7efff4e34640 1 -- 172.21.15.7:0/4197826803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff01038c0 msgr2=0x7efff019dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.926+0000 7efff4e34640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff01038c0 0x7efff019dfb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7effe400e9c0 tx=0x7effe400ee80 comp rx=0 tx=0).stop 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.927+0000 7efff4e34640 1 -- 172.21.15.7:0/4197826803 shutdown_connections 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.927+0000 7efff4e34640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7effd80a28d0 0x7effd80a4d90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.927+0000 7efff4e34640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efff0105470 0x7efff01a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.927+0000 7efff4e34640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff0104ab0 0x7efff019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.927+0000 7efff4e34640 1 --2- 172.21.15.7:0/4197826803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efff01038c0 0x7efff019dfb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.927+0000 7efff4e34640 1 -- 172.21.15.7:0/4197826803 >> 172.21.15.7:0/4197826803 conn(0x7efff00fee50 msgr2=0x7efff0077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.927+0000 7efff4e34640 1 -- 172.21.15.7:0/4197826803 shutdown_connections 2024-03-26T08:46:38.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:38.928+0000 7efff4e34640 1 -- 172.21.15.7:0/4197826803 wait complete. 2024-03-26T08:46:39.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:39 smithi016 ceph-mon[35851]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 770 op/s 2024-03-26T08:46:39.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:39 smithi007 ceph-mon[32462]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 770 op/s 2024-03-26T08:46:39.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:39 smithi044 ceph-mon[36461]: pgmap v487: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 770 op/s 2024-03-26T08:46:39.937 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:46:39.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:46:39.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:46:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.095+0000 7febcd636640 1 -- 172.21.15.7:0/1606143257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8102180 msgr2=0x7febc8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.095+0000 7febcd636640 1 --2- 172.21.15.7:0/1606143257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8102180 0x7febc8108a10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7febc0009ca0 tx=0x7febc002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.095+0000 7febcd636640 1 -- 172.21.15.7:0/1606143257 shutdown_connections 2024-03-26T08:46:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.095+0000 7febcd636640 1 --2- 172.21.15.7:0/1606143257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8102180 0x7febc8108a10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.095+0000 7febcd636640 1 --2- 172.21.15.7:0/1606143257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc81017c0 0x7febc8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.095+0000 7febcd636640 1 --2- 172.21.15.7:0/1606143257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc81005d0 0x7febc81009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.095+0000 7febcd636640 1 -- 172.21.15.7:0/1606143257 >> 172.21.15.7:0/1606143257 conn(0x7febc80fbb40 msgr2=0x7febc80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.095+0000 7febcd636640 1 -- 172.21.15.7:0/1606143257 shutdown_connections 2024-03-26T08:46:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.095+0000 7febcd636640 1 -- 172.21.15.7:0/1606143257 wait complete. 2024-03-26T08:46:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.096+0000 7febcd636640 1 Processor -- start 2024-03-26T08:46:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.096+0000 7febcd636640 1 -- start start 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.096+0000 7febcd636640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc81005d0 0x7febc81a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.096+0000 7febcd636640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc81017c0 0x7febc81a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.096+0000 7febcd636640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8102180 0x7febc81a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febcd636640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc810c250 con 0x7febc8102180 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febcd636640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc810c0d0 con 0x7febc81017c0 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febcd636640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc810c3d0 con 0x7febc81005d0 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc81005d0 0x7febc81a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8102180 0x7febc81a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc81005d0 0x7febc81a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59608/0 (socket says 172.21.15.7:59608) 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc6ffd640 1 -- 172.21.15.7:0/4000300818 learned_addr learned my addr 172.21.15.7:0/4000300818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc6ffd640 1 -- 172.21.15.7:0/4000300818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc81017c0 msgr2=0x7febc81a2c90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc6ffd640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc81017c0 0x7febc81a2c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc6ffd640 1 -- 172.21.15.7:0/4000300818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8102180 msgr2=0x7febc81a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc6ffd640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8102180 0x7febc81a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc6ffd640 1 -- 172.21.15.7:0/4000300818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc81a5be0 con 0x7febc81005d0 2024-03-26T08:46:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.097+0000 7febc6ffd640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc81005d0 0x7febc81a0740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7febb800e9c0 tx=0x7febb800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.098+0000 7febc4ff9640 1 -- 172.21.15.7:0/4000300818 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb800cd80 con 0x7febc81005d0 2024-03-26T08:46:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.098+0000 7febc4ff9640 1 -- 172.21.15.7:0/4000300818 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febb80035f0 con 0x7febc81005d0 2024-03-26T08:46:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.098+0000 7febc4ff9640 1 -- 172.21.15.7:0/4000300818 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8010690 con 0x7febc81005d0 2024-03-26T08:46:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.098+0000 7febcd636640 1 -- 172.21.15.7:0/4000300818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc81a5ed0 con 0x7febc81005d0 2024-03-26T08:46:40.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.098+0000 7febcd636640 1 -- 172.21.15.7:0/4000300818 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc81a62e0 con 0x7febc81005d0 2024-03-26T08:46:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.101+0000 7febc4ff9640 1 -- 172.21.15.7:0/4000300818 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7febb8004020 con 0x7febc81005d0 2024-03-26T08:46:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.102+0000 7febd0226740 1 Processor -- start 2024-03-26T08:46:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febd0226740 1 -- start start 2024-03-26T08:46:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febd0226740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f52d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febd0226740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f53310 0x55a6a6f57e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febd0226740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f5a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febd0226740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a6eb7b10 con 0x55a6a6f58480 2024-03-26T08:46:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febd0226740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a6e60900 con 0x55a6a6f53310 2024-03-26T08:46:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febd0226740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a6eb3bb0 con 0x55a6a6f52920 2024-03-26T08:46:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febc4ff9640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febb40a29c0 0x7febb40a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f53310 0x55a6a6f57e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f5a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f52d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f5a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55680/0 (socket says 172.21.15.7:55680) 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febbe7fc640 1 -- 172.21.15.7:0/1506197930 learned_addr learned my addr 172.21.15.7:0/1506197930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f52d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59610/0 (socket says 172.21.15.7:59610) 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febc77fe640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febb40a29c0 0x7febb40a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.103+0000 7febc4ff9640 1 -- 172.21.15.7:0/4000300818 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7febb80cedb0 con 0x7febc81005d0 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febc77fe640 1 -- 172.21.15.7:0/1506197930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f53310 msgr2=0x55a6a6f57e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febc77fe640 1 --2- 172.21.15.7:0/1506197930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f53310 0x55a6a6f57e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febc77fe640 1 -- 172.21.15.7:0/1506197930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 msgr2=0x55a6a6f5a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febc77fe640 1 --2- 172.21.15.7:0/1506197930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f5a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febc77fe640 1 -- 172.21.15.7:0/1506197930 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6a6bcc210 con 0x55a6a6f52920 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febbe7fc640 1 --2- 172.21.15.7:0/1506197930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f5a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febc77fe640 1 --2- 172.21.15.7:0/1506197930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f52d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7febc002c630 tx=0x7febc002cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a1d969fd0bc829 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febc77fe640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febb40a29c0 0x7febb40a4e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7febc003b000 tx=0x7febc0047000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febbd7fa640 1 -- 172.21.15.7:0/1506197930 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0050070 con 0x55a6a6f52920 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.104+0000 7febbd7fa640 1 -- 172.21.15.7:0/1506197930 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febc0047070 con 0x55a6a6f52920 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.105+0000 7febbd7fa640 1 -- 172.21.15.7:0/1506197930 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0039040 con 0x55a6a6f52920 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.105+0000 7febd0226740 1 -- 172.21.15.7:0/1506197930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 msgr2=0x55a6a6f52d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.105+0000 7febd0226740 1 --2- 172.21.15.7:0/1506197930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f52d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7febc002c630 tx=0x7febc002cdd0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.105+0000 7febd0226740 1 -- 172.21.15.7:0/1506197930 shutdown_connections 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.105+0000 7febd0226740 1 --2- 172.21.15.7:0/1506197930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f5a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.105+0000 7febd0226740 1 --2- 172.21.15.7:0/1506197930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f53310 0x55a6a6f57e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.105+0000 7febd0226740 1 --2- 172.21.15.7:0/1506197930 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f52d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.105+0000 7febd0226740 1 -- 172.21.15.7:0/1506197930 >> 172.21.15.7:0/1506197930 conn(0x55a6a6f4de90 msgr2=0x55a6a6f4e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.106+0000 7febd0226740 1 -- 172.21.15.7:0/1506197930 shutdown_connections 2024-03-26T08:46:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.106+0000 7febd0226740 1 -- 172.21.15.7:0/1506197930 wait complete. 2024-03-26T08:46:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.107+0000 7febd0226740 1 -- start start 2024-03-26T08:46:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.107+0000 7febd0226740 1 Processor -- start 2024-03-26T08:46:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.107+0000 7febd0226740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f52920 0x55a6a6f5cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.107+0000 7febd0226740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 0x55a6a6f5d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febd0226740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f58480 0x55a6a6f641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febd0226740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a6f5b470 con 0x55a6a6f53310 2024-03-26T08:46:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febd0226740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a6e73d50 con 0x55a6a6f52920 2024-03-26T08:46:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febd0226740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a6f5b5f0 con 0x55a6a6f58480 2024-03-26T08:46:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febd0226740 10 client.0 _ll_register_callbacks cb 0x55a6a6e73720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f58480 0x55a6a6f641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febc77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f58480 0x55a6a6f641d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59628/0 (socket says 172.21.15.7:59628) 2024-03-26T08:46:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febc77fe640 1 -- 172.21.15.7:0/3026052487 learned_addr learned my addr 172.21.15.7:0/3026052487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febc6ffd640 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f52920 0x55a6a6f5cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febc6ffd640 1 -- 172.21.15.7:0/3026052487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f58480 msgr2=0x55a6a6f641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febc6ffd640 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f58480 0x55a6a6f641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febc6ffd640 1 -- 172.21.15.7:0/3026052487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 msgr2=0x55a6a6f5d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febc6ffd640 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 0x55a6a6f5d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.108+0000 7febc6ffd640 1 -- 172.21.15.7:0/3026052487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6a6c98620 con 0x55a6a6f52920 2024-03-26T08:46:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.109+0000 7febc77fe640 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f58480 0x55a6a6f641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.109+0000 7f7f33fff640 1 -- 172.21.15.7:0/629594211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f340ffb60 msgr2=0x7f7f340fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.109+0000 7f7f33fff640 1 --2- 172.21.15.7:0/629594211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f340ffb60 0x7f7f340fff80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400aa10 tx=0x7f7f2402cf70 comp rx=0 tx=0).stop 2024-03-26T08:46:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.109+0000 7febc6ffd640 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f52920 0x55a6a6f5cc50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7febb8063f20 tx=0x7febb809b130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e051683e7fec0a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.109+0000 7febae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.109+0000 7f7f33fff640 1 -- 172.21.15.7:0/629594211 shutdown_connections 2024-03-26T08:46:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.109+0000 7f7f33fff640 1 --2- 172.21.15.7:0/629594211 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f34101710 0x7f7f34107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.109+0000 7f7f33fff640 1 --2- 172.21.15.7:0/629594211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f34100d50 0x7f7f341011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.109+0000 7f7f33fff640 1 --2- 172.21.15.7:0/629594211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f340ffb60 0x7f7f340fff80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.109+0000 7f7f33fff640 1 -- 172.21.15.7:0/629594211 >> 172.21.15.7:0/629594211 conn(0x7f7f340fb0d0 msgr2=0x7f7f340fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.109+0000 7f7f33fff640 1 -- 172.21.15.7:0/629594211 shutdown_connections 2024-03-26T08:46:40.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.109+0000 7f7f33fff640 1 -- 172.21.15.7:0/629594211 wait complete. 2024-03-26T08:46:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.109+0000 7febae7fc640 1 -- 172.21.15.7:0/3026052487 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb80b4060 con 0x55a6a6f52920 2024-03-26T08:46:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.109+0000 7febae7fc640 1 -- 172.21.15.7:0/3026052487 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7febb809ab60 con 0x55a6a6f52920 2024-03-26T08:46:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.109+0000 7febd0226740 1 -- 172.21.15.7:0/3026052487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febc0047070 con 0x55a6a6f52920 2024-03-26T08:46:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.109+0000 7febae7fc640 1 -- 172.21.15.7:0/3026052487 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7febb8045e60 con 0x55a6a6f52920 2024-03-26T08:46:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.110+0000 7f7f33fff640 1 Processor -- start 2024-03-26T08:46:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.110+0000 7febd0226740 10 client.26381 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.110+0000 7febd0226740 1 -- 172.21.15.7:0/3026052487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a6a6f785c0 con 0x55a6a6f52920 2024-03-26T08:46:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.110+0000 7febae7fc640 1 -- 172.21.15.7:0/3026052487 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febb80a6070 con 0x55a6a6f52920 2024-03-26T08:46:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.110+0000 7f7f33fff640 1 -- start start 2024-03-26T08:46:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.110+0000 7febae7fc640 1 -- 172.21.15.7:0/3026052487 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7febb80974f0 con 0x55a6a6f52920 2024-03-26T08:46:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.110+0000 7f7f33fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f340ffb60 0x7f7f34103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.110+0000 7febd0226740 10 client.26381 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.110+0000 7febd0226740 10 client.26381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.110+0000 7febd0226740 20 client.26381 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.110+0000 7febd0226740 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a6a6f7a0f0 0x55a6a6f7c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.111+0000 7febd0226740 4 client.26381 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.111+0000 7febd0226740 1 -- 172.21.15.7:0/3026052487 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6a6ec5050 con 0x55a6a6f7a0f0 2024-03-26T08:46:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.111+0000 7febbe7fc640 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a6a6f7a0f0 0x55a6a6f7c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.110+0000 7f7f33fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f34100d50 0x7f7f34102130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.110+0000 7f7f33fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f34101710 0x7f7f34102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.110+0000 7f7f33fff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3410b080 con 0x7f7f34100d50 2024-03-26T08:46:40.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.110+0000 7f7f33fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3410af00 con 0x7f7f34101710 2024-03-26T08:46:40.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.110+0000 7f7f33fff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3410b200 con 0x7f7f340ffb60 2024-03-26T08:46:40.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f34101710 0x7f7f34102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f34101710 0x7f7f34102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50938/0 (socket says 172.21.15.7:50938) 2024-03-26T08:46:40.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f337fe640 1 -- 172.21.15.7:0/212020020 learned_addr learned my addr 172.21.15.7:0/212020020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f32ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f340ffb60 0x7f7f34103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f327fc640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f34100d50 0x7f7f34102130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f32ffd640 1 -- 172.21.15.7:0/212020020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f34101710 msgr2=0x7f7f34102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.111+0000 7febbe7fc640 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a6a6f7a0f0 0x55a6a6f7c550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3621c3bd6ced481a server_cookie=e99905e354429e5d in_seq=0 out_seq=0 2024-03-26T08:46:40.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.111+0000 7febae7fc640 10 client.26381 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f34101710 0x7f7f34102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f32ffd640 1 -- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f34100d50 msgr2=0x7f7f34102130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f34100d50 0x7f7f34102130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f32ffd640 1 -- 172.21.15.7:0/212020020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f34102f30 con 0x7f7f340ffb60 2024-03-26T08:46:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f337fe640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f34101710 0x7f7f34102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f340ffb60 0x7f7f34103aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400a340 tx=0x7f7f24005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.111+0000 7f7f38974640 1 -- 172.21.15.7:0/212020020 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24042020 con 0x7f7f340ffb60 2024-03-26T08:46:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.112+0000 7f7f33fff640 1 -- 172.21.15.7:0/212020020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f341a1ac0 con 0x7f7f340ffb60 2024-03-26T08:46:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.112+0000 7f7f38974640 1 -- 172.21.15.7:0/212020020 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f24004ad0 con 0x7f7f340ffb60 2024-03-26T08:46:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.112+0000 7f7f38974640 1 -- 172.21.15.7:0/212020020 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24003940 con 0x7f7f340ffb60 2024-03-26T08:46:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.112+0000 7f7f33fff640 1 -- 172.21.15.7:0/212020020 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f341a2030 con 0x7f7f340ffb60 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febae7fc640 1 -- 172.21.15.7:0/3026052487 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a6a6ec5050 con 0x55a6a6f7a0f0 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febae7fc640 10 client.26381 handle_command_reply: tid=0 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 client.26381 shutdown 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 -- 172.21.15.7:0/3026052487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f52920 msgr2=0x55a6a6f5cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f52920 0x55a6a6f5cc50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7febb8063f20 tx=0x7febb809b130 comp rx=0 tx=0).stop 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 -- 172.21.15.7:0/3026052487 shutdown_connections 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a6a6f7a0f0 0x55a6a6f7c550 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f58480 0x55a6a6f641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 0x55a6a6f5d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 --2- 172.21.15.7:0/3026052487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f52920 0x55a6a6f5cc50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 -- 172.21.15.7:0/3026052487 >> 172.21.15.7:0/3026052487 conn(0x55a6a6f4de90 msgr2=0x55a6a6f4d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febae7fc640 0 client.26381 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 -- 172.21.15.7:0/3026052487 shutdown_connections 2024-03-26T08:46:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 1 -- 172.21.15.7:0/3026052487 wait complete. 2024-03-26T08:46:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.112+0000 7febd0226740 20 client.26381 trim_cache size 0 max 16384 2024-03-26T08:46:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.115+0000 7f7f3c635740 1 Processor -- start 2024-03-26T08:46:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.115+0000 7f7f38974640 1 -- 172.21.15.7:0/212020020 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7f24045020 con 0x7f7f340ffb60 2024-03-26T08:46:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.116+0000 7f7f3c635740 1 -- start start 2024-03-26T08:46:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.116+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b4a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.116+0000 7f7f337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b4a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.116+0000 7f7f337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b4a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55714/0 (socket says 172.21.15.7:55714) 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 0x5645b2b4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 0x5645b2b52330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f3c635740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2a81c90 con 0x5645b2b4a340 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f3c635740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2aa7070 con 0x5645b2b4fea0 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f3c635740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2a4fc80 con 0x5645b2b4ad30 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f337fe640 1 -- 172.21.15.7:0/2585556187 learned_addr learned my addr 172.21.15.7:0/2585556187 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f327fc640 1 --2- 172.21.15.7:0/2585556187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 0x5645b2b52330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f337fe640 1 -- 172.21.15.7:0/2585556187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 msgr2=0x5645b2b4f830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f337fe640 1 --2- 172.21.15.7:0/2585556187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 0x5645b2b4f830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f38974640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f1c0a2880 0x7f7f1c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f337fe640 1 -- 172.21.15.7:0/2585556187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 msgr2=0x5645b2b52330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f337fe640 1 --2- 172.21.15.7:0/2585556187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 0x5645b2b52330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f337fe640 1 -- 172.21.15.7:0/2585556187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b2a47d10 con 0x5645b2b4a340 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.117+0000 7f7f38974640 1 -- 172.21.15.7:0/212020020 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f240f1020 con 0x7f7f340ffb60 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.118+0000 7f7f337fe640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f1c0a2880 0x7f7f1c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.118+0000 7f7f337fe640 1 --2- 172.21.15.7:0/2585556187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b4a760 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00e5b0 tx=0x7f7f2c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8ec549043c2258 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.118+0000 7f7f19ffb640 1 -- 172.21.15.7:0/2585556187 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c019040 con 0x5645b2b4a340 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.118+0000 7f7f19ffb640 1 -- 172.21.15.7:0/2585556187 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c015bf0 con 0x5645b2b4a340 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.119+0000 7f7f19ffb640 1 -- 172.21.15.7:0/2585556187 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0133e0 con 0x5645b2b4a340 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.119+0000 7f7f3c635740 1 -- 172.21.15.7:0/2585556187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 msgr2=0x5645b2b4a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.119+0000 7f7f337fe640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f1c0a2880 0x7f7f1c0a4d40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c01f000 tx=0x7f7f2c022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.119+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2585556187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b4a760 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c00e5b0 tx=0x7f7f2c009710 comp rx=0 tx=0).stop 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.119+0000 7f7f3c635740 1 -- 172.21.15.7:0/2585556187 shutdown_connections 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.119+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2585556187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 0x5645b2b52330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.119+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2585556187 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 0x5645b2b4f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.119+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2585556187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b4a760 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.119+0000 7f7f3c635740 1 -- 172.21.15.7:0/2585556187 >> 172.21.15.7:0/2585556187 conn(0x5645b2b0a1a0 msgr2=0x5645b2b45cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.120+0000 7f7f3c635740 1 -- 172.21.15.7:0/2585556187 shutdown_connections 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.120+0000 7f7f3c635740 1 -- 172.21.15.7:0/2585556187 wait complete. 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.120+0000 7f7f3c635740 1 -- start start 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.120+0000 7f7f3c635740 1 Processor -- start 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.120+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4a340 0x5645b2b54720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.120+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4ad30 0x5645b2b54c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 0x5645b2b5bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f3c635740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2a630d0 con 0x5645b2b4ad30 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f3c635740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2aa3110 con 0x5645b2b4a340 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f3c635740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645b2b52ea0 con 0x5645b2b4fea0 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f327fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4ad30 0x5645b2b54c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f32ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4a340 0x5645b2b54720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f3c635740 10 client.0 _ll_register_callbacks cb 0x5645b2a9ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f32ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4a340 0x5645b2b54720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50946/0 (socket says 172.21.15.7:50946) 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f327fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4ad30 0x5645b2b54c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55724/0 (socket says 172.21.15.7:55724) 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f32ffd640 1 -- 172.21.15.7:0/949744681 learned_addr learned my addr 172.21.15.7:0/949744681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f337fe640 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 0x5645b2b5bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f32ffd640 1 -- 172.21.15.7:0/949744681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 msgr2=0x5645b2b5bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 0x5645b2b5bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f32ffd640 1 -- 172.21.15.7:0/949744681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4ad30 msgr2=0x5645b2b54c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4ad30 0x5645b2b54c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f32ffd640 1 -- 172.21.15.7:0/949744681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645b28837e0 con 0x5645b2b4a340 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4a340 0x5645b2b54720 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7f240b9e70 tx=0x7f7f240bd550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1acfd5c81a7d59c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f12ffd640 1 -- 172.21.15.7:0/949744681 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f240bda10 con 0x5645b2b4a340 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.121+0000 7f7f12ffd640 1 -- 172.21.15.7:0/949744681 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f240bdc70 con 0x5645b2b4a340 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.122+0000 7f7f3c635740 1 -- 172.21.15.7:0/949744681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f2c015bf0 con 0x5645b2b4a340 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.122+0000 7f7f12ffd640 1 -- 172.21.15.7:0/949744681 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f240c1b60 con 0x5645b2b4a340 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.122+0000 7f7f3c635740 10 client.26390 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:40.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.122+0000 7f7f3c635740 1 -- 172.21.15.7:0/949744681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5645b28e3f70 con 0x5645b2b4a340 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.122+0000 7f7f12ffd640 1 -- 172.21.15.7:0/949744681 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c015bf0 con 0x5645b2b4a340 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.122+0000 7f7f12ffd640 1 -- 172.21.15.7:0/949744681 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7f2400b040 con 0x5645b2b4a340 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.123+0000 7f7f3c635740 10 client.26390 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.123+0000 7f7f3c635740 10 client.26390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.123+0000 7f7f3c635740 20 client.26390 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.123+0000 7f7f3c635740 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5645b2b603b0 0x5645b2b627d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.123+0000 7f7f3c635740 4 client.26390 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.123+0000 7f7f3c635740 1 -- 172.21.15.7:0/949744681 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645b2ab45b0 con 0x5645b2b603b0 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.123+0000 7f7f327fc640 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5645b2b603b0 0x5645b2b627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.123+0000 7f7f12ffd640 10 client.26390 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.124+0000 7f7f327fc640 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5645b2b603b0 0x5645b2b627d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75bfb3fb228680e server_cookie=ec63b0b4d84177ef in_seq=0 out_seq=0 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.124+0000 7f7f12ffd640 1 -- 172.21.15.7:0/949744681 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5645b2ab45b0 con 0x5645b2b603b0 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.124+0000 7f7f12ffd640 10 client.26390 handle_command_reply: tid=0 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.124+0000 7f7f3c635740 1 client.26390 shutdown 2024-03-26T08:46:40.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.125+0000 7f7f3c635740 1 -- 172.21.15.7:0/949744681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4a340 msgr2=0x5645b2b54720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.125+0000 7f7f3c635740 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4a340 0x5645b2b54720 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7f240b9e70 tx=0x7f7f240bd550 comp rx=0 tx=0).stop 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.125+0000 7f7f3c635740 1 -- 172.21.15.7:0/949744681 shutdown_connections 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.125+0000 7f7f3c635740 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5645b2b603b0 0x5645b2b627d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.125+0000 7f7f3c635740 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 0x5645b2b5bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.125+0000 7f7f3c635740 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4ad30 0x5645b2b54c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.125+0000 7f7f12ffd640 0 client.26390 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.125+0000 7f7f3c635740 1 --2- 172.21.15.7:0/949744681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4a340 0x5645b2b54720 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.125+0000 7f7f3c635740 1 -- 172.21.15.7:0/949744681 >> 172.21.15.7:0/949744681 conn(0x5645b2b0a1a0 msgr2=0x5645b2b3c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.126+0000 7f7f3c635740 1 -- 172.21.15.7:0/949744681 shutdown_connections 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.126+0000 7f7f3c635740 1 -- 172.21.15.7:0/949744681 wait complete. 2024-03-26T08:46:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.126+0000 7f7f3c635740 20 client.26390 trim_cache size 0 max 16384 2024-03-26T08:46:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.136+0000 7febd0226740 1 Processor -- start 2024-03-26T08:46:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.136+0000 7febd0226740 1 -- start start 2024-03-26T08:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.137+0000 7febd0226740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.137+0000 7febd0226740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f53310 0x55a6a6f54b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.137+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.137+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f54630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59666/0 (socket says 172.21.15.7:59666) 2024-03-26T08:46:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.137+0000 7febc6ffd640 1 -- 172.21.15.7:0/2377389066 learned_addr learned my addr 172.21.15.7:0/2377389066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.137+0000 7febd0226740 1 --2- 172.21.15.7:0/2377389066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f64160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.138+0000 7febd0226740 1 -- 172.21.15.7:0/2377389066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba8010980 con 0x55a6a6f58480 2024-03-26T08:46:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.138+0000 7febc77fe640 1 --2- 172.21.15.7:0/2377389066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f64160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.138+0000 7febd0226740 1 -- 172.21.15.7:0/2377389066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba800e480 con 0x55a6a6f53310 2024-03-26T08:46:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.138+0000 7febd0226740 1 -- 172.21.15.7:0/2377389066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba800c500 con 0x55a6a6f52920 2024-03-26T08:46:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.138+0000 7febc6ffd640 1 -- 172.21.15.7:0/2377389066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f53310 msgr2=0x55a6a6f54b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:46:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.138+0000 7febc6ffd640 1 --2- 172.21.15.7:0/2377389066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f53310 0x55a6a6f54b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.138+0000 7febc6ffd640 1 -- 172.21.15.7:0/2377389066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 msgr2=0x55a6a6f64160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.138+0000 7febc6ffd640 1 --2- 172.21.15.7:0/2377389066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f64160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.138+0000 7febc6ffd640 1 -- 172.21.15.7:0/2377389066 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6a6f52d00 con 0x55a6a6f52920 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.139+0000 7febc6ffd640 1 --2- 172.21.15.7:0/2377389066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f54630 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7febb8000c00 tx=0x7febb8099c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7198cfac08a9eaf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.139+0000 7febbcff9640 1 -- 172.21.15.7:0/2377389066 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb80b4050 con 0x55a6a6f52920 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.139+0000 7febbcff9640 1 -- 172.21.15.7:0/2377389066 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febb80a6030 con 0x55a6a6f52920 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.139+0000 7febbcff9640 1 -- 172.21.15.7:0/2377389066 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb809ab60 con 0x55a6a6f52920 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 -- 172.21.15.7:0/2377389066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 msgr2=0x55a6a6f54630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 --2- 172.21.15.7:0/2377389066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f54630 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7febb8000c00 tx=0x7febb8099c20 comp rx=0 tx=0).stop 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 -- 172.21.15.7:0/2377389066 shutdown_connections 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 --2- 172.21.15.7:0/2377389066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f58480 0x55a6a6f64160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 --2- 172.21.15.7:0/2377389066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f53310 0x55a6a6f54b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 --2- 172.21.15.7:0/2377389066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f54630 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 -- 172.21.15.7:0/2377389066 >> 172.21.15.7:0/2377389066 conn(0x55a6a6f4de90 msgr2=0x55a6a6f4e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 -- 172.21.15.7:0/2377389066 shutdown_connections 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 -- 172.21.15.7:0/2377389066 wait complete. 2024-03-26T08:46:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.140+0000 7febd0226740 1 -- start start 2024-03-26T08:46:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febd0226740 1 Processor -- start 2024-03-26T08:46:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febd0226740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f61240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febd0226740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 0x55a6a6f61780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febd0226740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f58480 0x55a6a6fd3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 0x55a6a6f61780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f61240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febbe7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 0x55a6a6f61780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55746/0 (socket says 172.21.15.7:55746) 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f61240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59676/0 (socket says 172.21.15.7:59676) 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febbe7fc640 1 -- 172.21.15.7:0/4146926541 learned_addr learned my addr 172.21.15.7:0/4146926541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba800cb00 con 0x55a6a6f53310 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba800de20 con 0x55a6a6f58480 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.141+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba800d160 con 0x55a6a6f52920 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febd0226740 10 client.0 _ll_register_callbacks cb 0x7febb809b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febc77fe640 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f58480 0x55a6a6fd3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febbe7fc640 1 -- 172.21.15.7:0/4146926541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 msgr2=0x55a6a6f61240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febbe7fc640 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f61240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febbe7fc640 1 -- 172.21.15.7:0/4146926541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f58480 msgr2=0x55a6a6fd3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febbe7fc640 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f58480 0x55a6a6fd3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febbe7fc640 1 -- 172.21.15.7:0/4146926541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6a6fd3fd0 con 0x55a6a6f53310 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febc6ffd640 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febbe7fc640 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 0x55a6a6f61780 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7febb0004700 tx=0x7febb001a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43626ebed920ca48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febae7fc640 1 -- 172.21.15.7:0/4146926541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0010630 con 0x55a6a6f53310 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.142+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febb80a6030 con 0x55a6a6f53310 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.143+0000 7febae7fc640 1 -- 172.21.15.7:0/4146926541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7febb00107d0 con 0x55a6a6f53310 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.143+0000 7febae7fc640 1 -- 172.21.15.7:0/4146926541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7febb002a050 con 0x55a6a6f53310 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.143+0000 7febd0226740 10 client.16557 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.143+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a6a6fd53c0 con 0x55a6a6f53310 2024-03-26T08:46:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.144+0000 7febae7fc640 1 -- 172.21.15.7:0/4146926541 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febb0023410 con 0x55a6a6f53310 2024-03-26T08:46:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.145+0000 7febae7fc640 1 -- 172.21.15.7:0/4146926541 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7febb0022380 con 0x55a6a6f53310 2024-03-26T08:46:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.145+0000 7febd0226740 10 client.16557 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.145+0000 7febd0226740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.145+0000 7febd0226740 20 client.16557 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.145+0000 7febd0226740 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a6a6fd5d20 0x55a6a6fd8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.145+0000 7febd0226740 4 client.16557 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:46:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.146+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7febb80a6030 con 0x55a6a6fd5d20 2024-03-26T08:46:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.153+0000 7febc6ffd640 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a6a6fd5d20 0x55a6a6fd8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.163+0000 7febc6ffd640 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a6a6fd5d20 0x55a6a6fd8140 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cec57be6d3889f2 server_cookie=932d965c5f212ec0 in_seq=0 out_seq=0 2024-03-26T08:46:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.163+0000 7febae7fc640 10 client.16557 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.164+0000 7f7f3c635740 1 Processor -- start 2024-03-26T08:46:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.164+0000 7f7f3c635740 1 -- start start 2024-03-26T08:46:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.164+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b3c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.164+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 0x5645b2b5bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.164+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 0x5645b2b53d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.164+0000 7f7f3c635740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341a8ba0 con 0x5645b2b4a340 2024-03-26T08:46:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.164+0000 7f7f3c635740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341a66a0 con 0x5645b2b4fea0 2024-03-26T08:46:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.164+0000 7f7f3c635740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341a4740 con 0x5645b2b4ad30 2024-03-26T08:46:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.165+0000 7f7f327fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 0x5645b2b5bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.165+0000 7f7f327fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 0x5645b2b5bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59682/0 (socket says 172.21.15.7:59682) 2024-03-26T08:46:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.165+0000 7f7f327fc640 1 -- 172.21.15.7:0/812533755 learned_addr learned my addr 172.21.15.7:0/812533755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.165+0000 7f7f337fe640 1 --2- 172.21.15.7:0/812533755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 0x5645b2b53d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.166+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/812533755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b3c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.166+0000 7f7f337fe640 1 -- 172.21.15.7:0/812533755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 msgr2=0x5645b2b5bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.166+0000 7f7f337fe640 1 --2- 172.21.15.7:0/812533755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 0x5645b2b5bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.166+0000 7f7f337fe640 1 -- 172.21.15.7:0/812533755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 msgr2=0x5645b2b3c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.166+0000 7f7f337fe640 1 --2- 172.21.15.7:0/812533755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b3c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.166+0000 7f7f337fe640 1 -- 172.21.15.7:0/812533755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645b2b62b40 con 0x5645b2b4fea0 2024-03-26T08:46:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.167+0000 7f7f337fe640 1 --2- 172.21.15.7:0/812533755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 0x5645b2b53d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c018040 tx=0x7f7f2c023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=241563596008fee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.167+0000 7f7f197fa640 1 -- 172.21.15.7:0/812533755 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c030070 con 0x5645b2b4fea0 2024-03-26T08:46:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.167+0000 7f7f197fa640 1 -- 172.21.15.7:0/812533755 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c015bf0 con 0x5645b2b4fea0 2024-03-26T08:46:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.167+0000 7f7f197fa640 1 -- 172.21.15.7:0/812533755 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c026040 con 0x5645b2b4fea0 2024-03-26T08:46:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.168+0000 7f7f3c635740 1 -- 172.21.15.7:0/812533755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 msgr2=0x5645b2b53d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.168+0000 7f7f3c635740 1 --2- 172.21.15.7:0/812533755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 0x5645b2b53d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c018040 tx=0x7f7f2c023000 comp rx=0 tx=0).stop 2024-03-26T08:46:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.168+0000 7f7f3c635740 1 -- 172.21.15.7:0/812533755 shutdown_connections 2024-03-26T08:46:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.168+0000 7f7f3c635740 1 --2- 172.21.15.7:0/812533755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4fea0 0x5645b2b53d40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.168+0000 7f7f3c635740 1 --2- 172.21.15.7:0/812533755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4ad30 0x5645b2b5bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.168+0000 7f7f3c635740 1 --2- 172.21.15.7:0/812533755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b3c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.168+0000 7f7f3c635740 1 -- 172.21.15.7:0/812533755 >> 172.21.15.7:0/812533755 conn(0x5645b2b0a1a0 msgr2=0x5645b2b3bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.169+0000 7f7f3c635740 1 -- 172.21.15.7:0/812533755 shutdown_connections 2024-03-26T08:46:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.169+0000 7f7f3c635740 1 -- 172.21.15.7:0/812533755 wait complete. 2024-03-26T08:46:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.169+0000 7f7f3c635740 1 -- start start 2024-03-26T08:46:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f3c635740 1 Processor -- start 2024-03-26T08:46:40.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b57c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4ad30 0x5645b2b58160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f3c635740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 0x5645b2b5b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f3c635740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341a4d40 con 0x5645b2b4a340 2024-03-26T08:46:40.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f3c635740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341a6040 con 0x5645b2b4ad30 2024-03-26T08:46:40.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f3c635740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f341a5380 con 0x5645b2b4fea0 2024-03-26T08:46:40.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f3c635740 10 client.0 _ll_register_callbacks cb 0x7f7f240b9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:40.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f32ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b57c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f32ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b57c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55762/0 (socket says 172.21.15.7:55762) 2024-03-26T08:46:40.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f32ffd640 1 -- 172.21.15.7:0/2496751849 learned_addr learned my addr 172.21.15.7:0/2496751849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f327fc640 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4ad30 0x5645b2b58160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.170+0000 7f7f337fe640 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 0x5645b2b5b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.171+0000 7f7f32ffd640 1 -- 172.21.15.7:0/2496751849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 msgr2=0x5645b2b5b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.171+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 0x5645b2b5b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.171+0000 7f7f32ffd640 1 -- 172.21.15.7:0/2496751849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4ad30 msgr2=0x5645b2b58160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.171+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4ad30 0x5645b2b58160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.171+0000 7f7f32ffd640 1 -- 172.21.15.7:0/2496751849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645b277b170 con 0x5645b2b4a340 2024-03-26T08:46:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.171+0000 7f7f337fe640 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 0x5645b2b5b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.172+0000 7f7f32ffd640 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b57c20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7f240bcf00 tx=0x7f7f240bd680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62989298131fd551 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.172+0000 7f7f12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.173+0000 7f7f12ffd640 1 -- 172.21.15.7:0/2496751849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f240bd190 con 0x5645b2b4a340 2024-03-26T08:46:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.173+0000 7f7f12ffd640 1 -- 172.21.15.7:0/2496751849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f240bdc70 con 0x5645b2b4a340 2024-03-26T08:46:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.173+0000 7f7f3c635740 1 -- 172.21.15.7:0/2496751849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f2c015bf0 con 0x5645b2b4a340 2024-03-26T08:46:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.174+0000 7f7f12ffd640 1 -- 172.21.15.7:0/2496751849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f240bd370 con 0x5645b2b4a340 2024-03-26T08:46:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.174+0000 7f7f3c635740 10 client.16563 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.174+0000 7f7f3c635740 1 -- 172.21.15.7:0/2496751849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5645b2b3d570 con 0x5645b2b4a340 2024-03-26T08:46:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.176+0000 7f7f12ffd640 1 -- 172.21.15.7:0/2496751849 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f2c015bf0 con 0x5645b2b4a340 2024-03-26T08:46:40.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.176+0000 7f7f12ffd640 1 -- 172.21.15.7:0/2496751849 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7f240b9760 con 0x5645b2b4a340 2024-03-26T08:46:40.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.177+0000 7f7f3c635740 10 client.16563 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:40.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.177+0000 7f7f3c635740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:40.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.177+0000 7f7f3c635740 20 client.16563 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:40.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.177+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5645b2b3dde0 0x5645b2b40200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.177+0000 7f7f3c635740 4 client.16563 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:46:40.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.177+0000 7f7f3c635740 1 -- 172.21.15.7:0/2496751849 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5645b2aae900 con 0x5645b2b3dde0 2024-03-26T08:46:40.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.178+0000 7f7f327fc640 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5645b2b3dde0 0x5645b2b40200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.179+0000 7f7f12ffd640 10 client.16563 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.179+0000 7f7f327fc640 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5645b2b3dde0 0x5645b2b40200 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f349e2f923e04d server_cookie=d3294ec3ad423819 in_seq=0 out_seq=0 2024-03-26T08:46:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febae7fc640 1 -- 172.21.15.7:0/4146926541 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7febb80a6030 con 0x55a6a6fd5d20 2024-03-26T08:46:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febae7fc640 10 client.16557 handle_command_reply: tid=0 2024-03-26T08:46:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febd0226740 1 client.16557 shutdown 2024-03-26T08:46:40.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 msgr2=0x55a6a6f61780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febd0226740 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 0x55a6a6f61780 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7febb0004700 tx=0x7febb001a9d0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.193+0000 7f7f12ffd640 1 -- 172.21.15.7:0/2496751849 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5645b2aae900 con 0x5645b2b3dde0 2024-03-26T08:46:40.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.193+0000 7f7f12ffd640 10 client.16563 handle_command_reply: tid=0 2024-03-26T08:46:40.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.193+0000 7f7f3c635740 1 client.16563 shutdown 2024-03-26T08:46:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 shutdown_connections 2024-03-26T08:46:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febd0226740 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a6a6fd5d20 0x55a6a6fd8140 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febd0226740 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6a6f58480 0x55a6a6fd3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febd0226740 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6a6f53310 0x55a6a6f61780 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febd0226740 1 --2- 172.21.15.7:0/4146926541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a6a6f52920 0x55a6a6f61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 >> 172.21.15.7:0/4146926541 conn(0x55a6a6f4de90 msgr2=0x55a6a6f4e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.193+0000 7febae7fc640 0 client.16557 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 shutdown_connections 2024-03-26T08:46:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febd0226740 1 -- 172.21.15.7:0/4146926541 wait complete. 2024-03-26T08:46:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 -- 172.21.15.7:0/2496751849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 msgr2=0x5645b2b57c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b57c20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7f240bcf00 tx=0x7f7f240bd680 comp rx=0 tx=0).stop 2024-03-26T08:46:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 -- 172.21.15.7:0/2496751849 shutdown_connections 2024-03-26T08:46:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5645b2b3dde0 0x5645b2b40200 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febd0226740 20 client.16557 trim_cache size 0 max 16384 2024-03-26T08:46:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5645b2b4fea0 0x5645b2b5b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645b2b4ad30 0x5645b2b58160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 --2- 172.21.15.7:0/2496751849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645b2b4a340 0x5645b2b57c20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 -- 172.21.15.7:0/2496751849 >> 172.21.15.7:0/2496751849 conn(0x5645b2b0a1a0 msgr2=0x5645b2b3c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f12ffd640 0 client.16563 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 -- 172.21.15.7:0/2496751849 shutdown_connections 2024-03-26T08:46:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:46:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:46:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:46:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:46:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 1 -- 172.21.15.7:0/2496751849 wait complete. 2024-03-26T08:46:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.194+0000 7f7f3c635740 20 client.16563 trim_cache size 0 max 16384 2024-03-26T08:46:40.190 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:46:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:46:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "d891ad4f-8e60-45dd-bd55-438a06515d32", 2024-03-26T08:46:40.190 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:46:40.190 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 -- 172.21.15.7:0/4000300818 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febb40a29c0 msgr2=0x7febb40a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febb40a29c0 0x7febb40a4e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7febc003b000 tx=0x7febc0047000 comp rx=0 tx=0).stop 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 -- 172.21.15.7:0/4000300818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc81005d0 msgr2=0x7febc81a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc81005d0 0x7febc81a0740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7febb800e9c0 tx=0x7febb800ee80 comp rx=0 tx=0).stop 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 -- 172.21.15.7:0/4000300818 shutdown_connections 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febb40a29c0 0x7febb40a4e80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febc8102180 0x7febc81a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc81017c0 0x7febc81a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 --2- 172.21.15.7:0/4000300818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febc81005d0 0x7febc81a0740 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.194+0000 7febcd636640 1 -- 172.21.15.7:0/4000300818 >> 172.21.15.7:0/4000300818 conn(0x7febc80fbb40 msgr2=0x7febc80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 -- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f1c0a2880 msgr2=0x7f7f1c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.195+0000 7febcd636640 1 -- 172.21.15.7:0/4000300818 shutdown_connections 2024-03-26T08:46:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f1c0a2880 0x7f7f1c0a4d40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c01f000 tx=0x7f7f2c022000 comp rx=0 tx=0).stop 2024-03-26T08:46:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 -- 172.21.15.7:0/212020020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f340ffb60 msgr2=0x7f7f34103aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:40.195+0000 7febcd636640 1 -- 172.21.15.7:0/4000300818 wait complete. 2024-03-26T08:46:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f340ffb60 0x7f7f34103aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f2400a340 tx=0x7f7f24005300 comp rx=0 tx=0).stop 2024-03-26T08:46:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 -- 172.21.15.7:0/212020020 shutdown_connections 2024-03-26T08:46:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f1c0a2880 0x7f7f1c0a4d40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f34101710 0x7f7f34102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f34100d50 0x7f7f34102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 --2- 172.21.15.7:0/212020020 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f340ffb60 0x7f7f34103aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 -- 172.21.15.7:0/212020020 >> 172.21.15.7:0/212020020 conn(0x7f7f340fb0d0 msgr2=0x7f7f34106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 -- 172.21.15.7:0/212020020 shutdown_connections 2024-03-26T08:46:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.195+0000 7f7f33fff640 1 -- 172.21.15.7:0/212020020 wait complete. 2024-03-26T08:46:40.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:46:40.201 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:46:40.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.399+0000 7fbc75bab640 1 -- 172.21.15.7:0/1332023516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc701086a0 msgr2=0x7fbc7010aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.399+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1332023516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc701086a0 0x7fbc7010aad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009860 tx=0x7fbc5802bbb0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.399+0000 7fbc75bab640 1 -- 172.21.15.7:0/1332023516 shutdown_connections 2024-03-26T08:46:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.399+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1332023516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc7010d9c0 0x7fbc7010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.399+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1332023516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc7010b010 0x7fbc7010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.399+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1332023516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc701086a0 0x7fbc7010aad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.399+0000 7fbc75bab640 1 -- 172.21.15.7:0/1332023516 >> 172.21.15.7:0/1332023516 conn(0x7fbc7006e640 msgr2=0x7fbc70070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.400+0000 7fbc75bab640 1 -- 172.21.15.7:0/1332023516 shutdown_connections 2024-03-26T08:46:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.400+0000 7fbc75bab640 1 -- 172.21.15.7:0/1332023516 wait complete. 2024-03-26T08:46:40.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.401+0000 7fbc75bab640 1 Processor -- start 2024-03-26T08:46:40.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.402+0000 7fbc75bab640 1 -- start start 2024-03-26T08:46:40.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.402+0000 7fbc75bab640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc7010b010 0x7fbc7019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.402+0000 7fbc75bab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc7010d9c0 0x7fbc7019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.402+0000 7fbc75bab640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc701a36f0 0x7fbc701a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.402+0000 7fbc75bab640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70111950 con 0x7fbc701a36f0 2024-03-26T08:46:40.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.402+0000 7fbc75bab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701117d0 con 0x7fbc7010d9c0 2024-03-26T08:46:40.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.402+0000 7fbc75bab640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70111ad0 con 0x7fbc7010b010 2024-03-26T08:46:40.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.403+0000 7fbc6ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc701a36f0 0x7fbc701a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.403+0000 7fbc6ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc701a36f0 0x7fbc701a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55780/0 (socket says 172.21.15.7:55780) 2024-03-26T08:46:40.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.403+0000 7fbc6ffff640 1 -- 172.21.15.7:0/1432119177 learned_addr learned my addr 172.21.15.7:0/1432119177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.404+0000 7fbc6ffff640 1 -- 172.21.15.7:0/1432119177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc7010b010 msgr2=0x7fbc7019e1a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:46:40.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.404+0000 7fbc6ffff640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc7010b010 0x7fbc7019e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.404+0000 7fbc6ffff640 1 -- 172.21.15.7:0/1432119177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc7010d9c0 msgr2=0x7fbc7019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.404+0000 7fbc6ffff640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc7010d9c0 0x7fbc7019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.404+0000 7fbc6ffff640 1 -- 172.21.15.7:0/1432119177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc701a6150 con 0x7fbc701a36f0 2024-03-26T08:46:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.404+0000 7fbc6ffff640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc701a36f0 0x7fbc701a5ae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbc6801e920 tx=0x7fbc6801ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.405+0000 7fbc6cff9640 1 -- 172.21.15.7:0/1432119177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68028070 con 0x7fbc701a36f0 2024-03-26T08:46:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.405+0000 7fbc6cff9640 1 -- 172.21.15.7:0/1432119177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc68005020 con 0x7fbc701a36f0 2024-03-26T08:46:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.405+0000 7fbc6cff9640 1 -- 172.21.15.7:0/1432119177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68004d20 con 0x7fbc701a36f0 2024-03-26T08:46:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.405+0000 7fbc75bab640 1 -- 172.21.15.7:0/1432119177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc701a6440 con 0x7fbc701a36f0 2024-03-26T08:46:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.405+0000 7fbc75bab640 1 -- 172.21.15.7:0/1432119177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc701a6940 con 0x7fbc701a36f0 2024-03-26T08:46:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.409+0000 7fbc7878a740 1 Processor -- start 2024-03-26T08:46:40.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.409+0000 7fbc6cff9640 1 -- 172.21.15.7:0/1432119177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fbc680051d0 con 0x7fbc701a36f0 2024-03-26T08:46:40.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.410+0000 7fbc6cff9640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbc5c0a2a10 0x7fbc5c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.410+0000 7fbc7878a740 1 -- start start 2024-03-26T08:46:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.410+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d35da0 0x55e193d361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.410+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d36790 0x55e193d3b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.410+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d3b900 0x55e193d3dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.410+0000 7fbc7878a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e193c9b3d0 con 0x55e193d36790 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.410+0000 7fbc7878a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e193c43ad0 con 0x55e193d3b900 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.410+0000 7fbc7878a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e193c97470 con 0x55e193d35da0 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d35da0 0x55e193d361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d35da0 0x55e193d361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59720/0 (socket says 172.21.15.7:59720) 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/4217198098 learned_addr learned my addr 172.21.15.7:0/4217198098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6cff9640 1 -- 172.21.15.7:0/1432119177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc680df440 con 0x7fbc701a36f0 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/4217198098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d3b900 msgr2=0x55e193d3dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6ffff640 1 --2- 172.21.15.7:0/4217198098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d3b900 0x55e193d3dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6f7fe640 1 --2- 172.21.15.7:0/4217198098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d3b900 0x55e193d3dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/4217198098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d36790 msgr2=0x55e193d3b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6f7fe640 1 --2- 172.21.15.7:0/4217198098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d36790 0x55e193d3b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/4217198098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e193a797d0 con 0x55e193d35da0 2024-03-26T08:46:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6effd640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbc5c0a2a10 0x7fbc5c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.412+0000 7fbc6f7fe640 1 --2- 172.21.15.7:0/4217198098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d35da0 0x55e193d361c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc58037040 tx=0x7fbc5802c5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c104b046b1925e5b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.413+0000 7fbc65ffb640 1 -- 172.21.15.7:0/4217198098 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58039040 con 0x55e193d35da0 2024-03-26T08:46:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.413+0000 7fbc65ffb640 1 -- 172.21.15.7:0/4217198098 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc58034b50 con 0x55e193d35da0 2024-03-26T08:46:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.413+0000 7fbc65ffb640 1 -- 172.21.15.7:0/4217198098 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5802ca50 con 0x55e193d35da0 2024-03-26T08:46:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 -- 172.21.15.7:0/4217198098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d35da0 msgr2=0x55e193d361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc6effd640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbc5c0a2a10 0x7fbc5c0a4ed0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009270 tx=0x7fbc600091e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 --2- 172.21.15.7:0/4217198098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d35da0 0x55e193d361c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc58037040 tx=0x7fbc5802c5e0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 -- 172.21.15.7:0/4217198098 shutdown_connections 2024-03-26T08:46:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 --2- 172.21.15.7:0/4217198098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d3b900 0x55e193d3dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 --2- 172.21.15.7:0/4217198098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d36790 0x55e193d3b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 --2- 172.21.15.7:0/4217198098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d35da0 0x55e193d361c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 -- 172.21.15.7:0/4217198098 >> 172.21.15.7:0/4217198098 conn(0x55e193d31310 msgr2=0x55e193d31710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 -- 172.21.15.7:0/4217198098 shutdown_connections 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 -- 172.21.15.7:0/4217198098 wait complete. 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.414+0000 7fbc7878a740 1 -- start start 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc7878a740 1 Processor -- start 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d40040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d40580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d3b900 0x55e193d47550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc6effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d40580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc7878a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e193d3e850 con 0x55e193d3b900 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc7878a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e193c57060 con 0x55e193d35da0 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc7878a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e193d3e9d0 con 0x55e193d36790 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc6effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d40580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59724/0 (socket says 172.21.15.7:59724) 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d40040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc6effd640 1 -- 172.21.15.7:0/1239420248 learned_addr learned my addr 172.21.15.7:0/1239420248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.415+0000 7fbc7878a740 10 client.0 _ll_register_callbacks cb 0x55e193c396a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.416+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/1239420248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 msgr2=0x55e193d40580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.416+0000 7fbc6f7fe640 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d40580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.416+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/1239420248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d3b900 msgr2=0x55e193d47550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.416+0000 7fbc6f7fe640 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d3b900 0x55e193d47550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.416+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/1239420248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1939af3f0 con 0x55e193d35da0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.416+0000 7fbc6f7fe640 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d40040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbc5802c1c0 tx=0x7fbc58034440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e71e51815292a0f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.416+0000 7fbc4effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.416+0000 7fbc7878a740 1 -- 172.21.15.7:0/1239420248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc58034b50 con 0x55e193d35da0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc4effd640 1 -- 172.21.15.7:0/1239420248 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58044020 con 0x55e193d35da0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc4effd640 1 -- 172.21.15.7:0/1239420248 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc58005450 con 0x55e193d35da0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc4effd640 1 -- 172.21.15.7:0/1239420248 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc58039040 con 0x55e193d35da0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc7878a740 10 client.26405 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc7878a740 1 -- 172.21.15.7:0/1239420248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55e19396f0f0 con 0x55e193d35da0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc4effd640 1 -- 172.21.15.7:0/1239420248 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbc5803c070 con 0x55e193d35da0 2024-03-26T08:46:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc7878a740 10 client.26405 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc7878a740 10 client.26405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc7878a740 20 client.26405 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e193d4b570 0x55e193d4d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc7878a740 4 client.26405 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.417+0000 7fbc7878a740 1 -- 172.21.15.7:0/1239420248 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc58034b50 con 0x55e193d4b570 2024-03-26T08:46:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.418+0000 7fbc6effd640 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e193d4b570 0x55e193d4d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.418+0000 7fbc4effd640 10 client.26405 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.418+0000 7fbc6effd640 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e193d4b570 0x55e193d4d960 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26128f0f8092f355 server_cookie=d0825a0df9c6986a in_seq=0 out_seq=0 2024-03-26T08:46:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.420+0000 7fbc4effd640 1 -- 172.21.15.7:0/1239420248 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fbc58034b50 con 0x55e193d4b570 2024-03-26T08:46:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.420+0000 7fbc4effd640 10 client.26405 handle_command_reply: tid=0 2024-03-26T08:46:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.420+0000 7fbc7878a740 1 client.26405 shutdown 2024-03-26T08:46:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.420+0000 7fbc7878a740 1 -- 172.21.15.7:0/1239420248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 msgr2=0x55e193d40040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.421+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d40040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbc5802c1c0 tx=0x7fbc58034440 comp rx=0 tx=0).stop 2024-03-26T08:46:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.421+0000 7fbc7878a740 1 -- 172.21.15.7:0/1239420248 shutdown_connections 2024-03-26T08:46:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.421+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e193d4b570 0x55e193d4d960 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.421+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d3b900 0x55e193d47550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.421+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d40580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.421+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1239420248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d40040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.421+0000 7fbc7878a740 1 -- 172.21.15.7:0/1239420248 >> 172.21.15.7:0/1239420248 conn(0x55e193d31310 msgr2=0x55e193d30530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.421+0000 7fbc4effd640 0 client.26405 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.422+0000 7fbc7878a740 1 -- 172.21.15.7:0/1239420248 shutdown_connections 2024-03-26T08:46:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.422+0000 7fbc7878a740 1 -- 172.21.15.7:0/1239420248 wait complete. 2024-03-26T08:46:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.422+0000 7fbc7878a740 20 client.26405 trim_cache size 0 max 16384 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.446+0000 7fbc7878a740 1 Processor -- start 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc7878a740 1 -- start start 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d52040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d47470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d52040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc6f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d52040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51008/0 (socket says 172.21.15.7:51008) 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d3b900 0x55e193d52580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/130834012 learned_addr learned my addr 172.21.15.7:0/130834012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc7878a740 1 -- 172.21.15.7:0/130834012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701b1a50 con 0x55e193d3b900 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc7878a740 1 -- 172.21.15.7:0/130834012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701af550 con 0x55e193d35da0 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc7878a740 1 -- 172.21.15.7:0/130834012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701ad5d0 con 0x55e193d36790 2024-03-26T08:46:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.447+0000 7fbc6effd640 1 --2- 172.21.15.7:0/130834012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d47470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.448+0000 7fbc6ffff640 1 --2- 172.21.15.7:0/130834012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d3b900 0x55e193d52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.448+0000 7fbc6effd640 1 -- 172.21.15.7:0/130834012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 msgr2=0x55e193d52040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.448+0000 7fbc6effd640 1 --2- 172.21.15.7:0/130834012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d52040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.448+0000 7fbc6effd640 1 -- 172.21.15.7:0/130834012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d3b900 msgr2=0x55e193d52580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.448+0000 7fbc6effd640 1 --2- 172.21.15.7:0/130834012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d3b900 0x55e193d52580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.448+0000 7fbc6effd640 1 -- 172.21.15.7:0/130834012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e193d36b70 con 0x55e193d36790 2024-03-26T08:46:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.448+0000 7fbc6effd640 1 --2- 172.21.15.7:0/130834012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d47470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55e193d4df90 tx=0x7fbc60009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd0ceb19c20e1938 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.449+0000 7fbc657fa640 1 -- 172.21.15.7:0/130834012 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc600108a0 con 0x55e193d36790 2024-03-26T08:46:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.449+0000 7fbc657fa640 1 -- 172.21.15.7:0/130834012 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc6001d430 con 0x55e193d36790 2024-03-26T08:46:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.449+0000 7fbc657fa640 1 -- 172.21.15.7:0/130834012 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6001c6b0 con 0x55e193d36790 2024-03-26T08:46:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.450+0000 7fbc7878a740 1 -- 172.21.15.7:0/130834012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 msgr2=0x55e193d47470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.450+0000 7fbc7878a740 1 --2- 172.21.15.7:0/130834012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d47470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55e193d4df90 tx=0x7fbc60009f90 comp rx=0 tx=0).stop 2024-03-26T08:46:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.450+0000 7fbc7878a740 1 -- 172.21.15.7:0/130834012 shutdown_connections 2024-03-26T08:46:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.450+0000 7fbc7878a740 1 --2- 172.21.15.7:0/130834012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d3b900 0x55e193d52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.450+0000 7fbc7878a740 1 --2- 172.21.15.7:0/130834012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d36790 0x55e193d47470 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.450+0000 7fbc7878a740 1 --2- 172.21.15.7:0/130834012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d35da0 0x55e193d52040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.450+0000 7fbc7878a740 1 -- 172.21.15.7:0/130834012 >> 172.21.15.7:0/130834012 conn(0x55e193d31310 msgr2=0x55e193d34e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.450+0000 7fbc7878a740 1 -- 172.21.15.7:0/130834012 shutdown_connections 2024-03-26T08:46:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.450+0000 7fbc7878a740 1 -- 172.21.15.7:0/130834012 wait complete. 2024-03-26T08:46:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc7878a740 1 -- start start 2024-03-26T08:46:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc7878a740 1 Processor -- start 2024-03-26T08:46:40.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d35da0 0x55e193d45680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d36790 0x55e193d45bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc7878a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d3b900 0x55e193d46a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc7878a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701adbd0 con 0x55e193d35da0 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc7878a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701aeef0 con 0x55e193d36790 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc7878a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701ae230 con 0x55e193d3b900 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc6f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d35da0 0x55e193d45680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc7878a740 10 client.0 _ll_register_callbacks cb 0x7fbc600093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc6f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d35da0 0x55e193d45680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55818/0 (socket says 172.21.15.7:55818) 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/1728978219 learned_addr learned my addr 172.21.15.7:0/1728978219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc6effd640 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d36790 0x55e193d45bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.451+0000 7fbc6ffff640 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d3b900 0x55e193d46a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/1728978219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d3b900 msgr2=0x55e193d46a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc6f7fe640 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d3b900 0x55e193d46a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/1728978219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d36790 msgr2=0x55e193d45bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc6f7fe640 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d36790 0x55e193d45bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc6f7fe640 1 -- 172.21.15.7:0/1728978219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e193d470b0 con 0x55e193d35da0 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc6effd640 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d36790 0x55e193d45bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc6f7fe640 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d35da0 0x55e193d45680 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbc5802c1c0 tx=0x7fbc58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a80a5a71df5062b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc4effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc4effd640 1 -- 172.21.15.7:0/1728978219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58057060 con 0x55e193d35da0 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc4effd640 1 -- 172.21.15.7:0/1728978219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc5802c9f0 con 0x55e193d35da0 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc7878a740 1 -- 172.21.15.7:0/1728978219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc6001d430 con 0x55e193d35da0 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.452+0000 7fbc4effd640 1 -- 172.21.15.7:0/1728978219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc58044040 con 0x55e193d35da0 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.453+0000 7fbc7878a740 10 client.16575 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.453+0000 7fbc7878a740 1 -- 172.21.15.7:0/1728978219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e193da75a0 con 0x55e193d35da0 2024-03-26T08:46:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.454+0000 7fbc4effd640 1 -- 172.21.15.7:0/1728978219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc6001d430 con 0x55e193d35da0 2024-03-26T08:46:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.454+0000 7fbc4effd640 1 -- 172.21.15.7:0/1728978219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbc5803c070 con 0x55e193d35da0 2024-03-26T08:46:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.454+0000 7fbc7878a740 10 client.16575 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.454+0000 7fbc7878a740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.454+0000 7fbc7878a740 20 client.16575 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.454+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e193da7df0 0x55e193daa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:40.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.454+0000 7fbc7878a740 4 client.16575 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.454+0000 7fbc7878a740 1 -- 172.21.15.7:0/1728978219 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e193ca8910 con 0x55e193da7df0 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.454+0000 7fbc6effd640 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e193da7df0 0x55e193daa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc6effd640 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e193da7df0 0x55e193daa1e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e75e426a1d6565c server_cookie=f42515908cc4f9d6 in_seq=0 out_seq=0 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc4effd640 10 client.16575 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc4effd640 1 -- 172.21.15.7:0/1728978219 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e193ca8910 con 0x55e193da7df0 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc4effd640 10 client.16575 handle_command_reply: tid=0 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 client.16575 shutdown 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 -- 172.21.15.7:0/1728978219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d35da0 msgr2=0x55e193d45680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d35da0 0x55e193d45680 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbc5802c1c0 tx=0x7fbc58000bc0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 -- 172.21.15.7:0/1728978219 shutdown_connections 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e193da7df0 0x55e193daa1e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e193d3b900 0x55e193d46a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e193d36790 0x55e193d45bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc4effd640 0 client.16575 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:40.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 --2- 172.21.15.7:0/1728978219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e193d35da0 0x55e193d45680 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 -- 172.21.15.7:0/1728978219 >> 172.21.15.7:0/1728978219 conn(0x55e193d31310 msgr2=0x55e193d2fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 -- 172.21.15.7:0/1728978219 shutdown_connections 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.455+0000 7fbc7878a740 1 -- 172.21.15.7:0/1728978219 wait complete. 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc7878a740 20 client.16575 trim_cache size 0 max 16384 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "d891ad4f-8e60-45dd-bd55-438a06515d32": { 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "d891ad4f-8e60-45dd-bd55-438a06515d32", 2024-03-26T08:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:46:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 -- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbc5c0a2a10 msgr2=0x7fbc5c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbc5c0a2a10 0x7fbc5c0a4ed0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009270 tx=0x7fbc600091e0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 -- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc701a36f0 msgr2=0x7fbc701a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc701a36f0 0x7fbc701a5ae0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbc6801e920 tx=0x7fbc6801ede0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 -- 172.21.15.7:0/1432119177 shutdown_connections 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbc5c0a2a10 0x7fbc5c0a4ed0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc701a36f0 0x7fbc701a5ae0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc7010d9c0 0x7fbc7019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 --2- 172.21.15.7:0/1432119177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc7010b010 0x7fbc7019e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.456+0000 7fbc75bab640 1 -- 172.21.15.7:0/1432119177 >> 172.21.15.7:0/1432119177 conn(0x7fbc7006e640 msgr2=0x7fbc70108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.457+0000 7fbc75bab640 1 -- 172.21.15.7:0/1432119177 shutdown_connections 2024-03-26T08:46:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:46:40.457+0000 7fbc75bab640 1 -- 172.21.15.7:0/1432119177 wait complete. 2024-03-26T08:46:40.462 INFO:tasks.cephfs.filesystem:scrub status for tag:d891ad4f-8e60-45dd-bd55-438a06515d32 - {'path': '/', 'tag': 'd891ad4f-8e60-45dd-bd55-438a06515d32', 'options': 'recursive,force'} 2024-03-26T08:46:40.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:40 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:46:40.542 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:40 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:40.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:40 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:46:40.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:40 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:40.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:40 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:46:40.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:40 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:41.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 250.0 s, 2856.4 tps, lat 34.943 ms stddev 37.410 2024-03-26T08:46:41.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:41 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:46:41.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:41 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:41.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:41 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:46:41.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:41 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:41.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:41 smithi016 ceph-mon[35851]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 793 op/s 2024-03-26T08:46:41.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:41 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:46:41.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:41 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:41.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:41 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:46:41.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:41 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:41.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:41 smithi007 ceph-mon[32462]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 793 op/s 2024-03-26T08:46:41.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:41 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:46:41.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:41 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:41.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:41 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:46:41.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:41 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:46:41.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:41 smithi044 ceph-mon[36461]: pgmap v488: 97 pgs: 97 active+clean; 1.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 793 op/s 2024-03-26T08:46:43.743 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:43 smithi016 ceph-mon[35851]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.0 MiB/s wr, 782 op/s 2024-03-26T08:46:43.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:43 smithi007 ceph-mon[32462]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.0 MiB/s wr, 782 op/s 2024-03-26T08:46:43.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:43 smithi044 ceph-mon[36461]: pgmap v489: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.0 MiB/s wr, 782 op/s 2024-03-26T08:46:45.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:45 smithi016 ceph-mon[35851]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.3 MiB/s wr, 865 op/s 2024-03-26T08:46:45.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:45 smithi007 ceph-mon[32462]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.3 MiB/s wr, 865 op/s 2024-03-26T08:46:45.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:45 smithi044 ceph-mon[36461]: pgmap v490: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.3 MiB/s wr, 865 op/s 2024-03-26T08:46:46.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 255.0 s, 3536.4 tps, lat 28.288 ms stddev 31.907 2024-03-26T08:46:46.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:46 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:46:46.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:46 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:46:46.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:46 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:46:46.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:46 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:46:46.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:46 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:46:46.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:46 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:46:47.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:47 smithi007 ceph-mon[32462]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.4 MiB/s wr, 822 op/s 2024-03-26T08:46:47.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:47 smithi044 ceph-mon[36461]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.4 MiB/s wr, 822 op/s 2024-03-26T08:46:47.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:47 smithi016 ceph-mon[35851]: pgmap v491: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 8.4 MiB/s wr, 822 op/s 2024-03-26T08:46:49.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:49 smithi007 ceph-mon[32462]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.5 MiB/s wr, 816 op/s 2024-03-26T08:46:49.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:49 smithi044 ceph-mon[36461]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.5 MiB/s wr, 816 op/s 2024-03-26T08:46:49.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:49 smithi016 ceph-mon[35851]: pgmap v492: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.5 MiB/s wr, 816 op/s 2024-03-26T08:46:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 260.0 s, 2769.0 tps, lat 36.241 ms stddev 39.632 2024-03-26T08:46:51.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:51 smithi007 ceph-mon[32462]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.0 MiB/s wr, 784 op/s 2024-03-26T08:46:51.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:51 smithi044 ceph-mon[36461]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.0 MiB/s wr, 784 op/s 2024-03-26T08:46:51.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:51 smithi016 ceph-mon[35851]: pgmap v493: 97 pgs: 97 active+clean; 1.6 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 7.0 MiB/s wr, 784 op/s 2024-03-26T08:46:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.327+0000 7fe0911f3640 1 -- 172.21.15.7:0/2015586491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 msgr2=0x7fe08c110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.327+0000 7fe0911f3640 1 --2- 172.21.15.7:0/2015586491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 0x7fe08c110160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe084009860 tx=0x7fe08402bb90 comp rx=0 tx=0).stop 2024-03-26T08:46:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.327+0000 7fe0911f3640 1 -- 172.21.15.7:0/2015586491 shutdown_connections 2024-03-26T08:46:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.327+0000 7fe0911f3640 1 --2- 172.21.15.7:0/2015586491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 0x7fe08c110160 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.327+0000 7fe0911f3640 1 --2- 172.21.15.7:0/2015586491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe08c108f10 0x7fe08c109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.327+0000 7fe0911f3640 1 --2- 172.21.15.7:0/2015586491 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe08c107d20 0x7fe08c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.327+0000 7fe0911f3640 1 -- 172.21.15.7:0/2015586491 >> 172.21.15.7:0/2015586491 conn(0x7fe08c076cb0 msgr2=0x7fe08c0790f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.327+0000 7fe0911f3640 1 -- 172.21.15.7:0/2015586491 shutdown_connections 2024-03-26T08:46:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.327+0000 7fe0911f3640 1 -- 172.21.15.7:0/2015586491 wait complete. 2024-03-26T08:46:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.328+0000 7fe0911f3640 1 Processor -- start 2024-03-26T08:46:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.328+0000 7fe0911f3640 1 -- start start 2024-03-26T08:46:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.328+0000 7fe0911f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe08c107d20 0x7fe08c06ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.328+0000 7fe0911f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe08c108f10 0x7fe08c0704f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.328+0000 7fe0911f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 0x7fe08c1ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.328+0000 7fe0911f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c113980 con 0x7fe08c107d20 2024-03-26T08:46:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.328+0000 7fe0911f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c113800 con 0x7fe08c1098d0 2024-03-26T08:46:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.328+0000 7fe0911f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c113b00 con 0x7fe08c108f10 2024-03-26T08:46:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.330+0000 7fe08b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 0x7fe08c1ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.330+0000 7fe08b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 0x7fe08c1ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40520/0 (socket says 172.21.15.7:40520) 2024-03-26T08:46:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.330+0000 7fe08b59e640 1 -- 172.21.15.7:0/3505451166 learned_addr learned my addr 172.21.15.7:0/3505451166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.330+0000 7fe08a59c640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe08c108f10 0x7fe08c0704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.330+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe08c107d20 0x7fe08c06ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.331+0000 7fe08b59e640 1 -- 172.21.15.7:0/3505451166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe08c108f10 msgr2=0x7fe08c0704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.331+0000 7fe08b59e640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe08c108f10 0x7fe08c0704f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.331+0000 7fe08b59e640 1 -- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe08c107d20 msgr2=0x7fe08c06ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.331+0000 7fe08b59e640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe08c107d20 0x7fe08c06ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.331+0000 7fe08b59e640 1 -- 172.21.15.7:0/3505451166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c073670 con 0x7fe08c1098d0 2024-03-26T08:46:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.331+0000 7fe08b59e640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 0x7fe08c1ae090 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe0840323d0 tx=0x7fe084004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.331+0000 7fe083fff640 1 -- 172.21.15.7:0/3505451166 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084041020 con 0x7fe08c1098d0 2024-03-26T08:46:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.331+0000 7fe0911f3640 1 -- 172.21.15.7:0/3505451166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c079f50 con 0x7fe08c1098d0 2024-03-26T08:46:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.332+0000 7fe0911f3640 1 -- 172.21.15.7:0/3505451166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c07a460 con 0x7fe08c1098d0 2024-03-26T08:46:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.332+0000 7fe083fff640 1 -- 172.21.15.7:0/3505451166 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe08402ca10 con 0x7fe08c1098d0 2024-03-26T08:46:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.332+0000 7fe083fff640 1 -- 172.21.15.7:0/3505451166 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084039040 con 0x7fe08c1098d0 2024-03-26T08:46:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.335+0000 7fe093dd9740 1 Processor -- start 2024-03-26T08:46:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.335+0000 7fe083fff640 1 -- 172.21.15.7:0/3505451166 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe084044020 con 0x7fe08c1098d0 2024-03-26T08:46:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.335+0000 7fe093dd9740 1 -- start start 2024-03-26T08:46:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba4920 0x557b1aba4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 0x557b1aba9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1abaa480 0x557b1abac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe093dd9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b1ab09b10 con 0x557b1aba4920 2024-03-26T08:46:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe093dd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b1aab2900 con 0x557b1abaa480 2024-03-26T08:46:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe093dd9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b1ab05bb0 con 0x557b1aba5310 2024-03-26T08:46:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe08a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba4920 0x557b1aba4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe08a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba4920 0x557b1aba4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41790/0 (socket says 172.21.15.7:41790) 2024-03-26T08:46:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe08a59c640 1 -- 172.21.15.7:0/3735112146 learned_addr learned my addr 172.21.15.7:0/3735112146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.336+0000 7fe08a59c640 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1abaa480 0x557b1abac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 0x557b1aba9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe08a59c640 1 -- 172.21.15.7:0/3735112146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 msgr2=0x557b1aba9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe08a59c640 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 0x557b1aba9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe08a59c640 1 -- 172.21.15.7:0/3735112146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba4920 msgr2=0x557b1aba4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe08a59c640 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba4920 0x557b1aba4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe08a59c640 1 -- 172.21.15.7:0/3735112146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b1a81e210 con 0x557b1abaa480 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe083fff640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0780a2870 0x7fe0780a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe08a59c640 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1abaa480 0x557b1abac910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x557b1aba5b70 tx=0x7fe07c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1928fc6940c6a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe083fff640 1 -- 172.21.15.7:0/3505451166 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe0840f02a0 con 0x7fe08c1098d0 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 0x557b1aba9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.337+0000 7fe0817fa640 1 -- 172.21.15.7:0/3735112146 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c017040 con 0x557b1abaa480 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.338+0000 7fe0817fa640 1 -- 172.21.15.7:0/3735112146 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe07c00e870 con 0x557b1abaa480 2024-03-26T08:46:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.338+0000 7fe0817fa640 1 -- 172.21.15.7:0/3735112146 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c01d3e0 con 0x557b1abaa480 2024-03-26T08:46:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.338+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0780a2870 0x7fe0780a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.338+0000 7fe093dd9740 1 -- 172.21.15.7:0/3735112146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1abaa480 msgr2=0x557b1abac910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.338+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0780a2870 0x7fe0780a4d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557b1aba5710 tx=0x7fe07400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.338+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1abaa480 0x557b1abac910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x557b1aba5b70 tx=0x7fe07c004ab0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 -- 172.21.15.7:0/3735112146 shutdown_connections 2024-03-26T08:46:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1abaa480 0x557b1abac910 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 0x557b1aba9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3735112146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba4920 0x557b1aba4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 -- 172.21.15.7:0/3735112146 >> 172.21.15.7:0/3735112146 conn(0x557b1ab9fe90 msgr2=0x557b1aba0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 -- 172.21.15.7:0/3735112146 shutdown_connections 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 -- 172.21.15.7:0/3735112146 wait complete. 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 -- start start 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 Processor -- start 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.339+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abaecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba5310 0x557b1abaf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1abaa480 0x557b1abb6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba5310 0x557b1abaf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe093dd9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b1abad4d0 con 0x557b1aba5310 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe093dd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b1aac5d50 con 0x557b1aba4920 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba5310 0x557b1abaf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41800/0 (socket says 172.21.15.7:41800) 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08a59c640 1 -- 172.21.15.7:0/3693002295 learned_addr learned my addr 172.21.15.7:0/3693002295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe093dd9740 1 -- 172.21.15.7:0/3693002295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b1abad650 con 0x557b1abaa480 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abaecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08b59e640 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1abaa480 0x557b1abb6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe093dd9740 10 client.0 _ll_register_callbacks cb 0x557b1aac5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08a59c640 1 -- 172.21.15.7:0/3693002295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1abaa480 msgr2=0x557b1abb6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08a59c640 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1abaa480 0x557b1abb6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08a59c640 1 -- 172.21.15.7:0/3693002295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 msgr2=0x557b1abaecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08a59c640 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abaecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08a59c640 1 -- 172.21.15.7:0/3693002295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b1a8ea620 con 0x557b1aba5310 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe093dd9740 1 -- 172.21.15.7:0/3693002295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe07c00e870 con 0x557b1aba5310 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe08a59c640 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba5310 0x557b1abaf200 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009620 tx=0x7fe07c00de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6e85d0326fd051 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.340+0000 7fe0727fc640 10 client.16587 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.341+0000 7fe0727fc640 1 -- 172.21.15.7:0/3693002295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c004810 con 0x557b1aba5310 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.341+0000 7fe0727fc640 1 -- 172.21.15.7:0/3693002295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe07c003970 con 0x557b1aba5310 2024-03-26T08:46:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.341+0000 7fe0727fc640 1 -- 172.21.15.7:0/3693002295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe07c003b90 con 0x557b1aba5310 2024-03-26T08:46:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.342+0000 7fe093dd9740 10 client.16587 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.342+0000 7fe093dd9740 1 -- 172.21.15.7:0/3693002295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x557b1abb9450 con 0x557b1aba5310 2024-03-26T08:46:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe0727fc640 1 -- 172.21.15.7:0/3693002295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe07c00b040 con 0x557b1aba5310 2024-03-26T08:46:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe093dd9740 10 client.16587 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe093dd9740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe093dd9740 20 client.16587 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557b1abba7f0 0x557b1abbcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe093dd9740 4 client.16587 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe093dd9740 1 -- 172.21.15.7:0/3693002295 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b1ab17050 con 0x557b1abba7f0 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557b1abba7f0 0x557b1abbcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe0727fc640 10 client.16587 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.343+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557b1abba7f0 0x557b1abbcc00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7728b638472ee36 server_cookie=4a43b0c61b07be1 in_seq=0 out_seq=0 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe0727fc640 1 -- 172.21.15.7:0/3693002295 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557b1ab17050 con 0x557b1abba7f0 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe0727fc640 10 client.16587 handle_command_reply: tid=0 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 client.16587 shutdown 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 -- 172.21.15.7:0/3693002295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba5310 msgr2=0x557b1abaf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba5310 0x557b1abaf200 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009620 tx=0x7fe07c00de70 comp rx=0 tx=0).stop 2024-03-26T08:46:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 -- 172.21.15.7:0/3693002295 shutdown_connections 2024-03-26T08:46:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557b1abba7f0 0x557b1abbcc00 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1abaa480 0x557b1abb6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe0727fc640 0 client.16587 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1aba5310 0x557b1abaf200 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 --2- 172.21.15.7:0/3693002295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abaecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 -- 172.21.15.7:0/3693002295 >> 172.21.15.7:0/3693002295 conn(0x557b1ab9fe90 msgr2=0x557b1ab9f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 -- 172.21.15.7:0/3693002295 shutdown_connections 2024-03-26T08:46:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.344+0000 7fe093dd9740 1 -- 172.21.15.7:0/3693002295 wait complete. 2024-03-26T08:46:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.345+0000 7fe093dd9740 20 client.16587 trim_cache size 0 max 16384 2024-03-26T08:46:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe093dd9740 1 Processor -- start 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe093dd9740 1 -- start start 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba4920 0x557b1aba6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba5310 0x557b1aba6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe093dd9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c010970 con 0x557b1abaa480 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe093dd9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c00e470 con 0x557b1aba5310 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe093dd9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c00c4f0 con 0x557b1aba4920 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe08b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe08a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba5310 0x557b1aba6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe08b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41806/0 (socket says 172.21.15.7:41806) 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe08b59e640 1 -- 172.21.15.7:0/1532564984 learned_addr learned my addr 172.21.15.7:0/1532564984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.364+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/1532564984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba4920 0x557b1aba6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe08b59e640 1 -- 172.21.15.7:0/1532564984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba4920 msgr2=0x557b1aba6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe08b59e640 1 --2- 172.21.15.7:0/1532564984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba4920 0x557b1aba6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe08b59e640 1 -- 172.21.15.7:0/1532564984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba5310 msgr2=0x557b1aba6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe08b59e640 1 --2- 172.21.15.7:0/1532564984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba5310 0x557b1aba6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe08b59e640 1 -- 172.21.15.7:0/1532564984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b1aba6120 con 0x557b1abaa480 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe08a59c640 1 --2- 172.21.15.7:0/1532564984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba5310 0x557b1aba6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe08b59e640 1 --2- 172.21.15.7:0/1532564984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb6200 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe0840bcbd0 tx=0x7fe0840bc840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b9f9d0a09ba06b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe080ff9640 1 -- 172.21.15.7:0/1532564984 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084009460 con 0x557b1abaa480 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe080ff9640 1 -- 172.21.15.7:0/1532564984 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0840bd340 con 0x557b1abaa480 2024-03-26T08:46:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.365+0000 7fe080ff9640 1 -- 172.21.15.7:0/1532564984 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0840bad50 con 0x557b1abaa480 2024-03-26T08:46:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.366+0000 7fe093dd9740 1 -- 172.21.15.7:0/1532564984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 msgr2=0x557b1abb6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.366+0000 7fe093dd9740 1 --2- 172.21.15.7:0/1532564984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb6200 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe0840bcbd0 tx=0x7fe0840bc840 comp rx=0 tx=0).stop 2024-03-26T08:46:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.366+0000 7fe093dd9740 1 -- 172.21.15.7:0/1532564984 shutdown_connections 2024-03-26T08:46:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.366+0000 7fe093dd9740 1 --2- 172.21.15.7:0/1532564984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb6200 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.366+0000 7fe093dd9740 1 --2- 172.21.15.7:0/1532564984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba5310 0x557b1aba6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.366+0000 7fe093dd9740 1 --2- 172.21.15.7:0/1532564984 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba4920 0x557b1aba6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.366+0000 7fe093dd9740 1 -- 172.21.15.7:0/1532564984 >> 172.21.15.7:0/1532564984 conn(0x557b1ab9fe90 msgr2=0x557b1aba0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.368+0000 7fe093dd9740 1 -- 172.21.15.7:0/1532564984 shutdown_connections 2024-03-26T08:46:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.368+0000 7fe093dd9740 1 -- 172.21.15.7:0/1532564984 wait complete. 2024-03-26T08:46:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.368+0000 7fe093dd9740 1 -- start start 2024-03-26T08:46:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe093dd9740 1 Processor -- start 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abb3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 0x557b1abb3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe093dd9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe093dd9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c00caf0 con 0x557b1abaa480 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe08ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abb3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe08ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abb3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40534/0 (socket says 172.21.15.7:40534) 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe08ad9d640 1 -- 172.21.15.7:0/2904757141 learned_addr learned my addr 172.21.15.7:0/2904757141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe08a59c640 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 0x557b1abb3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c00de10 con 0x557b1aba4920 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c00d150 con 0x557b1aba5310 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe093dd9740 10 client.0 _ll_register_callbacks cb 0x7fe07c00ec40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:46:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.369+0000 7fe08b59e640 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.370+0000 7fe08ad9d640 1 -- 172.21.15.7:0/2904757141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 msgr2=0x557b1abb3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.370+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 0x557b1abb3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.370+0000 7fe08ad9d640 1 -- 172.21.15.7:0/2904757141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 msgr2=0x557b1abb4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:46:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.370+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.370+0000 7fe08ad9d640 1 -- 172.21.15.7:0/2904757141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b1abb51f0 con 0x557b1aba4920 2024-03-26T08:46:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.371+0000 7fe08ad9d640 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abb3870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x557b1aba0e00 tx=0x7fe074014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649013864ba4e621 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.371+0000 7fe0727fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.371+0000 7fe0727fc640 1 -- 172.21.15.7:0/2904757141 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074028bf0 con 0x557b1aba4920 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.371+0000 7fe0727fc640 1 -- 172.21.15.7:0/2904757141 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe074028d90 con 0x557b1aba4920 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.371+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0840bd340 con 0x557b1aba4920 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.371+0000 7fe0727fc640 1 -- 172.21.15.7:0/2904757141 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe074038050 con 0x557b1aba4920 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.371+0000 7fe093dd9740 10 client.26429 fetch_fsmap learned FSMap version 25 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.371+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557b1abb6570 con 0x557b1aba4920 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.372+0000 7fe0727fc640 1 -- 172.21.15.7:0/2904757141 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0840bd340 con 0x557b1aba4920 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.372+0000 7fe0727fc640 1 -- 172.21.15.7:0/2904757141 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe074003b30 con 0x557b1aba4920 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.372+0000 7fe093dd9740 10 client.26429 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.372+0000 7fe093dd9740 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.372+0000 7fe093dd9740 20 client.26429 populate_metadata read hostname 'smithi007' 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.372+0000 7fe093dd9740 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557b1ac16850 0x557b1ac18c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.372+0000 7fe093dd9740 4 client.26429 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:46:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.373+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557b1ab113a0 con 0x557b1ac16850 2024-03-26T08:46:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.373+0000 7fe08a59c640 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557b1ac16850 0x557b1ac18c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:46:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.374+0000 7fe08a59c640 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557b1ac16850 0x557b1ac18c70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b503e4ab514296a server_cookie=237978bce9e603ac in_seq=0 out_seq=0 2024-03-26T08:46:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.374+0000 7fe0727fc640 10 client.26429 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe0727fc640 1 -- 172.21.15.7:0/2904757141 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557b1ab113a0 con 0x557b1ac16850 2024-03-26T08:46:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe0727fc640 10 client.26429 handle_command_reply: tid=0 2024-03-26T08:46:53.380 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 client.26429 shutdown 2024-03-26T08:46:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 msgr2=0x557b1abb3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abb3870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x557b1aba0e00 tx=0x7fe074014000 comp rx=0 tx=0).stop 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 shutdown_connections 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557b1ac16850 0x557b1ac18c70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557b1abaa480 0x557b1abb4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe0727fc640 0 client.26429 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557b1aba5310 0x557b1abb3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 --2- 172.21.15.7:0/2904757141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557b1aba4920 0x557b1abb3870 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 >> 172.21.15.7:0/2904757141 conn(0x557b1ab9fe90 msgr2=0x557b1aba6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.389+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 shutdown_connections 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.390+0000 7fe093dd9740 1 -- 172.21.15.7:0/2904757141 wait complete. 2024-03-26T08:46:53.382 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.390+0000 7fe093dd9740 20 client.26429 trim_cache size 0 max 16384 2024-03-26T08:46:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:46:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:46:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:46:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:46:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.390+0000 7fe0911f3640 1 -- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0780a2870 msgr2=0x7fe0780a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:53.383 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.390+0000 7fe0911f3640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0780a2870 0x7fe0780a4d30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557b1aba5710 tx=0x7fe07400b390 comp rx=0 tx=0).stop 2024-03-26T08:46:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.391+0000 7fe0911f3640 1 -- 172.21.15.7:0/3505451166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 msgr2=0x7fe08c1ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:46:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.391+0000 7fe0911f3640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 0x7fe08c1ae090 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe0840323d0 tx=0x7fe084004270 comp rx=0 tx=0).stop 2024-03-26T08:46:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.392+0000 7fe0911f3640 1 -- 172.21.15.7:0/3505451166 shutdown_connections 2024-03-26T08:46:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.393+0000 7fe0911f3640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0780a2870 0x7fe0780a4d30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.393+0000 7fe0911f3640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe08c1098d0 0x7fe08c1ae090 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.384 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.393+0000 7fe0911f3640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe08c108f10 0x7fe08c0704f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.393+0000 7fe0911f3640 1 --2- 172.21.15.7:0/3505451166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe08c107d20 0x7fe08c06ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:46:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.393+0000 7fe0911f3640 1 -- 172.21.15.7:0/3505451166 >> 172.21.15.7:0/3505451166 conn(0x7fe08c076cb0 msgr2=0x7fe08c10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:46:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.394+0000 7fe0911f3640 1 -- 172.21.15.7:0/3505451166 shutdown_connections 2024-03-26T08:46:53.385 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:46:53.394+0000 7fe0911f3640 1 -- 172.21.15.7:0/3505451166 wait complete. 2024-03-26T08:46:53.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:53 smithi007 ceph-mon[32462]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.5 MiB/s wr, 852 op/s 2024-03-26T08:46:53.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:53 smithi044 ceph-mon[36461]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.5 MiB/s wr, 852 op/s 2024-03-26T08:46:53.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:53 smithi016 ceph-mon[35851]: pgmap v494: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.5 MiB/s wr, 852 op/s 2024-03-26T08:46:55.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:55 smithi007 ceph-mon[32462]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 853 op/s 2024-03-26T08:46:55.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:55 smithi044 ceph-mon[36461]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 853 op/s 2024-03-26T08:46:55.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:55 smithi016 ceph-mon[35851]: pgmap v495: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 853 op/s 2024-03-26T08:46:56.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 265.0 s, 3285.8 tps, lat 30.363 ms stddev 31.971 2024-03-26T08:46:57.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:57 smithi007 ceph-mon[32462]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 724 op/s 2024-03-26T08:46:57.815 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:57 smithi044 ceph-mon[36461]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 724 op/s 2024-03-26T08:46:57.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:57 smithi016 ceph-mon[35851]: pgmap v496: 97 pgs: 97 active+clean; 1.6 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 MiB/s wr, 724 op/s 2024-03-26T08:46:59.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:46:59 smithi007 ceph-mon[32462]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.9 MiB/s wr, 761 op/s 2024-03-26T08:46:59.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:46:59 smithi044 ceph-mon[36461]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.9 MiB/s wr, 761 op/s 2024-03-26T08:46:59.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:46:59 smithi016 ceph-mon[35851]: pgmap v497: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 7.9 MiB/s wr, 761 op/s 2024-03-26T08:47:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 270.0 s, 4014.8 tps, lat 24.957 ms stddev 28.072 2024-03-26T08:47:01.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:01 smithi007 ceph-mon[32462]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 705 op/s 2024-03-26T08:47:01.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:01 smithi044 ceph-mon[36461]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 705 op/s 2024-03-26T08:47:01.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:01 smithi016 ceph-mon[35851]: pgmap v498: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 705 op/s 2024-03-26T08:47:03.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:03 smithi007 ceph-mon[32462]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 787 op/s 2024-03-26T08:47:03.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:03 smithi044 ceph-mon[36461]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 787 op/s 2024-03-26T08:47:03.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:03 smithi016 ceph-mon[35851]: pgmap v499: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 787 op/s 2024-03-26T08:47:05.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:05 smithi007 ceph-mon[32462]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 807 op/s 2024-03-26T08:47:05.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:05 smithi044 ceph-mon[36461]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 807 op/s 2024-03-26T08:47:05.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:05 smithi016 ceph-mon[35851]: pgmap v500: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 MiB/s wr, 807 op/s 2024-03-26T08:47:06.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 275.0 s, 3718.1 tps, lat 26.769 ms stddev 28.349 2024-03-26T08:47:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.534+0000 7fe48d137640 1 -- 172.21.15.7:0/486973576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 msgr2=0x7fe48810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.534+0000 7fe48d137640 1 --2- 172.21.15.7:0/486973576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 0x7fe48810aad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe478009900 tx=0x7fe47802bc60 comp rx=0 tx=0).stop 2024-03-26T08:47:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.538+0000 7fe48d137640 1 -- 172.21.15.7:0/486973576 shutdown_connections 2024-03-26T08:47:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.538+0000 7fe48d137640 1 --2- 172.21.15.7:0/486973576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48810d9c0 0x7fe48810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.538+0000 7fe48d137640 1 --2- 172.21.15.7:0/486973576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe48810b010 0x7fe48810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.538+0000 7fe48d137640 1 --2- 172.21.15.7:0/486973576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 0x7fe48810aad0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.538+0000 7fe48d137640 1 -- 172.21.15.7:0/486973576 >> 172.21.15.7:0/486973576 conn(0x7fe48806e6b0 msgr2=0x7fe488070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.539+0000 7fe48d137640 1 -- 172.21.15.7:0/486973576 shutdown_connections 2024-03-26T08:47:06.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.539+0000 7fe48d137640 1 -- 172.21.15.7:0/486973576 wait complete. 2024-03-26T08:47:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.540+0000 7fe48d137640 1 Processor -- start 2024-03-26T08:47:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.541+0000 7fe48d137640 1 -- start start 2024-03-26T08:47:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.541+0000 7fe48d137640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 0x7fe4881a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.541+0000 7fe48d137640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe48810b010 0x7fe4881a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.542+0000 7fe486d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 0x7fe4881a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.542+0000 7fe486d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 0x7fe4881a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35764/0 (socket says 172.21.15.7:35764) 2024-03-26T08:47:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.542+0000 7fe48659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe48810b010 0x7fe4881a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.542+0000 7fe48659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe48810b010 0x7fe4881a6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35106/0 (socket says 172.21.15.7:35106) 2024-03-26T08:47:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.542+0000 7fe486d9d640 1 -- 172.21.15.7:0/2610208375 learned_addr learned my addr 172.21.15.7:0/2610208375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.542+0000 7fe48d137640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48810d9c0 0x7fe4881ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.542+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48811c280 con 0x7fe48810b010 2024-03-26T08:47:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.542+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48811c100 con 0x7fe48810d9c0 2024-03-26T08:47:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.542+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48811c400 con 0x7fe4881086a0 2024-03-26T08:47:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.543+0000 7fe48659c640 1 -- 172.21.15.7:0/2610208375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 msgr2=0x7fe4881a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.543+0000 7fe48659c640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 0x7fe4881a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.543+0000 7fe48659c640 1 -- 172.21.15.7:0/2610208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48810d9c0 msgr2=0x7fe4881ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.543+0000 7fe48759e640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48810d9c0 0x7fe4881ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.543+0000 7fe48659c640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48810d9c0 0x7fe4881ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.543+0000 7fe48659c640 1 -- 172.21.15.7:0/2610208375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4881ae500 con 0x7fe48810b010 2024-03-26T08:47:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.543+0000 7fe486d9d640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 0x7fe4881a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.543+0000 7fe48759e640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48810d9c0 0x7fe4881ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:47:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.543+0000 7fe48659c640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe48810b010 0x7fe4881a6e70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b3f0 tx=0x7fe47400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.544+0000 7fe47ffff640 1 -- 172.21.15.7:0/2610208375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474015020 con 0x7fe48810b010 2024-03-26T08:47:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.544+0000 7fe47ffff640 1 -- 172.21.15.7:0/2610208375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe474011d10 con 0x7fe48810b010 2024-03-26T08:47:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.544+0000 7fe47ffff640 1 -- 172.21.15.7:0/2610208375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400f2e0 con 0x7fe48810b010 2024-03-26T08:47:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.544+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4881ae7f0 con 0x7fe48810b010 2024-03-26T08:47:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.544+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4881aecf0 con 0x7fe48810b010 2024-03-26T08:47:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.547+0000 7fe47ffff640 1 -- 172.21.15.7:0/2610208375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe474018020 con 0x7fe48810b010 2024-03-26T08:47:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.547+0000 7fe48fdfb740 1 Processor -- start 2024-03-26T08:47:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.547+0000 7fe48fdfb740 1 -- start start 2024-03-26T08:47:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.547+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5935bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.547+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59360e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.547+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c59363910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.547+0000 7fe48fdfb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c592c0b10 con 0x556c5935c310 2024-03-26T08:47:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.547+0000 7fe48fdfb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c59269900 con 0x556c5935b920 2024-03-26T08:47:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.547+0000 7fe48fdfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c592bcbb0 con 0x556c59361480 2024-03-26T08:47:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe486d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59360e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe48659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c59363910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe486d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59360e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47452/0 (socket says 172.21.15.7:47452) 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe486d9d640 1 -- 172.21.15.7:0/677690991 learned_addr learned my addr 172.21.15.7:0/677690991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe48759e640 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5935bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe47ffff640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe46c0a2a00 0x7fe46c0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe486d9d640 1 -- 172.21.15.7:0/677690991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 msgr2=0x556c59363910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe486d9d640 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c59363910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe486d9d640 1 -- 172.21.15.7:0/677690991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 msgr2=0x556c5935bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe486d9d640 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5935bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe486d9d640 1 -- 172.21.15.7:0/677690991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c58fd5210 con 0x556c5935c310 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe48659c640 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c59363910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.548+0000 7fe47ffff640 1 -- 172.21.15.7:0/2610208375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe4740ce210 con 0x7fe48810b010 2024-03-26T08:47:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.549+0000 7fe48759e640 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5935bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.549+0000 7fe486d9d640 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59360e10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe4780096a0 tx=0x7fe47802c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c42df7403aeaad3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.549+0000 7fe48759e640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe46c0a2a00 0x7fe46c0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.549+0000 7fe47d7fa640 1 -- 172.21.15.7:0/677690991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478042070 con 0x556c5935c310 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.549+0000 7fe47d7fa640 1 -- 172.21.15.7:0/677690991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4780054e0 con 0x556c5935c310 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.549+0000 7fe47d7fa640 1 -- 172.21.15.7:0/677690991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4780035f0 con 0x556c5935c310 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 -- 172.21.15.7:0/677690991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 msgr2=0x556c59360e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59360e10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe4780096a0 tx=0x7fe47802c6f0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48759e640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe46c0a2a00 0x7fe46c0a4ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x556c5935cb70 tx=0x7fe480004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 -- 172.21.15.7:0/677690991 shutdown_connections 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c59363910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59360e10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/677690991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5935bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 -- 172.21.15.7:0/677690991 >> 172.21.15.7:0/677690991 conn(0x556c59356e90 msgr2=0x556c59357290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 -- 172.21.15.7:0/677690991 shutdown_connections 2024-03-26T08:47:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 -- 172.21.15.7:0/677690991 wait complete. 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 -- start start 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.550+0000 7fe48fdfb740 1 Processor -- start 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c59365c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59366140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c5936d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48fdfb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c593643d0 con 0x556c5935c310 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48fdfb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c5927cd50 con 0x556c5935b920 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48fdfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c59364550 con 0x556c59361480 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48fdfb740 10 client.0 _ll_register_callbacks cb 0x556c5927c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59366140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59366140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47468/0 (socket says 172.21.15.7:47468) 2024-03-26T08:47:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48659c640 1 -- 172.21.15.7:0/2248787199 learned_addr learned my addr 172.21.15.7:0/2248787199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.551+0000 7fe48759e640 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c5936d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe48659c640 1 -- 172.21.15.7:0/2248787199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 msgr2=0x556c5936d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe48659c640 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c5936d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe48659c640 1 -- 172.21.15.7:0/2248787199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 msgr2=0x556c59365c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe486d9d640 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c59365c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe48659c640 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c59365c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe48659c640 1 -- 172.21.15.7:0/2248787199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c590a1620 con 0x556c5935c310 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe486d9d640 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c59365c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe48659c640 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59366140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b3c0 tx=0x7fe47409a940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d416c2fef463589 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe46a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.552+0000 7fe46a7fc640 1 -- 172.21.15.7:0/2248787199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47409b270 con 0x556c5935c310 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe46a7fc640 1 -- 172.21.15.7:0/2248787199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe4740105a0 con 0x556c5935c310 2024-03-26T08:47:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe48fdfb740 1 -- 172.21.15.7:0/2248787199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4780054e0 con 0x556c5935c310 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe46a7fc640 1 -- 172.21.15.7:0/2248787199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe47409ab70 con 0x556c5935c310 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe48fdfb740 10 client.16602 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe48fdfb740 1 -- 172.21.15.7:0/2248787199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556c59370560 con 0x556c5935c310 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe46a7fc640 1 -- 172.21.15.7:0/2248787199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4780054e0 con 0x556c5935c310 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe46a7fc640 1 -- 172.21.15.7:0/2248787199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe474099fc0 con 0x556c5935c310 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe48fdfb740 10 client.16602 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe48fdfb740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe48fdfb740 20 client.16602 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c59371c30 0x556c59374060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe48fdfb740 4 client.16602 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.553+0000 7fe48fdfb740 1 -- 172.21.15.7:0/2248787199 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c592ce050 con 0x556c59371c30 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.554+0000 7fe486d9d640 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c59371c30 0x556c59374060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.554+0000 7fe486d9d640 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c59371c30 0x556c59374060 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72560ad5946c1fbd server_cookie=9e2a9637032ffeae in_seq=0 out_seq=0 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.554+0000 7fe46a7fc640 10 client.16602 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe46a7fc640 1 -- 172.21.15.7:0/2248787199 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556c592ce050 con 0x556c59371c30 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe46a7fc640 10 client.16602 handle_command_reply: tid=0 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 client.16602 shutdown 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 -- 172.21.15.7:0/2248787199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 msgr2=0x556c59366140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59366140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b3c0 tx=0x7fe47409a940 comp rx=0 tx=0).stop 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 -- 172.21.15.7:0/2248787199 shutdown_connections 2024-03-26T08:47:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c59371c30 0x556c59374060 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c5936d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935c310 0x556c59366140 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/2248787199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c59365c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 -- 172.21.15.7:0/2248787199 >> 172.21.15.7:0/2248787199 conn(0x556c59356e90 msgr2=0x556c59356020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe46a7fc640 0 client.16602 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 -- 172.21.15.7:0/2248787199 shutdown_connections 2024-03-26T08:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 1 -- 172.21.15.7:0/2248787199 wait complete. 2024-03-26T08:47:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.555+0000 7fe48fdfb740 20 client.16602 trim_cache size 0 max 16384 2024-03-26T08:47:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.576+0000 7fe48fdfb740 1 Processor -- start 2024-03-26T08:47:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.576+0000 7fe48fdfb740 1 -- start start 2024-03-26T08:47:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.576+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935b920 0x556c59355790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.576+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935c310 0x556c5936d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c59355cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe48fdfb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881b9df0 con 0x556c5935b920 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe48fdfb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881b78f0 con 0x556c5935c310 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe48fdfb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881b5970 con 0x556c59361480 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe486d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935b920 0x556c59355790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe486d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935b920 0x556c59355790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47472/0 (socket says 172.21.15.7:47472) 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe486d9d640 1 -- 172.21.15.7:0/3826054239 learned_addr learned my addr 172.21.15.7:0/3826054239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe48759e640 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c59355cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe48659c640 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935c310 0x556c5936d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe486d9d640 1 -- 172.21.15.7:0/3826054239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 msgr2=0x556c59355cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe486d9d640 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c59355cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe486d9d640 1 -- 172.21.15.7:0/3826054239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935c310 msgr2=0x556c5936d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:06.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe486d9d640 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935c310 0x556c5936d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe486d9d640 1 -- 172.21.15.7:0/3826054239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c5935cba0 con 0x556c5935b920 2024-03-26T08:47:06.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.577+0000 7fe48659c640 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935c310 0x556c5936d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.579+0000 7fe486d9d640 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935b920 0x556c59355790 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe47802cd60 tx=0x7fe47802bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128d5a94b236811b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.579+0000 7fe47cff9640 1 -- 172.21.15.7:0/3826054239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478042040 con 0x556c5935b920 2024-03-26T08:47:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.579+0000 7fe47cff9640 1 -- 172.21.15.7:0/3826054239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe478052230 con 0x556c5935b920 2024-03-26T08:47:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.579+0000 7fe47cff9640 1 -- 172.21.15.7:0/3826054239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478005260 con 0x556c5935b920 2024-03-26T08:47:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.580+0000 7fe48fdfb740 1 -- 172.21.15.7:0/3826054239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935b920 msgr2=0x556c59355790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.580+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935b920 0x556c59355790 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe47802cd60 tx=0x7fe47802bc60 comp rx=0 tx=0).stop 2024-03-26T08:47:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.580+0000 7fe48fdfb740 1 -- 172.21.15.7:0/3826054239 shutdown_connections 2024-03-26T08:47:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.580+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c59361480 0x556c59355cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.580+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935c310 0x556c5936d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.580+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/3826054239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c5935b920 0x556c59355790 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.580+0000 7fe48fdfb740 1 -- 172.21.15.7:0/3826054239 >> 172.21.15.7:0/3826054239 conn(0x556c59356e90 msgr2=0x556c5935d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.580+0000 7fe48fdfb740 1 -- 172.21.15.7:0/3826054239 shutdown_connections 2024-03-26T08:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.580+0000 7fe48fdfb740 1 -- 172.21.15.7:0/3826054239 wait complete. 2024-03-26T08:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.581+0000 7fe48fdfb740 1 -- start start 2024-03-26T08:47:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.581+0000 7fe48fdfb740 1 Processor -- start 2024-03-26T08:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.581+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5936a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c5935c310 0x556c5936aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48fdfb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c59361480 0x556c5936ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c5935c310 0x556c5936aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe486d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5936a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c5935c310 0x556c5936aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:55882/0 (socket says 172.21.15.7:55882) 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48659c640 1 -- 172.21.15.7:0/1419890060 learned_addr learned my addr 172.21.15.7:0/1419890060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881b5f70 con 0x556c59361480 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881b7290 con 0x556c5935b920 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881b65d0 con 0x556c5935c310 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48fdfb740 10 client.0 _ll_register_callbacks cb 0x7fe4740d1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48759e640 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c59361480 0x556c5936ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe486d9d640 1 -- 172.21.15.7:0/1419890060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c5935c310 msgr2=0x556c5936aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe486d9d640 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c5935c310 0x556c5936aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe486d9d640 1 -- 172.21.15.7:0/1419890060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c59361480 msgr2=0x556c5936ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe486d9d640 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c59361480 0x556c5936ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe486d9d640 1 -- 172.21.15.7:0/1419890060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c5936c040 con 0x556c5935b920 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48659c640 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c5935c310 0x556c5936aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.582+0000 7fe48759e640 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c59361480 0x556c5936ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.583+0000 7fe486d9d640 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5936a500 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe47802c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1f9087f218c235 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.583+0000 7fe46a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.583+0000 7fe46a7fc640 1 -- 172.21.15.7:0/1419890060 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478058070 con 0x556c5935b920 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.583+0000 7fe46a7fc640 1 -- 172.21.15.7:0/1419890060 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe478004990 con 0x556c5935b920 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.583+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe478052230 con 0x556c5935b920 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.583+0000 7fe46a7fc640 1 -- 172.21.15.7:0/1419890060 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe478042040 con 0x556c5935b920 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.583+0000 7fe48fdfb740 10 client.26438 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.583+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556c5936d330 con 0x556c5935b920 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.583+0000 7fe46a7fc640 1 -- 172.21.15.7:0/1419890060 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4780054e0 con 0x556c5935b920 2024-03-26T08:47:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe46a7fc640 1 -- 172.21.15.7:0/1419890060 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe478065050 con 0x556c5935b920 2024-03-26T08:47:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe48fdfb740 10 client.26438 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe48fdfb740 10 client.26438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe48fdfb740 20 client.26438 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c593cd680 0x556c593cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe48fdfb740 4 client.26438 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:47:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fe478052230 con 0x556c593cd680 2024-03-26T08:47:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe48659c640 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c593cd680 0x556c593cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe46a7fc640 10 client.26438 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.584+0000 7fe48659c640 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c593cd680 0x556c593cfa70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec8b39fb9ef3de4 server_cookie=ba5d0cb5fc699af6 in_seq=0 out_seq=0 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe46a7fc640 1 -- 172.21.15.7:0/1419890060 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fe478052230 con 0x556c593cd680 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe46a7fc640 10 client.26438 handle_command_reply: tid=0 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 client.26438 shutdown 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 msgr2=0x556c5936a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5936a500 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe47802c2e0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 shutdown_connections 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c593cd680 0x556c593cfa70 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c59361480 0x556c5936ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe46a7fc640 0 client.26438 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c5935c310 0x556c5936aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 --2- 172.21.15.7:0/1419890060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c5935b920 0x556c5936a500 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 >> 172.21.15.7:0/1419890060 conn(0x556c59356e90 msgr2=0x556c59355790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 shutdown_connections 2024-03-26T08:47:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 1 -- 172.21.15.7:0/1419890060 wait complete. 2024-03-26T08:47:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.603+0000 7fe48fdfb740 20 client.26438 trim_cache size 0 max 16384 2024-03-26T08:47:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:47:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:47:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:47:06.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:47:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe46c0a2a00 msgr2=0x7fe46c0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe46c0a2a00 0x7fe46c0a4ec0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x556c5935cb70 tx=0x7fe480004ad0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe48810b010 msgr2=0x7fe4881a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe48810b010 0x7fe4881a6e70 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe47400b3f0 tx=0x7fe47400b8b0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 shutdown_connections 2024-03-26T08:47:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe46c0a2a00 0x7fe46c0a4ec0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe48810d9c0 0x7fe4881ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe48810b010 0x7fe4881a6e70 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 --2- 172.21.15.7:0/2610208375 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe4881086a0 0x7fe4881a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 >> 172.21.15.7:0/2610208375 conn(0x7fe48806e6b0 msgr2=0x7fe48810b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 shutdown_connections 2024-03-26T08:47:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:06.604+0000 7fe48d137640 1 -- 172.21.15.7:0/2610208375 wait complete. 2024-03-26T08:47:07.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:07 smithi007 ceph-mon[32462]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 MiB/s wr, 724 op/s 2024-03-26T08:47:07.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:07 smithi044 ceph-mon[36461]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 MiB/s wr, 724 op/s 2024-03-26T08:47:07.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:07 smithi016 ceph-mon[35851]: pgmap v501: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.8 MiB/s wr, 724 op/s 2024-03-26T08:47:09.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:09 smithi044 ceph-mon[36461]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 MiB/s wr, 784 op/s 2024-03-26T08:47:09.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:09 smithi016 ceph-mon[35851]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 MiB/s wr, 784 op/s 2024-03-26T08:47:10.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:09 smithi007 ceph-mon[32462]: pgmap v502: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.7 MiB/s wr, 784 op/s 2024-03-26T08:47:10.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:47:10.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:47:10.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.638+0000 7fe3ee7fc640 1 -- 172.21.15.7:0/3838990317 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e002cc80 con 0x7fe3f01017b0 2024-03-26T08:47:10.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.639+0000 7fe3f6169640 1 -- 172.21.15.7:0/3838990317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 msgr2=0x7fe3f0101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.639+0000 7fe3f6169640 1 --2- 172.21.15.7:0/3838990317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 0x7fe3f0101c30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009900 tx=0x7fe3e002bc30 comp rx=0 tx=0).stop 2024-03-26T08:47:10.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.639+0000 7fe3f6169640 1 -- 172.21.15.7:0/3838990317 shutdown_connections 2024-03-26T08:47:10.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.639+0000 7fe3f6169640 1 --2- 172.21.15.7:0/3838990317 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f0102170 0x7fe3f0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.639+0000 7fe3f6169640 1 --2- 172.21.15.7:0/3838990317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 0x7fe3f0101c30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.639+0000 7fe3f6169640 1 --2- 172.21.15.7:0/3838990317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f01005c0 0x7fe3f01009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.639+0000 7fe3f6169640 1 -- 172.21.15.7:0/3838990317 >> 172.21.15.7:0/3838990317 conn(0x7fe3f00fbb30 msgr2=0x7fe3f00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.640+0000 7fe3f6169640 1 -- 172.21.15.7:0/3838990317 shutdown_connections 2024-03-26T08:47:10.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.640+0000 7fe3f6169640 1 -- 172.21.15.7:0/3838990317 wait complete. 2024-03-26T08:47:10.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3f6169640 1 Processor -- start 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3f6169640 1 -- start start 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3f6169640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f01005c0 0x7fe3f019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3f6169640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 0x7fe3f019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3f6169640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f0102170 0x7fe3f01a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3f6169640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f010b190 con 0x7fe3f0102170 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3f6169640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f010b010 con 0x7fe3f01017b0 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3f6169640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f010b310 con 0x7fe3f01005c0 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 0x7fe3f019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3ef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f01005c0 0x7fe3f019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 0x7fe3f019e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55200/0 (socket says 172.21.15.7:55200) 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3eeffd640 1 -- 172.21.15.7:0/671057292 learned_addr learned my addr 172.21.15.7:0/671057292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.641+0000 7fe3effff640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f0102170 0x7fe3f01a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3eeffd640 1 -- 172.21.15.7:0/671057292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f01005c0 msgr2=0x7fe3f019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3eeffd640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f01005c0 0x7fe3f019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3eeffd640 1 -- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f0102170 msgr2=0x7fe3f01a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3eeffd640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f0102170 0x7fe3f01a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3eeffd640 1 -- 172.21.15.7:0/671057292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f01a5c10 con 0x7fe3f01017b0 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3eeffd640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 0x7fe3f019e530 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0000c00 tx=0x7fe3e002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3ecff9640 1 -- 172.21.15.7:0/671057292 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0043020 con 0x7fe3f01017b0 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3f6169640 1 -- 172.21.15.7:0/671057292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f01a5ea0 con 0x7fe3f01017b0 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3f6169640 1 -- 172.21.15.7:0/671057292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f01a63b0 con 0x7fe3f01017b0 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.642+0000 7fe3ecff9640 1 -- 172.21.15.7:0/671057292 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3e00054e0 con 0x7fe3f01017b0 2024-03-26T08:47:10.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.643+0000 7fe3ecff9640 1 -- 172.21.15.7:0/671057292 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0003c40 con 0x7fe3f01017b0 2024-03-26T08:47:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3f8e32740 1 Processor -- start 2024-03-26T08:47:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3f8e32740 1 -- start start 2024-03-26T08:47:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed1da0 0x55f043ed21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed2790 0x55f043ed7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed1da0 0x55f043ed21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed2790 0x55f043ed7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed2790 0x55f043ed7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47500/0 (socket says 172.21.15.7:47500) 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed1da0 0x55f043ed21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55210/0 (socket says 172.21.15.7:55210) 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed7900 0x55f043ed9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3ef7fe640 1 -- 172.21.15.7:0/1604252479 learned_addr learned my addr 172.21.15.7:0/1604252479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1604252479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f043e373d0 con 0x55f043ed2790 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1604252479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f043ddfad0 con 0x55f043ed1da0 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1604252479 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f043e33470 con 0x55f043ed7900 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3ef7fe640 1 -- 172.21.15.7:0/1604252479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed7900 msgr2=0x55f043ed9d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/1604252479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed7900 0x55f043ed9d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3ef7fe640 1 -- 172.21.15.7:0/1604252479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed1da0 msgr2=0x55f043ed21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:10.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/1604252479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed1da0 0x55f043ed21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.646+0000 7fe3ef7fe640 1 -- 172.21.15.7:0/1604252479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f043c157d0 con 0x55f043ed2790 2024-03-26T08:47:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.647+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/1604252479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed2790 0x55f043ed7290 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80039f0 tx=0x7fe3e8003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35ddeefe23b0d25 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.647+0000 7fe3ecff9640 1 -- 172.21.15.7:0/671057292 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe3e0046020 con 0x7fe3f01017b0 2024-03-26T08:47:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.647+0000 7fe3d5ffb640 1 -- 172.21.15.7:0/1604252479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e801c070 con 0x55f043ed2790 2024-03-26T08:47:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.647+0000 7fe3d5ffb640 1 -- 172.21.15.7:0/1604252479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3e80178d0 con 0x55f043ed2790 2024-03-26T08:47:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.647+0000 7fe3d5ffb640 1 -- 172.21.15.7:0/1604252479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80168d0 con 0x55f043ed2790 2024-03-26T08:47:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.647+0000 7fe3ecff9640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3d80a2900 0x7fe3d80a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.648+0000 7fe3ecff9640 1 -- 172.21.15.7:0/671057292 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3e00f0a60 con 0x7fe3f01017b0 2024-03-26T08:47:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.648+0000 7fe3effff640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3d80a2900 0x7fe3d80a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.648+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1604252479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed2790 msgr2=0x55f043ed7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.649+0000 7fe3effff640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3d80a2900 0x7fe3d80a4dc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55f043ed2ff0 tx=0x7fe3e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.649+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1604252479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed2790 0x55f043ed7290 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80039f0 tx=0x7fe3e8003df0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.649+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1604252479 shutdown_connections 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.649+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1604252479 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed7900 0x55f043ed9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.649+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1604252479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed2790 0x55f043ed7290 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.649+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1604252479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed1da0 0x55f043ed21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.649+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1604252479 >> 172.21.15.7:0/1604252479 conn(0x55f043ecd310 msgr2=0x55f043ecd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.649+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1604252479 shutdown_connections 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.649+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1604252479 wait complete. 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3f8e32740 1 -- start start 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3f8e32740 1 Processor -- start 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed1da0 0x55f043edc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed2790 0x55f043edc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed7900 0x55f043ee3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3f8e32740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f043eda900 con 0x55f043ed7900 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3f8e32740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f043df3060 con 0x55f043ed2790 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3f8e32740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f043edaa80 con 0x55f043ed1da0 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3f8e32740 10 client.0 _ll_register_callbacks cb 0x55f043dd56a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed2790 0x55f043edc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.650+0000 7fe3effff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed7900 0x55f043ee3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed2790 0x55f043edc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55224/0 (socket says 172.21.15.7:55224) 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3eeffd640 1 -- 172.21.15.7:0/1644468104 learned_addr learned my addr 172.21.15.7:0/1644468104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3ef7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed1da0 0x55f043edc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3effff640 1 -- 172.21.15.7:0/1644468104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed1da0 msgr2=0x55f043edc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3effff640 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed1da0 0x55f043edc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3effff640 1 -- 172.21.15.7:0/1644468104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed2790 msgr2=0x55f043edc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3effff640 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed2790 0x55f043edc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3effff640 1 -- 172.21.15.7:0/1644468104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f043b4b3f0 con 0x55f043ed7900 2024-03-26T08:47:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3effff640 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed7900 0x55f043ee3880 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40045e0 tx=0x7fe3e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f27778bf3e097 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:47:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3ceffd640 1 -- 172.21.15.7:0/1644468104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e401e470 con 0x55f043ed7900 2024-03-26T08:47:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3ceffd640 1 -- 172.21.15.7:0/1644468104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3e401e610 con 0x55f043ed7900 2024-03-26T08:47:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.651+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1644468104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3e80178d0 con 0x55f043ed7900 2024-03-26T08:47:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3ceffd640 1 -- 172.21.15.7:0/1644468104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e401e810 con 0x55f043ed7900 2024-03-26T08:47:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3f8e32740 10 client.16620 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1644468104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f043b0b0f0 con 0x55f043ed7900 2024-03-26T08:47:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3ceffd640 1 -- 172.21.15.7:0/1644468104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3e80178d0 con 0x55f043ed7900 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3ceffd640 1 -- 172.21.15.7:0/1644468104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe3e4027070 con 0x55f043ed7900 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3f8e32740 10 client.16620 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3f8e32740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3f8e32740 20 client.16620 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f043ee7dd0 0x55f043eea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3f8e32740 4 client.16620 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.652+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1644468104 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f043e44910 con 0x55f043ee7dd0 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.653+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f043ee7dd0 0x55f043eea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.654+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f043ee7dd0 0x55f043eea210 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53cbe3d10e8529f7 server_cookie=a333f7e2c4e14391 in_seq=0 out_seq=0 2024-03-26T08:47:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.654+0000 7fe3ceffd640 10 client.16620 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.654+0000 7fe3ceffd640 1 -- 172.21.15.7:0/1644468104 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f043e44910 con 0x55f043ee7dd0 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.654+0000 7fe3ceffd640 10 client.16620 handle_command_reply: tid=0 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 client.16620 shutdown 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1644468104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed7900 msgr2=0x55f043ee3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed7900 0x55f043ee3880 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40045e0 tx=0x7fe3e4004730 comp rx=0 tx=0).stop 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1644468104 shutdown_connections 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f043ee7dd0 0x55f043eea210 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed7900 0x55f043ee3880 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3ceffd640 0 client.16620 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed2790 0x55f043edc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/1644468104 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed1da0 0x55f043edc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1644468104 >> 172.21.15.7:0/1644468104 conn(0x55f043ecd310 msgr2=0x55f043ecc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1644468104 shutdown_connections 2024-03-26T08:47:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 1 -- 172.21.15.7:0/1644468104 wait complete. 2024-03-26T08:47:10.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.655+0000 7fe3f8e32740 20 client.16620 trim_cache size 0 max 16384 2024-03-26T08:47:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3f8e32740 1 Processor -- start 2024-03-26T08:47:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3f8e32740 1 -- start start 2024-03-26T08:47:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ed3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 0x55f043ee3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ed3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 0x55f043ecc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3f8e32740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01b1660 con 0x55f043ed1da0 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3f8e32740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01af160 con 0x55f043ed7900 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3f8e32740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01ad1e0 con 0x55f043ed2790 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.679+0000 7fe3ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ed3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47510/0 (socket says 172.21.15.7:47510) 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3ef7fe640 1 -- 172.21.15.7:0/61726154 learned_addr learned my addr 172.21.15.7:0/61726154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3eeffd640 1 --2- 172.21.15.7:0/61726154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 0x55f043ee3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3effff640 1 --2- 172.21.15.7:0/61726154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 0x55f043ecc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3ef7fe640 1 -- 172.21.15.7:0/61726154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 msgr2=0x55f043ee3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/61726154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 0x55f043ee3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3ef7fe640 1 -- 172.21.15.7:0/61726154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 msgr2=0x55f043ecc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/61726154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 0x55f043ecc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3ef7fe640 1 -- 172.21.15.7:0/61726154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f043edb420 con 0x55f043ed1da0 2024-03-26T08:47:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/61726154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ed3460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8004410 tx=0x7fe3e8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9496c1729893e68e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3d57fa640 1 -- 172.21.15.7:0/61726154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e801c040 con 0x55f043ed1da0 2024-03-26T08:47:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3d57fa640 1 -- 172.21.15.7:0/61726154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3e802d030 con 0x55f043ed1da0 2024-03-26T08:47:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.680+0000 7fe3d57fa640 1 -- 172.21.15.7:0/61726154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80239f0 con 0x55f043ed1da0 2024-03-26T08:47:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.681+0000 7fe3f8e32740 1 -- 172.21.15.7:0/61726154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 msgr2=0x55f043ed3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.681+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/61726154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ed3460 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8004410 tx=0x7fe3e8005fd0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.681+0000 7fe3f8e32740 1 -- 172.21.15.7:0/61726154 shutdown_connections 2024-03-26T08:47:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.681+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/61726154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 0x55f043ecc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.681+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/61726154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 0x55f043ee3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.681+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/61726154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ed3460 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.681+0000 7fe3f8e32740 1 -- 172.21.15.7:0/61726154 >> 172.21.15.7:0/61726154 conn(0x55f043ecd310 msgr2=0x55f043ecbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.682+0000 7fe3f8e32740 1 -- 172.21.15.7:0/61726154 shutdown_connections 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.682+0000 7fe3f8e32740 1 -- 172.21.15.7:0/61726154 wait complete. 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.682+0000 7fe3f8e32740 1 -- start start 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.682+0000 7fe3f8e32740 1 Processor -- start 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.683+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ee11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 0x55f043ee1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3f8e32740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 0x55f043ee2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ee11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3ef7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ee11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47522/0 (socket says 172.21.15.7:47522) 2024-03-26T08:47:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3ef7fe640 1 -- 172.21.15.7:0/3609023107 learned_addr learned my addr 172.21.15.7:0/3609023107 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3effff640 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 0x55f043ee2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3eeffd640 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 0x55f043ee1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01ad7e0 con 0x55f043ed1da0 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01aeb00 con 0x55f043ed7900 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f01ade40 con 0x55f043ed2790 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3f8e32740 10 client.0 _ll_register_callbacks cb 0x7fe3e4009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3effff640 1 -- 172.21.15.7:0/3609023107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 msgr2=0x55f043ee1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3effff640 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 0x55f043ee1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3effff640 1 -- 172.21.15.7:0/3609023107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 msgr2=0x55f043ee11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3effff640 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ee11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3effff640 1 -- 172.21.15.7:0/3609023107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f043ee2d10 con 0x55f043ed7900 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3effff640 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 0x55f043ee2720 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40092e0 tx=0x7fe3e401e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=153018af3c89489e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3eeffd640 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 0x55f043ee1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.684+0000 7fe3ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.685+0000 7fe3ceffd640 1 -- 172.21.15.7:0/3609023107 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e401ed80 con 0x55f043ed7900 2024-03-26T08:47:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.685+0000 7fe3ceffd640 1 -- 172.21.15.7:0/3609023107 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3e4029440 con 0x55f043ed7900 2024-03-26T08:47:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.685+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3e802d030 con 0x55f043ed7900 2024-03-26T08:47:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.685+0000 7fe3ceffd640 1 -- 172.21.15.7:0/3609023107 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e4029640 con 0x55f043ed7900 2024-03-26T08:47:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.685+0000 7fe3f8e32740 10 client.26456 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.685+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f043f54890 con 0x55f043ed7900 2024-03-26T08:47:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.685+0000 7fe3ceffd640 1 -- 172.21.15.7:0/3609023107 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3e802d030 con 0x55f043ed7900 2024-03-26T08:47:10.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.686+0000 7fe3ceffd640 1 -- 172.21.15.7:0/3609023107 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe3e4027040 con 0x55f043ed7900 2024-03-26T08:47:10.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.686+0000 7fe3f8e32740 10 client.26456 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:10.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.686+0000 7fe3f8e32740 10 client.26456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.686+0000 7fe3f8e32740 20 client.26456 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.686+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f043ee7dd0 0x55f043f57140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.686+0000 7fe3f8e32740 4 client.26456 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.686+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f043e3ec60 con 0x55f043ee7dd0 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.686+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f043ee7dd0 0x55f043f57140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.687+0000 7fe3ef7fe640 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f043ee7dd0 0x55f043f57140 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9bbd7fd8fd6979b server_cookie=a2b2aafa75260e57 in_seq=0 out_seq=0 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.687+0000 7fe3ceffd640 10 client.26456 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.687+0000 7fe3ceffd640 1 -- 172.21.15.7:0/3609023107 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f043e3ec60 con 0x55f043ee7dd0 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.687+0000 7fe3ceffd640 10 client.26456 handle_command_reply: tid=0 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.687+0000 7fe3f8e32740 1 client.26456 shutdown 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 msgr2=0x55f043ee2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 0x55f043ee2720 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40092e0 tx=0x7fe3e401e720 comp rx=0 tx=0).stop 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 shutdown_connections 2024-03-26T08:47:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f043ee7dd0 0x55f043f57140 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f043ed7900 0x55f043ee2720 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3ceffd640 0 client.26456 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f043ed2790 0x55f043ee1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 --2- 172.21.15.7:0/3609023107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f043ed1da0 0x55f043ee11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 >> 172.21.15.7:0/3609023107 conn(0x55f043ecd310 msgr2=0x55f043ed0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 shutdown_connections 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.688+0000 7fe3f8e32740 1 -- 172.21.15.7:0/3609023107 wait complete. 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.689+0000 7fe3f8e32740 20 client.26456 trim_cache size 0 max 16384 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:47:10.684 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:47:10.685 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:47:10.685 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.689+0000 7fe3f6169640 1 -- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3d80a2900 msgr2=0x7fe3d80a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.689+0000 7fe3f6169640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3d80a2900 0x7fe3d80a4dc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55f043ed2ff0 tx=0x7fe3e4005f90 comp rx=0 tx=0).stop 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.689+0000 7fe3f6169640 1 -- 172.21.15.7:0/671057292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 msgr2=0x7fe3f019e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.689+0000 7fe3f6169640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 0x7fe3f019e530 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0000c00 tx=0x7fe3e002c650 comp rx=0 tx=0).stop 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.690+0000 7fe3f6169640 1 -- 172.21.15.7:0/671057292 shutdown_connections 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.690+0000 7fe3f6169640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3d80a2900 0x7fe3d80a4dc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.690+0000 7fe3f6169640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f0102170 0x7fe3f01a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.690+0000 7fe3f6169640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f01017b0 0x7fe3f019e530 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.690+0000 7fe3f6169640 1 --2- 172.21.15.7:0/671057292 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3f01005c0 0x7fe3f019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.690+0000 7fe3f6169640 1 -- 172.21.15.7:0/671057292 >> 172.21.15.7:0/671057292 conn(0x7fe3f00fbb30 msgr2=0x7fe3f01029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.690+0000 7fe3f6169640 1 -- 172.21.15.7:0/671057292 shutdown_connections 2024-03-26T08:47:10.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.690+0000 7fe3f6169640 1 -- 172.21.15.7:0/671057292 wait complete. 2024-03-26T08:47:10.697 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:47:10.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:47:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.870+0000 7f44ddc44640 1 -- 172.21.15.7:0/2151766620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d8101900 msgr2=0x7f44d8108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.870+0000 7f44ddc44640 1 --2- 172.21.15.7:0/2151766620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d8101900 0x7f44d8108190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009860 tx=0x7f44cc02bb70 comp rx=0 tx=0).stop 2024-03-26T08:47:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.871+0000 7f44ddc44640 1 -- 172.21.15.7:0/2151766620 shutdown_connections 2024-03-26T08:47:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.871+0000 7f44ddc44640 1 --2- 172.21.15.7:0/2151766620 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d8101900 0x7f44d8108190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.871+0000 7f44ddc44640 1 --2- 172.21.15.7:0/2151766620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d8100f40 0x7f44d81013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.871+0000 7f44ddc44640 1 --2- 172.21.15.7:0/2151766620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d80ffd50 0x7f44d8100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.871+0000 7f44ddc44640 1 -- 172.21.15.7:0/2151766620 >> 172.21.15.7:0/2151766620 conn(0x7f44d80fb2c0 msgr2=0x7f44d80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:10.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.872+0000 7f44ddc44640 1 -- 172.21.15.7:0/2151766620 shutdown_connections 2024-03-26T08:47:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.872+0000 7f44ddc44640 1 -- 172.21.15.7:0/2151766620 wait complete. 2024-03-26T08:47:10.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.872+0000 7f44ddc44640 1 Processor -- start 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.873+0000 7f44ddc44640 1 -- start start 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.873+0000 7f44ddc44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d80ffd50 0x7f44d819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.873+0000 7f44ddc44640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d8100f40 0x7f44d819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.873+0000 7f44ddc44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8101900 0x7f44d81a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.873+0000 7f44ddc44640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d810b170 con 0x7f44d80ffd50 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.873+0000 7f44ddc44640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d810aff0 con 0x7f44d8101900 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.873+0000 7f44ddc44640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d810b2f0 con 0x7f44d8100f40 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d80ffd50 0x7f44d819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8101900 0x7f44d81a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8101900 0x7f44d81a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55252/0 (socket says 172.21.15.7:55252) 2024-03-26T08:47:10.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d7fff640 1 -- 172.21.15.7:0/1417308501 learned_addr learned my addr 172.21.15.7:0/1417308501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d6ffd640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d8100f40 0x7f44d819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d7fff640 1 -- 172.21.15.7:0/1417308501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d8100f40 msgr2=0x7f44d819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d7fff640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d8100f40 0x7f44d819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d7fff640 1 -- 172.21.15.7:0/1417308501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d80ffd50 msgr2=0x7f44d819dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d7fff640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d80ffd50 0x7f44d819dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.874+0000 7f44d7fff640 1 -- 172.21.15.7:0/1417308501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d8110640 con 0x7f44d8101900 2024-03-26T08:47:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.875+0000 7f44d7fff640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8101900 0x7f44d81a5450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009780 tx=0x7f44cc004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.875+0000 7f44d4ff9640 1 -- 172.21.15.7:0/1417308501 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc039040 con 0x7f44d8101900 2024-03-26T08:47:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.876+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d81108d0 con 0x7f44d8101900 2024-03-26T08:47:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.876+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d8110dc0 con 0x7f44d8101900 2024-03-26T08:47:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.876+0000 7f44d4ff9640 1 -- 172.21.15.7:0/1417308501 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44cc042070 con 0x7f44d8101900 2024-03-26T08:47:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.876+0000 7f44d4ff9640 1 -- 172.21.15.7:0/1417308501 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc032800 con 0x7f44d8101900 2024-03-26T08:47:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.877+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c40052c0 con 0x7f44d8101900 2024-03-26T08:47:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.879+0000 7f44d4ff9640 1 -- 172.21.15.7:0/1417308501 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f44cc02c810 con 0x7f44d8101900 2024-03-26T08:47:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.880+0000 7f44d4ff9640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44c00a2a10 0x7f44c00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.880+0000 7f44d4ff9640 1 -- 172.21.15.7:0/1417308501 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44cc0f02c0 con 0x7f44d8101900 2024-03-26T08:47:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.880+0000 7f44d77fe640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44c00a2a10 0x7f44c00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:10.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.881+0000 7f44d77fe640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44c00a2a10 0x7f44c00a4ed0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f44d8100da0 tx=0x7f44d0009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:10.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:10.884+0000 7f44d4ff9640 1 -- 172.21.15.7:0/1417308501 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f44cc0bd320 con 0x7f44d8101900 2024-03-26T08:47:11.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 280.0 s, 3180.2 tps, lat 31.328 ms stddev 36.303 2024-03-26T08:47:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.282+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44c40055b0 con 0x7f44d8101900 2024-03-26T08:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:47:11.276 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:47:11.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.283+0000 7f44d4ff9640 1 -- 172.21.15.7:0/1417308501 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f44cc0bcc20 con 0x7f44d8101900 2024-03-26T08:47:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44c00a2a10 msgr2=0x7f44c00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44c00a2a10 0x7f44c00a4ed0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f44d8100da0 tx=0x7f44d0009670 comp rx=0 tx=0).stop 2024-03-26T08:47:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8101900 msgr2=0x7f44d81a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8101900 0x7f44d81a5450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009780 tx=0x7f44cc004530 comp rx=0 tx=0).stop 2024-03-26T08:47:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 shutdown_connections 2024-03-26T08:47:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44c00a2a10 0x7f44c00a4ed0 secure :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f44d8100da0 tx=0x7f44d0009670 comp rx=0 tx=0).stop 2024-03-26T08:47:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44d8101900 0x7f44d81a5450 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f44d8100f40 0x7f44d819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 --2- 172.21.15.7:0/1417308501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d80ffd50 0x7f44d819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 >> 172.21.15.7:0/1417308501 conn(0x7f44d80fb2c0 msgr2=0x7f44d810db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 shutdown_connections 2024-03-26T08:47:11.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.287+0000 7f44ddc44640 1 -- 172.21.15.7:0/1417308501 wait complete. 2024-03-26T08:47:11.296 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:47:11.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:47:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.468+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/3108552419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f18107360 msgr2=0x7f2f181097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.468+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/3108552419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f18107360 0x7f2f181097f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009ca0 tx=0x7f2f1002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.469+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/3108552419 shutdown_connections 2024-03-26T08:47:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.469+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/3108552419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f18107360 0x7f2f181097f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.469+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/3108552419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f180ffcd0 0x7f2f18106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.469+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/3108552419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f180ff2e0 0x7f2f180ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.469+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/3108552419 >> 172.21.15.7:0/3108552419 conn(0x7f2f180fae50 msgr2=0x7f2f180fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.469+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/3108552419 shutdown_connections 2024-03-26T08:47:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.469+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/3108552419 wait complete. 2024-03-26T08:47:11.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1f0e8640 1 Processor -- start 2024-03-26T08:47:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1f0e8640 1 -- start start 2024-03-26T08:47:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1f0e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f180ff2e0 0x7f2f18197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1f0e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f180ffcd0 0x7f2f18197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1f0e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f18107360 0x7f2f1819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1f0e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1810d330 con 0x7f2f180ffcd0 2024-03-26T08:47:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1f0e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1810d1b0 con 0x7f2f180ff2e0 2024-03-26T08:47:11.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1f0e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1810d4b0 con 0x7f2f18107360 2024-03-26T08:47:11.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1ce84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f180ff2e0 0x7f2f18197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f180ffcd0 0x7f2f18197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1ce84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f180ff2e0 0x7f2f18197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55268/0 (socket says 172.21.15.7:55268) 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1ce84640 1 -- 172.21.15.7:0/1781139428 learned_addr learned my addr 172.21.15.7:0/1781139428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.470+0000 7f2f1d685640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f18107360 0x7f2f1819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.471+0000 7f2f1d685640 1 -- 172.21.15.7:0/1781139428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f180ff2e0 msgr2=0x7f2f18197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.471+0000 7f2f1d685640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f180ff2e0 0x7f2f18197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.471+0000 7f2f1d685640 1 -- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f180ffcd0 msgr2=0x7f2f18197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.471+0000 7f2f1d685640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f180ffcd0 0x7f2f18197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.471+0000 7f2f1d685640 1 -- 172.21.15.7:0/1781139428 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1819f590 con 0x7f2f18107360 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.471+0000 7f2f17fff640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f180ffcd0 0x7f2f18197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.471+0000 7f2f1d685640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f18107360 0x7f2f1819ef20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2f1002c630 tx=0x7f2f10032f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.472+0000 7f2f15ffb640 1 -- 172.21.15.7:0/1781139428 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10044070 con 0x7f2f18107360 2024-03-26T08:47:11.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.472+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/1781139428 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1819f820 con 0x7f2f18107360 2024-03-26T08:47:11.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.472+0000 7f2f15ffb640 1 -- 172.21.15.7:0/1781139428 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f10034cd0 con 0x7f2f18107360 2024-03-26T08:47:11.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.472+0000 7f2f15ffb640 1 -- 172.21.15.7:0/1781139428 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10044070 con 0x7f2f18107360 2024-03-26T08:47:11.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.472+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/1781139428 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1819fd60 con 0x7f2f18107360 2024-03-26T08:47:11.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.474+0000 7f2f15ffb640 1 -- 172.21.15.7:0/1781139428 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2f10047070 con 0x7f2f18107360 2024-03-26T08:47:11.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.475+0000 7f2f21dab740 1 Processor -- start 2024-03-26T08:47:11.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.475+0000 7f2f21dab740 1 -- start start 2024-03-26T08:47:11.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.475+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.475+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 0x5606e5a5a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.475+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a5aa60 0x5606e5a5cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.475+0000 7f2f21dab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606e5a0f610 con 0x5606e5a55ea0 2024-03-26T08:47:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.475+0000 7f2f21dab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606e59b23c0 con 0x5606e5a5aa60 2024-03-26T08:47:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.475+0000 7f2f21dab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606e595ad60 con 0x5606e5a554b0 2024-03-26T08:47:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.475+0000 7f2f15ffb640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f080a2bf0 0x7f2f080a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f15ffb640 1 -- 172.21.15.7:0/1781139428 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f10034ad0 con 0x7f2f18107360 2024-03-26T08:47:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a5aa60 0x5606e5a5cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:55972/0 (socket says 172.21.15.7:55972) 2024-03-26T08:47:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f17fff640 1 -- 172.21.15.7:0/2834791625 learned_addr learned my addr 172.21.15.7:0/2834791625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f17fff640 1 -- 172.21.15.7:0/2834791625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a5aa60 msgr2=0x5606e5a5cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f17fff640 1 --2- 172.21.15.7:0/2834791625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a5aa60 0x5606e5a5cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f17fff640 1 -- 172.21.15.7:0/2834791625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 msgr2=0x5606e5a5a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f17fff640 1 --2- 172.21.15.7:0/2834791625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 0x5606e5a5a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.476+0000 7f2f17fff640 1 -- 172.21.15.7:0/2834791625 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606e56c63f0 con 0x5606e5a554b0 2024-03-26T08:47:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.477+0000 7f2f1d685640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f080a2bf0 0x7f2f080a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.477+0000 7f2f17fff640 1 --2- 172.21.15.7:0/2834791625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a558d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2f04013bf0 tx=0x7f2f04005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1706bb0d19e7df5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.477+0000 7f2f02ffd640 1 -- 172.21.15.7:0/2834791625 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f040130d0 con 0x5606e5a554b0 2024-03-26T08:47:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.477+0000 7f2f02ffd640 1 -- 172.21.15.7:0/2834791625 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f04013270 con 0x5606e5a554b0 2024-03-26T08:47:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.477+0000 7f2f02ffd640 1 -- 172.21.15.7:0/2834791625 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04004a00 con 0x5606e5a554b0 2024-03-26T08:47:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 -- 172.21.15.7:0/2834791625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 msgr2=0x5606e5a558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f1d685640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f080a2bf0 0x7f2f080a50b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2f100c48e0 tx=0x7f2f10004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 --2- 172.21.15.7:0/2834791625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a558d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2f04013bf0 tx=0x7f2f04005f90 comp rx=0 tx=0).stop 2024-03-26T08:47:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 -- 172.21.15.7:0/2834791625 shutdown_connections 2024-03-26T08:47:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 --2- 172.21.15.7:0/2834791625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a5aa60 0x5606e5a5cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 --2- 172.21.15.7:0/2834791625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 0x5606e5a5a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 --2- 172.21.15.7:0/2834791625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a558d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 -- 172.21.15.7:0/2834791625 >> 172.21.15.7:0/2834791625 conn(0x5606e5a47b80 msgr2=0x5606e5a47f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 -- 172.21.15.7:0/2834791625 shutdown_connections 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 -- 172.21.15.7:0/2834791625 wait complete. 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 -- start start 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.478+0000 7f2f21dab740 1 Processor -- start 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.479+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a554b0 0x5606e5a5f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.479+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 0x5606e5a5f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.479+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 0x5606e5a66820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.479+0000 7f2f21dab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606e596e240 con 0x5606e5a554b0 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.479+0000 7f2f21dab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606e59ae460 con 0x5606e5a55ea0 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.479+0000 7f2f21dab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606e5a5da50 con 0x5606e5a5aa60 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.479+0000 7f2f21dab740 10 client.0 _ll_register_callbacks cb 0x5606e596dc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.479+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a554b0 0x5606e5a5f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a554b0 0x5606e5a5f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47568/0 (socket says 172.21.15.7:47568) 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2f17fff640 1 -- 172.21.15.7:0/4025822213 learned_addr learned my addr 172.21.15.7:0/4025822213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2f17fff640 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 0x5606e5a66820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2f17fff640 1 -- 172.21.15.7:0/4025822213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 msgr2=0x5606e5a5f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2f17fff640 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 0x5606e5a5f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2f17fff640 1 -- 172.21.15.7:0/4025822213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a554b0 msgr2=0x5606e5a5f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2f17fff640 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a554b0 0x5606e5a5f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2f17fff640 1 -- 172.21.15.7:0/4025822213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606e57907d0 con 0x5606e5a5aa60 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2f17fff640 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 0x5606e5a66820 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f040058c0 tx=0x7f2f04013460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45706f331b688b91 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2ef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.480+0000 7f2ef7fff640 1 -- 172.21.15.7:0/4025822213 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04025020 con 0x5606e5a5aa60 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f21dab740 1 -- 172.21.15.7:0/4025822213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f04013270 con 0x5606e5a5aa60 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2ef7fff640 1 -- 172.21.15.7:0/4025822213 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f04004040 con 0x5606e5a5aa60 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2ef7fff640 1 -- 172.21.15.7:0/4025822213 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f04017070 con 0x5606e5a5aa60 2024-03-26T08:47:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f21dab740 10 client.26506 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f21dab740 1 -- 172.21.15.7:0/4025822213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5606e5a4e8d0 con 0x5606e5a5aa60 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2ef7fff640 1 -- 172.21.15.7:0/4025822213 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f04013270 con 0x5606e5a5aa60 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2ef7fff640 1 -- 172.21.15.7:0/4025822213 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2f04003680 con 0x5606e5a5aa60 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f21dab740 10 client.26506 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f21dab740 10 client.26506 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f21dab740 20 client.26506 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f21dab740 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5606e5a50290 0x5606e5a526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f21dab740 4 client.26506 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f21dab740 1 -- 172.21.15.7:0/4025822213 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606e59bf900 con 0x5606e5a50290 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.481+0000 7f2f1ce84640 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5606e5a50290 0x5606e5a526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.482+0000 7f2f1ce84640 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5606e5a50290 0x5606e5a526d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0d926c01d9c25ae server_cookie=562c6704a18e73ca in_seq=0 out_seq=0 2024-03-26T08:47:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.482+0000 7f2ef7fff640 10 client.26506 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:47:11.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.483+0000 7f2ef7fff640 1 -- 172.21.15.7:0/4025822213 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5606e59bf900 con 0x5606e5a50290 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.483+0000 7f2ef7fff640 10 client.26506 handle_command_reply: tid=0 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.483+0000 7f2f21dab740 1 client.26506 shutdown 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 -- 172.21.15.7:0/4025822213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 msgr2=0x5606e5a66820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 0x5606e5a66820 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f040058c0 tx=0x7f2f04013460 comp rx=0 tx=0).stop 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 -- 172.21.15.7:0/4025822213 shutdown_connections 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5606e5a50290 0x5606e5a526d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 0x5606e5a66820 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 0x5606e5a5f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 --2- 172.21.15.7:0/4025822213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a554b0 0x5606e5a5f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2ef7fff640 0 client.26506 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 -- 172.21.15.7:0/4025822213 >> 172.21.15.7:0/4025822213 conn(0x5606e5a47b80 msgr2=0x5606e5a54b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 -- 172.21.15.7:0/4025822213 shutdown_connections 2024-03-26T08:47:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 1 -- 172.21.15.7:0/4025822213 wait complete. 2024-03-26T08:47:11.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.484+0000 7f2f21dab740 20 client.26506 trim_cache size 0 max 16384 2024-03-26T08:47:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.505+0000 7f2f21dab740 1 Processor -- start 2024-03-26T08:47:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.505+0000 7f2f21dab740 1 -- start start 2024-03-26T08:47:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.505+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a7bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.505+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 0x5606e5a667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.505+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a5aa60 0x5606e5a7c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.505+0000 7f2f21dab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181a8db0 con 0x5606e5a5aa60 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.505+0000 7f2f21dab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181a68b0 con 0x5606e5a55ea0 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.505+0000 7f2f21dab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181a4930 con 0x5606e5a554b0 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.505+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a5aa60 0x5606e5a7c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a7bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a7bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:55976/0 (socket says 172.21.15.7:55976) 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f17fff640 1 -- 172.21.15.7:0/2435791431 learned_addr learned my addr 172.21.15.7:0/2435791431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f1ce84640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 0x5606e5a667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f1ce84640 1 -- 172.21.15.7:0/2435791431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 msgr2=0x5606e5a7bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f1ce84640 1 --2- 172.21.15.7:0/2435791431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a7bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f1ce84640 1 -- 172.21.15.7:0/2435791431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a5aa60 msgr2=0x5606e5a7c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f1ce84640 1 --2- 172.21.15.7:0/2435791431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a5aa60 0x5606e5a7c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f1ce84640 1 -- 172.21.15.7:0/2435791431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606e5a5d8f0 con 0x5606e5a55ea0 2024-03-26T08:47:11.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.506+0000 7f2f1ce84640 1 --2- 172.21.15.7:0/2435791431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 0x5606e5a667e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5606e5a60780 tx=0x7f2f0c009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c571190df28f7b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.507+0000 7f2f027fc640 1 -- 172.21.15.7:0/2435791431 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c005020 con 0x5606e5a55ea0 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.507+0000 7f2f027fc640 1 -- 172.21.15.7:0/2435791431 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f0c0047a0 con 0x5606e5a55ea0 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.507+0000 7f2f027fc640 1 -- 172.21.15.7:0/2435791431 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c012800 con 0x5606e5a55ea0 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 -- 172.21.15.7:0/2435791431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 msgr2=0x5606e5a667e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 --2- 172.21.15.7:0/2435791431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 0x5606e5a667e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5606e5a60780 tx=0x7f2f0c009740 comp rx=0 tx=0).stop 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 -- 172.21.15.7:0/2435791431 shutdown_connections 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 --2- 172.21.15.7:0/2435791431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a5aa60 0x5606e5a7c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 --2- 172.21.15.7:0/2435791431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a55ea0 0x5606e5a667e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 --2- 172.21.15.7:0/2435791431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a554b0 0x5606e5a7bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 -- 172.21.15.7:0/2435791431 >> 172.21.15.7:0/2435791431 conn(0x5606e5a47b80 msgr2=0x5606e5a47f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 -- 172.21.15.7:0/2435791431 shutdown_connections 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 -- 172.21.15.7:0/2435791431 wait complete. 2024-03-26T08:47:11.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 -- start start 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.508+0000 7f2f21dab740 1 Processor -- start 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a554b0 0x5606e5a64420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 0x5606e5a64960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f21dab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 0x5606e5a65970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f21dab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181a4f30 con 0x5606e5a55ea0 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f21dab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181a6250 con 0x5606e5a554b0 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f21dab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f181a5590 con 0x5606e5a5aa60 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f21dab740 10 client.0 _ll_register_callbacks cb 0x7f2f040054d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f1ce84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 0x5606e5a64960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a554b0 0x5606e5a64420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f1ce84640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 0x5606e5a64960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47592/0 (socket says 172.21.15.7:47592) 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f1ce84640 1 -- 172.21.15.7:0/1618859253 learned_addr learned my addr 172.21.15.7:0/1618859253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f17fff640 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 0x5606e5a65970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f17fff640 1 -- 172.21.15.7:0/1618859253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 msgr2=0x5606e5a65970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f17fff640 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 0x5606e5a65970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f17fff640 1 -- 172.21.15.7:0/1618859253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 msgr2=0x5606e5a64960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f17fff640 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 0x5606e5a64960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f17fff640 1 -- 172.21.15.7:0/1618859253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606e5a65f60 con 0x5606e5a554b0 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.509+0000 7f2f1ce84640 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 0x5606e5a64960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.510+0000 7f2f17fff640 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a554b0 0x5606e5a64420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2f04001eb0 tx=0x7f2f040130f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a00629d1e930fc64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.510+0000 7f2ef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.510+0000 7f2ef7fff640 1 -- 172.21.15.7:0/1618859253 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04025040 con 0x5606e5a554b0 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.510+0000 7f2ef7fff640 1 -- 172.21.15.7:0/1618859253 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f04004040 con 0x5606e5a554b0 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.510+0000 7f2f21dab740 1 -- 172.21.15.7:0/1618859253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f0c0047a0 con 0x5606e5a554b0 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.511+0000 7f2ef7fff640 1 -- 172.21.15.7:0/1618859253 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f04004240 con 0x5606e5a554b0 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.511+0000 7f2f21dab740 10 client.26477 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.511+0000 7f2f21dab740 1 -- 172.21.15.7:0/1618859253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5606e5acee90 con 0x5606e5a554b0 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.511+0000 7f2ef7fff640 1 -- 172.21.15.7:0/1618859253 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2f04004db0 con 0x5606e5a554b0 2024-03-26T08:47:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.511+0000 7f2ef7fff640 1 -- 172.21.15.7:0/1618859253 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2f04011640 con 0x5606e5a554b0 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.511+0000 7f2f21dab740 10 client.26477 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.512+0000 7f2f21dab740 10 client.26477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.512+0000 7f2f21dab740 20 client.26477 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.512+0000 7f2f21dab740 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5606e5acf680 0x5606e5ad1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.512+0000 7f2f21dab740 4 client.26477 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.512+0000 7f2f21dab740 1 -- 172.21.15.7:0/1618859253 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f2f0c0047a0 con 0x5606e5acf680 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.512+0000 7f2f1ce84640 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5606e5acf680 0x5606e5ad1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.512+0000 7f2ef7fff640 10 client.26477 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.512+0000 7f2f1ce84640 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5606e5acf680 0x5606e5ad1aa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb705444299b3d5a server_cookie=379e279f81fecef4 in_seq=0 out_seq=0 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2ef7fff640 1 -- 172.21.15.7:0/1618859253 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f2f0c0047a0 con 0x5606e5acf680 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2ef7fff640 10 client.26477 handle_command_reply: tid=0 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 client.26477 shutdown 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 -- 172.21.15.7:0/1618859253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a554b0 msgr2=0x5606e5a64420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a554b0 0x5606e5a64420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2f04001eb0 tx=0x7f2f040130f0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 -- 172.21.15.7:0/1618859253 shutdown_connections 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5606e5acf680 0x5606e5ad1aa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606e5a5aa60 0x5606e5a65970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606e5a55ea0 0x5606e5a64960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 --2- 172.21.15.7:0/1618859253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606e5a554b0 0x5606e5a64420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2ef7fff640 0 client.26477 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 -- 172.21.15.7:0/1618859253 >> 172.21.15.7:0/1618859253 conn(0x5606e5a47b80 msgr2=0x5606e5a5dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 -- 172.21.15.7:0/1618859253 shutdown_connections 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 1 -- 172.21.15.7:0/1618859253 wait complete. 2024-03-26T08:47:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.513+0000 7f2f21dab740 20 client.26477 trim_cache size 0 max 16384 2024-03-26T08:47:11.512 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:47:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f080a2bf0 msgr2=0x7f2f080a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f080a2bf0 0x7f2f080a50b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2f100c48e0 tx=0x7f2f10004a40 comp rx=0 tx=0).stop 2024-03-26T08:47:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/1781139428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f18107360 msgr2=0x7f2f1819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f18107360 0x7f2f1819ef20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2f1002c630 tx=0x7f2f10032f40 comp rx=0 tx=0).stop 2024-03-26T08:47:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/1781139428 shutdown_connections 2024-03-26T08:47:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f080a2bf0 0x7f2f080a50b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f18107360 0x7f2f1819ef20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f180ffcd0 0x7f2f18197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 --2- 172.21.15.7:0/1781139428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f180ff2e0 0x7f2f18197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.514+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/1781139428 >> 172.21.15.7:0/1781139428 conn(0x7f2f180fae50 msgr2=0x7f2f18102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.515+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/1781139428 shutdown_connections 2024-03-26T08:47:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.515+0000 7f2f1f0e8640 1 -- 172.21.15.7:0/1781139428 wait complete. 2024-03-26T08:47:11.523 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:47:11.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:47:11.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.704+0000 7f60a6b09640 1 -- 172.21.15.7:0/3630757377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a0103280 msgr2=0x7f60a0109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.704+0000 7f60a6b09640 1 --2- 172.21.15.7:0/3630757377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a0103280 0x7f60a0109b10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6098009860 tx=0x7f609802bb70 comp rx=0 tx=0).stop 2024-03-26T08:47:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.705+0000 7f60a6b09640 1 -- 172.21.15.7:0/3630757377 shutdown_connections 2024-03-26T08:47:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.705+0000 7f60a6b09640 1 --2- 172.21.15.7:0/3630757377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a0103280 0x7f60a0109b10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.705+0000 7f60a6b09640 1 --2- 172.21.15.7:0/3630757377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a01028c0 0x7f60a0102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.705+0000 7f60a6b09640 1 --2- 172.21.15.7:0/3630757377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a01016d0 0x7f60a0101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.705+0000 7f60a6b09640 1 -- 172.21.15.7:0/3630757377 >> 172.21.15.7:0/3630757377 conn(0x7f60a00fcc60 msgr2=0x7f60a00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.706+0000 7f60a6b09640 1 -- 172.21.15.7:0/3630757377 shutdown_connections 2024-03-26T08:47:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.706+0000 7f60a6b09640 1 -- 172.21.15.7:0/3630757377 wait complete. 2024-03-26T08:47:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a6b09640 1 Processor -- start 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a6b09640 1 -- start start 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a6b09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a01016d0 0x7f60a0197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a6b09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a01028c0 0x7f60a0197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a48a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a01016d0 0x7f60a0197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a48a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a01016d0 0x7f60a0197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47610/0 (socket says 172.21.15.7:47610) 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a48a5640 1 -- 172.21.15.7:0/1369358925 learned_addr learned my addr 172.21.15.7:0/1369358925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a6b09640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a0103280 0x7f60a019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a010d350 con 0x7f60a01016d0 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a010d1d0 con 0x7f60a01028c0 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f609ffff640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a01028c0 0x7f60a0197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.707+0000 7f60a50a6640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a0103280 0x7f60a019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.708+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a010d4d0 con 0x7f60a0103280 2024-03-26T08:47:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.708+0000 7f609ffff640 1 -- 172.21.15.7:0/1369358925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a0103280 msgr2=0x7f60a019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.708+0000 7f609ffff640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a0103280 0x7f60a019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.708+0000 7f609ffff640 1 -- 172.21.15.7:0/1369358925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a01016d0 msgr2=0x7f60a0197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.708+0000 7f609ffff640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a01016d0 0x7f60a0197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.708+0000 7f609ffff640 1 -- 172.21.15.7:0/1369358925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a019f5b0 con 0x7f60a01028c0 2024-03-26T08:47:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.709+0000 7f609ffff640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a01028c0 0x7f60a0197ed0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6088009e40 tx=0x7f608800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.709+0000 7f609dffb640 1 -- 172.21.15.7:0/1369358925 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608800ccc0 con 0x7f60a01028c0 2024-03-26T08:47:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.709+0000 7f609dffb640 1 -- 172.21.15.7:0/1369358925 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6088005020 con 0x7f60a01028c0 2024-03-26T08:47:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.709+0000 7f609dffb640 1 -- 172.21.15.7:0/1369358925 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088013a30 con 0x7f60a01028c0 2024-03-26T08:47:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.709+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a0073930 con 0x7f60a01028c0 2024-03-26T08:47:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.709+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a0073e30 con 0x7f60a01028c0 2024-03-26T08:47:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.712+0000 7f609dffb640 1 -- 172.21.15.7:0/1369358925 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6088022020 con 0x7f60a01028c0 2024-03-26T08:47:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.713+0000 7f609dffb640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f608c0a29d0 0x7f608c0a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.713+0000 7f609dffb640 1 -- 172.21.15.7:0/1369358925 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f60880d2a70 con 0x7f60a01028c0 2024-03-26T08:47:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.713+0000 7f60a48a5640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f608c0a29d0 0x7f608c0a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.714+0000 7f60a48a5640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f608c0a29d0 0x7f608c0a4e90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6090009a60 tx=0x7f6090010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.714+0000 7f60a97d4740 1 Processor -- start 2024-03-26T08:47:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a97d4740 1 -- start start 2024-03-26T08:47:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5822ea0 0x5652f58232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 0x5652f5828390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 0x5652f582ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a97d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f57dd610 con 0x5652f5823890 2024-03-26T08:47:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a97d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f57803c0 con 0x5652f5822ea0 2024-03-26T08:47:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a97d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f5728d60 con 0x5652f5828a00 2024-03-26T08:47:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f609ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 0x5652f5828390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a50a6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 0x5652f582ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a48a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5822ea0 0x5652f58232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f609ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 0x5652f5828390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47620/0 (socket says 172.21.15.7:47620) 2024-03-26T08:47:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f609ffff640 1 -- 172.21.15.7:0/2405359667 learned_addr learned my addr 172.21.15.7:0/2405359667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a48a5640 1 -- 172.21.15.7:0/2405359667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 msgr2=0x5652f582ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a48a5640 1 --2- 172.21.15.7:0/2405359667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 0x5652f582ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a48a5640 1 -- 172.21.15.7:0/2405359667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 msgr2=0x5652f5828390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a48a5640 1 --2- 172.21.15.7:0/2405359667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 0x5652f5828390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.715+0000 7f60a48a5640 1 -- 172.21.15.7:0/2405359667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652f54943f0 con 0x5652f5822ea0 2024-03-26T08:47:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.716+0000 7f60a48a5640 1 --2- 172.21.15.7:0/2405359667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5822ea0 0x5652f58232c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6090005c40 tx=0x7f6090014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fdc573936556e85 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.716+0000 7f6096ffd640 1 -- 172.21.15.7:0/2405359667 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090019a10 con 0x5652f5822ea0 2024-03-26T08:47:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.716+0000 7f6096ffd640 1 -- 172.21.15.7:0/2405359667 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6090028070 con 0x5652f5822ea0 2024-03-26T08:47:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.716+0000 7f6096ffd640 1 -- 172.21.15.7:0/2405359667 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60900185d0 con 0x5652f5822ea0 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.717+0000 7f60a97d4740 1 -- 172.21.15.7:0/2405359667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5822ea0 msgr2=0x5652f58232c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.717+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2405359667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5822ea0 0x5652f58232c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6090005c40 tx=0x7f6090014000 comp rx=0 tx=0).stop 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.717+0000 7f60a97d4740 1 -- 172.21.15.7:0/2405359667 shutdown_connections 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.717+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2405359667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 0x5652f582ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.717+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2405359667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 0x5652f5828390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.717+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2405359667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5822ea0 0x5652f58232c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.717+0000 7f60a97d4740 1 -- 172.21.15.7:0/2405359667 >> 172.21.15.7:0/2405359667 conn(0x5652f5815c00 msgr2=0x5652f5816000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.717+0000 7f60a97d4740 1 -- 172.21.15.7:0/2405359667 shutdown_connections 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.717+0000 7f60a97d4740 1 -- 172.21.15.7:0/2405359667 wait complete. 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.718+0000 7f60a97d4740 1 -- start start 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.718+0000 7f60a97d4740 1 Processor -- start 2024-03-26T08:47:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.718+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5822ea0 0x5652f582d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.718+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 0x5652f582d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.718+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 0x5652f58345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a97d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f573c240 con 0x5652f5823890 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a97d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f577c460 con 0x5652f5828a00 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a97d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652f582b9b0 con 0x5652f5822ea0 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a48a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5822ea0 0x5652f582d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a97d4740 10 client.0 _ll_register_callbacks cb 0x5652f573bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a48a5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5822ea0 0x5652f582d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56012/0 (socket says 172.21.15.7:56012) 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a48a5640 1 -- 172.21.15.7:0/3580418294 learned_addr learned my addr 172.21.15.7:0/3580418294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f609ffff640 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 0x5652f582d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a50a6640 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 0x5652f58345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a48a5640 1 -- 172.21.15.7:0/3580418294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 msgr2=0x5652f58345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a48a5640 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 0x5652f58345b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a48a5640 1 -- 172.21.15.7:0/3580418294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 msgr2=0x5652f582d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a48a5640 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 0x5652f582d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a48a5640 1 -- 172.21.15.7:0/3580418294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652f555e7d0 con 0x5652f5822ea0 2024-03-26T08:47:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a50a6640 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 0x5652f58345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f60a48a5640 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5822ea0 0x5652f582d030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6090009e10 tx=0x7f6090023620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd44ad101a85cba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.719+0000 7f607ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.720+0000 7f607ffff640 1 -- 172.21.15.7:0/3580418294 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090004e10 con 0x5652f5822ea0 2024-03-26T08:47:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.720+0000 7f60a97d4740 1 -- 172.21.15.7:0/3580418294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6090028070 con 0x5652f5822ea0 2024-03-26T08:47:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.720+0000 7f607ffff640 1 -- 172.21.15.7:0/3580418294 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f60900188d0 con 0x5652f5822ea0 2024-03-26T08:47:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.720+0000 7f607ffff640 1 -- 172.21.15.7:0/3580418294 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6090018ad0 con 0x5652f5822ea0 2024-03-26T08:47:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f60a97d4740 10 client.26524 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f60a97d4740 1 -- 172.21.15.7:0/3580418294 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5652f5837970 con 0x5652f5822ea0 2024-03-26T08:47:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f607ffff640 1 -- 172.21.15.7:0/3580418294 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6090028070 con 0x5652f5822ea0 2024-03-26T08:47:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f607ffff640 1 -- 172.21.15.7:0/3580418294 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f609000b040 con 0x5652f5822ea0 2024-03-26T08:47:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f60a97d4740 10 client.26524 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f60a97d4740 10 client.26524 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:47:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f60a97d4740 20 client.26524 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f60a97d4740 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5652f5838d20 0x5652f583b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f60a97d4740 4 client.26524 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.722+0000 7f60a97d4740 1 -- 172.21.15.7:0/3580418294 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652f578d900 con 0x5652f5838d20 2024-03-26T08:47:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.723+0000 7f609ffff640 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5652f5838d20 0x5652f583b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.723+0000 7f609ffff640 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5652f5838d20 0x5652f583b110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b68e8897b51e19a server_cookie=7962ed3eb60195f8 in_seq=0 out_seq=0 2024-03-26T08:47:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.723+0000 7f607ffff640 10 client.26524 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:47:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.725+0000 7f607ffff640 1 -- 172.21.15.7:0/3580418294 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5652f578d900 con 0x5652f5838d20 2024-03-26T08:47:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.725+0000 7f607ffff640 10 client.26524 handle_command_reply: tid=0 2024-03-26T08:47:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.725+0000 7f60a97d4740 1 client.26524 shutdown 2024-03-26T08:47:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 -- 172.21.15.7:0/3580418294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5822ea0 msgr2=0x5652f582d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5822ea0 0x5652f582d030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6090009e10 tx=0x7f6090023620 comp rx=0 tx=0).stop 2024-03-26T08:47:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 -- 172.21.15.7:0/3580418294 shutdown_connections 2024-03-26T08:47:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 0x5652f58345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5823890 0x5652f582d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5652f5838d20 0x5652f583b110 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f607ffff640 0 client.26524 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:47:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 --2- 172.21.15.7:0/3580418294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5822ea0 0x5652f582d030 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 -- 172.21.15.7:0/3580418294 >> 172.21.15.7:0/3580418294 conn(0x5652f5815c00 msgr2=0x5652f5814e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 -- 172.21.15.7:0/3580418294 shutdown_connections 2024-03-26T08:47:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 1 -- 172.21.15.7:0/3580418294 wait complete. 2024-03-26T08:47:11.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.726+0000 7f60a97d4740 20 client.26524 trim_cache size 0 max 16384 2024-03-26T08:47:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.746+0000 7f60a97d4740 1 Processor -- start 2024-03-26T08:47:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.746+0000 7f60a97d4740 1 -- start start 2024-03-26T08:47:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f582cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5823890 0x5652f5834570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 0x5652f582d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a97d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01a8db0 con 0x5652f5822ea0 2024-03-26T08:47:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a97d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01a68b0 con 0x5652f5828a00 2024-03-26T08:47:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a97d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01a4930 con 0x5652f5823890 2024-03-26T08:47:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a48a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f582cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a48a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f582cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47640/0 (socket says 172.21.15.7:47640) 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a48a5640 1 -- 172.21.15.7:0/2308132565 learned_addr learned my addr 172.21.15.7:0/2308132565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f609ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5823890 0x5652f5834570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a50a6640 1 --2- 172.21.15.7:0/2308132565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 0x5652f582d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a48a5640 1 -- 172.21.15.7:0/2308132565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5823890 msgr2=0x5652f5834570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a48a5640 1 --2- 172.21.15.7:0/2308132565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5823890 0x5652f5834570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a48a5640 1 -- 172.21.15.7:0/2308132565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 msgr2=0x5652f582d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a48a5640 1 --2- 172.21.15.7:0/2308132565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 0x5652f582d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a48a5640 1 -- 172.21.15.7:0/2308132565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652f583b480 con 0x5652f5822ea0 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.747+0000 7f60a48a5640 1 --2- 172.21.15.7:0/2308132565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f582cc40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6090025040 tx=0x7f6090009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d414bd996b07c4e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60967fc640 1 -- 172.21.15.7:0/2308132565 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090018c70 con 0x5652f5822ea0 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60967fc640 1 -- 172.21.15.7:0/2308132565 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6090028070 con 0x5652f5822ea0 2024-03-26T08:47:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60967fc640 1 -- 172.21.15.7:0/2308132565 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090005690 con 0x5652f5822ea0 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60a97d4740 1 -- 172.21.15.7:0/2308132565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 msgr2=0x5652f582cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2308132565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f582cc40 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6090025040 tx=0x7f6090009640 comp rx=0 tx=0).stop 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60a97d4740 1 -- 172.21.15.7:0/2308132565 shutdown_connections 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2308132565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5828a00 0x5652f582d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2308132565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5823890 0x5652f5834570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2308132565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f582cc40 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60a97d4740 1 -- 172.21.15.7:0/2308132565 >> 172.21.15.7:0/2308132565 conn(0x5652f5815c00 msgr2=0x5652f5824bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.748+0000 7f60a97d4740 1 -- 172.21.15.7:0/2308132565 shutdown_connections 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.749+0000 7f60a97d4740 1 -- 172.21.15.7:0/2308132565 wait complete. 2024-03-26T08:47:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.749+0000 7f60a97d4740 1 -- start start 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.749+0000 7f60a97d4740 1 Processor -- start 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.750+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f5832630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.750+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5823890 0x5652f5832b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.750+0000 7f60a97d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 0x5652f5833ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.750+0000 7f60a97d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01a4f30 con 0x5652f5822ea0 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.750+0000 7f60a97d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01a6250 con 0x5652f5823890 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.750+0000 7f60a48a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f5832630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a97d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01a5590 con 0x5652f5828a00 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a48a5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f5832630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47654/0 (socket says 172.21.15.7:47654) 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a48a5640 1 -- 172.21.15.7:0/2531577229 learned_addr learned my addr 172.21.15.7:0/2531577229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a50a6640 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 0x5652f5833ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a97d4740 10 client.0 _ll_register_callbacks cb 0x7f609000d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f609ffff640 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5823890 0x5652f5832b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a50a6640 1 -- 172.21.15.7:0/2531577229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5823890 msgr2=0x5652f5832b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a50a6640 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5823890 0x5652f5832b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a50a6640 1 -- 172.21.15.7:0/2531577229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 msgr2=0x5652f5832630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a50a6640 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f5832630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a50a6640 1 -- 172.21.15.7:0/2531577229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652f58340d0 con 0x5652f5828a00 2024-03-26T08:47:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f60a50a6640 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 0x5652f5833ae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6098009780 tx=0x7f60980324b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=466423cf9746d748 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.751+0000 7f607ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.752+0000 7f607ffff640 1 -- 172.21.15.7:0/2531577229 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098039040 con 0x5652f5828a00 2024-03-26T08:47:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.752+0000 7f607ffff640 1 -- 172.21.15.7:0/2531577229 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6098004380 con 0x5652f5828a00 2024-03-26T08:47:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.752+0000 7f60a97d4740 1 -- 172.21.15.7:0/2531577229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6090028070 con 0x5652f5828a00 2024-03-26T08:47:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f607ffff640 1 -- 172.21.15.7:0/2531577229 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f609802c510 con 0x5652f5828a00 2024-03-26T08:47:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f60a97d4740 10 client.26530 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f60a97d4740 1 -- 172.21.15.7:0/2531577229 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5652f5894e80 con 0x5652f5828a00 2024-03-26T08:47:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f607ffff640 1 -- 172.21.15.7:0/2531577229 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6090028070 con 0x5652f5828a00 2024-03-26T08:47:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f607ffff640 1 -- 172.21.15.7:0/2531577229 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6098032c60 con 0x5652f5828a00 2024-03-26T08:47:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f60a97d4740 10 client.26530 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f60a97d4740 10 client.26530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:47:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f60a97d4740 20 client.26530 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5652f5838d20 0x5652f5897820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f60a97d4740 4 client.26530 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:47:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.755+0000 7f60a97d4740 1 -- 172.21.15.7:0/2531577229 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5652f5787c50 con 0x5652f5838d20 2024-03-26T08:47:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.756+0000 7f60a48a5640 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5652f5838d20 0x5652f5897820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.757+0000 7f60a48a5640 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5652f5838d20 0x5652f5897820 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fd401c210312727 server_cookie=e1cdffca2f740825 in_seq=0 out_seq=0 2024-03-26T08:47:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.757+0000 7f607ffff640 10 client.26530 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:47:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.757+0000 7f607ffff640 1 -- 172.21.15.7:0/2531577229 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5652f5787c50 con 0x5652f5838d20 2024-03-26T08:47:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.757+0000 7f607ffff640 10 client.26530 handle_command_reply: tid=0 2024-03-26T08:47:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.757+0000 7f60a97d4740 1 client.26530 shutdown 2024-03-26T08:47:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 -- 172.21.15.7:0/2531577229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 msgr2=0x5652f5833ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 0x5652f5833ae0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6098009780 tx=0x7f60980324b0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 -- 172.21.15.7:0/2531577229 shutdown_connections 2024-03-26T08:47:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5652f5838d20 0x5652f5897820 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5652f5828a00 0x5652f5833ae0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5652f5823890 0x5652f5832b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 --2- 172.21.15.7:0/2531577229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5652f5822ea0 0x5652f5832630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 -- 172.21.15.7:0/2531577229 >> 172.21.15.7:0/2531577229 conn(0x5652f5815c00 msgr2=0x5652f582cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f607ffff640 0 client.26530 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:47:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 -- 172.21.15.7:0/2531577229 shutdown_connections 2024-03-26T08:47:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.758+0000 7f60a97d4740 1 -- 172.21.15.7:0/2531577229 wait complete. 2024-03-26T08:47:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.759+0000 7f60a97d4740 20 client.26530 trim_cache size 0 max 16384 2024-03-26T08:47:11.752 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:47:11.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.761+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f608c0a29d0 msgr2=0x7f608c0a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.762+0000 7f60a6b09640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f608c0a29d0 0x7f608c0a4e90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6090009a60 tx=0x7f6090010040 comp rx=0 tx=0).stop 2024-03-26T08:47:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.762+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a01028c0 msgr2=0x7f60a0197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.762+0000 7f60a6b09640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a01028c0 0x7f60a0197ed0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6088009e40 tx=0x7f608800edc0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.762+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 shutdown_connections 2024-03-26T08:47:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.762+0000 7f60a6b09640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f608c0a29d0 0x7f608c0a4e90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.762+0000 7f60a6b09640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60a0103280 0x7f60a019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.762+0000 7f60a6b09640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a01028c0 0x7f60a0197ed0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.762+0000 7f60a6b09640 1 --2- 172.21.15.7:0/1369358925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60a01016d0 0x7f60a0197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.762+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 >> 172.21.15.7:0/1369358925 conn(0x7f60a00fcc60 msgr2=0x7f60a0105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.763+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 shutdown_connections 2024-03-26T08:47:11.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.765+0000 7f60a6b09640 1 -- 172.21.15.7:0/1369358925 wait complete. 2024-03-26T08:47:11.775 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:47:11.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:47:11.815 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:11 smithi007 ceph-mon[32462]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 757 op/s 2024-03-26T08:47:11.815 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1417308501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:47:11.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:11 smithi044 ceph-mon[36461]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 757 op/s 2024-03-26T08:47:11.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1417308501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:47:11.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.951+0000 7fa829627640 1 -- 172.21.15.7:0/3754094049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 msgr2=0x7fa824109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.951+0000 7fa829627640 1 --2- 172.21.15.7:0/3754094049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 0x7fa824109b10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009ca0 tx=0x7fa81c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.952+0000 7fa829627640 1 -- 172.21.15.7:0/3754094049 shutdown_connections 2024-03-26T08:47:11.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.952+0000 7fa829627640 1 --2- 172.21.15.7:0/3754094049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 0x7fa824109b10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.952+0000 7fa829627640 1 --2- 172.21.15.7:0/3754094049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8241028c0 0x7fa824102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.952+0000 7fa829627640 1 --2- 172.21.15.7:0/3754094049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8241016d0 0x7fa824101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.952+0000 7fa829627640 1 -- 172.21.15.7:0/3754094049 >> 172.21.15.7:0/3754094049 conn(0x7fa8240fcc60 msgr2=0x7fa8240ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.952+0000 7fa829627640 1 -- 172.21.15.7:0/3754094049 shutdown_connections 2024-03-26T08:47:11.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.952+0000 7fa829627640 1 -- 172.21.15.7:0/3754094049 wait complete. 2024-03-26T08:47:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.953+0000 7fa829627640 1 Processor -- start 2024-03-26T08:47:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.953+0000 7fa829627640 1 -- start start 2024-03-26T08:47:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.953+0000 7fa829627640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8241016d0 0x7fa824075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa829627640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8241028c0 0x7fa824073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa829627640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 0x7fa824073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa829627640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82410d330 con 0x7fa824103280 2024-03-26T08:47:11.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa829627640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82410d1b0 con 0x7fa8241028c0 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa829627640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82410d4b0 con 0x7fa8241016d0 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 0x7fa824073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa822ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8241016d0 0x7fa824075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8241028c0 0x7fa824073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa822ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8241016d0 0x7fa824075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56048/0 (socket says 172.21.15.7:56048) 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 0x7fa824073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47670/0 (socket says 172.21.15.7:47670) 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa822ffd640 1 -- 172.21.15.7:0/286567335 learned_addr learned my addr 172.21.15.7:0/286567335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8237fe640 1 -- 172.21.15.7:0/286567335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8241016d0 msgr2=0x7fa824075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8237fe640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8241016d0 0x7fa824075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8237fe640 1 -- 172.21.15.7:0/286567335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8241028c0 msgr2=0x7fa824073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8237fe640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8241028c0 0x7fa824073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8237fe640 1 -- 172.21.15.7:0/286567335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824074860 con 0x7fa824103280 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa822ffd640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8241016d0 0x7fa824075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8227fc640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8241028c0 0x7fa824073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.954+0000 7fa8237fe640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 0x7fa824073fd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02c630 tx=0x7fa81c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.955+0000 7fa80bfff640 1 -- 172.21.15.7:0/286567335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c039040 con 0x7fa824103280 2024-03-26T08:47:11.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.955+0000 7fa80bfff640 1 -- 172.21.15.7:0/286567335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa81c044070 con 0x7fa824103280 2024-03-26T08:47:11.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.955+0000 7fa80bfff640 1 -- 172.21.15.7:0/286567335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c0042a0 con 0x7fa824103280 2024-03-26T08:47:11.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.955+0000 7fa829627640 1 -- 172.21.15.7:0/286567335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8241a1a10 con 0x7fa824103280 2024-03-26T08:47:11.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.955+0000 7fa829627640 1 -- 172.21.15.7:0/286567335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8241a1ee0 con 0x7fa824103280 2024-03-26T08:47:11.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.958+0000 7fa80bfff640 1 -- 172.21.15.7:0/286567335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa81c004440 con 0x7fa824103280 2024-03-26T08:47:11.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.959+0000 7fa80bfff640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa80c0a2900 0x7fa80c0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.960+0000 7fa80bfff640 1 -- 172.21.15.7:0/286567335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa81c0f0cb0 con 0x7fa824103280 2024-03-26T08:47:11.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.960+0000 7fa822ffd640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa80c0a2900 0x7fa80c0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.960+0000 7fa822ffd640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa80c0a2900 0x7fa80c0a4dc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa8140104e0 tx=0x7fa81400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.961+0000 7fa82c214740 1 Processor -- start 2024-03-26T08:47:11.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.961+0000 7fa82c214740 1 -- start start 2024-03-26T08:47:11.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.961+0000 7fa82c214740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5ed9ea0 0x5649d5eda2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa82c214740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5edf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa82c214740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 0x5649d5ee1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa82c214740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5e94610 con 0x5649d5eda890 2024-03-26T08:47:11.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa82c214740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5e373c0 con 0x5649d5ed9ea0 2024-03-26T08:47:11.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa82c214740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5ddfd60 con 0x5649d5edfa00 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 0x5649d5ee1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 0x5649d5ee1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56064/0 (socket says 172.21.15.7:56064) 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa8237fe640 1 -- 172.21.15.7:0/3554118135 learned_addr learned my addr 172.21.15.7:0/3554118135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa8227fc640 1 --2- 172.21.15.7:0/3554118135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5edf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa822ffd640 1 --2- 172.21.15.7:0/3554118135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5ed9ea0 0x5649d5eda2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa8227fc640 1 -- 172.21.15.7:0/3554118135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 msgr2=0x5649d5ee1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.962+0000 7fa8227fc640 1 --2- 172.21.15.7:0/3554118135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 0x5649d5ee1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.963+0000 7fa8227fc640 1 -- 172.21.15.7:0/3554118135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5ed9ea0 msgr2=0x5649d5eda2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.963+0000 7fa8227fc640 1 --2- 172.21.15.7:0/3554118135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5ed9ea0 0x5649d5eda2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.963+0000 7fa8227fc640 1 -- 172.21.15.7:0/3554118135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d5b4b3f0 con 0x5649d5eda890 2024-03-26T08:47:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.963+0000 7fa8227fc640 1 --2- 172.21.15.7:0/3554118135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5edf390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa818009fa0 tx=0x7fa818011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6611816e7707da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.964+0000 7fa8097fa640 1 -- 172.21.15.7:0/3554118135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818004230 con 0x5649d5eda890 2024-03-26T08:47:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.964+0000 7fa8097fa640 1 -- 172.21.15.7:0/3554118135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8180043d0 con 0x5649d5eda890 2024-03-26T08:47:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.965+0000 7fa8097fa640 1 -- 172.21.15.7:0/3554118135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818004230 con 0x5649d5eda890 2024-03-26T08:47:11.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.966+0000 7fa82c214740 1 -- 172.21.15.7:0/3554118135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 msgr2=0x5649d5edf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.966+0000 7fa82c214740 1 --2- 172.21.15.7:0/3554118135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5edf390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa818009fa0 tx=0x7fa818011810 comp rx=0 tx=0).stop 2024-03-26T08:47:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.969+0000 7fa82c214740 1 -- 172.21.15.7:0/3554118135 shutdown_connections 2024-03-26T08:47:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.969+0000 7fa82c214740 1 --2- 172.21.15.7:0/3554118135 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 0x5649d5ee1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.969+0000 7fa82c214740 1 --2- 172.21.15.7:0/3554118135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5edf390 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.969+0000 7fa82c214740 1 --2- 172.21.15.7:0/3554118135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5ed9ea0 0x5649d5eda2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.969+0000 7fa82c214740 1 -- 172.21.15.7:0/3554118135 >> 172.21.15.7:0/3554118135 conn(0x5649d5eccc00 msgr2=0x5649d5ecd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.970+0000 7fa82c214740 1 -- 172.21.15.7:0/3554118135 shutdown_connections 2024-03-26T08:47:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.970+0000 7fa82c214740 1 -- 172.21.15.7:0/3554118135 wait complete. 2024-03-26T08:47:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.970+0000 7fa82c214740 1 -- start start 2024-03-26T08:47:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.971+0000 7fa82c214740 1 Processor -- start 2024-03-26T08:47:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.971+0000 7fa82c214740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.971+0000 7fa82c214740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5ee4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.971+0000 7fa82c214740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 0x5649d5eeb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.971+0000 7fa822ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.971+0000 7fa822ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56066/0 (socket says 172.21.15.7:56066) 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa82c214740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5df3240 con 0x5649d5eda890 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa82c214740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5e33460 con 0x5649d5edfa00 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa82c214740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649d5ee29b0 con 0x5649d5ed9ea0 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa82c214740 10 client.0 _ll_register_callbacks cb 0x5649d5df2c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa822ffd640 1 -- 172.21.15.7:0/1011557193 learned_addr learned my addr 172.21.15.7:0/1011557193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa822ffd640 1 -- 172.21.15.7:0/1011557193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 msgr2=0x5649d5eeb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa822ffd640 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 0x5649d5eeb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa822ffd640 1 -- 172.21.15.7:0/1011557193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 msgr2=0x5649d5ee4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa822ffd640 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5ee4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.972+0000 7fa822ffd640 1 -- 172.21.15.7:0/1011557193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649d5c157d0 con 0x5649d5ed9ea0 2024-03-26T08:47:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.973+0000 7fa822ffd640 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee42c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa814011c50 tx=0x7fa814004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48ea663e06149950 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.975+0000 7fa806ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.975+0000 7fa806ff5640 1 -- 172.21.15.7:0/1011557193 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814028640 con 0x5649d5ed9ea0 2024-03-26T08:47:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.975+0000 7fa82c214740 1 -- 172.21.15.7:0/1011557193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8180043d0 con 0x5649d5ed9ea0 2024-03-26T08:47:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.975+0000 7fa806ff5640 1 -- 172.21.15.7:0/1011557193 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa8140273e0 con 0x5649d5ed9ea0 2024-03-26T08:47:11.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.978+0000 7fa806ff5640 1 -- 172.21.15.7:0/1011557193 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8140287e0 con 0x5649d5ed9ea0 2024-03-26T08:47:11.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.979+0000 7fa82c214740 10 client.26542 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:11.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.979+0000 7fa82c214740 1 -- 172.21.15.7:0/1011557193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5649d5eeecb0 con 0x5649d5ed9ea0 2024-03-26T08:47:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.982+0000 7fa806ff5640 1 -- 172.21.15.7:0/1011557193 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8180043d0 con 0x5649d5ed9ea0 2024-03-26T08:47:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.983+0000 7fa806ff5640 1 -- 172.21.15.7:0/1011557193 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa814026070 con 0x5649d5ed9ea0 2024-03-26T08:47:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.985+0000 7fa82c214740 10 client.26542 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.985+0000 7fa82c214740 10 client.26542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.985+0000 7fa82c214740 20 client.26542 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.985+0000 7fa82c214740 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5649d5ef03d0 0x5649d5ef2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.985+0000 7fa82c214740 4 client.26542 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.985+0000 7fa82c214740 1 -- 172.21.15.7:0/1011557193 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649d5e44900 con 0x5649d5ef03d0 2024-03-26T08:47:11.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:11.990+0000 7fa8227fc640 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5649d5ef03d0 0x5649d5ef2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:11.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:11 smithi016 ceph-mon[35851]: pgmap v503: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 MiB/s wr, 757 op/s 2024-03-26T08:47:11.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1417308501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.003+0000 7fa806ff5640 10 client.26542 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.003+0000 7fa8227fc640 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5649d5ef03d0 0x5649d5ef2890 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=596296ac499b35a6 server_cookie=2ad710b06fb5d3c3 in_seq=0 out_seq=0 2024-03-26T08:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.005+0000 7fa806ff5640 1 -- 172.21.15.7:0/1011557193 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5649d5e44900 con 0x5649d5ef03d0 2024-03-26T08:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.005+0000 7fa806ff5640 10 client.26542 handle_command_reply: tid=0 2024-03-26T08:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.005+0000 7fa82c214740 1 client.26542 shutdown 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa82c214740 1 -- 172.21.15.7:0/1011557193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 msgr2=0x5649d5ee42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa82c214740 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee42c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa814011c50 tx=0x7fa814004730 comp rx=0 tx=0).stop 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa82c214740 1 -- 172.21.15.7:0/1011557193 shutdown_connections 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa82c214740 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5649d5ef03d0 0x5649d5ef2890 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa82c214740 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 0x5649d5eeb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa82c214740 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5ee4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa82c214740 1 --2- 172.21.15.7:0/1011557193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee42c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa82c214740 1 -- 172.21.15.7:0/1011557193 >> 172.21.15.7:0/1011557193 conn(0x5649d5eccc00 msgr2=0x5649d5ecbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa806ff5640 0 client.26542 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.006+0000 7fa82c214740 1 -- 172.21.15.7:0/1011557193 shutdown_connections 2024-03-26T08:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.007+0000 7fa82c214740 1 -- 172.21.15.7:0/1011557193 wait complete. 2024-03-26T08:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.007+0000 7fa82c214740 20 client.26542 trim_cache size 0 max 16384 2024-03-26T08:47:12.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa82c214740 1 Processor -- start 2024-03-26T08:47:12.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa82c214740 1 -- start start 2024-03-26T08:47:12.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa82c214740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:12.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa82c214740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5eeb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa82c214740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 0x5649d5ecb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa82c214740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa810010820 con 0x5649d5eda890 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa8227fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5eeb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa8227fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5eeb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47698/0 (socket says 172.21.15.7:47698) 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa8227fc640 1 -- 172.21.15.7:0/433440539 learned_addr learned my addr 172.21.15.7:0/433440539 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa82c214740 1 -- 172.21.15.7:0/433440539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81000e320 con 0x5649d5edfa00 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa8237fe640 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 0x5649d5ecb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.028+0000 7fa82c214740 1 -- 172.21.15.7:0/433440539 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81000c470 con 0x5649d5ed9ea0 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.029+0000 7fa822ffd640 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.029+0000 7fa8227fc640 1 -- 172.21.15.7:0/433440539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 msgr2=0x5649d5ee45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.029+0000 7fa8227fc640 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.029+0000 7fa8227fc640 1 -- 172.21.15.7:0/433440539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 msgr2=0x5649d5ecb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:12.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.029+0000 7fa8227fc640 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 0x5649d5ecb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.029+0000 7fa8227fc640 1 -- 172.21.15.7:0/433440539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649d5edf750 con 0x5649d5eda890 2024-03-26T08:47:12.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.029+0000 7fa8237fe640 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 0x5649d5ecb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:12.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.029+0000 7fa8227fc640 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5eeb800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa818014040 tx=0x7fa818005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce452361014c844 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:12.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.029+0000 7fa808ff9640 1 -- 172.21.15.7:0/433440539 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818029020 con 0x5649d5eda890 2024-03-26T08:47:12.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.030+0000 7fa808ff9640 1 -- 172.21.15.7:0/433440539 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa81801f040 con 0x5649d5eda890 2024-03-26T08:47:12.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.030+0000 7fa808ff9640 1 -- 172.21.15.7:0/433440539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81801e040 con 0x5649d5eda890 2024-03-26T08:47:12.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.030+0000 7fa82c214740 1 -- 172.21.15.7:0/433440539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 msgr2=0x5649d5eeb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:12.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.030+0000 7fa82c214740 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5eeb800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa818014040 tx=0x7fa818005850 comp rx=0 tx=0).stop 2024-03-26T08:47:12.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.030+0000 7fa82c214740 1 -- 172.21.15.7:0/433440539 shutdown_connections 2024-03-26T08:47:12.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.030+0000 7fa82c214740 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5edfa00 0x5649d5ecb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.030+0000 7fa82c214740 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5eda890 0x5649d5eeb800 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.030+0000 7fa82c214740 1 --2- 172.21.15.7:0/433440539 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.030+0000 7fa82c214740 1 -- 172.21.15.7:0/433440539 >> 172.21.15.7:0/433440539 conn(0x5649d5eccc00 msgr2=0x5649d5ee3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.031+0000 7fa82c214740 1 -- 172.21.15.7:0/433440539 shutdown_connections 2024-03-26T08:47:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.031+0000 7fa82c214740 1 -- 172.21.15.7:0/433440539 wait complete. 2024-03-26T08:47:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.031+0000 7fa82c214740 1 -- start start 2024-03-26T08:47:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.031+0000 7fa82c214740 1 Processor -- start 2024-03-26T08:47:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.031+0000 7fa82c214740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.031+0000 7fa82c214740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5eda890 0x5649d5ee9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa82c214740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 0x5649d5eea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa82c214740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81000c9e0 con 0x5649d5ed9ea0 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa82c214740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81000dcc0 con 0x5649d5eda890 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa82c214740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81000d000 con 0x5649d5edfa00 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa822ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa82c214740 10 client.0 _ll_register_callbacks cb 0x7fa814011640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa822ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47702/0 (socket says 172.21.15.7:47702) 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa822ffd640 1 -- 172.21.15.7:0/1266697384 learned_addr learned my addr 172.21.15.7:0/1266697384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa8227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5eda890 0x5649d5ee9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa8237fe640 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 0x5649d5eea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa822ffd640 1 -- 172.21.15.7:0/1266697384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 msgr2=0x5649d5eea790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa822ffd640 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 0x5649d5eea790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa822ffd640 1 -- 172.21.15.7:0/1266697384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5eda890 msgr2=0x5649d5ee9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa822ffd640 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5eda890 0x5649d5ee9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa822ffd640 1 -- 172.21.15.7:0/1266697384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649d5eead80 con 0x5649d5ed9ea0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa822ffd640 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee9240 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa814022040 tx=0x7fa814028670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5950562fefe973 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.032+0000 7fa806ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa806ff5640 1 -- 172.21.15.7:0/1266697384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814028d90 con 0x5649d5ed9ea0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa806ff5640 1 -- 172.21.15.7:0/1266697384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa814004020 con 0x5649d5ed9ea0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa82c214740 1 -- 172.21.15.7:0/1266697384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa81801f040 con 0x5649d5ed9ea0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa806ff5640 1 -- 172.21.15.7:0/1266697384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa814004220 con 0x5649d5ed9ea0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa82c214740 10 client.16680 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa82c214740 1 -- 172.21.15.7:0/1266697384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5649d5f4c060 con 0x5649d5ed9ea0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa806ff5640 1 -- 172.21.15.7:0/1266697384 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa81801f040 con 0x5649d5ed9ea0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa806ff5640 1 -- 172.21.15.7:0/1266697384 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa814044050 con 0x5649d5ed9ea0 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa82c214740 10 client.16680 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa82c214740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa82c214740 20 client.16680 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa82c214740 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5649d5f4c980 0x5649d5f4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:12.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa82c214740 4 client.16680 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:47:12.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.033+0000 7fa82c214740 1 -- 172.21.15.7:0/1266697384 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5649d5e3ec50 con 0x5649d5f4c980 2024-03-26T08:47:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.035+0000 7fa8227fc640 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5649d5f4c980 0x5649d5f4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.036+0000 7fa806ff5640 10 client.16680 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.036+0000 7fa8227fc640 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5649d5f4c980 0x5649d5f4eda0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97c798c782e2a766 server_cookie=aec5ae492d567c90 in_seq=0 out_seq=0 2024-03-26T08:47:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.036+0000 7fa806ff5640 1 -- 172.21.15.7:0/1266697384 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5649d5e3ec50 con 0x5649d5f4c980 2024-03-26T08:47:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.036+0000 7fa806ff5640 10 client.16680 handle_command_reply: tid=0 2024-03-26T08:47:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.036+0000 7fa82c214740 1 client.16680 shutdown 2024-03-26T08:47:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 -- 172.21.15.7:0/1266697384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5ed9ea0 msgr2=0x5649d5ee9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee9240 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa814022040 tx=0x7fa814028670 comp rx=0 tx=0).stop 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 -- 172.21.15.7:0/1266697384 shutdown_connections 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5649d5f4c980 0x5649d5f4eda0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5649d5edfa00 0x5649d5eea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5649d5eda890 0x5649d5ee9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 --2- 172.21.15.7:0/1266697384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5649d5ed9ea0 0x5649d5ee9240 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa806ff5640 0 client.16680 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 -- 172.21.15.7:0/1266697384 >> 172.21.15.7:0/1266697384 conn(0x5649d5eccc00 msgr2=0x5649d5ecfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 -- 172.21.15.7:0/1266697384 shutdown_connections 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 1 -- 172.21.15.7:0/1266697384 wait complete. 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.037+0000 7fa82c214740 20 client.16680 trim_cache size 0 max 16384 2024-03-26T08:47:12.032 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:47:12.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 -- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa80c0a2900 msgr2=0x7fa80c0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:12.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa80c0a2900 0x7fa80c0a4dc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa8140104e0 tx=0x7fa81400b410 comp rx=0 tx=0).stop 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 -- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 msgr2=0x7fa824073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 0x7fa824073fd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02c630 tx=0x7fa81c032f20 comp rx=0 tx=0).stop 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 -- 172.21.15.7:0/286567335 shutdown_connections 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa80c0a2900 0x7fa80c0a4dc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa824103280 0x7fa824073fd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8241028c0 0x7fa824073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 --2- 172.21.15.7:0/286567335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8241016d0 0x7fa824075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.038+0000 7fa829627640 1 -- 172.21.15.7:0/286567335 >> 172.21.15.7:0/286567335 conn(0x7fa8240fcc60 msgr2=0x7fa824105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.039+0000 7fa829627640 1 -- 172.21.15.7:0/286567335 shutdown_connections 2024-03-26T08:47:12.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:12.039+0000 7fa829627640 1 -- 172.21.15.7:0/286567335 wait complete. 2024-03-26T08:47:13.044 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:47:13.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:47:13.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:47:13.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.191+0000 7fca9bbf4640 1 -- 172.21.15.7:0/4151046128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca941016d0 msgr2=0x7fca94101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.191+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/4151046128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca941016d0 0x7fca94101af0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fca90009900 tx=0x7fca9002bc30 comp rx=0 tx=0).stop 2024-03-26T08:47:13.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.191+0000 7fca9bbf4640 1 -- 172.21.15.7:0/4151046128 shutdown_connections 2024-03-26T08:47:13.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.191+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/4151046128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca94103280 0x7fca94109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.191+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/4151046128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca941028c0 0x7fca94102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.191+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/4151046128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca941016d0 0x7fca94101af0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.191+0000 7fca9bbf4640 1 -- 172.21.15.7:0/4151046128 >> 172.21.15.7:0/4151046128 conn(0x7fca940fcc60 msgr2=0x7fca940ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.191+0000 7fca9bbf4640 1 -- 172.21.15.7:0/4151046128 shutdown_connections 2024-03-26T08:47:13.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.191+0000 7fca9bbf4640 1 -- 172.21.15.7:0/4151046128 wait complete. 2024-03-26T08:47:13.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9bbf4640 1 Processor -- start 2024-03-26T08:47:13.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9bbf4640 1 -- start start 2024-03-26T08:47:13.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9bbf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca941016d0 0x7fca94199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9bbf4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca941028c0 0x7fca9419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9bbf4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca94103280 0x7fca941a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9bbf4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9410d340 con 0x7fca94103280 2024-03-26T08:47:13.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9bbf4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9410d1c0 con 0x7fca941016d0 2024-03-26T08:47:13.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9bbf4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9410d4c0 con 0x7fca941028c0 2024-03-26T08:47:13.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9a191640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca94103280 0x7fca941a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca99990640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca941016d0 0x7fca94199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca99990640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca941016d0 0x7fca94199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55448/0 (socket says 172.21.15.7:55448) 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9a191640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca94103280 0x7fca941a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47726/0 (socket says 172.21.15.7:47726) 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca99990640 1 -- 172.21.15.7:0/3660703468 learned_addr learned my addr 172.21.15.7:0/3660703468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.192+0000 7fca9918f640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca941028c0 0x7fca9419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.193+0000 7fca9a191640 1 -- 172.21.15.7:0/3660703468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca941028c0 msgr2=0x7fca9419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.193+0000 7fca9a191640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca941028c0 0x7fca9419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.193+0000 7fca9a191640 1 -- 172.21.15.7:0/3660703468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca941016d0 msgr2=0x7fca94199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.193+0000 7fca9a191640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca941016d0 0x7fca94199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.193+0000 7fca9a191640 1 -- 172.21.15.7:0/3660703468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca941a1810 con 0x7fca94103280 2024-03-26T08:47:13.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.193+0000 7fca9a191640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca94103280 0x7fca941a11a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009fd0 tx=0x7fca8c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.193+0000 7fca8affd640 1 -- 172.21.15.7:0/3660703468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00ce20 con 0x7fca94103280 2024-03-26T08:47:13.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.193+0000 7fca9bbf4640 1 -- 172.21.15.7:0/3660703468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca941a1b00 con 0x7fca94103280 2024-03-26T08:47:13.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.193+0000 7fca9bbf4640 1 -- 172.21.15.7:0/3660703468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca941a2050 con 0x7fca94103280 2024-03-26T08:47:13.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.194+0000 7fca8affd640 1 -- 172.21.15.7:0/3660703468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca8c010430 con 0x7fca94103280 2024-03-26T08:47:13.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.194+0000 7fca8affd640 1 -- 172.21.15.7:0/3660703468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00f620 con 0x7fca94103280 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.196+0000 7fca8affd640 1 -- 172.21.15.7:0/3660703468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fca8c00f7c0 con 0x7fca94103280 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9e7e4740 1 Processor -- start 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9e7e4740 1 -- start start 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5203960 0x560be5203d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5204350 0x560be5208e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca8affd640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fca800a2880 0x7fca800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be520b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be5143c90 con 0x560be5203960 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be5169070 con 0x560be52094c0 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be5111c80 con 0x560be5204350 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca99990640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5204350 0x560be5208e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9918f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5203960 0x560be5203d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca99990640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5204350 0x560be5208e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56098/0 (socket says 172.21.15.7:56098) 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca99990640 1 -- 172.21.15.7:0/2606262435 learned_addr learned my addr 172.21.15.7:0/2606262435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9a191640 1 --2- 172.21.15.7:0/2606262435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be520b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca9918f640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fca800a2880 0x7fca800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca8affd640 1 -- 172.21.15.7:0/3660703468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca8c0cfd60 con 0x7fca94103280 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca99990640 1 -- 172.21.15.7:0/2606262435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 msgr2=0x560be520b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca99990640 1 --2- 172.21.15.7:0/2606262435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be520b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca99990640 1 -- 172.21.15.7:0/2606262435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5203960 msgr2=0x560be5203d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca99990640 1 --2- 172.21.15.7:0/2606262435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5203960 0x560be5203d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.197+0000 7fca99990640 1 -- 172.21.15.7:0/2606262435 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560be5109d10 con 0x560be5204350 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.198+0000 7fca99990640 1 --2- 172.21.15.7:0/2606262435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5204350 0x560be5208e50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca9002cfd0 tx=0x7fca90009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e519f4320179e997 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.198+0000 7fca9918f640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fca800a2880 0x7fca800a4d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fca84009710 tx=0x7fca84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.198+0000 7fca7ffff640 1 -- 172.21.15.7:0/2606262435 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90042070 con 0x560be5204350 2024-03-26T08:47:13.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.198+0000 7fca7ffff640 1 -- 172.21.15.7:0/2606262435 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca9002cb80 con 0x560be5204350 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.199+0000 7fca7ffff640 1 -- 172.21.15.7:0/2606262435 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90042070 con 0x560be5204350 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.199+0000 7fca9e7e4740 1 -- 172.21.15.7:0/2606262435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5204350 msgr2=0x560be5208e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.199+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/2606262435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5204350 0x560be5208e50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca9002cfd0 tx=0x7fca90009560 comp rx=0 tx=0).stop 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.199+0000 7fca9e7e4740 1 -- 172.21.15.7:0/2606262435 shutdown_connections 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.199+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/2606262435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be520b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.200+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/2606262435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5204350 0x560be5208e50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.200+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/2606262435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5203960 0x560be5203d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.200+0000 7fca9e7e4740 1 -- 172.21.15.7:0/2606262435 >> 172.21.15.7:0/2606262435 conn(0x560be51cc1a0 msgr2=0x560be51ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.200+0000 7fca9e7e4740 1 -- 172.21.15.7:0/2606262435 shutdown_connections 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.200+0000 7fca9e7e4740 1 -- 172.21.15.7:0/2606262435 wait complete. 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.200+0000 7fca9e7e4740 1 -- start start 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.200+0000 7fca9e7e4740 1 Processor -- start 2024-03-26T08:47:13.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be520e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be52151f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be51250d0 con 0x560be5204350 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be5165110 con 0x560be52094c0 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be520c460 con 0x560be5203960 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9918f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be520e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca99990640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9918f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be520e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47756/0 (socket says 172.21.15.7:47756) 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9918f640 1 -- 172.21.15.7:0/3556599881 learned_addr learned my addr 172.21.15.7:0/3556599881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca99990640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56106/0 (socket says 172.21.15.7:56106) 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9a191640 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be52151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9e7e4740 10 client.0 _ll_register_callbacks cb 0x560be5160e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9918f640 1 -- 172.21.15.7:0/3556599881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 msgr2=0x560be520dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9918f640 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9918f640 1 -- 172.21.15.7:0/3556599881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 msgr2=0x560be52151f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9918f640 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be52151f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.201+0000 7fca9918f640 1 -- 172.21.15.7:0/3556599881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560be4f457e0 con 0x560be5204350 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.202+0000 7fca99990640 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.202+0000 7fca9918f640 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be520e1b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fca84004580 tx=0x7fca840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e814edddcbcb6b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.202+0000 7fca7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.202+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3556599881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8402d060 con 0x560be5204350 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.202+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3556599881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca9002cb80 con 0x560be5204350 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.202+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3556599881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca84003d70 con 0x560be5204350 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.202+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3556599881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca84020460 con 0x560be5204350 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.202+0000 7fca9e7e4740 10 client.16692 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3556599881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560be4fa5f70 con 0x560be5204350 2024-03-26T08:47:13.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3556599881 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca9002cb80 con 0x560be5204350 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3556599881 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fca84032070 con 0x560be5204350 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca9e7e4740 10 client.16692 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca9e7e4740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca9e7e4740 20 client.16692 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560be5219bc0 0x560be521c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca9e7e4740 4 client.16692 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3556599881 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560be51765b0 con 0x560be5219bc0 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.203+0000 7fca99990640 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560be5219bc0 0x560be521c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.204+0000 7fca7d7fa640 10 client.16692 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:13.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.204+0000 7fca99990640 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560be5219bc0 0x560be521c000 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93b50091519f019d server_cookie=f3c688998f4dd50f in_seq=0 out_seq=0 2024-03-26T08:47:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.204+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3556599881 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560be51765b0 con 0x560be5219bc0 2024-03-26T08:47:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.204+0000 7fca7d7fa640 10 client.16692 handle_command_reply: tid=0 2024-03-26T08:47:13.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 client.16692 shutdown 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3556599881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 msgr2=0x560be520e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be520e1b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fca84004580 tx=0x7fca840046c0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3556599881 shutdown_connections 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560be5219bc0 0x560be521c000 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be52151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca7d7fa640 0 client.16692 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be520e1b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3556599881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3556599881 >> 172.21.15.7:0/3556599881 conn(0x560be51cc1a0 msgr2=0x560be51fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3556599881 shutdown_connections 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3556599881 wait complete. 2024-03-26T08:47:13.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.205+0000 7fca9e7e4740 20 client.16692 trim_cache size 0 max 16384 2024-03-26T08:47:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.229+0000 7fca9e7e4740 1 Processor -- start 2024-03-26T08:47:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.230+0000 7fca9e7e4740 1 -- start start 2024-03-26T08:47:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.230+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.230+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be52151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.230+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be5202510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.230+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca780108c0 con 0x560be5204350 2024-03-26T08:47:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.230+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7800e3c0 con 0x560be52094c0 2024-03-26T08:47:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.230+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7800c440 con 0x560be5203960 2024-03-26T08:47:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.231+0000 7fca99990640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.231+0000 7fca9918f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be52151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.231+0000 7fca9918f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be52151b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47766/0 (socket says 172.21.15.7:47766) 2024-03-26T08:47:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.231+0000 7fca9918f640 1 -- 172.21.15.7:0/3152913819 learned_addr learned my addr 172.21.15.7:0/3152913819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.231+0000 7fca9918f640 1 -- 172.21.15.7:0/3152913819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 msgr2=0x560be520d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.231+0000 7fca9918f640 1 --2- 172.21.15.7:0/3152913819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.231+0000 7fca9918f640 1 -- 172.21.15.7:0/3152913819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 msgr2=0x560be5202510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:47:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.231+0000 7fca9918f640 1 --2- 172.21.15.7:0/3152913819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be5202510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.232+0000 7fca9918f640 1 -- 172.21.15.7:0/3152913819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560be4e3d170 con 0x560be5204350 2024-03-26T08:47:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.232+0000 7fca9918f640 1 --2- 172.21.15.7:0/3152913819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be52151b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fca8401e4d0 tx=0x7fca84004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249326ee5f75b39c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.234+0000 7fca7f7fe640 1 -- 172.21.15.7:0/3152913819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84038070 con 0x560be5204350 2024-03-26T08:47:13.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.235+0000 7fca7f7fe640 1 -- 172.21.15.7:0/3152913819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca84005d30 con 0x560be5204350 2024-03-26T08:47:13.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.235+0000 7fca7f7fe640 1 -- 172.21.15.7:0/3152913819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8402d040 con 0x560be5204350 2024-03-26T08:47:13.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.236+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3152913819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 msgr2=0x560be52151b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.236+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3152913819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be52151b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fca8401e4d0 tx=0x7fca84004380 comp rx=0 tx=0).stop 2024-03-26T08:47:13.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.238+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3152913819 shutdown_connections 2024-03-26T08:47:13.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.238+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3152913819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be52094c0 0x560be5202510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.238+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3152913819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5204350 0x560be52151b0 secure :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fca8401e4d0 tx=0x7fca84004380 comp rx=0 tx=0).stop 2024-03-26T08:47:13.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.238+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3152913819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be5203960 0x560be520d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.238+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3152913819 >> 172.21.15.7:0/3152913819 conn(0x560be51cc1a0 msgr2=0x560be520cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.243+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3152913819 shutdown_connections 2024-03-26T08:47:13.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.245+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3152913819 wait complete. 2024-03-26T08:47:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.246+0000 7fca9e7e4740 1 -- start start 2024-03-26T08:47:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.246+0000 7fca9e7e4740 1 Processor -- start 2024-03-26T08:47:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.246+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5203960 0x560be52151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.246+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be52094c0 0x560be52137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.247+0000 7fca9e7e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be5213d60 0x560be52141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.247+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7800ca40 con 0x560be5203960 2024-03-26T08:47:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.247+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7800dd60 con 0x560be5213d60 2024-03-26T08:47:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.247+0000 7fca9e7e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7800d0a0 con 0x560be52094c0 2024-03-26T08:47:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.247+0000 7fca9e7e4740 10 client.0 _ll_register_callbacks cb 0x7fca84004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:13.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.248+0000 7fca9a191640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be5213d60 0x560be52141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.248+0000 7fca9a191640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be5213d60 0x560be52141e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55474/0 (socket says 172.21.15.7:55474) 2024-03-26T08:47:13.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.248+0000 7fca9a191640 1 -- 172.21.15.7:0/3703524197 learned_addr learned my addr 172.21.15.7:0/3703524197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.249+0000 7fca9a191640 1 -- 172.21.15.7:0/3703524197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be52094c0 msgr2=0x560be52137d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:47:13.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.249+0000 7fca9a191640 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be52094c0 0x560be52137d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.249+0000 7fca9a191640 1 -- 172.21.15.7:0/3703524197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5203960 msgr2=0x560be52151c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.249+0000 7fca9a191640 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5203960 0x560be52151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.249+0000 7fca9a191640 1 -- 172.21.15.7:0/3703524197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560be528eef0 con 0x560be5213d60 2024-03-26T08:47:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.250+0000 7fca9a191640 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be5213d60 0x560be52141e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fca8c010de0 tx=0x7fca8c01dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9823d7f8d1e34094 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.252+0000 7fca7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.252+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3703524197 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c09c700 con 0x560be5213d60 2024-03-26T08:47:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.252+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3703524197 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca8c0cd970 con 0x560be5213d60 2024-03-26T08:47:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.256+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3703524197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca84005d30 con 0x560be5213d60 2024-03-26T08:47:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.258+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3703524197 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca8c0a0850 con 0x560be5213d60 2024-03-26T08:47:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.259+0000 7fca9e7e4740 10 client.26525 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.259+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3703524197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560be52902a0 con 0x560be5213d60 2024-03-26T08:47:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.261+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3703524197 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fca84005d30 con 0x560be5213d60 2024-03-26T08:47:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.261+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3703524197 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fca8c01f020 con 0x560be5213d60 2024-03-26T08:47:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.261+0000 7fca9e7e4740 10 client.26525 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.261+0000 7fca9e7e4740 10 client.26525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.261+0000 7fca9e7e4740 20 client.26525 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.261+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560be5290b30 0x560be5292f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.261+0000 7fca9e7e4740 4 client.26525 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:47:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.261+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3703524197 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560be5170900 con 0x560be5290b30 2024-03-26T08:47:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.262+0000 7fca99990640 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560be5290b30 0x560be5292f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.272+0000 7fca99990640 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560be5290b30 0x560be5292f50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf04ef3a1e0cd8a server_cookie=9aca31cc35f074ef in_seq=0 out_seq=0 2024-03-26T08:47:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.273+0000 7fca7d7fa640 10 client.26525 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.275+0000 7fca7d7fa640 1 -- 172.21.15.7:0/3703524197 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560be5170900 con 0x560be5290b30 2024-03-26T08:47:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.275+0000 7fca7d7fa640 10 client.26525 handle_command_reply: tid=0 2024-03-26T08:47:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.275+0000 7fca9e7e4740 1 client.26525 shutdown 2024-03-26T08:47:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.279+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3703524197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be5213d60 msgr2=0x560be52141e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.279+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be5213d60 0x560be52141e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fca8c010de0 tx=0x7fca8c01dea0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.281+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3703524197 shutdown_connections 2024-03-26T08:47:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.281+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560be5290b30 0x560be5292f50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.281+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560be5213d60 0x560be52141e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.281+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560be52094c0 0x560be52137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.281+0000 7fca9e7e4740 1 --2- 172.21.15.7:0/3703524197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560be5203960 0x560be52151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.281+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3703524197 >> 172.21.15.7:0/3703524197 conn(0x560be51cc1a0 msgr2=0x560be51fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.281+0000 7fca7d7fa640 0 client.26525 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.282+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3703524197 shutdown_connections 2024-03-26T08:47:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.284+0000 7fca9e7e4740 1 -- 172.21.15.7:0/3703524197 wait complete. 2024-03-26T08:47:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.284+0000 7fca9e7e4740 20 client.26525 trim_cache size 0 max 16384 2024-03-26T08:47:13.276 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:47:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:47:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "806816cb-e683-4563-a801-65dc3fc1a444", 2024-03-26T08:47:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:47:13.277 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 -- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fca800a2880 msgr2=0x7fca800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fca800a2880 0x7fca800a4d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fca84009710 tx=0x7fca84005f90 comp rx=0 tx=0).stop 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 -- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca94103280 msgr2=0x7fca941a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca94103280 0x7fca941a11a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009fd0 tx=0x7fca8c00ef10 comp rx=0 tx=0).stop 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 -- 172.21.15.7:0/3660703468 shutdown_connections 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fca800a2880 0x7fca800a4d40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca94103280 0x7fca941a11a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fca941028c0 0x7fca9419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 --2- 172.21.15.7:0/3660703468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca941016d0 0x7fca94199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 -- 172.21.15.7:0/3660703468 >> 172.21.15.7:0/3660703468 conn(0x7fca940fcc60 msgr2=0x7fca940738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 -- 172.21.15.7:0/3660703468 shutdown_connections 2024-03-26T08:47:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.285+0000 7fca9bbf4640 1 -- 172.21.15.7:0/3660703468 wait complete. 2024-03-26T08:47:13.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:47:13.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:47:13.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.463+0000 7fc874db2640 1 -- 172.21.15.7:0/3482083431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8701086a0 msgr2=0x7fc87010aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.463+0000 7fc874db2640 1 --2- 172.21.15.7:0/3482083431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8701086a0 0x7fc87010aad0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc86401a910 tx=0x7fc86403cca0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.466+0000 7fc874db2640 1 -- 172.21.15.7:0/3482083431 shutdown_connections 2024-03-26T08:47:13.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.466+0000 7fc874db2640 1 --2- 172.21.15.7:0/3482083431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87010d9c0 0x7fc87010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.466+0000 7fc874db2640 1 --2- 172.21.15.7:0/3482083431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc87010b010 0x7fc87010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.466+0000 7fc874db2640 1 --2- 172.21.15.7:0/3482083431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8701086a0 0x7fc87010aad0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.466+0000 7fc874db2640 1 -- 172.21.15.7:0/3482083431 >> 172.21.15.7:0/3482083431 conn(0x7fc87006e640 msgr2=0x7fc870070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.473+0000 7fc874db2640 1 -- 172.21.15.7:0/3482083431 shutdown_connections 2024-03-26T08:47:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.473+0000 7fc874db2640 1 -- 172.21.15.7:0/3482083431 wait complete. 2024-03-26T08:47:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.474+0000 7fc874db2640 1 Processor -- start 2024-03-26T08:47:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.474+0000 7fc874db2640 1 -- start start 2024-03-26T08:47:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.474+0000 7fc874db2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701086a0 0x7fc87019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.474+0000 7fc874db2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87010b010 0x7fc87019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.474+0000 7fc874db2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc87010d9c0 0x7fc8701a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.474+0000 7fc874db2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc870111a30 con 0x7fc87010d9c0 2024-03-26T08:47:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.474+0000 7fc874db2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8701118b0 con 0x7fc87010b010 2024-03-26T08:47:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.474+0000 7fc874db2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc870111bb0 con 0x7fc8701086a0 2024-03-26T08:47:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.475+0000 7fc86e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701086a0 0x7fc87019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.476+0000 7fc86e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701086a0 0x7fc87019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:56124/0 (socket says 172.21.15.7:56124) 2024-03-26T08:47:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.476+0000 7fc86e59c640 1 -- 172.21.15.7:0/503989787 learned_addr learned my addr 172.21.15.7:0/503989787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.477+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc87010d9c0 0x7fc8701a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.478+0000 7fc86e59c640 1 -- 172.21.15.7:0/503989787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87010b010 msgr2=0x7fc87019e690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:47:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.478+0000 7fc86e59c640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87010b010 0x7fc87019e690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.478+0000 7fc86e59c640 1 -- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc87010d9c0 msgr2=0x7fc8701a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.478+0000 7fc86e59c640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc87010d9c0 0x7fc8701a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.478+0000 7fc86e59c640 1 -- 172.21.15.7:0/503989787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8701a5d70 con 0x7fc8701086a0 2024-03-26T08:47:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.478+0000 7fc86e59c640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701086a0 0x7fc87019e150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc86401a440 tx=0x7fc864004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.481+0000 7fc85f7fe640 1 -- 172.21.15.7:0/503989787 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864052020 con 0x7fc8701086a0 2024-03-26T08:47:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.481+0000 7fc874db2640 1 -- 172.21.15.7:0/503989787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8701a6000 con 0x7fc8701086a0 2024-03-26T08:47:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.482+0000 7fc874db2640 1 -- 172.21.15.7:0/503989787 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8701a65d0 con 0x7fc8701086a0 2024-03-26T08:47:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.482+0000 7fc85f7fe640 1 -- 172.21.15.7:0/503989787 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc86403dd30 con 0x7fc8701086a0 2024-03-26T08:47:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.482+0000 7fc85f7fe640 1 -- 172.21.15.7:0/503989787 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86404e480 con 0x7fc8701086a0 2024-03-26T08:47:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.486+0000 7fc87799e740 1 Processor -- start 2024-03-26T08:47:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.487+0000 7fc87799e740 1 -- start start 2024-03-26T08:47:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.487+0000 7fc85f7fe640 1 -- 172.21.15.7:0/503989787 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc864055020 con 0x7fc8701086a0 2024-03-26T08:47:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.487+0000 7fc87799e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdc11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.487+0000 7fc87799e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdc6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.487+0000 7fc87799e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 0x556e7bdc8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.487+0000 7fc87799e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bd263d0 con 0x556e7bdc6900 2024-03-26T08:47:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.487+0000 7fc87799e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bccead0 con 0x556e7bdc0da0 2024-03-26T08:47:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.487+0000 7fc87799e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bd22470 con 0x556e7bdc1790 2024-03-26T08:47:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.487+0000 7fc86ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdc11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.488+0000 7fc86ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdc11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55494/0 (socket says 172.21.15.7:55494) 2024-03-26T08:47:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.488+0000 7fc86ed9d640 1 -- 172.21.15.7:0/533577093 learned_addr learned my addr 172.21.15.7:0/533577093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.488+0000 7fc86e59c640 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdc6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.488+0000 7fc86dd9b640 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 0x556e7bdc8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.489+0000 7fc85f7fe640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc8580a2a00 0x7fc8580a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.489+0000 7fc85f7fe640 1 -- 172.21.15.7:0/503989787 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc864101bc0 con 0x7fc8701086a0 2024-03-26T08:47:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.490+0000 7fc86ed9d640 1 -- 172.21.15.7:0/533577093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 msgr2=0x556e7bdc6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.490+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdc6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.490+0000 7fc86ed9d640 1 -- 172.21.15.7:0/533577093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 msgr2=0x556e7bdc8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.490+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 0x556e7bdc8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.490+0000 7fc86ed9d640 1 -- 172.21.15.7:0/533577093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e7bb047d0 con 0x556e7bdc0da0 2024-03-26T08:47:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.490+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc8580a2a00 0x7fc8580a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.490+0000 7fc86dd9b640 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 0x556e7bdc8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.491+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdc11c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c570 tx=0x7fc868009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4268d482570d3e4b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.492+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc8580a2a00 0x7fc8580a4ec0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc86801d000 tx=0x7fc868018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.492+0000 7fc85cff9640 1 -- 172.21.15.7:0/533577093 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868018440 con 0x556e7bdc0da0 2024-03-26T08:47:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.492+0000 7fc85cff9640 1 -- 172.21.15.7:0/533577093 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc86801e070 con 0x556e7bdc0da0 2024-03-26T08:47:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.492+0000 7fc85cff9640 1 -- 172.21.15.7:0/533577093 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868007800 con 0x556e7bdc0da0 2024-03-26T08:47:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.493+0000 7fc87799e740 1 -- 172.21.15.7:0/533577093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 msgr2=0x556e7bdc11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.494+0000 7fc87799e740 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdc11c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c570 tx=0x7fc868009f90 comp rx=0 tx=0).stop 2024-03-26T08:47:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.494+0000 7fc87799e740 1 -- 172.21.15.7:0/533577093 shutdown_connections 2024-03-26T08:47:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.494+0000 7fc87799e740 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 0x556e7bdc8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.494+0000 7fc87799e740 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdc6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.494+0000 7fc87799e740 1 --2- 172.21.15.7:0/533577093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdc11c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.494+0000 7fc87799e740 1 -- 172.21.15.7:0/533577093 >> 172.21.15.7:0/533577093 conn(0x556e7bdbc310 msgr2=0x556e7bdbc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.494+0000 7fc87799e740 1 -- 172.21.15.7:0/533577093 shutdown_connections 2024-03-26T08:47:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.494+0000 7fc87799e740 1 -- 172.21.15.7:0/533577093 wait complete. 2024-03-26T08:47:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.494+0000 7fc87799e740 1 -- start start 2024-03-26T08:47:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc87799e740 1 Processor -- start 2024-03-26T08:47:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc87799e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdcb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc87799e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdcb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc87799e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 0x556e7bdd2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc87799e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bdc98b0 con 0x556e7bdc6900 2024-03-26T08:47:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc87799e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bce2060 con 0x556e7bdc0da0 2024-03-26T08:47:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc87799e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e7bdc9a30 con 0x556e7bdc1790 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc87799e740 10 client.0 _ll_register_callbacks cb 0x556e7bcc46a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc86e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdcb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc86e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdcb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55502/0 (socket says 172.21.15.7:55502) 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc86e59c640 1 -- 172.21.15.7:0/1109280291 learned_addr learned my addr 172.21.15.7:0/1109280291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.495+0000 7fc86dd9b640 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdcb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.496+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 0x556e7bdd2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.496+0000 7fc86dd9b640 1 -- 172.21.15.7:0/1109280291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 msgr2=0x556e7bdcb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.496+0000 7fc86dd9b640 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdcb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.496+0000 7fc86dd9b640 1 -- 172.21.15.7:0/1109280291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 msgr2=0x556e7bdd2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.496+0000 7fc86dd9b640 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 0x556e7bdd2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.496+0000 7fc86dd9b640 1 -- 172.21.15.7:0/1109280291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e7ba3a3f0 con 0x556e7bdc1790 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.496+0000 7fc86dd9b640 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdcb7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x556e7bdbd280 tx=0x7fc86000d530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=434ae1c7452f6425 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.496+0000 7fc855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.496+0000 7fc855ffb640 1 -- 172.21.15.7:0/1109280291 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86000de60 con 0x556e7bdc1790 2024-03-26T08:47:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.497+0000 7fc87799e740 1 -- 172.21.15.7:0/1109280291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc86801e070 con 0x556e7bdc1790 2024-03-26T08:47:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.497+0000 7fc855ffb640 1 -- 172.21.15.7:0/1109280291 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc860014430 con 0x556e7bdc1790 2024-03-26T08:47:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.497+0000 7fc855ffb640 1 -- 172.21.15.7:0/1109280291 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc860014630 con 0x556e7bdc1790 2024-03-26T08:47:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.497+0000 7fc87799e740 10 client.26569 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.497+0000 7fc87799e740 1 -- 172.21.15.7:0/1109280291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556e7b9fa0f0 con 0x556e7bdc1790 2024-03-26T08:47:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.498+0000 7fc855ffb640 1 -- 172.21.15.7:0/1109280291 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc86801e070 con 0x556e7bdc1790 2024-03-26T08:47:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.498+0000 7fc855ffb640 1 -- 172.21.15.7:0/1109280291 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc860013070 con 0x556e7bdc1790 2024-03-26T08:47:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.499+0000 7fc87799e740 10 client.26569 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.499+0000 7fc87799e740 10 client.26569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.499+0000 7fc87799e740 20 client.26569 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.499+0000 7fc87799e740 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556e7bdd6df0 0x556e7bdd9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.499+0000 7fc87799e740 4 client.26569 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.499+0000 7fc87799e740 1 -- 172.21.15.7:0/1109280291 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e7bd33910 con 0x556e7bdd6df0 2024-03-26T08:47:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.500+0000 7fc86e59c640 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556e7bdd6df0 0x556e7bdd9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.501+0000 7fc86e59c640 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556e7bdd6df0 0x556e7bdd9210 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4841f4a0a340289c server_cookie=e2939a7206b6b580 in_seq=0 out_seq=0 2024-03-26T08:47:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.501+0000 7fc855ffb640 10 client.26569 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.503+0000 7fc855ffb640 1 -- 172.21.15.7:0/1109280291 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556e7bd33910 con 0x556e7bdd6df0 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.503+0000 7fc855ffb640 10 client.26569 handle_command_reply: tid=0 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.503+0000 7fc87799e740 1 client.26569 shutdown 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 -- 172.21.15.7:0/1109280291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 msgr2=0x556e7bdcb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdcb7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x556e7bdbd280 tx=0x7fc86000d530 comp rx=0 tx=0).stop 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 -- 172.21.15.7:0/1109280291 shutdown_connections 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556e7bdd6df0 0x556e7bdd9210 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc6900 0x556e7bdd2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdcb7c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 --2- 172.21.15.7:0/1109280291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc0da0 0x556e7bdcb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 -- 172.21.15.7:0/1109280291 >> 172.21.15.7:0/1109280291 conn(0x556e7bdbc310 msgr2=0x556e7bdbb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc855ffb640 0 client.26569 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:13.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 -- 172.21.15.7:0/1109280291 shutdown_connections 2024-03-26T08:47:13.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 1 -- 172.21.15.7:0/1109280291 wait complete. 2024-03-26T08:47:13.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.504+0000 7fc87799e740 20 client.26569 trim_cache size 0 max 16384 2024-03-26T08:47:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.529+0000 7fc87799e740 1 Processor -- start 2024-03-26T08:47:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.529+0000 7fc87799e740 1 -- start start 2024-03-26T08:47:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.529+0000 7fc87799e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc0da0 0x556e7bdcac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.529+0000 7fc87799e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdd27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.529+0000 7fc87799e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdb7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.529+0000 7fc87799e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8500108b0 con 0x556e7bdc0da0 2024-03-26T08:47:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.529+0000 7fc87799e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000e3b0 con 0x556e7bdc6900 2024-03-26T08:47:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.529+0000 7fc87799e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000c430 con 0x556e7bdc1790 2024-03-26T08:47:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.530+0000 7fc86ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdb7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.530+0000 7fc86ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdb7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55516/0 (socket says 172.21.15.7:55516) 2024-03-26T08:47:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.530+0000 7fc86ed9d640 1 -- 172.21.15.7:0/3195457772 learned_addr learned my addr 172.21.15.7:0/3195457772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.531+0000 7fc86ed9d640 1 -- 172.21.15.7:0/3195457772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 msgr2=0x556e7bdd27c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:47:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.531+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/3195457772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdd27c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.531+0000 7fc86ed9d640 1 -- 172.21.15.7:0/3195457772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc0da0 msgr2=0x556e7bdcac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.531+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/3195457772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc0da0 0x556e7bdcac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.531+0000 7fc86ed9d640 1 -- 172.21.15.7:0/3195457772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e7bdb6910 con 0x556e7bdc6900 2024-03-26T08:47:13.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.531+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/3195457772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdb7390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x556e7bdc1ff0 tx=0x7fc86800c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d3c68e63d2660e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.531+0000 7fc857fff640 1 -- 172.21.15.7:0/3195457772 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868009960 con 0x556e7bdc6900 2024-03-26T08:47:13.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.532+0000 7fc857fff640 1 -- 172.21.15.7:0/3195457772 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc868030070 con 0x556e7bdc6900 2024-03-26T08:47:13.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.532+0000 7fc857fff640 1 -- 172.21.15.7:0/3195457772 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8680227b0 con 0x556e7bdc6900 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.532+0000 7fc87799e740 1 -- 172.21.15.7:0/3195457772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 msgr2=0x556e7bdb7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.532+0000 7fc87799e740 1 --2- 172.21.15.7:0/3195457772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdb7390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x556e7bdc1ff0 tx=0x7fc86800c370 comp rx=0 tx=0).stop 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.532+0000 7fc87799e740 1 -- 172.21.15.7:0/3195457772 shutdown_connections 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.532+0000 7fc87799e740 1 --2- 172.21.15.7:0/3195457772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdb7390 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.532+0000 7fc87799e740 1 --2- 172.21.15.7:0/3195457772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc1790 0x556e7bdd27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.532+0000 7fc87799e740 1 --2- 172.21.15.7:0/3195457772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc0da0 0x556e7bdcac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.532+0000 7fc87799e740 1 -- 172.21.15.7:0/3195457772 >> 172.21.15.7:0/3195457772 conn(0x556e7bdbc310 msgr2=0x556e7bdca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 -- 172.21.15.7:0/3195457772 shutdown_connections 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 -- 172.21.15.7:0/3195457772 wait complete. 2024-03-26T08:47:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 -- start start 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 Processor -- start 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc0da0 0x556e7bdd0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc1790 0x556e7bdd0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdd1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000ca30 con 0x556e7bdc1790 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000dd50 con 0x556e7bdc6900 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000d090 con 0x556e7bdc0da0 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.533+0000 7fc87799e740 10 client.0 _ll_register_callbacks cb 0x7fc86000b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc1790 0x556e7bdd0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc1790 0x556e7bdd0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47818/0 (socket says 172.21.15.7:47818) 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86dd9b640 1 -- 172.21.15.7:0/2755216949 learned_addr learned my addr 172.21.15.7:0/2755216949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86e59c640 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc0da0 0x556e7bdd0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdd1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86ed9d640 1 -- 172.21.15.7:0/2755216949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc0da0 msgr2=0x556e7bdd0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc0da0 0x556e7bdd0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86ed9d640 1 -- 172.21.15.7:0/2755216949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc1790 msgr2=0x556e7bdd0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc1790 0x556e7bdd0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86ed9d640 1 -- 172.21.15.7:0/2755216949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e7bdd2230 con 0x556e7bdc6900 2024-03-26T08:47:13.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc86ed9d640 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdd1c40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8680098f0 tx=0x7fc86800aed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ee9498d702af68 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:13.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.534+0000 7fc855ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:13.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc855ffb640 1 -- 172.21.15.7:0/2755216949 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8680186d0 con 0x556e7bdc6900 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc855ffb640 1 -- 172.21.15.7:0/2755216949 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc868018870 con 0x556e7bdc6900 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc87799e740 1 -- 172.21.15.7:0/2755216949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc868030070 con 0x556e7bdc6900 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc855ffb640 1 -- 172.21.15.7:0/2755216949 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc868018a70 con 0x556e7bdc6900 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc87799e740 10 client.26537 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc87799e740 1 -- 172.21.15.7:0/2755216949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556e7be32f30 con 0x556e7bdc6900 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc855ffb640 1 -- 172.21.15.7:0/2755216949 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc868030070 con 0x556e7bdc6900 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc855ffb640 1 -- 172.21.15.7:0/2755216949 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc868034610 con 0x556e7bdc6900 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc87799e740 10 client.26537 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc87799e740 10 client.26537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc87799e740 20 client.26537 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc87799e740 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556e7be33530 0x556e7be35920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc87799e740 4 client.26537 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc87799e740 1 -- 172.21.15.7:0/2755216949 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556e7bd2dc60 con 0x556e7be33530 2024-03-26T08:47:13.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.535+0000 7fc86e59c640 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556e7be33530 0x556e7be35920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc855ffb640 10 client.26537 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc86e59c640 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556e7be33530 0x556e7be35920 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0748bb50531b9b1 server_cookie=19f5c894c6b0965a in_seq=0 out_seq=0 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc855ffb640 1 -- 172.21.15.7:0/2755216949 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556e7bd2dc60 con 0x556e7be33530 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc855ffb640 10 client.26537 handle_command_reply: tid=0 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc87799e740 1 client.26537 shutdown 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc87799e740 1 -- 172.21.15.7:0/2755216949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 msgr2=0x556e7bdd1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc87799e740 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdd1c40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8680098f0 tx=0x7fc86800aed0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc87799e740 1 -- 172.21.15.7:0/2755216949 shutdown_connections 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc87799e740 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556e7be33530 0x556e7be35920 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc87799e740 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e7bdc6900 0x556e7bdd1c40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc87799e740 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556e7bdc1790 0x556e7bdd0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc87799e740 1 --2- 172.21.15.7:0/2755216949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556e7bdc0da0 0x556e7bdd0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.536+0000 7fc87799e740 1 -- 172.21.15.7:0/2755216949 >> 172.21.15.7:0/2755216949 conn(0x556e7bdbc310 msgr2=0x556e7bdbf570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.537+0000 7fc855ffb640 0 client.26537 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.537+0000 7fc87799e740 1 -- 172.21.15.7:0/2755216949 shutdown_connections 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.537+0000 7fc87799e740 1 -- 172.21.15.7:0/2755216949 wait complete. 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.537+0000 7fc87799e740 20 client.26537 trim_cache size 0 max 16384 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "806816cb-e683-4563-a801-65dc3fc1a444": { 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "806816cb-e683-4563-a801-65dc3fc1a444", 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:47:13.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:47:13.534 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:47:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.537+0000 7fc874db2640 1 -- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc8580a2a00 msgr2=0x7fc8580a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.537+0000 7fc874db2640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc8580a2a00 0x7fc8580a4ec0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc86801d000 tx=0x7fc868018000 comp rx=0 tx=0).stop 2024-03-26T08:47:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.537+0000 7fc874db2640 1 -- 172.21.15.7:0/503989787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701086a0 msgr2=0x7fc87019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.538+0000 7fc874db2640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701086a0 0x7fc87019e150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc86401a440 tx=0x7fc864004270 comp rx=0 tx=0).stop 2024-03-26T08:47:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.538+0000 7fc874db2640 1 -- 172.21.15.7:0/503989787 shutdown_connections 2024-03-26T08:47:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.538+0000 7fc874db2640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc8580a2a00 0x7fc8580a4ec0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.538+0000 7fc874db2640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc87010d9c0 0x7fc8701a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.538+0000 7fc874db2640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc87010b010 0x7fc87019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.538+0000 7fc874db2640 1 --2- 172.21.15.7:0/503989787 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc8701086a0 0x7fc87019e150 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.538+0000 7fc874db2640 1 -- 172.21.15.7:0/503989787 >> 172.21.15.7:0/503989787 conn(0x7fc87006e640 msgr2=0x7fc870108e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.538+0000 7fc874db2640 1 -- 172.21.15.7:0/503989787 shutdown_connections 2024-03-26T08:47:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:13.540+0000 7fc874db2640 1 -- 172.21.15.7:0/503989787 wait complete. 2024-03-26T08:47:13.547 INFO:tasks.cephfs.filesystem:scrub status for tag:806816cb-e683-4563-a801-65dc3fc1a444 - {'path': '/', 'tag': '806816cb-e683-4563-a801-65dc3fc1a444', 'options': 'recursive,force'} 2024-03-26T08:47:13.792 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:13 smithi007 ceph-mon[32462]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 12 MiB/s wr, 913 op/s 2024-03-26T08:47:13.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:13 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:47:13.793 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:13 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:13.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:13 smithi044 ceph-mon[36461]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 12 MiB/s wr, 913 op/s 2024-03-26T08:47:13.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:13 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:47:13.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:13 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:13.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:13 smithi016 ceph-mon[35851]: pgmap v504: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 12 MiB/s wr, 913 op/s 2024-03-26T08:47:13.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:13 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:47:13.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:13 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:14.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:14 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:47:14.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:14 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:14.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:14 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:47:14.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:14 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:14.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:14 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:47:14.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:14 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:14.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:14 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:47:14.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:14 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:15.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:14 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:47:15.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:14 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:15.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:14 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:47:15.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:14 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:15.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:15 smithi044 ceph-mon[36461]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-03-26T08:47:15.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:15 smithi016 ceph-mon[35851]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-03-26T08:47:16.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:15 smithi007 ceph-mon[32462]: pgmap v505: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-03-26T08:47:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 285.0 s, 2814.0 tps, lat 35.784 ms stddev 41.335 2024-03-26T08:47:17.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:17 smithi044 ceph-mon[36461]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 16 MiB/s wr, 886 op/s 2024-03-26T08:47:17.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:17 smithi016 ceph-mon[35851]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 16 MiB/s wr, 886 op/s 2024-03-26T08:47:18.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:17 smithi007 ceph-mon[32462]: pgmap v506: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 16 MiB/s wr, 886 op/s 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.730+0000 7fba09485640 1 -- 172.21.15.7:0/1592260732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0410dd90 msgr2=0x7fba04110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.730+0000 7fba09485640 1 --2- 172.21.15.7:0/1592260732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0410dd90 0x7fba04110250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009860 tx=0x7fb9f402bb70 comp rx=0 tx=0).stop 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.730+0000 7fba09485640 1 -- 172.21.15.7:0/1592260732 shutdown_connections 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.730+0000 7fba09485640 1 --2- 172.21.15.7:0/1592260732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0410dd90 0x7fba04110250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.730+0000 7fba09485640 1 --2- 172.21.15.7:0/1592260732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba0410b2d0 0x7fba0410d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.730+0000 7fba09485640 1 --2- 172.21.15.7:0/1592260732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba04108910 0x7fba0410ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.730+0000 7fba09485640 1 -- 172.21.15.7:0/1592260732 >> 172.21.15.7:0/1592260732 conn(0x7fba0406e8b0 msgr2=0x7fba04070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.730+0000 7fba09485640 1 -- 172.21.15.7:0/1592260732 shutdown_connections 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.730+0000 7fba09485640 1 -- 172.21.15.7:0/1592260732 wait complete. 2024-03-26T08:47:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.731+0000 7fba09485640 1 Processor -- start 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba09485640 1 -- start start 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba09485640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba04108910 0x7fba0419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba09485640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba0410b2d0 0x7fba0419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba09485640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0410dd90 0x7fba041a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba09485640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04113980 con 0x7fba04108910 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba09485640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04113800 con 0x7fba0410dd90 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba09485640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04113b00 con 0x7fba0410b2d0 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba04108910 0x7fba0419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0410dd90 0x7fba041a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba04108910 0x7fba0419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38530/0 (socket says 172.21.15.7:38530) 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.732+0000 7fba02ffd640 1 -- 172.21.15.7:0/1780729177 learned_addr learned my addr 172.21.15.7:0/1780729177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.733+0000 7fba027fc640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba0410b2d0 0x7fba0419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.733+0000 7fba02ffd640 1 -- 172.21.15.7:0/1780729177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba0410b2d0 msgr2=0x7fba0419e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.733+0000 7fba02ffd640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba0410b2d0 0x7fba0419e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.733+0000 7fba02ffd640 1 -- 172.21.15.7:0/1780729177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0410dd90 msgr2=0x7fba041a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.733+0000 7fba02ffd640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0410dd90 0x7fba041a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.733+0000 7fba02ffd640 1 -- 172.21.15.7:0/1780729177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba041a5be0 con 0x7fba04108910 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.733+0000 7fba02ffd640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba04108910 0x7fba0419dfc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007920 tx=0x7fb9fc007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.734+0000 7fb9fbfff640 1 -- 172.21.15.7:0/1780729177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc024070 con 0x7fba04108910 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.734+0000 7fba09485640 1 -- 172.21.15.7:0/1780729177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba041a5ed0 con 0x7fba04108910 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.734+0000 7fb9fbfff640 1 -- 172.21.15.7:0/1780729177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc01f400 con 0x7fba04108910 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.734+0000 7fb9fbfff640 1 -- 172.21.15.7:0/1780729177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc01e600 con 0x7fba04108910 2024-03-26T08:47:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.734+0000 7fba09485640 1 -- 172.21.15.7:0/1780729177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba041a6390 con 0x7fba04108910 2024-03-26T08:47:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.737+0000 7fba0c075740 1 Processor -- start 2024-03-26T08:47:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.737+0000 7fb9fbfff640 1 -- 172.21.15.7:0/1780729177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb9fc004b20 con 0x7fba04108910 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.737+0000 7fba0c075740 1 -- start start 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.737+0000 7fba0c075740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.737+0000 7fba0c075740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a8310 0x5627fd5ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.737+0000 7fba0c075740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba0c075740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fd50cb10 con 0x5627fd5a7920 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba0c075740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fd4b5900 con 0x5627fd5ad480 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba0c075740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fd508bb0 con 0x5627fd5a8310 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a8310 0x5627fd5ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a8310 0x5627fd5ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:52812/0 (socket says 172.21.15.7:52812) 2024-03-26T08:47:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba02ffd640 1 -- 172.21.15.7:0/2338136046 learned_addr learned my addr 172.21.15.7:0/2338136046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba027fc640 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fb9fbfff640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb9e80a2870 0x7fb9e80a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba037fe640 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba02ffd640 1 -- 172.21.15.7:0/2338136046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 msgr2=0x5627fd5af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba02ffd640 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba02ffd640 1 -- 172.21.15.7:0/2338136046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 msgr2=0x5627fd5a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba02ffd640 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba02ffd640 1 -- 172.21.15.7:0/2338136046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627fd221210 con 0x5627fd5a8310 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba037fe640 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fb9fbfff640 1 -- 172.21.15.7:0/1780729177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb9fc0d6e80 con 0x7fba04108910 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.738+0000 7fba037fe640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb9e80a2870 0x7fb9e80a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.739+0000 7fba02ffd640 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a8310 0x5627fd5ace10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0d4940 tx=0x7fb9fc09f530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10bf32edae5493fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.739+0000 7fb9f97fa640 1 -- 172.21.15.7:0/2338136046 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0a3890 con 0x5627fd5a8310 2024-03-26T08:47:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.739+0000 7fb9f97fa640 1 -- 172.21.15.7:0/2338136046 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc0a7930 con 0x5627fd5a8310 2024-03-26T08:47:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.739+0000 7fb9f97fa640 1 -- 172.21.15.7:0/2338136046 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0a1c20 con 0x5627fd5a8310 2024-03-26T08:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.740+0000 7fba0c075740 1 -- 172.21.15.7:0/2338136046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a8310 msgr2=0x5627fd5ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.740+0000 7fba0c075740 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a8310 0x5627fd5ace10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0d4940 tx=0x7fb9fc09f530 comp rx=0 tx=0).stop 2024-03-26T08:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.740+0000 7fba037fe640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb9e80a2870 0x7fb9e80a4d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4005940 tx=0x7fb9f402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.740+0000 7fba0c075740 1 -- 172.21.15.7:0/2338136046 shutdown_connections 2024-03-26T08:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.740+0000 7fba0c075740 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.740+0000 7fba0c075740 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a8310 0x5627fd5ace10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.740+0000 7fba0c075740 1 --2- 172.21.15.7:0/2338136046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.740+0000 7fba0c075740 1 -- 172.21.15.7:0/2338136046 >> 172.21.15.7:0/2338136046 conn(0x5627fd5a2e90 msgr2=0x5627fd5a3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.741+0000 7fba0c075740 1 -- 172.21.15.7:0/2338136046 shutdown_connections 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.741+0000 7fba0c075740 1 -- 172.21.15.7:0/2338136046 wait complete. 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba0c075740 1 -- start start 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba0c075740 1 Processor -- start 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba0c075740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba0c075740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 0x5627fd5b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba0c075740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5ad480 0x5627fd5b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba0c075740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fd5b03d0 con 0x5627fd5ad480 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba0c075740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fd4c8d50 con 0x5627fd5a8310 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba0c075740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627fd5b0550 con 0x5627fd5a7920 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba0c075740 10 client.0 _ll_register_callbacks cb 0x5627fd4c8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:52826/0 (socket says 172.21.15.7:52826) 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba02ffd640 1 -- 172.21.15.7:0/3855112228 learned_addr learned my addr 172.21.15.7:0/3855112228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba037fe640 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5ad480 0x5627fd5b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.742+0000 7fba027fc640 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 0x5627fd5b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fba02ffd640 1 -- 172.21.15.7:0/3855112228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 msgr2=0x5627fd5b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fba02ffd640 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 0x5627fd5b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fba02ffd640 1 -- 172.21.15.7:0/3855112228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5ad480 msgr2=0x5627fd5b91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fba02ffd640 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5ad480 0x5627fd5b91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fba02ffd640 1 -- 172.21.15.7:0/3855112228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627fd2ed620 con 0x5627fd5a7920 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fba02ffd640 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b1c30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000f80 tx=0x7fb9fc0d48b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3fd2ea2f610a01d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fb9e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fb9e67fc640 1 -- 172.21.15.7:0/3855112228 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0a3890 con 0x5627fd5a7920 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fba0c075740 1 -- 172.21.15.7:0/3855112228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9fc0a7930 con 0x5627fd5a7920 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fb9e67fc640 1 -- 172.21.15.7:0/3855112228 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb9fc0d3d90 con 0x5627fd5a7920 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fb9e67fc640 1 -- 172.21.15.7:0/3855112228 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9fc0abf80 con 0x5627fd5a7920 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fba0c075740 10 client.26584 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.743+0000 7fba0c075740 1 -- 172.21.15.7:0/3855112228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5627fd5b0490 con 0x5627fd5a7920 2024-03-26T08:47:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.744+0000 7fb9e67fc640 1 -- 172.21.15.7:0/3855112228 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc0a7930 con 0x5627fd5a7920 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.744+0000 7fb9e67fc640 1 -- 172.21.15.7:0/3855112228 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb9fc0cace0 con 0x5627fd5a7920 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.744+0000 7fba0c075740 10 client.26584 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.744+0000 7fba0c075740 10 client.26584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.744+0000 7fba0c075740 20 client.26584 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.744+0000 7fba0c075740 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5627fd5bde00 0x5627fd5c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.744+0000 7fba0c075740 4 client.26584 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.744+0000 7fba0c075740 1 -- 172.21.15.7:0/3855112228 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627fd51a050 con 0x5627fd5bde00 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.744+0000 7fba027fc640 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5627fd5bde00 0x5627fd5c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.745+0000 7fba027fc640 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5627fd5bde00 0x5627fd5c0210 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44cc43e66a0824b server_cookie=ea978f955e4cc47c in_seq=0 out_seq=0 2024-03-26T08:47:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.745+0000 7fb9e67fc640 10 client.26584 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fb9e67fc640 1 -- 172.21.15.7:0/3855112228 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5627fd51a050 con 0x5627fd5bde00 2024-03-26T08:47:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fb9e67fc640 10 client.26584 handle_command_reply: tid=0 2024-03-26T08:47:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fba0c075740 1 client.26584 shutdown 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fba0c075740 1 -- 172.21.15.7:0/3855112228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 msgr2=0x5627fd5b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fba0c075740 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b1c30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc000f80 tx=0x7fb9fc0d48b0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fba0c075740 1 -- 172.21.15.7:0/3855112228 shutdown_connections 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fba0c075740 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5627fd5bde00 0x5627fd5c0210 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fba0c075740 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5ad480 0x5627fd5b91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fba0c075740 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 0x5627fd5b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fba0c075740 1 --2- 172.21.15.7:0/3855112228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b1c30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fba0c075740 1 -- 172.21.15.7:0/3855112228 >> 172.21.15.7:0/3855112228 conn(0x5627fd5a2e90 msgr2=0x5627fd5a2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.746+0000 7fb9e67fc640 0 client.26584 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.747+0000 7fba0c075740 1 -- 172.21.15.7:0/3855112228 shutdown_connections 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.747+0000 7fba0c075740 1 -- 172.21.15.7:0/3855112228 wait complete. 2024-03-26T08:47:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.747+0000 7fba0c075740 20 client.26584 trim_cache size 0 max 16384 2024-03-26T08:47:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.769+0000 7fba0c075740 1 Processor -- start 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.769+0000 7fba0c075740 1 -- start start 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.769+0000 7fba0c075740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.769+0000 7fba0c075740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 0x5627fd5b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba0c075740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5ad480 0x5627fd5b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba0c075740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041b1600 con 0x5627fd5a7920 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba0c075740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041af100 con 0x5627fd5a8310 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba0c075740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041ad180 con 0x5627fd5ad480 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba02ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38556/0 (socket says 172.21.15.7:38556) 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba027fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 0x5627fd5b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba02ffd640 1 -- 172.21.15.7:0/195656848 learned_addr learned my addr 172.21.15.7:0/195656848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba037fe640 1 --2- 172.21.15.7:0/195656848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5ad480 0x5627fd5b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba02ffd640 1 -- 172.21.15.7:0/195656848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5ad480 msgr2=0x5627fd5b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba02ffd640 1 --2- 172.21.15.7:0/195656848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5ad480 0x5627fd5b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba02ffd640 1 -- 172.21.15.7:0/195656848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 msgr2=0x5627fd5b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba02ffd640 1 --2- 172.21.15.7:0/195656848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 0x5627fd5b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba02ffd640 1 -- 172.21.15.7:0/195656848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627fd5c0890 con 0x5627fd5a7920 2024-03-26T08:47:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba027fc640 1 --2- 172.21.15.7:0/195656848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 0x5627fd5b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fba02ffd640 1 --2- 172.21.15.7:0/195656848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a3270 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0a3540 tx=0x7fb9fc0a1b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef66885fab7011ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.770+0000 7fb9f8ff9640 1 -- 172.21.15.7:0/195656848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc01f790 con 0x5627fd5a7920 2024-03-26T08:47:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.771+0000 7fb9f8ff9640 1 -- 172.21.15.7:0/195656848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc0a19a0 con 0x5627fd5a7920 2024-03-26T08:47:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.771+0000 7fb9f8ff9640 1 -- 172.21.15.7:0/195656848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc01f960 con 0x5627fd5a7920 2024-03-26T08:47:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.771+0000 7fba0c075740 1 -- 172.21.15.7:0/195656848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 msgr2=0x5627fd5a3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.771+0000 7fba0c075740 1 --2- 172.21.15.7:0/195656848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a3270 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0a3540 tx=0x7fb9fc0a1b90 comp rx=0 tx=0).stop 2024-03-26T08:47:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.773+0000 7fba0c075740 1 -- 172.21.15.7:0/195656848 shutdown_connections 2024-03-26T08:47:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.773+0000 7fba0c075740 1 --2- 172.21.15.7:0/195656848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5ad480 0x5627fd5b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.773+0000 7fba0c075740 1 --2- 172.21.15.7:0/195656848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5a8310 0x5627fd5b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.773+0000 7fba0c075740 1 --2- 172.21.15.7:0/195656848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a7920 0x5627fd5a3270 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.773+0000 7fba0c075740 1 -- 172.21.15.7:0/195656848 >> 172.21.15.7:0/195656848 conn(0x5627fd5a2e90 msgr2=0x5627fd59deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.773+0000 7fba0c075740 1 -- 172.21.15.7:0/195656848 shutdown_connections 2024-03-26T08:47:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.773+0000 7fba0c075740 1 -- 172.21.15.7:0/195656848 wait complete. 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba0c075740 1 -- start start 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba0c075740 1 Processor -- start 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba0c075740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba0c075740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a8310 0x5627fd5b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba0c075740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba0c075740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041ad780 con 0x5627fd5a8310 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba0c075740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041aeaa0 con 0x5627fd5ad480 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba0c075740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba041adde0 con 0x5627fd5a7920 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba0c075740 10 client.0 _ll_register_callbacks cb 0x7fb9fc0da3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a8310 0x5627fd5b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5b7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37720/0 (socket says 172.21.15.7:37720) 2024-03-26T08:47:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba037fe640 1 -- 172.21.15.7:0/4068790105 learned_addr learned my addr 172.21.15.7:0/4068790105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.774+0000 7fba02ffd640 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.775+0000 7fba027fc640 1 -- 172.21.15.7:0/4068790105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 msgr2=0x5627fd5b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.775+0000 7fba027fc640 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.775+0000 7fba027fc640 1 -- 172.21.15.7:0/4068790105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 msgr2=0x5627fd5b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.775+0000 7fba027fc640 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.775+0000 7fba027fc640 1 -- 172.21.15.7:0/4068790105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627fd5b8220 con 0x5627fd5a8310 2024-03-26T08:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.775+0000 7fba037fe640 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.775+0000 7fba027fc640 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a8310 0x5627fd5b6c20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5627fd5c0ac0 tx=0x7fb9f0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929f919709a2bc97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.775+0000 7fb9e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:47:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.775+0000 7fb9e67fc640 1 -- 172.21.15.7:0/4068790105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0003e50 con 0x5627fd5a8310 2024-03-26T08:47:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.776+0000 7fb9e67fc640 1 -- 172.21.15.7:0/4068790105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb9f0020520 con 0x5627fd5a8310 2024-03-26T08:47:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.776+0000 7fba0c075740 1 -- 172.21.15.7:0/4068790105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9fc0a19a0 con 0x5627fd5a8310 2024-03-26T08:47:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.776+0000 7fb9e67fc640 1 -- 172.21.15.7:0/4068790105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9f0020720 con 0x5627fd5a8310 2024-03-26T08:47:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.776+0000 7fba0c075740 10 client.16728 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.776+0000 7fba0c075740 1 -- 172.21.15.7:0/4068790105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5627fd6196e0 con 0x5627fd5a8310 2024-03-26T08:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.777+0000 7fb9e67fc640 1 -- 172.21.15.7:0/4068790105 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9f0020da0 con 0x5627fd5a8310 2024-03-26T08:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.777+0000 7fb9e67fc640 1 -- 172.21.15.7:0/4068790105 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb9f0032c40 con 0x5627fd5a8310 2024-03-26T08:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.777+0000 7fba0c075740 10 client.16728 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.777+0000 7fba0c075740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.777+0000 7fba0c075740 20 client.16728 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.777+0000 7fba0c075740 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5627fd619e20 0x5627fd61c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.777+0000 7fba0c075740 4 client.16728 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.777+0000 7fba0c075740 1 -- 172.21.15.7:0/4068790105 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fb9fc0a19a0 con 0x5627fd619e20 2024-03-26T08:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.778+0000 7fba02ffd640 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5627fd619e20 0x5627fd61c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.778+0000 7fb9e67fc640 10 client.16728 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.778+0000 7fba02ffd640 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5627fd619e20 0x5627fd61c240 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09d0bd36a75a9b server_cookie=28042b90337ff3f8 in_seq=0 out_seq=0 2024-03-26T08:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fb9e67fc640 1 -- 172.21.15.7:0/4068790105 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fb9fc0a19a0 con 0x5627fd619e20 2024-03-26T08:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fb9e67fc640 10 client.16728 handle_command_reply: tid=0 2024-03-26T08:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 client.16728 shutdown 2024-03-26T08:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 -- 172.21.15.7:0/4068790105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a8310 msgr2=0x5627fd5b6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a8310 0x5627fd5b6c20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5627fd5c0ac0 tx=0x7fb9f0005f90 comp rx=0 tx=0).stop 2024-03-26T08:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 -- 172.21.15.7:0/4068790105 shutdown_connections 2024-03-26T08:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5627fd619e20 0x5627fd61c240 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5627fd5ad480 0x5627fd5b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fb9e67fc640 0 client.16728 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5627fd5a8310 0x5627fd5b6c20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 --2- 172.21.15.7:0/4068790105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5627fd5a7920 0x5627fd5b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 -- 172.21.15.7:0/4068790105 >> 172.21.15.7:0/4068790105 conn(0x5627fd5a2e90 msgr2=0x5627fd5a3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.796+0000 7fba0c075740 1 -- 172.21.15.7:0/4068790105 shutdown_connections 2024-03-26T08:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.797+0000 7fba0c075740 1 -- 172.21.15.7:0/4068790105 wait complete. 2024-03-26T08:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.797+0000 7fba0c075740 20 client.16728 trim_cache size 0 max 16384 2024-03-26T08:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:47:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.798+0000 7fba09485640 1 -- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb9e80a2870 msgr2=0x7fb9e80a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.798+0000 7fba09485640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb9e80a2870 0x7fb9e80a4d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4005940 tx=0x7fb9f402e040 comp rx=0 tx=0).stop 2024-03-26T08:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.798+0000 7fba09485640 1 -- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba04108910 msgr2=0x7fba0419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:19.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.798+0000 7fba09485640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba04108910 0x7fba0419dfc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc007920 tx=0x7fb9fc007de0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.798+0000 7fba09485640 1 -- 172.21.15.7:0/1780729177 shutdown_connections 2024-03-26T08:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.798+0000 7fba09485640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb9e80a2870 0x7fb9e80a4d30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.798+0000 7fba09485640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba0410dd90 0x7fba041a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.799+0000 7fba09485640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba0410b2d0 0x7fba0419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.799+0000 7fba09485640 1 --2- 172.21.15.7:0/1780729177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba04108910 0x7fba0419dfc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.799+0000 7fba09485640 1 -- 172.21.15.7:0/1780729177 >> 172.21.15.7:0/1780729177 conn(0x7fba0406e8b0 msgr2=0x7fba04109250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.799+0000 7fba09485640 1 -- 172.21.15.7:0/1780729177 shutdown_connections 2024-03-26T08:47:19.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:19.799+0000 7fba09485640 1 -- 172.21.15.7:0/1780729177 wait complete. 2024-03-26T08:47:19.816 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:19 smithi044 ceph-mon[36461]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 18 MiB/s wr, 951 op/s 2024-03-26T08:47:19.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:19 smithi016 ceph-mon[35851]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 18 MiB/s wr, 951 op/s 2024-03-26T08:47:20.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:19 smithi007 ceph-mon[32462]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 18 MiB/s wr, 951 op/s 2024-03-26T08:47:21.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 290.0 s, 3483.2 tps, lat 28.714 ms stddev 30.798 2024-03-26T08:47:21.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:21 smithi016 ceph-mon[35851]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 20 MiB/s wr, 912 op/s 2024-03-26T08:47:21.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:21 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:47:21.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:21 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:47:22.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:21 smithi007 ceph-mon[32462]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 20 MiB/s wr, 912 op/s 2024-03-26T08:47:22.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:21 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:47:22.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:21 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:47:22.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:21 smithi044 ceph-mon[36461]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 20 MiB/s wr, 912 op/s 2024-03-26T08:47:22.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:21 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:47:22.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:21 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:47:23.845 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:23 smithi007 ceph-mon[32462]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 25 MiB/s wr, 1.01k op/s 2024-03-26T08:47:23.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:23 smithi016 ceph-mon[35851]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 25 MiB/s wr, 1.01k op/s 2024-03-26T08:47:24.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:23 smithi044 ceph-mon[36461]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 25 MiB/s wr, 1.01k op/s 2024-03-26T08:47:25.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:25 smithi016 ceph-mon[35851]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 25 MiB/s wr, 897 op/s 2024-03-26T08:47:26.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:25 smithi007 ceph-mon[32462]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 25 MiB/s wr, 897 op/s 2024-03-26T08:47:26.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:25 smithi044 ceph-mon[36461]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 25 MiB/s wr, 897 op/s 2024-03-26T08:47:26.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 295.0 s, 2963.4 tps, lat 33.764 ms stddev 37.573 2024-03-26T08:47:27.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:27 smithi016 ceph-mon[35851]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 23 MiB/s wr, 818 op/s 2024-03-26T08:47:28.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:27 smithi007 ceph-mon[32462]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 23 MiB/s wr, 818 op/s 2024-03-26T08:47:28.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:27 smithi044 ceph-mon[36461]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 23 MiB/s wr, 818 op/s 2024-03-26T08:47:29.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:29 smithi016 ceph-mon[35851]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 21 MiB/s wr, 767 op/s 2024-03-26T08:47:30.042 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:29 smithi007 ceph-mon[32462]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 21 MiB/s wr, 767 op/s 2024-03-26T08:47:30.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:29 smithi044 ceph-mon[36461]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 21 MiB/s wr, 767 op/s 2024-03-26T08:47:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 300.0 s, 3069.4 tps, lat 32.467 ms stddev 36.284 2024-03-26T08:47:31.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:31 smithi016 ceph-mon[35851]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 703 op/s 2024-03-26T08:47:32.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:31 smithi007 ceph-mon[32462]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 703 op/s 2024-03-26T08:47:32.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:31 smithi044 ceph-mon[36461]: pgmap v513: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 703 op/s 2024-03-26T08:47:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.972+0000 7fbe47d1b640 1 -- 172.21.15.7:0/2895564476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe401028c0 msgr2=0x7fbe40104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.972+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/2895564476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe401028c0 0x7fbe40104d30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009860 tx=0x7fbe3002bbb0 comp rx=0 tx=0).stop 2024-03-26T08:47:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.982+0000 7fbe47d1b640 1 -- 172.21.15.7:0/2895564476 shutdown_connections 2024-03-26T08:47:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.982+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/2895564476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe40105270 0x7fbe401056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.982+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/2895564476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe401028c0 0x7fbe40104d30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.982+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/2895564476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe400fff50 0x7fbe40102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.982+0000 7fbe47d1b640 1 -- 172.21.15.7:0/2895564476 >> 172.21.15.7:0/2895564476 conn(0x7fbe400fbb70 msgr2=0x7fbe400fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.983+0000 7fbe47d1b640 1 -- 172.21.15.7:0/2895564476 shutdown_connections 2024-03-26T08:47:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.984+0000 7fbe47d1b640 1 -- 172.21.15.7:0/2895564476 wait complete. 2024-03-26T08:47:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.985+0000 7fbe47d1b640 1 Processor -- start 2024-03-26T08:47:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.985+0000 7fbe47d1b640 1 -- start start 2024-03-26T08:47:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.985+0000 7fbe47d1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe400fff50 0x7fbe4010b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.986+0000 7fbe45ab7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe400fff50 0x7fbe4010b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.986+0000 7fbe45ab7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe400fff50 0x7fbe4010b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49610/0 (socket says 172.21.15.7:49610) 2024-03-26T08:47:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.987+0000 7fbe47d1b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401028c0 0x7fbe4010ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.988+0000 7fbe452b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401028c0 0x7fbe4010ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.988+0000 7fbe452b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401028c0 0x7fbe4010ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47826/0 (socket says 172.21.15.7:47826) 2024-03-26T08:47:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.990+0000 7fbe47d1b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe40105270 0x7fbe401ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.990+0000 7fbe47d1b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40113a10 con 0x7fbe40105270 2024-03-26T08:47:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.990+0000 7fbe47d1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40113890 con 0x7fbe400fff50 2024-03-26T08:47:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.990+0000 7fbe47d1b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40113b90 con 0x7fbe401028c0 2024-03-26T08:47:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.990+0000 7fbe45ab7640 1 -- 172.21.15.7:0/1928364518 learned_addr learned my addr 172.21.15.7:0/1928364518 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.991+0000 7fbe45ab7640 1 -- 172.21.15.7:0/1928364518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401028c0 msgr2=0x7fbe4010ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.992+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401028c0 0x7fbe4010ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.992+0000 7fbe45ab7640 1 -- 172.21.15.7:0/1928364518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe40105270 msgr2=0x7fbe401ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.992+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe40105270 0x7fbe401ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.992+0000 7fbe45ab7640 1 -- 172.21.15.7:0/1928364518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe401ae660 con 0x7fbe400fff50 2024-03-26T08:47:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.995+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe400fff50 0x7fbe4010b510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00b450 tx=0x7fbe3c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.996+0000 7fbe36ffd640 1 -- 172.21.15.7:0/1928364518 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c015020 con 0x7fbe400fff50 2024-03-26T08:47:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.996+0000 7fbe36ffd640 1 -- 172.21.15.7:0/1928364518 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe3c011d70 con 0x7fbe400fff50 2024-03-26T08:47:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.996+0000 7fbe36ffd640 1 -- 172.21.15.7:0/1928364518 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00f340 con 0x7fbe400fff50 2024-03-26T08:47:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.997+0000 7fbe47d1b640 1 -- 172.21.15.7:0/1928364518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe401ae830 con 0x7fbe400fff50 2024-03-26T08:47:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:32.997+0000 7fbe47d1b640 1 -- 172.21.15.7:0/1928364518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe401aed20 con 0x7fbe400fff50 2024-03-26T08:47:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.001+0000 7fbe4a9e6740 1 Processor -- start 2024-03-26T08:47:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.002+0000 7fbe4a9e6740 1 -- start start 2024-03-26T08:47:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.002+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.002+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 0x5634b9e80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.002+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e81480 0x5634b9e83910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.002+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b9de0b10 con 0x5634b9e7b920 2024-03-26T08:47:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.002+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b9d89900 con 0x5634b9e7c310 2024-03-26T08:47:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.002+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b9ddcbb0 con 0x5634b9e81480 2024-03-26T08:47:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.003+0000 7fbe462b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.003+0000 7fbe462b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43558/0 (socket says 172.21.15.7:43558) 2024-03-26T08:47:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.003+0000 7fbe462b8640 1 -- 172.21.15.7:0/2613425855 learned_addr learned my addr 172.21.15.7:0/2613425855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.003+0000 7fbe452b6640 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e81480 0x5634b9e83910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.003+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 0x5634b9e80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.003+0000 7fbe36ffd640 1 -- 172.21.15.7:0/1928364518 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fbe3c018020 con 0x7fbe400fff50 2024-03-26T08:47:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.004+0000 7fbe452b6640 1 -- 172.21.15.7:0/2613425855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 msgr2=0x5634b9e80e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.004+0000 7fbe452b6640 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 0x5634b9e80e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.004+0000 7fbe452b6640 1 -- 172.21.15.7:0/2613425855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7b920 msgr2=0x5634b9e7bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.004+0000 7fbe452b6640 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.004+0000 7fbe452b6640 1 -- 172.21.15.7:0/2613425855 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634b9af5210 con 0x5634b9e81480 2024-03-26T08:47:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.004+0000 7fbe462b8640 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.004+0000 7fbe36ffd640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbe2c0a2a00 0x7fbe2c0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.004+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 0x5634b9e80e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:47:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.005+0000 7fbe462b8640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbe2c0a2a00 0x7fbe2c0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.005+0000 7fbe452b6640 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e81480 0x5634b9e83910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009780 tx=0x7fbe30032340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f17ed2f4b49fe206 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.005+0000 7fbe462b8640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbe2c0a2a00 0x7fbe2c0a4ec0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009330 tx=0x7fbe38009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.005+0000 7fbe36ffd640 1 -- 172.21.15.7:0/1928364518 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbe3c0ce450 con 0x7fbe400fff50 2024-03-26T08:47:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.006+0000 7fbe2bfff640 1 -- 172.21.15.7:0/2613425855 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30043070 con 0x5634b9e81480 2024-03-26T08:47:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.006+0000 7fbe2bfff640 1 -- 172.21.15.7:0/2613425855 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe30034e50 con 0x5634b9e81480 2024-03-26T08:47:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.006+0000 7fbe2bfff640 1 -- 172.21.15.7:0/2613425855 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30039040 con 0x5634b9e81480 2024-03-26T08:47:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.007+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2613425855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e81480 msgr2=0x5634b9e83910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.007+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e81480 0x5634b9e83910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009780 tx=0x7fbe30032340 comp rx=0 tx=0).stop 2024-03-26T08:47:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.008+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2613425855 shutdown_connections 2024-03-26T08:47:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.008+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e81480 0x5634b9e83910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.008+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 0x5634b9e80e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.008+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2613425855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.008+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2613425855 >> 172.21.15.7:0/2613425855 conn(0x5634b9e76e90 msgr2=0x5634b9e77290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.008+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2613425855 shutdown_connections 2024-03-26T08:47:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.009+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2613425855 wait complete. 2024-03-26T08:47:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.009+0000 7fbe4a9e6740 1 -- start start 2024-03-26T08:47:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.009+0000 7fbe4a9e6740 1 Processor -- start 2024-03-26T08:47:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e96c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 0x5634b9e97170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e81480 0x5634b9e9e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b9e843d0 con 0x5634b9e81480 2024-03-26T08:47:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b9d9cd50 con 0x5634b9e7c310 2024-03-26T08:47:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634b9e84550 con 0x5634b9e7b920 2024-03-26T08:47:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe4a9e6740 10 client.0 _ll_register_callbacks cb 0x5634b9d9c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe462b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e81480 0x5634b9e9e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe462b8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e81480 0x5634b9e9e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43560/0 (socket says 172.21.15.7:43560) 2024-03-26T08:47:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe462b8640 1 -- 172.21.15.7:0/1859775294 learned_addr learned my addr 172.21.15.7:0/1859775294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.010+0000 7fbe452b6640 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 0x5634b9e97170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.011+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e96c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.014+0000 7fbe462b8640 1 -- 172.21.15.7:0/1859775294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 msgr2=0x5634b9e96c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.014+0000 7fbe462b8640 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e96c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.014+0000 7fbe462b8640 1 -- 172.21.15.7:0/1859775294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 msgr2=0x5634b9e97170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.014+0000 7fbe462b8640 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 0x5634b9e97170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.014+0000 7fbe462b8640 1 -- 172.21.15.7:0/1859775294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634b9bc1620 con 0x5634b9e81480 2024-03-26T08:47:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.014+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e96c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.014+0000 7fbe462b8640 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e81480 0x5634b9e9e1b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800ed00 tx=0x7fbe380092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401d3ce17eaae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.015+0000 7fbe297fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:47:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.015+0000 7fbe297fa640 1 -- 172.21.15.7:0/1859775294 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800ed30 con 0x5634b9e81480 2024-03-26T08:47:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.015+0000 7fbe297fa640 1 -- 172.21.15.7:0/1859775294 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbe38003970 con 0x5634b9e81480 2024-03-26T08:47:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.015+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/1859775294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe30034e50 con 0x5634b9e81480 2024-03-26T08:47:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.016+0000 7fbe297fa640 1 -- 172.21.15.7:0/1859775294 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe38003b70 con 0x5634b9e81480 2024-03-26T08:47:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.016+0000 7fbe4a9e6740 10 client.16737 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.016+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/1859775294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5634b9ea1560 con 0x5634b9e81480 2024-03-26T08:47:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.017+0000 7fbe297fa640 1 -- 172.21.15.7:0/1859775294 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe30034e50 con 0x5634b9e81480 2024-03-26T08:47:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.017+0000 7fbe297fa640 1 -- 172.21.15.7:0/1859775294 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbe38024070 con 0x5634b9e81480 2024-03-26T08:47:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.017+0000 7fbe4a9e6740 10 client.16737 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.017+0000 7fbe4a9e6740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.017+0000 7fbe4a9e6740 20 client.16737 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.017+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634b9ea3040 0x5634b9ea5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.017+0000 7fbe4a9e6740 4 client.16737 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.017+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/1859775294 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634b9dee050 con 0x5634b9ea3040 2024-03-26T08:47:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.019+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634b9ea3040 0x5634b9ea5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.024+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634b9ea3040 0x5634b9ea5480 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba35c7e1590e9da server_cookie=9bb1e6910a6b4429 in_seq=0 out_seq=0 2024-03-26T08:47:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.025+0000 7fbe297fa640 10 client.16737 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.029+0000 7fbe297fa640 1 -- 172.21.15.7:0/1859775294 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5634b9dee050 con 0x5634b9ea3040 2024-03-26T08:47:33.021 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.029+0000 7fbe297fa640 10 client.16737 handle_command_reply: tid=0 2024-03-26T08:47:33.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.029+0000 7fbe4a9e6740 1 client.16737 shutdown 2024-03-26T08:47:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.030+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/1859775294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e81480 msgr2=0x5634b9e9e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.030+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e81480 0x5634b9e9e1b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800ed00 tx=0x7fbe380092c0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.030+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/1859775294 shutdown_connections 2024-03-26T08:47:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.030+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634b9ea3040 0x5634b9ea5480 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.030+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e81480 0x5634b9e9e1b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.031+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e7c310 0x5634b9e97170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.031+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/1859775294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e96c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.031+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/1859775294 >> 172.21.15.7:0/1859775294 conn(0x5634b9e76e90 msgr2=0x5634b9e76020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.031+0000 7fbe297fa640 0 client.16737 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.031+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/1859775294 shutdown_connections 2024-03-26T08:47:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.031+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/1859775294 wait complete. 2024-03-26T08:47:33.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.032+0000 7fbe4a9e6740 20 client.16737 trim_cache size 0 max 16384 2024-03-26T08:47:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.056+0000 7fbe4a9e6740 1 Processor -- start 2024-03-26T08:47:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.057+0000 7fbe4a9e6740 1 -- start start 2024-03-26T08:47:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.057+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.057+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.057+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 0x5634b9e77270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.057+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe401b1870 con 0x5634b9e7c310 2024-03-26T08:47:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe45ab7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe452b6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe45ab7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47856/0 (socket says 172.21.15.7:47856) 2024-03-26T08:47:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe45ab7640 1 -- 172.21.15.7:0/977547516 learned_addr learned my addr 172.21.15.7:0/977547516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/977547516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe401af370 con 0x5634b9e81480 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/977547516 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40071d00 con 0x5634b9e7b920 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe452b6640 1 -- 172.21.15.7:0/977547516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 msgr2=0x5634b9e7bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe452b6640 1 --2- 172.21.15.7:0/977547516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe452b6640 1 -- 172.21.15.7:0/977547516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 msgr2=0x5634b9e77270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe452b6640 1 --2- 172.21.15.7:0/977547516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 0x5634b9e77270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe452b6640 1 -- 172.21.15.7:0/977547516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634b9e7b710 con 0x5634b9e7c310 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.058+0000 7fbe452b6640 1 --2- 172.21.15.7:0/977547516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9e170 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x5634b9e77e00 tx=0x7fbe30005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697a642b1ea4a9de server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.059+0000 7fbe2b7fe640 1 -- 172.21.15.7:0/977547516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3004c070 con 0x5634b9e7c310 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.059+0000 7fbe2b7fe640 1 -- 172.21.15.7:0/977547516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe30034e50 con 0x5634b9e7c310 2024-03-26T08:47:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.059+0000 7fbe2b7fe640 1 -- 172.21.15.7:0/977547516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30039040 con 0x5634b9e7c310 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.060+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/977547516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 msgr2=0x5634b9e9e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.060+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/977547516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9e170 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x5634b9e77e00 tx=0x7fbe30005f50 comp rx=0 tx=0).stop 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.060+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/977547516 shutdown_connections 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.060+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/977547516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 0x5634b9e77270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.060+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/977547516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9e170 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.060+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/977547516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e7bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.060+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/977547516 >> 172.21.15.7:0/977547516 conn(0x5634b9e76e90 msgr2=0x5634b9e71eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.060+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/977547516 shutdown_connections 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.060+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/977547516 wait complete. 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.061+0000 7fbe4a9e6740 1 -- start start 2024-03-26T08:47:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.061+0000 7fbe4a9e6740 1 Processor -- start 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.062+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e9b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.062+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.062+0000 7fbe4a9e6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 0x5634b9e9c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.062+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40072300 con 0x5634b9e7c310 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.062+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40073620 con 0x5634b9e81480 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.062+0000 7fbe4a9e6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40072960 con 0x5634b9e7b920 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe462b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 0x5634b9e9c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe462b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 0x5634b9e9c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:49662/0 (socket says 172.21.15.7:49662) 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe462b8640 1 -- 172.21.15.7:0/2217339321 learned_addr learned my addr 172.21.15.7:0/2217339321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e9b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe4a9e6740 10 client.0 _ll_register_callbacks cb 0x7fbe3800e980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe452b6640 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe462b8640 1 -- 172.21.15.7:0/2217339321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 msgr2=0x5634b9e9b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe462b8640 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e9b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe462b8640 1 -- 172.21.15.7:0/2217339321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 msgr2=0x5634b9e9b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe462b8640 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.063+0000 7fbe462b8640 1 -- 172.21.15.7:0/2217339321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634b9e9ce60 con 0x5634b9e81480 2024-03-26T08:47:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.064+0000 7fbe452b6640 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:47:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.064+0000 7fbe462b8640 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 0x5634b9e9c870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe38016040 tx=0x7fbe380308e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2678d37af325bd51 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.064+0000 7fbe297fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.065+0000 7fbe297fa640 1 -- 172.21.15.7:0/2217339321 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38030ce0 con 0x5634b9e81480 2024-03-26T08:47:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.065+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2217339321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe30034e50 con 0x5634b9e81480 2024-03-26T08:47:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.065+0000 7fbe297fa640 1 -- 172.21.15.7:0/2217339321 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbe38026440 con 0x5634b9e81480 2024-03-26T08:47:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.068+0000 7fbe297fa640 1 -- 172.21.15.7:0/2217339321 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe38041050 con 0x5634b9e81480 2024-03-26T08:47:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.068+0000 7fbe4a9e6740 10 client.26567 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.068+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2217339321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5634b9e9e180 con 0x5634b9e81480 2024-03-26T08:47:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.070+0000 7fbe297fa640 1 -- 172.21.15.7:0/2217339321 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe30034e50 con 0x5634b9e81480 2024-03-26T08:47:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.070+0000 7fbe297fa640 1 -- 172.21.15.7:0/2217339321 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbe38024070 con 0x5634b9e81480 2024-03-26T08:47:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.070+0000 7fbe4a9e6740 10 client.26567 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.070+0000 7fbe4a9e6740 10 client.26567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.070+0000 7fbe4a9e6740 20 client.26567 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.070+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634b9efed60 0x5634b9f01180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.070+0000 7fbe4a9e6740 4 client.26567 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:47:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.070+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2217339321 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5634b9de83a0 con 0x5634b9efed60 2024-03-26T08:47:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.071+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634b9efed60 0x5634b9f01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.073+0000 7fbe45ab7640 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634b9efed60 0x5634b9f01180 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc2e647ec30cb56 server_cookie=638abab643a6c8f5 in_seq=0 out_seq=0 2024-03-26T08:47:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.073+0000 7fbe297fa640 10 client.26567 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:33.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe297fa640 1 -- 172.21.15.7:0/2217339321 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5634b9de83a0 con 0x5634b9efed60 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe297fa640 10 client.26567 handle_command_reply: tid=0 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 client.26567 shutdown 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2217339321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 msgr2=0x5634b9e9c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 0x5634b9e9c870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe38016040 tx=0x7fbe380308e0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2217339321 shutdown_connections 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634b9efed60 0x5634b9f01180 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634b9e81480 0x5634b9e9c870 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634b9e7c310 0x5634b9e9b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe297fa640 0 client.26567 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 --2- 172.21.15.7:0/2217339321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634b9e7b920 0x5634b9e9b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2217339321 >> 172.21.15.7:0/2217339321 conn(0x5634b9e76e90 msgr2=0x5634b9e77270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2217339321 shutdown_connections 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 1 -- 172.21.15.7:0/2217339321 wait complete. 2024-03-26T08:47:33.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.275+0000 7fbe4a9e6740 20 client.26567 trim_cache size 0 max 16384 2024-03-26T08:47:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:47:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:47:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:47:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 -- 172.21.15.7:0/1928364518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbe2c0a2a00 msgr2=0x7fbe2c0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbe2c0a2a00 0x7fbe2c0a4ec0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009330 tx=0x7fbe38009680 comp rx=0 tx=0).stop 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 -- 172.21.15.7:0/1928364518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe400fff50 msgr2=0x7fbe4010b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe400fff50 0x7fbe4010b510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00b450 tx=0x7fbe3c00b910 comp rx=0 tx=0).stop 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 -- 172.21.15.7:0/1928364518 shutdown_connections 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbe2c0a2a00 0x7fbe2c0a4ec0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe40105270 0x7fbe401ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbe401028c0 0x7fbe4010ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 --2- 172.21.15.7:0/1928364518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe400fff50 0x7fbe4010b510 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:33.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 -- 172.21.15.7:0/1928364518 >> 172.21.15.7:0/1928364518 conn(0x7fbe400fbb70 msgr2=0x7fbe40100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 -- 172.21.15.7:0/1928364518 shutdown_connections 2024-03-26T08:47:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:33.276+0000 7fbe47d1b640 1 -- 172.21.15.7:0/1928364518 wait complete. 2024-03-26T08:47:33.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:33 smithi016 ceph-mon[35851]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 800 op/s 2024-03-26T08:47:34.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:33 smithi007 ceph-mon[32462]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 800 op/s 2024-03-26T08:47:34.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:33 smithi044 ceph-mon[36461]: pgmap v514: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 20 MiB/s wr, 800 op/s 2024-03-26T08:47:35.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:35 smithi016 ceph-mon[35851]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 MiB/s wr, 1.11k op/s 2024-03-26T08:47:36.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:35 smithi007 ceph-mon[32462]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 MiB/s wr, 1.11k op/s 2024-03-26T08:47:36.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:35 smithi044 ceph-mon[36461]: pgmap v515: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 31 MiB/s wr, 1.11k op/s 2024-03-26T08:47:36.071 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 305.0 s, 2494.6 tps, lat 39.892 ms stddev 81.126 2024-03-26T08:47:37.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:37 smithi016 ceph-mon[35851]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 1.38k op/s 2024-03-26T08:47:38.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:37 smithi007 ceph-mon[32462]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 1.38k op/s 2024-03-26T08:47:38.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:37 smithi044 ceph-mon[36461]: pgmap v516: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 1.38k op/s 2024-03-26T08:47:38.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:47:38.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:47:38.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:47:38.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:47:39.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:47:39.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:47:39.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:47:39.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:47:39.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:47:39.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:47:39.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:47:39.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:47:39.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:39 smithi016 ceph-mon[35851]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.51k op/s 2024-03-26T08:47:40.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:39 smithi007 ceph-mon[32462]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.51k op/s 2024-03-26T08:47:40.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:39 smithi044 ceph-mon[36461]: pgmap v517: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 45 MiB/s wr, 1.51k op/s 2024-03-26T08:47:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 310.0 s, 2928.0 tps, lat 34.390 ms stddev 38.959 2024-03-26T08:47:41.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:41 smithi016 ceph-mon[35851]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 46 MiB/s wr, 1.53k op/s 2024-03-26T08:47:42.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:41 smithi007 ceph-mon[32462]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 46 MiB/s wr, 1.53k op/s 2024-03-26T08:47:42.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:41 smithi044 ceph-mon[36461]: pgmap v518: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 46 MiB/s wr, 1.53k op/s 2024-03-26T08:47:43.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:47:43.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:47:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.713+0000 7f8e84953640 1 -- 172.21.15.7:0/1250455412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 msgr2=0x7f8e80110f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.713+0000 7f8e84953640 1 --2- 172.21.15.7:0/1250455412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 0x7f8e80110f40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009ca0 tx=0x7f8e7802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.713+0000 7f8e84953640 1 -- 172.21.15.7:0/1250455412 shutdown_connections 2024-03-26T08:47:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.713+0000 7f8e84953640 1 --2- 172.21.15.7:0/1250455412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 0x7f8e80110f40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.713+0000 7f8e84953640 1 --2- 172.21.15.7:0/1250455412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e8010c0e0 0x7f8e8010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.713+0000 7f8e84953640 1 --2- 172.21.15.7:0/1250455412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e801097b0 0x7f8e8010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.713+0000 7f8e84953640 1 -- 172.21.15.7:0/1250455412 >> 172.21.15.7:0/1250455412 conn(0x7f8e8006e880 msgr2=0x7f8e80070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.713+0000 7f8e84953640 1 -- 172.21.15.7:0/1250455412 shutdown_connections 2024-03-26T08:47:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.713+0000 7f8e84953640 1 -- 172.21.15.7:0/1250455412 wait complete. 2024-03-26T08:47:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e84953640 1 Processor -- start 2024-03-26T08:47:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e84953640 1 -- start start 2024-03-26T08:47:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e84953640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e801097b0 0x7f8e8019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e84953640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e8010c0e0 0x7f8e801a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e84953640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 0x7f8e801a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e84953640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80115b60 con 0x7f8e8010eaa0 2024-03-26T08:47:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e84953640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801159e0 con 0x7f8e8010c0e0 2024-03-26T08:47:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e84953640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80115ce0 con 0x7f8e801097b0 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e7ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 0x7f8e801a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e7e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e801097b0 0x7f8e8019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e8010c0e0 0x7f8e801a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e7ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 0x7f8e801a75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38096/0 (socket says 172.21.15.7:38096) 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.714+0000 7f8e7ed9d640 1 -- 172.21.15.7:0/1972957220 learned_addr learned my addr 172.21.15.7:0/1972957220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.715+0000 7f8e7ed9d640 1 -- 172.21.15.7:0/1972957220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e801097b0 msgr2=0x7f8e8019fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.715+0000 7f8e7ed9d640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e801097b0 0x7f8e8019fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.715+0000 7f8e7ed9d640 1 -- 172.21.15.7:0/1972957220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e8010c0e0 msgr2=0x7f8e801a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.715+0000 7f8e7ed9d640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e8010c0e0 0x7f8e801a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.715+0000 7f8e7ed9d640 1 -- 172.21.15.7:0/1972957220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e801a7c10 con 0x7f8e8010eaa0 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.715+0000 7f8e77fff640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e8010c0e0 0x7f8e801a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.715+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e801097b0 0x7f8e8019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.715+0000 7f8e7ed9d640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 0x7f8e801a75a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8e78005e30 tx=0x7f8e78035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.716+0000 7f8e777fe640 1 -- 172.21.15.7:0/1972957220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78039040 con 0x7f8e8010eaa0 2024-03-26T08:47:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.716+0000 7f8e777fe640 1 -- 172.21.15.7:0/1972957220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e78044070 con 0x7f8e8010eaa0 2024-03-26T08:47:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.716+0000 7f8e777fe640 1 -- 172.21.15.7:0/1972957220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78004280 con 0x7f8e8010eaa0 2024-03-26T08:47:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.716+0000 7f8e84953640 1 -- 172.21.15.7:0/1972957220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e801a7ea0 con 0x7f8e8010eaa0 2024-03-26T08:47:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.716+0000 7f8e84953640 1 -- 172.21.15.7:0/1972957220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e801a8250 con 0x7f8e8010eaa0 2024-03-26T08:47:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.718+0000 7f8e777fe640 1 -- 172.21.15.7:0/1972957220 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8e78004420 con 0x7f8e8010eaa0 2024-03-26T08:47:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e8761c740 1 Processor -- start 2024-03-26T08:47:43.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e8761c740 1 -- start start 2024-03-26T08:47:43.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb1d20 0x55b13ccb2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccb5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb5870 0x55b13ccc26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e8761c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13cc173d0 con 0x55b13ccb1d20 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e8761c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13cbbfad0 con 0x55b13ccb2710 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e8761c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13cc13470 con 0x55b13ccb5870 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e7e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccb5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e7e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccb5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41558/0 (socket says 172.21.15.7:41558) 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.719+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2500467443 learned_addr learned my addr 172.21.15.7:0/2500467443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.720+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2500467443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb5870 msgr2=0x55b13ccc26e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.720+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/2500467443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb5870 0x55b13ccc26e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.720+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2500467443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb1d20 msgr2=0x55b13ccb2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.720+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/2500467443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb1d20 0x55b13ccb2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.720+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2500467443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b13c9f57d0 con 0x55b13ccb2710 2024-03-26T08:47:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.720+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/2500467443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccb5200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8e700114f0 tx=0x7f8e700119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0b79030e3dc3828 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.720+0000 7f8e777fe640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8e6c0a2950 0x7f8e6c0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.720+0000 7f8e777fe640 1 -- 172.21.15.7:0/1972957220 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8e780f0790 con 0x7f8e8010eaa0 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.721+0000 7f8e74ff9640 1 -- 172.21.15.7:0/2500467443 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7001e070 con 0x55b13ccb2710 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.721+0000 7f8e74ff9640 1 -- 172.21.15.7:0/2500467443 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e700035f0 con 0x55b13ccb2710 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.721+0000 7f8e74ff9640 1 -- 172.21.15.7:0/2500467443 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7000f8b0 con 0x55b13ccb2710 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.721+0000 7f8e77fff640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8e6c0a2950 0x7f8e6c0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e77fff640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8e6c0a2950 0x7f8e6c0a4e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009710 tx=0x7f8e68009c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 -- 172.21.15.7:0/2500467443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 msgr2=0x55b13ccb5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2500467443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccb5200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8e700114f0 tx=0x7f8e700119b0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 -- 172.21.15.7:0/2500467443 shutdown_connections 2024-03-26T08:47:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2500467443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb5870 0x55b13ccc26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2500467443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccb5200 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2500467443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb1d20 0x55b13ccb2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 -- 172.21.15.7:0/2500467443 >> 172.21.15.7:0/2500467443 conn(0x55b13ccad310 msgr2=0x55b13ccad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 -- 172.21.15.7:0/2500467443 shutdown_connections 2024-03-26T08:47:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 -- 172.21.15.7:0/2500467443 wait complete. 2024-03-26T08:47:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.722+0000 7f8e8761c740 1 -- start start 2024-03-26T08:47:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.723+0000 7f8e8761c740 1 Processor -- start 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.723+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb1d20 0x55b13ccc4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.723+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb2710 0x55b13ccc5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.723+0000 7f8e7e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb1d20 0x55b13ccc4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.723+0000 7f8e7e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb1d20 0x55b13ccc4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41574/0 (socket says 172.21.15.7:41574) 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.723+0000 7f8e7e59c640 1 -- 172.21.15.7:0/3424264333 learned_addr learned my addr 172.21.15.7:0/3424264333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.723+0000 7f8e8761c740 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 0x55b13cccbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13ccc30e0 con 0x55b13ccb5870 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e77fff640 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb2710 0x55b13ccc5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13cbd3060 con 0x55b13ccb1d20 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13ccc3260 con 0x55b13ccb2710 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e7ed9d640 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 0x55b13cccbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e8761c740 10 client.0 _ll_register_callbacks cb 0x55b13cbb56a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e7e59c640 1 -- 172.21.15.7:0/3424264333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb2710 msgr2=0x55b13ccc5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb2710 0x55b13ccc5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e7e59c640 1 -- 172.21.15.7:0/3424264333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 msgr2=0x55b13cccbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 0x55b13cccbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e7e59c640 1 -- 172.21.15.7:0/3424264333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b13c92b3f0 con 0x55b13ccb1d20 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e77fff640 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb2710 0x55b13ccc5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.724+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e700035f0 con 0x55b13ccb1d20 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb1d20 0x55b13ccc4ad0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8e700114c0 tx=0x7f8e700041e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270e6a6b2aa02f4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e65ffb640 10 client.26579 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e65ffb640 1 -- 172.21.15.7:0/3424264333 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7001e040 con 0x55b13ccb1d20 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e65ffb640 1 -- 172.21.15.7:0/3424264333 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8e700042d0 con 0x55b13ccb1d20 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e65ffb640 1 -- 172.21.15.7:0/3424264333 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e70005030 con 0x55b13ccb1d20 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e8761c740 10 client.26579 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55b13c8eb0f0 con 0x55b13ccb1d20 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e65ffb640 1 -- 172.21.15.7:0/3424264333 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8e7002b030 con 0x55b13ccb1d20 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e8761c740 10 client.26579 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e8761c740 10 client.26579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e8761c740 20 client.26579 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e8761c740 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b13ccd0050 0x55b13ccd2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e8761c740 4 client.26579 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b13cc24910 con 0x55b13ccd0050 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.725+0000 7f8e77fff640 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b13ccd0050 0x55b13ccd2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.726+0000 7f8e77fff640 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b13ccd0050 0x55b13ccd2440 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0a76f7b9e751e2e server_cookie=4b269f2efca06445 in_seq=0 out_seq=0 2024-03-26T08:47:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.726+0000 7f8e65ffb640 10 client.26579 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:43.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e65ffb640 1 -- 172.21.15.7:0/3424264333 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b13cc24910 con 0x55b13ccd0050 2024-03-26T08:47:43.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e65ffb640 10 client.26579 handle_command_reply: tid=0 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 client.26579 shutdown 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb1d20 msgr2=0x55b13ccc4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb1d20 0x55b13ccc4ad0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8e700114c0 tx=0x7f8e700041e0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 shutdown_connections 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b13ccd0050 0x55b13ccd2440 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e65ffb640 0 client.26579 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 0x55b13cccbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb2710 0x55b13ccc5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 --2- 172.21.15.7:0/3424264333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb1d20 0x55b13ccc4ad0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 >> 172.21.15.7:0/3424264333 conn(0x55b13ccad310 msgr2=0x55b13ccac530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 shutdown_connections 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 1 -- 172.21.15.7:0/3424264333 wait complete. 2024-03-26T08:47:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.727+0000 7f8e8761c740 20 client.26579 trim_cache size 0 max 16384 2024-03-26T08:47:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e8761c740 1 Processor -- start 2024-03-26T08:47:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e8761c740 1 -- start start 2024-03-26T08:47:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccabca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb2710 0x55b13cccbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb5870 0x55b13ccac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e8761c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801b1370 con 0x55b13ccb2710 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e8761c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801aee70 con 0x55b13ccb5870 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e8761c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801acef0 con 0x55b13ccb1d20 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb2710 0x55b13cccbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e7e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccabca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb2710 0x55b13cccbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38114/0 (socket says 172.21.15.7:38114) 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e77fff640 1 -- 172.21.15.7:0/2623805126 learned_addr learned my addr 172.21.15.7:0/2623805126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.751+0000 7f8e7ed9d640 1 --2- 172.21.15.7:0/2623805126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb5870 0x55b13ccac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.752+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2623805126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb5870 msgr2=0x55b13ccac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.752+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/2623805126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb5870 0x55b13ccac1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.752+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2623805126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb2710 msgr2=0x55b13cccbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.752+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/2623805126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb2710 0x55b13cccbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.752+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2623805126 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b13ccc2370 con 0x55b13ccb1d20 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.752+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/2623805126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccabca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009da0 tx=0x7f8e7001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f85b151e66b830 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.753+0000 7f8e67fff640 1 -- 172.21.15.7:0/2623805126 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7001e070 con 0x55b13ccb1d20 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.753+0000 7f8e67fff640 1 -- 172.21.15.7:0/2623805126 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e700035f0 con 0x55b13ccb1d20 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.753+0000 7f8e67fff640 1 -- 172.21.15.7:0/2623805126 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7000fb90 con 0x55b13ccb1d20 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.754+0000 7f8e8761c740 1 -- 172.21.15.7:0/2623805126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 msgr2=0x55b13ccabca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.754+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2623805126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccabca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009da0 tx=0x7f8e7001f000 comp rx=0 tx=0).stop 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.754+0000 7f8e8761c740 1 -- 172.21.15.7:0/2623805126 shutdown_connections 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.754+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2623805126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb5870 0x55b13ccac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.754+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2623805126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb2710 0x55b13cccbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.754+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2623805126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccabca0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.754+0000 7f8e8761c740 1 -- 172.21.15.7:0/2623805126 >> 172.21.15.7:0/2623805126 conn(0x55b13ccad310 msgr2=0x55b13ccad6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.755+0000 7f8e8761c740 1 -- 172.21.15.7:0/2623805126 shutdown_connections 2024-03-26T08:47:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.755+0000 7f8e8761c740 1 -- 172.21.15.7:0/2623805126 wait complete. 2024-03-26T08:47:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.755+0000 7f8e8761c740 1 -- start start 2024-03-26T08:47:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.755+0000 7f8e8761c740 1 Processor -- start 2024-03-26T08:47:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.755+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccc8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.755+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccc9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.755+0000 7f8e8761c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 0x55b13cd29900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e7e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccc8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccc9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccc9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41594/0 (socket says 172.21.15.7:41594) 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e7e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccc8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39582/0 (socket says 172.21.15.7:39582) 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e7ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 0x55b13cd29900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e7ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 0x55b13cd29900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38118/0 (socket says 172.21.15.7:38118) 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e8761c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801ad4f0 con 0x55b13ccb5870 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801ae810 con 0x55b13ccb2710 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e77fff640 1 -- 172.21.15.7:0/2307638298 learned_addr learned my addr 172.21.15.7:0/2307638298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801adb50 con 0x55b13ccb1d20 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2307638298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 msgr2=0x55b13ccc9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccc9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2307638298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 msgr2=0x55b13cd29900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 0x55b13cd29900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e7e59c640 1 -- 172.21.15.7:0/2307638298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b13cd29e40 con 0x55b13ccb1d20 2024-03-26T08:47:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.756+0000 7f8e8761c740 10 client.0 _ll_register_callbacks cb 0x7f8e7001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.757+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e700035f0 con 0x55b13ccb1d20 2024-03-26T08:47:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.757+0000 7f8e7e59c640 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccc8f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8e700198e0 tx=0x7f8e70019b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9153fd16fccc4e43 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.757+0000 7f8e65ffb640 10 client.26620 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.757+0000 7f8e65ffb640 1 -- 172.21.15.7:0/2307638298 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70044050 con 0x55b13ccb1d20 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.757+0000 7f8e65ffb640 1 -- 172.21.15.7:0/2307638298 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8e7000fb10 con 0x55b13ccb1d20 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.757+0000 7f8e65ffb640 1 -- 172.21.15.7:0/2307638298 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e7003a070 con 0x55b13ccb1d20 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.757+0000 7f8e8761c740 10 client.26620 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.757+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55b13cd2afe0 con 0x55b13ccb1d20 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e65ffb640 1 -- 172.21.15.7:0/2307638298 <== mon.1 v2:172.21.15.44:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8e7002b040 con 0x55b13ccb1d20 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e8761c740 10 client.26620 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e8761c740 10 client.26620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e8761c740 20 client.26620 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b13cd2b7d0 0x55b13cd2dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e8761c740 4 client.26620 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b13cc1ec60 con 0x55b13cd2b7d0 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e77fff640 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b13cd2b7d0 0x55b13cd2dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e65ffb640 10 client.26620 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e77fff640 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b13cd2b7d0 0x55b13cd2dbc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2172cef76b18de01 server_cookie=cb81aa9aadcbc165 in_seq=0 out_seq=0 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e65ffb640 1 -- 172.21.15.7:0/2307638298 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b13cc1ec60 con 0x55b13cd2b7d0 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e65ffb640 10 client.26620 handle_command_reply: tid=0 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.758+0000 7f8e8761c740 1 client.26620 shutdown 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 msgr2=0x55b13ccc8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccc8f30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8e700198e0 tx=0x7f8e70019b90 comp rx=0 tx=0).stop 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 shutdown_connections 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b13cd2b7d0 0x55b13cd2dbc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b13ccb5870 0x55b13cd29900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b13ccb2710 0x55b13ccc9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 --2- 172.21.15.7:0/2307638298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b13ccb1d20 0x55b13ccc8f30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e65ffb640 0 client.26620 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 >> 172.21.15.7:0/2307638298 conn(0x55b13ccad310 msgr2=0x55b13ccb0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 shutdown_connections 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 1 -- 172.21.15.7:0/2307638298 wait complete. 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.759+0000 7f8e8761c740 20 client.26620 trim_cache size 0 max 16384 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:47:43.755 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:47:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 -- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8e6c0a2950 msgr2=0x7f8e6c0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8e6c0a2950 0x7f8e6c0a4e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009710 tx=0x7f8e68009c70 comp rx=0 tx=0).stop 2024-03-26T08:47:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 -- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 msgr2=0x7f8e801a75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 0x7f8e801a75a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8e78005e30 tx=0x7f8e78035d10 comp rx=0 tx=0).stop 2024-03-26T08:47:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 -- 172.21.15.7:0/1972957220 shutdown_connections 2024-03-26T08:47:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8e6c0a2950 0x7f8e6c0a4e10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8010eaa0 0x7f8e801a75a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e8010c0e0 0x7f8e801a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 --2- 172.21.15.7:0/1972957220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8e801097b0 0x7f8e8019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.760+0000 7f8e84953640 1 -- 172.21.15.7:0/1972957220 >> 172.21.15.7:0/1972957220 conn(0x7f8e8006e880 msgr2=0x7f8e8010ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.761+0000 7f8e84953640 1 -- 172.21.15.7:0/1972957220 shutdown_connections 2024-03-26T08:47:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.761+0000 7f8e84953640 1 -- 172.21.15.7:0/1972957220 wait complete. 2024-03-26T08:47:43.768 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:47:43.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:47:43.869 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:43 smithi044 ceph-mon[36461]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 MiB/s wr, 1.64k op/s 2024-03-26T08:47:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.924+0000 7fc3108b1640 1 -- 172.21.15.7:0/3761883404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304099170 msgr2=0x7fc3040995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.924+0000 7fc3108b1640 1 --2- 172.21.15.7:0/3761883404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304099170 0x7fc3040995f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009900 tx=0x7fc2fc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:47:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.924+0000 7fc3108b1640 1 -- 172.21.15.7:0/3761883404 shutdown_connections 2024-03-26T08:47:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.924+0000 7fc3108b1640 1 --2- 172.21.15.7:0/3761883404 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc304099b30 0x7fc3040a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.924+0000 7fc3108b1640 1 --2- 172.21.15.7:0/3761883404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304099170 0x7fc3040995f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.924+0000 7fc3108b1640 1 --2- 172.21.15.7:0/3761883404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc304097f80 0x7fc3040983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.924+0000 7fc3108b1640 1 -- 172.21.15.7:0/3761883404 >> 172.21.15.7:0/3761883404 conn(0x7fc304093530 msgr2=0x7fc304095950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.924+0000 7fc3108b1640 1 -- 172.21.15.7:0/3761883404 shutdown_connections 2024-03-26T08:47:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.924+0000 7fc3108b1640 1 -- 172.21.15.7:0/3761883404 wait complete. 2024-03-26T08:47:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.925+0000 7fc3108b1640 1 Processor -- start 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.925+0000 7fc3108b1640 1 -- start start 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.925+0000 7fc3108b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc304097f80 0x7fc30412e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc3108b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc304099170 0x7fc30412e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc3108b1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304099b30 0x7fc304135840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc3108b1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040a3c10 con 0x7fc304099b30 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc3108b1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040a3a90 con 0x7fc304097f80 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc3108b1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040a3d90 con 0x7fc304099170 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc304097f80 0x7fc30412e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc304099170 0x7fc30412e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc304099170 0x7fc30412e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39598/0 (socket says 172.21.15.7:39598) 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30affd640 1 -- 172.21.15.7:0/759413362 learned_addr learned my addr 172.21.15.7:0/759413362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc304097f80 0x7fc30412e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41616/0 (socket says 172.21.15.7:41616) 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30bfff640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304099b30 0x7fc304135840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30b7fe640 1 -- 172.21.15.7:0/759413362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc304099170 msgr2=0x7fc30412e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30b7fe640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc304099170 0x7fc30412e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30b7fe640 1 -- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304099b30 msgr2=0x7fc304135840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30b7fe640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304099b30 0x7fc304135840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30b7fe640 1 -- 172.21.15.7:0/759413362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc304135eb0 con 0x7fc304097f80 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30affd640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc304099170 0x7fc30412e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc30b7fe640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc304097f80 0x7fc30412e290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc30000e9c0 tx=0x7fc30000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.926+0000 7fc308ff9640 1 -- 172.21.15.7:0/759413362 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000ccc0 con 0x7fc304097f80 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.927+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3041361a0 con 0x7fc304097f80 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.927+0000 7fc308ff9640 1 -- 172.21.15.7:0/759413362 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3000035f0 con 0x7fc304097f80 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.927+0000 7fc308ff9640 1 -- 172.21.15.7:0/759413362 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30000ce60 con 0x7fc304097f80 2024-03-26T08:47:43.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.927+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc304136710 con 0x7fc304097f80 2024-03-26T08:47:43.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.928+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3041369d0 con 0x7fc304097f80 2024-03-26T08:47:43.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.930+0000 7fc308ff9640 1 -- 172.21.15.7:0/759413362 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc3000037a0 con 0x7fc304097f80 2024-03-26T08:47:43.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.930+0000 7fc308ff9640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2f40a2880 0x7fc2f40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:43.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.930+0000 7fc308ff9640 1 -- 172.21.15.7:0/759413362 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc3000ceda0 con 0x7fc304097f80 2024-03-26T08:47:43.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.930+0000 7fc30affd640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2f40a2880 0x7fc2f40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:43.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.931+0000 7fc30affd640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2f40a2880 0x7fc2f40a4d40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc000c00 tx=0x7fc2fc02c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:43.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:43.934+0000 7fc308ff9640 1 -- 172.21.15.7:0/759413362 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc30009be40 con 0x7fc304097f80 2024-03-26T08:47:43.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:43 smithi016 ceph-mon[35851]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 MiB/s wr, 1.64k op/s 2024-03-26T08:47:44.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:43 smithi007 ceph-mon[32462]: pgmap v519: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 47 MiB/s wr, 1.64k op/s 2024-03-26T08:47:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.274+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc304136ba0 con 0x7fc304097f80 2024-03-26T08:47:44.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.275+0000 7fc308ff9640 1 -- 172.21.15.7:0/759413362 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fc30009b740 con 0x7fc304097f80 2024-03-26T08:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:47:44.268 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:47:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2f40a2880 msgr2=0x7fc2f40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2f40a2880 0x7fc2f40a4d40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc000c00 tx=0x7fc2fc02c880 comp rx=0 tx=0).stop 2024-03-26T08:47:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc304097f80 msgr2=0x7fc30412e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc304097f80 0x7fc30412e290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc30000e9c0 tx=0x7fc30000ee80 comp rx=0 tx=0).stop 2024-03-26T08:47:44.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 shutdown_connections 2024-03-26T08:47:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2f40a2880 0x7fc2f40a4d40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc304099b30 0x7fc304135840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc304099170 0x7fc30412e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 --2- 172.21.15.7:0/759413362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc304097f80 0x7fc30412e290 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.277+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 >> 172.21.15.7:0/759413362 conn(0x7fc304093530 msgr2=0x7fc30409c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.278+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 shutdown_connections 2024-03-26T08:47:44.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.278+0000 7fc3108b1640 1 -- 172.21.15.7:0/759413362 wait complete. 2024-03-26T08:47:44.286 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:47:44.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:47:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.440+0000 7fb16ce75640 1 -- 172.21.15.7:0/580735315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb168074100 msgr2=0x7fb168074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.440+0000 7fb16ce75640 1 --2- 172.21.15.7:0/580735315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb168074100 0x7fb168074580 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb150009cd0 tx=0x7fb15002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.440+0000 7fb16ce75640 1 -- 172.21.15.7:0/580735315 shutdown_connections 2024-03-26T08:47:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.440+0000 7fb16ce75640 1 --2- 172.21.15.7:0/580735315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb16810dca0 0x7fb168110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.440+0000 7fb16ce75640 1 --2- 172.21.15.7:0/580735315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb168074100 0x7fb168074580 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.440+0000 7fb16ce75640 1 --2- 172.21.15.7:0/580735315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1680756e0 0x7fb168073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.440+0000 7fb16ce75640 1 -- 172.21.15.7:0/580735315 >> 172.21.15.7:0/580735315 conn(0x7fb1680fcc60 msgr2=0x7fb1680ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.441+0000 7fb16ce75640 1 -- 172.21.15.7:0/580735315 shutdown_connections 2024-03-26T08:47:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.442+0000 7fb16ce75640 1 -- 172.21.15.7:0/580735315 wait complete. 2024-03-26T08:47:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16ce75640 1 Processor -- start 2024-03-26T08:47:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16ce75640 1 -- start start 2024-03-26T08:47:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16ce75640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb168074100 0x7fb16819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16ce75640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1680756e0 0x7fb16819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16ce75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16810dca0 0x7fb1681a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16ce75640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168113940 con 0x7fb1680756e0 2024-03-26T08:47:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16ce75640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681137c0 con 0x7fb16810dca0 2024-03-26T08:47:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16ce75640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168113ac0 con 0x7fb168074100 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb165d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1680756e0 0x7fb16819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb168074100 0x7fb16819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb168074100 0x7fb16819e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39610/0 (socket says 172.21.15.7:39610) 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb165d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1680756e0 0x7fb16819e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38160/0 (socket says 172.21.15.7:38160) 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb16659c640 1 -- 172.21.15.7:0/2099680512 learned_addr learned my addr 172.21.15.7:0/2099680512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.443+0000 7fb166d9d640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16810dca0 0x7fb1681a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb166d9d640 1 -- 172.21.15.7:0/2099680512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb168074100 msgr2=0x7fb16819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb166d9d640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb168074100 0x7fb16819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb166d9d640 1 -- 172.21.15.7:0/2099680512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1680756e0 msgr2=0x7fb16819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb166d9d640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1680756e0 0x7fb16819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb166d9d640 1 -- 172.21.15.7:0/2099680512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1681a5c30 con 0x7fb16810dca0 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb16659c640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb168074100 0x7fb16819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb166d9d640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16810dca0 0x7fb1681a55c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb16000e970 tx=0x7fb16000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb15f7fe640 1 -- 172.21.15.7:0/2099680512 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000ccc0 con 0x7fb16810dca0 2024-03-26T08:47:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb15f7fe640 1 -- 172.21.15.7:0/2099680512 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb160005020 con 0x7fb16810dca0 2024-03-26T08:47:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb16ce75640 1 -- 172.21.15.7:0/2099680512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1681a5f20 con 0x7fb16810dca0 2024-03-26T08:47:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb15f7fe640 1 -- 172.21.15.7:0/2099680512 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160004a00 con 0x7fb16810dca0 2024-03-26T08:47:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.444+0000 7fb16ce75640 1 -- 172.21.15.7:0/2099680512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1681a63e0 con 0x7fb16810dca0 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.446+0000 7fb15f7fe640 1 -- 172.21.15.7:0/2099680512 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb1600051d0 con 0x7fb16810dca0 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb16fa59740 1 Processor -- start 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb16fa59740 1 -- start start 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb15f7fe640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1480a2800 0x7fb1480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c8690 0x55be051c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051ce1f0 0x55be051d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb16fa59740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0518b610 con 0x55be051c8690 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb16fa59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0512e3c0 con 0x55be051ce1f0 2024-03-26T08:47:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb16fa59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be050d6d60 con 0x55be051c9080 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb15f7fe640 1 -- 172.21.15.7:0/2099680512 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb1600ce9a0 con 0x7fb16810dca0 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.447+0000 7fb165d9b640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1480a2800 0x7fb1480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb165d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb16659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c8690 0x55be051c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb165d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39616/0 (socket says 172.21.15.7:39616) 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb165d9b640 1 -- 172.21.15.7:0/896494219 learned_addr learned my addr 172.21.15.7:0/896494219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb166d9d640 1 --2- 172.21.15.7:0/896494219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051ce1f0 0x55be051d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb165d9b640 1 -- 172.21.15.7:0/896494219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051ce1f0 msgr2=0x55be051d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb165d9b640 1 --2- 172.21.15.7:0/896494219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051ce1f0 0x55be051d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb165d9b640 1 -- 172.21.15.7:0/896494219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c8690 msgr2=0x55be051c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb165d9b640 1 --2- 172.21.15.7:0/896494219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c8690 0x55be051c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb165d9b640 1 -- 172.21.15.7:0/896494219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be04e423f0 con 0x55be051c9080 2024-03-26T08:47:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb16659c640 1 --2- 172.21.15.7:0/896494219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c8690 0x55be051c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.448+0000 7fb165d9b640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1480a2800 0x7fb1480a4cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb150005e30 tx=0x7fb150009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.449+0000 7fb165d9b640 1 --2- 172.21.15.7:0/896494219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051cdb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1500324c0 tx=0x7fb150047000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c088e8cfb6bba2a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.449+0000 7fb15cff9640 1 -- 172.21.15.7:0/896494219 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150051070 con 0x55be051c9080 2024-03-26T08:47:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.449+0000 7fb15cff9640 1 -- 172.21.15.7:0/896494219 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb150032830 con 0x55be051c9080 2024-03-26T08:47:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.449+0000 7fb15cff9640 1 -- 172.21.15.7:0/896494219 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150039040 con 0x55be051c9080 2024-03-26T08:47:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.450+0000 7fb16fa59740 1 -- 172.21.15.7:0/896494219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 msgr2=0x55be051cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.450+0000 7fb16fa59740 1 --2- 172.21.15.7:0/896494219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051cdb80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb1500324c0 tx=0x7fb150047000 comp rx=0 tx=0).stop 2024-03-26T08:47:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.450+0000 7fb16fa59740 1 -- 172.21.15.7:0/896494219 shutdown_connections 2024-03-26T08:47:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.450+0000 7fb16fa59740 1 --2- 172.21.15.7:0/896494219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051ce1f0 0x55be051d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.450+0000 7fb16fa59740 1 --2- 172.21.15.7:0/896494219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051cdb80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.450+0000 7fb16fa59740 1 --2- 172.21.15.7:0/896494219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c8690 0x55be051c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.450+0000 7fb16fa59740 1 -- 172.21.15.7:0/896494219 >> 172.21.15.7:0/896494219 conn(0x55be051c3c00 msgr2=0x55be051c4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.450+0000 7fb16fa59740 1 -- 172.21.15.7:0/896494219 shutdown_connections 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.450+0000 7fb16fa59740 1 -- 172.21.15.7:0/896494219 wait complete. 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16fa59740 1 -- start start 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16fa59740 1 Processor -- start 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 0x55be051d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051da000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16fa59740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be050ea240 con 0x55be051ce1f0 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16fa59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be0512a460 con 0x55be051c8690 2024-03-26T08:47:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16fa59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be051d1140 con 0x55be051c9080 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16fa59740 10 client.0 _ll_register_callbacks cb 0x55be050e9c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb165d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb166d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051da000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb165d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39624/0 (socket says 172.21.15.7:39624) 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb165d9b640 1 -- 172.21.15.7:0/1417012257 learned_addr learned my addr 172.21.15.7:0/1417012257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.451+0000 7fb16659c640 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 0x55be051d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb165d9b640 1 -- 172.21.15.7:0/1417012257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 msgr2=0x55be051d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb165d9b640 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 0x55be051d2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb165d9b640 1 -- 172.21.15.7:0/1417012257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 msgr2=0x55be051da000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb165d9b640 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051da000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb165d9b640 1 -- 172.21.15.7:0/1417012257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be04f0c7d0 con 0x55be051c9080 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb166d9d640 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051da000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb16659c640 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 0x55be051d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb165d9b640 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051d2fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb150045040 tx=0x7fb15002c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7883d0b649ed134 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb155ffb640 1 -- 172.21.15.7:0/1417012257 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150051040 con 0x55be051c9080 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb155ffb640 1 -- 172.21.15.7:0/1417012257 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb1500045d0 con 0x55be051c9080 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.452+0000 7fb16fa59740 1 -- 172.21.15.7:0/1417012257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb150032830 con 0x55be051c9080 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb155ffb640 1 -- 172.21.15.7:0/1417012257 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb150039040 con 0x55be051c9080 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb16fa59740 10 client.26638 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb16fa59740 1 -- 172.21.15.7:0/1417012257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be051dd370 con 0x55be051c9080 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb155ffb640 1 -- 172.21.15.7:0/1417012257 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb150032830 con 0x55be051c9080 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb155ffb640 1 -- 172.21.15.7:0/1417012257 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb15003f070 con 0x55be051c9080 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb16fa59740 10 client.26638 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb16fa59740 10 client.26638 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb16fa59740 20 client.26638 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55be051de6c0 0x55be051e0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb16fa59740 4 client.26638 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.453+0000 7fb16fa59740 1 -- 172.21.15.7:0/1417012257 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be0513b900 con 0x55be051de6c0 2024-03-26T08:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.454+0000 7fb16659c640 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55be051de6c0 0x55be051e0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.454+0000 7fb155ffb640 10 client.26638 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.454+0000 7fb16659c640 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55be051de6c0 0x55be051e0ab0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d78a38b60905ac4 server_cookie=957159f7b3d04e86 in_seq=0 out_seq=0 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.454+0000 7fb155ffb640 1 -- 172.21.15.7:0/1417012257 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55be0513b900 con 0x55be051de6c0 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.454+0000 7fb155ffb640 10 client.26638 handle_command_reply: tid=0 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.454+0000 7fb16fa59740 1 client.26638 shutdown 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 -- 172.21.15.7:0/1417012257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 msgr2=0x55be051d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051d2fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb150045040 tx=0x7fb15002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 -- 172.21.15.7:0/1417012257 shutdown_connections 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55be051de6c0 0x55be051e0ab0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051da000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb155ffb640 0 client.26638 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c9080 0x55be051d2fc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1417012257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 0x55be051d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 -- 172.21.15.7:0/1417012257 >> 172.21.15.7:0/1417012257 conn(0x55be051c3c00 msgr2=0x55be051c2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 -- 172.21.15.7:0/1417012257 shutdown_connections 2024-03-26T08:47:44.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 1 -- 172.21.15.7:0/1417012257 wait complete. 2024-03-26T08:47:44.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.455+0000 7fb16fa59740 20 client.26638 trim_cache size 0 max 16384 2024-03-26T08:47:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.471+0000 7fb16fa59740 1 Processor -- start 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb16fa59740 1 -- start start 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 0x55be051d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c9080 0x55be051d9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051ce1f0 0x55be051d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb16fa59740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681b16e0 con 0x55be051c9080 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb16fa59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681af1e0 con 0x55be051c8690 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb16fa59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681ad260 con 0x55be051ce1f0 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb166d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051ce1f0 0x55be051d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb166d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051ce1f0 0x55be051d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39640/0 (socket says 172.21.15.7:39640) 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb166d9d640 1 -- 172.21.15.7:0/3063002565 learned_addr learned my addr 172.21.15.7:0/3063002565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb165d9b640 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c9080 0x55be051d9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb16659c640 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 0x55be051d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb166d9d640 1 -- 172.21.15.7:0/3063002565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 msgr2=0x55be051d2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb166d9d640 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 0x55be051d2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb166d9d640 1 -- 172.21.15.7:0/3063002565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c9080 msgr2=0x55be051d9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb166d9d640 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c9080 0x55be051d9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb166d9d640 1 -- 172.21.15.7:0/3063002565 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be051d3c00 con 0x55be051ce1f0 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.472+0000 7fb165d9b640 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c9080 0x55be051d9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.473+0000 7fb166d9d640 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051ce1f0 0x55be051d2fc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb160097750 tx=0x7fb160005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd00088da66905f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.473+0000 7fb157fff640 1 -- 172.21.15.7:0/3063002565 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16001c040 con 0x55be051ce1f0 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.473+0000 7fb157fff640 1 -- 172.21.15.7:0/3063002565 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1600a3ae0 con 0x55be051ce1f0 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.473+0000 7fb157fff640 1 -- 172.21.15.7:0/3063002565 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1600a3650 con 0x55be051ce1f0 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 -- 172.21.15.7:0/3063002565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051ce1f0 msgr2=0x55be051d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051ce1f0 0x55be051d2fc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb160097750 tx=0x7fb160005750 comp rx=0 tx=0).stop 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 -- 172.21.15.7:0/3063002565 shutdown_connections 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051ce1f0 0x55be051d2fc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051c9080 0x55be051d9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 --2- 172.21.15.7:0/3063002565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c8690 0x55be051d2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 -- 172.21.15.7:0/3063002565 >> 172.21.15.7:0/3063002565 conn(0x55be051c3c00 msgr2=0x55be051c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 -- 172.21.15.7:0/3063002565 shutdown_connections 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 -- 172.21.15.7:0/3063002565 wait complete. 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 -- start start 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 Processor -- start 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c8690 0x55be051d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c9080 0x55be051d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681ad860 con 0x55be051ce1f0 2024-03-26T08:47:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681aeb80 con 0x55be051c9080 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1681adec0 con 0x55be051c8690 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.474+0000 7fb16fa59740 10 client.0 _ll_register_callbacks cb 0x7fb160005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb166d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb166d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051d8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38194/0 (socket says 172.21.15.7:38194) 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb16659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c8690 0x55be051d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb166d9d640 1 -- 172.21.15.7:0/1598205093 learned_addr learned my addr 172.21.15.7:0/1598205093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb165d9b640 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c9080 0x55be051d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb16659c640 1 -- 172.21.15.7:0/1598205093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c9080 msgr2=0x55be051d7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb16659c640 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c9080 0x55be051d7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb16659c640 1 -- 172.21.15.7:0/1598205093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 msgr2=0x55be051d8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb16659c640 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051d8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.475+0000 7fb16659c640 1 -- 172.21.15.7:0/1598205093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be051d9420 con 0x55be051c8690 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.476+0000 7fb16659c640 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c8690 0x55be051d7930 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55be051e1080 tx=0x7fb158010db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec53de913b325af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.476+0000 7fb155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.476+0000 7fb155ffb640 1 -- 172.21.15.7:0/1598205093 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1580200f0 con 0x55be051c8690 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.476+0000 7fb16fa59740 1 -- 172.21.15.7:0/1598205093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1600a3ae0 con 0x55be051c8690 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.476+0000 7fb155ffb640 1 -- 172.21.15.7:0/1598205093 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb158020290 con 0x55be051c8690 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.476+0000 7fb155ffb640 1 -- 172.21.15.7:0/1598205093 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb15802f050 con 0x55be051c8690 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.476+0000 7fb16fa59740 10 client.26644 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.476+0000 7fb16fa59740 1 -- 172.21.15.7:0/1598205093 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be05239fa0 con 0x55be051c8690 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.477+0000 7fb155ffb640 1 -- 172.21.15.7:0/1598205093 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1600a3ae0 con 0x55be051c8690 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.477+0000 7fb155ffb640 1 -- 172.21.15.7:0/1598205093 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb1580284d0 con 0x55be051c8690 2024-03-26T08:47:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.477+0000 7fb16fa59740 10 client.26644 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.477+0000 7fb16fa59740 10 client.26644 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.477+0000 7fb16fa59740 20 client.26644 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.477+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55be0523a840 0x55be0523cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.477+0000 7fb16fa59740 4 client.26644 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.477+0000 7fb16fa59740 1 -- 172.21.15.7:0/1598205093 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be05135c50 con 0x55be0523a840 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.477+0000 7fb165d9b640 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55be0523a840 0x55be0523cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb155ffb640 10 client.26644 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb165d9b640 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55be0523a840 0x55be0523cce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec2b0138c6120837 server_cookie=a93e35844160bef2 in_seq=0 out_seq=0 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb155ffb640 1 -- 172.21.15.7:0/1598205093 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be05135c50 con 0x55be0523a840 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb155ffb640 10 client.26644 handle_command_reply: tid=0 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 client.26644 shutdown 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 -- 172.21.15.7:0/1598205093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c8690 msgr2=0x55be051d7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c8690 0x55be051d7930 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55be051e1080 tx=0x7fb158010db0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 -- 172.21.15.7:0/1598205093 shutdown_connections 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55be0523a840 0x55be0523cce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be051ce1f0 0x55be051d8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be051c9080 0x55be051d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 --2- 172.21.15.7:0/1598205093 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be051c8690 0x55be051d7930 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb155ffb640 0 client.26644 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 -- 172.21.15.7:0/1598205093 >> 172.21.15.7:0/1598205093 conn(0x55be051c3c00 msgr2=0x55be051d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 -- 172.21.15.7:0/1598205093 shutdown_connections 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 1 -- 172.21.15.7:0/1598205093 wait complete. 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.478+0000 7fb16fa59740 20 client.26644 trim_cache size 0 max 16384 2024-03-26T08:47:44.475 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:47:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 -- 172.21.15.7:0/2099680512 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1480a2800 msgr2=0x7fb1480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1480a2800 0x7fb1480a4cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb150005e30 tx=0x7fb150009880 comp rx=0 tx=0).stop 2024-03-26T08:47:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 -- 172.21.15.7:0/2099680512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16810dca0 msgr2=0x7fb1681a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16810dca0 0x7fb1681a55c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb16000e970 tx=0x7fb16000ee30 comp rx=0 tx=0).stop 2024-03-26T08:47:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 -- 172.21.15.7:0/2099680512 shutdown_connections 2024-03-26T08:47:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1480a2800 0x7fb1480a4cc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16810dca0 0x7fb1681a55c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1680756e0 0x7fb16819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 --2- 172.21.15.7:0/2099680512 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb168074100 0x7fb16819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 -- 172.21.15.7:0/2099680512 >> 172.21.15.7:0/2099680512 conn(0x7fb1680fcc60 msgr2=0x7fb168101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 -- 172.21.15.7:0/2099680512 shutdown_connections 2024-03-26T08:47:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.479+0000 7fb16ce75640 1 -- 172.21.15.7:0/2099680512 wait complete. 2024-03-26T08:47:44.485 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:47:44.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:47:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.662+0000 7fe0d0263640 1 -- 172.21.15.7:0/482325391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c81017b0 msgr2=0x7fe0c8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.662+0000 7fe0d0263640 1 --2- 172.21.15.7:0/482325391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c81017b0 0x7fe0c8101c30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009900 tx=0x7fe0b802bc30 comp rx=0 tx=0).stop 2024-03-26T08:47:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.662+0000 7fe0d0263640 1 -- 172.21.15.7:0/482325391 shutdown_connections 2024-03-26T08:47:44.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.662+0000 7fe0d0263640 1 --2- 172.21.15.7:0/482325391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c8102170 0x7fe0c8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.662+0000 7fe0d0263640 1 --2- 172.21.15.7:0/482325391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c81017b0 0x7fe0c8101c30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.662+0000 7fe0d0263640 1 --2- 172.21.15.7:0/482325391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c81005c0 0x7fe0c81009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.662+0000 7fe0d0263640 1 -- 172.21.15.7:0/482325391 >> 172.21.15.7:0/482325391 conn(0x7fe0c80fbb30 msgr2=0x7fe0c80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.662+0000 7fe0d0263640 1 -- 172.21.15.7:0/482325391 shutdown_connections 2024-03-26T08:47:44.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.662+0000 7fe0d0263640 1 -- 172.21.15.7:0/482325391 wait complete. 2024-03-26T08:47:44.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.663+0000 7fe0d0263640 1 Processor -- start 2024-03-26T08:47:44.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.663+0000 7fe0d0263640 1 -- start start 2024-03-26T08:47:44.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.663+0000 7fe0d0263640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c81005c0 0x7fe0c8069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.663+0000 7fe0d0263640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c81017b0 0x7fe0c819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.663+0000 7fe0d0263640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c8102170 0x7fe0c819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0d0263640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c810c220 con 0x7fe0c81005c0 2024-03-26T08:47:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0d0263640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c810c0a0 con 0x7fe0c81017b0 2024-03-26T08:47:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0d0263640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c810c3a0 con 0x7fe0c8102170 2024-03-26T08:47:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0cd7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c81017b0 0x7fe0c819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0cdfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c81005c0 0x7fe0c8069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0ce800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c8102170 0x7fe0c819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0cdfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c81005c0 0x7fe0c8069540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38212/0 (socket says 172.21.15.7:38212) 2024-03-26T08:47:44.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0cdfff640 1 -- 172.21.15.7:0/1484713867 learned_addr learned my addr 172.21.15.7:0/1484713867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0cd7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c81017b0 0x7fe0c819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41702/0 (socket says 172.21.15.7:41702) 2024-03-26T08:47:44.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.665+0000 7fe0cd7fe640 1 -- 172.21.15.7:0/1484713867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c8102170 msgr2=0x7fe0c819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.664+0000 7fe0ce800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c8102170 0x7fe0c819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39678/0 (socket says 172.21.15.7:39678) 2024-03-26T08:47:44.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.665+0000 7fe0cd7fe640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c8102170 0x7fe0c819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.665+0000 7fe0cd7fe640 1 -- 172.21.15.7:0/1484713867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c81005c0 msgr2=0x7fe0c8069540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.665+0000 7fe0cd7fe640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c81005c0 0x7fe0c8069540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.665+0000 7fe0cd7fe640 1 -- 172.21.15.7:0/1484713867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c819b3f0 con 0x7fe0c81017b0 2024-03-26T08:47:44.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.665+0000 7fe0ce800640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c8102170 0x7fe0c819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:47:44.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.665+0000 7fe0cd7fe640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c81017b0 0x7fe0c819a4d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009230 tx=0x7fe0b802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.665+0000 7fe0beffd640 1 -- 172.21.15.7:0/1484713867 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8041070 con 0x7fe0c81017b0 2024-03-26T08:47:44.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.666+0000 7fe0beffd640 1 -- 172.21.15.7:0/1484713867 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0b8004450 con 0x7fe0c81017b0 2024-03-26T08:47:44.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.666+0000 7fe0beffd640 1 -- 172.21.15.7:0/1484713867 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8003df0 con 0x7fe0c81017b0 2024-03-26T08:47:44.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.666+0000 7fe0d0263640 1 -- 172.21.15.7:0/1484713867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c819b680 con 0x7fe0c81017b0 2024-03-26T08:47:44.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.666+0000 7fe0d0263640 1 -- 172.21.15.7:0/1484713867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c819bb20 con 0x7fe0c81017b0 2024-03-26T08:47:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.668+0000 7fe0beffd640 1 -- 172.21.15.7:0/1484713867 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe0b8044070 con 0x7fe0c81017b0 2024-03-26T08:47:44.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.668+0000 7fe0d2e48740 1 Processor -- start 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.668+0000 7fe0d2e48740 1 -- start start 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341af1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341af6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341af9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a341ab4610 con 0x55a341af71f0 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a341a573c0 con 0x55a341af2080 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3419ffd60 con 0x55a341af1690 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0beffd640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0b40a2880 0x7fe0b40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0cd7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341af9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0ce800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341af1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0ce800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341af1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39684/0 (socket says 172.21.15.7:39684) 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0ce800640 1 -- 172.21.15.7:0/2724782965 learned_addr learned my addr 172.21.15.7:0/2724782965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0cd7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341af9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38218/0 (socket says 172.21.15.7:38218) 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0beffd640 1 -- 172.21.15.7:0/1484713867 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe0b80f0520 con 0x7fe0c81017b0 2024-03-26T08:47:44.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0ce800640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0b40a2880 0x7fe0b40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/2724782965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341af6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0cdfff640 1 -- 172.21.15.7:0/2724782965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 msgr2=0x55a341af1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/2724782965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341af1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0cdfff640 1 -- 172.21.15.7:0/2724782965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 msgr2=0x55a341af9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/2724782965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341af9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0cdfff640 1 -- 172.21.15.7:0/2724782965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a34176b3f0 con 0x55a341af2080 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.669+0000 7fe0cd7fe640 1 --2- 172.21.15.7:0/2724782965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341af9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.670+0000 7fe0ce800640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0b40a2880 0x7fe0b40a4d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009270 tx=0x7fe0c0009be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.670+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/2724782965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341af6b80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40058c0 tx=0x7fe0c4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d012f1cfd75e1f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.671+0000 7fe0b3fff640 1 -- 172.21.15.7:0/2724782965 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4004220 con 0x55a341af2080 2024-03-26T08:47:44.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.671+0000 7fe0b3fff640 1 -- 172.21.15.7:0/2724782965 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0c40043c0 con 0x55a341af2080 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.671+0000 7fe0b3fff640 1 -- 172.21.15.7:0/2724782965 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40035f0 con 0x55a341af2080 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.672+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2724782965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 msgr2=0x55a341af6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.672+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2724782965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341af6b80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40058c0 tx=0x7fe0c4011440 comp rx=0 tx=0).stop 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.672+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2724782965 shutdown_connections 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.672+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2724782965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341af9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.672+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2724782965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341af6b80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.672+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2724782965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341af1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.672+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2724782965 >> 172.21.15.7:0/2724782965 conn(0x55a341aecc00 msgr2=0x55a341aed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2724782965 shutdown_connections 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2724782965 wait complete. 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 -- start start 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 Processor -- start 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341afbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341afbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b03030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a341a13240 con 0x55a341af71f0 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a341a53460 con 0x55a341af2080 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.673+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a341afa1a0 con 0x55a341af1690 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0d2e48740 10 client.0 _ll_register_callbacks cb 0x55a341a12c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0ce800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b03030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cdfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341afbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cd7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341afbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0ce800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b03030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38232/0 (socket says 172.21.15.7:38232) 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cdfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341afbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39692/0 (socket says 172.21.15.7:39692) 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cd7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341afbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41728/0 (socket says 172.21.15.7:41728) 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cdfff640 1 -- 172.21.15.7:0/2342212728 learned_addr learned my addr 172.21.15.7:0/2342212728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cd7fe640 1 -- 172.21.15.7:0/2342212728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 msgr2=0x55a341afbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cd7fe640 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341afbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cd7fe640 1 -- 172.21.15.7:0/2342212728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 msgr2=0x55a341b03030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cd7fe640 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b03030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cd7fe640 1 -- 172.21.15.7:0/2342212728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3418357d0 con 0x55a341af2080 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341afbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0ce800640 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b03030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0cd7fe640 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341afbff0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80bcb80 tx=0x7fe0b808bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=932352841b71c8a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0b17fa640 1 -- 172.21.15.7:0/2342212728 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b804b040 con 0x55a341af2080 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2342212728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0c40043c0 con 0x55a341af2080 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.674+0000 7fe0b17fa640 1 -- 172.21.15.7:0/2342212728 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe0b80f00f0 con 0x55a341af2080 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0b17fa640 1 -- 172.21.15.7:0/2342212728 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0b80bcc50 con 0x55a341af2080 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0d2e48740 10 client.26621 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2342212728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a341b063a0 con 0x55a341af2080 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0b17fa640 1 -- 172.21.15.7:0/2342212728 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0c40043c0 con 0x55a341af2080 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0b17fa640 1 -- 172.21.15.7:0/2342212728 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe0b80c5190 con 0x55a341af2080 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0d2e48740 10 client.26621 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0d2e48740 10 client.26621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0d2e48740 20 client.26621 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a341b07a80 0x55a341b09eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0d2e48740 4 client.26621 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2342212728 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a341a64900 con 0x55a341b07a80 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.675+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a341b07a80 0x55a341b09eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.677+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a341b07a80 0x55a341b09eb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=763229195589af5b server_cookie=e7704628da5ec0bd in_seq=0 out_seq=0 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.677+0000 7fe0b17fa640 10 client.26621 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0b17fa640 1 -- 172.21.15.7:0/2342212728 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a341a64900 con 0x55a341b07a80 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0b17fa640 10 client.26621 handle_command_reply: tid=0 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0d2e48740 1 client.26621 shutdown 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2342212728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 msgr2=0x55a341afbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341afbff0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80bcb80 tx=0x7fe0b808bce0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2342212728 shutdown_connections 2024-03-26T08:47:44.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b03030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341afbff0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a341b07a80 0x55a341b09eb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0b17fa640 0 client.26621 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:47:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/2342212728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af1690 0x55a341afbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.678+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2342212728 >> 172.21.15.7:0/2342212728 conn(0x55a341aecc00 msgr2=0x55a341aebe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.679+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2342212728 shutdown_connections 2024-03-26T08:47:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.679+0000 7fe0d2e48740 1 -- 172.21.15.7:0/2342212728 wait complete. 2024-03-26T08:47:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.679+0000 7fe0d2e48740 20 client.26621 trim_cache size 0 max 16384 2024-03-26T08:47:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0d2e48740 1 Processor -- start 2024-03-26T08:47:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0d2e48740 1 -- start start 2024-03-26T08:47:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af1690 0x55a341aeb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341b02ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af71f0 0x55a341aebad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c81b9df0 con 0x55a341af1690 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c81a48c0 con 0x55a341af2080 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c81a2940 con 0x55a341af71f0 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0cdfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af1690 0x55a341aeb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0cdfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af1690 0x55a341aeb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38242/0 (socket says 172.21.15.7:38242) 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0ce800640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af71f0 0x55a341aebad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0cdfff640 1 -- 172.21.15.7:0/3078835511 learned_addr learned my addr 172.21.15.7:0/3078835511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0cd7fe640 1 --2- 172.21.15.7:0/3078835511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341b02ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0cdfff640 1 -- 172.21.15.7:0/3078835511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af71f0 msgr2=0x55a341aebad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/3078835511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af71f0 0x55a341aebad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0cdfff640 1 -- 172.21.15.7:0/3078835511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 msgr2=0x55a341b02ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/3078835511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341b02ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.699+0000 7fe0cdfff640 1 -- 172.21.15.7:0/3078835511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a341af2460 con 0x55a341af1690 2024-03-26T08:47:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.700+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/3078835511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af1690 0x55a341aeb590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009bd0 tx=0x7fe0c4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebb4336d58a59ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.701+0000 7fe0b37fe640 1 -- 172.21.15.7:0/3078835511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4031020 con 0x55a341af1690 2024-03-26T08:47:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.701+0000 7fe0b37fe640 1 -- 172.21.15.7:0/3078835511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0c402d260 con 0x55a341af1690 2024-03-26T08:47:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.701+0000 7fe0b37fe640 1 -- 172.21.15.7:0/3078835511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c401d040 con 0x55a341af1690 2024-03-26T08:47:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 -- 172.21.15.7:0/3078835511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af1690 msgr2=0x55a341aeb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/3078835511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af1690 0x55a341aeb590 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009bd0 tx=0x7fe0c4004570 comp rx=0 tx=0).stop 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 -- 172.21.15.7:0/3078835511 shutdown_connections 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/3078835511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af71f0 0x55a341aebad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/3078835511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af2080 0x55a341b02ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/3078835511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af1690 0x55a341aeb590 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 -- 172.21.15.7:0/3078835511 >> 172.21.15.7:0/3078835511 conn(0x55a341aecc00 msgr2=0x55a341af3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 -- 172.21.15.7:0/3078835511 shutdown_connections 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 -- 172.21.15.7:0/3078835511 wait complete. 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.702+0000 7fe0d2e48740 1 -- start start 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0d2e48740 1 Processor -- start 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af1690 0x55a341b005e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af2080 0x55a341b00b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0d2e48740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b01b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c81a2f40 con 0x55a341af71f0 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c81a4260 con 0x55a341af1690 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0d2e48740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c81a35a0 con 0x55a341af2080 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0d2e48740 10 client.0 _ll_register_callbacks cb 0x7fe0b80b9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0ce800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b01b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0cdfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af1690 0x55a341b005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0ce800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b01b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38252/0 (socket says 172.21.15.7:38252) 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0cd7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af2080 0x55a341b00b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0ce800640 1 -- 172.21.15.7:0/4167374416 learned_addr learned my addr 172.21.15.7:0/4167374416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0ce800640 1 -- 172.21.15.7:0/4167374416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af2080 msgr2=0x55a341b00b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0ce800640 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af2080 0x55a341b00b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0ce800640 1 -- 172.21.15.7:0/4167374416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af1690 msgr2=0x55a341b005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0ce800640 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af1690 0x55a341b005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0ce800640 1 -- 172.21.15.7:0/4167374416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a341b02120 con 0x55a341af71f0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.703+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af1690 0x55a341b005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.704+0000 7fe0ce800640 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b01b30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0014f30 tx=0x7fe0c001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fec57be6b54912 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.704+0000 7fe0b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.704+0000 7fe0b17fa640 1 -- 172.21.15.7:0/4167374416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0020800 con 0x55a341af71f0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.704+0000 7fe0b17fa640 1 -- 172.21.15.7:0/4167374416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe0c00209a0 con 0x55a341af71f0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.704+0000 7fe0d2e48740 1 -- 172.21.15.7:0/4167374416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0c402d260 con 0x55a341af71f0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.704+0000 7fe0b17fa640 1 -- 172.21.15.7:0/4167374416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0c0020ba0 con 0x55a341af71f0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.704+0000 7fe0d2e48740 10 client.16797 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.704+0000 7fe0d2e48740 1 -- 172.21.15.7:0/4167374416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a341b63340 con 0x55a341af71f0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0b17fa640 1 -- 172.21.15.7:0/4167374416 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0c402d260 con 0x55a341af71f0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0b17fa640 1 -- 172.21.15.7:0/4167374416 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe0c0033c90 con 0x55a341af71f0 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0d2e48740 10 client.16797 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0d2e48740 10 client.16797 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0d2e48740 20 client.16797 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a341b63d40 0x55a341b66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0d2e48740 4 client.16797 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0d2e48740 1 -- 172.21.15.7:0/4167374416 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a341a5ec50 con 0x55a341b63d40 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a341b63d40 0x55a341b66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0b17fa640 10 client.16797 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0cdfff640 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a341b63d40 0x55a341b66160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f3ecbcbaa70aeac server_cookie=eb4a0f66694daa9d in_seq=0 out_seq=0 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0b17fa640 1 -- 172.21.15.7:0/4167374416 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a341a5ec50 con 0x55a341b63d40 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0b17fa640 10 client.16797 handle_command_reply: tid=0 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.705+0000 7fe0d2e48740 1 client.16797 shutdown 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 -- 172.21.15.7:0/4167374416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 msgr2=0x55a341b01b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b01b30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0014f30 tx=0x7fe0c001b000 comp rx=0 tx=0).stop 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 -- 172.21.15.7:0/4167374416 shutdown_connections 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a341af71f0 0x55a341b01b30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a341b63d40 0x55a341b66160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0b17fa640 0 client.16797 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:47:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a341af2080 0x55a341b00b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 --2- 172.21.15.7:0/4167374416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a341af1690 0x55a341b005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 -- 172.21.15.7:0/4167374416 >> 172.21.15.7:0/4167374416 conn(0x55a341aecc00 msgr2=0x55a341aefe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 -- 172.21.15.7:0/4167374416 shutdown_connections 2024-03-26T08:47:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 1 -- 172.21.15.7:0/4167374416 wait complete. 2024-03-26T08:47:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d2e48740 20 client.16797 trim_cache size 0 max 16384 2024-03-26T08:47:44.703 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 -- 172.21.15.7:0/1484713867 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0b40a2880 msgr2=0x7fe0b40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0b40a2880 0x7fe0b40a4d40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009270 tx=0x7fe0c0009be0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 -- 172.21.15.7:0/1484713867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c81017b0 msgr2=0x7fe0c819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c81017b0 0x7fe0c819a4d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009230 tx=0x7fe0b802c650 comp rx=0 tx=0).stop 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 -- 172.21.15.7:0/1484713867 shutdown_connections 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0b40a2880 0x7fe0b40a4d40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0c8102170 0x7fe0c819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c81017b0 0x7fe0c819a4d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 --2- 172.21.15.7:0/1484713867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0c81005c0 0x7fe0c8069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.706+0000 7fe0d0263640 1 -- 172.21.15.7:0/1484713867 >> 172.21.15.7:0/1484713867 conn(0x7fe0c80fbb30 msgr2=0x7fe0c8104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.707+0000 7fe0d0263640 1 -- 172.21.15.7:0/1484713867 shutdown_connections 2024-03-26T08:47:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.707+0000 7fe0d0263640 1 -- 172.21.15.7:0/1484713867 wait complete. 2024-03-26T08:47:44.715 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:47:44.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:47:44.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.877+0000 7f9e6691b640 1 -- 172.21.15.7:0/3055305765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e601017b0 msgr2=0x7f9e60101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.877+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3055305765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e601017b0 0x7f9e60101c30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009900 tx=0x7f9e5402bc40 comp rx=0 tx=0).stop 2024-03-26T08:47:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.878+0000 7f9e6691b640 1 -- 172.21.15.7:0/3055305765 shutdown_connections 2024-03-26T08:47:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.878+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3055305765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e60102170 0x7f9e60108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.878+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3055305765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e601017b0 0x7f9e60101c30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.878+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3055305765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e601005c0 0x7f9e601009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.878+0000 7f9e6691b640 1 -- 172.21.15.7:0/3055305765 >> 172.21.15.7:0/3055305765 conn(0x7f9e600fbb30 msgr2=0x7f9e600fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.878+0000 7f9e6691b640 1 -- 172.21.15.7:0/3055305765 shutdown_connections 2024-03-26T08:47:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.879+0000 7f9e6691b640 1 -- 172.21.15.7:0/3055305765 wait complete. 2024-03-26T08:47:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.879+0000 7f9e6691b640 1 Processor -- start 2024-03-26T08:47:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.879+0000 7f9e6691b640 1 -- start start 2024-03-26T08:47:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e6691b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e601005c0 0x7f9e601979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e6691b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e601017b0 0x7f9e60197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e6691b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60102170 0x7f9e6019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e6691b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6010c220 con 0x7f9e60102170 2024-03-26T08:47:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e6691b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6010c0a0 con 0x7f9e601017b0 2024-03-26T08:47:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e6691b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6010c3a0 con 0x7f9e601005c0 2024-03-26T08:47:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e64eb8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60102170 0x7f9e6019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e64eb8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60102170 0x7f9e6019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38280/0 (socket says 172.21.15.7:38280) 2024-03-26T08:47:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e64eb8640 1 -- 172.21.15.7:0/3555178356 learned_addr learned my addr 172.21.15.7:0/3555178356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e601017b0 0x7f9e60197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e601005c0 0x7f9e601979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e64eb8640 1 -- 172.21.15.7:0/3555178356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e601005c0 msgr2=0x7f9e601979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e64eb8640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e601005c0 0x7f9e601979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e64eb8640 1 -- 172.21.15.7:0/3555178356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e601017b0 msgr2=0x7f9e60197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e64eb8640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e601017b0 0x7f9e60197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.880+0000 7f9e64eb8640 1 -- 172.21.15.7:0/3555178356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6019f5d0 con 0x7f9e60102170 2024-03-26T08:47:44.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.881+0000 7f9e64eb8640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60102170 0x7f9e6019ef60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800eb10 tx=0x7f9e5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.881+0000 7f9e5d7fa640 1 -- 172.21.15.7:0/3555178356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004b20 con 0x7f9e60102170 2024-03-26T08:47:44.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.881+0000 7f9e5d7fa640 1 -- 172.21.15.7:0/3555178356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e58004cc0 con 0x7f9e60102170 2024-03-26T08:47:44.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.881+0000 7f9e5d7fa640 1 -- 172.21.15.7:0/3555178356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58005650 con 0x7f9e60102170 2024-03-26T08:47:44.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.881+0000 7f9e6691b640 1 -- 172.21.15.7:0/3555178356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e60102a30 con 0x7f9e60102170 2024-03-26T08:47:44.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.881+0000 7f9e6691b640 1 -- 172.21.15.7:0/3555178356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e60102f00 con 0x7f9e60102170 2024-03-26T08:47:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.884+0000 7f9e5d7fa640 1 -- 172.21.15.7:0/3555178356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9e58005810 con 0x7f9e60102170 2024-03-26T08:47:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e695e9740 1 Processor -- start 2024-03-26T08:47:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e695e9740 1 -- start start 2024-03-26T08:47:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 0x561ca97adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 0x561ca97b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e695e9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca9770610 con 0x561ca97b31f0 2024-03-26T08:47:44.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e695e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca97133c0 con 0x561ca97ad690 2024-03-26T08:47:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e695e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca96bbd60 con 0x561ca97ae080 2024-03-26T08:47:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e5d7fa640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e480a2900 0x7f9e480a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39738/0 (socket says 172.21.15.7:39738) 2024-03-26T08:47:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e5f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 0x561ca97b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e5ffff640 1 -- 172.21.15.7:0/4089933600 learned_addr learned my addr 172.21.15.7:0/4089933600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.885+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/4089933600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 0x561ca97adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e64eb8640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e480a2900 0x7f9e480a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e5d7fa640 1 -- 172.21.15.7:0/3555178356 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9e580d30e0 con 0x7f9e60102170 2024-03-26T08:47:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e5ffff640 1 -- 172.21.15.7:0/4089933600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 msgr2=0x561ca97adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/4089933600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 0x561ca97adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e5ffff640 1 -- 172.21.15.7:0/4089933600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 msgr2=0x561ca97b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/4089933600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 0x561ca97b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e5ffff640 1 -- 172.21.15.7:0/4089933600 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ca94273f0 con 0x561ca97ae080 2024-03-26T08:47:44.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/4089933600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 0x561ca97adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e64eb8640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e480a2900 0x7f9e480a4dc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9e5809f9d0 tx=0x7f9e58062040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/4089933600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b2b80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000bc90 tx=0x7f9e5000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1ea3397861cd81a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e467fc640 1 -- 172.21.15.7:0/4089933600 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50015070 con 0x561ca97ae080 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e467fc640 1 -- 172.21.15.7:0/4089933600 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e500035f0 con 0x561ca97ae080 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.886+0000 7f9e467fc640 1 -- 172.21.15.7:0/4089933600 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500116a0 con 0x561ca97ae080 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.887+0000 7f9e695e9740 1 -- 172.21.15.7:0/4089933600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 msgr2=0x561ca97b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.887+0000 7f9e695e9740 1 --2- 172.21.15.7:0/4089933600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b2b80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000bc90 tx=0x7f9e5000f730 comp rx=0 tx=0).stop 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.887+0000 7f9e695e9740 1 -- 172.21.15.7:0/4089933600 shutdown_connections 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.887+0000 7f9e695e9740 1 --2- 172.21.15.7:0/4089933600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 0x561ca97b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.887+0000 7f9e695e9740 1 --2- 172.21.15.7:0/4089933600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b2b80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.887+0000 7f9e695e9740 1 --2- 172.21.15.7:0/4089933600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 0x561ca97adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.887+0000 7f9e695e9740 1 -- 172.21.15.7:0/4089933600 >> 172.21.15.7:0/4089933600 conn(0x561ca97a8c00 msgr2=0x561ca97a9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.887+0000 7f9e695e9740 1 -- 172.21.15.7:0/4089933600 shutdown_connections 2024-03-26T08:47:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.887+0000 7f9e695e9740 1 -- 172.21.15.7:0/4089933600 wait complete. 2024-03-26T08:47:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e695e9740 1 -- start start 2024-03-26T08:47:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e695e9740 1 Processor -- start 2024-03-26T08:47:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 0x561ca97b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 0x561ca97bef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e695e9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca96cf240 con 0x561ca97b31f0 2024-03-26T08:47:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e695e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca970f460 con 0x561ca97ad690 2024-03-26T08:47:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e695e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca97b61f0 con 0x561ca97ae080 2024-03-26T08:47:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e695e9740 10 client.0 _ll_register_callbacks cb 0x561ca96cec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e5f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 0x561ca97bef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.888+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39748/0 (socket says 172.21.15.7:39748) 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e5ffff640 1 -- 172.21.15.7:0/156941337 learned_addr learned my addr 172.21.15.7:0/156941337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 0x561ca97b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e5ffff640 1 -- 172.21.15.7:0/156941337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 msgr2=0x561ca97b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 0x561ca97b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e5ffff640 1 -- 172.21.15.7:0/156941337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 msgr2=0x561ca97bef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 0x561ca97bef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e5ffff640 1 -- 172.21.15.7:0/156941337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ca94f17d0 con 0x561ca97ae080 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b7f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000fc30 tx=0x7f9e5000b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6036af2c4b73d1a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/156941337 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50028060 con 0x561ca97ae080 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e695e9740 1 -- 172.21.15.7:0/156941337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e500035f0 con 0x561ca97ae080 2024-03-26T08:47:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/156941337 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9e500039b0 con 0x561ca97ae080 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/156941337 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e50015070 con 0x561ca97ae080 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e695e9740 10 client.26671 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.889+0000 7f9e695e9740 1 -- 172.21.15.7:0/156941337 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561ca97c2350 con 0x561ca97ae080 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.890+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/156941337 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e500035f0 con 0x561ca97ae080 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.890+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/156941337 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9e5002d070 con 0x561ca97ae080 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.890+0000 7f9e695e9740 10 client.26671 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.890+0000 7f9e695e9740 10 client.26671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.890+0000 7f9e695e9740 20 client.26671 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.890+0000 7f9e695e9740 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ca97c3750 0x561ca97c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.890+0000 7f9e695e9740 4 client.26671 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.890+0000 7f9e695e9740 1 -- 172.21.15.7:0/156941337 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ca9720900 con 0x561ca97c3750 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.891+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ca97c3750 0x561ca97c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.892+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ca97c3750 0x561ca97c5b40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693d3b52410fcfe5 server_cookie=c15691572f68def4 in_seq=0 out_seq=0 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.892+0000 7f9e3f7fe640 10 client.26671 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.892+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/156941337 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561ca9720900 con 0x561ca97c3750 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.892+0000 7f9e3f7fe640 10 client.26671 handle_command_reply: tid=0 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.892+0000 7f9e695e9740 1 client.26671 shutdown 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.892+0000 7f9e695e9740 1 -- 172.21.15.7:0/156941337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 msgr2=0x561ca97b7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.892+0000 7f9e695e9740 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b7f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000fc30 tx=0x7f9e5000b9d0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e695e9740 1 -- 172.21.15.7:0/156941337 shutdown_connections 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e695e9740 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ca97c3750 0x561ca97c5b40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e695e9740 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97b31f0 0x561ca97bef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e695e9740 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97b7f50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e695e9740 1 --2- 172.21.15.7:0/156941337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ad690 0x561ca97b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e3f7fe640 0 client.26671 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e695e9740 1 -- 172.21.15.7:0/156941337 >> 172.21.15.7:0/156941337 conn(0x561ca97a8c00 msgr2=0x561ca97a7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e695e9740 1 -- 172.21.15.7:0/156941337 shutdown_connections 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e695e9740 1 -- 172.21.15.7:0/156941337 wait complete. 2024-03-26T08:47:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.893+0000 7f9e695e9740 20 client.26671 trim_cache size 0 max 16384 2024-03-26T08:47:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e695e9740 1 Processor -- start 2024-03-26T08:47:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e695e9740 1 -- start start 2024-03-26T08:47:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ae080 0x561ca97bef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97b31f0 0x561ca97ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e695e9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601a8d20 con 0x561ca97ad690 2024-03-26T08:47:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e695e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601a6820 con 0x561ca97ae080 2024-03-26T08:47:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e695e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601a48a0 con 0x561ca97b31f0 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e5f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e5f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97a7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38314/0 (socket says 172.21.15.7:38314) 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e5f7fe640 1 -- 172.21.15.7:0/1619200454 learned_addr learned my addr 172.21.15.7:0/1619200454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ae080 0x561ca97bef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.910+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97b31f0 0x561ca97ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.911+0000 7f9e5f7fe640 1 -- 172.21.15.7:0/1619200454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97b31f0 msgr2=0x561ca97ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.911+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97b31f0 0x561ca97ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.911+0000 7f9e5f7fe640 1 -- 172.21.15.7:0/1619200454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ae080 msgr2=0x561ca97bef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.911+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ae080 0x561ca97bef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.911+0000 7f9e5f7fe640 1 -- 172.21.15.7:0/1619200454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ca97a8550 con 0x561ca97ad690 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.911+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ae080 0x561ca97bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.911+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97a7ca0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009900 tx=0x7f9e5402bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b46addf8d6c5ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e45ffb640 1 -- 172.21.15.7:0/1619200454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54054070 con 0x561ca97ad690 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e45ffb640 1 -- 172.21.15.7:0/1619200454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e54005530 con 0x561ca97ad690 2024-03-26T08:47:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e45ffb640 1 -- 172.21.15.7:0/1619200454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e54004b70 con 0x561ca97ad690 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e695e9740 1 -- 172.21.15.7:0/1619200454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 msgr2=0x561ca97a7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e695e9740 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97a7ca0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009900 tx=0x7f9e5402bc40 comp rx=0 tx=0).stop 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e695e9740 1 -- 172.21.15.7:0/1619200454 shutdown_connections 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e695e9740 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97b31f0 0x561ca97ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e695e9740 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97ae080 0x561ca97bef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e695e9740 1 --2- 172.21.15.7:0/1619200454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97a7ca0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.912+0000 7f9e695e9740 1 -- 172.21.15.7:0/1619200454 >> 172.21.15.7:0/1619200454 conn(0x561ca97a8c00 msgr2=0x561ca97a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.913+0000 7f9e695e9740 1 -- 172.21.15.7:0/1619200454 shutdown_connections 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.913+0000 7f9e695e9740 1 -- 172.21.15.7:0/1619200454 wait complete. 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.913+0000 7f9e695e9740 1 -- start start 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.913+0000 7f9e695e9740 1 Processor -- start 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e695e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97b31f0 0x561ca97bcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e695e9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601a4ea0 con 0x561ca97ad690 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e695e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601a61c0 con 0x561ca97b31f0 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e695e9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601a5500 con 0x561ca97ae080 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e695e9740 10 client.0 _ll_register_callbacks cb 0x7f9e5000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e5ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97bec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39776/0 (socket says 172.21.15.7:39776) 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.914+0000 7f9e5ffff640 1 -- 172.21.15.7:0/497794096 learned_addr learned my addr 172.21.15.7:0/497794096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e5ffff640 1 -- 172.21.15.7:0/497794096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97b31f0 msgr2=0x561ca97bcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97b31f0 0x561ca97bcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e5ffff640 1 -- 172.21.15.7:0/497794096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 msgr2=0x561ca97be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97be750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e5ffff640 1 -- 172.21.15.7:0/497794096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ca97b6330 con 0x561ca97ae080 2024-03-26T08:47:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:47:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e5ffff640 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97bec90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9e500057c0 tx=0x7f9e50024830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b10d12c0de6aa31 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/497794096 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50033030 con 0x561ca97ae080 2024-03-26T08:47:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/497794096 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9e50020040 con 0x561ca97ae080 2024-03-26T08:47:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.915+0000 7f9e695e9740 1 -- 172.21.15.7:0/497794096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e54005530 con 0x561ca97ae080 2024-03-26T08:47:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/497794096 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e50028070 con 0x561ca97ae080 2024-03-26T08:47:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e695e9740 10 client.26677 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e695e9740 1 -- 172.21.15.7:0/497794096 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561ca981f700 con 0x561ca97ae080 2024-03-26T08:47:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/497794096 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e54005530 con 0x561ca97ae080 2024-03-26T08:47:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/497794096 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9e5002d070 con 0x561ca97ae080 2024-03-26T08:47:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e695e9740 10 client.26677 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e695e9740 10 client.26677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e695e9740 20 client.26677 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e695e9740 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ca9820030 0x561ca9822450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e695e9740 4 client.26677 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:47:44.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e695e9740 1 -- 172.21.15.7:0/497794096 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ca97b7800 con 0x561ca9820030 2024-03-26T08:47:44.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.916+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ca9820030 0x561ca9822450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.917+0000 7f9e5f7fe640 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ca9820030 0x561ca9822450 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa3e27ca247eed7 server_cookie=ad90218eecd89f8f in_seq=0 out_seq=0 2024-03-26T08:47:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.917+0000 7f9e3f7fe640 10 client.26677 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.917+0000 7f9e3f7fe640 1 -- 172.21.15.7:0/497794096 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ca97b7800 con 0x561ca9820030 2024-03-26T08:47:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.917+0000 7f9e3f7fe640 10 client.26677 handle_command_reply: tid=0 2024-03-26T08:47:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.917+0000 7f9e695e9740 1 client.26677 shutdown 2024-03-26T08:47:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 -- 172.21.15.7:0/497794096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 msgr2=0x561ca97bec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97bec90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9e500057c0 tx=0x7f9e50024830 comp rx=0 tx=0).stop 2024-03-26T08:47:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 -- 172.21.15.7:0/497794096 shutdown_connections 2024-03-26T08:47:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ca9820030 0x561ca9822450 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ca97b31f0 0x561ca97bcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ca97ae080 0x561ca97bec90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e3f7fe640 0 client.26677 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 --2- 172.21.15.7:0/497794096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ca97ad690 0x561ca97be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 -- 172.21.15.7:0/497794096 >> 172.21.15.7:0/497794096 conn(0x561ca97a8c00 msgr2=0x561ca97a7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 -- 172.21.15.7:0/497794096 shutdown_connections 2024-03-26T08:47:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 1 -- 172.21.15.7:0/497794096 wait complete. 2024-03-26T08:47:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e695e9740 20 client.26677 trim_cache size 0 max 16384 2024-03-26T08:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:47:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 -- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e480a2900 msgr2=0x7f9e480a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e480a2900 0x7f9e480a4dc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9e5809f9d0 tx=0x7f9e58062040 comp rx=0 tx=0).stop 2024-03-26T08:47:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 -- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60102170 msgr2=0x7f9e6019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60102170 0x7f9e6019ef60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800eb10 tx=0x7f9e5800c2d0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 -- 172.21.15.7:0/3555178356 shutdown_connections 2024-03-26T08:47:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9e480a2900 0x7f9e480a4dc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60102170 0x7f9e6019ef60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e601017b0 0x7f9e60197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 --2- 172.21.15.7:0/3555178356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e601005c0 0x7f9e601979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.918+0000 7f9e6691b640 1 -- 172.21.15.7:0/3555178356 >> 172.21.15.7:0/3555178356 conn(0x7f9e600fbb30 msgr2=0x7f9e600fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.919+0000 7f9e6691b640 1 -- 172.21.15.7:0/3555178356 shutdown_connections 2024-03-26T08:47:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:44.919+0000 7f9e6691b640 1 -- 172.21.15.7:0/3555178356 wait complete. 2024-03-26T08:47:44.923 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:44 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/759413362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:47:44.993 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:44 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/759413362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:47:45.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:44 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/759413362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:47:45.894 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:45 smithi016 ceph-mon[35851]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 45 MiB/s wr, 1.57k op/s 2024-03-26T08:47:45.925 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:47:45.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:47:45.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:47:45.956 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:45 smithi007 ceph-mon[32462]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 45 MiB/s wr, 1.57k op/s 2024-03-26T08:47:46.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:45 smithi044 ceph-mon[36461]: pgmap v520: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 45 MiB/s wr, 1.57k op/s 2024-03-26T08:47:46.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 315.0 s, 2884.4 tps, lat 34.492 ms stddev 38.951 2024-03-26T08:47:46.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.089+0000 7fc4d3177640 1 -- 172.21.15.7:0/2773933748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc1005c0 msgr2=0x7fc4cc1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.089+0000 7fc4d3177640 1 --2- 172.21.15.7:0/2773933748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc1005c0 0x7fc4cc1009e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009900 tx=0x7fc4c002bc30 comp rx=0 tx=0).stop 2024-03-26T08:47:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.090+0000 7fc4d3177640 1 -- 172.21.15.7:0/2773933748 shutdown_connections 2024-03-26T08:47:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.090+0000 7fc4d3177640 1 --2- 172.21.15.7:0/2773933748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4cc102170 0x7fc4cc108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.090+0000 7fc4d3177640 1 --2- 172.21.15.7:0/2773933748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc1017b0 0x7fc4cc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.090+0000 7fc4d3177640 1 --2- 172.21.15.7:0/2773933748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc1005c0 0x7fc4cc1009e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.090+0000 7fc4d3177640 1 -- 172.21.15.7:0/2773933748 >> 172.21.15.7:0/2773933748 conn(0x7fc4cc0fbb30 msgr2=0x7fc4cc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.090+0000 7fc4d3177640 1 -- 172.21.15.7:0/2773933748 shutdown_connections 2024-03-26T08:47:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.090+0000 7fc4d3177640 1 -- 172.21.15.7:0/2773933748 wait complete. 2024-03-26T08:47:46.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.090+0000 7fc4d3177640 1 Processor -- start 2024-03-26T08:47:46.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.090+0000 7fc4d3177640 1 -- start start 2024-03-26T08:47:46.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d3177640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc1017b0 0x7fc4cc104550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d3177640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc102170 0x7fc4cc102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d3177640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4cc1030d0 0x7fc4cc103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d3177640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc10c230 con 0x7fc4cc102170 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d3177640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc10c0b0 con 0x7fc4cc1017b0 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d3177640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc10c3b0 con 0x7fc4cc1030d0 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc102170 0x7fc4cc102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d0f13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc1017b0 0x7fc4cc104550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d1714640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4cc1030d0 0x7fc4cc103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d0f13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc1017b0 0x7fc4cc104550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41788/0 (socket says 172.21.15.7:41788) 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc102170 0x7fc4cc102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38350/0 (socket says 172.21.15.7:38350) 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d1714640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4cc1030d0 0x7fc4cc103580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39798/0 (socket says 172.21.15.7:39798) 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d0f13640 1 -- 172.21.15.7:0/1871712774 learned_addr learned my addr 172.21.15.7:0/1871712774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4cbfff640 1 -- 172.21.15.7:0/1871712774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4cc1030d0 msgr2=0x7fc4cc103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4cbfff640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4cc1030d0 0x7fc4cc103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4cbfff640 1 -- 172.21.15.7:0/1871712774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc1017b0 msgr2=0x7fc4cc104550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4cbfff640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc1017b0 0x7fc4cc104550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4cbfff640 1 -- 172.21.15.7:0/1871712774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc1a1e30 con 0x7fc4cc102170 2024-03-26T08:47:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d1714640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4cc1030d0 0x7fc4cc103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc1017b0 0x7fc4cc104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.091+0000 7fc4cbfff640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc102170 0x7fc4cc102b90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9f0 tx=0x7fc4b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.092+0000 7fc4c9ffb640 1 -- 172.21.15.7:0/1871712774 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800ccc0 con 0x7fc4cc102170 2024-03-26T08:47:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.092+0000 7fc4d3177640 1 -- 172.21.15.7:0/1871712774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4cc1a2120 con 0x7fc4cc102170 2024-03-26T08:47:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.092+0000 7fc4d3177640 1 -- 172.21.15.7:0/1871712774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc1a26c0 con 0x7fc4cc102170 2024-03-26T08:47:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.092+0000 7fc4c9ffb640 1 -- 172.21.15.7:0/1871712774 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4b8010910 con 0x7fc4cc102170 2024-03-26T08:47:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.092+0000 7fc4c9ffb640 1 -- 172.21.15.7:0/1871712774 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8018960 con 0x7fc4cc102170 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.095+0000 7fc4d59fd740 1 Processor -- start 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.095+0000 7fc4d59fd740 1 -- start start 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c54d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 0x562823c59e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 0x562823c5c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d59fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562823b94c90 con 0x562823c5a4c0 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d59fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562823bba070 con 0x562823c55350 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d59fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562823b62c80 con 0x562823c54960 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4c9ffb640 1 -- 172.21.15.7:0/1871712774 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc4b8010ac0 con 0x7fc4cc102170 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d1714640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c54d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d1714640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c54d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39804/0 (socket says 172.21.15.7:39804) 2024-03-26T08:47:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d1714640 1 -- 172.21.15.7:0/3524946791 learned_addr learned my addr 172.21.15.7:0/3524946791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/3524946791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 0x562823c59e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4cbfff640 1 --2- 172.21.15.7:0/3524946791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 0x562823c5c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d1714640 1 -- 172.21.15.7:0/3524946791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 msgr2=0x562823c59e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d1714640 1 --2- 172.21.15.7:0/3524946791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 0x562823c59e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d1714640 1 -- 172.21.15.7:0/3524946791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 msgr2=0x562823c5c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d1714640 1 --2- 172.21.15.7:0/3524946791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 0x562823c5c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d1714640 1 -- 172.21.15.7:0/3524946791 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562823b5ad10 con 0x562823c54960 2024-03-26T08:47:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.096+0000 7fc4d1714640 1 --2- 172.21.15.7:0/3524946791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c54d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40117e0 tx=0x7fc4c4011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=859d5218a07e992e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.097+0000 7fc4b6ffd640 1 -- 172.21.15.7:0/3524946791 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4011310 con 0x562823c54960 2024-03-26T08:47:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.097+0000 7fc4b6ffd640 1 -- 172.21.15.7:0/3524946791 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4003e50 con 0x562823c54960 2024-03-26T08:47:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.097+0000 7fc4b6ffd640 1 -- 172.21.15.7:0/3524946791 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4004620 con 0x562823c54960 2024-03-26T08:47:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4c9ffb640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4bc0a2880 0x7fc4bc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3524946791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 msgr2=0x562823c54d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3524946791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c54d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc4c40117e0 tx=0x7fc4c4011ca0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4bc0a2880 0x7fc4bc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3524946791 shutdown_connections 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3524946791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 0x562823c5c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3524946791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 0x562823c59e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3524946791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c54d80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3524946791 >> 172.21.15.7:0/3524946791 conn(0x562823c1d1a0 msgr2=0x562823c502d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3524946791 shutdown_connections 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3524946791 wait complete. 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.099+0000 7fc4d59fd740 1 -- start start 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.098+0000 7fc4c9ffb640 1 -- 172.21.15.7:0/1871712774 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4b80cec80 con 0x7fc4cc102170 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.099+0000 7fc4d59fd740 1 Processor -- start 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.099+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4bc0a2880 0x7fc4bc0a4d40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005ab0 tx=0x7fc4c002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.099+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c5eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.099+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 0x562823c5f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.099+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 0x562823c66060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.099+0000 7fc4d59fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562823b760d0 con 0x562823c5a4c0 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.099+0000 7fc4d59fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562823bb6110 con 0x562823c55350 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.099+0000 7fc4d59fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562823c5d470 con 0x562823c54960 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d0f13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c5eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 0x562823c5f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d0f13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c5eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39810/0 (socket says 172.21.15.7:39810) 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d0f13640 1 -- 172.21.15.7:0/2577563463 learned_addr learned my addr 172.21.15.7:0/2577563463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d59fd740 10 client.0 _ll_register_callbacks cb 0x562823bb1e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d1714640 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 0x562823c66060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d0f13640 1 -- 172.21.15.7:0/2577563463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 msgr2=0x562823c5f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 0x562823c5f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d0f13640 1 -- 172.21.15.7:0/2577563463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 msgr2=0x562823c66060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 0x562823c66060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d0f13640 1 -- 172.21.15.7:0/2577563463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628239967e0 con 0x562823c54960 2024-03-26T08:47:46.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d1714640 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 0x562823c66060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:46.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c5eae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0036040 tx=0x7fc4c002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e41d2170d947af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4abfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:46.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4abfff640 1 -- 172.21.15.7:0/2577563463 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c004f070 con 0x562823c54960 2024-03-26T08:47:46.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.100+0000 7fc4abfff640 1 -- 172.21.15.7:0/2577563463 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4c0004460 con 0x562823c54960 2024-03-26T08:47:46.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.101+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2577563463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4c4003e50 con 0x562823c54960 2024-03-26T08:47:46.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.101+0000 7fc4abfff640 1 -- 172.21.15.7:0/2577563463 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c005b050 con 0x562823c54960 2024-03-26T08:47:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.101+0000 7fc4d59fd740 10 client.26689 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.101+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2577563463 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5628239f6f70 con 0x562823c54960 2024-03-26T08:47:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.102+0000 7fc4abfff640 1 -- 172.21.15.7:0/2577563463 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c00053f0 con 0x562823c54960 2024-03-26T08:47:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.102+0000 7fc4abfff640 1 -- 172.21.15.7:0/2577563463 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc4c004b440 con 0x562823c54960 2024-03-26T08:47:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.102+0000 7fc4d59fd740 10 client.26689 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.102+0000 7fc4d59fd740 10 client.26689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.102+0000 7fc4d59fd740 20 client.26689 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.102+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562823c6a950 0x562823c6cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.102+0000 7fc4d59fd740 4 client.26689 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.102+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2577563463 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4c4003e50 con 0x562823c6a950 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.102+0000 7fc4cbfff640 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562823c6a950 0x562823c6cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.103+0000 7fc4cbfff640 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562823c6a950 0x562823c6cd90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505c2f3366ce586a server_cookie=98f5613547aee268 in_seq=0 out_seq=0 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.103+0000 7fc4abfff640 10 client.26689 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.103+0000 7fc4abfff640 1 -- 172.21.15.7:0/2577563463 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fc4c4003e50 con 0x562823c6a950 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.103+0000 7fc4abfff640 10 client.26689 handle_command_reply: tid=0 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.103+0000 7fc4d59fd740 1 client.26689 shutdown 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.103+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2577563463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 msgr2=0x562823c5eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.103+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c5eae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0036040 tx=0x7fc4c002bc30 comp rx=0 tx=0).stop 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.104+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2577563463 shutdown_connections 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.104+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562823c6a950 0x562823c6cd90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.104+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c5a4c0 0x562823c66060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.104+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c55350 0x562823c5f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.104+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2577563463 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c54960 0x562823c5eae0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.104+0000 7fc4abfff640 0 client.26689 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.104+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2577563463 >> 172.21.15.7:0/2577563463 conn(0x562823c1d1a0 msgr2=0x562823c4f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.105+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2577563463 shutdown_connections 2024-03-26T08:47:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.105+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2577563463 wait complete. 2024-03-26T08:47:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.105+0000 7fc4d59fd740 20 client.26689 trim_cache size 0 max 16384 2024-03-26T08:47:46.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d59fd740 1 Processor -- start 2024-03-26T08:47:46.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d59fd740 1 -- start start 2024-03-26T08:47:46.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 0x562823c66020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 0x562823c5ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d0f13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4cbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 0x562823c66020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d0f13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c53b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41818/0 (socket says 172.21.15.7:41818) 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d0f13640 1 -- 172.21.15.7:0/2523113234 learned_addr learned my addr 172.21.15.7:0/2523113234 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2523113234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00108c0 con 0x562823c55350 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2523113234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b000e3c0 con 0x562823c54960 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d1714640 1 --2- 172.21.15.7:0/2523113234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 0x562823c5ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.133+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2523113234 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b000c440 con 0x562823c5a4c0 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4d0f13640 1 -- 172.21.15.7:0/2523113234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 msgr2=0x562823c5ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/2523113234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 0x562823c5ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4d0f13640 1 -- 172.21.15.7:0/2523113234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 msgr2=0x562823c66020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/2523113234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 0x562823c66020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4d0f13640 1 -- 172.21.15.7:0/2523113234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56282388e170 con 0x562823c54960 2024-03-26T08:47:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4d1714640 1 --2- 172.21.15.7:0/2523113234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 0x562823c5ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/2523113234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c53b80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000c00 tx=0x7fc4c0005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f9c400ab7f2e59 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4b67fc640 1 -- 172.21.15.7:0/2523113234 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0063070 con 0x562823c54960 2024-03-26T08:47:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4b67fc640 1 -- 172.21.15.7:0/2523113234 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c004ba00 con 0x562823c54960 2024-03-26T08:47:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.134+0000 7fc4b67fc640 1 -- 172.21.15.7:0/2523113234 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c004f070 con 0x562823c54960 2024-03-26T08:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.135+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2523113234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 msgr2=0x562823c53b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.136+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2523113234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c53b80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000c00 tx=0x7fc4c0005d50 comp rx=0 tx=0).stop 2024-03-26T08:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.136+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2523113234 shutdown_connections 2024-03-26T08:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.136+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2523113234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 0x562823c5ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.136+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2523113234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 0x562823c66020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.136+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/2523113234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c53b80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.136+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2523113234 >> 172.21.15.7:0/2523113234 conn(0x562823c1d1a0 msgr2=0x562823c5e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.136+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2523113234 shutdown_connections 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.136+0000 7fc4d59fd740 1 -- 172.21.15.7:0/2523113234 wait complete. 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d59fd740 1 -- start start 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d59fd740 1 Processor -- start 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c64bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 0x562823c65130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d59fd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 0x562823ccea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d59fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b000ca40 con 0x562823c55350 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d59fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b000dd60 con 0x562823c54960 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d59fd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b000d0a0 con 0x562823c5a4c0 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d0f13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c64bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d0f13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c64bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41828/0 (socket says 172.21.15.7:41828) 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d0f13640 1 -- 172.21.15.7:0/3373788011 learned_addr learned my addr 172.21.15.7:0/3373788011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.137+0000 7fc4d59fd740 10 client.0 _ll_register_callbacks cb 0x7fc4c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.138+0000 7fc4cbfff640 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 0x562823c65130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.138+0000 7fc4d1714640 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 0x562823ccea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.138+0000 7fc4d0f13640 1 -- 172.21.15.7:0/3373788011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 msgr2=0x562823ccea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.138+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 0x562823ccea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.138+0000 7fc4d0f13640 1 -- 172.21.15.7:0/3373788011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 msgr2=0x562823c65130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.138+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 0x562823c65130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.138+0000 7fc4d0f13640 1 -- 172.21.15.7:0/3373788011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562823ccef70 con 0x562823c54960 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.139+0000 7fc4d0f13640 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c64bf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc4c002c7b0 tx=0x7fc4c004b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b6c66fec627071 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.139+0000 7fc4abfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4abfff640 1 -- 172.21.15.7:0/3373788011 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c004f070 con 0x562823c54960 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4abfff640 1 -- 172.21.15.7:0/3373788011 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4c0004f10 con 0x562823c54960 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3373788011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4c004ba00 con 0x562823c54960 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4abfff640 1 -- 172.21.15.7:0/3373788011 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c0063040 con 0x562823c54960 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4d59fd740 10 client.26660 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3373788011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562823cd00e0 con 0x562823c54960 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4abfff640 1 -- 172.21.15.7:0/3373788011 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c004ba00 con 0x562823c54960 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4abfff640 1 -- 172.21.15.7:0/3373788011 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc4c0070050 con 0x562823c54960 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4d59fd740 10 client.26660 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4d59fd740 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4d59fd740 20 client.26660 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562823cd0870 0x562823cd2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4d59fd740 4 client.26660 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.140+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3373788011 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562823bc75b0 con 0x562823cd0870 2024-03-26T08:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.141+0000 7fc4cbfff640 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562823cd0870 0x562823cd2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.141+0000 7fc4cbfff640 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562823cd0870 0x562823cd2c90 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42a49e0b80a4409 server_cookie=18a43b99a9f96372 in_seq=0 out_seq=0 2024-03-26T08:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.141+0000 7fc4abfff640 10 client.26660 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4abfff640 1 -- 172.21.15.7:0/3373788011 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562823bc75b0 con 0x562823cd0870 2024-03-26T08:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4abfff640 10 client.26660 handle_command_reply: tid=0 2024-03-26T08:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4d59fd740 1 client.26660 shutdown 2024-03-26T08:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3373788011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 msgr2=0x562823c64bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c64bf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc4c002c7b0 tx=0x7fc4c004b6f0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3373788011 shutdown_connections 2024-03-26T08:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562823cd0870 0x562823cd2c90 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562823c5a4c0 0x562823ccea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562823c55350 0x562823c65130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4abfff640 0 client.26660 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4d59fd740 1 --2- 172.21.15.7:0/3373788011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562823c54960 0x562823c64bf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.143+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3373788011 >> 172.21.15.7:0/3373788011 conn(0x562823c1d1a0 msgr2=0x562823c4eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.144+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3373788011 shutdown_connections 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.144+0000 7fc4d59fd740 1 -- 172.21.15.7:0/3373788011 wait complete. 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.144+0000 7fc4d59fd740 20 client.26660 trim_cache size 0 max 16384 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "22aa4737-0a88-4ef5-8fe2-30ec1e15ffa9", 2024-03-26T08:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:47:46.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.144+0000 7fc4d3177640 1 -- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4bc0a2880 msgr2=0x7fc4bc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.144+0000 7fc4d3177640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4bc0a2880 0x7fc4bc0a4d40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005ab0 tx=0x7fc4c002e040 comp rx=0 tx=0).stop 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.144+0000 7fc4d3177640 1 -- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc102170 msgr2=0x7fc4cc102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.144+0000 7fc4d3177640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc102170 0x7fc4cc102b90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9f0 tx=0x7fc4b800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.146+0000 7fc4d3177640 1 -- 172.21.15.7:0/1871712774 shutdown_connections 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.146+0000 7fc4d3177640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4bc0a2880 0x7fc4bc0a4d40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.146+0000 7fc4d3177640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4cc1030d0 0x7fc4cc103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.146+0000 7fc4d3177640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4cc102170 0x7fc4cc102b90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.146+0000 7fc4d3177640 1 --2- 172.21.15.7:0/1871712774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc1017b0 0x7fc4cc104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.146+0000 7fc4d3177640 1 -- 172.21.15.7:0/1871712774 >> 172.21.15.7:0/1871712774 conn(0x7fc4cc0fbb30 msgr2=0x7fc4cc069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.146+0000 7fc4d3177640 1 -- 172.21.15.7:0/1871712774 shutdown_connections 2024-03-26T08:47:46.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.146+0000 7fc4d3177640 1 -- 172.21.15.7:0/1871712774 wait complete. 2024-03-26T08:47:46.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:47:46.153 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:47:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.386+0000 7f539a1a9640 1 -- 172.21.15.7:0/2944473213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53941005c0 msgr2=0x7f53941009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.386+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2944473213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53941005c0 0x7f53941009e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5384009900 tx=0x7f538402bc40 comp rx=0 tx=0).stop 2024-03-26T08:47:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.386+0000 7f539a1a9640 1 -- 172.21.15.7:0/2944473213 shutdown_connections 2024-03-26T08:47:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.386+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2944473213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5394102170 0x7f5394108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.386+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2944473213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53941017b0 0x7f5394101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.386+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2944473213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53941005c0 0x7f53941009e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.386+0000 7f539a1a9640 1 -- 172.21.15.7:0/2944473213 >> 172.21.15.7:0/2944473213 conn(0x7f53940fbb30 msgr2=0x7f53940fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.386+0000 7f539a1a9640 1 -- 172.21.15.7:0/2944473213 shutdown_connections 2024-03-26T08:47:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.387+0000 7f539a1a9640 1 -- 172.21.15.7:0/2944473213 wait complete. 2024-03-26T08:47:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.387+0000 7f539a1a9640 1 Processor -- start 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.387+0000 7f539a1a9640 1 -- start start 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.388+0000 7f539a1a9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53941005c0 0x7f5394197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.388+0000 7f53937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53941005c0 0x7f5394197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.389+0000 7f53937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53941005c0 0x7f5394197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38404/0 (socket says 172.21.15.7:38404) 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.389+0000 7f53937fe640 1 -- 172.21.15.7:0/2630743048 learned_addr learned my addr 172.21.15.7:0/2630743048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.389+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53941017b0 0x7f5394197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.389+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5394102170 0x7f539419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.389+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539410c290 con 0x7f53941005c0 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.389+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539410c110 con 0x7f53941017b0 2024-03-26T08:47:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.389+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539410c410 con 0x7f5394102170 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.390+0000 7f5393fff640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5394102170 0x7f539419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.390+0000 7f53937fe640 1 -- 172.21.15.7:0/2630743048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5394102170 msgr2=0x7f539419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.390+0000 7f53937fe640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5394102170 0x7f539419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.390+0000 7f53937fe640 1 -- 172.21.15.7:0/2630743048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53941017b0 msgr2=0x7f5394197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.390+0000 7f5392ffd640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53941017b0 0x7f5394197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.390+0000 7f53937fe640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53941017b0 0x7f5394197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.390+0000 7f53937fe640 1 -- 172.21.15.7:0/2630743048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539419f630 con 0x7f53941005c0 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.390+0000 7f5392ffd640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53941017b0 0x7f5394197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.390+0000 7f53937fe640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53941005c0 0x7f5394197a80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f5384004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.391+0000 7f5390ff9640 1 -- 172.21.15.7:0/2630743048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384041070 con 0x7f53941005c0 2024-03-26T08:47:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.391+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539419f8c0 con 0x7f53941005c0 2024-03-26T08:47:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.391+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539419fe00 con 0x7f53941005c0 2024-03-26T08:47:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.391+0000 7f5390ff9640 1 -- 172.21.15.7:0/2630743048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5384004c80 con 0x7f53941005c0 2024-03-26T08:47:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.391+0000 7f5390ff9640 1 -- 172.21.15.7:0/2630743048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384003c30 con 0x7f53941005c0 2024-03-26T08:47:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.394+0000 7f5390ff9640 1 -- 172.21.15.7:0/2630743048 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5384044070 con 0x7f53941005c0 2024-03-26T08:47:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.394+0000 7f539cd8e740 1 Processor -- start 2024-03-26T08:47:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.394+0000 7f539cd8e740 1 -- start start 2024-03-26T08:47:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.394+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 0x5639ee81cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 0x5639ee821b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 0x5639ee824600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f539cd8e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee7793d0 con 0x5639ee822170 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f539cd8e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee721ad0 con 0x5639ee81cbc0 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f539cd8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee775470 con 0x5639ee81d5b0 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 0x5639ee81cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 0x5639ee81cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41870/0 (socket says 172.21.15.7:41870) 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5393fff640 1 -- 172.21.15.7:0/2844362828 learned_addr learned my addr 172.21.15.7:0/2844362828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f53937fe640 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 0x5639ee821b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5392ffd640 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 0x5639ee824600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5393fff640 1 -- 172.21.15.7:0/2844362828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 msgr2=0x5639ee821b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5393fff640 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 0x5639ee821b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5393fff640 1 -- 172.21.15.7:0/2844362828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 msgr2=0x5639ee824600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5393fff640 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 0x5639ee824600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5393fff640 1 -- 172.21.15.7:0/2844362828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639ee5577d0 con 0x5639ee81cbc0 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f53937fe640 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 0x5639ee821b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5390ff9640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f53780a2800 0x7f53780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.395+0000 7f5393fff640 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 0x5639ee81cfe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f538c008da0 tx=0x7f538c00d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646a71437ebbf742 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.396+0000 7f5390ff9640 1 -- 172.21.15.7:0/2630743048 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f53840f08a0 con 0x7f53941005c0 2024-03-26T08:47:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.396+0000 7f5389ffb640 1 -- 172.21.15.7:0/2844362828 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00f830 con 0x5639ee81cbc0 2024-03-26T08:47:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.396+0000 7f5389ffb640 1 -- 172.21.15.7:0/2844362828 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f538c00de20 con 0x5639ee81cbc0 2024-03-26T08:47:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.396+0000 7f5389ffb640 1 -- 172.21.15.7:0/2844362828 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00b3a0 con 0x5639ee81cbc0 2024-03-26T08:47:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.396+0000 7f53937fe640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f53780a2800 0x7f53780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.397+0000 7f539cd8e740 1 -- 172.21.15.7:0/2844362828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 msgr2=0x5639ee81cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.397+0000 7f539cd8e740 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 0x5639ee81cfe0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f538c008da0 tx=0x7f538c00d710 comp rx=0 tx=0).stop 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.397+0000 7f539cd8e740 1 -- 172.21.15.7:0/2844362828 shutdown_connections 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.397+0000 7f539cd8e740 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 0x5639ee824600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.397+0000 7f539cd8e740 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 0x5639ee821b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.397+0000 7f539cd8e740 1 --2- 172.21.15.7:0/2844362828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 0x5639ee81cfe0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.397+0000 7f539cd8e740 1 -- 172.21.15.7:0/2844362828 >> 172.21.15.7:0/2844362828 conn(0x5639ee80f2b0 msgr2=0x5639ee80f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.398+0000 7f539cd8e740 1 -- 172.21.15.7:0/2844362828 shutdown_connections 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.398+0000 7f539cd8e740 1 -- 172.21.15.7:0/2844362828 wait complete. 2024-03-26T08:47:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.398+0000 7f53937fe640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f53780a2800 0x7f53780a4cc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f538402c720 tx=0x7f538402bc40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.399+0000 7f539cd8e740 1 -- start start 2024-03-26T08:47:46.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.399+0000 7f539cd8e740 1 Processor -- start 2024-03-26T08:47:46.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81cbc0 0x5639ee826ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81d5b0 0x5639ee827020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f5393fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81cbc0 0x5639ee826ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f5393fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81cbc0 0x5639ee826ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39860/0 (socket says 172.21.15.7:39860) 2024-03-26T08:47:46.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f5393fff640 1 -- 172.21.15.7:0/1099600472 learned_addr learned my addr 172.21.15.7:0/1099600472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f539cd8e740 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 0x5639ee82e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee825120 con 0x5639ee822170 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee735060 con 0x5639ee81d5b0 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639ee8252a0 con 0x5639ee81cbc0 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.400+0000 7f539cd8e740 10 client.0 _ll_register_callbacks cb 0x5639ee7176a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5392ffd640 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81d5b0 0x5639ee827020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5393fff640 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 0x5639ee82e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5393fff640 1 -- 172.21.15.7:0/1099600472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81d5b0 msgr2=0x5639ee827020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5393fff640 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81d5b0 0x5639ee827020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5393fff640 1 -- 172.21.15.7:0/1099600472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 msgr2=0x5639ee82e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5393fff640 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 0x5639ee82e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5393fff640 1 -- 172.21.15.7:0/1099600472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639ee48d3f0 con 0x5639ee81cbc0 2024-03-26T08:47:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5393fff640 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81cbc0 0x5639ee826ae0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f538c008450 tx=0x7f538c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3ec413527c70da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5376ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5376ffd640 1 -- 172.21.15.7:0/1099600472 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00ee60 con 0x5639ee81cbc0 2024-03-26T08:47:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.401+0000 7f5376ffd640 1 -- 172.21.15.7:0/1099600472 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f538c020040 con 0x5639ee81cbc0 2024-03-26T08:47:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.402+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f538c00de20 con 0x5639ee81cbc0 2024-03-26T08:47:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.402+0000 7f5376ffd640 1 -- 172.21.15.7:0/1099600472 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f538c00f840 con 0x5639ee81cbc0 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.402+0000 7f539cd8e740 10 client.26704 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.402+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639ee44d0f0 con 0x5639ee81cbc0 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.402+0000 7f5376ffd640 1 -- 172.21.15.7:0/1099600472 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f538c00de20 con 0x5639ee81cbc0 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.402+0000 7f5376ffd640 1 -- 172.21.15.7:0/1099600472 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f538c023070 con 0x5639ee81cbc0 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.403+0000 7f539cd8e740 10 client.26704 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.403+0000 7f539cd8e740 10 client.26704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.403+0000 7f539cd8e740 20 client.26704 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.403+0000 7f539cd8e740 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639ee832280 0x5639ee834670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.403+0000 7f539cd8e740 4 client.26704 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.403+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639ee786910 con 0x5639ee832280 2024-03-26T08:47:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.403+0000 7f5392ffd640 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639ee832280 0x5639ee834670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.404+0000 7f5392ffd640 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639ee832280 0x5639ee834670 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a03e5dbe3d518 server_cookie=d9525cb7774422f3 in_seq=0 out_seq=0 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.404+0000 7f5376ffd640 10 client.26704 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f5376ffd640 1 -- 172.21.15.7:0/1099600472 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5639ee786910 con 0x5639ee832280 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f5376ffd640 10 client.26704 handle_command_reply: tid=0 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f539cd8e740 1 client.26704 shutdown 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81cbc0 msgr2=0x5639ee826ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f539cd8e740 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81cbc0 0x5639ee826ae0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f538c008450 tx=0x7f538c004a40 comp rx=0 tx=0).stop 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 shutdown_connections 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f539cd8e740 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639ee832280 0x5639ee834670 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f539cd8e740 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee822170 0x5639ee82e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f539cd8e740 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81d5b0 0x5639ee827020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f5376ffd640 0 client.26704 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f539cd8e740 1 --2- 172.21.15.7:0/1099600472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81cbc0 0x5639ee826ae0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.405+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 >> 172.21.15.7:0/1099600472 conn(0x5639ee80f2b0 msgr2=0x5639ee81c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.406+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 shutdown_connections 2024-03-26T08:47:46.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.406+0000 7f539cd8e740 1 -- 172.21.15.7:0/1099600472 wait complete. 2024-03-26T08:47:46.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.406+0000 7f539cd8e740 20 client.26704 trim_cache size 0 max 16384 2024-03-26T08:47:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.412+0000 7f6a5759e640 1 -- 172.21.15.7:0/3485361988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a50096830 msgr2=0x7f6a5009d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.412+0000 7f6a5759e640 1 --2- 172.21.15.7:0/3485361988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a50096830 0x7f6a5009d0c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6a5804f6d0 tx=0x7f6a580977d0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.412+0000 7f6a5759e640 1 -- 172.21.15.7:0/3485361988 shutdown_connections 2024-03-26T08:47:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.412+0000 7f6a5759e640 1 --2- 172.21.15.7:0/3485361988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a50096830 0x7f6a5009d0c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.412+0000 7f6a5759e640 1 --2- 172.21.15.7:0/3485361988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50095e70 0x7f6a500962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.412+0000 7f6a5759e640 1 --2- 172.21.15.7:0/3485361988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a50094c80 0x7f6a500950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.412+0000 7f6a5759e640 1 -- 172.21.15.7:0/3485361988 >> 172.21.15.7:0/3485361988 conn(0x7f6a500901d0 msgr2=0x7f6a50092630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.412+0000 7f6a5759e640 1 -- 172.21.15.7:0/3485361988 shutdown_connections 2024-03-26T08:47:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.412+0000 7f6a5759e640 1 -- 172.21.15.7:0/3485361988 wait complete. 2024-03-26T08:47:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.413+0000 7f6a5759e640 1 Processor -- start 2024-03-26T08:47:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.413+0000 7f6a5759e640 1 -- start start 2024-03-26T08:47:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a5759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50094c80 0x7f6a50132530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a5759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a50095e70 0x7f6a50132a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50094c80 0x7f6a50132530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a5659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50094c80 0x7f6a50132530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41900/0 (socket says 172.21.15.7:41900) 2024-03-26T08:47:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a5659c640 1 -- 172.21.15.7:0/1234763810 learned_addr learned my addr 172.21.15.7:0/1234763810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a55d9b640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a50095e70 0x7f6a50132a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a5759e640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a50096830 0x7f6a50139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5009f700 con 0x7f6a50096830 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5009f580 con 0x7f6a50094c80 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.414+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5009f880 con 0x7f6a50095e70 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.415+0000 7f6a5659c640 1 -- 172.21.15.7:0/1234763810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a50095e70 msgr2=0x7f6a50132a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.415+0000 7f6a5659c640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a50095e70 0x7f6a50132a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.415+0000 7f6a5659c640 1 -- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a50096830 msgr2=0x7f6a50139940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.415+0000 7f6a5659c640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a50096830 0x7f6a50139940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.415+0000 7f6a5659c640 1 -- 172.21.15.7:0/1234763810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a50139fe0 con 0x7f6a50094c80 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.415+0000 7f6a5659c640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50094c80 0x7f6a50132530 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800e9f0 tx=0x7f6a4800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.415+0000 7f6a477fe640 1 -- 172.21.15.7:0/1234763810 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800ccc0 con 0x7f6a50094c80 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.415+0000 7f6a477fe640 1 -- 172.21.15.7:0/1234763810 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a480035f0 con 0x7f6a50094c80 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.415+0000 7f6a477fe640 1 -- 172.21.15.7:0/1234763810 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800ce60 con 0x7f6a50094c80 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.416+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5013a2d0 con 0x7f6a50094c80 2024-03-26T08:47:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.416+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5013a770 con 0x7f6a50094c80 2024-03-26T08:47:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.420+0000 7f6a477fe640 1 -- 172.21.15.7:0/1234763810 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6a480037a0 con 0x7f6a50094c80 2024-03-26T08:47:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.420+0000 7f6a5f83b740 1 Processor -- start 2024-03-26T08:47:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a5f83b740 1 -- start start 2024-03-26T08:47:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d7349b20 0x5625d7349f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d734f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734f680 0x5625d7351b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d72aeb10 con 0x5625d7349b20 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d7257900 con 0x5625d734f680 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d72aabb0 con 0x5625d734a510 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d7349b20 0x5625d7349f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a5659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d734f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a5659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d734f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39886/0 (socket says 172.21.15.7:39886) 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d7349b20 0x5625d7349f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38446/0 (socket says 172.21.15.7:38446) 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a56d9d640 1 -- 172.21.15.7:0/1486843939 learned_addr learned my addr 172.21.15.7:0/1486843939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a477fe640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a400a28b0 0x7f6a400a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.421+0000 7f6a55d9b640 1 --2- 172.21.15.7:0/1486843939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734f680 0x5625d7351b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.422+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a400a28b0 0x7f6a400a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.422+0000 7f6a477fe640 1 -- 172.21.15.7:0/1234763810 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6a480ced20 con 0x7f6a50094c80 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.422+0000 7f6a56d9d640 1 -- 172.21.15.7:0/1486843939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 msgr2=0x5625d734f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.422+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/1486843939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d734f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.422+0000 7f6a56d9d640 1 -- 172.21.15.7:0/1486843939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734f680 msgr2=0x5625d7351b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.422+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/1486843939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734f680 0x5625d7351b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.422+0000 7f6a56d9d640 1 -- 172.21.15.7:0/1486843939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625d6fc3210 con 0x5625d7349b20 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.423+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/1486843939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d7349b20 0x5625d7349f40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6a580647e0 tx=0x7f6a5809ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f065954f7bf9a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.423+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a400a28b0 0x7f6a400a4d70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6a58097d10 tx=0x7f6a580a8000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.423+0000 7f6a44ff9640 1 -- 172.21.15.7:0/1486843939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580a4040 con 0x5625d7349b20 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.423+0000 7f6a44ff9640 1 -- 172.21.15.7:0/1486843939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a580b4070 con 0x5625d7349b20 2024-03-26T08:47:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.423+0000 7f6a44ff9640 1 -- 172.21.15.7:0/1486843939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580b8c80 con 0x5625d7349b20 2024-03-26T08:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.424+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1486843939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d7349b20 msgr2=0x5625d7349f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.424+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/1486843939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d7349b20 0x5625d7349f40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6a580647e0 tx=0x7f6a5809ed80 comp rx=0 tx=0).stop 2024-03-26T08:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.424+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1486843939 shutdown_connections 2024-03-26T08:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.424+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/1486843939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734f680 0x5625d7351b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.424+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/1486843939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d734f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.424+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/1486843939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d7349b20 0x5625d7349f40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.424+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1486843939 >> 172.21.15.7:0/1486843939 conn(0x5625d7345090 msgr2=0x5625d7345490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.424+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1486843939 shutdown_connections 2024-03-26T08:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.425+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1486843939 wait complete. 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.425+0000 7f6a5f83b740 1 -- start start 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.425+0000 7f6a5f83b740 1 Processor -- start 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.425+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 0x5625d7353e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.425+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d7354340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.425+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d735b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.425+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d7352620 con 0x5625d734f680 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.425+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d726ad50 con 0x5625d7349b20 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.425+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625d73527a0 con 0x5625d734a510 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.426+0000 7f6a5f83b740 10 client.0 _ll_register_callbacks cb 0x5625d726a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.426+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d735b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.426+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d735b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38460/0 (socket says 172.21.15.7:38460) 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.426+0000 7f6a56d9d640 1 -- 172.21.15.7:0/3921781904 learned_addr learned my addr 172.21.15.7:0/3921781904 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.426+0000 7f6a55d9b640 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d7354340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.427+0000 7f6a5659c640 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 0x5625d7353e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.427+0000 7f6a55d9b640 1 -- 172.21.15.7:0/3921781904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 msgr2=0x5625d7353e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.427+0000 7f6a55d9b640 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 0x5625d7353e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.427+0000 7f6a55d9b640 1 -- 172.21.15.7:0/3921781904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 msgr2=0x5625d735b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.427+0000 7f6a55d9b640 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d735b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.427+0000 7f6a55d9b640 1 -- 172.21.15.7:0/3921781904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625d708f620 con 0x5625d734a510 2024-03-26T08:47:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.427+0000 7f6a5659c640 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 0x5625d7353e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.427+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d735b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:47:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.428+0000 7f6a55d9b640 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d7354340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a501338a0 tx=0x7f6a4c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5d247682955e567 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.428+0000 7f6a3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.428+0000 7f6a3dffb640 1 -- 172.21.15.7:0/3921781904 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00f890 con 0x5625d734a510 2024-03-26T08:47:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.428+0000 7f6a5f83b740 1 -- 172.21.15.7:0/3921781904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a580b4070 con 0x5625d734a510 2024-03-26T08:47:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.428+0000 7f6a3dffb640 1 -- 172.21.15.7:0/3921781904 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6a4c00fa30 con 0x5625d734a510 2024-03-26T08:47:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.428+0000 7f6a3dffb640 1 -- 172.21.15.7:0/3921781904 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a4c00fc30 con 0x5625d734a510 2024-03-26T08:47:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.428+0000 7f6a5f83b740 10 client.26716 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.429+0000 7f6a5f83b740 1 -- 172.21.15.7:0/3921781904 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5625d736f7b0 con 0x5625d734a510 2024-03-26T08:47:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.435+0000 7f6a3dffb640 1 -- 172.21.15.7:0/3921781904 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a580b4070 con 0x5625d734a510 2024-03-26T08:47:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.435+0000 7f6a3dffb640 1 -- 172.21.15.7:0/3921781904 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6a4c01c420 con 0x5625d734a510 2024-03-26T08:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.435+0000 7f6a5f83b740 10 client.26716 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.435+0000 7f6a5f83b740 10 client.26716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.435+0000 7f6a5f83b740 20 client.26716 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.435+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5625d7370e50 0x5625d7373290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.435+0000 7f6a5f83b740 4 client.26716 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.435+0000 7f6a5f83b740 1 -- 172.21.15.7:0/3921781904 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625d72bc050 con 0x5625d7370e50 2024-03-26T08:47:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.436+0000 7f6a5659c640 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5625d7370e50 0x5625d7373290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.440+0000 7f6a5659c640 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5625d7370e50 0x5625d7373290 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2694b95b420c208 server_cookie=ec146087ab4d01f6 in_seq=0 out_seq=0 2024-03-26T08:47:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.441+0000 7f6a3dffb640 10 client.26716 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.440+0000 7f539cd8e740 1 Processor -- start 2024-03-26T08:47:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.443+0000 7f539cd8e740 1 -- start start 2024-03-26T08:47:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.443+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 0x5639ee80dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.443+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81d5b0 0x5639ee82dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.443+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee822170 0x5639ee80e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.443+0000 7f539cd8e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c010990 con 0x5639ee81d5b0 2024-03-26T08:47:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f539cd8e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c00e490 con 0x5639ee81cbc0 2024-03-26T08:47:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f539cd8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c00c510 con 0x5639ee822170 2024-03-26T08:47:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f5392ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81d5b0 0x5639ee82dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f5392ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81d5b0 0x5639ee82dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38474/0 (socket says 172.21.15.7:38474) 2024-03-26T08:47:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f5392ffd640 1 -- 172.21.15.7:0/2999787614 learned_addr learned my addr 172.21.15.7:0/2999787614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f5392ffd640 1 -- 172.21.15.7:0/2999787614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee822170 msgr2=0x5639ee80e1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:47:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f5392ffd640 1 --2- 172.21.15.7:0/2999787614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee822170 0x5639ee80e1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f5392ffd640 1 -- 172.21.15.7:0/2999787614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 msgr2=0x5639ee80dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f5392ffd640 1 --2- 172.21.15.7:0/2999787614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 0x5639ee80dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.444+0000 7f5392ffd640 1 -- 172.21.15.7:0/2999787614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639ee810020 con 0x5639ee81d5b0 2024-03-26T08:47:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.445+0000 7f5392ffd640 1 --2- 172.21.15.7:0/2999787614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81d5b0 0x5639ee82dff0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5380000c00 tx=0x7f538000a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3173b594815f78c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.445+0000 7f53897fa640 1 -- 172.21.15.7:0/2999787614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000a1f0 con 0x5639ee81d5b0 2024-03-26T08:47:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.445+0000 7f53897fa640 1 -- 172.21.15.7:0/2999787614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5380019150 con 0x5639ee81d5b0 2024-03-26T08:47:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.446+0000 7f539cd8e740 1 -- 172.21.15.7:0/2999787614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81d5b0 msgr2=0x5639ee82dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.446+0000 7f539cd8e740 1 --2- 172.21.15.7:0/2999787614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81d5b0 0x5639ee82dff0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5380000c00 tx=0x7f538000a7b0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.448+0000 7f539cd8e740 1 -- 172.21.15.7:0/2999787614 shutdown_connections 2024-03-26T08:47:46.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.448+0000 7f539cd8e740 1 --2- 172.21.15.7:0/2999787614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee822170 0x5639ee80e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.448+0000 7f539cd8e740 1 --2- 172.21.15.7:0/2999787614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81d5b0 0x5639ee82dff0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.448+0000 7f539cd8e740 1 --2- 172.21.15.7:0/2999787614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee81cbc0 0x5639ee80dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.448+0000 7f539cd8e740 1 -- 172.21.15.7:0/2999787614 >> 172.21.15.7:0/2999787614 conn(0x5639ee80f2b0 msgr2=0x5639ee825ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.448+0000 7f539cd8e740 1 -- 172.21.15.7:0/2999787614 shutdown_connections 2024-03-26T08:47:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.449+0000 7f539cd8e740 1 -- 172.21.15.7:0/2999787614 wait complete. 2024-03-26T08:47:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.450+0000 7f539cd8e740 1 -- start start 2024-03-26T08:47:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.450+0000 7f539cd8e740 1 Processor -- start 2024-03-26T08:47:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.450+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81cbc0 0x5639ee82bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.450+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 0x5639ee82c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.450+0000 7f539cd8e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee822170 0x5639ee82d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.450+0000 7f539cd8e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c00cb10 con 0x5639ee81cbc0 2024-03-26T08:47:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.450+0000 7f539cd8e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c00de30 con 0x5639ee822170 2024-03-26T08:47:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.450+0000 7f539cd8e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c00d170 con 0x5639ee81d5b0 2024-03-26T08:47:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.450+0000 7f539cd8e740 10 client.0 _ll_register_callbacks cb 0x7f538c008bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.451+0000 7f5393fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81cbc0 0x5639ee82bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.451+0000 7f5393fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81cbc0 0x5639ee82bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38476/0 (socket says 172.21.15.7:38476) 2024-03-26T08:47:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.451+0000 7f5393fff640 1 -- 172.21.15.7:0/652626117 learned_addr learned my addr 172.21.15.7:0/652626117 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.451+0000 7f5393fff640 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee822170 0x5639ee82d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.451+0000 7f5393fff640 1 -- 172.21.15.7:0/652626117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 msgr2=0x5639ee82c050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:47:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.451+0000 7f5393fff640 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 0x5639ee82c050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.451+0000 7f5393fff640 1 -- 172.21.15.7:0/652626117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee822170 msgr2=0x5639ee82d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.451+0000 7f5393fff640 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee822170 0x5639ee82d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.451+0000 7f5393fff640 1 -- 172.21.15.7:0/652626117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639ee82d650 con 0x5639ee81cbc0 2024-03-26T08:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.454+0000 7f6a3dffb640 1 -- 172.21.15.7:0/3921781904 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5625d72bc050 con 0x5625d7370e50 2024-03-26T08:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.454+0000 7f6a3dffb640 10 client.26716 handle_command_reply: tid=0 2024-03-26T08:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.454+0000 7f6a5f83b740 1 client.26716 shutdown 2024-03-26T08:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.454+0000 7f6a5f83b740 1 -- 172.21.15.7:0/3921781904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 msgr2=0x5625d7354340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.454+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d7354340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a501338a0 tx=0x7f6a4c012b60 comp rx=0 tx=0).stop 2024-03-26T08:47:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.453+0000 7f5393fff640 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81cbc0 0x5639ee82bb10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f538c00dd90 tx=0x7f538c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4671741ceb37b63d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.453+0000 7f5376ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:47:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.454+0000 7f5376ffd640 1 -- 172.21.15.7:0/652626117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00e990 con 0x5639ee81cbc0 2024-03-26T08:47:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.454+0000 7f5376ffd640 1 -- 172.21.15.7:0/652626117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f538c002030 con 0x5639ee81cbc0 2024-03-26T08:47:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.455+0000 7f539cd8e740 1 -- 172.21.15.7:0/652626117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639ee780c60 con 0x5639ee81cbc0 2024-03-26T08:47:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f5376ffd640 1 -- 172.21.15.7:0/652626117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f538c00eb30 con 0x5639ee81cbc0 2024-03-26T08:47:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f539cd8e740 10 client.16854 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f539cd8e740 1 -- 172.21.15.7:0/652626117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639ee813670 con 0x5639ee81cbc0 2024-03-26T08:47:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f6a5f83b740 1 -- 172.21.15.7:0/3921781904 shutdown_connections 2024-03-26T08:47:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5625d7370e50 0x5625d7373290 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d735b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734a510 0x5625d7354340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/3921781904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 0x5625d7353e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f6a5f83b740 1 -- 172.21.15.7:0/3921781904 >> 172.21.15.7:0/3921781904 conn(0x5625d7345090 msgr2=0x5625d7344290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.456+0000 7f6a3dffb640 0 client.26716 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.458+0000 7f5376ffd640 1 -- 172.21.15.7:0/652626117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5639ee780c60 con 0x5639ee81cbc0 2024-03-26T08:47:46.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.458+0000 7f5376ffd640 1 -- 172.21.15.7:0/652626117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f538c03d030 con 0x5639ee81cbc0 2024-03-26T08:47:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.461+0000 7f6a5f83b740 1 -- 172.21.15.7:0/3921781904 shutdown_connections 2024-03-26T08:47:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.461+0000 7f6a5f83b740 1 -- 172.21.15.7:0/3921781904 wait complete. 2024-03-26T08:47:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.461+0000 7f6a5f83b740 20 client.26716 trim_cache size 0 max 16384 2024-03-26T08:47:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.460+0000 7f539cd8e740 10 client.16854 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.460+0000 7f539cd8e740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.460+0000 7f539cd8e740 20 client.16854 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.460+0000 7f539cd8e740 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639ee832280 0x5639ee815e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.460+0000 7f539cd8e740 4 client.16854 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:47:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.460+0000 7f539cd8e740 1 -- 172.21.15.7:0/652626117 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5639ee6fe4e0 con 0x5639ee832280 2024-03-26T08:47:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.463+0000 7f5392ffd640 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639ee832280 0x5639ee815e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.464+0000 7f5392ffd640 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639ee832280 0x5639ee815e10 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e1ccbe470d766d server_cookie=25519fdc8e66b4d1 in_seq=0 out_seq=0 2024-03-26T08:47:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.465+0000 7f5376ffd640 10 client.16854 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.465+0000 7f5376ffd640 1 -- 172.21.15.7:0/652626117 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5639ee6fe4e0 con 0x5639ee832280 2024-03-26T08:47:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.465+0000 7f5376ffd640 10 client.16854 handle_command_reply: tid=0 2024-03-26T08:47:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.465+0000 7f539cd8e740 1 client.16854 shutdown 2024-03-26T08:47:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.466+0000 7f539cd8e740 1 -- 172.21.15.7:0/652626117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81cbc0 msgr2=0x5639ee82bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.466+0000 7f539cd8e740 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81cbc0 0x5639ee82bb10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f538c00dd90 tx=0x7f538c00dcc0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.469+0000 7f539cd8e740 1 -- 172.21.15.7:0/652626117 shutdown_connections 2024-03-26T08:47:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.469+0000 7f539cd8e740 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639ee832280 0x5639ee815e10 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.469+0000 7f539cd8e740 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639ee822170 0x5639ee82d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.469+0000 7f539cd8e740 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639ee81d5b0 0x5639ee82c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.469+0000 7f539cd8e740 1 --2- 172.21.15.7:0/652626117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639ee81cbc0 0x5639ee82bb10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.469+0000 7f539cd8e740 1 -- 172.21.15.7:0/652626117 >> 172.21.15.7:0/652626117 conn(0x5639ee80f2b0 msgr2=0x5639ee825660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.469+0000 7f5376ffd640 0 client.16854 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.470+0000 7f539cd8e740 1 -- 172.21.15.7:0/652626117 shutdown_connections 2024-03-26T08:47:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.470+0000 7f539cd8e740 1 -- 172.21.15.7:0/652626117 wait complete. 2024-03-26T08:47:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.470+0000 7f539cd8e740 20 client.16854 trim_cache size 0 max 16384 2024-03-26T08:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "22aa4737-0a88-4ef5-8fe2-30ec1e15ffa9": { 2024-03-26T08:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "22aa4737-0a88-4ef5-8fe2-30ec1e15ffa9", 2024-03-26T08:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:47:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f53780a2800 msgr2=0x7f53780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f53780a2800 0x7f53780a4cc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f538402c720 tx=0x7f538402bc40 comp rx=0 tx=0).stop 2024-03-26T08:47:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53941005c0 msgr2=0x7f5394197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53941005c0 0x7f5394197a80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f5384004040 comp rx=0 tx=0).stop 2024-03-26T08:47:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 shutdown_connections 2024-03-26T08:47:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f53780a2800 0x7f53780a4cc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5394102170 0x7f539419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53941017b0 0x7f5394197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 --2- 172.21.15.7:0/2630743048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53941005c0 0x7f5394197a80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.471+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 >> 172.21.15.7:0/2630743048 conn(0x7f53940fbb30 msgr2=0x7f53940fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.472+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 shutdown_connections 2024-03-26T08:47:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:47:46.472+0000 7f539a1a9640 1 -- 172.21.15.7:0/2630743048 wait complete. 2024-03-26T08:47:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5f83b740 1 Processor -- start 2024-03-26T08:47:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5f83b740 1 -- start start 2024-03-26T08:47:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d7349b20 0x5625d734b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734a510 0x5625d735b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d734bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5013d2c0 con 0x5625d734f680 2024-03-26T08:47:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5013adc0 con 0x5625d734a510 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500aba40 con 0x5625d7349b20 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a55d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734a510 0x5625d735b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d7349b20 0x5625d734b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a55d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734a510 0x5625d735b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41942/0 (socket says 172.21.15.7:41942) 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a55d9b640 1 -- 172.21.15.7:0/1849699860 learned_addr learned my addr 172.21.15.7:0/1849699860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d7349b20 0x5625d734b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39916/0 (socket says 172.21.15.7:39916) 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/1849699860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d734bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5659c640 1 -- 172.21.15.7:0/1849699860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734a510 msgr2=0x5625d735b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.487+0000 7f6a5659c640 1 --2- 172.21.15.7:0/1849699860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734a510 0x5625d735b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.488+0000 7f6a5659c640 1 -- 172.21.15.7:0/1849699860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 msgr2=0x5625d734bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.488+0000 7f6a5659c640 1 --2- 172.21.15.7:0/1849699860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d734bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.488+0000 7f6a5659c640 1 -- 172.21.15.7:0/1849699860 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625d736f5d0 con 0x5625d7349b20 2024-03-26T08:47:46.482 INFO:tasks.cephfs.filesystem:scrub status for tag:22aa4737-0a88-4ef5-8fe2-30ec1e15ffa9 - {'path': '/', 'tag': '22aa4737-0a88-4ef5-8fe2-30ec1e15ffa9', 'options': 'recursive,force'} 2024-03-26T08:47:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.488+0000 7f6a5659c640 1 --2- 172.21.15.7:0/1849699860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d7349b20 0x5625d734b830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a48098090 tx=0x7f6a48003c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6655eece8e0c6468 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.488+0000 7f6a3ffff640 1 -- 172.21.15.7:0/1849699860 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a480b3070 con 0x5625d7349b20 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.488+0000 7f6a3ffff640 1 -- 172.21.15.7:0/1849699860 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a4809b6c0 con 0x5625d7349b20 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.488+0000 7f6a3ffff640 1 -- 172.21.15.7:0/1849699860 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a480b4270 con 0x5625d7349b20 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.489+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1849699860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d7349b20 msgr2=0x5625d734b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.489+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/1849699860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d7349b20 0x5625d734b830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a48098090 tx=0x7f6a48003c90 comp rx=0 tx=0).stop 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.490+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1849699860 shutdown_connections 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.490+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/1849699860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734f680 0x5625d734bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.490+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/1849699860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d734a510 0x5625d735b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.490+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/1849699860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d7349b20 0x5625d734b830 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.490+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1849699860 >> 172.21.15.7:0/1849699860 conn(0x5625d7345090 msgr2=0x5625d7345470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.490+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1849699860 shutdown_connections 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 1 -- 172.21.15.7:0/1849699860 wait complete. 2024-03-26T08:47:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 1 -- start start 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 1 Processor -- start 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 0x5625d7359280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734a510 0x5625d73597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734f680 0x5625d735a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500ac010 con 0x5625d734a510 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500ad330 con 0x5625d7349b20 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500ac670 con 0x5625d734f680 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a5f83b740 10 client.0 _ll_register_callbacks cb 0x7f6a4c009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734f680 0x5625d735a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a55d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734a510 0x5625d73597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a56d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734f680 0x5625d735a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39920/0 (socket says 172.21.15.7:39920) 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.491+0000 7f6a56d9d640 1 -- 172.21.15.7:0/2418504236 learned_addr learned my addr 172.21.15.7:0/2418504236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.492+0000 7f6a5659c640 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 0x5625d7359280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.492+0000 7f6a56d9d640 1 -- 172.21.15.7:0/2418504236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 msgr2=0x5625d7359280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.492+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 0x5625d7359280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.492+0000 7f6a56d9d640 1 -- 172.21.15.7:0/2418504236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734a510 msgr2=0x5625d73597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.492+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734a510 0x5625d73597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.492+0000 7f6a56d9d640 1 -- 172.21.15.7:0/2418504236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625d735adc0 con 0x5625d734f680 2024-03-26T08:47:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.492+0000 7f6a56d9d640 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734f680 0x5625d735a7d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a58067100 tx=0x7f6a580b0b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7505fd696469fd47 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.492+0000 7f6a3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:47:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.492+0000 7f6a3dffb640 1 -- 172.21.15.7:0/2418504236 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580c5070 con 0x5625d734f680 2024-03-26T08:47:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a3dffb640 1 -- 172.21.15.7:0/2418504236 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6a580c1460 con 0x5625d734f680 2024-03-26T08:47:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a5f83b740 1 -- 172.21.15.7:0/2418504236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a4809b6c0 con 0x5625d734f680 2024-03-26T08:47:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a3dffb640 1 -- 172.21.15.7:0/2418504236 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a580cf050 con 0x5625d734f680 2024-03-26T08:47:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a5f83b740 10 client.26722 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a5f83b740 1 -- 172.21.15.7:0/2418504236 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5625d73cd0a0 con 0x5625d734f680 2024-03-26T08:47:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a3dffb640 1 -- 172.21.15.7:0/2418504236 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a580b4040 con 0x5625d734f680 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a3dffb640 1 -- 172.21.15.7:0/2418504236 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6a580c1c60 con 0x5625d734f680 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a5f83b740 10 client.26722 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a5f83b740 10 client.26722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a5f83b740 20 client.26722 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5625d73cd6a0 0x5625d73cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a5f83b740 4 client.26722 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.493+0000 7f6a5f83b740 1 -- 172.21.15.7:0/2418504236 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f6a4809b6c0 con 0x5625d73cd6a0 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.494+0000 7f6a5659c640 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5625d73cd6a0 0x5625d73cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.494+0000 7f6a5659c640 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5625d73cd6a0 0x5625d73cfa90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bced47400827c5c server_cookie=ad2299ba40ad5c8c in_seq=0 out_seq=0 2024-03-26T08:47:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.494+0000 7f6a3dffb640 10 client.26722 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.534+0000 7f6a3dffb640 1 -- 172.21.15.7:0/2418504236 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f6a4809b6c0 con 0x5625d73cd6a0 2024-03-26T08:47:46.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.534+0000 7f6a3dffb640 10 client.26722 handle_command_reply: tid=0 2024-03-26T08:47:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.534+0000 7f6a5f83b740 1 client.26722 shutdown 2024-03-26T08:47:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a5f83b740 1 -- 172.21.15.7:0/2418504236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734f680 msgr2=0x5625d735a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734f680 0x5625d735a7d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a58067100 tx=0x7f6a580b0b40 comp rx=0 tx=0).stop 2024-03-26T08:47:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a5f83b740 1 -- 172.21.15.7:0/2418504236 shutdown_connections 2024-03-26T08:47:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5625d73cd6a0 0x5625d73cfa90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5625d734f680 0x5625d735a7d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5625d734a510 0x5625d73597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a5f83b740 1 --2- 172.21.15.7:0/2418504236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5625d7349b20 0x5625d7359280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a5f83b740 1 -- 172.21.15.7:0/2418504236 >> 172.21.15.7:0/2418504236 conn(0x5625d7345090 msgr2=0x5625d7353da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a3dffb640 0 client.26722 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.535+0000 7f6a5f83b740 1 -- 172.21.15.7:0/2418504236 shutdown_connections 2024-03-26T08:47:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.536+0000 7f6a5f83b740 1 -- 172.21.15.7:0/2418504236 wait complete. 2024-03-26T08:47:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.536+0000 7f6a5f83b740 20 client.26722 trim_cache size 0 max 16384 2024-03-26T08:47:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:47:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:47:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:47:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:47:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a400a28b0 msgr2=0x7f6a400a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a400a28b0 0x7f6a400a4d70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6a58097d10 tx=0x7f6a580a8000 comp rx=0 tx=0).stop 2024-03-26T08:47:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50094c80 msgr2=0x7f6a50132530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:46.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50094c80 0x7f6a50132530 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800e9f0 tx=0x7f6a4800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 shutdown_connections 2024-03-26T08:47:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a400a28b0 0x7f6a400a4d70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a50096830 0x7f6a50139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a50095e70 0x7f6a50132a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 --2- 172.21.15.7:0/1234763810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50094c80 0x7f6a50132530 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 >> 172.21.15.7:0/1234763810 conn(0x7f6a500901d0 msgr2=0x7f6a500a20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 shutdown_connections 2024-03-26T08:47:46.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:46.537+0000 7f6a5759e640 1 -- 172.21.15.7:0/1234763810 wait complete. 2024-03-26T08:47:46.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:46 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:47:46.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:46 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:47.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:46 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:47:47.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:46 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:46 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:47:47.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:46 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:47.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:47 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:47:47.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:47 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:47.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:47 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:47:47.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:47 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:47.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:47 smithi016 ceph-mon[35851]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 29 MiB/s wr, 1.18k op/s 2024-03-26T08:47:48.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:47 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:47:48.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:47 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:48.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:47 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:47:48.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:47 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:48.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:47 smithi007 ceph-mon[32462]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 29 MiB/s wr, 1.18k op/s 2024-03-26T08:47:48.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:47 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:47:48.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:47 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:48.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:47 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:47:48.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:47 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:47:48.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:47 smithi044 ceph-mon[36461]: pgmap v521: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 29 MiB/s wr, 1.18k op/s 2024-03-26T08:47:49.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:49 smithi016 ceph-mon[35851]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 18 MiB/s wr, 937 op/s 2024-03-26T08:47:50.046 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:49 smithi007 ceph-mon[32462]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 18 MiB/s wr, 937 op/s 2024-03-26T08:47:50.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:49 smithi044 ceph-mon[36461]: pgmap v522: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 18 MiB/s wr, 937 op/s 2024-03-26T08:47:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 320.0 s, 3436.0 tps, lat 29.255 ms stddev 34.288 2024-03-26T08:47:51.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:51 smithi016 ceph-mon[35851]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 11 MiB/s wr, 798 op/s 2024-03-26T08:47:52.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:51 smithi007 ceph-mon[32462]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 11 MiB/s wr, 798 op/s 2024-03-26T08:47:52.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:51 smithi044 ceph-mon[36461]: pgmap v523: 97 pgs: 97 active+clean; 1.6 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 11 MiB/s wr, 798 op/s 2024-03-26T08:47:53.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:53 smithi016 ceph-mon[35851]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 887 op/s 2024-03-26T08:47:54.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:53 smithi007 ceph-mon[32462]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 887 op/s 2024-03-26T08:47:54.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:53 smithi044 ceph-mon[36461]: pgmap v524: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 887 op/s 2024-03-26T08:47:55.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:55 smithi016 ceph-mon[35851]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.8 MiB/s wr, 813 op/s 2024-03-26T08:47:56.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:55 smithi007 ceph-mon[32462]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.8 MiB/s wr, 813 op/s 2024-03-26T08:47:56.066 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:55 smithi044 ceph-mon[36461]: pgmap v525: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.8 MiB/s wr, 813 op/s 2024-03-26T08:47:56.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 325.0 s, 2817.0 tps, lat 35.457 ms stddev 37.526 2024-03-26T08:47:56.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:56 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:47:56.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:56 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:47:57.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:56 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:47:57.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:56 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:47:57.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:56 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:47:57.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:56 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:47:57.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:57 smithi016 ceph-mon[35851]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.7 MiB/s wr, 811 op/s 2024-03-26T08:47:58.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:57 smithi007 ceph-mon[32462]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.7 MiB/s wr, 811 op/s 2024-03-26T08:47:58.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:57 smithi044 ceph-mon[36461]: pgmap v526: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.7 MiB/s wr, 811 op/s 2024-03-26T08:47:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.675+0000 7f7c1720b640 1 -- 172.21.15.7:0/2443786186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c10102180 msgr2=0x7f7c10068e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.675+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2443786186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c10102180 0x7f7c10068e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009ca0 tx=0x7f7c0802c140 comp rx=0 tx=0).stop 2024-03-26T08:47:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.676+0000 7f7c1720b640 1 -- 172.21.15.7:0/2443786186 shutdown_connections 2024-03-26T08:47:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.676+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2443786186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c10102180 0x7f7c10068e90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.676+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2443786186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c101017c0 0x7f7c10101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.676+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2443786186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c101005d0 0x7f7c101009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.676+0000 7f7c1720b640 1 -- 172.21.15.7:0/2443786186 >> 172.21.15.7:0/2443786186 conn(0x7f7c100fbb40 msgr2=0x7f7c100fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.677+0000 7f7c1720b640 1 -- 172.21.15.7:0/2443786186 shutdown_connections 2024-03-26T08:47:59.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.679+0000 7f7c1720b640 1 -- 172.21.15.7:0/2443786186 wait complete. 2024-03-26T08:47:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.679+0000 7f7c1720b640 1 Processor -- start 2024-03-26T08:47:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.680+0000 7f7c1720b640 1 -- start start 2024-03-26T08:47:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.680+0000 7f7c1720b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c101005d0 0x7f7c10195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.680+0000 7f7c1720b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c101017c0 0x7f7c10195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.680+0000 7f7c1720b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c10102180 0x7f7c1019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.680+0000 7f7c1720b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1010f650 con 0x7f7c101017c0 2024-03-26T08:47:59.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.680+0000 7f7c1720b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1010f4d0 con 0x7f7c101005d0 2024-03-26T08:47:59.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.680+0000 7f7c1720b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1010f7d0 con 0x7f7c10102180 2024-03-26T08:47:59.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.680+0000 7f7c157a8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c10102180 0x7f7c1019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.681+0000 7f7c14fa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c101005d0 0x7f7c10195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.681+0000 7f7c14fa7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c101005d0 0x7f7c10195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59862/0 (socket says 172.21.15.7:59862) 2024-03-26T08:47:59.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.681+0000 7f7c14fa7640 1 -- 172.21.15.7:0/2537220646 learned_addr learned my addr 172.21.15.7:0/2537220646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.682+0000 7f7c0ffff640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c101017c0 0x7f7c10195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.682+0000 7f7c157a8640 1 -- 172.21.15.7:0/2537220646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c101005d0 msgr2=0x7f7c10195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.682+0000 7f7c157a8640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c101005d0 0x7f7c10195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.682+0000 7f7c157a8640 1 -- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c101017c0 msgr2=0x7f7c10195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.682+0000 7f7c157a8640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c101017c0 0x7f7c10195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.682+0000 7f7c157a8640 1 -- 172.21.15.7:0/2537220646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c101ae5d0 con 0x7f7c10102180 2024-03-26T08:47:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.682+0000 7f7c157a8640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c10102180 0x7f7c1019cf20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c08037040 tx=0x7f7c08035e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:59.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.683+0000 7f7c0dffb640 1 -- 172.21.15.7:0/2537220646 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08039040 con 0x7f7c10102180 2024-03-26T08:47:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.683+0000 7f7c0dffb640 1 -- 172.21.15.7:0/2537220646 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c08003970 con 0x7f7c10102180 2024-03-26T08:47:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.683+0000 7f7c0dffb640 1 -- 172.21.15.7:0/2537220646 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08004690 con 0x7f7c10102180 2024-03-26T08:47:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.683+0000 7f7c1720b640 1 -- 172.21.15.7:0/2537220646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c101ae860 con 0x7f7c10102180 2024-03-26T08:47:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.683+0000 7f7c1720b640 1 -- 172.21.15.7:0/2537220646 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c101aed80 con 0x7f7c10102180 2024-03-26T08:47:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.686+0000 7f7c19df9740 1 Processor -- start 2024-03-26T08:47:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.686+0000 7f7c0dffb640 1 -- 172.21.15.7:0/2537220646 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7c08003b20 con 0x7f7c10102180 2024-03-26T08:47:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.687+0000 7f7c0dffb640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bf80a28b0 0x7f7bf80a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.687+0000 7f7c0dffb640 1 -- 172.21.15.7:0/2537220646 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c080f0b10 con 0x7f7c10102180 2024-03-26T08:47:59.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.688+0000 7f7c0ffff640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bf80a28b0 0x7f7bf80a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.688+0000 7f7c19df9740 1 -- start start 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.688+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe3015d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.688+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 0x557fe301ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.688+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe301d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.688+0000 7f7c19df9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe2f7ab10 con 0x557fe3016310 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.688+0000 7f7c19df9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe2f23900 con 0x557fe301b480 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.688+0000 7f7c19df9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe2f76bb0 con 0x557fe3015920 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c157a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe301d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c157a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe301d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59870/0 (socket says 172.21.15.7:59870) 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c157a8640 1 -- 172.21.15.7:0/746440979 learned_addr learned my addr 172.21.15.7:0/746440979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c0ffff640 1 --2- 172.21.15.7:0/746440979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 0x557fe301ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c0ffff640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bf80a28b0 0x7f7bf80a4d70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7c10196e80 tx=0x7f7c04008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c157a8640 1 -- 172.21.15.7:0/746440979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 msgr2=0x557fe3015d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:47:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c157a8640 1 --2- 172.21.15.7:0/746440979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe3015d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c157a8640 1 -- 172.21.15.7:0/746440979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 msgr2=0x557fe301ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c157a8640 1 --2- 172.21.15.7:0/746440979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 0x557fe301ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c157a8640 1 -- 172.21.15.7:0/746440979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fe2c8f210 con 0x557fe301b480 2024-03-26T08:47:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.689+0000 7f7c157a8640 1 --2- 172.21.15.7:0/746440979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe301d910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7c080c56e0 tx=0x7f7c080bcfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944d0503c7d6e2d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.690+0000 7f7bf6ffd640 1 -- 172.21.15.7:0/746440979 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08004830 con 0x557fe301b480 2024-03-26T08:47:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.690+0000 7f7bf6ffd640 1 -- 172.21.15.7:0/746440979 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c080bdb70 con 0x557fe301b480 2024-03-26T08:47:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.690+0000 7f7bf6ffd640 1 -- 172.21.15.7:0/746440979 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080bd6d0 con 0x557fe301b480 2024-03-26T08:47:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 -- 172.21.15.7:0/746440979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 msgr2=0x557fe301d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 --2- 172.21.15.7:0/746440979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe301d910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7c080c56e0 tx=0x7f7c080bcfb0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 -- 172.21.15.7:0/746440979 shutdown_connections 2024-03-26T08:47:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 --2- 172.21.15.7:0/746440979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe301d910 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 --2- 172.21.15.7:0/746440979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 0x557fe301ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 --2- 172.21.15.7:0/746440979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe3015d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 -- 172.21.15.7:0/746440979 >> 172.21.15.7:0/746440979 conn(0x557fe3010e90 msgr2=0x557fe3011290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 -- 172.21.15.7:0/746440979 shutdown_connections 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 -- 172.21.15.7:0/746440979 wait complete. 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.691+0000 7f7c19df9740 1 -- start start 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c19df9740 1 Processor -- start 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe301fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 0x557fe301ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe3026f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c19df9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe301e1f0 con 0x557fe3016310 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c19df9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe2f36d50 con 0x557fe301b480 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c19df9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fe301e370 con 0x557fe3015920 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c19df9740 10 client.0 _ll_register_callbacks cb 0x557fe2f36720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c157a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe3026f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c157a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe3026f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59874/0 (socket says 172.21.15.7:59874) 2024-03-26T08:47:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c157a8640 1 -- 172.21.15.7:0/1843392492 learned_addr learned my addr 172.21.15.7:0/1843392492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c14fa7640 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe301fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.692+0000 7f7c0ffff640 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 0x557fe301ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7c157a8640 1 -- 172.21.15.7:0/1843392492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 msgr2=0x557fe301fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7c157a8640 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe301fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7c157a8640 1 -- 172.21.15.7:0/1843392492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 msgr2=0x557fe301ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7c157a8640 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 0x557fe301ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7c157a8640 1 -- 172.21.15.7:0/1843392492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fe2d5b620 con 0x557fe301b480 2024-03-26T08:47:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7c157a8640 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe3026f90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c080b9e40 tx=0x7f7c080095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=831a1b08c1637bfa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7beffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7beffff640 1 -- 172.21.15.7:0/1843392492 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08004830 con 0x557fe301b480 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7beffff640 1 -- 172.21.15.7:0/1843392492 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c080c5780 con 0x557fe301b480 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7c19df9740 1 -- 172.21.15.7:0/1843392492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c080bdb70 con 0x557fe301b480 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7beffff640 1 -- 172.21.15.7:0/1843392492 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c080c15c0 con 0x557fe301b480 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7c19df9740 10 client.26696 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.693+0000 7f7c19df9740 1 -- 172.21.15.7:0/1843392492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557fe301e1f0 con 0x557fe301b480 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7beffff640 1 -- 172.21.15.7:0/1843392492 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c080bdb70 con 0x557fe301b480 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7beffff640 1 -- 172.21.15.7:0/1843392492 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7c0803c030 con 0x557fe301b480 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7c19df9740 10 client.26696 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7c19df9740 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7c19df9740 20 client.26696 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fe302b5b0 0x557fe302d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7c19df9740 4 client.26696 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:47:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7c19df9740 1 -- 172.21.15.7:0/1843392492 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fe2f88050 con 0x557fe302b5b0 2024-03-26T08:47:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7c14fa7640 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fe302b5b0 0x557fe302d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.694+0000 7f7beffff640 10 client.26696 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.695+0000 7f7c14fa7640 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fe302b5b0 0x557fe302d9d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=534c24e00ae5b8a server_cookie=6b445e9dfe2fe9f2 in_seq=0 out_seq=0 2024-03-26T08:47:59.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.696+0000 7f7beffff640 1 -- 172.21.15.7:0/1843392492 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557fe2f88050 con 0x557fe302b5b0 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.696+0000 7f7beffff640 10 client.26696 handle_command_reply: tid=0 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.696+0000 7f7c19df9740 1 client.26696 shutdown 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.696+0000 7f7c19df9740 1 -- 172.21.15.7:0/1843392492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 msgr2=0x557fe3026f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.696+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe3026f90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c080b9e40 tx=0x7f7c080095c0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.697+0000 7f7c19df9740 1 -- 172.21.15.7:0/1843392492 shutdown_connections 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.697+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fe302b5b0 0x557fe302d9d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.697+0000 7f7beffff640 0 client.26696 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.697+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe301b480 0x557fe3026f90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.697+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3016310 0x557fe301ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.697+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1843392492 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe301fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.697+0000 7f7c19df9740 1 -- 172.21.15.7:0/1843392492 >> 172.21.15.7:0/1843392492 conn(0x557fe3010e90 msgr2=0x557fe3010020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.697+0000 7f7c19df9740 1 -- 172.21.15.7:0/1843392492 shutdown_connections 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.698+0000 7f7c19df9740 1 -- 172.21.15.7:0/1843392492 wait complete. 2024-03-26T08:47:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.698+0000 7f7c19df9740 20 client.26696 trim_cache size 0 max 16384 2024-03-26T08:47:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.717+0000 7f7c19df9740 1 Processor -- start 2024-03-26T08:47:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c19df9740 1 -- start start 2024-03-26T08:47:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3015920 0x557fe301e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 0x557fe3026f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c14fa7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3015920 0x557fe301e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe301b480 0x557fe30176d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c19df9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101b9e80 con 0x557fe3015920 2024-03-26T08:47:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c19df9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101b7980 con 0x557fe3016310 2024-03-26T08:47:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c19df9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101b5a00 con 0x557fe301b480 2024-03-26T08:47:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c14fa7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3015920 0x557fe301e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37106/0 (socket says 172.21.15.7:37106) 2024-03-26T08:47:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c14fa7640 1 -- 172.21.15.7:0/2874094549 learned_addr learned my addr 172.21.15.7:0/2874094549 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c0ffff640 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 0x557fe3026f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.718+0000 7f7c157a8640 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe301b480 0x557fe30176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.719+0000 7f7c14fa7640 1 -- 172.21.15.7:0/2874094549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe301b480 msgr2=0x557fe30176d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.719+0000 7f7c14fa7640 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe301b480 0x557fe30176d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.719+0000 7f7c14fa7640 1 -- 172.21.15.7:0/2874094549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 msgr2=0x557fe3026f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.719+0000 7f7c14fa7640 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 0x557fe3026f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.719+0000 7f7c14fa7640 1 -- 172.21.15.7:0/2874094549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fe302e030 con 0x557fe3015920 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.719+0000 7f7c157a8640 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe301b480 0x557fe30176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.719+0000 7f7c14fa7640 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3015920 0x557fe301e660 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x557fe3016b70 tx=0x7f7c00009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a9782d077f7fab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.720+0000 7f7bf67fc640 1 -- 172.21.15.7:0/2874094549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00024070 con 0x557fe3015920 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.720+0000 7f7bf67fc640 1 -- 172.21.15.7:0/2874094549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557fe2f88050 con 0x557fe3015920 2024-03-26T08:47:59.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.720+0000 7f7bf67fc640 1 -- 172.21.15.7:0/2874094549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0001fbd0 con 0x557fe3015920 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.721+0000 7f7c19df9740 1 -- 172.21.15.7:0/2874094549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3015920 msgr2=0x557fe301e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.721+0000 7f7c19df9740 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3015920 0x557fe301e660 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x557fe3016b70 tx=0x7f7c00009190 comp rx=0 tx=0).stop 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.721+0000 7f7c19df9740 1 -- 172.21.15.7:0/2874094549 shutdown_connections 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.721+0000 7f7c19df9740 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe301b480 0x557fe30176d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.721+0000 7f7c19df9740 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 0x557fe3026f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.721+0000 7f7c19df9740 1 --2- 172.21.15.7:0/2874094549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe3015920 0x557fe301e660 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.721+0000 7f7c19df9740 1 -- 172.21.15.7:0/2874094549 >> 172.21.15.7:0/2874094549 conn(0x557fe3010e90 msgr2=0x557fe300beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.722+0000 7f7c19df9740 1 -- 172.21.15.7:0/2874094549 shutdown_connections 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.722+0000 7f7c19df9740 1 -- 172.21.15.7:0/2874094549 wait complete. 2024-03-26T08:47:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.722+0000 7f7c19df9740 1 -- start start 2024-03-26T08:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.722+0000 7f7c19df9740 1 Processor -- start 2024-03-26T08:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe3024460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 0x557fe30249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c19df9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe301b480 0x557fe30259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c14fa7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe3024460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 0x557fe30249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c14fa7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe3024460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:40662/0 (socket says 172.21.15.7:40662) 2024-03-26T08:47:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 0x557fe30249a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:59888/0 (socket says 172.21.15.7:59888) 2024-03-26T08:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c14fa7640 1 -- 172.21.15.7:0/1489736089 learned_addr learned my addr 172.21.15.7:0/1489736089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101b6000 con 0x557fe301b480 2024-03-26T08:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101b7320 con 0x557fe3016310 2024-03-26T08:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101b6660 con 0x557fe3015920 2024-03-26T08:47:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.723+0000 7f7c157a8640 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe301b480 0x557fe30259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c157a8640 1 -- 172.21.15.7:0/1489736089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 msgr2=0x557fe3024460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c157a8640 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe3024460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c157a8640 1 -- 172.21.15.7:0/1489736089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 msgr2=0x557fe30249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:47:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c157a8640 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 0x557fe30249a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c157a8640 1 -- 172.21.15.7:0/1489736089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fe301b160 con 0x557fe301b480 2024-03-26T08:47:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c19df9740 10 client.0 _ll_register_callbacks cb 0x7f7c080c5c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c157a8640 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe301b480 0x557fe30259b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7c08032520 tx=0x7f7c08004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66a149403347715 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fe2f88050 con 0x557fe301b480 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7beffff640 10 client.16875 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7beffff640 1 -- 172.21.15.7:0/1489736089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080bcdd0 con 0x557fe301b480 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7beffff640 1 -- 172.21.15.7:0/1489736089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c080c5980 con 0x557fe301b480 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7beffff640 1 -- 172.21.15.7:0/1489736089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c08004040 con 0x557fe301b480 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c19df9740 10 client.16875 fetch_fsmap learned FSMap version 25 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.724+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x557fe301b160 con 0x557fe301b480 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7beffff640 1 -- 172.21.15.7:0/1489736089 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7c080bc8c0 con 0x557fe301b480 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7c19df9740 10 client.16875 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7c19df9740 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7c19df9740 20 client.16875 populate_metadata read hostname 'smithi007' 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fe3086e80 0x557fe3089270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:47:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7c19df9740 4 client.16875 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:47:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557fe2f823a0 con 0x557fe3086e80 2024-03-26T08:47:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7c14fa7640 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fe3086e80 0x557fe3089270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:47:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7beffff640 10 client.16875 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.725+0000 7f7c14fa7640 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fe3086e80 0x557fe3089270 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee18c44d7fa02d44 server_cookie=c250bf97fd6581bf in_seq=0 out_seq=0 2024-03-26T08:47:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7beffff640 1 -- 172.21.15.7:0/1489736089 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557fe2f823a0 con 0x557fe3086e80 2024-03-26T08:47:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7beffff640 10 client.16875 handle_command_reply: tid=0 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 client.16875 shutdown 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe301b480 msgr2=0x557fe30259b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe301b480 0x557fe30259b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7c08032520 tx=0x7f7c08004be0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 shutdown_connections 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fe3086e80 0x557fe3089270 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7beffff640 0 client.16875 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fe301b480 0x557fe30259b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fe3016310 0x557fe30249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 --2- 172.21.15.7:0/1489736089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fe3015920 0x557fe3024460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 >> 172.21.15.7:0/1489736089 conn(0x557fe3010e90 msgr2=0x557fe301f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.746+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 shutdown_connections 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.747+0000 7f7c19df9740 1 -- 172.21.15.7:0/1489736089 wait complete. 2024-03-26T08:47:59.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.747+0000 7f7c19df9740 20 client.16875 trim_cache size 0 max 16384 2024-03-26T08:47:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:47:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:47:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:47:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.747+0000 7f7c1720b640 1 -- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bf80a28b0 msgr2=0x7f7bf80a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.747+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bf80a28b0 0x7f7bf80a4d70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f7c10196e80 tx=0x7f7c04008040 comp rx=0 tx=0).stop 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.747+0000 7f7c1720b640 1 -- 172.21.15.7:0/2537220646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c10102180 msgr2=0x7f7c1019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.747+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c10102180 0x7f7c1019cf20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7c08037040 tx=0x7f7c08035e50 comp rx=0 tx=0).stop 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.748+0000 7f7c1720b640 1 -- 172.21.15.7:0/2537220646 shutdown_connections 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.748+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bf80a28b0 0x7f7bf80a4d70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.748+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c10102180 0x7f7c1019cf20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.748+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c101017c0 0x7f7c10195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.748+0000 7f7c1720b640 1 --2- 172.21.15.7:0/2537220646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c101005d0 0x7f7c10195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:47:59.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.748+0000 7f7c1720b640 1 -- 172.21.15.7:0/2537220646 >> 172.21.15.7:0/2537220646 conn(0x7f7c100fbb40 msgr2=0x7f7c10106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:47:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.748+0000 7f7c1720b640 1 -- 172.21.15.7:0/2537220646 shutdown_connections 2024-03-26T08:47:59.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:47:59.748+0000 7f7c1720b640 1 -- 172.21.15.7:0/2537220646 wait complete. 2024-03-26T08:47:59.994 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:47:59 smithi016 ceph-mon[35851]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.5 MiB/s wr, 801 op/s 2024-03-26T08:48:00.043 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:47:59 smithi007 ceph-mon[32462]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.5 MiB/s wr, 801 op/s 2024-03-26T08:48:00.067 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:47:59 smithi044 ceph-mon[36461]: pgmap v527: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.5 MiB/s wr, 801 op/s 2024-03-26T08:48:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 330.0 s, 3458.8 tps, lat 28.955 ms stddev 34.700 2024-03-26T08:48:02.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:01 smithi016 ceph-mon[35851]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.0 MiB/s wr, 745 op/s 2024-03-26T08:48:02.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:01 smithi007 ceph-mon[32462]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.0 MiB/s wr, 745 op/s 2024-03-26T08:48:02.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:01 smithi044 ceph-mon[36461]: pgmap v528: 97 pgs: 97 active+clean; 1.6 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 9.0 MiB/s wr, 745 op/s 2024-03-26T08:48:04.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:03 smithi016 ceph-mon[35851]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 804 op/s 2024-03-26T08:48:04.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:03 smithi007 ceph-mon[32462]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 804 op/s 2024-03-26T08:48:04.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:03 smithi044 ceph-mon[36461]: pgmap v529: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 10 MiB/s wr, 804 op/s 2024-03-26T08:48:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 335.0 s, 3286.8 tps, lat 30.231 ms stddev 34.394 2024-03-26T08:48:06.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:05 smithi016 ceph-mon[35851]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.4 MiB/s wr, 740 op/s 2024-03-26T08:48:06.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:05 smithi007 ceph-mon[32462]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.4 MiB/s wr, 740 op/s 2024-03-26T08:48:06.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:05 smithi044 ceph-mon[36461]: pgmap v530: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.4 MiB/s wr, 740 op/s 2024-03-26T08:48:08.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:07 smithi016 ceph-mon[35851]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 MiB/s wr, 752 op/s 2024-03-26T08:48:08.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:07 smithi007 ceph-mon[32462]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 MiB/s wr, 752 op/s 2024-03-26T08:48:08.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:07 smithi044 ceph-mon[36461]: pgmap v531: 97 pgs: 97 active+clean; 1.6 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 MiB/s wr, 752 op/s 2024-03-26T08:48:10.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:09 smithi016 ceph-mon[35851]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 746 op/s 2024-03-26T08:48:10.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:09 smithi007 ceph-mon[32462]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 746 op/s 2024-03-26T08:48:10.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:09 smithi044 ceph-mon[36461]: pgmap v532: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 746 op/s 2024-03-26T08:48:11.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 340.0 s, 3259.0 tps, lat 30.791 ms stddev 37.100 2024-03-26T08:48:12.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:11 smithi016 ceph-mon[35851]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 784 op/s 2024-03-26T08:48:12.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:11 smithi007 ceph-mon[32462]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 784 op/s 2024-03-26T08:48:12.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:11 smithi044 ceph-mon[36461]: pgmap v533: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 784 op/s 2024-03-26T08:48:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.877+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3467535385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c241086a0 msgr2=0x7f6c2410aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.877+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3467535385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c241086a0 0x7f6c2410aad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009860 tx=0x7f6c1c02bb70 comp rx=0 tx=0).stop 2024-03-26T08:48:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.877+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3467535385 shutdown_connections 2024-03-26T08:48:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.877+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3467535385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c2410d9c0 0x7f6c2410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.877+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3467535385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2410b010 0x7f6c2410d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.877+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3467535385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c241086a0 0x7f6c2410aad0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.877+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3467535385 >> 172.21.15.7:0/3467535385 conn(0x7f6c2406e6b0 msgr2=0x7f6c24070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.877+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3467535385 shutdown_connections 2024-03-26T08:48:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.877+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3467535385 wait complete. 2024-03-26T08:48:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.878+0000 7f6c2ad5b640 1 Processor -- start 2024-03-26T08:48:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c2ad5b640 1 -- start start 2024-03-26T08:48:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c2ad5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c241086a0 0x7f6c241a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c2ad5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c2410b010 0x7f6c241a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c2ad5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c2410d9c0 0x7f6c241add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c2ad5b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2411c190 con 0x7f6c2410b010 2024-03-26T08:48:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c2ad5b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2411c010 con 0x7f6c241086a0 2024-03-26T08:48:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c2ad5b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2411c310 con 0x7f6c2410d9c0 2024-03-26T08:48:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c292f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c2410d9c0 0x7f6c241add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c292f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c2410d9c0 0x7f6c241add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:37326/0 (socket says 172.21.15.7:37326) 2024-03-26T08:48:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c292f8640 1 -- 172.21.15.7:0/3208222551 learned_addr learned my addr 172.21.15.7:0/3208222551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.879+0000 7f6c292f8640 1 -- 172.21.15.7:0/3208222551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c241086a0 msgr2=0x7f6c241a67e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c292f8640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c241086a0 0x7f6c241a67e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c292f8640 1 -- 172.21.15.7:0/3208222551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c2410b010 msgr2=0x7f6c241a6d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c292f8640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c2410b010 0x7f6c241a6d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c292f8640 1 -- 172.21.15.7:0/3208222551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c241ae400 con 0x7f6c2410d9c0 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c292f8640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c2410d9c0 0x7f6c241add90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d750 tx=0x7f6c0c00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c21ffb640 1 -- 172.21.15.7:0/3208222551 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c017070 con 0x7f6c2410d9c0 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c21ffb640 1 -- 172.21.15.7:0/3208222551 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c005500 con 0x7f6c2410d9c0 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c21ffb640 1 -- 172.21.15.7:0/3208222551 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c004d90 con 0x7f6c2410d9c0 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3208222551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c241ae6f0 con 0x7f6c2410d9c0 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.880+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3208222551 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c241aebf0 con 0x7f6c2410d9c0 2024-03-26T08:48:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c21ffb640 1 -- 172.21.15.7:0/3208222551 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6c0c005020 con 0x7f6c2410d9c0 2024-03-26T08:48:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c2da24740 1 Processor -- start 2024-03-26T08:48:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c2da24740 1 -- start start 2024-03-26T08:48:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c21ffb640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c100a2870 0x7f6c100a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea402d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea407e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea40a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c2da24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ea367b10 con 0x5642ea403310 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c2da24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ea310900 con 0x5642ea408480 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.883+0000 7f6c2da24740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ea363bb0 con 0x5642ea402920 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c21ffb640 1 -- 172.21.15.7:0/3208222551 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c0c0ce4f0 con 0x7f6c2410d9c0 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c23fff640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c100a2870 0x7f6c100a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c292f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea40a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c292f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea40a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55702/0 (socket says 172.21.15.7:55702) 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c292f8640 1 -- 172.21.15.7:0/2178086180 learned_addr learned my addr 172.21.15.7:0/2178086180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c28af7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea402d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c23fff640 1 --2- 172.21.15.7:0/2178086180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea407e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c292f8640 1 -- 172.21.15.7:0/2178086180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 msgr2=0x5642ea402d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:12.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c292f8640 1 --2- 172.21.15.7:0/2178086180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea402d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c292f8640 1 -- 172.21.15.7:0/2178086180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 msgr2=0x5642ea407e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c292f8640 1 --2- 172.21.15.7:0/2178086180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea407e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c292f8640 1 -- 172.21.15.7:0/2178086180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642ea07c210 con 0x5642ea408480 2024-03-26T08:48:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c28af7640 1 --2- 172.21.15.7:0/2178086180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea402d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c23fff640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c100a2870 0x7f6c100a4d30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6c140098d0 tx=0x7f6c14009d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c292f8640 1 --2- 172.21.15.7:0/2178086180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea40a910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d720 tx=0x7f6c0c00de60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155600a3ad3072f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c1affd640 1 -- 172.21.15.7:0/2178086180 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c020040 con 0x5642ea408480 2024-03-26T08:48:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.884+0000 7f6c1affd640 1 -- 172.21.15.7:0/2178086180 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c09d020 con 0x5642ea408480 2024-03-26T08:48:12.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c1affd640 1 -- 172.21.15.7:0/2178086180 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c005310 con 0x5642ea408480 2024-03-26T08:48:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 -- 172.21.15.7:0/2178086180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 msgr2=0x5642ea40a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 --2- 172.21.15.7:0/2178086180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea40a910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d720 tx=0x7f6c0c00de60 comp rx=0 tx=0).stop 2024-03-26T08:48:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 -- 172.21.15.7:0/2178086180 shutdown_connections 2024-03-26T08:48:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 --2- 172.21.15.7:0/2178086180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea40a910 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 --2- 172.21.15.7:0/2178086180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea407e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 --2- 172.21.15.7:0/2178086180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea402d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 -- 172.21.15.7:0/2178086180 >> 172.21.15.7:0/2178086180 conn(0x5642ea3fde90 msgr2=0x5642ea3fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 -- 172.21.15.7:0/2178086180 shutdown_connections 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 -- 172.21.15.7:0/2178086180 wait complete. 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.885+0000 7f6c2da24740 1 -- start start 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.886+0000 7f6c2da24740 1 Processor -- start 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.886+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea402920 0x5642ea40cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.886+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea403310 0x5642ea40d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.886+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea414140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.886+0000 7f6c2da24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ea40b370 con 0x5642ea402920 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.886+0000 7f6c2da24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ea323d50 con 0x5642ea408480 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.886+0000 7f6c2da24740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ea40b4f0 con 0x5642ea403310 2024-03-26T08:48:12.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.886+0000 7f6c2da24740 10 client.0 _ll_register_callbacks cb 0x5642ea323720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c28af7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea402920 0x5642ea40cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c28af7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea402920 0x5642ea40cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58232/0 (socket says 172.21.15.7:58232) 2024-03-26T08:48:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c28af7640 1 -- 172.21.15.7:0/636167320 learned_addr learned my addr 172.21.15.7:0/636167320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c23fff640 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea403310 0x5642ea40d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c28af7640 1 -- 172.21.15.7:0/636167320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea403310 msgr2=0x5642ea40d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c28af7640 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea403310 0x5642ea40d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c28af7640 1 -- 172.21.15.7:0/636167320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 msgr2=0x5642ea414140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c28af7640 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea414140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c28af7640 1 -- 172.21.15.7:0/636167320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642ea148620 con 0x5642ea402920 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.887+0000 7f6c23fff640 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea403310 0x5642ea40d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.888+0000 7f6c28af7640 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea402920 0x5642ea40cbc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c02c0b0 tx=0x7f6c1c035f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64aac9af7e94cabc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.888+0000 7f6c03fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.888+0000 7f6c03fff640 1 -- 172.21.15.7:0/636167320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c039040 con 0x5642ea402920 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.888+0000 7f6c03fff640 1 -- 172.21.15.7:0/636167320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c1c034700 con 0x5642ea402920 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.888+0000 7f6c2da24740 1 -- 172.21.15.7:0/636167320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c0c09d020 con 0x5642ea402920 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c03fff640 1 -- 172.21.15.7:0/636167320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c1c004040 con 0x5642ea402920 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c2da24740 10 client.16884 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c2da24740 1 -- 172.21.15.7:0/636167320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5642ea417570 con 0x5642ea402920 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c03fff640 1 -- 172.21.15.7:0/636167320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c09d020 con 0x5642ea402920 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c03fff640 1 -- 172.21.15.7:0/636167320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6c1c004880 con 0x5642ea402920 2024-03-26T08:48:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c2da24740 10 client.16884 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c2da24740 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c2da24740 20 client.16884 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c2da24740 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5642ea418c90 0x5642ea41b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c2da24740 4 client.16884 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.889+0000 7f6c2da24740 1 -- 172.21.15.7:0/636167320 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642ea375050 con 0x5642ea418c90 2024-03-26T08:48:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.890+0000 7f6c23fff640 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5642ea418c90 0x5642ea41b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.890+0000 7f6c23fff640 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5642ea418c90 0x5642ea41b0c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ce003845ac6081 server_cookie=633e4da091bd3ff0 in_seq=0 out_seq=0 2024-03-26T08:48:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.890+0000 7f6c03fff640 10 client.16884 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c03fff640 1 -- 172.21.15.7:0/636167320 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5642ea375050 con 0x5642ea418c90 2024-03-26T08:48:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c03fff640 10 client.16884 handle_command_reply: tid=0 2024-03-26T08:48:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 client.16884 shutdown 2024-03-26T08:48:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 -- 172.21.15.7:0/636167320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea402920 msgr2=0x5642ea40cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea402920 0x5642ea40cbc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c02c0b0 tx=0x7f6c1c035f60 comp rx=0 tx=0).stop 2024-03-26T08:48:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 -- 172.21.15.7:0/636167320 shutdown_connections 2024-03-26T08:48:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5642ea418c90 0x5642ea41b0c0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea414140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea403310 0x5642ea40d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 --2- 172.21.15.7:0/636167320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea402920 0x5642ea40cbc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c03fff640 0 client.16884 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 -- 172.21.15.7:0/636167320 >> 172.21.15.7:0/636167320 conn(0x5642ea3fde90 msgr2=0x5642ea3fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 -- 172.21.15.7:0/636167320 shutdown_connections 2024-03-26T08:48:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 1 -- 172.21.15.7:0/636167320 wait complete. 2024-03-26T08:48:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.891+0000 7f6c2da24740 20 client.16884 trim_cache size 0 max 16384 2024-03-26T08:48:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c2da24740 1 Processor -- start 2024-03-26T08:48:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c2da24740 1 -- start start 2024-03-26T08:48:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea3fc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea414060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea3fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c2da24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241b9ed0 con 0x5642ea403310 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c2da24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241b79d0 con 0x5642ea408480 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c2da24740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241b5a50 con 0x5642ea402920 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c23fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea414060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c28af7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea3fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c23fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea414060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58244/0 (socket says 172.21.15.7:58244) 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c23fff640 1 -- 172.21.15.7:0/2842213845 learned_addr learned my addr 172.21.15.7:0/2842213845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.913+0000 7f6c292f8640 1 --2- 172.21.15.7:0/2842213845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea3fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.914+0000 7f6c23fff640 1 -- 172.21.15.7:0/2842213845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 msgr2=0x5642ea3fc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.914+0000 7f6c23fff640 1 --2- 172.21.15.7:0/2842213845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea3fc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.914+0000 7f6c23fff640 1 -- 172.21.15.7:0/2842213845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 msgr2=0x5642ea3fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.914+0000 7f6c23fff640 1 --2- 172.21.15.7:0/2842213845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea3fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.914+0000 7f6c23fff640 1 -- 172.21.15.7:0/2842213845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642ea40a8d0 con 0x5642ea403310 2024-03-26T08:48:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.914+0000 7f6c23fff640 1 --2- 172.21.15.7:0/2842213845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea414060 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6c241a7cf0 tx=0x7f6c1401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64dcda182b9dd5f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.914+0000 7f6c1a7fc640 1 -- 172.21.15.7:0/2842213845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1402c070 con 0x5642ea403310 2024-03-26T08:48:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.914+0000 7f6c1a7fc640 1 -- 172.21.15.7:0/2842213845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c14004040 con 0x5642ea403310 2024-03-26T08:48:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.914+0000 7f6c1a7fc640 1 -- 172.21.15.7:0/2842213845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14024040 con 0x5642ea403310 2024-03-26T08:48:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.915+0000 7f6c2da24740 1 -- 172.21.15.7:0/2842213845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 msgr2=0x5642ea414060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.915+0000 7f6c2da24740 1 --2- 172.21.15.7:0/2842213845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea414060 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6c241a7cf0 tx=0x7f6c1401e000 comp rx=0 tx=0).stop 2024-03-26T08:48:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.915+0000 7f6c2da24740 1 -- 172.21.15.7:0/2842213845 shutdown_connections 2024-03-26T08:48:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.915+0000 7f6c2da24740 1 --2- 172.21.15.7:0/2842213845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea3fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.915+0000 7f6c2da24740 1 --2- 172.21.15.7:0/2842213845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea403310 0x5642ea414060 secure :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6c241a7cf0 tx=0x7f6c1401e000 comp rx=0 tx=0).stop 2024-03-26T08:48:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.915+0000 7f6c2da24740 1 --2- 172.21.15.7:0/2842213845 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea3fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.915+0000 7f6c2da24740 1 -- 172.21.15.7:0/2842213845 >> 172.21.15.7:0/2842213845 conn(0x5642ea3fde90 msgr2=0x5642ea404670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.917+0000 7f6c2da24740 1 -- 172.21.15.7:0/2842213845 shutdown_connections 2024-03-26T08:48:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.917+0000 7f6c2da24740 1 -- 172.21.15.7:0/2842213845 wait complete. 2024-03-26T08:48:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.918+0000 7f6c2da24740 1 -- start start 2024-03-26T08:48:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.918+0000 7f6c2da24740 1 Processor -- start 2024-03-26T08:48:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea411240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea411780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c2da24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea412790 0x5642ea412c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c2da24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241b6050 con 0x5642ea412790 2024-03-26T08:48:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea411780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c28af7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea411240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c28af7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea411240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:37378/0 (socket says 172.21.15.7:37378) 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea411780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55722/0 (socket says 172.21.15.7:55722) 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c28af7640 1 -- 172.21.15.7:0/3788868529 learned_addr learned my addr 172.21.15.7:0/3788868529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241b7370 con 0x5642ea408480 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241b66b0 con 0x5642ea402920 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c292f8640 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea412790 0x5642ea412c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c2da24740 10 client.0 _ll_register_callbacks cb 0x7f6c1c005c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c292f8640 1 -- 172.21.15.7:0/3788868529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 msgr2=0x5642ea411240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c292f8640 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea411240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c292f8640 1 -- 172.21.15.7:0/3788868529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 msgr2=0x5642ea411780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c292f8640 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea411780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.919+0000 7f6c292f8640 1 -- 172.21.15.7:0/3788868529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642ea413150 con 0x5642ea412790 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c23fff640 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea411780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c28af7640 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea411240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c292f8640 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea412790 0x5642ea412c10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d720 tx=0x7f6c0c09b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d44301fba575d75 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c03fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c03fff640 1 -- 172.21.15.7:0/3788868529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c020040 con 0x5642ea412790 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c03fff640 1 -- 172.21.15.7:0/3788868529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c0c0974c0 con 0x5642ea412790 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c14004040 con 0x5642ea412790 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c03fff640 1 -- 172.21.15.7:0/3788868529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c0c09efa0 con 0x5642ea412790 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c2da24740 10 client.16890 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5642ea474590 con 0x5642ea412790 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c03fff640 1 -- 172.21.15.7:0/3788868529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c14004040 con 0x5642ea412790 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.920+0000 7f6c03fff640 1 -- 172.21.15.7:0/3788868529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6c0c088c20 con 0x5642ea412790 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.921+0000 7f6c2da24740 10 client.16890 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.921+0000 7f6c2da24740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.921+0000 7f6c2da24740 20 client.16890 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.921+0000 7f6c2da24740 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5642ea474900 0x5642ea476d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.921+0000 7f6c2da24740 4 client.16890 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:48:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.921+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5642ea36f3a0 con 0x5642ea474900 2024-03-26T08:48:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.921+0000 7f6c28af7640 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5642ea474900 0x5642ea476d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.921+0000 7f6c03fff640 10 client.16890 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.921+0000 7f6c28af7640 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5642ea474900 0x5642ea476d20 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b10984210b688f6 server_cookie=e67b7675cd521d04 in_seq=0 out_seq=0 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.939+0000 7f6c03fff640 1 -- 172.21.15.7:0/3788868529 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5642ea36f3a0 con 0x5642ea474900 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.939+0000 7f6c03fff640 10 client.16890 handle_command_reply: tid=0 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.939+0000 7f6c2da24740 1 client.16890 shutdown 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.939+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea412790 msgr2=0x5642ea412c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.939+0000 7f6c2da24740 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea412790 0x5642ea412c10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d720 tx=0x7f6c0c09b6c0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 shutdown_connections 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2da24740 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5642ea474900 0x5642ea476d20 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2da24740 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5642ea412790 0x5642ea412c10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c03fff640 0 client.16890 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2da24740 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ea408480 0x5642ea411780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2da24740 1 --2- 172.21.15.7:0/3788868529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5642ea402920 0x5642ea411240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 >> 172.21.15.7:0/3788868529 conn(0x5642ea3fde90 msgr2=0x5642ea3fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 shutdown_connections 2024-03-26T08:48:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2da24740 1 -- 172.21.15.7:0/3788868529 wait complete. 2024-03-26T08:48:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2da24740 20 client.16890 trim_cache size 0 max 16384 2024-03-26T08:48:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:48:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:48:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:48:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:48:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3208222551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c100a2870 msgr2=0x7f6c100a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c100a2870 0x7f6c100a4d30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6c140098d0 tx=0x7f6c14009d50 comp rx=0 tx=0).stop 2024-03-26T08:48:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3208222551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c2410d9c0 msgr2=0x7f6c241add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.940+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c2410d9c0 0x7f6c241add90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00d750 tx=0x7f6c0c00dc10 comp rx=0 tx=0).stop 2024-03-26T08:48:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.941+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3208222551 shutdown_connections 2024-03-26T08:48:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.941+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6c100a2870 0x7f6c100a4d30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.941+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6c2410d9c0 0x7f6c241add90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.941+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c2410b010 0x7f6c241a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.941+0000 7f6c2ad5b640 1 --2- 172.21.15.7:0/3208222551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c241086a0 0x7f6c241a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.941+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3208222551 >> 172.21.15.7:0/3208222551 conn(0x7f6c2406e6b0 msgr2=0x7f6c2410b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.941+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3208222551 shutdown_connections 2024-03-26T08:48:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:12.941+0000 7f6c2ad5b640 1 -- 172.21.15.7:0/3208222551 wait complete. 2024-03-26T08:48:14.118 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:13 smithi044 ceph-mon[36461]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 909 op/s 2024-03-26T08:48:14.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:13 smithi016 ceph-mon[35851]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 909 op/s 2024-03-26T08:48:14.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:13 smithi007 ceph-mon[32462]: pgmap v534: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 909 op/s 2024-03-26T08:48:16.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 345.0 s, 3451.4 tps, lat 29.090 ms stddev 32.515 2024-03-26T08:48:16.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:15 smithi016 ceph-mon[35851]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.8 MiB/s wr, 824 op/s 2024-03-26T08:48:16.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:15 smithi007 ceph-mon[32462]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.8 MiB/s wr, 824 op/s 2024-03-26T08:48:16.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:15 smithi044 ceph-mon[36461]: pgmap v535: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.8 MiB/s wr, 824 op/s 2024-03-26T08:48:16.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:48:16.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:48:16.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.659+0000 7f42de559640 1 -- 172.21.15.7:0/2715242391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 msgr2=0x7f42d8076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.659+0000 7f42de559640 1 --2- 172.21.15.7:0/2715242391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 0x7f42d8076480 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009900 tx=0x7f42cc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:16.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.659+0000 7f42de559640 1 -- 172.21.15.7:0/2715242391 shutdown_connections 2024-03-26T08:48:16.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.659+0000 7f42de559640 1 --2- 172.21.15.7:0/2715242391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d8077c40 0x7f42d80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.659+0000 7f42de559640 1 --2- 172.21.15.7:0/2715242391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80772a0 0x7f42d8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.659+0000 7f42de559640 1 --2- 172.21.15.7:0/2715242391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 0x7f42d8076480 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.659+0000 7f42de559640 1 -- 172.21.15.7:0/2715242391 >> 172.21.15.7:0/2715242391 conn(0x7f42d80fee60 msgr2=0x7f42d81012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.660+0000 7f42de559640 1 -- 172.21.15.7:0/2715242391 shutdown_connections 2024-03-26T08:48:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.660+0000 7f42de559640 1 -- 172.21.15.7:0/2715242391 wait complete. 2024-03-26T08:48:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.660+0000 7f42de559640 1 Processor -- start 2024-03-26T08:48:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42de559640 1 -- start start 2024-03-26T08:48:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42de559640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 0x7f42d819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42de559640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80772a0 0x7f42d819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42de559640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d8077c40 0x7f42d81a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42de559640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8113980 con 0x7f42d80772a0 2024-03-26T08:48:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42de559640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8113800 con 0x7f42d8077c40 2024-03-26T08:48:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42de559640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8113b00 con 0x7f42d8076060 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 0x7f42d819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 0x7f42d819e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:48924/0 (socket says 172.21.15.7:48924) 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.661+0000 7f42d7fff640 1 -- 172.21.15.7:0/334009848 learned_addr learned my addr 172.21.15.7:0/334009848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.662+0000 7f42d7fff640 1 -- 172.21.15.7:0/334009848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d8077c40 msgr2=0x7f42d81a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.662+0000 7f42d7fff640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d8077c40 0x7f42d81a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.662+0000 7f42d7fff640 1 -- 172.21.15.7:0/334009848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80772a0 msgr2=0x7f42d819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.662+0000 7f42d7fff640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80772a0 0x7f42d819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.662+0000 7f42d7fff640 1 -- 172.21.15.7:0/334009848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d81a5cf0 con 0x7f42d8076060 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.662+0000 7f42d7fff640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 0x7f42d819e0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009230 tx=0x7f42cc005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.662+0000 7f42d57fa640 1 -- 172.21.15.7:0/334009848 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc041070 con 0x7f42d8076060 2024-03-26T08:48:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.662+0000 7f42d57fa640 1 -- 172.21.15.7:0/334009848 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42cc0044d0 con 0x7f42d8076060 2024-03-26T08:48:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.662+0000 7f42d57fa640 1 -- 172.21.15.7:0/334009848 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc034440 con 0x7f42d8076060 2024-03-26T08:48:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.663+0000 7f42de559640 1 -- 172.21.15.7:0/334009848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d81a5f80 con 0x7f42d8076060 2024-03-26T08:48:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.663+0000 7f42de559640 1 -- 172.21.15.7:0/334009848 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d81a6390 con 0x7f42d8076060 2024-03-26T08:48:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.665+0000 7f42e1211740 1 Processor -- start 2024-03-26T08:48:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.666+0000 7f42d57fa640 1 -- 172.21.15.7:0/334009848 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f42cc044070 con 0x7f42d8076060 2024-03-26T08:48:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.666+0000 7f42e1211740 1 -- start start 2024-03-26T08:48:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.666+0000 7f42e1211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 0x55f2c363f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.666+0000 7f42e1211740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c363f790 0x55f2c3644290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.666+0000 7f42e1211740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c3644900 0x55f2c3646d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.666+0000 7f42e1211740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2c35a43d0 con 0x55f2c363f790 2024-03-26T08:48:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.666+0000 7f42e1211740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2c354cad0 con 0x55f2c3644900 2024-03-26T08:48:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.666+0000 7f42e1211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2c35a0470 con 0x55f2c363eda0 2024-03-26T08:48:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.667+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c363f790 0x55f2c3644290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.667+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c363f790 0x55f2c3644290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36192/0 (socket says 172.21.15.7:36192) 2024-03-26T08:48:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.667+0000 7f42d7fff640 1 -- 172.21.15.7:0/869883827 learned_addr learned my addr 172.21.15.7:0/869883827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.667+0000 7f42d77fe640 1 --2- 172.21.15.7:0/869883827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c3644900 0x55f2c3646d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.667+0000 7f42dcaf6640 1 --2- 172.21.15.7:0/869883827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 0x55f2c363f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42d57fa640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42c00a2950 0x7f42c00a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42d7fff640 1 -- 172.21.15.7:0/869883827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 msgr2=0x55f2c363f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42d7fff640 1 --2- 172.21.15.7:0/869883827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 0x55f2c363f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42d7fff640 1 -- 172.21.15.7:0/869883827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c3644900 msgr2=0x55f2c3646d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42d7fff640 1 --2- 172.21.15.7:0/869883827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c3644900 0x55f2c3646d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42d7fff640 1 -- 172.21.15.7:0/869883827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2c33827d0 con 0x55f2c363f790 2024-03-26T08:48:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42d57fa640 1 -- 172.21.15.7:0/334009848 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42cc0f0620 con 0x7f42d8076060 2024-03-26T08:48:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42dcaf6640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42c00a2950 0x7f42c00a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42d7fff640 1 --2- 172.21.15.7:0/869883827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c363f790 0x55f2c3644290 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0c58e0 tx=0x7f42cc0b9050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2249de0d35fb7d3a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.668+0000 7f42be7fc640 1 -- 172.21.15.7:0/869883827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc04b070 con 0x55f2c363f790 2024-03-26T08:48:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.669+0000 7f42be7fc640 1 -- 172.21.15.7:0/869883827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42cc0c5720 con 0x55f2c363f790 2024-03-26T08:48:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.669+0000 7f42be7fc640 1 -- 172.21.15.7:0/869883827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0bcac0 con 0x55f2c363f790 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 -- 172.21.15.7:0/869883827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c363f790 msgr2=0x55f2c3644290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 --2- 172.21.15.7:0/869883827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c363f790 0x55f2c3644290 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0c58e0 tx=0x7f42cc0b9050 comp rx=0 tx=0).stop 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42dcaf6640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42c00a2950 0x7f42c00a4e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55f2c363fff0 tx=0x7f42c80046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 -- 172.21.15.7:0/869883827 shutdown_connections 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 --2- 172.21.15.7:0/869883827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c3644900 0x55f2c3646d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 --2- 172.21.15.7:0/869883827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c363f790 0x55f2c3644290 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 --2- 172.21.15.7:0/869883827 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 0x55f2c363f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 -- 172.21.15.7:0/869883827 >> 172.21.15.7:0/869883827 conn(0x55f2c363a310 msgr2=0x55f2c363a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 -- 172.21.15.7:0/869883827 shutdown_connections 2024-03-26T08:48:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 -- 172.21.15.7:0/869883827 wait complete. 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.670+0000 7f42e1211740 1 -- start start 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42e1211740 1 Processor -- start 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42e1211740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c3659ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42e1211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 0x55f2c365a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42e1211740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3661470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42e1211740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2c3658680 con 0x55f2c3644900 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42e1211740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2c3560060 con 0x55f2c363eda0 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42e1211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2c3658800 con 0x55f2c363f790 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42e1211740 10 client.0 _ll_register_callbacks cb 0x55f2c35426a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c3659ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c3659ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51224/0 (socket says 172.21.15.7:51224) 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42d7fff640 1 -- 172.21.15.7:0/1538627266 learned_addr learned my addr 172.21.15.7:0/1538627266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42d77fe640 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 0x55f2c365a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42dcaf6640 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3661470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42d7fff640 1 -- 172.21.15.7:0/1538627266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 msgr2=0x55f2c365a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42d7fff640 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 0x55f2c365a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42d7fff640 1 -- 172.21.15.7:0/1538627266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 msgr2=0x55f2c3661470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42d7fff640 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3661470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42d7fff640 1 -- 172.21.15.7:0/1538627266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2c32b83f0 con 0x55f2c363eda0 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.671+0000 7f42dcaf6640 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3661470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.672+0000 7f42d7fff640 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c3659ef0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0b93e0 tx=0x7f42cc02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a365935436906e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.672+0000 7f42b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.672+0000 7f42b77fe640 1 -- 172.21.15.7:0/1538627266 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc04b070 con 0x55f2c363eda0 2024-03-26T08:48:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.672+0000 7f42b77fe640 1 -- 172.21.15.7:0/1538627266 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42cc0b8cd0 con 0x55f2c363eda0 2024-03-26T08:48:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.672+0000 7f42e1211740 1 -- 172.21.15.7:0/1538627266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2c35b1910 con 0x55f2c363eda0 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.673+0000 7f42b77fe640 1 -- 172.21.15.7:0/1538627266 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42cc0c5290 con 0x55f2c363eda0 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.673+0000 7f42e1211740 10 client.26723 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.673+0000 7f42e1211740 1 -- 172.21.15.7:0/1538627266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f2c32780f0 con 0x55f2c363eda0 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.674+0000 7f42b77fe640 1 -- 172.21.15.7:0/1538627266 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f2c35b1910 con 0x55f2c363eda0 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.674+0000 7f42b77fe640 1 -- 172.21.15.7:0/1538627266 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f42cc0bd1e0 con 0x55f2c363eda0 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.674+0000 7f42e1211740 10 client.26723 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.674+0000 7f42e1211740 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.674+0000 7f42e1211740 20 client.26723 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.674+0000 7f42e1211740 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f2c36655d0 0x55f2c36679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.674+0000 7f42e1211740 4 client.26723 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.674+0000 7f42e1211740 1 -- 172.21.15.7:0/1538627266 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2c35abc60 con 0x55f2c36655d0 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.675+0000 7f42d77fe640 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f2c36655d0 0x55f2c36679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.676+0000 7f42b77fe640 10 client.26723 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.676+0000 7f42d77fe640 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f2c36655d0 0x55f2c36679e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02392dbbe721b86 server_cookie=125666e0039eb136 in_seq=0 out_seq=0 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.676+0000 7f42b77fe640 1 -- 172.21.15.7:0/1538627266 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f2c35abc60 con 0x55f2c36655d0 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.676+0000 7f42b77fe640 10 client.26723 handle_command_reply: tid=0 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 client.26723 shutdown 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 -- 172.21.15.7:0/1538627266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 msgr2=0x55f2c3659ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c3659ef0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0b93e0 tx=0x7f42cc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 -- 172.21.15.7:0/1538627266 shutdown_connections 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f2c36655d0 0x55f2c36679e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3661470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42b77fe640 0 client.26723 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 0x55f2c365a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 --2- 172.21.15.7:0/1538627266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c3659ef0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 -- 172.21.15.7:0/1538627266 >> 172.21.15.7:0/1538627266 conn(0x55f2c363a310 msgr2=0x55f2c3639530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 -- 172.21.15.7:0/1538627266 shutdown_connections 2024-03-26T08:48:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 1 -- 172.21.15.7:0/1538627266 wait complete. 2024-03-26T08:48:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.677+0000 7f42e1211740 20 client.26723 trim_cache size 0 max 16384 2024-03-26T08:48:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.704+0000 7f42e1211740 1 Processor -- start 2024-03-26T08:48:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.704+0000 7f42e1211740 1 -- start start 2024-03-26T08:48:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42e1211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 0x55f2c3638ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42e1211740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363f790 0x55f2c3661400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42e1211740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c36391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42e1211740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d81a8fc0 con 0x55f2c3644900 2024-03-26T08:48:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42e1211740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d81a6ac0 con 0x55f2c363f790 2024-03-26T08:48:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42e1211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8071d50 con 0x55f2c363eda0 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363f790 0x55f2c3661400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363f790 0x55f2c3661400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51230/0 (socket says 172.21.15.7:51230) 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d77fe640 1 -- 172.21.15.7:0/1138941920 learned_addr learned my addr 172.21.15.7:0/1138941920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42dcaf6640 1 --2- 172.21.15.7:0/1138941920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c36391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d7fff640 1 --2- 172.21.15.7:0/1138941920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 0x55f2c3638ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d77fe640 1 -- 172.21.15.7:0/1138941920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 msgr2=0x55f2c3638ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d77fe640 1 --2- 172.21.15.7:0/1138941920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 0x55f2c3638ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d77fe640 1 -- 172.21.15.7:0/1138941920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 msgr2=0x55f2c36391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d77fe640 1 --2- 172.21.15.7:0/1138941920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c36391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d77fe640 1 -- 172.21.15.7:0/1138941920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2c3643e30 con 0x55f2c363f790 2024-03-26T08:48:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.705+0000 7f42d77fe640 1 --2- 172.21.15.7:0/1138941920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363f790 0x55f2c3661400 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009810 tx=0x7f42d0011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9032fe944705f3e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.706+0000 7f42bdffb640 1 -- 172.21.15.7:0/1138941920 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0021020 con 0x55f2c363f790 2024-03-26T08:48:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.706+0000 7f42bdffb640 1 -- 172.21.15.7:0/1138941920 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d000e470 con 0x55f2c363f790 2024-03-26T08:48:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.706+0000 7f42bdffb640 1 -- 172.21.15.7:0/1138941920 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0010680 con 0x55f2c363f790 2024-03-26T08:48:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.706+0000 7f42e1211740 1 -- 172.21.15.7:0/1138941920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363f790 msgr2=0x55f2c3661400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.706+0000 7f42e1211740 1 --2- 172.21.15.7:0/1138941920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363f790 0x55f2c3661400 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009810 tx=0x7f42d0011980 comp rx=0 tx=0).stop 2024-03-26T08:48:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.707+0000 7f42e1211740 1 -- 172.21.15.7:0/1138941920 shutdown_connections 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.707+0000 7f42e1211740 1 --2- 172.21.15.7:0/1138941920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c36391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.707+0000 7f42e1211740 1 --2- 172.21.15.7:0/1138941920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363f790 0x55f2c3661400 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.707+0000 7f42e1211740 1 --2- 172.21.15.7:0/1138941920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363eda0 0x55f2c3638ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.707+0000 7f42e1211740 1 -- 172.21.15.7:0/1138941920 >> 172.21.15.7:0/1138941920 conn(0x55f2c363a310 msgr2=0x55f2c3659b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.707+0000 7f42e1211740 1 -- 172.21.15.7:0/1138941920 shutdown_connections 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.707+0000 7f42e1211740 1 -- 172.21.15.7:0/1138941920 wait complete. 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.707+0000 7f42e1211740 1 -- start start 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.708+0000 7f42e1211740 1 Processor -- start 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.708+0000 7f42e1211740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c365f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.708+0000 7f42e1211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 0x55f2c365f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.708+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c365f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.708+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c365f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51242/0 (socket says 172.21.15.7:51242) 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.708+0000 7f42d7fff640 1 -- 172.21.15.7:0/1958116650 learned_addr learned my addr 172.21.15.7:0/1958116650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42d77fe640 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 0x55f2c365f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42e1211740 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3660800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8072350 con 0x55f2c3644900 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8073670 con 0x55f2c363eda0 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80729b0 con 0x55f2c363f790 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42e1211740 10 client.0 _ll_register_callbacks cb 0x7f42c8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42dcaf6640 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3660800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42d7fff640 1 -- 172.21.15.7:0/1958116650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 msgr2=0x55f2c365f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42d7fff640 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 0x55f2c365f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42d7fff640 1 -- 172.21.15.7:0/1958116650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 msgr2=0x55f2c3660800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42d7fff640 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3660800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42d7fff640 1 -- 172.21.15.7:0/1958116650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2c3660df0 con 0x55f2c363eda0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.709+0000 7f42dcaf6640 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3660800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.710+0000 7f42d7fff640 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c365f350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0c5430 tx=0x7f42cc0ee410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2960dc4e8788ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.710+0000 7f42b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.710+0000 7f42b77fe640 1 -- 172.21.15.7:0/1958116650 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc04b070 con 0x55f2c363eda0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.710+0000 7f42b77fe640 1 -- 172.21.15.7:0/1958116650 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42cc0bc3d0 con 0x55f2c363eda0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42d000e470 con 0x55f2c363eda0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42b77fe640 1 -- 172.21.15.7:0/1958116650 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42cc0bc8e0 con 0x55f2c363eda0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42e1211740 10 client.26729 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f2c3660df0 con 0x55f2c363eda0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42b77fe640 1 -- 172.21.15.7:0/1958116650 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d000e470 con 0x55f2c363eda0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42b77fe640 1 -- 172.21.15.7:0/1958116650 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f42cc0035f0 con 0x55f2c363eda0 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42e1211740 10 client.26729 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42e1211740 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42e1211740 20 client.26729 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42e1211740 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f2c36c1a10 0x55f2c36c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42e1211740 4 client.26729 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:48:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f2c35294e0 con 0x55f2c36c1a10 2024-03-26T08:48:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.711+0000 7f42d77fe640 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f2c36c1a10 0x55f2c36c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.712+0000 7f42b77fe640 10 client.26729 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:16.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.712+0000 7f42d77fe640 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f2c36c1a10 0x55f2c36c3e30 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6120691a29285b63 server_cookie=a0b2a7ba5f7b5bca in_seq=0 out_seq=0 2024-03-26T08:48:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.712+0000 7f42b77fe640 1 -- 172.21.15.7:0/1958116650 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f2c35294e0 con 0x55f2c36c1a10 2024-03-26T08:48:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.712+0000 7f42b77fe640 10 client.26729 handle_command_reply: tid=0 2024-03-26T08:48:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.712+0000 7f42e1211740 1 client.26729 shutdown 2024-03-26T08:48:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 msgr2=0x55f2c365f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c365f350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0c5430 tx=0x7f42cc0ee410 comp rx=0 tx=0).stop 2024-03-26T08:48:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 shutdown_connections 2024-03-26T08:48:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f2c36c1a10 0x55f2c36c3e30 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2c3644900 0x55f2c3660800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42b77fe640 0 client.26729 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2c363f790 0x55f2c365f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 --2- 172.21.15.7:0/1958116650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2c363eda0 0x55f2c365f350 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 >> 172.21.15.7:0/1958116650 conn(0x55f2c363a310 msgr2=0x55f2c363d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 shutdown_connections 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 1 -- 172.21.15.7:0/1958116650 wait complete. 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.714+0000 7f42e1211740 20 client.26729 trim_cache size 0 max 16384 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:48:16.709 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:48:16.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 -- 172.21.15.7:0/334009848 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42c00a2950 msgr2=0x7f42c00a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42c00a2950 0x7f42c00a4e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55f2c363fff0 tx=0x7f42c80046c0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 -- 172.21.15.7:0/334009848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 msgr2=0x7f42d819e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 0x7f42d819e0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009230 tx=0x7f42cc005d50 comp rx=0 tx=0).stop 2024-03-26T08:48:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 -- 172.21.15.7:0/334009848 shutdown_connections 2024-03-26T08:48:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42c00a2950 0x7f42c00a4e10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42d8077c40 0x7f42d81a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80772a0 0x7f42d819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 --2- 172.21.15.7:0/334009848 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42d8076060 0x7f42d819e0d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 -- 172.21.15.7:0/334009848 >> 172.21.15.7:0/334009848 conn(0x7f42d80fee60 msgr2=0x7f42d8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 -- 172.21.15.7:0/334009848 shutdown_connections 2024-03-26T08:48:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.715+0000 7f42de559640 1 -- 172.21.15.7:0/334009848 wait complete. 2024-03-26T08:48:16.724 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:48:16.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:48:16.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.890+0000 7f904037e640 1 -- 172.21.15.7:0/2246547965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90381032e0 msgr2=0x7f9038109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.890+0000 7f904037e640 1 --2- 172.21.15.7:0/2246547965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90381032e0 0x7f9038109b70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9030009be0 tx=0x7f903002bef0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.890+0000 7f904037e640 1 -- 172.21.15.7:0/2246547965 shutdown_connections 2024-03-26T08:48:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.890+0000 7f904037e640 1 --2- 172.21.15.7:0/2246547965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90381032e0 0x7f9038109b70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.890+0000 7f904037e640 1 --2- 172.21.15.7:0/2246547965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9038102920 0x7f9038102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.890+0000 7f904037e640 1 --2- 172.21.15.7:0/2246547965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9038101730 0x7f9038101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.890+0000 7f904037e640 1 -- 172.21.15.7:0/2246547965 >> 172.21.15.7:0/2246547965 conn(0x7f90380fccc0 msgr2=0x7f90380ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.890+0000 7f904037e640 1 -- 172.21.15.7:0/2246547965 shutdown_connections 2024-03-26T08:48:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.890+0000 7f904037e640 1 -- 172.21.15.7:0/2246547965 wait complete. 2024-03-26T08:48:16.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.891+0000 7f904037e640 1 Processor -- start 2024-03-26T08:48:16.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.891+0000 7f904037e640 1 -- start start 2024-03-26T08:48:16.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.891+0000 7f904037e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9038101730 0x7f90381101e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.891+0000 7f904037e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9038102920 0x7f9038110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f904037e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90381032e0 0x7f9038117790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f904037e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038075270 con 0x7f90381032e0 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f904037e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90380750f0 con 0x7f9038102920 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f904037e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90380753f0 con 0x7f9038101730 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903d919640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9038102920 0x7f9038110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903e11a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9038101730 0x7f90381101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903d919640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9038102920 0x7f9038110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51260/0 (socket says 172.21.15.7:51260) 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903d919640 1 -- 172.21.15.7:0/1783992142 learned_addr learned my addr 172.21.15.7:0/1783992142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903e91b640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90381032e0 0x7f9038117790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903d919640 1 -- 172.21.15.7:0/1783992142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9038101730 msgr2=0x7f90381101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903d919640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9038101730 0x7f90381101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903d919640 1 -- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90381032e0 msgr2=0x7f9038117790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903d919640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90381032e0 0x7f9038117790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903d919640 1 -- 172.21.15.7:0/1783992142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9038117e00 con 0x7f9038102920 2024-03-26T08:48:16.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.892+0000 7f903d919640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9038102920 0x7f9038110720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9028030a00 tx=0x7f9028030e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.893+0000 7f902f7fe640 1 -- 172.21.15.7:0/1783992142 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902802ecd0 con 0x7f9038102920 2024-03-26T08:48:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.893+0000 7f902f7fe640 1 -- 172.21.15.7:0/1783992142 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9028032920 con 0x7f9038102920 2024-03-26T08:48:16.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.893+0000 7f902f7fe640 1 -- 172.21.15.7:0/1783992142 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902803a8c0 con 0x7f9038102920 2024-03-26T08:48:16.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.893+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90381180f0 con 0x7f9038102920 2024-03-26T08:48:16.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.893+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903819db20 con 0x7f9038102920 2024-03-26T08:48:16.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.894+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903819de40 con 0x7f9038102920 2024-03-26T08:48:16.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.895+0000 7f902f7fe640 1 -- 172.21.15.7:0/1783992142 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9028032ad0 con 0x7f9038102920 2024-03-26T08:48:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.896+0000 7f902f7fe640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90240a2900 0x7f90240a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:16.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.896+0000 7f902f7fe640 1 -- 172.21.15.7:0/1783992142 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f90280f0e40 con 0x7f9038102920 2024-03-26T08:48:16.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.897+0000 7f903e11a640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90240a2900 0x7f90240a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:16.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.897+0000 7f903e11a640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90240a2900 0x7f90240a4dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9038102780 tx=0x7f9034009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:16.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:16.901+0000 7f902f7fe640 1 -- 172.21.15.7:0/1783992142 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f90280bdea0 con 0x7f9038102920 2024-03-26T08:48:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.265+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f903819e010 con 0x7f9038102920 2024-03-26T08:48:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.268+0000 7f902f7fe640 1 -- 172.21.15.7:0/1783992142 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f90280bd7a0 con 0x7f9038102920 2024-03-26T08:48:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:48:17.261 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:48:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:48:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90240a2900 msgr2=0x7f90240a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90240a2900 0x7f90240a4dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9038102780 tx=0x7f9034009260 comp rx=0 tx=0).stop 2024-03-26T08:48:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9038102920 msgr2=0x7f9038110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9038102920 0x7f9038110720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9028030a00 tx=0x7f9028030e00 comp rx=0 tx=0).stop 2024-03-26T08:48:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 shutdown_connections 2024-03-26T08:48:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f90240a2900 0x7f90240a4dc0 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9038102780 tx=0x7f9034009260 comp rx=0 tx=0).stop 2024-03-26T08:48:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90381032e0 0x7f9038117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9038102920 0x7f9038110720 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9028030a00 tx=0x7f9028030e00 comp rx=0 tx=0).stop 2024-03-26T08:48:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 --2- 172.21.15.7:0/1783992142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9038101730 0x7f90381101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.271+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 >> 172.21.15.7:0/1783992142 conn(0x7f90380fccc0 msgr2=0x7f903810dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.272+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 shutdown_connections 2024-03-26T08:48:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.273+0000 7f904037e640 1 -- 172.21.15.7:0/1783992142 wait complete. 2024-03-26T08:48:17.283 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:48:17.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:48:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.453+0000 7ff2f8b09640 1 -- 172.21.15.7:0/3134583772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4107cc0 msgr2=0x7ff2f41080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.453+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/3134583772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4107cc0 0x7ff2f41080e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009900 tx=0x7ff2e402bc80 comp rx=0 tx=0).stop 2024-03-26T08:48:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.456+0000 7ff2f8b09640 1 -- 172.21.15.7:0/3134583772 shutdown_connections 2024-03-26T08:48:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.456+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/3134583772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4109870 0x7ff2f4110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.456+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/3134583772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4108eb0 0x7ff2f4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.456+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/3134583772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4107cc0 0x7ff2f41080e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.456+0000 7ff2f8b09640 1 -- 172.21.15.7:0/3134583772 >> 172.21.15.7:0/3134583772 conn(0x7ff2f4076c60 msgr2=0x7ff2f40790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.458+0000 7ff2f8b09640 1 -- 172.21.15.7:0/3134583772 shutdown_connections 2024-03-26T08:48:17.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.459+0000 7ff2f8b09640 1 -- 172.21.15.7:0/3134583772 wait complete. 2024-03-26T08:48:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.460+0000 7ff2f8b09640 1 Processor -- start 2024-03-26T08:48:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.460+0000 7ff2f8b09640 1 -- start start 2024-03-26T08:48:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f8b09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4107cc0 0x7ff2f4072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f8b09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4108eb0 0x7ff2f4073190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f8b09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4109870 0x7ff2f4071290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f8b09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f406bb10 con 0x7ff2f4109870 2024-03-26T08:48:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f8b09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f406b990 con 0x7ff2f4108eb0 2024-03-26T08:48:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f8b09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f406bc90 con 0x7ff2f4107cc0 2024-03-26T08:48:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4109870 0x7ff2f4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4109870 0x7ff2f4071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36246/0 (socket says 172.21.15.7:36246) 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f2d9d640 1 -- 172.21.15.7:0/1026866403 learned_addr learned my addr 172.21.15.7:0/1026866403 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f259c640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4107cc0 0x7ff2f4072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4108eb0 0x7ff2f4073190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f2d9d640 1 -- 172.21.15.7:0/1026866403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4107cc0 msgr2=0x7ff2f4072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.461+0000 7ff2f2d9d640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4107cc0 0x7ff2f4072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.462+0000 7ff2f2d9d640 1 -- 172.21.15.7:0/1026866403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4108eb0 msgr2=0x7ff2f4073190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.462+0000 7ff2f2d9d640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4108eb0 0x7ff2f4073190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.462+0000 7ff2f2d9d640 1 -- 172.21.15.7:0/1026866403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4071a90 con 0x7ff2f4109870 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.462+0000 7ff2f2d9d640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4109870 0x7ff2f4071290 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800b3f0 tx=0x7ff2e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.462+0000 7ff2e37fe640 1 -- 172.21.15.7:0/1026866403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8015020 con 0x7ff2f4109870 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.462+0000 7ff2e37fe640 1 -- 172.21.15.7:0/1026866403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e8011d10 con 0x7ff2f4109870 2024-03-26T08:48:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.462+0000 7ff2e37fe640 1 -- 172.21.15.7:0/1026866403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e800f2e0 con 0x7ff2f4109870 2024-03-26T08:48:17.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.463+0000 7ff2f8b09640 1 -- 172.21.15.7:0/1026866403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f4071d80 con 0x7ff2f4109870 2024-03-26T08:48:17.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.463+0000 7ff2f8b09640 1 -- 172.21.15.7:0/1026866403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f41aed80 con 0x7ff2f4109870 2024-03-26T08:48:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.466+0000 7ff2fb7d1740 1 Processor -- start 2024-03-26T08:48:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.467+0000 7ff2e37fe640 1 -- 172.21.15.7:0/1026866403 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff2e8018020 con 0x7ff2f4109870 2024-03-26T08:48:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.467+0000 7ff2fb7d1740 1 -- start start 2024-03-26T08:48:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 0x5610587cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ce080 0x5610587d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561058790610 con 0x5610587d31f0 2024-03-26T08:48:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610587333c0 con 0x5610587ce080 2024-03-26T08:48:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610586dbd60 con 0x5610587cd690 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ce080 0x5610587d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587d5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36256/0 (socket says 172.21.15.7:36256) 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ce080 0x5610587d2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51300/0 (socket says 172.21.15.7:51300) 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f1d9b640 1 -- 172.21.15.7:0/2233702812 learned_addr learned my addr 172.21.15.7:0/2233702812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/2233702812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 0x5610587cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f259c640 1 -- 172.21.15.7:0/2233702812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 msgr2=0x5610587cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f259c640 1 --2- 172.21.15.7:0/2233702812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 0x5610587cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f259c640 1 -- 172.21.15.7:0/2233702812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 msgr2=0x5610587d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f259c640 1 --2- 172.21.15.7:0/2233702812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587d5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f259c640 1 -- 172.21.15.7:0/2233702812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610584473f0 con 0x5610587ce080 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.468+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/2233702812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.469+0000 7ff2e37fe640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2dc0a2980 0x7ff2dc0a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.469+0000 7ff2f259c640 1 --2- 172.21.15.7:0/2233702812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ce080 0x5610587d2b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4036040 tx=0x7ff2e4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e194b6fa4fec13 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.469+0000 7ff2e0ff9640 1 -- 172.21.15.7:0/2233702812 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4042070 con 0x5610587ce080 2024-03-26T08:48:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.469+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2dc0a2980 0x7ff2dc0a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.469+0000 7ff2e0ff9640 1 -- 172.21.15.7:0/2233702812 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e4005040 con 0x5610587ce080 2024-03-26T08:48:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.469+0000 7ff2e0ff9640 1 -- 172.21.15.7:0/2233702812 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4004a40 con 0x5610587ce080 2024-03-26T08:48:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.469+0000 7ff2e37fe640 1 -- 172.21.15.7:0/1026866403 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff2e80ce200 con 0x7ff2f4109870 2024-03-26T08:48:17.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/2233702812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ce080 msgr2=0x5610587d2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/2233702812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ce080 0x5610587d2b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4036040 tx=0x7ff2e4005c00 comp rx=0 tx=0).stop 2024-03-26T08:48:17.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2dc0a2980 0x7ff2dc0a4e40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00a300 tx=0x7ff2ec00a690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/2233702812 shutdown_connections 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/2233702812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/2233702812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ce080 0x5610587d2b80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/2233702812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 0x5610587cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/2233702812 >> 172.21.15.7:0/2233702812 conn(0x5610587c8c00 msgr2=0x5610587c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/2233702812 shutdown_connections 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/2233702812 wait complete. 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.470+0000 7ff2fb7d1740 1 -- start start 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2fb7d1740 1 Processor -- start 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587d8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610586ef240 con 0x5610587d31f0 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56105872f460 con 0x5610587cd690 2024-03-26T08:48:17.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610587d6140 con 0x5610587ce080 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2fb7d1740 10 client.0 _ll_register_callbacks cb 0x5610586eec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587d7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51310/0 (socket says 172.21.15.7:51310) 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f259c640 1 -- 172.21.15.7:0/1417284406 learned_addr learned my addr 172.21.15.7:0/1417284406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f259c640 1 -- 172.21.15.7:0/1417284406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 msgr2=0x5610587d8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f259c640 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587d8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f259c640 1 -- 172.21.15.7:0/1417284406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 msgr2=0x5610587df040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f259c640 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587df040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f259c640 1 -- 172.21.15.7:0/1417284406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610585117d0 con 0x5610587cd690 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f2d9d640 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2f259c640 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587d7ac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009900 tx=0x7ff2e4005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da44c31fc7bd37a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.471+0000 7ff2d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:17.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/1417284406 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e404e070 con 0x5610587cd690 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/1417284406 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff2e4004bd0 con 0x5610587cd690 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/1417284406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2e4005040 con 0x5610587cd690 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/1417284406 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2e405a050 con 0x5610587cd690 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2fb7d1740 10 client.26747 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/1417284406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5610587e23b0 con 0x5610587cd690 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/1417284406 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e4005040 con 0x5610587cd690 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/1417284406 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff2e4034440 con 0x5610587cd690 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2fb7d1740 10 client.26747 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2fb7d1740 10 client.26747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2fb7d1740 20 client.26747 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5610587e3700 0x5610587e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2fb7d1740 4 client.26747 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:17.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/1417284406 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561058740900 con 0x5610587e3700 2024-03-26T08:48:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.472+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5610587e3700 0x5610587e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.473+0000 7ff2d9ffb640 10 client.26747 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:48:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.473+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5610587e3700 0x5610587e5af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c44a2b0bb17aada8 server_cookie=5cc43e9b3cc6598 in_seq=0 out_seq=0 2024-03-26T08:48:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.474+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/1417284406 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561058740900 con 0x5610587e3700 2024-03-26T08:48:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.474+0000 7ff2d9ffb640 10 client.26747 handle_command_reply: tid=0 2024-03-26T08:48:17.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.474+0000 7ff2fb7d1740 1 client.26747 shutdown 2024-03-26T08:48:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/1417284406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 msgr2=0x5610587d7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587d7ac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009900 tx=0x7ff2e4005750 comp rx=0 tx=0).stop 2024-03-26T08:48:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/1417284406 shutdown_connections 2024-03-26T08:48:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5610587e3700 0x5610587e5af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587d8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/1417284406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587d7ac0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/1417284406 >> 172.21.15.7:0/1417284406 conn(0x5610587c8c00 msgr2=0x5610587c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2d9ffb640 0 client.26747 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:48:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/1417284406 shutdown_connections 2024-03-26T08:48:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/1417284406 wait complete. 2024-03-26T08:48:17.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.475+0000 7ff2fb7d1740 20 client.26747 trim_cache size 0 max 16384 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.493+0000 7ff2fb7d1740 1 Processor -- start 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.493+0000 7ff2fb7d1740 1 -- start start 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.493+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.493+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587def60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587ea200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41b9df0 con 0x5610587d31f0 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41b78f0 con 0x5610587cd690 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41b5970 con 0x5610587ce080 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587def60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587def60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49042/0 (socket says 172.21.15.7:49042) 2024-03-26T08:48:17.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f1d9b640 1 -- 172.21.15.7:0/3611150662 learned_addr learned my addr 172.21.15.7:0/3611150662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f2d9d640 1 --2- 172.21.15.7:0/3611150662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587ea200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f1d9b640 1 -- 172.21.15.7:0/3611150662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 msgr2=0x5610587cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/3611150662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f1d9b640 1 -- 172.21.15.7:0/3611150662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 msgr2=0x5610587ea200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/3611150662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587ea200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f1d9b640 1 -- 172.21.15.7:0/3611150662 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610587c9670 con 0x5610587ce080 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f2d9d640 1 --2- 172.21.15.7:0/3611150662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587ea200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.494+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/3611150662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587def60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5610587ce8e0 tx=0x7ff2ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0106327c77de9f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.495+0000 7ff2dbfff640 1 -- 172.21.15.7:0/3611150662 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec012ba0 con 0x5610587ce080 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.495+0000 7ff2dbfff640 1 -- 172.21.15.7:0/3611150662 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec012d40 con 0x5610587ce080 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.495+0000 7ff2dbfff640 1 -- 172.21.15.7:0/3611150662 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec02bde0 con 0x5610587ce080 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/3611150662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 msgr2=0x5610587def60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/3611150662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587def60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5610587ce8e0 tx=0x7ff2ec004a40 comp rx=0 tx=0).stop 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/3611150662 shutdown_connections 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/3611150662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587ea200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/3611150662 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587ce080 0x5610587def60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/3611150662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587cd690 0x5610587cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/3611150662 >> 172.21.15.7:0/3611150662 conn(0x5610587c8c00 msgr2=0x5610587cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/3611150662 shutdown_connections 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/3611150662 wait complete. 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 -- start start 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 Processor -- start 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 0x5610587dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.496+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2fb7d1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ddd00 0x5610587de180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f1d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587dccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36280/0 (socket says 172.21.15.7:36280) 2024-03-26T08:48:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 0x5610587dc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f1d9b640 1 -- 172.21.15.7:0/308505335 learned_addr learned my addr 172.21.15.7:0/308505335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2fb7d1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41b5f70 con 0x5610587d31f0 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41b7290 con 0x5610587ddd00 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41b65d0 con 0x5610587cd690 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2fb7d1740 10 client.0 _ll_register_callbacks cb 0x7ff2e402ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f2d9d640 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ddd00 0x5610587de180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f1d9b640 1 -- 172.21.15.7:0/308505335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 msgr2=0x5610587dc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 0x5610587dc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f1d9b640 1 -- 172.21.15.7:0/308505335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ddd00 msgr2=0x5610587de180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ddd00 0x5610587de180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f1d9b640 1 -- 172.21.15.7:0/308505335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610587de6c0 con 0x5610587d31f0 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f259c640 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 0x5610587dc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f2d9d640 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ddd00 0x5610587de180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2f1d9b640 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587dccf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005c40 tx=0x7ff2ec012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e3ab4fd67c8be0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/308505335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec026040 con 0x5610587d31f0 2024-03-26T08:48:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/308505335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff2ec02b320 con 0x5610587d31f0 2024-03-26T08:48:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.497+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2ec012d40 con 0x5610587d31f0 2024-03-26T08:48:17.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.498+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/308505335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2ec02b560 con 0x5610587d31f0 2024-03-26T08:48:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.498+0000 7ff2fb7d1740 10 client.16920 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.498+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56105883f0a0 con 0x5610587d31f0 2024-03-26T08:48:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.499+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/308505335 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec012d40 con 0x5610587d31f0 2024-03-26T08:48:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.499+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/308505335 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff2ec03f020 con 0x5610587d31f0 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.499+0000 7ff2fb7d1740 10 client.16920 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.499+0000 7ff2fb7d1740 10 client.16920 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.499+0000 7ff2fb7d1740 20 client.16920 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.499+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56105883f980 0x561058841da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.499+0000 7ff2fb7d1740 4 client.16920 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.499+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56105873ac50 con 0x56105883f980 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.499+0000 7ff2f259c640 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56105883f980 0x561058841da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2f259c640 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56105883f980 0x561058841da0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e36bfeb800ac648 server_cookie=b86e4e2141e0848e in_seq=0 out_seq=0 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2d9ffb640 10 client.16920 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2d9ffb640 1 -- 172.21.15.7:0/308505335 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56105873ac50 con 0x56105883f980 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2d9ffb640 10 client.16920 handle_command_reply: tid=0 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 client.16920 shutdown 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 msgr2=0x5610587dccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587dccf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005c40 tx=0x7ff2ec012f70 comp rx=0 tx=0).stop 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 shutdown_connections 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56105883f980 0x561058841da0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5610587ddd00 0x5610587de180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2d9ffb640 0 client.16920 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5610587d31f0 0x5610587dccf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 --2- 172.21.15.7:0/308505335 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5610587cd690 0x5610587dc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 >> 172.21.15.7:0/308505335 conn(0x5610587c8c00 msgr2=0x5610587c7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 shutdown_connections 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 1 -- 172.21.15.7:0/308505335 wait complete. 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.500+0000 7ff2fb7d1740 20 client.16920 trim_cache size 0 max 16384 2024-03-26T08:48:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:48:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 -- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2dc0a2980 msgr2=0x7ff2dc0a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2dc0a2980 0x7ff2dc0a4e40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00a300 tx=0x7ff2ec00a690 comp rx=0 tx=0).stop 2024-03-26T08:48:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 -- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4109870 msgr2=0x7ff2f4071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4109870 0x7ff2f4071290 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800b3f0 tx=0x7ff2e800b8b0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 -- 172.21.15.7:0/1026866403 shutdown_connections 2024-03-26T08:48:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2dc0a2980 0x7ff2dc0a4e40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4109870 0x7ff2f4071290 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4108eb0 0x7ff2f4073190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 --2- 172.21.15.7:0/1026866403 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4107cc0 0x7ff2f4072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 -- 172.21.15.7:0/1026866403 >> 172.21.15.7:0/1026866403 conn(0x7ff2f4076c60 msgr2=0x7ff2f410c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 -- 172.21.15.7:0/1026866403 shutdown_connections 2024-03-26T08:48:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.501+0000 7ff2f8b09640 1 -- 172.21.15.7:0/1026866403 wait complete. 2024-03-26T08:48:17.510 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:48:17.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:48:17.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.685+0000 7f3c8259d640 1 -- 172.21.15.7:0/3890621704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c108cc0 msgr2=0x7f3c7c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.685+0000 7f3c8259d640 1 --2- 172.21.15.7:0/3890621704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c108cc0 0x7f3c7c109140 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009860 tx=0x7f3c7802bbb0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.685+0000 7f3c8259d640 1 -- 172.21.15.7:0/3890621704 shutdown_connections 2024-03-26T08:48:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.685+0000 7f3c8259d640 1 --2- 172.21.15.7:0/3890621704 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c109680 0x7f3c7c10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.685+0000 7f3c8259d640 1 --2- 172.21.15.7:0/3890621704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c108cc0 0x7f3c7c109140 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.685+0000 7f3c8259d640 1 --2- 172.21.15.7:0/3890621704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c7c072a10 0x7f3c7c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.685+0000 7f3c8259d640 1 -- 172.21.15.7:0/3890621704 >> 172.21.15.7:0/3890621704 conn(0x7f3c7c06e4e0 msgr2=0x7f3c7c070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.686+0000 7f3c8259d640 1 -- 172.21.15.7:0/3890621704 shutdown_connections 2024-03-26T08:48:17.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.686+0000 7f3c8259d640 1 -- 172.21.15.7:0/3890621704 wait complete. 2024-03-26T08:48:17.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.687+0000 7f3c8259d640 1 Processor -- start 2024-03-26T08:48:17.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.687+0000 7f3c8259d640 1 -- start start 2024-03-26T08:48:17.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.687+0000 7f3c8259d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c072a10 0x7f3c7c1a6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.687+0000 7f3c8259d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c7c108cc0 0x7f3c7c1a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.687+0000 7f3c8259d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c109680 0x7f3c7c1adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.687+0000 7f3c8259d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c11c010 con 0x7f3c7c108cc0 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.687+0000 7f3c8259d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c11be90 con 0x7f3c7c109680 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.687+0000 7f3c8259d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c11c190 con 0x7f3c7c072a10 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.688+0000 7f3c81d9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c109680 0x7f3c7c1adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.688+0000 7f3c8159b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c072a10 0x7f3c7c1a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.688+0000 7f3c81d9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c109680 0x7f3c7c1adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51350/0 (socket says 172.21.15.7:51350) 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.688+0000 7f3c81d9c640 1 -- 172.21.15.7:0/4133556877 learned_addr learned my addr 172.21.15.7:0/4133556877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.689+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c7c108cc0 0x7f3c7c1a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.690+0000 7f3c80d9a640 1 -- 172.21.15.7:0/4133556877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c072a10 msgr2=0x7f3c7c1a6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.690+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c072a10 0x7f3c7c1a6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.690+0000 7f3c80d9a640 1 -- 172.21.15.7:0/4133556877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c109680 msgr2=0x7f3c7c1adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.690+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c109680 0x7f3c7c1adcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.690+0000 7f3c80d9a640 1 -- 172.21.15.7:0/4133556877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c1ae330 con 0x7f3c7c108cc0 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.690+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c7c108cc0 0x7f3c7c1a6c50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009780 tx=0x7f3c78004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.691+0000 7f3c727fc640 1 -- 172.21.15.7:0/4133556877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78041020 con 0x7f3c7c108cc0 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.691+0000 7f3c8259d640 1 -- 172.21.15.7:0/4133556877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c1ae5c0 con 0x7f3c7c108cc0 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.691+0000 7f3c727fc640 1 -- 172.21.15.7:0/4133556877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c7802ca30 con 0x7f3c7c108cc0 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.691+0000 7f3c727fc640 1 -- 172.21.15.7:0/4133556877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78041020 con 0x7f3c7c108cc0 2024-03-26T08:48:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.691+0000 7f3c8259d640 1 -- 172.21.15.7:0/4133556877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c1aeb50 con 0x7f3c7c108cc0 2024-03-26T08:48:17.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.694+0000 7f3c727fc640 1 -- 172.21.15.7:0/4133556877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3c78044020 con 0x7f3c7c108cc0 2024-03-26T08:48:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.695+0000 7f3c727fc640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c680a2c40 0x7f3c680a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.696+0000 7f3c727fc640 1 -- 172.21.15.7:0/4133556877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c78004460 con 0x7f3c7c108cc0 2024-03-26T08:48:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.696+0000 7f3c8159b640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c680a2c40 0x7f3c680a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c863ed740 1 Processor -- start 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c863ed740 1 -- start start 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c8159b640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c680a2c40 0x7f3c680a5100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009990 tx=0x7f3c6c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c470840 0x564d2c470c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c475d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c4763a0 0x564d2c478830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c863ed740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2c433610 con 0x564d2c4763a0 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c863ed740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2c3d63c0 con 0x564d2c470840 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c863ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2c37ed60 con 0x564d2c471230 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c80d9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c475d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c81d9c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c4763a0 0x564d2c478830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c80d9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c475d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49066/0 (socket says 172.21.15.7:49066) 2024-03-26T08:48:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c81d9c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c4763a0 0x564d2c478830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36318/0 (socket says 172.21.15.7:36318) 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c80d9a640 1 -- 172.21.15.7:0/1340600080 learned_addr learned my addr 172.21.15.7:0/1340600080 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.697+0000 7f3c8159b640 1 --2- 172.21.15.7:0/1340600080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c470840 0x564d2c470c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.698+0000 7f3c80d9a640 1 -- 172.21.15.7:0/1340600080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c470840 msgr2=0x564d2c470c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.698+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/1340600080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c470840 0x564d2c470c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.698+0000 7f3c80d9a640 1 -- 172.21.15.7:0/1340600080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c4763a0 msgr2=0x564d2c478830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.698+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/1340600080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c4763a0 0x564d2c478830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.698+0000 7f3c80d9a640 1 -- 172.21.15.7:0/1340600080 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d2c0ea3f0 con 0x564d2c471230 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.698+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/1340600080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c475d30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3c780b9190 tx=0x7f3c780bc650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=636c7e6e42108406 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.698+0000 7f3c677fe640 1 -- 172.21.15.7:0/1340600080 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78041070 con 0x564d2c471230 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.698+0000 7f3c677fe640 1 -- 172.21.15.7:0/1340600080 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c780bc7c0 con 0x564d2c471230 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.698+0000 7f3c677fe640 1 -- 172.21.15.7:0/1340600080 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78039050 con 0x564d2c471230 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.699+0000 7f3c863ed740 1 -- 172.21.15.7:0/1340600080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 msgr2=0x564d2c475d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.699+0000 7f3c863ed740 1 --2- 172.21.15.7:0/1340600080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c475d30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3c780b9190 tx=0x7f3c780bc650 comp rx=0 tx=0).stop 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.699+0000 7f3c863ed740 1 -- 172.21.15.7:0/1340600080 shutdown_connections 2024-03-26T08:48:17.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.699+0000 7f3c863ed740 1 --2- 172.21.15.7:0/1340600080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c4763a0 0x564d2c478830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.699+0000 7f3c863ed740 1 --2- 172.21.15.7:0/1340600080 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c475d30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.699+0000 7f3c863ed740 1 --2- 172.21.15.7:0/1340600080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c470840 0x564d2c470c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.699+0000 7f3c863ed740 1 -- 172.21.15.7:0/1340600080 >> 172.21.15.7:0/1340600080 conn(0x564d2c46bdb0 msgr2=0x564d2c46c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.699+0000 7f3c863ed740 1 -- 172.21.15.7:0/1340600080 shutdown_connections 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.699+0000 7f3c863ed740 1 -- 172.21.15.7:0/1340600080 wait complete. 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c863ed740 1 -- start start 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c863ed740 1 Processor -- start 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c481d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c863ed740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2c392240 con 0x564d2c470840 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c863ed740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2c3d2460 con 0x564d2c4763a0 2024-03-26T08:48:17.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c863ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d2c479110 con 0x564d2c471230 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c863ed740 10 client.0 _ll_register_callbacks cb 0x564d2c391c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c8159b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c80d9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.700+0000 7f3c80d9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49074/0 (socket says 172.21.15.7:49074) 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c80d9a640 1 -- 172.21.15.7:0/201541486 learned_addr learned my addr 172.21.15.7:0/201541486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c81d9c640 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c481d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c8159b640 1 -- 172.21.15.7:0/201541486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 msgr2=0x564d2c47ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c8159b640 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c8159b640 1 -- 172.21.15.7:0/201541486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 msgr2=0x564d2c481d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c8159b640 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c481d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c8159b640 1 -- 172.21.15.7:0/201541486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d2c1b47d0 con 0x564d2c470840 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c8159b640 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47a7e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009fd0 tx=0x7f3c6c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03cbb8905891929 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:17.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c64ff9640 1 -- 172.21.15.7:0/201541486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c025070 con 0x564d2c470840 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.701+0000 7f3c863ed740 1 -- 172.21.15.7:0/201541486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c780bc7c0 con 0x564d2c470840 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c64ff9640 1 -- 172.21.15.7:0/201541486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c6c0137e0 con 0x564d2c470840 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c64ff9640 1 -- 172.21.15.7:0/201541486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c6c031050 con 0x564d2c470840 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c863ed740 10 client.16929 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c863ed740 1 -- 172.21.15.7:0/201541486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564d2c4851f0 con 0x564d2c470840 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c64ff9640 1 -- 172.21.15.7:0/201541486 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c780bc7c0 con 0x564d2c470840 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c64ff9640 1 -- 172.21.15.7:0/201541486 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3c6c021c50 con 0x564d2c470840 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c863ed740 10 client.16929 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c863ed740 10 client.16929 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c863ed740 20 client.16929 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c863ed740 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564d2c4867f0 0x564d2c488c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c863ed740 4 client.16929 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c863ed740 1 -- 172.21.15.7:0/201541486 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d2c3e3900 con 0x564d2c4867f0 2024-03-26T08:48:17.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.702+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564d2c4867f0 0x564d2c488c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.703+0000 7f3c64ff9640 10 client.16929 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.703+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564d2c4867f0 0x564d2c488c40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21c0dbcf723b1cd server_cookie=2c20cfb98bc8a8af in_seq=0 out_seq=0 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c64ff9640 1 -- 172.21.15.7:0/201541486 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564d2c3e3900 con 0x564d2c4867f0 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c64ff9640 10 client.16929 handle_command_reply: tid=0 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 client.16929 shutdown 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 -- 172.21.15.7:0/201541486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 msgr2=0x564d2c47a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47a7e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009fd0 tx=0x7f3c6c0047d0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 -- 172.21.15.7:0/201541486 shutdown_connections 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c481d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564d2c4867f0 0x564d2c488c40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 --2- 172.21.15.7:0/201541486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47a7e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 -- 172.21.15.7:0/201541486 >> 172.21.15.7:0/201541486 conn(0x564d2c46bdb0 msgr2=0x564d2c46b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c64ff9640 0 client.16929 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:48:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 -- 172.21.15.7:0/201541486 shutdown_connections 2024-03-26T08:48:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.704+0000 7f3c863ed740 1 -- 172.21.15.7:0/201541486 wait complete. 2024-03-26T08:48:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.705+0000 7f3c863ed740 20 client.16929 trim_cache size 0 max 16384 2024-03-26T08:48:17.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.721+0000 7f3c863ed740 1 Processor -- start 2024-03-26T08:48:17.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.721+0000 7f3c863ed740 1 -- start start 2024-03-26T08:48:17.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.721+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c46f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c481cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c46f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c863ed740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c1c2560 con 0x564d2c470840 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c863ed740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c1c0060 con 0x564d2c4763a0 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c863ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c116830 con 0x564d2c471230 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c81d9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c46f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c81d9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c46f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51382/0 (socket says 172.21.15.7:51382) 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c81d9c640 1 -- 172.21.15.7:0/2524277966 learned_addr learned my addr 172.21.15.7:0/2524277966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c8159b640 1 --2- 172.21.15.7:0/2524277966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c46f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/2524277966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c481cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c81d9c640 1 -- 172.21.15.7:0/2524277966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 msgr2=0x564d2c481cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c81d9c640 1 --2- 172.21.15.7:0/2524277966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c481cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c81d9c640 1 -- 172.21.15.7:0/2524277966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 msgr2=0x564d2c46f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c81d9c640 1 --2- 172.21.15.7:0/2524277966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c46f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.722+0000 7f3c81d9c640 1 -- 172.21.15.7:0/2524277966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d2c471610 con 0x564d2c4763a0 2024-03-26T08:48:17.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.723+0000 7f3c81d9c640 1 --2- 172.21.15.7:0/2524277966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c46f550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400e610 tx=0x7f3c740095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ee9f6f3291e5f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.723+0000 7f3c66ffd640 1 -- 172.21.15.7:0/2524277966 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7401d070 con 0x564d2c4763a0 2024-03-26T08:48:17.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.723+0000 7f3c66ffd640 1 -- 172.21.15.7:0/2524277966 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c7400dcf0 con 0x564d2c4763a0 2024-03-26T08:48:17.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.723+0000 7f3c66ffd640 1 -- 172.21.15.7:0/2524277966 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004690 con 0x564d2c4763a0 2024-03-26T08:48:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 -- 172.21.15.7:0/2524277966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 msgr2=0x564d2c46f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 --2- 172.21.15.7:0/2524277966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c46f550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400e610 tx=0x7f3c740095c0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 -- 172.21.15.7:0/2524277966 shutdown_connections 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 --2- 172.21.15.7:0/2524277966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c46f550 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 --2- 172.21.15.7:0/2524277966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c481cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 --2- 172.21.15.7:0/2524277966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c46f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 -- 172.21.15.7:0/2524277966 >> 172.21.15.7:0/2524277966 conn(0x564d2c46bdb0 msgr2=0x564d2c4725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 -- 172.21.15.7:0/2524277966 shutdown_connections 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 -- 172.21.15.7:0/2524277966 wait complete. 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 -- start start 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.724+0000 7f3c863ed740 1 Processor -- start 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c863ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c8159b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c8159b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36334/0 (socket says 172.21.15.7:36334) 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c8159b640 1 -- 172.21.15.7:0/543742673 learned_addr learned my addr 172.21.15.7:0/543742673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c863ed740 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c480840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c116e30 con 0x564d2c470840 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c118150 con 0x564d2c4763a0 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.726+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c117490 con 0x564d2c471230 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c863ed740 10 client.0 _ll_register_callbacks cb 0x7f3c6c00ee10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c81d9c640 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c480840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c8159b640 1 -- 172.21.15.7:0/543742673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 msgr2=0x564d2c47f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c8159b640 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c8159b640 1 -- 172.21.15.7:0/543742673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 msgr2=0x564d2c480840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c8159b640 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c480840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c8159b640 1 -- 172.21.15.7:0/543742673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d2c480e30 con 0x564d2c470840 2024-03-26T08:48:17.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c8159b640 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47f2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00e5a0 tx=0x7f3c6c020bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cc112e91c91d71 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c64ff9640 1 -- 172.21.15.7:0/543742673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c025040 con 0x564d2c470840 2024-03-26T08:48:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c64ff9640 1 -- 172.21.15.7:0/543742673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c6c0139e0 con 0x564d2c470840 2024-03-26T08:48:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.727+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c7400dcf0 con 0x564d2c470840 2024-03-26T08:48:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c64ff9640 1 -- 172.21.15.7:0/543742673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c6c01b040 con 0x564d2c470840 2024-03-26T08:48:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c863ed740 10 client.16935 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564d2c4e20d0 con 0x564d2c470840 2024-03-26T08:48:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c64ff9640 1 -- 172.21.15.7:0/543742673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c7400dcf0 con 0x564d2c470840 2024-03-26T08:48:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c64ff9640 1 -- 172.21.15.7:0/543742673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3c6c021a20 con 0x564d2c470840 2024-03-26T08:48:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c863ed740 10 client.16935 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c863ed740 10 client.16935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:48:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c863ed740 20 client.16935 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:17.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c863ed740 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564d2c4e2990 0x564d2c4e4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c863ed740 4 client.16935 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.728+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d2c488fd0 con 0x564d2c4e2990 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.729+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564d2c4e2990 0x564d2c4e4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.729+0000 7f3c64ff9640 10 client.16935 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.729+0000 7f3c80d9a640 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564d2c4e2990 0x564d2c4e4db0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1b89165b1c4b182 server_cookie=430ad1606a45f11d in_seq=0 out_seq=0 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c64ff9640 1 -- 172.21.15.7:0/543742673 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d2c488fd0 con 0x564d2c4e2990 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c64ff9640 10 client.16935 handle_command_reply: tid=0 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 client.16935 shutdown 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 msgr2=0x564d2c47f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47f2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00e5a0 tx=0x7f3c6c020bb0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 shutdown_connections 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564d2c4763a0 0x564d2c480840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564d2c4e2990 0x564d2c4e4db0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564d2c471230 0x564d2c47f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c64ff9640 0 client.16935 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:48:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 --2- 172.21.15.7:0/543742673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564d2c470840 0x564d2c47f2f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 >> 172.21.15.7:0/543742673 conn(0x564d2c46bdb0 msgr2=0x564d2c46f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 shutdown_connections 2024-03-26T08:48:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 1 -- 172.21.15.7:0/543742673 wait complete. 2024-03-26T08:48:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.730+0000 7f3c863ed740 20 client.16935 trim_cache size 0 max 16384 2024-03-26T08:48:17.727 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:48:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 -- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c680a2c40 msgr2=0x7f3c680a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c680a2c40 0x7f3c680a5100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009990 tx=0x7f3c6c009110 comp rx=0 tx=0).stop 2024-03-26T08:48:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 -- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c7c108cc0 msgr2=0x7f3c7c1a6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c7c108cc0 0x7f3c7c1a6c50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009780 tx=0x7f3c78004270 comp rx=0 tx=0).stop 2024-03-26T08:48:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 -- 172.21.15.7:0/4133556877 shutdown_connections 2024-03-26T08:48:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3c680a2c40 0x7f3c680a5100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c7c109680 0x7f3c7c1adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c7c108cc0 0x7f3c7c1a6c50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 --2- 172.21.15.7:0/4133556877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c7c072a10 0x7f3c7c1a6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 -- 172.21.15.7:0/4133556877 >> 172.21.15.7:0/4133556877 conn(0x7f3c7c06e4e0 msgr2=0x7f3c7c070120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 -- 172.21.15.7:0/4133556877 shutdown_connections 2024-03-26T08:48:17.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.731+0000 7f3c8259d640 1 -- 172.21.15.7:0/4133556877 wait complete. 2024-03-26T08:48:17.739 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:48:17.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:48:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.898+0000 7f7fb0bba640 1 -- 172.21.15.7:0/1276354289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac1017d0 msgr2=0x7f7fac101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.898+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/1276354289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac1017d0 0x7f7fac101c50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400d910 tx=0x7f7f9402dbb0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.898+0000 7f7fb0bba640 1 -- 172.21.15.7:0/1276354289 shutdown_connections 2024-03-26T08:48:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.898+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/1276354289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fac102190 0x7f7fac108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.898+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/1276354289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac1017d0 0x7f7fac101c50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.898+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/1276354289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fac1005e0 0x7f7fac100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.898+0000 7f7fb0bba640 1 -- 172.21.15.7:0/1276354289 >> 172.21.15.7:0/1276354289 conn(0x7f7fac0fbb70 msgr2=0x7f7fac0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.898+0000 7f7fb0bba640 1 -- 172.21.15.7:0/1276354289 shutdown_connections 2024-03-26T08:48:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.898+0000 7f7fb0bba640 1 -- 172.21.15.7:0/1276354289 wait complete. 2024-03-26T08:48:17.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.899+0000 7f7fb0bba640 1 Processor -- start 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.899+0000 7f7fb0bba640 1 -- start start 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.899+0000 7f7fb0bba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fac1005e0 0x7f7fac199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.899+0000 7f7fb0bba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fac1017d0 0x7f7fac19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.899+0000 7f7fb0bba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac102190 0x7f7fac1a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.899+0000 7f7fb0bba640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac10b1c0 con 0x7f7fac102190 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.899+0000 7f7fb0bba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac10b040 con 0x7f7fac1017d0 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.899+0000 7f7fb0bba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac10b340 con 0x7f7fac1005e0 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac102190 0x7f7fac1a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7fa3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fac1017d0 0x7f7fac19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fac1005e0 0x7f7fac199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac102190 0x7f7fac1a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36352/0 (socket says 172.21.15.7:36352) 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7fa3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fac1017d0 0x7f7fac19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51400/0 (socket says 172.21.15.7:51400) 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faad9d640 1 -- 172.21.15.7:0/4288161812 learned_addr learned my addr 172.21.15.7:0/4288161812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faad9d640 1 -- 172.21.15.7:0/4288161812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fac1005e0 msgr2=0x7f7fac199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faad9d640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fac1005e0 0x7f7fac199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faad9d640 1 -- 172.21.15.7:0/4288161812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fac1017d0 msgr2=0x7f7fac19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faad9d640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fac1017d0 0x7f7fac19a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faad9d640 1 -- 172.21.15.7:0/4288161812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac1a18f0 con 0x7f7fac102190 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7faad9d640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac102190 0x7f7fac1a1250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400cce0 tx=0x7f7fa400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7fa37fe640 1 -- 172.21.15.7:0/4288161812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400f450 con 0x7f7fac102190 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7fa37fe640 1 -- 172.21.15.7:0/4288161812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fa400fad0 con 0x7f7fac102190 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7fb0bba640 1 -- 172.21.15.7:0/4288161812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fac1a1be0 con 0x7f7fac102190 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7fa37fe640 1 -- 172.21.15.7:0/4288161812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4018b60 con 0x7f7fac102190 2024-03-26T08:48:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.900+0000 7f7fb0bba640 1 -- 172.21.15.7:0/4288161812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac1a2130 con 0x7f7fac102190 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7fb379d740 1 Processor -- start 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7fb379d740 1 -- start start 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debadab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7fb379d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3deb70610 con 0x55e3debae080 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7fb379d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3deb133c0 con 0x55e3debb31f0 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7fb379d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3deabbd60 con 0x55e3debad690 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7faa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7faa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36362/0 (socket says 172.21.15.7:36362) 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7faa59c640 1 -- 172.21.15.7:0/567400537 learned_addr learned my addr 172.21.15.7:0/567400537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.904+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debadab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7faa59c640 1 -- 172.21.15.7:0/567400537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 msgr2=0x55e3debadab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7faa59c640 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debadab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7faa59c640 1 -- 172.21.15.7:0/567400537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 msgr2=0x55e3debb5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7faa59c640 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7faa59c640 1 -- 172.21.15.7:0/567400537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3de8273f0 con 0x55e3debae080 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7faa59c640 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb2b80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009a20 tx=0x7f7f9c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff78fa18c10e447 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7fa0ff9640 1 -- 172.21.15.7:0/567400537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c017070 con 0x55e3debae080 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7fa0ff9640 1 -- 172.21.15.7:0/567400537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f9c013470 con 0x55e3debae080 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.905+0000 7f7fa0ff9640 1 -- 172.21.15.7:0/567400537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c012610 con 0x55e3debae080 2024-03-26T08:48:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fa37fe640 1 -- 172.21.15.7:0/4288161812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7fa400fc80 con 0x7f7fac102190 2024-03-26T08:48:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fb379d740 1 -- 172.21.15.7:0/567400537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 msgr2=0x55e3debb2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fb379d740 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb2b80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009a20 tx=0x7f7f9c009ee0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fb379d740 1 -- 172.21.15.7:0/567400537 shutdown_connections 2024-03-26T08:48:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fb379d740 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fb379d740 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb2b80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fb379d740 1 --2- 172.21.15.7:0/567400537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debadab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fb379d740 1 -- 172.21.15.7:0/567400537 >> 172.21.15.7:0/567400537 conn(0x55e3deba8c00 msgr2=0x55e3deba9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fb379d740 1 -- 172.21.15.7:0/567400537 shutdown_connections 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.906+0000 7f7fb379d740 1 -- 172.21.15.7:0/567400537 wait complete. 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fa37fe640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f980a2710 0x7f7f980a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7faa59c640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f980a2710 0x7f7f980a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fa37fe640 1 -- 172.21.15.7:0/4288161812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fa40cef30 con 0x7f7fac102190 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fb379d740 1 -- start start 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fb379d740 1 Processor -- start 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7faa59c640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f980a2710 0x7f7f980a4bd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c013db0 tx=0x7f7f9c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debb7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fb379d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3deacf240 con 0x55e3debae080 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fb379d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3deb0f460 con 0x55e3debb31f0 2024-03-26T08:48:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fb379d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3debb61a0 con 0x55e3debad690 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fb379d740 10 client.0 _ll_register_callbacks cb 0x55e3deacec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7faa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debb7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7fa3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7faad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7faa59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debb7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49134/0 (socket says 172.21.15.7:49134) 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.907+0000 7f7faa59c640 1 -- 172.21.15.7:0/85658706 learned_addr learned my addr 172.21.15.7:0/85658706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7faa59c640 1 -- 172.21.15.7:0/85658706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 msgr2=0x55e3debbefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7faa59c640 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7faa59c640 1 -- 172.21.15.7:0/85658706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 msgr2=0x55e3debb7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7faa59c640 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7faa59c640 1 -- 172.21.15.7:0/85658706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3de8f17d0 con 0x55e3debad690 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7faad9d640 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7faa59c640 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debb7a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55e3debaeaf0 tx=0x7f7f9c0047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b01606822bb52963 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7f91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7f91ffb640 1 -- 172.21.15.7:0/85658706 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c02c070 con 0x55e3debad690 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.908+0000 7f7f91ffb640 1 -- 172.21.15.7:0/85658706 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f9c028620 con 0x55e3debad690 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.909+0000 7f7fb379d740 1 -- 172.21.15.7:0/85658706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f9c013470 con 0x55e3debad690 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.909+0000 7f7f91ffb640 1 -- 172.21.15.7:0/85658706 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f9c01ce30 con 0x55e3debad690 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.909+0000 7f7fb379d740 10 client.26818 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.909+0000 7f7fb379d740 1 -- 172.21.15.7:0/85658706 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e3debc23b0 con 0x55e3debad690 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.909+0000 7f7f91ffb640 1 -- 172.21.15.7:0/85658706 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f9c013470 con 0x55e3debad690 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.910+0000 7f7f91ffb640 1 -- 172.21.15.7:0/85658706 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7f9c03aa40 con 0x55e3debad690 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.910+0000 7f7fb379d740 10 client.26818 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.910+0000 7f7fb379d740 10 client.26818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.910+0000 7f7fb379d740 20 client.26818 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.910+0000 7f7fb379d740 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3debc3700 0x55e3debc5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.910+0000 7f7fb379d740 4 client.26818 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.910+0000 7f7fb379d740 1 -- 172.21.15.7:0/85658706 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3deb20900 con 0x55e3debc3700 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.910+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3debc3700 0x55e3debc5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.911+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3debc3700 0x55e3debc5af0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b82791ad7afeaebd server_cookie=e2626e42cee3a112 in_seq=0 out_seq=0 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.911+0000 7f7f91ffb640 10 client.26818 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7f91ffb640 1 -- 172.21.15.7:0/85658706 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e3deb20900 con 0x55e3debc3700 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7f91ffb640 10 client.26818 handle_command_reply: tid=0 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 client.26818 shutdown 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 -- 172.21.15.7:0/85658706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 msgr2=0x55e3debb7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debb7a50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55e3debaeaf0 tx=0x7f7f9c0047d0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 -- 172.21.15.7:0/85658706 shutdown_connections 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3debc3700 0x55e3debc5af0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debae080 0x55e3debb7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 --2- 172.21.15.7:0/85658706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debad690 0x55e3debb7a50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 -- 172.21.15.7:0/85658706 >> 172.21.15.7:0/85658706 conn(0x55e3deba8c00 msgr2=0x55e3deba7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7f91ffb640 0 client.26818 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 -- 172.21.15.7:0/85658706 shutdown_connections 2024-03-26T08:48:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 1 -- 172.21.15.7:0/85658706 wait complete. 2024-03-26T08:48:17.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.912+0000 7f7fb379d740 20 client.26818 trim_cache size 0 max 16384 2024-03-26T08:48:17.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.929+0000 7f7fb379d740 1 Processor -- start 2024-03-26T08:48:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fb379d740 1 -- start start 2024-03-26T08:48:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debb7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debbef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fb379d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac1ad340 con 0x55e3debad690 2024-03-26T08:48:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fb379d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac1aae40 con 0x55e3debb31f0 2024-03-26T08:48:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fb379d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac1a8ec0 con 0x55e3debae080 2024-03-26T08:48:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7faad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fa3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debbef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fa3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debbef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49138/0 (socket says 172.21.15.7:49138) 2024-03-26T08:48:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7fa3fff640 1 -- 172.21.15.7:0/3414255056 learned_addr learned my addr 172.21.15.7:0/3414255056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7faad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51422/0 (socket says 172.21.15.7:51422) 2024-03-26T08:48:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.930+0000 7f7faa59c640 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debb7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.931+0000 7f7fa3fff640 1 -- 172.21.15.7:0/3414255056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 msgr2=0x55e3debb7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.931+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.931+0000 7f7fa3fff640 1 -- 172.21.15.7:0/3414255056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 msgr2=0x55e3debb7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.931+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debb7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.931+0000 7f7fa3fff640 1 -- 172.21.15.7:0/3414255056 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3deba4670 con 0x55e3debae080 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.931+0000 7f7faa59c640 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debb7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.931+0000 7f7faad9d640 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.931+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debbef90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7f94036000 tx=0x7f7f94030040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2d66a78aa9bf46 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.931+0000 7f7f93fff640 1 -- 172.21.15.7:0/3414255056 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94004df0 con 0x55e3debae080 2024-03-26T08:48:17.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.932+0000 7f7f93fff640 1 -- 172.21.15.7:0/3414255056 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f94032040 con 0x55e3debae080 2024-03-26T08:48:17.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.932+0000 7f7f93fff640 1 -- 172.21.15.7:0/3414255056 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9404ba90 con 0x55e3debae080 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.932+0000 7f7fb379d740 1 -- 172.21.15.7:0/3414255056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 msgr2=0x55e3debbef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.932+0000 7f7fb379d740 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debbef90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7f94036000 tx=0x7f7f94030040 comp rx=0 tx=0).stop 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 -- 172.21.15.7:0/3414255056 shutdown_connections 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debb7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debbef90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 --2- 172.21.15.7:0/3414255056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debb7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 -- 172.21.15.7:0/3414255056 >> 172.21.15.7:0/3414255056 conn(0x55e3deba8c00 msgr2=0x55e3debaf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 -- 172.21.15.7:0/3414255056 shutdown_connections 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 -- 172.21.15.7:0/3414255056 wait complete. 2024-03-26T08:48:17.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 -- start start 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 Processor -- start 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debba400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac1a94c0 con 0x55e3debad690 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac1aa7e0 con 0x55e3debb31f0 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac1a9b20 con 0x55e3debae080 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.933+0000 7f7fb379d740 10 client.0 _ll_register_callbacks cb 0x7f7f9c009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7faa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debba400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fa3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7faad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fa3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debba940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49146/0 (socket says 172.21.15.7:49146) 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7faad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51430/0 (socket says 172.21.15.7:51430) 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fa3fff640 1 -- 172.21.15.7:0/2959415540 learned_addr learned my addr 172.21.15.7:0/2959415540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fa3fff640 1 -- 172.21.15.7:0/2959415540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 msgr2=0x55e3debbd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fa3fff640 1 -- 172.21.15.7:0/2959415540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 msgr2=0x55e3debba400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debba400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fa3fff640 1 -- 172.21.15.7:0/2959415540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3debbdf50 con 0x55e3debae080 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7faa59c640 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debba400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fa3fff640 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debba940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f94006010 tx=0x7f7f940047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fd1517aef3432e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7f91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7f91ffb640 1 -- 172.21.15.7:0/2959415540 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9403b040 con 0x55e3debae080 2024-03-26T08:48:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7f91ffb640 1 -- 172.21.15.7:0/2959415540 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f9404ba10 con 0x55e3debae080 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.934+0000 7f7fb379d740 1 -- 172.21.15.7:0/2959415540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f94032040 con 0x55e3debae080 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7f91ffb640 1 -- 172.21.15.7:0/2959415540 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f9404bc10 con 0x55e3debae080 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7fb379d740 10 client.26824 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7fb379d740 1 -- 172.21.15.7:0/2959415540 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e3debbf280 con 0x55e3debae080 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7f91ffb640 1 -- 172.21.15.7:0/2959415540 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f94032040 con 0x55e3debae080 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7f91ffb640 1 -- 172.21.15.7:0/2959415540 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7f9402e710 con 0x55e3debae080 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7fb379d740 10 client.26824 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7fb379d740 10 client.26824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7fb379d740 20 client.26824 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7fb379d740 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3dec30590 0x55e3dec329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7fb379d740 4 client.26824 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7fb379d740 1 -- 172.21.15.7:0/2959415540 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3deb1ac50 con 0x55e3dec30590 2024-03-26T08:48:17.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.935+0000 7f7faa59c640 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3dec30590 0x55e3dec329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7f91ffb640 10 client.26824 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7faa59c640 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3dec30590 0x55e3dec329b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8875741a0c5f3508 server_cookie=c735b15d8f4b7a5b in_seq=0 out_seq=0 2024-03-26T08:48:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7f91ffb640 1 -- 172.21.15.7:0/2959415540 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3deb1ac50 con 0x55e3dec30590 2024-03-26T08:48:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7f91ffb640 10 client.26824 handle_command_reply: tid=0 2024-03-26T08:48:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 client.26824 shutdown 2024-03-26T08:48:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 -- 172.21.15.7:0/2959415540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 msgr2=0x55e3debba940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debba940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f94006010 tx=0x7f7f940047e0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 -- 172.21.15.7:0/2959415540 shutdown_connections 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e3dec30590 0x55e3dec329b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e3debb31f0 0x55e3debbd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7f91ffb640 0 client.26824 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e3debae080 0x55e3debba940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 --2- 172.21.15.7:0/2959415540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e3debad690 0x55e3debba400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 -- 172.21.15.7:0/2959415540 >> 172.21.15.7:0/2959415540 conn(0x55e3deba8c00 msgr2=0x55e3debb7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 -- 172.21.15.7:0/2959415540 shutdown_connections 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.938+0000 7f7fb379d740 1 -- 172.21.15.7:0/2959415540 wait complete. 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.939+0000 7f7fb379d740 20 client.26824 trim_cache size 0 max 16384 2024-03-26T08:48:17.935 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:48:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.939+0000 7f7fb0bba640 1 -- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f980a2710 msgr2=0x7f7f980a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.939+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f980a2710 0x7f7f980a4bd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c013db0 tx=0x7f7f9c000bc0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.939+0000 7f7fb0bba640 1 -- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac102190 msgr2=0x7f7fac1a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.939+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac102190 0x7f7fac1a1250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400cce0 tx=0x7f7fa400a620 comp rx=0 tx=0).stop 2024-03-26T08:48:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.940+0000 7f7fb0bba640 1 -- 172.21.15.7:0/4288161812 shutdown_connections 2024-03-26T08:48:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.940+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f980a2710 0x7f7f980a4bd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.940+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fac102190 0x7f7fac1a1250 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.940+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fac1017d0 0x7f7fac19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.940+0000 7f7fb0bba640 1 --2- 172.21.15.7:0/4288161812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fac1005e0 0x7f7fac199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.940+0000 7f7fb0bba640 1 -- 172.21.15.7:0/4288161812 >> 172.21.15.7:0/4288161812 conn(0x7f7fac0fbb70 msgr2=0x7f7fac1908a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.940+0000 7f7fb0bba640 1 -- 172.21.15.7:0/4288161812 shutdown_connections 2024-03-26T08:48:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:17.940+0000 7f7fb0bba640 1 -- 172.21.15.7:0/4288161812 wait complete. 2024-03-26T08:48:18.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:17 smithi016 ceph-mon[35851]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 808 op/s 2024-03-26T08:48:18.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:17 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1783992142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:48:18.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:17 smithi007 ceph-mon[32462]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 808 op/s 2024-03-26T08:48:18.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:17 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1783992142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:48:18.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:17 smithi044 ceph-mon[36461]: pgmap v536: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 MiB/s wr, 808 op/s 2024-03-26T08:48:18.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:17 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1783992142' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:48:18.948 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:48:18.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:48:18.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:48:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.104+0000 7f51e88dc640 1 -- 172.21.15.7:0/3185459835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51e4102a10 msgr2=0x7f51e4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.104+0000 7f51e88dc640 1 --2- 172.21.15.7:0/3185459835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51e4102a10 0x7f51e4102e90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009860 tx=0x7f51d802bb70 comp rx=0 tx=0).stop 2024-03-26T08:48:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.104+0000 7f51e88dc640 1 -- 172.21.15.7:0/3185459835 shutdown_connections 2024-03-26T08:48:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.104+0000 7f51e88dc640 1 --2- 172.21.15.7:0/3185459835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e41033d0 0x7f51e4109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.104+0000 7f51e88dc640 1 --2- 172.21.15.7:0/3185459835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51e4102a10 0x7f51e4102e90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.104+0000 7f51e88dc640 1 --2- 172.21.15.7:0/3185459835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e40ff9a0 0x7f51e40ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.104+0000 7f51e88dc640 1 -- 172.21.15.7:0/3185459835 >> 172.21.15.7:0/3185459835 conn(0x7f51e40fb2b0 msgr2=0x7f51e40fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.104+0000 7f51e88dc640 1 -- 172.21.15.7:0/3185459835 shutdown_connections 2024-03-26T08:48:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.104+0000 7f51e88dc640 1 -- 172.21.15.7:0/3185459835 wait complete. 2024-03-26T08:48:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.105+0000 7f51e88dc640 1 Processor -- start 2024-03-26T08:48:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.105+0000 7f51e88dc640 1 -- start start 2024-03-26T08:48:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e88dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e40ff9a0 0x7f51e4102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e88dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e4102a10 0x7f51e41009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e88dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51e41033d0 0x7f51e4100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e88dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e410d330 con 0x7f51e40ff9a0 2024-03-26T08:48:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e88dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e410d1b0 con 0x7f51e4102a10 2024-03-26T08:48:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e88dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e410d4b0 con 0x7f51e41033d0 2024-03-26T08:48:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51e41033d0 0x7f51e4100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51e41033d0 0x7f51e4100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49174/0 (socket says 172.21.15.7:49174) 2024-03-26T08:48:19.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e2d9d640 1 -- 172.21.15.7:0/105882642 learned_addr learned my addr 172.21.15.7:0/105882642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e4102a10 0x7f51e41009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e259c640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e40ff9a0 0x7f51e4102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e1d9b640 1 -- 172.21.15.7:0/105882642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51e41033d0 msgr2=0x7f51e4100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51e41033d0 0x7f51e4100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e1d9b640 1 -- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e40ff9a0 msgr2=0x7f51e4102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e40ff9a0 0x7f51e4102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e1d9b640 1 -- 172.21.15.7:0/105882642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4101440 con 0x7f51e4102a10 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.106+0000 7f51e259c640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e40ff9a0 0x7f51e4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.107+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e4102a10 0x7f51e41009c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f51d802c0b0 tx=0x7f51d8034930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.107+0000 7f51d37fe640 1 -- 172.21.15.7:0/105882642 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8041020 con 0x7f51e4102a10 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.107+0000 7f51e88dc640 1 -- 172.21.15.7:0/105882642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e41016d0 con 0x7f51e4102a10 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.107+0000 7f51e88dc640 1 -- 172.21.15.7:0/105882642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e41a6430 con 0x7f51e4102a10 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.107+0000 7f51d37fe640 1 -- 172.21.15.7:0/105882642 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51d802c9f0 con 0x7f51e4102a10 2024-03-26T08:48:19.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.107+0000 7f51d37fe640 1 -- 172.21.15.7:0/105882642 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8039040 con 0x7f51e4102a10 2024-03-26T08:48:19.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.109+0000 7f51d37fe640 1 -- 172.21.15.7:0/105882642 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f51d8044020 con 0x7f51e4102a10 2024-03-26T08:48:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.110+0000 7f51d37fe640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f51cc0a2880 0x7f51cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.110+0000 7f51eb5a0740 1 Processor -- start 2024-03-26T08:48:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.110+0000 7f51d37fe640 1 -- 172.21.15.7:0/105882642 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f51d80f0180 con 0x7f51e4102a10 2024-03-26T08:48:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.110+0000 7f51e259c640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f51cc0a2880 0x7f51cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.110+0000 7f51eb5a0740 1 -- start start 2024-03-26T08:48:19.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.110+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 0x561ac52c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52ca4c0 0x561ac52cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac5204c90 con 0x561ac52c5350 2024-03-26T08:48:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac522a070 con 0x561ac52ca4c0 2024-03-26T08:48:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac51d2c80 con 0x561ac52c4960 2024-03-26T08:48:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52ca4c0 0x561ac52cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52ca4c0 0x561ac52cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51458/0 (socket says 172.21.15.7:51458) 2024-03-26T08:48:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51e2d9d640 1 -- 172.21.15.7:0/2337730090 learned_addr learned my addr 172.21.15.7:0/2337730090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51e2d9d640 1 -- 172.21.15.7:0/2337730090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 msgr2=0x561ac52c4d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:48:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51e2d9d640 1 --2- 172.21.15.7:0/2337730090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52c4d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51e2d9d640 1 -- 172.21.15.7:0/2337730090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 msgr2=0x561ac52c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51e2d9d640 1 --2- 172.21.15.7:0/2337730090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 0x561ac52c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.111+0000 7f51e2d9d640 1 -- 172.21.15.7:0/2337730090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ac51cad10 con 0x561ac52ca4c0 2024-03-26T08:48:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.112+0000 7f51e2d9d640 1 --2- 172.21.15.7:0/2337730090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52ca4c0 0x561ac52cc950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f51dc011570 tx=0x7f51dc011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be6bb0b826941af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.112+0000 7f51d0ff9640 1 -- 172.21.15.7:0/2337730090 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc004b20 con 0x561ac52ca4c0 2024-03-26T08:48:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.112+0000 7f51e259c640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f51cc0a2880 0x7f51cc0a4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f51e40686b0 tx=0x7f51d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.112+0000 7f51d0ff9640 1 -- 172.21.15.7:0/2337730090 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51dc004cc0 con 0x561ac52ca4c0 2024-03-26T08:48:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.112+0000 7f51d0ff9640 1 -- 172.21.15.7:0/2337730090 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc005520 con 0x561ac52ca4c0 2024-03-26T08:48:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 -- 172.21.15.7:0/2337730090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52ca4c0 msgr2=0x561ac52cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/2337730090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52ca4c0 0x561ac52cc950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f51dc011570 tx=0x7f51dc011a30 comp rx=0 tx=0).stop 2024-03-26T08:48:19.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 -- 172.21.15.7:0/2337730090 shutdown_connections 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/2337730090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52ca4c0 0x561ac52cc950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/2337730090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 0x561ac52c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/2337730090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 -- 172.21.15.7:0/2337730090 >> 172.21.15.7:0/2337730090 conn(0x561ac528d1a0 msgr2=0x561ac52c02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 -- 172.21.15.7:0/2337730090 shutdown_connections 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 -- 172.21.15.7:0/2337730090 wait complete. 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.113+0000 7f51eb5a0740 1 -- start start 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51eb5a0740 1 Processor -- start 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52d6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac51e60d0 con 0x561ac52ca4c0 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52d6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52d6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36430/0 (socket says 172.21.15.7:36430) 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e2d9d640 1 -- 172.21.15.7:0/3372057785 learned_addr learned my addr 172.21.15.7:0/3372057785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac5226110 con 0x561ac52c5350 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ac52cd510 con 0x561ac52c4960 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e259c640 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e1d9b640 1 -- 172.21.15.7:0/3372057785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 msgr2=0x561ac52ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e1d9b640 1 -- 172.21.15.7:0/3372057785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 msgr2=0x561ac52d6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52d6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e1d9b640 1 -- 172.21.15.7:0/3372057785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ac50067e0 con 0x561ac52c5350 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.114+0000 7f51e2d9d640 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52d6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.115+0000 7f51eb5a0740 10 client.0 _ll_register_callbacks cb 0x561ac5221e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.115+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51dc004cc0 con 0x561ac52c5350 2024-03-26T08:48:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.115+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52cf040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x561ac52c5dc0 tx=0x7f51d80bc660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a94acd1eb698a128 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.115+0000 7f51c9ffb640 10 client.26798 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.115+0000 7f51c9ffb640 1 -- 172.21.15.7:0/3372057785 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d80bcb20 con 0x561ac52c5350 2024-03-26T08:48:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.115+0000 7f51c9ffb640 1 -- 172.21.15.7:0/3372057785 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f51d80b9190 con 0x561ac52c5350 2024-03-26T08:48:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.115+0000 7f51c9ffb640 1 -- 172.21.15.7:0/3372057785 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d80c0c70 con 0x561ac52c5350 2024-03-26T08:48:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.115+0000 7f51eb5a0740 10 client.26798 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.115+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x561ac5066f70 con 0x561ac52c5350 2024-03-26T08:48:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.116+0000 7f51c9ffb640 1 -- 172.21.15.7:0/3372057785 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f51d803c030 con 0x561ac52c5350 2024-03-26T08:48:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.116+0000 7f51eb5a0740 10 client.26798 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.116+0000 7f51eb5a0740 10 client.26798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.116+0000 7f51eb5a0740 20 client.26798 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.116+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ac52da640 0x561ac52dcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.116+0000 7f51eb5a0740 4 client.26798 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.116+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ac52375b0 con 0x561ac52da640 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.116+0000 7f51e259c640 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ac52da640 0x561ac52dcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.116+0000 7f51e259c640 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ac52da640 0x561ac52dcae0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15a665926857e3 server_cookie=130de3e89da42fe7 in_seq=0 out_seq=0 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.117+0000 7f51c9ffb640 10 client.26798 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.117+0000 7f51c9ffb640 1 -- 172.21.15.7:0/3372057785 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561ac52375b0 con 0x561ac52da640 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.118+0000 7f51c9ffb640 10 client.26798 handle_command_reply: tid=0 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.118+0000 7f51eb5a0740 1 client.26798 shutdown 2024-03-26T08:48:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 msgr2=0x561ac52cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52cf040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x561ac52c5dc0 tx=0x7f51d80bc660 comp rx=0 tx=0).stop 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 shutdown_connections 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ac52da640 0x561ac52dcae0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52d6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52cf040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3372057785 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 >> 172.21.15.7:0/3372057785 conn(0x561ac528d1a0 msgr2=0x561ac52bf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51c9ffb640 0 client.26798 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 shutdown_connections 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3372057785 wait complete. 2024-03-26T08:48:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.119+0000 7f51eb5a0740 20 client.26798 trim_cache size 0 max 16384 2024-03-26T08:48:19.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.149+0000 7f51eb5a0740 1 Processor -- start 2024-03-26T08:48:19.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.150+0000 7f51eb5a0740 1 -- start start 2024-03-26T08:48:19.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.150+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.150+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52d6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.150+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.150+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4010920 con 0x561ac52ca4c0 2024-03-26T08:48:19.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.150+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c400e420 con 0x561ac52c5350 2024-03-26T08:48:19.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.150+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c400c4a0 con 0x561ac52c4960 2024-03-26T08:48:19.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.151+0000 7f51e1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52d6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.151+0000 7f51e1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52d6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51478/0 (socket says 172.21.15.7:51478) 2024-03-26T08:48:19.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.151+0000 7f51e1d9b640 1 -- 172.21.15.7:0/3926414262 learned_addr learned my addr 172.21.15.7:0/3926414262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.151+0000 7f51e2d9d640 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.154+0000 7f51e259c640 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.154+0000 7f51e1d9b640 1 -- 172.21.15.7:0/3926414262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 msgr2=0x561ac52c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.154+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.154+0000 7f51e1d9b640 1 -- 172.21.15.7:0/3926414262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 msgr2=0x561ac52c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.154+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.154+0000 7f51e1d9b640 1 -- 172.21.15.7:0/3926414262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ac4efe170 con 0x561ac52c5350 2024-03-26T08:48:19.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.154+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52d6040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f51d80b8f80 tx=0x7f51d803e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3376d920b1b1d68 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.155+0000 7f51e259c640 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:48:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.157+0000 7f51cbfff640 1 -- 172.21.15.7:0/3926414262 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004340 con 0x561ac52c5350 2024-03-26T08:48:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.157+0000 7f51cbfff640 1 -- 172.21.15.7:0/3926414262 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51dc004cc0 con 0x561ac52c5350 2024-03-26T08:48:19.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.157+0000 7f51cbfff640 1 -- 172.21.15.7:0/3926414262 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d80bcd80 con 0x561ac52c5350 2024-03-26T08:48:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.159+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3926414262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 msgr2=0x561ac52d6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.159+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52d6040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f51d80b8f80 tx=0x7f51d803e000 comp rx=0 tx=0).stop 2024-03-26T08:48:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.166+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3926414262 shutdown_connections 2024-03-26T08:48:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.166+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52ca4c0 0x561ac52c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.166+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c5350 0x561ac52d6040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.166+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/3926414262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52c4960 0x561ac52c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.166+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3926414262 >> 172.21.15.7:0/3926414262 conn(0x561ac528d1a0 msgr2=0x561ac52ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.174+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3926414262 shutdown_connections 2024-03-26T08:48:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.179+0000 7f51eb5a0740 1 -- 172.21.15.7:0/3926414262 wait complete. 2024-03-26T08:48:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.179+0000 7f51eb5a0740 1 -- start start 2024-03-26T08:48:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.179+0000 7f51eb5a0740 1 Processor -- start 2024-03-26T08:48:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.180+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c4960 0x561ac52d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.180+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 0x561ac52d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.180+0000 7f51eb5a0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52ca4c0 0x561ac52d53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.180+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c400caa0 con 0x561ac52c5350 2024-03-26T08:48:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.180+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c400ddc0 con 0x561ac52c4960 2024-03-26T08:48:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.180+0000 7f51eb5a0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c400d100 con 0x561ac52ca4c0 2024-03-26T08:48:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.180+0000 7f51eb5a0740 10 client.0 _ll_register_callbacks cb 0x7f51d80094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c4960 0x561ac52d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c4960 0x561ac52d1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51492/0 (socket says 172.21.15.7:51492) 2024-03-26T08:48:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e259c640 1 -- 172.21.15.7:0/425255589 learned_addr learned my addr 172.21.15.7:0/425255589 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e2d9d640 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52ca4c0 0x561ac52d53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 0x561ac52d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e1d9b640 1 -- 172.21.15.7:0/425255589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52ca4c0 msgr2=0x561ac52d53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52ca4c0 0x561ac52d53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e1d9b640 1 -- 172.21.15.7:0/425255589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c4960 msgr2=0x561ac52d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c4960 0x561ac52d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.181+0000 7f51e1d9b640 1 -- 172.21.15.7:0/425255589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ac52d59c0 con 0x561ac52c5350 2024-03-26T08:48:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.182+0000 7f51e1d9b640 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 0x561ac52d2420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f51d8005820 tx=0x7f51d8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbedec45b2b13e42 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.182+0000 7f51c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.182+0000 7f51c9ffb640 1 -- 172.21.15.7:0/425255589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d80bcd80 con 0x561ac52c5350 2024-03-26T08:48:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.182+0000 7f51c9ffb640 1 -- 172.21.15.7:0/425255589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f51d80bcf20 con 0x561ac52c5350 2024-03-26T08:48:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.184+0000 7f51eb5a0740 1 -- 172.21.15.7:0/425255589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51dc004cc0 con 0x561ac52c5350 2024-03-26T08:48:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.184+0000 7f51c9ffb640 1 -- 172.21.15.7:0/425255589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d80db150 con 0x561ac52c5350 2024-03-26T08:48:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.184+0000 7f51eb5a0740 10 client.16965 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.184+0000 7f51eb5a0740 1 -- 172.21.15.7:0/425255589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561ac52d59c0 con 0x561ac52c5350 2024-03-26T08:48:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.188+0000 7f51c9ffb640 1 -- 172.21.15.7:0/425255589 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51dc004cc0 con 0x561ac52c5350 2024-03-26T08:48:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.188+0000 7f51c9ffb640 1 -- 172.21.15.7:0/425255589 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f51d803c070 con 0x561ac52c5350 2024-03-26T08:48:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.190+0000 7f51eb5a0740 10 client.16965 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.190+0000 7f51eb5a0740 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.190+0000 7f51eb5a0740 20 client.16965 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.190+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ac5347b70 0x561ac5349f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.190+0000 7f51eb5a0740 4 client.16965 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:48:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.190+0000 7f51eb5a0740 1 -- 172.21.15.7:0/425255589 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561ac5231900 con 0x561ac5347b70 2024-03-26T08:48:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.191+0000 7f51e259c640 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ac5347b70 0x561ac5349f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.192+0000 7f51c9ffb640 10 client.16965 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.192+0000 7f51e259c640 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ac5347b70 0x561ac5349f60 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a63dbe6ba81163 server_cookie=d5db7f00237321da in_seq=0 out_seq=0 2024-03-26T08:48:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.193+0000 7f51c9ffb640 1 -- 172.21.15.7:0/425255589 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561ac5231900 con 0x561ac5347b70 2024-03-26T08:48:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.193+0000 7f51c9ffb640 10 client.16965 handle_command_reply: tid=0 2024-03-26T08:48:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.193+0000 7f51eb5a0740 1 client.16965 shutdown 2024-03-26T08:48:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.193+0000 7f51eb5a0740 1 -- 172.21.15.7:0/425255589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 msgr2=0x561ac52d2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.193+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 0x561ac52d2420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f51d8005820 tx=0x7f51d8005f90 comp rx=0 tx=0).stop 2024-03-26T08:48:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.194+0000 7f51eb5a0740 1 -- 172.21.15.7:0/425255589 shutdown_connections 2024-03-26T08:48:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.194+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561ac5347b70 0x561ac5349f60 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.194+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561ac52ca4c0 0x561ac52d53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.194+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561ac52c5350 0x561ac52d2420 secure :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f51d8005820 tx=0x7f51d8005f90 comp rx=0 tx=0).stop 2024-03-26T08:48:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.194+0000 7f51eb5a0740 1 --2- 172.21.15.7:0/425255589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561ac52c4960 0x561ac52d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.194+0000 7f51eb5a0740 1 -- 172.21.15.7:0/425255589 >> 172.21.15.7:0/425255589 conn(0x561ac528d1a0 msgr2=0x561ac52ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.194+0000 7f51c9ffb640 0 client.16965 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.194+0000 7f51eb5a0740 1 -- 172.21.15.7:0/425255589 shutdown_connections 2024-03-26T08:48:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.195+0000 7f51eb5a0740 1 -- 172.21.15.7:0/425255589 wait complete. 2024-03-26T08:48:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.196+0000 7f51eb5a0740 20 client.16965 trim_cache size 0 max 16384 2024-03-26T08:48:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:48:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:48:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "d4e4ebbd-7b4d-48fb-a940-47f3ff8645a5", 2024-03-26T08:48:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:48:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:48:19.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 -- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f51cc0a2880 msgr2=0x7f51cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f51cc0a2880 0x7f51cc0a4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f51e40686b0 tx=0x7f51d4009260 comp rx=0 tx=0).stop 2024-03-26T08:48:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 -- 172.21.15.7:0/105882642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e4102a10 msgr2=0x7f51e41009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e4102a10 0x7f51e41009c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f51d802c0b0 tx=0x7f51d8034930 comp rx=0 tx=0).stop 2024-03-26T08:48:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 -- 172.21.15.7:0/105882642 shutdown_connections 2024-03-26T08:48:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f51cc0a2880 0x7f51cc0a4d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f51e41033d0 0x7f51e4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51e4102a10 0x7f51e41009c0 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f51d802c0b0 tx=0x7f51d8034930 comp rx=0 tx=0).stop 2024-03-26T08:48:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 --2- 172.21.15.7:0/105882642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51e40ff9a0 0x7f51e4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.198+0000 7f51e88dc640 1 -- 172.21.15.7:0/105882642 >> 172.21.15.7:0/105882642 conn(0x7f51e40fb2b0 msgr2=0x7f51e4105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.199+0000 7f51e88dc640 1 -- 172.21.15.7:0/105882642 shutdown_connections 2024-03-26T08:48:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.200+0000 7f51e88dc640 1 -- 172.21.15.7:0/105882642 wait complete. 2024-03-26T08:48:19.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:48:19.214 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:48:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.381+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2409679916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb881017b0 msgr2=0x7fbb88101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.381+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2409679916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb881017b0 0x7fbb88101c30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00dd10 tx=0x7fbb7c0341a0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.381+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2409679916 shutdown_connections 2024-03-26T08:48:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.381+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2409679916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb88102170 0x7fbb88108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.381+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2409679916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb881017b0 0x7fbb88101c30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.381+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2409679916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb881005c0 0x7fbb881009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.381+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2409679916 >> 172.21.15.7:0/2409679916 conn(0x7fbb880fbb30 msgr2=0x7fbb880fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.382+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2409679916 shutdown_connections 2024-03-26T08:48:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.382+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2409679916 wait complete. 2024-03-26T08:48:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.383+0000 7fbb8c8a1640 1 Processor -- start 2024-03-26T08:48:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.383+0000 7fbb8c8a1640 1 -- start start 2024-03-26T08:48:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb8c8a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb881005c0 0x7fbb88195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb8c8a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb881017b0 0x7fbb88195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb8c8a1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb88102170 0x7fbb8819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb8659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb881005c0 0x7fbb88195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb8659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb881005c0 0x7fbb88195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49242/0 (socket says 172.21.15.7:49242) 2024-03-26T08:48:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb881017b0 0x7fbb88195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb881017b0 0x7fbb88195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51504/0 (socket says 172.21.15.7:51504) 2024-03-26T08:48:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb8659c640 1 -- 172.21.15.7:0/2253278661 learned_addr learned my addr 172.21.15.7:0/2253278661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8810b1f0 con 0x7fbb88102170 2024-03-26T08:48:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8810b070 con 0x7fbb881017b0 2024-03-26T08:48:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.384+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8810b370 con 0x7fbb881005c0 2024-03-26T08:48:19.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb86d9d640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb88102170 0x7fbb8819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb8659c640 1 -- 172.21.15.7:0/2253278661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb881017b0 msgr2=0x7fbb88195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb8659c640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb881017b0 0x7fbb88195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb8659c640 1 -- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb88102170 msgr2=0x7fbb8819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb8659c640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb88102170 0x7fbb8819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb8659c640 1 -- 172.21.15.7:0/2253278661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8819d480 con 0x7fbb881005c0 2024-03-26T08:48:19.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb8659c640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb881005c0 0x7fbb88195860 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800b450 tx=0x7fbb7800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb777fe640 1 -- 172.21.15.7:0/2253278661 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78015020 con 0x7fbb881005c0 2024-03-26T08:48:19.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb777fe640 1 -- 172.21.15.7:0/2253278661 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb78011d70 con 0x7fbb881005c0 2024-03-26T08:48:19.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8819d770 con 0x7fbb881005c0 2024-03-26T08:48:19.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb777fe640 1 -- 172.21.15.7:0/2253278661 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7800f340 con 0x7fbb881005c0 2024-03-26T08:48:19.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.385+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8819dd40 con 0x7fbb881005c0 2024-03-26T08:48:19.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.387+0000 7fbb777fe640 1 -- 172.21.15.7:0/2253278661 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fbb78018020 con 0x7fbb881005c0 2024-03-26T08:48:19.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.388+0000 7fbb8f56e740 1 Processor -- start 2024-03-26T08:48:19.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb777fe640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbb700a2850 0x7fbb700a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb777fe640 1 -- 172.21.15.7:0/2253278661 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbb780ce470 con 0x7fbb881005c0 2024-03-26T08:48:19.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb8f56e740 1 -- start start 2024-03-26T08:48:19.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47ec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c47513d0 con 0x55a7c47ebda0 2024-03-26T08:48:19.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c46f9ad0 con 0x55a7c47ec790 2024-03-26T08:48:19.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c474d470 con 0x55a7c47f1900 2024-03-26T08:48:19.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51506/0 (socket says 172.21.15.7:51506) 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb85d9b640 1 -- 172.21.15.7:0/898747736 learned_addr learned my addr 172.21.15.7:0/898747736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb8659c640 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47ec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.389+0000 7fbb86d9d640 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb86d9d640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbb700a2850 0x7fbb700a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb85d9b640 1 -- 172.21.15.7:0/898747736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 msgr2=0x55a7c47f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb85d9b640 1 -- 172.21.15.7:0/898747736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 msgr2=0x55a7c47ec1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47ec1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb85d9b640 1 -- 172.21.15.7:0/898747736 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c452f7d0 con 0x55a7c47ec790 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb86d9d640 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb8659c640 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47ec1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f1290 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00dd10 tx=0x7fbb7c00da20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913e069a3951170a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb74ff9640 1 -- 172.21.15.7:0/898747736 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c00d0d0 con 0x55a7c47ec790 2024-03-26T08:48:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb86d9d640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbb700a2850 0x7fbb700a4d10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbb80005fd0 tx=0x7fbb80009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb74ff9640 1 -- 172.21.15.7:0/898747736 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb7c00d270 con 0x55a7c47ec790 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.390+0000 7fbb74ff9640 1 -- 172.21.15.7:0/898747736 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c003de0 con 0x55a7c47ec790 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 -- 172.21.15.7:0/898747736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 msgr2=0x55a7c47f1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f1290 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00dd10 tx=0x7fbb7c00da20 comp rx=0 tx=0).stop 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 -- 172.21.15.7:0/898747736 shutdown_connections 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f1290 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/898747736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47ec1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 -- 172.21.15.7:0/898747736 >> 172.21.15.7:0/898747736 conn(0x55a7c47e7310 msgr2=0x55a7c47e7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 -- 172.21.15.7:0/898747736 shutdown_connections 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 -- 172.21.15.7:0/898747736 wait complete. 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 -- start start 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.391+0000 7fbb8f56e740 1 Processor -- start 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47fd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c47f4850 con 0x55a7c47ebda0 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c470d060 con 0x55a7c47ec790 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c47f49d0 con 0x55a7c47f1900 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8f56e740 10 client.0 _ll_register_callbacks cb 0x55a7c46ef6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36464/0 (socket says 172.21.15.7:36464) 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8659c640 1 -- 172.21.15.7:0/2202520060 learned_addr learned my addr 172.21.15.7:0/2202520060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb86d9d640 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8659c640 1 -- 172.21.15.7:0/2202520060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 msgr2=0x55a7c47fd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8659c640 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47fd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8659c640 1 -- 172.21.15.7:0/2202520060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 msgr2=0x55a7c47f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8659c640 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8659c640 1 -- 172.21.15.7:0/2202520060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7c44653f0 con 0x55a7c47ebda0 2024-03-26T08:48:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb8659c640 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f60d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbb7809ada0 tx=0x7fbb7809b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6beb592ea0d7e22 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.392+0000 7fbb6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.393+0000 7fbb6dffb640 1 -- 172.21.15.7:0/2202520060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7809ef60 con 0x55a7c47ebda0 2024-03-26T08:48:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.393+0000 7fbb6dffb640 1 -- 172.21.15.7:0/2202520060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbb780ce0b0 con 0x55a7c47ebda0 2024-03-26T08:48:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.393+0000 7fbb8f56e740 1 -- 172.21.15.7:0/2202520060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb7c00d270 con 0x55a7c47ebda0 2024-03-26T08:48:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.393+0000 7fbb6dffb640 1 -- 172.21.15.7:0/2202520060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb780ce2b0 con 0x55a7c47ebda0 2024-03-26T08:48:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.393+0000 7fbb8f56e740 10 client.16977 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.393+0000 7fbb8f56e740 1 -- 172.21.15.7:0/2202520060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a7c44250f0 con 0x55a7c47ebda0 2024-03-26T08:48:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.394+0000 7fbb6dffb640 1 -- 172.21.15.7:0/2202520060 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb780a35b0 con 0x55a7c47ebda0 2024-03-26T08:48:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.394+0000 7fbb6dffb640 1 -- 172.21.15.7:0/2202520060 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbb780b2070 con 0x55a7c47ebda0 2024-03-26T08:48:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.394+0000 7fbb8f56e740 10 client.16977 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.394+0000 7fbb8f56e740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.394+0000 7fbb8f56e740 20 client.16977 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.394+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a7c4801c20 0x55a7c4804070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.394+0000 7fbb8f56e740 4 client.16977 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.394+0000 7fbb8f56e740 1 -- 172.21.15.7:0/2202520060 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb7c00d270 con 0x55a7c4801c20 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.395+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a7c4801c20 0x55a7c4804070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.395+0000 7fbb6dffb640 10 client.16977 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.395+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a7c4801c20 0x55a7c4804070 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40432a83f22cdd06 server_cookie=ce908714aae9edcc in_seq=0 out_seq=0 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb6dffb640 1 -- 172.21.15.7:0/2202520060 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fbb7c00d270 con 0x55a7c4801c20 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb6dffb640 10 client.16977 handle_command_reply: tid=0 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 client.16977 shutdown 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 -- 172.21.15.7:0/2202520060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 msgr2=0x55a7c47f60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f60d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbb7809ada0 tx=0x7fbb7809b510 comp rx=0 tx=0).stop 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 -- 172.21.15.7:0/2202520060 shutdown_connections 2024-03-26T08:48:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a7c4801c20 0x55a7c4804070 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47fd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb6dffb640 0 client.16977 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/2202520060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f60d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 -- 172.21.15.7:0/2202520060 >> 172.21.15.7:0/2202520060 conn(0x55a7c47e7310 msgr2=0x55a7c47e6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 -- 172.21.15.7:0/2202520060 shutdown_connections 2024-03-26T08:48:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 1 -- 172.21.15.7:0/2202520060 wait complete. 2024-03-26T08:48:19.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.396+0000 7fbb8f56e740 20 client.16977 trim_cache size 0 max 16384 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.421+0000 7fbb8f56e740 1 Processor -- start 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8f56e740 1 -- start start 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47fd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47eac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb881a8e80 con 0x55a7c47ebda0 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb881a6980 con 0x55a7c47ec790 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb881a4a00 con 0x55a7c47f1900 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36476/0 (socket says 172.21.15.7:36476) 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8659c640 1 -- 172.21.15.7:0/522362695 learned_addr learned my addr 172.21.15.7:0/522362695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb86d9d640 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47eac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8659c640 1 -- 172.21.15.7:0/522362695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 msgr2=0x55a7c47eac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8659c640 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47eac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8659c640 1 -- 172.21.15.7:0/522362695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 msgr2=0x55a7c47fd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8659c640 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47fd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb8659c640 1 -- 172.21.15.7:0/522362695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c47e8080 con 0x55a7c47ebda0 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.422+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.423+0000 7fbb8659c640 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f6570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbb780977e0 tx=0x7fbb78099320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992df4f74ba5deb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.423+0000 7fbb6ffff640 1 -- 172.21.15.7:0/522362695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7809ef60 con 0x55a7c47ebda0 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.423+0000 7fbb6ffff640 1 -- 172.21.15.7:0/522362695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb78099fd0 con 0x55a7c47ebda0 2024-03-26T08:48:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.423+0000 7fbb6ffff640 1 -- 172.21.15.7:0/522362695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7809a260 con 0x55a7c47ebda0 2024-03-26T08:48:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 -- 172.21.15.7:0/522362695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 msgr2=0x55a7c47f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f6570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbb780977e0 tx=0x7fbb78099320 comp rx=0 tx=0).stop 2024-03-26T08:48:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 -- 172.21.15.7:0/522362695 shutdown_connections 2024-03-26T08:48:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47f1900 0x55a7c47eac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47ec790 0x55a7c47fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/522362695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c47ebda0 0x55a7c47f6570 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 -- 172.21.15.7:0/522362695 >> 172.21.15.7:0/522362695 conn(0x55a7c47e7310 msgr2=0x55a7c47ea570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 -- 172.21.15.7:0/522362695 shutdown_connections 2024-03-26T08:48:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 -- 172.21.15.7:0/522362695 wait complete. 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.424+0000 7fbb8f56e740 1 -- start start 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb8f56e740 1 Processor -- start 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47ec790 0x55a7c47fce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47f1900 0x55a7c47fd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb8f56e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c4801c20 0x55a7c47fb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb881a5000 con 0x55a7c4801c20 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb881a6320 con 0x55a7c47f1900 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb8f56e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb881a5660 con 0x55a7c47ec790 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb8f56e740 10 client.0 _ll_register_callbacks cb 0x7fbb7809a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb86d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c4801c20 0x55a7c47fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb8659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47ec790 0x55a7c47fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb86d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c4801c20 0x55a7c47fb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36478/0 (socket says 172.21.15.7:36478) 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb86d9d640 1 -- 172.21.15.7:0/1985367831 learned_addr learned my addr 172.21.15.7:0/1985367831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47f1900 0x55a7c47fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb85d9b640 1 -- 172.21.15.7:0/1985367831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47ec790 msgr2=0x55a7c47fce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47ec790 0x55a7c47fce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb85d9b640 1 -- 172.21.15.7:0/1985367831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c4801c20 msgr2=0x55a7c47fb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.425+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c4801c20 0x55a7c47fb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.426+0000 7fbb85d9b640 1 -- 172.21.15.7:0/1985367831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7c47fba90 con 0x55a7c47f1900 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.426+0000 7fbb85d9b640 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47f1900 0x55a7c47fd380 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55a7c47e8280 tx=0x7fbb7c0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c25b3932dfe6de server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.426+0000 7fbb6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.426+0000 7fbb6dffb640 1 -- 172.21.15.7:0/1985367831 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c05b060 con 0x55a7c47f1900 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.426+0000 7fbb6dffb640 1 -- 172.21.15.7:0/1985367831 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbb7c00b7e0 con 0x55a7c47f1900 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.426+0000 7fbb8f56e740 1 -- 172.21.15.7:0/1985367831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb78099fd0 con 0x55a7c47f1900 2024-03-26T08:48:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb6dffb640 1 -- 172.21.15.7:0/1985367831 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb7c00b9e0 con 0x55a7c47f1900 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb8f56e740 10 client.26819 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb8f56e740 1 -- 172.21.15.7:0/1985367831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a7c485dea0 con 0x55a7c47f1900 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb6dffb640 1 -- 172.21.15.7:0/1985367831 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb7c00bb80 con 0x55a7c47f1900 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb6dffb640 1 -- 172.21.15.7:0/1985367831 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbb7c004c30 con 0x55a7c47f1900 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb8f56e740 10 client.26819 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb8f56e740 10 client.26819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb8f56e740 20 client.26819 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a7c485e290 0x55a7c48606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb8f56e740 4 client.26819 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb8f56e740 1 -- 172.21.15.7:0/1985367831 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fbb78099fd0 con 0x55a7c485e290 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.427+0000 7fbb8659c640 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a7c485e290 0x55a7c48606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8659c640 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a7c485e290 0x55a7c48606e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f163a755c5e963 server_cookie=7296f57fdf4534c9 in_seq=0 out_seq=0 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb6dffb640 10 client.26819 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb6dffb640 1 -- 172.21.15.7:0/1985367831 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fbb78099fd0 con 0x55a7c485e290 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb6dffb640 10 client.26819 handle_command_reply: tid=0 2024-03-26T08:48:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8f56e740 1 client.26819 shutdown 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8f56e740 1 -- 172.21.15.7:0/1985367831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47f1900 msgr2=0x55a7c47fd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47f1900 0x55a7c47fd380 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55a7c47e8280 tx=0x7fbb7c0035f0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8f56e740 1 -- 172.21.15.7:0/1985367831 shutdown_connections 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a7c485e290 0x55a7c48606e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb6dffb640 0 client.26819 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a7c4801c20 0x55a7c47fb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a7c47f1900 0x55a7c47fd380 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8f56e740 1 --2- 172.21.15.7:0/1985367831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a7c47ec790 0x55a7c47fce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.428+0000 7fbb8f56e740 1 -- 172.21.15.7:0/1985367831 >> 172.21.15.7:0/1985367831 conn(0x55a7c47e7310 msgr2=0x55a7c47f6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8f56e740 1 -- 172.21.15.7:0/1985367831 shutdown_connections 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8f56e740 1 -- 172.21.15.7:0/1985367831 wait complete. 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8f56e740 20 client.26819 trim_cache size 0 max 16384 2024-03-26T08:48:19.425 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:48:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:48:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:48:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "d4e4ebbd-7b4d-48fb-a940-47f3ff8645a5": { 2024-03-26T08:48:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:48:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "d4e4ebbd-7b4d-48fb-a940-47f3ff8645a5", 2024-03-26T08:48:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:48:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:48:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:48:19.426 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:48:19.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbb700a2850 msgr2=0x7fbb700a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbb700a2850 0x7fbb700a4d10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbb80005fd0 tx=0x7fbb80009600 comp rx=0 tx=0).stop 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb881005c0 msgr2=0x7fbb88195860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb881005c0 0x7fbb88195860 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800b450 tx=0x7fbb7800b910 comp rx=0 tx=0).stop 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 shutdown_connections 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbb700a2850 0x7fbb700a4d10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb88102170 0x7fbb8819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb881017b0 0x7fbb88195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 --2- 172.21.15.7:0/2253278661 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb881005c0 0x7fbb88195860 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 >> 172.21.15.7:0/2253278661 conn(0x7fbb880fbb30 msgr2=0x7fbb88104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 shutdown_connections 2024-03-26T08:48:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:19.429+0000 7fbb8c8a1640 1 -- 172.21.15.7:0/2253278661 wait complete. 2024-03-26T08:48:19.436 INFO:tasks.cephfs.filesystem:scrub status for tag:d4e4ebbd-7b4d-48fb-a940-47f3ff8645a5 - {'path': '/', 'tag': 'd4e4ebbd-7b4d-48fb-a940-47f3ff8645a5', 'options': 'recursive,force'} 2024-03-26T08:48:20.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:19 smithi016 ceph-mon[35851]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.5 MiB/s wr, 840 op/s 2024-03-26T08:48:20.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:19 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:48:20.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:19 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:20.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:19 smithi007 ceph-mon[32462]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.5 MiB/s wr, 840 op/s 2024-03-26T08:48:20.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:19 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:48:20.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:19 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:20.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:19 smithi044 ceph-mon[36461]: pgmap v537: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.5 MiB/s wr, 840 op/s 2024-03-26T08:48:20.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:19 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:48:20.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:19 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:21.072 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 350.0 s, 3328.6 tps, lat 29.688 ms stddev 32.883 2024-03-26T08:48:21.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:20 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:48:21.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:20 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:21.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:20 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:48:21.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:20 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:21.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:20 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:48:21.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:20 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:21.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:20 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:48:21.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:20 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:21.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:20 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:48:21.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:20 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:21.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:20 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:48:21.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:20 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:22.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:21 smithi016 ceph-mon[35851]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 793 op/s 2024-03-26T08:48:22.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:21 smithi007 ceph-mon[32462]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 793 op/s 2024-03-26T08:48:22.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:21 smithi044 ceph-mon[36461]: pgmap v538: 97 pgs: 97 active+clean; 1.6 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 793 op/s 2024-03-26T08:48:24.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:23 smithi016 ceph-mon[35851]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.8 MiB/s wr, 892 op/s 2024-03-26T08:48:24.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:23 smithi007 ceph-mon[32462]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.8 MiB/s wr, 892 op/s 2024-03-26T08:48:24.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:23 smithi044 ceph-mon[36461]: pgmap v539: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.8 MiB/s wr, 892 op/s 2024-03-26T08:48:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.067+0000 7fae24c61640 1 -- 172.21.15.7:0/859437178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201032c0 msgr2=0x7fae20109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.067+0000 7fae24c61640 1 --2- 172.21.15.7:0/859437178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201032c0 0x7fae20109b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fae18009ca0 tx=0x7fae1802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.068+0000 7fae24c61640 1 -- 172.21.15.7:0/859437178 shutdown_connections 2024-03-26T08:48:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.068+0000 7fae24c61640 1 --2- 172.21.15.7:0/859437178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae201032c0 0x7fae20109b50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.068+0000 7fae24c61640 1 --2- 172.21.15.7:0/859437178 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae20102900 0x7fae20102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.068+0000 7fae24c61640 1 --2- 172.21.15.7:0/859437178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20101710 0x7fae20101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.068+0000 7fae24c61640 1 -- 172.21.15.7:0/859437178 >> 172.21.15.7:0/859437178 conn(0x7fae200fcc80 msgr2=0x7fae200ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.068+0000 7fae24c61640 1 -- 172.21.15.7:0/859437178 shutdown_connections 2024-03-26T08:48:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.069+0000 7fae24c61640 1 -- 172.21.15.7:0/859437178 wait complete. 2024-03-26T08:48:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae24c61640 1 Processor -- start 2024-03-26T08:48:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae24c61640 1 -- start start 2024-03-26T08:48:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae24c61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20101710 0x7fae20199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae24c61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20102900 0x7fae2019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae24c61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201032c0 0x7fae201a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae24c61640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20075290 con 0x7fae20102900 2024-03-26T08:48:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae24c61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20075110 con 0x7fae20101710 2024-03-26T08:48:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae24c61640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae20075410 con 0x7fae201032c0 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20101710 0x7fae20199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae1dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20102900 0x7fae2019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae1ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201032c0 0x7fae201a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae1dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20102900 0x7fae2019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36488/0 (socket says 172.21.15.7:36488) 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae1e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20101710 0x7fae20199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51568/0 (socket says 172.21.15.7:51568) 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae1ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201032c0 0x7fae201a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49312/0 (socket says 172.21.15.7:49312) 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.070+0000 7fae1dd9b640 1 -- 172.21.15.7:0/1025255046 learned_addr learned my addr 172.21.15.7:0/1025255046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.071+0000 7fae1ed9d640 1 -- 172.21.15.7:0/1025255046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20101710 msgr2=0x7fae20199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.071+0000 7fae1ed9d640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20101710 0x7fae20199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.071+0000 7fae1ed9d640 1 -- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20102900 msgr2=0x7fae2019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.071+0000 7fae1ed9d640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20102900 0x7fae2019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.071+0000 7fae1ed9d640 1 -- 172.21.15.7:0/1025255046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae201a1850 con 0x7fae201032c0 2024-03-26T08:48:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.071+0000 7fae1e59c640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20101710 0x7fae20199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.071+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20102900 0x7fae2019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.071+0000 7fae1ed9d640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201032c0 0x7fae201a11b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fae1802c630 tx=0x7fae18032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.071+0000 7fae0f7fe640 1 -- 172.21.15.7:0/1025255046 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18039040 con 0x7fae201032c0 2024-03-26T08:48:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.072+0000 7fae24c61640 1 -- 172.21.15.7:0/1025255046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae201a1ae0 con 0x7fae201032c0 2024-03-26T08:48:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.072+0000 7fae24c61640 1 -- 172.21.15.7:0/1025255046 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae201a1ee0 con 0x7fae201032c0 2024-03-26T08:48:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.072+0000 7fae0f7fe640 1 -- 172.21.15.7:0/1025255046 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae18044070 con 0x7fae201032c0 2024-03-26T08:48:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.072+0000 7fae0f7fe640 1 -- 172.21.15.7:0/1025255046 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180042a0 con 0x7fae201032c0 2024-03-26T08:48:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.074+0000 7fae0f7fe640 1 -- 172.21.15.7:0/1025255046 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fae18004440 con 0x7fae201032c0 2024-03-26T08:48:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.075+0000 7fae27852740 1 Processor -- start 2024-03-26T08:48:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae27852740 1 -- start start 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae27852740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae27852740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae27852740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 0x55cc7a9f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae27852740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7a945b10 con 0x55cc7a9e9b20 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae27852740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7a8ee900 con 0x55cc7a9eec90 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae27852740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7a941bb0 con 0x55cc7a9e9130 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae1e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae1e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9ee620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36494/0 (socket says 172.21.15.7:36494) 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae1dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae1e59c640 1 -- 172.21.15.7:0/824892768 learned_addr learned my addr 172.21.15.7:0/824892768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.076+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/824892768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 0x55cc7a9f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.077+0000 7fae0f7fe640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae080a2940 0x7fae080a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.077+0000 7fae1dd9b640 1 -- 172.21.15.7:0/824892768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 msgr2=0x55cc7a9f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.077+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/824892768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 0x55cc7a9f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.077+0000 7fae1dd9b640 1 -- 172.21.15.7:0/824892768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9b20 msgr2=0x55cc7a9ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.077+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/824892768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.077+0000 7fae1dd9b640 1 -- 172.21.15.7:0/824892768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc7a65a210 con 0x55cc7a9e9130 2024-03-26T08:48:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.077+0000 7fae1ed9d640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae080a2940 0x7fae080a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.077+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/824892768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e9550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae14011be0 tx=0x7fae14009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0d2c77979049ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.078+0000 7fae1ed9d640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae080a2940 0x7fae080a4e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae180b9360 tx=0x7fae180b91f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.078+0000 7fae0f7fe640 1 -- 172.21.15.7:0/1025255046 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae180f0ce0 con 0x7fae201032c0 2024-03-26T08:48:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.078+0000 7fae0cff9640 1 -- 172.21.15.7:0/824892768 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400fc50 con 0x55cc7a9e9130 2024-03-26T08:48:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.078+0000 7fae0cff9640 1 -- 172.21.15.7:0/824892768 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae1400fdf0 con 0x55cc7a9e9130 2024-03-26T08:48:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.078+0000 7fae0cff9640 1 -- 172.21.15.7:0/824892768 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14004a00 con 0x55cc7a9e9130 2024-03-26T08:48:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 -- 172.21.15.7:0/824892768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9130 msgr2=0x55cc7a9e9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 --2- 172.21.15.7:0/824892768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e9550 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae14011be0 tx=0x7fae14009550 comp rx=0 tx=0).stop 2024-03-26T08:48:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 -- 172.21.15.7:0/824892768 shutdown_connections 2024-03-26T08:48:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 --2- 172.21.15.7:0/824892768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 0x55cc7a9f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 --2- 172.21.15.7:0/824892768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 --2- 172.21.15.7:0/824892768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e9550 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 -- 172.21.15.7:0/824892768 >> 172.21.15.7:0/824892768 conn(0x55cc7a9dbe90 msgr2=0x55cc7a9dc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 -- 172.21.15.7:0/824892768 shutdown_connections 2024-03-26T08:48:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 -- 172.21.15.7:0/824892768 wait complete. 2024-03-26T08:48:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 355.0 s, 3031.4 tps, lat 33.306 ms stddev 38.677 2024-03-26T08:48:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.079+0000 7fae27852740 1 -- start start 2024-03-26T08:48:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae27852740 1 Processor -- start 2024-03-26T08:48:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae27852740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.079 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae27852740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae27852740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 0x55cc7a9e7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae27852740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7a9e0a20 con 0x55cc7a9e9130 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae27852740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7a901d50 con 0x55cc7a9eec90 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae27852740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc7a9e0ba0 con 0x55cc7a9e9b20 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae27852740 10 client.0 _ll_register_callbacks cb 0x55cc7a901720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36510/0 (socket says 172.21.15.7:36510) 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1dd9b640 1 -- 172.21.15.7:0/2592787709 learned_addr learned my addr 172.21.15.7:0/2592787709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 0x55cc7a9e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1e59c640 1 -- 172.21.15.7:0/2592787709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 msgr2=0x55cc7a9e7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1e59c640 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 0x55cc7a9e7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1e59c640 1 -- 172.21.15.7:0/2592787709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 msgr2=0x55cc7a9e22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:26.080 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1e59c640 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.080+0000 7fae1e59c640 1 -- 172.21.15.7:0/2592787709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc7a726620 con 0x55cc7a9e9b20 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.081+0000 7fae1e59c640 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9e2810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55cc7a9e9f20 tx=0x7fae10011cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58401ed3d250c667 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.081+0000 7fae05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.081+0000 7fae05ffb640 1 -- 172.21.15.7:0/2592787709 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1000f920 con 0x55cc7a9e9b20 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.081+0000 7fae05ffb640 1 -- 172.21.15.7:0/2592787709 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae1000fac0 con 0x55cc7a9e9b20 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.082+0000 7fae27852740 1 -- 172.21.15.7:0/2592787709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae1400fdf0 con 0x55cc7a9e9b20 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.082+0000 7fae05ffb640 1 -- 172.21.15.7:0/2592787709 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae1000fcc0 con 0x55cc7a9e9b20 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae27852740 10 client.26866 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae27852740 1 -- 172.21.15.7:0/2592787709 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cc7aa06660 con 0x55cc7a9e9b20 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae05ffb640 1 -- 172.21.15.7:0/2592787709 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae1400fdf0 con 0x55cc7a9e9b20 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae05ffb640 1 -- 172.21.15.7:0/2592787709 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fae10027420 con 0x55cc7a9e9b20 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae27852740 10 client.26866 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae27852740 10 client.26866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae27852740 20 client.26866 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:26.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae27852740 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc7aa07d30 0x55cc7aa0a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae27852740 4 client.26866 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae27852740 1 -- 172.21.15.7:0/2592787709 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc7a953050 con 0x55cc7aa07d30 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.083+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc7aa07d30 0x55cc7aa0a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.084+0000 7fae05ffb640 10 client.26866 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.084+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc7aa07d30 0x55cc7aa0a190 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4eeae54d036584b server_cookie=15c8f0509f0f00c4 in_seq=0 out_seq=0 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.085+0000 7fae05ffb640 1 -- 172.21.15.7:0/2592787709 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cc7a953050 con 0x55cc7aa07d30 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.085+0000 7fae05ffb640 10 client.26866 handle_command_reply: tid=0 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.085+0000 7fae27852740 1 client.26866 shutdown 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.085+0000 7fae27852740 1 -- 172.21.15.7:0/2592787709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9b20 msgr2=0x55cc7a9e2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.085+0000 7fae27852740 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9e2810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55cc7a9e9f20 tx=0x7fae10011cd0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae27852740 1 -- 172.21.15.7:0/2592787709 shutdown_connections 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae27852740 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc7aa07d30 0x55cc7aa0a190 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae27852740 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9eec90 0x55cc7a9e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae27852740 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9e2810 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae27852740 1 --2- 172.21.15.7:0/2592787709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9e22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae05ffb640 0 client.26866 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae27852740 1 -- 172.21.15.7:0/2592787709 >> 172.21.15.7:0/2592787709 conn(0x55cc7a9dbe90 msgr2=0x55cc7a9db020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae27852740 1 -- 172.21.15.7:0/2592787709 shutdown_connections 2024-03-26T08:48:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae27852740 1 -- 172.21.15.7:0/2592787709 wait complete. 2024-03-26T08:48:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.086+0000 7fae27852740 20 client.26866 trim_cache size 0 max 16384 2024-03-26T08:48:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae27852740 1 Processor -- start 2024-03-26T08:48:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae27852740 1 -- start start 2024-03-26T08:48:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae27852740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae27852740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae27852740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 0x55cc7a9df0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae1dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae1dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9ea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36518/0 (socket says 172.21.15.7:36518) 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae1dd9b640 1 -- 172.21.15.7:0/2094765257 learned_addr learned my addr 172.21.15.7:0/2094765257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae27852740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae00010870 con 0x55cc7a9e9130 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae1e59c640 1 --2- 172.21.15.7:0/2094765257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae27852740 1 -- 172.21.15.7:0/2094765257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000e370 con 0x55cc7a9e9b20 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae27852740 1 -- 172.21.15.7:0/2094765257 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000c3f0 con 0x55cc7a9eec90 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/2094765257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 0x55cc7a9df0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae1dd9b640 1 -- 172.21.15.7:0/2094765257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 msgr2=0x55cc7a9df0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.111+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/2094765257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 0x55cc7a9df0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.112+0000 7fae1dd9b640 1 -- 172.21.15.7:0/2094765257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 msgr2=0x55cc7a9dc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.112+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/2094765257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9dc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.112+0000 7fae1dd9b640 1 -- 172.21.15.7:0/2094765257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc7aa0a810 con 0x55cc7a9e9130 2024-03-26T08:48:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.112+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/2094765257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9ea330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fae14009620 tx=0x7fae14009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10978ceed8ed5cc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.112+0000 7fae07fff640 1 -- 172.21.15.7:0/2094765257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400fd90 con 0x55cc7a9e9130 2024-03-26T08:48:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.112+0000 7fae07fff640 1 -- 172.21.15.7:0/2094765257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae14005380 con 0x55cc7a9e9130 2024-03-26T08:48:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.112+0000 7fae07fff640 1 -- 172.21.15.7:0/2094765257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14003770 con 0x55cc7a9e9130 2024-03-26T08:48:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.113+0000 7fae27852740 1 -- 172.21.15.7:0/2094765257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 msgr2=0x55cc7a9ea330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.113+0000 7fae27852740 1 --2- 172.21.15.7:0/2094765257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9ea330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fae14009620 tx=0x7fae14009f50 comp rx=0 tx=0).stop 2024-03-26T08:48:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.115+0000 7fae27852740 1 -- 172.21.15.7:0/2094765257 shutdown_connections 2024-03-26T08:48:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.115+0000 7fae27852740 1 --2- 172.21.15.7:0/2094765257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 0x55cc7a9df0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.115+0000 7fae27852740 1 --2- 172.21.15.7:0/2094765257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 0x55cc7a9dc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.115+0000 7fae27852740 1 --2- 172.21.15.7:0/2094765257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7a9ea330 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.115+0000 7fae27852740 1 -- 172.21.15.7:0/2094765257 >> 172.21.15.7:0/2094765257 conn(0x55cc7a9dbe90 msgr2=0x55cc7a9e8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.115+0000 7fae27852740 1 -- 172.21.15.7:0/2094765257 shutdown_connections 2024-03-26T08:48:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.115+0000 7fae27852740 1 -- 172.21.15.7:0/2094765257 wait complete. 2024-03-26T08:48:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.116+0000 7fae27852740 1 -- start start 2024-03-26T08:48:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.116+0000 7fae27852740 1 Processor -- start 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae27852740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7aa18da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae27852740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 0x55cc7aa192e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae27852740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 0x55cc7aa19820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae27852740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000c9f0 con 0x55cc7a9e9130 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae27852740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000dd10 con 0x55cc7a9e9b20 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae27852740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000d050 con 0x55cc7a9eec90 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7aa18da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae27852740 10 client.0 _ll_register_callbacks cb 0x7fae100114f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7aa18da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36534/0 (socket says 172.21.15.7:36534) 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 -- 172.21.15.7:0/3064965847 learned_addr learned my addr 172.21.15.7:0/3064965847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1e59c640 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 0x55cc7aa192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 0x55cc7aa19820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 -- 172.21.15.7:0/3064965847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 msgr2=0x55cc7aa19820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 0x55cc7aa19820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 -- 172.21.15.7:0/3064965847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 msgr2=0x55cc7aa192e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 0x55cc7aa192e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 -- 172.21.15.7:0/3064965847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc7aa62d10 con 0x55cc7a9e9130 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1e59c640 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 0x55cc7aa192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae1dd9b640 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7aa18da0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fae140055b0 tx=0x7fae14026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101bd3fae58509bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.117+0000 7fae05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae05ffb640 1 -- 172.21.15.7:0/3064965847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14011110 con 0x55cc7a9e9130 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae05ffb640 1 -- 172.21.15.7:0/3064965847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae14019040 con 0x55cc7a9e9130 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae27852740 1 -- 172.21.15.7:0/3064965847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae14005380 con 0x55cc7a9e9130 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae05ffb640 1 -- 172.21.15.7:0/3064965847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae140112b0 con 0x55cc7a9e9130 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae27852740 10 client.16998 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae27852740 1 -- 172.21.15.7:0/3064965847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cc7aa64000 con 0x55cc7a9e9130 2024-03-26T08:48:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae05ffb640 1 -- 172.21.15.7:0/3064965847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae14005380 con 0x55cc7a9e9130 2024-03-26T08:48:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae05ffb640 1 -- 172.21.15.7:0/3064965847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fae1402f750 con 0x55cc7a9e9130 2024-03-26T08:48:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae27852740 10 client.16998 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae27852740 10 client.16998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae27852740 20 client.16998 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae27852740 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc7aa64720 0x55cc7aa66b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae27852740 4 client.16998 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:48:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.118+0000 7fae27852740 1 -- 172.21.15.7:0/3064965847 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc7a94d3a0 con 0x55cc7aa64720 2024-03-26T08:48:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.119+0000 7fae1e59c640 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc7aa64720 0x55cc7aa66b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.120+0000 7fae1e59c640 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc7aa64720 0x55cc7aa66b40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3861b0a249a3428 server_cookie=2f025f43cd71932 in_seq=0 out_seq=0 2024-03-26T08:48:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.120+0000 7fae05ffb640 10 client.16998 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.173+0000 7fae05ffb640 1 -- 172.21.15.7:0/3064965847 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc7a94d3a0 con 0x55cc7aa64720 2024-03-26T08:48:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.173+0000 7fae05ffb640 10 client.16998 handle_command_reply: tid=0 2024-03-26T08:48:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.173+0000 7fae27852740 1 client.16998 shutdown 2024-03-26T08:48:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.174+0000 7fae27852740 1 -- 172.21.15.7:0/3064965847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 msgr2=0x55cc7aa18da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.174+0000 7fae27852740 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7aa18da0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fae140055b0 tx=0x7fae14026040 comp rx=0 tx=0).stop 2024-03-26T08:48:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.174+0000 7fae27852740 1 -- 172.21.15.7:0/3064965847 shutdown_connections 2024-03-26T08:48:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.174+0000 7fae27852740 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc7aa64720 0x55cc7aa66b40 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.174+0000 7fae27852740 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc7a9eec90 0x55cc7aa19820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.174+0000 7fae27852740 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc7a9e9b20 0x55cc7aa192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.174+0000 7fae27852740 1 --2- 172.21.15.7:0/3064965847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc7a9e9130 0x55cc7aa18da0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.174+0000 7fae27852740 1 -- 172.21.15.7:0/3064965847 >> 172.21.15.7:0/3064965847 conn(0x55cc7a9dbe90 msgr2=0x55cc7a9dc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.174+0000 7fae05ffb640 0 client.16998 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.175+0000 7fae27852740 1 -- 172.21.15.7:0/3064965847 shutdown_connections 2024-03-26T08:48:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.175+0000 7fae27852740 1 -- 172.21.15.7:0/3064965847 wait complete. 2024-03-26T08:48:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.175+0000 7fae27852740 20 client.16998 trim_cache size 0 max 16384 2024-03-26T08:48:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:48:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:48:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:48:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:48:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 -- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae080a2940 msgr2=0x7fae080a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae080a2940 0x7fae080a4e00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae180b9360 tx=0x7fae180b91f0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 -- 172.21.15.7:0/1025255046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201032c0 msgr2=0x7fae201a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201032c0 0x7fae201a11b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fae1802c630 tx=0x7fae18032f20 comp rx=0 tx=0).stop 2024-03-26T08:48:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 -- 172.21.15.7:0/1025255046 shutdown_connections 2024-03-26T08:48:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae080a2940 0x7fae080a4e00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae201032c0 0x7fae201a11b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae20102900 0x7fae2019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 --2- 172.21.15.7:0/1025255046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20101710 0x7fae20199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 -- 172.21.15.7:0/1025255046 >> 172.21.15.7:0/1025255046 conn(0x7fae200fcc80 msgr2=0x7fae20105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 -- 172.21.15.7:0/1025255046 shutdown_connections 2024-03-26T08:48:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:26.176+0000 7fae24c61640 1 -- 172.21.15.7:0/1025255046 wait complete. 2024-03-26T08:48:26.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:25 smithi016 ceph-mon[35851]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.4 MiB/s wr, 798 op/s 2024-03-26T08:48:26.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:25 smithi007 ceph-mon[32462]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.4 MiB/s wr, 798 op/s 2024-03-26T08:48:26.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:25 smithi044 ceph-mon[36461]: pgmap v540: 97 pgs: 97 active+clean; 1.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 7.4 MiB/s wr, 798 op/s 2024-03-26T08:48:27.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:26 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:48:27.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:26 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:48:27.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:26 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:48:27.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:26 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:48:27.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:26 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:48:27.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:26 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:48:28.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:27 smithi016 ceph-mon[35851]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.4 MiB/s wr, 807 op/s 2024-03-26T08:48:28.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:27 smithi007 ceph-mon[32462]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.4 MiB/s wr, 807 op/s 2024-03-26T08:48:28.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:27 smithi044 ceph-mon[36461]: pgmap v541: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.4 MiB/s wr, 807 op/s 2024-03-26T08:48:30.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:29 smithi016 ceph-mon[35851]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.3 MiB/s wr, 851 op/s 2024-03-26T08:48:30.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:29 smithi007 ceph-mon[32462]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.3 MiB/s wr, 851 op/s 2024-03-26T08:48:30.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:29 smithi044 ceph-mon[36461]: pgmap v542: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.3 MiB/s wr, 851 op/s 2024-03-26T08:48:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 360.0 s, 3326.8 tps, lat 29.904 ms stddev 34.630 2024-03-26T08:48:32.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:31 smithi016 ceph-mon[35851]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.8 MiB/s wr, 799 op/s 2024-03-26T08:48:32.293 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:31 smithi007 ceph-mon[32462]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.8 MiB/s wr, 799 op/s 2024-03-26T08:48:32.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:31 smithi044 ceph-mon[36461]: pgmap v543: 97 pgs: 97 active+clean; 1.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.8 MiB/s wr, 799 op/s 2024-03-26T08:48:34.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:33 smithi016 ceph-mon[35851]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.1 MiB/s wr, 873 op/s 2024-03-26T08:48:34.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:33 smithi007 ceph-mon[32462]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.1 MiB/s wr, 873 op/s 2024-03-26T08:48:34.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:33 smithi044 ceph-mon[36461]: pgmap v544: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.1 MiB/s wr, 873 op/s 2024-03-26T08:48:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 365.0 s, 3510.6 tps, lat 28.608 ms stddev 32.974 2024-03-26T08:48:36.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:35 smithi016 ceph-mon[35851]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.5 MiB/s wr, 733 op/s 2024-03-26T08:48:36.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:35 smithi007 ceph-mon[32462]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.5 MiB/s wr, 733 op/s 2024-03-26T08:48:36.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:35 smithi044 ceph-mon[36461]: pgmap v545: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.5 MiB/s wr, 733 op/s 2024-03-26T08:48:38.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:37 smithi016 ceph-mon[35851]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 711 op/s 2024-03-26T08:48:38.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:37 smithi007 ceph-mon[32462]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 711 op/s 2024-03-26T08:48:38.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:37 smithi044 ceph-mon[36461]: pgmap v546: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 MiB/s wr, 711 op/s 2024-03-26T08:48:39.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:38 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:48:39.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:38 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:48:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.314+0000 7f2a9e374640 1 -- 172.21.15.7:0/592845578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981017c0 msgr2=0x7f2a98101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.314+0000 7f2a9e374640 1 --2- 172.21.15.7:0/592845578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981017c0 0x7f2a98101c40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009900 tx=0x7f2a8c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.314+0000 7f2a9e374640 1 -- 172.21.15.7:0/592845578 shutdown_connections 2024-03-26T08:48:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.314+0000 7f2a9e374640 1 --2- 172.21.15.7:0/592845578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a98102180 0x7f2a98108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.314+0000 7f2a9e374640 1 --2- 172.21.15.7:0/592845578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981017c0 0x7f2a98101c40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.314+0000 7f2a9e374640 1 --2- 172.21.15.7:0/592845578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a981005d0 0x7f2a981009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.314+0000 7f2a9e374640 1 -- 172.21.15.7:0/592845578 >> 172.21.15.7:0/592845578 conn(0x7f2a980fbb40 msgr2=0x7f2a980fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.314+0000 7f2a9e374640 1 -- 172.21.15.7:0/592845578 shutdown_connections 2024-03-26T08:48:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.314+0000 7f2a9e374640 1 -- 172.21.15.7:0/592845578 wait complete. 2024-03-26T08:48:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.316+0000 7f2a9e374640 1 Processor -- start 2024-03-26T08:48:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.316+0000 7f2a9e374640 1 -- start start 2024-03-26T08:48:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.316+0000 7f2a9e374640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a981005d0 0x7f2a98199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.316+0000 7f2a9e374640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981017c0 0x7f2a98199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a981005d0 0x7f2a98199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a9e374640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a98102180 0x7f2a981a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a9e374640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9810f5b0 con 0x7f2a981017c0 2024-03-26T08:48:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a9e374640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9810f430 con 0x7f2a98102180 2024-03-26T08:48:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a9e374640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9810f730 con 0x7f2a981005d0 2024-03-26T08:48:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981017c0 0x7f2a98199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a981005d0 0x7f2a98199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58168/0 (socket says 172.21.15.7:58168) 2024-03-26T08:48:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a97fff640 1 -- 172.21.15.7:0/951246720 learned_addr learned my addr 172.21.15.7:0/951246720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a9c911640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a98102180 0x7f2a981a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a97fff640 1 -- 172.21.15.7:0/951246720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a98102180 msgr2=0x7f2a981a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a97fff640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a98102180 0x7f2a981a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a97fff640 1 -- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981017c0 msgr2=0x7f2a98199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a97fff640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981017c0 0x7f2a98199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a97fff640 1 -- 172.21.15.7:0/951246720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a981a1630 con 0x7f2a981005d0 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.317+0000 7f2a977fe640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981017c0 0x7f2a98199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.318+0000 7f2a97fff640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a981005d0 0x7f2a98199a10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800e9c0 tx=0x7f2a8800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.318+0000 7f2a957fa640 1 -- 172.21.15.7:0/951246720 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800ccc0 con 0x7f2a981005d0 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.318+0000 7f2a9e374640 1 -- 172.21.15.7:0/951246720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a981a1920 con 0x7f2a981005d0 2024-03-26T08:48:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.318+0000 7f2a9e374640 1 -- 172.21.15.7:0/951246720 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a981a1e40 con 0x7f2a981005d0 2024-03-26T08:48:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.320+0000 7f2a957fa640 1 -- 172.21.15.7:0/951246720 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a880035f0 con 0x7f2a981005d0 2024-03-26T08:48:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.320+0000 7f2a957fa640 1 -- 172.21.15.7:0/951246720 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800ce60 con 0x7f2a981005d0 2024-03-26T08:48:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.321+0000 7f2a957fa640 1 -- 172.21.15.7:0/951246720 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2a88010750 con 0x7f2a981005d0 2024-03-26T08:48:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.322+0000 7f2a957fa640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a800a2860 0x7f2a800a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2a977fe640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a800a2860 0x7f2a800a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2aa1037740 1 Processor -- start 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2a957fa640 1 -- 172.21.15.7:0/951246720 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a8809a7f0 con 0x7f2a981005d0 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2aa1037740 1 -- start start 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 0x5639b90ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2aa1037740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b902bb10 con 0x5639b90c6920 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2aa1037740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b8fd4900 con 0x5639b90c7310 2024-03-26T08:48:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2aa1037740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b9027bb0 con 0x5639b90cc480 2024-03-26T08:48:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60826/0 (socket says 172.21.15.7:60826) 2024-03-26T08:48:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2a9c911640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 0x5639b90ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2a97fff640 1 -- 172.21.15.7:0/377024915 learned_addr learned my addr 172.21.15.7:0/377024915 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.323+0000 7f2a977fe640 1 --2- 172.21.15.7:0/377024915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:38 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:48:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.324+0000 7f2a977fe640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a800a2860 0x7f2a800a4d20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c02c170 tx=0x7f2a8c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.324+0000 7f2a97fff640 1 -- 172.21.15.7:0/377024915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 msgr2=0x5639b90ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.324+0000 7f2a97fff640 1 --2- 172.21.15.7:0/377024915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 0x5639b90ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.324+0000 7f2a97fff640 1 -- 172.21.15.7:0/377024915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 msgr2=0x5639b90cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.324+0000 7f2a97fff640 1 --2- 172.21.15.7:0/377024915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.324+0000 7f2a97fff640 1 -- 172.21.15.7:0/377024915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b8d40210 con 0x5639b90c6920 2024-03-26T08:48:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.324+0000 7f2a977fe640 1 --2- 172.21.15.7:0/377024915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.324+0000 7f2a97fff640 1 --2- 172.21.15.7:0/377024915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c6d40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2a880cd8b0 tx=0x7f2a8809ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9cc016319c6e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.325+0000 7f2a7e7fc640 1 -- 172.21.15.7:0/377024915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8809c750 con 0x5639b90c6920 2024-03-26T08:48:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.326+0000 7f2a7e7fc640 1 -- 172.21.15.7:0/377024915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a880a08a0 con 0x5639b90c6920 2024-03-26T08:48:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.326+0000 7f2a7e7fc640 1 -- 172.21.15.7:0/377024915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8809c750 con 0x5639b90c6920 2024-03-26T08:48:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.327+0000 7f2aa1037740 1 -- 172.21.15.7:0/377024915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 msgr2=0x5639b90c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.327+0000 7f2aa1037740 1 --2- 172.21.15.7:0/377024915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c6d40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2a880cd8b0 tx=0x7f2a8809ce50 comp rx=0 tx=0).stop 2024-03-26T08:48:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.327+0000 7f2aa1037740 1 -- 172.21.15.7:0/377024915 shutdown_connections 2024-03-26T08:48:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.327+0000 7f2aa1037740 1 --2- 172.21.15.7:0/377024915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 0x5639b90ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.327+0000 7f2aa1037740 1 --2- 172.21.15.7:0/377024915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.327+0000 7f2aa1037740 1 --2- 172.21.15.7:0/377024915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c6d40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.327+0000 7f2aa1037740 1 -- 172.21.15.7:0/377024915 >> 172.21.15.7:0/377024915 conn(0x5639b90c1e90 msgr2=0x5639b90c2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.327+0000 7f2aa1037740 1 -- 172.21.15.7:0/377024915 shutdown_connections 2024-03-26T08:48:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.327+0000 7f2aa1037740 1 -- 172.21.15.7:0/377024915 wait complete. 2024-03-26T08:48:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.328+0000 7f2aa1037740 1 -- start start 2024-03-26T08:48:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.328+0000 7f2aa1037740 1 Processor -- start 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c6920 0x5639b90d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90cc480 0x5639b90d8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c6920 0x5639b90d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c6920 0x5639b90d0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58188/0 (socket says 172.21.15.7:58188) 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56284/0 (socket says 172.21.15.7:56284) 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2a9c911640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90cc480 0x5639b90d8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2a97fff640 1 -- 172.21.15.7:0/3952439547 learned_addr learned my addr 172.21.15.7:0/3952439547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b90cf3d0 con 0x5639b90cc480 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b8fe7d50 con 0x5639b90c7310 2024-03-26T08:48:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.329+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639b90cf550 con 0x5639b90c6920 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2aa1037740 10 client.0 _ll_register_callbacks cb 0x5639b8fe7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a97fff640 1 -- 172.21.15.7:0/3952439547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 msgr2=0x5639b90d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a97fff640 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a97fff640 1 -- 172.21.15.7:0/3952439547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90cc480 msgr2=0x5639b90d8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a97fff640 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90cc480 0x5639b90d8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a97fff640 1 -- 172.21.15.7:0/3952439547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639b8e0c620 con 0x5639b90c6920 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a97fff640 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c6920 0x5639b90d0b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2a8809d000 tx=0x7f2a8800c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10ad2217ea7e75b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a777fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a777fe640 1 -- 172.21.15.7:0/3952439547 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a880b4070 con 0x5639b90c6920 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a777fe640 1 -- 172.21.15.7:0/3952439547 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a880a4bd0 con 0x5639b90c6920 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a880a08a0 con 0x5639b90c6920 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2a777fe640 1 -- 172.21.15.7:0/3952439547 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a8809c750 con 0x5639b90c6920 2024-03-26T08:48:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2aa1037740 10 client.26884 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.330+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639b90cf3d0 con 0x5639b90c6920 2024-03-26T08:48:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.331+0000 7f2a777fe640 1 -- 172.21.15.7:0/3952439547 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a880a08a0 con 0x5639b90c6920 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.332+0000 7f2a777fe640 1 -- 172.21.15.7:0/3952439547 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2a88085ce0 con 0x5639b90c6920 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.332+0000 7f2aa1037740 10 client.26884 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.332+0000 7f2aa1037740 10 client.26884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.332+0000 7f2aa1037740 20 client.26884 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.332+0000 7f2aa1037740 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639b90dc6a0 0x5639b90dea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.332+0000 7f2aa1037740 4 client.26884 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.332+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639b9039050 con 0x5639b90dc6a0 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.333+0000 7f2a977fe640 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639b90dc6a0 0x5639b90dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.334+0000 7f2a977fe640 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639b90dc6a0 0x5639b90dea90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b560e38b04c302 server_cookie=eb1045fec6893a51 in_seq=0 out_seq=0 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.334+0000 7f2a777fe640 10 client.26884 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.334+0000 7f2a777fe640 1 -- 172.21.15.7:0/3952439547 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5639b9039050 con 0x5639b90dc6a0 2024-03-26T08:48:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.334+0000 7f2a777fe640 10 client.26884 handle_command_reply: tid=0 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.334+0000 7f2aa1037740 1 client.26884 shutdown 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c6920 msgr2=0x5639b90d0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c6920 0x5639b90d0b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2a8809d000 tx=0x7f2a8800c7d0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 shutdown_connections 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639b90dc6a0 0x5639b90dea90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90cc480 0x5639b90d8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2a777fe640 0 client.26884 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 --2- 172.21.15.7:0/3952439547 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c6920 0x5639b90d0b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 >> 172.21.15.7:0/3952439547 conn(0x5639b90c1e90 msgr2=0x5639b90c1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 shutdown_connections 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 1 -- 172.21.15.7:0/3952439547 wait complete. 2024-03-26T08:48:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.335+0000 7f2aa1037740 20 client.26884 trim_cache size 0 max 16384 2024-03-26T08:48:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.364+0000 7f2aa1037740 1 Processor -- start 2024-03-26T08:48:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.364+0000 7f2aa1037740 1 -- start start 2024-03-26T08:48:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.364+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.364+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.364+0000 7f2a97fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60834/0 (socket says 172.21.15.7:60834) 2024-03-26T08:48:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.364+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a97fff640 1 -- 172.21.15.7:0/644378906 learned_addr learned my addr 172.21.15.7:0/644378906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2aa1037740 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 0x5639b90c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a977fe640 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2aa1037740 1 -- 172.21.15.7:0/644378906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a840108c0 con 0x5639b90c6920 2024-03-26T08:48:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2aa1037740 1 -- 172.21.15.7:0/644378906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8400e3c0 con 0x5639b90c7310 2024-03-26T08:48:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2aa1037740 1 -- 172.21.15.7:0/644378906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8400c440 con 0x5639b90cc480 2024-03-26T08:48:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a9c911640 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 0x5639b90c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a977fe640 1 -- 172.21.15.7:0/644378906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 msgr2=0x5639b90c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a977fe640 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 0x5639b90c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a977fe640 1 -- 172.21.15.7:0/644378906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 msgr2=0x5639b90c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a977fe640 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a977fe640 1 -- 172.21.15.7:0/644378906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639b90dee00 con 0x5639b90c7310 2024-03-26T08:48:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a97fff640 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.365+0000 7f2a977fe640 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d80d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009230 tx=0x7f2a8c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21595c1f5df92377 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.366+0000 7f2a7dffb640 1 -- 172.21.15.7:0/644378906 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c05a070 con 0x5639b90c7310 2024-03-26T08:48:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.366+0000 7f2a7dffb640 1 -- 172.21.15.7:0/644378906 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c046040 con 0x5639b90c7310 2024-03-26T08:48:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.366+0000 7f2a7dffb640 1 -- 172.21.15.7:0/644378906 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c005630 con 0x5639b90c7310 2024-03-26T08:48:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 -- 172.21.15.7:0/644378906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 msgr2=0x5639b90d80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d80d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009230 tx=0x7f2a8c03f000 comp rx=0 tx=0).stop 2024-03-26T08:48:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 -- 172.21.15.7:0/644378906 shutdown_connections 2024-03-26T08:48:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90cc480 0x5639b90c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90c7310 0x5639b90d80d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 --2- 172.21.15.7:0/644378906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 -- 172.21.15.7:0/644378906 >> 172.21.15.7:0/644378906 conn(0x5639b90c1e90 msgr2=0x5639b90c8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 -- 172.21.15.7:0/644378906 shutdown_connections 2024-03-26T08:48:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 -- 172.21.15.7:0/644378906 wait complete. 2024-03-26T08:48:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.367+0000 7f2aa1037740 1 -- start start 2024-03-26T08:48:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2aa1037740 1 Processor -- start 2024-03-26T08:48:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c7310 0x5639b90d6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2aa1037740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90cc480 0x5639b90d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2aa1037740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8400ca40 con 0x5639b90c6920 2024-03-26T08:48:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2aa1037740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8400dd60 con 0x5639b90cc480 2024-03-26T08:48:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2aa1037740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8400d0a0 con 0x5639b90c7310 2024-03-26T08:48:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c7310 0x5639b90d6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c7310 0x5639b90d6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58206/0 (socket says 172.21.15.7:58206) 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2a977fe640 1 -- 172.21.15.7:0/1276841761 learned_addr learned my addr 172.21.15.7:0/1276841761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2a9c911640 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90cc480 0x5639b90d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.368+0000 7f2a97fff640 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.369+0000 7f2aa1037740 10 client.0 _ll_register_callbacks cb 0x7f2a8809c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.369+0000 7f2a9c911640 1 -- 172.21.15.7:0/1276841761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c7310 msgr2=0x5639b90d6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.369+0000 7f2a9c911640 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c7310 0x5639b90d6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.369+0000 7f2a9c911640 1 -- 172.21.15.7:0/1276841761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 msgr2=0x5639b90d5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.369+0000 7f2a9c911640 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90d5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.369+0000 7f2a9c911640 1 -- 172.21.15.7:0/1276841761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639b90d7720 con 0x5639b90cc480 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.369+0000 7f2a9c911640 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90cc480 0x5639b90d7130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5639b90c2e00 tx=0x7f2a9000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac579ffea8363909 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.369+0000 7f2a777fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.370+0000 7f2a777fe640 1 -- 172.21.15.7:0/1276841761 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a90022020 con 0x5639b90cc480 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.370+0000 7f2a777fe640 1 -- 172.21.15.7:0/1276841761 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a90004d20 con 0x5639b90cc480 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.370+0000 7f2aa1037740 1 -- 172.21.15.7:0/1276841761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a8c046040 con 0x5639b90cc480 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.370+0000 7f2a777fe640 1 -- 172.21.15.7:0/1276841761 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a90005020 con 0x5639b90cc480 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.370+0000 7f2aa1037740 10 client.26855 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.370+0000 7f2aa1037740 1 -- 172.21.15.7:0/1276841761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5639b9140bb0 con 0x5639b90cc480 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.371+0000 7f2a777fe640 1 -- 172.21.15.7:0/1276841761 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a8c046040 con 0x5639b90cc480 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.371+0000 7f2a777fe640 1 -- 172.21.15.7:0/1276841761 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2a9001e600 con 0x5639b90cc480 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.371+0000 7f2aa1037740 10 client.26855 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.371+0000 7f2aa1037740 10 client.26855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.371+0000 7f2aa1037740 20 client.26855 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.371+0000 7f2aa1037740 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639b90dc6a0 0x5639b91435a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.371+0000 7f2aa1037740 4 client.26855 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:48:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.371+0000 7f2aa1037740 1 -- 172.21.15.7:0/1276841761 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639b90333a0 con 0x5639b90dc6a0 2024-03-26T08:48:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.371+0000 7f2a97fff640 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639b90dc6a0 0x5639b91435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.372+0000 7f2a777fe640 10 client.26855 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:39.372 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.372+0000 7f2a97fff640 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639b90dc6a0 0x5639b91435a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68375ac07372704a server_cookie=ae4f8c9397f0dd20 in_seq=0 out_seq=0 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.400+0000 7f2a777fe640 1 -- 172.21.15.7:0/1276841761 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639b90333a0 con 0x5639b90dc6a0 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.400+0000 7f2a777fe640 10 client.26855 handle_command_reply: tid=0 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.400+0000 7f2aa1037740 1 client.26855 shutdown 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.400+0000 7f2aa1037740 1 -- 172.21.15.7:0/1276841761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90cc480 msgr2=0x5639b90d7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.400+0000 7f2aa1037740 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90cc480 0x5639b90d7130 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5639b90c2e00 tx=0x7f2a9000f790 comp rx=0 tx=0).stop 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.401+0000 7f2aa1037740 1 -- 172.21.15.7:0/1276841761 shutdown_connections 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.401+0000 7f2aa1037740 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5639b90dc6a0 0x5639b91435a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.401+0000 7f2aa1037740 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639b90cc480 0x5639b90d7130 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.401+0000 7f2aa1037740 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5639b90c7310 0x5639b90d6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.401+0000 7f2aa1037740 1 --2- 172.21.15.7:0/1276841761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639b90c6920 0x5639b90d5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.401+0000 7f2aa1037740 1 -- 172.21.15.7:0/1276841761 >> 172.21.15.7:0/1276841761 conn(0x5639b90c1e90 msgr2=0x5639b90c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.401+0000 7f2a777fe640 0 client.26855 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.401+0000 7f2aa1037740 1 -- 172.21.15.7:0/1276841761 shutdown_connections 2024-03-26T08:48:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.402+0000 7f2aa1037740 1 -- 172.21.15.7:0/1276841761 wait complete. 2024-03-26T08:48:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.402+0000 7f2aa1037740 20 client.26855 trim_cache size 0 max 16384 2024-03-26T08:48:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:48:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:48:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:48:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:48:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 -- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a800a2860 msgr2=0x7f2a800a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a800a2860 0x7f2a800a4d20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c02c170 tx=0x7f2a8c02df90 comp rx=0 tx=0).stop 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 -- 172.21.15.7:0/951246720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a981005d0 msgr2=0x7f2a98199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a981005d0 0x7f2a98199a10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800e9c0 tx=0x7f2a8800edc0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 -- 172.21.15.7:0/951246720 shutdown_connections 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2a800a2860 0x7f2a800a4d20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a98102180 0x7f2a981a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a981017c0 0x7f2a98199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 --2- 172.21.15.7:0/951246720 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2a981005d0 0x7f2a98199a10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.403+0000 7f2a9e374640 1 -- 172.21.15.7:0/951246720 >> 172.21.15.7:0/951246720 conn(0x7f2a980fbb40 msgr2=0x7f2a981049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.404+0000 7f2a9e374640 1 -- 172.21.15.7:0/951246720 shutdown_connections 2024-03-26T08:48:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:39.404+0000 7f2a9e374640 1 -- 172.21.15.7:0/951246720 wait complete. 2024-03-26T08:48:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:39 smithi016 ceph-mon[35851]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 792 op/s 2024-03-26T08:48:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:48:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:48:40.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:39 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:48:40.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:39 smithi007 ceph-mon[32462]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 792 op/s 2024-03-26T08:48:40.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:48:40.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:48:40.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:39 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:48:40.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:39 smithi044 ceph-mon[36461]: pgmap v547: 97 pgs: 97 active+clean; 1.6 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 792 op/s 2024-03-26T08:48:40.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:48:40.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:48:40.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:48:40.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:39 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:48:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 370.0 s, 2062.6 tps, lat 47.998 ms stddev 51.933 2024-03-26T08:48:41.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:40 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-03-26T08:48:41.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:40 smithi016 ceph-mon[35851]: Unable to set osd_memory_target on smithi007 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-03-26T08:48:41.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:40 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-03-26T08:48:41.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:40 smithi007 ceph-mon[32462]: Unable to set osd_memory_target on smithi007 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-03-26T08:48:41.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:40 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi007 to 177.2M 2024-03-26T08:48:41.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:40 smithi044 ceph-mon[36461]: Unable to set osd_memory_target on smithi007 to 185894912: error parsing value: Value '185894912' is below minimum 939524096 2024-03-26T08:48:42.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:41 smithi016 ceph-mon[35851]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.5 MiB/s wr, 695 op/s 2024-03-26T08:48:42.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:41 smithi007 ceph-mon[32462]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.5 MiB/s wr, 695 op/s 2024-03-26T08:48:42.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:41 smithi044 ceph-mon[36461]: pgmap v548: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.5 MiB/s wr, 695 op/s 2024-03-26T08:48:44.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:43 smithi016 ceph-mon[35851]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.8 MiB/s wr, 765 op/s 2024-03-26T08:48:44.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:43 smithi007 ceph-mon[32462]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.8 MiB/s wr, 765 op/s 2024-03-26T08:48:44.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:43 smithi044 ceph-mon[36461]: pgmap v549: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.8 MiB/s wr, 765 op/s 2024-03-26T08:48:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 375.0 s, 2727.4 tps, lat 37.031 ms stddev 45.704 2024-03-26T08:48:46.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:45 smithi016 ceph-mon[35851]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.5 MiB/s wr, 754 op/s 2024-03-26T08:48:46.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:45 smithi007 ceph-mon[32462]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.5 MiB/s wr, 754 op/s 2024-03-26T08:48:46.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:45 smithi044 ceph-mon[36461]: pgmap v550: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.5 MiB/s wr, 754 op/s 2024-03-26T08:48:48.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:47 smithi016 ceph-mon[35851]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 781 op/s 2024-03-26T08:48:48.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:47 smithi007 ceph-mon[32462]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 781 op/s 2024-03-26T08:48:48.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:47 smithi044 ceph-mon[36461]: pgmap v551: 97 pgs: 97 active+clean; 1.6 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 781 op/s 2024-03-26T08:48:49.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:48:49.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:48:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.600+0000 7f2fb690a640 1 -- 172.21.15.7:0/2940611962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb0074100 msgr2=0x7f2fb0074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.600+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2940611962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb0074100 0x7f2fb0074580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009930 tx=0x7f2fa402bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.601+0000 7f2fb690a640 1 -- 172.21.15.7:0/2940611962 shutdown_connections 2024-03-26T08:48:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.601+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2940611962 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb010dca0 0x7f2fb0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.601+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2940611962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb0074100 0x7f2fb0074580 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.601+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2940611962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb00756e0 0x7f2fb0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.601+0000 7f2fb690a640 1 -- 172.21.15.7:0/2940611962 >> 172.21.15.7:0/2940611962 conn(0x7f2fb00fcc60 msgr2=0x7f2fb00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.601+0000 7f2fb690a640 1 -- 172.21.15.7:0/2940611962 shutdown_connections 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.601+0000 7f2fb690a640 1 -- 172.21.15.7:0/2940611962 wait complete. 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.601+0000 7f2fb690a640 1 Processor -- start 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.601+0000 7f2fb690a640 1 -- start start 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2fb690a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0074100 0x7f2fb019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2fb690a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb00756e0 0x7f2fb019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2fb690a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb010dca0 0x7f2fb01a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2fb690a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb0113990 con 0x7f2fb010dca0 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2fb690a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb0113810 con 0x7f2fb0074100 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2fb690a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb0113b10 con 0x7f2fb00756e0 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb00756e0 0x7f2fb019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0074100 0x7f2fb019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0074100 0x7f2fb019e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41324/0 (socket says 172.21.15.7:41324) 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb00756e0 0x7f2fb019e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43800/0 (socket says 172.21.15.7:43800) 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faffff640 1 -- 172.21.15.7:0/2094134833 learned_addr learned my addr 172.21.15.7:0/2094134833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2fb4ea7640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb010dca0 0x7f2fb01a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faf7fe640 1 -- 172.21.15.7:0/2094134833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0074100 msgr2=0x7f2fb019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0074100 0x7f2fb019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faf7fe640 1 -- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb010dca0 msgr2=0x7f2fb01a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb010dca0 0x7f2fb01a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faf7fe640 1 -- 172.21.15.7:0/2094134833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb01a5c70 con 0x7f2fb00756e0 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faffff640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0074100 0x7f2fb019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb00756e0 0x7f2fb019e590 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2fa402c170 tx=0x7f2fa402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.602+0000 7f2fad7fa640 1 -- 172.21.15.7:0/2094134833 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4043020 con 0x7f2fb00756e0 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.603+0000 7f2fb690a640 1 -- 172.21.15.7:0/2094134833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb01a5f00 con 0x7f2fb00756e0 2024-03-26T08:48:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.603+0000 7f2fad7fa640 1 -- 172.21.15.7:0/2094134833 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fa40054e0 con 0x7f2fb00756e0 2024-03-26T08:48:49.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.603+0000 7f2fad7fa640 1 -- 172.21.15.7:0/2094134833 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4003c30 con 0x7f2fb00756e0 2024-03-26T08:48:49.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.603+0000 7f2fb690a640 1 -- 172.21.15.7:0/2094134833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb01a6410 con 0x7f2fb00756e0 2024-03-26T08:48:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.606+0000 7f2fad7fa640 1 -- 172.21.15.7:0/2094134833 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2fa4046020 con 0x7f2fb00756e0 2024-03-26T08:48:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.606+0000 7f2fb95cd740 1 Processor -- start 2024-03-26T08:48:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.606+0000 7f2fb95cd740 1 -- start start 2024-03-26T08:48:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058d71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058dc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb95cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190583c3d0 con 0x5619058dc900 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb95cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619057e4ad0 con 0x5619058d6da0 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb95cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561905838470 con 0x5619058d7790 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb4ea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058d71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb4ea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058d71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41336/0 (socket says 172.21.15.7:41336) 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb4ea7640 1 -- 172.21.15.7:0/157601527 learned_addr learned my addr 172.21.15.7:0/157601527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2faffff640 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058dc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2faffff640 1 -- 172.21.15.7:0/157601527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 msgr2=0x5619058d71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2faffff640 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058d71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2faffff640 1 -- 172.21.15.7:0/157601527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 msgr2=0x5619058ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2faffff640 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058ded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2faffff640 1 -- 172.21.15.7:0/157601527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56190561a7d0 con 0x5619058d7790 2024-03-26T08:48:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058ded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fad7fa640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f980a2900 0x7f2f980a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb4ea7640 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058d71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2fb4ea7640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f980a2900 0x7f2f980a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.607+0000 7f2faffff640 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058dc290 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8011870 tx=0x7f2fa8011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff8451e69234ad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.608+0000 7f2fad7fa640 1 -- 172.21.15.7:0/2094134833 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2fa40f09d0 con 0x7f2fb00756e0 2024-03-26T08:48:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.608+0000 7f2f967fc640 1 -- 172.21.15.7:0/157601527 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80112f0 con 0x5619058d7790 2024-03-26T08:48:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.608+0000 7f2f967fc640 1 -- 172.21.15.7:0/157601527 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fa8003e50 con 0x5619058d7790 2024-03-26T08:48:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.608+0000 7f2f967fc640 1 -- 172.21.15.7:0/157601527 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8004530 con 0x5619058d7790 2024-03-26T08:48:49.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.608+0000 7f2fb95cd740 1 -- 172.21.15.7:0/157601527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 msgr2=0x5619058dc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.608+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058dc290 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8011870 tx=0x7f2fa8011d30 comp rx=0 tx=0).stop 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.608+0000 7f2fb4ea7640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f980a2900 0x7f2f980a4dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5619058d7ff0 tx=0x7f2fa0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.609+0000 7f2fb95cd740 1 -- 172.21.15.7:0/157601527 shutdown_connections 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.609+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058ded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.609+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058dc290 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.609+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/157601527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058d71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.609+0000 7f2fb95cd740 1 -- 172.21.15.7:0/157601527 >> 172.21.15.7:0/157601527 conn(0x5619058d2310 msgr2=0x5619058d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.609+0000 7f2fb95cd740 1 -- 172.21.15.7:0/157601527 shutdown_connections 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.609+0000 7f2fb95cd740 1 -- 172.21.15.7:0/157601527 wait complete. 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.609+0000 7f2fb95cd740 1 -- start start 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.609+0000 7f2fb95cd740 1 Processor -- start 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2faf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2faf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43816/0 (socket says 172.21.15.7:43816) 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2faf7fe640 1 -- 172.21.15.7:0/2166359428 learned_addr learned my addr 172.21.15.7:0/2166359428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619058f08c0 con 0x5619058dc900 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2faffff640 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619057f8060 con 0x5619058d6da0 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619058f0a40 con 0x5619058d7790 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2fb4ea7640 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.610+0000 7f2fb95cd740 10 client.0 _ll_register_callbacks cb 0x5619057da6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2faffff640 1 -- 172.21.15.7:0/2166359428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 msgr2=0x5619058f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2faffff640 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2faffff640 1 -- 172.21.15.7:0/2166359428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 msgr2=0x5619058f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:49.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2faffff640 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2faffff640 1 -- 172.21.15.7:0/2166359428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619055503f0 con 0x5619058d6da0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2faffff640 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058f21b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5619058d3280 tx=0x7f2fa8009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b45b911f36ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2f8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/2166359428 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa802a060 con 0x5619058d6da0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fa8003e50 con 0x5619058d6da0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.611+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/2166359428 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2fa8004a40 con 0x5619058d6da0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/2166359428 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa8004c40 con 0x5619058d6da0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2fb95cd740 10 client.26867 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5619055100f0 con 0x5619058d6da0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/2166359428 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fa8004de0 con 0x5619058d6da0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/2166359428 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2fa802f070 con 0x5619058d6da0 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2fb95cd740 10 client.26867 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2fb95cd740 10 client.26867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:49.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2fb95cd740 20 client.26867 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619058fd930 0x5619058ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2fb95cd740 4 client.26867 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fa8003e50 con 0x5619058fd930 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.612+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619058fd930 0x5619058ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.613+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619058fd930 0x5619058ffd20 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302ea4084b6c78ae server_cookie=a5c32875c82089af in_seq=0 out_seq=0 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.613+0000 7f2f8f7fe640 10 client.26867 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.613+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/2166359428 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f2fa8003e50 con 0x5619058fd930 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.613+0000 7f2f8f7fe640 10 client.26867 handle_command_reply: tid=0 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.613+0000 7f2fb95cd740 1 client.26867 shutdown 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 msgr2=0x5619058f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058f21b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5619058d3280 tx=0x7f2fa8009330 comp rx=0 tx=0).stop 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 shutdown_connections 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619058fd930 0x5619058ffd20 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/2166359428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d6da0 0x5619058f21b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2f8f7fe640 0 client.26867 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:49.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 >> 172.21.15.7:0/2166359428 conn(0x5619058d2310 msgr2=0x5619058d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:49.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 shutdown_connections 2024-03-26T08:48:49.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 1 -- 172.21.15.7:0/2166359428 wait complete. 2024-03-26T08:48:49.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.614+0000 7f2fb95cd740 20 client.26867 trim_cache size 0 max 16384 2024-03-26T08:48:49.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.636+0000 7f2fb95cd740 1 Processor -- start 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.636+0000 7f2fb95cd740 1 -- start start 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058d6da0 0x5619058d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058dc900 0x5619058f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2fb95cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb01a8f40 con 0x5619058d6da0 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2fb95cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb01a6a40 con 0x5619058dc900 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2fb95cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb0071db0 con 0x5619058d7790 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058d6da0 0x5619058d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058d6da0 0x5619058d5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38094/0 (socket says 172.21.15.7:38094) 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43822/0 (socket says 172.21.15.7:43822) 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faffff640 1 -- 172.21.15.7:0/108568098 learned_addr learned my addr 172.21.15.7:0/108568098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2fb4ea7640 1 --2- 172.21.15.7:0/108568098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058dc900 0x5619058f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faf7fe640 1 -- 172.21.15.7:0/108568098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058dc900 msgr2=0x5619058f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/108568098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058dc900 0x5619058f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faf7fe640 1 -- 172.21.15.7:0/108568098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058d6da0 msgr2=0x5619058d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/108568098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058d6da0 0x5619058d5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faf7fe640 1 -- 172.21.15.7:0/108568098 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619058d1c40 con 0x5619058d7790 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.637+0000 7f2faffff640 1 --2- 172.21.15.7:0/108568098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058d6da0 0x5619058d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.638+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/108568098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f96f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40bd2c0 tx=0x7f2fa40dd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e606d647fecd3b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.638+0000 7f2f95ffb640 1 -- 172.21.15.7:0/108568098 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40bda30 con 0x5619058d7790 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.638+0000 7f2f95ffb640 1 -- 172.21.15.7:0/108568098 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fa40bd330 con 0x5619058d7790 2024-03-26T08:48:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.638+0000 7f2f95ffb640 1 -- 172.21.15.7:0/108568098 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40bb630 con 0x5619058d7790 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.639+0000 7f2fb95cd740 1 -- 172.21.15.7:0/108568098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 msgr2=0x5619058f96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.639+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/108568098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f96f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40bd2c0 tx=0x7f2fa40dd040 comp rx=0 tx=0).stop 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.640+0000 7f2fb95cd740 1 -- 172.21.15.7:0/108568098 shutdown_connections 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.640+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/108568098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058dc900 0x5619058f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.640+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/108568098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d7790 0x5619058f96f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.640+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/108568098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058d6da0 0x5619058d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.640+0000 7f2fb95cd740 1 -- 172.21.15.7:0/108568098 >> 172.21.15.7:0/108568098 conn(0x5619058d2310 msgr2=0x5619058d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.640+0000 7f2fb95cd740 1 -- 172.21.15.7:0/108568098 shutdown_connections 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.640+0000 7f2fb95cd740 1 -- 172.21.15.7:0/108568098 wait complete. 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.640+0000 7f2fb95cd740 1 -- start start 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2fb95cd740 1 Processor -- start 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d6da0 0x5619058f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d7790 0x5619058f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2fb95cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2fb95cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00723b0 con 0x5619058dc900 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2fb95cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00736d0 con 0x5619058d7790 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2fb95cd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb0072a10 con 0x5619058d6da0 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2faffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d6da0 0x5619058f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2fb95cd740 10 client.0 _ll_register_callbacks cb 0x7f2fa800fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2faffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d6da0 0x5619058f8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43828/0 (socket says 172.21.15.7:43828) 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2faf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d7790 0x5619058f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2faffff640 1 -- 172.21.15.7:0/720077287 learned_addr learned my addr 172.21.15.7:0/720077287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2fb4ea7640 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2faffff640 1 -- 172.21.15.7:0/720077287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d7790 msgr2=0x5619058f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2faffff640 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d7790 0x5619058f93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2faffff640 1 -- 172.21.15.7:0/720077287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 msgr2=0x5619058f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2faffff640 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.641+0000 7f2faffff640 1 -- 172.21.15.7:0/720077287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619058f7a80 con 0x5619058d6da0 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2faffff640 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d6da0 0x5619058f8e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80053a0 tx=0x7f2fa8005010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0db70f8ea40d594 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2f8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/720077287 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8035070 con 0x5619058d6da0 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/720077287 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2fa8005710 con 0x5619058d6da0 2024-03-26T08:48:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2fb95cd740 1 -- 172.21.15.7:0/720077287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fa40bd330 con 0x5619058d6da0 2024-03-26T08:48:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/720077287 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa802a040 con 0x5619058d6da0 2024-03-26T08:48:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2fb95cd740 10 client.26908 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2fb95cd740 1 -- 172.21.15.7:0/720077287 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561905959a50 con 0x5619058d6da0 2024-03-26T08:48:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/720077287 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fa40bd330 con 0x5619058d6da0 2024-03-26T08:48:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/720077287 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2fa8038030 con 0x5619058d6da0 2024-03-26T08:48:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2fb95cd740 10 client.26908 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2fb95cd740 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2fb95cd740 20 client.26908 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.642+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56190595a3d0 0x56190595c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.643+0000 7f2fb95cd740 4 client.26908 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:48:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.643+0000 7f2fb95cd740 1 -- 172.21.15.7:0/720077287 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561905849910 con 0x56190595a3d0 2024-03-26T08:48:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.643+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56190595a3d0 0x56190595c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2faf7fe640 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56190595a3d0 0x56190595c7f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8f819fe4d10b4f server_cookie=1db9ac339acac32d in_seq=0 out_seq=0 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2f8f7fe640 10 client.26908 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2f8f7fe640 1 -- 172.21.15.7:0/720077287 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561905849910 con 0x56190595a3d0 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2f8f7fe640 10 client.26908 handle_command_reply: tid=0 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2fb95cd740 1 client.26908 shutdown 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2fb95cd740 1 -- 172.21.15.7:0/720077287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d6da0 msgr2=0x5619058f8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d6da0 0x5619058f8e80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80053a0 tx=0x7f2fa8005010 comp rx=0 tx=0).stop 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2fb95cd740 1 -- 172.21.15.7:0/720077287 shutdown_connections 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56190595a3d0 0x56190595c7f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619058dc900 0x5619058f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2f8f7fe640 0 client.26908 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619058d7790 0x5619058f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2fb95cd740 1 --2- 172.21.15.7:0/720077287 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619058d6da0 0x5619058f8e80 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80053a0 tx=0x7f2fa8005010 comp rx=0 tx=0).stop 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.645+0000 7f2fb95cd740 1 -- 172.21.15.7:0/720077287 >> 172.21.15.7:0/720077287 conn(0x5619058d2310 msgr2=0x5619059045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb95cd740 1 -- 172.21.15.7:0/720077287 shutdown_connections 2024-03-26T08:48:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb95cd740 1 -- 172.21.15.7:0/720077287 wait complete. 2024-03-26T08:48:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb95cd740 20 client.26908 trim_cache size 0 max 16384 2024-03-26T08:48:49.645 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:48:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:48:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:48:49.645 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:48:49.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb690a640 1 -- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f980a2900 msgr2=0x7f2f980a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f980a2900 0x7f2f980a4dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x5619058d7ff0 tx=0x7f2fa0005f90 comp rx=0 tx=0).stop 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb690a640 1 -- 172.21.15.7:0/2094134833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb00756e0 msgr2=0x7f2fb019e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb00756e0 0x7f2fb019e590 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2fa402c170 tx=0x7f2fa402c650 comp rx=0 tx=0).stop 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb690a640 1 -- 172.21.15.7:0/2094134833 shutdown_connections 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2f980a2900 0x7f2f980a4dc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb010dca0 0x7f2fb01a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.646+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2fb00756e0 0x7f2fb019e590 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.647+0000 7f2fb690a640 1 --2- 172.21.15.7:0/2094134833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb0074100 0x7f2fb019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.647+0000 7f2fb690a640 1 -- 172.21.15.7:0/2094134833 >> 172.21.15.7:0/2094134833 conn(0x7f2fb00fcc60 msgr2=0x7f2fb0101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.647+0000 7f2fb690a640 1 -- 172.21.15.7:0/2094134833 shutdown_connections 2024-03-26T08:48:49.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.647+0000 7f2fb690a640 1 -- 172.21.15.7:0/2094134833 wait complete. 2024-03-26T08:48:49.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:48:49.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:48:49.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.817+0000 7fd4bf40a640 1 -- 172.21.15.7:0/235454497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4b8108db0 msgr2=0x7fd4b8109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.818+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/235454497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4b8108db0 0x7fd4b8109230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd4a801a910 tx=0x7fd4a803cca0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.818+0000 7fd4bf40a640 1 -- 172.21.15.7:0/235454497 shutdown_connections 2024-03-26T08:48:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.818+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/235454497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b8109770 0x7fd4b8110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.818+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/235454497 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4b8108db0 0x7fd4b8109230 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.818+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/235454497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b8107bc0 0x7fd4b8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.818+0000 7fd4bf40a640 1 -- 172.21.15.7:0/235454497 >> 172.21.15.7:0/235454497 conn(0x7fd4b8076b60 msgr2=0x7fd4b8078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.818+0000 7fd4bf40a640 1 -- 172.21.15.7:0/235454497 shutdown_connections 2024-03-26T08:48:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.818+0000 7fd4bf40a640 1 -- 172.21.15.7:0/235454497 wait complete. 2024-03-26T08:48:49.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.819+0000 7fd4bf40a640 1 Processor -- start 2024-03-26T08:48:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.819+0000 7fd4bf40a640 1 -- start start 2024-03-26T08:48:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.819+0000 7fd4bf40a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4b8107bc0 0x7fd4b807a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.819+0000 7fd4bf40a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b8108db0 0x7fd4b807a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.819+0000 7fd4bf40a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b8109770 0x7fd4b8081220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.819+0000 7fd4bf40a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b806dd20 con 0x7fd4b8108db0 2024-03-26T08:48:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.819+0000 7fd4bf40a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b806dba0 con 0x7fd4b8109770 2024-03-26T08:48:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.819+0000 7fd4bf40a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b806dea0 con 0x7fd4b8107bc0 2024-03-26T08:48:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4be408640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4b8107bc0 0x7fd4b807a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4bec09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b8109770 0x7fd4b8081220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4bec09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b8109770 0x7fd4b8081220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41382/0 (socket says 172.21.15.7:41382) 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4bec09640 1 -- 172.21.15.7:0/2721949674 learned_addr learned my addr 172.21.15.7:0/2721949674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4bdc07640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b8108db0 0x7fd4b807a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4bec09640 1 -- 172.21.15.7:0/2721949674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4b8107bc0 msgr2=0x7fd4b807a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4bec09640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4b8107bc0 0x7fd4b807a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4bec09640 1 -- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b8108db0 msgr2=0x7fd4b807a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4bec09640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b8108db0 0x7fd4b807a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4bec09640 1 -- 172.21.15.7:0/2721949674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b8081890 con 0x7fd4b8109770 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.820+0000 7fd4be408640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4b8107bc0 0x7fd4b807a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.821+0000 7fd4bec09640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b8109770 0x7fd4b8081220 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000eaa0 tx=0x7fd4b000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.821+0000 7fd4af7fe640 1 -- 172.21.15.7:0/2721949674 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000cdb0 con 0x7fd4b8109770 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.821+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b8081b80 con 0x7fd4b8109770 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.821+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b8082120 con 0x7fd4b8109770 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.822+0000 7fd4af7fe640 1 -- 172.21.15.7:0/2721949674 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4b0005020 con 0x7fd4b8109770 2024-03-26T08:48:49.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.822+0000 7fd4af7fe640 1 -- 172.21.15.7:0/2721949674 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b00116b0 con 0x7fd4b8109770 2024-03-26T08:48:49.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.824+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49c0052c0 con 0x7fd4b8109770 2024-03-26T08:48:49.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.826+0000 7fd4af7fe640 1 -- 172.21.15.7:0/2721949674 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd4b0004a40 con 0x7fd4b8109770 2024-03-26T08:48:49.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.826+0000 7fd4af7fe640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4a40a29c0 0x7fd4a40a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:49.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.827+0000 7fd4be408640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4a40a29c0 0x7fd4a40a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:49.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.827+0000 7fd4af7fe640 1 -- 172.21.15.7:0/2721949674 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd4b00ceb20 con 0x7fd4b8109770 2024-03-26T08:48:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.828+0000 7fd4be408640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4a40a29c0 0x7fd4a40a4e80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4006fd0 tx=0x7fd4b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:49.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:49.835+0000 7fd4af7fe640 1 -- 172.21.15.7:0/2721949674 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd4b0016030 con 0x7fd4b8109770 2024-03-26T08:48:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.186+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd49c0055b0 con 0x7fd4b8109770 2024-03-26T08:48:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.187+0000 7fd4af7fe640 1 -- 172.21.15.7:0/2721949674 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fd4b009bb80 con 0x7fd4b8109770 2024-03-26T08:48:50.180 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:48:50.180 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:48:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4a40a29c0 msgr2=0x7fd4a40a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4a40a29c0 0x7fd4a40a4e80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4006fd0 tx=0x7fd4b4008040 comp rx=0 tx=0).stop 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b8109770 msgr2=0x7fd4b8081220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b8109770 0x7fd4b8081220 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000eaa0 tx=0x7fd4b000ef60 comp rx=0 tx=0).stop 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 shutdown_connections 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd4a40a29c0 0x7fd4a40a4e80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4b8109770 0x7fd4b8081220 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4b8108db0 0x7fd4b807a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 --2- 172.21.15.7:0/2721949674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd4b8107bc0 0x7fd4b807a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 >> 172.21.15.7:0/2721949674 conn(0x7fd4b8076b60 msgr2=0x7fd4b8070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 shutdown_connections 2024-03-26T08:48:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.189+0000 7fd4bf40a640 1 -- 172.21.15.7:0/2721949674 wait complete. 2024-03-26T08:48:50.200 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:48:50.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:48:50.200 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:49 smithi007 ceph-mon[32462]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 880 op/s 2024-03-26T08:48:50.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:49 smithi016 ceph-mon[35851]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 880 op/s 2024-03-26T08:48:50.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:49 smithi044 ceph-mon[36461]: pgmap v552: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 880 op/s 2024-03-26T08:48:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.363+0000 7f0d18a4b640 1 -- 172.21.15.7:0/1339732438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d141016f0 msgr2=0x7f0d14101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.363+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/1339732438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d141016f0 0x7f0d14101b10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009900 tx=0x7f0d0402bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.364+0000 7f0d18a4b640 1 -- 172.21.15.7:0/1339732438 shutdown_connections 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.364+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/1339732438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d141032a0 0x7f0d14109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.364+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/1339732438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141028e0 0x7f0d14102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.364+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/1339732438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d141016f0 0x7f0d14101b10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.364+0000 7f0d18a4b640 1 -- 172.21.15.7:0/1339732438 >> 172.21.15.7:0/1339732438 conn(0x7f0d140fcc60 msgr2=0x7f0d140ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.364+0000 7f0d18a4b640 1 -- 172.21.15.7:0/1339732438 shutdown_connections 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.364+0000 7f0d18a4b640 1 -- 172.21.15.7:0/1339732438 wait complete. 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.364+0000 7f0d18a4b640 1 Processor -- start 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.364+0000 7f0d18a4b640 1 -- start start 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d18a4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d141016f0 0x7f0d14199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d18a4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d141028e0 0x7f0d1419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d18a4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141032a0 0x7f0d141a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d18a4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140746d0 con 0x7f0d141028e0 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d18a4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14074550 con 0x7f0d141032a0 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d18a4b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14074850 con 0x7f0d141016f0 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d1259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d141016f0 0x7f0d14199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d1259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d141016f0 0x7f0d14199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43868/0 (socket says 172.21.15.7:43868) 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d1259c640 1 -- 172.21.15.7:0/4158592302 learned_addr learned my addr 172.21.15.7:0/4158592302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d12d9d640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141032a0 0x7f0d141a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d141028e0 0x7f0d1419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d1259c640 1 -- 172.21.15.7:0/4158592302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141032a0 msgr2=0x7f0d141a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d1259c640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141032a0 0x7f0d141a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d1259c640 1 -- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d141028e0 msgr2=0x7f0d1419a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d1259c640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d141028e0 0x7f0d1419a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d1259c640 1 -- 172.21.15.7:0/4158592302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d141a1800 con 0x7f0d141016f0 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.365+0000 7f0d1259c640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d141016f0 0x7f0d14199c30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000c00 tx=0x7f0d0402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.366+0000 7f0d037fe640 1 -- 172.21.15.7:0/4158592302 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04043020 con 0x7f0d141016f0 2024-03-26T08:48:50.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.366+0000 7f0d037fe640 1 -- 172.21.15.7:0/4158592302 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d040054e0 con 0x7f0d141016f0 2024-03-26T08:48:50.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.366+0000 7f0d037fe640 1 -- 172.21.15.7:0/4158592302 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04003c40 con 0x7f0d141016f0 2024-03-26T08:48:50.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.366+0000 7f0d18a4b640 1 -- 172.21.15.7:0/4158592302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d141a1a90 con 0x7f0d141016f0 2024-03-26T08:48:50.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.366+0000 7f0d18a4b640 1 -- 172.21.15.7:0/4158592302 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d141a1f90 con 0x7f0d141016f0 2024-03-26T08:48:50.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.369+0000 7f0d037fe640 1 -- 172.21.15.7:0/4158592302 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0d04046020 con 0x7f0d141016f0 2024-03-26T08:48:50.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.370+0000 7f0d037fe640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cfc0a2880 0x7f0cfc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.370+0000 7f0d037fe640 1 -- 172.21.15.7:0/4158592302 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0d040f0aa0 con 0x7f0d141016f0 2024-03-26T08:48:50.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.370+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cfc0a2880 0x7f0cfc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cfc0a2880 0x7f0cfc0a4d40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0d1419b0f0 tx=0x7f0d08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d1b633740 1 Processor -- start 2024-03-26T08:48:50.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d1b633740 1 -- start start 2024-03-26T08:48:50.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 0x562b2f34cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34d080 0x562b2f351b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f354680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d1b633740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2f30f610 con 0x562b2f34c690 2024-03-26T08:48:50.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d1b633740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2f2b23c0 con 0x562b2f3521f0 2024-03-26T08:48:50.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d1b633740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2f25ad60 con 0x562b2f34d080 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d1259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 0x562b2f34cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f354680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d12d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f354680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41416/0 (socket says 172.21.15.7:41416) 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.371+0000 7f0d12d9d640 1 -- 172.21.15.7:0/1763242122 learned_addr learned my addr 172.21.15.7:0/1763242122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/1763242122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34d080 0x562b2f351b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d12d9d640 1 -- 172.21.15.7:0/1763242122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34d080 msgr2=0x562b2f351b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d12d9d640 1 --2- 172.21.15.7:0/1763242122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34d080 0x562b2f351b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d12d9d640 1 -- 172.21.15.7:0/1763242122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 msgr2=0x562b2f34cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d12d9d640 1 --2- 172.21.15.7:0/1763242122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 0x562b2f34cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d12d9d640 1 -- 172.21.15.7:0/1763242122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b2efc63f0 con 0x562b2f3521f0 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/1763242122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34d080 0x562b2f351b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d12d9d640 1 --2- 172.21.15.7:0/1763242122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f354680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009fd0 tx=0x7f0d0c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=814cb10d9ec76693 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d00ff9640 1 -- 172.21.15.7:0/1763242122 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c004d20 con 0x562b2f3521f0 2024-03-26T08:48:50.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d00ff9640 1 -- 172.21.15.7:0/1763242122 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c004050 con 0x562b2f3521f0 2024-03-26T08:48:50.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.372+0000 7f0d00ff9640 1 -- 172.21.15.7:0/1763242122 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c005260 con 0x562b2f3521f0 2024-03-26T08:48:50.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.373+0000 7f0d1b633740 1 -- 172.21.15.7:0/1763242122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 msgr2=0x562b2f354680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.373+0000 7f0d1b633740 1 --2- 172.21.15.7:0/1763242122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f354680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009fd0 tx=0x7f0d0c0118d0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.373+0000 7f0d1b633740 1 -- 172.21.15.7:0/1763242122 shutdown_connections 2024-03-26T08:48:50.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.373+0000 7f0d1b633740 1 --2- 172.21.15.7:0/1763242122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f354680 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.373+0000 7f0d1b633740 1 --2- 172.21.15.7:0/1763242122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34d080 0x562b2f351b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.373+0000 7f0d1b633740 1 --2- 172.21.15.7:0/1763242122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 0x562b2f34cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.373+0000 7f0d1b633740 1 -- 172.21.15.7:0/1763242122 >> 172.21.15.7:0/1763242122 conn(0x562b2f347c00 msgr2=0x562b2f348000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.373+0000 7f0d1b633740 1 -- 172.21.15.7:0/1763242122 shutdown_connections 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.373+0000 7f0d1b633740 1 -- 172.21.15.7:0/1763242122 wait complete. 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1b633740 1 -- start start 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1b633740 1 Processor -- start 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34c690 0x562b2f3569f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f356f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 0x562b2f35df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1b633740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2f26e240 con 0x562b2f34d080 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1b633740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2f2ae460 con 0x562b2f34c690 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1b633740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2f3551a0 con 0x562b2f3521f0 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1b633740 10 client.0 _ll_register_callbacks cb 0x562b2f26dc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d11d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f356f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d1259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34c690 0x562b2f3569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d11d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f356f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38144/0 (socket says 172.21.15.7:38144) 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d11d9b640 1 -- 172.21.15.7:0/408995659 learned_addr learned my addr 172.21.15.7:0/408995659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.374+0000 7f0d12d9d640 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 0x562b2f35df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d1259c640 1 -- 172.21.15.7:0/408995659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 msgr2=0x562b2f35df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d1259c640 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 0x562b2f35df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d1259c640 1 -- 172.21.15.7:0/408995659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 msgr2=0x562b2f356f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d1259c640 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f356f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d1259c640 1 -- 172.21.15.7:0/408995659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b2f0907d0 con 0x562b2f34c690 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d12d9d640 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 0x562b2f35df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d1259c640 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34c690 0x562b2f3569f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d040bd3d0 tx=0x7f0d0402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31493e72ba4aa3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/408995659 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040d6060 con 0x562b2f34c690 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/408995659 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0d040bb740 con 0x562b2f34c690 2024-03-26T08:48:50.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d1b633740 1 -- 172.21.15.7:0/408995659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d0c004050 con 0x562b2f34c690 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/408995659 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d04005690 con 0x562b2f34c690 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d1b633740 10 client.26888 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.375+0000 7f0d1b633740 1 -- 172.21.15.7:0/408995659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b2f3612e0 con 0x562b2f34c690 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.376+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/408995659 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c004050 con 0x562b2f34c690 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.376+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/408995659 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0d040c1170 con 0x562b2f34c690 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.376+0000 7f0d1b633740 10 client.26888 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.376+0000 7f0d1b633740 10 client.26888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.376+0000 7f0d1b633740 20 client.26888 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.376+0000 7f0d1b633740 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562b2f362900 0x562b2f364d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.376+0000 7f0d1b633740 4 client.26888 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.376+0000 7f0d1b633740 1 -- 172.21.15.7:0/408995659 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b2f2bf900 con 0x562b2f362900 2024-03-26T08:48:50.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.376+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562b2f362900 0x562b2f364d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.379+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562b2f362900 0x562b2f364d80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b95eddd620f3aed7 server_cookie=aa2b5121db0c9592 in_seq=0 out_seq=0 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.379+0000 7f0cf9ffb640 10 client.26888 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.379+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/408995659 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562b2f2bf900 con 0x562b2f362900 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.379+0000 7f0cf9ffb640 10 client.26888 handle_command_reply: tid=0 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 client.26888 shutdown 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 -- 172.21.15.7:0/408995659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34c690 msgr2=0x562b2f3569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34c690 0x562b2f3569f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d040bd3d0 tx=0x7f0d0402bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 -- 172.21.15.7:0/408995659 shutdown_connections 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562b2f362900 0x562b2f364d80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0cf9ffb640 0 client.26888 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 0x562b2f35df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f356f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 --2- 172.21.15.7:0/408995659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34c690 0x562b2f3569f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 -- 172.21.15.7:0/408995659 >> 172.21.15.7:0/408995659 conn(0x562b2f347c00 msgr2=0x562b2f346e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 -- 172.21.15.7:0/408995659 shutdown_connections 2024-03-26T08:48:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 1 -- 172.21.15.7:0/408995659 wait complete. 2024-03-26T08:48:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.380+0000 7f0d1b633740 20 client.26888 trim_cache size 0 max 16384 2024-03-26T08:48:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.399+0000 7f0d1b633740 1 Processor -- start 2024-03-26T08:48:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.399+0000 7f0d1b633740 1 -- start start 2024-03-26T08:48:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 0x562b2f347fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34d080 0x562b2f35df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 0x562b2f34e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d1b633740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141ad240 con 0x562b2f34c690 2024-03-26T08:48:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d1b633740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141aad40 con 0x562b2f34d080 2024-03-26T08:48:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d1b633740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141a8dc0 con 0x562b2f3521f0 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d11d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34d080 0x562b2f35df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d1259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 0x562b2f347fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d12d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 0x562b2f34e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d11d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34d080 0x562b2f35df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41430/0 (socket says 172.21.15.7:41430) 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d11d9b640 1 -- 172.21.15.7:0/1921175481 learned_addr learned my addr 172.21.15.7:0/1921175481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d11d9b640 1 -- 172.21.15.7:0/1921175481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 msgr2=0x562b2f34e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/1921175481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 0x562b2f34e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d11d9b640 1 -- 172.21.15.7:0/1921175481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 msgr2=0x562b2f347fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/1921175481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 0x562b2f347fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d11d9b640 1 -- 172.21.15.7:0/1921175481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b2f348970 con 0x562b2f34d080 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/1921175481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34d080 0x562b2f35df30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x562b2f357ed0 tx=0x7f0d08014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b38bc8b66e24066 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.400+0000 7f0cfbfff640 1 -- 172.21.15.7:0/1921175481 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d08004020 con 0x562b2f34d080 2024-03-26T08:48:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0cfbfff640 1 -- 172.21.15.7:0/1921175481 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d0802bc20 con 0x562b2f34d080 2024-03-26T08:48:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0cfbfff640 1 -- 172.21.15.7:0/1921175481 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080039b0 con 0x562b2f34d080 2024-03-26T08:48:50.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0d1b633740 1 -- 172.21.15.7:0/1921175481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34d080 msgr2=0x562b2f35df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0d1b633740 1 --2- 172.21.15.7:0/1921175481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34d080 0x562b2f35df30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x562b2f357ed0 tx=0x7f0d08014000 comp rx=0 tx=0).stop 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0d1b633740 1 -- 172.21.15.7:0/1921175481 shutdown_connections 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0d1b633740 1 --2- 172.21.15.7:0/1921175481 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f3521f0 0x562b2f34e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0d1b633740 1 --2- 172.21.15.7:0/1921175481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f34d080 0x562b2f35df30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0d1b633740 1 --2- 172.21.15.7:0/1921175481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34c690 0x562b2f347fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0d1b633740 1 -- 172.21.15.7:0/1921175481 >> 172.21.15.7:0/1921175481 conn(0x562b2f347c00 msgr2=0x562b2f355540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0d1b633740 1 -- 172.21.15.7:0/1921175481 shutdown_connections 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.401+0000 7f0d1b633740 1 -- 172.21.15.7:0/1921175481 wait complete. 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.402+0000 7f0d1b633740 1 -- start start 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.402+0000 7f0d1b633740 1 Processor -- start 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.402+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34c690 0x562b2f35ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.402+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f35bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.402+0000 7f0d1b633740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f35cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.402+0000 7f0d1259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34c690 0x562b2f35ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.402+0000 7f0d1259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34c690 0x562b2f35ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43912/0 (socket says 172.21.15.7:43912) 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.402+0000 7f0d11d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f35bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.402+0000 7f0d11d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f35bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38168/0 (socket says 172.21.15.7:38168) 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1b633740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141a93c0 con 0x562b2f34d080 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1b633740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141aa6e0 con 0x562b2f3521f0 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1b633740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141a9a20 con 0x562b2f34c690 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1259c640 1 -- 172.21.15.7:0/3727448413 learned_addr learned my addr 172.21.15.7:0/3727448413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1b633740 10 client.0 _ll_register_callbacks cb 0x7f0d040bd030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d12d9d640 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f35cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1259c640 1 -- 172.21.15.7:0/3727448413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 msgr2=0x562b2f35cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1259c640 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f35cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1259c640 1 -- 172.21.15.7:0/3727448413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 msgr2=0x562b2f35bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1259c640 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f35bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1259c640 1 -- 172.21.15.7:0/3727448413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b2f35d550 con 0x562b2f34c690 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.403+0000 7f0d1259c640 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34c690 0x562b2f35ba60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0d04005e90 tx=0x7f0d040bd2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=338a099708cfc2d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.404+0000 7f0cf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.404+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/3727448413 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040bcda0 con 0x562b2f34c690 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.404+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/3727448413 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0d040c5750 con 0x562b2f34c690 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.404+0000 7f0d1b633740 1 -- 172.21.15.7:0/3727448413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d0802bc20 con 0x562b2f34c690 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.404+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/3727448413 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d040c5c30 con 0x562b2f34c690 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.405+0000 7f0d1b633740 10 client.26932 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:50.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.405+0000 7f0d1b633740 1 -- 172.21.15.7:0/3727448413 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b2f3be640 con 0x562b2f34c690 2024-03-26T08:48:50.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.406+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/3727448413 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d04004c60 con 0x562b2f34c690 2024-03-26T08:48:50.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.406+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/3727448413 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0d040bc890 con 0x562b2f34c690 2024-03-26T08:48:50.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.406+0000 7f0d1b633740 10 client.26932 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:50.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.406+0000 7f0d1b633740 10 client.26932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:48:50.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.406+0000 7f0d1b633740 20 client.26932 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:50.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.406+0000 7f0d1b633740 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562b2f362900 0x562b2f3c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.406+0000 7f0d1b633740 4 client.26932 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:48:50.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.406+0000 7f0d1b633740 1 -- 172.21.15.7:0/3727448413 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f0d0802bc20 con 0x562b2f362900 2024-03-26T08:48:50.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.407+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562b2f362900 0x562b2f3c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.420+0000 7f0d11d9b640 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562b2f362900 0x562b2f3c0de0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4a2d3312926795a server_cookie=5bfbfdc989b6ddb2 in_seq=0 out_seq=0 2024-03-26T08:48:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.420+0000 7f0cf9ffb640 10 client.26932 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:48:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.422+0000 7f0cf9ffb640 1 -- 172.21.15.7:0/3727448413 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0d0802bc20 con 0x562b2f362900 2024-03-26T08:48:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.422+0000 7f0cf9ffb640 10 client.26932 handle_command_reply: tid=0 2024-03-26T08:48:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.422+0000 7f0d1b633740 1 client.26932 shutdown 2024-03-26T08:48:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.426+0000 7f0d1b633740 1 -- 172.21.15.7:0/3727448413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34c690 msgr2=0x562b2f35ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.426+0000 7f0d1b633740 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34c690 0x562b2f35ba60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0d04005e90 tx=0x7f0d040bd2c0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.428+0000 7f0d1b633740 1 -- 172.21.15.7:0/3727448413 shutdown_connections 2024-03-26T08:48:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.428+0000 7f0d1b633740 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562b2f362900 0x562b2f3c0de0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.429+0000 7f0d1b633740 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b2f3521f0 0x562b2f35cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.429+0000 7f0d1b633740 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b2f34d080 0x562b2f35bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.429+0000 7f0d1b633740 1 --2- 172.21.15.7:0/3727448413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b2f34c690 0x562b2f35ba60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.429+0000 7f0d1b633740 1 -- 172.21.15.7:0/3727448413 >> 172.21.15.7:0/3727448413 conn(0x562b2f347c00 msgr2=0x562b2f347fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.429+0000 7f0cf9ffb640 0 client.26932 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:48:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.430+0000 7f0d1b633740 1 -- 172.21.15.7:0/3727448413 shutdown_connections 2024-03-26T08:48:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.430+0000 7f0d1b633740 1 -- 172.21.15.7:0/3727448413 wait complete. 2024-03-26T08:48:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.430+0000 7f0d1b633740 20 client.26932 trim_cache size 0 max 16384 2024-03-26T08:48:50.424 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:48:50.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.433+0000 7f0d18a4b640 1 -- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cfc0a2880 msgr2=0x7f0cfc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.434+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cfc0a2880 0x7f0cfc0a4d40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0d1419b0f0 tx=0x7f0d08005f90 comp rx=0 tx=0).stop 2024-03-26T08:48:50.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.434+0000 7f0d18a4b640 1 -- 172.21.15.7:0/4158592302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d141016f0 msgr2=0x7f0d14199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.434+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d141016f0 0x7f0d14199c30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000c00 tx=0x7f0d0402c650 comp rx=0 tx=0).stop 2024-03-26T08:48:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.434+0000 7f0d18a4b640 1 -- 172.21.15.7:0/4158592302 shutdown_connections 2024-03-26T08:48:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.434+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0cfc0a2880 0x7f0cfc0a4d40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.434+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d141032a0 0x7f0d141a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.434+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d141028e0 0x7f0d1419a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.434+0000 7f0d18a4b640 1 --2- 172.21.15.7:0/4158592302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0d141016f0 0x7f0d14199c30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.435+0000 7f0d18a4b640 1 -- 172.21.15.7:0/4158592302 >> 172.21.15.7:0/4158592302 conn(0x7f0d140fcc60 msgr2=0x7f0d140fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.435+0000 7f0d18a4b640 1 -- 172.21.15.7:0/4158592302 shutdown_connections 2024-03-26T08:48:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.435+0000 7f0d18a4b640 1 -- 172.21.15.7:0/4158592302 wait complete. 2024-03-26T08:48:50.446 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:48:50.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:48:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.605+0000 7f04a719f640 1 -- 172.21.15.7:0/2322667955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04a01003f0 msgr2=0x7f04a0100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.605+0000 7f04a719f640 1 --2- 172.21.15.7:0/2322667955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04a01003f0 0x7f04a0100810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f049c009900 tx=0x7f049c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.605+0000 7f04a719f640 1 -- 172.21.15.7:0/2322667955 shutdown_connections 2024-03-26T08:48:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.605+0000 7f04a719f640 1 --2- 172.21.15.7:0/2322667955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a0101fa0 0x7f04a0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.605+0000 7f04a719f640 1 --2- 172.21.15.7:0/2322667955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a01015e0 0x7f04a0101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.605+0000 7f04a719f640 1 --2- 172.21.15.7:0/2322667955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04a01003f0 0x7f04a0100810 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.605+0000 7f04a719f640 1 -- 172.21.15.7:0/2322667955 >> 172.21.15.7:0/2322667955 conn(0x7f04a00fb980 msgr2=0x7f04a00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.605+0000 7f04a719f640 1 -- 172.21.15.7:0/2322667955 shutdown_connections 2024-03-26T08:48:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.605+0000 7f04a719f640 1 -- 172.21.15.7:0/2322667955 wait complete. 2024-03-26T08:48:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.606+0000 7f04a719f640 1 Processor -- start 2024-03-26T08:48:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.606+0000 7f04a719f640 1 -- start start 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a719f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04a01003f0 0x7f04a00694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a719f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a01015e0 0x7f04a019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a719f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a0101fa0 0x7f04a019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a719f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a010c050 con 0x7f04a01015e0 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a719f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a010bed0 con 0x7f04a01003f0 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a719f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a010c1d0 con 0x7f04a0101fa0 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a599c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a01015e0 0x7f04a019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a599c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a01015e0 0x7f04a019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38198/0 (socket says 172.21.15.7:38198) 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a599c640 1 -- 172.21.15.7:0/2005321775 learned_addr learned my addr 172.21.15.7:0/2005321775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a699e640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a0101fa0 0x7f04a019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a619d640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04a01003f0 0x7f04a00694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a599c640 1 -- 172.21.15.7:0/2005321775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a0101fa0 msgr2=0x7f04a019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a599c640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a0101fa0 0x7f04a019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a599c640 1 -- 172.21.15.7:0/2005321775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04a01003f0 msgr2=0x7f04a00694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a599c640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04a01003f0 0x7f04a00694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a599c640 1 -- 172.21.15.7:0/2005321775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a019f4d0 con 0x7f04a01015e0 2024-03-26T08:48:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a619d640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04a01003f0 0x7f04a00694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.607+0000 7f04a599c640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a01015e0 0x7f04a019c5a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0498009d50 tx=0x7f049800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.608+0000 7f04977fe640 1 -- 172.21.15.7:0/2005321775 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498003b70 con 0x7f04a01015e0 2024-03-26T08:48:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.608+0000 7f04977fe640 1 -- 172.21.15.7:0/2005321775 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0498003d10 con 0x7f04a01015e0 2024-03-26T08:48:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.608+0000 7f04977fe640 1 -- 172.21.15.7:0/2005321775 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04980108f0 con 0x7f04a01015e0 2024-03-26T08:48:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.608+0000 7f04a719f640 1 -- 172.21.15.7:0/2005321775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a019f760 con 0x7f04a01015e0 2024-03-26T08:48:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.609+0000 7f04a719f640 1 -- 172.21.15.7:0/2005321775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a019fc60 con 0x7f04a01015e0 2024-03-26T08:48:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.613+0000 7f04977fe640 1 -- 172.21.15.7:0/2005321775 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0498004a40 con 0x7f04a01015e0 2024-03-26T08:48:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.613+0000 7f04977fe640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f048c0a2950 0x7f048c0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.613+0000 7f04977fe640 1 -- 172.21.15.7:0/2005321775 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f04980cf050 con 0x7f04a01015e0 2024-03-26T08:48:50.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.614+0000 7f04a619d640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f048c0a2950 0x7f048c0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.614+0000 7f04a619d640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f048c0a2950 0x7f048c0a4e10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f049c02c7b0 tx=0x7f049c02c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.615+0000 7f04aafea740 1 Processor -- start 2024-03-26T08:48:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.617+0000 7f04aafea740 1 -- start start 2024-03-26T08:48:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.618+0000 7f04aafea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a394a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.618+0000 7f04aafea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 0x56064a3995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.618+0000 7f04aafea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a399c30 0x56064a39c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.618+0000 7f04aafea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064a34e610 con 0x56064a394640 2024-03-26T08:48:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.618+0000 7f04aafea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064a2f13c0 con 0x56064a399c30 2024-03-26T08:48:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.618+0000 7f04aafea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064a299d60 con 0x56064a395030 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.618+0000 7f04a699e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a399c30 0x56064a39c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.619+0000 7f04a699e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a399c30 0x56064a39c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41476/0 (socket says 172.21.15.7:41476) 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.619+0000 7f04a699e640 1 -- 172.21.15.7:0/1998603686 learned_addr learned my addr 172.21.15.7:0/1998603686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.619+0000 7f04a619d640 1 --2- 172.21.15.7:0/1998603686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a394a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.619+0000 7f04a699e640 1 -- 172.21.15.7:0/1998603686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 msgr2=0x56064a3995c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.619+0000 7f04a699e640 1 --2- 172.21.15.7:0/1998603686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 0x56064a3995c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.619+0000 7f04a699e640 1 -- 172.21.15.7:0/1998603686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 msgr2=0x56064a394a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.619+0000 7f04a699e640 1 --2- 172.21.15.7:0/1998603686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a394a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.619+0000 7f04a699e640 1 -- 172.21.15.7:0/1998603686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56064a0053f0 con 0x56064a399c30 2024-03-26T08:48:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.620+0000 7f04a619d640 1 --2- 172.21.15.7:0/1998603686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a394a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:48:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.626+0000 7f04a699e640 1 --2- 172.21.15.7:0/1998603686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a399c30 0x56064a39c0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0490009c90 tx=0x7f0490011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=754b94a114e91a19 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.629+0000 7f0494bfe640 1 -- 172.21.15.7:0/1998603686 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490004230 con 0x56064a399c30 2024-03-26T08:48:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.629+0000 7f0494bfe640 1 -- 172.21.15.7:0/1998603686 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0490009e40 con 0x56064a399c30 2024-03-26T08:48:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.629+0000 7f0494bfe640 1 -- 172.21.15.7:0/1998603686 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04900197e0 con 0x56064a399c30 2024-03-26T08:48:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.633+0000 7f04aafea740 1 -- 172.21.15.7:0/1998603686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a399c30 msgr2=0x56064a39c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.633+0000 7f04aafea740 1 --2- 172.21.15.7:0/1998603686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a399c30 0x56064a39c0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0490009c90 tx=0x7f0490011810 comp rx=0 tx=0).stop 2024-03-26T08:48:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.638+0000 7f04aafea740 1 -- 172.21.15.7:0/1998603686 shutdown_connections 2024-03-26T08:48:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.638+0000 7f04aafea740 1 --2- 172.21.15.7:0/1998603686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a399c30 0x56064a39c0a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.638+0000 7f04aafea740 1 --2- 172.21.15.7:0/1998603686 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 0x56064a3995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.638+0000 7f04aafea740 1 --2- 172.21.15.7:0/1998603686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a394a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.638+0000 7f04aafea740 1 -- 172.21.15.7:0/1998603686 >> 172.21.15.7:0/1998603686 conn(0x56064a386d30 msgr2=0x56064a387130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.646+0000 7f04aafea740 1 -- 172.21.15.7:0/1998603686 shutdown_connections 2024-03-26T08:48:50.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.649+0000 7f04aafea740 1 -- 172.21.15.7:0/1998603686 wait complete. 2024-03-26T08:48:50.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.649+0000 7f04aafea740 1 -- start start 2024-03-26T08:48:50.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.650+0000 7f04aafea740 1 Processor -- start 2024-03-26T08:48:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.650+0000 7f04aafea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a39e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.650+0000 7f04aafea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a39ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.651+0000 7f04aafea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.651+0000 7f04aafea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064a2ad240 con 0x56064a394640 2024-03-26T08:48:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.651+0000 7f04aafea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064a2ed460 con 0x56064a395030 2024-03-26T08:48:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.651+0000 7f04aafea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064a39cbc0 con 0x56064a399c30 2024-03-26T08:48:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.651+0000 7f04a699e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.651+0000 7f04a699e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43958/0 (socket says 172.21.15.7:43958) 2024-03-26T08:48:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.651+0000 7f04a699e640 1 -- 172.21.15.7:0/1216719324 learned_addr learned my addr 172.21.15.7:0/1216719324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.651+0000 7f04aafea740 10 client.0 _ll_register_callbacks cb 0x56064a2acc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.652+0000 7f04a619d640 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a39e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.655+0000 7f04a599c640 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a39ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.656+0000 7f04a699e640 1 -- 172.21.15.7:0/1216719324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 msgr2=0x56064a39ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.656+0000 7f04a699e640 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a39ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.656+0000 7f04a699e640 1 -- 172.21.15.7:0/1216719324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 msgr2=0x56064a39e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.656+0000 7f04a699e640 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a39e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.656+0000 7f04a699e640 1 -- 172.21.15.7:0/1216719324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56064a0cf7d0 con 0x56064a399c30 2024-03-26T08:48:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.657+0000 7f04a699e640 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a5a40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0490011d10 tx=0x7f04900045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1963f1dd7e8816 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.658+0000 7f0489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.659+0000 7f0489ffb640 1 -- 172.21.15.7:0/1216719324 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0490027070 con 0x56064a399c30 2024-03-26T08:48:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.659+0000 7f04aafea740 1 -- 172.21.15.7:0/1216719324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56064a2fe900 con 0x56064a399c30 2024-03-26T08:48:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.659+0000 7f0489ffb640 1 -- 172.21.15.7:0/1216719324 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0490003800 con 0x56064a399c30 2024-03-26T08:48:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.663+0000 7f0489ffb640 1 -- 172.21.15.7:0/1216719324 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0490031050 con 0x56064a399c30 2024-03-26T08:48:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.665+0000 7f04aafea740 10 client.26941 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.665+0000 7f04aafea740 1 -- 172.21.15.7:0/1216719324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56064a3a8d20 con 0x56064a399c30 2024-03-26T08:48:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.666+0000 7f0489ffb640 1 -- 172.21.15.7:0/1216719324 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56064a2fe900 con 0x56064a399c30 2024-03-26T08:48:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.666+0000 7f0489ffb640 1 -- 172.21.15.7:0/1216719324 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0490018070 con 0x56064a399c30 2024-03-26T08:48:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.666+0000 7f04aafea740 10 client.26941 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.666+0000 7f04aafea740 10 client.26941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:48:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.666+0000 7f04aafea740 20 client.26941 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.666+0000 7f04aafea740 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56064a3aa120 0x56064a3ac510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.666+0000 7f04aafea740 4 client.26941 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:50.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.666+0000 7f04aafea740 1 -- 172.21.15.7:0/1216719324 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56064a2f8c50 con 0x56064a3aa120 2024-03-26T08:48:50.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.668+0000 7f04a619d640 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56064a3aa120 0x56064a3ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.670+0000 7f04a619d640 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56064a3aa120 0x56064a3ac510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae8a6f144ade59a7 server_cookie=388fa519db02c4d5 in_seq=0 out_seq=0 2024-03-26T08:48:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.670+0000 7f0489ffb640 10 client.26941 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:48:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.671+0000 7f0489ffb640 1 -- 172.21.15.7:0/1216719324 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56064a2f8c50 con 0x56064a3aa120 2024-03-26T08:48:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.671+0000 7f0489ffb640 10 client.26941 handle_command_reply: tid=0 2024-03-26T08:48:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.671+0000 7f04aafea740 1 client.26941 shutdown 2024-03-26T08:48:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.674+0000 7f04aafea740 1 -- 172.21.15.7:0/1216719324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 msgr2=0x56064a3a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.674+0000 7f04aafea740 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a5a40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0490011d10 tx=0x7f04900045b0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.676+0000 7f04aafea740 1 -- 172.21.15.7:0/1216719324 shutdown_connections 2024-03-26T08:48:50.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.676+0000 7f04aafea740 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56064a3aa120 0x56064a3ac510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.676+0000 7f04aafea740 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a5a40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.676+0000 7f04aafea740 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a39ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.676+0000 7f04aafea740 1 --2- 172.21.15.7:0/1216719324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a39e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.676+0000 7f04aafea740 1 -- 172.21.15.7:0/1216719324 >> 172.21.15.7:0/1216719324 conn(0x56064a386d30 msgr2=0x56064a393cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.680+0000 7f0489ffb640 0 client.26941 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:48:50.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.680+0000 7f04aafea740 1 -- 172.21.15.7:0/1216719324 shutdown_connections 2024-03-26T08:48:50.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.681+0000 7f04aafea740 1 -- 172.21.15.7:0/1216719324 wait complete. 2024-03-26T08:48:50.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.681+0000 7f04aafea740 20 client.26941 trim_cache size 0 max 16384 2024-03-26T08:48:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.705+0000 7f04aafea740 1 Processor -- start 2024-03-26T08:48:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.705+0000 7f04aafea740 1 -- start start 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04aafea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a394640 0x56064a3ac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04aafea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 0x56064a3a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04aafea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a399c30 0x56064a3b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04a619d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a394640 0x56064a3ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04a599c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 0x56064a3a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04a699e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a399c30 0x56064a3b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04a599c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 0x56064a3a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43962/0 (socket says 172.21.15.7:43962) 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04a699e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a399c30 0x56064a3b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38240/0 (socket says 172.21.15.7:38240) 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04a699e640 1 -- 172.21.15.7:0/1213601125 learned_addr learned my addr 172.21.15.7:0/1213601125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04aafea740 1 -- 172.21.15.7:0/1213601125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a8e10 con 0x56064a399c30 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04aafea740 1 -- 172.21.15.7:0/1213601125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a6910 con 0x56064a394640 2024-03-26T08:48:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.706+0000 7f04aafea740 1 -- 172.21.15.7:0/1213601125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a4990 con 0x56064a395030 2024-03-26T08:48:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f04a599c640 1 -- 172.21.15.7:0/1213601125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a394640 msgr2=0x56064a3ac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f04a599c640 1 --2- 172.21.15.7:0/1213601125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a394640 0x56064a3ac130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f04a599c640 1 -- 172.21.15.7:0/1213601125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a399c30 msgr2=0x56064a3b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f04a599c640 1 --2- 172.21.15.7:0/1213601125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a399c30 0x56064a3b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f04a599c640 1 -- 172.21.15.7:0/1213601125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56064a387aa0 con 0x56064a395030 2024-03-26T08:48:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f04a699e640 1 --2- 172.21.15.7:0/1213601125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a399c30 0x56064a3b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f04a599c640 1 --2- 172.21.15.7:0/1213601125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 0x56064a3a5a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x56064a395430 tx=0x7f049800cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3dcf0798cee754e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f048bfff640 1 -- 172.21.15.7:0/1213601125 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04980b3070 con 0x56064a395030 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f048bfff640 1 -- 172.21.15.7:0/1213601125 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f049809c0b0 con 0x56064a395030 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.707+0000 7f048bfff640 1 -- 172.21.15.7:0/1213601125 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0498020390 con 0x56064a395030 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.708+0000 7f04aafea740 1 -- 172.21.15.7:0/1213601125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 msgr2=0x56064a3a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.708+0000 7f04aafea740 1 --2- 172.21.15.7:0/1213601125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 0x56064a3a5a00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x56064a395430 tx=0x7f049800cf90 comp rx=0 tx=0).stop 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.708+0000 7f04aafea740 1 -- 172.21.15.7:0/1213601125 shutdown_connections 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.708+0000 7f04aafea740 1 --2- 172.21.15.7:0/1213601125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a399c30 0x56064a3b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.708+0000 7f04aafea740 1 --2- 172.21.15.7:0/1213601125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a395030 0x56064a3a5a00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.708+0000 7f04aafea740 1 --2- 172.21.15.7:0/1213601125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a394640 0x56064a3ac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.708+0000 7f04aafea740 1 -- 172.21.15.7:0/1213601125 >> 172.21.15.7:0/1213601125 conn(0x56064a386d30 msgr2=0x56064a39e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.708+0000 7f04aafea740 1 -- 172.21.15.7:0/1213601125 shutdown_connections 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.708+0000 7f04aafea740 1 -- 172.21.15.7:0/1213601125 wait complete. 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04aafea740 1 -- start start 2024-03-26T08:48:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04aafea740 1 Processor -- start 2024-03-26T08:48:50.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04aafea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a3a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04aafea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a3a3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04aafea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04aafea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a4f90 con 0x56064a394640 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04aafea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a62b0 con 0x56064a395030 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04aafea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a01a55f0 con 0x56064a399c30 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04aafea740 10 client.0 _ll_register_callbacks cb 0x7f0490004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04a619d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a3a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04a599c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a3a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04a599c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a3a3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41504/0 (socket says 172.21.15.7:41504) 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.709+0000 7f04a599c640 1 -- 172.21.15.7:0/3634932708 learned_addr learned my addr 172.21.15.7:0/3634932708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04a699e640 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04a599c640 1 -- 172.21.15.7:0/3634932708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 msgr2=0x56064a3a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04a599c640 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04a599c640 1 -- 172.21.15.7:0/3634932708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 msgr2=0x56064a3a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04a599c640 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a3a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04a599c640 1 -- 172.21.15.7:0/3634932708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56064a3a4940 con 0x56064a395030 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04a599c640 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a3a3390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0498009fd0 tx=0x7f049809b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db9b76f408d8549 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f0489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f0489ffb640 1 -- 172.21.15.7:0/3634932708 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04980b3050 con 0x56064a395030 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f0489ffb640 1 -- 172.21.15.7:0/3634932708 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f049809ae00 con 0x56064a395030 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04aafea740 1 -- 172.21.15.7:0/3634932708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f049809c0b0 con 0x56064a395030 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f0489ffb640 1 -- 172.21.15.7:0/3634932708 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0498020590 con 0x56064a395030 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04aafea740 10 client.26909 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.710+0000 7f04aafea740 1 -- 172.21.15.7:0/3634932708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56064a3a5c70 con 0x56064a395030 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.711+0000 7f0489ffb640 1 -- 172.21.15.7:0/3634932708 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f049809c0b0 con 0x56064a395030 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.711+0000 7f0489ffb640 1 -- 172.21.15.7:0/3634932708 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f049809cce0 con 0x56064a395030 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.711+0000 7f04aafea740 10 client.26909 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:50.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.711+0000 7f04aafea740 10 client.26909 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.711+0000 7f04aafea740 20 client.26909 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.711+0000 7f04aafea740 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56064a406010 0x56064a408430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.712+0000 7f04aafea740 4 client.26909 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.712+0000 7f04aafea740 1 -- 172.21.15.7:0/3634932708 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56064a276640 con 0x56064a406010 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.712+0000 7f04a619d640 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56064a406010 0x56064a408430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.712+0000 7f04a619d640 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56064a406010 0x56064a408430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93b350edf44191d8 server_cookie=5bafd7657c0eda3 in_seq=0 out_seq=0 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.712+0000 7f0489ffb640 10 client.26909 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.712+0000 7f0489ffb640 1 -- 172.21.15.7:0/3634932708 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56064a276640 con 0x56064a406010 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.712+0000 7f0489ffb640 10 client.26909 handle_command_reply: tid=0 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.712+0000 7f04aafea740 1 client.26909 shutdown 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.713+0000 7f04aafea740 1 -- 172.21.15.7:0/3634932708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 msgr2=0x56064a3a3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.713+0000 7f04aafea740 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a3a3390 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0498009fd0 tx=0x7f049809b9b0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.714+0000 7f04aafea740 1 -- 172.21.15.7:0/3634932708 shutdown_connections 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.714+0000 7f04aafea740 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56064a406010 0x56064a408430 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.714+0000 7f0489ffb640 0 client.26909 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.714+0000 7f04aafea740 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56064a399c30 0x56064a3a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.714+0000 7f04aafea740 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56064a395030 0x56064a3a3390 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.714+0000 7f04aafea740 1 --2- 172.21.15.7:0/3634932708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56064a394640 0x56064a3a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.715+0000 7f04aafea740 1 -- 172.21.15.7:0/3634932708 >> 172.21.15.7:0/3634932708 conn(0x56064a386d30 msgr2=0x56064a393440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.715+0000 7f04aafea740 1 -- 172.21.15.7:0/3634932708 shutdown_connections 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.715+0000 7f04aafea740 1 -- 172.21.15.7:0/3634932708 wait complete. 2024-03-26T08:48:50.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04aafea740 20 client.26909 trim_cache size 0 max 16384 2024-03-26T08:48:50.712 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:48:50.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 -- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f048c0a2950 msgr2=0x7f048c0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f048c0a2950 0x7f048c0a4e10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f049c02c7b0 tx=0x7f049c02c880 comp rx=0 tx=0).stop 2024-03-26T08:48:50.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 -- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a01015e0 msgr2=0x7f04a019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a01015e0 0x7f04a019c5a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0498009d50 tx=0x7f049800c6a0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 -- 172.21.15.7:0/2005321775 shutdown_connections 2024-03-26T08:48:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f048c0a2950 0x7f048c0a4e10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f04a0101fa0 0x7f04a019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04a01015e0 0x7f04a019c5a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 --2- 172.21.15.7:0/2005321775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04a01003f0 0x7f04a00694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.716+0000 7f04a719f640 1 -- 172.21.15.7:0/2005321775 >> 172.21.15.7:0/2005321775 conn(0x7f04a00fb980 msgr2=0x7f04a0104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.717+0000 7f04a719f640 1 -- 172.21.15.7:0/2005321775 shutdown_connections 2024-03-26T08:48:50.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.717+0000 7f04a719f640 1 -- 172.21.15.7:0/2005321775 wait complete. 2024-03-26T08:48:50.724 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:48:50.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:48:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.897+0000 7f4476144640 1 -- 172.21.15.7:0/2649269677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447010b010 msgr2=0x7f447010d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.897+0000 7f4476144640 1 --2- 172.21.15.7:0/2649269677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447010b010 0x7f447010d480 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4464009900 tx=0x7f446402bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.897+0000 7f4476144640 1 -- 172.21.15.7:0/2649269677 shutdown_connections 2024-03-26T08:48:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.897+0000 7f4476144640 1 --2- 172.21.15.7:0/2649269677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f447010d9c0 0x7f447010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.897+0000 7f4476144640 1 --2- 172.21.15.7:0/2649269677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f447010b010 0x7f447010d480 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.897+0000 7f4476144640 1 --2- 172.21.15.7:0/2649269677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44701086a0 0x7f447010aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.897+0000 7f4476144640 1 -- 172.21.15.7:0/2649269677 >> 172.21.15.7:0/2649269677 conn(0x7f447006e640 msgr2=0x7f4470070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.897+0000 7f4476144640 1 -- 172.21.15.7:0/2649269677 shutdown_connections 2024-03-26T08:48:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.897+0000 7f4476144640 1 -- 172.21.15.7:0/2649269677 wait complete. 2024-03-26T08:48:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.898+0000 7f4476144640 1 Processor -- start 2024-03-26T08:48:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.898+0000 7f4476144640 1 -- start start 2024-03-26T08:48:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f4476144640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44701086a0 0x7f44701a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f4476144640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447010b010 0x7f44701a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f4476144640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f447010d9c0 0x7f44701add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f4476144640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447011c190 con 0x7f44701086a0 2024-03-26T08:48:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f4476144640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447011c010 con 0x7f447010b010 2024-03-26T08:48:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f4476144640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447011c310 con 0x7f447010d9c0 2024-03-26T08:48:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f446effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447010b010 0x7f44701a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f446f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44701086a0 0x7f44701a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f446ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f447010d9c0 0x7f44701add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f446effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447010b010 0x7f44701a6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41530/0 (socket says 172.21.15.7:41530) 2024-03-26T08:48:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.899+0000 7f446effd640 1 -- 172.21.15.7:0/3825594926 learned_addr learned my addr 172.21.15.7:0/3825594926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.900+0000 7f446effd640 1 -- 172.21.15.7:0/3825594926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f447010d9c0 msgr2=0x7f44701add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.900+0000 7f446effd640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f447010d9c0 0x7f44701add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.900+0000 7f446effd640 1 -- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44701086a0 msgr2=0x7f44701a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.900+0000 7f446effd640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44701086a0 0x7f44701a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.900+0000 7f446effd640 1 -- 172.21.15.7:0/3825594926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44701ae3f0 con 0x7f447010b010 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.900+0000 7f446ffff640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f447010d9c0 0x7f44701add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.900+0000 7f446f7fe640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44701086a0 0x7f44701a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.900+0000 7f446effd640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447010b010 0x7f44701a6d10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f446402c240 tx=0x7f446402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.900+0000 7f446cff9640 1 -- 172.21.15.7:0/3825594926 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464043020 con 0x7f447010b010 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.901+0000 7f446cff9640 1 -- 172.21.15.7:0/3825594926 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44640054e0 con 0x7f447010b010 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.901+0000 7f4476144640 1 -- 172.21.15.7:0/3825594926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44701ae680 con 0x7f447010b010 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.901+0000 7f4476144640 1 -- 172.21.15.7:0/3825594926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44701aec20 con 0x7f447010b010 2024-03-26T08:48:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.901+0000 7f446cff9640 1 -- 172.21.15.7:0/3825594926 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464003d00 con 0x7f447010b010 2024-03-26T08:48:50.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.904+0000 7f446cff9640 1 -- 172.21.15.7:0/3825594926 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4464046020 con 0x7f447010b010 2024-03-26T08:48:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.905+0000 7f446cff9640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44580a2950 0x7f44580a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.906+0000 7f4478e05740 1 Processor -- start 2024-03-26T08:48:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.906+0000 7f446f7fe640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44580a2950 0x7f44580a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.906+0000 7f446cff9640 1 -- 172.21.15.7:0/3825594926 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44640f0cb0 con 0x7f447010b010 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.906+0000 7f4478e05740 1 -- start start 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.906+0000 7f4478e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9a690 0x55bbbef9aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f4478e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbef9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f4478e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f4478e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbbef5d610 con 0x55bbbefa01f0 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f4478e05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbbef003c0 con 0x55bbbef9b080 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f4478e05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbbeea8d60 con 0x55bbbef9a690 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9a690 0x55bbbef9aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbef9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbef9fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41538/0 (socket says 172.21.15.7:41538) 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446effd640 1 -- 172.21.15.7:0/3175315049 learned_addr learned my addr 172.21.15.7:0/3175315049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446f7fe640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44580a2950 0x7f44580a4e10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4460009b40 tx=0x7f4460005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446effd640 1 -- 172.21.15.7:0/3175315049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9a690 msgr2=0x55bbbef9aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446effd640 1 --2- 172.21.15.7:0/3175315049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9a690 0x55bbbef9aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446effd640 1 -- 172.21.15.7:0/3175315049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 msgr2=0x55bbbefa2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446effd640 1 --2- 172.21.15.7:0/3175315049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446effd640 1 -- 172.21.15.7:0/3175315049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbbec143f0 con 0x55bbbef9b080 2024-03-26T08:48:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f446effd640 1 --2- 172.21.15.7:0/3175315049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbef9fb80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f446402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67baebe822c19bb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.907+0000 7f4455ffb640 1 -- 172.21.15.7:0/3175315049 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640bcf70 con 0x55bbbef9b080 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4455ffb640 1 -- 172.21.15.7:0/3175315049 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44640b9040 con 0x55bbbef9b080 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4455ffb640 1 -- 172.21.15.7:0/3175315049 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446403cd80 con 0x55bbbef9b080 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4478e05740 1 -- 172.21.15.7:0/3175315049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 msgr2=0x55bbbef9fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4478e05740 1 --2- 172.21.15.7:0/3175315049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbef9fb80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f446402bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4478e05740 1 -- 172.21.15.7:0/3175315049 shutdown_connections 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4478e05740 1 --2- 172.21.15.7:0/3175315049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4478e05740 1 --2- 172.21.15.7:0/3175315049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbef9fb80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4478e05740 1 --2- 172.21.15.7:0/3175315049 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9a690 0x55bbbef9aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4478e05740 1 -- 172.21.15.7:0/3175315049 >> 172.21.15.7:0/3175315049 conn(0x55bbbef95c00 msgr2=0x55bbbef96000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4478e05740 1 -- 172.21.15.7:0/3175315049 shutdown_connections 2024-03-26T08:48:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.908+0000 7f4478e05740 1 -- 172.21.15.7:0/3175315049 wait complete. 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.909+0000 7f4478e05740 1 -- start start 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.909+0000 7f4478e05740 1 Processor -- start 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.909+0000 7f4478e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbefa4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.909+0000 7f4478e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefa4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f4478e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 0x55bbbefabc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f4478e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbbeebc240 con 0x55bbbef9a690 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbefa4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f4478e05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbbeefc460 con 0x55bbbef9b080 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f4478e05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbbefa2f60 con 0x55bbbefa01f0 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefa4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbefa4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38282/0 (socket says 172.21.15.7:38282) 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446f7fe640 1 -- 172.21.15.7:0/293670079 learned_addr learned my addr 172.21.15.7:0/293670079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446ffff640 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 0x55bbbefabc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f4478e05740 10 client.0 _ll_register_callbacks cb 0x55bbbeebbc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446effd640 1 -- 172.21.15.7:0/293670079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 msgr2=0x55bbbefabc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446effd640 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 0x55bbbefabc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446effd640 1 -- 172.21.15.7:0/293670079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 msgr2=0x55bbbefa4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446effd640 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbefa4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446effd640 1 -- 172.21.15.7:0/293670079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbbecde7d0 con 0x55bbbef9b080 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f446effd640 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefa4ca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f4464005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f1042d98cfda5a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f444effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.910+0000 7f444effd640 1 -- 172.21.15.7:0/293670079 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464005690 con 0x55bbbef9b080 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.911+0000 7f444effd640 1 -- 172.21.15.7:0/293670079 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44640bb910 con 0x55bbbef9b080 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.911+0000 7f4478e05740 1 -- 172.21.15.7:0/293670079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44640b9040 con 0x55bbbef9b080 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.911+0000 7f444effd640 1 -- 172.21.15.7:0/293670079 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44640bd610 con 0x55bbbef9b080 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.911+0000 7f4478e05740 10 client.26921 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.911+0000 7f4478e05740 1 -- 172.21.15.7:0/293670079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bbbefaefc0 con 0x55bbbef9b080 2024-03-26T08:48:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.911+0000 7f444effd640 1 -- 172.21.15.7:0/293670079 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44640bdd10 con 0x55bbbef9b080 2024-03-26T08:48:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.911+0000 7f444effd640 1 -- 172.21.15.7:0/293670079 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f44640a6ba0 con 0x55bbbef9b080 2024-03-26T08:48:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.913+0000 7f4478e05740 10 client.26921 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.913+0000 7f4478e05740 10 client.26921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.913+0000 7f4478e05740 20 client.26921 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.913+0000 7f4478e05740 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bbbefb0200 0x55bbbefb25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.914+0000 7f4478e05740 4 client.26921 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.914+0000 7f4478e05740 1 -- 172.21.15.7:0/293670079 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44640b9040 con 0x55bbbefb0200 2024-03-26T08:48:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.914+0000 7f446f7fe640 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bbbefb0200 0x55bbbefb25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f444effd640 10 client.26921 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f446f7fe640 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bbbefb0200 0x55bbbefb25f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967d3537926f2e73 server_cookie=1213a73f48442b0d in_seq=0 out_seq=0 2024-03-26T08:48:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f444effd640 1 -- 172.21.15.7:0/293670079 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f44640b9040 con 0x55bbbefb0200 2024-03-26T08:48:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f444effd640 10 client.26921 handle_command_reply: tid=0 2024-03-26T08:48:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f4478e05740 1 client.26921 shutdown 2024-03-26T08:48:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f4478e05740 1 -- 172.21.15.7:0/293670079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 msgr2=0x55bbbefa4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f4478e05740 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefa4ca0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4464000c00 tx=0x7f4464005b50 comp rx=0 tx=0).stop 2024-03-26T08:48:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f4478e05740 1 -- 172.21.15.7:0/293670079 shutdown_connections 2024-03-26T08:48:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f4478e05740 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bbbefb0200 0x55bbbefb25f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f4478e05740 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 0x55bbbefabc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f444effd640 0 client.26921 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f4478e05740 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefa4ca0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.915+0000 7f4478e05740 1 --2- 172.21.15.7:0/293670079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbefa4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.916+0000 7f4478e05740 1 -- 172.21.15.7:0/293670079 >> 172.21.15.7:0/293670079 conn(0x55bbbef95c00 msgr2=0x55bbbef94e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.916+0000 7f4478e05740 1 -- 172.21.15.7:0/293670079 shutdown_connections 2024-03-26T08:48:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.916+0000 7f4478e05740 1 -- 172.21.15.7:0/293670079 wait complete. 2024-03-26T08:48:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.916+0000 7f4478e05740 20 client.26921 trim_cache size 0 max 16384 2024-03-26T08:48:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.935+0000 7f4478e05740 1 Processor -- start 2024-03-26T08:48:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.935+0000 7f4478e05740 1 -- start start 2024-03-26T08:48:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f4478e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbef99570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f4478e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefabc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f4478e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 0x55bbbefb6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefabc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefabc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41564/0 (socket says 172.21.15.7:41564) 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446effd640 1 -- 172.21.15.7:0/323465200 learned_addr learned my addr 172.21.15.7:0/323465200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446ffff640 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 0x55bbbefb6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446f7fe640 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbef99570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f4478e05740 1 -- 172.21.15.7:0/323465200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0108c0 con 0x55bbbef9a690 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f4478e05740 1 -- 172.21.15.7:0/323465200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c00e3c0 con 0x55bbbef9b080 2024-03-26T08:48:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f4478e05740 1 -- 172.21.15.7:0/323465200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c00c440 con 0x55bbbefa01f0 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446effd640 1 -- 172.21.15.7:0/323465200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 msgr2=0x55bbbefb6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446effd640 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 0x55bbbefb6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446effd640 1 -- 172.21.15.7:0/323465200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 msgr2=0x55bbbef99570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446effd640 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbef99570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446effd640 1 -- 172.21.15.7:0/323465200 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbbefb2960 con 0x55bbbef9b080 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.936+0000 7f446f7fe640 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbef99570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.937+0000 7f446effd640 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefabc30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f44640b9fe0 tx=0x7f446402ce70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f59efcb35b4e58c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.937+0000 7f44557fa640 1 -- 172.21.15.7:0/323465200 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640bd150 con 0x55bbbef9b080 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.937+0000 7f44557fa640 1 -- 172.21.15.7:0/323465200 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44640b9040 con 0x55bbbef9b080 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.937+0000 7f44557fa640 1 -- 172.21.15.7:0/323465200 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44640a6ba0 con 0x55bbbef9b080 2024-03-26T08:48:50.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 -- 172.21.15.7:0/323465200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 msgr2=0x55bbbefabc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefabc30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f44640b9fe0 tx=0x7f446402ce70 comp rx=0 tx=0).stop 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 -- 172.21.15.7:0/323465200 shutdown_connections 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbefa01f0 0x55bbbefb6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9b080 0x55bbbefabc30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 --2- 172.21.15.7:0/323465200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbef9a690 0x55bbbef99570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 -- 172.21.15.7:0/323465200 >> 172.21.15.7:0/323465200 conn(0x55bbbef95c00 msgr2=0x55bbbef98e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 -- 172.21.15.7:0/323465200 shutdown_connections 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 -- 172.21.15.7:0/323465200 wait complete. 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 -- start start 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.938+0000 7f4478e05740 1 Processor -- start 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f4478e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9a690 0x55bbbefab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f4478e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9b080 0x55bbbefabc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f4478e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f4478e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c00ca40 con 0x55bbbefa01f0 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f4478e05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c00dd60 con 0x55bbbef9a690 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f4478e05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c00d0a0 con 0x55bbbef9b080 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9a690 0x55bbbefab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f4478e05740 10 client.0 _ll_register_callbacks cb 0x7f44640b98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9a690 0x55bbbefab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41566/0 (socket says 172.21.15.7:41566) 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446f7fe640 1 -- 172.21.15.7:0/233705590 learned_addr learned my addr 172.21.15.7:0/233705590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38304/0 (socket says 172.21.15.7:38304) 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446effd640 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9b080 0x55bbbefabc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446f7fe640 1 -- 172.21.15.7:0/233705590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9b080 msgr2=0x55bbbefabc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446f7fe640 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9b080 0x55bbbefabc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446f7fe640 1 -- 172.21.15.7:0/233705590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 msgr2=0x55bbbefa9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446f7fe640 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446f7fe640 1 -- 172.21.15.7:0/233705590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbbefaa630 con 0x55bbbef9a690 2024-03-26T08:48:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446ffff640 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f446f7fe640 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9a690 0x55bbbefab740 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f44600114e0 tx=0x7f44600046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec5cc761ccf2591 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.939+0000 7f444effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.942+0000 7f444effd640 1 -- 172.21.15.7:0/233705590 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4460004220 con 0x55bbbef9a690 2024-03-26T08:48:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.942+0000 7f444effd640 1 -- 172.21.15.7:0/233705590 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f44600043c0 con 0x55bbbef9a690 2024-03-26T08:48:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.942+0000 7f4478e05740 1 -- 172.21.15.7:0/233705590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44640b9040 con 0x55bbbef9a690 2024-03-26T08:48:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f444effd640 1 -- 172.21.15.7:0/233705590 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4460005030 con 0x55bbbef9a690 2024-03-26T08:48:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f4478e05740 10 client.26927 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f4478e05740 1 -- 172.21.15.7:0/233705590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bbbf00c6d0 con 0x55bbbef9a690 2024-03-26T08:48:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f444effd640 1 -- 172.21.15.7:0/233705590 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44640b9040 con 0x55bbbef9a690 2024-03-26T08:48:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f444effd640 1 -- 172.21.15.7:0/233705590 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f446000f040 con 0x55bbbef9a690 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f4478e05740 10 client.26927 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f4478e05740 10 client.26927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f4478e05740 20 client.26927 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f4478e05740 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bbbf00d010 0x55bbbf00f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f4478e05740 4 client.26927 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f4478e05740 1 -- 172.21.15.7:0/233705590 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bbbef0d900 con 0x55bbbf00d010 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.943+0000 7f446effd640 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bbbf00d010 0x55bbbf00f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f444effd640 10 client.26927 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f446effd640 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bbbf00d010 0x55bbbf00f430 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d44fa69c26f87 server_cookie=23707331497406d0 in_seq=0 out_seq=0 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f444effd640 1 -- 172.21.15.7:0/233705590 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bbbef0d900 con 0x55bbbf00d010 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f444effd640 10 client.26927 handle_command_reply: tid=0 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 client.26927 shutdown 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 -- 172.21.15.7:0/233705590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9a690 msgr2=0x55bbbefab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9a690 0x55bbbefab740 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f44600114e0 tx=0x7f44600046c0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 -- 172.21.15.7:0/233705590 shutdown_connections 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bbbf00d010 0x55bbbf00f430 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bbbefa01f0 0x55bbbefa9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bbbef9b080 0x55bbbefabc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 --2- 172.21.15.7:0/233705590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bbbef9a690 0x55bbbefab740 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f444effd640 0 client.26927 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 -- 172.21.15.7:0/233705590 >> 172.21.15.7:0/233705590 conn(0x55bbbef95c00 msgr2=0x55bbbef996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 -- 172.21.15.7:0/233705590 shutdown_connections 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 1 -- 172.21.15.7:0/233705590 wait complete. 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.944+0000 7f4478e05740 20 client.26927 trim_cache size 0 max 16384 2024-03-26T08:48:50.940 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:48:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 -- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44580a2950 msgr2=0x7f44580a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44580a2950 0x7f44580a4e10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4460009b40 tx=0x7f4460005e50 comp rx=0 tx=0).stop 2024-03-26T08:48:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 -- 172.21.15.7:0/3825594926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447010b010 msgr2=0x7f44701a6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447010b010 0x7f44701a6d10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f446402c240 tx=0x7f446402c5d0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 -- 172.21.15.7:0/3825594926 shutdown_connections 2024-03-26T08:48:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f44580a2950 0x7f44580a4e10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f447010d9c0 0x7f44701add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447010b010 0x7f44701a6d10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 --2- 172.21.15.7:0/3825594926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44701086a0 0x7f44701a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 -- 172.21.15.7:0/3825594926 >> 172.21.15.7:0/3825594926 conn(0x7f447006e640 msgr2=0x7f447010a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 -- 172.21.15.7:0/3825594926 shutdown_connections 2024-03-26T08:48:50.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:50.945+0000 7f4476144640 1 -- 172.21.15.7:0/3825594926 wait complete. 2024-03-26T08:48:51.044 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:50 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2721949674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:48:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 380.0 s, 2717.8 tps, lat 36.595 ms stddev 41.500 2024-03-26T08:48:51.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:50 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2721949674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:48:51.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:50 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2721949674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:48:51.953 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:48:51.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:48:51.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:48:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.108+0000 7fa1737c4640 1 -- 172.21.15.7:0/701503949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c1011e0 msgr2=0x7fa16c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.108+0000 7fa1737c4640 1 --2- 172.21.15.7:0/701503949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c1011e0 0x7fa16c108a80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa164009ca0 tx=0x7fa16402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.108+0000 7fa1737c4640 1 -- 172.21.15.7:0/701503949 shutdown_connections 2024-03-26T08:48:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.108+0000 7fa1737c4640 1 --2- 172.21.15.7:0/701503949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c1011e0 0x7fa16c108a80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.108+0000 7fa1737c4640 1 --2- 172.21.15.7:0/701503949 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa16c100820 0x7fa16c100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.108+0000 7fa1737c4640 1 --2- 172.21.15.7:0/701503949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16c101bd0 0x7fa16c101ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.108+0000 7fa1737c4640 1 -- 172.21.15.7:0/701503949 >> 172.21.15.7:0/701503949 conn(0x7fa16c0fb2b0 msgr2=0x7fa16c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.108+0000 7fa1737c4640 1 -- 172.21.15.7:0/701503949 shutdown_connections 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.109+0000 7fa1737c4640 1 -- 172.21.15.7:0/701503949 wait complete. 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.109+0000 7fa1737c4640 1 Processor -- start 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.109+0000 7fa1737c4640 1 -- start start 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.109+0000 7fa1737c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c100820 0x7fa16c10f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.109+0000 7fa1737c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16c1011e0 0x7fa16c10fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.109+0000 7fa1737c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa16c101bd0 0x7fa16c110be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.109+0000 7fa1737c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c109e30 con 0x7fa16c100820 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.109+0000 7fa1737c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c109cb0 con 0x7fa16c1011e0 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.109+0000 7fa1737c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c109fb0 con 0x7fa16c101bd0 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171560640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c100820 0x7fa16c10f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171560640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c100820 0x7fa16c10f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38328/0 (socket says 172.21.15.7:38328) 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171560640 1 -- 172.21.15.7:0/2042146762 learned_addr learned my addr 172.21.15.7:0/2042146762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa170d5f640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16c1011e0 0x7fa16c10fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171d61640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa16c101bd0 0x7fa16c110be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171560640 1 -- 172.21.15.7:0/2042146762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa16c101bd0 msgr2=0x7fa16c110be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171560640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa16c101bd0 0x7fa16c110be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171560640 1 -- 172.21.15.7:0/2042146762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16c1011e0 msgr2=0x7fa16c10fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171560640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16c1011e0 0x7fa16c10fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171560640 1 -- 172.21.15.7:0/2042146762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c111310 con 0x7fa16c100820 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa170d5f640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16c1011e0 0x7fa16c10fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171d61640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa16c101bd0 0x7fa16c110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa171560640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c100820 0x7fa16c10f7d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa16800e9c0 tx=0x7fa16800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa1627fc640 1 -- 172.21.15.7:0/2042146762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16800cd60 con 0x7fa16c100820 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa1737c4640 1 -- 172.21.15.7:0/2042146762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c111600 con 0x7fa16c100820 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.110+0000 7fa1737c4640 1 -- 172.21.15.7:0/2042146762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c1aed00 con 0x7fa16c100820 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.111+0000 7fa1627fc640 1 -- 172.21.15.7:0/2042146762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa168010910 con 0x7fa16c100820 2024-03-26T08:48:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.111+0000 7fa1627fc640 1 -- 172.21.15.7:0/2042146762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168018980 con 0x7fa16c100820 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.113+0000 7fa1763ab740 1 Processor -- start 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa1763ab740 1 -- start start 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e1960 0x564a1d8e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e2350 0x564a1d8e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 0x564a1d8e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa1763ab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d821c90 con 0x564a1d8e1960 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa1763ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d847070 con 0x564a1d8e2350 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa1763ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d7efc80 con 0x564a1d8e74c0 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa1627fc640 1 -- 172.21.15.7:0/2042146762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa16801f020 con 0x7fa16c100820 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa171d61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e1960 0x564a1d8e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa171d61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e1960 0x564a1d8e1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38336/0 (socket says 172.21.15.7:38336) 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa171d61640 1 -- 172.21.15.7:0/2649182009 learned_addr learned my addr 172.21.15.7:0/2649182009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa170d5f640 1 --2- 172.21.15.7:0/2649182009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 0x564a1d8e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa171560640 1 --2- 172.21.15.7:0/2649182009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e2350 0x564a1d8e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa170d5f640 1 -- 172.21.15.7:0/2649182009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e2350 msgr2=0x564a1d8e6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa170d5f640 1 --2- 172.21.15.7:0/2649182009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e2350 0x564a1d8e6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa170d5f640 1 -- 172.21.15.7:0/2649182009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e1960 msgr2=0x564a1d8e1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa170d5f640 1 --2- 172.21.15.7:0/2649182009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e1960 0x564a1d8e1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.114+0000 7fa170d5f640 1 -- 172.21.15.7:0/2649182009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a1d7e7d10 con 0x564a1d8e74c0 2024-03-26T08:48:52.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.115+0000 7fa1627fc640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa1580a2840 0x7fa1580a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.115+0000 7fa1627fc640 1 -- 172.21.15.7:0/2042146762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa1680ceba0 con 0x7fa16c100820 2024-03-26T08:48:52.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.116+0000 7fa170d5f640 1 --2- 172.21.15.7:0/2649182009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 0x564a1d8e9950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa16c068940 tx=0x7fa15c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb5b482a7e53fcde server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.116+0000 7fa1577fe640 1 -- 172.21.15.7:0/2649182009 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c01e070 con 0x564a1d8e74c0 2024-03-26T08:48:52.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.116+0000 7fa1577fe640 1 -- 172.21.15.7:0/2649182009 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c00fa20 con 0x564a1d8e74c0 2024-03-26T08:48:52.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.116+0000 7fa1577fe640 1 -- 172.21.15.7:0/2649182009 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c019b60 con 0x564a1d8e74c0 2024-03-26T08:48:52.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.116+0000 7fa171d61640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa1580a2840 0x7fa1580a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.117+0000 7fa171d61640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa1580a2840 0x7fa1580a4d00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa164032420 tx=0x7fa164004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.119+0000 7fa1763ab740 1 -- 172.21.15.7:0/2649182009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 msgr2=0x564a1d8e9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.119+0000 7fa1763ab740 1 --2- 172.21.15.7:0/2649182009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 0x564a1d8e9950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa16c068940 tx=0x7fa15c0118d0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.122+0000 7fa1763ab740 1 -- 172.21.15.7:0/2649182009 shutdown_connections 2024-03-26T08:48:52.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.122+0000 7fa1763ab740 1 --2- 172.21.15.7:0/2649182009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 0x564a1d8e9950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.122+0000 7fa1763ab740 1 --2- 172.21.15.7:0/2649182009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e2350 0x564a1d8e6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.122+0000 7fa1763ab740 1 --2- 172.21.15.7:0/2649182009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e1960 0x564a1d8e1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.122+0000 7fa1763ab740 1 -- 172.21.15.7:0/2649182009 >> 172.21.15.7:0/2649182009 conn(0x564a1d8aa1a0 msgr2=0x564a1d8dd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.123+0000 7fa1763ab740 1 -- 172.21.15.7:0/2649182009 shutdown_connections 2024-03-26T08:48:52.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.123+0000 7fa1763ab740 1 -- 172.21.15.7:0/2649182009 wait complete. 2024-03-26T08:48:52.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.123+0000 7fa1763ab740 1 -- start start 2024-03-26T08:48:52.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.124+0000 7fa1763ab740 1 Processor -- start 2024-03-26T08:48:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.125+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.125+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e2350 0x564a1d8ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.125+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e74c0 0x564a1d8f3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.125+0000 7fa1763ab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d8030d0 con 0x564a1d8e74c0 2024-03-26T08:48:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.125+0000 7fa1763ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d843110 con 0x564a1d8e1960 2024-03-26T08:48:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.125+0000 7fa1763ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a1d8ea470 con 0x564a1d8e2350 2024-03-26T08:48:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.125+0000 7fa1763ab740 10 client.0 _ll_register_callbacks cb 0x564a1d83ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.125+0000 7fa171560640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.128+0000 7fa170d5f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e2350 0x564a1d8ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.128+0000 7fa171560640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8ebd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41588/0 (socket says 172.21.15.7:41588) 2024-03-26T08:48:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.128+0000 7fa171560640 1 -- 172.21.15.7:0/422870567 learned_addr learned my addr 172.21.15.7:0/422870567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.130+0000 7fa171d61640 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e74c0 0x564a1d8f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.131+0000 7fa170d5f640 1 -- 172.21.15.7:0/422870567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 msgr2=0x564a1d8ebd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.131+0000 7fa170d5f640 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8ebd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.131+0000 7fa170d5f640 1 -- 172.21.15.7:0/422870567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e74c0 msgr2=0x564a1d8f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.131+0000 7fa170d5f640 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e74c0 0x564a1d8f3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.131+0000 7fa170d5f640 1 -- 172.21.15.7:0/422870567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a1d6237e0 con 0x564a1d8e2350 2024-03-26T08:48:52.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.131+0000 7fa170d5f640 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e2350 0x564a1d8ec240 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa15c000c00 tx=0x7fa15c00f770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa6cb3bde368eaed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.135+0000 7fa154ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:52.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.135+0000 7fa154ff9640 1 -- 172.21.15.7:0/422870567 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c026020 con 0x564a1d8e2350 2024-03-26T08:48:52.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.135+0000 7fa154ff9640 1 -- 172.21.15.7:0/422870567 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa15c004040 con 0x564a1d8e2350 2024-03-26T08:48:52.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.135+0000 7fa1763ab740 1 -- 172.21.15.7:0/422870567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa15c00fa20 con 0x564a1d8e2350 2024-03-26T08:48:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.141+0000 7fa154ff9640 1 -- 172.21.15.7:0/422870567 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa15c01e040 con 0x564a1d8e2350 2024-03-26T08:48:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.142+0000 7fa1763ab740 10 client.26974 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.142+0000 7fa1763ab740 1 -- 172.21.15.7:0/422870567 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564a1d683f70 con 0x564a1d8e2350 2024-03-26T08:48:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.142+0000 7fa154ff9640 1 -- 172.21.15.7:0/422870567 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c00fa20 con 0x564a1d8e2350 2024-03-26T08:48:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.142+0000 7fa154ff9640 1 -- 172.21.15.7:0/422870567 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa15c021030 con 0x564a1d8e2350 2024-03-26T08:48:52.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.143+0000 7fa1763ab740 10 client.26974 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:52.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.143+0000 7fa1763ab740 10 client.26974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:52.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.143+0000 7fa1763ab740 20 client.26974 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:52.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.143+0000 7fa1763ab740 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564a1d8f7770 0x564a1d8f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.143+0000 7fa1763ab740 4 client.26974 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:52.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.143+0000 7fa1763ab740 1 -- 172.21.15.7:0/422870567 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a1d8545b0 con 0x564a1d8f7770 2024-03-26T08:48:52.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.145+0000 7fa171560640 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564a1d8f7770 0x564a1d8f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.145+0000 7fa171560640 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564a1d8f7770 0x564a1d8f9b60 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab38e4879b66227d server_cookie=b3c6e672bf6ebf3 in_seq=0 out_seq=0 2024-03-26T08:48:52.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.145+0000 7fa154ff9640 10 client.26974 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.146+0000 7fa154ff9640 1 -- 172.21.15.7:0/422870567 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564a1d8545b0 con 0x564a1d8f7770 2024-03-26T08:48:52.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.146+0000 7fa154ff9640 10 client.26974 handle_command_reply: tid=0 2024-03-26T08:48:52.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.148+0000 7fa1763ab740 1 client.26974 shutdown 2024-03-26T08:48:52.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.148+0000 7fa1763ab740 1 -- 172.21.15.7:0/422870567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e2350 msgr2=0x564a1d8ec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.148+0000 7fa1763ab740 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e2350 0x564a1d8ec240 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa15c000c00 tx=0x7fa15c00f770 comp rx=0 tx=0).stop 2024-03-26T08:48:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.153+0000 7fa1763ab740 1 -- 172.21.15.7:0/422870567 shutdown_connections 2024-03-26T08:48:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.153+0000 7fa1763ab740 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564a1d8f7770 0x564a1d8f9b60 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.153+0000 7fa1763ab740 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e74c0 0x564a1d8f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.153+0000 7fa1763ab740 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e2350 0x564a1d8ec240 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.153+0000 7fa1763ab740 1 --2- 172.21.15.7:0/422870567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8ebd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.153+0000 7fa1763ab740 1 -- 172.21.15.7:0/422870567 >> 172.21.15.7:0/422870567 conn(0x564a1d8aa1a0 msgr2=0x564a1d8dc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.154+0000 7fa154ff9640 0 client.26974 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.156+0000 7fa1763ab740 1 -- 172.21.15.7:0/422870567 shutdown_connections 2024-03-26T08:48:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.157+0000 7fa1763ab740 1 -- 172.21.15.7:0/422870567 wait complete. 2024-03-26T08:48:52.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.161+0000 7fa1763ab740 20 client.26974 trim_cache size 0 max 16384 2024-03-26T08:48:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.194+0000 7fa1763ab740 1 Processor -- start 2024-03-26T08:48:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.194+0000 7fa1763ab740 1 -- start start 2024-03-26T08:48:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.194+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8e0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.194+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e2350 0x564a1d8f3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.194+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 0x564a1d8fe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.194+0000 7fa1763ab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1ba060 con 0x564a1d8e2350 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.194+0000 7fa1763ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1b7b60 con 0x564a1d8e1960 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.194+0000 7fa1763ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1b5be0 con 0x564a1d8e74c0 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa170d5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e2350 0x564a1d8f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa171560640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8e0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa170d5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e2350 0x564a1d8f3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38346/0 (socket says 172.21.15.7:38346) 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa170d5f640 1 -- 172.21.15.7:0/3154177098 learned_addr learned my addr 172.21.15.7:0/3154177098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa171d61640 1 --2- 172.21.15.7:0/3154177098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 0x564a1d8fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa170d5f640 1 -- 172.21.15.7:0/3154177098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 msgr2=0x564a1d8fe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa170d5f640 1 --2- 172.21.15.7:0/3154177098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 0x564a1d8fe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa170d5f640 1 -- 172.21.15.7:0/3154177098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 msgr2=0x564a1d8e0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa170d5f640 1 --2- 172.21.15.7:0/3154177098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8e0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa170d5f640 1 -- 172.21.15.7:0/3154177098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a1d51b170 con 0x564a1d8e2350 2024-03-26T08:48:52.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa171560640 1 --2- 172.21.15.7:0/3154177098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8e0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa170d5f640 1 --2- 172.21.15.7:0/3154177098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e2350 0x564a1d8f3240 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa15c02d040 tx=0x7fa15c00f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc8b4635056000 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.195+0000 7fa156ffd640 1 -- 172.21.15.7:0/3154177098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c026040 con 0x564a1d8e2350 2024-03-26T08:48:52.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.196+0000 7fa156ffd640 1 -- 172.21.15.7:0/3154177098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c016070 con 0x564a1d8e2350 2024-03-26T08:48:52.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.196+0000 7fa156ffd640 1 -- 172.21.15.7:0/3154177098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c004620 con 0x564a1d8e2350 2024-03-26T08:48:52.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.196+0000 7fa1763ab740 1 -- 172.21.15.7:0/3154177098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e2350 msgr2=0x564a1d8f3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.196+0000 7fa1763ab740 1 --2- 172.21.15.7:0/3154177098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e2350 0x564a1d8f3240 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa15c02d040 tx=0x7fa15c00f480 comp rx=0 tx=0).stop 2024-03-26T08:48:52.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.196+0000 7fa1763ab740 1 -- 172.21.15.7:0/3154177098 shutdown_connections 2024-03-26T08:48:52.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.196+0000 7fa1763ab740 1 --2- 172.21.15.7:0/3154177098 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e74c0 0x564a1d8fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.197+0000 7fa1763ab740 1 --2- 172.21.15.7:0/3154177098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8e2350 0x564a1d8f3240 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.197+0000 7fa1763ab740 1 --2- 172.21.15.7:0/3154177098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e1960 0x564a1d8e0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.197+0000 7fa1763ab740 1 -- 172.21.15.7:0/3154177098 >> 172.21.15.7:0/3154177098 conn(0x564a1d8aa1a0 msgr2=0x564a1d8e0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.197+0000 7fa1763ab740 1 -- 172.21.15.7:0/3154177098 shutdown_connections 2024-03-26T08:48:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.197+0000 7fa1763ab740 1 -- 172.21.15.7:0/3154177098 wait complete. 2024-03-26T08:48:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.198+0000 7fa1763ab740 1 -- start start 2024-03-26T08:48:52.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.198+0000 7fa1763ab740 1 Processor -- start 2024-03-26T08:48:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e1960 0x564a1d8f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e74c0 0x564a1d8f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa1763ab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8f1c90 0x564a1d8f2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa171560640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e1960 0x564a1d8f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa171560640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e1960 0x564a1d8f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44058/0 (socket says 172.21.15.7:44058) 2024-03-26T08:48:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa1763ab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1b61e0 con 0x564a1d8f1c90 2024-03-26T08:48:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa1763ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1b7500 con 0x564a1d8e74c0 2024-03-26T08:48:52.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa1763ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1b6840 con 0x564a1d8e1960 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa170d5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e74c0 0x564a1d8f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa170d5f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e74c0 0x564a1d8f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41598/0 (socket says 172.21.15.7:41598) 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.199+0000 7fa170d5f640 1 -- 172.21.15.7:0/177868242 learned_addr learned my addr 172.21.15.7:0/177868242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa1763ab740 10 client.0 _ll_register_callbacks cb 0x7fa15c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa171d61640 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8f1c90 0x564a1d8f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa171560640 1 -- 172.21.15.7:0/177868242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e74c0 msgr2=0x564a1d8f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa171560640 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e74c0 0x564a1d8f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa171560640 1 -- 172.21.15.7:0/177868242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8f1c90 msgr2=0x564a1d8f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa171560640 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8f1c90 0x564a1d8f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa171560640 1 -- 172.21.15.7:0/177868242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a1d9530d0 con 0x564a1d8e1960 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa171d61640 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8f1c90 0x564a1d8f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa170d5f640 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e74c0 0x564a1d8f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa171560640 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e1960 0x564a1d8f30f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa16809b490 tx=0x7fa1680cc530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=839373c6fce09f08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa154ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa154ff9640 1 -- 172.21.15.7:0/177868242 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16809ae60 con 0x564a1d8e1960 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa154ff9640 1 -- 172.21.15.7:0/177868242 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa168010430 con 0x564a1d8e1960 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.200+0000 7fa1763ab740 1 -- 172.21.15.7:0/177868242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa15c016070 con 0x564a1d8e1960 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa154ff9640 1 -- 172.21.15.7:0/177868242 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa16809bc00 con 0x564a1d8e1960 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa1763ab740 10 client.26980 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa1763ab740 1 -- 172.21.15.7:0/177868242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564a1d9544e0 con 0x564a1d8e1960 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa154ff9640 1 -- 172.21.15.7:0/177868242 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa15c016070 con 0x564a1d8e1960 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa154ff9640 1 -- 172.21.15.7:0/177868242 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa168010ac0 con 0x564a1d8e1960 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa1763ab740 10 client.26980 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa1763ab740 10 client.26980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:52.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa1763ab740 20 client.26980 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa1763ab740 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564a1d954dc0 0x564a1d9571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa1763ab740 4 client.26980 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:48:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.201+0000 7fa1763ab740 1 -- 172.21.15.7:0/177868242 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564a1d84e900 con 0x564a1d954dc0 2024-03-26T08:48:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.202+0000 7fa170d5f640 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564a1d954dc0 0x564a1d9571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.202+0000 7fa170d5f640 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564a1d954dc0 0x564a1d9571e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eb69db913814e58 server_cookie=24e1121a555332a4 in_seq=0 out_seq=0 2024-03-26T08:48:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.202+0000 7fa154ff9640 10 client.26980 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.203+0000 7fa154ff9640 1 -- 172.21.15.7:0/177868242 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564a1d84e900 con 0x564a1d954dc0 2024-03-26T08:48:52.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.203+0000 7fa154ff9640 10 client.26980 handle_command_reply: tid=0 2024-03-26T08:48:52.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa1763ab740 1 client.26980 shutdown 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa1763ab740 1 -- 172.21.15.7:0/177868242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e1960 msgr2=0x564a1d8f30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa1763ab740 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e1960 0x564a1d8f30f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa16809b490 tx=0x7fa1680cc530 comp rx=0 tx=0).stop 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa1763ab740 1 -- 172.21.15.7:0/177868242 shutdown_connections 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa1763ab740 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564a1d954dc0 0x564a1d9571e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa1763ab740 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a1d8f1c90 0x564a1d8f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa154ff9640 0 client.26980 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa1763ab740 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a1d8e74c0 0x564a1d8f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa1763ab740 1 --2- 172.21.15.7:0/177868242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564a1d8e1960 0x564a1d8f30f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.204+0000 7fa1763ab740 1 -- 172.21.15.7:0/177868242 >> 172.21.15.7:0/177868242 conn(0x564a1d8aa1a0 msgr2=0x564a1d8dc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.205+0000 7fa1763ab740 1 -- 172.21.15.7:0/177868242 shutdown_connections 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.205+0000 7fa1763ab740 1 -- 172.21.15.7:0/177868242 wait complete. 2024-03-26T08:48:52.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.205+0000 7fa1763ab740 20 client.26980 trim_cache size 0 max 16384 2024-03-26T08:48:52.202 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:48:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:48:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "159e724e-629c-407d-b00c-ca73c93a9df3", 2024-03-26T08:48:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:48:52.202 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:48:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.208+0000 7fa1737c4640 1 -- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa1580a2840 msgr2=0x7fa1580a4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.208+0000 7fa1737c4640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa1580a2840 0x7fa1580a4d00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa164032420 tx=0x7fa164004a40 comp rx=0 tx=0).stop 2024-03-26T08:48:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.208+0000 7fa1737c4640 1 -- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c100820 msgr2=0x7fa16c10f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.208+0000 7fa1737c4640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c100820 0x7fa16c10f7d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa16800e9c0 tx=0x7fa16800ee80 comp rx=0 tx=0).stop 2024-03-26T08:48:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.209+0000 7fa1737c4640 1 -- 172.21.15.7:0/2042146762 shutdown_connections 2024-03-26T08:48:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.209+0000 7fa1737c4640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa1580a2840 0x7fa1580a4d00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.209+0000 7fa1737c4640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa16c101bd0 0x7fa16c110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.209+0000 7fa1737c4640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa16c1011e0 0x7fa16c10fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.209+0000 7fa1737c4640 1 --2- 172.21.15.7:0/2042146762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c100820 0x7fa16c10f7d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.209+0000 7fa1737c4640 1 -- 172.21.15.7:0/2042146762 >> 172.21.15.7:0/2042146762 conn(0x7fa16c0fb2b0 msgr2=0x7fa16c104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.209+0000 7fa1737c4640 1 -- 172.21.15.7:0/2042146762 shutdown_connections 2024-03-26T08:48:52.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.209+0000 7fa1737c4640 1 -- 172.21.15.7:0/2042146762 wait complete. 2024-03-26T08:48:52.214 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:51 smithi007 ceph-mon[32462]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 772 op/s 2024-03-26T08:48:52.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:48:52.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:48:52.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:51 smithi016 ceph-mon[35851]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 772 op/s 2024-03-26T08:48:52.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:51 smithi044 ceph-mon[36461]: pgmap v553: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 772 op/s 2024-03-26T08:48:52.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.384+0000 7f9cd0144640 1 -- 172.21.15.7:0/3809646494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc8104a70 msgr2=0x7f9cc8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.384+0000 7f9cd0144640 1 --2- 172.21.15.7:0/3809646494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc8104a70 0x7f9cc8106e60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009900 tx=0x7f9cb802bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:52.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.384+0000 7f9cd0144640 1 -- 172.21.15.7:0/3809646494 shutdown_connections 2024-03-26T08:48:52.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.384+0000 7f9cd0144640 1 --2- 172.21.15.7:0/3809646494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cc81073a0 0x7f9cc81097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.384+0000 7f9cd0144640 1 --2- 172.21.15.7:0/3809646494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc8104a70 0x7f9cc8106e60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.384+0000 7f9cd0144640 1 --2- 172.21.15.7:0/3809646494 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc8102140 0x7f9cc8104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.384+0000 7f9cd0144640 1 -- 172.21.15.7:0/3809646494 >> 172.21.15.7:0/3809646494 conn(0x7f9cc80fbb30 msgr2=0x7f9cc80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.385+0000 7f9cd0144640 1 -- 172.21.15.7:0/3809646494 shutdown_connections 2024-03-26T08:48:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.385+0000 7f9cd0144640 1 -- 172.21.15.7:0/3809646494 wait complete. 2024-03-26T08:48:52.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.385+0000 7f9cd0144640 1 Processor -- start 2024-03-26T08:48:52.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.385+0000 7f9cd0144640 1 -- start start 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cd0144640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc8102140 0x7f9cc806fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cd0144640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cc8104a70 0x7f9cc80703b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cd0144640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc81073a0 0x7f9cc81a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cd0144640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc810d3a0 con 0x7f9cc81073a0 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cd0144640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc810d220 con 0x7f9cc8104a70 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cd0144640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc810d520 con 0x7f9cc8102140 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cce6e1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc81073a0 0x7f9cc81a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cce6e1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc81073a0 0x7f9cc81a7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38360/0 (socket says 172.21.15.7:38360) 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9ccdee0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc8102140 0x7f9cc806fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cce6e1640 1 -- 172.21.15.7:0/2716344548 learned_addr learned my addr 172.21.15.7:0/2716344548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9ccd6df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cc8104a70 0x7f9cc80703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9ccd6df640 1 -- 172.21.15.7:0/2716344548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc8102140 msgr2=0x7f9cc806fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc8102140 0x7f9cc806fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9ccd6df640 1 -- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc81073a0 msgr2=0x7f9cc81a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc81073a0 0x7f9cc81a7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9ccd6df640 1 -- 172.21.15.7:0/2716344548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8073530 con 0x7f9cc8104a70 2024-03-26T08:48:52.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9cce6e1640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc81073a0 0x7f9cc81a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.386+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc8102140 0x7f9cc806fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.387+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cc8104a70 0x7f9cc80703b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009230 tx=0x7f9cb8005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.387+0000 7f9cbeffd640 1 -- 172.21.15.7:0/2716344548 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8043020 con 0x7f9cc8104a70 2024-03-26T08:48:52.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.387+0000 7f9cd0144640 1 -- 172.21.15.7:0/2716344548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc80fff30 con 0x7f9cc8104a70 2024-03-26T08:48:52.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.387+0000 7f9cbeffd640 1 -- 172.21.15.7:0/2716344548 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cb80054e0 con 0x7f9cc8104a70 2024-03-26T08:48:52.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.387+0000 7f9cbeffd640 1 -- 172.21.15.7:0/2716344548 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb803e450 con 0x7f9cc8104a70 2024-03-26T08:48:52.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.387+0000 7f9cd0144640 1 -- 172.21.15.7:0/2716344548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc81004a0 con 0x7f9cc8104a70 2024-03-26T08:48:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.390+0000 7f9cd2e05740 1 Processor -- start 2024-03-26T08:48:52.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.391+0000 7f9cd2e05740 1 -- start start 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.391+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e11f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.391+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11ffa0 0x55f84e124aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.391+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e125110 0x55f84e1275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.391+0000 7f9cd2e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e07c3d0 con 0x55f84e125110 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.391+0000 7f9cd2e05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e024ad0 con 0x55f84e11f5b0 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.391+0000 7f9cd2e05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e078470 con 0x55f84e11ffa0 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cce6e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e11f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cce6e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e11f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41622/0 (socket says 172.21.15.7:41622) 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cce6e1640 1 -- 172.21.15.7:0/2815343575 learned_addr learned my addr 172.21.15.7:0/2815343575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/2815343575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11ffa0 0x55f84e124aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cce6e1640 1 -- 172.21.15.7:0/2815343575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11ffa0 msgr2=0x55f84e124aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cce6e1640 1 --2- 172.21.15.7:0/2815343575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11ffa0 0x55f84e124aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cce6e1640 1 -- 172.21.15.7:0/2815343575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e125110 msgr2=0x55f84e1275a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:48:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cce6e1640 1 --2- 172.21.15.7:0/2815343575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e125110 0x55f84e1275a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cce6e1640 1 -- 172.21.15.7:0/2815343575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84de5a7d0 con 0x55f84e11f5b0 2024-03-26T08:48:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/2815343575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11ffa0 0x55f84e124aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cbeffd640 1 -- 172.21.15.7:0/2716344548 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9cb8046020 con 0x7f9cc8104a70 2024-03-26T08:48:52.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.392+0000 7f9cce6e1640 1 --2- 172.21.15.7:0/2815343575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e11f9d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00118b0 tx=0x7f9cc0011cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7db86c1db94f4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.393+0000 7f9cb3fff640 1 -- 172.21.15.7:0/2815343575 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0011330 con 0x55f84e11f5b0 2024-03-26T08:48:52.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.393+0000 7f9cb3fff640 1 -- 172.21.15.7:0/2815343575 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cc0010430 con 0x55f84e11f5b0 2024-03-26T08:48:52.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.393+0000 7f9cb3fff640 1 -- 172.21.15.7:0/2815343575 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000fea0 con 0x55f84e11f5b0 2024-03-26T08:48:52.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.393+0000 7f9cbeffd640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cb40a2880 0x7f9cb40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.393+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2815343575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 msgr2=0x55f84e11f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.393+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/2815343575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e11f9d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00118b0 tx=0x7f9cc0011cb0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.393+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2815343575 shutdown_connections 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.393+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/2815343575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e125110 0x55f84e1275a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.393+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/2815343575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11ffa0 0x55f84e124aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.394+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/2815343575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e11f9d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.394+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2815343575 >> 172.21.15.7:0/2815343575 conn(0x55f84e11ab20 msgr2=0x55f84e11af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.394+0000 7f9cbeffd640 1 -- 172.21.15.7:0/2716344548 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cb80f0d70 con 0x7f9cc8104a70 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.394+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cb40a2880 0x7f9cb40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.394+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2815343575 shutdown_connections 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.394+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2815343575 wait complete. 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.394+0000 7f9cd2e05740 1 -- start start 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.394+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cb40a2880 0x7f9cb40a4d40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55f84e11ba90 tx=0x7f9cc400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.394+0000 7f9cd2e05740 1 Processor -- start 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11f5b0 0x55f84e129a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 0x55f84e129fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e125110 0x55f84e130fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9cd2e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e1280c0 con 0x55f84e11ffa0 2024-03-26T08:48:52.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9cd2e05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e038060 con 0x55f84e125110 2024-03-26T08:48:52.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9cd2e05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84e128240 con 0x55f84e11f5b0 2024-03-26T08:48:52.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccdee0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11f5b0 0x55f84e129a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9cd2e05740 10 client.0 _ll_register_callbacks cb 0x55f84e01a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:52.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccdee0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11f5b0 0x55f84e129a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44098/0 (socket says 172.21.15.7:44098) 2024-03-26T08:48:52.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccdee0640 1 -- 172.21.15.7:0/3267478996 learned_addr learned my addr 172.21.15.7:0/3267478996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 0x55f84e129fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9cce6e1640 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e125110 0x55f84e130fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccdee0640 1 -- 172.21.15.7:0/3267478996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e125110 msgr2=0x55f84e130fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e125110 0x55f84e130fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccdee0640 1 -- 172.21.15.7:0/3267478996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 msgr2=0x55f84e129fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 0x55f84e129fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccdee0640 1 -- 172.21.15.7:0/3267478996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f84dd903f0 con 0x55f84e11f5b0 2024-03-26T08:48:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.395+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 0x55f84e129fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.396+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11f5b0 0x55f84e129a90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9cc401f740 tx=0x7f9cc4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05171472484d592 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.396+0000 7f9cb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.396+0000 7f9cb17fa640 1 -- 172.21.15.7:0/3267478996 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4004260 con 0x55f84e11f5b0 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.396+0000 7f9cd2e05740 1 -- 172.21.15.7:0/3267478996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cc0010430 con 0x55f84e11f5b0 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.396+0000 7f9cb17fa640 1 -- 172.21.15.7:0/3267478996 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cc4004400 con 0x55f84e11f5b0 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.397+0000 7f9cb17fa640 1 -- 172.21.15.7:0/3267478996 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cc401f020 con 0x55f84e11f5b0 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.397+0000 7f9cd2e05740 10 client.26992 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.397+0000 7f9cd2e05740 1 -- 172.21.15.7:0/3267478996 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f84dd500f0 con 0x55f84e11f5b0 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.397+0000 7f9cb17fa640 1 -- 172.21.15.7:0/3267478996 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cc0010430 con 0x55f84e11f5b0 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.397+0000 7f9cb17fa640 1 -- 172.21.15.7:0/3267478996 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9cc4026070 con 0x55f84e11f5b0 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.398+0000 7f9cd2e05740 10 client.26992 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.398+0000 7f9cd2e05740 10 client.26992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.398+0000 7f9cd2e05740 20 client.26992 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.398+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f84e1358b0 0x55f84e137d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.398+0000 7f9cd2e05740 4 client.26992 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.398+0000 7f9cd2e05740 1 -- 172.21.15.7:0/3267478996 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f84e089910 con 0x55f84e1358b0 2024-03-26T08:48:52.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.398+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f84e1358b0 0x55f84e137d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.398+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f84e1358b0 0x55f84e137d10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc9cb27b54aa128 server_cookie=681d36090b19eca4 in_seq=0 out_seq=0 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.398+0000 7f9cb17fa640 10 client.26992 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.399+0000 7f9cb17fa640 1 -- 172.21.15.7:0/3267478996 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f84e089910 con 0x55f84e1358b0 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.399+0000 7f9cb17fa640 10 client.26992 handle_command_reply: tid=0 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.399+0000 7f9cd2e05740 1 client.26992 shutdown 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 -- 172.21.15.7:0/3267478996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11f5b0 msgr2=0x55f84e129a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11f5b0 0x55f84e129a90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9cc401f740 tx=0x7f9cc4004730 comp rx=0 tx=0).stop 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 -- 172.21.15.7:0/3267478996 shutdown_connections 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f84e1358b0 0x55f84e137d10 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e125110 0x55f84e130fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 0x55f84e129fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/3267478996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e11f5b0 0x55f84e129a90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 -- 172.21.15.7:0/3267478996 >> 172.21.15.7:0/3267478996 conn(0x55f84e11ab20 msgr2=0x55f84e111530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cb17fa640 0 client.26992 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 -- 172.21.15.7:0/3267478996 shutdown_connections 2024-03-26T08:48:52.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 1 -- 172.21.15.7:0/3267478996 wait complete. 2024-03-26T08:48:52.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.400+0000 7f9cd2e05740 20 client.26992 trim_cache size 0 max 16384 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.425+0000 7f9cd2e05740 1 Processor -- start 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.425+0000 7f9cd2e05740 1 -- start start 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.426+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e1296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.426+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 0x55f84e130f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.426+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 0x55f84e129be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.426+0000 7f9ccdee0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e1296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.426+0000 7f9ccdee0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e1296a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41650/0 (socket says 172.21.15.7:41650) 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.426+0000 7f9ccdee0640 1 -- 172.21.15.7:0/2633727941 learned_addr learned my addr 172.21.15.7:0/2633727941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.426+0000 7f9cce6e1640 1 --2- 172.21.15.7:0/2633727941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 0x55f84e129be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.426+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2633727941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81b13b0 con 0x55f84e11ffa0 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.427+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/2633727941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 0x55f84e130f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.426+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2633727941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81aeeb0 con 0x55f84e11f5b0 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.427+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2633727941 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81acf30 con 0x55f84e125110 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.427+0000 7f9ccdee0640 1 -- 172.21.15.7:0/2633727941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 msgr2=0x55f84e129be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.427+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/2633727941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 0x55f84e129be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.427+0000 7f9ccdee0640 1 -- 172.21.15.7:0/2633727941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 msgr2=0x55f84e130f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.427+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/2633727941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 0x55f84e130f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.427+0000 7f9ccdee0640 1 -- 172.21.15.7:0/2633727941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84e11b590 con 0x55f84e11f5b0 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.428+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/2633727941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e1296a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9cc401f710 tx=0x7f9cc4005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9342fadbdb32f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.428+0000 7f9cb37fe640 1 -- 172.21.15.7:0/2633727941 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4010730 con 0x55f84e11f5b0 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.428+0000 7f9cb37fe640 1 -- 172.21.15.7:0/2633727941 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cc4036070 con 0x55f84e11f5b0 2024-03-26T08:48:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.428+0000 7f9cb37fe640 1 -- 172.21.15.7:0/2633727941 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4031400 con 0x55f84e11f5b0 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2633727941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 msgr2=0x55f84e1296a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/2633727941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e1296a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9cc401f710 tx=0x7f9cc4005970 comp rx=0 tx=0).stop 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2633727941 shutdown_connections 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/2633727941 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 0x55f84e129be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/2633727941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11ffa0 0x55f84e130f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/2633727941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11f5b0 0x55f84e1296a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2633727941 >> 172.21.15.7:0/2633727941 conn(0x55f84e11ab20 msgr2=0x55f84e121440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2633727941 shutdown_connections 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 -- 172.21.15.7:0/2633727941 wait complete. 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.429+0000 7f9cd2e05740 1 -- start start 2024-03-26T08:48:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9cd2e05740 1 Processor -- start 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11f5b0 0x55f84e12f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11ffa0 0x55f84e12f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9cd2e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 0x55f84e1304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9cd2e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81ad530 con 0x55f84e11f5b0 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9ccdee0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11f5b0 0x55f84e12f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9ccd6df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11ffa0 0x55f84e12f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9ccdee0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11f5b0 0x55f84e12f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38374/0 (socket says 172.21.15.7:38374) 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9ccdee0640 1 -- 172.21.15.7:0/1932165202 learned_addr learned my addr 172.21.15.7:0/1932165202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9cce6e1640 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 0x55f84e1304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81ae850 con 0x55f84e11ffa0 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.430+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81adb90 con 0x55f84e125110 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.431+0000 7f9cd2e05740 10 client.0 _ll_register_callbacks cb 0x7f9cc401f5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.431+0000 7f9ccdee0640 1 -- 172.21.15.7:0/1932165202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 msgr2=0x55f84e1304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.431+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 0x55f84e1304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.431+0000 7f9ccdee0640 1 -- 172.21.15.7:0/1932165202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11ffa0 msgr2=0x55f84e12f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.431+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11ffa0 0x55f84e12f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.431+0000 7f9ccdee0640 1 -- 172.21.15.7:0/1932165202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f84e130ad0 con 0x55f84e11f5b0 2024-03-26T08:48:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.431+0000 7f9cce6e1640 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 0x55f84e1304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.431+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11ffa0 0x55f84e12f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.432+0000 7f9ccdee0640 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11f5b0 0x55f84e12f030 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4010930 tx=0x7f9cc4010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18dac54122633b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.432+0000 7f9cb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.432+0000 7f9cb17fa640 1 -- 172.21.15.7:0/1932165202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc403f030 con 0x55f84e11f5b0 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.432+0000 7f9cb17fa640 1 -- 172.21.15.7:0/1932165202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cc4004260 con 0x55f84e11f5b0 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.432+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cc4036070 con 0x55f84e11f5b0 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.432+0000 7f9cb17fa640 1 -- 172.21.15.7:0/1932165202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cc404a050 con 0x55f84e11f5b0 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cd2e05740 10 client.17121 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f84e117580 con 0x55f84e11f5b0 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cb17fa640 1 -- 172.21.15.7:0/1932165202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cc4004460 con 0x55f84e11f5b0 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cb17fa640 1 -- 172.21.15.7:0/1932165202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9cc4026070 con 0x55f84e11f5b0 2024-03-26T08:48:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cd2e05740 10 client.17121 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cd2e05740 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cd2e05740 20 client.17121 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f84e117e70 0x55f84e19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cd2e05740 4 client.17121 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f9cc4036070 con 0x55f84e117e70 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.433+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f84e117e70 0x55f84e19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cb17fa640 10 client.17121 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9ccd6df640 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f84e117e70 0x55f84e19c1c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcfe37ad37fcafb4 server_cookie=61f658a0e0916c9f in_seq=0 out_seq=0 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cb17fa640 1 -- 172.21.15.7:0/1932165202 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f9cc4036070 con 0x55f84e117e70 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cb17fa640 10 client.17121 handle_command_reply: tid=0 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cd2e05740 1 client.17121 shutdown 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11f5b0 msgr2=0x55f84e12f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11f5b0 0x55f84e12f030 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4010930 tx=0x7f9cc4010820 comp rx=0 tx=0).stop 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 shutdown_connections 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f84e117e70 0x55f84e19c1c0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f84e125110 0x55f84e1304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f84e11ffa0 0x55f84e12f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cd2e05740 1 --2- 172.21.15.7:0/1932165202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f84e11f5b0 0x55f84e12f030 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 >> 172.21.15.7:0/1932165202 conn(0x55f84e11ab20 msgr2=0x55f84e129610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.434+0000 7f9cb17fa640 0 client.17121 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.435+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 shutdown_connections 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.435+0000 7f9cd2e05740 1 -- 172.21.15.7:0/1932165202 wait complete. 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.435+0000 7f9cd2e05740 20 client.17121 trim_cache size 0 max 16384 2024-03-26T08:48:52.431 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:48:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:48:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:48:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: "159e724e-629c-407d-b00c-ca73c93a9df3": { 2024-03-26T08:48:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:48:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "159e724e-629c-407d-b00c-ca73c93a9df3", 2024-03-26T08:48:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:48:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:48:52.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:48:52.432 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:48:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.435+0000 7f9cd0144640 1 -- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cb40a2880 msgr2=0x7f9cb40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.435+0000 7f9cd0144640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cb40a2880 0x7f9cb40a4d40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55f84e11ba90 tx=0x7f9cc400b390 comp rx=0 tx=0).stop 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.435+0000 7f9cd0144640 1 -- 172.21.15.7:0/2716344548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cc8104a70 msgr2=0x7f9cc80703b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.435+0000 7f9cd0144640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cc8104a70 0x7f9cc80703b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009230 tx=0x7f9cb8005e60 comp rx=0 tx=0).stop 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.436+0000 7f9cd0144640 1 -- 172.21.15.7:0/2716344548 shutdown_connections 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.436+0000 7f9cd0144640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cb40a2880 0x7f9cb40a4d40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.436+0000 7f9cd0144640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cc81073a0 0x7f9cc81a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.436+0000 7f9cd0144640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cc8104a70 0x7f9cc80703b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.436+0000 7f9cd0144640 1 --2- 172.21.15.7:0/2716344548 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9cc8102140 0x7f9cc806fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.436+0000 7f9cd0144640 1 -- 172.21.15.7:0/2716344548 >> 172.21.15.7:0/2716344548 conn(0x7f9cc80fbb30 msgr2=0x7f9cc8105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.439+0000 7f9cd0144640 1 -- 172.21.15.7:0/2716344548 shutdown_connections 2024-03-26T08:48:52.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:48:52.439+0000 7f9cd0144640 1 -- 172.21.15.7:0/2716344548 wait complete. 2024-03-26T08:48:52.447 INFO:tasks.cephfs.filesystem:scrub status for tag:159e724e-629c-407d-b00c-ca73c93a9df3 - {'path': '/', 'tag': '159e724e-629c-407d-b00c-ca73c93a9df3', 'options': 'recursive,force'} 2024-03-26T08:48:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.525+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/1582994934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c440ff9a0 msgr2=0x7f7c440ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.525+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/1582994934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c440ff9a0 0x7f7c440ffdc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009930 tx=0x7f7c3802bc30 comp rx=0 tx=0).stop 2024-03-26T08:48:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.526+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/1582994934 shutdown_connections 2024-03-26T08:48:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.526+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/1582994934 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c44103330 0x7f7c44109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.526+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/1582994934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c44102970 0x7f7c44102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.526+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/1582994934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c440ff9a0 0x7f7c440ffdc0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.526+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/1582994934 >> 172.21.15.7:0/1582994934 conn(0x7f7c440fb2f0 msgr2=0x7f7c440fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.526+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/1582994934 shutdown_connections 2024-03-26T08:48:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.526+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/1582994934 wait complete. 2024-03-26T08:48:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4aeb2640 1 Processor -- start 2024-03-26T08:48:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4aeb2640 1 -- start start 2024-03-26T08:48:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4aeb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c440ff9a0 0x7f7c44197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4aeb2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c44102970 0x7f7c44197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4aeb2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c44103330 0x7f7c4419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4aeb2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4410d3c0 con 0x7f7c44103330 2024-03-26T08:48:52.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4aeb2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4410d240 con 0x7f7c440ff9a0 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4aeb2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4410d540 con 0x7f7c44102970 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4944f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c44103330 0x7f7c4419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4944f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c44103330 0x7f7c4419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38394/0 (socket says 172.21.15.7:38394) 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c44102970 0x7f7c44197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4944f640 1 -- 172.21.15.7:0/3349432073 learned_addr learned my addr 172.21.15.7:0/3349432073 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c48c4e640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c440ff9a0 0x7f7c44197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4944f640 1 -- 172.21.15.7:0/3349432073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c44102970 msgr2=0x7f7c44197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.527+0000 7f7c4944f640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c44102970 0x7f7c44197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.528+0000 7f7c4944f640 1 -- 172.21.15.7:0/3349432073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c440ff9a0 msgr2=0x7f7c44197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.528+0000 7f7c4944f640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c440ff9a0 0x7f7c44197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.528+0000 7f7c4944f640 1 -- 172.21.15.7:0/3349432073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4419f670 con 0x7f7c44103330 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.528+0000 7f7c48c4e640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c440ff9a0 0x7f7c44197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.528+0000 7f7c4944f640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c44103330 0x7f7c4419f000 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00eb10 tx=0x7f7c3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.528+0000 7f7c41ffb640 1 -- 172.21.15.7:0/3349432073 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c004b20 con 0x7f7c44103330 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.528+0000 7f7c41ffb640 1 -- 172.21.15.7:0/3349432073 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c004cc0 con 0x7f7c44103330 2024-03-26T08:48:52.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.528+0000 7f7c41ffb640 1 -- 172.21.15.7:0/3349432073 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c010760 con 0x7f7c44103330 2024-03-26T08:48:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.529+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/3349432073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4419f960 con 0x7f7c44103330 2024-03-26T08:48:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.529+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/3349432073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4419fe60 con 0x7f7c44103330 2024-03-26T08:48:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.532+0000 7f7c41ffb640 1 -- 172.21.15.7:0/3349432073 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7c3c005020 con 0x7f7c44103330 2024-03-26T08:48:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.533+0000 7f7c4db7e740 1 Processor -- start 2024-03-26T08:48:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.533+0000 7f7c4db7e740 1 -- start start 2024-03-26T08:48:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.533+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8519cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.533+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 0x563fc851cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.533+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 0x563fc852a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.533+0000 7f7c4db7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc847eb10 con 0x563fc85198a0 2024-03-26T08:48:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.533+0000 7f7c4db7e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc8427900 con 0x563fc851a290 2024-03-26T08:48:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.533+0000 7f7c4db7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc847abb0 con 0x563fc851d3f0 2024-03-26T08:48:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.534+0000 7f7c41ffb640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7c340a2a00 0x7f7c340a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.534+0000 7f7c41ffb640 1 -- 172.21.15.7:0/3349432073 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c3c0cf040 con 0x7f7c44103330 2024-03-26T08:48:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.536+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8519cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.536+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8519cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38400/0 (socket says 172.21.15.7:38400) 2024-03-26T08:48:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.536+0000 7f7c43fff640 1 -- 172.21.15.7:0/1043658419 learned_addr learned my addr 172.21.15.7:0/1043658419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.538+0000 7f7c43fff640 1 --2- 172.21.15.7:0/1043658419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 0x563fc852a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.539+0000 7f7c48c4e640 1 --2- 172.21.15.7:0/1043658419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 0x563fc851cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.540+0000 7f7c43fff640 1 -- 172.21.15.7:0/1043658419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 msgr2=0x563fc851cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.540+0000 7f7c43fff640 1 --2- 172.21.15.7:0/1043658419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 0x563fc851cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.540+0000 7f7c43fff640 1 -- 172.21.15.7:0/1043658419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 msgr2=0x563fc8519cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.540+0000 7f7c43fff640 1 --2- 172.21.15.7:0/1043658419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8519cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.540+0000 7f7c43fff640 1 -- 172.21.15.7:0/1043658419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc8193210 con 0x563fc851d3f0 2024-03-26T08:48:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.541+0000 7f7c43fff640 1 --2- 172.21.15.7:0/1043658419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 0x563fc852a260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009fd0 tx=0x7f7c30005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=919aeeb15ef034b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.542+0000 7f7c4944f640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7c340a2a00 0x7f7c340a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.546+0000 7f7c4944f640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7c340a2a00 0x7f7c340a4ec0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c004f90 tx=0x7f7c3c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.547+0000 7f7c2effd640 1 -- 172.21.15.7:0/1043658419 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30011df0 con 0x563fc851d3f0 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.547+0000 7f7c2effd640 1 -- 172.21.15.7:0/1043658419 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c3000f7c0 con 0x563fc851d3f0 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.548+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1043658419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 msgr2=0x563fc852a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.548+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1043658419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 0x563fc852a260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009fd0 tx=0x7f7c30005f90 comp rx=0 tx=0).stop 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.550+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1043658419 shutdown_connections 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.550+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1043658419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 0x563fc852a260 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.550+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1043658419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 0x563fc851cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.550+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1043658419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8519cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.550+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1043658419 >> 172.21.15.7:0/1043658419 conn(0x563fc8514e90 msgr2=0x563fc8515290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.550+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1043658419 shutdown_connections 2024-03-26T08:48:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.550+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1043658419 wait complete. 2024-03-26T08:48:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.551+0000 7f7c4db7e740 1 -- start start 2024-03-26T08:48:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.551+0000 7f7c4db7e740 1 Processor -- start 2024-03-26T08:48:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.555+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc85198a0 0x563fc8523b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.555+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc851a290 0x563fc85240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.555+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 0x563fc85290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.555+0000 7f7c4db7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc851de80 con 0x563fc851a290 2024-03-26T08:48:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.555+0000 7f7c4db7e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc843ad50 con 0x563fc851d3f0 2024-03-26T08:48:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.555+0000 7f7c4db7e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc851e000 con 0x563fc85198a0 2024-03-26T08:48:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.555+0000 7f7c4db7e740 10 client.0 _ll_register_callbacks cb 0x563fc843a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.559+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc85198a0 0x563fc8523b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.559+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc85198a0 0x563fc8523b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44170/0 (socket says 172.21.15.7:44170) 2024-03-26T08:48:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.559+0000 7f7c43fff640 1 -- 172.21.15.7:0/321276632 learned_addr learned my addr 172.21.15.7:0/321276632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.559+0000 7f7c43fff640 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 0x563fc85290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.561+0000 7f7c43fff640 1 -- 172.21.15.7:0/321276632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc85198a0 msgr2=0x563fc8523b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.561+0000 7f7c43fff640 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc85198a0 0x563fc8523b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.561+0000 7f7c43fff640 1 -- 172.21.15.7:0/321276632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc851a290 msgr2=0x563fc85240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.561+0000 7f7c43fff640 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc851a290 0x563fc85240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.561+0000 7f7c43fff640 1 -- 172.21.15.7:0/321276632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fc825f620 con 0x563fc851d3f0 2024-03-26T08:48:52.560 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.567+0000 7f7c43fff640 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 0x563fc85290f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7c300057c0 tx=0x7f7c300115c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94176ea0cc64c0fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.568+0000 7f7c23fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:48:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.571+0000 7f7c23fff640 1 -- 172.21.15.7:0/321276632 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30005030 con 0x563fc851d3f0 2024-03-26T08:48:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.571+0000 7f7c23fff640 1 -- 172.21.15.7:0/321276632 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c300051d0 con 0x563fc851d3f0 2024-03-26T08:48:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.573+0000 7f7c4db7e740 1 -- 172.21.15.7:0/321276632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c3000f7c0 con 0x563fc851d3f0 2024-03-26T08:48:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.574+0000 7f7c23fff640 1 -- 172.21.15.7:0/321276632 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c300053d0 con 0x563fc851d3f0 2024-03-26T08:48:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.574+0000 7f7c4db7e740 10 client.26975 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:52.567 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.574+0000 7f7c4db7e740 1 -- 172.21.15.7:0/321276632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563fc8550560 con 0x563fc851d3f0 2024-03-26T08:48:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.574+0000 7f7c23fff640 1 -- 172.21.15.7:0/321276632 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c3000f7c0 con 0x563fc851d3f0 2024-03-26T08:48:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.574+0000 7f7c23fff640 1 -- 172.21.15.7:0/321276632 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7c3001d020 con 0x563fc851d3f0 2024-03-26T08:48:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.575+0000 7f7c4db7e740 10 client.26975 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.575+0000 7f7c4db7e740 10 client.26975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.575+0000 7f7c4db7e740 20 client.26975 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.575+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563fc85519e0 0x563fc8553dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.575+0000 7f7c4db7e740 4 client.26975 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:48:52.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.575+0000 7f7c4db7e740 1 -- 172.21.15.7:0/321276632 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fc848c050 con 0x563fc85519e0 2024-03-26T08:48:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.580+0000 7f7c48c4e640 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563fc85519e0 0x563fc8553dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.581+0000 7f7c48c4e640 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563fc85519e0 0x563fc8553dd0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf39d8ae80c074e server_cookie=93e8fb1cf9dad0a5 in_seq=0 out_seq=0 2024-03-26T08:48:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.581+0000 7f7c23fff640 10 client.26975 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c23fff640 1 -- 172.21.15.7:0/321276632 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563fc848c050 con 0x563fc85519e0 2024-03-26T08:48:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c23fff640 10 client.26975 handle_command_reply: tid=0 2024-03-26T08:48:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 client.26975 shutdown 2024-03-26T08:48:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 -- 172.21.15.7:0/321276632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 msgr2=0x563fc85290f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 0x563fc85290f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7c300057c0 tx=0x7f7c300115c0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 -- 172.21.15.7:0/321276632 shutdown_connections 2024-03-26T08:48:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563fc85519e0 0x563fc8553dd0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 0x563fc85290f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc851a290 0x563fc85240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/321276632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc85198a0 0x563fc8523b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 -- 172.21.15.7:0/321276632 >> 172.21.15.7:0/321276632 conn(0x563fc8514e90 msgr2=0x563fc8514020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c23fff640 0 client.26975 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 -- 172.21.15.7:0/321276632 shutdown_connections 2024-03-26T08:48:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 1 -- 172.21.15.7:0/321276632 wait complete. 2024-03-26T08:48:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.582+0000 7f7c4db7e740 20 client.26975 trim_cache size 0 max 16384 2024-03-26T08:48:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.606+0000 7f7c4db7e740 1 Processor -- start 2024-03-26T08:48:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.606+0000 7f7c4db7e740 1 -- start start 2024-03-26T08:48:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.606+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8515270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.606+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851a290 0x563fc85180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 0x563fc8518630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c48c4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851a290 0x563fc85180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8515270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8515270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38412/0 (socket says 172.21.15.7:38412) 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c48c4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851a290 0x563fc85180f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44172/0 (socket says 172.21.15.7:44172) 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 -- 172.21.15.7:0/554197914 learned_addr learned my addr 172.21.15.7:0/554197914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c4db7e740 1 -- 172.21.15.7:0/554197914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c441a07f0 con 0x563fc85198a0 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c4db7e740 1 -- 172.21.15.7:0/554197914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44071ad0 con 0x563fc851d3f0 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c4db7e740 1 -- 172.21.15.7:0/554197914 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4406fb50 con 0x563fc851a290 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 --2- 172.21.15.7:0/554197914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 0x563fc8518630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 -- 172.21.15.7:0/554197914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851a290 msgr2=0x563fc85180f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 --2- 172.21.15.7:0/554197914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851a290 0x563fc85180f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 -- 172.21.15.7:0/554197914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 msgr2=0x563fc8518630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 --2- 172.21.15.7:0/554197914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 0x563fc8518630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 -- 172.21.15.7:0/554197914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc851a670 con 0x563fc85198a0 2024-03-26T08:48:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c48c4e640 1 --2- 172.21.15.7:0/554197914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851a290 0x563fc85180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.607+0000 7f7c43fff640 1 --2- 172.21.15.7:0/554197914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8515270 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009670 tx=0x7f7c30009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cce93ec096af29b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.608+0000 7f7c2e7fc640 1 -- 172.21.15.7:0/554197914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30024040 con 0x563fc85198a0 2024-03-26T08:48:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.608+0000 7f7c2e7fc640 1 -- 172.21.15.7:0/554197914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c30005570 con 0x563fc85198a0 2024-03-26T08:48:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.608+0000 7f7c2e7fc640 1 -- 172.21.15.7:0/554197914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30010690 con 0x563fc85198a0 2024-03-26T08:48:52.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.609+0000 7f7c4db7e740 1 -- 172.21.15.7:0/554197914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 msgr2=0x563fc8515270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.609+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/554197914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8515270 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009670 tx=0x7f7c30009c90 comp rx=0 tx=0).stop 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.610+0000 7f7c4db7e740 1 -- 172.21.15.7:0/554197914 shutdown_connections 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.610+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/554197914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851d3f0 0x563fc8518630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.610+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/554197914 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851a290 0x563fc85180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.610+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/554197914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc8515270 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.610+0000 7f7c4db7e740 1 -- 172.21.15.7:0/554197914 >> 172.21.15.7:0/554197914 conn(0x563fc8514e90 msgr2=0x563fc850feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.610+0000 7f7c4db7e740 1 -- 172.21.15.7:0/554197914 shutdown_connections 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.611+0000 7f7c4db7e740 1 -- 172.21.15.7:0/554197914 wait complete. 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.611+0000 7f7c4db7e740 1 -- start start 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.611+0000 7f7c4db7e740 1 Processor -- start 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc85622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 0x563fc85627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c4db7e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 0x563fc8563200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c4db7e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44070150 con 0x563fc85198a0 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc85622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c48c4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 0x563fc85627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc85622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38418/0 (socket says 172.21.15.7:38418) 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 -- 172.21.15.7:0/1791330755 learned_addr learned my addr 172.21.15.7:0/1791330755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44071470 con 0x563fc851a290 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440707b0 con 0x563fc851d3f0 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c4db7e740 10 client.0 _ll_register_callbacks cb 0x7f7c3000fd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 0x563fc8563200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 -- 172.21.15.7:0/1791330755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 msgr2=0x563fc8563200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 0x563fc8563200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 -- 172.21.15.7:0/1791330755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 msgr2=0x563fc85627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 0x563fc85627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 -- 172.21.15.7:0/1791330755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fc8563840 con 0x563fc85198a0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c48c4e640 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 0x563fc85627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c43fff640 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc85622a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009800 tx=0x7f7c30003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4314e5a4958eda75 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c23fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.612+0000 7f7c23fff640 1 -- 172.21.15.7:0/1791330755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3002a040 con 0x563fc85198a0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c23fff640 1 -- 172.21.15.7:0/1791330755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c3001f640 con 0x563fc85198a0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c30005570 con 0x563fc85198a0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c23fff640 1 -- 172.21.15.7:0/1791330755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c3001f840 con 0x563fc85198a0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c4db7e740 10 client.17136 fetch_fsmap learned FSMap version 25 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563fc85acd20 con 0x563fc85198a0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c23fff640 1 -- 172.21.15.7:0/1791330755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c30005570 con 0x563fc85198a0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c23fff640 1 -- 172.21.15.7:0/1791330755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7c3001d040 con 0x563fc85198a0 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c4db7e740 10 client.17136 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c4db7e740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:48:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c4db7e740 20 client.17136 populate_metadata read hostname 'smithi007' 2024-03-26T08:48:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563fc85ad640 0x563fc85afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:48:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c4db7e740 4 client.17136 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:48:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563fc84863a0 con 0x563fc85ad640 2024-03-26T08:48:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.613+0000 7f7c48c4e640 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563fc85ad640 0x563fc85afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:48:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.614+0000 7f7c48c4e640 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563fc85ad640 0x563fc85afa60 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab25e2d1dc1f5b2 server_cookie=4d352fb608cb4267 in_seq=0 out_seq=0 2024-03-26T08:48:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.614+0000 7f7c23fff640 10 client.17136 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c23fff640 1 -- 172.21.15.7:0/1791330755 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563fc84863a0 con 0x563fc85ad640 2024-03-26T08:48:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c23fff640 10 client.17136 handle_command_reply: tid=0 2024-03-26T08:48:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c4db7e740 1 client.17136 shutdown 2024-03-26T08:48:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 msgr2=0x563fc85622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc85622a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009800 tx=0x7f7c30003c60 comp rx=0 tx=0).stop 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 shutdown_connections 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563fc85ad640 0x563fc85afa60 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563fc851d3f0 0x563fc8563200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c23fff640 0 client.17136 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563fc851a290 0x563fc85627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c4db7e740 1 --2- 172.21.15.7:0/1791330755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563fc85198a0 0x563fc85622a0 secure :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009800 tx=0x7f7c30003c60 comp rx=0 tx=0).stop 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.635+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 >> 172.21.15.7:0/1791330755 conn(0x563fc8514e90 msgr2=0x563fc8515270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.636+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 shutdown_connections 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.636+0000 7f7c4db7e740 1 -- 172.21.15.7:0/1791330755 wait complete. 2024-03-26T08:48:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.636+0000 7f7c4db7e740 20 client.17136 trim_cache size 0 max 16384 2024-03-26T08:48:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:48:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:48:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:48:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:48:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.636+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7c340a2a00 msgr2=0x7f7c340a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.636+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7c340a2a00 0x7f7c340a4ec0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c004f90 tx=0x7f7c3c0058d0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.636+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c44103330 msgr2=0x7f7c4419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.636+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c44103330 0x7f7c4419f000 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00eb10 tx=0x7f7c3c00c2d0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.637+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/3349432073 shutdown_connections 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.637+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7c340a2a00 0x7f7c340a4ec0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.637+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c44103330 0x7f7c4419f000 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.637+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c44102970 0x7f7c44197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.637+0000 7f7c4aeb2640 1 --2- 172.21.15.7:0/3349432073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c440ff9a0 0x7f7c44197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.637+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/3349432073 >> 172.21.15.7:0/3349432073 conn(0x7f7c440fb2f0 msgr2=0x7f7c44105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.637+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/3349432073 shutdown_connections 2024-03-26T08:48:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:48:52.637+0000 7f7c4aeb2640 1 -- 172.21.15.7:0/3349432073 wait complete. 2024-03-26T08:48:53.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:52 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:48:53.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:52 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:53.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:52 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:48:53.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:52 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:53.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:52 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:48:53.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:52 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:54.194 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:53 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:48:54.195 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:53 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:54.195 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:53 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:48:54.195 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:53 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:54.195 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:53 smithi007 ceph-mon[32462]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 864 op/s 2024-03-26T08:48:54.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:53 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:48:54.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:53 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:54.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:53 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:48:54.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:53 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:54.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:53 smithi016 ceph-mon[35851]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 864 op/s 2024-03-26T08:48:54.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:53 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:48:54.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:53 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:54.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:53 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:48:54.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:53 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:48:54.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:53 smithi044 ceph-mon[36461]: pgmap v554: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 MiB/s wr, 864 op/s 2024-03-26T08:48:56.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 385.0 s, 2301.6 tps, lat 43.634 ms stddev 46.310 2024-03-26T08:48:56.244 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:55 smithi016 ceph-mon[35851]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 817 op/s 2024-03-26T08:48:56.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:55 smithi007 ceph-mon[32462]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 817 op/s 2024-03-26T08:48:56.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:55 smithi044 ceph-mon[36461]: pgmap v555: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 817 op/s 2024-03-26T08:48:58.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:57 smithi016 ceph-mon[35851]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 9.9 MiB/s wr, 836 op/s 2024-03-26T08:48:58.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:57 smithi007 ceph-mon[32462]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 9.9 MiB/s wr, 836 op/s 2024-03-26T08:48:58.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:57 smithi044 ceph-mon[36461]: pgmap v556: 97 pgs: 97 active+clean; 1.6 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 9.9 MiB/s wr, 836 op/s 2024-03-26T08:49:00.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:48:59 smithi016 ceph-mon[35851]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.6 MiB/s wr, 810 op/s 2024-03-26T08:49:00.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:48:59 smithi007 ceph-mon[32462]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.6 MiB/s wr, 810 op/s 2024-03-26T08:49:00.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:48:59 smithi044 ceph-mon[36461]: pgmap v557: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.6 MiB/s wr, 810 op/s 2024-03-26T08:49:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 390.0 s, 2474.8 tps, lat 40.014 ms stddev 44.616 2024-03-26T08:49:02.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:01 smithi016 ceph-mon[35851]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.4 MiB/s wr, 697 op/s 2024-03-26T08:49:02.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:01 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:49:02.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:01 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:49:02.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:01 smithi007 ceph-mon[32462]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.4 MiB/s wr, 697 op/s 2024-03-26T08:49:02.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:01 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:49:02.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:01 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:49:02.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:01 smithi044 ceph-mon[36461]: pgmap v558: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.4 MiB/s wr, 697 op/s 2024-03-26T08:49:02.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:01 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:49:02.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:01 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:49:04.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:03 smithi016 ceph-mon[35851]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.2 MiB/s wr, 780 op/s 2024-03-26T08:49:04.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:03 smithi007 ceph-mon[32462]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.2 MiB/s wr, 780 op/s 2024-03-26T08:49:04.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:03 smithi044 ceph-mon[36461]: pgmap v559: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.2 MiB/s wr, 780 op/s 2024-03-26T08:49:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.776+0000 7f05c559a640 1 -- 172.21.15.7:0/1413276438 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc004d10 con 0x7f05c8101600 2024-03-26T08:49:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.777+0000 7f05c759e640 1 -- 172.21.15.7:0/1413276438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 msgr2=0x7f05c8101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.777+0000 7f05c759e640 1 --2- 172.21.15.7:0/1413276438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 0x7f05c8101a80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f05bc01a910 tx=0x7f05bc03cca0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.781+0000 7f05c759e640 1 -- 172.21.15.7:0/1413276438 shutdown_connections 2024-03-26T08:49:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.781+0000 7f05c759e640 1 --2- 172.21.15.7:0/1413276438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c8101fc0 0x7f05c8108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.781+0000 7f05c759e640 1 --2- 172.21.15.7:0/1413276438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 0x7f05c8101a80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.781+0000 7f05c759e640 1 --2- 172.21.15.7:0/1413276438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c8100410 0x7f05c8100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.778 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.781+0000 7f05c759e640 1 -- 172.21.15.7:0/1413276438 >> 172.21.15.7:0/1413276438 conn(0x7f05c80fb960 msgr2=0x7f05c80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.788+0000 7f05c759e640 1 -- 172.21.15.7:0/1413276438 shutdown_connections 2024-03-26T08:49:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.788+0000 7f05c759e640 1 -- 172.21.15.7:0/1413276438 wait complete. 2024-03-26T08:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.790+0000 7f05c759e640 1 Processor -- start 2024-03-26T08:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.790+0000 7f05c759e640 1 -- start start 2024-03-26T08:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.790+0000 7f05c759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c8100410 0x7f05c8107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.790+0000 7f05c759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 0x7f05c8107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.790+0000 7f05c759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c8101fc0 0x7f05c819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.790+0000 7f05c759e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c806dd70 con 0x7f05c8101fc0 2024-03-26T08:49:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.790+0000 7f05c759e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c806dbf0 con 0x7f05c8100410 2024-03-26T08:49:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.790+0000 7f05c759e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c806def0 con 0x7f05c8101600 2024-03-26T08:49:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.790+0000 7f05c5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 0x7f05c8107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.791+0000 7f05c5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 0x7f05c8107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:51400/0 (socket says 172.21.15.7:51400) 2024-03-26T08:49:05.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.791+0000 7f05c5d9b640 1 -- 172.21.15.7:0/3666107614 learned_addr learned my addr 172.21.15.7:0/3666107614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.791+0000 7f05c6d9d640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c8101fc0 0x7f05c819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.791+0000 7f05c5d9b640 1 -- 172.21.15.7:0/3666107614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c8100410 msgr2=0x7f05c8107a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.791+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c8100410 0x7f05c8107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.791+0000 7f05c5d9b640 1 -- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c8101fc0 msgr2=0x7f05c819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.791+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c8101fc0 0x7f05c819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.791+0000 7f05c5d9b640 1 -- 172.21.15.7:0/3666107614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c819d320 con 0x7f05c8101600 2024-03-26T08:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.791+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 0x7f05c8107f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f05bc000c00 tx=0x7f05bc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:05.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.792+0000 7f05b77fe640 1 -- 172.21.15.7:0/3666107614 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc052020 con 0x7f05c8101600 2024-03-26T08:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.792+0000 7f05b77fe640 1 -- 172.21.15.7:0/3666107614 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05bc004410 con 0x7f05c8101600 2024-03-26T08:49:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.793+0000 7f05b77fe640 1 -- 172.21.15.7:0/3666107614 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc003e50 con 0x7f05c8101600 2024-03-26T08:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.794+0000 7f05c759e640 1 -- 172.21.15.7:0/3666107614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c819d5b0 con 0x7f05c8101600 2024-03-26T08:49:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.794+0000 7f05c759e640 1 -- 172.21.15.7:0/3666107614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c819da80 con 0x7f05c8101600 2024-03-26T08:49:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.797+0000 7f05b77fe640 1 -- 172.21.15.7:0/3666107614 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f05bc055020 con 0x7f05c8101600 2024-03-26T08:49:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.797+0000 7f05cf9ea740 1 Processor -- start 2024-03-26T08:49:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.797+0000 7f05cf9ea740 1 -- start start 2024-03-26T08:49:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.797+0000 7f05b77fe640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f05b00a2ae0 0x7f05b00a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc950b20 0x561efc950f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc951510 0x561efc956010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 0x561efc958b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efc8b5b10 con 0x561efc950b20 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efc85e900 con 0x561efc956680 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efc8b1bb0 con 0x561efc951510 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05b77fe640 1 -- 172.21.15.7:0/3666107614 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f05bc1015f0 con 0x7f05c8101600 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c6d9d640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f05b00a2ae0 0x7f05b00a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc950b20 0x561efc950f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc950b20 0x561efc950f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55518/0 (socket says 172.21.15.7:55518) 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc951510 0x561efc956010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c659c640 1 -- 172.21.15.7:0/222570465 learned_addr learned my addr 172.21.15.7:0/222570465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c6d9d640 1 --2- 172.21.15.7:0/222570465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 0x561efc958b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c659c640 1 -- 172.21.15.7:0/222570465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc951510 msgr2=0x561efc956010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c659c640 1 --2- 172.21.15.7:0/222570465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc951510 0x561efc956010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c659c640 1 -- 172.21.15.7:0/222570465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 msgr2=0x561efc958b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c659c640 1 --2- 172.21.15.7:0/222570465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 0x561efc958b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c659c640 1 -- 172.21.15.7:0/222570465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efc5ca210 con 0x561efc950b20 2024-03-26T08:49:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.798+0000 7f05c6d9d640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f05b00a2ae0 0x7f05b00a4fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009fd0 tx=0x7f05c0009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.799+0000 7f05c659c640 1 --2- 172.21.15.7:0/222570465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc950b20 0x561efc950f40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f05b800bb20 tx=0x7f05b800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183abc578f5b924c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.799+0000 7f05b4ff9640 1 -- 172.21.15.7:0/222570465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8016020 con 0x561efc950b20 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.799+0000 7f05b4ff9640 1 -- 172.21.15.7:0/222570465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b800db40 con 0x561efc950b20 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.799+0000 7f05b4ff9640 1 -- 172.21.15.7:0/222570465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800f690 con 0x561efc950b20 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.800+0000 7f05cf9ea740 1 -- 172.21.15.7:0/222570465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc950b20 msgr2=0x561efc950f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.800+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/222570465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc950b20 0x561efc950f40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f05b800bb20 tx=0x7f05b800bf20 comp rx=0 tx=0).stop 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.800+0000 7f05cf9ea740 1 -- 172.21.15.7:0/222570465 shutdown_connections 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.800+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/222570465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 0x561efc958b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.800+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/222570465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc951510 0x561efc956010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.800+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/222570465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc950b20 0x561efc950f40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.800+0000 7f05cf9ea740 1 -- 172.21.15.7:0/222570465 >> 172.21.15.7:0/222570465 conn(0x561efc94c090 msgr2=0x561efc94c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.800+0000 7f05cf9ea740 1 -- 172.21.15.7:0/222570465 shutdown_connections 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.800+0000 7f05cf9ea740 1 -- 172.21.15.7:0/222570465 wait complete. 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05cf9ea740 1 -- start start 2024-03-26T08:49:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05cf9ea740 1 Processor -- start 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc950b20 0x561efc95ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 0x561efc962400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efc959630 con 0x561efc951510 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efc871d50 con 0x561efc956680 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efc9597b0 con 0x561efc950b20 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05cf9ea740 10 client.0 _ll_register_callbacks cb 0x561efc871720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05c5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05c5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55520/0 (socket says 172.21.15.7:55520) 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05c5d9b640 1 -- 172.21.15.7:0/2420041056 learned_addr learned my addr 172.21.15.7:0/2420041056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05c6d9d640 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 0x561efc962400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.801+0000 7f05c659c640 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc950b20 0x561efc95ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05c5d9b640 1 -- 172.21.15.7:0/2420041056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc950b20 msgr2=0x561efc95ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc950b20 0x561efc95ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05c5d9b640 1 -- 172.21.15.7:0/2420041056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 msgr2=0x561efc962400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 0x561efc962400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05c5d9b640 1 -- 172.21.15.7:0/2420041056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561efc696620 con 0x561efc951510 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05c6d9d640 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 0x561efc962400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05adffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95b3c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0d61c0 tx=0x7f05bc0ca270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3230380aae9e4950 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05adffb640 1 -- 172.21.15.7:0/2420041056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc005020 con 0x561efc951510 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2420041056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05b800db40 con 0x561efc951510 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05adffb640 1 -- 172.21.15.7:0/2420041056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f05bc0ff1c0 con 0x561efc951510 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05adffb640 1 -- 172.21.15.7:0/2420041056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05bc0ce650 con 0x561efc951510 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05cf9ea740 10 client.17148 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.802+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2420041056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561efc959630 con 0x561efc951510 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.803+0000 7f05adffb640 1 -- 172.21.15.7:0/2420041056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b800db40 con 0x561efc951510 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.803+0000 7f05adffb640 1 -- 172.21.15.7:0/2420041056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f05bc05f060 con 0x561efc951510 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.803+0000 7f05cf9ea740 10 client.17148 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.803+0000 7f05cf9ea740 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.803+0000 7f05cf9ea740 20 client.17148 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.803+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561efc966e60 0x561efc969250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.803+0000 7f05cf9ea740 4 client.17148 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.803+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2420041056 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561efc8c3050 con 0x561efc966e60 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.803+0000 7f05c659c640 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561efc966e60 0x561efc969250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.804+0000 7f05c659c640 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561efc966e60 0x561efc969250 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a43a1d8336285d server_cookie=c503c108dee08088 in_seq=0 out_seq=0 2024-03-26T08:49:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.804+0000 7f05adffb640 10 client.17148 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05adffb640 1 -- 172.21.15.7:0/2420041056 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561efc8c3050 con 0x561efc966e60 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05adffb640 10 client.17148 handle_command_reply: tid=0 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 client.17148 shutdown 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2420041056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 msgr2=0x561efc95b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95b3c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0d61c0 tx=0x7f05bc0ca270 comp rx=0 tx=0).stop 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2420041056 shutdown_connections 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561efc966e60 0x561efc969250 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc956680 0x561efc962400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95b3c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2420041056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc950b20 0x561efc95ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2420041056 >> 172.21.15.7:0/2420041056 conn(0x561efc94c090 msgr2=0x561efc94b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:05.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05adffb640 0 client.17148 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2420041056 shutdown_connections 2024-03-26T08:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2420041056 wait complete. 2024-03-26T08:49:05.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.805+0000 7f05cf9ea740 20 client.17148 trim_cache size 0 max 16384 2024-03-26T08:49:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05cf9ea740 1 Processor -- start 2024-03-26T08:49:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05cf9ea740 1 -- start start 2024-03-26T08:49:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc950b20 0x561efc952830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc9623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc952d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81a8f50 con 0x561efc951510 2024-03-26T08:49:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81a6a50 con 0x561efc950b20 2024-03-26T08:49:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81a4a70 con 0x561efc956680 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05c5d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc9623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05c6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc952d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05c659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc950b20 0x561efc952830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05c6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc952d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:51412/0 (socket says 172.21.15.7:51412) 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05c6d9d640 1 -- 172.21.15.7:0/2717885242 learned_addr learned my addr 172.21.15.7:0/2717885242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.826+0000 7f05c659c640 1 --2- 172.21.15.7:0/2717885242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc950b20 0x561efc952830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38502/0 (socket says 172.21.15.7:38502) 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.827+0000 7f05c659c640 1 -- 172.21.15.7:0/2717885242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 msgr2=0x561efc952d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.827+0000 7f05c659c640 1 --2- 172.21.15.7:0/2717885242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc952d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.827+0000 7f05c659c640 1 -- 172.21.15.7:0/2717885242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 msgr2=0x561efc9623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.827+0000 7f05c659c640 1 --2- 172.21.15.7:0/2717885242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc9623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.827+0000 7f05c659c640 1 -- 172.21.15.7:0/2717885242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efc9695c0 con 0x561efc950b20 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.827+0000 7f05c659c640 1 --2- 172.21.15.7:0/2717885242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc950b20 0x561efc952830 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05b800d770 tx=0x7f05b8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78effebd44bc5f93 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.827+0000 7f05affff640 1 -- 172.21.15.7:0/2717885242 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b802a020 con 0x561efc950b20 2024-03-26T08:49:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.827+0000 7f05affff640 1 -- 172.21.15.7:0/2717885242 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b800db40 con 0x561efc950b20 2024-03-26T08:49:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.827+0000 7f05affff640 1 -- 172.21.15.7:0/2717885242 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8016040 con 0x561efc950b20 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.828+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2717885242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc950b20 msgr2=0x561efc952830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.828+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2717885242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc950b20 0x561efc952830 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05b800d770 tx=0x7f05b8000bc0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.828+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2717885242 shutdown_connections 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.828+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2717885242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc952d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.828+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2717885242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc9623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.828+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/2717885242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc950b20 0x561efc952830 secure :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05b800d770 tx=0x7f05b8000bc0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.829+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2717885242 >> 172.21.15.7:0/2717885242 conn(0x561efc94c090 msgr2=0x561efc95ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.829+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2717885242 shutdown_connections 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.829+0000 7f05cf9ea740 1 -- 172.21.15.7:0/2717885242 wait complete. 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.829+0000 7f05cf9ea740 1 -- start start 2024-03-26T08:49:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05cf9ea740 1 Processor -- start 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc95fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05cf9ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc960dc0 0x561efc961240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81a50d0 con 0x561efc951510 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81a63f0 con 0x561efc960dc0 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05cf9ea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c81a5730 con 0x561efc956680 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05c659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55550/0 (socket says 172.21.15.7:55550) 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05c659c640 1 -- 172.21.15.7:0/3837642566 learned_addr learned my addr 172.21.15.7:0/3837642566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05cf9ea740 10 client.0 _ll_register_callbacks cb 0x7f05bc01a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.830+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc95fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05c6d9d640 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc960dc0 0x561efc961240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05c659c640 1 -- 172.21.15.7:0/3837642566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 msgr2=0x561efc95fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05c659c640 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc95fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05c659c640 1 -- 172.21.15.7:0/3837642566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc960dc0 msgr2=0x561efc961240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05c659c640 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc960dc0 0x561efc961240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05c659c640 1 -- 172.21.15.7:0/3837642566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561efc961780 con 0x561efc951510 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc95fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05c6d9d640 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc960dc0 0x561efc961240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05c659c640 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95f870 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x561efc95bfc0 tx=0x7f05b8026830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ed28712a433d8b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05adffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.831+0000 7f05adffb640 1 -- 172.21.15.7:0/3837642566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8036070 con 0x561efc951510 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05adffb640 1 -- 172.21.15.7:0/3837642566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f05b8026bb0 con 0x561efc951510 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05cf9ea740 1 -- 172.21.15.7:0/3837642566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05b800db40 con 0x561efc951510 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05adffb640 1 -- 172.21.15.7:0/3837642566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b8016040 con 0x561efc951510 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05cf9ea740 10 client.17154 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05cf9ea740 1 -- 172.21.15.7:0/3837642566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561efc9c27f0 con 0x561efc951510 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05adffb640 1 -- 172.21.15.7:0/3837642566 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05b800db40 con 0x561efc951510 2024-03-26T08:49:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05adffb640 1 -- 172.21.15.7:0/3837642566 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f05b8039020 con 0x561efc951510 2024-03-26T08:49:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05cf9ea740 10 client.17154 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05cf9ea740 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05cf9ea740 20 client.17154 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561efc9c2f00 0x561efc9c5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05cf9ea740 4 client.17154 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:49:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.832+0000 7f05cf9ea740 1 -- 172.21.15.7:0/3837642566 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561efc8bd3a0 con 0x561efc9c2f00 2024-03-26T08:49:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.833+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561efc9c2f00 0x561efc9c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.833+0000 7f05c5d9b640 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561efc9c2f00 0x561efc9c5320 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeeb073754a43d28 server_cookie=ab67bca0a18564e7 in_seq=0 out_seq=0 2024-03-26T08:49:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.833+0000 7f05adffb640 10 client.17154 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.850+0000 7f05adffb640 1 -- 172.21.15.7:0/3837642566 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561efc8bd3a0 con 0x561efc9c2f00 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.850+0000 7f05adffb640 10 client.17154 handle_command_reply: tid=0 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.850+0000 7f05cf9ea740 1 client.17154 shutdown 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 -- 172.21.15.7:0/3837642566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 msgr2=0x561efc95f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95f870 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x561efc95bfc0 tx=0x7f05b8026830 comp rx=0 tx=0).stop 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 -- 172.21.15.7:0/3837642566 shutdown_connections 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561efc9c2f00 0x561efc9c5320 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561efc960dc0 0x561efc961240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561efc956680 0x561efc95fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 --2- 172.21.15.7:0/3837642566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561efc951510 0x561efc95f870 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 -- 172.21.15.7:0/3837642566 >> 172.21.15.7:0/3837642566 conn(0x561efc94c090 msgr2=0x561efc952830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05adffb640 0 client.17154 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 -- 172.21.15.7:0/3837642566 shutdown_connections 2024-03-26T08:49:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 1 -- 172.21.15.7:0/3837642566 wait complete. 2024-03-26T08:49:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.851+0000 7f05cf9ea740 20 client.17154 trim_cache size 0 max 16384 2024-03-26T08:49:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:49:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:49:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:49:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:49:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 -- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f05b00a2ae0 msgr2=0x7f05b00a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f05b00a2ae0 0x7f05b00a4fa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009fd0 tx=0x7f05c0009ee0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 -- 172.21.15.7:0/3666107614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 msgr2=0x7f05c8107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 0x7f05c8107f90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f05bc000c00 tx=0x7f05bc004730 comp rx=0 tx=0).stop 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 -- 172.21.15.7:0/3666107614 shutdown_connections 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f05b00a2ae0 0x7f05b00a4fa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c8101fc0 0x7f05c819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05c8101600 0x7f05c8107f90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 --2- 172.21.15.7:0/3666107614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c8100410 0x7f05c8107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 -- 172.21.15.7:0/3666107614 >> 172.21.15.7:0/3666107614 conn(0x7f05c80fb960 msgr2=0x7f05c80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 -- 172.21.15.7:0/3666107614 shutdown_connections 2024-03-26T08:49:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:05.852+0000 7f05c759e640 1 -- 172.21.15.7:0/3666107614 wait complete. 2024-03-26T08:49:06.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 395.0 s, 2271.0 tps, lat 44.505 ms stddev 52.151 2024-03-26T08:49:06.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:05 smithi016 ceph-mon[35851]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 6.1 MiB/s wr, 701 op/s 2024-03-26T08:49:06.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:05 smithi007 ceph-mon[32462]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 6.1 MiB/s wr, 701 op/s 2024-03-26T08:49:06.317 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:05 smithi044 ceph-mon[36461]: pgmap v560: 97 pgs: 97 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 6.1 MiB/s wr, 701 op/s 2024-03-26T08:49:08.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:07 smithi016 ceph-mon[35851]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.2 MiB/s wr, 704 op/s 2024-03-26T08:49:08.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:07 smithi007 ceph-mon[32462]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.2 MiB/s wr, 704 op/s 2024-03-26T08:49:08.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:07 smithi044 ceph-mon[36461]: pgmap v561: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.2 MiB/s wr, 704 op/s 2024-03-26T08:49:10.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:09 smithi016 ceph-mon[35851]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 663 op/s 2024-03-26T08:49:10.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:09 smithi007 ceph-mon[32462]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 663 op/s 2024-03-26T08:49:10.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:09 smithi044 ceph-mon[36461]: pgmap v562: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 663 op/s 2024-03-26T08:49:11.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 400.0 s, 2598.6 tps, lat 38.232 ms stddev 41.870 2024-03-26T08:49:12.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:11 smithi016 ceph-mon[35851]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 7.0 MiB/s wr, 680 op/s 2024-03-26T08:49:12.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:11 smithi007 ceph-mon[32462]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 7.0 MiB/s wr, 680 op/s 2024-03-26T08:49:12.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:11 smithi044 ceph-mon[36461]: pgmap v563: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 7.0 MiB/s wr, 680 op/s 2024-03-26T08:49:14.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:13 smithi016 ceph-mon[35851]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.8 MiB/s wr, 700 op/s 2024-03-26T08:49:14.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:13 smithi007 ceph-mon[32462]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.8 MiB/s wr, 700 op/s 2024-03-26T08:49:14.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:13 smithi044 ceph-mon[36461]: pgmap v564: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.8 MiB/s wr, 700 op/s 2024-03-26T08:49:16.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 405.0 s, 2681.0 tps, lat 37.489 ms stddev 41.511 2024-03-26T08:49:16.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:15 smithi016 ceph-mon[35851]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.3 MiB/s wr, 709 op/s 2024-03-26T08:49:16.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:15 smithi007 ceph-mon[32462]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.3 MiB/s wr, 709 op/s 2024-03-26T08:49:16.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:15 smithi044 ceph-mon[36461]: pgmap v565: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.3 MiB/s wr, 709 op/s 2024-03-26T08:49:18.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:17 smithi016 ceph-mon[35851]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.3 MiB/s wr, 724 op/s 2024-03-26T08:49:18.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:17 smithi007 ceph-mon[32462]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.3 MiB/s wr, 724 op/s 2024-03-26T08:49:18.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:17 smithi044 ceph-mon[36461]: pgmap v566: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.3 MiB/s wr, 724 op/s 2024-03-26T08:49:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:18.999+0000 7fa2af09a640 1 -- 172.21.15.7:0/4174243895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a80fff70 msgr2=0x7fa2a81023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:18.999+0000 7fa2af09a640 1 --2- 172.21.15.7:0/4174243895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a80fff70 0x7fa2a81023a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009ca0 tx=0x7fa2a002c120 comp rx=0 tx=0).stop 2024-03-26T08:49:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.000+0000 7fa2af09a640 1 -- 172.21.15.7:0/4174243895 shutdown_connections 2024-03-26T08:49:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.000+0000 7fa2af09a640 1 --2- 172.21.15.7:0/4174243895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8105290 0x7fa2a8105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.000+0000 7fa2af09a640 1 --2- 172.21.15.7:0/4174243895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a81028e0 0x7fa2a8104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.000+0000 7fa2af09a640 1 --2- 172.21.15.7:0/4174243895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a80fff70 0x7fa2a81023a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.000+0000 7fa2af09a640 1 -- 172.21.15.7:0/4174243895 >> 172.21.15.7:0/4174243895 conn(0x7fa2a80fbbb0 msgr2=0x7fa2a80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.002+0000 7fa2af09a640 1 -- 172.21.15.7:0/4174243895 shutdown_connections 2024-03-26T08:49:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.002+0000 7fa2af09a640 1 -- 172.21.15.7:0/4174243895 wait complete. 2024-03-26T08:49:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.003+0000 7fa2af09a640 1 Processor -- start 2024-03-26T08:49:18.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.004+0000 7fa2af09a640 1 -- start start 2024-03-26T08:49:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.005+0000 7fa2af09a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a80fff70 0x7fa2a8110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.005+0000 7fa2af09a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a81028e0 0x7fa2a81106d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.005+0000 7fa2af09a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8105290 0x7fa2a810e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.005+0000 7fa2af09a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a8106fe0 con 0x7fa2a80fff70 2024-03-26T08:49:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.005+0000 7fa2af09a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a8106e60 con 0x7fa2a8105290 2024-03-26T08:49:18.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.005+0000 7fa2af09a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a8107160 con 0x7fa2a81028e0 2024-03-26T08:49:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.005+0000 7fa2ace36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a80fff70 0x7fa2a8110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.005+0000 7fa2ace36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a80fff70 0x7fa2a8110190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34254/0 (socket says 172.21.15.7:34254) 2024-03-26T08:49:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.005+0000 7fa2ace36640 1 -- 172.21.15.7:0/1960412480 learned_addr learned my addr 172.21.15.7:0/1960412480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.006+0000 7fa2ad637640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8105290 0x7fa2a810e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.006+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a81028e0 0x7fa2a81106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.006+0000 7fa2ace36640 1 -- 172.21.15.7:0/1960412480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a81028e0 msgr2=0x7fa2a81106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.006+0000 7fa2ace36640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a81028e0 0x7fa2a81106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.006+0000 7fa2ace36640 1 -- 172.21.15.7:0/1960412480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8105290 msgr2=0x7fa2a810e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.006+0000 7fa2ace36640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8105290 0x7fa2a810e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.006+0000 7fa2ace36640 1 -- 172.21.15.7:0/1960412480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a810f0e0 con 0x7fa2a80fff70 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.006+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a81028e0 0x7fa2a81106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.006+0000 7fa2ace36640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a80fff70 0x7fa2a8110190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa2a002c730 tx=0x7fa2a0035e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:19.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.007+0000 7fa2a5ffb640 1 -- 172.21.15.7:0/1960412480 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0039040 con 0x7fa2a80fff70 2024-03-26T08:49:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.007+0000 7fa2a5ffb640 1 -- 172.21.15.7:0/1960412480 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0044070 con 0x7fa2a80fff70 2024-03-26T08:49:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.007+0000 7fa2a5ffb640 1 -- 172.21.15.7:0/1960412480 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a003f440 con 0x7fa2a80fff70 2024-03-26T08:49:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.007+0000 7fa2af09a640 1 -- 172.21.15.7:0/1960412480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a810f370 con 0x7fa2a80fff70 2024-03-26T08:49:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.007+0000 7fa2af09a640 1 -- 172.21.15.7:0/1960412480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a81aedb0 con 0x7fa2a80fff70 2024-03-26T08:49:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.012+0000 7fa2a5ffb640 1 -- 172.21.15.7:0/1960412480 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa2a0004040 con 0x7fa2a80fff70 2024-03-26T08:49:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.012+0000 7fa2b1d66740 1 Processor -- start 2024-03-26T08:49:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.013+0000 7fa2b1d66740 1 -- start start 2024-03-26T08:49:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.013+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e3011d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2ad637640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e3011d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2ad637640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e3011d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34264/0 (socket says 172.21.15.7:34264) 2024-03-26T08:49:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3016e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 0x55a0e3019910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e2f76b10 con 0x55a0e3011920 2024-03-26T08:49:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e2f1f900 con 0x55a0e3017480 2024-03-26T08:49:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e2f72bb0 con 0x55a0e3012310 2024-03-26T08:49:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2ad637640 1 -- 172.21.15.7:0/210694163 learned_addr learned my addr 172.21.15.7:0/210694163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2a5ffb640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2900a2930 0x7fa2900a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2ace36640 1 --2- 172.21.15.7:0/210694163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3016e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.014+0000 7fa2a5ffb640 1 -- 172.21.15.7:0/1960412480 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa2a00f0e10 con 0x7fa2a80fff70 2024-03-26T08:49:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.015+0000 7fa2ace36640 1 -- 172.21.15.7:0/210694163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 msgr2=0x55a0e3019910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:49:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.015+0000 7fa2ace36640 1 --2- 172.21.15.7:0/210694163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 0x55a0e3019910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.015+0000 7fa2ace36640 1 -- 172.21.15.7:0/210694163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 msgr2=0x55a0e3011d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.015+0000 7fa2ace36640 1 --2- 172.21.15.7:0/210694163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e3011d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.015+0000 7fa2ace36640 1 -- 172.21.15.7:0/210694163 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0e2c8b210 con 0x55a0e3012310 2024-03-26T08:49:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.015+0000 7fa2ace36640 1 --2- 172.21.15.7:0/210694163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3016e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00b9c60 tx=0x7fa2a0090530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f67d13fec16713af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.015+0000 7fa28effd640 1 -- 172.21.15.7:0/210694163 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0088a20 con 0x55a0e3012310 2024-03-26T08:49:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.015+0000 7fa28effd640 1 -- 172.21.15.7:0/210694163 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2a00bdf00 con 0x55a0e3012310 2024-03-26T08:49:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.015+0000 7fa2ad637640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2900a2930 0x7fa2900a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.016+0000 7fa2b1d66740 1 -- 172.21.15.7:0/210694163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 msgr2=0x55a0e3016e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.016+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/210694163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3016e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00b9c60 tx=0x7fa2a0090530 comp rx=0 tx=0).stop 2024-03-26T08:49:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.017+0000 7fa2b1d66740 1 -- 172.21.15.7:0/210694163 shutdown_connections 2024-03-26T08:49:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.017+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/210694163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 0x55a0e3019910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.017+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/210694163 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3016e10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.017+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/210694163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e3011d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.017+0000 7fa2b1d66740 1 -- 172.21.15.7:0/210694163 >> 172.21.15.7:0/210694163 conn(0x55a0e300ce90 msgr2=0x55a0e300d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.017+0000 7fa2ad637640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2900a2930 0x7fa2900a4df0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006fd0 tx=0x7fa29c004770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.017+0000 7fa2b1d66740 1 -- 172.21.15.7:0/210694163 shutdown_connections 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.017+0000 7fa2b1d66740 1 -- 172.21.15.7:0/210694163 wait complete. 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.018+0000 7fa2b1d66740 1 -- start start 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.018+0000 7fa2b1d66740 1 Processor -- start 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.018+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e301b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.018+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e301be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.018+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3022e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.018+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e301a1f0 con 0x55a0e3012310 2024-03-26T08:49:19.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.018+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e2f32d50 con 0x55a0e3011920 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.018+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e301a370 con 0x55a0e3017480 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.018+0000 7fa2b1d66740 10 client.0 _ll_register_callbacks cb 0x55a0e2f32720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e301be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2ace36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e301b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2ace36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e301b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34914/0 (socket says 172.21.15.7:34914) 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2ace36640 1 -- 172.21.15.7:0/301079878 learned_addr learned my addr 172.21.15.7:0/301079878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2ad637640 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3022e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2a7fff640 1 -- 172.21.15.7:0/301079878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 msgr2=0x55a0e3022e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3022e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2a7fff640 1 -- 172.21.15.7:0/301079878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 msgr2=0x55a0e301b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e301b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2a7fff640 1 -- 172.21.15.7:0/301079878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0e2d57620 con 0x55a0e3012310 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2ace36640 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e301b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:19.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2ad637640 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3022e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e301be10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa2a810ff00 tx=0x7fa29800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209eba122719a3f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa287fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa287fff640 1 -- 172.21.15.7:0/301079878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298015070 con 0x55a0e3012310 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa2b1d66740 1 -- 172.21.15.7:0/301079878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2a00bdf00 con 0x55a0e3012310 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa287fff640 1 -- 172.21.15.7:0/301079878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa2980115f0 con 0x55a0e3012310 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.019+0000 7fa287fff640 1 -- 172.21.15.7:0/301079878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa298021050 con 0x55a0e3012310 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa2b1d66740 10 client.17163 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa2b1d66740 1 -- 172.21.15.7:0/301079878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a0e30262e0 con 0x55a0e3012310 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa287fff640 1 -- 172.21.15.7:0/301079878 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2a00bdf00 con 0x55a0e3012310 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa287fff640 1 -- 172.21.15.7:0/301079878 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa298019990 con 0x55a0e3012310 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa2b1d66740 10 client.17163 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa2b1d66740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa2b1d66740 20 client.17163 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0e30279b0 0x55a0e3029dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa2b1d66740 4 client.17163 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa2b1d66740 1 -- 172.21.15.7:0/301079878 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0e2f84050 con 0x55a0e30279b0 2024-03-26T08:49:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.020+0000 7fa2ace36640 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0e30279b0 0x55a0e3029dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.021+0000 7fa287fff640 10 client.17163 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.021+0000 7fa2ace36640 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0e30279b0 0x55a0e3029dc0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d7080c4748c72b server_cookie=c8c8b0ac3c6e7429 in_seq=0 out_seq=0 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.021+0000 7fa287fff640 1 -- 172.21.15.7:0/301079878 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a0e2f84050 con 0x55a0e30279b0 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.021+0000 7fa287fff640 10 client.17163 handle_command_reply: tid=0 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.021+0000 7fa2b1d66740 1 client.17163 shutdown 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 -- 172.21.15.7:0/301079878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 msgr2=0x55a0e301be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e301be10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa2a810ff00 tx=0x7fa29800c520 comp rx=0 tx=0).stop 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 -- 172.21.15.7:0/301079878 shutdown_connections 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0e30279b0 0x55a0e3029dc0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3022e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa287fff640 0 client.17163 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e301be10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/301079878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e301b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 -- 172.21.15.7:0/301079878 >> 172.21.15.7:0/301079878 conn(0x55a0e300ce90 msgr2=0x55a0e300c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 -- 172.21.15.7:0/301079878 shutdown_connections 2024-03-26T08:49:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 1 -- 172.21.15.7:0/301079878 wait complete. 2024-03-26T08:49:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.022+0000 7fa2b1d66740 20 client.17163 trim_cache size 0 max 16384 2024-03-26T08:49:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2b1d66740 1 Processor -- start 2024-03-26T08:49:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2b1d66740 1 -- start start 2024-03-26T08:49:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e300d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3022e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 0x55a0e3013590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2ace36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e300d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81b9ff0 con 0x55a0e3011920 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81b7af0 con 0x55a0e3017480 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2ace36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e300d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34280/0 (socket says 172.21.15.7:34280) 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2ace36640 1 -- 172.21.15.7:0/3005966250 learned_addr learned my addr 172.21.15.7:0/3005966250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3022e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2b1d66740 1 -- 172.21.15.7:0/3005966250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81b5b70 con 0x55a0e3012310 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.043+0000 7fa2ad637640 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 0x55a0e3013590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa2a7fff640 1 -- 172.21.15.7:0/3005966250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 msgr2=0x55a0e3013590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 0x55a0e3013590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa2a7fff640 1 -- 172.21.15.7:0/3005966250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 msgr2=0x55a0e300d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e300d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa2a7fff640 1 -- 172.21.15.7:0/3005966250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0e3013080 con 0x55a0e3012310 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa2ace36640 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e300d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa2ad637640 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 0x55a0e3013590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3022e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa298019e30 tx=0x7fa2980105c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0c04bd55e6b7c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa28e7fc640 1 -- 172.21.15.7:0/3005966250 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298029070 con 0x55a0e3012310 2024-03-26T08:49:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa28e7fc640 1 -- 172.21.15.7:0/3005966250 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa298019a90 con 0x55a0e3012310 2024-03-26T08:49:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.044+0000 7fa28e7fc640 1 -- 172.21.15.7:0/3005966250 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298015040 con 0x55a0e3012310 2024-03-26T08:49:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.045+0000 7fa2b1d66740 1 -- 172.21.15.7:0/3005966250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 msgr2=0x55a0e3022e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.045+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3022e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa298019e30 tx=0x7fa2980105c0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.045+0000 7fa2b1d66740 1 -- 172.21.15.7:0/3005966250 shutdown_connections 2024-03-26T08:49:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.045+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3017480 0x55a0e3013590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.045+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3012310 0x55a0e3022e10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.045+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/3005966250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3011920 0x55a0e300d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.045+0000 7fa2b1d66740 1 -- 172.21.15.7:0/3005966250 >> 172.21.15.7:0/3005966250 conn(0x55a0e300ce90 msgr2=0x55a0e3016bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.045+0000 7fa2b1d66740 1 -- 172.21.15.7:0/3005966250 shutdown_connections 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.045+0000 7fa2b1d66740 1 -- 172.21.15.7:0/3005966250 wait complete. 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2b1d66740 1 -- start start 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2b1d66740 1 Processor -- start 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e3020090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e30205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2b1d66740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3021450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81b6170 con 0x55a0e3012310 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81b7490 con 0x55a0e3011920 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2b1d66740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81b67d0 con 0x55a0e3017480 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2b1d66740 10 client.0 _ll_register_callbacks cb 0x7fa29800ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2ad637640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3021450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2ace36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e3020090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2a7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e30205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2ace36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e3020090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34936/0 (socket says 172.21.15.7:34936) 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2ad637640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3021450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35270/0 (socket says 172.21.15.7:35270) 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.046+0000 7fa2ace36640 1 -- 172.21.15.7:0/2706197248 learned_addr learned my addr 172.21.15.7:0/2706197248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2ace36640 1 -- 172.21.15.7:0/2706197248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 msgr2=0x55a0e3021450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2ace36640 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3021450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2ace36640 1 -- 172.21.15.7:0/2706197248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 msgr2=0x55a0e30205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2ace36640 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e30205d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2ace36640 1 -- 172.21.15.7:0/2706197248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0e3021a40 con 0x55a0e3011920 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e30205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2ace36640 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e3020090 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00bd4b0 tx=0x7fa2a0004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5fdb5be4f5a80f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa287fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa287fff640 1 -- 172.21.15.7:0/2706197248 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00bdf00 con 0x55a0e3011920 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa287fff640 1 -- 172.21.15.7:0/2706197248 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa2a00a80b0 con 0x55a0e3011920 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2b1d66740 1 -- 172.21.15.7:0/2706197248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa298019a90 con 0x55a0e3011920 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa287fff640 1 -- 172.21.15.7:0/2706197248 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2a0009870 con 0x55a0e3011920 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2b1d66740 10 client.27005 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa2b1d66740 1 -- 172.21.15.7:0/2706197248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a0e3022dc0 con 0x55a0e3011920 2024-03-26T08:49:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.047+0000 7fa287fff640 1 -- 172.21.15.7:0/2706197248 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2a00c1950 con 0x55a0e3011920 2024-03-26T08:49:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.048+0000 7fa287fff640 1 -- 172.21.15.7:0/2706197248 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa2a003c030 con 0x55a0e3011920 2024-03-26T08:49:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.048+0000 7fa2b1d66740 10 client.27005 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.048+0000 7fa2b1d66740 10 client.27005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.048+0000 7fa2b1d66740 20 client.27005 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.048+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0e30832e0 0x55a0e30856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.048+0000 7fa2b1d66740 4 client.27005 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:49:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.048+0000 7fa2b1d66740 1 -- 172.21.15.7:0/2706197248 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fa298019a90 con 0x55a0e30832e0 2024-03-26T08:49:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.051+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0e30832e0 0x55a0e30856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.051+0000 7fa2a7fff640 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0e30832e0 0x55a0e30856d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2075bd6421f38f80 server_cookie=8f1254993c991d86 in_seq=0 out_seq=0 2024-03-26T08:49:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.051+0000 7fa287fff640 10 client.27005 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.070+0000 7fa287fff640 1 -- 172.21.15.7:0/2706197248 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fa298019a90 con 0x55a0e30832e0 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.070+0000 7fa287fff640 10 client.27005 handle_command_reply: tid=0 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 client.27005 shutdown 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 -- 172.21.15.7:0/2706197248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 msgr2=0x55a0e3020090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e3020090 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00bd4b0 tx=0x7fa2a0004990 comp rx=0 tx=0).stop 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 -- 172.21.15.7:0/2706197248 shutdown_connections 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0e30832e0 0x55a0e30856d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0e3017480 0x55a0e3021450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e3012310 0x55a0e30205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 --2- 172.21.15.7:0/2706197248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0e3011920 0x55a0e3020090 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa287fff640 0 client.27005 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 -- 172.21.15.7:0/2706197248 >> 172.21.15.7:0/2706197248 conn(0x55a0e300ce90 msgr2=0x55a0e300d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 -- 172.21.15.7:0/2706197248 shutdown_connections 2024-03-26T08:49:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 1 -- 172.21.15.7:0/2706197248 wait complete. 2024-03-26T08:49:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.071+0000 7fa2b1d66740 20 client.27005 trim_cache size 0 max 16384 2024-03-26T08:49:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:49:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:49:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:49:19.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:49:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 -- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2900a2930 msgr2=0x7fa2900a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2900a2930 0x7fa2900a4df0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006fd0 tx=0x7fa29c004770 comp rx=0 tx=0).stop 2024-03-26T08:49:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 -- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a80fff70 msgr2=0x7fa2a8110190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a80fff70 0x7fa2a8110190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa2a002c730 tx=0x7fa2a0035e70 comp rx=0 tx=0).stop 2024-03-26T08:49:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 -- 172.21.15.7:0/1960412480 shutdown_connections 2024-03-26T08:49:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2900a2930 0x7fa2900a4df0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a8105290 0x7fa2a810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2a81028e0 0x7fa2a81106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 --2- 172.21.15.7:0/1960412480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2a80fff70 0x7fa2a8110190 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.072+0000 7fa2af09a640 1 -- 172.21.15.7:0/1960412480 >> 172.21.15.7:0/1960412480 conn(0x7fa2a80fbbb0 msgr2=0x7fa2a80fbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.073+0000 7fa2af09a640 1 -- 172.21.15.7:0/1960412480 shutdown_connections 2024-03-26T08:49:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:19.073+0000 7fa2af09a640 1 -- 172.21.15.7:0/1960412480 wait complete. 2024-03-26T08:49:20.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:19 smithi016 ceph-mon[35851]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 MiB/s wr, 707 op/s 2024-03-26T08:49:20.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:19 smithi007 ceph-mon[32462]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 MiB/s wr, 707 op/s 2024-03-26T08:49:20.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:19 smithi044 ceph-mon[36461]: pgmap v567: 97 pgs: 97 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.0 MiB/s wr, 707 op/s 2024-03-26T08:49:21.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 410.0 s, 2611.4 tps, lat 38.346 ms stddev 41.283 2024-03-26T08:49:22.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:21 smithi016 ceph-mon[35851]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.8 MiB/s wr, 688 op/s 2024-03-26T08:49:22.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:21 smithi007 ceph-mon[32462]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.8 MiB/s wr, 688 op/s 2024-03-26T08:49:22.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:21 smithi044 ceph-mon[36461]: pgmap v568: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.8 MiB/s wr, 688 op/s 2024-03-26T08:49:22.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:49:22.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:49:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.587+0000 7f3f16acc640 1 -- 172.21.15.7:0/797617617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f10102890 msgr2=0x7f3f10102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.587+0000 7f3f16acc640 1 --2- 172.21.15.7:0/797617617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f10102890 0x7f3f10102d10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009900 tx=0x7f3efc02bc50 comp rx=0 tx=0).stop 2024-03-26T08:49:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.587+0000 7f3f16acc640 1 -- 172.21.15.7:0/797617617 shutdown_connections 2024-03-26T08:49:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.587+0000 7f3f16acc640 1 --2- 172.21.15.7:0/797617617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f10103250 0x7f3f10109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.587+0000 7f3f16acc640 1 --2- 172.21.15.7:0/797617617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f10102890 0x7f3f10102d10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.587+0000 7f3f16acc640 1 --2- 172.21.15.7:0/797617617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f101016a0 0x7f3f10101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 -- 172.21.15.7:0/797617617 >> 172.21.15.7:0/797617617 conn(0x7f3f100fcbf0 msgr2=0x7f3f100ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 -- 172.21.15.7:0/797617617 shutdown_connections 2024-03-26T08:49:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 -- 172.21.15.7:0/797617617 wait complete. 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 Processor -- start 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 -- start start 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f101016a0 0x7f3f101979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f10102890 0x7f3f10197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f10103250 0x7f3f1019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1010d350 con 0x7f3f101016a0 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1010d1d0 con 0x7f3f10102890 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f16acc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1010d4d0 con 0x7f3f10103250 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f14868640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f101016a0 0x7f3f101979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f14868640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f101016a0 0x7f3f101979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34310/0 (socket says 172.21.15.7:34310) 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.588+0000 7f3f14868640 1 -- 172.21.15.7:0/3658652873 learned_addr learned my addr 172.21.15.7:0/3658652873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f10102890 0x7f3f10197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f15069640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f10103250 0x7f3f1019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f0ffff640 1 -- 172.21.15.7:0/3658652873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f10103250 msgr2=0x7f3f1019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f10103250 0x7f3f1019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f0ffff640 1 -- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f101016a0 msgr2=0x7f3f101979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f101016a0 0x7f3f101979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f0ffff640 1 -- 172.21.15.7:0/3658652873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1019f610 con 0x7f3f10102890 2024-03-26T08:49:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f14868640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f101016a0 0x7f3f101979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f10102890 0x7f3f10197f30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3efc02c190 tx=0x7f3efc02c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f0dffb640 1 -- 172.21.15.7:0/3658652873 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc004450 con 0x7f3f10102890 2024-03-26T08:49:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f0dffb640 1 -- 172.21.15.7:0/3658652873 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3efc0054e0 con 0x7f3f10102890 2024-03-26T08:49:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.589+0000 7f3f16acc640 1 -- 172.21.15.7:0/3658652873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1019f8a0 con 0x7f3f10102890 2024-03-26T08:49:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.590+0000 7f3f0dffb640 1 -- 172.21.15.7:0/3658652873 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc034440 con 0x7f3f10102890 2024-03-26T08:49:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.590+0000 7f3f16acc640 1 -- 172.21.15.7:0/3658652873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1019fce0 con 0x7f3f10102890 2024-03-26T08:49:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.592+0000 7f3f0dffb640 1 -- 172.21.15.7:0/3658652873 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3efc004f80 con 0x7f3f10102890 2024-03-26T08:49:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.592+0000 7f3f0dffb640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f000a2950 0x7f3f000a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.592+0000 7f3f0dffb640 1 -- 172.21.15.7:0/3658652873 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3efc0f0920 con 0x7f3f10102890 2024-03-26T08:49:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.592+0000 7f3f14868640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f000a2950 0x7f3f000a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.592+0000 7f3f1978b740 1 Processor -- start 2024-03-26T08:49:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.592+0000 7f3f1978b740 1 -- start start 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.593+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 0x55b59413c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.593+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413c790 0x55b594141290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.593+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b594141900 0x55b594143d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.593+0000 7f3f1978b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5940a13d0 con 0x55b59413c790 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.593+0000 7f3f1978b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b594049ad0 con 0x55b594141900 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.593+0000 7f3f1978b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59409d470 con 0x55b59413bda0 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.593+0000 7f3f0ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413c790 0x55b594141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.593+0000 7f3f0ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413c790 0x55b594141290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34312/0 (socket says 172.21.15.7:34312) 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.593+0000 7f3f0ffff640 1 -- 172.21.15.7:0/3971382859 learned_addr learned my addr 172.21.15.7:0/3971382859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.594+0000 7f3f0ffff640 1 -- 172.21.15.7:0/3971382859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 msgr2=0x55b59413c1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:49:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.594+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/3971382859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 0x55b59413c1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.594+0000 7f3f0ffff640 1 -- 172.21.15.7:0/3971382859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b594141900 msgr2=0x55b594143d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:49:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.594+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/3971382859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b594141900 0x55b594143d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.594+0000 7f3f0ffff640 1 -- 172.21.15.7:0/3971382859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b593e7f7d0 con 0x55b59413c790 2024-03-26T08:49:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.594+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/3971382859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413c790 0x55b594141290 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0ee420 tx=0x7f3efc0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4c29e8b379d92c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.595+0000 7f3efaffd640 1 -- 172.21.15.7:0/3971382859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc0bd2c0 con 0x55b59413c790 2024-03-26T08:49:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.595+0000 7f3efaffd640 1 -- 172.21.15.7:0/3971382859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3efc0c1410 con 0x55b59413c790 2024-03-26T08:49:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.595+0000 7f3efaffd640 1 -- 172.21.15.7:0/3971382859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc0dd070 con 0x55b59413c790 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.595+0000 7f3f1978b740 1 -- 172.21.15.7:0/3971382859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413c790 msgr2=0x55b594141290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.595+0000 7f3f14868640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f000a2950 0x7f3f000a4e10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3f040058c0 tx=0x7f3f04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.595+0000 7f3f1978b740 1 --2- 172.21.15.7:0/3971382859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413c790 0x55b594141290 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0ee420 tx=0x7f3efc0045f0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 -- 172.21.15.7:0/3971382859 shutdown_connections 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 --2- 172.21.15.7:0/3971382859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b594141900 0x55b594143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 --2- 172.21.15.7:0/3971382859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413c790 0x55b594141290 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 --2- 172.21.15.7:0/3971382859 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 0x55b59413c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 -- 172.21.15.7:0/3971382859 >> 172.21.15.7:0/3971382859 conn(0x55b594137310 msgr2=0x55b594137710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 -- 172.21.15.7:0/3971382859 shutdown_connections 2024-03-26T08:49:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 -- 172.21.15.7:0/3971382859 wait complete. 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 -- start start 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 Processor -- start 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413bda0 0x55b594145f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 0x55b5941464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b594141900 0x55b59414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b594144670 con 0x55b59413bda0 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59405d060 con 0x55b59413c790 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5941447f0 con 0x55b594141900 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.596+0000 7f3f1978b740 10 client.0 _ll_register_callbacks cb 0x55b59403f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f14868640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413bda0 0x55b594145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f0ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 0x55b5941464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f14868640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413bda0 0x55b594145f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34324/0 (socket says 172.21.15.7:34324) 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f14868640 1 -- 172.21.15.7:0/501081942 learned_addr learned my addr 172.21.15.7:0/501081942 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f15069640 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b594141900 0x55b59414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f14868640 1 -- 172.21.15.7:0/501081942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b594141900 msgr2=0x55b59414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f14868640 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b594141900 0x55b59414d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f14868640 1 -- 172.21.15.7:0/501081942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 msgr2=0x55b5941464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f14868640 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 0x55b5941464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f14868640 1 -- 172.21.15.7:0/501081942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b593db53f0 con 0x55b59413bda0 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 0x55b5941464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f14868640 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413bda0 0x55b594145f70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009560 tx=0x7f3f04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1d00dd77d4fd9c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3eeffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3eeffff640 1 -- 172.21.15.7:0/501081942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0402a060 con 0x55b59413bda0 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3eeffff640 1 -- 172.21.15.7:0/501081942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3f0401db20 con 0x55b59413bda0 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f1978b740 1 -- 172.21.15.7:0/501081942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3efc0c1410 con 0x55b59413bda0 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3eeffff640 1 -- 172.21.15.7:0/501081942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f0401dd20 con 0x55b59413bda0 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f1978b740 10 client.17181 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.597+0000 7f3f1978b740 1 -- 172.21.15.7:0/501081942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b593d750f0 con 0x55b59413bda0 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3eeffff640 1 -- 172.21.15.7:0/501081942 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3efc0c1410 con 0x55b59413bda0 2024-03-26T08:49:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3eeffff640 1 -- 172.21.15.7:0/501081942 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3f0402f070 con 0x55b59413bda0 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3f1978b740 10 client.17181 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3f1978b740 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3f1978b740 20 client.17181 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3f1978b740 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b5941519f0 0x55b594153e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3f1978b740 4 client.17181 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3f1978b740 1 -- 172.21.15.7:0/501081942 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5940ae910 con 0x55b5941519f0 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b5941519f0 0x55b594153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b5941519f0 0x55b594153e50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de16a31f4807b9e7 server_cookie=ef5d6b7f7f071905 in_seq=0 out_seq=0 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.598+0000 7f3eeffff640 10 client.17181 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3eeffff640 1 -- 172.21.15.7:0/501081942 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b5940ae910 con 0x55b5941519f0 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3eeffff640 10 client.17181 handle_command_reply: tid=0 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3f1978b740 1 client.17181 shutdown 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3f1978b740 1 -- 172.21.15.7:0/501081942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413bda0 msgr2=0x55b594145f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3f1978b740 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413bda0 0x55b594145f70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009560 tx=0x7f3f04004730 comp rx=0 tx=0).stop 2024-03-26T08:49:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3f1978b740 1 -- 172.21.15.7:0/501081942 shutdown_connections 2024-03-26T08:49:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3f1978b740 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b5941519f0 0x55b594153e50 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3f1978b740 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b594141900 0x55b59414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3f1978b740 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 0x55b5941464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3f1978b740 1 --2- 172.21.15.7:0/501081942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b59413bda0 0x55b594145f70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3f1978b740 1 -- 172.21.15.7:0/501081942 >> 172.21.15.7:0/501081942 conn(0x55b594137310 msgr2=0x55b594136530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.599+0000 7f3eeffff640 0 client.17181 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.600+0000 7f3f1978b740 1 -- 172.21.15.7:0/501081942 shutdown_connections 2024-03-26T08:49:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.600+0000 7f3f1978b740 1 -- 172.21.15.7:0/501081942 wait complete. 2024-03-26T08:49:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.600+0000 7f3f1978b740 20 client.17181 trim_cache size 0 max 16384 2024-03-26T08:49:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.632+0000 7f3f1978b740 1 Processor -- start 2024-03-26T08:49:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f1978b740 1 -- start start 2024-03-26T08:49:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 0x55b594135ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 0x55b59414d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b5941361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f1978b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f101a2830 con 0x55b594141900 2024-03-26T08:49:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f1978b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f101a0330 con 0x55b59413c790 2024-03-26T08:49:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f1978b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f10071d40 con 0x55b59413bda0 2024-03-26T08:49:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f14868640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 0x55b594135ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f14868640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 0x55b594135ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35328/0 (socket says 172.21.15.7:35328) 2024-03-26T08:49:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.633+0000 7f3f14868640 1 -- 172.21.15.7:0/3693268815 learned_addr learned my addr 172.21.15.7:0/3693268815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.634+0000 7f3f15069640 1 --2- 172.21.15.7:0/3693268815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b5941361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.634+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/3693268815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 0x55b59414d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.637+0000 7f3f14868640 1 -- 172.21.15.7:0/3693268815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 msgr2=0x55b59414d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.637+0000 7f3f14868640 1 --2- 172.21.15.7:0/3693268815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 0x55b59414d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.637+0000 7f3f14868640 1 -- 172.21.15.7:0/3693268815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 msgr2=0x55b5941361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.637+0000 7f3f14868640 1 --2- 172.21.15.7:0/3693268815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b5941361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.637+0000 7f3f14868640 1 -- 172.21.15.7:0/3693268815 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b594138080 con 0x55b59413bda0 2024-03-26T08:49:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.641+0000 7f3f14868640 1 --2- 172.21.15.7:0/3693268815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 0x55b594135ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009f40 tx=0x7f3f040253f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=454488e997e05da5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.641+0000 7f3efa7fc640 1 -- 172.21.15.7:0/3693268815 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04035050 con 0x55b59413bda0 2024-03-26T08:49:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.641+0000 7f3efa7fc640 1 -- 172.21.15.7:0/3693268815 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f0401d9f0 con 0x55b59413bda0 2024-03-26T08:49:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.641+0000 7f3efa7fc640 1 -- 172.21.15.7:0/3693268815 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04005650 con 0x55b59413bda0 2024-03-26T08:49:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.642+0000 7f3f1978b740 1 -- 172.21.15.7:0/3693268815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 msgr2=0x55b594135ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.642+0000 7f3f1978b740 1 --2- 172.21.15.7:0/3693268815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 0x55b594135ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009f40 tx=0x7f3f040253f0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.642+0000 7f3f1978b740 1 -- 172.21.15.7:0/3693268815 shutdown_connections 2024-03-26T08:49:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.643+0000 7f3f1978b740 1 --2- 172.21.15.7:0/3693268815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b5941361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.643+0000 7f3f1978b740 1 --2- 172.21.15.7:0/3693268815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413c790 0x55b59414d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.643+0000 7f3f1978b740 1 --2- 172.21.15.7:0/3693268815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413bda0 0x55b594135ca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.643+0000 7f3f1978b740 1 -- 172.21.15.7:0/3693268815 >> 172.21.15.7:0/3693268815 conn(0x55b594137310 msgr2=0x55b594145b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.643+0000 7f3f1978b740 1 -- 172.21.15.7:0/3693268815 shutdown_connections 2024-03-26T08:49:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.643+0000 7f3f1978b740 1 -- 172.21.15.7:0/3693268815 wait complete. 2024-03-26T08:49:22.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.643+0000 7f3f1978b740 1 -- start start 2024-03-26T08:49:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.643+0000 7f3f1978b740 1 Processor -- start 2024-03-26T08:49:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413bda0 0x55b59414cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413c790 0x55b59414d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f1978b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b59414b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f1978b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f10072340 con 0x55b594141900 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f1978b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f10073660 con 0x55b59413bda0 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f1978b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f100729a0 con 0x55b59413c790 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f1978b740 10 client.0 _ll_register_callbacks cb 0x7f3f04009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f15069640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b59414b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f14868640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413bda0 0x55b59414cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f15069640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b59414b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34330/0 (socket says 172.21.15.7:34330) 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f14868640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413bda0 0x55b59414cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34980/0 (socket says 172.21.15.7:34980) 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f15069640 1 -- 172.21.15.7:0/699925027 learned_addr learned my addr 172.21.15.7:0/699925027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.644+0000 7f3f0ffff640 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413c790 0x55b59414d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.645+0000 7f3f15069640 1 -- 172.21.15.7:0/699925027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413c790 msgr2=0x55b59414d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.645+0000 7f3f15069640 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413c790 0x55b59414d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.645+0000 7f3f15069640 1 -- 172.21.15.7:0/699925027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413bda0 msgr2=0x55b59414cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.645+0000 7f3f15069640 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413bda0 0x55b59414cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.645+0000 7f3f15069640 1 -- 172.21.15.7:0/699925027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b59414b880 con 0x55b594141900 2024-03-26T08:49:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.645+0000 7f3f15069640 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b59414b270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3f08013820 tx=0x7f3f08013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4708315bbb6918 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.648+0000 7f3eeffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.648+0000 7f3eeffff640 1 -- 172.21.15.7:0/699925027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08021060 con 0x55b594141900 2024-03-26T08:49:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.648+0000 7f3eeffff640 1 -- 172.21.15.7:0/699925027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3f08011ad0 con 0x55b594141900 2024-03-26T08:49:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.648+0000 7f3f1978b740 1 -- 172.21.15.7:0/699925027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f0401d9f0 con 0x55b594141900 2024-03-26T08:49:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.648+0000 7f3eeffff640 1 -- 172.21.15.7:0/699925027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f08011cd0 con 0x55b594141900 2024-03-26T08:49:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.649+0000 7f3f1978b740 10 client.17187 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.649+0000 7f3f1978b740 1 -- 172.21.15.7:0/699925027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b5941b64d0 con 0x55b594141900 2024-03-26T08:49:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.649+0000 7f3eeffff640 1 -- 172.21.15.7:0/699925027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f0401d9f0 con 0x55b594141900 2024-03-26T08:49:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3eeffff640 1 -- 172.21.15.7:0/699925027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3f08026070 con 0x55b594141900 2024-03-26T08:49:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3f1978b740 10 client.17187 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3f1978b740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3f1978b740 20 client.17187 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3f1978b740 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b5941b6e40 0x55b5941b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3f1978b740 4 client.17187 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:49:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3f1978b740 1 -- 172.21.15.7:0/699925027 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b5940a8c60 con 0x55b5941b6e40 2024-03-26T08:49:22.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3f14868640 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b5941b6e40 0x55b5941b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3f14868640 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b5941b6e40 0x55b5941b9260 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7c595ce9ac9b911 server_cookie=5b5e0b2a88e75e1a in_seq=0 out_seq=0 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.650+0000 7f3eeffff640 10 client.17187 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3eeffff640 1 -- 172.21.15.7:0/699925027 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b5940a8c60 con 0x55b5941b6e40 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3eeffff640 10 client.17187 handle_command_reply: tid=0 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 client.17187 shutdown 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 -- 172.21.15.7:0/699925027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 msgr2=0x55b59414b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b59414b270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3f08013820 tx=0x7f3f08013ce0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 -- 172.21.15.7:0/699925027 shutdown_connections 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b5941b6e40 0x55b5941b9260 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b594141900 0x55b59414b270 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b59413c790 0x55b59414d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 --2- 172.21.15.7:0/699925027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b59413bda0 0x55b59414cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3eeffff640 0 client.17187 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 -- 172.21.15.7:0/699925027 >> 172.21.15.7:0/699925027 conn(0x55b594137310 msgr2=0x55b59413a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.651+0000 7f3f1978b740 1 -- 172.21.15.7:0/699925027 shutdown_connections 2024-03-26T08:49:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f1978b740 1 -- 172.21.15.7:0/699925027 wait complete. 2024-03-26T08:49:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f1978b740 20 client.17187 trim_cache size 0 max 16384 2024-03-26T08:49:22.647 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:49:22.647 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:49:22.647 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:49:22.647 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:49:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 -- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f000a2950 msgr2=0x7f3f000a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f000a2950 0x7f3f000a4e10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3f040058c0 tx=0x7f3f04005f90 comp rx=0 tx=0).stop 2024-03-26T08:49:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 -- 172.21.15.7:0/3658652873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f10102890 msgr2=0x7f3f10197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f10102890 0x7f3f10197f30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3efc02c190 tx=0x7f3efc02c670 comp rx=0 tx=0).stop 2024-03-26T08:49:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 -- 172.21.15.7:0/3658652873 shutdown_connections 2024-03-26T08:49:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f000a2950 0x7f3f000a4e10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f10103250 0x7f3f1019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f10102890 0x7f3f10197f30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 --2- 172.21.15.7:0/3658652873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f101016a0 0x7f3f101979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.652+0000 7f3f16acc640 1 -- 172.21.15.7:0/3658652873 >> 172.21.15.7:0/3658652873 conn(0x7f3f100fcbf0 msgr2=0x7f3f10105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:22.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.653+0000 7f3f16acc640 1 -- 172.21.15.7:0/3658652873 shutdown_connections 2024-03-26T08:49:22.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.653+0000 7f3f16acc640 1 -- 172.21.15.7:0/3658652873 wait complete. 2024-03-26T08:49:22.660 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:49:22.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:49:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.789+0000 7fa22c36f640 1 -- 172.21.15.7:0/1782658544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410e030 msgr2=0x7fa224110420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.789+0000 7fa22c36f640 1 --2- 172.21.15.7:0/1782658544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410e030 0x7fa224110420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009ca0 tx=0x7fa21c02c180 comp rx=0 tx=0).stop 2024-03-26T08:49:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.789+0000 7fa22c36f640 1 -- 172.21.15.7:0/1782658544 shutdown_connections 2024-03-26T08:49:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.789+0000 7fa22c36f640 1 --2- 172.21.15.7:0/1782658544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410e030 0x7fa224110420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.789+0000 7fa22c36f640 1 --2- 172.21.15.7:0/1782658544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22410b700 0x7fa22410daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.789+0000 7fa22c36f640 1 --2- 172.21.15.7:0/1782658544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa224108910 0x7fa224108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.789+0000 7fa22c36f640 1 -- 172.21.15.7:0/1782658544 >> 172.21.15.7:0/1782658544 conn(0x7fa22406e8f0 msgr2=0x7fa224070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:22.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.789+0000 7fa22c36f640 1 -- 172.21.15.7:0/1782658544 shutdown_connections 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.789+0000 7fa22c36f640 1 -- 172.21.15.7:0/1782658544 wait complete. 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22c36f640 1 Processor -- start 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22c36f640 1 -- start start 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22c36f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa224108910 0x7fa2241a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22c36f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410b700 0x7fa2241a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22c36f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22410e030 0x7fa2241a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22c36f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa224113940 con 0x7fa22410b700 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22c36f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2241137c0 con 0x7fa22410e030 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22c36f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa224113ac0 con 0x7fa224108910 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22990a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410b700 0x7fa2241a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22990a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410b700 0x7fa2241a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34342/0 (socket says 172.21.15.7:34342) 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22990a640 1 -- 172.21.15.7:0/506876688 learned_addr learned my addr 172.21.15.7:0/506876688 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22a10b640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa224108910 0x7fa2241a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22a90c640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22410e030 0x7fa2241a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22990a640 1 -- 172.21.15.7:0/506876688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa224108910 msgr2=0x7fa2241a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22990a640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa224108910 0x7fa2241a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22990a640 1 -- 172.21.15.7:0/506876688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22410e030 msgr2=0x7fa2241a76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22990a640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22410e030 0x7fa2241a76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.790+0000 7fa22990a640 1 -- 172.21.15.7:0/506876688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2241a7d20 con 0x7fa22410b700 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.791+0000 7fa22990a640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410b700 0x7fa2241a0640 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa21400e9f0 tx=0x7fa21400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.791+0000 7fa21b7fe640 1 -- 172.21.15.7:0/506876688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400ccc0 con 0x7fa22410b700 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.791+0000 7fa21b7fe640 1 -- 172.21.15.7:0/506876688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa214010910 con 0x7fa22410b700 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.791+0000 7fa21b7fe640 1 -- 172.21.15.7:0/506876688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140188a0 con 0x7fa22410b700 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.791+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2241a7fb0 con 0x7fa22410b700 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.791+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2241a84d0 con 0x7fa22410b700 2024-03-26T08:49:22.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.792+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa224068520 con 0x7fa22410b700 2024-03-26T08:49:22.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.794+0000 7fa21b7fe640 1 -- 172.21.15.7:0/506876688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa21401f020 con 0x7fa22410b700 2024-03-26T08:49:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.795+0000 7fa21b7fe640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2100a2800 0x7fa2100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.795+0000 7fa21b7fe640 1 -- 172.21.15.7:0/506876688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa2140cf4b0 con 0x7fa22410b700 2024-03-26T08:49:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.795+0000 7fa22a10b640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2100a2800 0x7fa2100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.795+0000 7fa22a10b640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2100a2800 0x7fa2100a4cc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa22000afd0 tx=0x7fa22000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:22.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:22.799+0000 7fa21b7fe640 1 -- 172.21.15.7:0/506876688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa214026070 con 0x7fa22410b700 2024-03-26T08:49:23.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.130+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2241a8930 con 0x7fa22410b700 2024-03-26T08:49:23.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.131+0000 7fa21b7fe640 1 -- 172.21.15.7:0/506876688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fa21409c550 con 0x7fa22410b700 2024-03-26T08:49:23.125 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:49:23.125 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:49:23.125 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:49:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2100a2800 msgr2=0x7fa2100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2100a2800 0x7fa2100a4cc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa22000afd0 tx=0x7fa22000c040 comp rx=0 tx=0).stop 2024-03-26T08:49:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410b700 msgr2=0x7fa2241a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410b700 0x7fa2241a0640 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa21400e9f0 tx=0x7fa21400edf0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 shutdown_connections 2024-03-26T08:49:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2100a2800 0x7fa2100a4cc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22410e030 0x7fa2241a76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22410b700 0x7fa2241a0640 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 --2- 172.21.15.7:0/506876688 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa224108910 0x7fa2241a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 >> 172.21.15.7:0/506876688 conn(0x7fa22406e8f0 msgr2=0x7fa224070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 shutdown_connections 2024-03-26T08:49:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.133+0000 7fa22c36f640 1 -- 172.21.15.7:0/506876688 wait complete. 2024-03-26T08:49:23.140 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:49:23.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:49:23.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.296+0000 7f429d976640 1 -- 172.21.15.7:0/1741300294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42981016b0 msgr2=0x7f4298101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.296+0000 7f429d976640 1 --2- 172.21.15.7:0/1741300294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42981016b0 0x7f4298101ad0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4288009860 tx=0x7f428802bbb0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.300+0000 7f429d976640 1 -- 172.21.15.7:0/1741300294 shutdown_connections 2024-03-26T08:49:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.300+0000 7f429d976640 1 --2- 172.21.15.7:0/1741300294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4298103260 0x7f4298109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.300+0000 7f429d976640 1 --2- 172.21.15.7:0/1741300294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42981028a0 0x7f4298102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.300+0000 7f429d976640 1 --2- 172.21.15.7:0/1741300294 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f42981016b0 0x7f4298101ad0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.300+0000 7f429d976640 1 -- 172.21.15.7:0/1741300294 >> 172.21.15.7:0/1741300294 conn(0x7f42980fcc20 msgr2=0x7f42980ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.305+0000 7f429d976640 1 -- 172.21.15.7:0/1741300294 shutdown_connections 2024-03-26T08:49:23.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.306+0000 7f429d976640 1 -- 172.21.15.7:0/1741300294 wait complete. 2024-03-26T08:49:23.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.306+0000 7f429d976640 1 Processor -- start 2024-03-26T08:49:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.308+0000 7f429d976640 1 -- start start 2024-03-26T08:49:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.309+0000 7f429d976640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42981016b0 0x7f4298110b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.309+0000 7f429d976640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42981028a0 0x7f4298111070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.309+0000 7f429d976640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4298103260 0x7f42981160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.309+0000 7f429d976640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429810b180 con 0x7f42981016b0 2024-03-26T08:49:23.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.309+0000 7f429d976640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429810b000 con 0x7f42981028a0 2024-03-26T08:49:23.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.309+0000 7f429d976640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429810b300 con 0x7f4298103260 2024-03-26T08:49:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.311+0000 7f42967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42981028a0 0x7f4298111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.311+0000 7f42967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42981028a0 0x7f4298111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35018/0 (socket says 172.21.15.7:35018) 2024-03-26T08:49:23.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.311+0000 7f42967fc640 1 -- 172.21.15.7:0/279436728 learned_addr learned my addr 172.21.15.7:0/279436728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.311+0000 7f42977fe640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4298103260 0x7f42981160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.312+0000 7f4296ffd640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42981016b0 0x7f4298110b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.312+0000 7f42967fc640 1 -- 172.21.15.7:0/279436728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4298103260 msgr2=0x7f42981160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.312+0000 7f42967fc640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4298103260 0x7f42981160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.312+0000 7f42967fc640 1 -- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42981016b0 msgr2=0x7f4298110b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.312+0000 7f42967fc640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42981016b0 0x7f4298110b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.312+0000 7f42967fc640 1 -- 172.21.15.7:0/279436728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4298116740 con 0x7f42981028a0 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.312+0000 7f4296ffd640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42981016b0 0x7f4298110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.312+0000 7f42977fe640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4298103260 0x7f42981160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.313+0000 7f42967fc640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42981028a0 0x7f4298111070 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f429000ed80 tx=0x7f429000c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.313+0000 7f429c974640 1 -- 172.21.15.7:0/279436728 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429001c070 con 0x7f42981028a0 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.313+0000 7f429c974640 1 -- 172.21.15.7:0/279436728 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4290017b20 con 0x7f42981028a0 2024-03-26T08:49:23.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.313+0000 7f429c974640 1 -- 172.21.15.7:0/279436728 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429000fc80 con 0x7f42981028a0 2024-03-26T08:49:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.313+0000 7f429d976640 1 -- 172.21.15.7:0/279436728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4298116a30 con 0x7f42981028a0 2024-03-26T08:49:23.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.314+0000 7f429d976640 1 -- 172.21.15.7:0/279436728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429806b440 con 0x7f42981028a0 2024-03-26T08:49:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.317+0000 7f42a0635740 1 Processor -- start 2024-03-26T08:49:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.317+0000 7f42a0635740 1 -- start start 2024-03-26T08:49:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.317+0000 7f429c974640 1 -- 172.21.15.7:0/279436728 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f429001f070 con 0x7f42981028a0 2024-03-26T08:49:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.317+0000 7f42a0635740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc113690 0x557ecc113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.317+0000 7f42a0635740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc118b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.317+0000 7f42a0635740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc1191f0 0x557ecc11b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.317+0000 7f42a0635740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ecc0d6610 con 0x557ecc1191f0 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.317+0000 7f42a0635740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ecc0793c0 con 0x557ecc114080 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.317+0000 7f42a0635740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ecc021d60 con 0x557ecc113690 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f4296ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc118b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f42967fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc1191f0 0x557ecc11b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f42967fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc1191f0 0x557ecc11b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34372/0 (socket says 172.21.15.7:34372) 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f42967fc640 1 -- 172.21.15.7:0/1685659055 learned_addr learned my addr 172.21.15.7:0/1685659055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f429c974640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42800a2950 0x7f42800a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f42977fe640 1 --2- 172.21.15.7:0/1685659055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc113690 0x557ecc113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f4296ffd640 1 -- 172.21.15.7:0/1685659055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc113690 msgr2=0x557ecc113ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f4296ffd640 1 --2- 172.21.15.7:0/1685659055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc113690 0x557ecc113ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f4296ffd640 1 -- 172.21.15.7:0/1685659055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc1191f0 msgr2=0x557ecc11b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f4296ffd640 1 --2- 172.21.15.7:0/1685659055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc1191f0 0x557ecc11b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f4296ffd640 1 -- 172.21.15.7:0/1685659055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ecbd8d3f0 con 0x557ecc114080 2024-03-26T08:49:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f42967fc640 1 --2- 172.21.15.7:0/1685659055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc1191f0 0x557ecc11b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.318+0000 7f429c974640 1 -- 172.21.15.7:0/279436728 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42900ceb00 con 0x7f42981028a0 2024-03-26T08:49:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.319+0000 7f4296ffd640 1 --2- 172.21.15.7:0/1685659055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc118b80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4288037040 tx=0x7f4288009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d149539384fa0a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.319+0000 7f42977fe640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42800a2950 0x7f42800a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.320+0000 7f427dffb640 1 -- 172.21.15.7:0/1685659055 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288042020 con 0x557ecc114080 2024-03-26T08:49:23.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.320+0000 7f427dffb640 1 -- 172.21.15.7:0/1685659055 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f428803a040 con 0x557ecc114080 2024-03-26T08:49:23.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.320+0000 7f427dffb640 1 -- 172.21.15.7:0/1685659055 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288039050 con 0x557ecc114080 2024-03-26T08:49:23.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42a0635740 1 -- 172.21.15.7:0/1685659055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 msgr2=0x557ecc118b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42a0635740 1 --2- 172.21.15.7:0/1685659055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc118b80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4288037040 tx=0x7f4288009590 comp rx=0 tx=0).stop 2024-03-26T08:49:23.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42a0635740 1 -- 172.21.15.7:0/1685659055 shutdown_connections 2024-03-26T08:49:23.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42a0635740 1 --2- 172.21.15.7:0/1685659055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc1191f0 0x557ecc11b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42a0635740 1 --2- 172.21.15.7:0/1685659055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc118b80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42977fe640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42800a2950 0x7f42800a4e10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x557ecc1148e0 tx=0x7f428c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42a0635740 1 --2- 172.21.15.7:0/1685659055 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc113690 0x557ecc113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42a0635740 1 -- 172.21.15.7:0/1685659055 >> 172.21.15.7:0/1685659055 conn(0x557ecc10ec00 msgr2=0x557ecc10f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42a0635740 1 -- 172.21.15.7:0/1685659055 shutdown_connections 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.321+0000 7f42a0635740 1 -- 172.21.15.7:0/1685659055 wait complete. 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42a0635740 1 -- start start 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42a0635740 1 Processor -- start 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42a0635740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc11daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42a0635740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 0x557ecc11e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42a0635740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 0x557ecc125000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42a0635740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ecc035240 con 0x557ecc113690 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42a0635740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ecc075460 con 0x557ecc1191f0 2024-03-26T08:49:23.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42a0635740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ecc11c140 con 0x557ecc114080 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42a0635740 10 client.0 _ll_register_callbacks cb 0x557ecc034c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f4296ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc11daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f4296ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc11daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34384/0 (socket says 172.21.15.7:34384) 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4296ffd640 1 -- 172.21.15.7:0/1735589877 learned_addr learned my addr 172.21.15.7:0/1735589877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.322+0000 7f42967fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 0x557ecc11e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f42977fe640 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 0x557ecc125000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4296ffd640 1 -- 172.21.15.7:0/1735589877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 msgr2=0x557ecc11e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4296ffd640 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 0x557ecc11e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4296ffd640 1 -- 172.21.15.7:0/1735589877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 msgr2=0x557ecc125000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4296ffd640 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 0x557ecc125000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4296ffd640 1 -- 172.21.15.7:0/1735589877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ecbe577d0 con 0x557ecc113690 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4296ffd640 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc11daf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4288009780 tx=0x7f4288005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83df6b90f2a29dba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4276ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:23.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4276ffd640 1 -- 172.21.15.7:0/1735589877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288042040 con 0x557ecc113690 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f4276ffd640 1 -- 172.21.15.7:0/1735589877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4288004040 con 0x557ecc113690 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.323+0000 7f42a0635740 1 -- 172.21.15.7:0/1735589877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f428803a040 con 0x557ecc113690 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f4276ffd640 1 -- 172.21.15.7:0/1735589877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4288004240 con 0x557ecc113690 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f42a0635740 10 client.17202 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f42a0635740 1 -- 172.21.15.7:0/1735589877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557ecc128370 con 0x557ecc113690 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f4276ffd640 1 -- 172.21.15.7:0/1735589877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f428804b070 con 0x557ecc113690 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f4276ffd640 1 -- 172.21.15.7:0/1735589877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f42880035f0 con 0x557ecc113690 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f42a0635740 10 client.17202 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f42a0635740 10 client.17202 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f42a0635740 20 client.17202 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f42a0635740 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557ecc1296c0 0x557ecc12bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f42a0635740 4 client.17202 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f42a0635740 1 -- 172.21.15.7:0/1735589877 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f428803a040 con 0x557ecc1296c0 2024-03-26T08:49:23.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.324+0000 7f42967fc640 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557ecc1296c0 0x557ecc12bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.325+0000 7f42967fc640 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557ecc1296c0 0x557ecc12bab0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2297ca572c6ba8ee server_cookie=8918584f963c8da6 in_seq=0 out_seq=0 2024-03-26T08:49:23.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.325+0000 7f4276ffd640 10 client.17202 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.327+0000 7f4276ffd640 1 -- 172.21.15.7:0/1735589877 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f428803a040 con 0x557ecc1296c0 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.327+0000 7f4276ffd640 10 client.17202 handle_command_reply: tid=0 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.327+0000 7f42a0635740 1 client.17202 shutdown 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.327+0000 7f42a0635740 1 -- 172.21.15.7:0/1735589877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 msgr2=0x557ecc11daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.327+0000 7f42a0635740 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc11daf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4288009780 tx=0x7f4288005780 comp rx=0 tx=0).stop 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f42a0635740 1 -- 172.21.15.7:0/1735589877 shutdown_connections 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f42a0635740 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557ecc1296c0 0x557ecc12bab0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f42a0635740 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 0x557ecc125000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f42a0635740 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 0x557ecc11e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f4276ffd640 0 client.17202 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f42a0635740 1 --2- 172.21.15.7:0/1735589877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc11daf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f42a0635740 1 -- 172.21.15.7:0/1735589877 >> 172.21.15.7:0/1735589877 conn(0x557ecc10ec00 msgr2=0x557ecc10de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f42a0635740 1 -- 172.21.15.7:0/1735589877 shutdown_connections 2024-03-26T08:49:23.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f42a0635740 1 -- 172.21.15.7:0/1735589877 wait complete. 2024-03-26T08:49:23.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.328+0000 7f42a0635740 20 client.17202 trim_cache size 0 max 16384 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.342+0000 7f42a0635740 1 Processor -- start 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.342+0000 7f42a0635740 1 -- start start 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.342+0000 7f42a0635740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc10dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42a0635740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 0x557ecc10e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f4296ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc10dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f4296ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc10dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34388/0 (socket says 172.21.15.7:34388) 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42a0635740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 0x557ecc124fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f4296ffd640 1 -- 172.21.15.7:0/4122708687 learned_addr learned my addr 172.21.15.7:0/4122708687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42a0635740 1 -- 172.21.15.7:0/4122708687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42981c2750 con 0x557ecc113690 2024-03-26T08:49:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42a0635740 1 -- 172.21.15.7:0/4122708687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42981c0250 con 0x557ecc1191f0 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42a0635740 1 -- 172.21.15.7:0/4122708687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298072210 con 0x557ecc114080 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42967fc640 1 --2- 172.21.15.7:0/4122708687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 0x557ecc10e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42977fe640 1 --2- 172.21.15.7:0/4122708687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 0x557ecc124fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42967fc640 1 -- 172.21.15.7:0/4122708687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 msgr2=0x557ecc124fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42967fc640 1 --2- 172.21.15.7:0/4122708687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 0x557ecc124fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42967fc640 1 -- 172.21.15.7:0/4122708687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 msgr2=0x557ecc10dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42967fc640 1 --2- 172.21.15.7:0/4122708687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc10dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.343+0000 7f42967fc640 1 -- 172.21.15.7:0/4122708687 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ecc114460 con 0x557ecc114080 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.344+0000 7f42967fc640 1 --2- 172.21.15.7:0/4122708687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 0x557ecc10e1e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42900a36d0 tx=0x7f4290097040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ebbd800620adca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.344+0000 7f427d7fa640 1 -- 172.21.15.7:0/4122708687 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429009adc0 con 0x557ecc114080 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.344+0000 7f427d7fa640 1 -- 172.21.15.7:0/4122708687 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f429000fe20 con 0x557ecc114080 2024-03-26T08:49:23.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.344+0000 7f427d7fa640 1 -- 172.21.15.7:0/4122708687 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290099760 con 0x557ecc114080 2024-03-26T08:49:23.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 -- 172.21.15.7:0/4122708687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 msgr2=0x557ecc10e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 --2- 172.21.15.7:0/4122708687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 0x557ecc10e1e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42900a36d0 tx=0x7f4290097040 comp rx=0 tx=0).stop 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 -- 172.21.15.7:0/4122708687 shutdown_connections 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 --2- 172.21.15.7:0/4122708687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc1191f0 0x557ecc124fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 --2- 172.21.15.7:0/4122708687 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc114080 0x557ecc10e1e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 --2- 172.21.15.7:0/4122708687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 -- 172.21.15.7:0/4122708687 >> 172.21.15.7:0/4122708687 conn(0x557ecc10ec00 msgr2=0x557ecc10d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 -- 172.21.15.7:0/4122708687 shutdown_connections 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 -- 172.21.15.7:0/4122708687 wait complete. 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.345+0000 7f42a0635740 1 -- start start 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42a0635740 1 Processor -- start 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42a0635740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc17ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42a0635740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc181250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42a0635740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc1191f0 0x557ecc1837a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42a0635740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298072810 con 0x557ecc113690 2024-03-26T08:49:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42a0635740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42981bfbf0 con 0x557ecc114080 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4296ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc17ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42a0635740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298072e70 con 0x557ecc1191f0 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4296ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc17ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34402/0 (socket says 172.21.15.7:34402) 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4296ffd640 1 -- 172.21.15.7:0/3724759087 learned_addr learned my addr 172.21.15.7:0/3724759087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42967fc640 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc181250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42a0635740 10 client.0 _ll_register_callbacks cb 0x7f42880096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42977fe640 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc1191f0 0x557ecc1837a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4296ffd640 1 -- 172.21.15.7:0/3724759087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc1191f0 msgr2=0x557ecc1837a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4296ffd640 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc1191f0 0x557ecc1837a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4296ffd640 1 -- 172.21.15.7:0/3724759087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 msgr2=0x557ecc181250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4296ffd640 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc181250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4296ffd640 1 -- 172.21.15.7:0/3724759087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ecc183d10 con 0x557ecc113690 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f42967fc640 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc181250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4296ffd640 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc17ed00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4288006010 tx=0x7f4288004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525ddaf4d62e04e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.346+0000 7f4276ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f4276ffd640 1 -- 172.21.15.7:0/3724759087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288039020 con 0x557ecc113690 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f4276ffd640 1 -- 172.21.15.7:0/3724759087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4288003b80 con 0x557ecc113690 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42a0635740 1 -- 172.21.15.7:0/3724759087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f429000fe20 con 0x557ecc113690 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f4276ffd640 1 -- 172.21.15.7:0/3724759087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4288060050 con 0x557ecc113690 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42a0635740 10 client.17208 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42a0635740 1 -- 172.21.15.7:0/3724759087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557ecc185010 con 0x557ecc113690 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f4276ffd640 1 -- 172.21.15.7:0/3724759087 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f429000fe20 con 0x557ecc113690 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f4276ffd640 1 -- 172.21.15.7:0/3724759087 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f428802c630 con 0x557ecc113690 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42a0635740 10 client.17208 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42a0635740 10 client.17208 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42a0635740 20 client.17208 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42a0635740 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557ecc185930 0x557ecc187d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42a0635740 4 client.17208 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42a0635740 1 -- 172.21.15.7:0/3724759087 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ecc086900 con 0x557ecc185930 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.347+0000 7f42967fc640 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557ecc185930 0x557ecc187d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f4276ffd640 10 client.17208 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42967fc640 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557ecc185930 0x557ecc187d50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a26c137d41394ced server_cookie=fbe69e81d3ca2c1a in_seq=0 out_seq=0 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f4276ffd640 1 -- 172.21.15.7:0/3724759087 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ecc086900 con 0x557ecc185930 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f4276ffd640 10 client.17208 handle_command_reply: tid=0 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 client.17208 shutdown 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 -- 172.21.15.7:0/3724759087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 msgr2=0x557ecc17ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc17ed00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4288006010 tx=0x7f4288004610 comp rx=0 tx=0).stop 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 -- 172.21.15.7:0/3724759087 shutdown_connections 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557ecc185930 0x557ecc187d50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ecc1191f0 0x557ecc1837a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ecc114080 0x557ecc181250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 --2- 172.21.15.7:0/3724759087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ecc113690 0x557ecc17ed00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 -- 172.21.15.7:0/3724759087 >> 172.21.15.7:0/3724759087 conn(0x557ecc10ec00 msgr2=0x557ecc10de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f4276ffd640 0 client.17208 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:49:23.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 -- 172.21.15.7:0/3724759087 shutdown_connections 2024-03-26T08:49:23.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.349+0000 7f42a0635740 1 -- 172.21.15.7:0/3724759087 wait complete. 2024-03-26T08:49:23.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f42a0635740 20 client.17208 trim_cache size 0 max 16384 2024-03-26T08:49:23.348 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:49:23.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 -- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42800a2950 msgr2=0x7f42800a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42800a2950 0x7f42800a4e10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x557ecc1148e0 tx=0x7f428c0046c0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 -- 172.21.15.7:0/279436728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42981028a0 msgr2=0x7f4298111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42981028a0 0x7f4298111070 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f429000ed80 tx=0x7f429000c6e0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 -- 172.21.15.7:0/279436728 shutdown_connections 2024-03-26T08:49:23.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f42800a2950 0x7f42800a4e10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4298103260 0x7f42981160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42981028a0 0x7f4298111070 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 --2- 172.21.15.7:0/279436728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42981016b0 0x7f4298110b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 -- 172.21.15.7:0/279436728 >> 172.21.15.7:0/279436728 conn(0x7f42980fcc20 msgr2=0x7f4298105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 -- 172.21.15.7:0/279436728 shutdown_connections 2024-03-26T08:49:23.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.350+0000 7f429d976640 1 -- 172.21.15.7:0/279436728 wait complete. 2024-03-26T08:49:23.356 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:49:23.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:49:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.505+0000 7f33b9909640 1 -- 172.21.15.7:0/3998735901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b41006f0 msgr2=0x7f33b4100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.505+0000 7f33b9909640 1 --2- 172.21.15.7:0/3998735901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b41006f0 0x7f33b4100b70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009ca0 tx=0x7f33ac02c120 comp rx=0 tx=0).stop 2024-03-26T08:49:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.509+0000 7f33b9909640 1 -- 172.21.15.7:0/3998735901 shutdown_connections 2024-03-26T08:49:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.509+0000 7f33b9909640 1 --2- 172.21.15.7:0/3998735901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b41010b0 0x7f33b4105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.509+0000 7f33b9909640 1 --2- 172.21.15.7:0/3998735901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b41006f0 0x7f33b4100b70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.509+0000 7f33b9909640 1 --2- 172.21.15.7:0/3998735901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40ff990 0x7f33b40ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.509+0000 7f33b9909640 1 -- 172.21.15.7:0/3998735901 >> 172.21.15.7:0/3998735901 conn(0x7f33b40fb2c0 msgr2=0x7f33b40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.518+0000 7f33b9909640 1 -- 172.21.15.7:0/3998735901 shutdown_connections 2024-03-26T08:49:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.521+0000 7f33b9909640 1 -- 172.21.15.7:0/3998735901 wait complete. 2024-03-26T08:49:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.521+0000 7f33b9909640 1 Processor -- start 2024-03-26T08:49:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.524+0000 7f33b9909640 1 -- start start 2024-03-26T08:49:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.524+0000 7f33b9909640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b40ff990 0x7f33b419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.524+0000 7f33b9909640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b41006f0 0x7f33b419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.524+0000 7f33b9909640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b41010b0 0x7f33b41a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.524+0000 7f33b9909640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4113a30 con 0x7f33b41010b0 2024-03-26T08:49:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.524+0000 7f33b9909640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41138b0 con 0x7f33b40ff990 2024-03-26T08:49:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.524+0000 7f33b9909640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4113bb0 con 0x7f33b41006f0 2024-03-26T08:49:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.525+0000 7f33b27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b41006f0 0x7f33b419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.525+0000 7f33b27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b41006f0 0x7f33b419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35438/0 (socket says 172.21.15.7:35438) 2024-03-26T08:49:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.525+0000 7f33b27fc640 1 -- 172.21.15.7:0/1199829692 learned_addr learned my addr 172.21.15.7:0/1199829692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.525+0000 7f33b27fc640 1 -- 172.21.15.7:0/1199829692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b40ff990 msgr2=0x7f33b419e120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:49:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.525+0000 7f33b27fc640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b40ff990 0x7f33b419e120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.525+0000 7f33b27fc640 1 -- 172.21.15.7:0/1199829692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b41010b0 msgr2=0x7f33b41a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.525+0000 7f33b27fc640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b41010b0 0x7f33b41a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.525+0000 7f33b27fc640 1 -- 172.21.15.7:0/1199829692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b406b1f0 con 0x7f33b41006f0 2024-03-26T08:49:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.526+0000 7f33b27fc640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b41006f0 0x7f33b419e660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005e30 tx=0x7f33ac032fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.527+0000 7f33b8907640 1 -- 172.21.15.7:0/1199829692 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac039040 con 0x7f33b41006f0 2024-03-26T08:49:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.527+0000 7f33b8907640 1 -- 172.21.15.7:0/1199829692 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33ac003970 con 0x7f33b41006f0 2024-03-26T08:49:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.527+0000 7f33b8907640 1 -- 172.21.15.7:0/1199829692 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac004570 con 0x7f33b41006f0 2024-03-26T08:49:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.527+0000 7f33b9909640 1 -- 172.21.15.7:0/1199829692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b406b480 con 0x7f33b41006f0 2024-03-26T08:49:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.527+0000 7f33b9909640 1 -- 172.21.15.7:0/1199829692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b406b9a0 con 0x7f33b41006f0 2024-03-26T08:49:23.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.531+0000 7f33b8907640 1 -- 172.21.15.7:0/1199829692 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f33ac004710 con 0x7f33b41006f0 2024-03-26T08:49:23.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.532+0000 7f33b8907640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f339c0a2a00 0x7f339c0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.533+0000 7f33b8907640 1 -- 172.21.15.7:0/1199829692 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f33ac0f0940 con 0x7f33b41006f0 2024-03-26T08:49:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.535+0000 7f33bc5d5740 1 Processor -- start 2024-03-26T08:49:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.535+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f339c0a2a00 0x7f339c0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33bc5d5740 1 -- start start 2024-03-26T08:49:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99fa1f0 0x562ec99fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33bc5d5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec99b7610 con 0x562ec99f4690 2024-03-26T08:49:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33bc5d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec995a3c0 con 0x562ec99f5080 2024-03-26T08:49:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33bc5d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec9902d60 con 0x562ec99fa1f0 2024-03-26T08:49:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99f4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34414/0 (socket says 172.21.15.7:34414) 2024-03-26T08:49:23.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.537+0000 7f33b2ffd640 1 -- 172.21.15.7:0/1273855916 learned_addr learned my addr 172.21.15.7:0/1273855916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f339c0a2a00 0x7f339c0a4ec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f33a800b990 tx=0x7f33a800b370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b27fc640 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b37fe640 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99fa1f0 0x562ec99fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b2ffd640 1 -- 172.21.15.7:0/1273855916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99fa1f0 msgr2=0x562ec99fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99fa1f0 0x562ec99fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b2ffd640 1 -- 172.21.15.7:0/1273855916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 msgr2=0x562ec99f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b2ffd640 1 -- 172.21.15.7:0/1273855916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ec966e3f0 con 0x562ec99f4690 2024-03-26T08:49:23.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b27fc640 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:49:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.539+0000 7f33b37fe640 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99fa1f0 0x562ec99fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:49:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.540+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99f4ab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f33a800dbe0 tx=0x7f33a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1fecba155ea176 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.540+0000 7f3399ffb640 1 -- 172.21.15.7:0/1273855916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8010da0 con 0x562ec99f4690 2024-03-26T08:49:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.540+0000 7f3399ffb640 1 -- 172.21.15.7:0/1273855916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33a801c900 con 0x562ec99f4690 2024-03-26T08:49:23.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.540+0000 7f3399ffb640 1 -- 172.21.15.7:0/1273855916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a800f980 con 0x562ec99f4690 2024-03-26T08:49:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.541+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1273855916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 msgr2=0x562ec99f4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.541+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99f4ab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f33a800dbe0 tx=0x7f33a8018000 comp rx=0 tx=0).stop 2024-03-26T08:49:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.541+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1273855916 shutdown_connections 2024-03-26T08:49:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.541+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99fa1f0 0x562ec99fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.541+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.541+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1273855916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99f4ab0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.541+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1273855916 >> 172.21.15.7:0/1273855916 conn(0x562ec99efc00 msgr2=0x562ec99f0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.541+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1273855916 shutdown_connections 2024-03-26T08:49:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.542+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1273855916 wait complete. 2024-03-26T08:49:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.543+0000 7f33bc5d5740 1 -- start start 2024-03-26T08:49:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.543+0000 7f33bc5d5740 1 Processor -- start 2024-03-26T08:49:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.543+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f4690 0x562ec99fe9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99fef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 0x562ec9a05f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99fef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99fef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35102/0 (socket says 172.21.15.7:35102) 2024-03-26T08:49:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33b27fc640 1 -- 172.21.15.7:0/559966406 learned_addr learned my addr 172.21.15.7:0/559966406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33b37fe640 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 0x562ec9a05f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec9916240 con 0x562ec99fa1f0 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec9956460 con 0x562ec99f5080 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ec99fd1a0 con 0x562ec99f4690 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.544+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f4690 0x562ec99fe9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f33b27fc640 1 -- 172.21.15.7:0/559966406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f4690 msgr2=0x562ec99fe9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f33b27fc640 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f4690 0x562ec99fe9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f33b27fc640 1 -- 172.21.15.7:0/559966406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 msgr2=0x562ec9a05f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f33b27fc640 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 0x562ec9a05f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f33b27fc640 1 -- 172.21.15.7:0/559966406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ec97387d0 con 0x562ec99f5080 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f33bc5d5740 10 client.0 _ll_register_callbacks cb 0x562ec9915c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f33b27fc640 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99fef30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0c5510 tx=0x7f33ac0bcc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989d6859a9e8a01b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f3392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f3392ffd640 1 -- 172.21.15.7:0/559966406 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0c13f0 con 0x562ec99f5080 2024-03-26T08:49:23.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.545+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33a801c900 con 0x562ec99f5080 2024-03-26T08:49:23.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.546+0000 7f3392ffd640 1 -- 172.21.15.7:0/559966406 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f33ac0dd0c0 con 0x562ec99f5080 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.546+0000 7f3392ffd640 1 -- 172.21.15.7:0/559966406 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33ac0c5a40 con 0x562ec99f5080 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.546+0000 7f33bc5d5740 10 client.27053 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.546+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x562ec9a09060 con 0x562ec99f5080 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f3392ffd640 1 -- 172.21.15.7:0/559966406 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f33ac03c030 con 0x562ec99f5080 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f33bc5d5740 10 client.27053 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f33bc5d5740 10 client.27053 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f33bc5d5740 20 client.27053 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x562ec9a0a770 0x562ec9a0cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f33bc5d5740 4 client.27053 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33a801c900 con 0x562ec9a0a770 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x562ec9a0a770 0x562ec9a0cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x562ec9a0a770 0x562ec9a0cc30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ed8ab5b6a5314e5 server_cookie=21d66e9d82973809 in_seq=0 out_seq=0 2024-03-26T08:49:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.547+0000 7f3392ffd640 10 client.27053 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:49:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.548+0000 7f3392ffd640 1 -- 172.21.15.7:0/559966406 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f33a801c900 con 0x562ec9a0a770 2024-03-26T08:49:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.548+0000 7f3392ffd640 10 client.27053 handle_command_reply: tid=0 2024-03-26T08:49:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.548+0000 7f33bc5d5740 1 client.27053 shutdown 2024-03-26T08:49:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 msgr2=0x562ec99fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99fef30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0c5510 tx=0x7f33ac0bcc00 comp rx=0 tx=0).stop 2024-03-26T08:49:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 shutdown_connections 2024-03-26T08:49:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 0x562ec9a05f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f5080 0x562ec99fef30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x562ec9a0a770 0x562ec9a0cc30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f3392ffd640 0 client.27053 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:49:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/559966406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f4690 0x562ec99fe9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 >> 172.21.15.7:0/559966406 conn(0x562ec99efc00 msgr2=0x562ec99eee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 shutdown_connections 2024-03-26T08:49:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 1 -- 172.21.15.7:0/559966406 wait complete. 2024-03-26T08:49:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.549+0000 7f33bc5d5740 20 client.27053 trim_cache size 0 max 16384 2024-03-26T08:49:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33bc5d5740 1 Processor -- start 2024-03-26T08:49:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33bc5d5740 1 -- start start 2024-03-26T08:49:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99ee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a05f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99fa1f0 0x562ec99eead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33bc5d5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0010790 con 0x562ec99f4690 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33bc5d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a000e290 con 0x562ec99fa1f0 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33bc5d5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a000c310 con 0x562ec99f5080 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99ee590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34428/0 (socket says 172.21.15.7:34428) 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33b2ffd640 1 -- 172.21.15.7:0/1720468550 learned_addr learned my addr 172.21.15.7:0/1720468550 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33b27fc640 1 --2- 172.21.15.7:0/1720468550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a05f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.568+0000 7f33b37fe640 1 --2- 172.21.15.7:0/1720468550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99fa1f0 0x562ec99eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.569+0000 7f33b2ffd640 1 -- 172.21.15.7:0/1720468550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 msgr2=0x562ec9a05f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.569+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1720468550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a05f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.569+0000 7f33b2ffd640 1 -- 172.21.15.7:0/1720468550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99fa1f0 msgr2=0x562ec99eead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.569+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1720468550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99fa1f0 0x562ec99eead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.569+0000 7f33b2ffd640 1 -- 172.21.15.7:0/1720468550 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ec99ea230 con 0x562ec99f4690 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.569+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1720468550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99ee590 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33a8010e40 tx=0x7f33a800f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffad428311225ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.569+0000 7f33997fa640 1 -- 172.21.15.7:0/1720468550 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8030020 con 0x562ec99f4690 2024-03-26T08:49:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.569+0000 7f33997fa640 1 -- 172.21.15.7:0/1720468550 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33a80055e0 con 0x562ec99f4690 2024-03-26T08:49:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.569+0000 7f33997fa640 1 -- 172.21.15.7:0/1720468550 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8030dd0 con 0x562ec99f4690 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.570+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1720468550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 msgr2=0x562ec99ee590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.570+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1720468550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99ee590 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f33a8010e40 tx=0x7f33a800f420 comp rx=0 tx=0).stop 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.570+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1720468550 shutdown_connections 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.570+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1720468550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99fa1f0 0x562ec99eead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.570+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1720468550 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a05f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.570+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1720468550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99f4690 0x562ec99ee590 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.570+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1720468550 >> 172.21.15.7:0/1720468550 conn(0x562ec99efc00 msgr2=0x562ec99f6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.570+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1720468550 shutdown_connections 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.570+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1720468550 wait complete. 2024-03-26T08:49:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.571+0000 7f33bc5d5740 1 -- start start 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.571+0000 7f33bc5d5740 1 Processor -- start 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.571+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f4690 0x562ec9a03b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33bc5d5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 0x562ec9a04ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f4690 0x562ec9a03b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f4690 0x562ec9a03b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35114/0 (socket says 172.21.15.7:35114) 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33b27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33b27fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35468/0 (socket says 172.21.15.7:35468) 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33b37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 0x562ec9a04ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33b37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 0x562ec9a04ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34432/0 (socket says 172.21.15.7:34432) 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33b2ffd640 1 -- 172.21.15.7:0/1680096010 learned_addr learned my addr 172.21.15.7:0/1680096010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a000c910 con 0x562ec99fa1f0 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a000dc30 con 0x562ec99f4690 2024-03-26T08:49:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a000cf70 con 0x562ec99f5080 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.572+0000 7f33bc5d5740 10 client.0 _ll_register_callbacks cb 0x7f33ac0bd070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33b27fc640 1 -- 172.21.15.7:0/1680096010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f4690 msgr2=0x562ec9a03b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33b27fc640 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f4690 0x562ec9a03b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33b27fc640 1 -- 172.21.15.7:0/1680096010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 msgr2=0x562ec9a04ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33b27fc640 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 0x562ec9a04ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33b27fc640 1 -- 172.21.15.7:0/1680096010 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ec9a055e0 con 0x562ec99f5080 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f4690 0x562ec9a03b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33b27fc640 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a040a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0bd230 tx=0x7f33ac0ee2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c7b04098ecbedf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f3392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f3392ffd640 1 -- 172.21.15.7:0/1680096010 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac004d40 con 0x562ec99f5080 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33a80055e0 con 0x562ec99f5080 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f3392ffd640 1 -- 172.21.15.7:0/1680096010 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f33ac0bc6f0 con 0x562ec99f5080 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f3392ffd640 1 -- 172.21.15.7:0/1680096010 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33ac0c13f0 con 0x562ec99f5080 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33bc5d5740 10 client.27097 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.573+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ec9a66560 con 0x562ec99f5080 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.574+0000 7f3392ffd640 1 -- 172.21.15.7:0/1680096010 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33ac0bc8f0 con 0x562ec99f5080 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.574+0000 7f3392ffd640 1 -- 172.21.15.7:0/1680096010 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f33ac0cd070 con 0x562ec99f5080 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.574+0000 7f33bc5d5740 10 client.27097 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.574+0000 7f33bc5d5740 10 client.27097 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.574+0000 7f33bc5d5740 20 client.27097 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.574+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x562ec9a0a770 0x562ec9a68db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.574+0000 7f33bc5d5740 4 client.27097 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.574+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f33a80055e0 con 0x562ec9a0a770 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.574+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x562ec9a0a770 0x562ec9a68db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f3392ffd640 10 client.27097 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33b2ffd640 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x562ec9a0a770 0x562ec9a68db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d363cb592bfc2c3 server_cookie=5df523921a11a6d3 in_seq=0 out_seq=0 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f3392ffd640 1 -- 172.21.15.7:0/1680096010 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f33a80055e0 con 0x562ec9a0a770 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f3392ffd640 10 client.27097 handle_command_reply: tid=0 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 client.27097 shutdown 2024-03-26T08:49:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 msgr2=0x562ec9a040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a040a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0bd230 tx=0x7f33ac0ee2d0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 shutdown_connections 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ec99fa1f0 0x562ec9a04ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x562ec9a0a770 0x562ec9a68db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ec99f5080 0x562ec9a040a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f3392ffd640 0 client.27097 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 --2- 172.21.15.7:0/1680096010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ec99f4690 0x562ec9a03b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 >> 172.21.15.7:0/1680096010 conn(0x562ec99efc00 msgr2=0x562ec99f2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 shutdown_connections 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 1 -- 172.21.15.7:0/1680096010 wait complete. 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.575+0000 7f33bc5d5740 20 client.27097 trim_cache size 0 max 16384 2024-03-26T08:49:23.574 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:49:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 -- 172.21.15.7:0/1199829692 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f339c0a2a00 msgr2=0x7f339c0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f339c0a2a00 0x7f339c0a4ec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f33a800b990 tx=0x7f33a800b370 comp rx=0 tx=0).stop 2024-03-26T08:49:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 -- 172.21.15.7:0/1199829692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b41006f0 msgr2=0x7f33b419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b41006f0 0x7f33b419e660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005e30 tx=0x7f33ac032fc0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 -- 172.21.15.7:0/1199829692 shutdown_connections 2024-03-26T08:49:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f339c0a2a00 0x7f339c0a4ec0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b41010b0 0x7f33b41a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b41006f0 0x7f33b419e660 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 --2- 172.21.15.7:0/1199829692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b40ff990 0x7f33b419e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 -- 172.21.15.7:0/1199829692 >> 172.21.15.7:0/1199829692 conn(0x7f33b40fb2c0 msgr2=0x7f33b4101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 -- 172.21.15.7:0/1199829692 shutdown_connections 2024-03-26T08:49:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.576+0000 7f33b9909640 1 -- 172.21.15.7:0/1199829692 wait complete. 2024-03-26T08:49:23.583 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:49:23.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:49:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.735+0000 7fad672e8640 1 -- 172.21.15.7:0/1239564064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad5809df40 msgr2=0x7fad580a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.735+0000 7fad672e8640 1 --2- 172.21.15.7:0/1239564064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad5809df40 0x7fad580a47d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fad50009860 tx=0x7fad5002bb70 comp rx=0 tx=0).stop 2024-03-26T08:49:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.735+0000 7fad672e8640 1 -- 172.21.15.7:0/1239564064 shutdown_connections 2024-03-26T08:49:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.735+0000 7fad672e8640 1 --2- 172.21.15.7:0/1239564064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad5809df40 0x7fad580a47d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.735+0000 7fad672e8640 1 --2- 172.21.15.7:0/1239564064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad5809d580 0x7fad5809da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.735+0000 7fad672e8640 1 --2- 172.21.15.7:0/1239564064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad5809c390 0x7fad5809c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.735+0000 7fad672e8640 1 -- 172.21.15.7:0/1239564064 >> 172.21.15.7:0/1239564064 conn(0x7fad5800b510 msgr2=0x7fad5800d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.736+0000 7fad672e8640 1 -- 172.21.15.7:0/1239564064 shutdown_connections 2024-03-26T08:49:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.736+0000 7fad672e8640 1 -- 172.21.15.7:0/1239564064 wait complete. 2024-03-26T08:49:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.736+0000 7fad672e8640 1 Processor -- start 2024-03-26T08:49:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.736+0000 7fad672e8640 1 -- start start 2024-03-26T08:49:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad672e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad5809c390 0x7fad581325e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad672e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad5809d580 0x7fad58132b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad672e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad5809df40 0x7fad58139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad672e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580a7ff0 con 0x7fad5809c390 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad672e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580a7e70 con 0x7fad5809df40 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad672e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580a8170 con 0x7fad5809d580 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad65084640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad5809c390 0x7fad581325e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad64883640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad5809d580 0x7fad58132b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad65084640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad5809c390 0x7fad581325e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34448/0 (socket says 172.21.15.7:34448) 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad65084640 1 -- 172.21.15.7:0/342860570 learned_addr learned my addr 172.21.15.7:0/342860570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad65885640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad5809df40 0x7fad58139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad64883640 1 -- 172.21.15.7:0/342860570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad5809df40 msgr2=0x7fad58139b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad64883640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad5809df40 0x7fad58139b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad64883640 1 -- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad5809c390 msgr2=0x7fad581325e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad64883640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad5809c390 0x7fad581325e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad64883640 1 -- 172.21.15.7:0/342860570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5813a200 con 0x7fad5809d580 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.737+0000 7fad65084640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad5809c390 0x7fad581325e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.738+0000 7fad64883640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad5809d580 0x7fad58132b20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00e9f0 tx=0x7fad5c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.738+0000 7fad567fc640 1 -- 172.21.15.7:0/342860570 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c00ccc0 con 0x7fad5809d580 2024-03-26T08:49:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.738+0000 7fad567fc640 1 -- 172.21.15.7:0/342860570 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad5c010910 con 0x7fad5809d580 2024-03-26T08:49:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.738+0000 7fad567fc640 1 -- 172.21.15.7:0/342860570 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c018800 con 0x7fad5809d580 2024-03-26T08:49:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.739+0000 7fad672e8640 1 -- 172.21.15.7:0/342860570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5813a4f0 con 0x7fad5809d580 2024-03-26T08:49:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.740+0000 7fad672e8640 1 -- 172.21.15.7:0/342860570 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5813a9c0 con 0x7fad5809d580 2024-03-26T08:49:23.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.742+0000 7fad567fc640 1 -- 172.21.15.7:0/342860570 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fad5c01f020 con 0x7fad5809d580 2024-03-26T08:49:23.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.743+0000 7fad567fc640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad4c0a2950 0x7fad4c0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.743+0000 7fad567fc640 1 -- 172.21.15.7:0/342860570 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fad5c0ceb50 con 0x7fad5809d580 2024-03-26T08:49:23.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.743+0000 7fad69faf740 1 Processor -- start 2024-03-26T08:49:23.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.743+0000 7fad65084640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad4c0a2950 0x7fad4c0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.744+0000 7fad69faf740 1 -- start start 2024-03-26T08:49:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.744+0000 7fad69faf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.744+0000 7fad69faf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127aed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.744+0000 7fad69faf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 0x559d127b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.744+0000 7fad69faf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1276c610 con 0x559d127a9860 2024-03-26T08:49:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.744+0000 7fad69faf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1270f3c0 con 0x559d127af3c0 2024-03-26T08:49:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.744+0000 7fad69faf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d126b7d60 con 0x559d127aa250 2024-03-26T08:49:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad65885640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 0x559d127b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad65084640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad64883640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad65885640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 0x559d127b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35138/0 (socket says 172.21.15.7:35138) 2024-03-26T08:49:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad65885640 1 -- 172.21.15.7:0/486456701 learned_addr learned my addr 172.21.15.7:0/486456701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad65084640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad4c0a2950 0x7fad4c0a4e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fad600501b0 tx=0x7fad60077040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad64883640 1 -- 172.21.15.7:0/486456701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 msgr2=0x559d127b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad64883640 1 --2- 172.21.15.7:0/486456701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 0x559d127b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad64883640 1 -- 172.21.15.7:0/486456701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 msgr2=0x559d127a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad64883640 1 --2- 172.21.15.7:0/486456701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad64883640 1 -- 172.21.15.7:0/486456701 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d124233f0 con 0x559d127aa250 2024-03-26T08:49:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.745+0000 7fad64883640 1 --2- 172.21.15.7:0/486456701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127aed50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0cc650 tx=0x7fad5c09af10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feb8d632370d999 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.746+0000 7fad4b7fe640 1 -- 172.21.15.7:0/486456701 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c07a170 con 0x559d127aa250 2024-03-26T08:49:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.746+0000 7fad4b7fe640 1 -- 172.21.15.7:0/486456701 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad5c09b4f0 con 0x559d127aa250 2024-03-26T08:49:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.746+0000 7fad4b7fe640 1 -- 172.21.15.7:0/486456701 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c09f170 con 0x559d127aa250 2024-03-26T08:49:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.746+0000 7fad69faf740 1 -- 172.21.15.7:0/486456701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 msgr2=0x559d127aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.746+0000 7fad69faf740 1 --2- 172.21.15.7:0/486456701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127aed50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0cc650 tx=0x7fad5c09af10 comp rx=0 tx=0).stop 2024-03-26T08:49:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.747+0000 7fad69faf740 1 -- 172.21.15.7:0/486456701 shutdown_connections 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.747+0000 7fad69faf740 1 --2- 172.21.15.7:0/486456701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 0x559d127b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.747+0000 7fad69faf740 1 --2- 172.21.15.7:0/486456701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127aed50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.747+0000 7fad69faf740 1 --2- 172.21.15.7:0/486456701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.747+0000 7fad69faf740 1 -- 172.21.15.7:0/486456701 >> 172.21.15.7:0/486456701 conn(0x559d127a4dd0 msgr2=0x559d127a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.747+0000 7fad69faf740 1 -- 172.21.15.7:0/486456701 shutdown_connections 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.747+0000 7fad69faf740 1 -- 172.21.15.7:0/486456701 wait complete. 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.747+0000 7fad69faf740 1 -- start start 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.747+0000 7fad69faf740 1 Processor -- start 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad69faf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127a9860 0x559d127b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad69faf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad69faf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127af3c0 0x559d127b9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad65885640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127af3c0 0x559d127b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad65084640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127a9860 0x559d127b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad69faf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d126cb240 con 0x559d127af3c0 2024-03-26T08:49:23.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad69faf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d1270b460 con 0x559d127a9860 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad69faf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d127c3320 con 0x559d127aa250 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad69faf740 10 client.0 _ll_register_callbacks cb 0x559d126cac10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad65084640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127a9860 0x559d127b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35150/0 (socket says 172.21.15.7:35150) 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad65084640 1 -- 172.21.15.7:0/3778295637 learned_addr learned my addr 172.21.15.7:0/3778295637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.748+0000 7fad64883640 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad65885640 1 -- 172.21.15.7:0/3778295637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 msgr2=0x559d127b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad65885640 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad65885640 1 -- 172.21.15.7:0/3778295637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127a9860 msgr2=0x559d127b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad65885640 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127a9860 0x559d127b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad65885640 1 -- 172.21.15.7:0/3778295637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d124ed7d0 con 0x559d127af3c0 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad65084640 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127a9860 0x559d127b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad65885640 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127af3c0 0x559d127b9820 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fad50009830 tx=0x7fad50009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfdd900fd6c9ba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad48ff9640 1 -- 172.21.15.7:0/3778295637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50043070 con 0x559d127af3c0 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad69faf740 1 -- 172.21.15.7:0/3778295637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad5c09b4f0 con 0x559d127af3c0 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.749+0000 7fad48ff9640 1 -- 172.21.15.7:0/3778295637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fad50004040 con 0x559d127af3c0 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad48ff9640 1 -- 172.21.15.7:0/3778295637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad5004f050 con 0x559d127af3c0 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad69faf740 10 client.17232 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad69faf740 1 -- 172.21.15.7:0/3778295637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559d127bcc50 con 0x559d127af3c0 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad48ff9640 1 -- 172.21.15.7:0/3778295637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad5c09b4f0 con 0x559d127af3c0 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad48ff9640 1 -- 172.21.15.7:0/3778295637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fad500326d0 con 0x559d127af3c0 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad69faf740 10 client.17232 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad69faf740 10 client.17232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad69faf740 20 client.17232 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad69faf740 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559d127be6a0 0x559d127c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad69faf740 4 client.17232 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad69faf740 1 -- 172.21.15.7:0/3778295637 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d1271c900 con 0x559d127be6a0 2024-03-26T08:49:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad65084640 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559d127be6a0 0x559d127c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad48ff9640 10 client.17232 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.750+0000 7fad65084640 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559d127be6a0 0x559d127c0b20 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a7925414515b6a7 server_cookie=5f06f064050717a in_seq=0 out_seq=0 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad48ff9640 1 -- 172.21.15.7:0/3778295637 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559d1271c900 con 0x559d127be6a0 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad48ff9640 10 client.17232 handle_command_reply: tid=0 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 client.17232 shutdown 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 -- 172.21.15.7:0/3778295637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127af3c0 msgr2=0x559d127b9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127af3c0 0x559d127b9820 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fad50009830 tx=0x7fad50009580 comp rx=0 tx=0).stop 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 -- 172.21.15.7:0/3778295637 shutdown_connections 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559d127be6a0 0x559d127c0b20 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127af3c0 0x559d127b9820 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 --2- 172.21.15.7:0/3778295637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127a9860 0x559d127b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 -- 172.21.15.7:0/3778295637 >> 172.21.15.7:0/3778295637 conn(0x559d127a4dd0 msgr2=0x559d127a4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad48ff9640 0 client.17232 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 -- 172.21.15.7:0/3778295637 shutdown_connections 2024-03-26T08:49:23.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.751+0000 7fad69faf740 1 -- 172.21.15.7:0/3778295637 wait complete. 2024-03-26T08:49:23.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.752+0000 7fad69faf740 20 client.17232 trim_cache size 0 max 16384 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad69faf740 1 Processor -- start 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad69faf740 1 -- start start 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad69faf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad69faf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127aa250 0x559d127b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad69faf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127af3c0 0x559d127a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad69faf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5813d350 con 0x559d127a9860 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad69faf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5813ae50 con 0x559d127aa250 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad69faf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad58014b90 con 0x559d127af3c0 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad65084640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad65084640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34480/0 (socket says 172.21.15.7:34480) 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad65084640 1 -- 172.21.15.7:0/259957239 learned_addr learned my addr 172.21.15.7:0/259957239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad64883640 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127aa250 0x559d127b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.768+0000 7fad65885640 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127af3c0 0x559d127a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad65084640 1 -- 172.21.15.7:0/259957239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127af3c0 msgr2=0x559d127a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad65084640 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127af3c0 0x559d127a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad65084640 1 -- 172.21.15.7:0/259957239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127aa250 msgr2=0x559d127b97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad65084640 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127aa250 0x559d127b97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad65084640 1 -- 172.21.15.7:0/259957239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d127c3490 con 0x559d127a9860 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad64883640 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127aa250 0x559d127b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad65084640 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a37e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fad60066fa0 tx=0x7fad600675b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4361ae8f76912fb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad4affd640 1 -- 172.21.15.7:0/259957239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60090170 con 0x559d127a9860 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad4affd640 1 -- 172.21.15.7:0/259957239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad60090790 con 0x559d127a9860 2024-03-26T08:49:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.769+0000 7fad4affd640 1 -- 172.21.15.7:0/259957239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60062260 con 0x559d127a9860 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 -- 172.21.15.7:0/259957239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 msgr2=0x559d127a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a37e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fad60066fa0 tx=0x7fad600675b0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 -- 172.21.15.7:0/259957239 shutdown_connections 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127af3c0 0x559d127a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127aa250 0x559d127b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 --2- 172.21.15.7:0/259957239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127a37e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 -- 172.21.15.7:0/259957239 >> 172.21.15.7:0/259957239 conn(0x559d127a4dd0 msgr2=0x559d127b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 -- 172.21.15.7:0/259957239 shutdown_connections 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 -- 172.21.15.7:0/259957239 wait complete. 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 -- start start 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 Processor -- start 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 0x559d127b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad58015190 con 0x559d127a9860 2024-03-26T08:49:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580164b0 con 0x559d127af3c0 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580157f0 con 0x559d127aa250 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.770+0000 7fad69faf740 10 client.0 _ll_register_callbacks cb 0x7fad50009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad65084640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad65084640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34488/0 (socket says 172.21.15.7:34488) 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad65084640 1 -- 172.21.15.7:0/1995769170 learned_addr learned my addr 172.21.15.7:0/1995769170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad65885640 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 0x559d127b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad64883640 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad64883640 1 -- 172.21.15.7:0/1995769170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 msgr2=0x559d127b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad64883640 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 0x559d127b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad64883640 1 -- 172.21.15.7:0/1995769170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 msgr2=0x559d127b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad64883640 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.771+0000 7fad64883640 1 -- 172.21.15.7:0/1995769170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d127b9140 con 0x559d127aa250 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.772+0000 7fad64883640 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127b5eb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x559d127aa650 tx=0x7fad5c00ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=846dd6d5fcb1ef3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.772+0000 7fad48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.772+0000 7fad48ff9640 1 -- 172.21.15.7:0/1995769170 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c07a170 con 0x559d127aa250 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.772+0000 7fad48ff9640 1 -- 172.21.15.7:0/1995769170 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fad5c0b4170 con 0x559d127aa250 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.772+0000 7fad69faf740 1 -- 172.21.15.7:0/1995769170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad60090790 con 0x559d127aa250 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.772+0000 7fad48ff9640 1 -- 172.21.15.7:0/1995769170 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad5c0b4370 con 0x559d127aa250 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.772+0000 7fad69faf740 10 client.27115 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.772+0000 7fad69faf740 1 -- 172.21.15.7:0/1995769170 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559d1283db80 con 0x559d127aa250 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.773+0000 7fad48ff9640 1 -- 172.21.15.7:0/1995769170 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad5c0a3c90 con 0x559d127aa250 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.773+0000 7fad48ff9640 1 -- 172.21.15.7:0/1995769170 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fad5c09a940 con 0x559d127aa250 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.773+0000 7fad69faf740 10 client.27115 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.773+0000 7fad69faf740 10 client.27115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.773+0000 7fad69faf740 20 client.27115 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.773+0000 7fad69faf740 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559d1283dfa0 0x559d12840390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.773+0000 7fad69faf740 4 client.27115 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.773+0000 7fad69faf740 1 -- 172.21.15.7:0/1995769170 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fad60090790 con 0x559d1283dfa0 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.773+0000 7fad65084640 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559d1283dfa0 0x559d12840390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad65084640 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559d1283dfa0 0x559d12840390 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f40527a52f03bc server_cookie=2588d38a5dcd6db0 in_seq=0 out_seq=0 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad48ff9640 10 client.27115 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad48ff9640 1 -- 172.21.15.7:0/1995769170 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fad60090790 con 0x559d1283dfa0 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad48ff9640 10 client.27115 handle_command_reply: tid=0 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 client.27115 shutdown 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 -- 172.21.15.7:0/1995769170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 msgr2=0x559d127b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127b5eb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x559d127aa650 tx=0x7fad5c00ce60 comp rx=0 tx=0).stop 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 -- 172.21.15.7:0/1995769170 shutdown_connections 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559d1283dfa0 0x559d12840390 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d127af3c0 0x559d127b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad48ff9640 0 client.27115 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d127aa250 0x559d127b5eb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 --2- 172.21.15.7:0/1995769170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d127a9860 0x559d127b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 -- 172.21.15.7:0/1995769170 >> 172.21.15.7:0/1995769170 conn(0x559d127a4dd0 msgr2=0x559d127a8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 -- 172.21.15.7:0/1995769170 shutdown_connections 2024-03-26T08:49:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 1 -- 172.21.15.7:0/1995769170 wait complete. 2024-03-26T08:49:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.774+0000 7fad69faf740 20 client.27115 trim_cache size 0 max 16384 2024-03-26T08:49:23.773 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 -- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad4c0a2950 msgr2=0x7fad4c0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad4c0a2950 0x7fad4c0a4e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fad600501b0 tx=0x7fad60077040 comp rx=0 tx=0).stop 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 -- 172.21.15.7:0/342860570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad5809d580 msgr2=0x7fad58132b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad5809d580 0x7fad58132b20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00e9f0 tx=0x7fad5c00edf0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 -- 172.21.15.7:0/342860570 shutdown_connections 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad4c0a2950 0x7fad4c0a4e10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad5809df40 0x7fad58139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad5809d580 0x7fad58132b20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 --2- 172.21.15.7:0/342860570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad5809c390 0x7fad581325e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 -- 172.21.15.7:0/342860570 >> 172.21.15.7:0/342860570 conn(0x7fad5800b510 msgr2=0x7fad580a0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 -- 172.21.15.7:0/342860570 shutdown_connections 2024-03-26T08:49:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:23.775+0000 7fad672e8640 1 -- 172.21.15.7:0/342860570 wait complete. 2024-03-26T08:49:24.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:23 smithi016 ceph-mon[35851]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.8 MiB/s wr, 702 op/s 2024-03-26T08:49:24.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:23 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/506876688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:49:24.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:23 smithi007 ceph-mon[32462]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.8 MiB/s wr, 702 op/s 2024-03-26T08:49:24.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:23 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/506876688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:49:24.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:23 smithi044 ceph-mon[36461]: pgmap v569: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.8 MiB/s wr, 702 op/s 2024-03-26T08:49:24.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:23 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/506876688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:49:24.782 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:49:24.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:49:24.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:49:24.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.921+0000 7f5abd13b640 1 -- 172.21.15.7:0/878028328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab8102190 msgr2=0x7f5ab8108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:24.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.921+0000 7f5abd13b640 1 --2- 172.21.15.7:0/878028328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab8102190 0x7f5ab8108a20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009ca0 tx=0x7f5ab002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.922+0000 7f5abd13b640 1 -- 172.21.15.7:0/878028328 shutdown_connections 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.922+0000 7f5abd13b640 1 --2- 172.21.15.7:0/878028328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab8102190 0x7f5ab8108a20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.922+0000 7f5abd13b640 1 --2- 172.21.15.7:0/878028328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab81017d0 0x7f5ab8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.922+0000 7f5abd13b640 1 --2- 172.21.15.7:0/878028328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab81005e0 0x7f5ab8100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.922+0000 7f5abd13b640 1 -- 172.21.15.7:0/878028328 >> 172.21.15.7:0/878028328 conn(0x7f5ab80fbb70 msgr2=0x7f5ab80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.922+0000 7f5abd13b640 1 -- 172.21.15.7:0/878028328 shutdown_connections 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.922+0000 7f5abd13b640 1 -- 172.21.15.7:0/878028328 wait complete. 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5abd13b640 1 Processor -- start 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5abd13b640 1 -- start start 2024-03-26T08:49:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5abd13b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab81005e0 0x7f5ab810c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5abd13b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab81017d0 0x7f5ab810a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5abd13b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab8102190 0x7f5ab810ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5abd13b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab810d370 con 0x7f5ab81017d0 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5abd13b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab810d1f0 con 0x7f5ab8102190 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5abd13b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab810d4f0 con 0x7f5ab81005e0 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab81017d0 0x7f5ab810a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5ab6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab81005e0 0x7f5ab810c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5ab759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab8102190 0x7f5ab810ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab81017d0 0x7f5ab810a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34494/0 (socket says 172.21.15.7:34494) 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5aaffff640 1 -- 172.21.15.7:0/4073303399 learned_addr learned my addr 172.21.15.7:0/4073303399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5ab759e640 1 -- 172.21.15.7:0/4073303399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab81005e0 msgr2=0x7f5ab810c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5ab759e640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab81005e0 0x7f5ab810c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5ab759e640 1 -- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab81017d0 msgr2=0x7f5ab810a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5ab759e640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab81017d0 0x7f5ab810a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5ab759e640 1 -- 172.21.15.7:0/4073303399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab810b4b0 con 0x7f5ab8102190 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.923+0000 7f5aaffff640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab81017d0 0x7f5ab810a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.924+0000 7f5ab759e640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab8102190 0x7f5ab810ac20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0037040 tx=0x7f5ab0032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.924+0000 7f5ab4d99640 1 -- 172.21.15.7:0/4073303399 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0041020 con 0x7f5ab8102190 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.924+0000 7f5abd13b640 1 -- 172.21.15.7:0/4073303399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab81a1a70 con 0x7f5ab8102190 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.924+0000 7f5ab4d99640 1 -- 172.21.15.7:0/4073303399 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab0034cd0 con 0x7f5ab8102190 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.924+0000 7f5ab4d99640 1 -- 172.21.15.7:0/4073303399 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0039040 con 0x7f5ab8102190 2024-03-26T08:49:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.924+0000 7f5abd13b640 1 -- 172.21.15.7:0/4073303399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab81a1fe0 con 0x7f5ab8102190 2024-03-26T08:49:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.927+0000 7f5ab4d99640 1 -- 172.21.15.7:0/4073303399 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5ab0044020 con 0x7f5ab8102190 2024-03-26T08:49:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.927+0000 7f5ab4d99640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa40a2880 0x7f5aa40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.927+0000 7f5ab4d99640 1 -- 172.21.15.7:0/4073303399 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5ab00f08c0 con 0x7f5ab8102190 2024-03-26T08:49:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.927+0000 7f5ab6d9d640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa40a2880 0x7f5aa40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.927+0000 7f5abfe09740 1 Processor -- start 2024-03-26T08:49:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.927+0000 7f5abfe09740 1 -- start start 2024-03-26T08:49:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe235590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe23d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5abfe09740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fe16cc90 con 0x5613fe235b60 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5abfe09740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fe192070 con 0x5613fe23acd0 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5abfe09740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fe13ac80 con 0x5613fe235170 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab6d9d640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa40a2880 0x7f5aa40a4d40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8101630 tx=0x7f5aa8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe23d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe23d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35212/0 (socket says 172.21.15.7:35212) 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab759e640 1 -- 172.21.15.7:0/1958111280 learned_addr learned my addr 172.21.15.7:0/1958111280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab6d9d640 1 --2- 172.21.15.7:0/1958111280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe235590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab759e640 1 -- 172.21.15.7:0/1958111280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 msgr2=0x5613fe235590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab759e640 1 --2- 172.21.15.7:0/1958111280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe235590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab759e640 1 -- 172.21.15.7:0/1958111280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 msgr2=0x5613fe23a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab759e640 1 --2- 172.21.15.7:0/1958111280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.928+0000 7f5ab759e640 1 -- 172.21.15.7:0/1958111280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613fe132d10 con 0x5613fe23acd0 2024-03-26T08:49:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.929+0000 7f5ab759e640 1 --2- 172.21.15.7:0/1958111280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe23d160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00b9710 tx=0x7f5ab0032340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8bef92840b6dbf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.929+0000 7f5aad7fa640 1 -- 172.21.15.7:0/1958111280 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00098b0 con 0x5613fe23acd0 2024-03-26T08:49:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.929+0000 7f5aad7fa640 1 -- 172.21.15.7:0/1958111280 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab00bd2b0 con 0x5613fe23acd0 2024-03-26T08:49:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.929+0000 7f5aad7fa640 1 -- 172.21.15.7:0/1958111280 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0003970 con 0x5613fe23acd0 2024-03-26T08:49:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.930+0000 7f5abfe09740 1 -- 172.21.15.7:0/1958111280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 msgr2=0x5613fe23d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.930+0000 7f5abfe09740 1 --2- 172.21.15.7:0/1958111280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe23d160 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00b9710 tx=0x7f5ab0032340 comp rx=0 tx=0).stop 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.931+0000 7f5abfe09740 1 -- 172.21.15.7:0/1958111280 shutdown_connections 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.931+0000 7f5abfe09740 1 --2- 172.21.15.7:0/1958111280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe23d160 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.931+0000 7f5abfe09740 1 --2- 172.21.15.7:0/1958111280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.931+0000 7f5abfe09740 1 --2- 172.21.15.7:0/1958111280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe235590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.931+0000 7f5abfe09740 1 -- 172.21.15.7:0/1958111280 >> 172.21.15.7:0/1958111280 conn(0x5613fe1f51a0 msgr2=0x5613fe230ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.931+0000 7f5abfe09740 1 -- 172.21.15.7:0/1958111280 shutdown_connections 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5abfe09740 1 -- 172.21.15.7:0/1958111280 wait complete. 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5abfe09740 1 -- start start 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5abfe09740 1 Processor -- start 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe23f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe2467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34516/0 (socket says 172.21.15.7:34516) 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5aaffff640 1 -- 172.21.15.7:0/2280790205 learned_addr learned my addr 172.21.15.7:0/2280790205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.932+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fe14e0d0 con 0x5613fe235b60 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5ab6d9d640 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe23f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fe18e110 con 0x5613fe23acd0 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613fe23da40 con 0x5613fe235170 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5ab759e640 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe2467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5abfe09740 10 client.0 _ll_register_callbacks cb 0x5613fe189e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5aaffff640 1 -- 172.21.15.7:0/2280790205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 msgr2=0x5613fe23f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5aaffff640 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe23f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5aaffff640 1 -- 172.21.15.7:0/2280790205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 msgr2=0x5613fe2467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5aaffff640 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe2467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5aaffff640 1 -- 172.21.15.7:0/2280790205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613fdf6e7e0 con 0x5613fe235b60 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5aaffff640 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23f770 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0012d20 tx=0x7f5aa000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8688bdf95cbfaa84 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5a9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/2280790205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0005020 con 0x5613fe235b60 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/2280790205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5aa00051c0 con 0x5613fe235b60 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ab00bd2b0 con 0x5613fe235b60 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/2280790205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aa00053c0 con 0x5613fe235b60 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5abfe09740 10 client.17250 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:24.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.933+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5613fdfcef70 con 0x5613fe235b60 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.934+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/2280790205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab00bd2b0 con 0x5613fe235b60 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.934+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/2280790205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5aa001c470 con 0x5613fe235b60 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.934+0000 7f5abfe09740 10 client.17250 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.934+0000 7f5abfe09740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.934+0000 7f5abfe09740 20 client.17250 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.934+0000 7f5abfe09740 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5613fe24b100 0x5613fe24d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.934+0000 7f5abfe09740 4 client.17250 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.934+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613fe19f5b0 con 0x5613fe24b100 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.934+0000 7f5ab6d9d640 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5613fe24b100 0x5613fe24d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.935+0000 7f5a9e7fc640 10 client.17250 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.935+0000 7f5ab6d9d640 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5613fe24b100 0x5613fe24d580 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dea94f5900e63 server_cookie=b3d62fc4165de050 in_seq=0 out_seq=0 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/2280790205 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5613fe19f5b0 con 0x5613fe24b100 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5a9e7fc640 10 client.17250 handle_command_reply: tid=0 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 client.17250 shutdown 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 msgr2=0x5613fe23f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23f770 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0012d20 tx=0x7f5aa000f790 comp rx=0 tx=0).stop 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 shutdown_connections 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5613fe24b100 0x5613fe24d580 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe23acd0 0x5613fe2467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5a9e7fc640 0 client.17250 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe23f770 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 --2- 172.21.15.7:0/2280790205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe23f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 >> 172.21.15.7:0/2280790205 conn(0x5613fe1f51a0 msgr2=0x5613fe227530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 shutdown_connections 2024-03-26T08:49:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 1 -- 172.21.15.7:0/2280790205 wait complete. 2024-03-26T08:49:24.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.936+0000 7f5abfe09740 20 client.17250 trim_cache size 0 max 16384 2024-03-26T08:49:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.964+0000 7f5abfe09740 1 Processor -- start 2024-03-26T08:49:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.964+0000 7f5abfe09740 1 -- start start 2024-03-26T08:49:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.965+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235170 0x5613fe2273b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.965+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe246770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.965+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe23acd0 0x5613fe251e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.965+0000 7f5abfe09740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81a8db0 con 0x5613fe235b60 2024-03-26T08:49:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.965+0000 7f5abfe09740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81a68b0 con 0x5613fe235170 2024-03-26T08:49:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.965+0000 7f5abfe09740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81a4950 con 0x5613fe23acd0 2024-03-26T08:49:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.966+0000 7f5ab759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe23acd0 0x5613fe251e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.966+0000 7f5ab759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe23acd0 0x5613fe251e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35582/0 (socket says 172.21.15.7:35582) 2024-03-26T08:49:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.966+0000 7f5ab759e640 1 -- 172.21.15.7:0/3047415939 learned_addr learned my addr 172.21.15.7:0/3047415939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.969+0000 7f5ab6d9d640 1 --2- 172.21.15.7:0/3047415939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235170 0x5613fe2273b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.970+0000 7f5ab759e640 1 -- 172.21.15.7:0/3047415939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235170 msgr2=0x5613fe2273b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.970+0000 7f5ab759e640 1 --2- 172.21.15.7:0/3047415939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235170 0x5613fe2273b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.970+0000 7f5ab759e640 1 -- 172.21.15.7:0/3047415939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 msgr2=0x5613fe246770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.970+0000 7f5ab759e640 1 --2- 172.21.15.7:0/3047415939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe246770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.970+0000 7f5ab759e640 1 -- 172.21.15.7:0/3047415939 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613fde66170 con 0x5613fe23acd0 2024-03-26T08:49:24.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.972+0000 7f5ab759e640 1 --2- 172.21.15.7:0/3047415939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe23acd0 0x5613fe251e40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009bf0 tx=0x7f5ab002c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a931363a96fc5726 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:24.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.973+0000 7f5aacff9640 1 -- 172.21.15.7:0/3047415939 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00098b0 con 0x5613fe23acd0 2024-03-26T08:49:24.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.973+0000 7f5aacff9640 1 -- 172.21.15.7:0/3047415939 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab00c1400 con 0x5613fe23acd0 2024-03-26T08:49:24.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.974+0000 7f5abfe09740 1 -- 172.21.15.7:0/3047415939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe23acd0 msgr2=0x5613fe251e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:24.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.974+0000 7f5abfe09740 1 --2- 172.21.15.7:0/3047415939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe23acd0 0x5613fe251e40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009bf0 tx=0x7f5ab002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.976+0000 7f5abfe09740 1 -- 172.21.15.7:0/3047415939 shutdown_connections 2024-03-26T08:49:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.976+0000 7f5abfe09740 1 --2- 172.21.15.7:0/3047415939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe23acd0 0x5613fe251e40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.976+0000 7f5abfe09740 1 --2- 172.21.15.7:0/3047415939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe235b60 0x5613fe246770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.976+0000 7f5abfe09740 1 --2- 172.21.15.7:0/3047415939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235170 0x5613fe2273b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.976+0000 7f5abfe09740 1 -- 172.21.15.7:0/3047415939 >> 172.21.15.7:0/3047415939 conn(0x5613fe1f51a0 msgr2=0x5613fe226ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:24.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.984+0000 7f5abfe09740 1 -- 172.21.15.7:0/3047415939 shutdown_connections 2024-03-26T08:49:24.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.988+0000 7f5abfe09740 1 -- 172.21.15.7:0/3047415939 wait complete. 2024-03-26T08:49:24.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.988+0000 7f5abfe09740 1 -- start start 2024-03-26T08:49:24.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.988+0000 7f5abfe09740 1 Processor -- start 2024-03-26T08:49:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.991+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe2466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.991+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235b60 0x5613fe244d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.991+0000 7f5abfe09740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe23acd0 0x5613fe245260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.991+0000 7f5abfe09740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81a4f50 con 0x5613fe23acd0 2024-03-26T08:49:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.991+0000 7f5abfe09740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81a6250 con 0x5613fe235b60 2024-03-26T08:49:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.991+0000 7f5abfe09740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab81a5590 con 0x5613fe235170 2024-03-26T08:49:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.991+0000 7f5abfe09740 10 client.0 _ll_register_callbacks cb 0x7f5aa0009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.992+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235b60 0x5613fe244d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.992+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235b60 0x5613fe244d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35242/0 (socket says 172.21.15.7:35242) 2024-03-26T08:49:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.992+0000 7f5aaffff640 1 -- 172.21.15.7:0/4164323155 learned_addr learned my addr 172.21.15.7:0/4164323155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.993+0000 7f5aaffff640 1 -- 172.21.15.7:0/4164323155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 msgr2=0x5613fe2466a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:49:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.993+0000 7f5aaffff640 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe2466a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.993+0000 7f5aaffff640 1 -- 172.21.15.7:0/4164323155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe23acd0 msgr2=0x5613fe245260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.993+0000 7f5aaffff640 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe23acd0 0x5613fe245260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.993+0000 7f5aaffff640 1 -- 172.21.15.7:0/4164323155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613fe245a60 con 0x5613fe235b60 2024-03-26T08:49:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.994+0000 7f5aaffff640 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235b60 0x5613fe244d20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00059f0 tx=0x7f5aa001cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bfd902d9813f822 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.994+0000 7f5a9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.994+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/4164323155 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa002b070 con 0x5613fe235b60 2024-03-26T08:49:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.995+0000 7f5abfe09740 1 -- 172.21.15.7:0/4164323155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613fe199900 con 0x5613fe235b60 2024-03-26T08:49:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.995+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/4164323155 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5aa001c470 con 0x5613fe235b60 2024-03-26T08:49:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.995+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/4164323155 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aa0037050 con 0x5613fe235b60 2024-03-26T08:49:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.995+0000 7f5abfe09740 10 client.27092 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.995+0000 7f5abfe09740 1 -- 172.21.15.7:0/4164323155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5613fe228df0 con 0x5613fe235b60 2024-03-26T08:49:24.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.996+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/4164323155 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5613fe199900 con 0x5613fe235b60 2024-03-26T08:49:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.998+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/4164323155 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5aa0027460 con 0x5613fe235b60 2024-03-26T08:49:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.998+0000 7f5abfe09740 10 client.27092 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.998+0000 7f5abfe09740 10 client.27092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.998+0000 7f5abfe09740 20 client.27092 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.998+0000 7f5abfe09740 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5613fe229780 0x5613fe22bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.998+0000 7f5abfe09740 4 client.27092 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:49:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.998+0000 7f5abfe09740 1 -- 172.21.15.7:0/4164323155 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5613fe117b80 con 0x5613fe229780 2024-03-26T08:49:24.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:24.999+0000 7f5ab6d9d640 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5613fe229780 0x5613fe22bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.006+0000 7f5ab6d9d640 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5613fe229780 0x5613fe22bba0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c65d407f0f119bf server_cookie=8883936bca7af725 in_seq=0 out_seq=0 2024-03-26T08:49:25.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.007+0000 7f5a9e7fc640 10 client.27092 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:25.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5a9e7fc640 1 -- 172.21.15.7:0/4164323155 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5613fe117b80 con 0x5613fe229780 2024-03-26T08:49:25.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5a9e7fc640 10 client.27092 handle_command_reply: tid=0 2024-03-26T08:49:25.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 client.27092 shutdown 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 -- 172.21.15.7:0/4164323155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235b60 msgr2=0x5613fe244d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235b60 0x5613fe244d20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00059f0 tx=0x7f5aa001cce0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 -- 172.21.15.7:0/4164323155 shutdown_connections 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5613fe229780 0x5613fe22bba0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5613fe23acd0 0x5613fe245260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5613fe235b60 0x5613fe244d20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 --2- 172.21.15.7:0/4164323155 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5613fe235170 0x5613fe2466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 -- 172.21.15.7:0/4164323155 >> 172.21.15.7:0/4164323155 conn(0x5613fe1f51a0 msgr2=0x5613fe2345b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5a9e7fc640 0 client.27092 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 -- 172.21.15.7:0/4164323155 shutdown_connections 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.014+0000 7f5abfe09740 1 -- 172.21.15.7:0/4164323155 wait complete. 2024-03-26T08:49:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abfe09740 20 client.27092 trim_cache size 0 max 16384 2024-03-26T08:49:25.010 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:49:25.010 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:49:25.010 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "04568886-f201-4e87-be8c-e22fdf67fccc", 2024-03-26T08:49:25.010 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:49:25.010 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:49:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 -- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa40a2880 msgr2=0x7f5aa40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa40a2880 0x7f5aa40a4d40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8101630 tx=0x7f5aa8005f90 comp rx=0 tx=0).stop 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 -- 172.21.15.7:0/4073303399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab8102190 msgr2=0x7f5ab810ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab8102190 0x7f5ab810ac20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0037040 tx=0x7f5ab0032ea0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 -- 172.21.15.7:0/4073303399 shutdown_connections 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa40a2880 0x7f5aa40a4d40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab8102190 0x7f5ab810ac20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab81017d0 0x7f5ab810a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 --2- 172.21.15.7:0/4073303399 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab81005e0 0x7f5ab810c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 -- 172.21.15.7:0/4073303399 >> 172.21.15.7:0/4073303399 conn(0x7f5ab80fbb70 msgr2=0x7f5ab8104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.015+0000 7f5abd13b640 1 -- 172.21.15.7:0/4073303399 shutdown_connections 2024-03-26T08:49:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.016+0000 7f5abd13b640 1 -- 172.21.15.7:0/4073303399 wait complete. 2024-03-26T08:49:25.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:49:25.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:49:25.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.168+0000 7f26e63ff640 1 -- 172.21.15.7:0/225583939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e01017b0 msgr2=0x7f26e0101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.168+0000 7f26e63ff640 1 --2- 172.21.15.7:0/225583939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e01017b0 0x7f26e0101c30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009900 tx=0x7f26d402bc30 comp rx=0 tx=0).stop 2024-03-26T08:49:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.176+0000 7f26e63ff640 1 -- 172.21.15.7:0/225583939 shutdown_connections 2024-03-26T08:49:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.176+0000 7f26e63ff640 1 --2- 172.21.15.7:0/225583939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e0102170 0x7f26e0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.176+0000 7f26e63ff640 1 --2- 172.21.15.7:0/225583939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e01017b0 0x7f26e0101c30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.176+0000 7f26e63ff640 1 --2- 172.21.15.7:0/225583939 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e01005c0 0x7f26e01009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.176+0000 7f26e63ff640 1 -- 172.21.15.7:0/225583939 >> 172.21.15.7:0/225583939 conn(0x7f26e00fbb30 msgr2=0x7f26e00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:25.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.184+0000 7f26e63ff640 1 -- 172.21.15.7:0/225583939 shutdown_connections 2024-03-26T08:49:25.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.188+0000 7f26e63ff640 1 -- 172.21.15.7:0/225583939 wait complete. 2024-03-26T08:49:25.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.188+0000 7f26e63ff640 1 Processor -- start 2024-03-26T08:49:25.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.188+0000 7f26e63ff640 1 -- start start 2024-03-26T08:49:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.189+0000 7f26e63ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e01005c0 0x7f26e006fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.189+0000 7f26e63ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e01017b0 0x7f26e00703b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.189+0000 7f26e63ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e0102170 0x7f26e01a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.189+0000 7f26e63ff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e010d3a0 con 0x7f26e01017b0 2024-03-26T08:49:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.189+0000 7f26e63ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e010d220 con 0x7f26e0102170 2024-03-26T08:49:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.189+0000 7f26e63ff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e010d520 con 0x7f26e01005c0 2024-03-26T08:49:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.190+0000 7f26e499c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e0102170 0x7f26e01a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.191+0000 7f26e499c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e0102170 0x7f26e01a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35254/0 (socket says 172.21.15.7:35254) 2024-03-26T08:49:25.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.191+0000 7f26e499c640 1 -- 172.21.15.7:0/760352024 learned_addr learned my addr 172.21.15.7:0/760352024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:25.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.193+0000 7f26df7fe640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e01017b0 0x7f26e00703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.193+0000 7f26dffff640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e01005c0 0x7f26e006fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.194+0000 7f26e499c640 1 -- 172.21.15.7:0/760352024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e01005c0 msgr2=0x7f26e006fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.194+0000 7f26e499c640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e01005c0 0x7f26e006fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.194+0000 7f26e499c640 1 -- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e01017b0 msgr2=0x7f26e00703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.194+0000 7f26e499c640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e01017b0 0x7f26e00703b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.194+0000 7f26e499c640 1 -- 172.21.15.7:0/760352024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e0073530 con 0x7f26e0102170 2024-03-26T08:49:25.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.195+0000 7f26e499c640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e0102170 0x7f26e01a7a70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26d800eb10 tx=0x7f26d800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.196+0000 7f26dd7fa640 1 -- 172.21.15.7:0/760352024 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8004b20 con 0x7f26e0102170 2024-03-26T08:49:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.196+0000 7f26dd7fa640 1 -- 172.21.15.7:0/760352024 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d8004cc0 con 0x7f26e0102170 2024-03-26T08:49:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.196+0000 7f26dd7fa640 1 -- 172.21.15.7:0/760352024 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8010790 con 0x7f26e0102170 2024-03-26T08:49:25.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.196+0000 7f26e63ff640 1 -- 172.21.15.7:0/760352024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e0109240 con 0x7f26e0102170 2024-03-26T08:49:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.196+0000 7f26e63ff640 1 -- 172.21.15.7:0/760352024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e0109710 con 0x7f26e0102170 2024-03-26T08:49:25.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.205+0000 7f26dd7fa640 1 -- 172.21.15.7:0/760352024 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f26d8005000 con 0x7f26e0102170 2024-03-26T08:49:25.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.206+0000 7f26dd7fa640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26c80a6de0 0x7f26c80a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.206+0000 7f26dd7fa640 1 -- 172.21.15.7:0/760352024 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f26d80cf100 con 0x7f26e0102170 2024-03-26T08:49:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.206+0000 7f26e8fed740 1 Processor -- start 2024-03-26T08:49:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.206+0000 7f26e8fed740 1 -- start start 2024-03-26T08:49:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.206+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e4601641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.206+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e460169290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.207+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 0x55e46016bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.207+0000 7f26e8fed740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4600c93d0 con 0x55e460169900 2024-03-26T08:49:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.207+0000 7f26e8fed740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e460071ad0 con 0x55e460163da0 2024-03-26T08:49:25.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.207+0000 7f26e8fed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4600c5470 con 0x55e460164790 2024-03-26T08:49:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.207+0000 7f26dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e4601641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.207+0000 7f26dffff640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26c80a6de0 0x7f26c80a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.207+0000 7f26df7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e460169290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.207+0000 7f26dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e4601641c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35258/0 (socket says 172.21.15.7:35258) 2024-03-26T08:49:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.207+0000 7f26dffff640 1 -- 172.21.15.7:0/3832199874 learned_addr learned my addr 172.21.15.7:0/3832199874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:25.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.208+0000 7f26e499c640 1 --2- 172.21.15.7:0/3832199874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 0x55e46016bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.208+0000 7f26dffff640 1 -- 172.21.15.7:0/3832199874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 msgr2=0x55e460169290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.208+0000 7f26dffff640 1 --2- 172.21.15.7:0/3832199874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e460169290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.208+0000 7f26dffff640 1 -- 172.21.15.7:0/3832199874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 msgr2=0x55e46016bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:25.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.208+0000 7f26dffff640 1 --2- 172.21.15.7:0/3832199874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 0x55e46016bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.208+0000 7f26dffff640 1 -- 172.21.15.7:0/3832199874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e45fea77d0 con 0x55e460163da0 2024-03-26T08:49:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.209+0000 7f26dffff640 1 --2- 172.21.15.7:0/3832199874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e4601641c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f26d0014b60 tx=0x7f26d00117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8971e5af03774a1d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.209+0000 7f26dffff640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26c80a6de0 0x7f26c80a92a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f26d0011cb0 tx=0x7f26d0008000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.209+0000 7f26c67fc640 1 -- 172.21.15.7:0/3832199874 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0014c50 con 0x55e460163da0 2024-03-26T08:49:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.209+0000 7f26c67fc640 1 -- 172.21.15.7:0/3832199874 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d0014df0 con 0x55e460163da0 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.210+0000 7f26c67fc640 1 -- 172.21.15.7:0/3832199874 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d000eea0 con 0x55e460163da0 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.210+0000 7f26e8fed740 1 -- 172.21.15.7:0/3832199874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 msgr2=0x55e4601641c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.210+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3832199874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e4601641c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f26d0014b60 tx=0x7f26d00117b0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.210+0000 7f26e8fed740 1 -- 172.21.15.7:0/3832199874 shutdown_connections 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.210+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3832199874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 0x55e46016bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.210+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3832199874 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e460169290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.210+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3832199874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e4601641c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.210+0000 7f26e8fed740 1 -- 172.21.15.7:0/3832199874 >> 172.21.15.7:0/3832199874 conn(0x55e46015f310 msgr2=0x55e46015f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.211+0000 7f26e8fed740 1 -- 172.21.15.7:0/3832199874 shutdown_connections 2024-03-26T08:49:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.211+0000 7f26e8fed740 1 -- 172.21.15.7:0/3832199874 wait complete. 2024-03-26T08:49:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.211+0000 7f26e8fed740 1 -- start start 2024-03-26T08:49:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.212+0000 7f26e8fed740 1 Processor -- start 2024-03-26T08:49:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.212+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e46016e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.212+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e46016e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.212+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 0x55e460175790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.212+0000 7f26df7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e46016e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.212+0000 7f26e499c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 0x55e460175790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.212+0000 7f26e499c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 0x55e460175790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34576/0 (socket says 172.21.15.7:34576) 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.212+0000 7f26e499c640 1 -- 172.21.15.7:0/3097255259 learned_addr learned my addr 172.21.15.7:0/3097255259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46016c8a0 con 0x55e460169900 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e460085060 con 0x55e460163da0 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e46016ca20 con 0x55e460164790 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26dffff640 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e46016e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26e8fed740 10 client.0 _ll_register_callbacks cb 0x55e4600676a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26dffff640 1 -- 172.21.15.7:0/3097255259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 msgr2=0x55e46016e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26dffff640 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e46016e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26dffff640 1 -- 172.21.15.7:0/3097255259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 msgr2=0x55e460175790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26dffff640 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 0x55e460175790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26dffff640 1 -- 172.21.15.7:0/3097255259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e45fddd3f0 con 0x55e460163da0 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26dffff640 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e46016e210 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26d0000c00 tx=0x7f26d0005340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4dcbaf19b7a549b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3097255259 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0014c50 con 0x55e460163da0 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26d0014df0 con 0x55e460163da0 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.213+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3097255259 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f26d0004050 con 0x55e460163da0 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3097255259 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26d0004290 con 0x55e460163da0 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26e8fed740 10 client.27104 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e45fd9d0f0 con 0x55e460163da0 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3097255259 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d0014df0 con 0x55e460163da0 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3097255259 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f26d0004430 con 0x55e460163da0 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26e8fed740 10 client.27104 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26e8fed740 10 client.27104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26e8fed740 20 client.27104 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e460179b00 0x55e46017bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26e8fed740 4 client.27104 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.214+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4600d6910 con 0x55e460179b00 2024-03-26T08:49:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.215+0000 7f26df7fe640 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e460179b00 0x55e46017bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.215+0000 7f26df7fe640 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e460179b00 0x55e46017bf70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ab8f9aae164949 server_cookie=4a16201dbcd5b39e in_seq=0 out_seq=0 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.215+0000 7f26bf7fe640 10 client.27104 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.216+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3097255259 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e4600d6910 con 0x55e460179b00 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.216+0000 7f26bf7fe640 10 client.27104 handle_command_reply: tid=0 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.216+0000 7f26e8fed740 1 client.27104 shutdown 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.216+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 msgr2=0x55e46016e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e46016e210 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f26d0000c00 tx=0x7f26d0005340 comp rx=0 tx=0).stop 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 shutdown_connections 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e460179b00 0x55e46017bf70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460169900 0x55e460175790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e46016e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3097255259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460163da0 0x55e46016e210 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 >> 172.21.15.7:0/3097255259 conn(0x55e46015f310 msgr2=0x55e46015e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26bf7fe640 0 client.27104 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 shutdown_connections 2024-03-26T08:49:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 1 -- 172.21.15.7:0/3097255259 wait complete. 2024-03-26T08:49:25.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.217+0000 7f26e8fed740 20 client.27104 trim_cache size 0 max 16384 2024-03-26T08:49:25.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.238+0000 7f26e8fed740 1 Processor -- start 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26e8fed740 1 -- start start 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e46016d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e460175750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460169900 0x55e46016e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26e8fed740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0108b0 con 0x55e460163da0 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26e8fed740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc00e3b0 con 0x55e460169900 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26e8fed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc00c430 con 0x55e460164790 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26dffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e46016d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26dffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e46016d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34584/0 (socket says 172.21.15.7:34584) 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26dffff640 1 -- 172.21.15.7:0/1353215606 learned_addr learned my addr 172.21.15.7:0/1353215606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26df7fe640 1 --2- 172.21.15.7:0/1353215606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e460175750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26e499c640 1 --2- 172.21.15.7:0/1353215606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460169900 0x55e46016e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26dffff640 1 -- 172.21.15.7:0/1353215606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 msgr2=0x55e460175750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:25.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26dffff640 1 --2- 172.21.15.7:0/1353215606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e460175750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26dffff640 1 -- 172.21.15.7:0/1353215606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460169900 msgr2=0x55e46016e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:25.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26dffff640 1 --2- 172.21.15.7:0/1353215606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460169900 0x55e46016e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.239+0000 7f26dffff640 1 -- 172.21.15.7:0/1353215606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e460168e70 con 0x55e460163da0 2024-03-26T08:49:25.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.240+0000 7f26dffff640 1 --2- 172.21.15.7:0/1353215606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e46016d500 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009570 tx=0x7f26d0005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63813bd164f8ebb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:25.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.240+0000 7f26c5ffb640 1 -- 172.21.15.7:0/1353215606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0041070 con 0x55e460163da0 2024-03-26T08:49:25.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.240+0000 7f26c5ffb640 1 -- 172.21.15.7:0/1353215606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d0004ca0 con 0x55e460163da0 2024-03-26T08:49:25.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.240+0000 7f26c5ffb640 1 -- 172.21.15.7:0/1353215606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d000ec50 con 0x55e460163da0 2024-03-26T08:49:25.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.240+0000 7f26e8fed740 1 -- 172.21.15.7:0/1353215606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 msgr2=0x55e46016d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.240+0000 7f26e8fed740 1 --2- 172.21.15.7:0/1353215606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e46016d500 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009570 tx=0x7f26d0005990 comp rx=0 tx=0).stop 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.240+0000 7f26e8fed740 1 -- 172.21.15.7:0/1353215606 shutdown_connections 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 --2- 172.21.15.7:0/1353215606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460169900 0x55e46016e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 --2- 172.21.15.7:0/1353215606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460164790 0x55e460175750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 --2- 172.21.15.7:0/1353215606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e46016d500 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 -- 172.21.15.7:0/1353215606 >> 172.21.15.7:0/1353215606 conn(0x55e46015f310 msgr2=0x55e46016de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 -- 172.21.15.7:0/1353215606 shutdown_connections 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 -- 172.21.15.7:0/1353215606 wait complete. 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 -- start start 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 Processor -- start 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e460173310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460164790 0x55e460173850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460169900 0x55e460174860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc00ca30 con 0x55e460163da0 2024-03-26T08:49:25.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc00dd50 con 0x55e460164790 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc00d090 con 0x55e460169900 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e8fed740 10 client.0 _ll_register_callbacks cb 0x7f26d8004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26e499c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460169900 0x55e460174860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26e499c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460169900 0x55e460174860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35652/0 (socket says 172.21.15.7:35652) 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460164790 0x55e460173850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26e499c640 1 -- 172.21.15.7:0/3659658264 learned_addr learned my addr 172.21.15.7:0/3659658264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.241+0000 7f26dffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e460173310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26df7fe640 1 -- 172.21.15.7:0/3659658264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460169900 msgr2=0x55e460174860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26df7fe640 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460169900 0x55e460174860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26df7fe640 1 -- 172.21.15.7:0/3659658264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 msgr2=0x55e460173310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26df7fe640 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e460173310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26df7fe640 1 -- 172.21.15.7:0/3659658264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e460174e50 con 0x55e460164790 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26e499c640 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460169900 0x55e460174860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26df7fe640 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460164790 0x55e460173850 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f26d4036040 tx=0x7f26d402cd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87dd9201a7c54fd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3659658264 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4004450 con 0x55e460164790 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3659658264 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f26d4005040 con 0x55e460164790 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26e8fed740 1 -- 172.21.15.7:0/3659658264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26d0004ca0 con 0x55e460164790 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3659658264 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26d4005240 con 0x55e460164790 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26e8fed740 10 client.27110 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.242+0000 7f26e8fed740 1 -- 172.21.15.7:0/3659658264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e4601d5850 con 0x55e460164790 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.243+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3659658264 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26d0004ca0 con 0x55e460164790 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.243+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3659658264 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f26d4003790 con 0x55e460164790 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.243+0000 7f26e8fed740 10 client.27110 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.243+0000 7f26e8fed740 10 client.27110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.243+0000 7f26e8fed740 20 client.27110 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.243+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e4601d6110 0x55e4601d8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.243+0000 7f26e8fed740 4 client.27110 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.243+0000 7f26e8fed740 1 -- 172.21.15.7:0/3659658264 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e4600d0c60 con 0x55e4601d6110 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.243+0000 7f26dffff640 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e4601d6110 0x55e4601d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26bf7fe640 10 client.27110 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:25.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26dffff640 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e4601d6110 0x55e4601d8500 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a122c8456090832 server_cookie=a8d5cc73b56eee42 in_seq=0 out_seq=0 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26bf7fe640 1 -- 172.21.15.7:0/3659658264 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e4600d0c60 con 0x55e4601d6110 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26bf7fe640 10 client.27110 handle_command_reply: tid=0 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 client.27110 shutdown 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 -- 172.21.15.7:0/3659658264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460164790 msgr2=0x55e460173850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460164790 0x55e460173850 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f26d4036040 tx=0x7f26d402cd10 comp rx=0 tx=0).stop 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 -- 172.21.15.7:0/3659658264 shutdown_connections 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e4601d6110 0x55e4601d8500 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26bf7fe640 0 client.27110 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e460169900 0x55e460174860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e460164790 0x55e460173850 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 --2- 172.21.15.7:0/3659658264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e460163da0 0x55e460173310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 -- 172.21.15.7:0/3659658264 >> 172.21.15.7:0/3659658264 conn(0x55e46015f310 msgr2=0x55e460162570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 -- 172.21.15.7:0/3659658264 shutdown_connections 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 1 -- 172.21.15.7:0/3659658264 wait complete. 2024-03-26T08:49:25.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.244+0000 7f26e8fed740 20 client.27110 trim_cache size 0 max 16384 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "04568886-f201-4e87-be8c-e22fdf67fccc": { 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "04568886-f201-4e87-be8c-e22fdf67fccc", 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:49:25.243 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:49:25.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 -- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26c80a6de0 msgr2=0x7f26c80a92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26c80a6de0 0x7f26c80a92a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f26d0011cb0 tx=0x7f26d0008000 comp rx=0 tx=0).stop 2024-03-26T08:49:25.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 -- 172.21.15.7:0/760352024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e0102170 msgr2=0x7f26e01a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:25.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e0102170 0x7f26e01a7a70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26d800eb10 tx=0x7f26d800c2d0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 -- 172.21.15.7:0/760352024 shutdown_connections 2024-03-26T08:49:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26c80a6de0 0x7f26c80a92a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26e0102170 0x7f26e01a7a70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26e01017b0 0x7f26e00703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 --2- 172.21.15.7:0/760352024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f26e01005c0 0x7f26e006fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 -- 172.21.15.7:0/760352024 >> 172.21.15.7:0/760352024 conn(0x7f26e00fbb30 msgr2=0x7f26e01049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 -- 172.21.15.7:0/760352024 shutdown_connections 2024-03-26T08:49:25.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:25.245+0000 7f26e63ff640 1 -- 172.21.15.7:0/760352024 wait complete. 2024-03-26T08:49:25.253 INFO:tasks.cephfs.filesystem:scrub status for tag:04568886-f201-4e87-be8c-e22fdf67fccc - {'path': '/', 'tag': '04568886-f201-4e87-be8c-e22fdf67fccc', 'options': 'recursive,force'} 2024-03-26T08:49:26.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 415.0 s, 2042.2 tps, lat 48.512 ms stddev 52.139 2024-03-26T08:49:26.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:26 smithi016 ceph-mon[35851]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 MiB/s wr, 695 op/s 2024-03-26T08:49:26.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:26 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:49:26.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:26 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:26.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:26 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:49:26.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:26 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:49:26.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:26 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:26.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:26 smithi007 ceph-mon[32462]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 MiB/s wr, 695 op/s 2024-03-26T08:49:26.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:26 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:49:26.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:26 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:26.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:26 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:49:26.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:26 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:49:26.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:26 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:26.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:26 smithi044 ceph-mon[36461]: pgmap v570: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.2 MiB/s wr, 695 op/s 2024-03-26T08:49:26.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:26 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:49:26.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:26 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:26.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:26 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:49:26.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:26 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:49:26.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:26 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:27.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:27 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:27.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:27 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:27.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:27 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:28.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:28 smithi007 ceph-mon[32462]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 712 op/s 2024-03-26T08:49:28.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:28 smithi044 ceph-mon[36461]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 712 op/s 2024-03-26T08:49:28.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:28 smithi016 ceph-mon[35851]: pgmap v571: 97 pgs: 97 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 712 op/s 2024-03-26T08:49:30.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:30 smithi007 ceph-mon[32462]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.5 MiB/s wr, 712 op/s 2024-03-26T08:49:30.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:30 smithi044 ceph-mon[36461]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.5 MiB/s wr, 712 op/s 2024-03-26T08:49:30.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:30 smithi016 ceph-mon[35851]: pgmap v572: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.5 MiB/s wr, 712 op/s 2024-03-26T08:49:31.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 420.0 s, 2203.6 tps, lat 45.675 ms stddev 48.961 2024-03-26T08:49:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.218+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/4065680604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f4410b010 msgr2=0x7f7f4410d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:32.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.218+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/4065680604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f4410b010 0x7f7f4410d480 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f4001a910 tx=0x7f7f4003cca0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.219+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/4065680604 shutdown_connections 2024-03-26T08:49:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.219+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/4065680604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4410d9c0 0x7f7f4410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.219+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/4065680604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f4410b010 0x7f7f4410d480 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.219+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/4065680604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f441086a0 0x7f7f4410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.219+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/4065680604 >> 172.21.15.7:0/4065680604 conn(0x7f7f4406e6b0 msgr2=0x7f7f44070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.221+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/4065680604 shutdown_connections 2024-03-26T08:49:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.222+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/4065680604 wait complete. 2024-03-26T08:49:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.223+0000 7f7f4b4c0640 1 Processor -- start 2024-03-26T08:49:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.223+0000 7f7f4b4c0640 1 -- start start 2024-03-26T08:49:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.224+0000 7f7f4b4c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f441086a0 0x7f7f4419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.224+0000 7f7f4b4c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f4410b010 0x7f7f4419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.224+0000 7f7f4b4c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f4410d9c0 0x7f7f441a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.224+0000 7f7f4b4c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44111a60 con 0x7f7f441086a0 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f4b4c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441118e0 con 0x7f7f4410d9c0 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f49a5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f4410d9c0 0x7f7f441a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f49a5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f4410d9c0 0x7f7f441a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50832/0 (socket says 172.21.15.7:50832) 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f4410b010 0x7f7f4419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f4410b010 0x7f7f4419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39452/0 (socket says 172.21.15.7:39452) 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f48a5b640 1 -- 172.21.15.7:0/1417883450 learned_addr learned my addr 172.21.15.7:0/1417883450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f4925c640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f441086a0 0x7f7f4419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/1417883450 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f44111be0 con 0x7f7f4410b010 2024-03-26T08:49:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f4925c640 1 -- 172.21.15.7:0/1417883450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f4410b010 msgr2=0x7f7f4419e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f4925c640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f4410b010 0x7f7f4419e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f4925c640 1 -- 172.21.15.7:0/1417883450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f4410d9c0 msgr2=0x7f7f441a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f4925c640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f4410d9c0 0x7f7f441a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.225+0000 7f7f4925c640 1 -- 172.21.15.7:0/1417883450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f441a5d40 con 0x7f7f441086a0 2024-03-26T08:49:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.226+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f4410b010 0x7f7f4419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.226+0000 7f7f4925c640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f441086a0 0x7f7f4419e120 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009fd0 tx=0x7f7f3400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.226+0000 7f7f3a7fc640 1 -- 172.21.15.7:0/1417883450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f34015020 con 0x7f7f441086a0 2024-03-26T08:49:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.226+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/1417883450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f441a5fd0 con 0x7f7f441086a0 2024-03-26T08:49:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.226+0000 7f7f3a7fc640 1 -- 172.21.15.7:0/1417883450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f34011d10 con 0x7f7f441086a0 2024-03-26T08:49:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.226+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/1417883450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f441a6560 con 0x7f7f441086a0 2024-03-26T08:49:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.226+0000 7f7f3a7fc640 1 -- 172.21.15.7:0/1417883450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400f2e0 con 0x7f7f441086a0 2024-03-26T08:49:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.230+0000 7f7f4e186740 1 Processor -- start 2024-03-26T08:49:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.230+0000 7f7f4e186740 1 -- start start 2024-03-26T08:49:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.230+0000 7f7f3a7fc640 1 -- 172.21.15.7:0/1417883450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7f34018020 con 0x7f7f441086a0 2024-03-26T08:49:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.230+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1043d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.230+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f1048e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.230+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1049480 0x5638f104b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.230+0000 7f7f4e186740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f0fa8b10 con 0x5638f1049480 2024-03-26T08:49:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.230+0000 7f7f4e186740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f0f51900 con 0x5638f1044310 2024-03-26T08:49:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.230+0000 7f7f4e186740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f0fa4bb0 con 0x5638f1043920 2024-03-26T08:49:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1049480 0x5638f104b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f4925c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f1048e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f49a5d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1043d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f49a5d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1043d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:39458/0 (socket says 172.21.15.7:39458) 2024-03-26T08:49:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1049480 0x5638f104b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49646/0 (socket says 172.21.15.7:49646) 2024-03-26T08:49:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f4925c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f1048e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50838/0 (socket says 172.21.15.7:50838) 2024-03-26T08:49:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f49a5d640 1 -- 172.21.15.7:0/1660399869 learned_addr learned my addr 172.21.15.7:0/1660399869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f48a5b640 1 -- 172.21.15.7:0/1660399869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 msgr2=0x5638f1043d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/1660399869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1043d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f48a5b640 1 -- 172.21.15.7:0/1660399869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 msgr2=0x5638f1048e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/1660399869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f1048e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f48a5b640 1 -- 172.21.15.7:0/1660399869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638f0cbd210 con 0x5638f1049480 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f3a7fc640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f300a2830 0x7f7f300a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f49a5d640 1 --2- 172.21.15.7:0/1660399869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1043d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f3a7fc640 1 -- 172.21.15.7:0/1417883450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f340ce2e0 con 0x7f7f441086a0 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f49a5d640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f300a2830 0x7f7f300a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.231+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/1660399869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1049480 0x5638f104b910 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7f40006010 tx=0x7f7f4001a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6039d46768951c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f2f7fe640 1 -- 172.21.15.7:0/1660399869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40004200 con 0x5638f1049480 2024-03-26T08:49:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f2f7fe640 1 -- 172.21.15.7:0/1660399869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f4003dd30 con 0x5638f1049480 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f2f7fe640 1 -- 172.21.15.7:0/1660399869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f400054e0 con 0x5638f1049480 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f4e186740 1 -- 172.21.15.7:0/1660399869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1049480 msgr2=0x5638f104b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f4e186740 1 --2- 172.21.15.7:0/1660399869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1049480 0x5638f104b910 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7f40006010 tx=0x7f7f4001a620 comp rx=0 tx=0).stop 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f49a5d640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f300a2830 0x7f7f300a4cf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x5638f1044b70 tx=0x7f7f3c00e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f4e186740 1 -- 172.21.15.7:0/1660399869 shutdown_connections 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f4e186740 1 --2- 172.21.15.7:0/1660399869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1049480 0x5638f104b910 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f4e186740 1 --2- 172.21.15.7:0/1660399869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f1048e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f4e186740 1 --2- 172.21.15.7:0/1660399869 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1043d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f4e186740 1 -- 172.21.15.7:0/1660399869 >> 172.21.15.7:0/1660399869 conn(0x5638f103ee90 msgr2=0x5638f103f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f4e186740 1 -- 172.21.15.7:0/1660399869 shutdown_connections 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.232+0000 7f7f4e186740 1 -- 172.21.15.7:0/1660399869 wait complete. 2024-03-26T08:49:32.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.233+0000 7f7f4e186740 1 -- start start 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.233+0000 7f7f4e186740 1 Processor -- start 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.233+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f104dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.233+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f104e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.234+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 0x5638f1055120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.234+0000 7f7f4e186740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f104c3d0 con 0x5638f1044310 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.234+0000 7f7f4e186740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f0f64d50 con 0x5638f1049480 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.234+0000 7f7f4e186740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f104c550 con 0x5638f1043920 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.234+0000 7f7f4e186740 10 client.0 _ll_register_callbacks cb 0x5638f0f64720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.234+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f104e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.234+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f104e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49648/0 (socket says 172.21.15.7:49648) 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.234+0000 7f7f48a5b640 1 -- 172.21.15.7:0/928320126 learned_addr learned my addr 172.21.15.7:0/928320126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.234+0000 7f7f49a5d640 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 0x5638f1055120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.235+0000 7f7f48a5b640 1 -- 172.21.15.7:0/928320126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 msgr2=0x5638f104dba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:49:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.235+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f104dba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.235+0000 7f7f48a5b640 1 -- 172.21.15.7:0/928320126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 msgr2=0x5638f1055120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.235+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 0x5638f1055120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.235+0000 7f7f48a5b640 1 -- 172.21.15.7:0/928320126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638f0d89620 con 0x5638f1044310 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.235+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f104e0e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7f4003dab0 tx=0x7f7f4003dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ddce24b29c575 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.235+0000 7f7f2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.235+0000 7f7f2cff9640 1 -- 172.21.15.7:0/928320126 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4005f020 con 0x5638f1044310 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.236+0000 7f7f2cff9640 1 -- 172.21.15.7:0/928320126 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f400054d0 con 0x5638f1044310 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.236+0000 7f7f4e186740 1 -- 172.21.15.7:0/928320126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f4003dd30 con 0x5638f1044310 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.236+0000 7f7f2cff9640 1 -- 172.21.15.7:0/928320126 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f400035f0 con 0x5638f1044310 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.236+0000 7f7f4e186740 10 client.17277 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.236+0000 7f7f4e186740 1 -- 172.21.15.7:0/928320126 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5638f1058490 con 0x5638f1044310 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.236+0000 7f7f2cff9640 1 -- 172.21.15.7:0/928320126 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f4003dd30 con 0x5638f1044310 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f2cff9640 1 -- 172.21.15.7:0/928320126 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7f40045840 con 0x5638f1044310 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f4e186740 10 client.17277 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f4e186740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f4e186740 20 client.17277 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f4e186740 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5638f10597e0 0x5638f105bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f4e186740 4 client.17277 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f4e186740 1 -- 172.21.15.7:0/928320126 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638f0fb6050 con 0x5638f10597e0 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f4925c640 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5638f10597e0 0x5638f105bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f4925c640 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5638f10597e0 0x5638f105bbd0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1e85899328fae9 server_cookie=bb99ba4864c5c014 in_seq=0 out_seq=0 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.237+0000 7f7f2cff9640 10 client.17277 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f2cff9640 1 -- 172.21.15.7:0/928320126 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5638f0fb6050 con 0x5638f10597e0 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f2cff9640 10 client.17277 handle_command_reply: tid=0 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 client.17277 shutdown 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 -- 172.21.15.7:0/928320126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 msgr2=0x5638f104e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f104e0e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7f4003dab0 tx=0x7f7f4003dae0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 -- 172.21.15.7:0/928320126 shutdown_connections 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5638f10597e0 0x5638f105bbd0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 0x5638f1055120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f104e0e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f2cff9640 0 client.17277 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 --2- 172.21.15.7:0/928320126 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f104dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 -- 172.21.15.7:0/928320126 >> 172.21.15.7:0/928320126 conn(0x5638f103ee90 msgr2=0x5638f103e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 -- 172.21.15.7:0/928320126 shutdown_connections 2024-03-26T08:49:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 1 -- 172.21.15.7:0/928320126 wait complete. 2024-03-26T08:49:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.238+0000 7f7f4e186740 20 client.17277 trim_cache size 0 max 16384 2024-03-26T08:49:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.256+0000 7f7f4e186740 1 Processor -- start 2024-03-26T08:49:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.256+0000 7f7f4e186740 1 -- start start 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.256+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1043920 0x5638f104b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.256+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f10550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.256+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1049480 0x5638f10455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.256+0000 7f7f4e186740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441b1710 con 0x5638f1043920 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.256+0000 7f7f4e186740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441af210 con 0x5638f1044310 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.256+0000 7f7f4e186740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441ad290 con 0x5638f1049480 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.257+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f10550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.257+0000 7f7f4925c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1043920 0x5638f104b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.257+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f10550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50866/0 (socket says 172.21.15.7:50866) 2024-03-26T08:49:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.257+0000 7f7f48a5b640 1 -- 172.21.15.7:0/2374936407 learned_addr learned my addr 172.21.15.7:0/2374936407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.257+0000 7f7f48a5b640 1 -- 172.21.15.7:0/2374936407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1049480 msgr2=0x5638f10455d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:49:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.257+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/2374936407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1049480 0x5638f10455d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.257+0000 7f7f48a5b640 1 -- 172.21.15.7:0/2374936407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1043920 msgr2=0x5638f104b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.257+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/2374936407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1043920 0x5638f104b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.257+0000 7f7f48a5b640 1 -- 172.21.15.7:0/2374936407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638f1044ba0 con 0x5638f1044310 2024-03-26T08:49:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.258+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/2374936407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f10550e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f40005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb136220fed5979 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.258+0000 7f7f2effd640 1 -- 172.21.15.7:0/2374936407 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4005f070 con 0x5638f1044310 2024-03-26T08:49:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.258+0000 7f7f2effd640 1 -- 172.21.15.7:0/2374936407 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f400459c0 con 0x5638f1044310 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.258+0000 7f7f4e186740 1 -- 172.21.15.7:0/2374936407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 msgr2=0x5638f10550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.258+0000 7f7f4e186740 1 --2- 172.21.15.7:0/2374936407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f10550e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7f40000c00 tx=0x7f7f40005e70 comp rx=0 tx=0).stop 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 -- 172.21.15.7:0/2374936407 shutdown_connections 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 --2- 172.21.15.7:0/2374936407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1049480 0x5638f10455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 --2- 172.21.15.7:0/2374936407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1044310 0x5638f10550e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 --2- 172.21.15.7:0/2374936407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1043920 0x5638f104b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 -- 172.21.15.7:0/2374936407 >> 172.21.15.7:0/2374936407 conn(0x5638f103ee90 msgr2=0x5638f103f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 -- 172.21.15.7:0/2374936407 shutdown_connections 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 -- 172.21.15.7:0/2374936407 wait complete. 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 -- start start 2024-03-26T08:49:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 Processor -- start 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1052be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f1053120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 0x5638f10540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441ad890 con 0x5638f1044310 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441aebb0 con 0x5638f1049480 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441adef0 con 0x5638f1043920 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.259+0000 7f7f4e186740 10 client.0 _ll_register_callbacks cb 0x7f7f4001a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f4925c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1052be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f1053120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f48a5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f1053120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49660/0 (socket says 172.21.15.7:49660) 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f49a5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 0x5638f10540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f48a5b640 1 -- 172.21.15.7:0/3157483074 learned_addr learned my addr 172.21.15.7:0/3157483074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f49a5d640 1 -- 172.21.15.7:0/3157483074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 msgr2=0x5638f1052be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f49a5d640 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1052be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f49a5d640 1 -- 172.21.15.7:0/3157483074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 msgr2=0x5638f1053120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f49a5d640 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f1053120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f49a5d640 1 -- 172.21.15.7:0/3157483074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638f10546b0 con 0x5638f1049480 2024-03-26T08:49:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f48a5b640 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f1053120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f49a5d640 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 0x5638f10540c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c007a40 tx=0x7f7f3c004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a85e80d50ad160 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f2cff9640 1 -- 172.21.15.7:0/3157483074 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c018960 con 0x5638f1049480 2024-03-26T08:49:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f2cff9640 1 -- 172.21.15.7:0/3157483074 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f3c018b00 con 0x5638f1049480 2024-03-26T08:49:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.260+0000 7f7f4e186740 1 -- 172.21.15.7:0/3157483074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f400459c0 con 0x5638f1049480 2024-03-26T08:49:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f2cff9640 1 -- 172.21.15.7:0/3157483074 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f3c018d00 con 0x5638f1049480 2024-03-26T08:49:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f4e186740 10 client.27128 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f4e186740 1 -- 172.21.15.7:0/3157483074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5638f10b53e0 con 0x5638f1049480 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f2cff9640 1 -- 172.21.15.7:0/3157483074 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f400459c0 con 0x5638f1049480 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f2cff9640 1 -- 172.21.15.7:0/3157483074 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7f3c023510 con 0x5638f1049480 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f4e186740 10 client.27128 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f4e186740 10 client.27128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f4e186740 20 client.27128 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f4e186740 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5638f10b5d50 0x5638f10b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f4e186740 4 client.27128 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.261+0000 7f7f4e186740 1 -- 172.21.15.7:0/3157483074 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5638f0fb03a0 con 0x5638f10b5d50 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.262+0000 7f7f4925c640 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5638f10b5d50 0x5638f10b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.263+0000 7f7f4925c640 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5638f10b5d50 0x5638f10b8170 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dfa721792c90c68 server_cookie=38ea4fae5a82119f in_seq=0 out_seq=0 2024-03-26T08:49:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.263+0000 7f7f2cff9640 10 client.27128 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.284+0000 7f7f2cff9640 1 -- 172.21.15.7:0/3157483074 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5638f0fb03a0 con 0x5638f10b5d50 2024-03-26T08:49:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.284+0000 7f7f2cff9640 10 client.27128 handle_command_reply: tid=0 2024-03-26T08:49:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.284+0000 7f7f4e186740 1 client.27128 shutdown 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 -- 172.21.15.7:0/3157483074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 msgr2=0x5638f10540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 0x5638f10540c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c007a40 tx=0x7f7f3c004b50 comp rx=0 tx=0).stop 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 -- 172.21.15.7:0/3157483074 shutdown_connections 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5638f10b5d50 0x5638f10b8170 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5638f1049480 0x5638f10540c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5638f1044310 0x5638f1053120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f2cff9640 0 client.27128 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 --2- 172.21.15.7:0/3157483074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5638f1043920 0x5638f1052be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 -- 172.21.15.7:0/3157483074 >> 172.21.15.7:0/3157483074 conn(0x5638f103ee90 msgr2=0x5638f104db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 -- 172.21.15.7:0/3157483074 shutdown_connections 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 1 -- 172.21.15.7:0/3157483074 wait complete. 2024-03-26T08:49:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.285+0000 7f7f4e186740 20 client.27128 trim_cache size 0 max 16384 2024-03-26T08:49:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:49:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:49:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:49:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f300a2830 msgr2=0x7f7f300a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f300a2830 0x7f7f300a4cf0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x5638f1044b70 tx=0x7f7f3c00e040 comp rx=0 tx=0).stop 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f441086a0 msgr2=0x7f7f4419e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f441086a0 0x7f7f4419e120 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009fd0 tx=0x7f7f3400b870 comp rx=0 tx=0).stop 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/1417883450 shutdown_connections 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f300a2830 0x7f7f300a4cf0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f4410d9c0 0x7f7f441a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f4410b010 0x7f7f4419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 --2- 172.21.15.7:0/1417883450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f441086a0 0x7f7f4419e120 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/1417883450 >> 172.21.15.7:0/1417883450 conn(0x7f7f4406e6b0 msgr2=0x7f7f4410b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/1417883450 shutdown_connections 2024-03-26T08:49:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:32.286+0000 7f7f4b4c0640 1 -- 172.21.15.7:0/1417883450 wait complete. 2024-03-26T08:49:32.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:32 smithi007 ceph-mon[32462]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.7 MiB/s wr, 747 op/s 2024-03-26T08:49:32.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:32 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:49:32.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:32 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:49:32.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:32 smithi044 ceph-mon[36461]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.7 MiB/s wr, 747 op/s 2024-03-26T08:49:32.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:32 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:49:32.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:32 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:49:32.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:32 smithi016 ceph-mon[35851]: pgmap v573: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 6.7 MiB/s wr, 747 op/s 2024-03-26T08:49:32.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:32 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:49:32.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:32 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:49:34.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:34 smithi007 ceph-mon[32462]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.7 MiB/s wr, 767 op/s 2024-03-26T08:49:34.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:34 smithi044 ceph-mon[36461]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.7 MiB/s wr, 767 op/s 2024-03-26T08:49:34.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:34 smithi016 ceph-mon[35851]: pgmap v574: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.7 MiB/s wr, 767 op/s 2024-03-26T08:49:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 425.0 s, 2034.8 tps, lat 49.047 ms stddev 54.151 2024-03-26T08:49:36.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:36 smithi007 ceph-mon[32462]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.3 MiB/s wr, 726 op/s 2024-03-26T08:49:36.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:36 smithi044 ceph-mon[36461]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.3 MiB/s wr, 726 op/s 2024-03-26T08:49:36.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:36 smithi016 ceph-mon[35851]: pgmap v575: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.3 MiB/s wr, 726 op/s 2024-03-26T08:49:38.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:38 smithi007 ceph-mon[32462]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.2 MiB/s wr, 759 op/s 2024-03-26T08:49:38.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:38 smithi044 ceph-mon[36461]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.2 MiB/s wr, 759 op/s 2024-03-26T08:49:38.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:38 smithi016 ceph-mon[35851]: pgmap v576: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.2 MiB/s wr, 759 op/s 2024-03-26T08:49:40.294 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:40 smithi007 ceph-mon[32462]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.4 MiB/s wr, 650 op/s 2024-03-26T08:49:40.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:40 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:49:40.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:40 smithi044 ceph-mon[36461]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.4 MiB/s wr, 650 op/s 2024-03-26T08:49:40.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:40 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:49:40.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:40 smithi016 ceph-mon[35851]: pgmap v577: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.4 MiB/s wr, 650 op/s 2024-03-26T08:49:40.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:40 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:49:41.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 430.0 s, 2703.4 tps, lat 36.914 ms stddev 45.795 2024-03-26T08:49:41.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:49:41.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:49:41.296 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:41 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:49:41.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:49:41.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:41 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:49:41.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T08:49:41.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:49:41.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:49:41.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:41 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:49:42.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:42 smithi007 ceph-mon[32462]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-03-26T08:49:42.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:42 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi016 to 175.2M 2024-03-26T08:49:42.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:42 smithi007 ceph-mon[32462]: Unable to set osd_memory_target on smithi016 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-03-26T08:49:42.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:42 smithi007 ceph-mon[32462]: Adjusting osd_memory_target on smithi044 to 175.2M 2024-03-26T08:49:42.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:42 smithi007 ceph-mon[32462]: Unable to set osd_memory_target on smithi044 to 183743795: error parsing value: Value '183743795' is below minimum 939524096 2024-03-26T08:49:42.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:42 smithi044 ceph-mon[36461]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-03-26T08:49:42.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:42 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi016 to 175.2M 2024-03-26T08:49:42.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:42 smithi044 ceph-mon[36461]: Unable to set osd_memory_target on smithi016 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-03-26T08:49:42.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:42 smithi044 ceph-mon[36461]: Adjusting osd_memory_target on smithi044 to 175.2M 2024-03-26T08:49:42.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:42 smithi044 ceph-mon[36461]: Unable to set osd_memory_target on smithi044 to 183743795: error parsing value: Value '183743795' is below minimum 939524096 2024-03-26T08:49:42.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:42 smithi016 ceph-mon[35851]: pgmap v578: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.3 MiB/s wr, 665 op/s 2024-03-26T08:49:42.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:42 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi016 to 175.2M 2024-03-26T08:49:42.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:42 smithi016 ceph-mon[35851]: Unable to set osd_memory_target on smithi016 to 183745228: error parsing value: Value '183745228' is below minimum 939524096 2024-03-26T08:49:42.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:42 smithi016 ceph-mon[35851]: Adjusting osd_memory_target on smithi044 to 175.2M 2024-03-26T08:49:42.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:42 smithi016 ceph-mon[35851]: Unable to set osd_memory_target on smithi044 to 183743795: error parsing value: Value '183743795' is below minimum 939524096 2024-03-26T08:49:44.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:44 smithi007 ceph-mon[32462]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 698 op/s 2024-03-26T08:49:44.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:44 smithi044 ceph-mon[36461]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 698 op/s 2024-03-26T08:49:44.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:44 smithi016 ceph-mon[35851]: pgmap v579: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 MiB/s wr, 698 op/s 2024-03-26T08:49:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.414+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3510369257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01005d0 msgr2=0x7f7ab01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.414+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3510369257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01005d0 0x7f7ab01009f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009ca0 tx=0x7f7aa802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.414+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3510369257 shutdown_connections 2024-03-26T08:49:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.414+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3510369257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab0102180 0x7f7ab0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.414+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3510369257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab01017c0 0x7f7ab0101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.414+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3510369257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01005d0 0x7f7ab01009f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.414+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3510369257 >> 172.21.15.7:0/3510369257 conn(0x7f7ab00fbb40 msgr2=0x7f7ab00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.414+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3510369257 shutdown_connections 2024-03-26T08:49:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.414+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3510369257 wait complete. 2024-03-26T08:49:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.415+0000 7f7ab6f52640 1 Processor -- start 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.415+0000 7f7ab6f52640 1 -- start start 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7ab6f52640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab01005d0 0x7f7ab00700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7ab6f52640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01017c0 0x7f7ab0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7ab6f52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab0102180 0x7f7ab01a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7ab6f52640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab010c240 con 0x7f7ab01017c0 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7ab6f52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab010c0c0 con 0x7f7ab0102180 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7ab6f52640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab010c3c0 con 0x7f7ab01005d0 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7aaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01017c0 0x7f7ab0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7aaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01017c0 0x7f7ab0070600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47934/0 (socket says 172.21.15.7:47934) 2024-03-26T08:49:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7aaffff640 1 -- 172.21.15.7:0/3781775767 learned_addr learned my addr 172.21.15.7:0/3781775767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.416+0000 7f7ab54ef640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab0102180 0x7f7ab01a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab01005d0 0x7f7ab00700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7ab54ef640 1 -- 172.21.15.7:0/3781775767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab01005d0 msgr2=0x7f7ab00700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7ab54ef640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab01005d0 0x7f7ab00700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7ab54ef640 1 -- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01017c0 msgr2=0x7f7ab0070600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7ab54ef640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01017c0 0x7f7ab0070600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7ab54ef640 1 -- 172.21.15.7:0/3781775767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab01a6e90 con 0x7f7ab0102180 2024-03-26T08:49:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7aaffff640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01017c0 0x7f7ab0070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7ab54ef640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab0102180 0x7f7ab01a6950 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009e40 tx=0x7f7a9800ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7aadffb640 1 -- 172.21.15.7:0/3781775767 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9800ccc0 con 0x7f7ab0102180 2024-03-26T08:49:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3781775767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab0069cf0 con 0x7f7ab0102180 2024-03-26T08:49:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7aadffb640 1 -- 172.21.15.7:0/3781775767 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7a98005020 con 0x7f7ab0102180 2024-03-26T08:49:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7aadffb640 1 -- 172.21.15.7:0/3781775767 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98004a00 con 0x7f7ab0102180 2024-03-26T08:49:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.417+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3781775767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab006a240 con 0x7f7ab0102180 2024-03-26T08:49:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.419+0000 7f7aadffb640 1 -- 172.21.15.7:0/3781775767 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7a98020020 con 0x7f7ab0102180 2024-03-26T08:49:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab9c0e740 1 Processor -- start 2024-03-26T08:49:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7aadffb640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a9c0a2870 0x7f7a9c0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab9c0e740 1 -- start start 2024-03-26T08:49:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234416cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234419d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234427260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab9c0e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23437bb10 con 0x55d23441a3f0 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab9c0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234324900 con 0x55d2344168a0 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab9c0e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234377bb0 con 0x55d234417290 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7aaffff640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a9c0a2870 0x7f7a9c0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7aadffb640 1 -- 172.21.15.7:0/3781775767 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7a980d0920 con 0x7f7ab0102180 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7aaffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234419d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab4cee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234416cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab4cee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234416cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57164/0 (socket says 172.21.15.7:57164) 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7aaffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234419d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60204/0 (socket says 172.21.15.7:60204) 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab4cee640 1 -- 172.21.15.7:0/2292404023 learned_addr learned my addr 172.21.15.7:0/2292404023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.420+0000 7f7ab54ef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234427260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7ab4cee640 1 -- 172.21.15.7:0/2292404023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 msgr2=0x55d234419d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/2292404023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234419d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7ab4cee640 1 -- 172.21.15.7:0/2292404023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 msgr2=0x55d234427260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/2292404023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234427260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7ab4cee640 1 -- 172.21.15.7:0/2292404023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d234090210 con 0x55d2344168a0 2024-03-26T08:49:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7aaffff640 1 --2- 172.21.15.7:0/2292404023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234419d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/2292404023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234416cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8000f80 tx=0x7f7aa80093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2455d257eb723c5c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7aaffff640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a9c0a2870 0x7f7a9c0a4d30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009810 tx=0x7f7aa0009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7aa6ffd640 1 -- 172.21.15.7:0/2292404023 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8043020 con 0x55d2344168a0 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.421+0000 7f7aa6ffd640 1 -- 172.21.15.7:0/2292404023 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aa8032c90 con 0x55d2344168a0 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.422+0000 7f7aa6ffd640 1 -- 172.21.15.7:0/2292404023 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8039040 con 0x55d2344168a0 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.422+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/2292404023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 msgr2=0x55d234416cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.422+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/2292404023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234416cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8000f80 tx=0x7f7aa80093f0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.422+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/2292404023 shutdown_connections 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.422+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/2292404023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234427260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.422+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/2292404023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234419d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.422+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/2292404023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234416cc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.422+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/2292404023 >> 172.21.15.7:0/2292404023 conn(0x55d234411e90 msgr2=0x55d234412290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/2292404023 shutdown_connections 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/2292404023 wait complete. 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 -- start start 2024-03-26T08:49:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 Processor -- start 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234420af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234421030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234426060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23441ada0 con 0x55d23441a3f0 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d234337d50 con 0x55d2344168a0 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d23441af20 con 0x55d234417290 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab9c0e740 10 client.0 _ll_register_callbacks cb 0x55d234337720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab54ef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234426060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab4cee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234420af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7aaffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234421030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab54ef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234426060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47962/0 (socket says 172.21.15.7:47962) 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.423+0000 7f7ab54ef640 1 -- 172.21.15.7:0/1860737523 learned_addr learned my addr 172.21.15.7:0/1860737523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab4cee640 1 -- 172.21.15.7:0/1860737523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 msgr2=0x55d234421030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234421030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab4cee640 1 -- 172.21.15.7:0/1860737523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 msgr2=0x55d234426060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234426060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab4cee640 1 -- 172.21.15.7:0/1860737523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d23415c620 con 0x55d2344168a0 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab54ef640 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234426060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7aaffff640 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234421030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234420af0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7aa802c630 tx=0x7f7aa80095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c336a43763fadc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1860737523 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa804c020 con 0x55d2344168a0 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1860737523 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7aa803f670 con 0x55d2344168a0 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1860737523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7aa8032c90 con 0x55d2344168a0 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1860737523 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aa8043070 con 0x55d2344168a0 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab9c0e740 10 client.27140 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1860737523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d23443c540 con 0x55d2344168a0 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.424+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1860737523 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aa8032c90 con 0x55d2344168a0 2024-03-26T08:49:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.425+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1860737523 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7aa80484e0 con 0x55d2344168a0 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.425+0000 7f7ab9c0e740 10 client.27140 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.425+0000 7f7ab9c0e740 10 client.27140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.425+0000 7f7ab9c0e740 20 client.27140 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.425+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d23443d890 0x55d23443fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.425+0000 7f7ab9c0e740 4 client.27140 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.425+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1860737523 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d234389050 con 0x55d23443d890 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.425+0000 7f7aaffff640 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d23443d890 0x55d23443fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7aaffff640 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d23443d890 0x55d23443fc80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6842daded954d301 server_cookie=1d9d5d5f96457dc8 in_seq=0 out_seq=0 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7a8ffff640 10 client.27140 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1860737523 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d234389050 con 0x55d23443d890 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7a8ffff640 10 client.27140 handle_command_reply: tid=0 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7ab9c0e740 1 client.27140 shutdown 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1860737523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 msgr2=0x55d234420af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234420af0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7aa802c630 tx=0x7f7aa80095e0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1860737523 shutdown_connections 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d23443d890 0x55d23443fc80 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d234426060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7a8ffff640 0 client.27140 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d234417290 0x55d234421030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1860737523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2344168a0 0x55d234420af0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.426+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1860737523 >> 172.21.15.7:0/1860737523 conn(0x55d234411e90 msgr2=0x55d234411020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.427+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1860737523 shutdown_connections 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.427+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1860737523 wait complete. 2024-03-26T08:49:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.427+0000 7f7ab9c0e740 20 client.27140 trim_cache size 0 max 16384 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.445+0000 7f7ab9c0e740 1 Processor -- start 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.445+0000 7f7ab9c0e740 1 -- start start 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.446+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2344168a0 0x55d234410790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.446+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 0x55d234410cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.446+0000 7f7ab4cee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2344168a0 0x55d234410790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.446+0000 7f7ab4cee640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2344168a0 0x55d234410790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47968/0 (socket says 172.21.15.7:47968) 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.446+0000 7f7ab4cee640 1 -- 172.21.15.7:0/639017685 learned_addr learned my addr 172.21.15.7:0/639017685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.446+0000 7f7aaffff640 1 --2- 172.21.15.7:0/639017685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 0x55d234410cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.446+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/639017685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d23441a3f0 0x55d234420110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.446+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/639017685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab01b1610 con 0x55d2344168a0 2024-03-26T08:49:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.447+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/639017685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab01af110 con 0x55d234417290 2024-03-26T08:49:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.447+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/639017685 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab01ad190 con 0x55d23441a3f0 2024-03-26T08:49:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.447+0000 7f7ab4cee640 1 -- 172.21.15.7:0/639017685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d23441a3f0 msgr2=0x55d234420110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.447+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/639017685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d23441a3f0 0x55d234420110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.447+0000 7f7ab4cee640 1 -- 172.21.15.7:0/639017685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 msgr2=0x55d234410cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.447+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/639017685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 0x55d234410cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.447+0000 7f7ab4cee640 1 -- 172.21.15.7:0/639017685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2344128e0 con 0x55d2344168a0 2024-03-26T08:49:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.448+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/639017685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2344168a0 0x55d234410790 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7aa802c130 tx=0x7f7aa8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4891a16d7fe1606 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.448+0000 7f7aa67fc640 1 -- 172.21.15.7:0/639017685 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa804c040 con 0x55d2344168a0 2024-03-26T08:49:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.448+0000 7f7aa67fc640 1 -- 172.21.15.7:0/639017685 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aa8032c90 con 0x55d2344168a0 2024-03-26T08:49:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.448+0000 7f7aa67fc640 1 -- 172.21.15.7:0/639017685 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8039070 con 0x55d2344168a0 2024-03-26T08:49:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.449+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/639017685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2344168a0 msgr2=0x55d234410790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.449+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/639017685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2344168a0 0x55d234410790 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7aa802c130 tx=0x7f7aa8009650 comp rx=0 tx=0).stop 2024-03-26T08:49:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.449+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/639017685 shutdown_connections 2024-03-26T08:49:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.449+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/639017685 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d23441a3f0 0x55d234420110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.449+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/639017685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 0x55d234410cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.449+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/639017685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d2344168a0 0x55d234410790 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.449+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/639017685 >> 172.21.15.7:0/639017685 conn(0x55d234411e90 msgr2=0x55d2344150f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.449+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/639017685 shutdown_connections 2024-03-26T08:49:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.449+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/639017685 wait complete. 2024-03-26T08:49:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.450+0000 7f7ab9c0e740 1 -- start start 2024-03-26T08:49:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.450+0000 7f7ab9c0e740 1 Processor -- start 2024-03-26T08:49:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2344168a0 0x55d23444f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 0x55d23444de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab9c0e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d23444e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab9c0e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab01ad790 con 0x55d23441a3f0 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab9c0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab01aeab0 con 0x55d234417290 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab9c0e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab01addf0 con 0x55d2344168a0 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab9c0e740 10 client.0 _ll_register_callbacks cb 0x7f7a98005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab4cee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2344168a0 0x55d23444f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab4cee640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2344168a0 0x55d23444f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60232/0 (socket says 172.21.15.7:60232) 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab54ef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d23444e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab4cee640 1 -- 172.21.15.7:0/1016859281 learned_addr learned my addr 172.21.15.7:0/1016859281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7aaffff640 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 0x55d23444de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab4cee640 1 -- 172.21.15.7:0/1016859281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 msgr2=0x55d23444de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 0x55d23444de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab4cee640 1 -- 172.21.15.7:0/1016859281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 msgr2=0x55d23444e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d23444e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab4cee640 1 -- 172.21.15.7:0/1016859281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d23444e900 con 0x55d2344168a0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7ab4cee640 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2344168a0 0x55d23444f870 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009620 tx=0x7f7aa8048920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a902b31d3dc248 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.451+0000 7f7a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1016859281 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8039070 con 0x55d2344168a0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1016859281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7aa8032c90 con 0x55d2344168a0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1016859281 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7aa803f470 con 0x55d2344168a0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1016859281 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aa804c040 con 0x55d2344168a0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7ab9c0e740 10 client.27166 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1016859281 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d234499150 con 0x55d2344168a0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1016859281 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aa8032c90 con 0x55d2344168a0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1016859281 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7aa803fbe0 con 0x55d2344168a0 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7ab9c0e740 10 client.27166 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7ab9c0e740 10 client.27166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7ab9c0e740 20 client.27166 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d234499990 0x55d23449bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7ab9c0e740 4 client.27166 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:49:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1016859281 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d23441b630 con 0x55d234499990 2024-03-26T08:49:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.452+0000 7f7aaffff640 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d234499990 0x55d23449bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.453+0000 7f7a8ffff640 10 client.27166 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.453+0000 7f7aaffff640 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d234499990 0x55d23449bd80 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f98d28055ff24e server_cookie=9b7d01c96a0f0bc8 in_seq=0 out_seq=0 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7a8ffff640 1 -- 172.21.15.7:0/1016859281 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d23441b630 con 0x55d234499990 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7a8ffff640 10 client.27166 handle_command_reply: tid=0 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 client.27166 shutdown 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1016859281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2344168a0 msgr2=0x55d23444f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2344168a0 0x55d23444f870 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009620 tx=0x7f7aa8048920 comp rx=0 tx=0).stop 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1016859281 shutdown_connections 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d234499990 0x55d23449bd80 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7a8ffff640 0 client.27166 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d23441a3f0 0x55d23444e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d234417290 0x55d23444de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 --2- 172.21.15.7:0/1016859281 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d2344168a0 0x55d23444f870 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1016859281 >> 172.21.15.7:0/1016859281 conn(0x55d234411e90 msgr2=0x55d234410790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1016859281 shutdown_connections 2024-03-26T08:49:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 1 -- 172.21.15.7:0/1016859281 wait complete. 2024-03-26T08:49:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.469+0000 7f7ab9c0e740 20 client.27166 trim_cache size 0 max 16384 2024-03-26T08:49:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:49:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:49:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:49:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:49:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a9c0a2870 msgr2=0x7f7a9c0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a9c0a2870 0x7f7a9c0a4d30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009810 tx=0x7f7aa0009f90 comp rx=0 tx=0).stop 2024-03-26T08:49:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3781775767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab0102180 msgr2=0x7f7ab01a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab0102180 0x7f7ab01a6950 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009e40 tx=0x7f7a9800ed40 comp rx=0 tx=0).stop 2024-03-26T08:49:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3781775767 shutdown_connections 2024-03-26T08:49:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7a9c0a2870 0x7f7a9c0a4d30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab0102180 0x7f7ab01a6950 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab01017c0 0x7f7ab0070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 --2- 172.21.15.7:0/3781775767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab01005d0 0x7f7ab00700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3781775767 >> 172.21.15.7:0/3781775767 conn(0x7f7ab00fbb40 msgr2=0x7f7ab0104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3781775767 shutdown_connections 2024-03-26T08:49:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:45.470+0000 7f7ab6f52640 1 -- 172.21.15.7:0/3781775767 wait complete. 2024-03-26T08:49:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 435.0 s, 2880.2 tps, lat 34.902 ms stddev 39.813 2024-03-26T08:49:46.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:46 smithi007 ceph-mon[32462]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 MiB/s wr, 708 op/s 2024-03-26T08:49:46.297 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:46 smithi016 ceph-mon[35851]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 MiB/s wr, 708 op/s 2024-03-26T08:49:46.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:46 smithi044 ceph-mon[36461]: pgmap v580: 97 pgs: 97 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.2 MiB/s wr, 708 op/s 2024-03-26T08:49:48.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:48 smithi007 ceph-mon[32462]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 727 op/s 2024-03-26T08:49:48.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:48 smithi044 ceph-mon[36461]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 727 op/s 2024-03-26T08:49:48.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:48 smithi016 ceph-mon[35851]: pgmap v581: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.4 MiB/s wr, 727 op/s 2024-03-26T08:49:50.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:50 smithi007 ceph-mon[32462]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.3 MiB/s wr, 704 op/s 2024-03-26T08:49:50.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:50 smithi044 ceph-mon[36461]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.3 MiB/s wr, 704 op/s 2024-03-26T08:49:50.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:50 smithi016 ceph-mon[35851]: pgmap v582: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.3 MiB/s wr, 704 op/s 2024-03-26T08:49:51.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 440.0 s, 2719.8 tps, lat 36.433 ms stddev 41.265 2024-03-26T08:49:52.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:52 smithi007 ceph-mon[32462]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.3 MiB/s wr, 710 op/s 2024-03-26T08:49:52.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:52 smithi044 ceph-mon[36461]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.3 MiB/s wr, 710 op/s 2024-03-26T08:49:52.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:52 smithi016 ceph-mon[35851]: pgmap v583: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.3 MiB/s wr, 710 op/s 2024-03-26T08:49:54.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:54 smithi044 ceph-mon[36461]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 MiB/s wr, 769 op/s 2024-03-26T08:49:54.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:54 smithi016 ceph-mon[35851]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 MiB/s wr, 769 op/s 2024-03-26T08:49:54.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:54 smithi007 ceph-mon[32462]: pgmap v584: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.1 MiB/s wr, 769 op/s 2024-03-26T08:49:55.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:49:55.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:49:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.424+0000 7fea8e3f4640 1 -- 172.21.15.7:0/884386515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea88102160 msgr2=0x7fea88104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.424+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/884386515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea88102160 0x7fea88104550 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fea78009900 tx=0x7fea7802bc30 comp rx=0 tx=0).stop 2024-03-26T08:49:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.424+0000 7fea8e3f4640 1 -- 172.21.15.7:0/884386515 shutdown_connections 2024-03-26T08:49:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.424+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/884386515 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea881073c0 0x7fea88109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.424+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/884386515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88104a90 0x7fea88106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.424+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/884386515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea88102160 0x7fea88104550 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.424+0000 7fea8e3f4640 1 -- 172.21.15.7:0/884386515 >> 172.21.15.7:0/884386515 conn(0x7fea880fbb70 msgr2=0x7fea880fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.424+0000 7fea8e3f4640 1 -- 172.21.15.7:0/884386515 shutdown_connections 2024-03-26T08:49:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.424+0000 7fea8e3f4640 1 -- 172.21.15.7:0/884386515 wait complete. 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.425+0000 7fea8e3f4640 1 Processor -- start 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.425+0000 7fea8e3f4640 1 -- start start 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.425+0000 7fea8e3f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea88102160 0x7fea88197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.425+0000 7fea8e3f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88104a90 0x7fea88197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.425+0000 7fea8e3f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea881073c0 0x7fea8819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.425+0000 7fea8e3f4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8810d370 con 0x7fea881073c0 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.425+0000 7fea8e3f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8810d1f0 con 0x7fea88104a90 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.425+0000 7fea8e3f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8810d4f0 con 0x7fea88102160 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.425+0000 7fea87fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea88102160 0x7fea88197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea8c991640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea881073c0 0x7fea8819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea87fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea88102160 0x7fea88197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59604/0 (socket says 172.21.15.7:59604) 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea87fff640 1 -- 172.21.15.7:0/1583317727 learned_addr learned my addr 172.21.15.7:0/1583317727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea8c991640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea881073c0 0x7fea8819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42324/0 (socket says 172.21.15.7:42324) 2024-03-26T08:49:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea877fe640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88104a90 0x7fea88197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea87fff640 1 -- 172.21.15.7:0/1583317727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88104a90 msgr2=0x7fea88197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea87fff640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88104a90 0x7fea88197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea87fff640 1 -- 172.21.15.7:0/1583317727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea881073c0 msgr2=0x7fea8819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea87fff640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea881073c0 0x7fea8819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.426+0000 7fea87fff640 1 -- 172.21.15.7:0/1583317727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8819f630 con 0x7fea88102160 2024-03-26T08:49:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.427+0000 7fea87fff640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea88102160 0x7fea88197a10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea78009230 tx=0x7fea78005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.427+0000 7fea857fa640 1 -- 172.21.15.7:0/1583317727 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78041070 con 0x7fea88102160 2024-03-26T08:49:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.427+0000 7fea857fa640 1 -- 172.21.15.7:0/1583317727 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea780044d0 con 0x7fea88102160 2024-03-26T08:49:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.427+0000 7fea857fa640 1 -- 172.21.15.7:0/1583317727 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78034440 con 0x7fea88102160 2024-03-26T08:49:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.428+0000 7fea8e3f4640 1 -- 172.21.15.7:0/1583317727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8819f8c0 con 0x7fea88102160 2024-03-26T08:49:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.428+0000 7fea8e3f4640 1 -- 172.21.15.7:0/1583317727 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8819fd60 con 0x7fea88102160 2024-03-26T08:49:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.431+0000 7fea90fe4740 1 Processor -- start 2024-03-26T08:49:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.431+0000 7fea90fe4740 1 -- start start 2024-03-26T08:49:55.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.431+0000 7fea857fa640 1 -- 172.21.15.7:0/1583317727 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fea78044070 con 0x7fea88102160 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d33e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 0x55732d3436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea8c991640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d33e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea8c991640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d33e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42340/0 (socket says 172.21.15.7:42340) 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea8c991640 1 -- 172.21.15.7:0/2455179243 learned_addr learned my addr 172.21.15.7:0/2455179243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3461a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea90fe4740 1 -- 172.21.15.7:0/2455179243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732d29f3d0 con 0x55732d33e1b0 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea90fe4740 1 -- 172.21.15.7:0/2455179243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732d247ad0 con 0x55732d343d10 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea90fe4740 1 -- 172.21.15.7:0/2455179243 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732d29b470 con 0x55732d33eba0 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea87fff640 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 0x55732d3436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.432+0000 7fea877fe640 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea857fa640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea700a2900 0x7fea700a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea877fe640 1 -- 172.21.15.7:0/2455179243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 msgr2=0x55732d3436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea877fe640 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 0x55732d3436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea877fe640 1 -- 172.21.15.7:0/2455179243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 msgr2=0x55732d33e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea877fe640 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d33e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea877fe640 1 -- 172.21.15.7:0/2455179243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55732d07d7d0 con 0x55732d343d10 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea857fa640 1 -- 172.21.15.7:0/1583317727 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea780f0590 con 0x7fea88102160 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea877fe640 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3461a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fea7c005d00 tx=0x7fea7c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32e02b5f6b02f86 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea8c991640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea700a2900 0x7fea700a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea6e7fc640 1 -- 172.21.15.7:0/2455179243 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c01c680 con 0x55732d343d10 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea6e7fc640 1 -- 172.21.15.7:0/2455179243 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea7c00be20 con 0x55732d343d10 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.433+0000 7fea6e7fc640 1 -- 172.21.15.7:0/2455179243 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c011b90 con 0x55732d343d10 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea90fe4740 1 -- 172.21.15.7:0/2455179243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 msgr2=0x55732d3461a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3461a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fea7c005d00 tx=0x7fea7c0107a0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea8c991640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea700a2900 0x7fea700a4dc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55732d33f400 tx=0x7fea80004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea90fe4740 1 -- 172.21.15.7:0/2455179243 shutdown_connections 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3461a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 0x55732d3436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2455179243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d33e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea90fe4740 1 -- 172.21.15.7:0/2455179243 >> 172.21.15.7:0/2455179243 conn(0x55732d335310 msgr2=0x55732d335710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea90fe4740 1 -- 172.21.15.7:0/2455179243 shutdown_connections 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.434+0000 7fea90fe4740 1 -- 172.21.15.7:0/2455179243 wait complete. 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea90fe4740 1 -- start start 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea90fe4740 1 Processor -- start 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33e1b0 0x55732d348600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d348b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d34fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d348b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d348b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42346/0 (socket says 172.21.15.7:42346) 2024-03-26T08:49:55.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea87fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33e1b0 0x55732d348600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea877fe640 1 -- 172.21.15.7:0/2547065035 learned_addr learned my addr 172.21.15.7:0/2547065035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732d346cc0 con 0x55732d33eba0 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732d25b060 con 0x55732d343d10 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732d346e40 con 0x55732d33e1b0 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea90fe4740 10 client.0 _ll_register_callbacks cb 0x55732d23d6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.435+0000 7fea8c991640 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d34fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.436+0000 7fea877fe640 1 -- 172.21.15.7:0/2547065035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33e1b0 msgr2=0x55732d348600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.436+0000 7fea877fe640 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33e1b0 0x55732d348600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.436+0000 7fea877fe640 1 -- 172.21.15.7:0/2547065035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 msgr2=0x55732d34fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.436+0000 7fea877fe640 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d34fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.436+0000 7fea877fe640 1 -- 172.21.15.7:0/2547065035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55732cfb33f0 con 0x55732d33eba0 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.436+0000 7fea87fff640 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33e1b0 0x55732d348600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.436+0000 7fea877fe640 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d348b40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fea7c000c00 tx=0x7fea7c011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313ac3c1cd2991e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.436+0000 7fea677fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.437+0000 7fea677fe640 1 -- 172.21.15.7:0/2547065035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c02b060 con 0x55732d33eba0 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.437+0000 7fea677fe640 1 -- 172.21.15.7:0/2547065035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea7c011ba0 con 0x55732d33eba0 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.437+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea7c00be20 con 0x55732d33eba0 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.438+0000 7fea677fe640 1 -- 172.21.15.7:0/2547065035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea7c011da0 con 0x55732d33eba0 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.438+0000 7fea90fe4740 10 client.17313 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.438+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55732cf730f0 con 0x55732d33eba0 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.439+0000 7fea677fe640 1 -- 172.21.15.7:0/2547065035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea7c00be20 con 0x55732d33eba0 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.439+0000 7fea677fe640 1 -- 172.21.15.7:0/2547065035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fea7c026770 con 0x55732d33eba0 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.439+0000 7fea90fe4740 10 client.17313 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.439+0000 7fea90fe4740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.439+0000 7fea90fe4740 20 client.17313 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.439+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55732d353cc0 0x55732d3560b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.439+0000 7fea90fe4740 4 client.17313 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.439+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55732d2ac910 con 0x55732d353cc0 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.439+0000 7fea87fff640 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55732d353cc0 0x55732d3560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.440+0000 7fea677fe640 10 client.17313 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.440+0000 7fea87fff640 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55732d353cc0 0x55732d3560b0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a13cd4cebc7de1c5 server_cookie=693422cc96e268c5 in_seq=0 out_seq=0 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.440+0000 7fea677fe640 1 -- 172.21.15.7:0/2547065035 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55732d2ac910 con 0x55732d353cc0 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.440+0000 7fea677fe640 10 client.17313 handle_command_reply: tid=0 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.440+0000 7fea90fe4740 1 client.17313 shutdown 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.440+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 msgr2=0x55732d348b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.440+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d348b40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fea7c000c00 tx=0x7fea7c011670 comp rx=0 tx=0).stop 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.440+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 shutdown_connections 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.440+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55732d353cc0 0x55732d3560b0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.441+0000 7fea677fe640 0 client.17313 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.441+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d34fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.441+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d348b40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.441+0000 7fea90fe4740 1 --2- 172.21.15.7:0/2547065035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33e1b0 0x55732d348600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.441+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 >> 172.21.15.7:0/2547065035 conn(0x55732d335310 msgr2=0x55732d334530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.441+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 shutdown_connections 2024-03-26T08:49:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.441+0000 7fea90fe4740 1 -- 172.21.15.7:0/2547065035 wait complete. 2024-03-26T08:49:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.441+0000 7fea90fe4740 20 client.17313 trim_cache size 0 max 16384 2024-03-26T08:49:55.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea90fe4740 1 Processor -- start 2024-03-26T08:49:55.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea90fe4740 1 -- start start 2024-03-26T08:49:55.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d33e1b0 0x55732d348600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d34fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d343d10 0x55732d3343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea90fe4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a8ee0 con 0x55732d33eba0 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea90fe4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a69e0 con 0x55732d33e1b0 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea90fe4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a4a60 con 0x55732d343d10 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea877fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d34fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d33e1b0 0x55732d348600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d33e1b0 0x55732d348600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56418/0 (socket says 172.21.15.7:56418) 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea8c991640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d343d10 0x55732d3343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea87fff640 1 -- 172.21.15.7:0/3452677810 learned_addr learned my addr 172.21.15.7:0/3452677810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea87fff640 1 -- 172.21.15.7:0/3452677810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d343d10 msgr2=0x55732d3343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea87fff640 1 --2- 172.21.15.7:0/3452677810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d343d10 0x55732d3343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea87fff640 1 -- 172.21.15.7:0/3452677810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 msgr2=0x55732d34fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea87fff640 1 --2- 172.21.15.7:0/3452677810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d34fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.463+0000 7fea87fff640 1 -- 172.21.15.7:0/3452677810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55732d33f430 con 0x55732d33e1b0 2024-03-26T08:49:55.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.464+0000 7fea8c991640 1 --2- 172.21.15.7:0/3452677810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d343d10 0x55732d3343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.465+0000 7fea87fff640 1 --2- 172.21.15.7:0/3452677810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d33e1b0 0x55732d348600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fea78009430 tx=0x7fea780edf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e29871bd02485233 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.465+0000 7fea6dffb640 1 -- 172.21.15.7:0/3452677810 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7804b040 con 0x55732d33e1b0 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.465+0000 7fea6dffb640 1 -- 172.21.15.7:0/3452677810 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea780f3e20 con 0x55732d33e1b0 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.465+0000 7fea6dffb640 1 -- 172.21.15.7:0/3452677810 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea780c1040 con 0x55732d33e1b0 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.466+0000 7fea90fe4740 1 -- 172.21.15.7:0/3452677810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d33e1b0 msgr2=0x55732d348600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.466+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3452677810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d33e1b0 0x55732d348600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fea78009430 tx=0x7fea780edf20 comp rx=0 tx=0).stop 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.466+0000 7fea90fe4740 1 -- 172.21.15.7:0/3452677810 shutdown_connections 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.466+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3452677810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d343d10 0x55732d3343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.466+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3452677810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33eba0 0x55732d34fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.466+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3452677810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d33e1b0 0x55732d348600 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.466+0000 7fea90fe4740 1 -- 172.21.15.7:0/3452677810 >> 172.21.15.7:0/3452677810 conn(0x55732d335310 msgr2=0x55732d333ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.466+0000 7fea90fe4740 1 -- 172.21.15.7:0/3452677810 shutdown_connections 2024-03-26T08:49:55.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.466+0000 7fea90fe4740 1 -- 172.21.15.7:0/3452677810 wait complete. 2024-03-26T08:49:55.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea90fe4740 1 -- start start 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea90fe4740 1 Processor -- start 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d34eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 0x55732d34f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea90fe4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea90fe4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a5060 con 0x55732d33e1b0 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea90fe4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a6380 con 0x55732d343d10 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea90fe4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881a56c0 con 0x55732d33eba0 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea90fe4740 10 client.0 _ll_register_callbacks cb 0x7fea80009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea87fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d34eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea877fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 0x55732d34f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea87fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d34eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42358/0 (socket says 172.21.15.7:42358) 2024-03-26T08:49:55.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea8c991640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.467+0000 7fea87fff640 1 -- 172.21.15.7:0/3852318840 learned_addr learned my addr 172.21.15.7:0/3852318840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea8c991640 1 -- 172.21.15.7:0/3852318840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 msgr2=0x55732d34f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea8c991640 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 0x55732d34f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea8c991640 1 -- 172.21.15.7:0/3852318840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 msgr2=0x55732d34eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea8c991640 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d34eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea8c991640 1 -- 172.21.15.7:0/3852318840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55732d3af8f0 con 0x55732d343d10 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea877fe640 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 0x55732d34f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea8c991640 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3af3b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fea8000f790 tx=0x7fea80004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d44f93461769869 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea677fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea677fe640 1 -- 172.21.15.7:0/3852318840 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8001a040 con 0x55732d343d10 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea677fe640 1 -- 172.21.15.7:0/3852318840 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea80004d20 con 0x55732d343d10 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.468+0000 7fea90fe4740 1 -- 172.21.15.7:0/3852318840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea780f3e20 con 0x55732d343d10 2024-03-26T08:49:55.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea677fe640 1 -- 172.21.15.7:0/3852318840 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea80004400 con 0x55732d343d10 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea90fe4740 10 client.27164 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea90fe4740 1 -- 172.21.15.7:0/3852318840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55732d3b0c70 con 0x55732d343d10 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea677fe640 1 -- 172.21.15.7:0/3852318840 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea780f3e20 con 0x55732d343d10 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea677fe640 1 -- 172.21.15.7:0/3852318840 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fea8002a070 con 0x55732d343d10 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea90fe4740 10 client.27164 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea90fe4740 10 client.27164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea90fe4740 20 client.27164 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55732d353cc0 0x55732d3b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea90fe4740 4 client.27164 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea90fe4740 1 -- 172.21.15.7:0/3852318840 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55732d2a6c60 con 0x55732d353cc0 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.469+0000 7fea87fff640 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55732d353cc0 0x55732d3b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea677fe640 10 client.27164 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea87fff640 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55732d353cc0 0x55732d3b36c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2fe4b4c7e3377c server_cookie=c60ba626b5671643 in_seq=0 out_seq=0 2024-03-26T08:49:55.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea677fe640 1 -- 172.21.15.7:0/3852318840 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55732d2a6c60 con 0x55732d353cc0 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea677fe640 10 client.27164 handle_command_reply: tid=0 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 client.27164 shutdown 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 -- 172.21.15.7:0/3852318840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 msgr2=0x55732d3af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3af3b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fea8000f790 tx=0x7fea80004a40 comp rx=0 tx=0).stop 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 -- 172.21.15.7:0/3852318840 shutdown_connections 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55732d353cc0 0x55732d3b36c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55732d343d10 0x55732d3af3b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55732d33eba0 0x55732d34f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 --2- 172.21.15.7:0/3852318840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55732d33e1b0 0x55732d34eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 -- 172.21.15.7:0/3852318840 >> 172.21.15.7:0/3852318840 conn(0x55732d335310 msgr2=0x55732d334530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea677fe640 0 client.27164 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 -- 172.21.15.7:0/3852318840 shutdown_connections 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.470+0000 7fea90fe4740 1 -- 172.21.15.7:0/3852318840 wait complete. 2024-03-26T08:49:55.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea90fe4740 20 client.27164 trim_cache size 0 max 16384 2024-03-26T08:49:55.470 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:49:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:49:55.470 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:49:55.470 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:49:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 -- 172.21.15.7:0/1583317727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea700a2900 msgr2=0x7fea700a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea700a2900 0x7fea700a4dc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55732d33f400 tx=0x7fea80004ab0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 -- 172.21.15.7:0/1583317727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea88102160 msgr2=0x7fea88197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea88102160 0x7fea88197a10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fea78009230 tx=0x7fea78005d50 comp rx=0 tx=0).stop 2024-03-26T08:49:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 -- 172.21.15.7:0/1583317727 shutdown_connections 2024-03-26T08:49:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea700a2900 0x7fea700a4dc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea881073c0 0x7fea8819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88104a90 0x7fea88197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 --2- 172.21.15.7:0/1583317727 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea88102160 0x7fea88197a10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 -- 172.21.15.7:0/1583317727 >> 172.21.15.7:0/1583317727 conn(0x7fea880fbb70 msgr2=0x7fea88102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 -- 172.21.15.7:0/1583317727 shutdown_connections 2024-03-26T08:49:55.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.471+0000 7fea8e3f4640 1 -- 172.21.15.7:0/1583317727 wait complete. 2024-03-26T08:49:55.481 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:49:55.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:49:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.650+0000 7f259cbf4640 1 -- 172.21.15.7:0/1376574179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 msgr2=0x7f2598109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.650+0000 7f259cbf4640 1 --2- 172.21.15.7:0/1376574179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 0x7f2598109bb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2590009cd0 tx=0x7f259002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.651+0000 7f259cbf4640 1 -- 172.21.15.7:0/1376574179 shutdown_connections 2024-03-26T08:49:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.651+0000 7f259cbf4640 1 --2- 172.21.15.7:0/1376574179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 0x7f2598109bb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.651+0000 7f259cbf4640 1 --2- 172.21.15.7:0/1376574179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2598102960 0x7f2598102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.651+0000 7f259cbf4640 1 --2- 172.21.15.7:0/1376574179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25980ff990 0x7f25980ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.651+0000 7f259cbf4640 1 -- 172.21.15.7:0/1376574179 >> 172.21.15.7:0/1376574179 conn(0x7f25980fb2c0 msgr2=0x7f25980fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.651+0000 7f259cbf4640 1 -- 172.21.15.7:0/1376574179 shutdown_connections 2024-03-26T08:49:55.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.652+0000 7f259cbf4640 1 -- 172.21.15.7:0/1376574179 wait complete. 2024-03-26T08:49:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.652+0000 7f259cbf4640 1 Processor -- start 2024-03-26T08:49:55.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.652+0000 7f259cbf4640 1 -- start start 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f259cbf4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25980ff990 0x7f2598199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f259cbf4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2598102960 0x7f259819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f259cbf4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 0x7f25981a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f259cbf4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259810d370 con 0x7f2598103320 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f259cbf4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259810d1f0 con 0x7f2598102960 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f259cbf4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259810d4f0 con 0x7f25980ff990 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2596d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 0x7f25981a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2596d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 0x7f25981a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42376/0 (socket says 172.21.15.7:42376) 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2596d9d640 1 -- 172.21.15.7:0/2386031633 learned_addr learned my addr 172.21.15.7:0/2386031633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2595d9b640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2598102960 0x7f259819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f259659c640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25980ff990 0x7f2598199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2595d9b640 1 -- 172.21.15.7:0/2386031633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25980ff990 msgr2=0x7f2598199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2595d9b640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25980ff990 0x7f2598199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2595d9b640 1 -- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 msgr2=0x7f25981a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:55.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2595d9b640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 0x7f25981a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2595d9b640 1 -- 172.21.15.7:0/2386031633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25981a17c0 con 0x7f2598102960 2024-03-26T08:49:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2596d9d640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 0x7f25981a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.653+0000 7f2595d9b640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2598102960 0x7f259819a0e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f258c00e9f0 tx=0x7f258c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.654+0000 7f25877fe640 1 -- 172.21.15.7:0/2386031633 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c00ccc0 con 0x7f2598102960 2024-03-26T08:49:55.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.654+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25981a1ab0 con 0x7f2598102960 2024-03-26T08:49:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.654+0000 7f25877fe640 1 -- 172.21.15.7:0/2386031633 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f258c005020 con 0x7f2598102960 2024-03-26T08:49:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.654+0000 7f25877fe640 1 -- 172.21.15.7:0/2386031633 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0049e0 con 0x7f2598102960 2024-03-26T08:49:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.654+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25981a2000 con 0x7f2598102960 2024-03-26T08:49:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.655+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25981a2230 con 0x7f2598102960 2024-03-26T08:49:55.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.656+0000 7f25877fe640 1 -- 172.21.15.7:0/2386031633 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f258c005560 con 0x7f2598102960 2024-03-26T08:49:55.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.657+0000 7f25877fe640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25800a2900 0x7f25800a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:55.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.657+0000 7f25877fe640 1 -- 172.21.15.7:0/2386031633 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f258c0ce9e0 con 0x7f2598102960 2024-03-26T08:49:55.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.657+0000 7f259659c640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25800a2900 0x7f25800a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:55.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.657+0000 7f259659c640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25800a2900 0x7f25800a4dc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2598068770 tx=0x7f2588005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:55.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:55.662+0000 7f25877fe640 1 -- 172.21.15.7:0/2386031633 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f258c0051d0 con 0x7f2598102960 2024-03-26T08:49:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.006+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25981a24a0 con 0x7f2598102960 2024-03-26T08:49:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.007+0000 7f25877fe640 1 -- 172.21.15.7:0/2386031633 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f258c09ba40 con 0x7f2598102960 2024-03-26T08:49:56.002 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:49:56.002 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:49:56.002 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25800a2900 msgr2=0x7f25800a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25800a2900 0x7f25800a4dc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2598068770 tx=0x7f2588005f90 comp rx=0 tx=0).stop 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2598102960 msgr2=0x7f259819a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2598102960 0x7f259819a0e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f258c00e9f0 tx=0x7f258c00eeb0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 shutdown_connections 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25800a2900 0x7f25800a4dc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2598103320 0x7f25981a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2598102960 0x7f259819a0e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 --2- 172.21.15.7:0/2386031633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25980ff990 0x7f2598199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 >> 172.21.15.7:0/2386031633 conn(0x7f25980fb2c0 msgr2=0x7f2598100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 shutdown_connections 2024-03-26T08:49:56.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.010+0000 7f259cbf4640 1 -- 172.21.15.7:0/2386031633 wait complete. 2024-03-26T08:49:56.018 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:49:56.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:49:56.073 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 445.0 s, 2005.0 tps, lat 50.183 ms stddev 54.018 2024-03-26T08:49:56.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.175+0000 7f0bde390640 1 -- 172.21.15.7:0/2273675591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd81017d0 msgr2=0x7f0bd8101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.175+0000 7f0bde390640 1 --2- 172.21.15.7:0/2273675591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd81017d0 0x7f0bd8101c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009900 tx=0x7f0bcc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:49:56.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.175+0000 7f0bde390640 1 -- 172.21.15.7:0/2273675591 shutdown_connections 2024-03-26T08:49:56.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.175+0000 7f0bde390640 1 --2- 172.21.15.7:0/2273675591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd8102190 0x7f0bd8108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.175+0000 7f0bde390640 1 --2- 172.21.15.7:0/2273675591 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd81017d0 0x7f0bd8101c50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.175+0000 7f0bde390640 1 --2- 172.21.15.7:0/2273675591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd81005e0 0x7f0bd8100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.175+0000 7f0bde390640 1 -- 172.21.15.7:0/2273675591 >> 172.21.15.7:0/2273675591 conn(0x7f0bd80fbb70 msgr2=0x7f0bd80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.175+0000 7f0bde390640 1 -- 172.21.15.7:0/2273675591 shutdown_connections 2024-03-26T08:49:56.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.175+0000 7f0bde390640 1 -- 172.21.15.7:0/2273675591 wait complete. 2024-03-26T08:49:56.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.176+0000 7f0bde390640 1 Processor -- start 2024-03-26T08:49:56.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.176+0000 7f0bde390640 1 -- start start 2024-03-26T08:49:56.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.176+0000 7f0bde390640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd81005e0 0x7f0bd80714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.176+0000 7f0bde390640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd81017d0 0x7f0bd80719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.176+0000 7f0bde390640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd8102190 0x7f0bd8072890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bde390640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd806bad0 con 0x7f0bd81005e0 2024-03-26T08:49:56.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bde390640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd806b950 con 0x7f0bd81017d0 2024-03-26T08:49:56.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bde390640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd806bc50 con 0x7f0bd8102190 2024-03-26T08:49:56.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd81005e0 0x7f0bd80714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bdc92d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd8102190 0x7f0bd8072890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd81005e0 0x7f0bd80714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42402/0 (socket says 172.21.15.7:42402) 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bdc92d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd8102190 0x7f0bd8072890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59678/0 (socket says 172.21.15.7:59678) 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bd7fff640 1 -- 172.21.15.7:0/2133153379 learned_addr learned my addr 172.21.15.7:0/2133153379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd81017d0 0x7f0bd80719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bdc92d640 1 -- 172.21.15.7:0/2133153379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd81017d0 msgr2=0x7f0bd80719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bdc92d640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd81017d0 0x7f0bd80719e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bdc92d640 1 -- 172.21.15.7:0/2133153379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd81005e0 msgr2=0x7f0bd80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bdc92d640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd81005e0 0x7f0bd80714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bdc92d640 1 -- 172.21.15.7:0/2133153379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd8073010 con 0x7f0bd8102190 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd81017d0 0x7f0bd80719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.177+0000 7f0bdc92d640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd8102190 0x7f0bd8072890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009fd0 tx=0x7f0bd000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.178+0000 7f0bd57fa640 1 -- 172.21.15.7:0/2133153379 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000ce20 con 0x7f0bd8102190 2024-03-26T08:49:56.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.178+0000 7f0bde390640 1 -- 172.21.15.7:0/2133153379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd8073300 con 0x7f0bd8102190 2024-03-26T08:49:56.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.178+0000 7f0bd57fa640 1 -- 172.21.15.7:0/2133153379 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bd0005020 con 0x7f0bd8102190 2024-03-26T08:49:56.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.178+0000 7f0bde390640 1 -- 172.21.15.7:0/2133153379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd81090e0 con 0x7f0bd8102190 2024-03-26T08:49:56.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.178+0000 7f0bd57fa640 1 -- 172.21.15.7:0/2133153379 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0010720 con 0x7f0bd8102190 2024-03-26T08:49:56.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.181+0000 7f0bd57fa640 1 -- 172.21.15.7:0/2133153379 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0bd0004a40 con 0x7f0bd8102190 2024-03-26T08:49:56.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.181+0000 7f0be0f7d740 1 Processor -- start 2024-03-26T08:49:56.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.181+0000 7f0be0f7d740 1 -- start start 2024-03-26T08:49:56.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb040690 0x562dcb040ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb045b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 0x562dcb048680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcb003610 con 0x562dcb040690 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcafa63c0 con 0x562dcb0461f0 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcaf4ed60 con 0x562dcb041080 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb040690 0x562dcb040ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd57fa640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0bc00a2950 0x7f0bc00a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb040690 0x562dcb040ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42404/0 (socket says 172.21.15.7:42404) 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 -- 172.21.15.7:0/2118185097 learned_addr learned my addr 172.21.15.7:0/2118185097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd7fff640 1 --2- 172.21.15.7:0/2118185097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb045b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2118185097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 0x562dcb048680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bdc92d640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0bc00a2950 0x7f0bc00a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd57fa640 1 -- 172.21.15.7:0/2133153379 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0bd00cf040 con 0x7f0bd8102190 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 -- 172.21.15.7:0/2118185097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 msgr2=0x562dcb045b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2118185097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb045b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 -- 172.21.15.7:0/2118185097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 msgr2=0x562dcb048680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2118185097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 0x562dcb048680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 -- 172.21.15.7:0/2118185097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dcacba3f0 con 0x562dcb040690 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2118185097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb040690 0x562dcb040ab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc02c7b0 tx=0x7f0bcc02c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6941312dc90433dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.182+0000 7f0bdc92d640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0bc00a2950 0x7f0bc00a4e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c760 tx=0x7f0bd0090e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.183+0000 7f0bbe7fc640 1 -- 172.21.15.7:0/2118185097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc049070 con 0x562dcb040690 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.183+0000 7f0bbe7fc640 1 -- 172.21.15.7:0/2118185097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bcc004230 con 0x562dcb040690 2024-03-26T08:49:56.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.183+0000 7f0bbe7fc640 1 -- 172.21.15.7:0/2118185097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0437b0 con 0x562dcb040690 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2118185097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb040690 msgr2=0x562dcb040ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2118185097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb040690 0x562dcb040ab0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc02c7b0 tx=0x7f0bcc02c880 comp rx=0 tx=0).stop 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2118185097 shutdown_connections 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2118185097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 0x562dcb048680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2118185097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb045b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2118185097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb040690 0x562dcb040ab0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2118185097 >> 172.21.15.7:0/2118185097 conn(0x562dcb03bc00 msgr2=0x562dcb03c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2118185097 shutdown_connections 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2118185097 wait complete. 2024-03-26T08:49:56.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 -- start start 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.184+0000 7f0be0f7d740 1 Processor -- start 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb053270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb041080 0x562dcb0537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 0x562dcb05a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcaf62240 con 0x562dcb041080 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcafa2460 con 0x562dcb0461f0 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dcb049140 con 0x562dcb040690 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0be0f7d740 10 client.0 _ll_register_callbacks cb 0x562dcaf61c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb041080 0x562dcb0537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb041080 0x562dcb0537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42408/0 (socket says 172.21.15.7:42408) 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0bd7fff640 1 -- 172.21.15.7:0/998170009 learned_addr learned my addr 172.21.15.7:0/998170009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.185+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb053270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.186+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 0x562dcb05a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.186+0000 7f0bd77fe640 1 -- 172.21.15.7:0/998170009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 msgr2=0x562dcb05a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.186+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 0x562dcb05a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.186+0000 7f0bd77fe640 1 -- 172.21.15.7:0/998170009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb041080 msgr2=0x562dcb0537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.186+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb041080 0x562dcb0537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.186+0000 7f0bd77fe640 1 -- 172.21.15.7:0/998170009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dcad847d0 con 0x562dcb040690 2024-03-26T08:49:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.186+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb053270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009850 tx=0x7f0bcc02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c2775385466378d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.186+0000 7f0bb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:49:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.187+0000 7f0bb77fe640 1 -- 172.21.15.7:0/998170009 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc043730 con 0x562dcb040690 2024-03-26T08:49:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.187+0000 7f0be0f7d740 1 -- 172.21.15.7:0/998170009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bcc004230 con 0x562dcb040690 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.187+0000 7f0bb77fe640 1 -- 172.21.15.7:0/998170009 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0bcc0438d0 con 0x562dcb040690 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.187+0000 7f0bb77fe640 1 -- 172.21.15.7:0/998170009 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bcc004460 con 0x562dcb040690 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.187+0000 7f0be0f7d740 10 client.27202 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.187+0000 7f0be0f7d740 1 -- 172.21.15.7:0/998170009 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562dcb0495f0 con 0x562dcb040690 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.187+0000 7f0bb77fe640 1 -- 172.21.15.7:0/998170009 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bcc004230 con 0x562dcb040690 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.188+0000 7f0bb77fe640 1 -- 172.21.15.7:0/998170009 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0bcc040040 con 0x562dcb040690 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.188+0000 7f0be0f7d740 10 client.27202 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.188+0000 7f0be0f7d740 10 client.27202 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.188+0000 7f0be0f7d740 20 client.27202 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.188+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562dcb04aa00 0x562dcb04ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.188+0000 7f0be0f7d740 4 client.27202 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.188+0000 7f0be0f7d740 1 -- 172.21.15.7:0/998170009 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dcafb3900 con 0x562dcb04aa00 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.188+0000 7f0bd7fff640 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562dcb04aa00 0x562dcb04ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.188+0000 7f0bb77fe640 10 client.27202 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.189+0000 7f0bd7fff640 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562dcb04aa00 0x562dcb04ce20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2363e0b856ddc4c9 server_cookie=1a3259a1566be9b5 in_seq=0 out_seq=0 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.189+0000 7f0bb77fe640 1 -- 172.21.15.7:0/998170009 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562dcafb3900 con 0x562dcb04aa00 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.189+0000 7f0bb77fe640 10 client.27202 handle_command_reply: tid=0 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.189+0000 7f0be0f7d740 1 client.27202 shutdown 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 -- 172.21.15.7:0/998170009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 msgr2=0x562dcb053270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb053270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009850 tx=0x7f0bcc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 -- 172.21.15.7:0/998170009 shutdown_connections 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562dcb04aa00 0x562dcb04ce20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb0461f0 0x562dcb05a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb041080 0x562dcb0537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/998170009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb053270 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 -- 172.21.15.7:0/998170009 >> 172.21.15.7:0/998170009 conn(0x562dcb03bc00 msgr2=0x562dcb03ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0bb77fe640 0 client.27202 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:49:56.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 -- 172.21.15.7:0/998170009 shutdown_connections 2024-03-26T08:49:56.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 1 -- 172.21.15.7:0/998170009 wait complete. 2024-03-26T08:49:56.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.190+0000 7f0be0f7d740 20 client.27202 trim_cache size 0 max 16384 2024-03-26T08:49:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.206+0000 7f0be0f7d740 1 Processor -- start 2024-03-26T08:49:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0be0f7d740 1 -- start start 2024-03-26T08:49:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb040690 0x562dcb04ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb053590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb0513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb040690 0x562dcb04ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb0513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4010980 con 0x562dcb0461f0 2024-03-26T08:49:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb053590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb053590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59702/0 (socket says 172.21.15.7:59702) 2024-03-26T08:49:56.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd7fff640 1 -- 172.21.15.7:0/4223299246 learned_addr learned my addr 172.21.15.7:0/4223299246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb0513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42416/0 (socket says 172.21.15.7:42416) 2024-03-26T08:49:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0be0f7d740 1 -- 172.21.15.7:0/4223299246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc400e480 con 0x562dcb040690 2024-03-26T08:49:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0be0f7d740 1 -- 172.21.15.7:0/4223299246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc400c500 con 0x562dcb041080 2024-03-26T08:49:56.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd7fff640 1 -- 172.21.15.7:0/4223299246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb040690 msgr2=0x562dcb04ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd7fff640 1 --2- 172.21.15.7:0/4223299246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb040690 0x562dcb04ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd7fff640 1 -- 172.21.15.7:0/4223299246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 msgr2=0x562dcb0513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd7fff640 1 --2- 172.21.15.7:0/4223299246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb0513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd7fff640 1 -- 172.21.15.7:0/4223299246 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dcb042ea0 con 0x562dcb041080 2024-03-26T08:49:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.207+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/4223299246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb0513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.208+0000 7f0bd7fff640 1 --2- 172.21.15.7:0/4223299246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb053590 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0bc801d6b0 tx=0x7f0bc801db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b7ad8d90e7b80a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.208+0000 7f0bbdffb640 1 -- 172.21.15.7:0/4223299246 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80130d0 con 0x562dcb041080 2024-03-26T08:49:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.208+0000 7f0bbdffb640 1 -- 172.21.15.7:0/4223299246 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8013270 con 0x562dcb041080 2024-03-26T08:49:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.208+0000 7f0bbdffb640 1 -- 172.21.15.7:0/4223299246 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8011a30 con 0x562dcb041080 2024-03-26T08:49:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 -- 172.21.15.7:0/4223299246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 msgr2=0x562dcb053590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/4223299246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb053590 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0bc801d6b0 tx=0x7f0bc801db70 comp rx=0 tx=0).stop 2024-03-26T08:49:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 -- 172.21.15.7:0/4223299246 shutdown_connections 2024-03-26T08:49:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/4223299246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb0513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/4223299246 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb041080 0x562dcb053590 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/4223299246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb040690 0x562dcb04ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 -- 172.21.15.7:0/4223299246 >> 172.21.15.7:0/4223299246 conn(0x562dcb03bc00 msgr2=0x562dcb052e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 -- 172.21.15.7:0/4223299246 shutdown_connections 2024-03-26T08:49:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 -- 172.21.15.7:0/4223299246 wait complete. 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 -- start start 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.209+0000 7f0be0f7d740 1 Processor -- start 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb061460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb041080 0x562dcb0619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0be0f7d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb061ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc400cb00 con 0x562dcb0461f0 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc400de20 con 0x562dcb041080 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0be0f7d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc400d160 con 0x562dcb040690 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0be0f7d740 10 client.0 _ll_register_callbacks cb 0x7f0bcc0039b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb061460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb061460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59714/0 (socket says 172.21.15.7:59714) 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 -- 172.21.15.7:0/2460807853 learned_addr learned my addr 172.21.15.7:0/2460807853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb061ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd7fff640 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb041080 0x562dcb0619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 -- 172.21.15.7:0/2460807853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb041080 msgr2=0x562dcb0619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb041080 0x562dcb0619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 -- 172.21.15.7:0/2460807853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 msgr2=0x562dcb061ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb061ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 -- 172.21.15.7:0/2460807853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dcb0b0920 con 0x562dcb040690 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bd77fe640 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb061460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc003ac0 tx=0x7f0bcc005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86d5dc27697c2fc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.210+0000 7f0bb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0bb77fe640 1 -- 172.21.15.7:0/2460807853 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc064640 con 0x562dcb040690 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0bb77fe640 1 -- 172.21.15.7:0/2460807853 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0bcc0647e0 con 0x562dcb040690 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2460807853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bc8013270 con 0x562dcb040690 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0bb77fe640 1 -- 172.21.15.7:0/2460807853 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bcc02c980 con 0x562dcb040690 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0be0f7d740 10 client.27208 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2460807853 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562dcb0b1c00 con 0x562dcb040690 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0bb77fe640 1 -- 172.21.15.7:0/2460807853 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0bc8013270 con 0x562dcb040690 2024-03-26T08:49:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0bb77fe640 1 -- 172.21.15.7:0/2460807853 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0bcc004960 con 0x562dcb040690 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0be0f7d740 10 client.27208 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0be0f7d740 10 client.27208 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0be0f7d740 20 client.27208 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562dcb0b24b0 0x562dcb0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0be0f7d740 4 client.27208 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.211+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2460807853 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562dcafadc50 con 0x562dcb0b24b0 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0bd7fff640 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562dcb0b24b0 0x562dcb0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0bb77fe640 10 client.27208 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0bd7fff640 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562dcb0b24b0 0x562dcb0b48d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a70b1f0ffa1ee329 server_cookie=af802ce25207a22f in_seq=0 out_seq=0 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0bb77fe640 1 -- 172.21.15.7:0/2460807853 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562dcafadc50 con 0x562dcb0b24b0 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0bb77fe640 10 client.27208 handle_command_reply: tid=0 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0be0f7d740 1 client.27208 shutdown 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2460807853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 msgr2=0x562dcb061460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb061460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc003ac0 tx=0x7f0bcc005850 comp rx=0 tx=0).stop 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2460807853 shutdown_connections 2024-03-26T08:49:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562dcb0b24b0 0x562dcb0b48d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0bb77fe640 0 client.27208 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:49:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562dcb0461f0 0x562dcb061ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562dcb041080 0x562dcb0619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0be0f7d740 1 --2- 172.21.15.7:0/2460807853 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562dcb040690 0x562dcb061460 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.212+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2460807853 >> 172.21.15.7:0/2460807853 conn(0x562dcb03bc00 msgr2=0x562dcb03ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2460807853 shutdown_connections 2024-03-26T08:49:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0be0f7d740 1 -- 172.21.15.7:0/2460807853 wait complete. 2024-03-26T08:49:56.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0be0f7d740 20 client.27208 trim_cache size 0 max 16384 2024-03-26T08:49:56.214 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:49:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 -- 172.21.15.7:0/2133153379 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0bc00a2950 msgr2=0x7f0bc00a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0bc00a2950 0x7f0bc00a4e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c760 tx=0x7f0bd0090e80 comp rx=0 tx=0).stop 2024-03-26T08:49:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 -- 172.21.15.7:0/2133153379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd8102190 msgr2=0x7f0bd8072890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd8102190 0x7f0bd8072890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009fd0 tx=0x7f0bd000ef10 comp rx=0 tx=0).stop 2024-03-26T08:49:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 -- 172.21.15.7:0/2133153379 shutdown_connections 2024-03-26T08:49:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0bc00a2950 0x7f0bc00a4e10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0bd8102190 0x7f0bd8072890 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bd81017d0 0x7f0bd80719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 --2- 172.21.15.7:0/2133153379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bd81005e0 0x7f0bd80714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 -- 172.21.15.7:0/2133153379 >> 172.21.15.7:0/2133153379 conn(0x7f0bd80fbb70 msgr2=0x7f0bd8104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 -- 172.21.15.7:0/2133153379 shutdown_connections 2024-03-26T08:49:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.213+0000 7f0bde390640 1 -- 172.21.15.7:0/2133153379 wait complete. 2024-03-26T08:49:56.223 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:49:56.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:49:56.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:56 smithi007 ceph-mon[32462]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.3 MiB/s wr, 745 op/s 2024-03-26T08:49:56.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:56 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2386031633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:49:56.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:56 smithi044 ceph-mon[36461]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.3 MiB/s wr, 745 op/s 2024-03-26T08:49:56.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:56 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2386031633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:49:56.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.403+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669453473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2881086a0 msgr2=0x7ff28810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.403+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669453473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2881086a0 0x7ff28810aad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff278009900 tx=0x7ff27802bc50 comp rx=0 tx=0).stop 2024-03-26T08:49:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.403+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669453473 shutdown_connections 2024-03-26T08:49:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.403+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669453473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28810d9c0 0x7ff28810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.403+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669453473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28810b010 0x7ff28810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.403+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669453473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2881086a0 0x7ff28810aad0 secure :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff278009900 tx=0x7ff27802bc50 comp rx=0 tx=0).stop 2024-03-26T08:49:56.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.403+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669453473 >> 172.21.15.7:0/1669453473 conn(0x7ff28806e640 msgr2=0x7ff288070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.405+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669453473 shutdown_connections 2024-03-26T08:49:56.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.406+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669453473 wait complete. 2024-03-26T08:49:56.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.407+0000 7ff28ccff640 1 Processor -- start 2024-03-26T08:49:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff28ccff640 1 -- start start 2024-03-26T08:49:56.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff28ccff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28810b010 0x7ff2881a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff28ccff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff28810d9c0 0x7ff2881a6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff28ccff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2881abfe0 0x7ff2881ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff28ccff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28811c250 con 0x7ff28810d9c0 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff28ccff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28811c0d0 con 0x7ff2881abfe0 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff28ccff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28811c3d0 con 0x7ff28810b010 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2881abfe0 0x7ff2881ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2881abfe0 0x7ff2881ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56498/0 (socket says 172.21.15.7:56498) 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff286d9d640 1 -- 172.21.15.7:0/1669255563 learned_addr learned my addr 172.21.15.7:0/1669255563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff28659c640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28810b010 0x7ff2881a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.408+0000 7ff285d9b640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff28810d9c0 0x7ff2881a6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff286d9d640 1 -- 172.21.15.7:0/1669255563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28810b010 msgr2=0x7ff2881a6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff286d9d640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28810b010 0x7ff2881a6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff286d9d640 1 -- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff28810d9c0 msgr2=0x7ff2881a6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff286d9d640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff28810d9c0 0x7ff2881a6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff286d9d640 1 -- 172.21.15.7:0/1669255563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2881aea40 con 0x7ff2881abfe0 2024-03-26T08:49:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff286d9d640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2881abfe0 0x7ff2881ae3d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff28000b990 tx=0x7ff28000be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff2777fe640 1 -- 172.21.15.7:0/1669255563 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280019070 con 0x7ff2881abfe0 2024-03-26T08:49:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669255563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2881aed30 con 0x7ff2881abfe0 2024-03-26T08:49:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669255563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2881af1d0 con 0x7ff2881abfe0 2024-03-26T08:49:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.409+0000 7ff2777fe640 1 -- 172.21.15.7:0/1669255563 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff28000cba0 con 0x7ff2881abfe0 2024-03-26T08:49:56.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.410+0000 7ff2777fe640 1 -- 172.21.15.7:0/1669255563 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280007650 con 0x7ff2881abfe0 2024-03-26T08:49:56.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.413+0000 7ff2777fe640 1 -- 172.21.15.7:0/1669255563 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff28001c070 con 0x7ff2881abfe0 2024-03-26T08:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.414+0000 7ff2777fe640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2700a2940 0x7ff2700a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.414+0000 7ff28f9c8740 1 Processor -- start 2024-03-26T08:49:56.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.414+0000 7ff2777fe640 1 -- 172.21.15.7:0/1669255563 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff2800ce050 con 0x7ff2881abfe0 2024-03-26T08:49:56.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.415+0000 7ff28659c640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2700a2940 0x7ff2700a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.415+0000 7ff28f9c8740 1 -- start start 2024-03-26T08:49:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d33ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d38b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d3b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a738cf6610 con 0x55a738d33690 2024-03-26T08:49:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a738c993c0 con 0x55a738d391f0 2024-03-26T08:49:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d3b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d3b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56500/0 (socket says 172.21.15.7:56500) 2024-03-26T08:49:56.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff286d9d640 1 -- 172.21.15.7:0/3500255064 learned_addr learned my addr 172.21.15.7:0/3500255064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff28659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d33ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff285d9b640 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d38b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.416+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a738c41d60 con 0x55a738d34080 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.417+0000 7ff28659c640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2700a2940 0x7ff2700a4e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff278009850 tx=0x7ff278005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.417+0000 7ff28659c640 1 -- 172.21.15.7:0/3500255064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 msgr2=0x55a738d38b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.417+0000 7ff28659c640 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d38b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.417+0000 7ff28659c640 1 -- 172.21.15.7:0/3500255064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 msgr2=0x55a738d3b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.417+0000 7ff28659c640 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d3b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.417+0000 7ff28659c640 1 -- 172.21.15.7:0/3500255064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7389ad3f0 con 0x55a738d33690 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.417+0000 7ff285d9b640 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d38b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.418+0000 7ff28659c640 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d33ab0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff278008000 tx=0x7ff27803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ace954c761daac3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.418+0000 7ff286d9d640 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d3b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.418+0000 7ff274ff9640 1 -- 172.21.15.7:0/3500255064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278044070 con 0x55a738d33690 2024-03-26T08:49:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.418+0000 7ff274ff9640 1 -- 172.21.15.7:0/3500255064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff27802cc90 con 0x55a738d33690 2024-03-26T08:49:56.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.418+0000 7ff274ff9640 1 -- 172.21.15.7:0/3500255064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278044070 con 0x55a738d33690 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.419+0000 7ff28f9c8740 1 -- 172.21.15.7:0/3500255064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 msgr2=0x55a738d33ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.419+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d33ab0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff278008000 tx=0x7ff27803e000 comp rx=0 tx=0).stop 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.419+0000 7ff28f9c8740 1 -- 172.21.15.7:0/3500255064 shutdown_connections 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.419+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d3b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.419+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d38b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.419+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/3500255064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d33ab0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.419+0000 7ff28f9c8740 1 -- 172.21.15.7:0/3500255064 >> 172.21.15.7:0/3500255064 conn(0x55a738d2ec00 msgr2=0x55a738d2f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.420+0000 7ff28f9c8740 1 -- 172.21.15.7:0/3500255064 shutdown_connections 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.420+0000 7ff28f9c8740 1 -- 172.21.15.7:0/3500255064 wait complete. 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.420+0000 7ff28f9c8740 1 -- start start 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.420+0000 7ff28f9c8740 1 Processor -- start 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.420+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d33690 0x55a738d3dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.420+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d3e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff28659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d33690 0x55a738d3dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff28659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d33690 0x55a738d3dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56504/0 (socket says 172.21.15.7:56504) 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d391f0 0x55a738d44ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a738c55240 con 0x55a738d391f0 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a738c95460 con 0x55a738d33690 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a738d3c140 con 0x55a738d34080 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff285d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d3e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff28659c640 1 -- 172.21.15.7:0/186526362 learned_addr learned my addr 172.21.15.7:0/186526362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff28f9c8740 10 client.0 _ll_register_callbacks cb 0x55a738c54c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.421+0000 7ff286d9d640 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d391f0 0x55a738d44ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.422+0000 7ff286d9d640 1 -- 172.21.15.7:0/186526362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 msgr2=0x55a738d3e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.422+0000 7ff286d9d640 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d3e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.422+0000 7ff286d9d640 1 -- 172.21.15.7:0/186526362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d33690 msgr2=0x55a738d3dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.422+0000 7ff286d9d640 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d33690 0x55a738d3dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.422+0000 7ff286d9d640 1 -- 172.21.15.7:0/186526362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a738a777d0 con 0x55a738d391f0 2024-03-26T08:49:56.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.422+0000 7ff286d9d640 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d391f0 0x55a738d44ff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff28009a8f0 tx=0x7ff28009b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3527effccd5a8ff5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.422+0000 7ff26dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:56.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.423+0000 7ff26dffb640 1 -- 172.21.15.7:0/186526362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800b3060 con 0x55a738d391f0 2024-03-26T08:49:56.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.423+0000 7ff26dffb640 1 -- 172.21.15.7:0/186526362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff280097020 con 0x55a738d391f0 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.423+0000 7ff28f9c8740 1 -- 172.21.15.7:0/186526362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff27802cc90 con 0x55a738d391f0 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.423+0000 7ff26dffb640 1 -- 172.21.15.7:0/186526362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2800a3150 con 0x55a738d391f0 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.424+0000 7ff28f9c8740 10 client.17355 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.424+0000 7ff28f9c8740 1 -- 172.21.15.7:0/186526362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a738d48430 con 0x55a738d391f0 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.424+0000 7ff26dffb640 1 -- 172.21.15.7:0/186526362 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2800b9170 con 0x55a738d391f0 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.424+0000 7ff26dffb640 1 -- 172.21.15.7:0/186526362 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff28009bce0 con 0x55a738d391f0 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.425+0000 7ff28f9c8740 10 client.17355 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.425+0000 7ff28f9c8740 10 client.17355 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.425+0000 7ff28f9c8740 20 client.17355 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.425+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a738d49a30 0x55a738d4be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.425+0000 7ff28f9c8740 4 client.17355 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.425+0000 7ff28f9c8740 1 -- 172.21.15.7:0/186526362 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff27802cc90 con 0x55a738d49a30 2024-03-26T08:49:56.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.426+0000 7ff28659c640 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a738d49a30 0x55a738d4be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.427+0000 7ff26dffb640 10 client.17355 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:49:56.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.427+0000 7ff28659c640 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a738d49a30 0x55a738d4be70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e34d64d24f1b6ab server_cookie=7ed91ce6448e187d in_seq=0 out_seq=0 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.428+0000 7ff26dffb640 1 -- 172.21.15.7:0/186526362 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7ff27802cc90 con 0x55a738d49a30 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.428+0000 7ff26dffb640 10 client.17355 handle_command_reply: tid=0 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 client.17355 shutdown 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 -- 172.21.15.7:0/186526362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d391f0 msgr2=0x55a738d44ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d391f0 0x55a738d44ff0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff28009a8f0 tx=0x7ff28009b1f0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 -- 172.21.15.7:0/186526362 shutdown_connections 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d391f0 0x55a738d44ff0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a738d49a30 0x55a738d4be70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d3e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff26dffb640 0 client.17355 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/186526362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d33690 0x55a738d3dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 -- 172.21.15.7:0/186526362 >> 172.21.15.7:0/186526362 conn(0x55a738d2ec00 msgr2=0x55a738d2de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 -- 172.21.15.7:0/186526362 shutdown_connections 2024-03-26T08:49:56.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.429+0000 7ff28f9c8740 1 -- 172.21.15.7:0/186526362 wait complete. 2024-03-26T08:49:56.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.430+0000 7ff28f9c8740 20 client.17355 trim_cache size 0 max 16384 2024-03-26T08:49:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.450+0000 7ff28f9c8740 1 Processor -- start 2024-03-26T08:49:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.450+0000 7ff28f9c8740 1 -- start start 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.450+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.450+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d34080 0x55a738d44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.450+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d391f0 0x55a738d2dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680109d0 con 0x55a738d33690 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26800e4d0 con 0x55a738d34080 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d2d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42466/0 (socket says 172.21.15.7:42466) 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26800c550 con 0x55a738d391f0 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28659c640 1 -- 172.21.15.7:0/742313711 learned_addr learned my addr 172.21.15.7:0/742313711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff285d9b640 1 --2- 172.21.15.7:0/742313711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d34080 0x55a738d44fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff286d9d640 1 --2- 172.21.15.7:0/742313711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d391f0 0x55a738d2dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28659c640 1 -- 172.21.15.7:0/742313711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d391f0 msgr2=0x55a738d2dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28659c640 1 --2- 172.21.15.7:0/742313711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d391f0 0x55a738d2dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28659c640 1 -- 172.21.15.7:0/742313711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d34080 msgr2=0x55a738d44fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28659c640 1 --2- 172.21.15.7:0/742313711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d34080 0x55a738d44fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28659c640 1 -- 172.21.15.7:0/742313711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a738d29230 con 0x55a738d33690 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff28659c640 1 --2- 172.21.15.7:0/742313711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d2d590 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff27802c900 tx=0x7ff278000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cace28499e38a243 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff26ffff640 1 -- 172.21.15.7:0/742313711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27805f070 con 0x55a738d33690 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff26ffff640 1 -- 172.21.15.7:0/742313711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff278059030 con 0x55a738d33690 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.451+0000 7ff26ffff640 1 -- 172.21.15.7:0/742313711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278044070 con 0x55a738d33690 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.452+0000 7ff28f9c8740 1 -- 172.21.15.7:0/742313711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 msgr2=0x55a738d2d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.452+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/742313711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d2d590 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff27802c900 tx=0x7ff278000bc0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.452+0000 7ff28f9c8740 1 -- 172.21.15.7:0/742313711 shutdown_connections 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.452+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/742313711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d391f0 0x55a738d2dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.452+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/742313711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d34080 0x55a738d44fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.452+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/742313711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d2d590 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.452+0000 7ff28f9c8740 1 -- 172.21.15.7:0/742313711 >> 172.21.15.7:0/742313711 conn(0x55a738d2ec00 msgr2=0x55a738d31e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.452+0000 7ff28f9c8740 1 -- 172.21.15.7:0/742313711 shutdown_connections 2024-03-26T08:49:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.452+0000 7ff28f9c8740 1 -- 172.21.15.7:0/742313711 wait complete. 2024-03-26T08:49:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.453+0000 7ff28f9c8740 1 -- start start 2024-03-26T08:49:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.453+0000 7ff28f9c8740 1 Processor -- start 2024-03-26T08:49:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d42ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d43530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff28f9c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26800cb50 con 0x55a738d33690 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26800de70 con 0x55a738d391f0 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff28f9c8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26800d1b0 con 0x55a738d34080 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff28f9c8740 10 client.0 _ll_register_callbacks cb 0x7ff28009a3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff285d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d43530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff286d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56532/0 (socket says 172.21.15.7:56532) 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.454+0000 7ff286d9d640 1 -- 172.21.15.7:0/1223803983 learned_addr learned my addr 172.21.15.7:0/1223803983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.455+0000 7ff285d9b640 1 -- 172.21.15.7:0/1223803983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 msgr2=0x55a738d444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.455+0000 7ff285d9b640 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.455+0000 7ff285d9b640 1 -- 172.21.15.7:0/1223803983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 msgr2=0x55a738d42ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.455+0000 7ff285d9b640 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d42ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.455+0000 7ff285d9b640 1 -- 172.21.15.7:0/1223803983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a738d44ac0 con 0x55a738d34080 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.455+0000 7ff285d9b640 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d43530 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55a738d3ef50 tx=0x7ff27c015940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2acbbbb09ff041b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.455+0000 7ff26dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:49:56.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.455+0000 7ff26dffb640 1 -- 172.21.15.7:0/1223803983 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c01f020 con 0x55a738d34080 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff26dffb640 1 -- 172.21.15.7:0/1223803983 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff27c01b930 con 0x55a738d34080 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28f9c8740 1 -- 172.21.15.7:0/1223803983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff278059030 con 0x55a738d34080 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff26dffb640 1 -- 172.21.15.7:0/1223803983 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff27c01bb30 con 0x55a738d34080 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28f9c8740 10 client.27223 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28f9c8740 1 -- 172.21.15.7:0/1223803983 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a738da5b90 con 0x55a738d34080 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff26dffb640 1 -- 172.21.15.7:0/1223803983 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff278059030 con 0x55a738d34080 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff26dffb640 1 -- 172.21.15.7:0/1223803983 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff27c0183b0 con 0x55a738d34080 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28f9c8740 10 client.27223 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28f9c8740 10 client.27223 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28f9c8740 20 client.27223 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a738d49a30 0x55a738da85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28f9c8740 4 client.27223 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28f9c8740 1 -- 172.21.15.7:0/1223803983 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a738ca6900 con 0x55a738d49a30 2024-03-26T08:49:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.456+0000 7ff28659c640 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a738d49a30 0x55a738da85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28659c640 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a738d49a30 0x55a738da85b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45300b5e95b5f262 server_cookie=cfdde8dc2451494d in_seq=0 out_seq=0 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff26dffb640 10 client.27223 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff26dffb640 1 -- 172.21.15.7:0/1223803983 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a738ca6900 con 0x55a738d49a30 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff26dffb640 10 client.27223 handle_command_reply: tid=0 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 client.27223 shutdown 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 -- 172.21.15.7:0/1223803983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 msgr2=0x55a738d43530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d43530 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55a738d3ef50 tx=0x7ff27c015940 comp rx=0 tx=0).stop 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 -- 172.21.15.7:0/1223803983 shutdown_connections 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a738d391f0 0x55a738d444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55a738d49a30 0x55a738da85b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a738d34080 0x55a738d43530 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 --2- 172.21.15.7:0/1223803983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a738d33690 0x55a738d42ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff26dffb640 0 client.27223 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:49:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 -- 172.21.15.7:0/1223803983 >> 172.21.15.7:0/1223803983 conn(0x55a738d2ec00 msgr2=0x55a738d3c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 -- 172.21.15.7:0/1223803983 shutdown_connections 2024-03-26T08:49:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 1 -- 172.21.15.7:0/1223803983 wait complete. 2024-03-26T08:49:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.457+0000 7ff28f9c8740 20 client.27223 trim_cache size 0 max 16384 2024-03-26T08:49:56.456 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:49:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2700a2940 msgr2=0x7ff2700a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2700a2940 0x7ff2700a4e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff278009850 tx=0x7ff278005fd0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669255563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2881abfe0 msgr2=0x7ff2881ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2881abfe0 0x7ff2881ae3d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff28000b990 tx=0x7ff28000be50 comp rx=0 tx=0).stop 2024-03-26T08:49:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669255563 shutdown_connections 2024-03-26T08:49:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2700a2940 0x7ff2700a4e00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2881abfe0 0x7ff2881ae3d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff28810d9c0 0x7ff2881a6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 --2- 172.21.15.7:0/1669255563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff28810b010 0x7ff2881a6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669255563 >> 172.21.15.7:0/1669255563 conn(0x7ff28806e640 msgr2=0x7ff28810b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669255563 shutdown_connections 2024-03-26T08:49:56.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.458+0000 7ff28ccff640 1 -- 172.21.15.7:0/1669255563 wait complete. 2024-03-26T08:49:56.467 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:49:56.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:49:56.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:56 smithi016 ceph-mon[35851]: pgmap v585: 97 pgs: 97 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 8.3 MiB/s wr, 745 op/s 2024-03-26T08:49:56.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:56 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2386031633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:49:56.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.624+0000 7f110e55b640 1 -- 172.21.15.7:0/1414518813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11081021a0 msgr2=0x7f1108104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.624+0000 7f110e55b640 1 --2- 172.21.15.7:0/1414518813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11081021a0 0x7f1108104590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009900 tx=0x7f10f802bc30 comp rx=0 tx=0).stop 2024-03-26T08:49:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.624+0000 7f110e55b640 1 -- 172.21.15.7:0/1414518813 shutdown_connections 2024-03-26T08:49:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.624+0000 7f110e55b640 1 --2- 172.21.15.7:0/1414518813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1108107400 0x7f11081097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.624+0000 7f110e55b640 1 --2- 172.21.15.7:0/1414518813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1108104ad0 0x7f1108106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.624+0000 7f110e55b640 1 --2- 172.21.15.7:0/1414518813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11081021a0 0x7f1108104590 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.624+0000 7f110e55b640 1 -- 172.21.15.7:0/1414518813 >> 172.21.15.7:0/1414518813 conn(0x7f11080fbb30 msgr2=0x7f11080fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.624+0000 7f110e55b640 1 -- 172.21.15.7:0/1414518813 shutdown_connections 2024-03-26T08:49:56.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.624+0000 7f110e55b640 1 -- 172.21.15.7:0/1414518813 wait complete. 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f110e55b640 1 Processor -- start 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f110e55b640 1 -- start start 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f110e55b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11081021a0 0x7f11081121c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f110e55b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1108104ad0 0x7f1108112700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f110e55b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1108107400 0x7f1108110800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f110e55b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110810b130 con 0x7f1108104ad0 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f110e55b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110810afb0 con 0x7f1108107400 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f110e55b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110810b2b0 con 0x7f11081021a0 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f11077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1108104ad0 0x7f1108112700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f11077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1108104ad0 0x7f1108112700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60950/0 (socket says 172.21.15.7:60950) 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f11077fe640 1 -- 172.21.15.7:0/340512953 learned_addr learned my addr 172.21.15.7:0/340512953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f1107fff640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11081021a0 0x7f11081121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.625+0000 7f110caf8640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1108107400 0x7f1108110800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f11077fe640 1 -- 172.21.15.7:0/340512953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11081021a0 msgr2=0x7f11081121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f11077fe640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11081021a0 0x7f11081121c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f11077fe640 1 -- 172.21.15.7:0/340512953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1108107400 msgr2=0x7f1108110800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f11077fe640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1108107400 0x7f1108110800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f11077fe640 1 -- 172.21.15.7:0/340512953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1108110f30 con 0x7f1108104ad0 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f11077fe640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1108104ad0 0x7f1108112700 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00e9a0 tx=0x7f10fc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f11057fa640 1 -- 172.21.15.7:0/340512953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc00cd80 con 0x7f1108104ad0 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f110e55b640 1 -- 172.21.15.7:0/340512953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1108111220 con 0x7f1108104ad0 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f11057fa640 1 -- 172.21.15.7:0/340512953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10fc0035f0 con 0x7f1108104ad0 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f11057fa640 1 -- 172.21.15.7:0/340512953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc010670 con 0x7f1108104ad0 2024-03-26T08:49:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.626+0000 7f110e55b640 1 -- 172.21.15.7:0/340512953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11081aecd0 con 0x7f1108104ad0 2024-03-26T08:49:56.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f11057fa640 1 -- 172.21.15.7:0/340512953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f10fc0037a0 con 0x7f1108104ad0 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f1111228740 1 Processor -- start 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f1111228740 1 -- start start 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f1111228740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f11057fa640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f10f00a27f0 0x7f10f00a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f1111228740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c44070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f1111228740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c44073680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f1111228740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4402e610 con 0x558c4406c080 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f1111228740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c43fd13c0 con 0x558c440711f0 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.629+0000 7f1111228740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c43f79d60 con 0x558c4406b690 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f11077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c44070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f1107fff640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f10f00a27f0 0x7f10f00a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46954/0 (socket says 172.21.15.7:46954) 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 -- 172.21.15.7:0/736620534 learned_addr learned my addr 172.21.15.7:0/736620534 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 --2- 172.21.15.7:0/736620534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c44073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 -- 172.21.15.7:0/736620534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 msgr2=0x558c44073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 --2- 172.21.15.7:0/736620534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c44073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 -- 172.21.15.7:0/736620534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 msgr2=0x558c44070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 --2- 172.21.15.7:0/736620534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c44070b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f11057fa640 1 -- 172.21.15.7:0/340512953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f10fc0cec50 con 0x7f1108104ad0 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 -- 172.21.15.7:0/736620534 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c43ce53f0 con 0x558c4406b690 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f110caf8640 1 --2- 172.21.15.7:0/736620534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4406bab0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1100005f40 tx=0x7f1100009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bf6391794663476 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f1107fff640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f10f00a27f0 0x7f10f00a4cb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f10f8006010 tx=0x7f10f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f10ee7fc640 1 -- 172.21.15.7:0/736620534 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100011070 con 0x558c4406b690 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.630+0000 7f10ee7fc640 1 -- 172.21.15.7:0/736620534 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f110000fd60 con 0x558c4406b690 2024-03-26T08:49:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f10ee7fc640 1 -- 172.21.15.7:0/736620534 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100010c10 con 0x558c4406b690 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 -- 172.21.15.7:0/736620534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 msgr2=0x558c4406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 --2- 172.21.15.7:0/736620534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4406bab0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1100005f40 tx=0x7f1100009b80 comp rx=0 tx=0).stop 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 -- 172.21.15.7:0/736620534 shutdown_connections 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 --2- 172.21.15.7:0/736620534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c44073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 --2- 172.21.15.7:0/736620534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c44070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 --2- 172.21.15.7:0/736620534 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4406bab0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 -- 172.21.15.7:0/736620534 >> 172.21.15.7:0/736620534 conn(0x558c44066c00 msgr2=0x558c44067000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 -- 172.21.15.7:0/736620534 shutdown_connections 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 -- 172.21.15.7:0/736620534 wait complete. 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.631+0000 7f1111228740 1 -- start start 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1111228740 1 Processor -- start 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1111228740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406b690 0x558c44075a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1111228740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c4406c080 0x558c44075f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1111228740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c440711f0 0x558c4407cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1111228740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c43f8d240 con 0x558c4406b690 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1111228740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c43fcd460 con 0x558c4406c080 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1111228740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c440741f0 con 0x558c440711f0 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1107fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406b690 0x558c44075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1111228740 10 client.0 _ll_register_callbacks cb 0x558c43f8cc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1107fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406b690 0x558c44075a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60964/0 (socket says 172.21.15.7:60964) 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1107fff640 1 -- 172.21.15.7:0/269190797 learned_addr learned my addr 172.21.15.7:0/269190797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f11077fe640 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c4406c080 0x558c44075f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f110caf8640 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c440711f0 0x558c4407cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f11077fe640 1 -- 172.21.15.7:0/269190797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c440711f0 msgr2=0x558c4407cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f11077fe640 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c440711f0 0x558c4407cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f11077fe640 1 -- 172.21.15.7:0/269190797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406b690 msgr2=0x558c44075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f11077fe640 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406b690 0x558c44075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f11077fe640 1 -- 172.21.15.7:0/269190797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c43daf7d0 con 0x558c4406c080 2024-03-26T08:49:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f110caf8640 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c440711f0 0x558c4407cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:49:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.632+0000 7f1107fff640 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406b690 0x558c44075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.633+0000 7f11077fe640 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c4406c080 0x558c44075f50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0a3860 tx=0x7f10fc06a450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4f29213cee1f11d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.633+0000 7f10e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.633+0000 7f10e77fe640 1 -- 172.21.15.7:0/269190797 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0a8950 con 0x558c4406c080 2024-03-26T08:49:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.633+0000 7f10e77fe640 1 -- 172.21.15.7:0/269190797 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f10fc07a0a0 con 0x558c4406c080 2024-03-26T08:49:56.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.633+0000 7f1111228740 1 -- 172.21.15.7:0/269190797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f110000fd60 con 0x558c4406c080 2024-03-26T08:49:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.633+0000 7f10e77fe640 1 -- 172.21.15.7:0/269190797 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10fc09bcf0 con 0x558c4406c080 2024-03-26T08:49:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.635+0000 7f1111228740 10 client.27215 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.635+0000 7f1111228740 1 -- 172.21.15.7:0/269190797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558c44080370 con 0x558c4406c080 2024-03-26T08:49:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.636+0000 7f10e77fe640 1 -- 172.21.15.7:0/269190797 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f110000fd60 con 0x558c4406c080 2024-03-26T08:49:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.636+0000 7f10e77fe640 1 -- 172.21.15.7:0/269190797 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f10fc0ab320 con 0x558c4406c080 2024-03-26T08:49:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.636+0000 7f1111228740 10 client.27215 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.636+0000 7f1111228740 10 client.27215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:56.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.636+0000 7f1111228740 20 client.27215 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.636+0000 7f1111228740 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558c440818a0 0x558c44083cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.636+0000 7f1111228740 4 client.27215 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.636+0000 7f1111228740 1 -- 172.21.15.7:0/269190797 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c43fde900 con 0x558c440818a0 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.636+0000 7f1107fff640 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558c440818a0 0x558c44083cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.637+0000 7f1107fff640 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558c440818a0 0x558c44083cb0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b900e3e0eee256f server_cookie=624281956f8a611a in_seq=0 out_seq=0 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.637+0000 7f10e77fe640 10 client.27215 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.637+0000 7f10e77fe640 1 -- 172.21.15.7:0/269190797 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558c43fde900 con 0x558c440818a0 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.637+0000 7f10e77fe640 10 client.27215 handle_command_reply: tid=0 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.637+0000 7f1111228740 1 client.27215 shutdown 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 -- 172.21.15.7:0/269190797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c4406c080 msgr2=0x558c44075f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c4406c080 0x558c44075f50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0a3860 tx=0x7f10fc06a450 comp rx=0 tx=0).stop 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 -- 172.21.15.7:0/269190797 shutdown_connections 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558c440818a0 0x558c44083cb0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f10e77fe640 0 client.27215 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c440711f0 0x558c4407cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c4406c080 0x558c44075f50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 --2- 172.21.15.7:0/269190797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406b690 0x558c44075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 -- 172.21.15.7:0/269190797 >> 172.21.15.7:0/269190797 conn(0x558c44066c00 msgr2=0x558c44065e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 -- 172.21.15.7:0/269190797 shutdown_connections 2024-03-26T08:49:56.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 1 -- 172.21.15.7:0/269190797 wait complete. 2024-03-26T08:49:56.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.638+0000 7f1111228740 20 client.27215 trim_cache size 0 max 16384 2024-03-26T08:49:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.659+0000 7f1111228740 1 Processor -- start 2024-03-26T08:49:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.659+0000 7f1111228740 1 -- start start 2024-03-26T08:49:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.659+0000 7f1111228740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4406d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.659+0000 7f1111228740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.659+0000 7f1111228740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c4406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f1111228740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11081b9ee0 con 0x558c4406c080 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f1111228740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11081b79e0 con 0x558c440711f0 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f11077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f1111228740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11081b5a60 con 0x558c4406b690 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f11077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60976/0 (socket says 172.21.15.7:60976) 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f11077fe640 1 -- 172.21.15.7:0/308201412 learned_addr learned my addr 172.21.15.7:0/308201412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f11077fe640 1 -- 172.21.15.7:0/308201412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 msgr2=0x558c4406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f11077fe640 1 --2- 172.21.15.7:0/308201412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f11077fe640 1 -- 172.21.15.7:0/308201412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 msgr2=0x558c4406da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f11077fe640 1 --2- 172.21.15.7:0/308201412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c4406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.660+0000 7f11077fe640 1 -- 172.21.15.7:0/308201412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c44067970 con 0x558c4406c080 2024-03-26T08:49:56.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.661+0000 7f11077fe640 1 --2- 172.21.15.7:0/308201412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407cf50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00c6a0 tx=0x7f10fc009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44137d0fa6cdb80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.661+0000 7f10edffb640 1 -- 172.21.15.7:0/308201412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0a3d90 con 0x558c4406c080 2024-03-26T08:49:56.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.661+0000 7f10edffb640 1 -- 172.21.15.7:0/308201412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10fc06c030 con 0x558c4406c080 2024-03-26T08:49:56.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.661+0000 7f10edffb640 1 -- 172.21.15.7:0/308201412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc003c30 con 0x558c4406c080 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.662+0000 7f1111228740 1 -- 172.21.15.7:0/308201412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 msgr2=0x558c4407cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.662+0000 7f1111228740 1 --2- 172.21.15.7:0/308201412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407cf50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00c6a0 tx=0x7f10fc009540 comp rx=0 tx=0).stop 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.662+0000 7f1111228740 1 -- 172.21.15.7:0/308201412 shutdown_connections 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.662+0000 7f1111228740 1 --2- 172.21.15.7:0/308201412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c4406da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.662+0000 7f1111228740 1 --2- 172.21.15.7:0/308201412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407cf50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.662+0000 7f1111228740 1 --2- 172.21.15.7:0/308201412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.662+0000 7f1111228740 1 -- 172.21.15.7:0/308201412 >> 172.21.15.7:0/308201412 conn(0x558c44066c00 msgr2=0x558c44075a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.662+0000 7f1111228740 1 -- 172.21.15.7:0/308201412 shutdown_connections 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.662+0000 7f1111228740 1 -- 172.21.15.7:0/308201412 wait complete. 2024-03-26T08:49:56.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.663+0000 7f1111228740 1 -- start start 2024-03-26T08:49:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.663+0000 7f1111228740 1 Processor -- start 2024-03-26T08:49:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.663+0000 7f1111228740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4407adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.663+0000 7f1111228740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f1111228740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c4407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f1107fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4407adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f11077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f1107fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4407adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46994/0 (socket says 172.21.15.7:46994) 2024-03-26T08:49:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f11077fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60986/0 (socket says 172.21.15.7:60986) 2024-03-26T08:49:56.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f1107fff640 1 -- 172.21.15.7:0/3563042504 learned_addr learned my addr 172.21.15.7:0/3563042504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11081b6060 con 0x558c4406c080 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f110caf8640 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c4407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11081b7380 con 0x558c440711f0 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11081b66c0 con 0x558c4406b690 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.664+0000 7f1111228740 10 client.0 _ll_register_callbacks cb 0x7f10fc009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f110caf8640 1 -- 172.21.15.7:0/3563042504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 msgr2=0x558c4407adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f110caf8640 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4407adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f110caf8640 1 -- 172.21.15.7:0/3563042504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 msgr2=0x558c4407b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f110caf8640 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f110caf8640 1 -- 172.21.15.7:0/3563042504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c4407c8f0 con 0x558c440711f0 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f110caf8640 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c4407c300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1100009a70 tx=0x7f1100003e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fc93f166b7584bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f10e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f10e77fe640 1 -- 172.21.15.7:0/3563042504 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100009e70 con 0x558c440711f0 2024-03-26T08:49:56.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f10e77fe640 1 -- 172.21.15.7:0/3563042504 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1100011020 con 0x558c440711f0 2024-03-26T08:49:56.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10fc06c030 con 0x558c440711f0 2024-03-26T08:49:56.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f10e77fe640 1 -- 172.21.15.7:0/3563042504 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1100011240 con 0x558c440711f0 2024-03-26T08:49:56.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f1111228740 10 client.27218 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:56.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.665+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558c440dd880 con 0x558c440711f0 2024-03-26T08:49:56.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.666+0000 7f10e77fe640 1 -- 172.21.15.7:0/3563042504 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f10fc06c030 con 0x558c440711f0 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.666+0000 7f10e77fe640 1 -- 172.21.15.7:0/3563042504 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f110001d020 con 0x558c440711f0 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.666+0000 7f1111228740 10 client.27218 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.666+0000 7f1111228740 10 client.27218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.666+0000 7f1111228740 20 client.27218 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.666+0000 7f1111228740 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558c440818a0 0x558c440dff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.666+0000 7f1111228740 4 client.27218 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.666+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558c43fd8c50 con 0x558c440818a0 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.666+0000 7f1107fff640 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558c440818a0 0x558c440dff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.667+0000 7f1107fff640 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558c440818a0 0x558c440dff30 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce304eaed23daf93 server_cookie=f6fd045b5e61e46f in_seq=0 out_seq=0 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.667+0000 7f10e77fe640 10 client.27218 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.668+0000 7f10e77fe640 1 -- 172.21.15.7:0/3563042504 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558c43fd8c50 con 0x558c440818a0 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.668+0000 7f10e77fe640 10 client.27218 handle_command_reply: tid=0 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.668+0000 7f1111228740 1 client.27218 shutdown 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 msgr2=0x558c4407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c4407c300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1100009a70 tx=0x7f1100003e40 comp rx=0 tx=0).stop 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 shutdown_connections 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558c440818a0 0x558c440dff30 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c440711f0 0x558c4407c300 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558c4406c080 0x558c4407b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 --2- 172.21.15.7:0/3563042504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558c4406b690 0x558c4407adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 >> 172.21.15.7:0/3563042504 conn(0x558c44066c00 msgr2=0x558c440759d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f10e77fe640 0 client.27218 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 shutdown_connections 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 1 -- 172.21.15.7:0/3563042504 wait complete. 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.669+0000 7f1111228740 20 client.27218 trim_cache size 0 max 16384 2024-03-26T08:49:56.665 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:49:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.670+0000 7f110e55b640 1 -- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f10f00a27f0 msgr2=0x7f10f00a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.670+0000 7f110e55b640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f10f00a27f0 0x7f10f00a4cb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f10f8006010 tx=0x7f10f8000bc0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.670+0000 7f110e55b640 1 -- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1108104ad0 msgr2=0x7f1108112700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.670+0000 7f110e55b640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1108104ad0 0x7f1108112700 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00e9a0 tx=0x7f10fc00ee60 comp rx=0 tx=0).stop 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.671+0000 7f110e55b640 1 -- 172.21.15.7:0/340512953 shutdown_connections 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.671+0000 7f110e55b640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f10f00a27f0 0x7f10f00a4cb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.671+0000 7f110e55b640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1108107400 0x7f1108110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.671+0000 7f110e55b640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1108104ad0 0x7f1108112700 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.671+0000 7f110e55b640 1 --2- 172.21.15.7:0/340512953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f11081021a0 0x7f11081121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.671+0000 7f110e55b640 1 -- 172.21.15.7:0/340512953 >> 172.21.15.7:0/340512953 conn(0x7f11080fbb30 msgr2=0x7f1108102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.671+0000 7f110e55b640 1 -- 172.21.15.7:0/340512953 shutdown_connections 2024-03-26T08:49:56.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:56.671+0000 7f110e55b640 1 -- 172.21.15.7:0/340512953 wait complete. 2024-03-26T08:49:57.682 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:49:57.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:49:57.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:49:57.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.841+0000 7fe3c17b9640 1 -- 172.21.15.7:0/858611823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 msgr2=0x7fe3bc101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:57.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.841+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/858611823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 0x7fe3bc101c50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009900 tx=0x7fe3b002bc40 comp rx=0 tx=0).stop 2024-03-26T08:49:57.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.841+0000 7fe3c17b9640 1 -- 172.21.15.7:0/858611823 shutdown_connections 2024-03-26T08:49:57.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.841+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/858611823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3bc102190 0x7fe3bc108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.841+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/858611823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 0x7fe3bc101c50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.841+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/858611823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc1005e0 0x7fe3bc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.841+0000 7fe3c17b9640 1 -- 172.21.15.7:0/858611823 >> 172.21.15.7:0/858611823 conn(0x7fe3bc0fbb70 msgr2=0x7fe3bc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:57.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.844+0000 7fe3c17b9640 1 -- 172.21.15.7:0/858611823 shutdown_connections 2024-03-26T08:49:57.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.848+0000 7fe3c17b9640 1 -- 172.21.15.7:0/858611823 wait complete. 2024-03-26T08:49:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.848+0000 7fe3c17b9640 1 Processor -- start 2024-03-26T08:49:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3c17b9640 1 -- start start 2024-03-26T08:49:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3c17b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3bc1005e0 0x7fe3bc1110e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3c17b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 0x7fe3bc111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3c17b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc102190 0x7fe3bc10f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3c17b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc10a080 con 0x7fe3bc1017d0 2024-03-26T08:49:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3c17b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc109f00 con 0x7fe3bc1005e0 2024-03-26T08:49:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3c17b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc10a200 con 0x7fe3bc102190 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3ba7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 0x7fe3bc111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3ba7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 0x7fe3bc111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32778/0 (socket says 172.21.15.7:32778) 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.849+0000 7fe3ba7fc640 1 -- 172.21.15.7:0/1389785280 learned_addr learned my addr 172.21.15.7:0/1389785280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.850+0000 7fe3ba7fc640 1 -- 172.21.15.7:0/1389785280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc102190 msgr2=0x7fe3bc10f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.850+0000 7fe3ba7fc640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc102190 0x7fe3bc10f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.850+0000 7fe3ba7fc640 1 -- 172.21.15.7:0/1389785280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3bc1005e0 msgr2=0x7fe3bc1110e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.850+0000 7fe3ba7fc640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3bc1005e0 0x7fe3bc1110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.850+0000 7fe3ba7fc640 1 -- 172.21.15.7:0/1389785280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc10fe90 con 0x7fe3bc1017d0 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.851+0000 7fe3ba7fc640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 0x7fe3bc111620 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe3b002c8f0 tx=0x7fe3b0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.851+0000 7fe3a3fff640 1 -- 172.21.15.7:0/1389785280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0043020 con 0x7fe3bc1017d0 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.851+0000 7fe3c17b9640 1 -- 172.21.15.7:0/1389785280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc110120 con 0x7fe3bc1017d0 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.851+0000 7fe3c17b9640 1 -- 172.21.15.7:0/1389785280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc118f90 con 0x7fe3bc1017d0 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.851+0000 7fe3a3fff640 1 -- 172.21.15.7:0/1389785280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3b0004c60 con 0x7fe3bc1017d0 2024-03-26T08:49:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.851+0000 7fe3a3fff640 1 -- 172.21.15.7:0/1389785280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0043020 con 0x7fe3bc1017d0 2024-03-26T08:49:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.855+0000 7fe3a3fff640 1 -- 172.21.15.7:0/1389785280 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe3b0046020 con 0x7fe3bc1017d0 2024-03-26T08:49:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.856+0000 7fe3a3fff640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3a40a2bf0 0x7fe3a40a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.856+0000 7fe3a3fff640 1 -- 172.21.15.7:0/1389785280 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3b003a040 con 0x7fe3bc1017d0 2024-03-26T08:49:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.857+0000 7fe3baffd640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3a40a2bf0 0x7fe3a40a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.872+0000 7fe3c43a7740 1 Processor -- start 2024-03-26T08:49:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.874+0000 7fe3baffd640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3a40a2bf0 0x7fe3a40a50b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc110e90 tx=0x7fe3ac009630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.876+0000 7fe3c43a7740 1 -- start start 2024-03-26T08:49:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.877+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7ede7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.877+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 0x55bb7edece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.877+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edef950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.877+0000 7fe3c43a7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb7ed27c90 con 0x55bb7ede7960 2024-03-26T08:49:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.877+0000 7fe3c43a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb7ed4d070 con 0x55bb7eded4c0 2024-03-26T08:49:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.877+0000 7fe3c43a7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb7ecf5c80 con 0x55bb7ede8350 2024-03-26T08:49:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.879+0000 7fe3ba7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 0x55bb7edece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.879+0000 7fe3ba7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 0x55bb7edece50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47014/0 (socket says 172.21.15.7:47014) 2024-03-26T08:49:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.879+0000 7fe3ba7fc640 1 -- 172.21.15.7:0/2679429758 learned_addr learned my addr 172.21.15.7:0/2679429758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.880+0000 7fe3bb7fe640 1 --2- 172.21.15.7:0/2679429758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.880+0000 7fe3baffd640 1 --2- 172.21.15.7:0/2679429758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7ede7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.880+0000 7fe3bb7fe640 1 -- 172.21.15.7:0/2679429758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 msgr2=0x55bb7edece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.880+0000 7fe3bb7fe640 1 --2- 172.21.15.7:0/2679429758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 0x55bb7edece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.880+0000 7fe3bb7fe640 1 -- 172.21.15.7:0/2679429758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 msgr2=0x55bb7ede7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.880+0000 7fe3bb7fe640 1 --2- 172.21.15.7:0/2679429758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7ede7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.880+0000 7fe3bb7fe640 1 -- 172.21.15.7:0/2679429758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb7ecedd10 con 0x55bb7eded4c0 2024-03-26T08:49:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.880+0000 7fe3bb7fe640 1 --2- 172.21.15.7:0/2679429758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edef950 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4005fd0 tx=0x7fe3b400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bedf607afaf3a588 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:57.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.882+0000 7fe3a17fa640 1 -- 172.21.15.7:0/2679429758 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4004d20 con 0x55bb7eded4c0 2024-03-26T08:49:57.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.883+0000 7fe3a17fa640 1 -- 172.21.15.7:0/2679429758 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3b4003e50 con 0x55bb7eded4c0 2024-03-26T08:49:57.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.883+0000 7fe3a17fa640 1 -- 172.21.15.7:0/2679429758 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4005020 con 0x55bb7eded4c0 2024-03-26T08:49:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.884+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2679429758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 msgr2=0x55bb7edef950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:57.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.884+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2679429758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edef950 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4005fd0 tx=0x7fe3b400e720 comp rx=0 tx=0).stop 2024-03-26T08:49:57.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.888+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2679429758 shutdown_connections 2024-03-26T08:49:57.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.888+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2679429758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edef950 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.888+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2679429758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 0x55bb7edece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.888+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2679429758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7ede7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.888+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2679429758 >> 172.21.15.7:0/2679429758 conn(0x55bb7edb01a0 msgr2=0x55bb7ede32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:57.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.889+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2679429758 shutdown_connections 2024-03-26T08:49:57.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.889+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2679429758 wait complete. 2024-03-26T08:49:57.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.889+0000 7fe3c43a7740 1 -- start start 2024-03-26T08:49:57.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.889+0000 7fe3c43a7740 1 Processor -- start 2024-03-26T08:49:57.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.890+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7edf1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.890+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 0x55bb7edf2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.890+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edf92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.890+0000 7fe3c43a7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb7ed090d0 con 0x55bb7ede7960 2024-03-26T08:49:57.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.890+0000 7fe3c43a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb7ed49110 con 0x55bb7eded4c0 2024-03-26T08:49:57.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.890+0000 7fe3c43a7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb7edf0470 con 0x55bb7ede8350 2024-03-26T08:49:57.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.890+0000 7fe3c43a7740 10 client.0 _ll_register_callbacks cb 0x55bb7ed44e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:57.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.891+0000 7fe3baffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7edf1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.891+0000 7fe3baffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7edf1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32800/0 (socket says 172.21.15.7:32800) 2024-03-26T08:49:57.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.891+0000 7fe3baffd640 1 -- 172.21.15.7:0/2750693265 learned_addr learned my addr 172.21.15.7:0/2750693265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:57.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.891+0000 7fe3baffd640 1 -- 172.21.15.7:0/2750693265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 msgr2=0x55bb7edf2260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:49:57.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.891+0000 7fe3baffd640 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 0x55bb7edf2260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.891+0000 7fe3baffd640 1 -- 172.21.15.7:0/2750693265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 msgr2=0x55bb7edf92a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:49:57.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.891+0000 7fe3baffd640 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edf92a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.891+0000 7fe3baffd640 1 -- 172.21.15.7:0/2750693265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb7eb297e0 con 0x55bb7ede7960 2024-03-26T08:49:57.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.893+0000 7fe3baffd640 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7edf1d20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0106e0 tx=0x7fe3ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7b1ab8d04376da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:57.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.893+0000 7fe39eff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:57.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.894+0000 7fe39eff5640 1 -- 172.21.15.7:0/2750693265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac01c990 con 0x55bb7ede7960 2024-03-26T08:49:57.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.894+0000 7fe39eff5640 1 -- 172.21.15.7:0/2750693265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3ac01cb30 con 0x55bb7ede7960 2024-03-26T08:49:57.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.894+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2750693265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3b4003e50 con 0x55bb7ede7960 2024-03-26T08:49:57.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.894+0000 7fe39eff5640 1 -- 172.21.15.7:0/2750693265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3ac01cd30 con 0x55bb7ede7960 2024-03-26T08:49:57.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.894+0000 7fe3c43a7740 10 client.17388 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:57.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.894+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2750693265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb7eb89f70 con 0x55bb7ede7960 2024-03-26T08:49:57.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.895+0000 7fe39eff5640 1 -- 172.21.15.7:0/2750693265 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3b4003e50 con 0x55bb7ede7960 2024-03-26T08:49:57.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.895+0000 7fe39eff5640 1 -- 172.21.15.7:0/2750693265 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe3ac0325a0 con 0x55bb7ede7960 2024-03-26T08:49:57.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.895+0000 7fe3c43a7740 10 client.17388 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:57.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.895+0000 7fe3c43a7740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:57.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.895+0000 7fe3c43a7740 20 client.17388 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:57.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.895+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bb7edfdc10 0x55bb7ee000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.895+0000 7fe3c43a7740 4 client.17388 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:57.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.895+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2750693265 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb7ed5a5b0 con 0x55bb7edfdc10 2024-03-26T08:49:57.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.896+0000 7fe3ba7fc640 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bb7edfdc10 0x55bb7ee000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.897+0000 7fe3ba7fc640 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bb7edfdc10 0x55bb7ee000b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b646d7287096b35e server_cookie=d76110bf6da44345 in_seq=0 out_seq=0 2024-03-26T08:49:57.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.897+0000 7fe39eff5640 10 client.17388 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe39eff5640 1 -- 172.21.15.7:0/2750693265 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bb7ed5a5b0 con 0x55bb7edfdc10 2024-03-26T08:49:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe39eff5640 10 client.17388 handle_command_reply: tid=0 2024-03-26T08:49:57.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe3c43a7740 1 client.17388 shutdown 2024-03-26T08:49:57.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2750693265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 msgr2=0x55bb7edf1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:57.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7edf1d20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0106e0 tx=0x7fe3ac0046c0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2750693265 shutdown_connections 2024-03-26T08:49:57.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bb7edfdc10 0x55bb7ee000b0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edf92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede8350 0x55bb7edf2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2750693265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede7960 0x55bb7edf1d20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2750693265 >> 172.21.15.7:0/2750693265 conn(0x55bb7edb01a0 msgr2=0x55bb7ede2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:57.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.898+0000 7fe39eff5640 0 client.17388 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:57.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.899+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2750693265 shutdown_connections 2024-03-26T08:49:57.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.899+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2750693265 wait complete. 2024-03-26T08:49:57.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.899+0000 7fe3c43a7740 20 client.17388 trim_cache size 0 max 16384 2024-03-26T08:49:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.925+0000 7fe3c43a7740 1 Processor -- start 2024-03-26T08:49:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.925+0000 7fe3c43a7740 1 -- start start 2024-03-26T08:49:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7ede7960 0x55bb7edf2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 0x55bb7edf9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7ede7960 0x55bb7edf2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7ede7960 0x55bb7edf2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50518/0 (socket says 172.21.15.7:50518) 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7eded4c0 0x55bb7ede6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3c43a7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8010990 con 0x55bb7ede8350 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3c43a7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a800e490 con 0x55bb7ede7960 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3c43a7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a800c510 con 0x55bb7eded4c0 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3baffd640 1 -- 172.21.15.7:0/2633676067 learned_addr learned my addr 172.21.15.7:0/2633676067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3ba7fc640 1 --2- 172.21.15.7:0/2633676067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 0x55bb7edf9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.926+0000 7fe3bb7fe640 1 --2- 172.21.15.7:0/2633676067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7eded4c0 0x55bb7ede6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.927+0000 7fe3baffd640 1 -- 172.21.15.7:0/2633676067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7eded4c0 msgr2=0x55bb7ede6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.927+0000 7fe3baffd640 1 --2- 172.21.15.7:0/2633676067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7eded4c0 0x55bb7ede6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.927+0000 7fe3baffd640 1 -- 172.21.15.7:0/2633676067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 msgr2=0x55bb7edf9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.927+0000 7fe3baffd640 1 --2- 172.21.15.7:0/2633676067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 0x55bb7edf9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.927+0000 7fe3baffd640 1 -- 172.21.15.7:0/2633676067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb7ea21170 con 0x55bb7ede7960 2024-03-26T08:49:57.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.927+0000 7fe3baffd640 1 --2- 172.21.15.7:0/2633676067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7ede7960 0x55bb7edf2040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00e790 tx=0x7fe3ac005ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb27599d3c3e4d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:57.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.927+0000 7fe3a0ff9640 1 -- 172.21.15.7:0/2633676067 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac040070 con 0x55bb7ede7960 2024-03-26T08:49:57.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3a0ff9640 1 -- 172.21.15.7:0/2633676067 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3ac032b60 con 0x55bb7ede7960 2024-03-26T08:49:57.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3a0ff9640 1 -- 172.21.15.7:0/2633676067 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac037070 con 0x55bb7ede7960 2024-03-26T08:49:57.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2633676067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7ede7960 msgr2=0x55bb7edf2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2633676067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7ede7960 0x55bb7edf2040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00e790 tx=0x7fe3ac005ca0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2633676067 shutdown_connections 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2633676067 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7eded4c0 0x55bb7ede6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2633676067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 0x55bb7edf9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/2633676067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7ede7960 0x55bb7edf2040 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2633676067 >> 172.21.15.7:0/2633676067 conn(0x55bb7edb01a0 msgr2=0x55bb7edf1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2633676067 shutdown_connections 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.928+0000 7fe3c43a7740 1 -- 172.21.15.7:0/2633676067 wait complete. 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.929+0000 7fe3c43a7740 1 -- start start 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.929+0000 7fe3c43a7740 1 Processor -- start 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.929+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede7960 0x55bb7edf6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 0x55bb7edf7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3baffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede7960 0x55bb7edf6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3ba7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 0x55bb7edf7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3baffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede7960 0x55bb7edf6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47044/0 (socket says 172.21.15.7:47044) 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3ba7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 0x55bb7edf7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32804/0 (socket says 172.21.15.7:32804) 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3c43a7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edf84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3baffd640 1 -- 172.21.15.7:0/3927324723 learned_addr learned my addr 172.21.15.7:0/3927324723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a800cb10 con 0x55bb7ede8350 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a800de30 con 0x55bb7eded4c0 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a800d170 con 0x55bb7ede7960 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3bb7fe640 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edf84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3c43a7740 10 client.0 _ll_register_callbacks cb 0x7fe3ac014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3baffd640 1 -- 172.21.15.7:0/3927324723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 msgr2=0x55bb7edf84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3baffd640 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edf84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3baffd640 1 -- 172.21.15.7:0/3927324723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 msgr2=0x55bb7edf7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3baffd640 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 0x55bb7edf7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3baffd640 1 -- 172.21.15.7:0/3927324723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb7edf8a90 con 0x55bb7ede7960 2024-03-26T08:49:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.930+0000 7fe3bb7fe640 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edf84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.931+0000 7fe3baffd640 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede7960 0x55bb7edf6f50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac03e040 tx=0x7fe3ac004680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b66ef68f857fbb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.931+0000 7fe39eff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:49:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.931+0000 7fe39eff5640 1 -- 172.21.15.7:0/3927324723 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac040070 con 0x55bb7ede7960 2024-03-26T08:49:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.931+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3ac032b60 con 0x55bb7ede7960 2024-03-26T08:49:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.931+0000 7fe39eff5640 1 -- 172.21.15.7:0/3927324723 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3ac032d10 con 0x55bb7ede7960 2024-03-26T08:49:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.931+0000 7fe39eff5640 1 -- 172.21.15.7:0/3927324723 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3ac037040 con 0x55bb7ede7960 2024-03-26T08:49:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.931+0000 7fe3c43a7740 10 client.27250 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.931+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bb7ee59a80 con 0x55bb7ede7960 2024-03-26T08:49:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.931+0000 7fe39eff5640 1 -- 172.21.15.7:0/3927324723 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe3ac032b60 con 0x55bb7ede7960 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.932+0000 7fe39eff5640 1 -- 172.21.15.7:0/3927324723 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe3ac03b650 con 0x55bb7ede7960 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.932+0000 7fe3c43a7740 10 client.27250 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.932+0000 7fe3c43a7740 10 client.27250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.932+0000 7fe3c43a7740 20 client.27250 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.932+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bb7ee5a340 0x55bb7ee5c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.932+0000 7fe3c43a7740 4 client.27250 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.932+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bb7ed54900 con 0x55bb7ee5a340 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.932+0000 7fe3ba7fc640 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bb7ee5a340 0x55bb7ee5c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.933+0000 7fe39eff5640 10 client.27250 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.933+0000 7fe3ba7fc640 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bb7ee5a340 0x55bb7ee5c760 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8841b30f02ead1 server_cookie=d7c098bbebd7c824 in_seq=0 out_seq=0 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe39eff5640 1 -- 172.21.15.7:0/3927324723 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bb7ed54900 con 0x55bb7ee5a340 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe39eff5640 10 client.27250 handle_command_reply: tid=0 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 client.27250 shutdown 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede7960 msgr2=0x55bb7edf6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede7960 0x55bb7edf6f50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac03e040 tx=0x7fe3ac004680 comp rx=0 tx=0).stop 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 shutdown_connections 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bb7ee5a340 0x55bb7ee5c760 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe39eff5640 0 client.27250 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb7eded4c0 0x55bb7edf84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bb7ede8350 0x55bb7edf7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 --2- 172.21.15.7:0/3927324723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bb7ede7960 0x55bb7edf6f50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 >> 172.21.15.7:0/3927324723 conn(0x55bb7edb01a0 msgr2=0x55bb7ede1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.935+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 shutdown_connections 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c43a7740 1 -- 172.21.15.7:0/3927324723 wait complete. 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c43a7740 20 client.27250 trim_cache size 0 max 16384 2024-03-26T08:49:57.932 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:49:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:49:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "8d692580-7d82-4ba3-b750-bcf2720f917c", 2024-03-26T08:49:57.933 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:49:57.933 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:49:57.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 -- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3a40a2bf0 msgr2=0x7fe3a40a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:57.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3a40a2bf0 0x7fe3a40a50b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc110e90 tx=0x7fe3ac009630 comp rx=0 tx=0).stop 2024-03-26T08:49:57.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 -- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 msgr2=0x7fe3bc111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:57.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 0x7fe3bc111620 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe3b002c8f0 tx=0x7fe3b0005d50 comp rx=0 tx=0).stop 2024-03-26T08:49:57.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 -- 172.21.15.7:0/1389785280 shutdown_connections 2024-03-26T08:49:57.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe3a40a2bf0 0x7fe3a40a50b0 secure :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc110e90 tx=0x7fe3ac009630 comp rx=0 tx=0).stop 2024-03-26T08:49:57.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe3bc102190 0x7fe3bc10f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3bc1017d0 0x7fe3bc111620 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 --2- 172.21.15.7:0/1389785280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3bc1005e0 0x7fe3bc1110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:57.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.936+0000 7fe3c17b9640 1 -- 172.21.15.7:0/1389785280 >> 172.21.15.7:0/1389785280 conn(0x7fe3bc0fbb70 msgr2=0x7fe3bc104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:57.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.937+0000 7fe3c17b9640 1 -- 172.21.15.7:0/1389785280 shutdown_connections 2024-03-26T08:49:57.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:57.937+0000 7fe3c17b9640 1 -- 172.21.15.7:0/1389785280 wait complete. 2024-03-26T08:49:57.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:49:57.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:49:58.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.117+0000 7fc25ae7e640 1 -- 172.21.15.7:0/1314588867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254103310 msgr2=0x7fc254109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.117+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/1314588867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254103310 0x7fc254109ba0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009c10 tx=0x7fc24c02bee0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.117+0000 7fc25ae7e640 1 -- 172.21.15.7:0/1314588867 shutdown_connections 2024-03-26T08:49:58.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.117+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/1314588867 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254103310 0x7fc254109ba0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.117+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/1314588867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc254102950 0x7fc254102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.117+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/1314588867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2540ff980 0x7fc2540ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.117+0000 7fc25ae7e640 1 -- 172.21.15.7:0/1314588867 >> 172.21.15.7:0/1314588867 conn(0x7fc2540fb2b0 msgr2=0x7fc2540fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.117+0000 7fc25ae7e640 1 -- 172.21.15.7:0/1314588867 shutdown_connections 2024-03-26T08:49:58.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.117+0000 7fc25ae7e640 1 -- 172.21.15.7:0/1314588867 wait complete. 2024-03-26T08:49:58.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc25ae7e640 1 Processor -- start 2024-03-26T08:49:58.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc25ae7e640 1 -- start start 2024-03-26T08:49:58.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc25ae7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2540ff980 0x7fc254199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc25ae7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254102950 0x7fc25419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc25ae7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc254103310 0x7fc2541a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc25ae7e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25410d330 con 0x7fc254103310 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc25ae7e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25410d1b0 con 0x7fc2540ff980 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc25ae7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25410d4b0 con 0x7fc254102950 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc253fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254102950 0x7fc25419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc258c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2540ff980 0x7fc254199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc253fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254102950 0x7fc25419a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47048/0 (socket says 172.21.15.7:47048) 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc258c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2540ff980 0x7fc254199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50548/0 (socket says 172.21.15.7:50548) 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.118+0000 7fc253fff640 1 -- 172.21.15.7:0/2392944105 learned_addr learned my addr 172.21.15.7:0/2392944105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.119+0000 7fc25941b640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc254103310 0x7fc2541a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.119+0000 7fc253fff640 1 -- 172.21.15.7:0/2392944105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2540ff980 msgr2=0x7fc254199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.119+0000 7fc253fff640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2540ff980 0x7fc254199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.119+0000 7fc253fff640 1 -- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc254103310 msgr2=0x7fc2541a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.119+0000 7fc253fff640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc254103310 0x7fc2541a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.119+0000 7fc253fff640 1 -- 172.21.15.7:0/2392944105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2541a17a0 con 0x7fc254102950 2024-03-26T08:49:58.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.119+0000 7fc253fff640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254102950 0x7fc25419a0c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc24400e9c0 tx=0x7fc24400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.119+0000 7fc251ffb640 1 -- 172.21.15.7:0/2392944105 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24400ccc0 con 0x7fc254102950 2024-03-26T08:49:58.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.119+0000 7fc25ae7e640 1 -- 172.21.15.7:0/2392944105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2541a1a90 con 0x7fc254102950 2024-03-26T08:49:58.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.120+0000 7fc25ae7e640 1 -- 172.21.15.7:0/2392944105 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2541a2030 con 0x7fc254102950 2024-03-26T08:49:58.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.120+0000 7fc251ffb640 1 -- 172.21.15.7:0/2392944105 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc244005500 con 0x7fc254102950 2024-03-26T08:49:58.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.120+0000 7fc251ffb640 1 -- 172.21.15.7:0/2392944105 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244010430 con 0x7fc254102950 2024-03-26T08:49:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.124+0000 7fc25da69740 1 Processor -- start 2024-03-26T08:49:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25da69740 1 -- start start 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25da69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15900c5b0 0x55f15900c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25da69740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15900cfa0 0x55f159011aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25da69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f159012110 0x55f1590145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25da69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f158f693d0 con 0x55f15900c5b0 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25da69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f158f11ad0 con 0x55f159012110 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25da69740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f158f65470 con 0x55f15900cfa0 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25941b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15900c5b0 0x55f15900c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25941b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15900c5b0 0x55f15900c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32848/0 (socket says 172.21.15.7:32848) 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc25941b640 1 -- 172.21.15.7:0/2351555092 learned_addr learned my addr 172.21.15.7:0/2351555092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.125+0000 7fc258c1a640 1 --2- 172.21.15.7:0/2351555092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15900cfa0 0x55f159011aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc251ffb640 1 -- 172.21.15.7:0/2392944105 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc2440046c0 con 0x7fc254102950 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc258c1a640 1 -- 172.21.15.7:0/2351555092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f159012110 msgr2=0x55f1590145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc258c1a640 1 --2- 172.21.15.7:0/2351555092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f159012110 0x55f1590145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc258c1a640 1 -- 172.21.15.7:0/2351555092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15900c5b0 msgr2=0x55f15900c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc258c1a640 1 --2- 172.21.15.7:0/2351555092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15900c5b0 0x55f15900c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc258c1a640 1 -- 172.21.15.7:0/2351555092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f158d477d0 con 0x55f15900cfa0 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc25941b640 1 --2- 172.21.15.7:0/2351555092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15900c5b0 0x55f15900c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc258c1a640 1 --2- 172.21.15.7:0/2351555092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15900cfa0 0x55f159011aa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009bf0 tx=0x7fc23c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c201a7416d13f347 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc24affd640 1 -- 172.21.15.7:0/2351555092 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c009da0 con 0x55f15900cfa0 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc24affd640 1 -- 172.21.15.7:0/2351555092 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc23c005020 con 0x55f15900cfa0 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc24affd640 1 -- 172.21.15.7:0/2351555092 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c004bc0 con 0x55f15900cfa0 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.126+0000 7fc251ffb640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2400a28d0 0x7fc2400a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.127+0000 7fc25941b640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2400a28d0 0x7fc2400a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.127+0000 7fc251ffb640 1 -- 172.21.15.7:0/2392944105 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc2440ceb80 con 0x7fc254102950 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.127+0000 7fc25da69740 1 -- 172.21.15.7:0/2351555092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15900cfa0 msgr2=0x55f159011aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25da69740 1 --2- 172.21.15.7:0/2351555092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15900cfa0 0x55f159011aa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009bf0 tx=0x7fc23c011810 comp rx=0 tx=0).stop 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25da69740 1 -- 172.21.15.7:0/2351555092 shutdown_connections 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25da69740 1 --2- 172.21.15.7:0/2351555092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f159012110 0x55f1590145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25da69740 1 --2- 172.21.15.7:0/2351555092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15900cfa0 0x55f159011aa0 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009bf0 tx=0x7fc23c011810 comp rx=0 tx=0).stop 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25da69740 1 --2- 172.21.15.7:0/2351555092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15900c5b0 0x55f15900c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25da69740 1 -- 172.21.15.7:0/2351555092 >> 172.21.15.7:0/2351555092 conn(0x55f159007b20 msgr2=0x55f159007f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25941b640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2400a28d0 0x7fc2400a4d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0099a0 tx=0x7fc24c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25da69740 1 -- 172.21.15.7:0/2351555092 shutdown_connections 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25da69740 1 -- 172.21.15.7:0/2351555092 wait complete. 2024-03-26T08:49:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.128+0000 7fc25da69740 1 -- start start 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc25da69740 1 Processor -- start 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc25da69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159016d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc25da69740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f159012110 0x55f159017240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc25da69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15901c270 0x55f15901e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc25da69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1590150c0 con 0x55f15901c270 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc25da69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f158f25060 con 0x55f15900c5b0 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc25da69740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f159015240 con 0x55f159012110 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc25da69740 10 client.0 _ll_register_callbacks cb 0x55f158f076a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc258c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159016d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc253fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f159012110 0x55f159017240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc253fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f159012110 0x55f159017240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47068/0 (socket says 172.21.15.7:47068) 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc258c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159016d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50558/0 (socket says 172.21.15.7:50558) 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc253fff640 1 -- 172.21.15.7:0/272558255 learned_addr learned my addr 172.21.15.7:0/272558255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc25941b640 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15901c270 0x55f15901e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc253fff640 1 -- 172.21.15.7:0/272558255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 msgr2=0x55f159016d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc253fff640 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159016d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc253fff640 1 -- 172.21.15.7:0/272558255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15901c270 msgr2=0x55f15901e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc253fff640 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15901c270 0x55f15901e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.129+0000 7fc253fff640 1 -- 172.21.15.7:0/272558255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f158c7d3f0 con 0x55f159012110 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.130+0000 7fc253fff640 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f159012110 0x55f159017240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc24409b470 tx=0x7fc24409ae40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3929fa618143029e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.130+0000 7fc233fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.130+0000 7fc233fff640 1 -- 172.21.15.7:0/272558255 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2440a39f0 con 0x55f159012110 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.130+0000 7fc233fff640 1 -- 172.21.15.7:0/272558255 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc2440b4190 con 0x55f159012110 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.130+0000 7fc25da69740 1 -- 172.21.15.7:0/272558255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc23c005020 con 0x55f159012110 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.130+0000 7fc233fff640 1 -- 172.21.15.7:0/272558255 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc24409b4e0 con 0x55f159012110 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.130+0000 7fc25da69740 10 client.27262 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.130+0000 7fc25da69740 1 -- 172.21.15.7:0/272558255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f158c3d0f0 con 0x55f159012110 2024-03-26T08:49:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.130+0000 7fc233fff640 1 -- 172.21.15.7:0/272558255 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc23c005020 con 0x55f159012110 2024-03-26T08:49:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.131+0000 7fc233fff640 1 -- 172.21.15.7:0/272558255 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc24409f240 con 0x55f159012110 2024-03-26T08:49:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.131+0000 7fc25da69740 10 client.27262 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.131+0000 7fc25da69740 10 client.27262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.131+0000 7fc25da69740 20 client.27262 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.131+0000 7fc25da69740 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f159022ba0 0x55f159024fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.131+0000 7fc25da69740 4 client.27262 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.131+0000 7fc25da69740 1 -- 172.21.15.7:0/272558255 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f158f76910 con 0x55f159022ba0 2024-03-26T08:49:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.132+0000 7fc258c1a640 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f159022ba0 0x55f159024fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.132+0000 7fc258c1a640 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f159022ba0 0x55f159024fc0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d7afeeef61e935 server_cookie=bde032d81cd64c0a in_seq=0 out_seq=0 2024-03-26T08:49:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.134+0000 7fc233fff640 10 client.27262 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.134+0000 7fc233fff640 1 -- 172.21.15.7:0/272558255 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f158f76910 con 0x55f159022ba0 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.134+0000 7fc233fff640 10 client.27262 handle_command_reply: tid=0 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.134+0000 7fc25da69740 1 client.27262 shutdown 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.134+0000 7fc25da69740 1 -- 172.21.15.7:0/272558255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f159012110 msgr2=0x55f159017240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.134+0000 7fc25da69740 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f159012110 0x55f159017240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc24409b470 tx=0x7fc24409ae40 comp rx=0 tx=0).stop 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc25da69740 1 -- 172.21.15.7:0/272558255 shutdown_connections 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc25da69740 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f159022ba0 0x55f159024fc0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc25da69740 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f15901c270 0x55f15901e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc25da69740 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f159012110 0x55f159017240 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc233fff640 0 client.27262 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc25da69740 1 --2- 172.21.15.7:0/272558255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159016d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc25da69740 1 -- 172.21.15.7:0/272558255 >> 172.21.15.7:0/272558255 conn(0x55f159007b20 msgr2=0x55f158ffe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc25da69740 1 -- 172.21.15.7:0/272558255 shutdown_connections 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc25da69740 1 -- 172.21.15.7:0/272558255 wait complete. 2024-03-26T08:49:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.135+0000 7fc25da69740 20 client.27262 trim_cache size 0 max 16384 2024-03-26T08:49:58.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.160+0000 7fc25da69740 1 Processor -- start 2024-03-26T08:49:58.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.161+0000 7fc25da69740 1 -- start start 2024-03-26T08:49:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.161+0000 7fc25da69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159015fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.161+0000 7fc25da69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159016510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.161+0000 7fc25da69740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 0x55f158ffdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.161+0000 7fc25da69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc234010870 con 0x55f159012110 2024-03-26T08:49:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.161+0000 7fc25da69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400e370 con 0x55f15900c5b0 2024-03-26T08:49:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.161+0000 7fc25da69740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400c3f0 con 0x55f15901c270 2024-03-26T08:49:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc258c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159015fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc25941b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 0x55f158ffdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc258c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159015fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50562/0 (socket says 172.21.15.7:50562) 2024-03-26T08:49:58.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc258c1a640 1 -- 172.21.15.7:0/2712963002 learned_addr learned my addr 172.21.15.7:0/2712963002 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc253fff640 1 --2- 172.21.15.7:0/2712963002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159016510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc258c1a640 1 -- 172.21.15.7:0/2712963002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 msgr2=0x55f158ffdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc258c1a640 1 --2- 172.21.15.7:0/2712963002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 0x55f158ffdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc258c1a640 1 -- 172.21.15.7:0/2712963002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 msgr2=0x55f159016510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc258c1a640 1 --2- 172.21.15.7:0/2712963002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159016510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc258c1a640 1 -- 172.21.15.7:0/2712963002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f159025620 con 0x55f15900c5b0 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc258c1a640 1 --2- 172.21.15.7:0/2712963002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159015fd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01b040 tx=0x7fc23c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fcb99ade07fdf28 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.162+0000 7fc24a7fc640 1 -- 172.21.15.7:0/2712963002 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0042d0 con 0x55f15900c5b0 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.163+0000 7fc24a7fc640 1 -- 172.21.15.7:0/2712963002 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc23c004470 con 0x55f15900c5b0 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.163+0000 7fc24a7fc640 1 -- 172.21.15.7:0/2712963002 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c004c90 con 0x55f15900c5b0 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.163+0000 7fc25da69740 1 -- 172.21.15.7:0/2712963002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 msgr2=0x55f159015fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.163+0000 7fc25da69740 1 --2- 172.21.15.7:0/2712963002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159015fd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01b040 tx=0x7fc23c009310 comp rx=0 tx=0).stop 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.163+0000 7fc25da69740 1 -- 172.21.15.7:0/2712963002 shutdown_connections 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.163+0000 7fc25da69740 1 --2- 172.21.15.7:0/2712963002 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 0x55f158ffdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.163+0000 7fc25da69740 1 --2- 172.21.15.7:0/2712963002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159016510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.163+0000 7fc25da69740 1 --2- 172.21.15.7:0/2712963002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159015fd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.163+0000 7fc25da69740 1 -- 172.21.15.7:0/2712963002 >> 172.21.15.7:0/2712963002 conn(0x55f159007b20 msgr2=0x55f15900e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 -- 172.21.15.7:0/2712963002 shutdown_connections 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 -- 172.21.15.7:0/2712963002 wait complete. 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 -- start start 2024-03-26T08:49:58.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 Processor -- start 2024-03-26T08:49:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159035100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159033760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 0x55f159033d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400c9f0 con 0x55f159012110 2024-03-26T08:49:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400dd10 con 0x55f15900c5b0 2024-03-26T08:49:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400d050 con 0x55f15901c270 2024-03-26T08:49:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.164+0000 7fc25da69740 10 client.0 _ll_register_callbacks cb 0x7fc2440970c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc253fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159033760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc258c1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159035100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc253fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159033760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32868/0 (socket says 172.21.15.7:32868) 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc253fff640 1 -- 172.21.15.7:0/1922716724 learned_addr learned my addr 172.21.15.7:0/1922716724 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc25941b640 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 0x55f159033d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc253fff640 1 -- 172.21.15.7:0/1922716724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 msgr2=0x55f159033d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc253fff640 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 0x55f159033d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc253fff640 1 -- 172.21.15.7:0/1922716724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 msgr2=0x55f159035100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc253fff640 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159035100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc253fff640 1 -- 172.21.15.7:0/1922716724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f15907d8c0 con 0x55f159012110 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc258c1a640 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159035100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc253fff640 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159033760 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc244004ba0 tx=0x7fc244009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f92201926ac1ef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc233fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc233fff640 1 -- 172.21.15.7:0/1922716724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24409b2b0 con 0x55f159012110 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc25da69740 1 -- 172.21.15.7:0/1922716724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc23c004470 con 0x55f159012110 2024-03-26T08:49:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc233fff640 1 -- 172.21.15.7:0/1922716724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc24409f120 con 0x55f159012110 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc233fff640 1 -- 172.21.15.7:0/1922716724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc24409f630 con 0x55f159012110 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc25da69740 10 client.17412 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.165+0000 7fc25da69740 1 -- 172.21.15.7:0/1922716724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f15907e9e0 con 0x55f159012110 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.166+0000 7fc233fff640 1 -- 172.21.15.7:0/1922716724 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc23c004470 con 0x55f159012110 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.166+0000 7fc233fff640 1 -- 172.21.15.7:0/1922716724 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc244004fd0 con 0x55f159012110 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.166+0000 7fc25da69740 10 client.17412 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.166+0000 7fc25da69740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.166+0000 7fc25da69740 20 client.17412 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.166+0000 7fc25da69740 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f15907eec0 0x55f1590812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.166+0000 7fc25da69740 4 client.17412 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.166+0000 7fc25da69740 1 -- 172.21.15.7:0/1922716724 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f158f70c60 con 0x55f15907eec0 2024-03-26T08:49:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.166+0000 7fc258c1a640 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f15907eec0 0x55f1590812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.168+0000 7fc258c1a640 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f15907eec0 0x55f1590812e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67099c6333cf8f8d server_cookie=f04f5ff795112cad in_seq=0 out_seq=0 2024-03-26T08:49:58.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.168+0000 7fc233fff640 10 client.17412 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:58.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.168+0000 7fc233fff640 1 -- 172.21.15.7:0/1922716724 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f158f70c60 con 0x55f15907eec0 2024-03-26T08:49:58.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.168+0000 7fc233fff640 10 client.17412 handle_command_reply: tid=0 2024-03-26T08:49:58.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.168+0000 7fc25da69740 1 client.17412 shutdown 2024-03-26T08:49:58.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.168+0000 7fc25da69740 1 -- 172.21.15.7:0/1922716724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 msgr2=0x55f159033760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159033760 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc244004ba0 tx=0x7fc244009f90 comp rx=0 tx=0).stop 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 1 -- 172.21.15.7:0/1922716724 shutdown_connections 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f15907eec0 0x55f1590812e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f15901c270 0x55f159033d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f159012110 0x55f159033760 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 1 --2- 172.21.15.7:0/1922716724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f15900c5b0 0x55f159035100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 1 -- 172.21.15.7:0/1922716724 >> 172.21.15.7:0/1922716724 conn(0x55f159007b20 msgr2=0x55f15900b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc233fff640 0 client.17412 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 1 -- 172.21.15.7:0/1922716724 shutdown_connections 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 1 -- 172.21.15.7:0/1922716724 wait complete. 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.169+0000 7fc25da69740 20 client.17412 trim_cache size 0 max 16384 2024-03-26T08:49:58.166 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:49:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:49:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:49:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "8d692580-7d82-4ba3-b750-bcf2720f917c": { 2024-03-26T08:49:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:49:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "8d692580-7d82-4ba3-b750-bcf2720f917c", 2024-03-26T08:49:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:49:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:49:58.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:49:58.167 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:49:58.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 -- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2400a28d0 msgr2=0x7fc2400a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2400a28d0 0x7fc2400a4d90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0099a0 tx=0x7fc24c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 -- 172.21.15.7:0/2392944105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254102950 msgr2=0x7fc25419a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254102950 0x7fc25419a0c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc24400e9c0 tx=0x7fc24400ee80 comp rx=0 tx=0).stop 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 -- 172.21.15.7:0/2392944105 shutdown_connections 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc2400a28d0 0x7fc2400a4d90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc254103310 0x7fc2541a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc254102950 0x7fc25419a0c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 --2- 172.21.15.7:0/2392944105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2540ff980 0x7fc254199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 -- 172.21.15.7:0/2392944105 >> 172.21.15.7:0/2392944105 conn(0x7fc2540fb2b0 msgr2=0x7fc254100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 -- 172.21.15.7:0/2392944105 shutdown_connections 2024-03-26T08:49:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:49:58.170+0000 7fc25ae7e640 1 -- 172.21.15.7:0/2392944105 wait complete. 2024-03-26T08:49:58.179 INFO:tasks.cephfs.filesystem:scrub status for tag:8d692580-7d82-4ba3-b750-bcf2720f917c - {'path': '/', 'tag': '8d692580-7d82-4ba3-b750-bcf2720f917c', 'options': 'recursive,force'} 2024-03-26T08:49:58.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:58 smithi007 ceph-mon[32462]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 MiB/s wr, 815 op/s 2024-03-26T08:49:58.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:58 smithi044 ceph-mon[36461]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 MiB/s wr, 815 op/s 2024-03-26T08:49:58.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:58 smithi016 ceph-mon[35851]: pgmap v586: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.8 MiB/s wr, 815 op/s 2024-03-26T08:49:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.597+0000 7fa54bfff640 1 -- 172.21.15.7:0/2848328611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54c1004d0 msgr2=0x7fa54c100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.597+0000 7fa54bfff640 1 --2- 172.21.15.7:0/2848328611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54c1004d0 0x7fa54c100930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009900 tx=0x7fa53c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:49:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.597+0000 7fa54bfff640 1 -- 172.21.15.7:0/2848328611 shutdown_connections 2024-03-26T08:49:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.597+0000 7fa54bfff640 1 --2- 172.21.15.7:0/2848328611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa54c100e70 0x7fa54c105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.597+0000 7fa54bfff640 1 --2- 172.21.15.7:0/2848328611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54c1004d0 0x7fa54c100930 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.597+0000 7fa54bfff640 1 --2- 172.21.15.7:0/2848328611 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa54c0ff3a0 0x7fa54c0ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.597+0000 7fa54bfff640 1 -- 172.21.15.7:0/2848328611 >> 172.21.15.7:0/2848328611 conn(0x7fa54c0faca0 msgr2=0x7fa54c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.597+0000 7fa54bfff640 1 -- 172.21.15.7:0/2848328611 shutdown_connections 2024-03-26T08:49:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.597+0000 7fa54bfff640 1 -- 172.21.15.7:0/2848328611 wait complete. 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.598+0000 7fa54bfff640 1 Processor -- start 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54bfff640 1 -- start start 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa54c0ff3a0 0x7fa54c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa54c1004d0 0x7fa54c19e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54c100e70 0x7fa54c1a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54bfff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c113740 con 0x7fa54c100e70 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1135c0 con 0x7fa54c1004d0 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54bfff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1138c0 con 0x7fa54c0ff3a0 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54c100e70 0x7fa54c1a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa54c1004d0 0x7fa54c19e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa54c0ff3a0 0x7fa54c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54c100e70 0x7fa54c1a53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32884/0 (socket says 172.21.15.7:32884) 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa54c1004d0 0x7fa54c19e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50590/0 (socket says 172.21.15.7:50590) 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54b7fe640 1 -- 172.21.15.7:0/3313028703 learned_addr learned my addr 172.21.15.7:0/3313028703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54affd640 1 -- 172.21.15.7:0/3313028703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa54c1004d0 msgr2=0x7fa54c19e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54affd640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa54c1004d0 0x7fa54c19e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54affd640 1 -- 172.21.15.7:0/3313028703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54c100e70 msgr2=0x7fa54c1a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54affd640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54c100e70 0x7fa54c1a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54affd640 1 -- 172.21.15.7:0/3313028703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c1a5a30 con 0x7fa54c0ff3a0 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.599+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa54c1004d0 0x7fa54c19e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.600+0000 7fa54affd640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa54c0ff3a0 0x7fa54c19de10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9f0 tx=0x7fa54000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.600+0000 7fa533fff640 1 -- 172.21.15.7:0/3313028703 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54000ccc0 con 0x7fa54c0ff3a0 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.600+0000 7fa533fff640 1 -- 172.21.15.7:0/3313028703 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa540010910 con 0x7fa54c0ff3a0 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.600+0000 7fa54bfff640 1 -- 172.21.15.7:0/3313028703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54c1a5d20 con 0x7fa54c0ff3a0 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.600+0000 7fa533fff640 1 -- 172.21.15.7:0/3313028703 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540018960 con 0x7fa54c0ff3a0 2024-03-26T08:49:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.600+0000 7fa54bfff640 1 -- 172.21.15.7:0/3313028703 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c1a6190 con 0x7fa54c0ff3a0 2024-03-26T08:49:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.603+0000 7fa533fff640 1 -- 172.21.15.7:0/3313028703 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa540018b00 con 0x7fa54c0ff3a0 2024-03-26T08:49:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.603+0000 7fa533fff640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5340a27e0 0x7fa5340a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.604+0000 7fa533fff640 1 -- 172.21.15.7:0/3313028703 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa5400cfd40 con 0x7fa54c0ff3a0 2024-03-26T08:49:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.604+0000 7fa553fea740 1 Processor -- start 2024-03-26T08:49:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa553fea740 1 -- start start 2024-03-26T08:49:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa553fea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef60b20 0x5556bef60f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa553fea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef61510 0x5556bef66010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa553fea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 0x5556bef68b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa553fea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556beec5b10 con 0x5556bef61510 2024-03-26T08:49:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa553fea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556bee6e900 con 0x5556bef66680 2024-03-26T08:49:58.601 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa553fea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556beec1bb0 con 0x5556bef60b20 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa54a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef61510 0x5556bef66010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa54b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 0x5556bef68b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa54a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef61510 0x5556bef66010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32886/0 (socket says 172.21.15.7:32886) 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.605+0000 7fa54a7fc640 1 -- 172.21.15.7:0/3184829069 learned_addr learned my addr 172.21.15.7:0/3184829069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.606+0000 7fa54a7fc640 1 -- 172.21.15.7:0/3184829069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef60b20 msgr2=0x5556bef60f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.606+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/3184829069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef60b20 0x5556bef60f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.606+0000 7fa54a7fc640 1 -- 172.21.15.7:0/3184829069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 msgr2=0x5556bef68b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.606+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/3184829069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 0x5556bef68b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.606+0000 7fa54a7fc640 1 -- 172.21.15.7:0/3184829069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556bebda210 con 0x5556bef61510 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.606+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5340a27e0 0x7fa5340a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.606+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/3184829069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef61510 0x5556bef66010 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa53c036040 tx=0x7fa53c02cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c652dba1ba6169a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.606+0000 7fa5317fa640 1 -- 172.21.15.7:0/3184829069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53c04f070 con 0x5556bef61510 2024-03-26T08:49:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.607+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5340a27e0 0x7fa5340a4ca0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa53c02cfd0 tx=0x7fa53c04d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.607+0000 7fa5317fa640 1 -- 172.21.15.7:0/3184829069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa53c0046c0 con 0x5556bef61510 2024-03-26T08:49:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.608+0000 7fa553fea740 1 -- 172.21.15.7:0/3184829069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef61510 msgr2=0x5556bef66010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.608+0000 7fa553fea740 1 --2- 172.21.15.7:0/3184829069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef61510 0x5556bef66010 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa53c036040 tx=0x7fa53c02cdb0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.608+0000 7fa553fea740 1 -- 172.21.15.7:0/3184829069 shutdown_connections 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.608+0000 7fa553fea740 1 --2- 172.21.15.7:0/3184829069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 0x5556bef68b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.608+0000 7fa553fea740 1 --2- 172.21.15.7:0/3184829069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef61510 0x5556bef66010 secure :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa53c036040 tx=0x7fa53c02cdb0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.608+0000 7fa553fea740 1 --2- 172.21.15.7:0/3184829069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef60b20 0x5556bef60f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.608+0000 7fa553fea740 1 -- 172.21.15.7:0/3184829069 >> 172.21.15.7:0/3184829069 conn(0x5556bef5c090 msgr2=0x5556bef5c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.609+0000 7fa553fea740 1 -- 172.21.15.7:0/3184829069 shutdown_connections 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.609+0000 7fa553fea740 1 -- 172.21.15.7:0/3184829069 wait complete. 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.609+0000 7fa553fea740 1 -- start start 2024-03-26T08:49:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.609+0000 7fa553fea740 1 Processor -- start 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa553fea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef6b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa553fea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef6b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa553fea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 0x5556bef72c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa553fea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556bef69680 con 0x5556bef66680 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa553fea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556bee81d50 con 0x5556bef60b20 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa553fea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556bef69800 con 0x5556bef70860 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa553fea740 10 client.0 _ll_register_callbacks cb 0x5556bee81720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef6b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef6b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32898/0 (socket says 172.21.15.7:32898) 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54a7fc640 1 -- 172.21.15.7:0/2661544425 learned_addr learned my addr 172.21.15.7:0/2661544425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54affd640 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef6b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54b7fe640 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 0x5556bef72c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54affd640 1 -- 172.21.15.7:0/2661544425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 msgr2=0x5556bef72c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54affd640 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 0x5556bef72c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54affd640 1 -- 172.21.15.7:0/2661544425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 msgr2=0x5556bef6b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54affd640 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef6b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.610+0000 7fa54affd640 1 -- 172.21.15.7:0/2661544425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556beca6620 con 0x5556bef60b20 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.611+0000 7fa54affd640 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef6b2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5556bef61d70 tx=0x7fa54009c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21dff51d652b576d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.611+0000 7fa52eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.611+0000 7fa52eff5640 1 -- 172.21.15.7:0/2661544425 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54009aa90 con 0x5556bef60b20 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.611+0000 7fa52eff5640 1 -- 172.21.15.7:0/2661544425 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa540087070 con 0x5556bef60b20 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.611+0000 7fa553fea740 1 -- 172.21.15.7:0/2661544425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556beed3050 con 0x5556bef60b20 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.611+0000 7fa52eff5640 1 -- 172.21.15.7:0/2661544425 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa540010ac0 con 0x5556bef60b20 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.611+0000 7fa553fea740 10 client.27257 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.611+0000 7fa553fea740 1 -- 172.21.15.7:0/2661544425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5556bef760c0 con 0x5556bef60b20 2024-03-26T08:49:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.612+0000 7fa52eff5640 1 -- 172.21.15.7:0/2661544425 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5556beed3050 con 0x5556bef60b20 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.612+0000 7fa52eff5640 1 -- 172.21.15.7:0/2661544425 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa5400984e0 con 0x5556bef60b20 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.612+0000 7fa553fea740 10 client.27257 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.612+0000 7fa553fea740 10 client.27257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.612+0000 7fa553fea740 20 client.27257 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.612+0000 7fa553fea740 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5556bef776a0 0x5556bef79a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.612+0000 7fa553fea740 4 client.27257 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.612+0000 7fa553fea740 1 -- 172.21.15.7:0/2661544425 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556beecd3a0 con 0x5556bef776a0 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.612+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5556bef776a0 0x5556bef79a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.613+0000 7fa52eff5640 10 client.27257 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.613+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5556bef776a0 0x5556bef79a90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8177ab39a960bff server_cookie=bb35b76b9a661671 in_seq=0 out_seq=0 2024-03-26T08:49:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa52eff5640 1 -- 172.21.15.7:0/2661544425 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5556beecd3a0 con 0x5556bef776a0 2024-03-26T08:49:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa52eff5640 10 client.27257 handle_command_reply: tid=0 2024-03-26T08:49:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 client.27257 shutdown 2024-03-26T08:49:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 -- 172.21.15.7:0/2661544425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 msgr2=0x5556bef6b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef6b2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x5556bef61d70 tx=0x7fa54009c790 comp rx=0 tx=0).stop 2024-03-26T08:49:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 -- 172.21.15.7:0/2661544425 shutdown_connections 2024-03-26T08:49:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5556bef776a0 0x5556bef79a90 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 0x5556bef72c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef6b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 --2- 172.21.15.7:0/2661544425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef6b2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa52eff5640 0 client.27257 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 -- 172.21.15.7:0/2661544425 >> 172.21.15.7:0/2661544425 conn(0x5556bef5c090 msgr2=0x5556bef5b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 -- 172.21.15.7:0/2661544425 shutdown_connections 2024-03-26T08:49:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 1 -- 172.21.15.7:0/2661544425 wait complete. 2024-03-26T08:49:58.613 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.614+0000 7fa553fea740 20 client.27257 trim_cache size 0 max 16384 2024-03-26T08:49:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa553fea740 1 Processor -- start 2024-03-26T08:49:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa553fea740 1 -- start start 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa553fea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef60b20 0x5556bef62c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa553fea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 0x5556bef63170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa553fea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef70860 0x5556bef6af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa553fea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1b1480 con 0x5556bef70860 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa553fea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1aef80 con 0x5556bef66680 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa553fea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1ad000 con 0x5556bef60b20 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef70860 0x5556bef6af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef70860 0x5556bef6af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32914/0 (socket says 172.21.15.7:32914) 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef60b20 0x5556bef62c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54b7fe640 1 -- 172.21.15.7:0/1024051008 learned_addr learned my addr 172.21.15.7:0/1024051008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/1024051008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 0x5556bef63170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54b7fe640 1 -- 172.21.15.7:0/1024051008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef60b20 msgr2=0x5556bef62c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54b7fe640 1 --2- 172.21.15.7:0/1024051008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef60b20 0x5556bef62c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54b7fe640 1 -- 172.21.15.7:0/1024051008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 msgr2=0x5556bef63170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54b7fe640 1 --2- 172.21.15.7:0/1024051008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 0x5556bef63170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.633+0000 7fa54b7fe640 1 -- 172.21.15.7:0/1024051008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556bef6c470 con 0x5556bef70860 2024-03-26T08:49:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.634+0000 7fa54b7fe640 1 --2- 172.21.15.7:0/1024051008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef70860 0x5556bef6af00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa544011790 tx=0x7fa544011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5664156bafe5d46f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.634+0000 7fa530ff9640 1 -- 172.21.15.7:0/1024051008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544004b20 con 0x5556bef70860 2024-03-26T08:49:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.634+0000 7fa530ff9640 1 -- 172.21.15.7:0/1024051008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa544010df0 con 0x5556bef70860 2024-03-26T08:49:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.634+0000 7fa530ff9640 1 -- 172.21.15.7:0/1024051008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa544005830 con 0x5556bef70860 2024-03-26T08:49:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 -- 172.21.15.7:0/1024051008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef70860 msgr2=0x5556bef6af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 --2- 172.21.15.7:0/1024051008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef70860 0x5556bef6af00 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa544011790 tx=0x7fa544011c50 comp rx=0 tx=0).stop 2024-03-26T08:49:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 -- 172.21.15.7:0/1024051008 shutdown_connections 2024-03-26T08:49:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 --2- 172.21.15.7:0/1024051008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef70860 0x5556bef6af00 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 --2- 172.21.15.7:0/1024051008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef66680 0x5556bef63170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 --2- 172.21.15.7:0/1024051008 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef60b20 0x5556bef62c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 -- 172.21.15.7:0/1024051008 >> 172.21.15.7:0/1024051008 conn(0x5556bef5c090 msgr2=0x5556bef5c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 -- 172.21.15.7:0/1024051008 shutdown_connections 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 -- 172.21.15.7:0/1024051008 wait complete. 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.635+0000 7fa553fea740 1 -- start start 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa553fea740 1 Processor -- start 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa553fea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef87630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa553fea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef87b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa553fea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 0x5556bef88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa553fea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1ad600 con 0x5556bef66680 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa553fea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1ae920 con 0x5556bef60b20 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef87b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef87b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32920/0 (socket says 172.21.15.7:32920) 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef87630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54a7fc640 1 -- 172.21.15.7:0/2238893177 learned_addr learned my addr 172.21.15.7:0/2238893177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa553fea740 1 -- 172.21.15.7:0/2238893177 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c1adc60 con 0x5556bef70860 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54b7fe640 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 0x5556bef88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa553fea740 10 client.0 _ll_register_callbacks cb 0x7fa54009c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54affd640 1 -- 172.21.15.7:0/2238893177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 msgr2=0x5556bef88b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54affd640 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 0x5556bef88b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54affd640 1 -- 172.21.15.7:0/2238893177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 msgr2=0x5556bef87b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54affd640 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef87b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.636+0000 7fa54affd640 1 -- 172.21.15.7:0/2238893177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556bef89170 con 0x5556bef60b20 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.637+0000 7fa54affd640 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef87630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c6a0 tx=0x7fa540005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21a7d30c9bbab2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.637+0000 7fa52eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.637+0000 7fa52eff5640 1 -- 172.21.15.7:0/2238893177 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540080060 con 0x5556bef60b20 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.637+0000 7fa52eff5640 1 -- 172.21.15.7:0/2238893177 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa54009bbe0 con 0x5556bef60b20 2024-03-26T08:49:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.637+0000 7fa553fea740 1 -- 172.21.15.7:0/2238893177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa544010df0 con 0x5556bef60b20 2024-03-26T08:49:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.637+0000 7fa52eff5640 1 -- 172.21.15.7:0/2238893177 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa540010c60 con 0x5556bef60b20 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa553fea740 10 client.27263 fetch_fsmap learned FSMap version 25 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa553fea740 1 -- 172.21.15.7:0/2238893177 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5556befd2710 con 0x5556bef60b20 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa52eff5640 1 -- 172.21.15.7:0/2238893177 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5400a8070 con 0x5556bef60b20 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa52eff5640 1 -- 172.21.15.7:0/2238893177 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa5400a1750 con 0x5556bef60b20 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa553fea740 10 client.27263 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa553fea740 10 client.27263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa553fea740 20 client.27263 populate_metadata read hostname 'smithi007' 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa553fea740 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5556befd2f80 0x5556befd53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa553fea740 4 client.27263 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.639+0000 7fa553fea740 1 -- 172.21.15.7:0/2238893177 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fa544010df0 con 0x5556befd2f80 2024-03-26T08:49:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.640+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5556befd2f80 0x5556befd53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:49:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.640+0000 7fa52eff5640 10 client.27263 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.640+0000 7fa54a7fc640 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5556befd2f80 0x5556befd53a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eecf2e5e0ca14844 server_cookie=764cab1b6dba4a0a in_seq=0 out_seq=0 2024-03-26T08:49:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.656+0000 7fa52eff5640 1 -- 172.21.15.7:0/2238893177 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fa544010df0 con 0x5556befd2f80 2024-03-26T08:49:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.656+0000 7fa52eff5640 10 client.27263 handle_command_reply: tid=0 2024-03-26T08:49:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.656+0000 7fa553fea740 1 client.27263 shutdown 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.656+0000 7fa553fea740 1 -- 172.21.15.7:0/2238893177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 msgr2=0x5556bef87630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.656+0000 7fa553fea740 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef87630 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa54000c6a0 tx=0x7fa540005f10 comp rx=0 tx=0).stop 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.656+0000 7fa553fea740 1 -- 172.21.15.7:0/2238893177 shutdown_connections 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.656+0000 7fa553fea740 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5556befd2f80 0x5556befd53a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa553fea740 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5556bef70860 0x5556bef88b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa52eff5640 0 client.27263 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa553fea740 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5556bef66680 0x5556bef87b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa553fea740 1 --2- 172.21.15.7:0/2238893177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5556bef60b20 0x5556bef87630 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa553fea740 1 -- 172.21.15.7:0/2238893177 >> 172.21.15.7:0/2238893177 conn(0x5556bef5c090 msgr2=0x5556bef62c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa553fea740 1 -- 172.21.15.7:0/2238893177 shutdown_connections 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa553fea740 1 -- 172.21.15.7:0/2238893177 wait complete. 2024-03-26T08:49:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa553fea740 20 client.27263 trim_cache size 0 max 16384 2024-03-26T08:49:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:49:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:49:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:49:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 -- 172.21.15.7:0/3313028703 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5340a27e0 msgr2=0x7fa5340a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5340a27e0 0x7fa5340a4ca0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa53c02cfd0 tx=0x7fa53c04d000 comp rx=0 tx=0).stop 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 -- 172.21.15.7:0/3313028703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa54c0ff3a0 msgr2=0x7fa54c19de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa54c0ff3a0 0x7fa54c19de10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa54000e9f0 tx=0x7fa54000eeb0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 -- 172.21.15.7:0/3313028703 shutdown_connections 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5340a27e0 0x7fa5340a4ca0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54c100e70 0x7fa54c1a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa54c1004d0 0x7fa54c19e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 --2- 172.21.15.7:0/3313028703 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa54c0ff3a0 0x7fa54c19de10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.657+0000 7fa54bfff640 1 -- 172.21.15.7:0/3313028703 >> 172.21.15.7:0/3313028703 conn(0x7fa54c0faca0 msgr2=0x7fa54c0fb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.658+0000 7fa54bfff640 1 -- 172.21.15.7:0/3313028703 shutdown_connections 2024-03-26T08:49:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:49:58.658+0000 7fa54bfff640 1 -- 172.21.15.7:0/3313028703 wait complete. 2024-03-26T08:49:59.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:59 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:49:59.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:59 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:59.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:59 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:49:59.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:59 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:49:59.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:59 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:59.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:49:59 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:59.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:59 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:49:59.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:59 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:59.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:59 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:49:59.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:59 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:49:59.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:59 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:59.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:49:59 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:59.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:59 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:49:59.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:59 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:59.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:59 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:49:59.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:59 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:49:59.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:59 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:49:59.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:49:59 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:00.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:00 smithi044 ceph-mon[36461]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 845 op/s 2024-03-26T08:50:00.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:00 smithi044 ceph-mon[36461]: overall HEALTH_OK 2024-03-26T08:50:00.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:00 smithi016 ceph-mon[35851]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 845 op/s 2024-03-26T08:50:00.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:00 smithi016 ceph-mon[35851]: overall HEALTH_OK 2024-03-26T08:50:00.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:00 smithi007 ceph-mon[32462]: pgmap v587: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 845 op/s 2024-03-26T08:50:00.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:00 smithi007 ceph-mon[32462]: overall HEALTH_OK 2024-03-26T08:50:01.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 450.0 s, 2935.8 tps, lat 34.068 ms stddev 40.189 2024-03-26T08:50:02.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:02 smithi044 ceph-mon[36461]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.4 MiB/s wr, 827 op/s 2024-03-26T08:50:02.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:02 smithi016 ceph-mon[35851]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.4 MiB/s wr, 827 op/s 2024-03-26T08:50:02.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:02 smithi007 ceph-mon[32462]: pgmap v588: 97 pgs: 97 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 9.4 MiB/s wr, 827 op/s 2024-03-26T08:50:04.245 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:04 smithi016 ceph-mon[35851]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 10 MiB/s wr, 962 op/s 2024-03-26T08:50:04.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:04 smithi044 ceph-mon[36461]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 10 MiB/s wr, 962 op/s 2024-03-26T08:50:04.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:04 smithi007 ceph-mon[32462]: pgmap v589: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 10 MiB/s wr, 962 op/s 2024-03-26T08:50:06.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 455.0 s, 2238.6 tps, lat 44.631 ms stddev 48.234 2024-03-26T08:50:06.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:06 smithi044 ceph-mon[36461]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.0 MiB/s wr, 869 op/s 2024-03-26T08:50:06.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:06 smithi016 ceph-mon[35851]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.0 MiB/s wr, 869 op/s 2024-03-26T08:50:06.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:06 smithi007 ceph-mon[32462]: pgmap v590: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.0 MiB/s wr, 869 op/s 2024-03-26T08:50:07.318 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:07 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:50:07.319 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:07 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:50:07.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:07 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:50:07.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:07 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:50:07.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:07 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:50:07.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:07 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:50:08.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:08 smithi016 ceph-mon[35851]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.3 MiB/s wr, 847 op/s 2024-03-26T08:50:08.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:08 smithi007 ceph-mon[32462]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.3 MiB/s wr, 847 op/s 2024-03-26T08:50:08.568 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:08 smithi044 ceph-mon[36461]: pgmap v591: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.3 MiB/s wr, 847 op/s 2024-03-26T08:50:10.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:10 smithi016 ceph-mon[35851]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.6 MiB/s wr, 743 op/s 2024-03-26T08:50:10.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:10 smithi007 ceph-mon[32462]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.6 MiB/s wr, 743 op/s 2024-03-26T08:50:10.568 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:10 smithi044 ceph-mon[36461]: pgmap v592: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.6 MiB/s wr, 743 op/s 2024-03-26T08:50:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 460.0 s, 1848.8 tps, lat 53.881 ms stddev 60.941 2024-03-26T08:50:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.789+0000 7fc4daa8c640 1 -- 172.21.15.7:0/3040911184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40ff750 msgr2=0x7fc4d40ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.789+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/3040911184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40ff750 0x7fc4d40ffbd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009900 tx=0x7fc4c002bc40 comp rx=0 tx=0).stop 2024-03-26T08:50:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.789+0000 7fc4daa8c640 1 -- 172.21.15.7:0/3040911184 shutdown_connections 2024-03-26T08:50:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.789+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/3040911184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d4100110 0x7fc4d4109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.789+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/3040911184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40ff750 0x7fc4d40ffbd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.789+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/3040911184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d4102ce0 0x7fc4d4103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.789+0000 7fc4daa8c640 1 -- 172.21.15.7:0/3040911184 >> 172.21.15.7:0/3040911184 conn(0x7fc4d40fb2f0 msgr2=0x7fc4d40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.790+0000 7fc4daa8c640 1 -- 172.21.15.7:0/3040911184 shutdown_connections 2024-03-26T08:50:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.790+0000 7fc4daa8c640 1 -- 172.21.15.7:0/3040911184 wait complete. 2024-03-26T08:50:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.790+0000 7fc4daa8c640 1 Processor -- start 2024-03-26T08:50:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4daa8c640 1 -- start start 2024-03-26T08:50:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4daa8c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d40ff750 0x7fc4d41023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4daa8c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d4100110 0x7fc4d4100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4daa8c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d4102ce0 0x7fc4d4100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4daa8c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d410d390 con 0x7fc4d4102ce0 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4daa8c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d410d210 con 0x7fc4d4100110 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4daa8c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d410d510 con 0x7fc4d40ff750 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d8828640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d40ff750 0x7fc4d41023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d8828640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d40ff750 0x7fc4d41023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58432/0 (socket says 172.21.15.7:58432) 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d8828640 1 -- 172.21.15.7:0/2980226762 learned_addr learned my addr 172.21.15.7:0/2980226762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d4100110 0x7fc4d4100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d9029640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d4102ce0 0x7fc4d4100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d3fff640 1 -- 172.21.15.7:0/2980226762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d40ff750 msgr2=0x7fc4d41023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d40ff750 0x7fc4d41023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d3fff640 1 -- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d4102ce0 msgr2=0x7fc4d4100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d4102ce0 0x7fc4d4100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.791+0000 7fc4d3fff640 1 -- 172.21.15.7:0/2980226762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d41017f0 con 0x7fc4d4100110 2024-03-26T08:50:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.792+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d4100110 0x7fc4d4100a20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0036040 tx=0x7fc4c002c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.792+0000 7fc4d1ffb640 1 -- 172.21.15.7:0/2980226762 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c004e070 con 0x7fc4d4100110 2024-03-26T08:50:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.792+0000 7fc4d1ffb640 1 -- 172.21.15.7:0/2980226762 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c0005000 con 0x7fc4d4100110 2024-03-26T08:50:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.792+0000 7fc4d1ffb640 1 -- 172.21.15.7:0/2980226762 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0003a20 con 0x7fc4d4100110 2024-03-26T08:50:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.792+0000 7fc4daa8c640 1 -- 172.21.15.7:0/2980226762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d4197420 con 0x7fc4d4100110 2024-03-26T08:50:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.793+0000 7fc4daa8c640 1 -- 172.21.15.7:0/2980226762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d41978f0 con 0x7fc4d4100110 2024-03-26T08:50:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.796+0000 7fc4d1ffb640 1 -- 172.21.15.7:0/2980226762 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc4c0051070 con 0x7fc4d4100110 2024-03-26T08:50:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.796+0000 7fc4dd672740 1 Processor -- start 2024-03-26T08:50:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.796+0000 7fc4dd672740 1 -- start start 2024-03-26T08:50:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.796+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ee920 0x5606c69eed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.796+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 0x5606c69f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.796+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 0x5606c69f6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.796+0000 7fc4dd672740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c6953b10 con 0x5606c69ef310 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.796+0000 7fc4dd672740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c68fc900 con 0x5606c69f4480 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.796+0000 7fc4dd672740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c694fbb0 con 0x5606c69ee920 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d9029640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ee920 0x5606c69eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 0x5606c69f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d9029640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ee920 0x5606c69eed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58448/0 (socket says 172.21.15.7:58448) 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d9029640 1 -- 172.21.15.7:0/666836241 learned_addr learned my addr 172.21.15.7:0/666836241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d8828640 1 --2- 172.21.15.7:0/666836241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 0x5606c69f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d3fff640 1 -- 172.21.15.7:0/666836241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ee920 msgr2=0x5606c69eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/666836241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ee920 0x5606c69eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d3fff640 1 -- 172.21.15.7:0/666836241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 msgr2=0x5606c69f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/666836241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 0x5606c69f3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d3fff640 1 -- 172.21.15.7:0/666836241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606c6668210 con 0x5606c69f4480 2024-03-26T08:50:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.797+0000 7fc4d1ffb640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4c40a27f0 0x7fc4c40a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.798+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/666836241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 0x5606c69f6910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00f60a0 tx=0x7fc4c00fe620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ddc4b11fc189b8b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.798+0000 7fc4d9029640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4c40a27f0 0x7fc4c40a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.798+0000 7fc4beffd640 1 -- 172.21.15.7:0/666836241 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00c1210 con 0x5606c69f4480 2024-03-26T08:50:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.798+0000 7fc4beffd640 1 -- 172.21.15.7:0/666836241 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c00fee20 con 0x5606c69f4480 2024-03-26T08:50:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.798+0000 7fc4beffd640 1 -- 172.21.15.7:0/666836241 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00c48d0 con 0x5606c69f4480 2024-03-26T08:50:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.798+0000 7fc4d1ffb640 1 -- 172.21.15.7:0/2980226762 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4c00f8610 con 0x7fc4d4100110 2024-03-26T08:50:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.799+0000 7fc4dd672740 1 -- 172.21.15.7:0/666836241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 msgr2=0x5606c69f6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.799+0000 7fc4dd672740 1 --2- 172.21.15.7:0/666836241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 0x5606c69f6910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00f60a0 tx=0x7fc4c00fe620 comp rx=0 tx=0).stop 2024-03-26T08:50:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.799+0000 7fc4d9029640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4c40a27f0 0x7fc4c40a4cb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5606c69efb70 tx=0x7fc4cc0095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.799+0000 7fc4dd672740 1 -- 172.21.15.7:0/666836241 shutdown_connections 2024-03-26T08:50:11.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.799+0000 7fc4dd672740 1 --2- 172.21.15.7:0/666836241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 0x5606c69f6910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.799+0000 7fc4dd672740 1 --2- 172.21.15.7:0/666836241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 0x5606c69f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.799+0000 7fc4dd672740 1 --2- 172.21.15.7:0/666836241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ee920 0x5606c69eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.799+0000 7fc4dd672740 1 -- 172.21.15.7:0/666836241 >> 172.21.15.7:0/666836241 conn(0x5606c69e9e90 msgr2=0x5606c69ea290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.800+0000 7fc4dd672740 1 -- 172.21.15.7:0/666836241 shutdown_connections 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.800+0000 7fc4dd672740 1 -- 172.21.15.7:0/666836241 wait complete. 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.800+0000 7fc4dd672740 1 -- start start 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.800+0000 7fc4dd672740 1 Processor -- start 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 0x5606c69f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69f4480 0x5606c69ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d8828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d8828640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57770/0 (socket says 172.21.15.7:57770) 2024-03-26T08:50:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d8828640 1 -- 172.21.15.7:0/2000771711 learned_addr learned my addr 172.21.15.7:0/2000771711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c68fc900 con 0x5606c69ef310 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 0x5606c69f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c690fd50 con 0x5606c69ee920 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606c69f71f0 con 0x5606c69f4480 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d9029640 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69f4480 0x5606c69ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4dd672740 10 client.0 _ll_register_callbacks cb 0x5606c690f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d8828640 1 -- 172.21.15.7:0/2000771711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69f4480 msgr2=0x5606c69ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d8828640 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69f4480 0x5606c69ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d8828640 1 -- 172.21.15.7:0/2000771711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 msgr2=0x5606c69f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d8828640 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 0x5606c69f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.801+0000 7fc4d8828640 1 -- 172.21.15.7:0/2000771711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606c6734620 con 0x5606c69ee920 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.802+0000 7fc4d8828640 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69f8800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8022450 tx=0x7fc4c8022910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef7bde7d9cd5dba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.802+0000 7fc4b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:50:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.802+0000 7fc4b3fff640 1 -- 172.21.15.7:0/2000771711 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8004020 con 0x5606c69ee920 2024-03-26T08:50:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.802+0000 7fc4b3fff640 1 -- 172.21.15.7:0/2000771711 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4c80041c0 con 0x5606c69ee920 2024-03-26T08:50:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.802+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4c00fee20 con 0x5606c69ee920 2024-03-26T08:50:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.802+0000 7fc4b3fff640 1 -- 172.21.15.7:0/2000771711 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c8004400 con 0x5606c69ee920 2024-03-26T08:50:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.802+0000 7fc4dd672740 10 client.27275 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.802+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5606c6a03140 con 0x5606c69ee920 2024-03-26T08:50:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.803+0000 7fc4b3fff640 1 -- 172.21.15.7:0/2000771711 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c00fee20 con 0x5606c69ee920 2024-03-26T08:50:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.803+0000 7fc4b3fff640 1 -- 172.21.15.7:0/2000771711 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc4c8005020 con 0x5606c69ee920 2024-03-26T08:50:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.803+0000 7fc4dd672740 10 client.27275 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.803+0000 7fc4dd672740 10 client.27275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.803+0000 7fc4dd672740 20 client.27275 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:11.801 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.803+0000 7fc4dd672740 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5606c6a047b0 0x5606c6a06be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.803+0000 7fc4dd672740 4 client.27275 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.803+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606c6961050 con 0x5606c6a047b0 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.803+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5606c6a047b0 0x5606c6a06be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.804+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5606c6a047b0 0x5606c6a06be0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8579f9ec007ec3 server_cookie=a8c1c0536d6f0cee in_seq=0 out_seq=0 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.804+0000 7fc4b3fff640 10 client.27275 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.804+0000 7fc4b3fff640 1 -- 172.21.15.7:0/2000771711 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5606c6961050 con 0x5606c6a047b0 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.804+0000 7fc4b3fff640 10 client.27275 handle_command_reply: tid=0 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.804+0000 7fc4dd672740 1 client.27275 shutdown 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 msgr2=0x5606c69f8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69f8800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8022450 tx=0x7fc4c8022910 comp rx=0 tx=0).stop 2024-03-26T08:50:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 shutdown_connections 2024-03-26T08:50:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5606c6a047b0 0x5606c6a06be0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69f4480 0x5606c69ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ef310 0x5606c69f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 --2- 172.21.15.7:0/2000771711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69f8800 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 >> 172.21.15.7:0/2000771711 conn(0x5606c69e9e90 msgr2=0x5606c69e9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4b3fff640 0 client.27275 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 shutdown_connections 2024-03-26T08:50:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 1 -- 172.21.15.7:0/2000771711 wait complete. 2024-03-26T08:50:11.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.805+0000 7fc4dd672740 20 client.27275 trim_cache size 0 max 16384 2024-03-26T08:50:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.831+0000 7fc4dd672740 1 Processor -- start 2024-03-26T08:50:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.831+0000 7fc4dd672740 1 -- start start 2024-03-26T08:50:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ee920 0x5606c69ed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 0x5606c69ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4dd672740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4072440 con 0x5606c69ee920 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4dd672740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d406ff40 con 0x5606c69f4480 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4dd672740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d406dfe0 con 0x5606c69ef310 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d8828640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ee920 0x5606c69ed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d8828640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ee920 0x5606c69ed0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53324/0 (socket says 172.21.15.7:53324) 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d8828640 1 -- 172.21.15.7:0/3807079808 learned_addr learned my addr 172.21.15.7:0/3807079808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d9029640 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 0x5606c69ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d3fff640 1 -- 172.21.15.7:0/3807079808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 msgr2=0x5606c69ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 0x5606c69ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d3fff640 1 -- 172.21.15.7:0/3807079808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ee920 msgr2=0x5606c69ed0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ee920 0x5606c69ed0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d3fff640 1 -- 172.21.15.7:0/3807079808 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606c69f9530 con 0x5606c69ef310 2024-03-26T08:50:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.832+0000 7fc4d8828640 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ee920 0x5606c69ed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.833+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69ffd40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00db000 tx=0x7fc4c00cd710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6c37b03dc786a69 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.833+0000 7fc4be7fc640 1 -- 172.21.15.7:0/3807079808 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00c4ab0 con 0x5606c69ef310 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.833+0000 7fc4be7fc640 1 -- 172.21.15.7:0/3807079808 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c00fee20 con 0x5606c69ef310 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.833+0000 7fc4be7fc640 1 -- 172.21.15.7:0/3807079808 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00c4d40 con 0x5606c69ef310 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.834+0000 7fc4dd672740 1 -- 172.21.15.7:0/3807079808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 msgr2=0x5606c69ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.834+0000 7fc4dd672740 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69ffd40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00db000 tx=0x7fc4c00cd710 comp rx=0 tx=0).stop 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.834+0000 7fc4dd672740 1 -- 172.21.15.7:0/3807079808 shutdown_connections 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.834+0000 7fc4dd672740 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69f4480 0x5606c69ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.834+0000 7fc4dd672740 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69ffd40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.834+0000 7fc4dd672740 1 --2- 172.21.15.7:0/3807079808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69ee920 0x5606c69ed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.834+0000 7fc4dd672740 1 -- 172.21.15.7:0/3807079808 >> 172.21.15.7:0/3807079808 conn(0x5606c69e9e90 msgr2=0x5606c69ea270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.835+0000 7fc4dd672740 1 -- 172.21.15.7:0/3807079808 shutdown_connections 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.835+0000 7fc4dd672740 1 -- 172.21.15.7:0/3807079808 wait complete. 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.835+0000 7fc4dd672740 1 -- start start 2024-03-26T08:50:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.835+0000 7fc4dd672740 1 Processor -- start 2024-03-26T08:50:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69fd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4dd672740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69f4480 0x5606c69fe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4dd672740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d406e5e0 con 0x5606c69f4480 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4d3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69fd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58468/0 (socket says 172.21.15.7:58468) 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4d3fff640 1 -- 172.21.15.7:0/681847321 learned_addr learned my addr 172.21.15.7:0/681847321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4d9029640 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69f4480 0x5606c69fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4d8828640 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69fd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4dd672740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d406f8e0 con 0x5606c69ee920 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.836+0000 7fc4dd672740 1 -- 172.21.15.7:0/681847321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d406ec20 con 0x5606c69ef310 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4dd672740 10 client.0 _ll_register_callbacks cb 0x7fc4c800bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4d3fff640 1 -- 172.21.15.7:0/681847321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 msgr2=0x5606c69fd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69fd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4d3fff640 1 -- 172.21.15.7:0/681847321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69f4480 msgr2=0x5606c69fe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69f4480 0x5606c69fe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4d3fff640 1 -- 172.21.15.7:0/681847321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606c69fecf0 con 0x5606c69ef310 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4d8828640 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4d3fff640 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69fd6f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc4c002e040 tx=0x7fc4c002cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1bf304196f82a89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:50:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4b3fff640 1 -- 172.21.15.7:0/681847321 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00c0ae0 con 0x5606c69ef310 2024-03-26T08:50:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4b3fff640 1 -- 172.21.15.7:0/681847321 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc4c0005520 con 0x5606c69ef310 2024-03-26T08:50:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.837+0000 7fc4dd672740 1 -- 172.21.15.7:0/681847321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4c00fee20 con 0x5606c69ef310 2024-03-26T08:50:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.838+0000 7fc4b3fff640 1 -- 172.21.15.7:0/681847321 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c00d4050 con 0x5606c69ef310 2024-03-26T08:50:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.838+0000 7fc4dd672740 10 client.27298 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.838+0000 7fc4dd672740 1 -- 172.21.15.7:0/681847321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5606c69fffc0 con 0x5606c69ef310 2024-03-26T08:50:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.839+0000 7fc4b3fff640 1 -- 172.21.15.7:0/681847321 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c00fee20 con 0x5606c69ef310 2024-03-26T08:50:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.839+0000 7fc4b3fff640 1 -- 172.21.15.7:0/681847321 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc4c00c15e0 con 0x5606c69ef310 2024-03-26T08:50:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.839+0000 7fc4dd672740 10 client.27298 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.839+0000 7fc4dd672740 10 client.27298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.839+0000 7fc4dd672740 20 client.27298 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.839+0000 7fc4dd672740 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5606c6a604c0 0x5606c6a628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.839+0000 7fc4dd672740 4 client.27298 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:50:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.840+0000 7fc4dd672740 1 -- 172.21.15.7:0/681847321 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5606c695b3a0 con 0x5606c6a604c0 2024-03-26T08:50:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.840+0000 7fc4d8828640 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5606c6a604c0 0x5606c6a628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.840+0000 7fc4b3fff640 10 client.27298 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.840+0000 7fc4d8828640 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5606c6a604c0 0x5606c6a628e0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc93e745d21d0e92 server_cookie=8786cd4d280815d1 in_seq=0 out_seq=0 2024-03-26T08:50:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.905+0000 7fc4b3fff640 1 -- 172.21.15.7:0/681847321 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5606c695b3a0 con 0x5606c6a604c0 2024-03-26T08:50:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.905+0000 7fc4b3fff640 10 client.27298 handle_command_reply: tid=0 2024-03-26T08:50:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.906+0000 7fc4dd672740 1 client.27298 shutdown 2024-03-26T08:50:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.907+0000 7fc4dd672740 1 -- 172.21.15.7:0/681847321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 msgr2=0x5606c69fd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.907+0000 7fc4dd672740 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69fd6f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc4c002e040 tx=0x7fc4c002cf90 comp rx=0 tx=0).stop 2024-03-26T08:50:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.911+0000 7fc4dd672740 1 -- 172.21.15.7:0/681847321 shutdown_connections 2024-03-26T08:50:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.911+0000 7fc4dd672740 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5606c6a604c0 0x5606c6a628e0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.911+0000 7fc4dd672740 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606c69f4480 0x5606c69fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.911+0000 7fc4dd672740 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5606c69ef310 0x5606c69fd6f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.911+0000 7fc4dd672740 1 --2- 172.21.15.7:0/681847321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606c69ee920 0x5606c69fd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.911+0000 7fc4dd672740 1 -- 172.21.15.7:0/681847321 >> 172.21.15.7:0/681847321 conn(0x5606c69e9e90 msgr2=0x5606c69ed0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:11.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.911+0000 7fc4b3fff640 0 client.27298 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.912+0000 7fc4dd672740 1 -- 172.21.15.7:0/681847321 shutdown_connections 2024-03-26T08:50:11.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.914+0000 7fc4dd672740 1 -- 172.21.15.7:0/681847321 wait complete. 2024-03-26T08:50:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.914+0000 7fc4dd672740 20 client.27298 trim_cache size 0 max 16384 2024-03-26T08:50:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:50:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:50:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:50:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:50:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.919+0000 7fc4daa8c640 1 -- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4c40a27f0 msgr2=0x7fc4c40a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.919+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4c40a27f0 0x7fc4c40a4cb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5606c69efb70 tx=0x7fc4cc0095e0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.919+0000 7fc4daa8c640 1 -- 172.21.15.7:0/2980226762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d4100110 msgr2=0x7fc4d4100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.919+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d4100110 0x7fc4d4100a20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0036040 tx=0x7fc4c002c980 comp rx=0 tx=0).stop 2024-03-26T08:50:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.920+0000 7fc4daa8c640 1 -- 172.21.15.7:0/2980226762 shutdown_connections 2024-03-26T08:50:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.920+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc4c40a27f0 0x7fc4c40a4cb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.920+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d4102ce0 0x7fc4d4100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.920+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d4100110 0x7fc4d4100a20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.921+0000 7fc4daa8c640 1 --2- 172.21.15.7:0/2980226762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc4d40ff750 0x7fc4d41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.921+0000 7fc4daa8c640 1 -- 172.21.15.7:0/2980226762 >> 172.21.15.7:0/2980226762 conn(0x7fc4d40fb2f0 msgr2=0x7fc4d4105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.921+0000 7fc4daa8c640 1 -- 172.21.15.7:0/2980226762 shutdown_connections 2024-03-26T08:50:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:11.922+0000 7fc4daa8c640 1 -- 172.21.15.7:0/2980226762 wait complete. 2024-03-26T08:50:12.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:12 smithi016 ceph-mon[35851]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.1 MiB/s wr, 726 op/s 2024-03-26T08:50:12.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:12 smithi007 ceph-mon[32462]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.1 MiB/s wr, 726 op/s 2024-03-26T08:50:12.568 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:12 smithi044 ceph-mon[36461]: pgmap v593: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 8.1 MiB/s wr, 726 op/s 2024-03-26T08:50:14.370 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:14 smithi044 ceph-mon[36461]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 821 op/s 2024-03-26T08:50:14.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:14 smithi016 ceph-mon[35851]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 821 op/s 2024-03-26T08:50:14.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:14 smithi007 ceph-mon[32462]: pgmap v594: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 821 op/s 2024-03-26T08:50:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 465.0 s, 2085.0 tps, lat 47.995 ms stddev 54.356 2024-03-26T08:50:16.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:16 smithi016 ceph-mon[35851]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 746 op/s 2024-03-26T08:50:16.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:16 smithi007 ceph-mon[32462]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 746 op/s 2024-03-26T08:50:16.568 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:16 smithi044 ceph-mon[36461]: pgmap v595: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 14 MiB/s wr, 746 op/s 2024-03-26T08:50:18.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:18 smithi016 ceph-mon[35851]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 797 op/s 2024-03-26T08:50:18.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:18 smithi007 ceph-mon[32462]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 797 op/s 2024-03-26T08:50:18.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:18 smithi044 ceph-mon[36461]: pgmap v596: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 MiB/s wr, 797 op/s 2024-03-26T08:50:20.495 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:20 smithi016 ceph-mon[35851]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 767 op/s 2024-03-26T08:50:20.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:20 smithi007 ceph-mon[32462]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 767 op/s 2024-03-26T08:50:20.568 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:20 smithi044 ceph-mon[36461]: pgmap v597: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 MiB/s wr, 767 op/s 2024-03-26T08:50:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 470.0 s, 2206.2 tps, lat 45.557 ms stddev 52.780 2024-03-26T08:50:22.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:22 smithi016 ceph-mon[35851]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 797 op/s 2024-03-26T08:50:22.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:22 smithi007 ceph-mon[32462]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 797 op/s 2024-03-26T08:50:22.568 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:22 smithi044 ceph-mon[36461]: pgmap v598: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 MiB/s wr, 797 op/s 2024-03-26T08:50:24.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:24 smithi016 ceph-mon[35851]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 813 op/s 2024-03-26T08:50:24.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:24 smithi007 ceph-mon[32462]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 813 op/s 2024-03-26T08:50:24.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:24 smithi044 ceph-mon[36461]: pgmap v599: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 813 op/s 2024-03-26T08:50:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.059+0000 7f18a037e640 1 -- 172.21.15.7:0/2316613841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18981086a0 msgr2=0x7f189810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.059+0000 7f18a037e640 1 --2- 172.21.15.7:0/2316613841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18981086a0 0x7f189810aad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1894009860 tx=0x7f189402bb70 comp rx=0 tx=0).stop 2024-03-26T08:50:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.060+0000 7f18a037e640 1 -- 172.21.15.7:0/2316613841 shutdown_connections 2024-03-26T08:50:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.060+0000 7f18a037e640 1 --2- 172.21.15.7:0/2316613841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f189810d9c0 0x7f189810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.060+0000 7f18a037e640 1 --2- 172.21.15.7:0/2316613841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f189810b010 0x7f189810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.060+0000 7f18a037e640 1 --2- 172.21.15.7:0/2316613841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18981086a0 0x7f189810aad0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.060+0000 7f18a037e640 1 -- 172.21.15.7:0/2316613841 >> 172.21.15.7:0/2316613841 conn(0x7f189806e6b0 msgr2=0x7f1898070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.060+0000 7f18a037e640 1 -- 172.21.15.7:0/2316613841 shutdown_connections 2024-03-26T08:50:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.060+0000 7f18a037e640 1 -- 172.21.15.7:0/2316613841 wait complete. 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f18a037e640 1 Processor -- start 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f18a037e640 1 -- start start 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f18a037e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18981086a0 0x7f18981a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f18a037e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189810b010 0x7f18981a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f18a037e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f189810d9c0 0x7f18981addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f18a037e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189811c1a0 con 0x7f189810d9c0 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f18a037e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189811c020 con 0x7f189810b010 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f18a037e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189811c320 con 0x7f18981086a0 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f189e11a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18981086a0 0x7f18981a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f189e11a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18981086a0 0x7f18981a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35610/0 (socket says 172.21.15.7:35610) 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.061+0000 7f189e11a640 1 -- 172.21.15.7:0/374145885 learned_addr learned my addr 172.21.15.7:0/374145885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:25.057 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.062+0000 7f189e91b640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f189810d9c0 0x7f18981addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.062+0000 7f189d919640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189810b010 0x7f18981a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.062+0000 7f189e11a640 1 -- 172.21.15.7:0/374145885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189810b010 msgr2=0x7f18981a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.062+0000 7f189e11a640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189810b010 0x7f18981a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.062+0000 7f189e11a640 1 -- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f189810d9c0 msgr2=0x7f18981addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.062+0000 7f189e11a640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f189810d9c0 0x7f18981addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.062+0000 7f189e11a640 1 -- 172.21.15.7:0/374145885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18981ae430 con 0x7f18981086a0 2024-03-26T08:50:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.063+0000 7f189e11a640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18981086a0 0x7f18981a6860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894032460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.063+0000 7f188f7fe640 1 -- 172.21.15.7:0/374145885 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894039040 con 0x7f18981086a0 2024-03-26T08:50:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.063+0000 7f188f7fe640 1 -- 172.21.15.7:0/374145885 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1894042070 con 0x7f18981086a0 2024-03-26T08:50:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.063+0000 7f188f7fe640 1 -- 172.21.15.7:0/374145885 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940046c0 con 0x7f18981086a0 2024-03-26T08:50:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.063+0000 7f18a037e640 1 -- 172.21.15.7:0/374145885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18981ae6c0 con 0x7f18981086a0 2024-03-26T08:50:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.063+0000 7f18a037e640 1 -- 172.21.15.7:0/374145885 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18981aec00 con 0x7f18981086a0 2024-03-26T08:50:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.067+0000 7f188f7fe640 1 -- 172.21.15.7:0/374145885 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1894004340 con 0x7f18981086a0 2024-03-26T08:50:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.067+0000 7f18a2f6b740 1 Processor -- start 2024-03-26T08:50:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.068+0000 7f18a2f6b740 1 -- start start 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.068+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a98920 0x55b761a98d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 0x55b761a9de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f188f7fe640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f18840a28b0 0x7f18840a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f18a2f6b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7619fdb10 con 0x55b761a98920 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f189d919640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f18840a28b0 0x7f18840a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f189e91b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a98920 0x55b761a98d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f188f7fe640 1 -- 172.21.15.7:0/374145885 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f18940f02e0 con 0x7f18981086a0 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f189e91b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a98920 0x55b761a98d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38932/0 (socket says 172.21.15.7:38932) 2024-03-26T08:50:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f189e91b640 1 -- 172.21.15.7:0/3762384841 learned_addr learned my addr 172.21.15.7:0/3762384841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f189e11a640 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 0x55b761a9de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3762384841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7619a6900 con 0x55b761a99310 2024-03-26T08:50:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.069+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3762384841 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7619f9bb0 con 0x55b761a9e480 2024-03-26T08:50:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f189e91b640 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f189e11a640 1 -- 172.21.15.7:0/3762384841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 msgr2=0x55b761aa0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f189d919640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f18840a28b0 0x7f18840a4d70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f18981a7d20 tx=0x7f1888009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f189e11a640 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f189e11a640 1 -- 172.21.15.7:0/3762384841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a98920 msgr2=0x55b761a98d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f189e11a640 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a98920 0x55b761a98d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f189e11a640 1 -- 172.21.15.7:0/3762384841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b761712210 con 0x55b761a99310 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f189e91b640 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f189e11a640 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 0x55b761a9de10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f18940349e0 tx=0x7f18940bcad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64d90a60370dcf70 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f188cbfe640 1 -- 172.21.15.7:0/3762384841 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940bb200 con 0x55b761a99310 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f188cbfe640 1 -- 172.21.15.7:0/3762384841 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f18940bd340 con 0x55b761a99310 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.070+0000 7f188cbfe640 1 -- 172.21.15.7:0/3762384841 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940c4d40 con 0x55b761a99310 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.071+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3762384841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 msgr2=0x55b761a9de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.071+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 0x55b761a9de10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f18940349e0 tx=0x7f18940bcad0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.071+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3762384841 shutdown_connections 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.071+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.071+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 0x55b761a9de10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.071+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3762384841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a98920 0x55b761a98d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.072+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3762384841 >> 172.21.15.7:0/3762384841 conn(0x55b761a93e90 msgr2=0x55b761a94290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.072+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3762384841 shutdown_connections 2024-03-26T08:50:25.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.072+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3762384841 wait complete. 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.072+0000 7f18a2f6b740 1 -- start start 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.072+0000 7f18a2f6b740 1 Processor -- start 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f18a2f6b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b761aa11f0 con 0x55b761a99310 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f189e11a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f18a2f6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7619b9d50 con 0x55b761a98920 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f189e11a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58332/0 (socket says 172.21.15.7:58332) 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f189e11a640 1 -- 172.21.15.7:0/4210509523 learned_addr learned my addr 172.21.15.7:0/4210509523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:25.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f18a2f6b740 1 -- 172.21.15.7:0/4210509523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b761aa1370 con 0x55b761a9e480 2024-03-26T08:50:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f189d919640 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f189e91b640 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.073+0000 7f18a2f6b740 10 client.0 _ll_register_callbacks cb 0x55b7619b9720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f189d919640 1 -- 172.21.15.7:0/4210509523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 msgr2=0x55b761aa9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f189d919640 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f189d919640 1 -- 172.21.15.7:0/4210509523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 msgr2=0x55b761aa2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f189d919640 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f189d919640 1 -- 172.21.15.7:0/4210509523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7617de620 con 0x55b761a99310 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f189e11a640 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f18a2f6b740 1 -- 172.21.15.7:0/4210509523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18940bd340 con 0x55b761a99310 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f189d919640 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa2e80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55b761a94e00 tx=0x7f1888004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143a7a2b456be97a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f1881ffb640 10 client.17460 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f1881ffb640 1 -- 172.21.15.7:0/4210509523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888022070 con 0x55b761a99310 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f1881ffb640 1 -- 172.21.15.7:0/4210509523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f188800faa0 con 0x55b761a99310 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f1881ffb640 1 -- 172.21.15.7:0/4210509523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f188800fd00 con 0x55b761a99310 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f18a2f6b740 10 client.17460 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.075+0000 7f18a2f6b740 1 -- 172.21.15.7:0/4210509523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55b761aad1a0 con 0x55b761a99310 2024-03-26T08:50:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.076+0000 7f1881ffb640 1 -- 172.21.15.7:0/4210509523 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1888031420 con 0x55b761a99310 2024-03-26T08:50:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.076+0000 7f18a2f6b740 10 client.17460 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.076+0000 7f18a2f6b740 10 client.17460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.076+0000 7f18a2f6b740 20 client.17460 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.076+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b761aae7d0 0x55b761ab0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.076+0000 7f18a2f6b740 4 client.17460 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.076+0000 7f18a2f6b740 1 -- 172.21.15.7:0/4210509523 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b761a0b050 con 0x55b761aae7d0 2024-03-26T08:50:25.073 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.077+0000 7f189e11a640 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b761aae7d0 0x55b761ab0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.077+0000 7f189e11a640 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b761aae7d0 0x55b761ab0bc0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16488b61104b458c server_cookie=8e2932f1acfcabac in_seq=0 out_seq=0 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.077+0000 7f1881ffb640 10 client.17460 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.078+0000 7f1881ffb640 1 -- 172.21.15.7:0/4210509523 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b761a0b050 con 0x55b761aae7d0 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.078+0000 7f1881ffb640 10 client.17460 handle_command_reply: tid=0 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.078+0000 7f18a2f6b740 1 client.17460 shutdown 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.078+0000 7f18a2f6b740 1 -- 172.21.15.7:0/4210509523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 msgr2=0x55b761aa2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.078+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa2e80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55b761a94e00 tx=0x7f1888004730 comp rx=0 tx=0).stop 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.078+0000 7f18a2f6b740 1 -- 172.21.15.7:0/4210509523 shutdown_connections 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.078+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b761aae7d0 0x55b761ab0bc0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.079+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.079+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa2e80 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.079+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/4210509523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.079+0000 7f18a2f6b740 1 -- 172.21.15.7:0/4210509523 >> 172.21.15.7:0/4210509523 conn(0x55b761a93e90 msgr2=0x55b761a93020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.079+0000 7f1881ffb640 0 client.17460 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.079+0000 7f18a2f6b740 1 -- 172.21.15.7:0/4210509523 shutdown_connections 2024-03-26T08:50:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.079+0000 7f18a2f6b740 1 -- 172.21.15.7:0/4210509523 wait complete. 2024-03-26T08:50:25.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.079+0000 7f18a2f6b740 20 client.17460 trim_cache size 0 max 16384 2024-03-26T08:50:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.105+0000 7f18a2f6b740 1 Processor -- start 2024-03-26T08:50:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.105+0000 7f18a2f6b740 1 -- start start 2024-03-26T08:50:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.105+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a98920 0x55b761a9a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.105+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 0x55b761aa9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.105+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a9e480 0x55b761a9ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f18a2f6b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981b9df0 con 0x55b761a9e480 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e11a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a98920 0x55b761a9a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189d919640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 0x55b761aa9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e91b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a9e480 0x55b761a9ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e11a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a98920 0x55b761a9a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35638/0 (socket says 172.21.15.7:35638) 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e11a640 1 -- 172.21.15.7:0/1474952092 learned_addr learned my addr 172.21.15.7:0/1474952092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f18a2f6b740 1 -- 172.21.15.7:0/1474952092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981b78f0 con 0x55b761a99310 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f18a2f6b740 1 -- 172.21.15.7:0/1474952092 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981b5970 con 0x55b761a98920 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e91b640 1 -- 172.21.15.7:0/1474952092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a98920 msgr2=0x55b761a9a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e91b640 1 --2- 172.21.15.7:0/1474952092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a98920 0x55b761a9a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e91b640 1 -- 172.21.15.7:0/1474952092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 msgr2=0x55b761aa9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e91b640 1 --2- 172.21.15.7:0/1474952092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 0x55b761aa9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e91b640 1 -- 172.21.15.7:0/1474952092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b761a94c00 con 0x55b761a9e480 2024-03-26T08:50:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.106+0000 7f189e91b640 1 --2- 172.21.15.7:0/1474952092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a9e480 0x55b761a9ab10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1890009fd0 tx=0x7f1890014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2967d30a1c2bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.107+0000 7f1883fff640 1 -- 172.21.15.7:0/1474952092 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890023070 con 0x55b761a9e480 2024-03-26T08:50:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.107+0000 7f1883fff640 1 -- 172.21.15.7:0/1474952092 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1890003e50 con 0x55b761a9e480 2024-03-26T08:50:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.107+0000 7f1883fff640 1 -- 172.21.15.7:0/1474952092 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890004290 con 0x55b761a9e480 2024-03-26T08:50:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.107+0000 7f18a2f6b740 1 -- 172.21.15.7:0/1474952092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a9e480 msgr2=0x55b761a9ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.107+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/1474952092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a9e480 0x55b761a9ab10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1890009fd0 tx=0x7f1890014040 comp rx=0 tx=0).stop 2024-03-26T08:50:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 -- 172.21.15.7:0/1474952092 shutdown_connections 2024-03-26T08:50:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/1474952092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a9e480 0x55b761a9ab10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/1474952092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a99310 0x55b761aa9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/1474952092 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a98920 0x55b761a9a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 -- 172.21.15.7:0/1474952092 >> 172.21.15.7:0/1474952092 conn(0x55b761a93e90 msgr2=0x55b761aa2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:25.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 -- 172.21.15.7:0/1474952092 shutdown_connections 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 -- 172.21.15.7:0/1474952092 wait complete. 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 -- start start 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 Processor -- start 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981b5f70 con 0x55b761a99310 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981b7290 con 0x55b761a98920 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18981b65d0 con 0x55b761a9e480 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f18a2f6b740 10 client.0 _ll_register_callbacks cb 0x7f1888012490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f189d919640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f189e11a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f189d919640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38946/0 (socket says 172.21.15.7:38946) 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f189d919640 1 -- 172.21.15.7:0/3073225899 learned_addr learned my addr 172.21.15.7:0/3073225899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.108+0000 7f189e11a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58346/0 (socket says 172.21.15.7:58346) 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f189e91b640 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f189d919640 1 -- 172.21.15.7:0/3073225899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 msgr2=0x55b761aa8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f189d919640 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f189d919640 1 -- 172.21.15.7:0/3073225899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 msgr2=0x55b761aa7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f189d919640 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f189d919640 1 -- 172.21.15.7:0/3073225899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b761aa91b0 con 0x55b761a99310 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f189e91b640 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f189e11a640 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f189d919640 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa7bb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55b761a94e00 tx=0x7f1888018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecc9f0e3c8ad759 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:25.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.109+0000 7f1881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:50:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.112+0000 7f1881ffb640 1 -- 172.21.15.7:0/3073225899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888040060 con 0x55b761a99310 2024-03-26T08:50:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.112+0000 7f1881ffb640 1 -- 172.21.15.7:0/3073225899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f188800f7a0 con 0x55b761a99310 2024-03-26T08:50:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.112+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3073225899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1890003e50 con 0x55b761a99310 2024-03-26T08:50:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f1881ffb640 1 -- 172.21.15.7:0/3073225899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1888022070 con 0x55b761a99310 2024-03-26T08:50:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f18a2f6b740 10 client.17466 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3073225899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b761b0a140 con 0x55b761a99310 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f1881ffb640 1 -- 172.21.15.7:0/3073225899 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1890003e50 con 0x55b761a99310 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f1881ffb640 1 -- 172.21.15.7:0/3073225899 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1888045070 con 0x55b761a99310 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f18a2f6b740 10 client.17466 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f18a2f6b740 10 client.17466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f18a2f6b740 20 client.17466 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b761b0aa90 0x55b761b0ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f18a2f6b740 4 client.17466 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3073225899 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b761a053a0 con 0x55b761b0aa90 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.113+0000 7f189e11a640 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b761b0aa90 0x55b761b0ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.114+0000 7f189e11a640 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b761b0aa90 0x55b761b0ceb0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c442f4967e8710cf server_cookie=a7b6b2bf784a6314 in_seq=0 out_seq=0 2024-03-26T08:50:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.114+0000 7f1881ffb640 10 client.17466 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f1881ffb640 1 -- 172.21.15.7:0/3073225899 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b761a053a0 con 0x55b761b0aa90 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f1881ffb640 10 client.17466 handle_command_reply: tid=0 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 client.17466 shutdown 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3073225899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 msgr2=0x55b761aa7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa7bb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55b761a94e00 tx=0x7f1888018000 comp rx=0 tx=0).stop 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3073225899 shutdown_connections 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b761b0aa90 0x55b761b0ceb0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b761a9e480 0x55b761aa8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b761a99310 0x55b761aa7bb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 --2- 172.21.15.7:0/3073225899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b761a98920 0x55b761aa7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3073225899 >> 172.21.15.7:0/3073225899 conn(0x55b761a93e90 msgr2=0x55b761aa2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f1881ffb640 0 client.17466 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3073225899 shutdown_connections 2024-03-26T08:50:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 1 -- 172.21.15.7:0/3073225899 wait complete. 2024-03-26T08:50:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.143+0000 7f18a2f6b740 20 client.17466 trim_cache size 0 max 16384 2024-03-26T08:50:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:50:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:50:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:50:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:50:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 -- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f18840a28b0 msgr2=0x7f18840a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f18840a28b0 0x7f18840a4d70 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f18981a7d20 tx=0x7f1888009260 comp rx=0 tx=0).stop 2024-03-26T08:50:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 -- 172.21.15.7:0/374145885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18981086a0 msgr2=0x7f18981a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:25.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18981086a0 0x7f18981a6860 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f1894032460 comp rx=0 tx=0).stop 2024-03-26T08:50:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 -- 172.21.15.7:0/374145885 shutdown_connections 2024-03-26T08:50:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f18840a28b0 0x7f18840a4d70 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f189810d9c0 0x7f18981addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f189810b010 0x7f18981a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 --2- 172.21.15.7:0/374145885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f18981086a0 0x7f18981a6860 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 -- 172.21.15.7:0/374145885 >> 172.21.15.7:0/374145885 conn(0x7f189806e6b0 msgr2=0x7f189810b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 -- 172.21.15.7:0/374145885 shutdown_connections 2024-03-26T08:50:25.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:25.144+0000 7f18a037e640 1 -- 172.21.15.7:0/374145885 wait complete. 2024-03-26T08:50:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 475.0 s, 2472.0 tps, lat 39.947 ms stddev 45.682 2024-03-26T08:50:26.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:26 smithi016 ceph-mon[35851]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 19 MiB/s wr, 724 op/s 2024-03-26T08:50:26.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:26 smithi007 ceph-mon[32462]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 19 MiB/s wr, 724 op/s 2024-03-26T08:50:26.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:26 smithi044 ceph-mon[36461]: pgmap v600: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 19 MiB/s wr, 724 op/s 2024-03-26T08:50:28.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:50:28.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:50:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.321+0000 7fad2db53640 1 -- 172.21.15.7:0/2912444993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810c0b0 msgr2=0x7fad2810c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.321+0000 7fad2db53640 1 --2- 172.21.15.7:0/2912444993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810c0b0 0x7fad2810c4d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fad18009900 tx=0x7fad1802bc30 comp rx=0 tx=0).stop 2024-03-26T08:50:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.321+0000 7fad2db53640 1 -- 172.21.15.7:0/2912444993 shutdown_connections 2024-03-26T08:50:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.321+0000 7fad2db53640 1 --2- 172.21.15.7:0/2912444993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2810dc60 0x7fad281144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.321+0000 7fad2db53640 1 --2- 172.21.15.7:0/2912444993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad2810d2a0 0x7fad2810d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.321+0000 7fad2db53640 1 --2- 172.21.15.7:0/2912444993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810c0b0 0x7fad2810c4d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.321+0000 7fad2db53640 1 -- 172.21.15.7:0/2912444993 >> 172.21.15.7:0/2912444993 conn(0x7fad2806c170 msgr2=0x7fad2806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.321+0000 7fad2db53640 1 -- 172.21.15.7:0/2912444993 shutdown_connections 2024-03-26T08:50:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.321+0000 7fad2db53640 1 -- 172.21.15.7:0/2912444993 wait complete. 2024-03-26T08:50:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad2db53640 1 Processor -- start 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad2db53640 1 -- start start 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad2db53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad2810c0b0 0x7fad281a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad2db53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2810d2a0 0x7fad281a2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad2db53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810dc60 0x7fad281a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad2db53640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad28117d40 con 0x7fad2810dc60 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad2db53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad28117bc0 con 0x7fad2810c0b0 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad2db53640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad28117ec0 con 0x7fad2810d2a0 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810dc60 0x7fad281a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810dc60 0x7fad281a99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59654/0 (socket says 172.21.15.7:59654) 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad2810c0b0 0x7fad281a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad27fff640 1 -- 172.21.15.7:0/1669549058 learned_addr learned my addr 172.21.15.7:0/1669549058 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.322+0000 7fad26ffd640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2810d2a0 0x7fad281a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.323+0000 7fad277fe640 1 -- 172.21.15.7:0/1669549058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2810d2a0 msgr2=0x7fad281a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.323+0000 7fad277fe640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2810d2a0 0x7fad281a2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.323+0000 7fad277fe640 1 -- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810dc60 msgr2=0x7fad281a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.323+0000 7fad277fe640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810dc60 0x7fad281a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.323+0000 7fad277fe640 1 -- 172.21.15.7:0/1669549058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad281aa020 con 0x7fad2810c0b0 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.323+0000 7fad27fff640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810dc60 0x7fad281a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.323+0000 7fad277fe640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad2810c0b0 0x7fad281a2400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fad1802c170 tx=0x7fad1802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.323+0000 7fad24ff9640 1 -- 172.21.15.7:0/1669549058 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18041070 con 0x7fad2810c0b0 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.324+0000 7fad24ff9640 1 -- 172.21.15.7:0/1669549058 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad18004450 con 0x7fad2810c0b0 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.324+0000 7fad24ff9640 1 -- 172.21.15.7:0/1669549058 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18003e50 con 0x7fad2810c0b0 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.324+0000 7fad2db53640 1 -- 172.21.15.7:0/1669549058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad281aa2b0 con 0x7fad2810c0b0 2024-03-26T08:50:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.324+0000 7fad2db53640 1 -- 172.21.15.7:0/1669549058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2806d240 con 0x7fad2810c0b0 2024-03-26T08:50:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad3081c740 1 Processor -- start 2024-03-26T08:50:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad3081c740 1 -- start start 2024-03-26T08:50:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad3081c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f2da0 0x5611c56f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad3081c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f3790 0x5611c56f8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad3081c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f8900 0x5611c56fad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad3081c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c56583d0 con 0x5611c56f8900 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad3081c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c5600ad0 con 0x5611c56f3790 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad3081c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c5654470 con 0x5611c56f2da0 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f2da0 0x5611c56f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad24ff9640 1 -- 172.21.15.7:0/1669549058 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fad18044070 con 0x7fad2810c0b0 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.327+0000 7fad27fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f2da0 0x5611c56f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43406/0 (socket says 172.21.15.7:43406) 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad27fff640 1 -- 172.21.15.7:0/2889421844 learned_addr learned my addr 172.21.15.7:0/2889421844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad26ffd640 1 --2- 172.21.15.7:0/2889421844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f8900 0x5611c56fad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad277fe640 1 --2- 172.21.15.7:0/2889421844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f3790 0x5611c56f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad27fff640 1 -- 172.21.15.7:0/2889421844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f3790 msgr2=0x5611c56f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad27fff640 1 --2- 172.21.15.7:0/2889421844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f3790 0x5611c56f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad27fff640 1 -- 172.21.15.7:0/2889421844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f8900 msgr2=0x5611c56fad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:28.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad27fff640 1 --2- 172.21.15.7:0/2889421844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f8900 0x5611c56fad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad27fff640 1 -- 172.21.15.7:0/2889421844 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611c54367d0 con 0x5611c56f2da0 2024-03-26T08:50:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad24ff9640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad100a2af0 0x7fad100a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad27fff640 1 --2- 172.21.15.7:0/2889421844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f2da0 0x5611c56f31c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fad200118b0 tx=0x7fad20011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74f86a826e515173 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad277fe640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad100a2af0 0x7fad100a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad0dffb640 1 -- 172.21.15.7:0/2889421844 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2001e070 con 0x5611c56f2da0 2024-03-26T08:50:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad24ff9640 1 -- 172.21.15.7:0/1669549058 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fad180f03c0 con 0x7fad2810c0b0 2024-03-26T08:50:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad0dffb640 1 -- 172.21.15.7:0/2889421844 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad20011310 con 0x5611c56f2da0 2024-03-26T08:50:28.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.328+0000 7fad0dffb640 1 -- 172.21.15.7:0/2889421844 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20019e60 con 0x5611c56f2da0 2024-03-26T08:50:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.329+0000 7fad3081c740 1 -- 172.21.15.7:0/2889421844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f2da0 msgr2=0x5611c56f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.329+0000 7fad3081c740 1 --2- 172.21.15.7:0/2889421844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f2da0 0x5611c56f31c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fad200118b0 tx=0x7fad20011d70 comp rx=0 tx=0).stop 2024-03-26T08:50:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.329+0000 7fad277fe640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad100a2af0 0x7fad100a4fb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5611c56f3b90 tx=0x7fad180f3e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.331+0000 7fad3081c740 1 -- 172.21.15.7:0/2889421844 shutdown_connections 2024-03-26T08:50:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.331+0000 7fad3081c740 1 --2- 172.21.15.7:0/2889421844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f8900 0x5611c56fad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.331+0000 7fad3081c740 1 --2- 172.21.15.7:0/2889421844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f3790 0x5611c56f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.331+0000 7fad3081c740 1 --2- 172.21.15.7:0/2889421844 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f2da0 0x5611c56f31c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.331+0000 7fad3081c740 1 -- 172.21.15.7:0/2889421844 >> 172.21.15.7:0/2889421844 conn(0x5611c56ee310 msgr2=0x5611c56ee710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:28.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.331+0000 7fad3081c740 1 -- 172.21.15.7:0/2889421844 shutdown_connections 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.332+0000 7fad3081c740 1 -- 172.21.15.7:0/2889421844 wait complete. 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.332+0000 7fad3081c740 1 -- start start 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.332+0000 7fad3081c740 1 Processor -- start 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.332+0000 7fad3081c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad3081c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 0x5611c56fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad3081c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 0x5611c57046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34116/0 (socket says 172.21.15.7:34116) 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 -- 172.21.15.7:0/1797301684 learned_addr learned my addr 172.21.15.7:0/1797301684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c56fb8b0 con 0x5611c56f3790 2024-03-26T08:50:28.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c5614060 con 0x5611c56f2da0 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611c56fba30 con 0x5611c56f8900 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad26ffd640 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 0x5611c56fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad3081c740 10 client.0 _ll_register_callbacks cb 0x5611c55f66a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 0x5611c57046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 -- 172.21.15.7:0/1797301684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 msgr2=0x5611c57046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 0x5611c57046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 -- 172.21.15.7:0/1797301684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 msgr2=0x5611c56fd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 0x5611c56fd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 -- 172.21.15.7:0/1797301684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611c536c3f0 con 0x5611c56f2da0 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad26ffd640 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 0x5611c56fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad27fff640 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd170 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fad20011500 tx=0x7fad20011180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d18314c1a3866bd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.333+0000 7fad06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:50:28.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad06ffd640 1 -- 172.21.15.7:0/1797301684 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2001e040 con 0x5611c56f2da0 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad06ffd640 1 -- 172.21.15.7:0/1797301684 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fad20019ba0 con 0x5611c56f2da0 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad20011310 con 0x5611c56f2da0 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad06ffd640 1 -- 172.21.15.7:0/1797301684 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad20019da0 con 0x5611c56f2da0 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad3081c740 10 client.27311 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5611c532c0f0 con 0x5611c56f2da0 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad06ffd640 1 -- 172.21.15.7:0/1797301684 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad20011310 con 0x5611c56f2da0 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad06ffd640 1 -- 172.21.15.7:0/1797301684 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fad20031050 con 0x5611c56f2da0 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad3081c740 10 client.27311 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad3081c740 10 client.27311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad3081c740 20 client.27311 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad3081c740 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5611c57088b0 0x5611c570ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad3081c740 4 client.27311 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.334+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611c5665910 con 0x5611c57088b0 2024-03-26T08:50:28.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.335+0000 7fad26ffd640 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5611c57088b0 0x5611c570ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.335+0000 7fad06ffd640 10 client.27311 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:28.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.335+0000 7fad26ffd640 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5611c57088b0 0x5611c570ace0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce879f9999e31335 server_cookie=27311781733b8da in_seq=0 out_seq=0 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.336+0000 7fad06ffd640 1 -- 172.21.15.7:0/1797301684 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5611c5665910 con 0x5611c57088b0 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.336+0000 7fad06ffd640 10 client.27311 handle_command_reply: tid=0 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.336+0000 7fad3081c740 1 client.27311 shutdown 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.336+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 msgr2=0x5611c56fd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.336+0000 7fad3081c740 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd170 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fad20011500 tx=0x7fad20011180 comp rx=0 tx=0).stop 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.337+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 shutdown_connections 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.337+0000 7fad3081c740 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5611c57088b0 0x5611c570ace0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.337+0000 7fad3081c740 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 0x5611c57046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.337+0000 7fad06ffd640 0 client.27311 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.337+0000 7fad3081c740 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 0x5611c56fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.337+0000 7fad3081c740 1 --2- 172.21.15.7:0/1797301684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd170 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.337+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 >> 172.21.15.7:0/1797301684 conn(0x5611c56ee310 msgr2=0x5611c56ed530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:28.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.337+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 shutdown_connections 2024-03-26T08:50:28.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.338+0000 7fad3081c740 1 -- 172.21.15.7:0/1797301684 wait complete. 2024-03-26T08:50:28.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.338+0000 7fad3081c740 20 client.27311 trim_cache size 0 max 16384 2024-03-26T08:50:28.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.363+0000 7fad3081c740 1 Processor -- start 2024-03-26T08:50:28.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.363+0000 7fad3081c740 1 -- start start 2024-03-26T08:50:28.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.363+0000 7fad3081c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.363+0000 7fad3081c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 0x5611c57046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.363+0000 7fad3081c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 0x5611c56ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.363+0000 7fad3081c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281b9e00 con 0x5611c56f3790 2024-03-26T08:50:28.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.363+0000 7fad3081c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281b7900 con 0x5611c56f2da0 2024-03-26T08:50:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.363+0000 7fad3081c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281b5980 con 0x5611c56f8900 2024-03-26T08:50:28.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.365+0000 7fad27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.365+0000 7fad27fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34120/0 (socket says 172.21.15.7:34120) 2024-03-26T08:50:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.365+0000 7fad27fff640 1 -- 172.21.15.7:0/3717849412 learned_addr learned my addr 172.21.15.7:0/3717849412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.368+0000 7fad26ffd640 1 --2- 172.21.15.7:0/3717849412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 0x5611c57046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.368+0000 7fad27fff640 1 --2- 172.21.15.7:0/3717849412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 0x5611c56ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.370+0000 7fad27fff640 1 -- 172.21.15.7:0/3717849412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 msgr2=0x5611c56ecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:28.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.370+0000 7fad27fff640 1 --2- 172.21.15.7:0/3717849412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 0x5611c56ecca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.370+0000 7fad27fff640 1 -- 172.21.15.7:0/3717849412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 msgr2=0x5611c57046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:28.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.370+0000 7fad27fff640 1 --2- 172.21.15.7:0/3717849412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 0x5611c57046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.370+0000 7fad27fff640 1 -- 172.21.15.7:0/3717849412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611c56f7e20 con 0x5611c56f2da0 2024-03-26T08:50:28.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.371+0000 7fad27fff640 1 --2- 172.21.15.7:0/3717849412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fad20011150 tx=0x7fad200021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2125c161e714854 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:28.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.372+0000 7fad0d7fa640 1 -- 172.21.15.7:0/3717849412 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2001e040 con 0x5611c56f2da0 2024-03-26T08:50:28.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.373+0000 7fad0d7fa640 1 -- 172.21.15.7:0/3717849412 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad2001a4c0 con 0x5611c56f2da0 2024-03-26T08:50:28.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.373+0000 7fad0d7fa640 1 -- 172.21.15.7:0/3717849412 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000fba0 con 0x5611c56f2da0 2024-03-26T08:50:28.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.374+0000 7fad3081c740 1 -- 172.21.15.7:0/3717849412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 msgr2=0x5611c56fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.374+0000 7fad3081c740 1 --2- 172.21.15.7:0/3717849412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fad20011150 tx=0x7fad200021d0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.374+0000 7fad3081c740 1 -- 172.21.15.7:0/3717849412 shutdown_connections 2024-03-26T08:50:28.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.374+0000 7fad3081c740 1 --2- 172.21.15.7:0/3717849412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f8900 0x5611c56ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.374+0000 7fad3081c740 1 --2- 172.21.15.7:0/3717849412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f3790 0x5611c57046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.374+0000 7fad3081c740 1 --2- 172.21.15.7:0/3717849412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f2da0 0x5611c56fd490 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.374+0000 7fad3081c740 1 -- 172.21.15.7:0/3717849412 >> 172.21.15.7:0/3717849412 conn(0x5611c56ee310 msgr2=0x5611c56fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:28.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.378+0000 7fad3081c740 1 -- 172.21.15.7:0/3717849412 shutdown_connections 2024-03-26T08:50:28.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.378+0000 7fad3081c740 1 -- 172.21.15.7:0/3717849412 wait complete. 2024-03-26T08:50:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.379+0000 7fad3081c740 1 -- start start 2024-03-26T08:50:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.379+0000 7fad3081c740 1 Processor -- start 2024-03-26T08:50:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.379+0000 7fad3081c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f2da0 0x5611c5701810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.379+0000 7fad3081c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f3790 0x5611c5701d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.379+0000 7fad3081c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f8900 0x5611c5702d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.379+0000 7fad3081c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281b5f80 con 0x5611c56f2da0 2024-03-26T08:50:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.379+0000 7fad3081c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281b72a0 con 0x5611c56f8900 2024-03-26T08:50:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.379+0000 7fad3081c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281b65e0 con 0x5611c56f3790 2024-03-26T08:50:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad3081c740 10 client.0 _ll_register_callbacks cb 0x7fad180f3cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f2da0 0x5611c5701810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f2da0 0x5611c5701810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59702/0 (socket says 172.21.15.7:59702) 2024-03-26T08:50:28.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 -- 172.21.15.7:0/1679401487 learned_addr learned my addr 172.21.15.7:0/1679401487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad26ffd640 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f3790 0x5611c5701d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f8900 0x5611c5702d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 -- 172.21.15.7:0/1679401487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f3790 msgr2=0x5611c5701d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f3790 0x5611c5701d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 -- 172.21.15.7:0/1679401487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f8900 msgr2=0x5611c5702d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:28.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f8900 0x5611c5702d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 -- 172.21.15.7:0/1679401487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611c5703300 con 0x5611c56f2da0 2024-03-26T08:50:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.380+0000 7fad27fff640 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f2da0 0x5611c5701810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fad200116c0 tx=0x7fad200323a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83790e56f38c11d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:28.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.383+0000 7fad06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:50:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.384+0000 7fad06ffd640 1 -- 172.21.15.7:0/1679401487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2001e070 con 0x5611c56f2da0 2024-03-26T08:50:28.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.384+0000 7fad06ffd640 1 -- 172.21.15.7:0/1679401487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fad20019960 con 0x5611c56f2da0 2024-03-26T08:50:28.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.384+0000 7fad3081c740 1 -- 172.21.15.7:0/1679401487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad2001a4c0 con 0x5611c56f2da0 2024-03-26T08:50:28.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.392+0000 7fad06ffd640 1 -- 172.21.15.7:0/1679401487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad20019b60 con 0x5611c56f2da0 2024-03-26T08:50:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.395+0000 7fad3081c740 10 client.17484 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.395+0000 7fad3081c740 1 -- 172.21.15.7:0/1679401487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5611c5704610 con 0x5611c56f2da0 2024-03-26T08:50:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.400+0000 7fad06ffd640 1 -- 172.21.15.7:0/1679401487 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad2001a4c0 con 0x5611c56f2da0 2024-03-26T08:50:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.400+0000 7fad06ffd640 1 -- 172.21.15.7:0/1679401487 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fad2003d030 con 0x5611c56f2da0 2024-03-26T08:50:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.401+0000 7fad3081c740 10 client.17484 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.401+0000 7fad3081c740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.401+0000 7fad3081c740 20 client.17484 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.401+0000 7fad3081c740 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5611c57088b0 0x5611c57664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.401+0000 7fad3081c740 4 client.17484 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:50:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.401+0000 7fad3081c740 1 -- 172.21.15.7:0/1679401487 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5611c565fc60 con 0x5611c57088b0 2024-03-26T08:50:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.405+0000 7fad26ffd640 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5611c57088b0 0x5611c57664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.415+0000 7fad26ffd640 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5611c57088b0 0x5611c57664f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d16715544b3b9b server_cookie=124416101ef8e503 in_seq=0 out_seq=0 2024-03-26T08:50:28.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.415+0000 7fad06ffd640 10 client.17484 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:28.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.421+0000 7fad06ffd640 1 -- 172.21.15.7:0/1679401487 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5611c565fc60 con 0x5611c57088b0 2024-03-26T08:50:28.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.421+0000 7fad06ffd640 10 client.17484 handle_command_reply: tid=0 2024-03-26T08:50:28.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.422+0000 7fad3081c740 1 client.17484 shutdown 2024-03-26T08:50:28.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.423+0000 7fad3081c740 1 -- 172.21.15.7:0/1679401487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f2da0 msgr2=0x5611c5701810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.423+0000 7fad3081c740 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f2da0 0x5611c5701810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fad200116c0 tx=0x7fad200323a0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad3081c740 1 -- 172.21.15.7:0/1679401487 shutdown_connections 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad3081c740 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5611c57088b0 0x5611c57664f0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad3081c740 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611c56f8900 0x5611c5702d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad3081c740 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5611c56f3790 0x5611c5701d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad3081c740 1 --2- 172.21.15.7:0/1679401487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5611c56f2da0 0x5611c5701810 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad3081c740 1 -- 172.21.15.7:0/1679401487 >> 172.21.15.7:0/1679401487 conn(0x5611c56ee310 msgr2=0x5611c56f1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad06ffd640 0 client.17484 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad3081c740 1 -- 172.21.15.7:0/1679401487 shutdown_connections 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad3081c740 1 -- 172.21.15.7:0/1679401487 wait complete. 2024-03-26T08:50:28.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.424+0000 7fad3081c740 20 client.17484 trim_cache size 0 max 16384 2024-03-26T08:50:28.420 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:50:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:50:28.420 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:50:28.420 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:50:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 -- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad100a2af0 msgr2=0x7fad100a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad100a2af0 0x7fad100a4fb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5611c56f3b90 tx=0x7fad180f3e00 comp rx=0 tx=0).stop 2024-03-26T08:50:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 -- 172.21.15.7:0/1669549058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad2810c0b0 msgr2=0x7fad281a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad2810c0b0 0x7fad281a2400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fad1802c170 tx=0x7fad1802c650 comp rx=0 tx=0).stop 2024-03-26T08:50:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 -- 172.21.15.7:0/1669549058 shutdown_connections 2024-03-26T08:50:28.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fad100a2af0 0x7fad100a4fb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad2810dc60 0x7fad281a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad2810d2a0 0x7fad281a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 --2- 172.21.15.7:0/1669549058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad2810c0b0 0x7fad281a2400 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 -- 172.21.15.7:0/1669549058 >> 172.21.15.7:0/1669549058 conn(0x7fad2806c170 msgr2=0x7fad28110620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 -- 172.21.15.7:0/1669549058 shutdown_connections 2024-03-26T08:50:28.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.426+0000 7fad2db53640 1 -- 172.21.15.7:0/1669549058 wait complete. 2024-03-26T08:50:28.434 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:28 smithi007 ceph-mon[32462]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 684 op/s 2024-03-26T08:50:28.435 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:50:28.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:50:28.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:28 smithi016 ceph-mon[35851]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 684 op/s 2024-03-26T08:50:28.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:28 smithi044 ceph-mon[36461]: pgmap v601: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 18 MiB/s wr, 684 op/s 2024-03-26T08:50:28.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.575+0000 7f408eac6640 1 -- 172.21.15.7:0/189119702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4088102180 msgr2=0x7f4088108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.575+0000 7f408eac6640 1 --2- 172.21.15.7:0/189119702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4088102180 0x7f4088108a10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4080009ca0 tx=0x7f408002dfd0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.588+0000 7f408eac6640 1 -- 172.21.15.7:0/189119702 shutdown_connections 2024-03-26T08:50:28.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.588+0000 7f408eac6640 1 --2- 172.21.15.7:0/189119702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4088102180 0x7f4088108a10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.588+0000 7f408eac6640 1 --2- 172.21.15.7:0/189119702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40881017c0 0x7f4088101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.588+0000 7f408eac6640 1 --2- 172.21.15.7:0/189119702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40881005d0 0x7f40881009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.588+0000 7f408eac6640 1 -- 172.21.15.7:0/189119702 >> 172.21.15.7:0/189119702 conn(0x7f40880fbb40 msgr2=0x7f40880fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:28.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.595+0000 7f408eac6640 1 -- 172.21.15.7:0/189119702 shutdown_connections 2024-03-26T08:50:28.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.598+0000 7f408eac6640 1 -- 172.21.15.7:0/189119702 wait complete. 2024-03-26T08:50:28.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.600+0000 7f408eac6640 1 Processor -- start 2024-03-26T08:50:28.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.600+0000 7f408eac6640 1 -- start start 2024-03-26T08:50:28.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.600+0000 7f408eac6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40881005d0 0x7f4088198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.601+0000 7f408eac6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40881017c0 0x7f4088199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.601+0000 7f408eac6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4088102180 0x7f4088195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.601+0000 7f408eac6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408810b150 con 0x7f40881005d0 2024-03-26T08:50:28.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.601+0000 7f408eac6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408810afd0 con 0x7f4088102180 2024-03-26T08:50:28.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.601+0000 7f408eac6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408810b2d0 con 0x7f40881017c0 2024-03-26T08:50:28.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.602+0000 7f4087fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40881017c0 0x7f4088199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.602+0000 7f408c862640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40881005d0 0x7f4088198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.602+0000 7f408c862640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40881005d0 0x7f4088198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59714/0 (socket says 172.21.15.7:59714) 2024-03-26T08:50:28.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.602+0000 7f408c862640 1 -- 172.21.15.7:0/1733202804 learned_addr learned my addr 172.21.15.7:0/1733202804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:28.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.602+0000 7f408d063640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4088102180 0x7f4088195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.604+0000 7f408c862640 1 -- 172.21.15.7:0/1733202804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40881017c0 msgr2=0x7f4088199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.604+0000 7f408c862640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40881017c0 0x7f4088199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.604+0000 7f408c862640 1 -- 172.21.15.7:0/1733202804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4088102180 msgr2=0x7f4088195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.604+0000 7f408c862640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4088102180 0x7f4088195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.604+0000 7f408c862640 1 -- 172.21.15.7:0/1733202804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40881960f0 con 0x7f40881005d0 2024-03-26T08:50:28.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.604+0000 7f408c862640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40881005d0 0x7f4088198cc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f407c00d950 tx=0x7f407c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:28.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.606+0000 7f4085ffb640 1 -- 172.21.15.7:0/1733202804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c017020 con 0x7f40881005d0 2024-03-26T08:50:28.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.606+0000 7f4085ffb640 1 -- 172.21.15.7:0/1733202804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f407c009c30 con 0x7f40881005d0 2024-03-26T08:50:28.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.606+0000 7f4085ffb640 1 -- 172.21.15.7:0/1733202804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c00f740 con 0x7f40881005d0 2024-03-26T08:50:28.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.607+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4088199760 con 0x7f40881005d0 2024-03-26T08:50:28.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.607+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4088199c60 con 0x7f40881005d0 2024-03-26T08:50:28.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.608+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4088068520 con 0x7f40881005d0 2024-03-26T08:50:28.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.621+0000 7f4085ffb640 1 -- 172.21.15.7:0/1733202804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f407c01a020 con 0x7f40881005d0 2024-03-26T08:50:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.622+0000 7f4085ffb640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40780a29d0 0x7f40780a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.622+0000 7f4085ffb640 1 -- 172.21.15.7:0/1733202804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f407c0ce9e0 con 0x7f40881005d0 2024-03-26T08:50:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.622+0000 7f4085ffb640 1 -- 172.21.15.7:0/1733202804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f407c021020 con 0x7f40881005d0 2024-03-26T08:50:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.623+0000 7f4087fff640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40780a29d0 0x7f40780a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.623+0000 7f4087fff640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40780a29d0 0x7f40780a4e90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4074005d80 tx=0x7f4074005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.948+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4088107890 con 0x7f40881005d0 2024-03-26T08:50:28.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.949+0000 7f4085ffb640 1 -- 172.21.15.7:0/1733202804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f407c009de0 con 0x7f40881005d0 2024-03-26T08:50:28.944 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:50:28.944 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:50:28.944 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:50:28.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.951+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40780a29d0 msgr2=0x7f40780a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.951+0000 7f408eac6640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40780a29d0 0x7f40780a4e90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4074005d80 tx=0x7f4074005c90 comp rx=0 tx=0).stop 2024-03-26T08:50:28.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.951+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40881005d0 msgr2=0x7f4088198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:28.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.951+0000 7f408eac6640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40881005d0 0x7f4088198cc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f407c00d950 tx=0x7f407c00de10 comp rx=0 tx=0).stop 2024-03-26T08:50:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.952+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 shutdown_connections 2024-03-26T08:50:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.953+0000 7f408eac6640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f40780a29d0 0x7f40780a4e90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.953+0000 7f408eac6640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4088102180 0x7f4088195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.953+0000 7f408eac6640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f40881017c0 0x7f4088199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.953+0000 7f408eac6640 1 --2- 172.21.15.7:0/1733202804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40881005d0 0x7f4088198cc0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.953+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 >> 172.21.15.7:0/1733202804 conn(0x7f40880fbb40 msgr2=0x7f4088104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.953+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 shutdown_connections 2024-03-26T08:50:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:28.953+0000 7f408eac6640 1 -- 172.21.15.7:0/1733202804 wait complete. 2024-03-26T08:50:28.964 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:50:28.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:50:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.111+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/76776118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb81017b0 msgr2=0x7fcdb8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.111+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/76776118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb81017b0 0x7fcdb8101c30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009900 tx=0x7fcda002bc30 comp rx=0 tx=0).stop 2024-03-26T08:50:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.111+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/76776118 shutdown_connections 2024-03-26T08:50:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.111+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/76776118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb8102170 0x7fcdb8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.111+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/76776118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb81017b0 0x7fcdb8101c30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.111+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/76776118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb81005c0 0x7fcdb81009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.111+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/76776118 >> 172.21.15.7:0/76776118 conn(0x7fcdb80fbb30 msgr2=0x7fcdb80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.111+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/76776118 shutdown_connections 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.111+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/76776118 wait complete. 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdbd0f1640 1 Processor -- start 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdbd0f1640 1 -- start start 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdbd0f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb81005c0 0x7fcdb810af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdbd0f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb81017b0 0x7fcdb8109580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdbd0f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb8102170 0x7fcdb8109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdbd0f1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb810d3c0 con 0x7fcdb8102170 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdbd0f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb810d240 con 0x7fcdb81017b0 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdbd0f1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb810d540 con 0x7fcdb81005c0 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdb759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb8102170 0x7fcdb8109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb81005c0 0x7fcdb810af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb81005c0 0x7fcdb810af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43454/0 (socket says 172.21.15.7:43454) 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdb6d9d640 1 -- 172.21.15.7:0/3388355523 learned_addr learned my addr 172.21.15.7:0/3388355523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdb759e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb8102170 0x7fcdb8109ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59740/0 (socket says 172.21.15.7:59740) 2024-03-26T08:50:29.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.112+0000 7fcdaffff640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb81017b0 0x7fcdb8109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdaffff640 1 -- 172.21.15.7:0/3388355523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb81005c0 msgr2=0x7fcdb810af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdaffff640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb81005c0 0x7fcdb810af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdaffff640 1 -- 172.21.15.7:0/3388355523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb8102170 msgr2=0x7fcdb8109ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdaffff640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb8102170 0x7fcdb8109ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdaffff640 1 -- 172.21.15.7:0/3388355523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb810a350 con 0x7fcdb81017b0 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdb6d9d640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb81005c0 0x7fcdb810af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdaffff640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb81017b0 0x7fcdb8109580 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcda0036040 tx=0x7fcda0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdb4d99640 1 -- 172.21.15.7:0/3388355523 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0043020 con 0x7fcdb81017b0 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdb4d99640 1 -- 172.21.15.7:0/3388355523 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda00054e0 con 0x7fcdb81017b0 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdb4d99640 1 -- 172.21.15.7:0/3388355523 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0003e20 con 0x7fcdb81017b0 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/3388355523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb81a1910 con 0x7fcdb81017b0 2024-03-26T08:50:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.113+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/3388355523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb81a1d20 con 0x7fcdb81017b0 2024-03-26T08:50:29.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.116+0000 7fcdb4d99640 1 -- 172.21.15.7:0/3388355523 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcda0046020 con 0x7fcdb81017b0 2024-03-26T08:50:29.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.117+0000 7fcdb4d99640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda40a2950 0x7fcda40a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.117+0000 7fcdb4d99640 1 -- 172.21.15.7:0/3388355523 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcda00f0cf0 con 0x7fcdb81017b0 2024-03-26T08:50:29.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.117+0000 7fcdb6d9d640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda40a2950 0x7fcda40a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.117+0000 7fcdbfdb9740 1 Processor -- start 2024-03-26T08:50:29.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.117+0000 7fcdbfdb9740 1 -- start start 2024-03-26T08:50:29.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdb6d9d640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda40a2950 0x7fcda40a4e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcda80058c0 tx=0x7fcda8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68208e610 con 0x55c6820cc080 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6820313c0 con 0x55c6820d11f0 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c681fd9d60 con 0x55c6820cb690 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59752/0 (socket says 172.21.15.7:59752) 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdaffff640 1 -- 172.21.15.7:0/1122673760 learned_addr learned my addr 172.21.15.7:0/1122673760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdb759e640 1 --2- 172.21.15.7:0/1122673760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdaffff640 1 -- 172.21.15.7:0/1122673760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 msgr2=0x55c6820cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdaffff640 1 --2- 172.21.15.7:0/1122673760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdaffff640 1 -- 172.21.15.7:0/1122673760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 msgr2=0x55c6820d3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdaffff640 1 --2- 172.21.15.7:0/1122673760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820d3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdaffff640 1 -- 172.21.15.7:0/1122673760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c681d453f0 con 0x55c6820cc080 2024-03-26T08:50:29.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdb759e640 1 --2- 172.21.15.7:0/1122673760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820d3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:50:29.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.118+0000 7fcdaffff640 1 --2- 172.21.15.7:0/1122673760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820d0b80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcda00ba020 tx=0x7fcda0003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9da65cd1b71042 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.120+0000 7fcdad7fa640 1 -- 172.21.15.7:0/1122673760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00bcfb0 con 0x55c6820cc080 2024-03-26T08:50:29.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.120+0000 7fcdad7fa640 1 -- 172.21.15.7:0/1122673760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcda0005690 con 0x55c6820cc080 2024-03-26T08:50:29.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.120+0000 7fcdad7fa640 1 -- 172.21.15.7:0/1122673760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00ee8c0 con 0x55c6820cc080 2024-03-26T08:50:29.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.121+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1122673760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 msgr2=0x55c6820d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.121+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1122673760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820d0b80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcda00ba020 tx=0x7fcda0003840 comp rx=0 tx=0).stop 2024-03-26T08:50:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.123+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1122673760 shutdown_connections 2024-03-26T08:50:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.123+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1122673760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820d3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.123+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1122673760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820d0b80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.123+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1122673760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.123+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1122673760 >> 172.21.15.7:0/1122673760 conn(0x55c6820c6c00 msgr2=0x55c6820c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.123+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1122673760 shutdown_connections 2024-03-26T08:50:29.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.124+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1122673760 wait complete. 2024-03-26T08:50:29.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.124+0000 7fcdbfdb9740 1 -- start start 2024-03-26T08:50:29.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdbfdb9740 1 Processor -- start 2024-03-26T08:50:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 0x55c6820d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c681fed240 con 0x55c6820cb690 2024-03-26T08:50:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68202d460 con 0x55c6820cc080 2024-03-26T08:50:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c68208e610 con 0x55c6820d11f0 2024-03-26T08:50:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdbfdb9740 10 client.0 _ll_register_callbacks cb 0x55c681fecc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 0x55c6820d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdb759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59766/0 (socket says 172.21.15.7:59766) 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdb759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820dcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43468/0 (socket says 172.21.15.7:43468) 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 0x55c6820d5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34186/0 (socket says 172.21.15.7:34186) 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.125+0000 7fcdb6d9d640 1 -- 172.21.15.7:0/1312278883 learned_addr learned my addr 172.21.15.7:0/1312278883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcdb759e640 1 -- 172.21.15.7:0/1312278883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 msgr2=0x55c6820d5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcdb759e640 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 0x55c6820d5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcdb759e640 1 -- 172.21.15.7:0/1312278883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 msgr2=0x55c6820d5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcdb759e640 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcdb759e640 1 -- 172.21.15.7:0/1312278883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c681e0f7d0 con 0x55c6820d11f0 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcdb759e640 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820dcbb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0005fd0 tx=0x7fcdb0011e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7784b27d73b3939e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcd9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/1312278883 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0003c40 con 0x55c6820d11f0 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/1312278883 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcdb0003de0 con 0x55c6820d11f0 2024-03-26T08:50:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1312278883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c68203e900 con 0x55c6820d11f0 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/1312278883 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb000f950 con 0x55c6820d11f0 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcdbfdb9740 10 client.27346 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1312278883 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c6820dfff0 con 0x55c6820d11f0 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.126+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/1312278883 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c68203e900 con 0x55c6820d11f0 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/1312278883 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcdb001a410 con 0x55c6820d11f0 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcdbfdb9740 10 client.27346 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcdbfdb9740 10 client.27346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcdbfdb9740 20 client.27346 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55c6820e1b00 0x55c6820e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcdbfdb9740 4 client.27346 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1312278883 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c682038c50 con 0x55c6820e1b00 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcdb6d9d640 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55c6820e1b00 0x55c6820e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcd9e7fc640 10 client.27346 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:50:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.127+0000 7fcdb6d9d640 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55c6820e1b00 0x55c6820e3f60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96543acb8e718711 server_cookie=2f32654cee6862f8 in_seq=0 out_seq=0 2024-03-26T08:50:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/1312278883 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c682038c50 con 0x55c6820e1b00 2024-03-26T08:50:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcd9e7fc640 10 client.27346 handle_command_reply: tid=0 2024-03-26T08:50:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 client.27346 shutdown 2024-03-26T08:50:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1312278883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 msgr2=0x55c6820dcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820dcbb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0005fd0 tx=0x7fcdb0011e60 comp rx=0 tx=0).stop 2024-03-26T08:50:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1312278883 shutdown_connections 2024-03-26T08:50:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55c6820e1b00 0x55c6820e3f60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820dcbb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcd9e7fc640 0 client.27346 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:50:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 0x55c6820d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/1312278883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1312278883 >> 172.21.15.7:0/1312278883 conn(0x55c6820c6c00 msgr2=0x55c6820c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.128+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1312278883 shutdown_connections 2024-03-26T08:50:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.129+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/1312278883 wait complete. 2024-03-26T08:50:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.129+0000 7fcdbfdb9740 20 client.27346 trim_cache size 0 max 16384 2024-03-26T08:50:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.145+0000 7fcdbfdb9740 1 Processor -- start 2024-03-26T08:50:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.145+0000 7fcdbfdb9740 1 -- start start 2024-03-26T08:50:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 0x55c6820dcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb81acfb0 con 0x55c6820cb690 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb81aaab0 con 0x55c6820cc080 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb81a8b30 con 0x55c6820d11f0 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb759e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820d5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43470/0 (socket says 172.21.15.7:43470) 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 0x55c6820dcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59774/0 (socket says 172.21.15.7:59774) 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb759e640 1 -- 172.21.15.7:0/3624399782 learned_addr learned my addr 172.21.15.7:0/3624399782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb759e640 1 -- 172.21.15.7:0/3624399782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 msgr2=0x55c6820dcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb759e640 1 --2- 172.21.15.7:0/3624399782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 0x55c6820dcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb759e640 1 -- 172.21.15.7:0/3624399782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 msgr2=0x55c6820d5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb759e640 1 --2- 172.21.15.7:0/3624399782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb759e640 1 -- 172.21.15.7:0/3624399782 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6820c6550 con 0x55c6820d11f0 2024-03-26T08:50:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.146+0000 7fcdb759e640 1 --2- 172.21.15.7:0/3624399782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820d5b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcdb001a8d0 tx=0x7fcdb000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6a3477cd8ef157 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.148+0000 7fcdacff9640 1 -- 172.21.15.7:0/3624399782 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0029070 con 0x55c6820d11f0 2024-03-26T08:50:29.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.148+0000 7fcdacff9640 1 -- 172.21.15.7:0/3624399782 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdb001aac0 con 0x55c6820d11f0 2024-03-26T08:50:29.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.148+0000 7fcdacff9640 1 -- 172.21.15.7:0/3624399782 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0005800 con 0x55c6820d11f0 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.149+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/3624399782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 msgr2=0x55c6820d5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.149+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/3624399782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820d5b70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcdb001a8d0 tx=0x7fcdb000f790 comp rx=0 tx=0).stop 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.149+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/3624399782 shutdown_connections 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.149+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/3624399782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820d11f0 0x55c6820d5b70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.149+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/3624399782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820cc080 0x55c6820dcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.149+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/3624399782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cb690 0x55c6820d5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.149+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/3624399782 >> 172.21.15.7:0/3624399782 conn(0x55c6820c6c00 msgr2=0x55c6820cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.149+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/3624399782 shutdown_connections 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.149+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/3624399782 wait complete. 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdbfdb9740 1 -- start start 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdbfdb9740 1 Processor -- start 2024-03-26T08:50:29.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820da360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820da8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdbfdb9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820db860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb81a9130 con 0x55c6820cc080 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb81aa450 con 0x55c6820d11f0 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdbfdb9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb81a9790 con 0x55c6820cb690 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdbfdb9740 10 client.0 _ll_register_callbacks cb 0x7fcdb0011610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdaffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820da8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820da360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820da360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43482/0 (socket says 172.21.15.7:43482) 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb6d9d640 1 -- 172.21.15.7:0/41439786 learned_addr learned my addr 172.21.15.7:0/41439786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb759e640 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb6d9d640 1 -- 172.21.15.7:0/41439786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 msgr2=0x55c6820db860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb6d9d640 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820db860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb6d9d640 1 -- 172.21.15.7:0/41439786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 msgr2=0x55c6820da8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb6d9d640 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820da8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb6d9d640 1 -- 172.21.15.7:0/41439786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6820dbe50 con 0x55c6820cb690 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdb759e640 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820db860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.150+0000 7fcdaffff640 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820da8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcdb6d9d640 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820da360 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55c6820e47e0 tx=0x7fcda8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1729813aef55777 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcd9e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/41439786 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda802b0f0 con 0x55c6820cb690 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/41439786 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcda802b290 con 0x55c6820cb690 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/41439786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdb001aac0 con 0x55c6820cb690 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/41439786 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcda802bbc0 con 0x55c6820cb690 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcdbfdb9740 10 client.27352 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/41439786 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c68213d560 con 0x55c6820cb690 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/41439786 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdb001aac0 con 0x55c6820cb690 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/41439786 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcda801c410 con 0x55c6820cb690 2024-03-26T08:50:29.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcdbfdb9740 10 client.27352 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:29.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcdbfdb9740 10 client.27352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:50:29.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcdbfdb9740 20 client.27352 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:29.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.151+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55c6820e1b00 0x55c68213ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.152+0000 7fcdbfdb9740 4 client.27352 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:50:29.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.152+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/41439786 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c681fb6640 con 0x55c6820e1b00 2024-03-26T08:50:29.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.152+0000 7fcdaffff640 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55c6820e1b00 0x55c68213ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.152+0000 7fcd9e7fc640 10 client.27352 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:50:29.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.152+0000 7fcdaffff640 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55c6820e1b00 0x55c68213ff80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0edaa3462cf256a server_cookie=ec33ccda48d0896f in_seq=0 out_seq=0 2024-03-26T08:50:29.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.152+0000 7fcd9e7fc640 1 -- 172.21.15.7:0/41439786 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c681fb6640 con 0x55c6820e1b00 2024-03-26T08:50:29.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.152+0000 7fcd9e7fc640 10 client.27352 handle_command_reply: tid=0 2024-03-26T08:50:29.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.152+0000 7fcdbfdb9740 1 client.27352 shutdown 2024-03-26T08:50:29.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/41439786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 msgr2=0x55c6820da360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820da360 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x55c6820e47e0 tx=0x7fcda8004730 comp rx=0 tx=0).stop 2024-03-26T08:50:29.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/41439786 shutdown_connections 2024-03-26T08:50:29.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55c6820e1b00 0x55c68213ff80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6820d11f0 0x55c6820db860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c6820cc080 0x55c6820da8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcdbfdb9740 1 --2- 172.21.15.7:0/41439786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c6820cb690 0x55c6820da360 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/41439786 >> 172.21.15.7:0/41439786 conn(0x55c6820c6c00 msgr2=0x55c6820d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcd9e7fc640 0 client.27352 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:50:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.153+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/41439786 shutdown_connections 2024-03-26T08:50:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbfdb9740 1 -- 172.21.15.7:0/41439786 wait complete. 2024-03-26T08:50:29.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbfdb9740 20 client.27352 trim_cache size 0 max 16384 2024-03-26T08:50:29.152 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:50:29.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/3388355523 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda40a2950 msgr2=0x7fcda40a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda40a2950 0x7fcda40a4e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcda80058c0 tx=0x7fcda8005f90 comp rx=0 tx=0).stop 2024-03-26T08:50:29.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/3388355523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb81017b0 msgr2=0x7fcdb8109580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb81017b0 0x7fcdb8109580 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcda0036040 tx=0x7fcda0004b40 comp rx=0 tx=0).stop 2024-03-26T08:50:29.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/3388355523 shutdown_connections 2024-03-26T08:50:29.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda40a2950 0x7fcda40a4e10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb8102170 0x7fcdb8109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb81017b0 0x7fcdb8109580 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 --2- 172.21.15.7:0/3388355523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdb81005c0 0x7fcdb810af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/3388355523 >> 172.21.15.7:0/3388355523 conn(0x7fcdb80fbb30 msgr2=0x7fcdb81049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/3388355523 shutdown_connections 2024-03-26T08:50:29.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.154+0000 7fcdbd0f1640 1 -- 172.21.15.7:0/3388355523 wait complete. 2024-03-26T08:50:29.162 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:50:29.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:50:29.246 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:29 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1733202804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:50:29.295 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:29 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1733202804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:50:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.326+0000 7f1a2ceac640 1 -- 172.21.15.7:0/3582269660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a281016d0 msgr2=0x7f1a28101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.326+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/3582269660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a281016d0 0x7f1a28101af0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009900 tx=0x7f1a1802bc30 comp rx=0 tx=0).stop 2024-03-26T08:50:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.326+0000 7f1a2ceac640 1 -- 172.21.15.7:0/3582269660 shutdown_connections 2024-03-26T08:50:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.326+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/3582269660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a28103280 0x7f1a28109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.326+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/3582269660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a281028c0 0x7f1a28102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.326+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/3582269660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a281016d0 0x7f1a28101af0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.326+0000 7f1a2ceac640 1 -- 172.21.15.7:0/3582269660 >> 172.21.15.7:0/3582269660 conn(0x7f1a280fcc60 msgr2=0x7f1a280ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.326+0000 7f1a2ceac640 1 -- 172.21.15.7:0/3582269660 shutdown_connections 2024-03-26T08:50:29.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.326+0000 7f1a2ceac640 1 -- 172.21.15.7:0/3582269660 wait complete. 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a2ceac640 1 Processor -- start 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a2ceac640 1 -- start start 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a2ceac640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a281016d0 0x7f1a28199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a2ceac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a281028c0 0x7f1a2819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a2ceac640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a28103280 0x7f1a281a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a2ceac640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a28074460 con 0x7f1a28103280 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a2ceac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280742e0 con 0x7f1a281028c0 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a2ceac640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a280745e0 con 0x7f1a281016d0 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a26d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a28103280 0x7f1a281a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a281028c0 0x7f1a2819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a26d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a28103280 0x7f1a281a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59790/0 (socket says 172.21.15.7:59790) 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a26d9d640 1 -- 172.21.15.7:0/1264759553 learned_addr learned my addr 172.21.15.7:0/1264759553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.327+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a281016d0 0x7f1a28199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a25d9b640 1 -- 172.21.15.7:0/1264759553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a281016d0 msgr2=0x7f1a28199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a281016d0 0x7f1a28199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a25d9b640 1 -- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a28103280 msgr2=0x7f1a281a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a28103280 0x7f1a281a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a25d9b640 1 -- 172.21.15.7:0/1264759553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a281a1830 con 0x7f1a281028c0 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a26d9d640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a28103280 0x7f1a281a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a281016d0 0x7f1a28199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a281028c0 0x7f1a2819a160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00e9f0 tx=0x7f1a1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a177fe640 1 -- 172.21.15.7:0/1264759553 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00cd80 con 0x7f1a281028c0 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a177fe640 1 -- 172.21.15.7:0/1264759553 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c0035f0 con 0x7f1a281028c0 2024-03-26T08:50:29.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a2ceac640 1 -- 172.21.15.7:0/1264759553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a281a1b20 con 0x7f1a281028c0 2024-03-26T08:50:29.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a177fe640 1 -- 172.21.15.7:0/1264759553 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c010690 con 0x7f1a281028c0 2024-03-26T08:50:29.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.328+0000 7f1a2ceac640 1 -- 172.21.15.7:0/1264759553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a281a20c0 con 0x7f1a281028c0 2024-03-26T08:50:29.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.332+0000 7f1a2fb74740 1 Processor -- start 2024-03-26T08:50:29.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.332+0000 7f1a2fb74740 1 -- start start 2024-03-26T08:50:29.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.332+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac04690 0x56515ac04ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.332+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac05080 0x56515ac09b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.332+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 0x56515ac0c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.332+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515abc7610 con 0x56515ac05080 2024-03-26T08:50:29.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.332+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515ab6a3c0 con 0x56515ac0a1f0 2024-03-26T08:50:29.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.332+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515ab12d60 con 0x56515ac04690 2024-03-26T08:50:29.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a177fe640 1 -- 172.21.15.7:0/1264759553 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1a1c004020 con 0x7f1a281028c0 2024-03-26T08:50:29.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a26d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac04690 0x56515ac04ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a25d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 0x56515ac0c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a26d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac04690 0x56515ac04ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43502/0 (socket says 172.21.15.7:43502) 2024-03-26T08:50:29.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a26d9d640 1 -- 172.21.15.7:0/2889304810 learned_addr learned my addr 172.21.15.7:0/2889304810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a2659c640 1 --2- 172.21.15.7:0/2889304810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac05080 0x56515ac09b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a26d9d640 1 -- 172.21.15.7:0/2889304810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 msgr2=0x56515ac0c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a26d9d640 1 --2- 172.21.15.7:0/2889304810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 0x56515ac0c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a26d9d640 1 -- 172.21.15.7:0/2889304810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac05080 msgr2=0x56515ac09b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a26d9d640 1 --2- 172.21.15.7:0/2889304810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac05080 0x56515ac09b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a26d9d640 1 -- 172.21.15.7:0/2889304810 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56515a87e3f0 con 0x56515ac04690 2024-03-26T08:50:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a26d9d640 1 --2- 172.21.15.7:0/2889304810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac04690 0x56515ac04ab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1a200118b0 tx=0x7f1a20011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d163b04e8a25696 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.333+0000 7f1a14ff9640 1 -- 172.21.15.7:0/2889304810 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2001e070 con 0x56515ac04690 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.334+0000 7f1a14ff9640 1 -- 172.21.15.7:0/2889304810 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a20011310 con 0x56515ac04690 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.334+0000 7f1a177fe640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a100a2900 0x7f1a100a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.334+0000 7f1a14ff9640 1 -- 172.21.15.7:0/2889304810 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a20019e60 con 0x56515ac04690 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.334+0000 7f1a177fe640 1 -- 172.21.15.7:0/1264759553 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1a1c0ced80 con 0x7f1a281028c0 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.334+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a100a2900 0x7f1a100a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a100a2900 0x7f1a100a4dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1a2819b0f0 tx=0x7f1a1802c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2889304810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac04690 msgr2=0x56515ac04ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2889304810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac04690 0x56515ac04ab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1a200118b0 tx=0x7f1a20011d70 comp rx=0 tx=0).stop 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2889304810 shutdown_connections 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2889304810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 0x56515ac0c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2889304810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac05080 0x56515ac09b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2889304810 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac04690 0x56515ac04ab0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2889304810 >> 172.21.15.7:0/2889304810 conn(0x56515abffc00 msgr2=0x56515ac00000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2889304810 shutdown_connections 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2889304810 wait complete. 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 -- start start 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.335+0000 7f1a2fb74740 1 Processor -- start 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac04690 0x56515ac0ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac0ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac0a1f0 0x56515ac15f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515ab26240 con 0x56515ac0a1f0 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515ab66460 con 0x56515ac04690 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56515ac0d1f0 con 0x56515ac05080 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a26d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac0a1f0 0x56515ac15f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac0ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a26d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac0a1f0 0x56515ac15f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59798/0 (socket says 172.21.15.7:59798) 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac0ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43514/0 (socket says 172.21.15.7:43514) 2024-03-26T08:50:29.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a25d9b640 1 -- 172.21.15.7:0/1171730289 learned_addr learned my addr 172.21.15.7:0/1171730289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac04690 0x56515ac0ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a2fb74740 10 client.0 _ll_register_callbacks cb 0x56515ab25c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a25d9b640 1 -- 172.21.15.7:0/1171730289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac04690 msgr2=0x56515ac0ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac04690 0x56515ac0ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a25d9b640 1 -- 172.21.15.7:0/1171730289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac0a1f0 msgr2=0x56515ac15f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac0a1f0 0x56515ac15f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a25d9b640 1 -- 172.21.15.7:0/1171730289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56515a9487d0 con 0x56515ac05080 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a26d9d640 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac0a1f0 0x56515ac15f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac0ef80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0a3950 tx=0x7f1a1c09b220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5c7463004b0eaf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.336+0000 7f1a0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a0dffb640 1 -- 172.21.15.7:0/1171730289 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c009450 con 0x56515ac05080 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a0dffb640 1 -- 172.21.15.7:0/1171730289 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1a1c004310 con 0x56515ac05080 2024-03-26T08:50:29.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1171730289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a20011310 con 0x56515ac05080 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a0dffb640 1 -- 172.21.15.7:0/1171730289 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a1c086070 con 0x56515ac05080 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a2fb74740 10 client.27364 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1171730289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56515ac19330 con 0x56515ac05080 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a0dffb640 1 -- 172.21.15.7:0/1171730289 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a20011310 con 0x56515ac05080 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a0dffb640 1 -- 172.21.15.7:0/1171730289 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1a1c097cc0 con 0x56515ac05080 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a2fb74740 10 client.27364 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a2fb74740 10 client.27364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a2fb74740 20 client.27364 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56515ac1a9a0 0x56515ac1cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a2fb74740 4 client.27364 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.337+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1171730289 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56515ab77900 con 0x56515ac1a9a0 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.338+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56515ac1a9a0 0x56515ac1cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.338+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56515ac1a9a0 0x56515ac1cd90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e58b6487aec02342 server_cookie=41794955fd977e9e in_seq=0 out_seq=0 2024-03-26T08:50:29.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.338+0000 7f1a0dffb640 10 client.27364 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:50:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.339+0000 7f1a0dffb640 1 -- 172.21.15.7:0/1171730289 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56515ab77900 con 0x56515ac1a9a0 2024-03-26T08:50:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.339+0000 7f1a0dffb640 10 client.27364 handle_command_reply: tid=0 2024-03-26T08:50:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.339+0000 7f1a2fb74740 1 client.27364 shutdown 2024-03-26T08:50:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.339+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1171730289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 msgr2=0x56515ac0ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.339+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac0ef80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0a3950 tx=0x7f1a1c09b220 comp rx=0 tx=0).stop 2024-03-26T08:50:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.339+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1171730289 shutdown_connections 2024-03-26T08:50:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.339+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac0a1f0 0x56515ac15f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.339+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56515ac1a9a0 0x56515ac1cd90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.339+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac0ef80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.340+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1171730289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac04690 0x56515ac0ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.340+0000 7f1a0dffb640 0 client.27364 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:50:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.340+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1171730289 >> 172.21.15.7:0/1171730289 conn(0x56515abffc00 msgr2=0x56515abfee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.340+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1171730289 shutdown_connections 2024-03-26T08:50:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.341+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1171730289 wait complete. 2024-03-26T08:50:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.341+0000 7f1a2fb74740 20 client.27364 trim_cache size 0 max 16384 2024-03-26T08:50:29.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.357+0000 7f1a2fb74740 1 Processor -- start 2024-03-26T08:50:29.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2fb74740 1 -- start start 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515abfacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac15f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 0x56515ac06540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281ad310 con 0x56515ac04690 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281aae10 con 0x56515ac0a1f0 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281a8e90 con 0x56515ac05080 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515abfacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a25d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac15f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515abfacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59806/0 (socket says 172.21.15.7:59806) 2024-03-26T08:50:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a2659c640 1 -- 172.21.15.7:0/1296975617 learned_addr learned my addr 172.21.15.7:0/1296975617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.358+0000 7f1a26d9d640 1 --2- 172.21.15.7:0/1296975617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 0x56515ac06540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a2659c640 1 -- 172.21.15.7:0/1296975617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 msgr2=0x56515ac15f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1296975617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac15f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a2659c640 1 -- 172.21.15.7:0/1296975617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 msgr2=0x56515ac06540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1296975617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 0x56515ac06540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a2659c640 1 -- 172.21.15.7:0/1296975617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56515ac1a740 con 0x56515ac04690 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a26d9d640 1 --2- 172.21.15.7:0/1296975617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 0x56515ac06540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a2659c640 1 --2- 172.21.15.7:0/1296975617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515abfacb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009230 tx=0x7f1a18041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821769039583767f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a0ffff640 1 -- 172.21.15.7:0/1296975617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18045070 con 0x56515ac04690 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a0ffff640 1 -- 172.21.15.7:0/1296975617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a18005330 con 0x56515ac04690 2024-03-26T08:50:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.359+0000 7f1a0ffff640 1 -- 172.21.15.7:0/1296975617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a180035f0 con 0x56515ac04690 2024-03-26T08:50:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1296975617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 msgr2=0x56515abfacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1296975617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515abfacb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009230 tx=0x7f1a18041000 comp rx=0 tx=0).stop 2024-03-26T08:50:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1296975617 shutdown_connections 2024-03-26T08:50:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1296975617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac0a1f0 0x56515ac06540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1296975617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac05080 0x56515ac15f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/1296975617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515abfacb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1296975617 >> 172.21.15.7:0/1296975617 conn(0x56515abffc00 msgr2=0x56515ac0ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1296975617 shutdown_connections 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 -- 172.21.15.7:0/1296975617 wait complete. 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.360+0000 7f1a2fb74740 1 -- start start 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2fb74740 1 Processor -- start 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515ac138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac05080 0x56515ac13e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2fb74740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac0a1f0 0x56515ac14dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281a9490 con 0x56515ac04690 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281aa7b0 con 0x56515ac05080 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2fb74740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a281a9af0 con 0x56515ac0a1f0 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515ac138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2fb74740 10 client.0 _ll_register_callbacks cb 0x56515ab25c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515ac138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59814/0 (socket says 172.21.15.7:59814) 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2659c640 1 -- 172.21.15.7:0/2971929477 learned_addr learned my addr 172.21.15.7:0/2971929477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac05080 0x56515ac13e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a26d9d640 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac0a1f0 0x56515ac14dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a25d9b640 1 -- 172.21.15.7:0/2971929477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac0a1f0 msgr2=0x56515ac14dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac0a1f0 0x56515ac14dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a25d9b640 1 -- 172.21.15.7:0/2971929477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 msgr2=0x56515ac138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515ac138d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a25d9b640 1 -- 172.21.15.7:0/2971929477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56515ac153c0 con 0x56515ac05080 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.361+0000 7f1a2659c640 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515ac138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a25d9b640 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac05080 0x56515ac13e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c097ab0 tx=0x7f1a1c0d2430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed75a8685933ed6b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a0dffb640 1 -- 172.21.15.7:0/2971929477 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c09b040 con 0x56515ac05080 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a0dffb640 1 -- 172.21.15.7:0/2971929477 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1a1c09ab30 con 0x56515ac05080 2024-03-26T08:50:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2971929477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a18005330 con 0x56515ac05080 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a0dffb640 1 -- 172.21.15.7:0/2971929477 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a1c009430 con 0x56515ac05080 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a2fb74740 10 client.27359 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2971929477 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56515ac7ed00 con 0x56515ac05080 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a0dffb640 1 -- 172.21.15.7:0/2971929477 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a18005330 con 0x56515ac05080 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.362+0000 7f1a0dffb640 1 -- 172.21.15.7:0/2971929477 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1a1c0afcc0 con 0x56515ac05080 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.363+0000 7f1a2fb74740 10 client.27359 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.363+0000 7f1a2fb74740 10 client.27359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.363+0000 7f1a2fb74740 20 client.27359 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.363+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56515ac7f5f0 0x56515ac81a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.363+0000 7f1a2fb74740 4 client.27359 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.363+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2971929477 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56515ab71c50 con 0x56515ac7f5f0 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.363+0000 7f1a2659c640 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56515ac7f5f0 0x56515ac81a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.363+0000 7f1a2659c640 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56515ac7f5f0 0x56515ac81a10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=286019e4d9081643 server_cookie=7fbf03ef2c3c7d03 in_seq=0 out_seq=0 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.363+0000 7f1a0dffb640 10 client.27359 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a0dffb640 1 -- 172.21.15.7:0/2971929477 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56515ab71c50 con 0x56515ac7f5f0 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a0dffb640 10 client.27359 handle_command_reply: tid=0 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 client.27359 shutdown 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2971929477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac05080 msgr2=0x56515ac13e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac05080 0x56515ac13e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c097ab0 tx=0x7f1a1c0d2430 comp rx=0 tx=0).stop 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2971929477 shutdown_connections 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56515ac7f5f0 0x56515ac81a10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56515ac0a1f0 0x56515ac14dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a0dffb640 0 client.27359 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515ac05080 0x56515ac13e10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 --2- 172.21.15.7:0/2971929477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56515ac04690 0x56515ac138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2971929477 >> 172.21.15.7:0/2971929477 conn(0x56515abffc00 msgr2=0x56515ac0e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2971929477 shutdown_connections 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.364+0000 7f1a2fb74740 1 -- 172.21.15.7:0/2971929477 wait complete. 2024-03-26T08:50:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2fb74740 20 client.27359 trim_cache size 0 max 16384 2024-03-26T08:50:29.365 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 -- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a100a2900 msgr2=0x7f1a100a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a100a2900 0x7f1a100a4dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1a2819b0f0 tx=0x7f1a1802c880 comp rx=0 tx=0).stop 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 -- 172.21.15.7:0/1264759553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a281028c0 msgr2=0x7f1a2819a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a281028c0 0x7f1a2819a160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00e9f0 tx=0x7f1a1c00eeb0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 -- 172.21.15.7:0/1264759553 shutdown_connections 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1a100a2900 0x7f1a100a4dc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a28103280 0x7f1a281a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a281028c0 0x7f1a2819a160 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 --2- 172.21.15.7:0/1264759553 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a281016d0 0x7f1a28199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 -- 172.21.15.7:0/1264759553 >> 172.21.15.7:0/1264759553 conn(0x7f1a280fcc60 msgr2=0x7f1a280fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 -- 172.21.15.7:0/1264759553 shutdown_connections 2024-03-26T08:50:29.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.365+0000 7f1a2ceac640 1 -- 172.21.15.7:0/1264759553 wait complete. 2024-03-26T08:50:29.374 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:50:29.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:50:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.536+0000 7f46281b9640 1 -- 172.21.15.7:0/3280878005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46201073a0 msgr2=0x7f46201097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.536+0000 7f46281b9640 1 --2- 172.21.15.7:0/3280878005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46201073a0 0x7f46201097f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4618009ca0 tx=0x7f461802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.537+0000 7f46281b9640 1 -- 172.21.15.7:0/3280878005 shutdown_connections 2024-03-26T08:50:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.537+0000 7f46281b9640 1 --2- 172.21.15.7:0/3280878005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46201073a0 0x7f46201097f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.537+0000 7f46281b9640 1 --2- 172.21.15.7:0/3280878005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4620104a70 0x7f4620106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.537+0000 7f46281b9640 1 --2- 172.21.15.7:0/3280878005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102140 0x7f4620104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.537+0000 7f46281b9640 1 -- 172.21.15.7:0/3280878005 >> 172.21.15.7:0/3280878005 conn(0x7f46200fbb30 msgr2=0x7f46200fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.537+0000 7f46281b9640 1 -- 172.21.15.7:0/3280878005 shutdown_connections 2024-03-26T08:50:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.537+0000 7f46281b9640 1 -- 172.21.15.7:0/3280878005 wait complete. 2024-03-26T08:50:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f46281b9640 1 Processor -- start 2024-03-26T08:50:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f46281b9640 1 -- start start 2024-03-26T08:50:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f46281b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102140 0x7f46201957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f46281b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620104a70 0x7f4620195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f46281b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201073a0 0x7f462019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f46281b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462010d360 con 0x7f4620104a70 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f46281b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462010d1e0 con 0x7f4620102140 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f46281b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462010d4e0 con 0x7f46201073a0 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4625754640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620104a70 0x7f4620195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4626756640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201073a0 0x7f462019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4626756640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201073a0 0x7f462019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43550/0 (socket says 172.21.15.7:43550) 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4625f55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102140 0x7f46201957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4626756640 1 -- 172.21.15.7:0/884542942 learned_addr learned my addr 172.21.15.7:0/884542942 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4626756640 1 -- 172.21.15.7:0/884542942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102140 msgr2=0x7f46201957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4626756640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102140 0x7f46201957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4626756640 1 -- 172.21.15.7:0/884542942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620104a70 msgr2=0x7f4620195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4626756640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620104a70 0x7f4620195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.538+0000 7f4626756640 1 -- 172.21.15.7:0/884542942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462019d400 con 0x7f46201073a0 2024-03-26T08:50:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.539+0000 7f4626756640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201073a0 0x7f462019cd90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4618009bf0 tx=0x7f4618032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.539+0000 7f4616ffd640 1 -- 172.21.15.7:0/884542942 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618041020 con 0x7f46201073a0 2024-03-26T08:50:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.539+0000 7f46281b9640 1 -- 172.21.15.7:0/884542942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462019d690 con 0x7f46201073a0 2024-03-26T08:50:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.539+0000 7f4616ffd640 1 -- 172.21.15.7:0/884542942 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4618034cd0 con 0x7f46201073a0 2024-03-26T08:50:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.539+0000 7f4616ffd640 1 -- 172.21.15.7:0/884542942 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618039040 con 0x7f46201073a0 2024-03-26T08:50:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.539+0000 7f46281b9640 1 -- 172.21.15.7:0/884542942 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462019dbd0 con 0x7f46201073a0 2024-03-26T08:50:29.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.541+0000 7f4616ffd640 1 -- 172.21.15.7:0/884542942 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4618044020 con 0x7f46201073a0 2024-03-26T08:50:29.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.542+0000 7f4616ffd640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0a28d0 0x7f460c0a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.542+0000 7f4625f55640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0a28d0 0x7f460c0a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.542+0000 7f4616ffd640 1 -- 172.21.15.7:0/884542942 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f46180f07d0 con 0x7f46201073a0 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.542+0000 7f462ada5740 1 Processor -- start 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.542+0000 7f462ada5740 1 -- start start 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.542+0000 7f4625f55640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0a28d0 0x7f460c0a4d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f461c0095f0 tx=0x7f461c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f462ada5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9efab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f462ada5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f462ada5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19a9f7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f462ada5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19a9b2610 con 0x55e19a9f0080 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f462ada5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19a9553c0 con 0x55e19a9f51f0 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f462ada5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19a8fdd60 con 0x55e19a9ef690 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4625754640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4625f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4625754640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59838/0 (socket says 172.21.15.7:59838) 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4625f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9efab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43558/0 (socket says 172.21.15.7:43558) 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4625754640 1 -- 172.21.15.7:0/1108203184 learned_addr learned my addr 172.21.15.7:0/1108203184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4626756640 1 --2- 172.21.15.7:0/1108203184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19a9f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4626756640 1 -- 172.21.15.7:0/1108203184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 msgr2=0x55e19a9efab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4626756640 1 --2- 172.21.15.7:0/1108203184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9efab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4626756640 1 -- 172.21.15.7:0/1108203184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 msgr2=0x55e19a9f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4626756640 1 --2- 172.21.15.7:0/1108203184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4626756640 1 -- 172.21.15.7:0/1108203184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e19a6693f0 con 0x55e19a9f51f0 2024-03-26T08:50:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.543+0000 7f4626756640 1 --2- 172.21.15.7:0/1108203184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19a9f7680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4618006010 tx=0x7f461802c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71536b4a7205df86 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.544+0000 7f460bfff640 1 -- 172.21.15.7:0/1108203184 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180098b0 con 0x55e19a9f51f0 2024-03-26T08:50:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.544+0000 7f460bfff640 1 -- 172.21.15.7:0/1108203184 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46180348f0 con 0x55e19a9f51f0 2024-03-26T08:50:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.544+0000 7f460bfff640 1 -- 172.21.15.7:0/1108203184 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180bb500 con 0x55e19a9f51f0 2024-03-26T08:50:29.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.545+0000 7f462ada5740 1 -- 172.21.15.7:0/1108203184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 msgr2=0x55e19a9f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.545+0000 7f462ada5740 1 --2- 172.21.15.7:0/1108203184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19a9f7680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4618006010 tx=0x7f461802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.546+0000 7f462ada5740 1 -- 172.21.15.7:0/1108203184 shutdown_connections 2024-03-26T08:50:29.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.546+0000 7f462ada5740 1 --2- 172.21.15.7:0/1108203184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19a9f7680 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.546+0000 7f462ada5740 1 --2- 172.21.15.7:0/1108203184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.546+0000 7f462ada5740 1 --2- 172.21.15.7:0/1108203184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9efab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.546+0000 7f462ada5740 1 -- 172.21.15.7:0/1108203184 >> 172.21.15.7:0/1108203184 conn(0x55e19a9eac00 msgr2=0x55e19a9eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.547+0000 7f462ada5740 1 -- 172.21.15.7:0/1108203184 shutdown_connections 2024-03-26T08:50:29.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.547+0000 7f462ada5740 1 -- 172.21.15.7:0/1108203184 wait complete. 2024-03-26T08:50:29.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.547+0000 7f462ada5740 1 -- start start 2024-03-26T08:50:29.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f462ada5740 1 Processor -- start 2024-03-26T08:50:29.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f462ada5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f462ada5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f462ada5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19aa00d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f462ada5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19a911240 con 0x55e19a9f0080 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f462ada5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19a951460 con 0x55e19a9f51f0 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f462ada5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e19a9f7f60 con 0x55e19a9ef690 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f4625754640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f4625f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f4625f55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43564/0 (socket says 172.21.15.7:43564) 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f4625754640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59852/0 (socket says 172.21.15.7:59852) 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f4625f55640 1 -- 172.21.15.7:0/318421083 learned_addr learned my addr 172.21.15.7:0/318421083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f4626756640 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19aa00d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.548+0000 7f462ada5740 10 client.0 _ll_register_callbacks cb 0x55e19a910c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f4625f55640 1 -- 172.21.15.7:0/318421083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 msgr2=0x55e19aa00d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f4625f55640 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19aa00d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f4625f55640 1 -- 172.21.15.7:0/318421083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 msgr2=0x55e19a9f9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f4625f55640 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f4625f55640 1 -- 172.21.15.7:0/318421083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e19a7337d0 con 0x55e19a9ef690 2024-03-26T08:50:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f4626756640 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19aa00d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f4625f55640 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f461c00ed80 tx=0x7f461c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1106b0c35865836 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f46097fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:50:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f46097fa640 1 -- 172.21.15.7:0/318421083 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c01db20 con 0x55e19a9ef690 2024-03-26T08:50:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f46097fa640 1 -- 172.21.15.7:0/318421083 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f461c01dcc0 con 0x55e19a9ef690 2024-03-26T08:50:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f462ada5740 1 -- 172.21.15.7:0/318421083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46180348f0 con 0x55e19a9ef690 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.549+0000 7f46097fa640 1 -- 172.21.15.7:0/318421083 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f461c004030 con 0x55e19a9ef690 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f462ada5740 10 client.27382 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f462ada5740 1 -- 172.21.15.7:0/318421083 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e19aa040c0 con 0x55e19a9ef690 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f46097fa640 1 -- 172.21.15.7:0/318421083 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46180348f0 con 0x55e19a9ef690 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f46097fa640 1 -- 172.21.15.7:0/318421083 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f461c003c60 con 0x55e19a9ef690 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f462ada5740 10 client.27382 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f462ada5740 10 client.27382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f462ada5740 20 client.27382 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f462ada5740 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e19aa057e0 0x55e19aa07c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f462ada5740 4 client.27382 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f462ada5740 1 -- 172.21.15.7:0/318421083 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e19a962900 con 0x55e19aa057e0 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.550+0000 7f4625754640 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e19aa057e0 0x55e19aa07c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.551+0000 7f4625754640 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e19aa057e0 0x55e19aa07c50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb932aba876a9701 server_cookie=be22908525f4e750 in_seq=0 out_seq=0 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.551+0000 7f46097fa640 10 client.27382 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f46097fa640 1 -- 172.21.15.7:0/318421083 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e19a962900 con 0x55e19aa057e0 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f46097fa640 10 client.27382 handle_command_reply: tid=0 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 client.27382 shutdown 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 -- 172.21.15.7:0/318421083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 msgr2=0x55e19a9f9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9780 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f461c00ed80 tx=0x7f461c004730 comp rx=0 tx=0).stop 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 -- 172.21.15.7:0/318421083 shutdown_connections 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e19aa057e0 0x55e19aa07c50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19aa00d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f46097fa640 0 client.27382 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19a9f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 --2- 172.21.15.7:0/318421083 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9780 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 -- 172.21.15.7:0/318421083 >> 172.21.15.7:0/318421083 conn(0x55e19a9eac00 msgr2=0x55e19a9e9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 -- 172.21.15.7:0/318421083 shutdown_connections 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 1 -- 172.21.15.7:0/318421083 wait complete. 2024-03-26T08:50:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.552+0000 7f462ada5740 20 client.27382 trim_cache size 0 max 16384 2024-03-26T08:50:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.569+0000 7f462ada5740 1 Processor -- start 2024-03-26T08:50:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.570+0000 7f462ada5740 1 -- start start 2024-03-26T08:50:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.570+0000 7f462ada5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f462ada5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19aa00c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f462ada5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19a9f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f462ada5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201b9e10 con 0x55e19a9f0080 2024-03-26T08:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f462ada5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201b7910 con 0x55e19a9f51f0 2024-03-26T08:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f462ada5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201b5990 con 0x55e19a9ef690 2024-03-26T08:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4625754640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19aa00c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4625754640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19aa00c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59864/0 (socket says 172.21.15.7:59864) 2024-03-26T08:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4625754640 1 -- 172.21.15.7:0/506548033 learned_addr learned my addr 172.21.15.7:0/506548033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4625f55640 1 --2- 172.21.15.7:0/506548033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4626756640 1 --2- 172.21.15.7:0/506548033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19a9f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4625754640 1 -- 172.21.15.7:0/506548033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 msgr2=0x55e19a9f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4625754640 1 --2- 172.21.15.7:0/506548033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4625754640 1 -- 172.21.15.7:0/506548033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 msgr2=0x55e19a9f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4625754640 1 --2- 172.21.15.7:0/506548033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19a9f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.571+0000 7f4625754640 1 -- 172.21.15.7:0/506548033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e19aa05580 con 0x55e19a9f0080 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.572+0000 7f4625754640 1 --2- 172.21.15.7:0/506548033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19aa00c90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55e19a9ebb70 tx=0x7f4610005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce8daab033722ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.572+0000 7f460b7fe640 1 -- 172.21.15.7:0/506548033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610020270 con 0x55e19a9f0080 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.572+0000 7f460b7fe640 1 -- 172.21.15.7:0/506548033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4610004490 con 0x55e19a9f0080 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.572+0000 7f460b7fe640 1 -- 172.21.15.7:0/506548033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610010be0 con 0x55e19a9f0080 2024-03-26T08:50:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.573+0000 7f462ada5740 1 -- 172.21.15.7:0/506548033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 msgr2=0x55e19aa00c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.573+0000 7f462ada5740 1 --2- 172.21.15.7:0/506548033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19aa00c90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55e19a9ebb70 tx=0x7f4610005f90 comp rx=0 tx=0).stop 2024-03-26T08:50:29.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:29 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1733202804' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.573+0000 7f462ada5740 1 -- 172.21.15.7:0/506548033 shutdown_connections 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.573+0000 7f462ada5740 1 --2- 172.21.15.7:0/506548033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f51f0 0x55e19a9f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.573+0000 7f462ada5740 1 --2- 172.21.15.7:0/506548033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9f0080 0x55e19aa00c90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.573+0000 7f462ada5740 1 --2- 172.21.15.7:0/506548033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9ef690 0x55e19a9f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.573+0000 7f462ada5740 1 -- 172.21.15.7:0/506548033 >> 172.21.15.7:0/506548033 conn(0x55e19a9eac00 msgr2=0x55e19a9f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.574+0000 7f462ada5740 1 -- 172.21.15.7:0/506548033 shutdown_connections 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.574+0000 7f462ada5740 1 -- 172.21.15.7:0/506548033 wait complete. 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.574+0000 7f462ada5740 1 -- start start 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.574+0000 7f462ada5740 1 Processor -- start 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f462ada5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9ef690 0x55e19a9fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f462ada5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f0080 0x55e19a9fe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f462ada5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9f51f0 0x55e19a9ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f462ada5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201b5f90 con 0x55e19a9ef690 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f462ada5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201b72b0 con 0x55e19a9f0080 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f4625f55640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9ef690 0x55e19a9fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f4625754640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f0080 0x55e19a9fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f4625f55640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9ef690 0x55e19a9fe1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59874/0 (socket says 172.21.15.7:59874) 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f4625f55640 1 -- 172.21.15.7:0/2466585525 learned_addr learned my addr 172.21.15.7:0/2466585525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f4625754640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f0080 0x55e19a9fe710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34300/0 (socket says 172.21.15.7:34300) 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f4626756640 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9f51f0 0x55e19a9ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.575+0000 7f462ada5740 1 -- 172.21.15.7:0/2466585525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201b65f0 con 0x55e19a9f51f0 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f462ada5740 10 client.0 _ll_register_callbacks cb 0x7f461c00ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f4626756640 1 -- 172.21.15.7:0/2466585525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f0080 msgr2=0x55e19a9fe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f4626756640 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f0080 0x55e19a9fe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f4626756640 1 -- 172.21.15.7:0/2466585525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9ef690 msgr2=0x55e19a9fe1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f4626756640 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9ef690 0x55e19a9fe1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f4626756640 1 -- 172.21.15.7:0/2466585525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e19a9ffc70 con 0x55e19a9f51f0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f4625754640 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f0080 0x55e19a9fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f4625f55640 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9ef690 0x55e19a9fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f4626756640 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9f51f0 0x55e19a9ff680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f46180bd330 tx=0x7f46180c1170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc01ae17b4966f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f46097fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f46097fa640 1 -- 172.21.15.7:0/2466585525 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180bd170 con 0x55e19a9f51f0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f46097fa640 1 -- 172.21.15.7:0/2466585525 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f46180c5480 con 0x55e19a9f51f0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f462ada5740 1 -- 172.21.15.7:0/2466585525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4610004490 con 0x55e19a9f51f0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.576+0000 7f46097fa640 1 -- 172.21.15.7:0/2466585525 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46180348f0 con 0x55e19a9f51f0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f462ada5740 10 client.27388 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f462ada5740 1 -- 172.21.15.7:0/2466585525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e19aa00f30 con 0x55e19a9f51f0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f46097fa640 1 -- 172.21.15.7:0/2466585525 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4610004490 con 0x55e19a9f51f0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f46097fa640 1 -- 172.21.15.7:0/2466585525 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f461803c030 con 0x55e19a9f51f0 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f462ada5740 10 client.27388 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f462ada5740 10 client.27388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:29.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f462ada5740 20 client.27388 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f462ada5740 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e19aa057e0 0x55e19aa63460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f462ada5740 4 client.27388 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:50:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f462ada5740 1 -- 172.21.15.7:0/2466585525 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e19a95cc50 con 0x55e19aa057e0 2024-03-26T08:50:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.577+0000 7f4625f55640 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e19aa057e0 0x55e19aa63460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.580+0000 7f4625f55640 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e19aa057e0 0x55e19aa63460 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee4a90f70a10db server_cookie=565e291ea643dd81 in_seq=0 out_seq=0 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.580+0000 7f46097fa640 10 client.27388 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f46097fa640 1 -- 172.21.15.7:0/2466585525 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e19a95cc50 con 0x55e19aa057e0 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f46097fa640 10 client.27388 handle_command_reply: tid=0 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f462ada5740 1 client.27388 shutdown 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f462ada5740 1 -- 172.21.15.7:0/2466585525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9f51f0 msgr2=0x55e19a9ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f462ada5740 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9f51f0 0x55e19a9ff680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f46180bd330 tx=0x7f46180c1170 comp rx=0 tx=0).stop 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f462ada5740 1 -- 172.21.15.7:0/2466585525 shutdown_connections 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f462ada5740 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e19aa057e0 0x55e19aa63460 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f462ada5740 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e19a9f51f0 0x55e19a9ff680 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f462ada5740 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e19a9f0080 0x55e19a9fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f462ada5740 1 --2- 172.21.15.7:0/2466585525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e19a9ef690 0x55e19a9fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f462ada5740 1 -- 172.21.15.7:0/2466585525 >> 172.21.15.7:0/2466585525 conn(0x55e19a9eac00 msgr2=0x55e19a9f9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.581+0000 7f46097fa640 0 client.27388 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.582+0000 7f462ada5740 1 -- 172.21.15.7:0/2466585525 shutdown_connections 2024-03-26T08:50:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.582+0000 7f462ada5740 1 -- 172.21.15.7:0/2466585525 wait complete. 2024-03-26T08:50:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.582+0000 7f462ada5740 20 client.27388 trim_cache size 0 max 16384 2024-03-26T08:50:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.582+0000 7f46281b9640 1 -- 172.21.15.7:0/884542942 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0a28d0 msgr2=0x7f460c0a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.582+0000 7f46281b9640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0a28d0 0x7f460c0a4d90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f461c0095f0 tx=0x7f461c005f90 comp rx=0 tx=0).stop 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.582+0000 7f46281b9640 1 -- 172.21.15.7:0/884542942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201073a0 msgr2=0x7f462019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.582+0000 7f46281b9640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201073a0 0x7f462019cd90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4618009bf0 tx=0x7f4618032ea0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.583+0000 7f46281b9640 1 -- 172.21.15.7:0/884542942 shutdown_connections 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.583+0000 7f46281b9640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0a28d0 0x7f460c0a4d90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.583+0000 7f46281b9640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201073a0 0x7f462019cd90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.583+0000 7f46281b9640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620104a70 0x7f4620195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.583+0000 7f46281b9640 1 --2- 172.21.15.7:0/884542942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102140 0x7f46201957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.583+0000 7f46281b9640 1 -- 172.21.15.7:0/884542942 >> 172.21.15.7:0/884542942 conn(0x7f46200fbb30 msgr2=0x7f4620101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.583+0000 7f46281b9640 1 -- 172.21.15.7:0/884542942 shutdown_connections 2024-03-26T08:50:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:29.583+0000 7f46281b9640 1 -- 172.21.15.7:0/884542942 wait complete. 2024-03-26T08:50:30.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:30 smithi016 ceph-mon[35851]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 687 op/s 2024-03-26T08:50:30.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:30 smithi007 ceph-mon[32462]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 687 op/s 2024-03-26T08:50:30.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:30 smithi044 ceph-mon[36461]: pgmap v602: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 18 MiB/s wr, 687 op/s 2024-03-26T08:50:30.593 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:50:30.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:50:30.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.763+0000 7f7cba1f4640 1 -- 172.21.15.7:0/3308650675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cb41021c0 msgr2=0x7f7cb41045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.763+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/3308650675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cb41021c0 0x7f7cb41045b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009860 tx=0x7f7ca402bb70 comp rx=0 tx=0).stop 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.763+0000 7f7cba1f4640 1 -- 172.21.15.7:0/3308650675 shutdown_connections 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.763+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/3308650675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb4107420 0x7f7cb4109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.763+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/3308650675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cb4104af0 0x7f7cb4106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.763+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/3308650675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cb41021c0 0x7f7cb41045b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.764+0000 7f7cba1f4640 1 -- 172.21.15.7:0/3308650675 >> 172.21.15.7:0/3308650675 conn(0x7f7cb40fbb70 msgr2=0x7f7cb40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.764+0000 7f7cba1f4640 1 -- 172.21.15.7:0/3308650675 shutdown_connections 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.764+0000 7f7cba1f4640 1 -- 172.21.15.7:0/3308650675 wait complete. 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.764+0000 7f7cba1f4640 1 Processor -- start 2024-03-26T08:50:30.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.764+0000 7f7cba1f4640 1 -- start start 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.764+0000 7f7cba1f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cb41021c0 0x7f7cb4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.764+0000 7f7cba1f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cb4104af0 0x7f7cb4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.764+0000 7f7cba1f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb4107420 0x7f7cb419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cba1f4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb410d360 con 0x7f7cb4107420 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cba1f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb410d1e0 con 0x7f7cb4104af0 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cba1f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb410d4e0 con 0x7f7cb41021c0 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb4107420 0x7f7cb419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cb4104af0 0x7f7cb4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cb41021c0 0x7f7cb4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb4107420 0x7f7cb419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59894/0 (socket says 172.21.15.7:59894) 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cb41021c0 0x7f7cb4197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43606/0 (socket says 172.21.15.7:43606) 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb3fff640 1 -- 172.21.15.7:0/53583831 learned_addr learned my addr 172.21.15.7:0/53583831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cb4104af0 0x7f7cb4197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34312/0 (socket says 172.21.15.7:34312) 2024-03-26T08:50:30.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb37fe640 1 -- 172.21.15.7:0/53583831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cb4104af0 msgr2=0x7f7cb4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cb4104af0 0x7f7cb4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb37fe640 1 -- 172.21.15.7:0/53583831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb4107420 msgr2=0x7f7cb419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb4107420 0x7f7cb419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb37fe640 1 -- 172.21.15.7:0/53583831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb419f680 con 0x7f7cb41021c0 2024-03-26T08:50:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.765+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cb41021c0 0x7f7cb4197a60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7ca402c0b0 tx=0x7f7ca4032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.766+0000 7f7cb17fa640 1 -- 172.21.15.7:0/53583831 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4041020 con 0x7f7cb41021c0 2024-03-26T08:50:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.766+0000 7f7cb17fa640 1 -- 172.21.15.7:0/53583831 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ca402c9f0 con 0x7f7cb41021c0 2024-03-26T08:50:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.766+0000 7f7cb17fa640 1 -- 172.21.15.7:0/53583831 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4039040 con 0x7f7cb41021c0 2024-03-26T08:50:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.766+0000 7f7cba1f4640 1 -- 172.21.15.7:0/53583831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb40fffb0 con 0x7f7cb41021c0 2024-03-26T08:50:30.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.766+0000 7f7cba1f4640 1 -- 172.21.15.7:0/53583831 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb4100470 con 0x7f7cb41021c0 2024-03-26T08:50:30.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.769+0000 7f7cbcde5740 1 Processor -- start 2024-03-26T08:50:30.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.769+0000 7f7cbcde5740 1 -- start start 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.769+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b345970 0x55d63b345d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.769+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b346360 0x55d63b34ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.769+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b34d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.769+0000 7f7cbcde5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d63b274c90 con 0x55d63b34b4d0 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.769+0000 7f7cbcde5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d63b29a070 con 0x55d63b346360 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.769+0000 7f7cbcde5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d63b242c80 con 0x55d63b345970 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b34d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b34d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59896/0 (socket says 172.21.15.7:59896) 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cabfff640 1 -- 172.21.15.7:0/3485466269 learned_addr learned my addr 172.21.15.7:0/3485466269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cb3fff640 1 --2- 172.21.15.7:0/3485466269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b345970 0x55d63b345d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/3485466269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b346360 0x55d63b34ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cabfff640 1 -- 172.21.15.7:0/3485466269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b345970 msgr2=0x55d63b345d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cabfff640 1 --2- 172.21.15.7:0/3485466269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b345970 0x55d63b345d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cabfff640 1 -- 172.21.15.7:0/3485466269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b346360 msgr2=0x55d63b34ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cabfff640 1 --2- 172.21.15.7:0/3485466269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b346360 0x55d63b34ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cb17fa640 1 -- 172.21.15.7:0/53583831 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7ca4044020 con 0x7f7cb41021c0 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.770+0000 7f7cabfff640 1 -- 172.21.15.7:0/3485466269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d63b23ad10 con 0x55d63b34b4d0 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.771+0000 7f7cabfff640 1 --2- 172.21.15.7:0/3485466269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b34d960 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009c60 tx=0x7f7c9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee560d6aedf2637 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.771+0000 7f7cb17fa640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7ca00a2880 0x7f7ca00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.771+0000 7f7ca9ffb640 1 -- 172.21.15.7:0/3485466269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c004020 con 0x55d63b34b4d0 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.771+0000 7f7cb3fff640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7ca00a2880 0x7f7ca00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.771+0000 7f7cb17fa640 1 -- 172.21.15.7:0/53583831 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7ca40f0280 con 0x7f7cb41021c0 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.771+0000 7f7ca9ffb640 1 -- 172.21.15.7:0/3485466269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c9c0041c0 con 0x55d63b34b4d0 2024-03-26T08:50:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.771+0000 7f7ca9ffb640 1 -- 172.21.15.7:0/3485466269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c019720 con 0x55d63b34b4d0 2024-03-26T08:50:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cbcde5740 1 -- 172.21.15.7:0/3485466269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 msgr2=0x55d63b34d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cb3fff640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7ca00a2880 0x7f7ca00a4d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x55d63b346bc0 tx=0x7f7cac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/3485466269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b34d960 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009c60 tx=0x7f7c9c011810 comp rx=0 tx=0).stop 2024-03-26T08:50:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cbcde5740 1 -- 172.21.15.7:0/3485466269 shutdown_connections 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/3485466269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b34d960 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/3485466269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b346360 0x55d63b34ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/3485466269 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b345970 0x55d63b345d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cbcde5740 1 -- 172.21.15.7:0/3485466269 >> 172.21.15.7:0/3485466269 conn(0x55d63b2fd1a0 msgr2=0x55d63b3302d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cbcde5740 1 -- 172.21.15.7:0/3485466269 shutdown_connections 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.772+0000 7f7cbcde5740 1 -- 172.21.15.7:0/3485466269 wait complete. 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cbcde5740 1 -- start start 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cbcde5740 1 Processor -- start 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b345970 0x55d63b34fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b350240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b34b4d0 0x55d63b357280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cbcde5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d63b2560d0 con 0x55d63b345970 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cbcde5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d63b296110 con 0x55d63b34b4d0 2024-03-26T08:50:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cbcde5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d63b34e480 con 0x55d63b346360 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cbcde5740 10 client.0 _ll_register_callbacks cb 0x55d63b291e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b345970 0x55d63b34fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b350240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b345970 0x55d63b34fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59904/0 (socket says 172.21.15.7:59904) 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cb37fe640 1 -- 172.21.15.7:0/2357894533 learned_addr learned my addr 172.21.15.7:0/2357894533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.773+0000 7f7cb3fff640 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b34b4d0 0x55d63b357280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7cabfff640 1 -- 172.21.15.7:0/2357894533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b34b4d0 msgr2=0x55d63b357280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7cabfff640 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b34b4d0 0x55d63b357280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7cabfff640 1 -- 172.21.15.7:0/2357894533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b345970 msgr2=0x55d63b34fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7cabfff640 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b345970 0x55d63b34fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7cabfff640 1 -- 172.21.15.7:0/2357894533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d63b0767e0 con 0x55d63b346360 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7cabfff640 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b350240 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009230 tx=0x7f7c9c003b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c20d80b900e0cfb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7c92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:50:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7c92ffd640 1 -- 172.21.15.7:0/2357894533 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c029020 con 0x55d63b346360 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7c92ffd640 1 -- 172.21.15.7:0/2357894533 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c9c005040 con 0x55d63b346360 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7cbcde5740 1 -- 172.21.15.7:0/2357894533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c9c0041c0 con 0x55d63b346360 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7c92ffd640 1 -- 172.21.15.7:0/2357894533 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c9c005260 con 0x55d63b346360 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7cbcde5740 10 client.27400 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7cbcde5740 1 -- 172.21.15.7:0/2357894533 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d63b0d6f70 con 0x55d63b346360 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.774+0000 7f7c92ffd640 1 -- 172.21.15.7:0/2357894533 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c9c0041c0 con 0x55d63b346360 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.775+0000 7f7c92ffd640 1 -- 172.21.15.7:0/2357894533 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7c9c0194a0 con 0x55d63b346360 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.775+0000 7f7cbcde5740 10 client.27400 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.775+0000 7f7cbcde5740 10 client.27400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.775+0000 7f7cbcde5740 20 client.27400 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.775+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d63b35b7d0 0x55d63b35dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.775+0000 7f7cbcde5740 4 client.27400 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.775+0000 7f7cbcde5740 1 -- 172.21.15.7:0/2357894533 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d63b2a75b0 con 0x55d63b35b7d0 2024-03-26T08:50:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.775+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d63b35b7d0 0x55d63b35dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.777+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d63b35b7d0 0x55d63b35dc00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=414f3248e7a0ec0b server_cookie=50f0c0a2982c3018 in_seq=0 out_seq=0 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.777+0000 7f7c92ffd640 10 client.27400 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.778+0000 7f7c92ffd640 1 -- 172.21.15.7:0/2357894533 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d63b2a75b0 con 0x55d63b35b7d0 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.778+0000 7f7c92ffd640 10 client.27400 handle_command_reply: tid=0 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.778+0000 7f7cbcde5740 1 client.27400 shutdown 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.778+0000 7f7cbcde5740 1 -- 172.21.15.7:0/2357894533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 msgr2=0x55d63b350240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.778+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b350240 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009230 tx=0x7f7c9c003b70 comp rx=0 tx=0).stop 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7cbcde5740 1 -- 172.21.15.7:0/2357894533 shutdown_connections 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d63b35b7d0 0x55d63b35dc00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b34b4d0 0x55d63b357280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b350240 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/2357894533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b345970 0x55d63b34fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7c92ffd640 0 client.27400 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7cbcde5740 1 -- 172.21.15.7:0/2357894533 >> 172.21.15.7:0/2357894533 conn(0x55d63b2fd1a0 msgr2=0x55d63b32f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7cbcde5740 1 -- 172.21.15.7:0/2357894533 shutdown_connections 2024-03-26T08:50:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7cbcde5740 1 -- 172.21.15.7:0/2357894533 wait complete. 2024-03-26T08:50:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.779+0000 7f7cbcde5740 20 client.27400 trim_cache size 0 max 16384 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.801+0000 7f7cbcde5740 1 Processor -- start 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.801+0000 7f7cbcde5740 1 -- start start 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 0x55d63b32eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b357240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b32f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cbcde5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb41a4a60 con 0x55d63b34b4d0 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cbcde5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4072110 con 0x55d63b345970 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cbcde5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4070190 con 0x55d63b346360 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b357240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b357240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43634/0 (socket says 172.21.15.7:43634) 2024-03-26T08:50:30.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cabfff640 1 -- 172.21.15.7:0/1569591289 learned_addr learned my addr 172.21.15.7:0/1569591289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b32f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/1569591289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 0x55d63b32eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cabfff640 1 -- 172.21.15.7:0/1569591289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 msgr2=0x55d63b32eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cabfff640 1 --2- 172.21.15.7:0/1569591289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 0x55d63b32eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cabfff640 1 -- 172.21.15.7:0/1569591289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 msgr2=0x55d63b32f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cabfff640 1 --2- 172.21.15.7:0/1569591289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b32f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cabfff640 1 -- 172.21.15.7:0/1569591289 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d63b34edc0 con 0x55d63b346360 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/1569591289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 0x55d63b32eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7cabfff640 1 --2- 172.21.15.7:0/1569591289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b357240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0198f0 tx=0x7f7c9c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1a5b087033dd59e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7ca97fa640 1 -- 172.21.15.7:0/1569591289 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c03d070 con 0x55d63b346360 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7ca97fa640 1 -- 172.21.15.7:0/1569591289 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c9c0041c0 con 0x55d63b346360 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.802+0000 7f7ca97fa640 1 -- 172.21.15.7:0/1569591289 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c029040 con 0x55d63b346360 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.803+0000 7f7cbcde5740 1 -- 172.21.15.7:0/1569591289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 msgr2=0x55d63b357240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.803+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/1569591289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b357240 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0198f0 tx=0x7f7c9c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.803+0000 7f7cbcde5740 1 -- 172.21.15.7:0/1569591289 shutdown_connections 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.803+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/1569591289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b34b4d0 0x55d63b32f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.803+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/1569591289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b346360 0x55d63b357240 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.803+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/1569591289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 0x55d63b32eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.803+0000 7f7cbcde5740 1 -- 172.21.15.7:0/1569591289 >> 172.21.15.7:0/1569591289 conn(0x55d63b2fd1a0 msgr2=0x55d63b347860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.804+0000 7f7cbcde5740 1 -- 172.21.15.7:0/1569591289 shutdown_connections 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.804+0000 7f7cbcde5740 1 -- 172.21.15.7:0/1569591289 wait complete. 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.804+0000 7f7cbcde5740 1 -- start start 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.804+0000 7f7cbcde5740 1 Processor -- start 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.804+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 0x55d63b3550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b346360 0x55d63b355630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cbcde5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b34b4d0 0x55d63b356640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b346360 0x55d63b355630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cabfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b346360 0x55d63b355630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59926/0 (socket says 172.21.15.7:59926) 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cabfff640 1 -- 172.21.15.7:0/4129532765 learned_addr learned my addr 172.21.15.7:0/4129532765 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4070790 con 0x55d63b346360 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4071ab0 con 0x55d63b345970 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4070df0 con 0x55d63b34b4d0 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cb3fff640 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b34b4d0 0x55d63b356640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cbcde5740 10 client.0 _ll_register_callbacks cb 0x7f7ca40f3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cabfff640 1 -- 172.21.15.7:0/4129532765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b34b4d0 msgr2=0x55d63b356640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cabfff640 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b34b4d0 0x55d63b356640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cabfff640 1 -- 172.21.15.7:0/4129532765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 msgr2=0x55d63b3550f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cabfff640 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 0x55d63b3550f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.805+0000 7f7cabfff640 1 -- 172.21.15.7:0/4129532765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d63af6e170 con 0x55d63b346360 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.806+0000 7f7cabfff640 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b346360 0x55d63b355630 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c03b040 tx=0x7f7c9c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947ce6d1563a50ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.806+0000 7f7c92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.806+0000 7f7c92ffd640 1 -- 172.21.15.7:0/4129532765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c03d030 con 0x55d63b346360 2024-03-26T08:50:30.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.806+0000 7f7c92ffd640 1 -- 172.21.15.7:0/4129532765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c9c009d50 con 0x55d63b346360 2024-03-26T08:50:30.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.806+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c9c0041c0 con 0x55d63b346360 2024-03-26T08:50:30.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7c92ffd640 1 -- 172.21.15.7:0/4129532765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c9c029070 con 0x55d63b346360 2024-03-26T08:50:30.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7cbcde5740 10 client.17562 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:30.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d63b3b76e0 con 0x55d63b346360 2024-03-26T08:50:30.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7c92ffd640 1 -- 172.21.15.7:0/4129532765 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c9c0041c0 con 0x55d63b346360 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7c92ffd640 1 -- 172.21.15.7:0/4129532765 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7c9c037030 con 0x55d63b346360 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7cbcde5740 10 client.17562 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7cbcde5740 10 client.17562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7cbcde5740 20 client.17562 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d63b3b7ea0 0x55d63b3b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7cbcde5740 4 client.17562 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.807+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d63b32baa0 con 0x55d63b3b7ea0 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.808+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d63b3b7ea0 0x55d63b3b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.808+0000 7f7c92ffd640 10 client.17562 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.808+0000 7f7cb37fe640 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d63b3b7ea0 0x55d63b3b82b0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6864c42dbdbd151 server_cookie=804207685ce2ab2a in_seq=0 out_seq=0 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.809+0000 7f7c92ffd640 1 -- 172.21.15.7:0/4129532765 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d63b32baa0 con 0x55d63b3b7ea0 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.809+0000 7f7c92ffd640 10 client.17562 handle_command_reply: tid=0 2024-03-26T08:50:30.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 client.17562 shutdown 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b346360 msgr2=0x55d63b355630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b346360 0x55d63b355630 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c03b040 tx=0x7f7c9c009b60 comp rx=0 tx=0).stop 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 shutdown_connections 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d63b3b7ea0 0x55d63b3b82b0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d63b34b4d0 0x55d63b356640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d63b346360 0x55d63b355630 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7c92ffd640 0 client.17562 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 --2- 172.21.15.7:0/4129532765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d63b345970 0x55d63b3550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 >> 172.21.15.7:0/4129532765 conn(0x55d63b2fd1a0 msgr2=0x55d63b32eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 shutdown_connections 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 1 -- 172.21.15.7:0/4129532765 wait complete. 2024-03-26T08:50:30.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.810+0000 7f7cbcde5740 20 client.17562 trim_cache size 0 max 16384 2024-03-26T08:50:30.808 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:50:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:50:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "7ba6260a-1cda-4963-9a80-6127c4fcb2f6", 2024-03-26T08:50:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:50:30.808 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 -- 172.21.15.7:0/53583831 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7ca00a2880 msgr2=0x7f7ca00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7ca00a2880 0x7f7ca00a4d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x55d63b346bc0 tx=0x7f7cac009500 comp rx=0 tx=0).stop 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 -- 172.21.15.7:0/53583831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cb41021c0 msgr2=0x7f7cb4197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cb41021c0 0x7f7cb4197a60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7ca402c0b0 tx=0x7f7ca4032430 comp rx=0 tx=0).stop 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 -- 172.21.15.7:0/53583831 shutdown_connections 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7ca00a2880 0x7f7ca00a4d40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb4107420 0x7f7cb419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cb4104af0 0x7f7cb4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 --2- 172.21.15.7:0/53583831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7cb41021c0 0x7f7cb4197a60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.811+0000 7f7cba1f4640 1 -- 172.21.15.7:0/53583831 >> 172.21.15.7:0/53583831 conn(0x7f7cb40fbb70 msgr2=0x7f7cb4102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.812+0000 7f7cba1f4640 1 -- 172.21.15.7:0/53583831 shutdown_connections 2024-03-26T08:50:30.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.812+0000 7f7cba1f4640 1 -- 172.21.15.7:0/53583831 wait complete. 2024-03-26T08:50:30.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:50:30.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:50:30.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.983+0000 7f71f1c34640 1 -- 172.21.15.7:0/2338451014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec101c40 msgr2=0x7f71ec102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.983+0000 7f71f1c34640 1 --2- 172.21.15.7:0/2338451014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec101c40 0x7f71ec102060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009860 tx=0x7f71dc02bb70 comp rx=0 tx=0).stop 2024-03-26T08:50:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.984+0000 7f71f1c34640 1 -- 172.21.15.7:0/2338451014 shutdown_connections 2024-03-26T08:50:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.984+0000 7f71f1c34640 1 --2- 172.21.15.7:0/2338451014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ec1000f0 0x7f71ec1005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.984+0000 7f71f1c34640 1 --2- 172.21.15.7:0/2338451014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ec0ff730 0x7f71ec0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.984+0000 7f71f1c34640 1 --2- 172.21.15.7:0/2338451014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec101c40 0x7f71ec102060 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.984+0000 7f71f1c34640 1 -- 172.21.15.7:0/2338451014 >> 172.21.15.7:0/2338451014 conn(0x7f71ec0fb2b0 msgr2=0x7f71ec0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:30.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.985+0000 7f71f1c34640 1 -- 172.21.15.7:0/2338451014 shutdown_connections 2024-03-26T08:50:30.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.986+0000 7f71f1c34640 1 -- 172.21.15.7:0/2338451014 wait complete. 2024-03-26T08:50:30.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.986+0000 7f71f1c34640 1 Processor -- start 2024-03-26T08:50:30.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71f1c34640 1 -- start start 2024-03-26T08:50:30.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71f1c34640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec0ff730 0x7f71ec199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71f1c34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ec1000f0 0x7f71ec19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71f1c34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ec101c40 0x7f71ec1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71f1c34640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec10f520 con 0x7f71ec1000f0 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71eaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ec1000f0 0x7f71ec19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec0ff730 0x7f71ec199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec0ff730 0x7f71ec199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43666/0 (socket says 172.21.15.7:43666) 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71eaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ec1000f0 0x7f71ec19a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59950/0 (socket says 172.21.15.7:59950) 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71eb7fe640 1 -- 172.21.15.7:0/3396320344 learned_addr learned my addr 172.21.15.7:0/3396320344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec10f3a0 con 0x7f71ec101c40 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec10f6a0 con 0x7f71ec0ff730 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.987+0000 7f71ebfff640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ec101c40 0x7f71ec1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71eb7fe640 1 -- 172.21.15.7:0/3396320344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ec101c40 msgr2=0x7f71ec1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ec101c40 0x7f71ec1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71eb7fe640 1 -- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ec1000f0 msgr2=0x7f71ec19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ec1000f0 0x7f71ec19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71eb7fe640 1 -- 172.21.15.7:0/3396320344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ec1a1810 con 0x7f71ec0ff730 2024-03-26T08:50:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71eaffd640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ec1000f0 0x7f71ec19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec0ff730 0x7f71ec199bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005940 tx=0x7f71dc032470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71e8ff9640 1 -- 172.21.15.7:0/3396320344 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc041020 con 0x7f71ec0ff730 2024-03-26T08:50:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71e8ff9640 1 -- 172.21.15.7:0/3396320344 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71dc02c9f0 con 0x7f71ec0ff730 2024-03-26T08:50:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.988+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec1a1aa0 con 0x7f71ec0ff730 2024-03-26T08:50:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.989+0000 7f71e8ff9640 1 -- 172.21.15.7:0/3396320344 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc039040 con 0x7f71ec0ff730 2024-03-26T08:50:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.989+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec1a1fe0 con 0x7f71ec0ff730 2024-03-26T08:50:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.991+0000 7f71e8ff9640 1 -- 172.21.15.7:0/3396320344 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f71dc044020 con 0x7f71ec0ff730 2024-03-26T08:50:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.992+0000 7f71e8ff9640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71d40a2880 0x7f71d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.992+0000 7f71eaffd640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71d40a2880 0x7f71d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.992+0000 7f71e8ff9640 1 -- 172.21.15.7:0/3396320344 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f71dc0f0330 con 0x7f71ec0ff730 2024-03-26T08:50:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.993+0000 7f71eaffd640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71d40a2880 0x7f71d40a4d40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f71e00096c0 tx=0x7f71e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.993+0000 7f71f481a740 1 Processor -- start 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.993+0000 7f71f481a740 1 -- start start 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71f481a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71f481a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71f481a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71f481a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b05233d0 con 0x55c4b05be790 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71f481a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b04cbad0 con 0x55c4b05bdda0 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71f481a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b051f470 con 0x55c4b05c3900 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71eaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71ebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05c5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43672/0 (socket says 172.21.15.7:43672) 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71ebfff640 1 -- 172.21.15.7:0/3520477944 learned_addr learned my addr 172.21.15.7:0/3520477944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.994+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/3520477944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.995+0000 7f71ebfff640 1 -- 172.21.15.7:0/3520477944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 msgr2=0x55c4b05be1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.995+0000 7f71ebfff640 1 --2- 172.21.15.7:0/3520477944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05be1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.995+0000 7f71ebfff640 1 -- 172.21.15.7:0/3520477944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 msgr2=0x55c4b05c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.995+0000 7f71ebfff640 1 --2- 172.21.15.7:0/3520477944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.995+0000 7f71ebfff640 1 -- 172.21.15.7:0/3520477944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4b03017d0 con 0x55c4b05c3900 2024-03-26T08:50:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.995+0000 7f71ebfff640 1 --2- 172.21.15.7:0/3520477944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05c5d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009fd0 tx=0x7f71e40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d4e7b3781bf6108 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.996+0000 7f71d1ffb640 1 -- 172.21.15.7:0/3520477944 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4004d20 con 0x55c4b05c3900 2024-03-26T08:50:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.996+0000 7f71d1ffb640 1 -- 172.21.15.7:0/3520477944 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e4004050 con 0x55c4b05c3900 2024-03-26T08:50:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.996+0000 7f71d1ffb640 1 -- 172.21.15.7:0/3520477944 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4005280 con 0x55c4b05c3900 2024-03-26T08:50:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.997+0000 7f71f481a740 1 -- 172.21.15.7:0/3520477944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 msgr2=0x55c4b05c5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.997+0000 7f71f481a740 1 --2- 172.21.15.7:0/3520477944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05c5d90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009fd0 tx=0x7f71e40118d0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.997+0000 7f71f481a740 1 -- 172.21.15.7:0/3520477944 shutdown_connections 2024-03-26T08:50:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.997+0000 7f71f481a740 1 --2- 172.21.15.7:0/3520477944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05c5d90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.997+0000 7f71f481a740 1 --2- 172.21.15.7:0/3520477944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.997+0000 7f71f481a740 1 --2- 172.21.15.7:0/3520477944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05be1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.997+0000 7f71f481a740 1 -- 172.21.15.7:0/3520477944 >> 172.21.15.7:0/3520477944 conn(0x55c4b05b9310 msgr2=0x55c4b05b9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.997+0000 7f71f481a740 1 -- 172.21.15.7:0/3520477944 shutdown_connections 2024-03-26T08:50:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.997+0000 7f71f481a740 1 -- 172.21.15.7:0/3520477944 wait complete. 2024-03-26T08:50:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.998+0000 7f71f481a740 1 -- start start 2024-03-26T08:50:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.998+0000 7f71f481a740 1 Processor -- start 2024-03-26T08:50:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.998+0000 7f71f481a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.998+0000 7f71f481a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.998+0000 7f71f481a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05c3900 0x55c4b05d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.998+0000 7f71eaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.998+0000 7f71eaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59974/0 (socket says 172.21.15.7:59974) 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.998+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05d09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43678/0 (socket says 172.21.15.7:43678) 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71eaffd640 1 -- 172.21.15.7:0/4007167380 learned_addr learned my addr 172.21.15.7:0/4007167380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b05c68b0 con 0x55c4b05be790 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b04df060 con 0x55c4b05c3900 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b05c6a30 con 0x55c4b05bdda0 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71f481a740 10 client.0 _ll_register_callbacks cb 0x55c4b04c16a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71ebfff640 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05c3900 0x55c4b05d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71eaffd640 1 -- 172.21.15.7:0/4007167380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 msgr2=0x55c4b05d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71eaffd640 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71eaffd640 1 -- 172.21.15.7:0/4007167380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05c3900 msgr2=0x55c4b05d7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71eaffd640 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05c3900 0x55c4b05d7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71eaffd640 1 -- 172.21.15.7:0/4007167380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4b02373f0 con 0x55c4b05be790 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71eaffd640 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d0f10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009530 tx=0x7f71e0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53b244026a9082e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71caffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:50:30.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:30.999+0000 7f71caffd640 1 -- 172.21.15.7:0/4007167380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e002c060 con 0x55c4b05be790 2024-03-26T08:50:30.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.000+0000 7f71caffd640 1 -- 172.21.15.7:0/4007167380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f71e001cb70 con 0x55c4b05be790 2024-03-26T08:50:30.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.000+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71e4004050 con 0x55c4b05be790 2024-03-26T08:50:30.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.000+0000 7f71caffd640 1 -- 172.21.15.7:0/4007167380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e001cd70 con 0x55c4b05be790 2024-03-26T08:50:30.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.000+0000 7f71f481a740 10 client.17574 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:30.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.000+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c4b01f70f0 con 0x55c4b05be790 2024-03-26T08:50:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.000+0000 7f71caffd640 1 -- 172.21.15.7:0/4007167380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71e4004050 con 0x55c4b05be790 2024-03-26T08:50:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.001+0000 7f71caffd640 1 -- 172.21.15.7:0/4007167380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f71e0031070 con 0x55c4b05be790 2024-03-26T08:50:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.001+0000 7f71f481a740 10 client.17574 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.001+0000 7f71f481a740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.001+0000 7f71f481a740 20 client.17574 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.001+0000 7f71f481a740 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c4b05dc580 0x55c4b05de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.001+0000 7f71f481a740 4 client.17574 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.001+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4b0530910 con 0x55c4b05dc580 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.002+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c4b05dc580 0x55c4b05de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.002+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c4b05dc580 0x55c4b05de9e0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2392ea2e764e667 server_cookie=d3b69543212bd805 in_seq=0 out_seq=0 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.002+0000 7f71caffd640 10 client.17574 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.003+0000 7f71caffd640 1 -- 172.21.15.7:0/4007167380 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c4b0530910 con 0x55c4b05dc580 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.003+0000 7f71caffd640 10 client.17574 handle_command_reply: tid=0 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.003+0000 7f71f481a740 1 client.17574 shutdown 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 msgr2=0x55c4b05d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d0f10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009530 tx=0x7f71e0013040 comp rx=0 tx=0).stop 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 shutdown_connections 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c4b05dc580 0x55c4b05de9e0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05c3900 0x55c4b05d7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d0f10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 --2- 172.21.15.7:0/4007167380 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71caffd640 0 client.17574 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:31.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 >> 172.21.15.7:0/4007167380 conn(0x55c4b05b9310 msgr2=0x55c4b05b8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:31.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 shutdown_connections 2024-03-26T08:50:31.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 1 -- 172.21.15.7:0/4007167380 wait complete. 2024-03-26T08:50:31.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.004+0000 7f71f481a740 20 client.17574 trim_cache size 0 max 16384 2024-03-26T08:50:31.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71f481a740 1 Processor -- start 2024-03-26T08:50:31.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71f481a740 1 -- start start 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71f481a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71f481a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05be790 0x55c4b05d7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71f481a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05c3900 0x55c4b05b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71f481a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec1a8d90 con 0x55c4b05c3900 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71f481a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec1a6890 con 0x55c4b05be790 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71f481a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec1a4910 con 0x55c4b05bdda0 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05c3900 0x55c4b05b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05b7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43694/0 (socket says 172.21.15.7:43694) 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71eb7fe640 1 -- 172.21.15.7:0/2024365410 learned_addr learned my addr 172.21.15.7:0/2024365410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.031+0000 7f71eaffd640 1 --2- 172.21.15.7:0/2024365410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05be790 0x55c4b05d7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.032+0000 7f71eb7fe640 1 -- 172.21.15.7:0/2024365410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05be790 msgr2=0x55c4b05d7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:31.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.032+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/2024365410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05be790 0x55c4b05d7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.032+0000 7f71eb7fe640 1 -- 172.21.15.7:0/2024365410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05c3900 msgr2=0x55c4b05b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:31.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.032+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/2024365410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05c3900 0x55c4b05b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.032+0000 7f71eb7fe640 1 -- 172.21.15.7:0/2024365410 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4b05b4b60 con 0x55c4b05bdda0 2024-03-26T08:50:31.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.032+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/2024365410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05b7ca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b9130 tx=0x7f71dc0329e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b76d95b4194b2b5f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.032+0000 7f71d17fa640 1 -- 172.21.15.7:0/2024365410 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0c5470 con 0x55c4b05bdda0 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.032+0000 7f71d17fa640 1 -- 172.21.15.7:0/2024365410 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71dc0edf40 con 0x55c4b05bdda0 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.032+0000 7f71d17fa640 1 -- 172.21.15.7:0/2024365410 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0c4fe0 con 0x55c4b05bdda0 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 -- 172.21.15.7:0/2024365410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 msgr2=0x55c4b05b7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 --2- 172.21.15.7:0/2024365410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05b7ca0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b9130 tx=0x7f71dc0329e0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 -- 172.21.15.7:0/2024365410 shutdown_connections 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 --2- 172.21.15.7:0/2024365410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05c3900 0x55c4b05b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 --2- 172.21.15.7:0/2024365410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05be790 0x55c4b05d7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 --2- 172.21.15.7:0/2024365410 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05bdda0 0x55c4b05b7ca0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 -- 172.21.15.7:0/2024365410 >> 172.21.15.7:0/2024365410 conn(0x55c4b05b9310 msgr2=0x55c4b05bfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 -- 172.21.15.7:0/2024365410 shutdown_connections 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 -- 172.21.15.7:0/2024365410 wait complete. 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.033+0000 7f71f481a740 1 -- start start 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71f481a740 1 Processor -- start 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71f481a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71f481a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71f481a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71f481a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec1a4f10 con 0x55c4b05be790 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71f481a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec1a6230 con 0x55c4b05bdda0 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71f481a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec1a5570 con 0x55c4b05c3900 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71f481a740 10 client.0 _ll_register_callbacks cb 0x7f71e0005a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eaffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:31.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05d5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34380/0 (socket says 172.21.15.7:34380) 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eb7fe640 1 -- 172.21.15.7:0/1281434793 learned_addr learned my addr 172.21.15.7:0/1281434793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eaffd640 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59984/0 (socket says 172.21.15.7:59984) 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71ebfff640 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eb7fe640 1 -- 172.21.15.7:0/1281434793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 msgr2=0x55c4b05d6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05d6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eb7fe640 1 -- 172.21.15.7:0/1281434793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 msgr2=0x55c4b05d5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eb7fe640 1 -- 172.21.15.7:0/1281434793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4b05d7260 con 0x55c4b05bdda0 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.034+0000 7f71eaffd640 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71eb7fe640 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05d5720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f71dc02cbc0 tx=0x7f71dc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a99573237f212de7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71caffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71caffd640 1 -- 172.21.15.7:0/1281434793 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0a0060 con 0x55c4b05bdda0 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71f481a740 1 -- 172.21.15.7:0/1281434793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71dc0edf40 con 0x55c4b05bdda0 2024-03-26T08:50:31.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71caffd640 1 -- 172.21.15.7:0/1281434793 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f71dc0bafd0 con 0x55c4b05bdda0 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71caffd640 1 -- 172.21.15.7:0/1281434793 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71dc0c5470 con 0x55c4b05bdda0 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71f481a740 10 client.27410 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71f481a740 1 -- 172.21.15.7:0/1281434793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c4b0648fb0 con 0x55c4b05bdda0 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71caffd640 1 -- 172.21.15.7:0/1281434793 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71dc0edf40 con 0x55c4b05bdda0 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71caffd640 1 -- 172.21.15.7:0/1281434793 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f71dc0ef040 con 0x55c4b05bdda0 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71f481a740 10 client.27410 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71f481a740 10 client.27410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71f481a740 20 client.27410 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71f481a740 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c4b06497f0 0x55c4b064bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71f481a740 4 client.27410 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.035+0000 7f71f481a740 1 -- 172.21.15.7:0/1281434793 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c4b052ac60 con 0x55c4b06497f0 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71eaffd640 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c4b06497f0 0x55c4b064bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71eaffd640 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c4b06497f0 0x55c4b064bbe0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=822bbba681acc2b2 server_cookie=7881a6e92de57fda in_seq=0 out_seq=0 2024-03-26T08:50:31.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71caffd640 10 client.27410 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71caffd640 1 -- 172.21.15.7:0/1281434793 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c4b052ac60 con 0x55c4b06497f0 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71caffd640 10 client.27410 handle_command_reply: tid=0 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 client.27410 shutdown 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 -- 172.21.15.7:0/1281434793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 msgr2=0x55c4b05d5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05d5720 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f71dc02cbc0 tx=0x7f71dc005f90 comp rx=0 tx=0).stop 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 -- 172.21.15.7:0/1281434793 shutdown_connections 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c4b06497f0 0x55c4b064bbe0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c4b05c3900 0x55c4b05d6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71caffd640 0 client.27410 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c4b05be790 0x55c4b05d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 --2- 172.21.15.7:0/1281434793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c4b05bdda0 0x55c4b05d5720 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 -- 172.21.15.7:0/1281434793 >> 172.21.15.7:0/1281434793 conn(0x55c4b05b9310 msgr2=0x55c4b05bc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 -- 172.21.15.7:0/1281434793 shutdown_connections 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.036+0000 7f71f481a740 1 -- 172.21.15.7:0/1281434793 wait complete. 2024-03-26T08:50:31.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f481a740 20 client.27410 trim_cache size 0 max 16384 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "7ba6260a-1cda-4963-9a80-6127c4fcb2f6": { 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "7ba6260a-1cda-4963-9a80-6127c4fcb2f6", 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:50:31.037 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71d40a2880 msgr2=0x7f71d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71d40a2880 0x7f71d40a4d40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f71e00096c0 tx=0x7f71e0009260 comp rx=0 tx=0).stop 2024-03-26T08:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec0ff730 msgr2=0x7f71ec199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:31.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec0ff730 0x7f71ec199bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f71dc005940 tx=0x7f71dc032470 comp rx=0 tx=0).stop 2024-03-26T08:50:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 shutdown_connections 2024-03-26T08:50:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71d40a2880 0x7f71d40a4d40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ec101c40 0x7f71ec1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ec1000f0 0x7f71ec19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 --2- 172.21.15.7:0/3396320344 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71ec0ff730 0x7f71ec199bf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 >> 172.21.15.7:0/3396320344 conn(0x7f71ec0fb2b0 msgr2=0x7f71ec0fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 shutdown_connections 2024-03-26T08:50:31.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:50:31.037+0000 7f71f1c34640 1 -- 172.21.15.7:0/3396320344 wait complete. 2024-03-26T08:50:31.047 INFO:tasks.cephfs.filesystem:scrub status for tag:7ba6260a-1cda-4963-9a80-6127c4fcb2f6 - {'path': '/', 'tag': '7ba6260a-1cda-4963-9a80-6127c4fcb2f6', 'options': 'recursive,force'} 2024-03-26T08:50:31.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 480.0 s, 2045.2 tps, lat 49.465 ms stddev 57.300 2024-03-26T08:50:32.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:32 smithi016 ceph-mon[35851]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 648 op/s 2024-03-26T08:50:32.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:32 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:50:32.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:32 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:32.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:32 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:50:32.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:32 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:50:32.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:32 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:32.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:32 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:32.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:32 smithi007 ceph-mon[32462]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 648 op/s 2024-03-26T08:50:32.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:32 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:50:32.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:32 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:32.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:32 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:50:32.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:32 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:50:32.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:32 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:32.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:32 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:32.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:32 smithi044 ceph-mon[36461]: pgmap v603: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 648 op/s 2024-03-26T08:50:32.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:32 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:50:32.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:32 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:32.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:32 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:50:32.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:32 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:50:32.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:32 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:32.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:32 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:50:34.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:34 smithi016 ceph-mon[35851]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 756 op/s 2024-03-26T08:50:34.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:34 smithi007 ceph-mon[32462]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 756 op/s 2024-03-26T08:50:34.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:34 smithi044 ceph-mon[36461]: pgmap v604: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 16 MiB/s wr, 756 op/s 2024-03-26T08:50:36.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 485.0 s, 2466.6 tps, lat 40.619 ms stddev 50.365 2024-03-26T08:50:36.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:36 smithi016 ceph-mon[35851]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 15 MiB/s wr, 753 op/s 2024-03-26T08:50:36.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:36 smithi007 ceph-mon[32462]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 15 MiB/s wr, 753 op/s 2024-03-26T08:50:36.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:36 smithi044 ceph-mon[36461]: pgmap v605: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 15 MiB/s wr, 753 op/s 2024-03-26T08:50:37.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:37 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:50:37.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:37 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:50:37.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:37 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:50:37.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:37 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:50:37.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:37 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:50:37.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:37 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:50:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.300+0000 7fea7147e640 1 -- 172.21.15.7:0/822789835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6c101910 msgr2=0x7fea6c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.300+0000 7fea7147e640 1 --2- 172.21.15.7:0/822789835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6c101910 0x7fea6c1081a0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fea6401acb0 tx=0x7fea6403d170 comp rx=0 tx=0).stop 2024-03-26T08:50:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.300+0000 7fea7147e640 1 -- 172.21.15.7:0/822789835 shutdown_connections 2024-03-26T08:50:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.300+0000 7fea7147e640 1 --2- 172.21.15.7:0/822789835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6c101910 0x7fea6c1081a0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.300+0000 7fea7147e640 1 --2- 172.21.15.7:0/822789835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c100f50 0x7fea6c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.300+0000 7fea7147e640 1 --2- 172.21.15.7:0/822789835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea6c0ffd60 0x7fea6c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.300+0000 7fea7147e640 1 -- 172.21.15.7:0/822789835 >> 172.21.15.7:0/822789835 conn(0x7fea6c0fb2f0 msgr2=0x7fea6c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.304+0000 7fea7147e640 1 -- 172.21.15.7:0/822789835 shutdown_connections 2024-03-26T08:50:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.307+0000 7fea7147e640 1 -- 172.21.15.7:0/822789835 wait complete. 2024-03-26T08:50:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.308+0000 7fea7147e640 1 Processor -- start 2024-03-26T08:50:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.308+0000 7fea7147e640 1 -- start start 2024-03-26T08:50:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.308+0000 7fea7147e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6c0ffd60 0x7fea6c1ba330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.308+0000 7fea7147e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c100f50 0x7fea6c1bc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.308+0000 7fea7147e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea6c101910 0x7fea6c1bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.308+0000 7fea7147e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c06d470 con 0x7fea6c0ffd60 2024-03-26T08:50:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.308+0000 7fea7147e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c06d2f0 con 0x7fea6c100f50 2024-03-26T08:50:38.305 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.308+0000 7fea7147e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c06d5f0 con 0x7fea6c101910 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.310+0000 7fea6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea6c101910 0x7fea6c1bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.310+0000 7fea6b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea6c101910 0x7fea6c1bedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:40420/0 (socket says 172.21.15.7:40420) 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.310+0000 7fea6b7fe640 1 -- 172.21.15.7:0/2506459425 learned_addr learned my addr 172.21.15.7:0/2506459425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.312+0000 7fea6b7fe640 1 -- 172.21.15.7:0/2506459425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c100f50 msgr2=0x7fea6c1bc880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.312+0000 7fea6b7fe640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c100f50 0x7fea6c1bc880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.312+0000 7fea6b7fe640 1 -- 172.21.15.7:0/2506459425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6c0ffd60 msgr2=0x7fea6c1ba330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.312+0000 7fea6b7fe640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6c0ffd60 0x7fea6c1ba330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.312+0000 7fea6b7fe640 1 -- 172.21.15.7:0/2506459425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c1bf4a0 con 0x7fea6c101910 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.312+0000 7fea6b7fe640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea6c101910 0x7fea6c1bedd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fea64005c40 tx=0x7fea64003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.313+0000 7fea53fff640 1 -- 172.21.15.7:0/2506459425 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64012020 con 0x7fea6c101910 2024-03-26T08:50:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.313+0000 7fea7147e640 1 -- 172.21.15.7:0/2506459425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c1bf730 con 0x7fea6c101910 2024-03-26T08:50:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.313+0000 7fea7147e640 1 -- 172.21.15.7:0/2506459425 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c1bfca0 con 0x7fea6c101910 2024-03-26T08:50:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.314+0000 7fea53fff640 1 -- 172.21.15.7:0/2506459425 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea64004520 con 0x7fea6c101910 2024-03-26T08:50:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.314+0000 7fea53fff640 1 -- 172.21.15.7:0/2506459425 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea64045bd0 con 0x7fea6c101910 2024-03-26T08:50:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.317+0000 7fea74065740 1 Processor -- start 2024-03-26T08:50:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.318+0000 7fea74065740 1 -- start start 2024-03-26T08:50:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.320+0000 7fea74065740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.320+0000 7fea74065740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6febe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.320+0000 7fea74065740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fec480 0x55a0b6fee910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.320+0000 7fea74065740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0b6f4bb10 con 0x55a0b6fec480 2024-03-26T08:50:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.320+0000 7fea74065740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0b6ef4900 con 0x55a0b6fe7310 2024-03-26T08:50:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.320+0000 7fea74065740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0b6f47bb0 con 0x55a0b6fe6920 2024-03-26T08:50:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.321+0000 7fea6a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.316 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.321+0000 7fea6a7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:40426/0 (socket says 172.21.15.7:40426) 2024-03-26T08:50:38.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.321+0000 7fea6a7fc640 1 -- 172.21.15.7:0/3082657029 learned_addr learned my addr 172.21.15.7:0/3082657029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.322+0000 7fea6affd640 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6febe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.323+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fec480 0x55a0b6fee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.323+0000 7fea53fff640 1 -- 172.21.15.7:0/2506459425 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fea6404d020 con 0x7fea6c101910 2024-03-26T08:50:38.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.324+0000 7fea53fff640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea540a2a00 0x7fea540a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.324+0000 7fea53fff640 1 -- 172.21.15.7:0/2506459425 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea640f0fc0 con 0x7fea6c101910 2024-03-26T08:50:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.326+0000 7fea6a7fc640 1 -- 172.21.15.7:0/3082657029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 msgr2=0x55a0b6febe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.326+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6febe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.326+0000 7fea6a7fc640 1 -- 172.21.15.7:0/3082657029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fec480 msgr2=0x55a0b6fee910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.326+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fec480 0x55a0b6fee910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.326+0000 7fea6a7fc640 1 -- 172.21.15.7:0/3082657029 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0b6c60210 con 0x55a0b6fe6920 2024-03-26T08:50:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.326+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe6d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fea60001e50 tx=0x7fea60009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef90542e1acfa72a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.327+0000 7fea517fa640 1 -- 172.21.15.7:0/3082657029 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6000ed00 con 0x55a0b6fe6920 2024-03-26T08:50:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.329+0000 7fea6b7fe640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea540a2a00 0x7fea540a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.324 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.329+0000 7fea517fa640 1 -- 172.21.15.7:0/3082657029 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea60007080 con 0x55a0b6fe6920 2024-03-26T08:50:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.330+0000 7fea74065740 1 -- 172.21.15.7:0/3082657029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe6920 msgr2=0x55a0b6fe6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:38.325 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.330+0000 7fea74065740 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe6d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fea60001e50 tx=0x7fea60009f90 comp rx=0 tx=0).stop 2024-03-26T08:50:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.331+0000 7fea6b7fe640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea540a2a00 0x7fea540a4ec0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fea640ba2f0 tx=0x7fea64043f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:38.326 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.331+0000 7fea6affd640 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6febe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:50:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.331+0000 7fea74065740 1 -- 172.21.15.7:0/3082657029 shutdown_connections 2024-03-26T08:50:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.331+0000 7fea74065740 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fec480 0x55a0b6fee910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.331+0000 7fea74065740 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6febe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.331+0000 7fea74065740 1 --2- 172.21.15.7:0/3082657029 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe6d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.327 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.331+0000 7fea74065740 1 -- 172.21.15.7:0/3082657029 >> 172.21.15.7:0/3082657029 conn(0x55a0b6fe1e90 msgr2=0x55a0b6fe2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.337+0000 7fea74065740 1 -- 172.21.15.7:0/3082657029 shutdown_connections 2024-03-26T08:50:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.337+0000 7fea74065740 1 -- 172.21.15.7:0/3082657029 wait complete. 2024-03-26T08:50:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.338+0000 7fea74065740 1 -- start start 2024-03-26T08:50:38.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.338+0000 7fea74065740 1 Processor -- start 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.341+0000 7fea74065740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6ff0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.341+0000 7fea74065740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.341+0000 7fea74065740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fec480 0x55a0b6ff8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.341+0000 7fea74065740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0b6fef4d0 con 0x55a0b6fe6920 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.341+0000 7fea74065740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0b6f07d50 con 0x55a0b6fec480 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.341+0000 7fea74065740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0b6fef650 con 0x55a0b6fe7310 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.341+0000 7fea74065740 10 client.0 _ll_register_callbacks cb 0x55a0b6f07720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.343+0000 7fea6a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6ff0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.343+0000 7fea6a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6ff0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41796/0 (socket says 172.21.15.7:41796) 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.343+0000 7fea6a7fc640 1 -- 172.21.15.7:0/76996692 learned_addr learned my addr 172.21.15.7:0/76996692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.343+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fec480 0x55a0b6ff8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.343+0000 7fea6a7fc640 1 -- 172.21.15.7:0/76996692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe7310 msgr2=0x55a0b6ff1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.343+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.343+0000 7fea6a7fc640 1 -- 172.21.15.7:0/76996692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fec480 msgr2=0x55a0b6ff8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.343+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fec480 0x55a0b6ff8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.343+0000 7fea6a7fc640 1 -- 172.21.15.7:0/76996692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0b6d2c620 con 0x55a0b6fe6920 2024-03-26T08:50:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.344+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6ff0d10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fea60009000 tx=0x7fea600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637d1d33c81d79ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.345+0000 7fea4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:50:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.347+0000 7fea4eff5640 1 -- 172.21.15.7:0/76996692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60007430 con 0x55a0b6fe6920 2024-03-26T08:50:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.347+0000 7fea4eff5640 1 -- 172.21.15.7:0/76996692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea60009470 con 0x55a0b6fe6920 2024-03-26T08:50:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.348+0000 7fea74065740 1 -- 172.21.15.7:0/76996692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea60007080 con 0x55a0b6fe6920 2024-03-26T08:50:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.349+0000 7fea4eff5640 1 -- 172.21.15.7:0/76996692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea600075d0 con 0x55a0b6fe6920 2024-03-26T08:50:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.351+0000 7fea74065740 10 client.17589 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.351+0000 7fea74065740 1 -- 172.21.15.7:0/76996692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a0b700c640 con 0x55a0b6fe6920 2024-03-26T08:50:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.353+0000 7fea4eff5640 1 -- 172.21.15.7:0/76996692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea60007080 con 0x55a0b6fe6920 2024-03-26T08:50:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.353+0000 7fea4eff5640 1 -- 172.21.15.7:0/76996692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fea600183d0 con 0x55a0b6fe6920 2024-03-26T08:50:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.354+0000 7fea74065740 10 client.17589 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.354+0000 7fea74065740 10 client.17589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.354+0000 7fea74065740 20 client.17589 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.354+0000 7fea74065740 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0b700daa0 0x55a0b700fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.354+0000 7fea74065740 4 client.17589 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.354+0000 7fea74065740 1 -- 172.21.15.7:0/76996692 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0b6f59050 con 0x55a0b700daa0 2024-03-26T08:50:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.356+0000 7fea6affd640 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0b700daa0 0x55a0b700fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.365+0000 7fea6affd640 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0b700daa0 0x55a0b700fee0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebea466b13cfe6f server_cookie=efcf374d5b366d06 in_seq=0 out_seq=0 2024-03-26T08:50:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.365+0000 7fea4eff5640 10 client.17589 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.367+0000 7fea4eff5640 1 -- 172.21.15.7:0/76996692 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a0b6f59050 con 0x55a0b700daa0 2024-03-26T08:50:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.367+0000 7fea4eff5640 10 client.17589 handle_command_reply: tid=0 2024-03-26T08:50:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.367+0000 7fea74065740 1 client.17589 shutdown 2024-03-26T08:50:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.368+0000 7fea74065740 1 -- 172.21.15.7:0/76996692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 msgr2=0x55a0b6ff0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.368+0000 7fea74065740 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6ff0d10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fea60009000 tx=0x7fea600046c0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.371+0000 7fea74065740 1 -- 172.21.15.7:0/76996692 shutdown_connections 2024-03-26T08:50:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.371+0000 7fea74065740 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0b700daa0 0x55a0b700fee0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.371+0000 7fea74065740 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fec480 0x55a0b6ff8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.371+0000 7fea74065740 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.371+0000 7fea74065740 1 --2- 172.21.15.7:0/76996692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6ff0d10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.371+0000 7fea74065740 1 -- 172.21.15.7:0/76996692 >> 172.21.15.7:0/76996692 conn(0x55a0b6fe1e90 msgr2=0x55a0b6fe1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.373+0000 7fea4eff5640 0 client.17589 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.374+0000 7fea74065740 1 -- 172.21.15.7:0/76996692 shutdown_connections 2024-03-26T08:50:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.375+0000 7fea74065740 1 -- 172.21.15.7:0/76996692 wait complete. 2024-03-26T08:50:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.375+0000 7fea74065740 20 client.17589 trim_cache size 0 max 16384 2024-03-26T08:50:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.409+0000 7fea74065740 1 Processor -- start 2024-03-26T08:50:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.409+0000 7fea74065740 1 -- start start 2024-03-26T08:50:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.409+0000 7fea74065740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.410+0000 7fea74065740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.410+0000 7fea74065740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 0x55a0b6ff0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.410+0000 7fea74065740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea58010a20 con 0x55a0b6fe6920 2024-03-26T08:50:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.410+0000 7fea74065740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5800e520 con 0x55a0b6fe7310 2024-03-26T08:50:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.410+0000 7fea6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.410+0000 7fea6affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58440/0 (socket says 172.21.15.7:58440) 2024-03-26T08:50:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.410+0000 7fea6affd640 1 -- 172.21.15.7:0/3721545792 learned_addr learned my addr 172.21.15.7:0/3721545792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.410+0000 7fea74065740 1 -- 172.21.15.7:0/3721545792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5800c5a0 con 0x55a0b6fec480 2024-03-26T08:50:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.411+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/3721545792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 0x55a0b6ff0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.411+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/3721545792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.412+0000 7fea6a7fc640 1 -- 172.21.15.7:0/3721545792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 msgr2=0x55a0b6ff8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.412+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/3721545792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.412+0000 7fea6a7fc640 1 -- 172.21.15.7:0/3721545792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 msgr2=0x55a0b6fe2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.412+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/3721545792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.412+0000 7fea6a7fc640 1 -- 172.21.15.7:0/3721545792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0b6feecd0 con 0x55a0b6fec480 2024-03-26T08:50:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.415+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/3721545792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 0x55a0b6ff0920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fea60001eb0 tx=0x7fea60000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3f2a2ff0d8e5012 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.416+0000 7fea50ff9640 1 -- 172.21.15.7:0/3721545792 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60001430 con 0x55a0b6fec480 2024-03-26T08:50:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.416+0000 7fea50ff9640 1 -- 172.21.15.7:0/3721545792 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea60023040 con 0x55a0b6fec480 2024-03-26T08:50:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.416+0000 7fea50ff9640 1 -- 172.21.15.7:0/3721545792 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea60009a00 con 0x55a0b6fec480 2024-03-26T08:50:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.419+0000 7fea74065740 1 -- 172.21.15.7:0/3721545792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 msgr2=0x55a0b6ff0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.419+0000 7fea74065740 1 --2- 172.21.15.7:0/3721545792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 0x55a0b6ff0920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fea60001eb0 tx=0x7fea60000bc0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.420+0000 7fea74065740 1 -- 172.21.15.7:0/3721545792 shutdown_connections 2024-03-26T08:50:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.420+0000 7fea74065740 1 --2- 172.21.15.7:0/3721545792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 0x55a0b6ff0920 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.420+0000 7fea74065740 1 --2- 172.21.15.7:0/3721545792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.420+0000 7fea74065740 1 --2- 172.21.15.7:0/3721545792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe6920 0x55a0b6fe2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.420+0000 7fea74065740 1 -- 172.21.15.7:0/3721545792 >> 172.21.15.7:0/3721545792 conn(0x55a0b6fe1e90 msgr2=0x55a0b6fefad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.424+0000 7fea74065740 1 -- 172.21.15.7:0/3721545792 shutdown_connections 2024-03-26T08:50:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.424+0000 7fea74065740 1 -- 172.21.15.7:0/3721545792 wait complete. 2024-03-26T08:50:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.424+0000 7fea74065740 1 -- start start 2024-03-26T08:50:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.425+0000 7fea74065740 1 Processor -- start 2024-03-26T08:50:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.425+0000 7fea74065740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe6920 0x55a0b6ff5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.425+0000 7fea74065740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.425+0000 7fea74065740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 0x55a0b7067360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.425+0000 7fea74065740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5800cba0 con 0x55a0b6fe7310 2024-03-26T08:50:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.425+0000 7fea74065740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5800dec0 con 0x55a0b6fe6920 2024-03-26T08:50:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea74065740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5800d200 con 0x55a0b6fec480 2024-03-26T08:50:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea6affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea74065740 10 client.0 _ll_register_callbacks cb 0x7fea6000ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea6affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41816/0 (socket says 172.21.15.7:41816) 2024-03-26T08:50:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea6affd640 1 -- 172.21.15.7:0/1538713838 learned_addr learned my addr 172.21.15.7:0/1538713838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea6affd640 1 -- 172.21.15.7:0/1538713838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 msgr2=0x55a0b7067360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:50:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea6affd640 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 0x55a0b7067360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea6affd640 1 -- 172.21.15.7:0/1538713838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe6920 msgr2=0x55a0b6ff5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea6affd640 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe6920 0x55a0b6ff5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.426+0000 7fea6affd640 1 -- 172.21.15.7:0/1538713838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0b7067920 con 0x55a0b6fe7310 2024-03-26T08:50:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.427+0000 7fea6affd640 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff57c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fea6c100db0 tx=0x7fea5c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bcb89e1b07f8af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.427+0000 7fea4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:50:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.428+0000 7fea4eff5640 1 -- 172.21.15.7:0/1538713838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c005220 con 0x55a0b6fe7310 2024-03-26T08:50:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.428+0000 7fea4eff5640 1 -- 172.21.15.7:0/1538713838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea5c0053c0 con 0x55a0b6fe7310 2024-03-26T08:50:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.429+0000 7fea74065740 1 -- 172.21.15.7:0/1538713838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea60023040 con 0x55a0b6fe7310 2024-03-26T08:50:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.432+0000 7fea4eff5640 1 -- 172.21.15.7:0/1538713838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea5c025050 con 0x55a0b6fe7310 2024-03-26T08:50:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.433+0000 7fea74065740 10 client.17595 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.433+0000 7fea74065740 1 -- 172.21.15.7:0/1538713838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a0b7068d30 con 0x55a0b6fe7310 2024-03-26T08:50:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.434+0000 7fea4eff5640 1 -- 172.21.15.7:0/1538713838 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea60023040 con 0x55a0b6fe7310 2024-03-26T08:50:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.434+0000 7fea4eff5640 1 -- 172.21.15.7:0/1538713838 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fea5c01e070 con 0x55a0b6fe7310 2024-03-26T08:50:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.435+0000 7fea74065740 10 client.17595 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.435+0000 7fea74065740 10 client.17595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.435+0000 7fea74065740 20 client.17595 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.435+0000 7fea74065740 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0b7069710 0x55a0b706bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.436+0000 7fea74065740 4 client.17595 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:50:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.436+0000 7fea74065740 1 -- 172.21.15.7:0/1538713838 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a0b6f533a0 con 0x55a0b7069710 2024-03-26T08:50:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.440+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0b7069710 0x55a0b706bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.450+0000 7fea6a7fc640 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0b7069710 0x55a0b706bb30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1386e127aa5b9c3 server_cookie=215a30efcb1d593d in_seq=0 out_seq=0 2024-03-26T08:50:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.451+0000 7fea4eff5640 10 client.17595 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:38.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:38 smithi016 ceph-mon[35851]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 13 MiB/s wr, 727 op/s 2024-03-26T08:50:38.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:38 smithi007 ceph-mon[32462]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 13 MiB/s wr, 727 op/s 2024-03-26T08:50:38.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:38 smithi044 ceph-mon[36461]: pgmap v606: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 13 MiB/s wr, 727 op/s 2024-03-26T08:50:38.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.836+0000 7fea4eff5640 1 -- 172.21.15.7:0/1538713838 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a0b6f533a0 con 0x55a0b7069710 2024-03-26T08:50:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.836+0000 7fea4eff5640 10 client.17595 handle_command_reply: tid=0 2024-03-26T08:50:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.836+0000 7fea74065740 1 client.17595 shutdown 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.840+0000 7fea74065740 1 -- 172.21.15.7:0/1538713838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe7310 msgr2=0x55a0b6ff57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.841+0000 7fea74065740 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff57c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fea6c100db0 tx=0x7fea5c00ced0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.841+0000 7fea74065740 1 -- 172.21.15.7:0/1538713838 shutdown_connections 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.841+0000 7fea74065740 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a0b7069710 0x55a0b706bb30 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.841+0000 7fea74065740 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a0b6fec480 0x55a0b7067360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.841+0000 7fea4eff5640 0 client.17595 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.841+0000 7fea74065740 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0b6fe7310 0x55a0b6ff57c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.841+0000 7fea74065740 1 --2- 172.21.15.7:0/1538713838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a0b6fe6920 0x55a0b6ff5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.841+0000 7fea74065740 1 -- 172.21.15.7:0/1538713838 >> 172.21.15.7:0/1538713838 conn(0x55a0b6fe1e90 msgr2=0x55a0b6fe2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.842+0000 7fea74065740 1 -- 172.21.15.7:0/1538713838 shutdown_connections 2024-03-26T08:50:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.842+0000 7fea74065740 1 -- 172.21.15.7:0/1538713838 wait complete. 2024-03-26T08:50:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.842+0000 7fea74065740 20 client.17595 trim_cache size 0 max 16384 2024-03-26T08:50:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:50:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:50:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:50:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:50:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.843+0000 7fea7147e640 1 -- 172.21.15.7:0/2506459425 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea540a2a00 msgr2=0x7fea540a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea540a2a00 0x7fea540a4ec0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fea640ba2f0 tx=0x7fea64043f90 comp rx=0 tx=0).stop 2024-03-26T08:50:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 -- 172.21.15.7:0/2506459425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea6c101910 msgr2=0x7fea6c1bedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea6c101910 0x7fea6c1bedd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fea64005c40 tx=0x7fea64003ba0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 -- 172.21.15.7:0/2506459425 shutdown_connections 2024-03-26T08:50:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fea540a2a00 0x7fea540a4ec0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea6c101910 0x7fea6c1bedd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea6c100f50 0x7fea6c1bc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 --2- 172.21.15.7:0/2506459425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea6c0ffd60 0x7fea6c1ba330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 -- 172.21.15.7:0/2506459425 >> 172.21.15.7:0/2506459425 conn(0x7fea6c0fb2f0 msgr2=0x7fea6c06ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 -- 172.21.15.7:0/2506459425 shutdown_connections 2024-03-26T08:50:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:38.844+0000 7fea7147e640 1 -- 172.21.15.7:0/2506459425 wait complete. 2024-03-26T08:50:40.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:40 smithi016 ceph-mon[35851]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 15 MiB/s wr, 865 op/s 2024-03-26T08:50:40.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:40 smithi007 ceph-mon[32462]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 15 MiB/s wr, 865 op/s 2024-03-26T08:50:40.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:40 smithi044 ceph-mon[36461]: pgmap v607: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 15 MiB/s wr, 865 op/s 2024-03-26T08:50:41.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 490.0 s, 1592.0 tps, lat 62.415 ms stddev 118.522 2024-03-26T08:50:42.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:42 smithi016 ceph-mon[35851]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 23 MiB/s wr, 1.14k op/s 2024-03-26T08:50:42.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:42 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:50:42.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:42 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:50:42.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:42 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:50:42.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:42 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:50:42.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:42 smithi007 ceph-mon[32462]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 23 MiB/s wr, 1.14k op/s 2024-03-26T08:50:42.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:42 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:50:42.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:42 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:50:42.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:42 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:50:42.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:42 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:50:42.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:42 smithi044 ceph-mon[36461]: pgmap v608: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 23 MiB/s wr, 1.14k op/s 2024-03-26T08:50:42.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:42 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:50:42.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:42 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:50:42.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:42 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:50:42.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:42 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:50:44.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:44 smithi016 ceph-mon[35851]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 40 MiB/s wr, 1.74k op/s 2024-03-26T08:50:44.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:44 smithi007 ceph-mon[32462]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 40 MiB/s wr, 1.74k op/s 2024-03-26T08:50:44.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:44 smithi044 ceph-mon[36461]: pgmap v609: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 40 MiB/s wr, 1.74k op/s 2024-03-26T08:50:46.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 495.0 s, 2056.6 tps, lat 48.727 ms stddev 58.255 2024-03-26T08:50:46.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:46 smithi016 ceph-mon[35851]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 38 MiB/s wr, 1.62k op/s 2024-03-26T08:50:46.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:46 smithi007 ceph-mon[32462]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 38 MiB/s wr, 1.62k op/s 2024-03-26T08:50:46.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:46 smithi044 ceph-mon[36461]: pgmap v610: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 38 MiB/s wr, 1.62k op/s 2024-03-26T08:50:48.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:48 smithi016 ceph-mon[35851]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 37 MiB/s wr, 1.55k op/s 2024-03-26T08:50:48.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:48 smithi007 ceph-mon[32462]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 37 MiB/s wr, 1.55k op/s 2024-03-26T08:50:48.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:48 smithi044 ceph-mon[36461]: pgmap v611: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 37 MiB/s wr, 1.55k op/s 2024-03-26T08:50:50.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:50 smithi016 ceph-mon[35851]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 39 MiB/s wr, 1.64k op/s 2024-03-26T08:50:50.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:50 smithi007 ceph-mon[32462]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 39 MiB/s wr, 1.64k op/s 2024-03-26T08:50:50.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:50 smithi044 ceph-mon[36461]: pgmap v612: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 39 MiB/s wr, 1.64k op/s 2024-03-26T08:50:51.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 500.0 s, 2157.4 tps, lat 46.412 ms stddev 52.327 2024-03-26T08:50:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:51.991+0000 7f2d46797640 1 -- 172.21.15.7:0/2303863743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 msgr2=0x7f2d40102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:51.991+0000 7f2d46797640 1 --2- 172.21.15.7:0/2303863743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 0x7f2d40102380 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009900 tx=0x7f2d3002bc60 comp rx=0 tx=0).stop 2024-03-26T08:50:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:51.997+0000 7f2d46797640 1 -- 172.21.15.7:0/2303863743 shutdown_connections 2024-03-26T08:50:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:51.997+0000 7f2d46797640 1 --2- 172.21.15.7:0/2303863743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d40105270 0x7f2d401056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:51.997+0000 7f2d46797640 1 --2- 172.21.15.7:0/2303863743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d401028c0 0x7f2d40104d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:51.997+0000 7f2d46797640 1 --2- 172.21.15.7:0/2303863743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 0x7f2d40102380 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:51.997+0000 7f2d46797640 1 -- 172.21.15.7:0/2303863743 >> 172.21.15.7:0/2303863743 conn(0x7f2d400fbb70 msgr2=0x7f2d400fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.005+0000 7f2d46797640 1 -- 172.21.15.7:0/2303863743 shutdown_connections 2024-03-26T08:50:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.006+0000 7f2d46797640 1 -- 172.21.15.7:0/2303863743 wait complete. 2024-03-26T08:50:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.007+0000 7f2d46797640 1 Processor -- start 2024-03-26T08:50:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.007+0000 7f2d46797640 1 -- start start 2024-03-26T08:50:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.007+0000 7f2d46797640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 0x7f2d40070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.007+0000 7f2d46797640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d401028c0 0x7f2d40070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.007+0000 7f2d46797640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d40105270 0x7f2d401ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.007+0000 7f2d46797640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401139f0 con 0x7f2d401028c0 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.007+0000 7f2d46797640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d40113870 con 0x7f2d400fff50 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.007+0000 7f2d46797640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d40113b70 con 0x7f2d40105270 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.008+0000 7f2d3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 0x7f2d40070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.008+0000 7f2d3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 0x7f2d40070040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57392/0 (socket says 172.21.15.7:57392) 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.008+0000 7f2d3ffff640 1 -- 172.21.15.7:0/525050561 learned_addr learned my addr 172.21.15.7:0/525050561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.008+0000 7f2d44d34640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d40105270 0x7f2d401ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.008+0000 7f2d3ffff640 1 -- 172.21.15.7:0/525050561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d40105270 msgr2=0x7f2d401ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.008+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d40105270 0x7f2d401ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.008+0000 7f2d3ffff640 1 -- 172.21.15.7:0/525050561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d401028c0 msgr2=0x7f2d40070580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.008+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d401028c0 0x7f2d40070580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.008+0000 7f2d3ffff640 1 -- 172.21.15.7:0/525050561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d40070c60 con 0x7f2d400fff50 2024-03-26T08:50:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.009+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 0x7f2d40070040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d30036040 tx=0x7f2d30004f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.009+0000 7f2d3dffb640 1 -- 172.21.15.7:0/525050561 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30063070 con 0x7f2d400fff50 2024-03-26T08:50:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.009+0000 7f2d46797640 1 -- 172.21.15.7:0/525050561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d401143c0 con 0x7f2d400fff50 2024-03-26T08:50:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.009+0000 7f2d46797640 1 -- 172.21.15.7:0/525050561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d40114900 con 0x7f2d400fff50 2024-03-26T08:50:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.009+0000 7f2d3dffb640 1 -- 172.21.15.7:0/525050561 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d300044d0 con 0x7f2d400fff50 2024-03-26T08:50:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.009+0000 7f2d3dffb640 1 -- 172.21.15.7:0/525050561 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3005f460 con 0x7f2d400fff50 2024-03-26T08:50:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.015+0000 7f2d3dffb640 1 -- 172.21.15.7:0/525050561 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2d30066070 con 0x7f2d400fff50 2024-03-26T08:50:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.015+0000 7f2d3dffb640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2d2c0a28b0 0x7f2d2c0a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.016+0000 7f2d3dffb640 1 -- 172.21.15.7:0/525050561 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2d30112800 con 0x7f2d400fff50 2024-03-26T08:50:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.016+0000 7f2d37fff640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2d2c0a28b0 0x7f2d2c0a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.018+0000 7f2d37fff640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2d2c0a28b0 0x7f2d2c0a4d70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2d40071550 tx=0x7f2d28006c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.019+0000 7f2d49385740 1 Processor -- start 2024-03-26T08:50:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d49385740 1 -- start start 2024-03-26T08:50:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d49385740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d410920 0x55b82d410d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d49385740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 0x55b82d415e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d49385740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d418910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d410920 0x55b82d410d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d410920 0x55b82d410d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57394/0 (socket says 172.21.15.7:57394) 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d3ffff640 1 -- 172.21.15.7:0/2392899552 learned_addr learned my addr 172.21.15.7:0/2392899552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d49385740 1 -- 172.21.15.7:0/2392899552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82d375b10 con 0x55b82d416480 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d49385740 1 -- 172.21.15.7:0/2392899552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82d31e900 con 0x55b82d410920 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d49385740 1 -- 172.21.15.7:0/2392899552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82d371bb0 con 0x55b82d411310 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d44d34640 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d418910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.020+0000 7f2d37fff640 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 0x55b82d415e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d44d34640 1 -- 172.21.15.7:0/2392899552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 msgr2=0x55b82d415e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d44d34640 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 0x55b82d415e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d44d34640 1 -- 172.21.15.7:0/2392899552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d410920 msgr2=0x55b82d410d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d44d34640 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d410920 0x55b82d410d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d44d34640 1 -- 172.21.15.7:0/2392899552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b82d08a210 con 0x55b82d416480 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d410920 0x55b82d410d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d44d34640 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d418910 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009f30 tx=0x7f2d38010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db58adf5c8bd96a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d367fc640 1 -- 172.21.15.7:0/2392899552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38016070 con 0x55b82d416480 2024-03-26T08:50:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d367fc640 1 -- 172.21.15.7:0/2392899552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d38004d20 con 0x55b82d416480 2024-03-26T08:50:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.021+0000 7f2d367fc640 1 -- 172.21.15.7:0/2392899552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d380127d0 con 0x55b82d416480 2024-03-26T08:50:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 -- 172.21.15.7:0/2392899552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 msgr2=0x55b82d418910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d418910 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009f30 tx=0x7f2d38010740 comp rx=0 tx=0).stop 2024-03-26T08:50:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 -- 172.21.15.7:0/2392899552 shutdown_connections 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d418910 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 0x55b82d415e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 --2- 172.21.15.7:0/2392899552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d410920 0x55b82d410d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 -- 172.21.15.7:0/2392899552 >> 172.21.15.7:0/2392899552 conn(0x55b82d40be90 msgr2=0x55b82d40c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 -- 172.21.15.7:0/2392899552 shutdown_connections 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 -- 172.21.15.7:0/2392899552 wait complete. 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.022+0000 7f2d49385740 1 -- start start 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d49385740 1 Processor -- start 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d49385740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d41abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d49385740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d411310 0x55b82d41b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d49385740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d422170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d49385740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82d419430 con 0x55b82d416480 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d49385740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82d331d50 con 0x55b82d411310 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d49385740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b82d4195b0 con 0x55b82d410920 2024-03-26T08:50:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d44d34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d422170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d411310 0x55b82d41b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d44d34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d422170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58638/0 (socket says 172.21.15.7:58638) 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d44d34640 1 -- 172.21.15.7:0/4283127596 learned_addr learned my addr 172.21.15.7:0/4283127596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d41abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.023+0000 7f2d49385740 10 client.0 _ll_register_callbacks cb 0x55b82d331720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d44d34640 1 -- 172.21.15.7:0/4283127596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 msgr2=0x55b82d41abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d44d34640 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d41abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d44d34640 1 -- 172.21.15.7:0/4283127596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d411310 msgr2=0x55b82d41b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d44d34640 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d411310 0x55b82d41b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d44d34640 1 -- 172.21.15.7:0/4283127596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b82d156620 con 0x55b82d416480 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d37fff640 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d411310 0x55b82d41b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d41abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d44d34640 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d422170 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2d38010fe0 tx=0x7f2d38012550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6566dde4f5afad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/4283127596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38016070 con 0x55b82d416480 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/4283127596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2d38012b90 con 0x55b82d416480 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d49385740 1 -- 172.21.15.7:0/4283127596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d38004d20 con 0x55b82d416480 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.024+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/4283127596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d38012d90 con 0x55b82d416480 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.025+0000 7f2d49385740 10 client.17601 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.025+0000 7f2d49385740 1 -- 172.21.15.7:0/4283127596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b82d4254f0 con 0x55b82d416480 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.025+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/4283127596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d38028070 con 0x55b82d416480 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.025+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/4283127596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2d3800f040 con 0x55b82d416480 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.025+0000 7f2d49385740 10 client.17601 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.025+0000 7f2d49385740 10 client.17601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.025+0000 7f2d49385740 20 client.17601 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.025+0000 7f2d49385740 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b82d4268f0 0x55b82d428ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.026+0000 7f2d49385740 4 client.17601 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.026+0000 7f2d49385740 1 -- 172.21.15.7:0/4283127596 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d38004d20 con 0x55b82d4268f0 2024-03-26T08:50:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.026+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b82d4268f0 0x55b82d428ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.027+0000 7f2d1f7fe640 10 client.17601 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.027+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b82d4268f0 0x55b82d428ce0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763f91e6360ae1ab server_cookie=2374128ba50b667d in_seq=0 out_seq=0 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/4283127596 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f2d38004d20 con 0x55b82d4268f0 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d1f7fe640 10 client.17601 handle_command_reply: tid=0 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 client.17601 shutdown 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 -- 172.21.15.7:0/4283127596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 msgr2=0x55b82d422170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d422170 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2d38010fe0 tx=0x7f2d38012550 comp rx=0 tx=0).stop 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 -- 172.21.15.7:0/4283127596 shutdown_connections 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b82d4268f0 0x55b82d428ce0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d416480 0x55b82d422170 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d411310 0x55b82d41b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 --2- 172.21.15.7:0/4283127596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d41abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 -- 172.21.15.7:0/4283127596 >> 172.21.15.7:0/4283127596 conn(0x55b82d40be90 msgr2=0x55b82d40b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d1f7fe640 0 client.17601 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 -- 172.21.15.7:0/4283127596 shutdown_connections 2024-03-26T08:50:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 1 -- 172.21.15.7:0/4283127596 wait complete. 2024-03-26T08:50:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.028+0000 7f2d49385740 20 client.17601 trim_cache size 0 max 16384 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.045+0000 7f2d49385740 1 Processor -- start 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.045+0000 7f2d49385740 1 -- start start 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d49385740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d418870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d49385740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d411310 0x55b82d422130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d49385740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 0x55b82d40c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d49385740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200108c0 con 0x55b82d411310 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d49385740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2000e3c0 con 0x55b82d416480 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d49385740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2000c440 con 0x55b82d410920 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d411310 0x55b82d422130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d3ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d418870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d411310 0x55b82d422130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58646/0 (socket says 172.21.15.7:58646) 2024-03-26T08:50:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d37fff640 1 -- 172.21.15.7:0/237842928 learned_addr learned my addr 172.21.15.7:0/237842928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d44d34640 1 --2- 172.21.15.7:0/237842928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 0x55b82d40c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d3ffff640 1 -- 172.21.15.7:0/237842928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 msgr2=0x55b82d40c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/237842928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 0x55b82d40c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d3ffff640 1 -- 172.21.15.7:0/237842928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d411310 msgr2=0x55b82d422130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/237842928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d411310 0x55b82d422130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d3ffff640 1 -- 172.21.15.7:0/237842928 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b82d40cc00 con 0x55b82d410920 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.046+0000 7f2d37fff640 1 --2- 172.21.15.7:0/237842928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d411310 0x55b82d422130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.047+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/237842928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d418870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2d300db570 tx=0x7f2d300ff040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4544a508a64c5b9b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.047+0000 7f2d35ffb640 1 -- 172.21.15.7:0/237842928 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300f7070 con 0x55b82d410920 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.047+0000 7f2d35ffb640 1 -- 172.21.15.7:0/237842928 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d300df1a0 con 0x55b82d410920 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.047+0000 7f2d35ffb640 1 -- 172.21.15.7:0/237842928 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3006d040 con 0x55b82d410920 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.047+0000 7f2d49385740 1 -- 172.21.15.7:0/237842928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 msgr2=0x55b82d418870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 --2- 172.21.15.7:0/237842928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d418870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2d300db570 tx=0x7f2d300ff040 comp rx=0 tx=0).stop 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 -- 172.21.15.7:0/237842928 shutdown_connections 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 --2- 172.21.15.7:0/237842928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 0x55b82d40c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 --2- 172.21.15.7:0/237842928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d411310 0x55b82d422130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 --2- 172.21.15.7:0/237842928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d410920 0x55b82d418870 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 -- 172.21.15.7:0/237842928 >> 172.21.15.7:0/237842928 conn(0x55b82d40be90 msgr2=0x55b82d406eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 -- 172.21.15.7:0/237842928 shutdown_connections 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 -- 172.21.15.7:0/237842928 wait complete. 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 -- start start 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 Processor -- start 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d410920 0x55b82d41f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 0x55b82d41f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 0x55b82d4801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2000ca40 con 0x55b82d410920 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2000dd60 con 0x55b82d416480 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2000d0a0 con 0x55b82d411310 2024-03-26T08:50:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.048+0000 7f2d49385740 10 client.0 _ll_register_callbacks cb 0x7f2d38009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d3ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d410920 0x55b82d41f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d3ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d410920 0x55b82d41f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58650/0 (socket says 172.21.15.7:58650) 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d3ffff640 1 -- 172.21.15.7:0/1572042740 learned_addr learned my addr 172.21.15.7:0/1572042740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d44d34640 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 0x55b82d4801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d37fff640 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 0x55b82d41f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d3ffff640 1 -- 172.21.15.7:0/1572042740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 msgr2=0x55b82d41f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 0x55b82d41f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d3ffff640 1 -- 172.21.15.7:0/1572042740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 msgr2=0x55b82d4801d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 0x55b82d4801d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d3ffff640 1 -- 172.21.15.7:0/1572042740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b82d422270 con 0x55b82d410920 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d37fff640 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 0x55b82d41f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d3ffff640 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d410920 0x55b82d41f310 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2d300df130 tx=0x7f2d3002bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25959ae9eb852f7e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/1572042740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d300c3060 con 0x55b82d410920 2024-03-26T08:50:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d49385740 1 -- 172.21.15.7:0/1572042740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d300df1a0 con 0x55b82d410920 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/1572042740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2d300e76b0 con 0x55b82d410920 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/1572042740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d300f7070 con 0x55b82d410920 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d49385740 10 client.17607 fetch_fsmap learned FSMap version 25 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.049+0000 7f2d49385740 1 -- 172.21.15.7:0/1572042740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b82d481600 con 0x55b82d410920 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.050+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/1572042740 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d300df1a0 con 0x55b82d410920 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.050+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/1572042740 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2d300de5f0 con 0x55b82d410920 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.050+0000 7f2d49385740 10 client.17607 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.050+0000 7f2d49385740 10 client.17607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.050+0000 7f2d49385740 20 client.17607 populate_metadata read hostname 'smithi007' 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.050+0000 7f2d49385740 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b82d481ed0 0x55b82d4842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.050+0000 7f2d49385740 4 client.17607 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.050+0000 7f2d49385740 1 -- 172.21.15.7:0/1572042740 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b82d383050 con 0x55b82d481ed0 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.050+0000 7f2d37fff640 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b82d481ed0 0x55b82d4842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:50:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.051+0000 7f2d1f7fe640 10 client.17607 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.051+0000 7f2d37fff640 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b82d481ed0 0x55b82d4842f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd15c2ef6b632d server_cookie=2bfbc9f58f21b1e1 in_seq=0 out_seq=0 2024-03-26T08:50:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.108+0000 7f2d1f7fe640 1 -- 172.21.15.7:0/1572042740 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b82d383050 con 0x55b82d481ed0 2024-03-26T08:50:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.108+0000 7f2d1f7fe640 10 client.17607 handle_command_reply: tid=0 2024-03-26T08:50:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.108+0000 7f2d49385740 1 client.17607 shutdown 2024-03-26T08:50:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 -- 172.21.15.7:0/1572042740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d410920 msgr2=0x55b82d41f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d410920 0x55b82d41f310 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2d300df130 tx=0x7f2d3002bc60 comp rx=0 tx=0).stop 2024-03-26T08:50:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 -- 172.21.15.7:0/1572042740 shutdown_connections 2024-03-26T08:50:52.104 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b82d481ed0 0x55b82d4842f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b82d416480 0x55b82d4801d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d1f7fe640 0 client.17607 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b82d411310 0x55b82d41f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 --2- 172.21.15.7:0/1572042740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b82d410920 0x55b82d41f310 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 -- 172.21.15.7:0/1572042740 >> 172.21.15.7:0/1572042740 conn(0x55b82d40be90 msgr2=0x55b82d40c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 -- 172.21.15.7:0/1572042740 shutdown_connections 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 1 -- 172.21.15.7:0/1572042740 wait complete. 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d49385740 20 client.17607 trim_cache size 0 max 16384 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:50:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:50:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 -- 172.21.15.7:0/525050561 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2d2c0a28b0 msgr2=0x7f2d2c0a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2d2c0a28b0 0x7f2d2c0a4d70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2d40071550 tx=0x7f2d28006c20 comp rx=0 tx=0).stop 2024-03-26T08:50:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 -- 172.21.15.7:0/525050561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 msgr2=0x7f2d40070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:50:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 0x7f2d40070040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2d30036040 tx=0x7f2d30004f00 comp rx=0 tx=0).stop 2024-03-26T08:50:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 -- 172.21.15.7:0/525050561 shutdown_connections 2024-03-26T08:50:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2d2c0a28b0 0x7f2d2c0a4d70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2d40105270 0x7f2d401ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d401028c0 0x7f2d40070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 --2- 172.21.15.7:0/525050561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d400fff50 0x7f2d40070040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:50:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.109+0000 7f2d46797640 1 -- 172.21.15.7:0/525050561 >> 172.21.15.7:0/525050561 conn(0x7f2d400fbb70 msgr2=0x7f2d40103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:50:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.110+0000 7f2d46797640 1 -- 172.21.15.7:0/525050561 shutdown_connections 2024-03-26T08:50:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:50:52.110+0000 7f2d46797640 1 -- 172.21.15.7:0/525050561 wait complete. 2024-03-26T08:50:52.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:52 smithi016 ceph-mon[35851]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 35 MiB/s wr, 1.50k op/s 2024-03-26T08:50:52.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:52 smithi007 ceph-mon[32462]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 35 MiB/s wr, 1.50k op/s 2024-03-26T08:50:52.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:52 smithi044 ceph-mon[36461]: pgmap v613: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 35 MiB/s wr, 1.50k op/s 2024-03-26T08:50:54.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:54 smithi016 ceph-mon[35851]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 1.24k op/s 2024-03-26T08:50:54.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:54 smithi007 ceph-mon[32462]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 1.24k op/s 2024-03-26T08:50:54.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:54 smithi044 ceph-mon[36461]: pgmap v614: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 27 MiB/s wr, 1.24k op/s 2024-03-26T08:50:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 505.0 s, 2366.6 tps, lat 42.080 ms stddev 49.426 2024-03-26T08:50:56.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:56 smithi016 ceph-mon[35851]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 653 op/s 2024-03-26T08:50:56.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:56 smithi007 ceph-mon[32462]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 653 op/s 2024-03-26T08:50:56.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:56 smithi044 ceph-mon[36461]: pgmap v615: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 10 MiB/s wr, 653 op/s 2024-03-26T08:50:58.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:50:58 smithi016 ceph-mon[35851]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.8 MiB/s wr, 667 op/s 2024-03-26T08:50:58.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:50:58 smithi007 ceph-mon[32462]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.8 MiB/s wr, 667 op/s 2024-03-26T08:50:58.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:50:58 smithi044 ceph-mon[36461]: pgmap v616: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.8 MiB/s wr, 667 op/s 2024-03-26T08:51:00.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:00 smithi016 ceph-mon[35851]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 697 op/s 2024-03-26T08:51:00.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:00 smithi007 ceph-mon[32462]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 697 op/s 2024-03-26T08:51:00.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:00 smithi044 ceph-mon[36461]: pgmap v617: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 10 MiB/s wr, 697 op/s 2024-03-26T08:51:01.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:51:01.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:51:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 510.0 s, 2184.6 tps, lat 46.054 ms stddev 49.662 2024-03-26T08:51:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 -- 172.21.15.7:0/1297451201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994100840 msgr2=0x7f1994100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 --2- 172.21.15.7:0/1297451201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994100840 0x7f1994100cc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1984009930 tx=0x7f198402bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 -- 172.21.15.7:0/1297451201 shutdown_connections 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 --2- 172.21.15.7:0/1297451201 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1994101200 0x7f1994108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 --2- 172.21.15.7:0/1297451201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994100840 0x7f1994100cc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 --2- 172.21.15.7:0/1297451201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1994101ba0 0x7f1994101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 -- 172.21.15.7:0/1297451201 >> 172.21.15.7:0/1297451201 conn(0x7f19940fb2b0 msgr2=0x7f19940fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 -- 172.21.15.7:0/1297451201 shutdown_connections 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 -- 172.21.15.7:0/1297451201 wait complete. 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 Processor -- start 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 -- start start 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.207+0000 7f1999b59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1994100840 0x7f1994197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1999b59640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994101200 0x7f1994197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1999b59640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1994101ba0 0x7f199419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1999b59640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199410c250 con 0x7f1994101200 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1999b59640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199410c0d0 con 0x7f1994100840 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1999b59640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199410c3d0 con 0x7f1994101ba0 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1992ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994101200 0x7f1994197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1992ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994101200 0x7f1994197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52532/0 (socket says 172.21.15.7:52532) 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1992ffd640 1 -- 172.21.15.7:0/725039744 learned_addr learned my addr 172.21.15.7:0/725039744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f19937fe640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1994100840 0x7f1994197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1993fff640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1994101ba0 0x7f199419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1992ffd640 1 -- 172.21.15.7:0/725039744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1994101ba0 msgr2=0x7f199419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1992ffd640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1994101ba0 0x7f199419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1992ffd640 1 -- 172.21.15.7:0/725039744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1994100840 msgr2=0x7f1994197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1992ffd640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1994100840 0x7f1994197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1992ffd640 1 -- 172.21.15.7:0/725039744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199419f690 con 0x7f1994101200 2024-03-26T08:51:01.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.208+0000 7f1992ffd640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994101200 0x7f1994197fb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1984000c00 tx=0x7f198402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:01.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.209+0000 7f1990ff9640 1 -- 172.21.15.7:0/725039744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984041070 con 0x7f1994101200 2024-03-26T08:51:01.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.209+0000 7f1990ff9640 1 -- 172.21.15.7:0/725039744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1984004450 con 0x7f1994101200 2024-03-26T08:51:01.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.209+0000 7f1999b59640 1 -- 172.21.15.7:0/725039744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1994102a70 con 0x7f1994101200 2024-03-26T08:51:01.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.209+0000 7f1999b59640 1 -- 172.21.15.7:0/725039744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1994102f60 con 0x7f1994101200 2024-03-26T08:51:01.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.209+0000 7f1990ff9640 1 -- 172.21.15.7:0/725039744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984034440 con 0x7f1994101200 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.212+0000 7f1990ff9640 1 -- 172.21.15.7:0/725039744 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1984044070 con 0x7f1994101200 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f199c812740 1 Processor -- start 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f199c812740 1 -- start start 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f199c812740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be18f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f199c812740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f199c812740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f199c812740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8be0ec3d0 con 0x55f8be18f5b0 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f199c812740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8be094ad0 con 0x55f8be195110 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f199c812740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8be0e8470 con 0x55f8be18ffa0 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f1993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be18f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f19937fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f1993fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be18f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52548/0 (socket says 172.21.15.7:52548) 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f1993fff640 1 -- 172.21.15.7:0/1891393074 learned_addr learned my addr 172.21.15.7:0/1891393074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.213+0000 7f1992ffd640 1 --2- 172.21.15.7:0/1891393074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1990ff9640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f197c0a2950 0x7f197c0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1993fff640 1 -- 172.21.15.7:0/1891393074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 msgr2=0x55f8be194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1993fff640 1 --2- 172.21.15.7:0/1891393074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be194aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1993fff640 1 -- 172.21.15.7:0/1891393074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 msgr2=0x55f8be1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1993fff640 1 --2- 172.21.15.7:0/1891393074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be1975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1993fff640 1 -- 172.21.15.7:0/1891393074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8bdeca7d0 con 0x55f8be18f5b0 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1990ff9640 1 -- 172.21.15.7:0/725039744 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f19840f0310 con 0x7f1994101200 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f19937fe640 1 --2- 172.21.15.7:0/1891393074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1993fff640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f197c0a2950 0x7f197c0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1993fff640 1 --2- 172.21.15.7:0/1891393074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be18f9d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1988009ca0 tx=0x7f1988011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d499c3a6263240 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.214+0000 7f1979ffb640 1 -- 172.21.15.7:0/1891393074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198801b870 con 0x55f8be18f5b0 2024-03-26T08:51:01.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.215+0000 7f1979ffb640 1 -- 172.21.15.7:0/1891393074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19880193b0 con 0x55f8be18f5b0 2024-03-26T08:51:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.215+0000 7f1979ffb640 1 -- 172.21.15.7:0/1891393074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198800e610 con 0x55f8be18f5b0 2024-03-26T08:51:01.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.215+0000 7f199c812740 1 -- 172.21.15.7:0/1891393074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 msgr2=0x55f8be18f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.215+0000 7f1993fff640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f197c0a2950 0x7f197c0a4e10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1988020000 tx=0x7f198800ecc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.215+0000 7f199c812740 1 --2- 172.21.15.7:0/1891393074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be18f9d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1988009ca0 tx=0x7f1988011810 comp rx=0 tx=0).stop 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 -- 172.21.15.7:0/1891393074 shutdown_connections 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 --2- 172.21.15.7:0/1891393074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 --2- 172.21.15.7:0/1891393074 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 --2- 172.21.15.7:0/1891393074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be18f9d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 -- 172.21.15.7:0/1891393074 >> 172.21.15.7:0/1891393074 conn(0x55f8be18ab20 msgr2=0x55f8be18af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 -- 172.21.15.7:0/1891393074 shutdown_connections 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 -- 172.21.15.7:0/1891393074 wait complete. 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 -- start start 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 Processor -- start 2024-03-26T08:51:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.216+0000 7f199c812740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be199a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f199c812740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be199f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f199c812740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f199c812740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8be1980c0 con 0x55f8be18f5b0 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f199c812740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8be0a8060 con 0x55f8be195110 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f199c812740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8be198240 con 0x55f8be18ffa0 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f199c812740 10 client.0 _ll_register_callbacks cb 0x55f8be08a6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f19937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f19937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52550/0 (socket says 172.21.15.7:52550) 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f19937fe640 1 -- 172.21.15.7:0/3791523645 learned_addr learned my addr 172.21.15.7:0/3791523645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1992ffd640 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1993fff640 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1992ffd640 1 -- 172.21.15.7:0/3791523645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 msgr2=0x55f8be1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1992ffd640 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1992ffd640 1 -- 172.21.15.7:0/3791523645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 msgr2=0x55f8be199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1992ffd640 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1992ffd640 1 -- 172.21.15.7:0/3791523645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8bde003f0 con 0x55f8be18ffa0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f19937fe640 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1992ffd640 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be199f70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f198402c7b0 tx=0x7f1984009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12589fcba763a7cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1972ffd640 1 -- 172.21.15.7:0/3791523645 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198404b040 con 0x55f8be18ffa0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f199c812740 1 -- 172.21.15.7:0/3791523645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19880193b0 con 0x55f8be18ffa0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.217+0000 7f1972ffd640 1 -- 172.21.15.7:0/3791523645 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f19840bd1c0 con 0x55f8be18ffa0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f1972ffd640 1 -- 172.21.15.7:0/3791523645 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19840bcd60 con 0x55f8be18ffa0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f199c812740 10 client.27463 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f199c812740 1 -- 172.21.15.7:0/3791523645 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8bddc00f0 con 0x55f8be18ffa0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f1972ffd640 1 -- 172.21.15.7:0/3791523645 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19880193b0 con 0x55f8be18ffa0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f1972ffd640 1 -- 172.21.15.7:0/3791523645 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f19840035f0 con 0x55f8be18ffa0 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f199c812740 10 client.27463 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f199c812740 10 client.27463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:01.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f199c812740 20 client.27463 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f199c812740 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f8be1a5530 0x55f8be1a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f199c812740 4 client.27463 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.218+0000 7f199c812740 1 -- 172.21.15.7:0/3791523645 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8be0f9910 con 0x55f8be1a5530 2024-03-26T08:51:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.220+0000 7f19937fe640 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f8be1a5530 0x55f8be1a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.221+0000 7f19937fe640 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f8be1a5530 0x55f8be1a79b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53d67d3dfe184a84 server_cookie=15b595ea32373dc in_seq=0 out_seq=0 2024-03-26T08:51:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.221+0000 7f1972ffd640 10 client.27463 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.221+0000 7f1972ffd640 1 -- 172.21.15.7:0/3791523645 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f8be0f9910 con 0x55f8be1a5530 2024-03-26T08:51:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.221+0000 7f1972ffd640 10 client.27463 handle_command_reply: tid=0 2024-03-26T08:51:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.221+0000 7f199c812740 1 client.27463 shutdown 2024-03-26T08:51:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 -- 172.21.15.7:0/3791523645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 msgr2=0x55f8be199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be199f70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f198402c7b0 tx=0x7f1984009260 comp rx=0 tx=0).stop 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 -- 172.21.15.7:0/3791523645 shutdown_connections 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f8be1a5530 0x55f8be1a79b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f1972ffd640 0 client.27463 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be199f70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 --2- 172.21.15.7:0/3791523645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 -- 172.21.15.7:0/3791523645 >> 172.21.15.7:0/3791523645 conn(0x55f8be18ab20 msgr2=0x55f8be181530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 -- 172.21.15.7:0/3791523645 shutdown_connections 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 1 -- 172.21.15.7:0/3791523645 wait complete. 2024-03-26T08:51:01.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.222+0000 7f199c812740 20 client.27463 trim_cache size 0 max 16384 2024-03-26T08:51:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.248+0000 7f199c812740 1 Processor -- start 2024-03-26T08:51:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.248+0000 7f199c812740 1 -- start start 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.248+0000 7f199c812740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be1949d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.248+0000 7f199c812740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be1a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.248+0000 7f199c812740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be180ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.248+0000 7f199c812740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980010830 con 0x55f8be18f5b0 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.248+0000 7f199c812740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198000e330 con 0x55f8be195110 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.248+0000 7f199c812740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198000c3b0 con 0x55f8be18ffa0 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f19937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be1949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f1992ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be1a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f19937fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be1949d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52566/0 (socket says 172.21.15.7:52566) 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f1992ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be1a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60888/0 (socket says 172.21.15.7:60888) 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f19937fe640 1 -- 172.21.15.7:0/2364539555 learned_addr learned my addr 172.21.15.7:0/2364539555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f1993fff640 1 --2- 172.21.15.7:0/2364539555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be180ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f1992ffd640 1 -- 172.21.15.7:0/2364539555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 msgr2=0x55f8be180ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f1992ffd640 1 --2- 172.21.15.7:0/2364539555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be180ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f1992ffd640 1 -- 172.21.15.7:0/2364539555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 msgr2=0x55f8be1949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f1992ffd640 1 --2- 172.21.15.7:0/2364539555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be1949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f1992ffd640 1 -- 172.21.15.7:0/2364539555 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8be1a8030 con 0x55f8be18ffa0 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f19937fe640 1 --2- 172.21.15.7:0/2364539555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be1949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f1992ffd640 1 --2- 172.21.15.7:0/2364539555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be1a0f70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1984009900 tx=0x7f198402cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177d17c0cd50fa89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f19797fa640 1 -- 172.21.15.7:0/2364539555 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198404b070 con 0x55f8be18ffa0 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f19797fa640 1 -- 172.21.15.7:0/2364539555 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19840d9020 con 0x55f8be18ffa0 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.249+0000 7f19797fa640 1 -- 172.21.15.7:0/2364539555 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840c5200 con 0x55f8be18ffa0 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.250+0000 7f199c812740 1 -- 172.21.15.7:0/2364539555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 msgr2=0x55f8be1a0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.250+0000 7f199c812740 1 --2- 172.21.15.7:0/2364539555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be1a0f70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1984009900 tx=0x7f198402cf90 comp rx=0 tx=0).stop 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.250+0000 7f199c812740 1 -- 172.21.15.7:0/2364539555 shutdown_connections 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.250+0000 7f199c812740 1 --2- 172.21.15.7:0/2364539555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be195110 0x55f8be180ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.250+0000 7f199c812740 1 --2- 172.21.15.7:0/2364539555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be18ffa0 0x55f8be1a0f70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.250+0000 7f199c812740 1 --2- 172.21.15.7:0/2364539555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18f5b0 0x55f8be1949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.250+0000 7f199c812740 1 -- 172.21.15.7:0/2364539555 >> 172.21.15.7:0/2364539555 conn(0x55f8be18ab20 msgr2=0x55f8be199a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.250+0000 7f199c812740 1 -- 172.21.15.7:0/2364539555 shutdown_connections 2024-03-26T08:51:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.250+0000 7f199c812740 1 -- 172.21.15.7:0/2364539555 wait complete. 2024-03-26T08:51:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.251+0000 7f199c812740 1 -- start start 2024-03-26T08:51:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.251+0000 7f199c812740 1 Processor -- start 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f199c812740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be18f5b0 0x55f8be19f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f199c812740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18ffa0 0x55f8be19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f199c812740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be195110 0x55f8be1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f199c812740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198000c9b0 con 0x55f8be18ffa0 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f199c812740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198000dcd0 con 0x55f8be18f5b0 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f199c812740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198000d010 con 0x55f8be195110 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f199c812740 10 client.0 _ll_register_callbacks cb 0x7f19840f38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f19937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be18f5b0 0x55f8be19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f19937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be18f5b0 0x55f8be19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46510/0 (socket says 172.21.15.7:46510) 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f19937fe640 1 -- 172.21.15.7:0/1546619828 learned_addr learned my addr 172.21.15.7:0/1546619828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f1992ffd640 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18ffa0 0x55f8be19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f1993fff640 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be195110 0x55f8be1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f19937fe640 1 -- 172.21.15.7:0/1546619828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be195110 msgr2=0x55f8be1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f19937fe640 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be195110 0x55f8be1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f19937fe640 1 -- 172.21.15.7:0/1546619828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18ffa0 msgr2=0x55f8be19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f19937fe640 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18ffa0 0x55f8be19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f19937fe640 1 -- 172.21.15.7:0/1546619828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8be1a0b00 con 0x55f8be18f5b0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f19937fe640 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be18f5b0 0x55f8be19f080 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55f8be1a8260 tx=0x7f198c012ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=855f46be4ec00145 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.252+0000 7f1972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f1972ffd640 1 -- 172.21.15.7:0/1546619828 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c004d20 con 0x55f8be18f5b0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f1972ffd640 1 -- 172.21.15.7:0/1546619828 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f198c0207e0 con 0x55f8be18f5b0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f199c812740 1 -- 172.21.15.7:0/1546619828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19840d9020 con 0x55f8be18f5b0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f1972ffd640 1 -- 172.21.15.7:0/1546619828 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f198c0202f0 con 0x55f8be18f5b0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f199c812740 10 client.27455 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f199c812740 1 -- 172.21.15.7:0/1546619828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8be2018e0 con 0x55f8be18f5b0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f1972ffd640 1 -- 172.21.15.7:0/1546619828 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f19840d9020 con 0x55f8be18f5b0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f1972ffd640 1 -- 172.21.15.7:0/1546619828 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f198c004620 con 0x55f8be18f5b0 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f199c812740 10 client.27455 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f199c812740 10 client.27455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f199c812740 20 client.27455 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f199c812740 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f8be2022a0 0x55f8be2046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f199c812740 4 client.27455 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:51:01.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.253+0000 7f199c812740 1 -- 172.21.15.7:0/1546619828 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f8be0f3c60 con 0x55f8be2022a0 2024-03-26T08:51:01.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.254+0000 7f1992ffd640 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f8be2022a0 0x55f8be2046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.254+0000 7f1972ffd640 10 client.27455 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:01.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.254+0000 7f1992ffd640 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f8be2022a0 0x55f8be2046c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da2f55e74abaf76 server_cookie=8a6a0af8c646aeda in_seq=0 out_seq=0 2024-03-26T08:51:01.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.255+0000 7f1972ffd640 1 -- 172.21.15.7:0/1546619828 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f8be0f3c60 con 0x55f8be2022a0 2024-03-26T08:51:01.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.255+0000 7f1972ffd640 10 client.27455 handle_command_reply: tid=0 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f199c812740 1 client.27455 shutdown 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f199c812740 1 -- 172.21.15.7:0/1546619828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be18f5b0 msgr2=0x55f8be19f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f199c812740 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be18f5b0 0x55f8be19f080 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55f8be1a8260 tx=0x7f198c012ea0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f199c812740 1 -- 172.21.15.7:0/1546619828 shutdown_connections 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f199c812740 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f8be2022a0 0x55f8be2046c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f199c812740 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f8be195110 0x55f8be1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f199c812740 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8be18ffa0 0x55f8be19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f199c812740 1 --2- 172.21.15.7:0/1546619828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f8be18f5b0 0x55f8be19f080 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f199c812740 1 -- 172.21.15.7:0/1546619828 >> 172.21.15.7:0/1546619828 conn(0x55f8be18ab20 msgr2=0x55f8be1999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.256+0000 7f1972ffd640 0 client.27455 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.257+0000 7f199c812740 1 -- 172.21.15.7:0/1546619828 shutdown_connections 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.257+0000 7f199c812740 1 -- 172.21.15.7:0/1546619828 wait complete. 2024-03-26T08:51:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.257+0000 7f199c812740 20 client.27455 trim_cache size 0 max 16384 2024-03-26T08:51:01.255 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:51:01.255 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:51:01.255 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:51:01.255 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.257+0000 7f1999b59640 1 -- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f197c0a2950 msgr2=0x7f197c0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.257+0000 7f1999b59640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f197c0a2950 0x7f197c0a4e10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1988020000 tx=0x7f198800ecc0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.257+0000 7f1999b59640 1 -- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994101200 msgr2=0x7f1994197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.257+0000 7f1999b59640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994101200 0x7f1994197fb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1984000c00 tx=0x7f198402c650 comp rx=0 tx=0).stop 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.258+0000 7f1999b59640 1 -- 172.21.15.7:0/725039744 shutdown_connections 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.258+0000 7f1999b59640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f197c0a2950 0x7f197c0a4e10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.258+0000 7f1999b59640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1994101ba0 0x7f199419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.258+0000 7f1999b59640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1994101200 0x7f1994197fb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.258+0000 7f1999b59640 1 --2- 172.21.15.7:0/725039744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1994100840 0x7f1994197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.258+0000 7f1999b59640 1 -- 172.21.15.7:0/725039744 >> 172.21.15.7:0/725039744 conn(0x7f19940fb2b0 msgr2=0x7f19940fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.258+0000 7f1999b59640 1 -- 172.21.15.7:0/725039744 shutdown_connections 2024-03-26T08:51:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.258+0000 7f1999b59640 1 -- 172.21.15.7:0/725039744 wait complete. 2024-03-26T08:51:01.266 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:51:01.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:51:01.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/2402273180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdbc102a30 msgr2=0x7fcdbc102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/2402273180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdbc102a30 0x7fcdbc102eb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009930 tx=0x7fcda402bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/2402273180 shutdown_connections 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/2402273180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdbc1033f0 0x7fcdbc109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/2402273180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdbc102a30 0x7fcdbc102eb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/2402273180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ff9c0 0x7fcdbc0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/2402273180 >> 172.21.15.7:0/2402273180 conn(0x7fcdbc0fb2d0 msgr2=0x7fcdbc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/2402273180 shutdown_connections 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/2402273180 wait complete. 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 Processor -- start 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.470+0000 7fcdc1bcd640 1 -- start start 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdc1bcd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ff9c0 0x7fcdbc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdc1bcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdbc102a30 0x7fcdbc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdc1bcd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdbc1033f0 0x7fcdbc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdc1bcd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc10d3a0 con 0x7fcdbc1033f0 2024-03-26T08:51:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdc1bcd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc10d220 con 0x7fcdbc102a30 2024-03-26T08:51:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdc1bcd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdbc10d520 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdbb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ff9c0 0x7fcdbc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdbc102a30 0x7fcdbc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdbc102a30 0x7fcdbc197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46518/0 (socket says 172.21.15.7:46518) 2024-03-26T08:51:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdb2ffd640 1 -- 172.21.15.7:0/1186296278 learned_addr learned my addr 172.21.15.7:0/1186296278 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdbbfff640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdbc1033f0 0x7fcdbc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.471+0000 7fcdbb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ff9c0 0x7fcdbc197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60926/0 (socket says 172.21.15.7:60926) 2024-03-26T08:51:01.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdbb7fe640 1 -- 172.21.15.7:0/1186296278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdbc102a30 msgr2=0x7fcdbc197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdbb7fe640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdbc102a30 0x7fcdbc197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdbb7fe640 1 -- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdbc1033f0 msgr2=0x7fcdbc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdbb7fe640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdbc1033f0 0x7fcdbc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdbb7fe640 1 -- 172.21.15.7:0/1186296278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc19f660 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdb2ffd640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdbc102a30 0x7fcdbc197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdbbfff640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdbc1033f0 0x7fcdbc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdbb7fe640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ff9c0 0x7fcdbc197a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e9c0 tx=0x7fcdac00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdb97fa640 1 -- 172.21.15.7:0/1186296278 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ccc0 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc19f950 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdb97fa640 1 -- 172.21.15.7:0/1186296278 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcdac0035f0 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdb97fa640 1 -- 172.21.15.7:0/1186296278 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ce60 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.472+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc19fe70 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.473+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdbc068970 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.474+0000 7fcdb97fa640 1 -- 172.21.15.7:0/1186296278 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcdac0037a0 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.475+0000 7fcdb97fa640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda80a2800 0x7fcda80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.475+0000 7fcdb97fa640 1 -- 172.21.15.7:0/1186296278 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcdac0cee40 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.475+0000 7fcdb2ffd640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda80a2800 0x7fcda80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:01.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.476+0000 7fcdb2ffd640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda80a2800 0x7fcda80a4cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcda4006010 tx=0x7fcda402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:01.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.479+0000 7fcdb97fa640 1 -- 172.21.15.7:0/1186296278 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcdac09bee0 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.834+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdbc108b60 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.835+0000 7fcdb97fa640 1 -- 172.21.15.7:0/1186296278 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fcdac09b7e0 con 0x7fcdbc0ff9c0 2024-03-26T08:51:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:51:01.830 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:51:01.831 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:51:01.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.838+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda80a2800 msgr2=0x7fcda80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.838+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda80a2800 0x7fcda80a4cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcda4006010 tx=0x7fcda402df90 comp rx=0 tx=0).stop 2024-03-26T08:51:01.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.838+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ff9c0 msgr2=0x7fcdbc197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:01.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.838+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ff9c0 0x7fcdbc197a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e9c0 tx=0x7fcdac00ee80 comp rx=0 tx=0).stop 2024-03-26T08:51:01.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.839+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 shutdown_connections 2024-03-26T08:51:01.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.839+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcda80a2800 0x7fcda80a4cc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.839+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdbc1033f0 0x7fcdbc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.839+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdbc102a30 0x7fcdbc197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.839+0000 7fcdc1bcd640 1 --2- 172.21.15.7:0/1186296278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcdbc0ff9c0 0x7fcdbc197a40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.839+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 >> 172.21.15.7:0/1186296278 conn(0x7fcdbc0fb2d0 msgr2=0x7fcdbc105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.839+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 shutdown_connections 2024-03-26T08:51:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:01.839+0000 7fcdc1bcd640 1 -- 172.21.15.7:0/1186296278 wait complete. 2024-03-26T08:51:01.847 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:51:01.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.006+0000 7f12a3af7640 1 -- 172.21.15.7:0/3767344140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f129c103280 msgr2=0x7f129c109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.006+0000 7f12a3af7640 1 --2- 172.21.15.7:0/3767344140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f129c103280 0x7f129c109b10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1294013490 tx=0x7f1294034350 comp rx=0 tx=0).stop 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.006+0000 7f12a3af7640 1 -- 172.21.15.7:0/3767344140 shutdown_connections 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.006+0000 7f12a3af7640 1 --2- 172.21.15.7:0/3767344140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f129c103280 0x7f129c109b10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.006+0000 7f12a3af7640 1 --2- 172.21.15.7:0/3767344140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c1028c0 0x7f129c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.006+0000 7f12a3af7640 1 --2- 172.21.15.7:0/3767344140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c1016d0 0x7f129c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.006+0000 7f12a3af7640 1 -- 172.21.15.7:0/3767344140 >> 172.21.15.7:0/3767344140 conn(0x7f129c0fcc60 msgr2=0x7f129c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.006+0000 7f12a3af7640 1 -- 172.21.15.7:0/3767344140 shutdown_connections 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.006+0000 7f12a3af7640 1 -- 172.21.15.7:0/3767344140 wait complete. 2024-03-26T08:51:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a3af7640 1 Processor -- start 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a3af7640 1 -- start start 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a3af7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f129c1016d0 0x7f129c1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a3af7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c1028c0 0x7f129c195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a3af7640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c103280 0x7f129c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a3af7640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c10b250 con 0x7f129c103280 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a3af7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c10b0d0 con 0x7f129c1028c0 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a3af7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c10b3d0 con 0x7f129c1016d0 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a1092640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c1028c0 0x7f129c195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a1092640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c1028c0 0x7f129c195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46548/0 (socket says 172.21.15.7:46548) 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a2094640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c103280 0x7f129c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a1092640 1 -- 172.21.15.7:0/886077140 learned_addr learned my addr 172.21.15.7:0/886077140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.007+0000 7f12a1893640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f129c1016d0 0x7f129c1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f12a1893640 1 -- 172.21.15.7:0/886077140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c1028c0 msgr2=0x7f129c195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f12a1893640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c1028c0 0x7f129c195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f12a1893640 1 -- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c103280 msgr2=0x7f129c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f12a1893640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c103280 0x7f129c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f12a1893640 1 -- 172.21.15.7:0/886077140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c19d500 con 0x7f129c1016d0 2024-03-26T08:51:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f12a2094640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c103280 0x7f129c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f12a1893640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f129c1016d0 0x7f129c1958e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1298009fd0 tx=0x7f129800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f1292ffd640 1 -- 172.21.15.7:0/886077140 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298015020 con 0x7f129c1016d0 2024-03-26T08:51:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f1292ffd640 1 -- 172.21.15.7:0/886077140 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1298011d10 con 0x7f129c1016d0 2024-03-26T08:51:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.008+0000 7f1292ffd640 1 -- 172.21.15.7:0/886077140 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800f2e0 con 0x7f129c1016d0 2024-03-26T08:51:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.009+0000 7f12a3af7640 1 -- 172.21.15.7:0/886077140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c19d7f0 con 0x7f129c1016d0 2024-03-26T08:51:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.009+0000 7f12a3af7640 1 -- 172.21.15.7:0/886077140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c19dd70 con 0x7f129c1016d0 2024-03-26T08:51:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a67b7740 1 Processor -- start 2024-03-26T08:51:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a67b7740 1 -- start start 2024-03-26T08:51:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ff1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff2080 0x563901ff6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff71f0 0x563901ff9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a67b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563901fb4610 con 0x563901ff1690 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a67b7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563901f573c0 con 0x563901ff71f0 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a67b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563901effd60 con 0x563901ff2080 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f1292ffd640 1 -- 172.21.15.7:0/886077140 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1298018020 con 0x7f129c1016d0 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a2094640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ff1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a2094640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ff1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52616/0 (socket says 172.21.15.7:52616) 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a2094640 1 -- 172.21.15.7:0/2121128928 learned_addr learned my addr 172.21.15.7:0/2121128928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a1092640 1 --2- 172.21.15.7:0/2121128928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff71f0 0x563901ff9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.012+0000 7f12a1893640 1 --2- 172.21.15.7:0/2121128928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff2080 0x563901ff6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.013+0000 7f12a1092640 1 -- 172.21.15.7:0/2121128928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff2080 msgr2=0x563901ff6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.013+0000 7f12a1092640 1 --2- 172.21.15.7:0/2121128928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff2080 0x563901ff6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.013+0000 7f12a1092640 1 -- 172.21.15.7:0/2121128928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 msgr2=0x563901ff1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.013+0000 7f12a1092640 1 --2- 172.21.15.7:0/2121128928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ff1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.013+0000 7f12a1092640 1 -- 172.21.15.7:0/2121128928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563901c6b3f0 con 0x563901ff71f0 2024-03-26T08:51:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.013+0000 7f12a1092640 1 --2- 172.21.15.7:0/2121128928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff71f0 0x563901ff9680 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f128c009a20 tx=0x7f128c009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9801c7e4da01599 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.013+0000 7f1292ffd640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f12880a2950 0x7f12880a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.013+0000 7f1292ffd640 1 -- 172.21.15.7:0/886077140 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f12980ce150 con 0x7f129c1016d0 2024-03-26T08:51:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.014+0000 7f1287fff640 1 -- 172.21.15.7:0/2121128928 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c017070 con 0x563901ff71f0 2024-03-26T08:51:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.014+0000 7f1287fff640 1 -- 172.21.15.7:0/2121128928 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f128c013470 con 0x563901ff71f0 2024-03-26T08:51:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.014+0000 7f1287fff640 1 -- 172.21.15.7:0/2121128928 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c012630 con 0x563901ff71f0 2024-03-26T08:51:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.014+0000 7f12a2094640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f12880a2950 0x7f12880a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.015+0000 7f12a67b7740 1 -- 172.21.15.7:0/2121128928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff71f0 msgr2=0x563901ff9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.015+0000 7f12a2094640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f12880a2950 0x7f12880a4e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f129403cc30 tx=0x7f1294036040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.015+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2121128928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff71f0 0x563901ff9680 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f128c009a20 tx=0x7f128c009ee0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.015+0000 7f12a67b7740 1 -- 172.21.15.7:0/2121128928 shutdown_connections 2024-03-26T08:51:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.015+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2121128928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff71f0 0x563901ff9680 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.015+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2121128928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff2080 0x563901ff6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.015+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2121128928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ff1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.015+0000 7f12a67b7740 1 -- 172.21.15.7:0/2121128928 >> 172.21.15.7:0/2121128928 conn(0x563901fecc00 msgr2=0x563901fed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 -- 172.21.15.7:0/2121128928 shutdown_connections 2024-03-26T08:51:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 -- 172.21.15.7:0/2121128928 wait complete. 2024-03-26T08:51:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 -- start start 2024-03-26T08:51:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 Processor -- start 2024-03-26T08:51:02.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ffba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563901ffbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563902003000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563901f13240 con 0x563901ff1690 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563901f53460 con 0x563901ff2080 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563901ffa1a0 con 0x563901ff71f0 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.016+0000 7f12a67b7740 10 client.0 _ll_register_callbacks cb 0x563901f12c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1893640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ffba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1092640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563901ffbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1893640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ffba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52618/0 (socket says 172.21.15.7:52618) 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1893640 1 -- 172.21.15.7:0/1700440852 learned_addr learned my addr 172.21.15.7:0/1700440852 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a2094640 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563902003000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1092640 1 -- 172.21.15.7:0/1700440852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 msgr2=0x563902003000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1092640 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563902003000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1092640 1 -- 172.21.15.7:0/1700440852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 msgr2=0x563901ffba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1092640 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ffba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1092640 1 -- 172.21.15.7:0/1700440852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563901d357d0 con 0x563901ff2080 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a67b7740 1 -- 172.21.15.7:0/1700440852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f128c013470 con 0x563901ff2080 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12a1092640 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563901ffbfc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f128c000c00 tx=0x7f128c013bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8ea4fc3daa5f84 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.017+0000 7f12857fa640 10 client.27470 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12857fa640 1 -- 172.21.15.7:0/1700440852 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c017050 con 0x563901ff2080 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12857fa640 1 -- 172.21.15.7:0/1700440852 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f128c01c450 con 0x563901ff2080 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12857fa640 1 -- 172.21.15.7:0/1700440852 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f128c01c6b0 con 0x563901ff2080 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12a67b7740 10 client.27470 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12a67b7740 1 -- 172.21.15.7:0/1700440852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x5639020062a0 con 0x563901ff2080 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12857fa640 1 -- 172.21.15.7:0/1700440852 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f128c01c850 con 0x563901ff2080 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12a67b7740 10 client.27470 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12a67b7740 10 client.27470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12a67b7740 20 client.27470 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5639020075f0 0x5639020099e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12a67b7740 4 client.27470 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.018+0000 7f12a67b7740 1 -- 172.21.15.7:0/1700440852 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563901f64900 con 0x5639020075f0 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.019+0000 7f12a1893640 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5639020075f0 0x5639020099e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.019+0000 7f12a1893640 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5639020075f0 0x5639020099e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e1ad2930d82ca15 server_cookie=fbcd1e19bfb5d5f0 in_seq=0 out_seq=0 2024-03-26T08:51:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.019+0000 7f12857fa640 10 client.27470 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:51:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12857fa640 1 -- 172.21.15.7:0/1700440852 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563901f64900 con 0x5639020075f0 2024-03-26T08:51:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12857fa640 10 client.27470 handle_command_reply: tid=0 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 client.27470 shutdown 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 -- 172.21.15.7:0/1700440852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 msgr2=0x563901ffbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563901ffbfc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f128c000c00 tx=0x7f128c013bc0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 -- 172.21.15.7:0/1700440852 shutdown_connections 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5639020075f0 0x5639020099e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563902003000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563901ffbfc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12857fa640 0 client.27470 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1700440852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ffba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 -- 172.21.15.7:0/1700440852 >> 172.21.15.7:0/1700440852 conn(0x563901fecc00 msgr2=0x563901febe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 -- 172.21.15.7:0/1700440852 shutdown_connections 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 1 -- 172.21.15.7:0/1700440852 wait complete. 2024-03-26T08:51:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.020+0000 7f12a67b7740 20 client.27470 trim_cache size 0 max 16384 2024-03-26T08:51:02.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.038+0000 7f12a67b7740 1 Processor -- start 2024-03-26T08:51:02.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.038+0000 7f12a67b7740 1 -- start start 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.038+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ffad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.038+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563902002fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.038+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563901ffb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a67b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1a8f10 con 0x563901ff1690 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a67b7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1a6a10 con 0x563901ff2080 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a67b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1a4a90 con 0x563901ff71f0 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a2094640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563901ffb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a2094640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563901ffb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60966/0 (socket says 172.21.15.7:60966) 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a2094640 1 -- 172.21.15.7:0/1105904649 learned_addr learned my addr 172.21.15.7:0/1105904649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a1893640 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ffad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a1092640 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563902002fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a2094640 1 -- 172.21.15.7:0/1105904649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 msgr2=0x563902002fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a2094640 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563902002fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a2094640 1 -- 172.21.15.7:0/1105904649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 msgr2=0x563901ffad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a2094640 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ffad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a2094640 1 -- 172.21.15.7:0/1105904649 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563901fed970 con 0x563901ff71f0 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a1092640 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563902002fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.039+0000 7f12a2094640 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563901ffb290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1294013490 tx=0x7f1294048000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39fa53165ae146c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.040+0000 7f12877fe640 1 -- 172.21.15.7:0/1105904649 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129400fe60 con 0x563901ff71f0 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.040+0000 7f12877fe640 1 -- 172.21.15.7:0/1105904649 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1294003e50 con 0x563901ff71f0 2024-03-26T08:51:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.040+0000 7f12877fe640 1 -- 172.21.15.7:0/1105904649 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294004280 con 0x563901ff71f0 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.040+0000 7f12a67b7740 1 -- 172.21.15.7:0/1105904649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 msgr2=0x563901ffb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.040+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563901ffb290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1294013490 tx=0x7f1294048000 comp rx=0 tx=0).stop 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.041+0000 7f12a67b7740 1 -- 172.21.15.7:0/1105904649 shutdown_connections 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.041+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563901ffb290 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.041+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff2080 0x563902002fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.041+0000 7f12a67b7740 1 --2- 172.21.15.7:0/1105904649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff1690 0x563901ffad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.041+0000 7f12a67b7740 1 -- 172.21.15.7:0/1105904649 >> 172.21.15.7:0/1105904649 conn(0x563901fecc00 msgr2=0x563901ff3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.041+0000 7f12a67b7740 1 -- 172.21.15.7:0/1105904649 shutdown_connections 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.041+0000 7f12a67b7740 1 -- 172.21.15.7:0/1105904649 wait complete. 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.041+0000 7f12a67b7740 1 -- start start 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.041+0000 7f12a67b7740 1 Processor -- start 2024-03-26T08:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff1690 0x5639020008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff2080 0x563902000e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a67b7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563902001dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a67b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1a5090 con 0x563901ff2080 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a67b7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1a63b0 con 0x563901ff1690 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a67b7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1a56f0 con 0x563901ff71f0 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a67b7740 10 client.0 _ll_register_callbacks cb 0x7f128c0098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a1092640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff2080 0x563902000e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a1893640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff1690 0x5639020008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a1893640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff1690 0x5639020008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46582/0 (socket says 172.21.15.7:46582) 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a1893640 1 -- 172.21.15.7:0/2294751451 learned_addr learned my addr 172.21.15.7:0/2294751451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a2094640 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563902001dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a1893640 1 -- 172.21.15.7:0/2294751451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 msgr2=0x563902001dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a1893640 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563902001dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a1893640 1 -- 172.21.15.7:0/2294751451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff2080 msgr2=0x563902000e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a1893640 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff2080 0x563902000e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a1893640 1 -- 172.21.15.7:0/2294751451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563901ffcbb0 con 0x563901ff1690 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.042+0000 7f12a2094640 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563902001dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a1893640 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff1690 0x5639020008f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f12980a2d60 tx=0x7f129809a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b77be623f097bbce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12857fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12857fa640 1 -- 172.21.15.7:0/2294751451 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12980091f0 con 0x563901ff1690 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a67b7740 1 -- 172.21.15.7:0/2294751451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1294003e50 con 0x563901ff1690 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12857fa640 1 -- 172.21.15.7:0/2294751451 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f12980d16a0 con 0x563901ff1690 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12857fa640 1 -- 172.21.15.7:0/2294751451 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f129809b1f0 con 0x563901ff1690 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a67b7740 10 client.27476 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a67b7740 1 -- 172.21.15.7:0/2294751451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563902062d00 con 0x563901ff1690 2024-03-26T08:51:02.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12857fa640 1 -- 172.21.15.7:0/2294751451 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1294003e50 con 0x563901ff1690 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12857fa640 1 -- 172.21.15.7:0/2294751451 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1298097160 con 0x563901ff1690 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a67b7740 10 client.27476 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a67b7740 10 client.27476 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a67b7740 20 client.27476 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563902063500 0x563902065930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a67b7740 4 client.27476 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.043+0000 7f12a67b7740 1 -- 172.21.15.7:0/2294751451 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563901f5ec50 con 0x563902063500 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.044+0000 7f12a1092640 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563902063500 0x563902065930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.044+0000 7f12a1092640 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563902063500 0x563902065930 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0146aa3587b3f91 server_cookie=17e35c934018974b in_seq=0 out_seq=0 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.044+0000 7f12857fa640 10 client.27476 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.044+0000 7f12857fa640 1 -- 172.21.15.7:0/2294751451 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563901f5ec50 con 0x563902063500 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.044+0000 7f12857fa640 10 client.27476 handle_command_reply: tid=0 2024-03-26T08:51:02.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.044+0000 7f12a67b7740 1 client.27476 shutdown 2024-03-26T08:51:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.044+0000 7f12a67b7740 1 -- 172.21.15.7:0/2294751451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff1690 msgr2=0x5639020008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.044+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff1690 0x5639020008f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f12980a2d60 tx=0x7f129809a8c0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.045+0000 7f12a67b7740 1 -- 172.21.15.7:0/2294751451 shutdown_connections 2024-03-26T08:51:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.045+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x563902063500 0x563902065930 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.045+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563901ff71f0 0x563902001dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.045+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563901ff2080 0x563902000e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.045+0000 7f12a67b7740 1 --2- 172.21.15.7:0/2294751451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563901ff1690 0x5639020008f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.045+0000 7f12857fa640 0 client.27476 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:51:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.045+0000 7f12a67b7740 1 -- 172.21.15.7:0/2294751451 >> 172.21.15.7:0/2294751451 conn(0x563901fecc00 msgr2=0x563901ffad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.045+0000 7f12a67b7740 1 -- 172.21.15.7:0/2294751451 shutdown_connections 2024-03-26T08:51:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.046+0000 7f12a67b7740 1 -- 172.21.15.7:0/2294751451 wait complete. 2024-03-26T08:51:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.046+0000 7f12a67b7740 20 client.27476 trim_cache size 0 max 16384 2024-03-26T08:51:02.045 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:51:02.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 -- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f12880a2950 msgr2=0x7f12880a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f12880a2950 0x7f12880a4e10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f129403cc30 tx=0x7f1294036040 comp rx=0 tx=0).stop 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 -- 172.21.15.7:0/886077140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f129c1016d0 msgr2=0x7f129c1958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f129c1016d0 0x7f129c1958e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1298009fd0 tx=0x7f129800b870 comp rx=0 tx=0).stop 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 -- 172.21.15.7:0/886077140 shutdown_connections 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f12880a2950 0x7f12880a4e10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c103280 0x7f129c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129c1028c0 0x7f129c195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 --2- 172.21.15.7:0/886077140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f129c1016d0 0x7f129c1958e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 -- 172.21.15.7:0/886077140 >> 172.21.15.7:0/886077140 conn(0x7f129c0fcc60 msgr2=0x7f129c105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 -- 172.21.15.7:0/886077140 shutdown_connections 2024-03-26T08:51:02.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.047+0000 7f12a3af7640 1 -- 172.21.15.7:0/886077140 wait complete. 2024-03-26T08:51:02.055 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:51:02.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:51:02.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.214+0000 7fb59b7fe640 1 -- 172.21.15.7:0/1286510389 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588032340 con 0x7fb59c1017b0 2024-03-26T08:51:02.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.215+0000 7fb5a2d05640 1 -- 172.21.15.7:0/1286510389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb59c1017b0 msgr2=0x7fb59c101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.215+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/1286510389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb59c1017b0 0x7fb59c101c30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb588009860 tx=0x7fb58802bb90 comp rx=0 tx=0).stop 2024-03-26T08:51:02.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.216+0000 7fb5a2d05640 1 -- 172.21.15.7:0/1286510389 shutdown_connections 2024-03-26T08:51:02.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.216+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/1286510389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb59c102170 0x7fb59c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.216+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/1286510389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb59c1017b0 0x7fb59c101c30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.216+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/1286510389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c1005c0 0x7fb59c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.216+0000 7fb5a2d05640 1 -- 172.21.15.7:0/1286510389 >> 172.21.15.7:0/1286510389 conn(0x7fb59c0fbb30 msgr2=0x7fb59c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.216+0000 7fb5a2d05640 1 -- 172.21.15.7:0/1286510389 shutdown_connections 2024-03-26T08:51:02.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.216+0000 7fb5a2d05640 1 -- 172.21.15.7:0/1286510389 wait complete. 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.216+0000 7fb5a2d05640 1 Processor -- start 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb5a2d05640 1 -- start start 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb5a2d05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c1005c0 0x7fb59c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb5a2d05640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb59c1017b0 0x7fb59c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb5a2d05640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb59c102170 0x7fb59c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb5a2d05640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c10d370 con 0x7fb59c1017b0 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb59c1017b0 0x7fb59c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb5a0aa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c1005c0 0x7fb59c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb5a2d05640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c10d1f0 con 0x7fb59c1005c0 2024-03-26T08:51:02.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb59c1017b0 0x7fb59c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52648/0 (socket says 172.21.15.7:52648) 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb59bfff640 1 -- 172.21.15.7:0/3550912964 learned_addr learned my addr 172.21.15.7:0/3550912964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb5a2d05640 1 -- 172.21.15.7:0/3550912964 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c10d4f0 con 0x7fb59c102170 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.217+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb59c102170 0x7fb59c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.218+0000 7fb59bfff640 1 -- 172.21.15.7:0/3550912964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb59c102170 msgr2=0x7fb59c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.218+0000 7fb59bfff640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb59c102170 0x7fb59c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.218+0000 7fb59bfff640 1 -- 172.21.15.7:0/3550912964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c1005c0 msgr2=0x7fb59c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.218+0000 7fb59bfff640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c1005c0 0x7fb59c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.218+0000 7fb59bfff640 1 -- 172.21.15.7:0/3550912964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c19f620 con 0x7fb59c1017b0 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.218+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb59c102170 0x7fb59c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:02.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.218+0000 7fb59bfff640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb59c1017b0 0x7fb59c197f40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb588005940 tx=0x7fb588032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.218+0000 7fb599ffb640 1 -- 172.21.15.7:0/3550912964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588041020 con 0x7fb59c1017b0 2024-03-26T08:51:02.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.219+0000 7fb599ffb640 1 -- 172.21.15.7:0/3550912964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb58802ca10 con 0x7fb59c1017b0 2024-03-26T08:51:02.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.219+0000 7fb599ffb640 1 -- 172.21.15.7:0/3550912964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588039040 con 0x7fb59c1017b0 2024-03-26T08:51:02.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.219+0000 7fb5a2d05640 1 -- 172.21.15.7:0/3550912964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c19f8b0 con 0x7fb59c1017b0 2024-03-26T08:51:02.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.219+0000 7fb5a2d05640 1 -- 172.21.15.7:0/3550912964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c19fd50 con 0x7fb59c1017b0 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb599ffb640 1 -- 172.21.15.7:0/3550912964 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb588044020 con 0x7fb59c1017b0 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a59c5740 1 Processor -- start 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a59c5740 1 -- start start 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612476ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 0x56261247bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 0x56261247e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a59c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562612439610 con 0x562612476690 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a59c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626123dc3c0 con 0x56261247c1f0 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a59c5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562612384d60 con 0x562612477080 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a12a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612476ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a12a2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612476ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52656/0 (socket says 172.21.15.7:52656) 2024-03-26T08:51:02.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a12a2640 1 -- 172.21.15.7:0/2866575283 learned_addr learned my addr 172.21.15.7:0/2866575283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb59bfff640 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 0x56261247e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.222+0000 7fb5a0aa1640 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 0x56261247bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb599ffb640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb58c0a2950 0x7fb58c0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb5a12a2640 1 -- 172.21.15.7:0/2866575283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 msgr2=0x56261247bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 0x56261247bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb5a12a2640 1 -- 172.21.15.7:0/2866575283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 msgr2=0x56261247e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 0x56261247e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb5a12a2640 1 -- 172.21.15.7:0/2866575283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626120f03f0 con 0x562612476690 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb59bfff640 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 0x56261247e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb599ffb640 1 -- 172.21.15.7:0/3550912964 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb5880f02e0 con 0x7fb59c1017b0 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb58c0a2950 0x7fb58c0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612476ab0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb59400e9c0 tx=0x7fb59400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6159cf0998e96159 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb586ffd640 1 -- 172.21.15.7:0/2866575283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594004510 con 0x562612476690 2024-03-26T08:51:02.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb586ffd640 1 -- 172.21.15.7:0/2866575283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5940046b0 con 0x562612476690 2024-03-26T08:51:02.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb586ffd640 1 -- 172.21.15.7:0/2866575283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594025440 con 0x562612476690 2024-03-26T08:51:02.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.223+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb58c0a2950 0x7fb58c0a4e10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb594022000 tx=0x7fb594023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.224+0000 7fb5a59c5740 1 -- 172.21.15.7:0/2866575283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 msgr2=0x562612476ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.224+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612476ab0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb59400e9c0 tx=0x7fb59400ee80 comp rx=0 tx=0).stop 2024-03-26T08:51:02.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.224+0000 7fb5a59c5740 1 -- 172.21.15.7:0/2866575283 shutdown_connections 2024-03-26T08:51:02.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.224+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 0x56261247e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.224+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 0x56261247bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.224+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/2866575283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612476ab0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.224+0000 7fb5a59c5740 1 -- 172.21.15.7:0/2866575283 >> 172.21.15.7:0/2866575283 conn(0x562612471c00 msgr2=0x562612472000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.224+0000 7fb5a59c5740 1 -- 172.21.15.7:0/2866575283 shutdown_connections 2024-03-26T08:51:02.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.224+0000 7fb5a59c5740 1 -- 172.21.15.7:0/2866575283 wait complete. 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.225+0000 7fb5a59c5740 1 -- start start 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.225+0000 7fb5a59c5740 1 Processor -- start 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.225+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 0x562612480a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612480fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x562612487fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a59c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562612398240 con 0x562612477080 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a12a2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x562612487fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a0aa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 0x562612480a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a12a2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x562612487fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:32778/0 (socket says 172.21.15.7:32778) 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a12a2640 1 -- 172.21.15.7:0/3761275387 learned_addr learned my addr 172.21.15.7:0/3761275387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb59bfff640 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612480fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626123d8460 con 0x562612476690 2024-03-26T08:51:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261247f1a0 con 0x56261247c1f0 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a59c5740 10 client.0 _ll_register_callbacks cb 0x562612397c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a12a2640 1 -- 172.21.15.7:0/3761275387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 msgr2=0x562612480a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 0x562612480a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a12a2640 1 -- 172.21.15.7:0/3761275387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 msgr2=0x562612480fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612480fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.226+0000 7fb5a12a2640 1 -- 172.21.15.7:0/3761275387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626121ba7d0 con 0x56261247c1f0 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.227+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x562612487fe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb59400d7b0 tx=0x7fb59401aa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c812d53ef8943eed server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.227+0000 7fb57bfff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.227+0000 7fb57bfff640 1 -- 172.21.15.7:0/3761275387 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594003d60 con 0x56261247c1f0 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.227+0000 7fb57bfff640 1 -- 172.21.15.7:0/3761275387 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb594025760 con 0x56261247c1f0 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.227+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5940046b0 con 0x56261247c1f0 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.227+0000 7fb57bfff640 1 -- 172.21.15.7:0/3761275387 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb594025960 con 0x56261247c1f0 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.227+0000 7fb5a59c5740 10 client.27505 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.227+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56261248b310 con 0x56261247c1f0 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.227+0000 7fb57bfff640 1 -- 172.21.15.7:0/3761275387 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5940046b0 con 0x56261247c1f0 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.228+0000 7fb57bfff640 1 -- 172.21.15.7:0/3761275387 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb59400b040 con 0x56261247c1f0 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.228+0000 7fb5a59c5740 10 client.27505 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.228+0000 7fb5a59c5740 10 client.27505 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.228+0000 7fb5a59c5740 20 client.27505 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.228+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56261248c850 0x56261248eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.228+0000 7fb5a59c5740 4 client.27505 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.228+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626123e9900 con 0x56261248c850 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.228+0000 7fb5a0aa1640 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56261248c850 0x56261248eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.229+0000 7fb5a0aa1640 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56261248c850 0x56261248eca0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1aa14d56bd343d1e server_cookie=121582b03c186cb9 in_seq=0 out_seq=0 2024-03-26T08:51:02.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.229+0000 7fb57bfff640 10 client.27505 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:51:02.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb57bfff640 1 -- 172.21.15.7:0/3761275387 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5626123e9900 con 0x56261248c850 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb57bfff640 10 client.27505 handle_command_reply: tid=0 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb5a59c5740 1 client.27505 shutdown 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 msgr2=0x562612487fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x562612487fe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb59400d7b0 tx=0x7fb59401aa80 comp rx=0 tx=0).stop 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 shutdown_connections 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56261248c850 0x56261248eca0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x562612487fe0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb57bfff640 0 client.27505 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612480fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3761275387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 0x562612480a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.230+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 >> 172.21.15.7:0/3761275387 conn(0x562612471c00 msgr2=0x562612470e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.231+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 shutdown_connections 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.231+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3761275387 wait complete. 2024-03-26T08:51:02.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.231+0000 7fb5a59c5740 20 client.27505 trim_cache size 0 max 16384 2024-03-26T08:51:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a59c5740 1 Processor -- start 2024-03-26T08:51:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a59c5740 1 -- start start 2024-03-26T08:51:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612477d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 0x562612487fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a0aa1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612477d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a0aa1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612477d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52676/0 (socket says 172.21.15.7:52676) 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a0aa1640 1 -- 172.21.15.7:0/186549990 learned_addr learned my addr 172.21.15.7:0/186549990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 0x562612470590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a59c5740 1 -- 172.21.15.7:0/186549990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c1a8e20 con 0x562612476690 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a59c5740 1 -- 172.21.15.7:0/186549990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c1a6920 con 0x56261247c1f0 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb5a59c5740 1 -- 172.21.15.7:0/186549990 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c1a49a0 con 0x562612477080 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.249+0000 7fb59bfff640 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 0x562612487fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 0x562612470590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5a0aa1640 1 -- 172.21.15.7:0/186549990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 msgr2=0x562612487fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5a0aa1640 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 0x562612487fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5a0aa1640 1 -- 172.21.15.7:0/186549990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 msgr2=0x562612470590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5a0aa1640 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 0x562612470590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5a0aa1640 1 -- 172.21.15.7:0/186549990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626124778e0 con 0x562612476690 2024-03-26T08:51:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5a0aa1640 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612477d50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x562612477af0 tx=0x7fb590005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c2a5543ce16f94 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5867fc640 1 -- 172.21.15.7:0/186549990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5900200f0 con 0x562612476690 2024-03-26T08:51:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5867fc640 1 -- 172.21.15.7:0/186549990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb590024030 con 0x562612476690 2024-03-26T08:51:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.250+0000 7fb5867fc640 1 -- 172.21.15.7:0/186549990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590004e40 con 0x562612476690 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.251+0000 7fb5a59c5740 1 -- 172.21.15.7:0/186549990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 msgr2=0x562612477d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.251+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612477d50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x562612477af0 tx=0x7fb590005f90 comp rx=0 tx=0).stop 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.251+0000 7fb5a59c5740 1 -- 172.21.15.7:0/186549990 shutdown_connections 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.251+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56261247c1f0 0x562612470590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.252+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562612477080 0x562612487fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.252+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/186549990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612476690 0x562612477d50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.252+0000 7fb5a59c5740 1 -- 172.21.15.7:0/186549990 >> 172.21.15.7:0/186549990 conn(0x562612471c00 msgr2=0x56261247f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.252+0000 7fb5a59c5740 1 -- 172.21.15.7:0/186549990 shutdown_connections 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.252+0000 7fb5a59c5740 1 -- 172.21.15.7:0/186549990 wait complete. 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a59c5740 1 -- start start 2024-03-26T08:51:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a59c5740 1 Processor -- start 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 0x562612485c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612486170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a59c5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x5626124870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a59c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c1a4fa0 con 0x562612477080 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a59c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c1a62c0 con 0x562612476690 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a59c5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c1a5600 con 0x56261247c1f0 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a59c5740 10 client.0 _ll_register_callbacks cb 0x7fb59400e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612486170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb59bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612486170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52688/0 (socket says 172.21.15.7:52688) 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a0aa1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 0x562612485c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb59bfff640 1 -- 172.21.15.7:0/3170890694 learned_addr learned my addr 172.21.15.7:0/3170890694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.253+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x5626124870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a0aa1640 1 -- 172.21.15.7:0/3170890694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 msgr2=0x5626124870a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a0aa1640 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x5626124870a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a0aa1640 1 -- 172.21.15.7:0/3170890694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 msgr2=0x562612486170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a0aa1640 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612486170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a0aa1640 1 -- 172.21.15.7:0/3170890694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562612487690 con 0x562612476690 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb59bfff640 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612486170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a12a2640 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x5626124870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a0aa1640 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 0x562612485c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x562612471bd0 tx=0x7fb5900122a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f65d1fe6de5b9fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb57bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb57bfff640 1 -- 172.21.15.7:0/3170890694 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59002d070 con 0x562612476690 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3170890694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb590024030 con 0x562612476690 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb57bfff640 1 -- 172.21.15.7:0/3170890694 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb5900125d0 con 0x562612476690 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb57bfff640 1 -- 172.21.15.7:0/3170890694 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb590039050 con 0x562612476690 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a59c5740 10 client.27494 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.254+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3170890694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5626124e8600 con 0x562612476690 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb57bfff640 1 -- 172.21.15.7:0/3170890694 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb590024030 con 0x562612476690 2024-03-26T08:51:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb57bfff640 1 -- 172.21.15.7:0/3170890694 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb590029890 con 0x562612476690 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb5a59c5740 10 client.27494 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb5a59c5740 10 client.27494 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb5a59c5740 20 client.27494 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56261248c850 0x5626124eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb5a59c5740 4 client.27494 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3170890694 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626123e3c50 con 0x56261248c850 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb59bfff640 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56261248c850 0x5626124eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb57bfff640 10 client.27494 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.255+0000 7fb59bfff640 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56261248c850 0x5626124eaf70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3483894a7d813b2 server_cookie=20dc587840f79fb1 in_seq=0 out_seq=0 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb57bfff640 1 -- 172.21.15.7:0/3170890694 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626123e3c50 con 0x56261248c850 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb57bfff640 10 client.27494 handle_command_reply: tid=0 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb5a59c5740 1 client.27494 shutdown 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3170890694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 msgr2=0x562612485c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 0x562612485c30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x562612471bd0 tx=0x7fb5900122a0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3170890694 shutdown_connections 2024-03-26T08:51:02.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56261248c850 0x5626124eaf70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56261247c1f0 0x5626124870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562612477080 0x562612486170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb5a59c5740 1 --2- 172.21.15.7:0/3170890694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562612476690 0x562612485c30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3170890694 >> 172.21.15.7:0/3170890694 conn(0x562612471c00 msgr2=0x562612474e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.256+0000 7fb57bfff640 0 client.27494 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:51:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.258+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3170890694 shutdown_connections 2024-03-26T08:51:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.258+0000 7fb5a59c5740 1 -- 172.21.15.7:0/3170890694 wait complete. 2024-03-26T08:51:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.258+0000 7fb5a59c5740 20 client.27494 trim_cache size 0 max 16384 2024-03-26T08:51:02.256 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.258+0000 7fb5a2d05640 1 -- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb58c0a2950 msgr2=0x7fb58c0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.258+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb58c0a2950 0x7fb58c0a4e10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb594022000 tx=0x7fb594023000 comp rx=0 tx=0).stop 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.258+0000 7fb5a2d05640 1 -- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb59c1017b0 msgr2=0x7fb59c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.258+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb59c1017b0 0x7fb59c197f40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb588005940 tx=0x7fb588032470 comp rx=0 tx=0).stop 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.259+0000 7fb5a2d05640 1 -- 172.21.15.7:0/3550912964 shutdown_connections 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.259+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb58c0a2950 0x7fb58c0a4e10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.259+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb59c102170 0x7fb59c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.259+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb59c1017b0 0x7fb59c197f40 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.259+0000 7fb5a2d05640 1 --2- 172.21.15.7:0/3550912964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c1005c0 0x7fb59c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.259+0000 7fb5a2d05640 1 -- 172.21.15.7:0/3550912964 >> 172.21.15.7:0/3550912964 conn(0x7fb59c0fbb30 msgr2=0x7fb59c104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.259+0000 7fb5a2d05640 1 -- 172.21.15.7:0/3550912964 shutdown_connections 2024-03-26T08:51:02.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.259+0000 7fb5a2d05640 1 -- 172.21.15.7:0/3550912964 wait complete. 2024-03-26T08:51:02.268 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:51:02.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:51:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.423+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1886096865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a58100f30 msgr2=0x7f5a581013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.423+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1886096865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a58100f30 0x7f5a581013b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009900 tx=0x7f5a4c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.423+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1886096865 shutdown_connections 2024-03-26T08:51:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.423+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1886096865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a581018f0 0x7f5a58108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.423+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1886096865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a58100f30 0x7f5a581013b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.423+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1886096865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a580ffd40 0x7f5a58100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.423+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1886096865 >> 172.21.15.7:0/1886096865 conn(0x7f5a580fb2b0 msgr2=0x7f5a580fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.423+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1886096865 shutdown_connections 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.423+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1886096865 wait complete. 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.424+0000 7f5a5e9e1640 1 Processor -- start 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.424+0000 7f5a5e9e1640 1 -- start start 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.424+0000 7f5a5e9e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a580ffd40 0x7f5a581979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5e9e1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a58100f30 0x7f5a58197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5e9e1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a581018f0 0x7f5a5819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5e9e1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5810b150 con 0x7f5a581018f0 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5e9e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5810afd0 con 0x7f5a580ffd40 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5e9e1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5810b2d0 con 0x7f5a58100f30 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a581018f0 0x7f5a5819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a57fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a580ffd40 0x7f5a581979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a581018f0 0x7f5a5819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52694/0 (socket says 172.21.15.7:52694) 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/1842876945 learned_addr learned my addr 172.21.15.7:0/1842876945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a577fe640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a58100f30 0x7f5a58197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a57fff640 1 -- 172.21.15.7:0/1842876945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a58100f30 msgr2=0x7f5a58197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a57fff640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a58100f30 0x7f5a58197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a57fff640 1 -- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a581018f0 msgr2=0x7f5a5819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a57fff640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a581018f0 0x7f5a5819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a57fff640 1 -- 172.21.15.7:0/1842876945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5819f5f0 con 0x7f5a580ffd40 2024-03-26T08:51:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a581018f0 0x7f5a5819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a57fff640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a580ffd40 0x7f5a581979d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800e9c0 tx=0x7f5a4800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a557fa640 1 -- 172.21.15.7:0/1842876945 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800cd80 con 0x7f5a580ffd40 2024-03-26T08:51:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a557fa640 1 -- 172.21.15.7:0/1842876945 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a48010910 con 0x7f5a580ffd40 2024-03-26T08:51:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1842876945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5819f8e0 con 0x7f5a580ffd40 2024-03-26T08:51:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.425+0000 7f5a557fa640 1 -- 172.21.15.7:0/1842876945 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48018980 con 0x7f5a580ffd40 2024-03-26T08:51:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.426+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1842876945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5819fe00 con 0x7f5a580ffd40 2024-03-26T08:51:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a615c5740 1 Processor -- start 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a615c5740 1 -- start start 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20bd690 0x555ee20bdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 0x555ee20c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 0x555ee20c5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a615c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee2080610 con 0x555ee20bd690 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a615c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee20233c0 con 0x555ee20be080 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a615c5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee1fcbd60 con 0x555ee20c31f0 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20bd690 0x555ee20bdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20bd690 0x555ee20bdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52698/0 (socket says 172.21.15.7:52698) 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/332313280 learned_addr learned my addr 172.21.15.7:0/332313280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a577fe640 1 --2- 172.21.15.7:0/332313280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 0x555ee20c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a557fa640 1 -- 172.21.15.7:0/1842876945 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5a4801f020 con 0x7f5a580ffd40 2024-03-26T08:51:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/332313280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 msgr2=0x555ee20c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/332313280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 0x555ee20c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/332313280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 msgr2=0x555ee20c2b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:51:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/332313280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 0x555ee20c2b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.430+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/332313280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ee1d373f0 con 0x555ee20bd690 2024-03-26T08:51:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.431+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/332313280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20bd690 0x555ee20bdab0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000e920 tx=0x7f5a5000a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcb6082e7046 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.431+0000 7f5a3e7fc640 1 -- 172.21.15.7:0/332313280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5001d070 con 0x555ee20bd690 2024-03-26T08:51:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.431+0000 7f5a3e7fc640 1 -- 172.21.15.7:0/332313280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a5000dbc0 con 0x555ee20bd690 2024-03-26T08:51:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.431+0000 7f5a3e7fc640 1 -- 172.21.15.7:0/332313280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50017760 con 0x555ee20bd690 2024-03-26T08:51:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.431+0000 7f5a557fa640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a400a2800 0x7f5a400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.431+0000 7f5a57fff640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a400a2800 0x7f5a400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.431+0000 7f5a557fa640 1 -- 172.21.15.7:0/1842876945 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5a480cea50 con 0x7f5a580ffd40 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 -- 172.21.15.7:0/332313280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20bd690 msgr2=0x555ee20bdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 --2- 172.21.15.7:0/332313280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20bd690 0x555ee20bdab0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000e920 tx=0x7f5a5000a610 comp rx=0 tx=0).stop 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a57fff640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a400a2800 0x7f5a400a4cc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5a480cc550 tx=0x7f5a480908d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 -- 172.21.15.7:0/332313280 shutdown_connections 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 --2- 172.21.15.7:0/332313280 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 0x555ee20c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 --2- 172.21.15.7:0/332313280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 0x555ee20c2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 --2- 172.21.15.7:0/332313280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20bd690 0x555ee20bdab0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 -- 172.21.15.7:0/332313280 >> 172.21.15.7:0/332313280 conn(0x555ee20b8c00 msgr2=0x555ee20b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 -- 172.21.15.7:0/332313280 shutdown_connections 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 -- 172.21.15.7:0/332313280 wait complete. 2024-03-26T08:51:02.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.432+0000 7f5a615c5740 1 -- start start 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a615c5740 1 Processor -- start 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20c31f0 0x555ee20d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a615c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee1fdf240 con 0x555ee20be080 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a615c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee201f460 con 0x555ee20c31f0 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:32812/0 (socket says 172.21.15.7:32812) 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a615c5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee20ca5b0 con 0x555ee20bd690 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.433+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/437314421 learned_addr learned my addr 172.21.15.7:0/437314421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20c31f0 0x555ee20d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a615c5740 10 client.0 _ll_register_callbacks cb 0x555ee1fdec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/437314421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20c31f0 msgr2=0x555ee20d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20c31f0 0x555ee20d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/437314421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 msgr2=0x555ee20cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/437314421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ee1e017d0 con 0x555ee20bd690 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a577fe640 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cbe40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000b010 tx=0x7f5a50018c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e028d9f9b364119 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a377fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a377fe640 1 -- 172.21.15.7:0/437314421 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5002e060 con 0x555ee20bd690 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a615c5740 1 -- 172.21.15.7:0/437314421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a5000dbc0 con 0x555ee20bd690 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a377fe640 1 -- 172.21.15.7:0/437314421 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5a50018890 con 0x555ee20bd690 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a377fe640 1 -- 172.21.15.7:0/437314421 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a5001d030 con 0x555ee20bd690 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a615c5740 10 client.27523 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.434+0000 7f5a615c5740 1 -- 172.21.15.7:0/437314421 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555ee20d66f0 con 0x555ee20bd690 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.435+0000 7f5a377fe640 1 -- 172.21.15.7:0/437314421 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a5000dbc0 con 0x555ee20bd690 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.435+0000 7f5a377fe640 1 -- 172.21.15.7:0/437314421 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5a50033070 con 0x555ee20bd690 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.435+0000 7f5a615c5740 10 client.27523 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:02.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.435+0000 7f5a615c5740 10 client.27523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.435+0000 7f5a615c5740 20 client.27523 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.435+0000 7f5a615c5740 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555ee20d7b50 0x555ee20d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.435+0000 7f5a615c5740 4 client.27523 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.435+0000 7f5a615c5740 1 -- 172.21.15.7:0/437314421 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ee2030900 con 0x555ee20d7b50 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.435+0000 7f5a577fe640 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555ee20d7b50 0x555ee20d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.436+0000 7f5a377fe640 10 client.27523 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.436+0000 7f5a577fe640 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555ee20d7b50 0x555ee20d9f40 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11a74d80b7057c67 server_cookie=55fec65bbb27c967 in_seq=0 out_seq=0 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.436+0000 7f5a377fe640 1 -- 172.21.15.7:0/437314421 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555ee2030900 con 0x555ee20d7b50 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.436+0000 7f5a377fe640 10 client.27523 handle_command_reply: tid=0 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.436+0000 7f5a615c5740 1 client.27523 shutdown 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.436+0000 7f5a615c5740 1 -- 172.21.15.7:0/437314421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 msgr2=0x555ee20cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.436+0000 7f5a615c5740 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cbe40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000b010 tx=0x7f5a50018c50 comp rx=0 tx=0).stop 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.439+0000 7f5a615c5740 1 -- 172.21.15.7:0/437314421 shutdown_connections 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.439+0000 7f5a615c5740 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555ee20d7b50 0x555ee20d9f40 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.439+0000 7f5a615c5740 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20c31f0 0x555ee20d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.439+0000 7f5a615c5740 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.439+0000 7f5a615c5740 1 --2- 172.21.15.7:0/437314421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cbe40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.439+0000 7f5a377fe640 0 client.27523 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.439+0000 7f5a615c5740 1 -- 172.21.15.7:0/437314421 >> 172.21.15.7:0/437314421 conn(0x555ee20b8c00 msgr2=0x555ee20b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.440+0000 7f5a615c5740 1 -- 172.21.15.7:0/437314421 shutdown_connections 2024-03-26T08:51:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.440+0000 7f5a615c5740 1 -- 172.21.15.7:0/437314421 wait complete. 2024-03-26T08:51:02.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.440+0000 7f5a615c5740 20 client.27523 trim_cache size 0 max 16384 2024-03-26T08:51:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.457+0000 7f5a615c5740 1 Processor -- start 2024-03-26T08:51:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.457+0000 7f5a615c5740 1 -- start start 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.457+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 0x555ee20d3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20c31f0 0x555ee20de6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a615c5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a581a8e00 con 0x555ee20c31f0 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a615c5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a581a6900 con 0x555ee20be080 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a615c5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a581a4980 con 0x555ee20bd690 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 0x555ee20d3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:32814/0 (socket says 172.21.15.7:32814) 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a5cf7e640 1 -- 172.21.15.7:0/1350767115 learned_addr learned my addr 172.21.15.7:0/1350767115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/1350767115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20c31f0 0x555ee20de6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a577fe640 1 -- 172.21.15.7:0/1350767115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 msgr2=0x555ee20cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a577fe640 1 --2- 172.21.15.7:0/1350767115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a577fe640 1 -- 172.21.15.7:0/1350767115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20c31f0 msgr2=0x555ee20de6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a577fe640 1 --2- 172.21.15.7:0/1350767115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20c31f0 0x555ee20de6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a577fe640 1 -- 172.21.15.7:0/1350767115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ee20ca6f0 con 0x555ee20be080 2024-03-26T08:51:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a577fe640 1 --2- 172.21.15.7:0/1350767115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 0x555ee20d3350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009230 tx=0x7f5a4c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f397f1b92aa1567a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.458+0000 7f5a3dffb640 1 -- 172.21.15.7:0/1350767115 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c004240 con 0x555ee20be080 2024-03-26T08:51:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.459+0000 7f5a3dffb640 1 -- 172.21.15.7:0/1350767115 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a4c0043e0 con 0x555ee20be080 2024-03-26T08:51:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.459+0000 7f5a3dffb640 1 -- 172.21.15.7:0/1350767115 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c004ca0 con 0x555ee20be080 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.459+0000 7f5a615c5740 1 -- 172.21.15.7:0/1350767115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 msgr2=0x555ee20d3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.459+0000 7f5a615c5740 1 --2- 172.21.15.7:0/1350767115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 0x555ee20d3350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c009230 tx=0x7f5a4c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.459+0000 7f5a615c5740 1 -- 172.21.15.7:0/1350767115 shutdown_connections 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.460+0000 7f5a615c5740 1 --2- 172.21.15.7:0/1350767115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20c31f0 0x555ee20de6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.460+0000 7f5a615c5740 1 --2- 172.21.15.7:0/1350767115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20be080 0x555ee20d3350 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.460+0000 7f5a615c5740 1 --2- 172.21.15.7:0/1350767115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20bd690 0x555ee20cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.460+0000 7f5a615c5740 1 -- 172.21.15.7:0/1350767115 >> 172.21.15.7:0/1350767115 conn(0x555ee20b8c00 msgr2=0x555ee20b8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.460+0000 7f5a615c5740 1 -- 172.21.15.7:0/1350767115 shutdown_connections 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.460+0000 7f5a615c5740 1 -- 172.21.15.7:0/1350767115 wait complete. 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.460+0000 7f5a615c5740 1 -- start start 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.460+0000 7f5a615c5740 1 Processor -- start 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20bd690 0x555ee20d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a615c5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 0x555ee20d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20bd690 0x555ee20d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a577fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52734/0 (socket says 172.21.15.7:52734) 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a577fe640 1 -- 172.21.15.7:0/3878249531 learned_addr learned my addr 172.21.15.7:0/3878249531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a5cf7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20bd690 0x555ee20d0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46698/0 (socket says 172.21.15.7:46698) 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a581a4f80 con 0x555ee20be080 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a581a62a0 con 0x555ee20bd690 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a581a55e0 con 0x555ee20c31f0 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a615c5740 10 client.0 _ll_register_callbacks cb 0x7f5a5000df00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 0x555ee20d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a577fe640 1 -- 172.21.15.7:0/3878249531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 msgr2=0x555ee20d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a577fe640 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 0x555ee20d23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a577fe640 1 -- 172.21.15.7:0/3878249531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20bd690 msgr2=0x555ee20d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a577fe640 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20bd690 0x555ee20d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a577fe640 1 -- 172.21.15.7:0/3878249531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ee20d29e0 con 0x555ee20be080 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 0x555ee20d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:51:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a577fe640 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20d14c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c036040 tx=0x7f5a4c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85e5e29757a34fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.461+0000 7f5a377fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a377fe640 1 -- 172.21.15.7:0/3878249531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c056060 con 0x555ee20be080 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a4c0043e0 con 0x555ee20be080 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a377fe640 1 -- 172.21.15.7:0/3878249531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5a4c046dc0 con 0x555ee20be080 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a377fe640 1 -- 172.21.15.7:0/3878249531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a4c051470 con 0x555ee20be080 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a615c5740 10 client.17682 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555ee2133710 con 0x555ee20be080 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a377fe640 1 -- 172.21.15.7:0/3878249531 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a4c051af0 con 0x555ee20be080 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a377fe640 1 -- 172.21.15.7:0/3878249531 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5a4c034850 con 0x555ee20be080 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a615c5740 10 client.17682 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a615c5740 10 client.17682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a615c5740 20 client.17682 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a615c5740 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555ee2133ff0 0x555ee2136410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a615c5740 4 client.17682 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f5a4c0043e0 con 0x555ee2133ff0 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.462+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555ee2133ff0 0x555ee2136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a377fe640 10 client.17682 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a5cf7e640 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555ee2133ff0 0x555ee2136410 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b68aca3dcc1e05 server_cookie=e069c8cd8058ec3b in_seq=0 out_seq=0 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a377fe640 1 -- 172.21.15.7:0/3878249531 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f5a4c0043e0 con 0x555ee2133ff0 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a377fe640 10 client.17682 handle_command_reply: tid=0 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a615c5740 1 client.17682 shutdown 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 msgr2=0x555ee20d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a615c5740 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20d14c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c036040 tx=0x7f5a4c000bc0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 shutdown_connections 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a615c5740 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555ee2133ff0 0x555ee2136410 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a615c5740 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ee20c31f0 0x555ee20d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a615c5740 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555ee20be080 0x555ee20d14c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a377fe640 0 client.17682 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a615c5740 1 --2- 172.21.15.7:0/3878249531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555ee20bd690 0x555ee20d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.463+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 >> 172.21.15.7:0/3878249531 conn(0x555ee20b8c00 msgr2=0x555ee20bbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.464+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 shutdown_connections 2024-03-26T08:51:02.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.464+0000 7f5a615c5740 1 -- 172.21.15.7:0/3878249531 wait complete. 2024-03-26T08:51:02.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.464+0000 7f5a615c5740 20 client.17682 trim_cache size 0 max 16384 2024-03-26T08:51:02.464 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a400a2800 msgr2=0x7f5a400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a400a2800 0x7f5a400a4cc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5a480cc550 tx=0x7f5a480908d0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1842876945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a580ffd40 msgr2=0x7f5a581979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a580ffd40 0x7f5a581979d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800e9c0 tx=0x7f5a4800ee80 comp rx=0 tx=0).stop 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1842876945 shutdown_connections 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a400a2800 0x7f5a400a4cc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a581018f0 0x7f5a5819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5a58100f30 0x7f5a58197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 --2- 172.21.15.7:0/1842876945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a580ffd40 0x7f5a581979d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.466+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1842876945 >> 172.21.15.7:0/1842876945 conn(0x7f5a580fb2b0 msgr2=0x7f5a581063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.467+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1842876945 shutdown_connections 2024-03-26T08:51:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:02.467+0000 7f5a5e9e1640 1 -- 172.21.15.7:0/1842876945 wait complete. 2024-03-26T08:51:02.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:02 smithi007 ceph-mon[32462]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.5 MiB/s wr, 633 op/s 2024-03-26T08:51:02.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:02 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1186296278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:51:02.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:02 smithi016 ceph-mon[35851]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.5 MiB/s wr, 633 op/s 2024-03-26T08:51:02.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:02 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1186296278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:51:02.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:02 smithi044 ceph-mon[36461]: pgmap v618: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 8.5 MiB/s wr, 633 op/s 2024-03-26T08:51:02.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:02 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1186296278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:51:03.475 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:51:03.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:51:03.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:51:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.628+0000 7fbbf9d10640 1 -- 172.21.15.7:0/1031070567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbf41028a0 msgr2=0x7fbbf4104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.628+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/1031070567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbf41028a0 0x7fbbf4104d10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009860 tx=0x7fbbe802bbb0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.639+0000 7fbbf9d10640 1 -- 172.21.15.7:0/1031070567 shutdown_connections 2024-03-26T08:51:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.639+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/1031070567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbf4105250 0x7fbbf41056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.639+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/1031070567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbf41028a0 0x7fbbf4104d10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.639+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/1031070567 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbf40fff30 0x7fbbf4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.639+0000 7fbbf9d10640 1 -- 172.21.15.7:0/1031070567 >> 172.21.15.7:0/1031070567 conn(0x7fbbf40fbb30 msgr2=0x7fbbf40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.645+0000 7fbbf9d10640 1 -- 172.21.15.7:0/1031070567 shutdown_connections 2024-03-26T08:51:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.647+0000 7fbbf9d10640 1 -- 172.21.15.7:0/1031070567 wait complete. 2024-03-26T08:51:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.647+0000 7fbbf9d10640 1 Processor -- start 2024-03-26T08:51:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.649+0000 7fbbf9d10640 1 -- start start 2024-03-26T08:51:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.649+0000 7fbbf9d10640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbf40fff30 0x7fbbf4070040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.649+0000 7fbbf9d10640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbf41028a0 0x7fbbf4070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.649+0000 7fbbf9d10640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbf4105250 0x7fbbf41ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.649+0000 7fbbf9d10640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf4113960 con 0x7fbbf40fff30 2024-03-26T08:51:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.649+0000 7fbbf9d10640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf41137e0 con 0x7fbbf4105250 2024-03-26T08:51:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.649+0000 7fbbf9d10640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf4113ae0 con 0x7fbbf41028a0 2024-03-26T08:51:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.652+0000 7fbbf37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbf40fff30 0x7fbbf4070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.652+0000 7fbbf37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbf40fff30 0x7fbbf4070040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52752/0 (socket says 172.21.15.7:52752) 2024-03-26T08:51:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.652+0000 7fbbf37fe640 1 -- 172.21.15.7:0/3542855733 learned_addr learned my addr 172.21.15.7:0/3542855733 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.653+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbf41028a0 0x7fbbf4070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.655+0000 7fbbf3fff640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbf4105250 0x7fbbf41ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.655+0000 7fbbf37fe640 1 -- 172.21.15.7:0/3542855733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbf41028a0 msgr2=0x7fbbf4070580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.655+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbf41028a0 0x7fbbf4070580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.655+0000 7fbbf37fe640 1 -- 172.21.15.7:0/3542855733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbf4105250 msgr2=0x7fbbf41ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.655+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbf4105250 0x7fbbf41ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.655+0000 7fbbf37fe640 1 -- 172.21.15.7:0/3542855733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf4070c60 con 0x7fbbf40fff30 2024-03-26T08:51:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.655+0000 7fbbf3fff640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbf4105250 0x7fbbf41ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:51:03.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.657+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbf40fff30 0x7fbbf4070040 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00efb0 tx=0x7fbbec00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.658+0000 7fbbf0ff9640 1 -- 172.21.15.7:0/3542855733 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec01b070 con 0x7fbbf40fff30 2024-03-26T08:51:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.658+0000 7fbbf0ff9640 1 -- 172.21.15.7:0/3542855733 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbec00ba50 con 0x7fbbf40fff30 2024-03-26T08:51:03.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.658+0000 7fbbf0ff9640 1 -- 172.21.15.7:0/3542855733 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec016c00 con 0x7fbbf40fff30 2024-03-26T08:51:03.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.658+0000 7fbbf9d10640 1 -- 172.21.15.7:0/3542855733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf41ae6d0 con 0x7fbbf40fff30 2024-03-26T08:51:03.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.658+0000 7fbbf9d10640 1 -- 172.21.15.7:0/3542855733 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf41aeba0 con 0x7fbbf40fff30 2024-03-26T08:51:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.662+0000 7fbbf0ff9640 1 -- 172.21.15.7:0/3542855733 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fbbec01e070 con 0x7fbbf40fff30 2024-03-26T08:51:03.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.663+0000 7fbbf0ff9640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbdc0a2a10 0x7fbbdc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.664+0000 7fbbf0ff9640 1 -- 172.21.15.7:0/3542855733 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbbec0ce940 con 0x7fbbf40fff30 2024-03-26T08:51:03.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.665+0000 7fbbfc9d9740 1 Processor -- start 2024-03-26T08:51:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.666+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbdc0a2a10 0x7fbbdc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.666+0000 7fbbfc9d9740 1 -- start start 2024-03-26T08:51:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e533324960 0x55e533324d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e533329e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 0x55e53332c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e533264c90 con 0x55e53332a4c0 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53328a070 con 0x55e533325350 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e533232c80 con 0x55e533324960 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbf2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e533329e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbdc0a2a10 0x7fbbdc0a4ed0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbbe802c1c0 tx=0x7fbbe8009490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbf2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e533329e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46728/0 (socket says 172.21.15.7:46728) 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbf2ffd640 1 -- 172.21.15.7:0/819084122 learned_addr learned my addr 172.21.15.7:0/819084122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.667+0000 7fbbf3fff640 1 --2- 172.21.15.7:0/819084122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 0x55e53332c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.668+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/819084122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e533324960 0x55e533324d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.668+0000 7fbbf2ffd640 1 -- 172.21.15.7:0/819084122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e533324960 msgr2=0x55e533324d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.668+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/819084122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e533324960 0x55e533324d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.668+0000 7fbbf2ffd640 1 -- 172.21.15.7:0/819084122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 msgr2=0x55e53332c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.668+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/819084122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 0x55e53332c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.668+0000 7fbbf2ffd640 1 -- 172.21.15.7:0/819084122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e53322ad10 con 0x55e533325350 2024-03-26T08:51:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.669+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/819084122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e533329e50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8034b60 tx=0x7fbbe8042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4032c95d0bba1455 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.673+0000 7fbbd9ffb640 1 -- 172.21.15.7:0/819084122 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8044030 con 0x55e533325350 2024-03-26T08:51:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.673+0000 7fbbd9ffb640 1 -- 172.21.15.7:0/819084122 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbe804f070 con 0x55e533325350 2024-03-26T08:51:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.673+0000 7fbbd9ffb640 1 -- 172.21.15.7:0/819084122 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8039040 con 0x55e533325350 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/819084122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 msgr2=0x55e533329e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/819084122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e533329e50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8034b60 tx=0x7fbbe8042000 comp rx=0 tx=0).stop 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/819084122 shutdown_connections 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/819084122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 0x55e53332c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/819084122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e533329e50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/819084122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e533324960 0x55e533324d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/819084122 >> 172.21.15.7:0/819084122 conn(0x55e5332ed1a0 msgr2=0x55e5333202d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/819084122 shutdown_connections 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/819084122 wait complete. 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.674+0000 7fbbfc9d9740 1 -- start start 2024-03-26T08:51:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbfc9d9740 1 Processor -- start 2024-03-26T08:51:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e53332ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e533325350 0x55e53332f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 0x55e533336300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5332460d0 con 0x55e53332a4c0 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e533286110 con 0x55e533324960 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e53332d410 con 0x55e533325350 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbfc9d9740 10 client.0 _ll_register_callbacks cb 0x55e533281e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbf37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e53332ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbf37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e53332ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46738/0 (socket says 172.21.15.7:46738) 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbf37fe640 1 -- 172.21.15.7:0/2856521326 learned_addr learned my addr 172.21.15.7:0/2856521326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.675+0000 7fbbf3fff640 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 0x55e533336300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbf37fe640 1 -- 172.21.15.7:0/2856521326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e533325350 msgr2=0x55e53332f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e533325350 0x55e53332f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbf37fe640 1 -- 172.21.15.7:0/2856521326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 msgr2=0x55e533336300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 0x55e533336300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbf37fe640 1 -- 172.21.15.7:0/2856521326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5330667e0 con 0x55e533324960 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e53332ed80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbbec097c70 tx=0x7fbbec09bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c51b30bbe3da6a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/2856521326 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec0b4060 con 0x55e533324960 2024-03-26T08:51:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/2856521326 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbbec0cc510 con 0x55e533324960 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/2856521326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbe804f070 con 0x55e533324960 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.676+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/2856521326 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbec00bc00 con 0x55e533324960 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbfc9d9740 10 client.27518 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/2856521326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e5330c6f70 con 0x55e533324960 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/2856521326 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbe804f070 con 0x55e533324960 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/2856521326 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbbec00b040 con 0x55e533324960 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbfc9d9740 10 client.27518 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbfc9d9740 10 client.27518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbfc9d9740 20 client.27518 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e53333abb0 0x55e53333d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbfc9d9740 4 client.27518 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/2856521326 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5332975b0 con 0x55e53333abb0 2024-03-26T08:51:03.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.677+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e53333abb0 0x55e53333d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.678+0000 7fbbd2ffd640 10 client.27518 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.678+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e53333abb0 0x55e53333d010 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12015acde613202 server_cookie=e4ff5bbb5dc4f215 in_seq=0 out_seq=0 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/2856521326 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e5332975b0 con 0x55e53333abb0 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbd2ffd640 10 client.27518 handle_command_reply: tid=0 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 client.27518 shutdown 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/2856521326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 msgr2=0x55e53332ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e53332ed80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbbec097c70 tx=0x7fbbec09bb10 comp rx=0 tx=0).stop 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/2856521326 shutdown_connections 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e53333abb0 0x55e53333d010 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbd2ffd640 0 client.27518 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e53332a4c0 0x55e533336300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e533325350 0x55e53332f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/2856521326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e53332ed80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/2856521326 >> 172.21.15.7:0/2856521326 conn(0x55e5332ed1a0 msgr2=0x55e53331f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/2856521326 shutdown_connections 2024-03-26T08:51:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.679+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/2856521326 wait complete. 2024-03-26T08:51:03.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.680+0000 7fbbfc9d9740 20 client.27518 trim_cache size 0 max 16384 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.699+0000 7fbbfc9d9740 1 Processor -- start 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.699+0000 7fbbfc9d9740 1 -- start start 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533324960 0x55e53331eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e5333362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 0x55e53331f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00108b0 con 0x55e533324960 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe000e3b0 con 0x55e533325350 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe000c430 con 0x55e53332a4c0 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533324960 0x55e53331eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533324960 0x55e53331eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52782/0 (socket says 172.21.15.7:52782) 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf37fe640 1 -- 172.21.15.7:0/1765123506 learned_addr learned my addr 172.21.15.7:0/1765123506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e5333362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf3fff640 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 0x55e53331f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf37fe640 1 -- 172.21.15.7:0/1765123506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 msgr2=0x55e53331f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 0x55e53331f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf37fe640 1 -- 172.21.15.7:0/1765123506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 msgr2=0x55e5333362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e5333362c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf37fe640 1 -- 172.21.15.7:0/1765123506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e532f5e170 con 0x55e533324960 2024-03-26T08:51:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf3fff640 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 0x55e53331f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:51:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533324960 0x55e53331eca0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbbec016770 tx=0x7fbbec09b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd00e3b956f3ec4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.700+0000 7fbbd97fa640 1 -- 172.21.15.7:0/1765123506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec00bda0 con 0x55e533324960 2024-03-26T08:51:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbd97fa640 1 -- 172.21.15.7:0/1765123506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbec0cc510 con 0x55e533324960 2024-03-26T08:51:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbd97fa640 1 -- 172.21.15.7:0/1765123506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec0971e0 con 0x55e533324960 2024-03-26T08:51:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/1765123506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533324960 msgr2=0x55e53331eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533324960 0x55e53331eca0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbbec016770 tx=0x7fbbec09b9a0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/1765123506 shutdown_connections 2024-03-26T08:51:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 0x55e53331f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533325350 0x55e5333362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/1765123506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533324960 0x55e53331eca0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/1765123506 >> 172.21.15.7:0/1765123506 conn(0x55e5332ed1a0 msgr2=0x55e533323da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.701+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/1765123506 shutdown_connections 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/1765123506 wait complete. 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 1 -- start start 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 1 Processor -- start 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e533336020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533325350 0x55e533334630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 0x55e533334b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe000ca30 con 0x55e533325350 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe000dd50 con 0x55e533324960 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe000d090 con 0x55e53332a4c0 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbfc9d9740 10 client.0 _ll_register_callbacks cb 0x7fbbec0975c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.702+0000 7fbbf2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533325350 0x55e533334630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.703+0000 7fbbf37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e533336020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.703+0000 7fbbf2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533325350 0x55e533334630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52786/0 (socket says 172.21.15.7:52786) 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.703+0000 7fbbf2ffd640 1 -- 172.21.15.7:0/3526785508 learned_addr learned my addr 172.21.15.7:0/3526785508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.703+0000 7fbbf3fff640 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 0x55e533334b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.703+0000 7fbbf2ffd640 1 -- 172.21.15.7:0/3526785508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 msgr2=0x55e533334b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.703+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 0x55e533334b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.703+0000 7fbbf2ffd640 1 -- 172.21.15.7:0/3526785508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 msgr2=0x55e533336020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.703+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e533336020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.703+0000 7fbbf2ffd640 1 -- 172.21.15.7:0/3526785508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5333351b0 con 0x55e533325350 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.704+0000 7fbbf2ffd640 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533325350 0x55e533334630 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009780 tx=0x7fbbe80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b80bec0d4d444a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.704+0000 7fbbd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.705+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/3526785508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8044040 con 0x55e533325350 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.705+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/3526785508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbbe805a120 con 0x55e533325350 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.705+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/3526785508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbec0cc510 con 0x55e533325350 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.705+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/3526785508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbe8039040 con 0x55e533325350 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.705+0000 7fbbfc9d9740 10 client.17700 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.705+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/3526785508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e5333973c0 con 0x55e533325350 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.705+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/3526785508 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbe804f040 con 0x55e533325350 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.706+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/3526785508 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbbe802ca00 con 0x55e533325350 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.706+0000 7fbbfc9d9740 10 client.17700 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.706+0000 7fbbfc9d9740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.706+0000 7fbbfc9d9740 20 client.17700 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.706+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e53333abb0 0x55e5333998a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.706+0000 7fbbfc9d9740 4 client.17700 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.706+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/3526785508 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7fbbec0cc510 con 0x55e53333abb0 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.706+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e53333abb0 0x55e5333998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.707+0000 7fbbf37fe640 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e53333abb0 0x55e5333998a0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b82b36b35a061d server_cookie=21bdbb4dd7b172cf in_seq=0 out_seq=0 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.707+0000 7fbbd2ffd640 10 client.17700 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbd2ffd640 1 -- 172.21.15.7:0/3526785508 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7fbbec0cc510 con 0x55e53333abb0 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbd2ffd640 10 client.17700 handle_command_reply: tid=0 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 client.17700 shutdown 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/3526785508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533325350 msgr2=0x55e533334630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533325350 0x55e533334630 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009780 tx=0x7fbbe80058b0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/3526785508 shutdown_connections 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e53333abb0 0x55e5333998a0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbd2ffd640 0 client.17700 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e53332a4c0 0x55e533334b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e533325350 0x55e533334630 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 --2- 172.21.15.7:0/3526785508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e533324960 0x55e533336020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/3526785508 >> 172.21.15.7:0/3526785508 conn(0x55e5332ed1a0 msgr2=0x55e53331eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/3526785508 shutdown_connections 2024-03-26T08:51:03.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 1 -- 172.21.15.7:0/3526785508 wait complete. 2024-03-26T08:51:03.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.708+0000 7fbbfc9d9740 20 client.17700 trim_cache size 0 max 16384 2024-03-26T08:51:03.707 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:51:03.707 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:51:03.707 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "6ccc866b-47a0-43bb-99d6-93e15115ea17", 2024-03-26T08:51:03.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:51:03.707 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 -- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbdc0a2a10 msgr2=0x7fbbdc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbdc0a2a10 0x7fbbdc0a4ed0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbbe802c1c0 tx=0x7fbbe8009490 comp rx=0 tx=0).stop 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 -- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbf40fff30 msgr2=0x7fbbf4070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbf40fff30 0x7fbbf4070040 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00efb0 tx=0x7fbbec00c800 comp rx=0 tx=0).stop 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 -- 172.21.15.7:0/3542855733 shutdown_connections 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbdc0a2a10 0x7fbbdc0a4ed0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbf4105250 0x7fbbf41ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbbf41028a0 0x7fbbf4070580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 --2- 172.21.15.7:0/3542855733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbf40fff30 0x7fbbf4070040 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.709+0000 7fbbf9d10640 1 -- 172.21.15.7:0/3542855733 >> 172.21.15.7:0/3542855733 conn(0x7fbbf40fbb30 msgr2=0x7fbbf4100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.710+0000 7fbbf9d10640 1 -- 172.21.15.7:0/3542855733 shutdown_connections 2024-03-26T08:51:03.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.710+0000 7fbbf9d10640 1 -- 172.21.15.7:0/3542855733 wait complete. 2024-03-26T08:51:03.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:51:03.719 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.873+0000 7f49209c4640 1 -- 172.21.15.7:0/976372543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 msgr2=0x7f491c106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.873+0000 7f49209c4640 1 --2- 172.21.15.7:0/976372543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 0x7f491c106e80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4904009900 tx=0x7f490402bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.873+0000 7f49209c4640 1 -- 172.21.15.7:0/976372543 shutdown_connections 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.873+0000 7f49209c4640 1 --2- 172.21.15.7:0/976372543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491c1073c0 0x7f491c109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.873+0000 7f49209c4640 1 --2- 172.21.15.7:0/976372543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 0x7f491c106e80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.873+0000 7f49209c4640 1 --2- 172.21.15.7:0/976372543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f491c102160 0x7f491c104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.873+0000 7f49209c4640 1 -- 172.21.15.7:0/976372543 >> 172.21.15.7:0/976372543 conn(0x7f491c0fbb70 msgr2=0x7f491c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.873+0000 7f49209c4640 1 -- 172.21.15.7:0/976372543 shutdown_connections 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.873+0000 7f49209c4640 1 -- 172.21.15.7:0/976372543 wait complete. 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f49209c4640 1 Processor -- start 2024-03-26T08:51:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f49209c4640 1 -- start start 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f49209c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491c102160 0x7f491c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f49209c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 0x7f491c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f49209c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f491c1073c0 0x7f491c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f49209c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c10d350 con 0x7f491c104a90 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f49209c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c10d1d0 con 0x7f491c1073c0 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f49209c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c10d4d0 con 0x7f491c102160 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f4913fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 0x7f491c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f4913fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 0x7f491c197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52798/0 (socket says 172.21.15.7:52798) 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f4913fff640 1 -- 172.21.15.7:0/863861807 learned_addr learned my addr 172.21.15.7:0/863861807 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f491a59c640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491c102160 0x7f491c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.874+0000 7f491ad9d640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f491c1073c0 0x7f491c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.875+0000 7f491ad9d640 1 -- 172.21.15.7:0/863861807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491c102160 msgr2=0x7f491c1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.875+0000 7f491ad9d640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491c102160 0x7f491c1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.875+0000 7f491ad9d640 1 -- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 msgr2=0x7f491c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.875+0000 7f491ad9d640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 0x7f491c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.875+0000 7f491ad9d640 1 -- 172.21.15.7:0/863861807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c19f610 con 0x7f491c1073c0 2024-03-26T08:51:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.875+0000 7f491a59c640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491c102160 0x7f491c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.875+0000 7f4913fff640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 0x7f491c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.875+0000 7f491ad9d640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f491c1073c0 0x7f491c19efa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f491400da10 tx=0x7f491400ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.876+0000 7f49137fe640 1 -- 172.21.15.7:0/863861807 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491400bd70 con 0x7f491c1073c0 2024-03-26T08:51:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.876+0000 7f49137fe640 1 -- 172.21.15.7:0/863861807 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4914003e50 con 0x7f491c1073c0 2024-03-26T08:51:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.876+0000 7f49209c4640 1 -- 172.21.15.7:0/863861807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c0fffb0 con 0x7f491c1073c0 2024-03-26T08:51:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.876+0000 7f49137fe640 1 -- 172.21.15.7:0/863861807 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49140106a0 con 0x7f491c1073c0 2024-03-26T08:51:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.876+0000 7f49209c4640 1 -- 172.21.15.7:0/863861807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c1004d0 con 0x7f491c1073c0 2024-03-26T08:51:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.879+0000 7f49137fe640 1 -- 172.21.15.7:0/863861807 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4914003970 con 0x7f491c1073c0 2024-03-26T08:51:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.879+0000 7f49235ac740 1 Processor -- start 2024-03-26T08:51:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f49235ac740 1 -- start start 2024-03-26T08:51:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f49137fe640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f49080a2900 0x7f49080a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f49235ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc8b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f49235ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc8b790 0x55a11bc90290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f49235ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc90900 0x55a11bc92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f49235ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11bbf03d0 con 0x55a11bc8ada0 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f49235ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11bb98ad0 con 0x55a11bc90900 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f49235ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11bbec470 con 0x55a11bc8b790 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f4913fff640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f49080a2900 0x7f49080a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f49137fe640 1 -- 172.21.15.7:0/863861807 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f49140cee00 con 0x7f491c1073c0 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f491a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc8b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f491a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc8b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52800/0 (socket says 172.21.15.7:52800) 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f491a59c640 1 -- 172.21.15.7:0/789148486 learned_addr learned my addr 172.21.15.7:0/789148486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f4913fff640 1 --2- 172.21.15.7:0/789148486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc8b790 0x55a11bc90290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.880+0000 7f491ad9d640 1 --2- 172.21.15.7:0/789148486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc90900 0x55a11bc92d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.881+0000 7f491ad9d640 1 -- 172.21.15.7:0/789148486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc8b790 msgr2=0x55a11bc90290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.881+0000 7f491ad9d640 1 --2- 172.21.15.7:0/789148486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc8b790 0x55a11bc90290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.881+0000 7f491ad9d640 1 -- 172.21.15.7:0/789148486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 msgr2=0x55a11bc8b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.881+0000 7f491ad9d640 1 --2- 172.21.15.7:0/789148486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc8b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.881+0000 7f491ad9d640 1 -- 172.21.15.7:0/789148486 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a11b9ce7d0 con 0x55a11bc90900 2024-03-26T08:51:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.881+0000 7f491ad9d640 1 --2- 172.21.15.7:0/789148486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc90900 0x55a11bc92d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f491400b630 tx=0x7f491409b360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3249189ff0f7b6e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.882+0000 7f4910ff9640 1 -- 172.21.15.7:0/789148486 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491409f8b0 con 0x55a11bc90900 2024-03-26T08:51:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.882+0000 7f4910ff9640 1 -- 172.21.15.7:0/789148486 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49140a3f00 con 0x55a11bc90900 2024-03-26T08:51:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.882+0000 7f4910ff9640 1 -- 172.21.15.7:0/789148486 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49140042e0 con 0x55a11bc90900 2024-03-26T08:51:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f4913fff640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f49080a2900 0x7f49080a4dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f490402c8e0 tx=0x7f4904009610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 -- 172.21.15.7:0/789148486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc90900 msgr2=0x55a11bc92d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 --2- 172.21.15.7:0/789148486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc90900 0x55a11bc92d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f491400b630 tx=0x7f491409b360 comp rx=0 tx=0).stop 2024-03-26T08:51:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 -- 172.21.15.7:0/789148486 shutdown_connections 2024-03-26T08:51:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 --2- 172.21.15.7:0/789148486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc90900 0x55a11bc92d90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 --2- 172.21.15.7:0/789148486 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc8b790 0x55a11bc90290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 --2- 172.21.15.7:0/789148486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc8b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 -- 172.21.15.7:0/789148486 >> 172.21.15.7:0/789148486 conn(0x55a11bc86310 msgr2=0x55a11bc86710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 -- 172.21.15.7:0/789148486 shutdown_connections 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 -- 172.21.15.7:0/789148486 wait complete. 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 -- start start 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.884+0000 7f49235ac740 1 Processor -- start 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f49235ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8ada0 0x55a11bc95230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f49235ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8b790 0x55a11bc95770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f49235ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc9c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f49235ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11bc93850 con 0x55a11bc8b790 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f49235ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11bbac060 con 0x55a11bc8ada0 2024-03-26T08:51:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f49235ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a11bc939d0 con 0x55a11bc90900 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f49235ac740 10 client.0 _ll_register_callbacks cb 0x55a11bb8e6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f491a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8ada0 0x55a11bc95230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f491ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc9c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f4913fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8b790 0x55a11bc95770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f491ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc9c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:32916/0 (socket says 172.21.15.7:32916) 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f491ad9d640 1 -- 172.21.15.7:0/3628544798 learned_addr learned my addr 172.21.15.7:0/3628544798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f4913fff640 1 -- 172.21.15.7:0/3628544798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 msgr2=0x55a11bc9c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f4913fff640 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc9c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f4913fff640 1 -- 172.21.15.7:0/3628544798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8ada0 msgr2=0x55a11bc95230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f4913fff640 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8ada0 0x55a11bc95230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f4913fff640 1 -- 172.21.15.7:0/3628544798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a11b9043f0 con 0x55a11bc8b790 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f4913fff640 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8b790 0x55a11bc95770 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f490402c7b0 tx=0x7f4904045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed4cd396e8b863d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.885+0000 7f4901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.886+0000 7f4901ffb640 1 -- 172.21.15.7:0/3628544798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904040650 con 0x55a11bc8b790 2024-03-26T08:51:03.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.886+0000 7f4901ffb640 1 -- 172.21.15.7:0/3628544798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f49040407f0 con 0x55a11bc8b790 2024-03-26T08:51:03.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.886+0000 7f49235ac740 1 -- 172.21.15.7:0/3628544798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49140a3f00 con 0x55a11bc8b790 2024-03-26T08:51:03.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.886+0000 7f4901ffb640 1 -- 172.21.15.7:0/3628544798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49040056f0 con 0x55a11bc8b790 2024-03-26T08:51:03.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.886+0000 7f49235ac740 10 client.17712 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:03.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.886+0000 7f49235ac740 1 -- 172.21.15.7:0/3628544798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a11b8c40f0 con 0x55a11bc8b790 2024-03-26T08:51:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.887+0000 7f4901ffb640 1 -- 172.21.15.7:0/3628544798 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49140a3f00 con 0x55a11bc8b790 2024-03-26T08:51:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.887+0000 7f4901ffb640 1 -- 172.21.15.7:0/3628544798 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4904057a10 con 0x55a11bc8b790 2024-03-26T08:51:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.887+0000 7f49235ac740 10 client.17712 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.887+0000 7f49235ac740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.887+0000 7f49235ac740 20 client.17712 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.887+0000 7f49235ac740 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a11bca0cc0 0x55a11bca3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.887+0000 7f49235ac740 4 client.17712 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.887+0000 7f49235ac740 1 -- 172.21.15.7:0/3628544798 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a11bbfd910 con 0x55a11bca0cc0 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.887+0000 7f491a59c640 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a11bca0cc0 0x55a11bca3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.888+0000 7f491a59c640 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a11bca0cc0 0x55a11bca3120 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6267a026f8e9ed75 server_cookie=248bad760a24b24 in_seq=0 out_seq=0 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.888+0000 7f4901ffb640 10 client.17712 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f4901ffb640 1 -- 172.21.15.7:0/3628544798 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a11bbfd910 con 0x55a11bca0cc0 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f4901ffb640 10 client.17712 handle_command_reply: tid=0 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f49235ac740 1 client.17712 shutdown 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f49235ac740 1 -- 172.21.15.7:0/3628544798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8b790 msgr2=0x55a11bc95770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f49235ac740 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8b790 0x55a11bc95770 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f490402c7b0 tx=0x7f4904045040 comp rx=0 tx=0).stop 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f49235ac740 1 -- 172.21.15.7:0/3628544798 shutdown_connections 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f49235ac740 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a11bca0cc0 0x55a11bca3120 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f4901ffb640 0 client.17712 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f49235ac740 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc9c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f49235ac740 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8b790 0x55a11bc95770 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f49235ac740 1 --2- 172.21.15.7:0/3628544798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8ada0 0x55a11bc95230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.889+0000 7f49235ac740 1 -- 172.21.15.7:0/3628544798 >> 172.21.15.7:0/3628544798 conn(0x55a11bc86310 msgr2=0x55a11bc85530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.890+0000 7f49235ac740 1 -- 172.21.15.7:0/3628544798 shutdown_connections 2024-03-26T08:51:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.890+0000 7f49235ac740 1 -- 172.21.15.7:0/3628544798 wait complete. 2024-03-26T08:51:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.890+0000 7f49235ac740 20 client.17712 trim_cache size 0 max 16384 2024-03-26T08:51:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f49235ac740 1 Processor -- start 2024-03-26T08:51:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f49235ac740 1 -- start start 2024-03-26T08:51:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f49235ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc84ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f49235ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 0x55a11bc9c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f49235ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f49235ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1a8e40 con 0x55a11bc8ada0 2024-03-26T08:51:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f49235ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1a6940 con 0x55a11bc8b790 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f49235ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1a49c0 con 0x55a11bc90900 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f491a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc84ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f491a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc84ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52804/0 (socket says 172.21.15.7:52804) 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f491a59c640 1 -- 172.21.15.7:0/2366992144 learned_addr learned my addr 172.21.15.7:0/2366992144 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f491ad9d640 1 --2- 172.21.15.7:0/2366992144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f4913fff640 1 --2- 172.21.15.7:0/2366992144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 0x55a11bc9c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f491a59c640 1 -- 172.21.15.7:0/2366992144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 msgr2=0x55a11bc851e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f491a59c640 1 --2- 172.21.15.7:0/2366992144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc851e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f491a59c640 1 -- 172.21.15.7:0/2366992144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 msgr2=0x55a11bc9c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f491a59c640 1 --2- 172.21.15.7:0/2366992144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 0x55a11bc9c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.912+0000 7f491a59c640 1 -- 172.21.15.7:0/2366992144 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a11bc8bb70 con 0x55a11bc8ada0 2024-03-26T08:51:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.913+0000 7f491a59c640 1 --2- 172.21.15.7:0/2366992144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc84ca0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55a11bc8bff0 tx=0x7f490c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdd677b41f7f4ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.913+0000 7f4903fff640 1 -- 172.21.15.7:0/2366992144 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c010d20 con 0x55a11bc8ada0 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.913+0000 7f4903fff640 1 -- 172.21.15.7:0/2366992144 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f490c005500 con 0x55a11bc8ada0 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.913+0000 7f4903fff640 1 -- 172.21.15.7:0/2366992144 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c027890 con 0x55a11bc8ada0 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.913+0000 7f49235ac740 1 -- 172.21.15.7:0/2366992144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 msgr2=0x55a11bc84ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.913+0000 7f49235ac740 1 --2- 172.21.15.7:0/2366992144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc84ca0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x55a11bc8bff0 tx=0x7f490c005f90 comp rx=0 tx=0).stop 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 -- 172.21.15.7:0/2366992144 shutdown_connections 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 --2- 172.21.15.7:0/2366992144 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc851e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 --2- 172.21.15.7:0/2366992144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 0x55a11bc9c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 --2- 172.21.15.7:0/2366992144 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc84ca0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 -- 172.21.15.7:0/2366992144 >> 172.21.15.7:0/2366992144 conn(0x55a11bc86310 msgr2=0x55a11bc8cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 -- 172.21.15.7:0/2366992144 shutdown_connections 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 -- 172.21.15.7:0/2366992144 wait complete. 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 -- start start 2024-03-26T08:51:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 Processor -- start 2024-03-26T08:51:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc9c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 0x55a11bc9c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.914+0000 7f49235ac740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc9a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f49235ac740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1a4fc0 con 0x55a11bc8ada0 2024-03-26T08:51:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f49235ac740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1a62e0 con 0x55a11bc8b790 2024-03-26T08:51:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f49235ac740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1a5620 con 0x55a11bc90900 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc9c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f49235ac740 10 client.0 _ll_register_callbacks cb 0x7f4904009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc9c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52812/0 (socket says 172.21.15.7:52812) 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491a59c640 1 -- 172.21.15.7:0/46724907 learned_addr learned my addr 172.21.15.7:0/46724907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f4913fff640 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 0x55a11bc9c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491ad9d640 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc9a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491a59c640 1 -- 172.21.15.7:0/46724907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 msgr2=0x55a11bc9a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491a59c640 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc9a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491a59c640 1 -- 172.21.15.7:0/46724907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 msgr2=0x55a11bc9c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491a59c640 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 0x55a11bc9c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491a59c640 1 -- 172.21.15.7:0/46724907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a11bc9ad40 con 0x55a11bc8ada0 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f491a59c640 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc9c020 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f490c010c90 tx=0x7f490c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7028d91ed2eb5b41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f4901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:03.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f4901ffb640 1 -- 172.21.15.7:0/46724907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c004e50 con 0x55a11bc8ada0 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f4901ffb640 1 -- 172.21.15.7:0/46724907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f490c004ff0 con 0x55a11bc8ada0 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.915+0000 7f49235ac740 1 -- 172.21.15.7:0/46724907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f490c005500 con 0x55a11bc8ada0 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.916+0000 7f4901ffb640 1 -- 172.21.15.7:0/46724907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f490c0051f0 con 0x55a11bc8ada0 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.916+0000 7f49235ac740 10 client.17718 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.916+0000 7f49235ac740 1 -- 172.21.15.7:0/46724907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a11bcfcfc0 con 0x55a11bc8ada0 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.917+0000 7f4901ffb640 1 -- 172.21.15.7:0/46724907 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f490c027b10 con 0x55a11bc8ada0 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.917+0000 7f4901ffb640 1 -- 172.21.15.7:0/46724907 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f490c0306f0 con 0x55a11bc8ada0 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.917+0000 7f49235ac740 10 client.17718 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.917+0000 7f49235ac740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.917+0000 7f49235ac740 20 client.17718 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.917+0000 7f49235ac740 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a11bca0cc0 0x55a11bcffa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.918+0000 7f49235ac740 4 client.17718 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.918+0000 7f49235ac740 1 -- 172.21.15.7:0/46724907 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f490c005500 con 0x55a11bca0cc0 2024-03-26T08:51:03.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.918+0000 7f4913fff640 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a11bca0cc0 0x55a11bcffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f4901ffb640 10 client.17718 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f4913fff640 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a11bca0cc0 0x55a11bcffa30 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=726a028dd1f524a7 server_cookie=2ef57ddcb25eeab4 in_seq=0 out_seq=0 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f4901ffb640 1 -- 172.21.15.7:0/46724907 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f490c005500 con 0x55a11bca0cc0 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f4901ffb640 10 client.17718 handle_command_reply: tid=0 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 client.17718 shutdown 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 -- 172.21.15.7:0/46724907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 msgr2=0x55a11bc9c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc9c020 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f490c010c90 tx=0x7f490c005730 comp rx=0 tx=0).stop 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 -- 172.21.15.7:0/46724907 shutdown_connections 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a11bca0cc0 0x55a11bcffa30 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a11bc90900 0x55a11bc9a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11bc8b790 0x55a11bc9c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f4901ffb640 0 client.17718 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 --2- 172.21.15.7:0/46724907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11bc8ada0 0x55a11bc9c020 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 -- 172.21.15.7:0/46724907 >> 172.21.15.7:0/46724907 conn(0x55a11bc86310 msgr2=0x55a11bc89570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 -- 172.21.15.7:0/46724907 shutdown_connections 2024-03-26T08:51:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 1 -- 172.21.15.7:0/46724907 wait complete. 2024-03-26T08:51:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.919+0000 7f49235ac740 20 client.17718 trim_cache size 0 max 16384 2024-03-26T08:51:03.919 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:51:03.919 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:51:03.919 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:51:03.919 INFO:teuthology.orchestra.run.smithi007.stdout: "6ccc866b-47a0-43bb-99d6-93e15115ea17": { 2024-03-26T08:51:03.919 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:51:03.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "6ccc866b-47a0-43bb-99d6-93e15115ea17", 2024-03-26T08:51:03.920 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:51:03.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:51:03.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:51:03.920 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 -- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f49080a2900 msgr2=0x7f49080a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f49080a2900 0x7f49080a4dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f490402c8e0 tx=0x7f4904009610 comp rx=0 tx=0).stop 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 -- 172.21.15.7:0/863861807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f491c1073c0 msgr2=0x7f491c19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f491c1073c0 0x7f491c19efa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f491400da10 tx=0x7f491400ded0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 -- 172.21.15.7:0/863861807 shutdown_connections 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f49080a2900 0x7f49080a4dc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f491c1073c0 0x7f491c19efa0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491c104a90 0x7f491c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 --2- 172.21.15.7:0/863861807 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f491c102160 0x7f491c1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 -- 172.21.15.7:0/863861807 >> 172.21.15.7:0/863861807 conn(0x7f491c0fbb70 msgr2=0x7f491c102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 -- 172.21.15.7:0/863861807 shutdown_connections 2024-03-26T08:51:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:03.920+0000 7f49209c4640 1 -- 172.21.15.7:0/863861807 wait complete. 2024-03-26T08:51:03.929 INFO:tasks.cephfs.filesystem:scrub status for tag:6ccc866b-47a0-43bb-99d6-93e15115ea17 - {'path': '/', 'tag': '6ccc866b-47a0-43bb-99d6-93e15115ea17', 'options': 'recursive,force'} 2024-03-26T08:51:04.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:04 smithi016 ceph-mon[35851]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 9.0 MiB/s wr, 692 op/s 2024-03-26T08:51:04.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:04 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:51:04.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:04 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:04.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:04 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:51:04.497 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:04 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:04.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:04 smithi007 ceph-mon[32462]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 9.0 MiB/s wr, 692 op/s 2024-03-26T08:51:04.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:04 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:51:04.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:04 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:04.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:04 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:51:04.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:04 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:04.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:04 smithi044 ceph-mon[36461]: pgmap v619: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 9.0 MiB/s wr, 692 op/s 2024-03-26T08:51:04.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:04 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:51:04.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:04 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:04.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:04 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:51:04.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:04 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.229+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2878060394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 msgr2=0x7f7ab8102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.229+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2878060394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 0x7f7ab8102d60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009900 tx=0x7f7aac02bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:05.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.229+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2878060394 shutdown_connections 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.229+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2878060394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab81032a0 0x7f7ab8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.229+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2878060394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 0x7f7ab8102d60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.229+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2878060394 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab81016f0 0x7f7ab8101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.229+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2878060394 >> 172.21.15.7:0/2878060394 conn(0x7f7ab80fcc60 msgr2=0x7f7ab80ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.229+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2878060394 shutdown_connections 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.229+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2878060394 wait complete. 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.230+0000 7f7abe0ea640 1 Processor -- start 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.230+0000 7f7abe0ea640 1 -- start start 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.230+0000 7f7abe0ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab81016f0 0x7f7ab8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.230+0000 7f7abe0ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 0x7f7ab8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.230+0000 7f7abe0ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab81032a0 0x7f7ab819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.230+0000 7f7abe0ea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab810d3c0 con 0x7f7ab81028e0 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.230+0000 7f7abe0ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab810d240 con 0x7f7ab81032a0 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.230+0000 7f7abe0ea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab810d540 con 0x7f7ab81016f0 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.230+0000 7f7ab6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 0x7f7ab8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab81016f0 0x7f7ab8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 0x7f7ab8197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52822/0 (socket says 172.21.15.7:52822) 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/2706455817 learned_addr learned my addr 172.21.15.7:0/2706455817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab7fff640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab81032a0 0x7f7ab819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/2706455817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab81016f0 msgr2=0x7f7ab8197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab81016f0 0x7f7ab8197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/2706455817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab81032a0 msgr2=0x7f7ab819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab81032a0 0x7f7ab819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/2706455817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab819f690 con 0x7f7ab81028e0 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab81016f0 0x7f7ab8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab7fff640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab81032a0 0x7f7ab819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 0x7f7ab8197fb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7aac02c240 tx=0x7f7aac005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab4ff9640 1 -- 172.21.15.7:0/2706455817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac043020 con 0x7f7ab81028e0 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab4ff9640 1 -- 172.21.15.7:0/2706455817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aac0054e0 con 0x7f7ab81028e0 2024-03-26T08:51:05.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2706455817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab819f920 con 0x7f7ab81028e0 2024-03-26T08:51:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7ab4ff9640 1 -- 172.21.15.7:0/2706455817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac003d30 con 0x7f7ab81028e0 2024-03-26T08:51:05.230 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.231+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2706455817 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab819fe70 con 0x7f7ab81028e0 2024-03-26T08:51:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.234+0000 7f7ab4ff9640 1 -- 172.21.15.7:0/2706455817 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7aac046020 con 0x7f7ab81028e0 2024-03-26T08:51:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.234+0000 7f7ac0db1740 1 Processor -- start 2024-03-26T08:51:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.234+0000 7f7ac0db1740 1 -- start start 2024-03-26T08:51:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a983cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a986d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a994260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46a8e8b10 con 0x55c46a984290 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46a891900 con 0x55c46a9838a0 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46a8e4bb0 con 0x55c46a9873f0 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a986d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a983cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a986d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52828/0 (socket says 172.21.15.7:52828) 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab77fe640 1 -- 172.21.15.7:0/1132154247 learned_addr learned my addr 172.21.15.7:0/1132154247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a983cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46826/0 (socket says 172.21.15.7:46826) 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/1132154247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a994260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab4ff9640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7aa00a28b0 0x7f7aa00a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab77fe640 1 -- 172.21.15.7:0/1132154247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 msgr2=0x55c46a994260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/1132154247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a994260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab77fe640 1 -- 172.21.15.7:0/1132154247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 msgr2=0x55c46a983cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/1132154247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a983cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab77fe640 1 -- 172.21.15.7:0/1132154247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c46a5fd210 con 0x55c46a984290 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab7fff640 1 --2- 172.21.15.7:0/1132154247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a983cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab7fff640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7aa00a28b0 0x7f7aa00a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab4ff9640 1 -- 172.21.15.7:0/2706455817 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7aac0f09d0 con 0x7f7ab81028e0 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.235+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/1132154247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a986d80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80114f0 tx=0x7f7aa80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ca5d0a241c6283 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7a9dffb640 1 -- 172.21.15.7:0/1132154247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa801e070 con 0x55c46a984290 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7a9dffb640 1 -- 172.21.15.7:0/1132154247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aa80035f0 con 0x55c46a984290 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7a9dffb640 1 -- 172.21.15.7:0/1132154247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800f800 con 0x55c46a984290 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1132154247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 msgr2=0x55c46a986d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1132154247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a986d80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80114f0 tx=0x7f7aa80119b0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7ab7fff640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7aa00a28b0 0x7f7aa00a4d70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55c46a984af0 tx=0x7f7ab0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1132154247 shutdown_connections 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1132154247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a994260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1132154247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a986d80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1132154247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a983cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.236+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1132154247 >> 172.21.15.7:0/1132154247 conn(0x55c46a97ee90 msgr2=0x55c46a97f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:05.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1132154247 shutdown_connections 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1132154247 wait complete. 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 -- start start 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 Processor -- start 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a9838a0 0x55c46a996480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a984290 0x55c46a9969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a99da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46a994c60 con 0x55c46a9838a0 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46a8a4d50 con 0x55c46a984290 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46a994de0 con 0x55c46a9873f0 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ac0db1740 10 client.0 _ll_register_callbacks cb 0x55c46a8a4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ab77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a9838a0 0x55c46a996480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ab77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a9838a0 0x55c46a996480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52834/0 (socket says 172.21.15.7:52834) 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ab77fe640 1 -- 172.21.15.7:0/2796112290 learned_addr learned my addr 172.21.15.7:0/2796112290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.237+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a984290 0x55c46a9969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ab7fff640 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a99da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ab77fe640 1 -- 172.21.15.7:0/2796112290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 msgr2=0x55c46a99da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a99da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ab77fe640 1 -- 172.21.15.7:0/2796112290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a984290 msgr2=0x55c46a9969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a984290 0x55c46a9969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ab77fe640 1 -- 172.21.15.7:0/2796112290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c46a6c9620 con 0x55c46a9838a0 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a984290 0x55c46a9969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ab7fff640 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a99da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a9838a0 0x55c46a996480 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8011eb0 tx=0x7f7aa80042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29dd387e4e18850 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7a96ffd640 1 -- 172.21.15.7:0/2796112290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8026020 con 0x55c46a9838a0 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7a96ffd640 1 -- 172.21.15.7:0/2796112290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7aa8005040 con 0x55c46a9838a0 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2796112290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7aa80035f0 con 0x55c46a9838a0 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7a96ffd640 1 -- 172.21.15.7:0/2796112290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aa801e040 con 0x55c46a9838a0 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ac0db1740 10 client.17730 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:05.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2796112290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c46a9a0d70 con 0x55c46a9838a0 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.238+0000 7f7a96ffd640 1 -- 172.21.15.7:0/2796112290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aa80035f0 con 0x55c46a9838a0 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.239+0000 7f7a96ffd640 1 -- 172.21.15.7:0/2796112290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7aa8021030 con 0x55c46a9838a0 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.239+0000 7f7ac0db1740 10 client.17730 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.239+0000 7f7ac0db1740 10 client.17730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.239+0000 7f7ac0db1740 20 client.17730 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.239+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c46a9a2190 0x55c46a9a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.239+0000 7f7ac0db1740 4 client.17730 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.239+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2796112290 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c46a8f6050 con 0x55c46a9a2190 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.239+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c46a9a2190 0x55c46a9a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.240+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c46a9a2190 0x55c46a9a4580 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a40334070ff08f server_cookie=eb7a838442ba53d5 in_seq=0 out_seq=0 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.240+0000 7f7a96ffd640 10 client.17730 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.240+0000 7f7a96ffd640 1 -- 172.21.15.7:0/2796112290 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c46a8f6050 con 0x55c46a9a2190 2024-03-26T08:51:05.240 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.240+0000 7f7a96ffd640 10 client.17730 handle_command_reply: tid=0 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.241+0000 7f7ac0db1740 1 client.17730 shutdown 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.241+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2796112290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a9838a0 msgr2=0x55c46a996480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.241+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a9838a0 0x55c46a996480 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8011eb0 tx=0x7f7aa80042b0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2796112290 shutdown_connections 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c46a9a2190 0x55c46a9a4580 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a99da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7a96ffd640 0 client.17730 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a984290 0x55c46a9969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2796112290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a9838a0 0x55c46a996480 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2796112290 >> 172.21.15.7:0/2796112290 conn(0x55c46a97ee90 msgr2=0x55c46a97e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2796112290 shutdown_connections 2024-03-26T08:51:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2796112290 wait complete. 2024-03-26T08:51:05.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.242+0000 7f7ac0db1740 20 client.17730 trim_cache size 0 max 16384 2024-03-26T08:51:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ac0db1740 1 Processor -- start 2024-03-26T08:51:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ac0db1740 1 -- start start 2024-03-26T08:51:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a9820f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a982630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab81a8e20 con 0x55c46a984290 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab81a6920 con 0x55c46a9838a0 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab81a49a0 con 0x55c46a9873f0 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ab6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ab77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a9820f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ab6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52844/0 (socket says 172.21.15.7:52844) 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/2516807197 learned_addr learned my addr 172.21.15.7:0/2516807197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.262+0000 7f7ab7fff640 1 --2- 172.21.15.7:0/2516807197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a982630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/2516807197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 msgr2=0x55c46a982630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2516807197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a982630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/2516807197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 msgr2=0x55c46a9820f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2516807197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a9820f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/2516807197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c46a9a48f0 con 0x55c46a984290 2024-03-26T08:51:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/2516807197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a9820f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/2516807197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99d9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0b9d00 tx=0x7f7aac0ee360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01e293a57d39c41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7a9d7fa640 1 -- 172.21.15.7:0/2516807197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0f6020 con 0x55c46a984290 2024-03-26T08:51:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7a9d7fa640 1 -- 172.21.15.7:0/2516807197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aac0035f0 con 0x55c46a984290 2024-03-26T08:51:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.263+0000 7f7a9d7fa640 1 -- 172.21.15.7:0/2516807197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0c1480 con 0x55c46a984290 2024-03-26T08:51:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.264+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2516807197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 msgr2=0x55c46a99d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.264+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2516807197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99d9c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0b9d00 tx=0x7f7aac0ee360 comp rx=0 tx=0).stop 2024-03-26T08:51:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.264+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2516807197 shutdown_connections 2024-03-26T08:51:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.264+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2516807197 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9873f0 0x55c46a982630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.264+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2516807197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99d9c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.264+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/2516807197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9838a0 0x55c46a9820f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.264+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2516807197 >> 172.21.15.7:0/2516807197 conn(0x55c46a97ee90 msgr2=0x55c46a97f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:05.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.265+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2516807197 shutdown_connections 2024-03-26T08:51:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.266+0000 7f7ac0db1740 1 -- 172.21.15.7:0/2516807197 wait complete. 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.266+0000 7f7ac0db1740 1 -- start start 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ac0db1740 1 Processor -- start 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9838a0 0x55c46a99a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ac0db1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9873f0 0x55c46a9fba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab81a4fa0 con 0x55c46a984290 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab81a62c0 con 0x55c46a9873f0 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ac0db1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab81a5600 con 0x55c46a9838a0 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ac0db1740 10 client.0 _ll_register_callbacks cb 0x7f7aac0f6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9873f0 0x55c46a9fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52848/0 (socket says 172.21.15.7:52848) 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/1817744921 learned_addr learned my addr 172.21.15.7:0/1817744921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9838a0 0x55c46a99a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/1817744921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9838a0 msgr2=0x55c46a99a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9838a0 0x55c46a99a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/1817744921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9873f0 msgr2=0x55c46a9fba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9873f0 0x55c46a9fba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab6ffd640 1 -- 172.21.15.7:0/1817744921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c46a99b3d0 con 0x55c46a984290 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.267+0000 7f7ab6ffd640 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99ad30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0bd030 tx=0x7f7aac005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab67944d8d029a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.268+0000 7f7a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.268+0000 7f7a96ffd640 1 -- 172.21.15.7:0/1817744921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0f6020 con 0x55c46a984290 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.268+0000 7f7a96ffd640 1 -- 172.21.15.7:0/1817744921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7aac0bb630 con 0x55c46a984290 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.268+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1817744921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c46a8f03a0 con 0x55c46a984290 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.268+0000 7f7a96ffd640 1 -- 172.21.15.7:0/1817744921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aac0d5070 con 0x55c46a984290 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.268+0000 7f7ac0db1740 10 client.17736 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.268+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1817744921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c46a9fd0d0 con 0x55c46a984290 2024-03-26T08:51:05.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.268+0000 7f7a96ffd640 1 -- 172.21.15.7:0/1817744921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7aac005690 con 0x55c46a984290 2024-03-26T08:51:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.269+0000 7f7a96ffd640 1 -- 172.21.15.7:0/1817744921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7aac0a20d0 con 0x55c46a984290 2024-03-26T08:51:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.269+0000 7f7ac0db1740 10 client.17736 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.269+0000 7f7ac0db1740 10 client.17736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.269+0000 7f7ac0db1740 20 client.17736 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.269+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c46a9fd7a0 0x55c46a9ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.269+0000 7f7ac0db1740 4 client.17736 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:51:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.269+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1817744921 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c46a8f03a0 con 0x55c46a9fd7a0 2024-03-26T08:51:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.269+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c46a9fd7a0 0x55c46a9ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.270+0000 7f7ab77fe640 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c46a9fd7a0 0x55c46a9ffbc0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15b181641abf1a7 server_cookie=b4722b54952f8a35 in_seq=0 out_seq=0 2024-03-26T08:51:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.270+0000 7f7a96ffd640 10 client.17736 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7a96ffd640 1 -- 172.21.15.7:0/1817744921 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c46a8f03a0 con 0x55c46a9fd7a0 2024-03-26T08:51:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7a96ffd640 10 client.17736 handle_command_reply: tid=0 2024-03-26T08:51:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 client.17736 shutdown 2024-03-26T08:51:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1817744921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 msgr2=0x55c46a99ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99ad30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0bd030 tx=0x7f7aac005fd0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1817744921 shutdown_connections 2024-03-26T08:51:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c46a9fd7a0 0x55c46a9ffbc0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c46a9873f0 0x55c46a9fba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c46a984290 0x55c46a99ad30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7a96ffd640 0 client.17736 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 --2- 172.21.15.7:0/1817744921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c46a9838a0 0x55c46a99a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1817744921 >> 172.21.15.7:0/1817744921 conn(0x55c46a97ee90 msgr2=0x55c46a97f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1817744921 shutdown_connections 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 1 -- 172.21.15.7:0/1817744921 wait complete. 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.348+0000 7f7ac0db1740 20 client.17736 trim_cache size 0 max 16384 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:51:05.345 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:51:05.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7aa00a28b0 msgr2=0x7f7aa00a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:05.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7aa00a28b0 0x7f7aa00a4d70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55c46a984af0 tx=0x7f7ab0004a40 comp rx=0 tx=0).stop 2024-03-26T08:51:05.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 msgr2=0x7f7ab8197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:05.346 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 0x7f7ab8197fb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7aac02c240 tx=0x7f7aac005d50 comp rx=0 tx=0).stop 2024-03-26T08:51:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2706455817 shutdown_connections 2024-03-26T08:51:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7aa00a28b0 0x7f7aa00a4d70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab81032a0 0x7f7ab819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ab81028e0 0x7f7ab8197fb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 --2- 172.21.15.7:0/2706455817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7ab81016f0 0x7f7ab8197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2706455817 >> 172.21.15.7:0/2706455817 conn(0x7f7ab80fcc60 msgr2=0x7f7ab8105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2706455817 shutdown_connections 2024-03-26T08:51:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:05.349+0000 7f7abe0ea640 1 -- 172.21.15.7:0/2706455817 wait complete. 2024-03-26T08:51:05.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:05 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:51:05.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:05 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:05.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:05 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:51:05.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:05 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:05.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:05 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:51:05.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:05 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:06.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 515.0 s, 1829.9 tps, lat 54.389 ms stddev 63.410 2024-03-26T08:51:06.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:06 smithi016 ceph-mon[35851]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.3 MiB/s wr, 657 op/s 2024-03-26T08:51:06.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:06 smithi007 ceph-mon[32462]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.3 MiB/s wr, 657 op/s 2024-03-26T08:51:06.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:06 smithi044 ceph-mon[36461]: pgmap v620: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.3 MiB/s wr, 657 op/s 2024-03-26T08:51:08.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:08 smithi016 ceph-mon[35851]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 9.1 MiB/s wr, 720 op/s 2024-03-26T08:51:08.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:08 smithi007 ceph-mon[32462]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 9.1 MiB/s wr, 720 op/s 2024-03-26T08:51:08.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:08 smithi044 ceph-mon[36461]: pgmap v621: 97 pgs: 97 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 9.1 MiB/s wr, 720 op/s 2024-03-26T08:51:10.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:10 smithi016 ceph-mon[35851]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.8 MiB/s wr, 688 op/s 2024-03-26T08:51:10.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:10 smithi007 ceph-mon[32462]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.8 MiB/s wr, 688 op/s 2024-03-26T08:51:10.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:10 smithi044 ceph-mon[36461]: pgmap v622: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 7.8 MiB/s wr, 688 op/s 2024-03-26T08:51:11.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 520.0 s, 2936.8 tps, lat 33.992 ms stddev 40.334 2024-03-26T08:51:11.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:11 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:51:11.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:11 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:51:11.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:11 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:51:12.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:12 smithi016 ceph-mon[35851]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 8.3 MiB/s wr, 757 op/s 2024-03-26T08:51:12.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:12 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:51:12.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:12 smithi007 ceph-mon[32462]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 8.3 MiB/s wr, 757 op/s 2024-03-26T08:51:12.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:12 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:51:12.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:12 smithi044 ceph-mon[36461]: pgmap v623: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 8.3 MiB/s wr, 757 op/s 2024-03-26T08:51:12.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:12 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:51:14.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:14 smithi016 ceph-mon[35851]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 9.6 MiB/s wr, 836 op/s 2024-03-26T08:51:14.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:14 smithi007 ceph-mon[32462]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 9.6 MiB/s wr, 836 op/s 2024-03-26T08:51:14.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:14 smithi044 ceph-mon[36461]: pgmap v624: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 9.6 MiB/s wr, 836 op/s 2024-03-26T08:51:16.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 525.0 s, 2147.0 tps, lat 46.940 ms stddev 53.085 2024-03-26T08:51:16.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:16 smithi016 ceph-mon[35851]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 9.4 MiB/s wr, 814 op/s 2024-03-26T08:51:16.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:16 smithi007 ceph-mon[32462]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 9.4 MiB/s wr, 814 op/s 2024-03-26T08:51:16.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:16 smithi044 ceph-mon[36461]: pgmap v625: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 9.4 MiB/s wr, 814 op/s 2024-03-26T08:51:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.478+0000 7f731a252640 1 -- 172.21.15.7:0/2695310193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140ff910 msgr2=0x7f73140ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.478+0000 7f731a252640 1 --2- 172.21.15.7:0/2695310193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140ff910 0x7f73140ffd30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f730c009ca0 tx=0x7f730c02c100 comp rx=0 tx=0).stop 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.478+0000 7f731a252640 1 -- 172.21.15.7:0/2695310193 shutdown_connections 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.478+0000 7f731a252640 1 --2- 172.21.15.7:0/2695310193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7314101040 0x7f73141014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.478+0000 7f731a252640 1 --2- 172.21.15.7:0/2695310193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73140680e0 0x7f7314100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.478+0000 7f731a252640 1 --2- 172.21.15.7:0/2695310193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140ff910 0x7f73140ffd30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.478+0000 7f731a252640 1 -- 172.21.15.7:0/2695310193 >> 172.21.15.7:0/2695310193 conn(0x7f73140fae80 msgr2=0x7f73140fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.478+0000 7f731a252640 1 -- 172.21.15.7:0/2695310193 shutdown_connections 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.478+0000 7f731a252640 1 -- 172.21.15.7:0/2695310193 wait complete. 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.479+0000 7f731a252640 1 Processor -- start 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.479+0000 7f731a252640 1 -- start start 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.479+0000 7f731a252640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140680e0 0x7f731410af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.479+0000 7f731a252640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73140ff910 0x7f73141095a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.479+0000 7f731a252640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7314101040 0x7f7314109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f731a252640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731410d3b0 con 0x7f73140ff910 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f731a252640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731410d230 con 0x7f7314101040 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f731a252640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731410d530 con 0x7f73140680e0 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73140ff910 0x7f73141095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f73137fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140680e0 0x7f731410af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f73137fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140680e0 0x7f731410af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34750/0 (socket says 172.21.15.7:34750) 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73140ff910 0x7f73141095a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54128/0 (socket says 172.21.15.7:54128) 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f73137fe640 1 -- 172.21.15.7:0/1077955372 learned_addr learned my addr 172.21.15.7:0/1077955372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f7313fff640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7314101040 0x7f7314109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f73137fe640 1 -- 172.21.15.7:0/1077955372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7314101040 msgr2=0x7f7314109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f73137fe640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7314101040 0x7f7314109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f73137fe640 1 -- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73140ff910 msgr2=0x7f73141095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f73137fe640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73140ff910 0x7f73141095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f73137fe640 1 -- 172.21.15.7:0/1077955372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731410a020 con 0x7f73140680e0 2024-03-26T08:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.480+0000 7f7312ffd640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73140ff910 0x7f73141095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.481+0000 7f73137fe640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140680e0 0x7f731410af60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f730c000f80 tx=0x7f730c035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.481+0000 7f7310ff9640 1 -- 172.21.15.7:0/1077955372 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c039040 con 0x7f73140680e0 2024-03-26T08:51:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.481+0000 7f731a252640 1 -- 172.21.15.7:0/1077955372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731410a2b0 con 0x7f73140680e0 2024-03-26T08:51:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.481+0000 7f731a252640 1 -- 172.21.15.7:0/1077955372 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73141a6480 con 0x7f73140680e0 2024-03-26T08:51:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.481+0000 7f7310ff9640 1 -- 172.21.15.7:0/1077955372 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f730c044070 con 0x7f73140680e0 2024-03-26T08:51:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.481+0000 7f7310ff9640 1 -- 172.21.15.7:0/1077955372 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730c004440 con 0x7f73140680e0 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.484+0000 7f731ce3f740 1 Processor -- start 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.484+0000 7f731ce3f740 1 -- start start 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.484+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe86d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.484+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 0x5604ffe8be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe8c480 0x5604ffe8e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f731ce3f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ffdebb10 con 0x5604ffe86920 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f731ce3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ffd94900 con 0x5604ffe87310 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f731ce3f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ffde7bb0 con 0x5604ffe8c480 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe86d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe86d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54138/0 (socket says 172.21.15.7:54138) 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7313fff640 1 -- 172.21.15.7:0/1604599033 learned_addr learned my addr 172.21.15.7:0/1604599033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7310ff9640 1 -- 172.21.15.7:0/1077955372 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f730c0045e0 con 0x7f73140680e0 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f73137fe640 1 --2- 172.21.15.7:0/1604599033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 0x5604ffe8be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7313fff640 1 -- 172.21.15.7:0/1604599033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe8c480 msgr2=0x5604ffe8e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7313fff640 1 --2- 172.21.15.7:0/1604599033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe8c480 0x5604ffe8e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7313fff640 1 -- 172.21.15.7:0/1604599033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 msgr2=0x5604ffe8be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7313fff640 1 --2- 172.21.15.7:0/1604599033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 0x5604ffe8be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7313fff640 1 -- 172.21.15.7:0/1604599033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604ffb00210 con 0x5604ffe86920 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f73137fe640 1 --2- 172.21.15.7:0/1604599033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 0x5604ffe8be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7313fff640 1 --2- 172.21.15.7:0/1604599033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe86d40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f730400b700 tx=0x7f730400bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f14feae9a76699 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f7310ff9640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72fc0a2870 0x7f72fc0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f72f9ffb640 1 -- 172.21.15.7:0/1604599033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304003dd0 con 0x5604ffe86920 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.485+0000 7f72f9ffb640 1 -- 172.21.15.7:0/1604599033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7304015070 con 0x5604ffe86920 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f72f9ffb640 1 -- 172.21.15.7:0/1604599033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73040106a0 con 0x5604ffe86920 2024-03-26T08:51:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f7310ff9640 1 -- 172.21.15.7:0/1077955372 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f730c0f0e80 con 0x7f73140680e0 2024-03-26T08:51:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f73137fe640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72fc0a2870 0x7f72fc0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f73137fe640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72fc0a2870 0x7f72fc0a4d30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f730c0ba240 tx=0x7f730c032f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f731ce3f740 1 -- 172.21.15.7:0/1604599033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 msgr2=0x5604ffe86d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:18.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f731ce3f740 1 --2- 172.21.15.7:0/1604599033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe86d40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f730400b700 tx=0x7f730400bbc0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f731ce3f740 1 -- 172.21.15.7:0/1604599033 shutdown_connections 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f731ce3f740 1 --2- 172.21.15.7:0/1604599033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe8c480 0x5604ffe8e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f731ce3f740 1 --2- 172.21.15.7:0/1604599033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 0x5604ffe8be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f731ce3f740 1 --2- 172.21.15.7:0/1604599033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe86d40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f731ce3f740 1 -- 172.21.15.7:0/1604599033 >> 172.21.15.7:0/1604599033 conn(0x5604ffe81e90 msgr2=0x5604ffe82290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f731ce3f740 1 -- 172.21.15.7:0/1604599033 shutdown_connections 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.486+0000 7f731ce3f740 1 -- 172.21.15.7:0/1604599033 wait complete. 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.487+0000 7f731ce3f740 1 -- start start 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.487+0000 7f731ce3f740 1 Processor -- start 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.487+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe90ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.487+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 0x5604ffe91220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe8c480 0x5604ffe98260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f731ce3f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ffe8f480 con 0x5604ffe86920 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f731ce3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ffda7d50 con 0x5604ffe8c480 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe90ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f731ce3f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604ffe8f600 con 0x5604ffe87310 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f731ce3f740 10 client.0 _ll_register_callbacks cb 0x5604ffda7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7313fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe8c480 0x5604ffe98260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe90ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54144/0 (socket says 172.21.15.7:54144) 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7313fff640 1 -- 172.21.15.7:0/26081641 learned_addr learned my addr 172.21.15.7:0/26081641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7312ffd640 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 0x5604ffe91220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7313fff640 1 -- 172.21.15.7:0/26081641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 msgr2=0x5604ffe91220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7313fff640 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 0x5604ffe91220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7313fff640 1 -- 172.21.15.7:0/26081641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 msgr2=0x5604ffe90ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7313fff640 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe90ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.488+0000 7f7313fff640 1 -- 172.21.15.7:0/26081641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604ffbcc620 con 0x5604ffe8c480 2024-03-26T08:51:18.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.489+0000 7f7313fff640 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe8c480 0x5604ffe98260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f730400bf50 tx=0x7f730400b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83989bf0a1eb778c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.489+0000 7f72f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.489+0000 7f72f2ffd640 1 -- 172.21.15.7:0/26081641 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304024040 con 0x5604ffe8c480 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.489+0000 7f72f2ffd640 1 -- 172.21.15.7:0/26081641 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f730401a470 con 0x5604ffe8c480 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.489+0000 7f731ce3f740 1 -- 172.21.15.7:0/26081641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7304015070 con 0x5604ffe8c480 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.489+0000 7f72f2ffd640 1 -- 172.21.15.7:0/26081641 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f730401a670 con 0x5604ffe8c480 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.489+0000 7f731ce3f740 10 client.27569 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.489+0000 7f731ce3f740 1 -- 172.21.15.7:0/26081641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5604ffe9b540 con 0x5604ffe8c480 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.490+0000 7f72f2ffd640 1 -- 172.21.15.7:0/26081641 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7304015070 con 0x5604ffe8c480 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.490+0000 7f72f2ffd640 1 -- 172.21.15.7:0/26081641 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7304029870 con 0x5604ffe8c480 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.490+0000 7f731ce3f740 10 client.27569 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.490+0000 7f731ce3f740 10 client.27569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.490+0000 7f731ce3f740 20 client.27569 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.490+0000 7f731ce3f740 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604ffe9c9f0 0x5604ffe9ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.490+0000 7f731ce3f740 4 client.27569 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.490+0000 7f731ce3f740 1 -- 172.21.15.7:0/26081641 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604ffdf9050 con 0x5604ffe9c9f0 2024-03-26T08:51:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.490+0000 7f7312ffd640 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604ffe9c9f0 0x5604ffe9ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.491+0000 7f7312ffd640 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604ffe9c9f0 0x5604ffe9ee10 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0af08b7441abb6 server_cookie=5e82436a7a2863d3 in_seq=0 out_seq=0 2024-03-26T08:51:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.491+0000 7f72f2ffd640 10 client.27569 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.491+0000 7f72f2ffd640 1 -- 172.21.15.7:0/26081641 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5604ffdf9050 con 0x5604ffe9c9f0 2024-03-26T08:51:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.491+0000 7f72f2ffd640 10 client.27569 handle_command_reply: tid=0 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.491+0000 7f731ce3f740 1 client.27569 shutdown 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 -- 172.21.15.7:0/26081641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe8c480 msgr2=0x5604ffe98260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe8c480 0x5604ffe98260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f730400bf50 tx=0x7f730400b2e0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 -- 172.21.15.7:0/26081641 shutdown_connections 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604ffe9c9f0 0x5604ffe9ee10 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe8c480 0x5604ffe98260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f72f2ffd640 0 client.27569 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 0x5604ffe91220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 --2- 172.21.15.7:0/26081641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe90ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 -- 172.21.15.7:0/26081641 >> 172.21.15.7:0/26081641 conn(0x5604ffe81e90 msgr2=0x5604ffe81020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 -- 172.21.15.7:0/26081641 shutdown_connections 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 1 -- 172.21.15.7:0/26081641 wait complete. 2024-03-26T08:51:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.492+0000 7f731ce3f740 20 client.27569 trim_cache size 0 max 16384 2024-03-26T08:51:18.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:18 smithi016 ceph-mon[35851]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 9.9 MiB/s wr, 862 op/s 2024-03-26T08:51:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.512+0000 7f731ce3f740 1 Processor -- start 2024-03-26T08:51:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.512+0000 7f731ce3f740 1 -- start start 2024-03-26T08:51:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe86920 0x5604ffe82270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 0x5604ffe98220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe8c480 0x5604ffe80790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f7313fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe86920 0x5604ffe82270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f7313fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe86920 0x5604ffe82270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53774/0 (socket says 172.21.15.7:53774) 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 0x5604ffe98220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f7312ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 0x5604ffe98220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34778/0 (socket says 172.21.15.7:34778) 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f7313fff640 1 -- 172.21.15.7:0/515330674 learned_addr learned my addr 172.21.15.7:0/515330674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f731ce3f740 1 -- 172.21.15.7:0/515330674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73141b16b0 con 0x5604ffe8c480 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.513+0000 7f731ce3f740 1 -- 172.21.15.7:0/515330674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73141af1b0 con 0x5604ffe86920 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f731ce3f740 1 -- 172.21.15.7:0/515330674 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73141ad230 con 0x5604ffe87310 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f7313fff640 1 --2- 172.21.15.7:0/515330674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe8c480 0x5604ffe80790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f7313fff640 1 -- 172.21.15.7:0/515330674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 msgr2=0x5604ffe98220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f7313fff640 1 --2- 172.21.15.7:0/515330674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 0x5604ffe98220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f7313fff640 1 -- 172.21.15.7:0/515330674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe8c480 msgr2=0x5604ffe80790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f7313fff640 1 --2- 172.21.15.7:0/515330674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe8c480 0x5604ffe80790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f7313fff640 1 -- 172.21.15.7:0/515330674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604ffe88120 con 0x5604ffe86920 2024-03-26T08:51:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f7313fff640 1 --2- 172.21.15.7:0/515330674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe86920 0x5604ffe82270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7304001eb0 tx=0x7f730400bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9329f2cf01cadf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f72f97fa640 1 -- 172.21.15.7:0/515330674 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304029de0 con 0x5604ffe86920 2024-03-26T08:51:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f72f97fa640 1 -- 172.21.15.7:0/515330674 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7304015030 con 0x5604ffe86920 2024-03-26T08:51:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.514+0000 7f72f97fa640 1 -- 172.21.15.7:0/515330674 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730401a810 con 0x5604ffe86920 2024-03-26T08:51:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 -- 172.21.15.7:0/515330674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe86920 msgr2=0x5604ffe82270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 --2- 172.21.15.7:0/515330674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe86920 0x5604ffe82270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7304001eb0 tx=0x7f730400bbc0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 -- 172.21.15.7:0/515330674 shutdown_connections 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 --2- 172.21.15.7:0/515330674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe8c480 0x5604ffe80790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 --2- 172.21.15.7:0/515330674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe87310 0x5604ffe98220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 --2- 172.21.15.7:0/515330674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe86920 0x5604ffe82270 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 -- 172.21.15.7:0/515330674 >> 172.21.15.7:0/515330674 conn(0x5604ffe81e90 msgr2=0x5604ffe8bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 -- 172.21.15.7:0/515330674 shutdown_connections 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 -- 172.21.15.7:0/515330674 wait complete. 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.515+0000 7f731ce3f740 1 -- start start 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f731ce3f740 1 Processor -- start 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe95c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 0x5604ffe961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f731ce3f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe8c480 0x5604ffe970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f731ce3f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73141ad830 con 0x5604ffe86920 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f731ce3f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73141aeb50 con 0x5604ffe87310 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f731ce3f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73141ade90 con 0x5604ffe8c480 2024-03-26T08:51:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f731ce3f740 10 client.0 _ll_register_callbacks cb 0x7f7304003d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe95c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7313fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe95c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54168/0 (socket says 172.21.15.7:54168) 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7313fff640 1 -- 172.21.15.7:0/973976161 learned_addr learned my addr 172.21.15.7:0/973976161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7312ffd640 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 0x5604ffe961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7313fff640 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe8c480 0x5604ffe970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7313fff640 1 -- 172.21.15.7:0/973976161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe8c480 msgr2=0x5604ffe970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7313fff640 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe8c480 0x5604ffe970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7313fff640 1 -- 172.21.15.7:0/973976161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 msgr2=0x5604ffe961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7313fff640 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 0x5604ffe961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.516+0000 7f7313fff640 1 -- 172.21.15.7:0/973976161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604ffe976d0 con 0x5604ffe86920 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f7313fff640 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe95c70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7304000c00 tx=0x7f73040059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9caf927aa2b7b4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f72f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f72f2ffd640 1 -- 172.21.15.7:0/973976161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400be50 con 0x5604ffe86920 2024-03-26T08:51:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f72f2ffd640 1 -- 172.21.15.7:0/973976161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f730401a470 con 0x5604ffe86920 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f731ce3f740 1 -- 172.21.15.7:0/973976161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7304015030 con 0x5604ffe86920 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f72f2ffd640 1 -- 172.21.15.7:0/973976161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f730401aa20 con 0x5604ffe86920 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f731ce3f740 10 client.17754 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f731ce3f740 1 -- 172.21.15.7:0/973976161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5604ffef8520 con 0x5604ffe86920 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f72f2ffd640 1 -- 172.21.15.7:0/973976161 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7304015030 con 0x5604ffe86920 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f72f2ffd640 1 -- 172.21.15.7:0/973976161 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7304010560 con 0x5604ffe86920 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f731ce3f740 10 client.17754 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f731ce3f740 10 client.17754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f731ce3f740 20 client.17754 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.517+0000 7f731ce3f740 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604ffef8d30 0x5604ffefb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.518+0000 7f731ce3f740 4 client.17754 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.518+0000 7f731ce3f740 1 -- 172.21.15.7:0/973976161 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5604ffdf33a0 con 0x5604ffef8d30 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.518+0000 7f7312ffd640 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604ffef8d30 0x5604ffefb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.518+0000 7f7312ffd640 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604ffef8d30 0x5604ffefb150 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cec08f304d14b56 server_cookie=cbb0dd5f626b3d88 in_seq=0 out_seq=0 2024-03-26T08:51:18.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.518+0000 7f72f2ffd640 10 client.17754 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.535+0000 7f72f2ffd640 1 -- 172.21.15.7:0/973976161 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5604ffdf33a0 con 0x5604ffef8d30 2024-03-26T08:51:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.535+0000 7f72f2ffd640 10 client.17754 handle_command_reply: tid=0 2024-03-26T08:51:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.535+0000 7f731ce3f740 1 client.17754 shutdown 2024-03-26T08:51:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.535+0000 7f731ce3f740 1 -- 172.21.15.7:0/973976161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 msgr2=0x5604ffe95c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.535+0000 7f731ce3f740 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe95c70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7304000c00 tx=0x7f73040059b0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731ce3f740 1 -- 172.21.15.7:0/973976161 shutdown_connections 2024-03-26T08:51:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731ce3f740 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604ffef8d30 0x5604ffefb150 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731ce3f740 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5604ffe8c480 0x5604ffe970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731ce3f740 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ffe87310 0x5604ffe961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731ce3f740 1 --2- 172.21.15.7:0/973976161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5604ffe86920 0x5604ffe95c70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731ce3f740 1 -- 172.21.15.7:0/973976161 >> 172.21.15.7:0/973976161 conn(0x5604ffe81e90 msgr2=0x5604ffe8b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f72f2ffd640 0 client.17754 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731ce3f740 1 -- 172.21.15.7:0/973976161 shutdown_connections 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731ce3f740 1 -- 172.21.15.7:0/973976161 wait complete. 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731ce3f740 20 client.17754 trim_cache size 0 max 16384 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:51:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:51:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 -- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72fc0a2870 msgr2=0x7f72fc0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72fc0a2870 0x7f72fc0a4d30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f730c0ba240 tx=0x7f730c032f90 comp rx=0 tx=0).stop 2024-03-26T08:51:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 -- 172.21.15.7:0/1077955372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140680e0 msgr2=0x7f731410af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140680e0 0x7f731410af60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f730c000f80 tx=0x7f730c035d10 comp rx=0 tx=0).stop 2024-03-26T08:51:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 -- 172.21.15.7:0/1077955372 shutdown_connections 2024-03-26T08:51:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f72fc0a2870 0x7f72fc0a4d30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7314101040 0x7f7314109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73140ff910 0x7f73141095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 --2- 172.21.15.7:0/1077955372 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73140680e0 0x7f731410af60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.536+0000 7f731a252640 1 -- 172.21.15.7:0/1077955372 >> 172.21.15.7:0/1077955372 conn(0x7f73140fae80 msgr2=0x7f73140fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.537+0000 7f731a252640 1 -- 172.21.15.7:0/1077955372 shutdown_connections 2024-03-26T08:51:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:18.537+0000 7f731a252640 1 -- 172.21.15.7:0/1077955372 wait complete. 2024-03-26T08:51:18.545 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:18 smithi007 ceph-mon[32462]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 9.9 MiB/s wr, 862 op/s 2024-03-26T08:51:18.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:18 smithi044 ceph-mon[36461]: pgmap v626: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 9.9 MiB/s wr, 862 op/s 2024-03-26T08:51:20.496 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:20 smithi016 ceph-mon[35851]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.2 MiB/s wr, 814 op/s 2024-03-26T08:51:20.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:20 smithi007 ceph-mon[32462]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.2 MiB/s wr, 814 op/s 2024-03-26T08:51:20.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:20 smithi044 ceph-mon[36461]: pgmap v627: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.2 MiB/s wr, 814 op/s 2024-03-26T08:51:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 530.0 s, 2546.8 tps, lat 38.914 ms stddev 44.904 2024-03-26T08:51:22.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:22 smithi007 ceph-mon[32462]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 846 op/s 2024-03-26T08:51:22.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:22 smithi044 ceph-mon[36461]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 846 op/s 2024-03-26T08:51:22.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:22 smithi016 ceph-mon[35851]: pgmap v628: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 846 op/s 2024-03-26T08:51:24.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:24 smithi007 ceph-mon[32462]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.6 MiB/s wr, 824 op/s 2024-03-26T08:51:24.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:24 smithi044 ceph-mon[36461]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.6 MiB/s wr, 824 op/s 2024-03-26T08:51:24.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:24 smithi016 ceph-mon[35851]: pgmap v629: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.6 MiB/s wr, 824 op/s 2024-03-26T08:51:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 535.0 s, 2161.6 tps, lat 46.440 ms stddev 53.473 2024-03-26T08:51:26.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:26 smithi007 ceph-mon[32462]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.9 MiB/s wr, 711 op/s 2024-03-26T08:51:26.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:26 smithi044 ceph-mon[36461]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.9 MiB/s wr, 711 op/s 2024-03-26T08:51:26.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:26 smithi016 ceph-mon[35851]: pgmap v630: 97 pgs: 97 active+clean; 1.6 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 7.9 MiB/s wr, 711 op/s 2024-03-26T08:51:28.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:28 smithi007 ceph-mon[32462]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 695 op/s 2024-03-26T08:51:28.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:28 smithi044 ceph-mon[36461]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 695 op/s 2024-03-26T08:51:28.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:28 smithi016 ceph-mon[35851]: pgmap v631: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 695 op/s 2024-03-26T08:51:30.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:30 smithi007 ceph-mon[32462]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 MiB/s wr, 623 op/s 2024-03-26T08:51:30.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:30 smithi044 ceph-mon[36461]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 MiB/s wr, 623 op/s 2024-03-26T08:51:30.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:30 smithi016 ceph-mon[35851]: pgmap v632: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.5 MiB/s wr, 623 op/s 2024-03-26T08:51:31.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 540.0 s, 1812.6 tps, lat 55.301 ms stddev 63.161 2024-03-26T08:51:31.669 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.674+0000 7f9d0aac0640 1 -- 172.21.15.7:0/3689190946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0410b010 msgr2=0x7f9d0410d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:31.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.674+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/3689190946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0410b010 0x7f9d0410d480 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc01acb0 tx=0x7f9cfc03d130 comp rx=0 tx=0).stop 2024-03-26T08:51:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.678+0000 7f9d0aac0640 1 -- 172.21.15.7:0/3689190946 shutdown_connections 2024-03-26T08:51:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.678+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/3689190946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410d9c0 0x7f9d0410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.678+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/3689190946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d0410b010 0x7f9d0410d480 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.678+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/3689190946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d041086a0 0x7f9d0410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.673 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.678+0000 7f9d0aac0640 1 -- 172.21.15.7:0/3689190946 >> 172.21.15.7:0/3689190946 conn(0x7f9d0406e6b0 msgr2=0x7f9d04070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:31.674 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.678+0000 7f9d0aac0640 1 -- 172.21.15.7:0/3689190946 shutdown_connections 2024-03-26T08:51:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.679+0000 7f9d0aac0640 1 -- 172.21.15.7:0/3689190946 wait complete. 2024-03-26T08:51:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0aac0640 1 Processor -- start 2024-03-26T08:51:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0aac0640 1 -- start start 2024-03-26T08:51:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0aac0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d041086a0 0x7f9d0419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0aac0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d0410b010 0x7f9d0419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0aac0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410d9c0 0x7f9d041c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0aac0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041119a0 con 0x7f9d041086a0 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0aac0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04111820 con 0x7f9d0410d9c0 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0aac0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04111b20 con 0x7f9d0410b010 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0905d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410d9c0 0x7f9d041c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0905d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410d9c0 0x7f9d041c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37278/0 (socket says 172.21.15.7:37278) 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.680+0000 7f9d0905d640 1 -- 172.21.15.7:0/695925501 learned_addr learned my addr 172.21.15.7:0/695925501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.681+0000 7f9d0885c640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d041086a0 0x7f9d0419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.681+0000 7f9d0905d640 1 -- 172.21.15.7:0/695925501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d0410b010 msgr2=0x7f9d0419e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.681+0000 7f9d0905d640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d0410b010 0x7f9d0419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.681+0000 7f9d0905d640 1 -- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d041086a0 msgr2=0x7f9d0419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.681+0000 7f9d0905d640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d041086a0 0x7f9d0419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.681+0000 7f9d0905d640 1 -- 172.21.15.7:0/695925501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d041c7c60 con 0x7f9d0410d9c0 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.681+0000 7f9d0885c640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d041086a0 0x7f9d0419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:31.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.682+0000 7f9d0905d640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410d9c0 0x7f9d041c75c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400d750 tx=0x7f9cf400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.682+0000 7f9d01ffb640 1 -- 172.21.15.7:0/695925501 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4017070 con 0x7f9d0410d9c0 2024-03-26T08:51:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.682+0000 7f9d01ffb640 1 -- 172.21.15.7:0/695925501 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cf4005500 con 0x7f9d0410d9c0 2024-03-26T08:51:31.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.682+0000 7f9d01ffb640 1 -- 172.21.15.7:0/695925501 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004d90 con 0x7f9d0410d9c0 2024-03-26T08:51:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.682+0000 7f9d0aac0640 1 -- 172.21.15.7:0/695925501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d041c7f50 con 0x7f9d0410d9c0 2024-03-26T08:51:31.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.683+0000 7f9d0aac0640 1 -- 172.21.15.7:0/695925501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d041c8420 con 0x7f9d0410d9c0 2024-03-26T08:51:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.687+0000 7f9d01ffb640 1 -- 172.21.15.7:0/695925501 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9cf4004f30 con 0x7f9d0410d9c0 2024-03-26T08:51:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.687+0000 7f9d0d786740 1 Processor -- start 2024-03-26T08:51:31.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.687+0000 7f9d0d786740 1 -- start start 2024-03-26T08:51:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.687+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.687+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119bb310 0x5629119bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.687+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.687+0000 7f9d0d786740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291191fb10 con 0x5629119ba920 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.687+0000 7f9d0d786740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629118c8900 con 0x5629119bb310 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.687+0000 7f9d0d786740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56291191bbb0 con 0x5629119c0480 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119bad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40940/0 (socket says 172.21.15.7:40940) 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 -- 172.21.15.7:0/2146489219 learned_addr learned my addr 172.21.15.7:0/2146489219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d01ffb640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cec0a2980 0x7f9cec0a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 --2- 172.21.15.7:0/2146489219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d0885c640 1 --2- 172.21.15.7:0/2146489219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119bb310 0x5629119bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d0905d640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cec0a2980 0x7f9cec0a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d01ffb640 1 -- 172.21.15.7:0/695925501 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cf40ce330 con 0x7f9d0410d9c0 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 -- 172.21.15.7:0/2146489219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 msgr2=0x5629119c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 --2- 172.21.15.7:0/2146489219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 -- 172.21.15.7:0/2146489219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119bb310 msgr2=0x5629119bfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:31.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 --2- 172.21.15.7:0/2146489219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119bb310 0x5629119bfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 -- 172.21.15.7:0/2146489219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562911634210 con 0x5629119ba920 2024-03-26T08:51:31.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9d03fff640 1 --2- 172.21.15.7:0/2146489219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119bad40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc005e30 tx=0x7f9cfc03d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8002928751bbbedf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.688+0000 7f9ceaffd640 1 -- 172.21.15.7:0/2146489219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc043800 con 0x5629119ba920 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.689+0000 7f9ceaffd640 1 -- 172.21.15.7:0/2146489219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cfc016070 con 0x5629119ba920 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.689+0000 7f9d0905d640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cec0a2980 0x7f9cec0a4e40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40047e0 tx=0x7f9cf409d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.689+0000 7f9ceaffd640 1 -- 172.21.15.7:0/2146489219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc0042a0 con 0x5629119ba920 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 -- 172.21.15.7:0/2146489219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 msgr2=0x5629119bad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2146489219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119bad40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc005e30 tx=0x7f9cfc03d650 comp rx=0 tx=0).stop 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 -- 172.21.15.7:0/2146489219 shutdown_connections 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2146489219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2146489219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119bb310 0x5629119bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2146489219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119bad40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 -- 172.21.15.7:0/2146489219 >> 172.21.15.7:0/2146489219 conn(0x5629119b5e90 msgr2=0x5629119b6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 -- 172.21.15.7:0/2146489219 shutdown_connections 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 -- 172.21.15.7:0/2146489219 wait complete. 2024-03-26T08:51:31.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 -- start start 2024-03-26T08:51:31.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.690+0000 7f9d0d786740 1 Processor -- start 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.691+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 0x5629119c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.691+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.691+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.691+0000 7f9d0885c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.691+0000 7f9d0885c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119c51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40956/0 (socket says 172.21.15.7:40956) 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.691+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 0x5629119c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.691+0000 7f9d03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 0x5629119c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37284/0 (socket says 172.21.15.7:37284) 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.691+0000 7f9d0885c640 1 -- 172.21.15.7:0/2908996426 learned_addr learned my addr 172.21.15.7:0/2908996426 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629119c3470 con 0x5629119bb310 2024-03-26T08:51:31.689 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629118dbd50 con 0x5629119ba920 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629119c35f0 con 0x5629119c0480 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0d786740 10 client.0 _ll_register_callbacks cb 0x5629118db720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d03fff640 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0885c640 1 -- 172.21.15.7:0/2908996426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 msgr2=0x5629119cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0885c640 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0885c640 1 -- 172.21.15.7:0/2908996426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 msgr2=0x5629119c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0885c640 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 0x5629119c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0885c640 1 -- 172.21.15.7:0/2908996426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562911700620 con 0x5629119bb310 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d03fff640 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.692+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cfc016070 con 0x5629119bb310 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9d0885c640 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119c51c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8012ab0 tx=0x7f9cf8012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6acc52116d396b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9ce3fff640 10 client.17766 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9ce3fff640 1 -- 172.21.15.7:0/2908996426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8010da0 con 0x5629119bb310 2024-03-26T08:51:31.690 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9ce3fff640 1 -- 172.21.15.7:0/2908996426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cf8004020 con 0x5629119bb310 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9ce3fff640 1 -- 172.21.15.7:0/2908996426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf8004260 con 0x5629119bb310 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9d0d786740 10 client.17766 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x5629119cf320 con 0x5629119bb310 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9ce3fff640 1 -- 172.21.15.7:0/2908996426 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9cf8005020 con 0x5629119bb310 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9d0d786740 10 client.17766 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9d0d786740 10 client.17766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9d0d786740 20 client.17766 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.693+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5629119d08e0 0x5629119d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.694+0000 7f9d0d786740 4 client.17766 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.694+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56291192d050 con 0x5629119d08e0 2024-03-26T08:51:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.694+0000 7f9d03fff640 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5629119d08e0 0x5629119d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.694+0000 7f9d03fff640 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5629119d08e0 0x5629119d2d70 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e412d8269211eab6 server_cookie=a6ba02816110fe7 in_seq=0 out_seq=0 2024-03-26T08:51:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.694+0000 7f9ce3fff640 10 client.17766 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9ce3fff640 1 -- 172.21.15.7:0/2908996426 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56291192d050 con 0x5629119d08e0 2024-03-26T08:51:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9ce3fff640 10 client.17766 handle_command_reply: tid=0 2024-03-26T08:51:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 client.17766 shutdown 2024-03-26T08:51:31.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 msgr2=0x5629119c51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119c51c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8012ab0 tx=0x7f9cf8012f70 comp rx=0 tx=0).stop 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 shutdown_connections 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5629119d08e0 0x5629119d2d70 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119c51c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 --2- 172.21.15.7:0/2908996426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 0x5629119c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9ce3fff640 0 client.17766 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 >> 172.21.15.7:0/2908996426 conn(0x5629119b5e90 msgr2=0x5629119b5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 shutdown_connections 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 1 -- 172.21.15.7:0/2908996426 wait complete. 2024-03-26T08:51:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.695+0000 7f9d0d786740 20 client.17766 trim_cache size 0 max 16384 2024-03-26T08:51:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.717+0000 7f9d0d786740 1 Processor -- start 2024-03-26T08:51:31.713 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.717+0000 7f9d0d786740 1 -- start start 2024-03-26T08:51:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119bb310 0x5629119cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119c0480 0x5629119c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d0d786740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041d3650 con 0x5629119ba920 2024-03-26T08:51:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d0d786740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041d1150 con 0x5629119c0480 2024-03-26T08:51:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d0d786740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041cf1d0 con 0x5629119bb310 2024-03-26T08:51:31.714 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d0885c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119bb310 0x5629119cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d0885c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119bb310 0x5629119cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49400/0 (socket says 172.21.15.7:49400) 2024-03-26T08:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d0885c640 1 -- 172.21.15.7:0/1427359944 learned_addr learned my addr 172.21.15.7:0/1427359944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d03fff640 1 --2- 172.21.15.7:0/1427359944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119c0480 0x5629119c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.718+0000 7f9d03fff640 1 --2- 172.21.15.7:0/1427359944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.719+0000 7f9d0885c640 1 -- 172.21.15.7:0/1427359944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119c0480 msgr2=0x5629119c51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.719+0000 7f9d0885c640 1 --2- 172.21.15.7:0/1427359944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119c0480 0x5629119c51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.719+0000 7f9d0885c640 1 -- 172.21.15.7:0/1427359944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 msgr2=0x5629119c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.719+0000 7f9d0885c640 1 --2- 172.21.15.7:0/1427359944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.719+0000 7f9d0885c640 1 -- 172.21.15.7:0/1427359944 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629119bf8a0 con 0x5629119bb310 2024-03-26T08:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.719+0000 7f9d0885c640 1 --2- 172.21.15.7:0/1427359944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119bb310 0x5629119cc1c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8001eb0 tx=0x7f9cf8015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66383f0154816a62 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.719+0000 7f9cea7fc640 1 -- 172.21.15.7:0/1427359944 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8000e60 con 0x5629119bb310 2024-03-26T08:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.719+0000 7f9cea7fc640 1 -- 172.21.15.7:0/1427359944 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cfc016070 con 0x5629119bb310 2024-03-26T08:51:31.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.719+0000 7f9cea7fc640 1 -- 172.21.15.7:0/1427359944 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8024db0 con 0x5629119bb310 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.720+0000 7f9d0d786740 1 -- 172.21.15.7:0/1427359944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119bb310 msgr2=0x5629119cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.720+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1427359944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119bb310 0x5629119cc1c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8001eb0 tx=0x7f9cf8015000 comp rx=0 tx=0).stop 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.720+0000 7f9d0d786740 1 -- 172.21.15.7:0/1427359944 shutdown_connections 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.720+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1427359944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119c0480 0x5629119c51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.720+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1427359944 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119bb310 0x5629119cc1c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.720+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1427359944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119ba920 0x5629119c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.720+0000 7f9d0d786740 1 -- 172.21.15.7:0/1427359944 >> 172.21.15.7:0/1427359944 conn(0x5629119b5e90 msgr2=0x5629119bc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.720+0000 7f9d0d786740 1 -- 172.21.15.7:0/1427359944 shutdown_connections 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.720+0000 7f9d0d786740 1 -- 172.21.15.7:0/1427359944 wait complete. 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0d786740 1 -- start start 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0d786740 1 Processor -- start 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 0x5629119c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0d786740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0d786740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041cf7d0 con 0x5629119bb310 2024-03-26T08:51:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0d786740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041d0af0 con 0x5629119ba920 2024-03-26T08:51:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0d786740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041cfe30 con 0x5629119c0480 2024-03-26T08:51:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0d786740 10 client.0 _ll_register_callbacks cb 0x7f9cf80126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.721+0000 7f9d0885c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d0885c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40970/0 (socket says 172.21.15.7:40970) 2024-03-26T08:51:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d0885c640 1 -- 172.21.15.7:0/1805545533 learned_addr learned my addr 172.21.15.7:0/1805545533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d03fff640 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d03fff640 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 0x5629119c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d0885c640 1 -- 172.21.15.7:0/1805545533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 msgr2=0x5629119cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d0885c640 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d0885c640 1 -- 172.21.15.7:0/1805545533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 msgr2=0x5629119c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d0885c640 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 0x5629119c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d0885c640 1 -- 172.21.15.7:0/1805545533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629119cb690 con 0x5629119bb310 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9d0885c640 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119ca0e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80058c0 tx=0x7f9cf8005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7410fcdbbe4f6ae5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.722+0000 7f9ce3fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9ce3fff640 1 -- 172.21.15.7:0/1805545533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8010da0 con 0x5629119bb310 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9d0d786740 1 -- 172.21.15.7:0/1805545533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cfc016070 con 0x5629119bb310 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9ce3fff640 1 -- 172.21.15.7:0/1805545533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9cf8024cc0 con 0x5629119bb310 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9ce3fff640 1 -- 172.21.15.7:0/1805545533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf8040050 con 0x5629119bb310 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9d0d786740 10 client.17772 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9d0d786740 1 -- 172.21.15.7:0/1805545533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562911a2c500 con 0x5629119bb310 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9ce3fff640 1 -- 172.21.15.7:0/1805545533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cfc016070 con 0x5629119bb310 2024-03-26T08:51:31.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9ce3fff640 1 -- 172.21.15.7:0/1805545533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9cf802e3f0 con 0x5629119bb310 2024-03-26T08:51:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9d0d786740 10 client.17772 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9d0d786740 10 client.17772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9d0d786740 20 client.17772 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562911a2ce80 0x562911a2f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9d0d786740 4 client.17772 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:51:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.723+0000 7f9d0d786740 1 -- 172.21.15.7:0/1805545533 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629119273a0 con 0x562911a2ce80 2024-03-26T08:51:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.724+0000 7f9d03fff640 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562911a2ce80 0x562911a2f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.725+0000 7f9d03fff640 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562911a2ce80 0x562911a2f270 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c9f3fc863e87aa server_cookie=99a9245f7883cb44 in_seq=0 out_seq=0 2024-03-26T08:51:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.725+0000 7f9ce3fff640 10 client.17772 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.747+0000 7f9ce3fff640 1 -- 172.21.15.7:0/1805545533 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629119273a0 con 0x562911a2ce80 2024-03-26T08:51:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9ce3fff640 10 client.17772 handle_command_reply: tid=0 2024-03-26T08:51:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9d0d786740 1 client.17772 shutdown 2024-03-26T08:51:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9d0d786740 1 -- 172.21.15.7:0/1805545533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 msgr2=0x5629119ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119ca0e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80058c0 tx=0x7f9cf8005760 comp rx=0 tx=0).stop 2024-03-26T08:51:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9d0d786740 1 -- 172.21.15.7:0/1805545533 shutdown_connections 2024-03-26T08:51:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562911a2ce80 0x562911a2f270 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5629119c0480 0x5629119cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5629119bb310 0x5629119ca0e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9d0d786740 1 --2- 172.21.15.7:0/1805545533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5629119ba920 0x5629119c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9d0d786740 1 -- 172.21.15.7:0/1805545533 >> 172.21.15.7:0/1805545533 conn(0x5629119b5e90 msgr2=0x5629119bc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.748+0000 7f9ce3fff640 0 client.17772 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.749+0000 7f9d0d786740 1 -- 172.21.15.7:0/1805545533 shutdown_connections 2024-03-26T08:51:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.749+0000 7f9d0d786740 1 -- 172.21.15.7:0/1805545533 wait complete. 2024-03-26T08:51:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.750+0000 7f9d0d786740 20 client.17772 trim_cache size 0 max 16384 2024-03-26T08:51:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:51:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:51:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:51:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:51:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.750+0000 7f9d0aac0640 1 -- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cec0a2980 msgr2=0x7f9cec0a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.750+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cec0a2980 0x7f9cec0a4e40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40047e0 tx=0x7f9cf409d040 comp rx=0 tx=0).stop 2024-03-26T08:51:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.750+0000 7f9d0aac0640 1 -- 172.21.15.7:0/695925501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410d9c0 msgr2=0x7f9d041c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.750+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410d9c0 0x7f9d041c75c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400d750 tx=0x7f9cf400dc10 comp rx=0 tx=0).stop 2024-03-26T08:51:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.751+0000 7f9d0aac0640 1 -- 172.21.15.7:0/695925501 shutdown_connections 2024-03-26T08:51:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.751+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9cec0a2980 0x7f9cec0a4e40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.751+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d0410d9c0 0x7f9d041c75c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.751+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9d0410b010 0x7f9d0419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.751+0000 7f9d0aac0640 1 --2- 172.21.15.7:0/695925501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d041086a0 0x7f9d0419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.751+0000 7f9d0aac0640 1 -- 172.21.15.7:0/695925501 >> 172.21.15.7:0/695925501 conn(0x7f9d0406e6b0 msgr2=0x7f9d0410b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.752+0000 7f9d0aac0640 1 -- 172.21.15.7:0/695925501 shutdown_connections 2024-03-26T08:51:31.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:31.752+0000 7f9d0aac0640 1 -- 172.21.15.7:0/695925501 wait complete. 2024-03-26T08:51:32.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:32 smithi007 ceph-mon[32462]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 644 op/s 2024-03-26T08:51:32.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:32 smithi044 ceph-mon[36461]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 644 op/s 2024-03-26T08:51:32.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:32 smithi016 ceph-mon[35851]: pgmap v633: 97 pgs: 97 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 644 op/s 2024-03-26T08:51:33.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:51:33.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:51:34.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.092+0000 7fc7f06a0640 1 -- 172.21.15.7:0/775877804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8107ce0 msgr2=0x7fc7e8108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.092+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/775877804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8107ce0 0x7fc7e8108100 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009900 tx=0x7fc7d802bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.093+0000 7fc7f06a0640 1 -- 172.21.15.7:0/775877804 shutdown_connections 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.093+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/775877804 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e8109890 0x7fc7e8110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.093+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/775877804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8108ed0 0x7fc7e8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.093+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/775877804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8107ce0 0x7fc7e8108100 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.093+0000 7fc7f06a0640 1 -- 172.21.15.7:0/775877804 >> 172.21.15.7:0/775877804 conn(0x7fc7e8076c90 msgr2=0x7fc7e80790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.093+0000 7fc7f06a0640 1 -- 172.21.15.7:0/775877804 shutdown_connections 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.093+0000 7fc7f06a0640 1 -- 172.21.15.7:0/775877804 wait complete. 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.093+0000 7fc7f06a0640 1 Processor -- start 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.093+0000 7fc7f06a0640 1 -- start start 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7f06a0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e8107ce0 0x7fc7e807ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7f06a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8108ed0 0x7fc7e807f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7f06a0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8109890 0x7fc7e806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7f06a0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8113930 con 0x7fc7e8109890 2024-03-26T08:51:34.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7f06a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81137b0 con 0x7fc7e8108ed0 2024-03-26T08:51:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7ee43c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e8107ce0 0x7fc7e807ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7eec3d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8109890 0x7fc7e806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8108ed0 0x7fc7e807f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7f06a0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8113ab0 con 0x7fc7e8107ce0 2024-03-26T08:51:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8108ed0 0x7fc7e807f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37328/0 (socket says 172.21.15.7:37328) 2024-03-26T08:51:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7eec3d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8109890 0x7fc7e806f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40982/0 (socket says 172.21.15.7:40982) 2024-03-26T08:51:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7edc3b640 1 -- 172.21.15.7:0/3883893562 learned_addr learned my addr 172.21.15.7:0/3883893562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.094+0000 7fc7ee43c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e8107ce0 0x7fc7e807ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49432/0 (socket says 172.21.15.7:49432) 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.095+0000 7fc7eec3d640 1 -- 172.21.15.7:0/3883893562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e8107ce0 msgr2=0x7fc7e807ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.095+0000 7fc7eec3d640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e8107ce0 0x7fc7e807ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.095+0000 7fc7eec3d640 1 -- 172.21.15.7:0/3883893562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8108ed0 msgr2=0x7fc7e807f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.095+0000 7fc7eec3d640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8108ed0 0x7fc7e807f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.095+0000 7fc7eec3d640 1 -- 172.21.15.7:0/3883893562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e806f620 con 0x7fc7e8109890 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.096+0000 7fc7eec3d640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8109890 0x7fc7e806f0e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400e970 tx=0x7fc7e400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.096+0000 7fc7df7fe640 1 -- 172.21.15.7:0/3883893562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400ccc0 con 0x7fc7e8109890 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.096+0000 7fc7df7fe640 1 -- 172.21.15.7:0/3883893562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7e4005020 con 0x7fc7e8109890 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.096+0000 7fc7df7fe640 1 -- 172.21.15.7:0/3883893562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4004a00 con 0x7fc7e8109890 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.096+0000 7fc7f06a0640 1 -- 172.21.15.7:0/3883893562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e806f910 con 0x7fc7e8109890 2024-03-26T08:51:34.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.096+0000 7fc7f06a0640 1 -- 172.21.15.7:0/3883893562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e806fe10 con 0x7fc7e8109890 2024-03-26T08:51:34.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.099+0000 7fc7df7fe640 1 -- 172.21.15.7:0/3883893562 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc7e40051d0 con 0x7fc7e8109890 2024-03-26T08:51:34.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.099+0000 7fc7f3285740 1 Processor -- start 2024-03-26T08:51:34.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7f3285740 1 -- start start 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a564c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a5651290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 0x55e9a5653d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7f3285740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a55b13d0 con 0x55e9a564c790 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7f3285740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a5559ad0 con 0x55e9a564bda0 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7f3285740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a55ad470 con 0x55e9a5651900 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7ee43c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a5651290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a564c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a564c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37334/0 (socket says 172.21.15.7:37334) 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7ee43c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a5651290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40984/0 (socket says 172.21.15.7:40984) 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7edc3b640 1 -- 172.21.15.7:0/2769387385 learned_addr learned my addr 172.21.15.7:0/2769387385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7df7fe640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a29d0 0x7fc7d40a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/2769387385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 0x55e9a5653d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7edc3b640 1 -- 172.21.15.7:0/2769387385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 msgr2=0x55e9a5653d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/2769387385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 0x55e9a5653d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7eec3d640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a29d0 0x7fc7d40a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7edc3b640 1 -- 172.21.15.7:0/2769387385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 msgr2=0x55e9a5651290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7df7fe640 1 -- 172.21.15.7:0/3883893562 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7e40ce8c0 con 0x7fc7e8109890 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/2769387385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a5651290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7edc3b640 1 -- 172.21.15.7:0/2769387385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9a538f7d0 con 0x55e9a564bda0 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.100+0000 7fc7ee43c640 1 --2- 172.21.15.7:0/2769387385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a5651290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.101+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/2769387385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a564c1c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0013ca0 tx=0x7fc7e0009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31659961451626e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.101+0000 7fc7eec3d640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a29d0 0x7fc7d40a4e90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400e940 tx=0x7fc7e401c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.101+0000 7fc7dcff9640 1 -- 172.21.15.7:0/2769387385 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0013d90 con 0x55e9a564bda0 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.101+0000 7fc7dcff9640 1 -- 172.21.15.7:0/2769387385 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7e0011c70 con 0x55e9a564bda0 2024-03-26T08:51:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.101+0000 7fc7dcff9640 1 -- 172.21.15.7:0/2769387385 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000f3c0 con 0x55e9a564bda0 2024-03-26T08:51:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.102+0000 7fc7f3285740 1 -- 172.21.15.7:0/2769387385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 msgr2=0x55e9a564c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.102+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2769387385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a564c1c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0013ca0 tx=0x7fc7e0009c40 comp rx=0 tx=0).stop 2024-03-26T08:51:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.103+0000 7fc7f3285740 1 -- 172.21.15.7:0/2769387385 shutdown_connections 2024-03-26T08:51:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.103+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2769387385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 0x55e9a5653d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.103+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2769387385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a5651290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.103+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2769387385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a564c1c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.103+0000 7fc7f3285740 1 -- 172.21.15.7:0/2769387385 >> 172.21.15.7:0/2769387385 conn(0x55e9a5647310 msgr2=0x55e9a5647710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 -- 172.21.15.7:0/2769387385 shutdown_connections 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 -- 172.21.15.7:0/2769387385 wait complete. 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 -- start start 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 Processor -- start 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a5656070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a56565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 0x55e9a565d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a56548b0 con 0x55e9a564c790 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a556d060 con 0x55e9a564bda0 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.104+0000 7fc7f3285740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9a5654a30 con 0x55e9a5651900 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7f3285740 10 client.0 _ll_register_callbacks cb 0x55e9a554f6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a5656070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a5656070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37338/0 (socket says 172.21.15.7:37338) 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 -- 172.21.15.7:0/2177821119 learned_addr learned my addr 172.21.15.7:0/2177821119 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7ee43c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a56565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 0x55e9a565d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 -- 172.21.15.7:0/2177821119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 msgr2=0x55e9a565d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 0x55e9a565d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 -- 172.21.15.7:0/2177821119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 msgr2=0x55e9a56565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a56565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 -- 172.21.15.7:0/2177821119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9a52c53f0 con 0x55e9a564bda0 2024-03-26T08:51:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a5656070 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009b30 tx=0x7fc7e001c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86d2d2a8daf8e55 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.105+0000 7fc7d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.106+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/2177821119 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0020b10 con 0x55e9a564bda0 2024-03-26T08:51:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.106+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/2177821119 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7e000e5c0 con 0x55e9a564bda0 2024-03-26T08:51:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.106+0000 7fc7f3285740 1 -- 172.21.15.7:0/2177821119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7e0011c70 con 0x55e9a564bda0 2024-03-26T08:51:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.106+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/2177821119 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7e000e7c0 con 0x55e9a564bda0 2024-03-26T08:51:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.106+0000 7fc7f3285740 10 client.27605 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.106+0000 7fc7f3285740 1 -- 172.21.15.7:0/2177821119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e9a52850f0 con 0x55e9a564bda0 2024-03-26T08:51:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.106+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/2177821119 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7e0011c70 con 0x55e9a564bda0 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/2177821119 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc7e002e990 con 0x55e9a564bda0 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7f3285740 10 client.27605 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7f3285740 10 client.27605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7f3285740 20 client.27605 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e9a56617a0 0x55e9a5663b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7f3285740 4 client.27605 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7f3285740 1 -- 172.21.15.7:0/2177821119 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9a55be910 con 0x55e9a56617a0 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7ee43c640 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e9a56617a0 0x55e9a5663b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7d1ffb640 10 client.27605 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.107+0000 7fc7ee43c640 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e9a56617a0 0x55e9a5663b90 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395da5fc0102acfc server_cookie=fbb2b680351056f6 in_seq=0 out_seq=0 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/2177821119 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e9a55be910 con 0x55e9a56617a0 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7d1ffb640 10 client.27605 handle_command_reply: tid=0 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 client.27605 shutdown 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 -- 172.21.15.7:0/2177821119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 msgr2=0x55e9a5656070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a5656070 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009b30 tx=0x7fc7e001c040 comp rx=0 tx=0).stop 2024-03-26T08:51:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 -- 172.21.15.7:0/2177821119 shutdown_connections 2024-03-26T08:51:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e9a56617a0 0x55e9a5663b90 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a5651900 0x55e9a565d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a564c790 0x55e9a56565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 --2- 172.21.15.7:0/2177821119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a5656070 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7d1ffb640 0 client.27605 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 -- 172.21.15.7:0/2177821119 >> 172.21.15.7:0/2177821119 conn(0x55e9a5647310 msgr2=0x55e9a5646530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 -- 172.21.15.7:0/2177821119 shutdown_connections 2024-03-26T08:51:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 1 -- 172.21.15.7:0/2177821119 wait complete. 2024-03-26T08:51:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.108+0000 7fc7f3285740 20 client.27605 trim_cache size 0 max 16384 2024-03-26T08:51:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.134+0000 7fc7f3285740 1 Processor -- start 2024-03-26T08:51:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.134+0000 7fc7f3285740 1 -- start start 2024-03-26T08:51:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.134+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564bda0 0x55e9a5655c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.134+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564c790 0x55e9a565d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.134+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 0x55e9a56561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.134+0000 7fc7f3285740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81c26f0 con 0x55e9a5651900 2024-03-26T08:51:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.134+0000 7fc7f3285740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81c01f0 con 0x55e9a564c790 2024-03-26T08:51:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.134+0000 7fc7f3285740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81be270 con 0x55e9a564bda0 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564bda0 0x55e9a5655c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564bda0 0x55e9a5655c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49474/0 (socket says 172.21.15.7:49474) 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 -- 172.21.15.7:0/82306248 learned_addr learned my addr 172.21.15.7:0/82306248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7ee43c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564c790 0x55e9a565d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/82306248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 0x55e9a56561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 -- 172.21.15.7:0/82306248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564c790 msgr2=0x55e9a565d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/82306248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564c790 0x55e9a565d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 -- 172.21.15.7:0/82306248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 msgr2=0x55e9a56561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/82306248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 0x55e9a56561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 -- 172.21.15.7:0/82306248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9a564cb70 con 0x55e9a564bda0 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/82306248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564bda0 0x55e9a5655c80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0005d70 tx=0x7fc7e00021f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd7f60439517c14a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7d3fff640 1 -- 172.21.15.7:0/82306248 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e003e040 con 0x55e9a564bda0 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7d3fff640 1 -- 172.21.15.7:0/82306248 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7e0011c40 con 0x55e9a564bda0 2024-03-26T08:51:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.135+0000 7fc7d3fff640 1 -- 172.21.15.7:0/82306248 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00049c0 con 0x55e9a564bda0 2024-03-26T08:51:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.137+0000 7fc7f3285740 1 -- 172.21.15.7:0/82306248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564bda0 msgr2=0x55e9a5655c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.137+0000 7fc7f3285740 1 --2- 172.21.15.7:0/82306248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564bda0 0x55e9a5655c80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0005d70 tx=0x7fc7e00021f0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.137+0000 7fc7f3285740 1 -- 172.21.15.7:0/82306248 shutdown_connections 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.137+0000 7fc7f3285740 1 --2- 172.21.15.7:0/82306248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 0x55e9a56561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.137+0000 7fc7f3285740 1 --2- 172.21.15.7:0/82306248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564c790 0x55e9a565d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.137+0000 7fc7f3285740 1 --2- 172.21.15.7:0/82306248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564bda0 0x55e9a5655c80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.137+0000 7fc7f3285740 1 -- 172.21.15.7:0/82306248 >> 172.21.15.7:0/82306248 conn(0x55e9a5647310 msgr2=0x55e9a56476f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.137+0000 7fc7f3285740 1 -- 172.21.15.7:0/82306248 shutdown_connections 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.138+0000 7fc7f3285740 1 -- 172.21.15.7:0/82306248 wait complete. 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.138+0000 7fc7f3285740 1 -- start start 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.138+0000 7fc7f3285740 1 Processor -- start 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.138+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a565af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.138+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564c790 0x55e9a565b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a565af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7edc3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a565af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37344/0 (socket says 172.21.15.7:37344) 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7f3285740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 0x55e9a565c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7f3285740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81be870 con 0x55e9a5651900 2024-03-26T08:51:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7edc3b640 1 -- 172.21.15.7:0/1132784032 learned_addr learned my addr 172.21.15.7:0/1132784032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81bfb90 con 0x55e9a564bda0 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81beed0 con 0x55e9a564c790 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7ee43c640 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564c790 0x55e9a565b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7f3285740 10 client.0 _ll_register_callbacks cb 0x7fc7e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 0x55e9a565c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7ee43c640 1 -- 172.21.15.7:0/1132784032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 msgr2=0x55e9a565af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7ee43c640 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a565af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7ee43c640 1 -- 172.21.15.7:0/1132784032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 msgr2=0x55e9a565c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7ee43c640 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 0x55e9a565c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7ee43c640 1 -- 172.21.15.7:0/1132784032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9a565ca50 con 0x55e9a564c790 2024-03-26T08:51:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.139+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a565af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.140+0000 7fc7ee43c640 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564c790 0x55e9a565b4a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8036040 tx=0x7fc7d8005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691accb1987fb3dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.140+0000 7fc7d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:51:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.140+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/1132784032 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8005220 con 0x55e9a564c790 2024-03-26T08:51:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.140+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/1132784032 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc7d8005020 con 0x55e9a564c790 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.140+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7e0011c40 con 0x55e9a564c790 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.140+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/1132784032 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7d8004250 con 0x55e9a564c790 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.140+0000 7fc7f3285740 10 client.27616 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.140+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e9a56bd250 con 0x55e9a564c790 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/1132784032 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc7e0011c40 con 0x55e9a564c790 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/1132784032 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc7d804a600 con 0x55e9a564c790 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7f3285740 10 client.27616 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7f3285740 10 client.27616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7f3285740 20 client.27616 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7f3285740 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e9a56bdc30 0x55e9a56c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7f3285740 4 client.27616 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e9a55b8c60 con 0x55e9a56bdc30 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e9a56bdc30 0x55e9a56c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7d1ffb640 10 client.27616 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.141+0000 7fc7edc3b640 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e9a56bdc30 0x55e9a56c0050 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2032d3e9d1226252 server_cookie=2517f5de7bbe972a in_seq=0 out_seq=0 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7d1ffb640 1 -- 172.21.15.7:0/1132784032 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e9a55b8c60 con 0x55e9a56bdc30 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7d1ffb640 10 client.27616 handle_command_reply: tid=0 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 client.27616 shutdown 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564c790 msgr2=0x55e9a565b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564c790 0x55e9a565b4a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8036040 tx=0x7fc7d8005900 comp rx=0 tx=0).stop 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 shutdown_connections 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e9a56bdc30 0x55e9a56c0050 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e9a5651900 0x55e9a565c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e9a564c790 0x55e9a565b4a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 --2- 172.21.15.7:0/1132784032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e9a564bda0 0x55e9a565af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7d1ffb640 0 client.27616 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 >> 172.21.15.7:0/1132784032 conn(0x55e9a5647310 msgr2=0x55e9a5655c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 shutdown_connections 2024-03-26T08:51:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.142+0000 7fc7f3285740 1 -- 172.21.15.7:0/1132784032 wait complete. 2024-03-26T08:51:34.141 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:51:34.141 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:51:34.141 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:51:34.141 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.143+0000 7fc7f3285740 20 client.27616 trim_cache size 0 max 16384 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 -- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a29d0 msgr2=0x7fc7d40a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a29d0 0x7fc7d40a4e90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400e940 tx=0x7fc7e401c000 comp rx=0 tx=0).stop 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 -- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8109890 msgr2=0x7fc7e806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8109890 0x7fc7e806f0e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400e970 tx=0x7fc7e400ee30 comp rx=0 tx=0).stop 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 -- 172.21.15.7:0/3883893562 shutdown_connections 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc7d40a29d0 0x7fc7d40a4e90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8109890 0x7fc7e806f0e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8108ed0 0x7fc7e807f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 --2- 172.21.15.7:0/3883893562 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7e8107ce0 0x7fc7e807ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 -- 172.21.15.7:0/3883893562 >> 172.21.15.7:0/3883893562 conn(0x7fc7e8076c90 msgr2=0x7fc7e810c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 -- 172.21.15.7:0/3883893562 shutdown_connections 2024-03-26T08:51:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.144+0000 7fc7f06a0640 1 -- 172.21.15.7:0/3883893562 wait complete. 2024-03-26T08:51:34.154 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:51:34.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:51:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.327+0000 7f6843960640 1 -- 172.21.15.7:0/2294029868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c067c90 msgr2=0x7f683c0ff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.327+0000 7f6843960640 1 --2- 172.21.15.7:0/2294029868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c067c90 0x7f683c0ff8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6838009900 tx=0x7f683802bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.327+0000 7f6843960640 1 -- 172.21.15.7:0/2294029868 shutdown_connections 2024-03-26T08:51:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.327+0000 7f6843960640 1 --2- 172.21.15.7:0/2294029868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c101520 0x7f683c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.327+0000 7f6843960640 1 --2- 172.21.15.7:0/2294029868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c100b60 0x7f683c100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.327+0000 7f6843960640 1 --2- 172.21.15.7:0/2294029868 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c067c90 0x7f683c0ff8e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.327+0000 7f6843960640 1 -- 172.21.15.7:0/2294029868 >> 172.21.15.7:0/2294029868 conn(0x7f683c0faec0 msgr2=0x7f683c0fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.327+0000 7f6843960640 1 -- 172.21.15.7:0/2294029868 shutdown_connections 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.327+0000 7f6843960640 1 -- 172.21.15.7:0/2294029868 wait complete. 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6843960640 1 Processor -- start 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6843960640 1 -- start start 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6843960640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c067c90 0x7f683c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6843960640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c100b60 0x7f683c19e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6843960640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c101520 0x7f683c1a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6843960640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c106e00 con 0x7f683c101520 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6843960640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c106c80 con 0x7f683c067c90 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6843960640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c10b430 con 0x7f683c100b60 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f68416fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c067c90 0x7f683c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f68416fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c067c90 0x7f683c19de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37358/0 (socket says 172.21.15.7:37358) 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f68416fc640 1 -- 172.21.15.7:0/2842956242 learned_addr learned my addr 172.21.15.7:0/2842956242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6840efb640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c100b60 0x7f683c19e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f6841efd640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c101520 0x7f683c1a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f68416fc640 1 -- 172.21.15.7:0/2842956242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c100b60 msgr2=0x7f683c19e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f68416fc640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c100b60 0x7f683c19e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f68416fc640 1 -- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c101520 msgr2=0x7f683c1a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f68416fc640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c101520 0x7f683c1a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.328+0000 7f68416fc640 1 -- 172.21.15.7:0/2842956242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c1a5a20 con 0x7f683c067c90 2024-03-26T08:51:34.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.329+0000 7f68416fc640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c067c90 0x7f683c19de00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f683802efd0 tx=0x7f6838005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.329+0000 7f68327fc640 1 -- 172.21.15.7:0/2842956242 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838041070 con 0x7f683c067c90 2024-03-26T08:51:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.330+0000 7f68327fc640 1 -- 172.21.15.7:0/2842956242 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68380054e0 con 0x7f683c067c90 2024-03-26T08:51:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.330+0000 7f68327fc640 1 -- 172.21.15.7:0/2842956242 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838034440 con 0x7f683c067c90 2024-03-26T08:51:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.330+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c1a5cb0 con 0x7f683c067c90 2024-03-26T08:51:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.330+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c1a6150 con 0x7f683c067c90 2024-03-26T08:51:34.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.332+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c10ea70 con 0x7f683c067c90 2024-03-26T08:51:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.333+0000 7f68327fc640 1 -- 172.21.15.7:0/2842956242 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6838044070 con 0x7f683c067c90 2024-03-26T08:51:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.334+0000 7f68327fc640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f68280a29d0 0x7f68280a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.334+0000 7f68327fc640 1 -- 172.21.15.7:0/2842956242 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f68380f01c0 con 0x7f683c067c90 2024-03-26T08:51:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.334+0000 7f6840efb640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f68280a29d0 0x7f68280a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.335+0000 7f6840efb640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f68280a29d0 0x7f68280a4e90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f683c19f310 tx=0x7f682c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.341+0000 7f68327fc640 1 -- 172.21.15.7:0/2842956242 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f68380bd220 con 0x7f683c067c90 2024-03-26T08:51:34.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:34 smithi007 ceph-mon[32462]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 662 op/s 2024-03-26T08:51:34.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:34 smithi044 ceph-mon[36461]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 662 op/s 2024-03-26T08:51:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.704+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f683c0622a0 con 0x7f683c067c90 2024-03-26T08:51:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.705+0000 7f68327fc640 1 -- 172.21.15.7:0/2842956242 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f68380bcb20 con 0x7f683c067c90 2024-03-26T08:51:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:51:34.701 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:51:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:51:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f68280a29d0 msgr2=0x7f68280a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f68280a29d0 0x7f68280a4e90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f683c19f310 tx=0x7f682c005f90 comp rx=0 tx=0).stop 2024-03-26T08:51:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c067c90 msgr2=0x7f683c19de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c067c90 0x7f683c19de00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f683802efd0 tx=0x7f6838005dd0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 shutdown_connections 2024-03-26T08:51:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f68280a29d0 0x7f68280a4e90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c101520 0x7f683c1a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f683c100b60 0x7f683c19e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 --2- 172.21.15.7:0/2842956242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c067c90 0x7f683c19de00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 >> 172.21.15.7:0/2842956242 conn(0x7f683c0faec0 msgr2=0x7f683c10bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 shutdown_connections 2024-03-26T08:51:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.708+0000 7f6843960640 1 -- 172.21.15.7:0/2842956242 wait complete. 2024-03-26T08:51:34.717 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:51:34.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:51:34.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:34 smithi016 ceph-mon[35851]: pgmap v634: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.8 MiB/s wr, 662 op/s 2024-03-26T08:51:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.885+0000 7fb214c4b640 1 -- 172.21.15.7:0/1688760137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21010b010 msgr2=0x7fb21010d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.885+0000 7fb214c4b640 1 --2- 172.21.15.7:0/1688760137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21010b010 0x7fb21010d480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009860 tx=0x7fb1f802bb70 comp rx=0 tx=0).stop 2024-03-26T08:51:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.886+0000 7fb214c4b640 1 -- 172.21.15.7:0/1688760137 shutdown_connections 2024-03-26T08:51:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.886+0000 7fb214c4b640 1 --2- 172.21.15.7:0/1688760137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb21010d9c0 0x7fb21010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.886+0000 7fb214c4b640 1 --2- 172.21.15.7:0/1688760137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21010b010 0x7fb21010d480 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.886+0000 7fb214c4b640 1 --2- 172.21.15.7:0/1688760137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2101086a0 0x7fb21010aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.886+0000 7fb214c4b640 1 -- 172.21.15.7:0/1688760137 >> 172.21.15.7:0/1688760137 conn(0x7fb21006e640 msgr2=0x7fb210070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.886+0000 7fb214c4b640 1 -- 172.21.15.7:0/1688760137 shutdown_connections 2024-03-26T08:51:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.886+0000 7fb214c4b640 1 -- 172.21.15.7:0/1688760137 wait complete. 2024-03-26T08:51:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.887+0000 7fb214c4b640 1 Processor -- start 2024-03-26T08:51:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.888+0000 7fb214c4b640 1 -- start start 2024-03-26T08:51:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.888+0000 7fb214c4b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2101086a0 0x7fb2101a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.888+0000 7fb214c4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21010b010 0x7fb2101a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.888+0000 7fb214c4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb21010d9c0 0x7fb2101adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.888+0000 7fb214c4b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21011c0e0 con 0x7fb2101086a0 2024-03-26T08:51:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.888+0000 7fb214c4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21011bf60 con 0x7fb21010d9c0 2024-03-26T08:51:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.888+0000 7fb214c4b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21011c260 con 0x7fb21010b010 2024-03-26T08:51:34.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.890+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2101086a0 0x7fb2101a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.892+0000 7fb20dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21010b010 0x7fb2101a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.892+0000 7fb20dd9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21010b010 0x7fb2101a6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49504/0 (socket says 172.21.15.7:49504) 2024-03-26T08:51:34.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.892+0000 7fb20dd9b640 1 -- 172.21.15.7:0/3531929837 learned_addr learned my addr 172.21.15.7:0/3531929837 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.892+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb21010d9c0 0x7fb2101adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.893+0000 7fb20ed9d640 1 -- 172.21.15.7:0/3531929837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21010b010 msgr2=0x7fb2101a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.893+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21010b010 0x7fb2101a6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.893+0000 7fb20ed9d640 1 -- 172.21.15.7:0/3531929837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2101086a0 msgr2=0x7fb2101a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.893+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2101086a0 0x7fb2101a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.893+0000 7fb20ed9d640 1 -- 172.21.15.7:0/3531929837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2101ae1d0 con 0x7fb21010d9c0 2024-03-26T08:51:34.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.893+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb21010d9c0 0x7fb2101adb60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb208009fd0 tx=0x7fb20800eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.896+0000 7fb2077fe640 1 -- 172.21.15.7:0/3531929837 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208004220 con 0x7fb21010d9c0 2024-03-26T08:51:34.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.896+0000 7fb2077fe640 1 -- 172.21.15.7:0/3531929837 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb20800cd90 con 0x7fb21010d9c0 2024-03-26T08:51:34.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.896+0000 7fb2077fe640 1 -- 172.21.15.7:0/3531929837 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208004a00 con 0x7fb21010d9c0 2024-03-26T08:51:34.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.897+0000 7fb214c4b640 1 -- 172.21.15.7:0/3531929837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2101ae4c0 con 0x7fb21010d9c0 2024-03-26T08:51:34.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.897+0000 7fb214c4b640 1 -- 172.21.15.7:0/3531929837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2101ae960 con 0x7fb21010d9c0 2024-03-26T08:51:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.900+0000 7fb217836740 1 Processor -- start 2024-03-26T08:51:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.901+0000 7fb217836740 1 -- start start 2024-03-26T08:51:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.901+0000 7fb217836740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e478da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.901+0000 7fb217836740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e47de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.901+0000 7fb217836740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 0x56171e480970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.901+0000 7fb217836740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171e42a610 con 0x56171e479370 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.901+0000 7fb217836740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171e3cd3c0 con 0x56171e478980 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.901+0000 7fb217836740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171e375d60 con 0x56171e47e4e0 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.903+0000 7fb20dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e478da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.903+0000 7fb20dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e478da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37390/0 (socket says 172.21.15.7:37390) 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.903+0000 7fb20dd9b640 1 -- 172.21.15.7:0/409926599 learned_addr learned my addr 172.21.15.7:0/409926599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.903+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 0x56171e480970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.903+0000 7fb20e59c640 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e47de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.904+0000 7fb2077fe640 1 -- 172.21.15.7:0/3531929837 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb208004ba0 con 0x7fb21010d9c0 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.904+0000 7fb20dd9b640 1 -- 172.21.15.7:0/409926599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 msgr2=0x56171e480970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.904+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 0x56171e480970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.904+0000 7fb20dd9b640 1 -- 172.21.15.7:0/409926599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 msgr2=0x56171e47de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.904+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e47de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.904+0000 7fb20dd9b640 1 -- 172.21.15.7:0/409926599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56171e0e13f0 con 0x56171e478980 2024-03-26T08:51:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.904+0000 7fb20e59c640 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e47de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:51:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.905+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e478da0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009780 tx=0x7fb1f802bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cbc4073e367c5b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.905+0000 7fb2077fe640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1fc0a29d0 0x7fb1fc0a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.905+0000 7fb1f7fff640 1 -- 172.21.15.7:0/409926599 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f806b070 con 0x56171e478980 2024-03-26T08:51:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.905+0000 7fb1f7fff640 1 -- 172.21.15.7:0/409926599 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1f8004040 con 0x56171e478980 2024-03-26T08:51:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.906+0000 7fb2077fe640 1 -- 172.21.15.7:0/3531929837 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb2080ce890 con 0x7fb21010d9c0 2024-03-26T08:51:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.906+0000 7fb20e59c640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1fc0a29d0 0x7fb1fc0a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.906+0000 7fb1f7fff640 1 -- 172.21.15.7:0/409926599 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8039040 con 0x56171e478980 2024-03-26T08:51:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.906+0000 7fb217836740 1 -- 172.21.15.7:0/409926599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 msgr2=0x56171e478da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.906+0000 7fb217836740 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e478da0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009780 tx=0x7fb1f802bb70 comp rx=0 tx=0).stop 2024-03-26T08:51:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.907+0000 7fb20e59c640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1fc0a29d0 0x7fb1fc0a4e90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x56171e479770 tx=0x7fb200009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.907+0000 7fb217836740 1 -- 172.21.15.7:0/409926599 shutdown_connections 2024-03-26T08:51:34.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.907+0000 7fb217836740 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 0x56171e480970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.907+0000 7fb217836740 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e47de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.907+0000 7fb217836740 1 --2- 172.21.15.7:0/409926599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e478da0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.907+0000 7fb217836740 1 -- 172.21.15.7:0/409926599 >> 172.21.15.7:0/409926599 conn(0x56171e462ee0 msgr2=0x56171e4632e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.907+0000 7fb217836740 1 -- 172.21.15.7:0/409926599 shutdown_connections 2024-03-26T08:51:34.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.907+0000 7fb217836740 1 -- 172.21.15.7:0/409926599 wait complete. 2024-03-26T08:51:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb217836740 1 -- start start 2024-03-26T08:51:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb217836740 1 Processor -- start 2024-03-26T08:51:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb217836740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e478980 0x56171e468ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb217836740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 0x56171e469410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb217836740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e47e4e0 0x56171e470450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb217836740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171e389240 con 0x56171e478980 2024-03-26T08:51:34.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb217836740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171e3c9460 con 0x56171e47e4e0 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb217836740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171e467800 con 0x56171e479370 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e478980 0x56171e468ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e478980 0x56171e468ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41054/0 (socket says 172.21.15.7:41054) 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb20e59c640 1 -- 172.21.15.7:0/243226760 learned_addr learned my addr 172.21.15.7:0/243226760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb217836740 10 client.0 _ll_register_callbacks cb 0x56171e388c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 0x56171e469410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.908+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e47e4e0 0x56171e470450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb20e59c640 1 -- 172.21.15.7:0/243226760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 msgr2=0x56171e469410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb20e59c640 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 0x56171e469410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb20e59c640 1 -- 172.21.15.7:0/243226760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e47e4e0 msgr2=0x56171e470450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb20e59c640 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e47e4e0 0x56171e470450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb20e59c640 1 -- 172.21.15.7:0/243226760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56171e1ab7d0 con 0x56171e478980 2024-03-26T08:51:34.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e47e4e0 0x56171e470450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb20e59c640 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e478980 0x56171e468ed0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb200012900 tx=0x7fb200004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1990a97e6a8d15f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb1f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb1f5ffb640 1 -- 172.21.15.7:0/243226760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200011cf0 con 0x56171e478980 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb1f5ffb640 1 -- 172.21.15.7:0/243226760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb20001f420 con 0x56171e478980 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb217836740 1 -- 172.21.15.7:0/243226760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1f8004040 con 0x56171e478980 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb1f5ffb640 1 -- 172.21.15.7:0/243226760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb200011e90 con 0x56171e478980 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb217836740 10 client.17805 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.909+0000 7fb217836740 1 -- 172.21.15.7:0/243226760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56171e473800 con 0x56171e478980 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.910+0000 7fb1f5ffb640 1 -- 172.21.15.7:0/243226760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb1f8004040 con 0x56171e478980 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.910+0000 7fb1f5ffb640 1 -- 172.21.15.7:0/243226760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb200032760 con 0x56171e478980 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.910+0000 7fb217836740 10 client.17805 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.910+0000 7fb217836740 10 client.17805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.910+0000 7fb217836740 20 client.17805 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.910+0000 7fb217836740 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56171e4752e0 0x56171e477760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.910+0000 7fb217836740 4 client.17805 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:34.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.910+0000 7fb217836740 1 -- 172.21.15.7:0/243226760 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56171e3da900 con 0x56171e4752e0 2024-03-26T08:51:34.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.910+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56171e4752e0 0x56171e477760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.911+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56171e4752e0 0x56171e477760 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba216f6ce9b1456 server_cookie=c1c5b62d2f630742 in_seq=0 out_seq=0 2024-03-26T08:51:34.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.911+0000 7fb1f5ffb640 10 client.17805 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:51:34.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb1f5ffb640 1 -- 172.21.15.7:0/243226760 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56171e3da900 con 0x56171e4752e0 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb1f5ffb640 10 client.17805 handle_command_reply: tid=0 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb217836740 1 client.17805 shutdown 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb217836740 1 -- 172.21.15.7:0/243226760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e478980 msgr2=0x56171e468ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb217836740 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e478980 0x56171e468ed0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb200012900 tx=0x7fb200004a40 comp rx=0 tx=0).stop 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb217836740 1 -- 172.21.15.7:0/243226760 shutdown_connections 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb217836740 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56171e4752e0 0x56171e477760 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb1f5ffb640 0 client.17805 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb217836740 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e47e4e0 0x56171e470450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb217836740 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 0x56171e469410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb217836740 1 --2- 172.21.15.7:0/243226760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e478980 0x56171e468ed0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.912+0000 7fb217836740 1 -- 172.21.15.7:0/243226760 >> 172.21.15.7:0/243226760 conn(0x56171e462ee0 msgr2=0x56171e462070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.913+0000 7fb217836740 1 -- 172.21.15.7:0/243226760 shutdown_connections 2024-03-26T08:51:34.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.913+0000 7fb217836740 1 -- 172.21.15.7:0/243226760 wait complete. 2024-03-26T08:51:34.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.913+0000 7fb217836740 20 client.17805 trim_cache size 0 max 16384 2024-03-26T08:51:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.935+0000 7fb217836740 1 Processor -- start 2024-03-26T08:51:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.935+0000 7fb217836740 1 -- start start 2024-03-26T08:51:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.935+0000 7fb217836740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e461ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.935+0000 7fb217836740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 0x56171e485210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.935+0000 7fb217836740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e47e4e0 0x56171e485750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.935+0000 7fb217836740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101b1350 con 0x56171e47e4e0 2024-03-26T08:51:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.935+0000 7fb217836740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101aee50 con 0x56171e478980 2024-03-26T08:51:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.935+0000 7fb217836740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb210116890 con 0x56171e479370 2024-03-26T08:51:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e461ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e461ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37404/0 (socket says 172.21.15.7:37404) 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20e59c640 1 -- 172.21.15.7:0/3437366906 learned_addr learned my addr 172.21.15.7:0/3437366906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e47e4e0 0x56171e485750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 0x56171e485210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20ed9d640 1 -- 172.21.15.7:0/3437366906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 msgr2=0x56171e485210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 0x56171e485210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20ed9d640 1 -- 172.21.15.7:0/3437366906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 msgr2=0x56171e461ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e461ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20ed9d640 1 -- 172.21.15.7:0/3437366906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56171e455ae0 con 0x56171e47e4e0 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20e59c640 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e461ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e47e4e0 0x56171e485750 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb2080cc350 tx=0x7fb20809ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a88cf460db3db7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb1f77fe640 1 -- 172.21.15.7:0/3437366906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20801c070 con 0x56171e47e4e0 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb1f77fe640 1 -- 172.21.15.7:0/3437366906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb20809b1f0 con 0x56171e47e4e0 2024-03-26T08:51:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.936+0000 7fb1f77fe640 1 -- 172.21.15.7:0/3437366906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208005020 con 0x56171e47e4e0 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.937+0000 7fb217836740 1 -- 172.21.15.7:0/3437366906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e47e4e0 msgr2=0x56171e485750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.937+0000 7fb217836740 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e47e4e0 0x56171e485750 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb2080cc350 tx=0x7fb20809ab50 comp rx=0 tx=0).stop 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.937+0000 7fb217836740 1 -- 172.21.15.7:0/3437366906 shutdown_connections 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.937+0000 7fb217836740 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e47e4e0 0x56171e485750 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.937+0000 7fb217836740 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e479370 0x56171e485210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.937+0000 7fb217836740 1 --2- 172.21.15.7:0/3437366906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e461ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.937+0000 7fb217836740 1 -- 172.21.15.7:0/3437366906 >> 172.21.15.7:0/3437366906 conn(0x56171e462ee0 msgr2=0x56171e4617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.937+0000 7fb217836740 1 -- 172.21.15.7:0/3437366906 shutdown_connections 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.937+0000 7fb217836740 1 -- 172.21.15.7:0/3437366906 wait complete. 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb217836740 1 -- start start 2024-03-26T08:51:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb217836740 1 Processor -- start 2024-03-26T08:51:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb217836740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e485890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb217836740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e485dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb217836740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 0x56171e4d84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb217836740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb210116e90 con 0x56171e479370 2024-03-26T08:51:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb217836740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101181b0 con 0x56171e478980 2024-03-26T08:51:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb217836740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2101174f0 con 0x56171e47e4e0 2024-03-26T08:51:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb217836740 10 client.0 _ll_register_callbacks cb 0x7fb2000125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb20dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e485dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e485890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb20e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e485890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37406/0 (socket says 172.21.15.7:37406) 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb20dd9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e485dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41062/0 (socket says 172.21.15.7:41062) 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb20e59c640 1 -- 172.21.15.7:0/523682206 learned_addr learned my addr 172.21.15.7:0/523682206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.938+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 0x56171e4d84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb20e59c640 1 -- 172.21.15.7:0/523682206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 msgr2=0x56171e4d84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb20e59c640 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 0x56171e4d84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb20e59c640 1 -- 172.21.15.7:0/523682206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 msgr2=0x56171e485dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb20e59c640 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e485dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb20e59c640 1 -- 172.21.15.7:0/523682206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56171e4d8af0 con 0x56171e478980 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e485dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb20ed9d640 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 0x56171e4d84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb20e59c640 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e485890 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb20000eca0 tx=0x7fb200009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc3fef9a221b270b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb1f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb1f57fa640 1 -- 172.21.15.7:0/523682206 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200011e90 con 0x56171e478980 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb1f57fa640 1 -- 172.21.15.7:0/523682206 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb20000fbe0 con 0x56171e478980 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb217836740 1 -- 172.21.15.7:0/523682206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb20809b1f0 con 0x56171e478980 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb1f57fa640 1 -- 172.21.15.7:0/523682206 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb20000fde0 con 0x56171e478980 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb217836740 10 client.27635 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.939+0000 7fb217836740 1 -- 172.21.15.7:0/523682206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56171e4d9e90 con 0x56171e478980 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.940+0000 7fb1f57fa640 1 -- 172.21.15.7:0/523682206 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb20809b1f0 con 0x56171e478980 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.940+0000 7fb1f57fa640 1 -- 172.21.15.7:0/523682206 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb200046050 con 0x56171e478980 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.940+0000 7fb217836740 10 client.27635 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.940+0000 7fb217836740 10 client.27635 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.940+0000 7fb217836740 20 client.27635 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.940+0000 7fb217836740 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56171e4da7c0 0x56171e4dcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.940+0000 7fb217836740 4 client.27635 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.940+0000 7fb217836740 1 -- 172.21.15.7:0/523682206 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56171e4785e0 con 0x56171e4da7c0 2024-03-26T08:51:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.940+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56171e4da7c0 0x56171e4dcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.941+0000 7fb20dd9b640 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56171e4da7c0 0x56171e4dcbe0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bff5d1544734716 server_cookie=4a728faa24a2d81a in_seq=0 out_seq=0 2024-03-26T08:51:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.941+0000 7fb1f57fa640 10 client.27635 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:51:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.942+0000 7fb1f57fa640 1 -- 172.21.15.7:0/523682206 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56171e4785e0 con 0x56171e4da7c0 2024-03-26T08:51:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.942+0000 7fb1f57fa640 10 client.27635 handle_command_reply: tid=0 2024-03-26T08:51:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.942+0000 7fb217836740 1 client.27635 shutdown 2024-03-26T08:51:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb217836740 1 -- 172.21.15.7:0/523682206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 msgr2=0x56171e485890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb217836740 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e485890 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb20000eca0 tx=0x7fb200009570 comp rx=0 tx=0).stop 2024-03-26T08:51:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb217836740 1 -- 172.21.15.7:0/523682206 shutdown_connections 2024-03-26T08:51:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb217836740 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56171e4da7c0 0x56171e4dcbe0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb217836740 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56171e47e4e0 0x56171e4d84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb1f57fa640 0 client.27635 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:51:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb217836740 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56171e479370 0x56171e485dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb217836740 1 --2- 172.21.15.7:0/523682206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56171e478980 0x56171e485890 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb217836740 1 -- 172.21.15.7:0/523682206 >> 172.21.15.7:0/523682206 conn(0x56171e462ee0 msgr2=0x56171e4617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.943+0000 7fb217836740 1 -- 172.21.15.7:0/523682206 shutdown_connections 2024-03-26T08:51:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.944+0000 7fb217836740 1 -- 172.21.15.7:0/523682206 wait complete. 2024-03-26T08:51:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.944+0000 7fb217836740 20 client.27635 trim_cache size 0 max 16384 2024-03-26T08:51:34.943 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:51:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.944+0000 7fb2057fa640 1 -- 172.21.15.7:0/3531929837 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1fc0a29d0 msgr2=0x7fb1fc0a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.944+0000 7fb2057fa640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1fc0a29d0 0x7fb1fc0a4e90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x56171e479770 tx=0x7fb200009500 comp rx=0 tx=0).stop 2024-03-26T08:51:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.944+0000 7fb2057fa640 1 -- 172.21.15.7:0/3531929837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb21010d9c0 msgr2=0x7fb2101adb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.944+0000 7fb2057fa640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb21010d9c0 0x7fb2101adb60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb208009fd0 tx=0x7fb20800eba0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.945+0000 7fb2057fa640 1 -- 172.21.15.7:0/3531929837 shutdown_connections 2024-03-26T08:51:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.945+0000 7fb2057fa640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb1fc0a29d0 0x7fb1fc0a4e90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.945+0000 7fb2057fa640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb21010d9c0 0x7fb2101adb60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.945+0000 7fb2057fa640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb21010b010 0x7fb2101a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.945+0000 7fb2057fa640 1 --2- 172.21.15.7:0/3531929837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2101086a0 0x7fb2101a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.945+0000 7fb2057fa640 1 -- 172.21.15.7:0/3531929837 >> 172.21.15.7:0/3531929837 conn(0x7fb21006e640 msgr2=0x7fb210109460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.945+0000 7fb2057fa640 1 -- 172.21.15.7:0/3531929837 shutdown_connections 2024-03-26T08:51:34.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:34.945+0000 7fb2057fa640 1 -- 172.21.15.7:0/3531929837 wait complete. 2024-03-26T08:51:34.954 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:51:34.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:51:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.106+0000 7efc611d3640 1 -- 172.21.15.7:0/1407951644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c0fff90 msgr2=0x7efc5c1003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.106+0000 7efc611d3640 1 --2- 172.21.15.7:0/1407951644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c0fff90 0x7efc5c1003b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009900 tx=0x7efc4c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.106+0000 7efc611d3640 1 -- 172.21.15.7:0/1407951644 shutdown_connections 2024-03-26T08:51:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.106+0000 7efc611d3640 1 --2- 172.21.15.7:0/1407951644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5c101290 0x7efc5c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.106+0000 7efc611d3640 1 --2- 172.21.15.7:0/1407951644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5c1008f0 0x7efc5c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.106+0000 7efc611d3640 1 --2- 172.21.15.7:0/1407951644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c0fff90 0x7efc5c1003b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.106+0000 7efc611d3640 1 -- 172.21.15.7:0/1407951644 >> 172.21.15.7:0/1407951644 conn(0x7efc5c0fbb30 msgr2=0x7efc5c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.106+0000 7efc611d3640 1 -- 172.21.15.7:0/1407951644 shutdown_connections 2024-03-26T08:51:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.107+0000 7efc611d3640 1 -- 172.21.15.7:0/1407951644 wait complete. 2024-03-26T08:51:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.107+0000 7efc611d3640 1 Processor -- start 2024-03-26T08:51:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc611d3640 1 -- start start 2024-03-26T08:51:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc611d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c1008f0 0x7efc5c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc611d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5c101290 0x7efc5c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc611d3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5c19f390 0x7efc5c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc611d3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c10f570 con 0x7efc5c1008f0 2024-03-26T08:51:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc611d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c10f3f0 con 0x7efc5c101290 2024-03-26T08:51:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc611d3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c10f6f0 con 0x7efc5c19f390 2024-03-26T08:51:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c1008f0 0x7efc5c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5c101290 0x7efc5c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c1008f0 0x7efc5c199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41082/0 (socket says 172.21.15.7:41082) 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5ad9d640 1 -- 172.21.15.7:0/1668117063 learned_addr learned my addr 172.21.15.7:0/1668117063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5b59e640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5c19f390 0x7efc5c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5a59c640 1 -- 172.21.15.7:0/1668117063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5c19f390 msgr2=0x7efc5c1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5a59c640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5c19f390 0x7efc5c1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5a59c640 1 -- 172.21.15.7:0/1668117063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c1008f0 msgr2=0x7efc5c199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5a59c640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c1008f0 0x7efc5c199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5a59c640 1 -- 172.21.15.7:0/1668117063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c1a1df0 con 0x7efc5c101290 2024-03-26T08:51:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.108+0000 7efc5a59c640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5c101290 0x7efc5c19a330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e9f0 tx=0x7efc5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.109+0000 7efc43fff640 1 -- 172.21.15.7:0/1668117063 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5000ccc0 con 0x7efc5c101290 2024-03-26T08:51:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.109+0000 7efc611d3640 1 -- 172.21.15.7:0/1668117063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c1a20e0 con 0x7efc5c101290 2024-03-26T08:51:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.109+0000 7efc611d3640 1 -- 172.21.15.7:0/1668117063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c1a2680 con 0x7efc5c101290 2024-03-26T08:51:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.109+0000 7efc43fff640 1 -- 172.21.15.7:0/1668117063 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc50010910 con 0x7efc5c101290 2024-03-26T08:51:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.109+0000 7efc43fff640 1 -- 172.21.15.7:0/1668117063 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50018960 con 0x7efc5c101290 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.111+0000 7efc43fff640 1 -- 172.21.15.7:0/1668117063 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7efc50018b00 con 0x7efc5c101290 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.111+0000 7efc63db6740 1 Processor -- start 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.111+0000 7efc63db6740 1 -- start start 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc63db6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6178690 0x5614e6178ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc63db6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6179080 0x5614e617db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc63db6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 0x5614e6180680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc63db6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e613b610 con 0x5614e6178690 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc63db6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e60de3c0 con 0x5614e617e1f0 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc63db6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e6086d60 con 0x5614e6179080 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc43fff640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc440a28d0 0x7efc440a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6178690 0x5614e6178ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6178690 0x5614e6178ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41094/0 (socket says 172.21.15.7:41094) 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 -- 172.21.15.7:0/3225061451 learned_addr learned my addr 172.21.15.7:0/3225061451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5ad9d640 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6179080 0x5614e617db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5a59c640 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 0x5614e6180680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc440a28d0 0x7efc440a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 -- 172.21.15.7:0/3225061451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6179080 msgr2=0x5614e617db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc43fff640 1 -- 172.21.15.7:0/1668117063 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efc500cece0 con 0x7efc5c101290 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6179080 0x5614e617db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 -- 172.21.15.7:0/3225061451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 msgr2=0x5614e6180680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 0x5614e6180680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 -- 172.21.15.7:0/3225061451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e5df23f0 con 0x5614e6178690 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5ad9d640 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6179080 0x5614e617db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5a59c640 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 0x5614e6180680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.112+0000 7efc5b59e640 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6178690 0x5614e6178ab0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efc54005c40 tx=0x7efc5400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469a78d3747c76a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.113+0000 7efc5b59e640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc440a28d0 0x7efc440a4d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efc5400f8c0 tx=0x7efc5401f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.113+0000 7efc417fa640 1 -- 172.21.15.7:0/3225061451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5400fd30 con 0x5614e6178690 2024-03-26T08:51:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.113+0000 7efc417fa640 1 -- 172.21.15.7:0/3225061451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc54022030 con 0x5614e6178690 2024-03-26T08:51:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.113+0000 7efc417fa640 1 -- 172.21.15.7:0/3225061451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54010930 con 0x5614e6178690 2024-03-26T08:51:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.114+0000 7efc63db6740 1 -- 172.21.15.7:0/3225061451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6178690 msgr2=0x5614e6178ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.114+0000 7efc63db6740 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6178690 0x5614e6178ab0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efc54005c40 tx=0x7efc5400f3c0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.115+0000 7efc63db6740 1 -- 172.21.15.7:0/3225061451 shutdown_connections 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.115+0000 7efc63db6740 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 0x5614e6180680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.115+0000 7efc63db6740 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6179080 0x5614e617db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.115+0000 7efc63db6740 1 --2- 172.21.15.7:0/3225061451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6178690 0x5614e6178ab0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.115+0000 7efc63db6740 1 -- 172.21.15.7:0/3225061451 >> 172.21.15.7:0/3225061451 conn(0x5614e6173c00 msgr2=0x5614e6174000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.115+0000 7efc63db6740 1 -- 172.21.15.7:0/3225061451 shutdown_connections 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.115+0000 7efc63db6740 1 -- 172.21.15.7:0/3225061451 wait complete. 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.116+0000 7efc63db6740 1 -- start start 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.116+0000 7efc63db6740 1 Processor -- start 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.116+0000 7efc63db6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e61829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.116+0000 7efc63db6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6179080 0x5614e6182ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.116+0000 7efc63db6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 0x5614e6189f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.116+0000 7efc5a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6179080 0x5614e6182ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e61829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6179080 0x5614e6182ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41104/0 (socket says 172.21.15.7:41104) 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc63db6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e609a240 con 0x5614e6179080 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5a59c640 1 -- 172.21.15.7:0/2738064476 learned_addr learned my addr 172.21.15.7:0/2738064476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e60da460 con 0x5614e617e1f0 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e6181140 con 0x5614e6178690 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5b59e640 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 0x5614e6189f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc63db6740 10 client.0 _ll_register_callbacks cb 0x5614e6099c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5ad9d640 1 -- 172.21.15.7:0/2738064476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 msgr2=0x5614e6189f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5ad9d640 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 0x5614e6189f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5ad9d640 1 -- 172.21.15.7:0/2738064476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6179080 msgr2=0x5614e6182ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5ad9d640 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6179080 0x5614e6182ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5ad9d640 1 -- 172.21.15.7:0/2738064476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614e5ebc7d0 con 0x5614e6178690 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc5ad9d640 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e61829a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009850 tx=0x7efc4c004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b64df9a6194ce42c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:51:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc3eff5640 1 -- 172.21.15.7:0/2738064476 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c005030 con 0x5614e6178690 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc3eff5640 1 -- 172.21.15.7:0/2738064476 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efc4c0051d0 con 0x5614e6178690 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.117+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc54022030 con 0x5614e6178690 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc3eff5640 1 -- 172.21.15.7:0/2738064476 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc4c0053d0 con 0x5614e6178690 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc63db6740 10 client.27652 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5614e618d2c0 con 0x5614e6178690 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc3eff5640 1 -- 172.21.15.7:0/2738064476 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc54022030 con 0x5614e6178690 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc3eff5640 1 -- 172.21.15.7:0/2738064476 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7efc4c0035f0 con 0x5614e6178690 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc63db6740 10 client.27652 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc63db6740 10 client.27652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc63db6740 20 client.27652 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc63db6740 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5614e618e8b0 0x5614e6190d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc63db6740 4 client.27652 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614e60eb900 con 0x5614e618e8b0 2024-03-26T08:51:35.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.118+0000 7efc5a59c640 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5614e618e8b0 0x5614e6190d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.119+0000 7efc3eff5640 10 client.27652 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:51:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.119+0000 7efc5a59c640 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5614e618e8b0 0x5614e6190d00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=466102e7fc8bf1b server_cookie=9d8bb44d87a8340c in_seq=0 out_seq=0 2024-03-26T08:51:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.119+0000 7efc3eff5640 1 -- 172.21.15.7:0/2738064476 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5614e60eb900 con 0x5614e618e8b0 2024-03-26T08:51:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc3eff5640 10 client.27652 handle_command_reply: tid=0 2024-03-26T08:51:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 client.27652 shutdown 2024-03-26T08:51:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 msgr2=0x5614e61829a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e61829a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009850 tx=0x7efc4c004250 comp rx=0 tx=0).stop 2024-03-26T08:51:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 shutdown_connections 2024-03-26T08:51:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e617e1f0 0x5614e6189f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e6179080 0x5614e6182ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5614e618e8b0 0x5614e6190d00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc3eff5640 0 client.27652 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:51:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 --2- 172.21.15.7:0/2738064476 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e61829a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 >> 172.21.15.7:0/2738064476 conn(0x5614e6173c00 msgr2=0x5614e6172e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 shutdown_connections 2024-03-26T08:51:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 1 -- 172.21.15.7:0/2738064476 wait complete. 2024-03-26T08:51:35.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.120+0000 7efc63db6740 20 client.27652 trim_cache size 0 max 16384 2024-03-26T08:51:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.136+0000 7efc63db6740 1 Processor -- start 2024-03-26T08:51:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc63db6740 1 -- start start 2024-03-26T08:51:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc63db6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e6195600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc63db6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e6189ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc63db6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 0x5614e6195b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc63db6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc48010820 con 0x5614e617e1f0 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc63db6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4800e320 con 0x5614e6179080 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc63db6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4800c3a0 con 0x5614e6178690 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc5a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e6189ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc5ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e6195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc5a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e6189ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37454/0 (socket says 172.21.15.7:37454) 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc5a59c640 1 -- 172.21.15.7:0/669883606 learned_addr learned my addr 172.21.15.7:0/669883606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.137+0000 7efc5b59e640 1 --2- 172.21.15.7:0/669883606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 0x5614e6195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.138+0000 7efc5a59c640 1 -- 172.21.15.7:0/669883606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 msgr2=0x5614e6195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.138+0000 7efc5a59c640 1 --2- 172.21.15.7:0/669883606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e6195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.138+0000 7efc5a59c640 1 -- 172.21.15.7:0/669883606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 msgr2=0x5614e6195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.138+0000 7efc5a59c640 1 --2- 172.21.15.7:0/669883606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 0x5614e6195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.138+0000 7efc5a59c640 1 -- 172.21.15.7:0/669883606 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e6173a30 con 0x5614e6179080 2024-03-26T08:51:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.138+0000 7efc5a59c640 1 --2- 172.21.15.7:0/669883606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e6189ee0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efc5009b680 tx=0x7efc50086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=662c20161d2e3968 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.138+0000 7efc40ff9640 1 -- 172.21.15.7:0/669883606 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50099790 con 0x5614e6179080 2024-03-26T08:51:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.138+0000 7efc40ff9640 1 -- 172.21.15.7:0/669883606 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc500999f0 con 0x5614e6179080 2024-03-26T08:51:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.138+0000 7efc40ff9640 1 -- 172.21.15.7:0/669883606 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5009ab40 con 0x5614e6179080 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 -- 172.21.15.7:0/669883606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 msgr2=0x5614e6189ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 --2- 172.21.15.7:0/669883606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e6189ee0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7efc5009b680 tx=0x7efc50086040 comp rx=0 tx=0).stop 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 -- 172.21.15.7:0/669883606 shutdown_connections 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 --2- 172.21.15.7:0/669883606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 0x5614e6195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 --2- 172.21.15.7:0/669883606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e6189ee0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 --2- 172.21.15.7:0/669883606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e6195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 -- 172.21.15.7:0/669883606 >> 172.21.15.7:0/669883606 conn(0x5614e6173c00 msgr2=0x5614e61776f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 -- 172.21.15.7:0/669883606 shutdown_connections 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 -- 172.21.15.7:0/669883606 wait complete. 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 -- start start 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 Processor -- start 2024-03-26T08:51:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e6187d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e61882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 0x5614e6189220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.139+0000 7efc63db6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4800c9a0 con 0x5614e617e1f0 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc63db6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4800dcc0 con 0x5614e6179080 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc63db6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4800d000 con 0x5614e6178690 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc63db6740 10 client.0 _ll_register_callbacks cb 0x7efc4c02cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e61882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 0x5614e6189220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e61882d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37466/0 (socket says 172.21.15.7:37466) 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5a59c640 1 -- 172.21.15.7:0/2409329574 learned_addr learned my addr 172.21.15.7:0/2409329574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5ad9d640 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e6187d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5a59c640 1 -- 172.21.15.7:0/2409329574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 msgr2=0x5614e6187d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5a59c640 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e6187d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5a59c640 1 -- 172.21.15.7:0/2409329574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 msgr2=0x5614e6189220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5a59c640 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 0x5614e6189220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5a59c640 1 -- 172.21.15.7:0/2409329574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614e6189810 con 0x5614e6179080 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc5a59c640 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e61882d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efc500c5010 tx=0x7efc50009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac00fdca3a31a63 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc3eff5640 1 -- 172.21.15.7:0/2409329574 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5009b6f0 con 0x5614e6179080 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc3eff5640 1 -- 172.21.15.7:0/2409329574 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efc50097d60 con 0x5614e6179080 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.140+0000 7efc63db6740 1 -- 172.21.15.7:0/2409329574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc500999f0 con 0x5614e6179080 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc3eff5640 1 -- 172.21.15.7:0/2409329574 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc500cc8b0 con 0x5614e6179080 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc63db6740 10 client.27653 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc63db6740 1 -- 172.21.15.7:0/2409329574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5614e61ea970 con 0x5614e6179080 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc3eff5640 1 -- 172.21.15.7:0/2409329574 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc500999f0 con 0x5614e6179080 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc3eff5640 1 -- 172.21.15.7:0/2409329574 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7efc500b41a0 con 0x5614e6179080 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc63db6740 10 client.27653 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc63db6740 10 client.27653 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc63db6740 20 client.27653 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc63db6740 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5614e61eb270 0x5614e61ed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc63db6740 4 client.27653 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc63db6740 1 -- 172.21.15.7:0/2409329574 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614e60e5c50 con 0x5614e61eb270 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.141+0000 7efc5ad9d640 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5614e61eb270 0x5614e61ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.142+0000 7efc5ad9d640 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5614e61eb270 0x5614e61ed690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc719ec4dd1a268e server_cookie=65841e5df3bc918c in_seq=0 out_seq=0 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.142+0000 7efc3eff5640 10 client.27653 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.142+0000 7efc3eff5640 1 -- 172.21.15.7:0/2409329574 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614e60e5c50 con 0x5614e61eb270 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.142+0000 7efc3eff5640 10 client.27653 handle_command_reply: tid=0 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.142+0000 7efc63db6740 1 client.27653 shutdown 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.142+0000 7efc63db6740 1 -- 172.21.15.7:0/2409329574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 msgr2=0x5614e61882d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.142+0000 7efc63db6740 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e61882d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efc500c5010 tx=0x7efc50009700 comp rx=0 tx=0).stop 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.143+0000 7efc63db6740 1 -- 172.21.15.7:0/2409329574 shutdown_connections 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.143+0000 7efc63db6740 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614e617e1f0 0x5614e6189220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.143+0000 7efc63db6740 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614e6179080 0x5614e61882d0 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efc500c5010 tx=0x7efc50009700 comp rx=0 tx=0).stop 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.143+0000 7efc63db6740 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5614e61eb270 0x5614e61ed690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.143+0000 7efc63db6740 1 --2- 172.21.15.7:0/2409329574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5614e6178690 0x5614e6187d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.143+0000 7efc63db6740 1 -- 172.21.15.7:0/2409329574 >> 172.21.15.7:0/2409329574 conn(0x5614e6173c00 msgr2=0x5614e6172590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.143+0000 7efc3eff5640 0 client.27653 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:51:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.143+0000 7efc63db6740 1 -- 172.21.15.7:0/2409329574 shutdown_connections 2024-03-26T08:51:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.143+0000 7efc63db6740 1 -- 172.21.15.7:0/2409329574 wait complete. 2024-03-26T08:51:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc63db6740 20 client.27653 trim_cache size 0 max 16384 2024-03-26T08:51:35.144 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:51:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc611d3640 1 -- 172.21.15.7:0/1668117063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc440a28d0 msgr2=0x7efc440a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc611d3640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc440a28d0 0x7efc440a4d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efc5400f8c0 tx=0x7efc5401f000 comp rx=0 tx=0).stop 2024-03-26T08:51:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc611d3640 1 -- 172.21.15.7:0/1668117063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5c101290 msgr2=0x7efc5c19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc611d3640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5c101290 0x7efc5c19a330 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e9f0 tx=0x7efc5000eeb0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc611d3640 1 -- 172.21.15.7:0/1668117063 shutdown_connections 2024-03-26T08:51:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc611d3640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc440a28d0 0x7efc440a4d90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc611d3640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efc5c19f390 0x7efc5c1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc611d3640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5c101290 0x7efc5c19a330 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.144+0000 7efc611d3640 1 --2- 172.21.15.7:0/1668117063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c1008f0 0x7efc5c199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.145+0000 7efc611d3640 1 -- 172.21.15.7:0/1668117063 >> 172.21.15.7:0/1668117063 conn(0x7efc5c0fbb30 msgr2=0x7efc5c0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.145+0000 7efc611d3640 1 -- 172.21.15.7:0/1668117063 shutdown_connections 2024-03-26T08:51:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.145+0000 7efc611d3640 1 -- 172.21.15.7:0/1668117063 wait complete. 2024-03-26T08:51:35.154 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:51:35.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:51:35.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.317+0000 7f3685ffb640 1 -- 172.21.15.7:0/3014234302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368809d360 con 0x7f3680096810 2024-03-26T08:51:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.318+0000 7f3687fff640 1 -- 172.21.15.7:0/3014234302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3680096810 msgr2=0x7f368009d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.318+0000 7f3687fff640 1 --2- 172.21.15.7:0/3014234302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3680096810 0x7f368009d0a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3688067bb0 tx=0x7f36880976e0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.318+0000 7f3687fff640 1 -- 172.21.15.7:0/3014234302 shutdown_connections 2024-03-26T08:51:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.318+0000 7f3687fff640 1 --2- 172.21.15.7:0/3014234302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3680096810 0x7f368009d0a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.318+0000 7f3687fff640 1 --2- 172.21.15.7:0/3014234302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3680095e50 0x7f36800962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.318+0000 7f3687fff640 1 --2- 172.21.15.7:0/3014234302 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680094c60 0x7f3680095080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.318+0000 7f3687fff640 1 -- 172.21.15.7:0/3014234302 >> 172.21.15.7:0/3014234302 conn(0x7f36800901d0 msgr2=0x7f3680092610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.318+0000 7f3687fff640 1 -- 172.21.15.7:0/3014234302 shutdown_connections 2024-03-26T08:51:35.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.318+0000 7f3687fff640 1 -- 172.21.15.7:0/3014234302 wait complete. 2024-03-26T08:51:35.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.319+0000 7f3687fff640 1 Processor -- start 2024-03-26T08:51:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.319+0000 7f3687fff640 1 -- start start 2024-03-26T08:51:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.319+0000 7f3687fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3680094c60 0x7f36800a3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.319+0000 7f3687fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680095e50 0x7f36800a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.319+0000 7f3687fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3680096810 0x7f368009fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.319+0000 7f3687fff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800a3c00 con 0x7f3680094c60 2024-03-26T08:51:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.319+0000 7f3687fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368009f5c0 con 0x7f3680096810 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.319+0000 7f3687fff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800a3d80 con 0x7f3680095e50 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3680096810 0x7f368009fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3680096810 0x7f368009fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37486/0 (socket says 172.21.15.7:37486) 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36877fe640 1 -- 172.21.15.7:0/3532167864 learned_addr learned my addr 172.21.15.7:0/3532167864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3680094c60 0x7f36800a3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36867fc640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680095e50 0x7f36800a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36877fe640 1 -- 172.21.15.7:0/3532167864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680095e50 msgr2=0x7f36800a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36877fe640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680095e50 0x7f36800a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36877fe640 1 -- 172.21.15.7:0/3532167864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3680094c60 msgr2=0x7f36800a3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36877fe640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3680094c60 0x7f36800a3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36877fe640 1 -- 172.21.15.7:0/3532167864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800a0590 con 0x7f3680096810 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f36877fe640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3680096810 0x7f368009fcd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3688050670 tx=0x7f3688069880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f366ffff640 1 -- 172.21.15.7:0/3532167864 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880a3040 con 0x7f3680096810 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f3687fff640 1 -- 172.21.15.7:0/3532167864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368013a540 con 0x7f3680096810 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.320+0000 7f3687fff640 1 -- 172.21.15.7:0/3532167864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368013a940 con 0x7f3680096810 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.321+0000 7f366ffff640 1 -- 172.21.15.7:0/3532167864 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36880a7af0 con 0x7f3680096810 2024-03-26T08:51:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.321+0000 7f366ffff640 1 -- 172.21.15.7:0/3532167864 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880aeb00 con 0x7f3680096810 2024-03-26T08:51:35.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.323+0000 7f368fe82740 1 Processor -- start 2024-03-26T08:51:35.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.323+0000 7f366ffff640 1 -- 172.21.15.7:0/3532167864 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f36880699c0 con 0x7f3680096810 2024-03-26T08:51:35.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.323+0000 7f368fe82740 1 -- start start 2024-03-26T08:51:35.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.323+0000 7f368fe82740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cb1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.323+0000 7f368fe82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cb6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.323+0000 7f368fe82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cb9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.323+0000 7f368fe82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a2c74610 con 0x55b2a2cb7400 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.323+0000 7f368fe82740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a2c173c0 con 0x55b2a2cb18a0 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.323+0000 7f368fe82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a2bbfd60 con 0x55b2a2cb2290 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cb6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cb6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49648/0 (socket says 172.21.15.7:49648) 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f3686ffd640 1 -- 172.21.15.7:0/2939008816 learned_addr learned my addr 172.21.15.7:0/2939008816 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f36867fc640 1 --2- 172.21.15.7:0/2939008816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cb1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f36867fc640 1 --2- 172.21.15.7:0/2939008816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cb9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f3686ffd640 1 -- 172.21.15.7:0/2939008816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb18a0 msgr2=0x55b2a2cb1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f3686ffd640 1 --2- 172.21.15.7:0/2939008816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cb1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f3686ffd640 1 -- 172.21.15.7:0/2939008816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 msgr2=0x55b2a2cb9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f3686ffd640 1 --2- 172.21.15.7:0/2939008816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cb9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f3686ffd640 1 -- 172.21.15.7:0/2939008816 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a292b3f0 con 0x55b2a2cb2290 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f366ffff640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2850 0x7f36700a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f36867fc640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2850 0x7f36700a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f3686ffd640 1 --2- 172.21.15.7:0/2939008816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cb6d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f367c00abf0 tx=0x7f367c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81916c5bde7e7565 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.324+0000 7f366ffff640 1 -- 172.21.15.7:0/3532167864 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f368815be50 con 0x7f3680096810 2024-03-26T08:51:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.325+0000 7f366d7fa640 1 -- 172.21.15.7:0/2939008816 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c004220 con 0x55b2a2cb2290 2024-03-26T08:51:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.325+0000 7f366d7fa640 1 -- 172.21.15.7:0/2939008816 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f367c0043c0 con 0x55b2a2cb2290 2024-03-26T08:51:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.325+0000 7f366d7fa640 1 -- 172.21.15.7:0/2939008816 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c018900 con 0x55b2a2cb2290 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.325+0000 7f368fe82740 1 -- 172.21.15.7:0/2939008816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 msgr2=0x55b2a2cb6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.325+0000 7f36867fc640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2850 0x7f36700a4d10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55b2a2cb2af0 tx=0x7f367800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.325+0000 7f368fe82740 1 --2- 172.21.15.7:0/2939008816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cb6d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f367c00abf0 tx=0x7f367c00f790 comp rx=0 tx=0).stop 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 -- 172.21.15.7:0/2939008816 shutdown_connections 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 --2- 172.21.15.7:0/2939008816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cb9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 --2- 172.21.15.7:0/2939008816 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cb6d90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 --2- 172.21.15.7:0/2939008816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cb1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 -- 172.21.15.7:0/2939008816 >> 172.21.15.7:0/2939008816 conn(0x55b2a2cace10 msgr2=0x55b2a2cad210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 -- 172.21.15.7:0/2939008816 shutdown_connections 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 -- 172.21.15.7:0/2939008816 wait complete. 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 -- start start 2024-03-26T08:51:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 Processor -- start 2024-03-26T08:51:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cbbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.326+0000 7f368fe82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cbc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.327+0000 7f368fe82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cc31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.327+0000 7f368fe82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a2bd3240 con 0x55b2a2cb7400 2024-03-26T08:51:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.327+0000 7f368fe82740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a2c13460 con 0x55b2a2cb18a0 2024-03-26T08:51:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.327+0000 7f368fe82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2a2cba3b0 con 0x55b2a2cb2290 2024-03-26T08:51:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.327+0000 7f368fe82740 10 client.0 _ll_register_callbacks cb 0x55b2a2bd2c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.327+0000 7f36867fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cbc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.327+0000 7f36867fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cbc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49660/0 (socket says 172.21.15.7:49660) 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.327+0000 7f36867fc640 1 -- 172.21.15.7:0/29298762 learned_addr learned my addr 172.21.15.7:0/29298762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.328+0000 7f36877fe640 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cc31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.328+0000 7f36867fc640 1 -- 172.21.15.7:0/29298762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb18a0 msgr2=0x55b2a2cbbc40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.328+0000 7f36867fc640 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cbbc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.328+0000 7f36867fc640 1 -- 172.21.15.7:0/29298762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 msgr2=0x55b2a2cc31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.328+0000 7f36867fc640 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cc31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.328+0000 7f36867fc640 1 -- 172.21.15.7:0/29298762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2a29f57d0 con 0x55b2a2cb2290 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.328+0000 7f36877fe640 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cc31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.329+0000 7f36867fc640 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cbc180 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f367800ab90 tx=0x7f3678004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e355e9df2d8672c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.329+0000 7f366aff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:51:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.329+0000 7f366aff5640 1 -- 172.21.15.7:0/29298762 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367803e060 con 0x55b2a2cb2290 2024-03-26T08:51:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.329+0000 7f366aff5640 1 -- 172.21.15.7:0/29298762 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3678011050 con 0x55b2a2cb2290 2024-03-26T08:51:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.329+0000 7f368fe82740 1 -- 172.21.15.7:0/29298762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f367c0043c0 con 0x55b2a2cb2290 2024-03-26T08:51:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f366aff5640 1 -- 172.21.15.7:0/29298762 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3678011250 con 0x55b2a2cb2290 2024-03-26T08:51:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f368fe82740 10 client.27667 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f368fe82740 1 -- 172.21.15.7:0/29298762 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b2a2cc65d0 con 0x55b2a2cb2290 2024-03-26T08:51:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f366aff5640 1 -- 172.21.15.7:0/29298762 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f367c0043c0 con 0x55b2a2cb2290 2024-03-26T08:51:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f366aff5640 1 -- 172.21.15.7:0/29298762 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3678043070 con 0x55b2a2cb2290 2024-03-26T08:51:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f368fe82740 10 client.27667 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f368fe82740 10 client.27667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f368fe82740 20 client.27667 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f368fe82740 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b2a2cc7cd0 0x55b2a2cca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f368fe82740 4 client.27667 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f368fe82740 1 -- 172.21.15.7:0/29298762 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2a2c24900 con 0x55b2a2cc7cd0 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.330+0000 7f3686ffd640 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b2a2cc7cd0 0x55b2a2cca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.331+0000 7f366aff5640 10 client.27667 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.331+0000 7f3686ffd640 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b2a2cc7cd0 0x55b2a2cca190 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4aeaedb7e0e8f server_cookie=b94d472e1bc62a35 in_seq=0 out_seq=0 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f366aff5640 1 -- 172.21.15.7:0/29298762 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b2a2c24900 con 0x55b2a2cc7cd0 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f366aff5640 10 client.27667 handle_command_reply: tid=0 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 client.27667 shutdown 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 -- 172.21.15.7:0/29298762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 msgr2=0x55b2a2cbc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cbc180 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f367800ab90 tx=0x7f3678004730 comp rx=0 tx=0).stop 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 -- 172.21.15.7:0/29298762 shutdown_connections 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b2a2cc7cd0 0x55b2a2cca190 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cc31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cbc180 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 --2- 172.21.15.7:0/29298762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cbbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 -- 172.21.15.7:0/29298762 >> 172.21.15.7:0/29298762 conn(0x55b2a2cace10 msgr2=0x55b2a2cac070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f366aff5640 0 client.27667 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 -- 172.21.15.7:0/29298762 shutdown_connections 2024-03-26T08:51:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 1 -- 172.21.15.7:0/29298762 wait complete. 2024-03-26T08:51:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.332+0000 7f368fe82740 20 client.27667 trim_cache size 0 max 16384 2024-03-26T08:51:35.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f368fe82740 1 Processor -- start 2024-03-26T08:51:35.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f368fe82740 1 -- start start 2024-03-26T08:51:35.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f368fe82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cb0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f368fe82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f368fe82740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb7400 0x55b2a2cba7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f368fe82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674010980 con 0x55b2a2cb18a0 2024-03-26T08:51:35.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f368fe82740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367400e480 con 0x55b2a2cb7400 2024-03-26T08:51:35.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f368fe82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367400c500 con 0x55b2a2cb2290 2024-03-26T08:51:35.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cb0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f36867fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f36867fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49664/0 (socket says 172.21.15.7:49664) 2024-03-26T08:51:35.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f36867fc640 1 -- 172.21.15.7:0/1211959373 learned_addr learned my addr 172.21.15.7:0/1211959373 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.349+0000 7f36877fe640 1 --2- 172.21.15.7:0/1211959373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb7400 0x55b2a2cba7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.350+0000 7f36867fc640 1 -- 172.21.15.7:0/1211959373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb7400 msgr2=0x55b2a2cba7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.350+0000 7f36867fc640 1 --2- 172.21.15.7:0/1211959373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb7400 0x55b2a2cba7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.350+0000 7f36867fc640 1 -- 172.21.15.7:0/1211959373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb18a0 msgr2=0x55b2a2cb0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.350+0000 7f36867fc640 1 --2- 172.21.15.7:0/1211959373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cb0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.350+0000 7f36867fc640 1 -- 172.21.15.7:0/1211959373 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2a2ca89d0 con 0x55b2a2cb2290 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.350+0000 7f36867fc640 1 --2- 172.21.15.7:0/1211959373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc3180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f367800ffd0 tx=0x7f367800af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f6d4745a10fde89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.350+0000 7f366cff9640 1 -- 172.21.15.7:0/1211959373 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678052070 con 0x55b2a2cb2290 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f366cff9640 1 -- 172.21.15.7:0/1211959373 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36780446e0 con 0x55b2a2cb2290 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f366cff9640 1 -- 172.21.15.7:0/1211959373 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678049030 con 0x55b2a2cb2290 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f368fe82740 1 -- 172.21.15.7:0/1211959373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 msgr2=0x55b2a2cc3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f368fe82740 1 --2- 172.21.15.7:0/1211959373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc3180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f367800ffd0 tx=0x7f367800af40 comp rx=0 tx=0).stop 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f368fe82740 1 -- 172.21.15.7:0/1211959373 shutdown_connections 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f368fe82740 1 --2- 172.21.15.7:0/1211959373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb7400 0x55b2a2cba7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f368fe82740 1 --2- 172.21.15.7:0/1211959373 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc3180 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f368fe82740 1 --2- 172.21.15.7:0/1211959373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cb0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f368fe82740 1 -- 172.21.15.7:0/1211959373 >> 172.21.15.7:0/1211959373 conn(0x55b2a2cace10 msgr2=0x55b2a2cb0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f368fe82740 1 -- 172.21.15.7:0/1211959373 shutdown_connections 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.351+0000 7f368fe82740 1 -- 172.21.15.7:0/1211959373 wait complete. 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.352+0000 7f368fe82740 1 -- start start 2024-03-26T08:51:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.352+0000 7f368fe82740 1 Processor -- start 2024-03-26T08:51:35.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.352+0000 7f368fe82740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cc0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.352+0000 7f368fe82740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.352+0000 7f368fe82740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cc2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.352+0000 7f368fe82740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367400cb00 con 0x55b2a2cb7400 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.352+0000 7f368fe82740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367400de20 con 0x55b2a2cb2290 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.352+0000 7f368fe82740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367400d160 con 0x55b2a2cb18a0 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.352+0000 7f368fe82740 10 client.0 _ll_register_callbacks cb 0x7f3678011cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cc0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f3686ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cc0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49670/0 (socket says 172.21.15.7:49670) 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f3686ffd640 1 -- 172.21.15.7:0/2028257382 learned_addr learned my addr 172.21.15.7:0/2028257382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f36867fc640 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f36877fe640 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cc2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f3686ffd640 1 -- 172.21.15.7:0/2028257382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb2290 msgr2=0x55b2a2cc11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f3686ffd640 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f3686ffd640 1 -- 172.21.15.7:0/2028257382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 msgr2=0x55b2a2cc2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f3686ffd640 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cc2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f3686ffd640 1 -- 172.21.15.7:0/2028257382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2a2cc2720 con 0x55b2a2cb18a0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f36867fc640 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f3686ffd640 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cc0ca0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f367c01a040 tx=0x7f367c005aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=175250ee5d35ca04 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.353+0000 7f366aff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f366aff5640 1 -- 172.21.15.7:0/2028257382 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c003fd0 con 0x55b2a2cb18a0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f366aff5640 1 -- 172.21.15.7:0/2028257382 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f367c004170 con 0x55b2a2cb18a0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f368fe82740 1 -- 172.21.15.7:0/2028257382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36780446e0 con 0x55b2a2cb18a0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f366aff5640 1 -- 172.21.15.7:0/2028257382 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f367c036050 con 0x55b2a2cb18a0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f368fe82740 10 client.27673 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f368fe82740 1 -- 172.21.15.7:0/2028257382 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b2a2d23990 con 0x55b2a2cb18a0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f366aff5640 1 -- 172.21.15.7:0/2028257382 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f36780446e0 con 0x55b2a2cb18a0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f366aff5640 1 -- 172.21.15.7:0/2028257382 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f367c005140 con 0x55b2a2cb18a0 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f368fe82740 10 client.27673 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f368fe82740 10 client.27673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f368fe82740 20 client.27673 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:35.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f368fe82740 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b2a2d24080 0x55b2a2d264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f368fe82740 4 client.27673 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.354+0000 7f368fe82740 1 -- 172.21.15.7:0/2028257382 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b2a2c1ec50 con 0x55b2a2d24080 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f36867fc640 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b2a2d24080 0x55b2a2d264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f366aff5640 10 client.27673 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f36867fc640 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b2a2d24080 0x55b2a2d264a0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c81b29d4be4532ac server_cookie=8486c8324769c7ef in_seq=0 out_seq=0 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f366aff5640 1 -- 172.21.15.7:0/2028257382 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b2a2c1ec50 con 0x55b2a2d24080 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f366aff5640 10 client.27673 handle_command_reply: tid=0 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 client.27673 shutdown 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 -- 172.21.15.7:0/2028257382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb18a0 msgr2=0x55b2a2cc0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cc0ca0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f367c01a040 tx=0x7f367c005aa0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 -- 172.21.15.7:0/2028257382 shutdown_connections 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b2a2d24080 0x55b2a2d264a0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b2a2cb7400 0x55b2a2cc2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b2a2cb2290 0x55b2a2cc11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 --2- 172.21.15.7:0/2028257382 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b2a2cb18a0 0x55b2a2cc0ca0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f366aff5640 0 client.27673 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:35.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 -- 172.21.15.7:0/2028257382 >> 172.21.15.7:0/2028257382 conn(0x55b2a2cace10 msgr2=0x55b2a2cb0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 -- 172.21.15.7:0/2028257382 shutdown_connections 2024-03-26T08:51:35.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 1 -- 172.21.15.7:0/2028257382 wait complete. 2024-03-26T08:51:35.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.355+0000 7f368fe82740 20 client.27673 trim_cache size 0 max 16384 2024-03-26T08:51:35.357 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:51:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 -- 172.21.15.7:0/3532167864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2850 msgr2=0x7f36700a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2850 0x7f36700a4d10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55b2a2cb2af0 tx=0x7f367800a2d0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 -- 172.21.15.7:0/3532167864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3680096810 msgr2=0x7f368009fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3680096810 0x7f368009fcd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3688050670 tx=0x7f3688069880 comp rx=0 tx=0).stop 2024-03-26T08:51:35.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 -- 172.21.15.7:0/3532167864 shutdown_connections 2024-03-26T08:51:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36700a2850 0x7f36700a4d10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3680096810 0x7f368009fcd0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3680095e50 0x7f36800a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 --2- 172.21.15.7:0/3532167864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3680094c60 0x7f36800a3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 -- 172.21.15.7:0/3532167864 >> 172.21.15.7:0/3532167864 conn(0x7f36800901d0 msgr2=0x7f3680099150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.356+0000 7f3687fff640 1 -- 172.21.15.7:0/3532167864 shutdown_connections 2024-03-26T08:51:35.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:35.357+0000 7f3687fff640 1 -- 172.21.15.7:0/3532167864 wait complete. 2024-03-26T08:51:35.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:35 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2842956242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:51:35.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:35 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2842956242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:51:35.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:35 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2842956242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:51:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 545.0 s, 2492.2 tps, lat 40.144 ms stddev 47.728 2024-03-26T08:51:36.370 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:51:36.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:51:36.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:51:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.513+0000 7f20d586a640 1 -- 172.21.15.7:0/1034764964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d0108730 msgr2=0x7f20d010ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.513+0000 7f20d586a640 1 --2- 172.21.15.7:0/1034764964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d0108730 0x7f20d010ab20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009370 tx=0x7f20c002bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:36.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.513+0000 7f20d586a640 1 -- 172.21.15.7:0/1034764964 shutdown_connections 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.513+0000 7f20d586a640 1 --2- 172.21.15.7:0/1034764964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d010db70 0x7f20d0110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.513+0000 7f20d586a640 1 --2- 172.21.15.7:0/1034764964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d010b0f0 0x7f20d010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.513+0000 7f20d586a640 1 --2- 172.21.15.7:0/1034764964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d0108730 0x7f20d010ab20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.513+0000 7f20d586a640 1 -- 172.21.15.7:0/1034764964 >> 172.21.15.7:0/1034764964 conn(0x7f20d006e6a0 msgr2=0x7f20d0070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.513+0000 7f20d586a640 1 -- 172.21.15.7:0/1034764964 shutdown_connections 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.513+0000 7f20d586a640 1 -- 172.21.15.7:0/1034764964 wait complete. 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d586a640 1 Processor -- start 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d586a640 1 -- start start 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d586a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d0108730 0x7f20d01a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d586a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d010b0f0 0x7f20d01a2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d586a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d010db70 0x7f20d01a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d586a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d01137b0 con 0x7f20d010db70 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d586a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0113630 con 0x7f20d010b0f0 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d586a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0113930 con 0x7f20d0108730 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d5069640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d010db70 0x7f20d01a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d5069640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d010db70 0x7f20d01a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41190/0 (socket says 172.21.15.7:41190) 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d5069640 1 -- 172.21.15.7:0/3190559460 learned_addr learned my addr 172.21.15.7:0/3190559460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20cffff640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d010b0f0 0x7f20d01a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.514+0000 7f20d4868640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d0108730 0x7f20d01a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20d5069640 1 -- 172.21.15.7:0/3190559460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d0108730 msgr2=0x7f20d01a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20d5069640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d0108730 0x7f20d01a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20d5069640 1 -- 172.21.15.7:0/3190559460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d010b0f0 msgr2=0x7f20d01a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20d5069640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d010b0f0 0x7f20d01a2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20d5069640 1 -- 172.21.15.7:0/3190559460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d01a9e60 con 0x7f20d010db70 2024-03-26T08:51:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20cffff640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d010b0f0 0x7f20d01a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20d5069640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d010db70 0x7f20d01a97f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ec60 tx=0x7f20c800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20cdffb640 1 -- 172.21.15.7:0/3190559460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8004d20 con 0x7f20d010db70 2024-03-26T08:51:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20d586a640 1 -- 172.21.15.7:0/3190559460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d01aa150 con 0x7f20d010db70 2024-03-26T08:51:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.515+0000 7f20cdffb640 1 -- 172.21.15.7:0/3190559460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c800ba50 con 0x7f20d010db70 2024-03-26T08:51:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.516+0000 7f20cdffb640 1 -- 172.21.15.7:0/3190559460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8005740 con 0x7f20d010db70 2024-03-26T08:51:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.520+0000 7f20d586a640 1 -- 172.21.15.7:0/3190559460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d01aa6f0 con 0x7f20d010db70 2024-03-26T08:51:36.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.533+0000 7f20d978b740 1 Processor -- start 2024-03-26T08:51:36.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.534+0000 7f20d978b740 1 -- start start 2024-03-26T08:51:36.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.534+0000 7f20d978b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8180760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.534+0000 7f20d978b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8185830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.534+0000 7f20d978b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8188330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.534+0000 7f20d978b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc80b7c90 con 0x55bcc8180d30 2024-03-26T08:51:36.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.534+0000 7f20d978b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc80dd070 con 0x55bcc8185ea0 2024-03-26T08:51:36.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.534+0000 7f20d978b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc8085c80 con 0x55bcc8180340 2024-03-26T08:51:36.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.535+0000 7f20cdffb640 1 -- 172.21.15.7:0/3190559460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f20c80058e0 con 0x7f20d010db70 2024-03-26T08:51:36.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.535+0000 7f20cdffb640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f20b80a26e0 0x7f20b80a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.536+0000 7f20cdffb640 1 -- 172.21.15.7:0/3190559460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f20c8099110 con 0x7f20d010db70 2024-03-26T08:51:36.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.537+0000 7f20cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8180760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.537+0000 7f20cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8188330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.539+0000 7f20cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8180760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49706/0 (socket says 172.21.15.7:49706) 2024-03-26T08:51:36.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.539+0000 7f20cffff640 1 -- 172.21.15.7:0/2077707625 learned_addr learned my addr 172.21.15.7:0/2077707625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.540+0000 7f20d4868640 1 --2- 172.21.15.7:0/2077707625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8185830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.540+0000 7f20d5069640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f20b80a26e0 0x7f20b80a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.543+0000 7f20d4868640 1 -- 172.21.15.7:0/2077707625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 msgr2=0x55bcc8180760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.543+0000 7f20d4868640 1 --2- 172.21.15.7:0/2077707625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8180760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.543+0000 7f20d4868640 1 -- 172.21.15.7:0/2077707625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 msgr2=0x55bcc8188330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.543+0000 7f20d4868640 1 --2- 172.21.15.7:0/2077707625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8188330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.543+0000 7f20d4868640 1 -- 172.21.15.7:0/2077707625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcc807dd10 con 0x55bcc8180d30 2024-03-26T08:51:36.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.543+0000 7f20d4868640 1 --2- 172.21.15.7:0/2077707625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8185830 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f20c002c170 tx=0x7f20c0009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5537f65c453fea40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.544+0000 7f20b6ffd640 1 -- 172.21.15.7:0/2077707625 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c002cb10 con 0x55bcc8180d30 2024-03-26T08:51:36.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.544+0000 7f20b6ffd640 1 -- 172.21.15.7:0/2077707625 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c002c5d0 con 0x55bcc8180d30 2024-03-26T08:51:36.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.545+0000 7f20d978b740 1 -- 172.21.15.7:0/2077707625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 msgr2=0x55bcc8185830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.545+0000 7f20d978b740 1 --2- 172.21.15.7:0/2077707625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8185830 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f20c002c170 tx=0x7f20c0009df0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.549+0000 7f20d978b740 1 -- 172.21.15.7:0/2077707625 shutdown_connections 2024-03-26T08:51:36.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.549+0000 7f20d978b740 1 --2- 172.21.15.7:0/2077707625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8188330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.549+0000 7f20d978b740 1 --2- 172.21.15.7:0/2077707625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8185830 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.549+0000 7f20d978b740 1 --2- 172.21.15.7:0/2077707625 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8180760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.549+0000 7f20d978b740 1 -- 172.21.15.7:0/2077707625 >> 172.21.15.7:0/2077707625 conn(0x55bcc81401a0 msgr2=0x55bcc817bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.550+0000 7f20d5069640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f20b80a26e0 0x7f20b80a4ba0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f20c800cca0 tx=0x7f20c800ce50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:36 smithi007 ceph-mon[32462]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 628 op/s 2024-03-26T08:51:36.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.552+0000 7f20d978b740 1 -- 172.21.15.7:0/2077707625 shutdown_connections 2024-03-26T08:51:36.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.557+0000 7f20d978b740 1 -- 172.21.15.7:0/2077707625 wait complete. 2024-03-26T08:51:36.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.557+0000 7f20d978b740 1 -- start start 2024-03-26T08:51:36.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.557+0000 7f20d978b740 1 Processor -- start 2024-03-26T08:51:36.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.561+0000 7f20d978b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc818a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.561+0000 7f20d978b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc818abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.561+0000 7f20d978b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.561+0000 7f20d978b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc80990d0 con 0x55bcc8180d30 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.561+0000 7f20d978b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc80d9110 con 0x55bcc8185ea0 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.561+0000 7f20d978b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcc8188e40 con 0x55bcc8180340 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.561+0000 7f20d978b740 10 client.0 _ll_register_callbacks cb 0x55bcc80d4e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.562+0000 7f20cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc818a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.562+0000 7f20cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.562+0000 7f20cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc818a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:52988/0 (socket says 172.21.15.7:52988) 2024-03-26T08:51:36.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.562+0000 7f20cffff640 1 -- 172.21.15.7:0/1893157892 learned_addr learned my addr 172.21.15.7:0/1893157892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.564+0000 7f20d4868640 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc818abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.566+0000 7f20cffff640 1 -- 172.21.15.7:0/1893157892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 msgr2=0x55bcc818a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.566+0000 7f20cffff640 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc818a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.566+0000 7f20cffff640 1 -- 172.21.15.7:0/1893157892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 msgr2=0x55bcc818abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.566+0000 7f20cffff640 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc818abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.566+0000 7f20cffff640 1 -- 172.21.15.7:0/1893157892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcc7eb97e0 con 0x55bcc8185ea0 2024-03-26T08:51:36.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.569+0000 7f20cffff640 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191c30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f20c4013b60 tx=0x7f20c4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d94c3a26d1d76f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.570+0000 7f20affff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:36.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:36 smithi044 ceph-mon[36461]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 628 op/s 2024-03-26T08:51:36.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.574+0000 7f20affff640 1 -- 172.21.15.7:0/1893157892 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4013e60 con 0x55bcc8185ea0 2024-03-26T08:51:36.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.574+0000 7f20d978b740 1 -- 172.21.15.7:0/1893157892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20c002c5d0 con 0x55bcc8185ea0 2024-03-26T08:51:36.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.574+0000 7f20affff640 1 -- 172.21.15.7:0/1893157892 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f20c4005020 con 0x55bcc8185ea0 2024-03-26T08:51:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.578+0000 7f20affff640 1 -- 172.21.15.7:0/1893157892 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20c4005220 con 0x55bcc8185ea0 2024-03-26T08:51:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.580+0000 7f20d978b740 10 client.27677 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.580+0000 7f20d978b740 1 -- 172.21.15.7:0/1893157892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bcc7f19f70 con 0x55bcc8185ea0 2024-03-26T08:51:36.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.584+0000 7f20affff640 1 -- 172.21.15.7:0/1893157892 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c002c5d0 con 0x55bcc8185ea0 2024-03-26T08:51:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.584+0000 7f20affff640 1 -- 172.21.15.7:0/1893157892 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f20c402c580 con 0x55bcc8185ea0 2024-03-26T08:51:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.584+0000 7f20d978b740 10 client.27677 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.584+0000 7f20d978b740 10 client.27677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.584+0000 7f20d978b740 20 client.27677 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.584+0000 7f20d978b740 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bcc8196910 0x55bcc8198d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.584+0000 7f20d978b740 4 client.27677 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.584+0000 7f20d978b740 1 -- 172.21.15.7:0/1893157892 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcc80ea5b0 con 0x55bcc8196910 2024-03-26T08:51:36.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.585+0000 7f20d4868640 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bcc8196910 0x55bcc8198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.590+0000 7f20affff640 10 client.27677 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.590+0000 7f20d4868640 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bcc8196910 0x55bcc8198d70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e915f4a79e24e7 server_cookie=7725692715a11cbb in_seq=0 out_seq=0 2024-03-26T08:51:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.592+0000 7f20affff640 1 -- 172.21.15.7:0/1893157892 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bcc80ea5b0 con 0x55bcc8196910 2024-03-26T08:51:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.592+0000 7f20affff640 10 client.27677 handle_command_reply: tid=0 2024-03-26T08:51:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.592+0000 7f20d978b740 1 client.27677 shutdown 2024-03-26T08:51:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.593+0000 7f20d978b740 1 -- 172.21.15.7:0/1893157892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 msgr2=0x55bcc8191c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.593+0000 7f20d978b740 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191c30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f20c4013b60 tx=0x7f20c4005f90 comp rx=0 tx=0).stop 2024-03-26T08:51:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.593+0000 7f20d978b740 1 -- 172.21.15.7:0/1893157892 shutdown_connections 2024-03-26T08:51:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.593+0000 7f20d978b740 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bcc8196910 0x55bcc8198d70 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.593+0000 7f20d978b740 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191c30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.593+0000 7f20d978b740 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc818abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.593+0000 7f20d978b740 1 --2- 172.21.15.7:0/1893157892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc818a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.593+0000 7f20affff640 0 client.27677 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.593+0000 7f20d978b740 1 -- 172.21.15.7:0/1893157892 >> 172.21.15.7:0/1893157892 conn(0x55bcc81401a0 msgr2=0x55bcc81727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.594+0000 7f20d978b740 1 -- 172.21.15.7:0/1893157892 shutdown_connections 2024-03-26T08:51:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.594+0000 7f20d978b740 1 -- 172.21.15.7:0/1893157892 wait complete. 2024-03-26T08:51:36.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.594+0000 7f20d978b740 20 client.27677 trim_cache size 0 max 16384 2024-03-26T08:51:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.626+0000 7f20d978b740 1 Processor -- start 2024-03-26T08:51:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.626+0000 7f20d978b740 1 -- start start 2024-03-26T08:51:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.626+0000 7f20d978b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180340 0x55bcc8182190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.626+0000 7f20d978b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180d30 0x55bcc81826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.626+0000 7f20d978b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.626+0000 7f20d978b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d01b1490 con 0x55bcc8180340 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.626+0000 7f20d978b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d01aef90 con 0x55bcc8185ea0 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.626+0000 7f20d978b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d01ad010 con 0x55bcc8180d30 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180340 0x55bcc8182190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180340 0x55bcc8182190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56278/0 (socket says 172.21.15.7:56278) 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 -- 172.21.15.7:0/3616177683 learned_addr learned my addr 172.21.15.7:0/3616177683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 --2- 172.21.15.7:0/3616177683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20d4868640 1 --2- 172.21.15.7:0/3616177683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180d30 0x55bcc81826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 -- 172.21.15.7:0/3616177683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180d30 msgr2=0x55bcc81826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 --2- 172.21.15.7:0/3616177683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180d30 0x55bcc81826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 -- 172.21.15.7:0/3616177683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180340 msgr2=0x55bcc8182190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 --2- 172.21.15.7:0/3616177683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180340 0x55bcc8182190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 -- 172.21.15.7:0/3616177683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcc7db1170 con 0x55bcc8185ea0 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20cffff640 1 --2- 172.21.15.7:0/3616177683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191bc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f20c4011ca0 tx=0x7f20c4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a80eed563b46da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20b67fc640 1 -- 172.21.15.7:0/3616177683 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4032070 con 0x55bcc8185ea0 2024-03-26T08:51:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20b67fc640 1 -- 172.21.15.7:0/3616177683 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c4013920 con 0x55bcc8185ea0 2024-03-26T08:51:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.627+0000 7f20b67fc640 1 -- 172.21.15.7:0/3616177683 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c402c6d0 con 0x55bcc8185ea0 2024-03-26T08:51:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.630+0000 7f20d978b740 1 -- 172.21.15.7:0/3616177683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 msgr2=0x55bcc8191bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.630+0000 7f20d978b740 1 --2- 172.21.15.7:0/3616177683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191bc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f20c4011ca0 tx=0x7f20c4000bc0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.630+0000 7f20d978b740 1 -- 172.21.15.7:0/3616177683 shutdown_connections 2024-03-26T08:51:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.630+0000 7f20d978b740 1 --2- 172.21.15.7:0/3616177683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc8185ea0 0x55bcc8191bc0 secure :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f20c4011ca0 tx=0x7f20c4000bc0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.630+0000 7f20d978b740 1 --2- 172.21.15.7:0/3616177683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180d30 0x55bcc81826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.630+0000 7f20d978b740 1 --2- 172.21.15.7:0/3616177683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180340 0x55bcc8182190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.630+0000 7f20d978b740 1 -- 172.21.15.7:0/3616177683 >> 172.21.15.7:0/3616177683 conn(0x55bcc81401a0 msgr2=0x55bcc818a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.631+0000 7f20d978b740 1 -- 172.21.15.7:0/3616177683 shutdown_connections 2024-03-26T08:51:36.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.631+0000 7f20d978b740 1 -- 172.21.15.7:0/3616177683 wait complete. 2024-03-26T08:51:36.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.631+0000 7f20d978b740 1 -- start start 2024-03-26T08:51:36.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.631+0000 7f20d978b740 1 Processor -- start 2024-03-26T08:51:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20d978b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8191350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20d978b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8191890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20d978b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc818f960 0x55bcc818fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20d978b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d01ad610 con 0x55bcc8180d30 2024-03-26T08:51:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20d978b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d01ae930 con 0x55bcc818f960 2024-03-26T08:51:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20d978b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d01adc70 con 0x55bcc8180340 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8191350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20d4868640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8191890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20cffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8191350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53016/0 (socket says 172.21.15.7:53016) 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20cffff640 1 -- 172.21.15.7:0/4222049889 learned_addr learned my addr 172.21.15.7:0/4222049889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20cffff640 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc818f960 0x55bcc818fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.632+0000 7f20d978b740 10 client.0 _ll_register_callbacks cb 0x7f20c4013200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20cffff640 1 -- 172.21.15.7:0/4222049889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 msgr2=0x55bcc8191350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20cffff640 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8191350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20cffff640 1 -- 172.21.15.7:0/4222049889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 msgr2=0x55bcc8191890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20cffff640 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8191890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20cffff640 1 -- 172.21.15.7:0/4222049889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcc8190350 con 0x55bcc818f960 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20d4868640 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8191890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:36.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20cffff640 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc818f960 0x55bcc818fde0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f20c4013b60 tx=0x7f20c4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56891b012c77413 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20affff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20affff640 1 -- 172.21.15.7:0/4222049889 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4032030 con 0x55bcc818f960 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20affff640 1 -- 172.21.15.7:0/4222049889 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f20c40035f0 con 0x55bcc818f960 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.633+0000 7f20d978b740 1 -- 172.21.15.7:0/4222049889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20c4013920 con 0x55bcc818f960 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20affff640 1 -- 172.21.15.7:0/4222049889 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20c4005430 con 0x55bcc818f960 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20d978b740 10 client.27683 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20d978b740 1 -- 172.21.15.7:0/4222049889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bcc81f2f20 con 0x55bcc818f960 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20affff640 1 -- 172.21.15.7:0/4222049889 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c4013920 con 0x55bcc818f960 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20affff640 1 -- 172.21.15.7:0/4222049889 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f20c4047720 con 0x55bcc818f960 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20d978b740 10 client.27683 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20d978b740 10 client.27683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20d978b740 20 client.27683 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20d978b740 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bcc81f3740 0x55bcc81f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20d978b740 4 client.27683 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:51:36.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20d978b740 1 -- 172.21.15.7:0/4222049889 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bcc80e4900 con 0x55bcc81f3740 2024-03-26T08:51:36.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.634+0000 7f20d4868640 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bcc81f3740 0x55bcc81f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.635+0000 7f20affff640 10 client.27683 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:36.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.635+0000 7f20d4868640 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bcc81f3740 0x55bcc81f5b60 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac40d168f5fafb4 server_cookie=2597b893014f3d5a in_seq=0 out_seq=0 2024-03-26T08:51:36.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20affff640 1 -- 172.21.15.7:0/4222049889 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bcc80e4900 con 0x55bcc81f3740 2024-03-26T08:51:36.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20affff640 10 client.27683 handle_command_reply: tid=0 2024-03-26T08:51:36.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20d978b740 1 client.27683 shutdown 2024-03-26T08:51:36.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20d978b740 1 -- 172.21.15.7:0/4222049889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc818f960 msgr2=0x55bcc818fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20d978b740 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc818f960 0x55bcc818fde0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f20c4013b60 tx=0x7f20c4009b50 comp rx=0 tx=0).stop 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20d978b740 1 -- 172.21.15.7:0/4222049889 shutdown_connections 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20d978b740 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bcc81f3740 0x55bcc81f5b60 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20d978b740 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bcc818f960 0x55bcc818fde0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20affff640 0 client.27683 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20d978b740 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bcc8180d30 0x55bcc8191890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20d978b740 1 --2- 172.21.15.7:0/4222049889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bcc8180340 0x55bcc8191350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.636+0000 7f20d978b740 1 -- 172.21.15.7:0/4222049889 >> 172.21.15.7:0/4222049889 conn(0x55bcc81401a0 msgr2=0x55bcc818a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.637+0000 7f20d978b740 1 -- 172.21.15.7:0/4222049889 shutdown_connections 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.637+0000 7f20d978b740 1 -- 172.21.15.7:0/4222049889 wait complete. 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.637+0000 7f20d978b740 20 client.27683 trim_cache size 0 max 16384 2024-03-26T08:51:36.635 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:51:36.636 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:51:36.636 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "fad0c253-fb4f-4059-9002-b89c4087f74c", 2024-03-26T08:51:36.636 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:51:36.636 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:51:36.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 -- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f20b80a26e0 msgr2=0x7f20b80a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f20b80a26e0 0x7f20b80a4ba0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f20c800cca0 tx=0x7f20c800ce50 comp rx=0 tx=0).stop 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 -- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d010db70 msgr2=0x7f20d01a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d010db70 0x7f20d01a97f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ec60 tx=0x7f20c800c7a0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 -- 172.21.15.7:0/3190559460 shutdown_connections 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f20b80a26e0 0x7f20b80a4ba0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d010db70 0x7f20d01a97f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d010b0f0 0x7f20d01a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 --2- 172.21.15.7:0/3190559460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f20d0108730 0x7f20d01a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 -- 172.21.15.7:0/3190559460 >> 172.21.15.7:0/3190559460 conn(0x7f20d006e6a0 msgr2=0x7f20d0109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 -- 172.21.15.7:0/3190559460 shutdown_connections 2024-03-26T08:51:36.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.640+0000 7f20d586a640 1 -- 172.21.15.7:0/3190559460 wait complete. 2024-03-26T08:51:36.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:51:36.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:51:36.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:36 smithi016 ceph-mon[35851]: pgmap v635: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 628 op/s 2024-03-26T08:51:36.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.814+0000 7ff2fbf13640 1 -- 172.21.15.7:0/532570035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4104b10 msgr2=0x7ff2f4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.814+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/532570035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4104b10 0x7ff2f4106f00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009ca0 tx=0x7ff2ec02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.815+0000 7ff2fbf13640 1 -- 172.21.15.7:0/532570035 shutdown_connections 2024-03-26T08:51:36.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.815+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/532570035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f4107440 0x7ff2f4109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.815+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/532570035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4104b10 0x7ff2f4106f00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.815+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/532570035 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f41021e0 0x7ff2f41045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.815+0000 7ff2fbf13640 1 -- 172.21.15.7:0/532570035 >> 172.21.15.7:0/532570035 conn(0x7ff2f40fbb30 msgr2=0x7ff2f40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.815+0000 7ff2fbf13640 1 -- 172.21.15.7:0/532570035 shutdown_connections 2024-03-26T08:51:36.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.815+0000 7ff2fbf13640 1 -- 172.21.15.7:0/532570035 wait complete. 2024-03-26T08:51:36.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.815+0000 7ff2fbf13640 1 Processor -- start 2024-03-26T08:51:36.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.815+0000 7ff2fbf13640 1 -- start start 2024-03-26T08:51:36.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fbf13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f41021e0 0x7ff2f4101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fbf13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4104b10 0x7ff2f4100190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fbf13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4107440 0x7ff2f41006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fbf13640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f410d330 con 0x7ff2f4107440 2024-03-26T08:51:36.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fbf13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f410d1b0 con 0x7ff2f41021e0 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fbf13640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f410d4b0 con 0x7ff2f4104b10 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fa4b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4107440 0x7ff2f41006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fa4b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4107440 0x7ff2f41006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56298/0 (socket says 172.21.15.7:56298) 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/2859464459 learned_addr learned my addr 172.21.15.7:0/2859464459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2f9caf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f41021e0 0x7ff2f4101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2f94ae640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4104b10 0x7ff2f4100190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/2859464459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4104b10 msgr2=0x7ff2f4100190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4104b10 0x7ff2f4100190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/2859464459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f41021e0 msgr2=0x7ff2f4101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f41021e0 0x7ff2f4101b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/2859464459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4100f60 con 0x7ff2f4107440 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.816+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4107440 0x7ff2f41006d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400e970 tx=0x7ff2e400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.817+0000 7ff2eaffd640 1 -- 172.21.15.7:0/2859464459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e400ccc0 con 0x7ff2f4107440 2024-03-26T08:51:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.817+0000 7ff2eaffd640 1 -- 172.21.15.7:0/2859464459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e400ce60 con 0x7ff2f4107440 2024-03-26T08:51:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.818+0000 7ff2eaffd640 1 -- 172.21.15.7:0/2859464459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4010490 con 0x7ff2f4107440 2024-03-26T08:51:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.818+0000 7ff2fbf13640 1 -- 172.21.15.7:0/2859464459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f41a1b50 con 0x7ff2f4107440 2024-03-26T08:51:36.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.818+0000 7ff2fbf13640 1 -- 172.21.15.7:0/2859464459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f41a2020 con 0x7ff2f4107440 2024-03-26T08:51:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.821+0000 7ff2febd0740 1 Processor -- start 2024-03-26T08:51:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.821+0000 7ff2eaffd640 1 -- 172.21.15.7:0/2859464459 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff2e4005000 con 0x7ff2f4107440 2024-03-26T08:51:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.822+0000 7ff2febd0740 1 -- start start 2024-03-26T08:51:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.822+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b75b0 0x5634cc5b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.822+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5bcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.822+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5bd110 0x5634cc5bf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.822+0000 7ff2febd0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634cc5143d0 con 0x5634cc5b75b0 2024-03-26T08:51:36.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.822+0000 7ff2febd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634cc4bcad0 con 0x5634cc5b7fa0 2024-03-26T08:51:36.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.822+0000 7ff2febd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634cc510470 con 0x5634cc5bd110 2024-03-26T08:51:36.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f9caf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5bcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f94ae640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b75b0 0x5634cc5b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f9caf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5bcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36596/0 (socket says 172.21.15.7:36596) 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f9caf640 1 -- 172.21.15.7:0/1771692558 learned_addr learned my addr 172.21.15.7:0/1771692558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f94ae640 1 --2- 172.21.15.7:0/1771692558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5bd110 0x5634cc5bf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f9caf640 1 -- 172.21.15.7:0/1771692558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5bd110 msgr2=0x5634cc5bf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f9caf640 1 --2- 172.21.15.7:0/1771692558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5bd110 0x5634cc5bf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f9caf640 1 -- 172.21.15.7:0/1771692558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b75b0 msgr2=0x5634cc5b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f9caf640 1 --2- 172.21.15.7:0/1771692558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b75b0 0x5634cc5b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f9caf640 1 -- 172.21.15.7:0/1771692558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634cc2f27d0 con 0x5634cc5b7fa0 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2eaffd640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2e00a2900 0x7ff2e00a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.823+0000 7ff2f94ae640 1 --2- 172.21.15.7:0/1771692558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5bd110 0x5634cc5bf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.824+0000 7ff2eaffd640 1 -- 172.21.15.7:0/2859464459 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff2e40cef60 con 0x7ff2f4107440 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.824+0000 7ff2f9caf640 1 --2- 172.21.15.7:0/1771692558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5bcaa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009d20 tx=0x7ff2f0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a63194956a0072 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.824+0000 7ff2f94ae640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2e00a2900 0x7ff2e00a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.824+0000 7ff2dffff640 1 -- 172.21.15.7:0/1771692558 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f001e070 con 0x5634cc5b7fa0 2024-03-26T08:51:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.824+0000 7ff2dffff640 1 -- 172.21.15.7:0/1771692558 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2f0019b10 con 0x5634cc5b7fa0 2024-03-26T08:51:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.824+0000 7ff2dffff640 1 -- 172.21.15.7:0/1771692558 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000eb40 con 0x5634cc5b7fa0 2024-03-26T08:51:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.826+0000 7ff2f94ae640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2e00a2900 0x7ff2e00a4dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec02c630 tx=0x7ff2ec02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.826+0000 7ff2febd0740 1 -- 172.21.15.7:0/1771692558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 msgr2=0x5634cc5bcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.826+0000 7ff2febd0740 1 --2- 172.21.15.7:0/1771692558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5bcaa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009d20 tx=0x7ff2f0011810 comp rx=0 tx=0).stop 2024-03-26T08:51:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.826+0000 7ff2febd0740 1 -- 172.21.15.7:0/1771692558 shutdown_connections 2024-03-26T08:51:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.826+0000 7ff2febd0740 1 --2- 172.21.15.7:0/1771692558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5bd110 0x5634cc5bf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.826+0000 7ff2febd0740 1 --2- 172.21.15.7:0/1771692558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5bcaa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.826+0000 7ff2febd0740 1 --2- 172.21.15.7:0/1771692558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b75b0 0x5634cc5b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.826+0000 7ff2febd0740 1 -- 172.21.15.7:0/1771692558 >> 172.21.15.7:0/1771692558 conn(0x5634cc5aa310 msgr2=0x5634cc5aa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.827+0000 7ff2febd0740 1 -- 172.21.15.7:0/1771692558 shutdown_connections 2024-03-26T08:51:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.827+0000 7ff2febd0740 1 -- 172.21.15.7:0/1771692558 wait complete. 2024-03-26T08:51:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.827+0000 7ff2febd0740 1 -- start start 2024-03-26T08:51:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.827+0000 7ff2febd0740 1 Processor -- start 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5bd110 0x5634cc5b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2febd0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634cc5aeea0 con 0x5634cc5bd110 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2f9caf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2fa4b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5bd110 0x5634cc5b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2f9caf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53054/0 (socket says 172.21.15.7:53054) 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2f9caf640 1 -- 172.21.15.7:0/3884625640 learned_addr learned my addr 172.21.15.7:0/3884625640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2f94ae640 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634cc4d0060 con 0x5634cc5b7fa0 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634cc5af020 con 0x5634cc5b75b0 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2febd0740 10 client.0 _ll_register_callbacks cb 0x5634cc4b26a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2f9caf640 1 -- 172.21.15.7:0/3884625640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 msgr2=0x5634cc5b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2f9caf640 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2f9caf640 1 -- 172.21.15.7:0/3884625640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5bd110 msgr2=0x5634cc5b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2f9caf640 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5bd110 0x5634cc5b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2f9caf640 1 -- 172.21.15.7:0/3884625640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634cc2283f0 con 0x5634cc5b75b0 2024-03-26T08:51:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.828+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5bd110 0x5634cc5b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.829+0000 7ff2f9caf640 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5b0850 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009cf0 tx=0x7ff2f0019a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa523b7b504dec4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.829+0000 7ff2dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:51:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.829+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3884625640 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0027070 con 0x5634cc5b75b0 2024-03-26T08:51:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.829+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2f0019b10 con 0x5634cc5b75b0 2024-03-26T08:51:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.829+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3884625640 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff2f0005040 con 0x5634cc5b75b0 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.829+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3884625640 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2f0033050 con 0x5634cc5b75b0 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.829+0000 7ff2febd0740 10 client.27703 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.829+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5634cc1e80f0 con 0x5634cc5b75b0 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.830+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3884625640 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2f0019b10 con 0x5634cc5b75b0 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.830+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3884625640 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff2f00046c0 con 0x5634cc5b75b0 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.830+0000 7ff2febd0740 10 client.27703 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.830+0000 7ff2febd0740 10 client.27703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.830+0000 7ff2febd0740 20 client.27703 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.830+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634cc5d5eb0 0x5634cc5d8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.830+0000 7ff2febd0740 4 client.27703 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.830+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634cc521910 con 0x5634cc5d5eb0 2024-03-26T08:51:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.831+0000 7ff2f94ae640 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634cc5d5eb0 0x5634cc5d8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.831+0000 7ff2f94ae640 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634cc5d5eb0 0x5634cc5d8320 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d39f7cc5ccdd60 server_cookie=5bd2a18dd38fc1d5 in_seq=0 out_seq=0 2024-03-26T08:51:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.831+0000 7ff2dd7fa640 10 client.27703 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.832+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3884625640 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5634cc521910 con 0x5634cc5d5eb0 2024-03-26T08:51:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.832+0000 7ff2dd7fa640 10 client.27703 handle_command_reply: tid=0 2024-03-26T08:51:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.832+0000 7ff2febd0740 1 client.27703 shutdown 2024-03-26T08:51:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 msgr2=0x5634cc5b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5b0850 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009cf0 tx=0x7ff2f0019a70 comp rx=0 tx=0).stop 2024-03-26T08:51:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 shutdown_connections 2024-03-26T08:51:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634cc5d5eb0 0x5634cc5d8320 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5bd110 0x5634cc5b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b7fa0 0x5634cc5b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3884625640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5b0850 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2dd7fa640 0 client.27703 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 >> 172.21.15.7:0/3884625640 conn(0x5634cc5aa310 msgr2=0x5634cc5a9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 shutdown_connections 2024-03-26T08:51:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 1 -- 172.21.15.7:0/3884625640 wait complete. 2024-03-26T08:51:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.833+0000 7ff2febd0740 20 client.27703 trim_cache size 0 max 16384 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.858+0000 7ff2febd0740 1 Processor -- start 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2febd0740 1 -- start start 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5aa310 0x5634cc5a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5a9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2f9caf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5aa310 0x5634cc5a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2f9caf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5aa310 0x5634cc5a8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36618/0 (socket says 172.21.15.7:36618) 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2fa4b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2f94ae640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2fa4b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5a9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56324/0 (socket says 172.21.15.7:56324) 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2f9caf640 1 -- 172.21.15.7:0/2784998255 learned_addr learned my addr 172.21.15.7:0/2784998255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2febd0740 1 -- 172.21.15.7:0/2784998255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41a8e30 con 0x5634cc5b7fa0 2024-03-26T08:51:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2febd0740 1 -- 172.21.15.7:0/2784998255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41a6930 con 0x5634cc5aa310 2024-03-26T08:51:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.859+0000 7ff2febd0740 1 -- 172.21.15.7:0/2784998255 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41a49b0 con 0x5634cc5b75b0 2024-03-26T08:51:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/2784998255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 msgr2=0x5634cc5a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/2784998255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/2784998255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5aa310 msgr2=0x5634cc5a8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/2784998255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5aa310 0x5634cc5a8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/2784998255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634cc5a9c60 con 0x5634cc5b7fa0 2024-03-26T08:51:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2f94ae640 1 --2- 172.21.15.7:0/2784998255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/2784998255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5a9720 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c6a0 tx=0x7ff2e409bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03e6f1744e899cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2df7fe640 1 -- 172.21.15.7:0/2784998255 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40b3070 con 0x5634cc5b7fa0 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2df7fe640 1 -- 172.21.15.7:0/2784998255 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e4084e50 con 0x5634cc5b7fa0 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.860+0000 7ff2df7fe640 1 -- 172.21.15.7:0/2784998255 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e40052f0 con 0x5634cc5b7fa0 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.861+0000 7ff2febd0740 1 -- 172.21.15.7:0/2784998255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 msgr2=0x5634cc5a9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.861+0000 7ff2febd0740 1 --2- 172.21.15.7:0/2784998255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5a9720 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c6a0 tx=0x7ff2e409bfc0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.861+0000 7ff2febd0740 1 -- 172.21.15.7:0/2784998255 shutdown_connections 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.861+0000 7ff2febd0740 1 --2- 172.21.15.7:0/2784998255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5a9720 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.861+0000 7ff2febd0740 1 --2- 172.21.15.7:0/2784998255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5b75b0 0x5634cc5a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.861+0000 7ff2febd0740 1 --2- 172.21.15.7:0/2784998255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5aa310 0x5634cc5a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.861+0000 7ff2febd0740 1 -- 172.21.15.7:0/2784998255 >> 172.21.15.7:0/2784998255 conn(0x5634cc5b8d80 msgr2=0x5634cc5aa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.862+0000 7ff2febd0740 1 -- 172.21.15.7:0/2784998255 shutdown_connections 2024-03-26T08:51:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.862+0000 7ff2febd0740 1 -- 172.21.15.7:0/2784998255 wait complete. 2024-03-26T08:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.862+0000 7ff2febd0740 1 -- start start 2024-03-26T08:51:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.862+0000 7ff2febd0740 1 Processor -- start 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5aa310 0x5634cc5e7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b75b0 0x5634cc5e8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2febd0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5e6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2febd0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41a4fb0 con 0x5634cc5b7fa0 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2febd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41a62d0 con 0x5634cc5b75b0 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2febd0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41a5610 con 0x5634cc5aa310 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2f94ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b75b0 0x5634cc5e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2fa4b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2fa4b0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5e6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56338/0 (socket says 172.21.15.7:56338) 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2febd0740 10 client.0 _ll_register_callbacks cb 0x7ff2f001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/3109127854 learned_addr learned my addr 172.21.15.7:0/3109127854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2f9caf640 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5aa310 0x5634cc5e7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2f94ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b75b0 0x5634cc5e8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36628/0 (socket says 172.21.15.7:36628) 2024-03-26T08:51:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/3109127854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5aa310 msgr2=0x5634cc5e7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5aa310 0x5634cc5e7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/3109127854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b75b0 msgr2=0x5634cc5e8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b75b0 0x5634cc5e8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2fa4b0640 1 -- 172.21.15.7:0/3109127854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634cc5e6d90 con 0x5634cc5b7fa0 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2fa4b0640 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5e6560 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4097d20 tx=0x7ff2e4005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4529bdc65a2e7ac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.863+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3109127854 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e407f060 con 0x5634cc5b7fa0 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2febd0740 1 -- 172.21.15.7:0/3109127854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2e4084e50 con 0x5634cc5b7fa0 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3109127854 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff2e409ad10 con 0x5634cc5b7fa0 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3109127854 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2e40b3070 con 0x5634cc5b7fa0 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2febd0740 10 client.17880 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2febd0740 1 -- 172.21.15.7:0/3109127854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5634cc631820 con 0x5634cc5b7fa0 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3109127854 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff2e4084e50 con 0x5634cc5b7fa0 2024-03-26T08:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3109127854 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff2e40a0930 con 0x5634cc5b7fa0 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2febd0740 10 client.17880 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2febd0740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2febd0740 20 client.17880 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634cc5bd110 0x5634cc634140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2febd0740 4 client.17880 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2febd0740 1 -- 172.21.15.7:0/3109127854 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5634cc51bc60 con 0x5634cc5bd110 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.864+0000 7ff2f9caf640 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634cc5bd110 0x5634cc634140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.865+0000 7ff2dd7fa640 10 client.17880 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.865+0000 7ff2f9caf640 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634cc5bd110 0x5634cc634140 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ede8aaa1d495ff00 server_cookie=1d696f5d74776959 in_seq=0 out_seq=0 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.866+0000 7ff2dd7fa640 1 -- 172.21.15.7:0/3109127854 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5634cc51bc60 con 0x5634cc5bd110 2024-03-26T08:51:36.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.866+0000 7ff2dd7fa640 10 client.17880 handle_command_reply: tid=0 2024-03-26T08:51:36.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.868+0000 7ff2febd0740 1 client.17880 shutdown 2024-03-26T08:51:36.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.868+0000 7ff2febd0740 1 -- 172.21.15.7:0/3109127854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 msgr2=0x5634cc5e6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.868+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5e6560 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4097d20 tx=0x7ff2e4005b20 comp rx=0 tx=0).stop 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.868+0000 7ff2febd0740 1 -- 172.21.15.7:0/3109127854 shutdown_connections 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.868+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5634cc5bd110 0x5634cc634140 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.868+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5634cc5b7fa0 0x5634cc5e6560 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.868+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5634cc5b75b0 0x5634cc5e8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2febd0740 1 --2- 172.21.15.7:0/3109127854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5634cc5aa310 0x5634cc5e7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2febd0740 1 -- 172.21.15.7:0/3109127854 >> 172.21.15.7:0/3109127854 conn(0x5634cc5b8d80 msgr2=0x5634cc5a8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2dd7fa640 0 client.17880 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2febd0740 1 -- 172.21.15.7:0/3109127854 shutdown_connections 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2febd0740 1 -- 172.21.15.7:0/3109127854 wait complete. 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2febd0740 20 client.17880 trim_cache size 0 max 16384 2024-03-26T08:51:36.866 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:51:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (128 inodes in the stack)", 2024-03-26T08:51:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:51:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "fad0c253-fb4f-4059-9002-b89c4087f74c": { 2024-03-26T08:51:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:51:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "fad0c253-fb4f-4059-9002-b89c4087f74c", 2024-03-26T08:51:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:51:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:51:36.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:51:36.867 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2fbf13640 1 -- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2e00a2900 msgr2=0x7ff2e00a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2e00a2900 0x7ff2e00a4dc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec02c630 tx=0x7ff2ec02e040 comp rx=0 tx=0).stop 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2fbf13640 1 -- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4107440 msgr2=0x7ff2f41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.869+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4107440 0x7ff2f41006d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400e970 tx=0x7ff2e400ed70 comp rx=0 tx=0).stop 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.870+0000 7ff2fbf13640 1 -- 172.21.15.7:0/2859464459 shutdown_connections 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.870+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff2e00a2900 0x7ff2e00a4dc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.870+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2f4107440 0x7ff2f41006d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.870+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff2f4104b10 0x7ff2f4100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.870+0000 7ff2fbf13640 1 --2- 172.21.15.7:0/2859464459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f41021e0 0x7ff2f4101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.870+0000 7ff2fbf13640 1 -- 172.21.15.7:0/2859464459 >> 172.21.15.7:0/2859464459 conn(0x7ff2f40fbb30 msgr2=0x7ff2f4102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:36.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.870+0000 7ff2fbf13640 1 -- 172.21.15.7:0/2859464459 shutdown_connections 2024-03-26T08:51:36.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:51:36.870+0000 7ff2fbf13640 1 -- 172.21.15.7:0/2859464459 wait complete. 2024-03-26T08:51:36.879 INFO:tasks.cephfs.filesystem:scrub status for tag:fad0c253-fb4f-4059-9002-b89c4087f74c - {'path': '/', 'tag': 'fad0c253-fb4f-4059-9002-b89c4087f74c', 'options': 'recursive,force'} 2024-03-26T08:51:37.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:37 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:51:37.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:37 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:37.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:37 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:51:37.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:37 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:37.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:37 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:51:37.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:37 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:37.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:37 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:51:37.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:37 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:37.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:37 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:51:37.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:37 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:37.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:37 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:51:37.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:37 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:38.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:38 smithi007 ceph-mon[32462]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 688 op/s 2024-03-26T08:51:38.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:38 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:51:38.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:38 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:38.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:38 smithi044 ceph-mon[36461]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 688 op/s 2024-03-26T08:51:38.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:38 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:51:38.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:38 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:38.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:38 smithi016 ceph-mon[35851]: pgmap v636: 97 pgs: 97 active+clean; 1.6 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.0 MiB/s wr, 688 op/s 2024-03-26T08:51:38.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:38 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:51:38.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:38 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:51:40.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:40 smithi007 ceph-mon[32462]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.2 MiB/s wr, 653 op/s 2024-03-26T08:51:40.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:40 smithi044 ceph-mon[36461]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.2 MiB/s wr, 653 op/s 2024-03-26T08:51:40.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:40 smithi016 ceph-mon[35851]: pgmap v637: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 7.2 MiB/s wr, 653 op/s 2024-03-26T08:51:41.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 550.0 s, 1865.6 tps, lat 53.338 ms stddev 68.540 2024-03-26T08:51:42.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:42 smithi007 ceph-mon[32462]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.2 MiB/s wr, 758 op/s 2024-03-26T08:51:42.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:42 smithi044 ceph-mon[36461]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.2 MiB/s wr, 758 op/s 2024-03-26T08:51:42.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:42 smithi016 ceph-mon[35851]: pgmap v638: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 8.2 MiB/s wr, 758 op/s 2024-03-26T08:51:43.569 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:43 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:51:43.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:43 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:51:43.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:43 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:51:44.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:44 smithi044 ceph-mon[36461]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.2 MiB/s wr, 788 op/s 2024-03-26T08:51:44.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:44 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:51:44.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:44 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:51:44.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:44 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:51:44.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:44 smithi016 ceph-mon[35851]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.2 MiB/s wr, 788 op/s 2024-03-26T08:51:44.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:44 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:51:44.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:44 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:51:44.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:44 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:51:44.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:44 smithi007 ceph-mon[32462]: pgmap v639: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.2 MiB/s wr, 788 op/s 2024-03-26T08:51:44.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:44 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:51:44.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:44 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:51:44.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:44 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:51:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.865+0000 7f86e2386640 1 -- 172.21.15.7:0/2842269865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc0734d0 msgr2=0x7f86dc10a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.865+0000 7f86e2386640 1 --2- 172.21.15.7:0/2842269865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc0734d0 0x7f86dc10a6b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009900 tx=0x7f86d802bc30 comp rx=0 tx=0).stop 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.865+0000 7f86e2386640 1 -- 172.21.15.7:0/2842269865 shutdown_connections 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.865+0000 7f86e2386640 1 --2- 172.21.15.7:0/2842269865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc10d780 0x7f86dc1185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.865+0000 7f86e2386640 1 --2- 172.21.15.7:0/2842269865 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86dc10ac80 0x7f86dc10d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.865+0000 7f86e2386640 1 --2- 172.21.15.7:0/2842269865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc0734d0 0x7f86dc10a6b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.865+0000 7f86e2386640 1 -- 172.21.15.7:0/2842269865 >> 172.21.15.7:0/2842269865 conn(0x7f86dc06e580 msgr2=0x7f86dc0709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 -- 172.21.15.7:0/2842269865 shutdown_connections 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 -- 172.21.15.7:0/2842269865 wait complete. 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 Processor -- start 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 -- start start 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86dc0734d0 0x7f86dc1a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc10ac80 0x7f86dc1a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc10d780 0x7f86dc1adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc11bf90 con 0x7f86dc10d780 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc11be10 con 0x7f86dc10ac80 2024-03-26T08:51:44.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e2386640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc11c110 con 0x7f86dc0734d0 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e1b85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc10d780 0x7f86dc1adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e1b85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc10d780 0x7f86dc1adbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56354/0 (socket says 172.21.15.7:56354) 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e1384640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86dc0734d0 0x7f86dc1a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.866+0000 7f86e1b85640 1 -- 172.21.15.7:0/1771330877 learned_addr learned my addr 172.21.15.7:0/1771330877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.867+0000 7f86e0b83640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc10ac80 0x7f86dc1a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.867+0000 7f86e1b85640 1 -- 172.21.15.7:0/1771330877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86dc0734d0 msgr2=0x7f86dc1a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.867+0000 7f86e1b85640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86dc0734d0 0x7f86dc1a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.867+0000 7f86e1b85640 1 -- 172.21.15.7:0/1771330877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc10ac80 msgr2=0x7f86dc1a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.867+0000 7f86e1b85640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc10ac80 0x7f86dc1a6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.867+0000 7f86e1b85640 1 -- 172.21.15.7:0/1771330877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc1ae220 con 0x7f86dc10d780 2024-03-26T08:51:44.864 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.867+0000 7f86e1384640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86dc0734d0 0x7f86dc1a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.868+0000 7f86e1b85640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc10d780 0x7f86dc1adbb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f86d400eb10 tx=0x7f86d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.868+0000 7f86d27fc640 1 -- 172.21.15.7:0/1771330877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4004b20 con 0x7f86dc10d780 2024-03-26T08:51:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.868+0000 7f86d27fc640 1 -- 172.21.15.7:0/1771330877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86d4004cc0 con 0x7f86dc10d780 2024-03-26T08:51:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.868+0000 7f86d27fc640 1 -- 172.21.15.7:0/1771330877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d4010760 con 0x7f86dc10d780 2024-03-26T08:51:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.868+0000 7f86e2386640 1 -- 172.21.15.7:0/1771330877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc1ae510 con 0x7f86dc10d780 2024-03-26T08:51:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.868+0000 7f86e2386640 1 -- 172.21.15.7:0/1771330877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc1aea10 con 0x7f86dc10d780 2024-03-26T08:51:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.871+0000 7f86e61cb740 1 Processor -- start 2024-03-26T08:51:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.871+0000 7f86d27fc640 1 -- 172.21.15.7:0/1771330877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f86d4005020 con 0x7f86dc10d780 2024-03-26T08:51:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e61cb740 1 -- start start 2024-03-26T08:51:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e3b20 0x55c34d6e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e9680 0x55c34d6ebb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e61cb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34d648b10 con 0x55c34d6e3b20 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e61cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34d5f1900 con 0x55c34d6e9680 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e61cb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34d644bb0 con 0x55c34d6e4510 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e0b83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e3b20 0x55c34d6e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e1384640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86d27fc640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86c80a2930 0x7f86c80a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e1384640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6e9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53084/0 (socket says 172.21.15.7:53084) 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e1384640 1 -- 172.21.15.7:0/2879385976 learned_addr learned my addr 172.21.15.7:0/2879385976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e0b83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e3b20 0x55c34d6e3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56370/0 (socket says 172.21.15.7:56370) 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e0b83640 1 --2- 172.21.15.7:0/2879385976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e9680 0x55c34d6ebb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e1b85640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86c80a2930 0x7f86c80a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e1384640 1 -- 172.21.15.7:0/2879385976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e9680 msgr2=0x55c34d6ebb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e1384640 1 --2- 172.21.15.7:0/2879385976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e9680 0x55c34d6ebb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86d27fc640 1 -- 172.21.15.7:0/1771330877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f86d40ceec0 con 0x7f86dc10d780 2024-03-26T08:51:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e1384640 1 -- 172.21.15.7:0/2879385976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e3b20 msgr2=0x55c34d6e3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e1384640 1 --2- 172.21.15.7:0/2879385976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e3b20 0x55c34d6e3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.872+0000 7f86e1384640 1 -- 172.21.15.7:0/2879385976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c34d35d210 con 0x55c34d6e4510 2024-03-26T08:51:44.870 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.873+0000 7f86e1384640 1 --2- 172.21.15.7:0/2879385976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6e9010 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009900 tx=0x7f86d802cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b1690ab809050d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.874+0000 7f86c77fe640 1 -- 172.21.15.7:0/2879385976 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8005d50 con 0x55c34d6e4510 2024-03-26T08:51:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.874+0000 7f86c77fe640 1 -- 172.21.15.7:0/2879385976 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86d8005520 con 0x55c34d6e4510 2024-03-26T08:51:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.874+0000 7f86c77fe640 1 -- 172.21.15.7:0/2879385976 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8004bf0 con 0x55c34d6e4510 2024-03-26T08:51:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 -- 172.21.15.7:0/2879385976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 msgr2=0x55c34d6e9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2879385976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6e9010 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009900 tx=0x7f86d802cde0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e1b85640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86c80a2930 0x7f86c80a4df0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f86d40a3ad0 tx=0x7f86d400aea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 -- 172.21.15.7:0/2879385976 shutdown_connections 2024-03-26T08:51:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2879385976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e9680 0x55c34d6ebb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2879385976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6e9010 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2879385976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e3b20 0x55c34d6e3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 -- 172.21.15.7:0/2879385976 >> 172.21.15.7:0/2879385976 conn(0x55c34d6df090 msgr2=0x55c34d6df490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 -- 172.21.15.7:0/2879385976 shutdown_connections 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 -- 172.21.15.7:0/2879385976 wait complete. 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.875+0000 7f86e61cb740 1 -- start start 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e61cb740 1 Processor -- start 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 0x55c34d6ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e4510 0x55c34d6ee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e9680 0x55c34d6f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e61cb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34d6ec5d0 con 0x55c34d6e4510 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e61cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34d604d50 con 0x55c34d6e3b20 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e61cb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c34d6ec750 con 0x55c34d6e9680 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e61cb740 10 client.0 _ll_register_callbacks cb 0x55c34d604720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e1384640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e4510 0x55c34d6ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e1384640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e4510 0x55c34d6ee360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56372/0 (socket says 172.21.15.7:56372) 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e1384640 1 -- 172.21.15.7:0/2218693589 learned_addr learned my addr 172.21.15.7:0/2218693589 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e0b83640 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 0x55c34d6ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.876+0000 7f86e0b83640 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e9680 0x55c34d6f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e1384640 1 -- 172.21.15.7:0/2218693589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e9680 msgr2=0x55c34d6f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e1384640 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e9680 0x55c34d6f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e1384640 1 -- 172.21.15.7:0/2218693589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 msgr2=0x55c34d6ede20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e1384640 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 0x55c34d6ede20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e1384640 1 -- 172.21.15.7:0/2218693589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c34d429620 con 0x55c34d6e4510 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e1384640 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e4510 0x55c34d6ee360 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f86d802c170 tx=0x7f86d8004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e19c864c501b7da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86c4ff9640 1 -- 172.21.15.7:0/2218693589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8005d50 con 0x55c34d6e4510 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e61cb740 1 -- 172.21.15.7:0/2218693589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86d8005520 con 0x55c34d6e4510 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86c4ff9640 1 -- 172.21.15.7:0/2218693589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f86d8005220 con 0x55c34d6e4510 2024-03-26T08:51:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86c4ff9640 1 -- 172.21.15.7:0/2218693589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d8034440 con 0x55c34d6e4510 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e61cb740 10 client.17892 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e61cb740 1 -- 172.21.15.7:0/2218693589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c34d6f8710 con 0x55c34d6e4510 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86c4ff9640 1 -- 172.21.15.7:0/2218693589 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86d8005520 con 0x55c34d6e4510 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86c4ff9640 1 -- 172.21.15.7:0/2218693589 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f86d8045520 con 0x55c34d6e4510 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e61cb740 10 client.17892 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.877+0000 7f86e61cb740 10 client.17892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.878+0000 7f86e61cb740 20 client.17892 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.878+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c34d6f9a60 0x55c34d6fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.878+0000 7f86e61cb740 4 client.17892 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.878+0000 7f86e61cb740 1 -- 172.21.15.7:0/2218693589 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c34d656050 con 0x55c34d6f9a60 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.878+0000 7f86e0b83640 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c34d6f9a60 0x55c34d6fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.878+0000 7f86c4ff9640 10 client.17892 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.878+0000 7f86e0b83640 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c34d6f9a60 0x55c34d6fbe50 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85bd10b66839a0d server_cookie=779dd6a5d878c0ed in_seq=0 out_seq=0 2024-03-26T08:51:44.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.880+0000 7f86c4ff9640 1 -- 172.21.15.7:0/2218693589 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c34d656050 con 0x55c34d6f9a60 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.880+0000 7f86c4ff9640 10 client.17892 handle_command_reply: tid=0 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.880+0000 7f86e61cb740 1 client.17892 shutdown 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.880+0000 7f86e61cb740 1 -- 172.21.15.7:0/2218693589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e4510 msgr2=0x55c34d6ee360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.880+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e4510 0x55c34d6ee360 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f86d802c170 tx=0x7f86d8004fa0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86e61cb740 1 -- 172.21.15.7:0/2218693589 shutdown_connections 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c34d6f9a60 0x55c34d6fbe50 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e9680 0x55c34d6f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6e4510 0x55c34d6ee360 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86e61cb740 1 --2- 172.21.15.7:0/2218693589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 0x55c34d6ede20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86e61cb740 1 -- 172.21.15.7:0/2218693589 >> 172.21.15.7:0/2218693589 conn(0x55c34d6df090 msgr2=0x55c34d6de290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86c4ff9640 0 client.17892 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86e61cb740 1 -- 172.21.15.7:0/2218693589 shutdown_connections 2024-03-26T08:51:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86e61cb740 1 -- 172.21.15.7:0/2218693589 wait complete. 2024-03-26T08:51:44.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.881+0000 7f86e61cb740 20 client.17892 trim_cache size 0 max 16384 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e61cb740 1 Processor -- start 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e61cb740 1 -- start start 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6dda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e3b20 0x55c34d6f5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e4510 0x55c34d6ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e61cb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1b9ad0 con 0x55c34d6df090 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e61cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1b75d0 con 0x55c34d6e4510 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e61cb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1b5650 con 0x55c34d6e3b20 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e0b83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6dda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e0b83640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6dda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56376/0 (socket says 172.21.15.7:56376) 2024-03-26T08:51:44.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e0b83640 1 -- 172.21.15.7:0/1363153249 learned_addr learned my addr 172.21.15.7:0/1363153249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e1384640 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e3b20 0x55c34d6f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e0b83640 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e4510 0x55c34d6ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e0b83640 1 -- 172.21.15.7:0/1363153249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e3b20 msgr2=0x55c34d6f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.900+0000 7f86e0b83640 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e3b20 0x55c34d6f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.901+0000 7f86e0b83640 1 -- 172.21.15.7:0/1363153249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e4510 msgr2=0x55c34d6ddf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.901+0000 7f86e0b83640 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e4510 0x55c34d6ddf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.901+0000 7f86e0b83640 1 -- 172.21.15.7:0/1363153249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c34d6fc1c0 con 0x55c34d6df090 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.901+0000 7f86e1384640 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e3b20 0x55c34d6f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.901+0000 7f86e0b83640 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6dda00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009fd0 tx=0x7f86cc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477a7bfad6da55d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.901+0000 7f86c6ffd640 1 -- 172.21.15.7:0/1363153249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc01c040 con 0x55c34d6df090 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.901+0000 7f86c6ffd640 1 -- 172.21.15.7:0/1363153249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86cc012390 con 0x55c34d6df090 2024-03-26T08:51:44.898 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.901+0000 7f86c6ffd640 1 -- 172.21.15.7:0/1363153249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0047a0 con 0x55c34d6df090 2024-03-26T08:51:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 -- 172.21.15.7:0/1363153249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 msgr2=0x55c34d6dda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:44.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6dda00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009fd0 tx=0x7f86cc015040 comp rx=0 tx=0).stop 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 -- 172.21.15.7:0/1363153249 shutdown_connections 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e4510 0x55c34d6ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e3b20 0x55c34d6f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 --2- 172.21.15.7:0/1363153249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6dda00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 -- 172.21.15.7:0/1363153249 >> 172.21.15.7:0/1363153249 conn(0x55c34d6e5110 msgr2=0x55c34d6e5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 -- 172.21.15.7:0/1363153249 shutdown_connections 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 -- 172.21.15.7:0/1363153249 wait complete. 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 -- start start 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.902+0000 7f86e61cb740 1 Processor -- start 2024-03-26T08:51:44.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.903+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.903+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 0x55c34d6f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.903+0000 7f86e61cb740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.903+0000 7f86e61cb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1b5c50 con 0x55c34d6df090 2024-03-26T08:51:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.903+0000 7f86e61cb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1b6f70 con 0x55c34d6e3b20 2024-03-26T08:51:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.904+0000 7f86e61cb740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86dc1b62b0 con 0x55c34d6e4510 2024-03-26T08:51:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.904+0000 7f86e61cb740 10 client.0 _ll_register_callbacks cb 0x7f86d8004ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.907+0000 7f86e0b83640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.910+0000 7f86e1384640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 0x55c34d6f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.911+0000 7f86e0b83640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6f42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53120/0 (socket says 172.21.15.7:53120) 2024-03-26T08:51:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.911+0000 7f86e0b83640 1 -- 172.21.15.7:0/3532686603 learned_addr learned my addr 172.21.15.7:0/3532686603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.911+0000 7f86e0b83640 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.915+0000 7f86e1384640 1 -- 172.21.15.7:0/3532686603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 msgr2=0x55c34d6f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.915+0000 7f86e1384640 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.915+0000 7f86e1384640 1 -- 172.21.15.7:0/3532686603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 msgr2=0x55c34d6f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.915+0000 7f86e1384640 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:44.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.915+0000 7f86e1384640 1 -- 172.21.15.7:0/3532686603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c34d6f48b0 con 0x55c34d6e3b20 2024-03-26T08:51:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.920+0000 7f86e1384640 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 0x55c34d6f32b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f86d802c7b0 tx=0x7f86d8045650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85bbb249b47f7a77 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.920+0000 7f86c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.925+0000 7f86c4ff9640 1 -- 172.21.15.7:0/3532686603 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d805c070 con 0x55c34d6e3b20 2024-03-26T08:51:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.925+0000 7f86c4ff9640 1 -- 172.21.15.7:0/3532686603 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f86d8005430 con 0x55c34d6e3b20 2024-03-26T08:51:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.925+0000 7f86e61cb740 1 -- 172.21.15.7:0/3532686603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86cc012390 con 0x55c34d6e3b20 2024-03-26T08:51:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.929+0000 7f86c4ff9640 1 -- 172.21.15.7:0/3532686603 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d8005d50 con 0x55c34d6e3b20 2024-03-26T08:51:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.929+0000 7f86e61cb740 10 client.27719 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.929+0000 7f86e61cb740 1 -- 172.21.15.7:0/3532686603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c34d755470 con 0x55c34d6e3b20 2024-03-26T08:51:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.931+0000 7f86c4ff9640 1 -- 172.21.15.7:0/3532686603 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86d8005630 con 0x55c34d6e3b20 2024-03-26T08:51:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.931+0000 7f86c4ff9640 1 -- 172.21.15.7:0/3532686603 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f86d805f030 con 0x55c34d6e3b20 2024-03-26T08:51:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.931+0000 7f86e61cb740 10 client.27719 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.931+0000 7f86e61cb740 10 client.27719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.931+0000 7f86e61cb740 20 client.27719 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.931+0000 7f86e61cb740 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c34d6e9680 0x55c34d757d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.931+0000 7f86e61cb740 4 client.27719 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:51:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.931+0000 7f86e61cb740 1 -- 172.21.15.7:0/3532686603 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f86cc012390 con 0x55c34d6e9680 2024-03-26T08:51:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.933+0000 7f86e0b83640 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c34d6e9680 0x55c34d757d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.946+0000 7f86e0b83640 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c34d6e9680 0x55c34d757d80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60d66ca5b7bcbe8 server_cookie=edd9b0fc2681e2f4 in_seq=0 out_seq=0 2024-03-26T08:51:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:44.946+0000 7f86c4ff9640 10 client.27719 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86c4ff9640 1 -- 172.21.15.7:0/3532686603 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f86cc012390 con 0x55c34d6e9680 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86c4ff9640 10 client.27719 handle_command_reply: tid=0 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 client.27719 shutdown 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 -- 172.21.15.7:0/3532686603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 msgr2=0x55c34d6f32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 0x55c34d6f32b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f86d802c7b0 tx=0x7f86d8045650 comp rx=0 tx=0).stop 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 -- 172.21.15.7:0/3532686603 shutdown_connections 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c34d6e9680 0x55c34d757d80 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c34d6e4510 0x55c34d6f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c34d6e3b20 0x55c34d6f32b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86c4ff9640 0 client.27719 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 --2- 172.21.15.7:0/3532686603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c34d6df090 0x55c34d6f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 -- 172.21.15.7:0/3532686603 >> 172.21.15.7:0/3532686603 conn(0x55c34d6e5110 msgr2=0x55c34d6e22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 -- 172.21.15.7:0/3532686603 shutdown_connections 2024-03-26T08:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 1 -- 172.21.15.7:0/3532686603 wait complete. 2024-03-26T08:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.013+0000 7f86e61cb740 20 client.27719 trim_cache size 0 max 16384 2024-03-26T08:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:51:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 -- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86c80a2930 msgr2=0x7f86c80a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86c80a2930 0x7f86c80a4df0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f86d40a3ad0 tx=0x7f86d400aea0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 -- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc10d780 msgr2=0x7f86dc1adbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc10d780 0x7f86dc1adbb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f86d400eb10 tx=0x7f86d400c2d0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 -- 172.21.15.7:0/1771330877 shutdown_connections 2024-03-26T08:51:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f86c80a2930 0x7f86c80a4df0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86dc10d780 0x7f86dc1adbb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc10ac80 0x7f86dc1a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 --2- 172.21.15.7:0/1771330877 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f86dc0734d0 0x7f86dc1a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 -- 172.21.15.7:0/1771330877 >> 172.21.15.7:0/1771330877 conn(0x7f86dc06e580 msgr2=0x7f86dc1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 -- 172.21.15.7:0/1771330877 shutdown_connections 2024-03-26T08:51:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:45.014+0000 7f86e2386640 1 -- 172.21.15.7:0/1771330877 wait complete. 2024-03-26T08:51:46.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 555.0 s, 1796.6 tps, lat 55.684 ms stddev 65.599 2024-03-26T08:51:46.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:46 smithi044 ceph-mon[36461]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.9 MiB/s wr, 750 op/s 2024-03-26T08:51:46.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:46 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:51:46.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:46 smithi016 ceph-mon[35851]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.9 MiB/s wr, 750 op/s 2024-03-26T08:51:46.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:46 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:51:46.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:46 smithi007 ceph-mon[32462]: pgmap v640: 97 pgs: 97 active+clean; 1.6 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.9 MiB/s wr, 750 op/s 2024-03-26T08:51:46.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:46 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:51:47.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:47 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:51:47.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:47 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:51:47.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:47 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:51:48.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:48 smithi044 ceph-mon[36461]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.5 MiB/s wr, 756 op/s 2024-03-26T08:51:48.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:48 smithi016 ceph-mon[35851]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.5 MiB/s wr, 756 op/s 2024-03-26T08:51:48.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:48 smithi007 ceph-mon[32462]: pgmap v641: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 7.5 MiB/s wr, 756 op/s 2024-03-26T08:51:50.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:50 smithi044 ceph-mon[36461]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.1 MiB/s wr, 742 op/s 2024-03-26T08:51:50.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:50 smithi016 ceph-mon[35851]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.1 MiB/s wr, 742 op/s 2024-03-26T08:51:50.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:50 smithi007 ceph-mon[32462]: pgmap v642: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.1 MiB/s wr, 742 op/s 2024-03-26T08:51:51.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 560.0 s, 1903.2 tps, lat 52.434 ms stddev 62.760 2024-03-26T08:51:52.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:52 smithi016 ceph-mon[35851]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.3 MiB/s wr, 757 op/s 2024-03-26T08:51:52.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:52 smithi007 ceph-mon[32462]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.3 MiB/s wr, 757 op/s 2024-03-26T08:51:52.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:52 smithi044 ceph-mon[36461]: pgmap v643: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.3 MiB/s wr, 757 op/s 2024-03-26T08:51:54.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:54 smithi016 ceph-mon[35851]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.2 MiB/s wr, 734 op/s 2024-03-26T08:51:54.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:54 smithi007 ceph-mon[32462]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.2 MiB/s wr, 734 op/s 2024-03-26T08:51:54.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:54 smithi044 ceph-mon[36461]: pgmap v644: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.2 MiB/s wr, 734 op/s 2024-03-26T08:51:56.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 565.0 s, 2564.4 tps, lat 39.232 ms stddev 48.108 2024-03-26T08:51:56.598 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:56 smithi007 ceph-mon[32462]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-03-26T08:51:56.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:56 smithi016 ceph-mon[35851]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-03-26T08:51:56.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:56 smithi044 ceph-mon[36461]: pgmap v645: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.4 MiB/s wr, 650 op/s 2024-03-26T08:51:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 -- 172.21.15.7:0/891393916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c1021c0 msgr2=0x7f6e4c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 --2- 172.21.15.7:0/891393916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c1021c0 0x7f6e4c108a50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009ca0 tx=0x7f6e4402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 -- 172.21.15.7:0/891393916 shutdown_connections 2024-03-26T08:51:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 --2- 172.21.15.7:0/891393916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c1021c0 0x7f6e4c108a50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 --2- 172.21.15.7:0/891393916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c101800 0x7f6e4c101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 --2- 172.21.15.7:0/891393916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4c100610 0x7f6e4c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 -- 172.21.15.7:0/891393916 >> 172.21.15.7:0/891393916 conn(0x7f6e4c0fbb80 msgr2=0x7f6e4c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 -- 172.21.15.7:0/891393916 shutdown_connections 2024-03-26T08:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 -- 172.21.15.7:0/891393916 wait complete. 2024-03-26T08:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.130+0000 7f6e51eea640 1 Processor -- start 2024-03-26T08:51:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e51eea640 1 -- start start 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e51eea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4c100610 0x7f6e4c1045a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e51eea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c101800 0x7f6e4c102be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e51eea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c1021c0 0x7f6e4c103120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e51eea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c10c290 con 0x7f6e4c101800 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e51eea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c10c110 con 0x7f6e4c1021c0 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e51eea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c10c410 con 0x7f6e4c100610 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c101800 0x7f6e4c102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c101800 0x7f6e4c102be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33296/0 (socket says 172.21.15.7:33296) 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4affd640 1 -- 172.21.15.7:0/3940753652 learned_addr learned my addr 172.21.15.7:0/3940753652 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4c100610 0x7f6e4c1045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c1021c0 0x7f6e4c103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4affd640 1 -- 172.21.15.7:0/3940753652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4c100610 msgr2=0x7f6e4c1045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4affd640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4c100610 0x7f6e4c1045a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4affd640 1 -- 172.21.15.7:0/3940753652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c1021c0 msgr2=0x7f6e4c103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4affd640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c1021c0 0x7f6e4c103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4affd640 1 -- 172.21.15.7:0/3940753652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c1039b0 con 0x7f6e4c101800 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e4affd640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c101800 0x7f6e4c102be0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000e9f0 tx=0x7f6e4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e48ff9640 1 -- 172.21.15.7:0/3940753652 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000cd80 con 0x7f6e4c101800 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e48ff9640 1 -- 172.21.15.7:0/3940753652 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e40010910 con 0x7f6e4c101800 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e51eea640 1 -- 172.21.15.7:0/3940753652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c1a1af0 con 0x7f6e4c101800 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.131+0000 7f6e48ff9640 1 -- 172.21.15.7:0/3940753652 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40018990 con 0x7f6e4c101800 2024-03-26T08:51:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.132+0000 7f6e51eea640 1 -- 172.21.15.7:0/3940753652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c1a2040 con 0x7f6e4c101800 2024-03-26T08:51:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.135+0000 7f6e48ff9640 1 -- 172.21.15.7:0/3940753652 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6e40010ac0 con 0x7f6e4c101800 2024-03-26T08:51:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.135+0000 7f6e54bb3740 1 Processor -- start 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.135+0000 7f6e54bb3740 1 -- start start 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.135+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec4f920 0x564bdec4fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.135+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec50310 0x564bdec54e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.135+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 0x564bdec57910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.135+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdebb4b10 con 0x564bdec4f920 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.135+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdeb5d900 con 0x564bdec50310 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.135+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdebb0bb0 con 0x564bdec55480 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec4f920 0x564bdec4fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec4f920 0x564bdec4fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33308/0 (socket says 172.21.15.7:33308) 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4bfff640 1 -- 172.21.15.7:0/708598397 learned_addr learned my addr 172.21.15.7:0/708598397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4affd640 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 0x564bdec57910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec50310 0x564bdec54e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e48ff9640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6e340a27f0 0x7f6e340a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4b7fe640 1 -- 172.21.15.7:0/708598397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 msgr2=0x564bdec57910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 0x564bdec57910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4b7fe640 1 -- 172.21.15.7:0/708598397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec4f920 msgr2=0x564bdec4fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec4f920 0x564bdec4fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4b7fe640 1 -- 172.21.15.7:0/708598397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bde8c9210 con 0x564bdec50310 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e48ff9640 1 -- 172.21.15.7:0/3940753652 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e400ced80 con 0x7f6e4c101800 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec4f920 0x564bdec4fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6e340a27f0 0x7f6e340a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.136+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec50310 0x564bdec54e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009bd0 tx=0x7f6e3c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811c030527a6be6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.137+0000 7f6e31ffb640 1 -- 172.21.15.7:0/708598397 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c004230 con 0x564bdec50310 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.137+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6e340a27f0 0x7f6e340a4cb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6e44006010 tx=0x7f6e4402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.137+0000 7f6e31ffb640 1 -- 172.21.15.7:0/708598397 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e3c009e40 con 0x564bdec50310 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.137+0000 7f6e31ffb640 1 -- 172.21.15.7:0/708598397 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c004230 con 0x564bdec50310 2024-03-26T08:51:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.137+0000 7f6e54bb3740 1 -- 172.21.15.7:0/708598397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec50310 msgr2=0x564bdec54e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.137+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec50310 0x564bdec54e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009bd0 tx=0x7f6e3c011810 comp rx=0 tx=0).stop 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.137+0000 7f6e54bb3740 1 -- 172.21.15.7:0/708598397 shutdown_connections 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.137+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 0x564bdec57910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec50310 0x564bdec54e10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/708598397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec4f920 0x564bdec4fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 -- 172.21.15.7:0/708598397 >> 172.21.15.7:0/708598397 conn(0x564bdec4ae90 msgr2=0x564bdec4b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 -- 172.21.15.7:0/708598397 shutdown_connections 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 -- 172.21.15.7:0/708598397 wait complete. 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 -- start start 2024-03-26T08:51:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 Processor -- start 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec59c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec5a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 0x564bdec61180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdec58430 con 0x564bdec50310 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdeb70d50 con 0x564bdec4f920 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bdec585b0 con 0x564bdec55480 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e54bb3740 10 client.0 _ll_register_callbacks cb 0x564bdeb70720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e4affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec5a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec59c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e4affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec5a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33320/0 (socket says 172.21.15.7:33320) 2024-03-26T08:51:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e4affd640 1 -- 172.21.15.7:0/3207469890 learned_addr learned my addr 172.21.15.7:0/3207469890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.138+0000 7f6e4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec59c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38310/0 (socket says 172.21.15.7:38310) 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 0x564bdec61180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e4affd640 1 -- 172.21.15.7:0/3207469890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 msgr2=0x564bdec61180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e4affd640 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 0x564bdec61180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e4affd640 1 -- 172.21.15.7:0/3207469890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 msgr2=0x564bdec59c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e4affd640 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec59c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e4affd640 1 -- 172.21.15.7:0/3207469890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bde995620 con 0x564bdec50310 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec59c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e4affd640 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec5a140 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c6a0 tx=0x7f6e4009b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac227e8468b91d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e2affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e2affd640 1 -- 172.21.15.7:0/3207469890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4009f870 con 0x564bdec50310 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e2affd640 1 -- 172.21.15.7:0/3207469890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e40018b70 con 0x564bdec50310 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3207469890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bdebc2050 con 0x564bdec50310 2024-03-26T08:51:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e2affd640 1 -- 172.21.15.7:0/3207469890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e400a3ec0 con 0x564bdec50310 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e54bb3740 10 client.17910 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3207469890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564bdec64560 con 0x564bdec50310 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.139+0000 7f6e2affd640 1 -- 172.21.15.7:0/3207469890 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564bdebc2050 con 0x564bdec50310 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e2affd640 1 -- 172.21.15.7:0/3207469890 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6e40097520 con 0x564bdec50310 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e54bb3740 10 client.17910 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e54bb3740 10 client.17910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e54bb3740 20 client.17910 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564bdec65be0 0x564bdec68030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e54bb3740 4 client.17910 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3207469890 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bdebbc3a0 con 0x564bdec65be0 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564bdec65be0 0x564bdec68030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564bdec65be0 0x564bdec68030 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4efa644af5ec9db server_cookie=ec3f1f6b9756dc0a in_seq=0 out_seq=0 2024-03-26T08:51:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.140+0000 7f6e2affd640 10 client.17910 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.141+0000 7f6e2affd640 1 -- 172.21.15.7:0/3207469890 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564bdebbc3a0 con 0x564bdec65be0 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.141+0000 7f6e2affd640 10 client.17910 handle_command_reply: tid=0 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.141+0000 7f6e54bb3740 1 client.17910 shutdown 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.143+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3207469890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 msgr2=0x564bdec5a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.143+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec5a140 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c6a0 tx=0x7f6e4009b310 comp rx=0 tx=0).stop 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.143+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3207469890 shutdown_connections 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.143+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564bdec65be0 0x564bdec68030 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.143+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec55480 0x564bdec61180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.143+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec5a140 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.143+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3207469890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec59c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.143+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3207469890 >> 172.21.15.7:0/3207469890 conn(0x564bdec4ae90 msgr2=0x564bdec4a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.144+0000 7f6e2affd640 0 client.17910 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.144+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3207469890 shutdown_connections 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.144+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3207469890 wait complete. 2024-03-26T08:51:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.144+0000 7f6e54bb3740 20 client.17910 trim_cache size 0 max 16384 2024-03-26T08:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.161+0000 7f6e54bb3740 1 Processor -- start 2024-03-26T08:51:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.161+0000 7f6e54bb3740 1 -- start start 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.161+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec4f920 0x564bdec4b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.161+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec61140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.161+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec55480 0x564bdec67a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.161+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c1ad140 con 0x564bdec50310 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c1aac40 con 0x564bdec55480 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c1a8cc0 con 0x564bdec4f920 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec4f920 0x564bdec4b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec4f920 0x564bdec4b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:55644/0 (socket says 172.21.15.7:55644) 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4b7fe640 1 -- 172.21.15.7:0/2952962955 learned_addr learned my addr 172.21.15.7:0/2952962955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec55480 0x564bdec67a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.158 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4affd640 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec61140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4bfff640 1 -- 172.21.15.7:0/2952962955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec4f920 msgr2=0x564bdec4b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec4f920 0x564bdec4b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4bfff640 1 -- 172.21.15.7:0/2952962955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 msgr2=0x564bdec61140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec61140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4bfff640 1 -- 172.21.15.7:0/2952962955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bdec62600 con 0x564bdec55480 2024-03-26T08:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.162+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec4f920 0x564bdec4b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec55480 0x564bdec67a30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6e44032420 tx=0x7f6e44041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468f8355d5b20630 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e317fa640 1 -- 172.21.15.7:0/2952962955 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44039040 con 0x564bdec55480 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e317fa640 1 -- 172.21.15.7:0/2952962955 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e44053070 con 0x564bdec55480 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e317fa640 1 -- 172.21.15.7:0/2952962955 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44041470 con 0x564bdec55480 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e54bb3740 1 -- 172.21.15.7:0/2952962955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec55480 msgr2=0x564bdec67a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec55480 0x564bdec67a30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6e44032420 tx=0x7f6e44041000 comp rx=0 tx=0).stop 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e54bb3740 1 -- 172.21.15.7:0/2952962955 shutdown_connections 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec55480 0x564bdec67a30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec50310 0x564bdec61140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.163+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/2952962955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec4f920 0x564bdec4b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.164+0000 7f6e54bb3740 1 -- 172.21.15.7:0/2952962955 >> 172.21.15.7:0/2952962955 conn(0x564bdec4ae90 msgr2=0x564bdec54ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.164+0000 7f6e54bb3740 1 -- 172.21.15.7:0/2952962955 shutdown_connections 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.164+0000 7f6e54bb3740 1 -- 172.21.15.7:0/2952962955 wait complete. 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.164+0000 7f6e54bb3740 1 -- start start 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.164+0000 7f6e54bb3740 1 Processor -- start 2024-03-26T08:51:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.164+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec5e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.164+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec50310 0x564bdec5e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.164+0000 7f6e54bb3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec55480 0x564bdec5f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c1a92c0 con 0x564bdec55480 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c1aa5e0 con 0x564bdec4f920 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e54bb3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c1a9920 con 0x564bdec50310 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec50310 0x564bdec5e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec5e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec50310 0x564bdec5e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:55660/0 (socket says 172.21.15.7:55660) 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4affd640 1 -- 172.21.15.7:0/3765688065 learned_addr learned my addr 172.21.15.7:0/3765688065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e54bb3740 10 client.0 _ll_register_callbacks cb 0x7f6e4000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4bfff640 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec55480 0x564bdec5f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4b7fe640 1 -- 172.21.15.7:0/3765688065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec50310 msgr2=0x564bdec5e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec50310 0x564bdec5e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4b7fe640 1 -- 172.21.15.7:0/3765688065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec55480 msgr2=0x564bdec5f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec55480 0x564bdec5f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4b7fe640 1 -- 172.21.15.7:0/3765688065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bdec5fdf0 con 0x564bdec4f920 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e4b7fe640 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec5e2b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c01b040 tx=0x7f6e3c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5fa08100c8d74f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e2affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.165+0000 7f6e2affd640 1 -- 172.21.15.7:0/3765688065 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c004230 con 0x564bdec4f920 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e2affd640 1 -- 172.21.15.7:0/3765688065 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e3c004030 con 0x564bdec4f920 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3765688065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e44053070 con 0x564bdec4f920 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e2affd640 1 -- 172.21.15.7:0/3765688065 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e3c003de0 con 0x564bdec4f920 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e54bb3740 10 client.27737 fetch_fsmap learned FSMap version 25 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3765688065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564bdec610e0 con 0x564bdec4f920 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e2affd640 1 -- 172.21.15.7:0/3765688065 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e44053070 con 0x564bdec4f920 2024-03-26T08:51:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e2affd640 1 -- 172.21.15.7:0/3765688065 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6e3c0045c0 con 0x564bdec4f920 2024-03-26T08:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e54bb3740 10 client.27737 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e54bb3740 10 client.27737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e54bb3740 20 client.27737 populate_metadata read hostname 'smithi007' 2024-03-26T08:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564bdecc9f90 0x564bdeccc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e54bb3740 4 client.27737 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3765688065 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564bdeb3a710 con 0x564bdecc9f90 2024-03-26T08:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.166+0000 7f6e4affd640 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564bdecc9f90 0x564bdeccc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.167+0000 7f6e2affd640 10 client.27737 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.167+0000 7f6e4affd640 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564bdecc9f90 0x564bdeccc380 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1e348b15a1c38 server_cookie=f6eaeb9c167cbe81 in_seq=0 out_seq=0 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.179+0000 7f6e2affd640 1 -- 172.21.15.7:0/3765688065 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564bdeb3a710 con 0x564bdecc9f90 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.179+0000 7f6e2affd640 10 client.27737 handle_command_reply: tid=0 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.179+0000 7f6e54bb3740 1 client.27737 shutdown 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3765688065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 msgr2=0x564bdec5e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec5e2b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c01b040 tx=0x7f6e3c005d30 comp rx=0 tx=0).stop 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3765688065 shutdown_connections 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564bdecc9f90 0x564bdeccc380 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bdec55480 0x564bdec5f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e2affd640 0 client.27737 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564bdec50310 0x564bdec5e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 --2- 172.21.15.7:0/3765688065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bdec4f920 0x564bdec5e2b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3765688065 >> 172.21.15.7:0/3765688065 conn(0x564bdec4ae90 msgr2=0x564bdec67a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3765688065 shutdown_connections 2024-03-26T08:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 1 -- 172.21.15.7:0/3765688065 wait complete. 2024-03-26T08:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e54bb3740 20 client.27737 trim_cache size 0 max 16384 2024-03-26T08:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:51:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:51:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e51eea640 1 -- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6e340a27f0 msgr2=0x7f6e340a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e51eea640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6e340a27f0 0x7f6e340a4cb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6e44006010 tx=0x7f6e4402e040 comp rx=0 tx=0).stop 2024-03-26T08:51:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.180+0000 7f6e51eea640 1 -- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c101800 msgr2=0x7f6e4c102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:51:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.181+0000 7f6e51eea640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c101800 0x7f6e4c102be0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000e9f0 tx=0x7f6e4000eeb0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.181+0000 7f6e51eea640 1 -- 172.21.15.7:0/3940753652 shutdown_connections 2024-03-26T08:51:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.181+0000 7f6e51eea640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6e340a27f0 0x7f6e340a4cb0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.181+0000 7f6e51eea640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c1021c0 0x7f6e4c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.181+0000 7f6e51eea640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e4c101800 0x7f6e4c102be0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.181+0000 7f6e51eea640 1 --2- 172.21.15.7:0/3940753652 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6e4c100610 0x7f6e4c1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:51:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.181+0000 7f6e51eea640 1 -- 172.21.15.7:0/3940753652 >> 172.21.15.7:0/3940753652 conn(0x7f6e4c0fbb80 msgr2=0x7f6e4c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:51:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.181+0000 7f6e51eea640 1 -- 172.21.15.7:0/3940753652 shutdown_connections 2024-03-26T08:51:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:51:58.181+0000 7f6e51eea640 1 -- 172.21.15.7:0/3940753652 wait complete. 2024-03-26T08:51:58.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:51:58 smithi016 ceph-mon[35851]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 626 op/s 2024-03-26T08:51:58.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:51:58 smithi007 ceph-mon[32462]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 626 op/s 2024-03-26T08:51:58.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:51:58 smithi044 ceph-mon[36461]: pgmap v646: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 626 op/s 2024-03-26T08:52:00.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:00 smithi016 ceph-mon[35851]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 632 op/s 2024-03-26T08:52:00.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:00 smithi007 ceph-mon[32462]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 632 op/s 2024-03-26T08:52:00.819 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:00 smithi044 ceph-mon[36461]: pgmap v647: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 MiB/s wr, 632 op/s 2024-03-26T08:52:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 570.0 s, 1830.0 tps, lat 54.242 ms stddev 64.624 2024-03-26T08:52:02.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:02 smithi016 ceph-mon[35851]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 MiB/s wr, 674 op/s 2024-03-26T08:52:02.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:02 smithi007 ceph-mon[32462]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 MiB/s wr, 674 op/s 2024-03-26T08:52:02.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:02 smithi044 ceph-mon[36461]: pgmap v648: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 MiB/s wr, 674 op/s 2024-03-26T08:52:04.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:04 smithi016 ceph-mon[35851]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 699 op/s 2024-03-26T08:52:04.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:04 smithi007 ceph-mon[32462]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 699 op/s 2024-03-26T08:52:04.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:04 smithi044 ceph-mon[36461]: pgmap v649: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.9 MiB/s wr, 699 op/s 2024-03-26T08:52:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 575.0 s, 1569.6 tps, lat 64.243 ms stddev 74.946 2024-03-26T08:52:06.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:06 smithi016 ceph-mon[35851]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 617 op/s 2024-03-26T08:52:06.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:06 smithi007 ceph-mon[32462]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 617 op/s 2024-03-26T08:52:06.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:06 smithi044 ceph-mon[36461]: pgmap v650: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.9 MiB/s wr, 617 op/s 2024-03-26T08:52:06.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:52:06.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:52:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.043+0000 7f3369190640 1 -- 172.21.15.7:0/1684685191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33641086a0 msgr2=0x7f336410aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.043+0000 7f3369190640 1 --2- 172.21.15.7:0/1684685191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33641086a0 0x7f336410aad0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3354009900 tx=0x7f335402bc60 comp rx=0 tx=0).stop 2024-03-26T08:52:07.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.049+0000 7f3369190640 1 -- 172.21.15.7:0/1684685191 shutdown_connections 2024-03-26T08:52:07.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.049+0000 7f3369190640 1 --2- 172.21.15.7:0/1684685191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336410d9c0 0x7f336410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.049+0000 7f3369190640 1 --2- 172.21.15.7:0/1684685191 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f336410b010 0x7f336410d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.049+0000 7f3369190640 1 --2- 172.21.15.7:0/1684685191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33641086a0 0x7f336410aad0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.049+0000 7f3369190640 1 -- 172.21.15.7:0/1684685191 >> 172.21.15.7:0/1684685191 conn(0x7f336406e640 msgr2=0x7f3364070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.053+0000 7f3369190640 1 -- 172.21.15.7:0/1684685191 shutdown_connections 2024-03-26T08:52:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.053+0000 7f3369190640 1 -- 172.21.15.7:0/1684685191 wait complete. 2024-03-26T08:52:07.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.054+0000 7f3369190640 1 Processor -- start 2024-03-26T08:52:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.054+0000 7f3369190640 1 -- start start 2024-03-26T08:52:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.054+0000 7f3369190640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33641086a0 0x7f33641a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.054+0000 7f3369190640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336410b010 0x7f33641a6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.054+0000 7f3369190640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336410d9c0 0x7f33641add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.054+0000 7f3369190640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336411c150 con 0x7f336410b010 2024-03-26T08:52:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.054+0000 7f3369190640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336411bfd0 con 0x7f336410d9c0 2024-03-26T08:52:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.054+0000 7f3369190640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336411c2d0 con 0x7f33641086a0 2024-03-26T08:52:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.055+0000 7f336259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336410b010 0x7f33641a6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.055+0000 7f336259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336410b010 0x7f33641a6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40806/0 (socket says 172.21.15.7:40806) 2024-03-26T08:52:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.055+0000 7f336259c640 1 -- 172.21.15.7:0/4075701771 learned_addr learned my addr 172.21.15.7:0/4075701771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.055+0000 7f336359e640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336410d9c0 0x7f33641add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.055+0000 7f3362d9d640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33641086a0 0x7f33641a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.058+0000 7f336259c640 1 -- 172.21.15.7:0/4075701771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33641086a0 msgr2=0x7f33641a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.058+0000 7f336259c640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33641086a0 0x7f33641a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.058+0000 7f336259c640 1 -- 172.21.15.7:0/4075701771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336410d9c0 msgr2=0x7f33641add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.058+0000 7f336259c640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336410d9c0 0x7f33641add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.058+0000 7f336259c640 1 -- 172.21.15.7:0/4075701771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33641ae3f0 con 0x7f336410b010 2024-03-26T08:52:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.058+0000 7f3362d9d640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33641086a0 0x7f33641a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:52:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.059+0000 7f336259c640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336410b010 0x7f33641a6d10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f335800d950 tx=0x7f335800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.059+0000 7f334bfff640 1 -- 172.21.15.7:0/4075701771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358017070 con 0x7f336410b010 2024-03-26T08:52:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.060+0000 7f334bfff640 1 -- 172.21.15.7:0/4075701771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3358009c30 con 0x7f336410b010 2024-03-26T08:52:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.060+0000 7f334bfff640 1 -- 172.21.15.7:0/4075701771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335800f6b0 con 0x7f336410b010 2024-03-26T08:52:07.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.060+0000 7f3369190640 1 -- 172.21.15.7:0/4075701771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33641ae6e0 con 0x7f336410b010 2024-03-26T08:52:07.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.060+0000 7f3369190640 1 -- 172.21.15.7:0/4075701771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33641aebe0 con 0x7f336410b010 2024-03-26T08:52:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.063+0000 7f336bd7d740 1 Processor -- start 2024-03-26T08:52:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.064+0000 7f334bfff640 1 -- 172.21.15.7:0/4075701771 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f33580109d0 con 0x7f336410b010 2024-03-26T08:52:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.065+0000 7f334bfff640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f334c0a2980 0x7f334c0a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.066+0000 7f334bfff640 1 -- 172.21.15.7:0/4075701771 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3358099020 con 0x7f336410b010 2024-03-26T08:52:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336bd7d740 1 -- start start 2024-03-26T08:52:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be3e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 0x55f06be40d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336359e640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f334c0a2980 0x7f334c0a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336bd7d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06bd9e3d0 con 0x55f06be3e900 2024-03-26T08:52:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336bd7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06bd46ad0 con 0x55f06be38da0 2024-03-26T08:52:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336bd7d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06bd9a470 con 0x55f06be39790 2024-03-26T08:52:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be3e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be3e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49520/0 (socket says 172.21.15.7:49520) 2024-03-26T08:52:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336259c640 1 -- 172.21.15.7:0/1476442965 learned_addr learned my addr 172.21.15.7:0/1476442965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.067+0000 7f336359e640 1 --2- 172.21.15.7:0/1476442965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 0x55f06be40d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.068+0000 7f336259c640 1 -- 172.21.15.7:0/1476442965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 msgr2=0x55f06be391c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:52:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.068+0000 7f336259c640 1 --2- 172.21.15.7:0/1476442965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be391c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.068+0000 7f336259c640 1 -- 172.21.15.7:0/1476442965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 msgr2=0x55f06be40d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.068+0000 7f336259c640 1 --2- 172.21.15.7:0/1476442965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 0x55f06be40d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.068+0000 7f336259c640 1 -- 172.21.15.7:0/1476442965 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f06bb7c7d0 con 0x55f06be39790 2024-03-26T08:52:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.068+0000 7f336259c640 1 --2- 172.21.15.7:0/1476442965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be3e290 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f33580973e0 tx=0x7f33580a2170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45356d60d86b4cfb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.069+0000 7f33497fa640 1 -- 172.21.15.7:0/1476442965 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33580979b0 con 0x55f06be39790 2024-03-26T08:52:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.069+0000 7f33497fa640 1 -- 172.21.15.7:0/1476442965 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3358015e80 con 0x55f06be39790 2024-03-26T08:52:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.069+0000 7f33497fa640 1 -- 172.21.15.7:0/1476442965 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358097b50 con 0x55f06be39790 2024-03-26T08:52:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336bd7d740 1 -- 172.21.15.7:0/1476442965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 msgr2=0x55f06be3e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336bd7d740 1 --2- 172.21.15.7:0/1476442965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be3e290 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f33580973e0 tx=0x7f33580a2170 comp rx=0 tx=0).stop 2024-03-26T08:52:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336359e640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f334c0a2980 0x7f334c0a4e40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f335c00afd0 tx=0x7f335c00ad40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336bd7d740 1 -- 172.21.15.7:0/1476442965 shutdown_connections 2024-03-26T08:52:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336bd7d740 1 --2- 172.21.15.7:0/1476442965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 0x55f06be40d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336bd7d740 1 --2- 172.21.15.7:0/1476442965 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be3e290 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336bd7d740 1 --2- 172.21.15.7:0/1476442965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336bd7d740 1 -- 172.21.15.7:0/1476442965 >> 172.21.15.7:0/1476442965 conn(0x55f06be34310 msgr2=0x55f06be34710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336bd7d740 1 -- 172.21.15.7:0/1476442965 shutdown_connections 2024-03-26T08:52:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.070+0000 7f336bd7d740 1 -- 172.21.15.7:0/1476442965 wait complete. 2024-03-26T08:52:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.071+0000 7f336bd7d740 1 -- start start 2024-03-26T08:52:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.071+0000 7f336bd7d740 1 Processor -- start 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.071+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be54230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.071+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be54770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.071+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 0x55f06be5b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.071+0000 7f336bd7d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06be52770 con 0x55f06be3e900 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.071+0000 7f336bd7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06bd5a060 con 0x55f06be38da0 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.071+0000 7f336bd7d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f06be528f0 con 0x55f06be39790 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.071+0000 7f336bd7d740 10 client.0 _ll_register_callbacks cb 0x55f06bd3c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3362d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be54230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3362d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be54230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44090/0 (socket says 172.21.15.7:44090) 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3362d9d640 1 -- 172.21.15.7:0/3780010440 learned_addr learned my addr 172.21.15.7:0/3780010440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f336259c640 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be54770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f336359e640 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 0x55f06be5b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3362d9d640 1 -- 172.21.15.7:0/3780010440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 msgr2=0x55f06be54770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3362d9d640 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be54770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3362d9d640 1 -- 172.21.15.7:0/3780010440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 msgr2=0x55f06be5b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3362d9d640 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 0x55f06be5b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3362d9d640 1 -- 172.21.15.7:0/3780010440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f06bab23f0 con 0x55f06be38da0 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3362d9d640 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be54230 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3354009850 tx=0x7f3354004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a4153dbe4cd245c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.072+0000 7f3346ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.073+0000 7f3346ff5640 1 -- 172.21.15.7:0/3780010440 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354041070 con 0x55f06be38da0 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.073+0000 7f336bd7d740 1 -- 172.21.15.7:0/3780010440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3358015e80 con 0x55f06be38da0 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.073+0000 7f3346ff5640 1 -- 172.21.15.7:0/3780010440 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3354005240 con 0x55f06be38da0 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.073+0000 7f3346ff5640 1 -- 172.21.15.7:0/3780010440 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335404d050 con 0x55f06be38da0 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.073+0000 7f336bd7d740 10 client.27743 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.073+0000 7f336bd7d740 1 -- 172.21.15.7:0/3780010440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f06ba720f0 con 0x55f06be38da0 2024-03-26T08:52:07.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.076+0000 7f3346ff5640 1 -- 172.21.15.7:0/3780010440 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3358015e80 con 0x55f06be38da0 2024-03-26T08:52:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.076+0000 7f3346ff5640 1 -- 172.21.15.7:0/3780010440 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f33540035f0 con 0x55f06be38da0 2024-03-26T08:52:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.076+0000 7f336bd7d740 10 client.27743 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.076+0000 7f336bd7d740 10 client.27743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.076+0000 7f336bd7d740 20 client.27743 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.076+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f06be5fb60 0x55f06be61fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.076+0000 7f336bd7d740 4 client.27743 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.076+0000 7f336bd7d740 1 -- 172.21.15.7:0/3780010440 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f06bdab910 con 0x55f06be5fb60 2024-03-26T08:52:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.076+0000 7f336259c640 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f06be5fb60 0x55f06be61fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.077+0000 7f336259c640 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f06be5fb60 0x55f06be61fc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7659baaaa076f66d server_cookie=9e773fe6542dda38 in_seq=0 out_seq=0 2024-03-26T08:52:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.077+0000 7f3346ff5640 10 client.27743 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.078+0000 7f3346ff5640 1 -- 172.21.15.7:0/3780010440 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f06bdab910 con 0x55f06be5fb60 2024-03-26T08:52:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.078+0000 7f3346ff5640 10 client.27743 handle_command_reply: tid=0 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.078+0000 7f336bd7d740 1 client.27743 shutdown 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.078+0000 7f336bd7d740 1 -- 172.21.15.7:0/3780010440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 msgr2=0x55f06be54230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.078+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be54230 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3354009850 tx=0x7f3354004270 comp rx=0 tx=0).stop 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f336bd7d740 1 -- 172.21.15.7:0/3780010440 shutdown_connections 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f06be5fb60 0x55f06be61fc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be3e900 0x55f06be5b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be39790 0x55f06be54770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3780010440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be54230 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f336bd7d740 1 -- 172.21.15.7:0/3780010440 >> 172.21.15.7:0/3780010440 conn(0x55f06be34310 msgr2=0x55f06be33530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f3346ff5640 0 client.27743 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f336bd7d740 1 -- 172.21.15.7:0/3780010440 shutdown_connections 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f336bd7d740 1 -- 172.21.15.7:0/3780010440 wait complete. 2024-03-26T08:52:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.079+0000 7f336bd7d740 20 client.27743 trim_cache size 0 max 16384 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.103+0000 7f336bd7d740 1 Processor -- start 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336bd7d740 1 -- start start 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be38da0 0x55f06be37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be39790 0x55f06be5b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be3e900 0x55f06be66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336bd7d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33641b1640 con 0x55f06be39790 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336bd7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33641af140 con 0x55f06be3e900 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336bd7d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33641168a0 con 0x55f06be38da0 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be39790 0x55f06be5b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be39790 0x55f06be5b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40826/0 (socket says 172.21.15.7:40826) 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336259c640 1 -- 172.21.15.7:0/3947750994 learned_addr learned my addr 172.21.15.7:0/3947750994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f3362d9d640 1 --2- 172.21.15.7:0/3947750994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be38da0 0x55f06be37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336359e640 1 --2- 172.21.15.7:0/3947750994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be3e900 0x55f06be66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336259c640 1 -- 172.21.15.7:0/3947750994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be38da0 msgr2=0x55f06be37c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336259c640 1 --2- 172.21.15.7:0/3947750994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be38da0 0x55f06be37c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336259c640 1 -- 172.21.15.7:0/3947750994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be3e900 msgr2=0x55f06be66880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336259c640 1 --2- 172.21.15.7:0/3947750994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be3e900 0x55f06be66880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336259c640 1 -- 172.21.15.7:0/3947750994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f06be34d60 con 0x55f06be39790 2024-03-26T08:52:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.104+0000 7f336259c640 1 --2- 172.21.15.7:0/3947750994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be39790 0x55f06be5b690 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f335809ac80 tx=0x7f335800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c509207620431eb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.105+0000 7f3348ff9640 1 -- 172.21.15.7:0/3947750994 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358020040 con 0x55f06be39790 2024-03-26T08:52:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.105+0000 7f3348ff9640 1 -- 172.21.15.7:0/3947750994 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3358004470 con 0x55f06be39790 2024-03-26T08:52:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.105+0000 7f3348ff9640 1 -- 172.21.15.7:0/3947750994 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358014040 con 0x55f06be39790 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.106+0000 7f336bd7d740 1 -- 172.21.15.7:0/3947750994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be39790 msgr2=0x55f06be5b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.106+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3947750994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be39790 0x55f06be5b690 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f335809ac80 tx=0x7f335800bf60 comp rx=0 tx=0).stop 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.106+0000 7f336bd7d740 1 -- 172.21.15.7:0/3947750994 shutdown_connections 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.106+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3947750994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be3e900 0x55f06be66880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.106+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3947750994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be39790 0x55f06be5b690 secure :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f335809ac80 tx=0x7f335800bf60 comp rx=0 tx=0).stop 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.106+0000 7f336bd7d740 1 --2- 172.21.15.7:0/3947750994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be38da0 0x55f06be37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.106+0000 7f336bd7d740 1 -- 172.21.15.7:0/3947750994 >> 172.21.15.7:0/3947750994 conn(0x55f06be34310 msgr2=0x55f06be37570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.106+0000 7f336bd7d740 1 -- 172.21.15.7:0/3947750994 shutdown_connections 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 1 -- 172.21.15.7:0/3947750994 wait complete. 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 1 -- start start 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 1 Processor -- start 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be59370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be3e900 0x55f06be598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be5a8c0 0x55f06be5ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3364116ea0 con 0x55f06be5a8c0 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33641181c0 con 0x55f06be38da0 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3364117500 con 0x55f06be3e900 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336bd7d740 10 client.0 _ll_register_callbacks cb 0x7f3354009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be5a8c0 0x55f06be5ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be5a8c0 0x55f06be5ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40828/0 (socket says 172.21.15.7:40828) 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336359e640 1 -- 172.21.15.7:0/405140353 learned_addr learned my addr 172.21.15.7:0/405140353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f336259c640 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be3e900 0x55f06be598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.107+0000 7f3362d9d640 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be59370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f336359e640 1 -- 172.21.15.7:0/405140353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be3e900 msgr2=0x55f06be598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f336359e640 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be3e900 0x55f06be598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f336359e640 1 -- 172.21.15.7:0/405140353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 msgr2=0x55f06be59370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f336359e640 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be59370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f336359e640 1 -- 172.21.15.7:0/405140353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f06be5b280 con 0x55f06be5a8c0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f336359e640 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be5a8c0 0x55f06be5ad40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f335c011c40 tx=0x7f335c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c339a1ceec78c9ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f3346ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f3346ff5640 1 -- 172.21.15.7:0/405140353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335c02f060 con 0x55f06be5a8c0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f3346ff5640 1 -- 172.21.15.7:0/405140353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f335c01a070 con 0x55f06be5a8c0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f336bd7d740 1 -- 172.21.15.7:0/405140353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3358004470 con 0x55f06be5a8c0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f3346ff5640 1 -- 172.21.15.7:0/405140353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335c012d20 con 0x55f06be5a8c0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f336bd7d740 10 client.17934 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.108+0000 7f336bd7d740 1 -- 172.21.15.7:0/405140353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f06bebbe90 con 0x55f06be5a8c0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.109+0000 7f3346ff5640 1 -- 172.21.15.7:0/405140353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f335c00f900 con 0x55f06be5a8c0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.109+0000 7f3346ff5640 1 -- 172.21.15.7:0/405140353 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f335c034070 con 0x55f06be5a8c0 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.109+0000 7f336bd7d740 10 client.17934 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.109+0000 7f336bd7d740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.109+0000 7f336bd7d740 20 client.17934 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.109+0000 7f336bd7d740 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f06bebc850 0x55f06bebec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.109+0000 7f336bd7d740 4 client.17934 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.109+0000 7f336bd7d740 1 -- 172.21.15.7:0/405140353 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f3358004470 con 0x55f06bebc850 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.109+0000 7f3362d9d640 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f06bebc850 0x55f06bebec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f3362d9d640 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f06bebc850 0x55f06bebec70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c8bc145e44d613a server_cookie=5a0939c1bf3a5b08 in_seq=0 out_seq=0 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f3346ff5640 10 client.17934 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f3346ff5640 1 -- 172.21.15.7:0/405140353 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f3358004470 con 0x55f06bebc850 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f3346ff5640 10 client.17934 handle_command_reply: tid=0 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 client.17934 shutdown 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 -- 172.21.15.7:0/405140353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be5a8c0 msgr2=0x55f06be5ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be5a8c0 0x55f06be5ad40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f335c011c40 tx=0x7f335c004ab0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 -- 172.21.15.7:0/405140353 shutdown_connections 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f06bebc850 0x55f06bebec70 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f3346ff5640 0 client.17934 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f06be5a8c0 0x55f06be5ad40 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f06be3e900 0x55f06be598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 --2- 172.21.15.7:0/405140353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06be38da0 0x55f06be59370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 -- 172.21.15.7:0/405140353 >> 172.21.15.7:0/405140353 conn(0x55f06be34310 msgr2=0x55f06be32ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 -- 172.21.15.7:0/405140353 shutdown_connections 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 1 -- 172.21.15.7:0/405140353 wait complete. 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.110+0000 7f336bd7d740 20 client.17934 trim_cache size 0 max 16384 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:52:07.111 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 -- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f334c0a2980 msgr2=0x7f334c0a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f334c0a2980 0x7f334c0a4e40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f335c00afd0 tx=0x7f335c00ad40 comp rx=0 tx=0).stop 2024-03-26T08:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 -- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336410b010 msgr2=0x7f33641a6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336410b010 0x7f33641a6d10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f335800d950 tx=0x7f335800de10 comp rx=0 tx=0).stop 2024-03-26T08:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 -- 172.21.15.7:0/4075701771 shutdown_connections 2024-03-26T08:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f334c0a2980 0x7f334c0a4e40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336410d9c0 0x7f33641add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336410b010 0x7f33641a6d10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 --2- 172.21.15.7:0/4075701771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33641086a0 0x7f33641a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 -- 172.21.15.7:0/4075701771 >> 172.21.15.7:0/4075701771 conn(0x7f336406e640 msgr2=0x7f336410b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 -- 172.21.15.7:0/4075701771 shutdown_connections 2024-03-26T08:52:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.111+0000 7f3369190640 1 -- 172.21.15.7:0/4075701771 wait complete. 2024-03-26T08:52:07.121 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:52:07.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.277+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2195188622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd8109890 msgr2=0x7f7bd8110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.277+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2195188622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd8109890 0x7f7bd8110120 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009ca0 tx=0x7f7bd002c100 comp rx=0 tx=0).stop 2024-03-26T08:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2195188622 shutdown_connections 2024-03-26T08:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2195188622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd8109890 0x7f7bd8110120 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2195188622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd8108ed0 0x7f7bd8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2195188622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd8107ce0 0x7f7bd8108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2195188622 >> 172.21.15.7:0/2195188622 conn(0x7f7bd8076ca0 msgr2=0x7f7bd80790c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2195188622 shutdown_connections 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2195188622 wait complete. 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 Processor -- start 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 -- start start 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd8107ce0 0x7f7bd819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd8108ed0 0x7f7bd819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd8109890 0x7f7bd81a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8113940 con 0x7f7bd8108ed0 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd81137c0 con 0x7f7bd8109890 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.278+0000 7f7bdf0f1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd8113ac0 con 0x7f7bd8107ce0 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd8108ed0 0x7f7bd819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd8108ed0 0x7f7bd819e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40846/0 (socket says 172.21.15.7:40846) 2024-03-26T08:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bd7fff640 1 -- 172.21.15.7:0/2173674221 learned_addr learned my addr 172.21.15.7:0/2173674221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bdd68e640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd8109890 0x7f7bd81a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bdce8d640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd8107ce0 0x7f7bd819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bd7fff640 1 -- 172.21.15.7:0/2173674221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd8107ce0 msgr2=0x7f7bd819dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bd7fff640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd8107ce0 0x7f7bd819dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bd7fff640 1 -- 172.21.15.7:0/2173674221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd8109890 msgr2=0x7f7bd81a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bd7fff640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd8109890 0x7f7bd81a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.279+0000 7f7bd7fff640 1 -- 172.21.15.7:0/2173674221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd81a5c10 con 0x7f7bd8108ed0 2024-03-26T08:52:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.280+0000 7f7bd7fff640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd8108ed0 0x7f7bd819e530 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800e9f0 tx=0x7f7bc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.281+0000 7f7bd5ffb640 1 -- 172.21.15.7:0/2173674221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800cd80 con 0x7f7bd8108ed0 2024-03-26T08:52:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.281+0000 7f7bd5ffb640 1 -- 172.21.15.7:0/2173674221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7bc8010910 con 0x7f7bd8108ed0 2024-03-26T08:52:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.281+0000 7f7bd5ffb640 1 -- 172.21.15.7:0/2173674221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8018990 con 0x7f7bd8108ed0 2024-03-26T08:52:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.281+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd81a5f00 con 0x7f7bd8108ed0 2024-03-26T08:52:07.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.282+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd81a62b0 con 0x7f7bd8108ed0 2024-03-26T08:52:07.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.283+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd810efa0 con 0x7f7bd8108ed0 2024-03-26T08:52:07.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.284+0000 7f7bd5ffb640 1 -- 172.21.15.7:0/2173674221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7bc8010ac0 con 0x7f7bd8108ed0 2024-03-26T08:52:07.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.284+0000 7f7bd5ffb640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bc00a2880 0x7f7bc00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.285+0000 7f7bd5ffb640 1 -- 172.21.15.7:0/2173674221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7bc80cef50 con 0x7f7bd8108ed0 2024-03-26T08:52:07.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.285+0000 7f7bdce8d640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bc00a2880 0x7f7bc00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.285+0000 7f7bdce8d640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bc00a2880 0x7f7bc00a4d40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8108d30 tx=0x7f7bcc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.290+0000 7f7bd5ffb640 1 -- 172.21.15.7:0/2173674221 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7bc809bfb0 con 0x7f7bd8108ed0 2024-03-26T08:52:07.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.622+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bd802d070 con 0x7f7bd8108ed0 2024-03-26T08:52:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:52:07.621 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:52:07.621 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:52:07.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.625+0000 7f7bd5ffb640 1 -- 172.21.15.7:0/2173674221 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f7bc809b8b0 con 0x7f7bd8108ed0 2024-03-26T08:52:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bc00a2880 msgr2=0x7f7bc00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bc00a2880 0x7f7bc00a4d40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8108d30 tx=0x7f7bcc00b390 comp rx=0 tx=0).stop 2024-03-26T08:52:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd8108ed0 msgr2=0x7f7bd819e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd8108ed0 0x7f7bd819e530 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800e9f0 tx=0x7f7bc800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 shutdown_connections 2024-03-26T08:52:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7bc00a2880 0x7f7bc00a4d40 secure :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8108d30 tx=0x7f7bcc00b390 comp rx=0 tx=0).stop 2024-03-26T08:52:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bd8109890 0x7f7bd81a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd8108ed0 0x7f7bd819e530 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 --2- 172.21.15.7:0/2173674221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7bd8107ce0 0x7f7bd819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.629+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 >> 172.21.15.7:0/2173674221 conn(0x7f7bd8076ca0 msgr2=0x7f7bd810c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.631+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 shutdown_connections 2024-03-26T08:52:07.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.632+0000 7f7bdf0f1640 1 -- 172.21.15.7:0/2173674221 wait complete. 2024-03-26T08:52:07.655 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:52:07.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:52:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.797+0000 7f3236144640 1 -- 172.21.15.7:0/2351388065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230103280 msgr2=0x7f3230109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.797+0000 7f3236144640 1 --2- 172.21.15.7:0/2351388065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230103280 0x7f3230109b10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3224009900 tx=0x7f322402bc30 comp rx=0 tx=0).stop 2024-03-26T08:52:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.797+0000 7f3236144640 1 -- 172.21.15.7:0/2351388065 shutdown_connections 2024-03-26T08:52:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.797+0000 7f3236144640 1 --2- 172.21.15.7:0/2351388065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3230103280 0x7f3230109b10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.797+0000 7f3236144640 1 --2- 172.21.15.7:0/2351388065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32301028c0 0x7f3230102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.797+0000 7f3236144640 1 --2- 172.21.15.7:0/2351388065 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f32301016d0 0x7f3230101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.797+0000 7f3236144640 1 -- 172.21.15.7:0/2351388065 >> 172.21.15.7:0/2351388065 conn(0x7f32300fcc60 msgr2=0x7f32300ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.797+0000 7f3236144640 1 -- 172.21.15.7:0/2351388065 shutdown_connections 2024-03-26T08:52:07.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.797+0000 7f3236144640 1 -- 172.21.15.7:0/2351388065 wait complete. 2024-03-26T08:52:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f3236144640 1 Processor -- start 2024-03-26T08:52:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f3236144640 1 -- start start 2024-03-26T08:52:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f3236144640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301016d0 0x7f3230195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f3236144640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32301028c0 0x7f3230195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f3236144640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3230103280 0x7f32301ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f3236144640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323010d350 con 0x7f32301016d0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f3236144640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323010d1d0 con 0x7f32301028c0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f3236144640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323010d4d0 con 0x7f3230103280 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301016d0 0x7f3230195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301016d0 0x7f3230195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40854/0 (socket says 172.21.15.7:40854) 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322f7fe640 1 -- 172.21.15.7:0/1940954806 learned_addr learned my addr 172.21.15.7:0/1940954806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322ffff640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3230103280 0x7f32301ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322effd640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32301028c0 0x7f3230195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322f7fe640 1 -- 172.21.15.7:0/1940954806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3230103280 msgr2=0x7f32301ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322f7fe640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3230103280 0x7f32301ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322f7fe640 1 -- 172.21.15.7:0/1940954806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32301028c0 msgr2=0x7f3230195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322f7fe640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32301028c0 0x7f3230195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.799+0000 7f322f7fe640 1 -- 172.21.15.7:0/1940954806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323019ae60 con 0x7f32301016d0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.800+0000 7f322f7fe640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301016d0 0x7f3230195830 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9c0 tx=0x7f322000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.800+0000 7f322cff9640 1 -- 172.21.15.7:0/1940954806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000cd80 con 0x7f32301016d0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.800+0000 7f322cff9640 1 -- 172.21.15.7:0/1940954806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32200035f0 con 0x7f32301016d0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.800+0000 7f322cff9640 1 -- 172.21.15.7:0/1940954806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220010690 con 0x7f32301016d0 2024-03-26T08:52:07.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.800+0000 7f3236144640 1 -- 172.21.15.7:0/1940954806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32301ae7a0 con 0x7f32301016d0 2024-03-26T08:52:07.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.800+0000 7f3236144640 1 -- 172.21.15.7:0/1940954806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32301aeb50 con 0x7f32301016d0 2024-03-26T08:52:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f3238e05740 1 Processor -- start 2024-03-26T08:52:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f3238e05740 1 -- start start 2024-03-26T08:52:07.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f322cff9640 1 -- 172.21.15.7:0/1940954806 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f32200037a0 con 0x7f32301016d0 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f3238e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f0fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f3238e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 0x561f33f14b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f3238e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 0x561f33f17680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f3238e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f33ed2610 con 0x561f33f0f690 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f3238e05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f33e753c0 con 0x561f33f10080 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f3238e05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f33e1dd60 con 0x561f33f151f0 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f322ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f0fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f322ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f0fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40870/0 (socket says 172.21.15.7:40870) 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f322ffff640 1 -- 172.21.15.7:0/1264744108 learned_addr learned my addr 172.21.15.7:0/1264744108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f322effd640 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 0x561f33f17680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.803+0000 7f322f7fe640 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 0x561f33f14b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322effd640 1 -- 172.21.15.7:0/1264744108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 msgr2=0x561f33f14b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322effd640 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 0x561f33f14b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322effd640 1 -- 172.21.15.7:0/1264744108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 msgr2=0x561f33f0fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322cff9640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32180a2880 0x7f32180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322effd640 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f0fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322effd640 1 -- 172.21.15.7:0/1264744108 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f33b893f0 con 0x561f33f151f0 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322ffff640 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f0fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322effd640 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 0x561f33f17680 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3228005fd0 tx=0x7f32280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d256f39f4f75a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322cff9640 1 -- 172.21.15.7:0/1940954806 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f32200ced30 con 0x7f32301016d0 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.804+0000 7f322f7fe640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32180a2880 0x7f32180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.805+0000 7f3215ffb640 1 -- 172.21.15.7:0/1264744108 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228003c30 con 0x561f33f151f0 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.805+0000 7f3215ffb640 1 -- 172.21.15.7:0/1264744108 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3228003dd0 con 0x561f33f151f0 2024-03-26T08:52:07.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.805+0000 7f3215ffb640 1 -- 172.21.15.7:0/1264744108 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322800fb00 con 0x561f33f151f0 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 -- 172.21.15.7:0/1264744108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 msgr2=0x561f33f17680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f322f7fe640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32180a2880 0x7f32180a4d40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f32200cc830 tx=0x7f3220003f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 0x561f33f17680 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3228005fd0 tx=0x7f32280118d0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 -- 172.21.15.7:0/1264744108 shutdown_connections 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 0x561f33f17680 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 0x561f33f14b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 --2- 172.21.15.7:0/1264744108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f0fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 -- 172.21.15.7:0/1264744108 >> 172.21.15.7:0/1264744108 conn(0x561f33f0ac00 msgr2=0x561f33f0b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 -- 172.21.15.7:0/1264744108 shutdown_connections 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 -- 172.21.15.7:0/1264744108 wait complete. 2024-03-26T08:52:07.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 -- start start 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.806+0000 7f3238e05740 1 Processor -- start 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f3238e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f0f690 0x561f33f19a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f3238e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 0x561f33f19fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f3238e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f151f0 0x561f33f21010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f3238e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f33e31240 con 0x561f33f151f0 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f3238e05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f33e71460 con 0x561f33f0f690 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f3238e05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f33f181a0 con 0x561f33f10080 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f3238e05740 10 client.0 _ll_register_callbacks cb 0x561f33e30c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f0f690 0x561f33f19a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f0f690 0x561f33f19a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44164/0 (socket says 172.21.15.7:44164) 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 -- 172.21.15.7:0/3524311420 learned_addr learned my addr 172.21.15.7:0/3524311420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322effd640 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 0x561f33f19fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f151f0 0x561f33f21010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 -- 172.21.15.7:0/3524311420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 msgr2=0x561f33f19fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 0x561f33f19fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 -- 172.21.15.7:0/3524311420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f151f0 msgr2=0x561f33f21010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f151f0 0x561f33f21010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 -- 172.21.15.7:0/3524311420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f33c537d0 con 0x561f33f0f690 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322effd640 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 0x561f33f19fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f322ffff640 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f0f690 0x561f33f19a90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f322402c170 tx=0x7f322402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8624da742272d6ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.807+0000 7f320effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f320effd640 1 -- 172.21.15.7:0/3524311420 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322402ccc0 con 0x561f33f0f690 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f320effd640 1 -- 172.21.15.7:0/3524311420 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3224005330 con 0x561f33f0f690 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f3238e05740 1 -- 172.21.15.7:0/3524311420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3228003dd0 con 0x561f33f0f690 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f320effd640 1 -- 172.21.15.7:0/3524311420 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3224004ce0 con 0x561f33f0f690 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f3238e05740 10 client.27767 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:07.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f3238e05740 1 -- 172.21.15.7:0/3524311420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561f33f243f0 con 0x561f33f0f690 2024-03-26T08:52:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f320effd640 1 -- 172.21.15.7:0/3524311420 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3228003dd0 con 0x561f33f0f690 2024-03-26T08:52:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f320effd640 1 -- 172.21.15.7:0/3524311420 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f322404c420 con 0x561f33f0f690 2024-03-26T08:52:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f3238e05740 10 client.27767 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f3238e05740 10 client.27767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:52:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f3238e05740 20 client.27767 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f3238e05740 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x561f33f259d0 0x561f33f27e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f3238e05740 4 client.27767 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.808+0000 7f3238e05740 1 -- 172.21.15.7:0/3524311420 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f33e82900 con 0x561f33f259d0 2024-03-26T08:52:07.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.809+0000 7f322effd640 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x561f33f259d0 0x561f33f27e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.823+0000 7f322effd640 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x561f33f259d0 0x561f33f27e00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=397409aefc8017d1 server_cookie=ffd818d709314d5d in_seq=0 out_seq=0 2024-03-26T08:52:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.823+0000 7f320effd640 10 client.27767 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:52:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.828+0000 7f320effd640 1 -- 172.21.15.7:0/3524311420 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561f33e82900 con 0x561f33f259d0 2024-03-26T08:52:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.828+0000 7f320effd640 10 client.27767 handle_command_reply: tid=0 2024-03-26T08:52:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.828+0000 7f3238e05740 1 client.27767 shutdown 2024-03-26T08:52:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.828+0000 7f3238e05740 1 -- 172.21.15.7:0/3524311420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f0f690 msgr2=0x561f33f19a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.828+0000 7f3238e05740 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f0f690 0x561f33f19a90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f322402c170 tx=0x7f322402bc30 comp rx=0 tx=0).stop 2024-03-26T08:52:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.829+0000 7f3238e05740 1 -- 172.21.15.7:0/3524311420 shutdown_connections 2024-03-26T08:52:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.829+0000 7f3238e05740 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x561f33f259d0 0x561f33f27e00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.829+0000 7f320effd640 0 client.27767 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:52:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.829+0000 7f3238e05740 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f151f0 0x561f33f21010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.829+0000 7f3238e05740 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 0x561f33f19fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.829+0000 7f3238e05740 1 --2- 172.21.15.7:0/3524311420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f0f690 0x561f33f19a90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.829+0000 7f3238e05740 1 -- 172.21.15.7:0/3524311420 >> 172.21.15.7:0/3524311420 conn(0x561f33f0ac00 msgr2=0x561f33f09e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.830+0000 7f3238e05740 1 -- 172.21.15.7:0/3524311420 shutdown_connections 2024-03-26T08:52:07.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.830+0000 7f3238e05740 1 -- 172.21.15.7:0/3524311420 wait complete. 2024-03-26T08:52:07.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.830+0000 7f3238e05740 20 client.27767 trim_cache size 0 max 16384 2024-03-26T08:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f3238e05740 1 Processor -- start 2024-03-26T08:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f3238e05740 1 -- start start 2024-03-26T08:52:07.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f3238e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f19f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f3238e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 0x561f33f20fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f3238e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 0x561f33f0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f3238e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32301b9e30 con 0x561f33f0f690 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f3238e05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32301b7930 con 0x561f33f10080 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f3238e05740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32301b59b0 con 0x561f33f151f0 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f322ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f19f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f322ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f19f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40884/0 (socket says 172.21.15.7:40884) 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f322ffff640 1 -- 172.21.15.7:0/610290320 learned_addr learned my addr 172.21.15.7:0/610290320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.854+0000 7f322effd640 1 --2- 172.21.15.7:0/610290320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 0x561f33f20fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f322ffff640 1 --2- 172.21.15.7:0/610290320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 0x561f33f0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f322ffff640 1 -- 172.21.15.7:0/610290320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 msgr2=0x561f33f0e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f322ffff640 1 --2- 172.21.15.7:0/610290320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 0x561f33f0e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f322ffff640 1 -- 172.21.15.7:0/610290320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 msgr2=0x561f33f20fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f322ffff640 1 --2- 172.21.15.7:0/610290320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 0x561f33f20fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f322ffff640 1 -- 172.21.15.7:0/610290320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f33f11080 con 0x561f33f0f690 2024-03-26T08:52:07.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f322ffff640 1 --2- 172.21.15.7:0/610290320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f19f30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3224005ec0 tx=0x7f3224005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d00af257bd499ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f32157fa640 1 -- 172.21.15.7:0/610290320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322402cb70 con 0x561f33f0f690 2024-03-26T08:52:07.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f32157fa640 1 -- 172.21.15.7:0/610290320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3224043040 con 0x561f33f0f690 2024-03-26T08:52:07.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.855+0000 7f32157fa640 1 -- 172.21.15.7:0/610290320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322403e960 con 0x561f33f0f690 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.856+0000 7f3238e05740 1 -- 172.21.15.7:0/610290320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 msgr2=0x561f33f19f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.856+0000 7f3238e05740 1 --2- 172.21.15.7:0/610290320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f19f30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3224005ec0 tx=0x7f3224005970 comp rx=0 tx=0).stop 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.857+0000 7f3238e05740 1 -- 172.21.15.7:0/610290320 shutdown_connections 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.857+0000 7f3238e05740 1 --2- 172.21.15.7:0/610290320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f151f0 0x561f33f0e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.857+0000 7f3238e05740 1 --2- 172.21.15.7:0/610290320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f10080 0x561f33f20fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.857+0000 7f3238e05740 1 --2- 172.21.15.7:0/610290320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f19f30 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.857+0000 7f3238e05740 1 -- 172.21.15.7:0/610290320 >> 172.21.15.7:0/610290320 conn(0x561f33f0ac00 msgr2=0x561f33f0de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.857+0000 7f3238e05740 1 -- 172.21.15.7:0/610290320 shutdown_connections 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.857+0000 7f3238e05740 1 -- 172.21.15.7:0/610290320 wait complete. 2024-03-26T08:52:07.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.858+0000 7f3238e05740 1 -- start start 2024-03-26T08:52:07.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.858+0000 7f3238e05740 1 Processor -- start 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f3238e05740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f20870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f3238e05740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 0x561f33f20db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f3238e05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f151f0 0x561f33f1ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f3238e05740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32301b5fb0 con 0x561f33f0f690 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f20870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f20870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40886/0 (socket says 172.21.15.7:40886) 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322ffff640 1 -- 172.21.15.7:0/1164000784 learned_addr learned my addr 172.21.15.7:0/1164000784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32301b72d0 con 0x561f33f151f0 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32301b6610 con 0x561f33f10080 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f3238e05740 10 client.0 _ll_register_callbacks cb 0x7f3224004ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322effd640 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 0x561f33f20db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322ffff640 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f151f0 0x561f33f1ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322ffff640 1 -- 172.21.15.7:0/1164000784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 msgr2=0x561f33f20db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322ffff640 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 0x561f33f20db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322ffff640 1 -- 172.21.15.7:0/1164000784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f151f0 msgr2=0x561f33f1ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322ffff640 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f151f0 0x561f33f1ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.859+0000 7f322ffff640 1 -- 172.21.15.7:0/1164000784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f33f1f490 con 0x561f33f0f690 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f322ffff640 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f20870 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3224005ab0 tx=0x7f322403e400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f38229abfefb37b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f320effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f320effd640 1 -- 172.21.15.7:0/1164000784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224004050 con 0x561f33f0f690 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f320effd640 1 -- 172.21.15.7:0/1164000784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f32240041f0 con 0x561f33f0f690 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3224043040 con 0x561f33f0f690 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f320effd640 1 -- 172.21.15.7:0/1164000784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32240043f0 con 0x561f33f0f690 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f3238e05740 10 client.17958 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561f33f81bb0 con 0x561f33f0f690 2024-03-26T08:52:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f320effd640 1 -- 172.21.15.7:0/1164000784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3224043040 con 0x561f33f0f690 2024-03-26T08:52:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.860+0000 7f320effd640 1 -- 172.21.15.7:0/1164000784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f322403ebd0 con 0x561f33f0f690 2024-03-26T08:52:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f3238e05740 10 client.17958 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f3238e05740 10 client.17958 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f3238e05740 20 client.17958 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f3238e05740 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x561f33f259d0 0x561f33f84570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f3238e05740 4 client.17958 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561f33e7cc50 con 0x561f33f259d0 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f322effd640 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x561f33f259d0 0x561f33f84570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f322effd640 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x561f33f259d0 0x561f33f84570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=103cf712e40e76cd server_cookie=362dc26a93b76d28 in_seq=0 out_seq=0 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f320effd640 10 client.17958 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f320effd640 1 -- 172.21.15.7:0/1164000784 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561f33e7cc50 con 0x561f33f259d0 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f320effd640 10 client.17958 handle_command_reply: tid=0 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.862+0000 7f3238e05740 1 client.17958 shutdown 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 msgr2=0x561f33f20870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f20870 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3224005ab0 tx=0x7f322403e400 comp rx=0 tx=0).stop 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 shutdown_connections 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x561f33f259d0 0x561f33f84570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f33f151f0 0x561f33f1ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561f33f10080 0x561f33f20db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f320effd640 0 client.17958 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:52:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 --2- 172.21.15.7:0/1164000784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f33f0f690 0x561f33f20870 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 >> 172.21.15.7:0/1164000784 conn(0x561f33f0ac00 msgr2=0x561f33f19db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 shutdown_connections 2024-03-26T08:52:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 1 -- 172.21.15.7:0/1164000784 wait complete. 2024-03-26T08:52:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3238e05740 20 client.17958 trim_cache size 0 max 16384 2024-03-26T08:52:07.862 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3236144640 1 -- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32180a2880 msgr2=0x7f32180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3236144640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32180a2880 0x7f32180a4d40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f32200cc830 tx=0x7f3220003f10 comp rx=0 tx=0).stop 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3236144640 1 -- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301016d0 msgr2=0x7f3230195830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3236144640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301016d0 0x7f3230195830 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f322000e9c0 tx=0x7f322000ee80 comp rx=0 tx=0).stop 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.863+0000 7f3236144640 1 -- 172.21.15.7:0/1940954806 shutdown_connections 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.864+0000 7f3236144640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f32180a2880 0x7f32180a4d40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.864+0000 7f3236144640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3230103280 0x7f32301ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.864+0000 7f3236144640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32301028c0 0x7f3230195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.864+0000 7f3236144640 1 --2- 172.21.15.7:0/1940954806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32301016d0 0x7f3230195830 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.864+0000 7f3236144640 1 -- 172.21.15.7:0/1940954806 >> 172.21.15.7:0/1940954806 conn(0x7f32300fcc60 msgr2=0x7f32300738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.864+0000 7f3236144640 1 -- 172.21.15.7:0/1940954806 shutdown_connections 2024-03-26T08:52:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:07.864+0000 7f3236144640 1 -- 172.21.15.7:0/1940954806 wait complete. 2024-03-26T08:52:07.871 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:52:07.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.007+0000 7fbc13836640 1 -- 172.21.15.7:0/41083966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 msgr2=0x7fbc0c108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.008+0000 7fbc13836640 1 --2- 172.21.15.7:0/41083966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 0x7fbc0c108eb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009900 tx=0x7fbbfc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.008+0000 7fbc13836640 1 -- 172.21.15.7:0/41083966 shutdown_connections 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.008+0000 7fbc13836640 1 --2- 172.21.15.7:0/41083966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c10a0f0 0x7fbc0c1109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.008+0000 7fbc13836640 1 --2- 172.21.15.7:0/41083966 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc0c109750 0x7fbc0c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.008+0000 7fbc13836640 1 --2- 172.21.15.7:0/41083966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 0x7fbc0c108eb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.008+0000 7fbc13836640 1 -- 172.21.15.7:0/41083966 >> 172.21.15.7:0/41083966 conn(0x7fbc0c06e640 msgr2=0x7fbc0c070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.008+0000 7fbc13836640 1 -- 172.21.15.7:0/41083966 shutdown_connections 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.008+0000 7fbc13836640 1 -- 172.21.15.7:0/41083966 wait complete. 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc13836640 1 Processor -- start 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc13836640 1 -- start start 2024-03-26T08:52:08.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc13836640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 0x7fbc0c10c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc13836640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc0c109750 0x7fbc0c10ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc13836640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c10a0f0 0x7fbc0c10b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc13836640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c113930 con 0x7fbc0c108a90 2024-03-26T08:52:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc115d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 0x7fbc0c10c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc13836640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1137b0 con 0x7fbc0c10a0f0 2024-03-26T08:52:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc13836640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c113ab0 con 0x7fbc0c109750 2024-03-26T08:52:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc115d2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 0x7fbc0c10c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40906/0 (socket says 172.21.15.7:40906) 2024-03-26T08:52:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc115d2640 1 -- 172.21.15.7:0/1311842945 learned_addr learned my addr 172.21.15.7:0/1311842945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.009+0000 7fbc10dd1640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc0c109750 0x7fbc0c10ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.010+0000 7fbc115d2640 1 -- 172.21.15.7:0/1311842945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc0c109750 msgr2=0x7fbc0c10ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.010+0000 7fbc115d2640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc0c109750 0x7fbc0c10ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.010+0000 7fbc115d2640 1 -- 172.21.15.7:0/1311842945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c10a0f0 msgr2=0x7fbc0c10b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.010+0000 7fbc115d2640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c10a0f0 0x7fbc0c10b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.010+0000 7fbc115d2640 1 -- 172.21.15.7:0/1311842945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c10b900 con 0x7fbc0c108a90 2024-03-26T08:52:08.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.010+0000 7fbc115d2640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 0x7fbc0c10c4f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009230 tx=0x7fbbfc02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.010+0000 7fbc027fc640 1 -- 172.21.15.7:0/1311842945 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc041070 con 0x7fbc0c108a90 2024-03-26T08:52:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.010+0000 7fbc027fc640 1 -- 172.21.15.7:0/1311842945 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbfc004450 con 0x7fbc0c108a90 2024-03-26T08:52:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.010+0000 7fbc027fc640 1 -- 172.21.15.7:0/1311842945 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc003e20 con 0x7fbc0c108a90 2024-03-26T08:52:08.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.013+0000 7fbc13836640 1 -- 172.21.15.7:0/1311842945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0c1aa2b0 con 0x7fbc0c108a90 2024-03-26T08:52:08.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.013+0000 7fbc13836640 1 -- 172.21.15.7:0/1311842945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0c19fff0 con 0x7fbc0c108a90 2024-03-26T08:52:08.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.021+0000 7fbc027fc640 1 -- 172.21.15.7:0/1311842945 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fbbfc044070 con 0x7fbc0c108a90 2024-03-26T08:52:08.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.021+0000 7fbc027fc640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbf80a29d0 0x7fbbf80a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.022+0000 7fbc027fc640 1 -- 172.21.15.7:0/1311842945 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbbfc0f0120 con 0x7fbc0c108a90 2024-03-26T08:52:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.025+0000 7fbc16429740 1 Processor -- start 2024-03-26T08:52:08.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.026+0000 7fbc10dd1640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbf80a29d0 0x7fbbf80a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.027+0000 7fbc10dd1640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbf80a29d0 0x7fbbf80a4e90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c10c260 tx=0x7fbc08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.029+0000 7fbc16429740 1 -- start start 2024-03-26T08:52:08.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.029+0000 7fbc16429740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330aa690 0x563c330aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.029+0000 7fbc16429740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.029+0000 7fbc16429740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330b01f0 0x563c330b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.029+0000 7fbc16429740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3306d610 con 0x563c330ab080 2024-03-26T08:52:08.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.030+0000 7fbc16429740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c330103c0 con 0x563c330b01f0 2024-03-26T08:52:08.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.030+0000 7fbc16429740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c32fb8d60 con 0x563c330aa690 2024-03-26T08:52:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.032+0000 7fbc10dd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.032+0000 7fbc10dd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40918/0 (socket says 172.21.15.7:40918) 2024-03-26T08:52:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.032+0000 7fbc10dd1640 1 -- 172.21.15.7:0/258124154 learned_addr learned my addr 172.21.15.7:0/258124154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.032+0000 7fbc11dd3640 1 --2- 172.21.15.7:0/258124154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330b01f0 0x563c330b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.033+0000 7fbc115d2640 1 --2- 172.21.15.7:0/258124154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330aa690 0x563c330aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.038+0000 7fbc115d2640 1 -- 172.21.15.7:0/258124154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330b01f0 msgr2=0x563c330b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.038+0000 7fbc115d2640 1 --2- 172.21.15.7:0/258124154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330b01f0 0x563c330b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.038+0000 7fbc115d2640 1 -- 172.21.15.7:0/258124154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 msgr2=0x563c330afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.038+0000 7fbc115d2640 1 --2- 172.21.15.7:0/258124154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.038+0000 7fbc115d2640 1 -- 172.21.15.7:0/258124154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c32d243f0 con 0x563c330aa690 2024-03-26T08:52:08.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.043+0000 7fbc115d2640 1 --2- 172.21.15.7:0/258124154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330aa690 0x563c330aaab0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009430 tx=0x7fbbfc005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86d9d78e7448908f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.047+0000 7fbbf77fe640 1 -- 172.21.15.7:0/258124154 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc0d4020 con 0x563c330aa690 2024-03-26T08:52:08.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.049+0000 7fbbf77fe640 1 -- 172.21.15.7:0/258124154 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbfc0bd180 con 0x563c330aa690 2024-03-26T08:52:08.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.049+0000 7fbbf77fe640 1 -- 172.21.15.7:0/258124154 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc0d0e60 con 0x563c330aa690 2024-03-26T08:52:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.051+0000 7fbc16429740 1 -- 172.21.15.7:0/258124154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330aa690 msgr2=0x563c330aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.051+0000 7fbc16429740 1 --2- 172.21.15.7:0/258124154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330aa690 0x563c330aaab0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009430 tx=0x7fbbfc005e50 comp rx=0 tx=0).stop 2024-03-26T08:52:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.053+0000 7fbc16429740 1 -- 172.21.15.7:0/258124154 shutdown_connections 2024-03-26T08:52:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.053+0000 7fbc16429740 1 --2- 172.21.15.7:0/258124154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330b01f0 0x563c330b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.053+0000 7fbc16429740 1 --2- 172.21.15.7:0/258124154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.053+0000 7fbc16429740 1 --2- 172.21.15.7:0/258124154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330aa690 0x563c330aaab0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.053+0000 7fbc16429740 1 -- 172.21.15.7:0/258124154 >> 172.21.15.7:0/258124154 conn(0x563c330a5c00 msgr2=0x563c330a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.058+0000 7fbc16429740 1 -- 172.21.15.7:0/258124154 shutdown_connections 2024-03-26T08:52:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.063+0000 7fbc16429740 1 -- 172.21.15.7:0/258124154 wait complete. 2024-03-26T08:52:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.063+0000 7fbc16429740 1 -- start start 2024-03-26T08:52:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.063+0000 7fbc16429740 1 Processor -- start 2024-03-26T08:52:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.066+0000 7fbc16429740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.066+0000 7fbc16429740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.066+0000 7fbc16429740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 0x563c330bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.066+0000 7fbc16429740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c32fcc240 con 0x563c330b01f0 2024-03-26T08:52:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.066+0000 7fbc16429740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c3300c460 con 0x563c330aa690 2024-03-26T08:52:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.066+0000 7fbc16429740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c330b2f60 con 0x563c330ab080 2024-03-26T08:52:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.066+0000 7fbc16429740 10 client.0 _ll_register_callbacks cb 0x563c32fcbc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.067+0000 7fbc10dd1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.067+0000 7fbc10dd1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49656/0 (socket says 172.21.15.7:49656) 2024-03-26T08:52:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.067+0000 7fbc10dd1640 1 -- 172.21.15.7:0/82205101 learned_addr learned my addr 172.21.15.7:0/82205101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.067+0000 7fbc10dd1640 1 -- 172.21.15.7:0/82205101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 msgr2=0x563c330b47a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:52:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.067+0000 7fbc10dd1640 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b47a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.067+0000 7fbc10dd1640 1 -- 172.21.15.7:0/82205101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 msgr2=0x563c330bbd20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:52:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.067+0000 7fbc10dd1640 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 0x563c330bbd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.067+0000 7fbc10dd1640 1 -- 172.21.15.7:0/82205101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c32dee7d0 con 0x563c330ab080 2024-03-26T08:52:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.070+0000 7fbc10dd1640 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330b4ce0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbc08004610 tx=0x7fbc08004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4bbc775755c22c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.071+0000 7fbbf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:52:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.075+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/82205101 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0802a060 con 0x563c330ab080 2024-03-26T08:52:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.075+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/82205101 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc0801b640 con 0x563c330ab080 2024-03-26T08:52:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.075+0000 7fbc16429740 1 -- 172.21.15.7:0/82205101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbfc0bd180 con 0x563c330ab080 2024-03-26T08:52:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.077+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/82205101 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc0801b840 con 0x563c330ab080 2024-03-26T08:52:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.078+0000 7fbc16429740 10 client.27796 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.078+0000 7fbc16429740 1 -- 172.21.15.7:0/82205101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563c330d01c0 con 0x563c330ab080 2024-03-26T08:52:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.079+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/82205101 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbfc0bd180 con 0x563c330ab080 2024-03-26T08:52:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.079+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/82205101 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbc0802f070 con 0x563c330ab080 2024-03-26T08:52:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.079+0000 7fbc16429740 10 client.27796 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.079+0000 7fbc16429740 10 client.27796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:52:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.079+0000 7fbc16429740 20 client.27796 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.079+0000 7fbc16429740 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x563c330d1870 0x563c330d3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.079+0000 7fbc16429740 4 client.27796 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:08.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.079+0000 7fbc16429740 1 -- 172.21.15.7:0/82205101 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c3301d900 con 0x563c330d1870 2024-03-26T08:52:08.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.083+0000 7fbc115d2640 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x563c330d1870 0x563c330d3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.085+0000 7fbc115d2640 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x563c330d1870 0x563c330d3d10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4682f160e35f7651 server_cookie=a75c86bd39b4627e in_seq=0 out_seq=0 2024-03-26T08:52:08.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.085+0000 7fbbf4ff9640 10 client.27796 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:52:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.086+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/82205101 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563c3301d900 con 0x563c330d1870 2024-03-26T08:52:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.086+0000 7fbbf4ff9640 10 client.27796 handle_command_reply: tid=0 2024-03-26T08:52:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.086+0000 7fbc16429740 1 client.27796 shutdown 2024-03-26T08:52:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.086+0000 7fbc16429740 1 -- 172.21.15.7:0/82205101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 msgr2=0x563c330b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.086+0000 7fbc16429740 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330b4ce0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbc08004610 tx=0x7fbc08004730 comp rx=0 tx=0).stop 2024-03-26T08:52:08.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.087+0000 7fbc16429740 1 -- 172.21.15.7:0/82205101 shutdown_connections 2024-03-26T08:52:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.087+0000 7fbc16429740 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 0x563c330bbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.087+0000 7fbc16429740 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x563c330d1870 0x563c330d3d10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.087+0000 7fbc16429740 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330b4ce0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.087+0000 7fbc16429740 1 --2- 172.21.15.7:0/82205101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.087+0000 7fbc16429740 1 -- 172.21.15.7:0/82205101 >> 172.21.15.7:0/82205101 conn(0x563c330a5c00 msgr2=0x563c330a4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.087+0000 7fbbf4ff9640 0 client.27796 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:52:08.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.087+0000 7fbc16429740 1 -- 172.21.15.7:0/82205101 shutdown_connections 2024-03-26T08:52:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.087+0000 7fbc16429740 1 -- 172.21.15.7:0/82205101 wait complete. 2024-03-26T08:52:08.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.088+0000 7fbc16429740 20 client.27796 trim_cache size 0 max 16384 2024-03-26T08:52:08.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.111+0000 7fbc16429740 1 Processor -- start 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.111+0000 7fbc16429740 1 -- start start 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc16429740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc16429740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc16429740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 0x563c330b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc16429740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1b16b0 con 0x563c330b01f0 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc16429740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1af1b0 con 0x563c330aa690 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc16429740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1ad230 con 0x563c330ab080 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc11dd3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 0x563c330b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc10dd1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc11dd3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 0x563c330b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40926/0 (socket says 172.21.15.7:40926) 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc11dd3640 1 -- 172.21.15.7:0/630390220 learned_addr learned my addr 172.21.15.7:0/630390220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc115d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc11dd3640 1 -- 172.21.15.7:0/630390220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 msgr2=0x563c330bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc11dd3640 1 --2- 172.21.15.7:0/630390220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330bbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc11dd3640 1 -- 172.21.15.7:0/630390220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 msgr2=0x563c330b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc11dd3640 1 --2- 172.21.15.7:0/630390220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc11dd3640 1 -- 172.21.15.7:0/630390220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c330ac0a0 con 0x563c330b01f0 2024-03-26T08:52:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbc11dd3640 1 --2- 172.21.15.7:0/630390220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 0x563c330b3790 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009fb0 tx=0x7fbc04010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87162ab7647a62b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/630390220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04004d20 con 0x563c330b01f0 2024-03-26T08:52:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/630390220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc04003e50 con 0x563c330b01f0 2024-03-26T08:52:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.113+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/630390220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04005280 con 0x563c330b01f0 2024-03-26T08:52:08.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.113+0000 7fbc16429740 1 -- 172.21.15.7:0/630390220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 msgr2=0x563c330b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.113+0000 7fbc16429740 1 --2- 172.21.15.7:0/630390220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 0x563c330b3790 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009fb0 tx=0x7fbc04010860 comp rx=0 tx=0).stop 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.113+0000 7fbc16429740 1 -- 172.21.15.7:0/630390220 shutdown_connections 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.113+0000 7fbc16429740 1 --2- 172.21.15.7:0/630390220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330b01f0 0x563c330b3790 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.113+0000 7fbc16429740 1 --2- 172.21.15.7:0/630390220 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330ab080 0x563c330bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.113+0000 7fbc16429740 1 --2- 172.21.15.7:0/630390220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.113+0000 7fbc16429740 1 -- 172.21.15.7:0/630390220 >> 172.21.15.7:0/630390220 conn(0x563c330a5c00 msgr2=0x563c330b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.114+0000 7fbc16429740 1 -- 172.21.15.7:0/630390220 shutdown_connections 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.114+0000 7fbc16429740 1 -- 172.21.15.7:0/630390220 wait complete. 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.114+0000 7fbc16429740 1 -- start start 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.114+0000 7fbc16429740 1 Processor -- start 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc16429740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc16429740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330b9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc115d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc115d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44232/0 (socket says 172.21.15.7:44232) 2024-03-26T08:52:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc16429740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330b01f0 0x563c330ba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc10dd1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc115d2640 1 -- 172.21.15.7:0/3679119173 learned_addr learned my addr 172.21.15.7:0/3679119173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1ad830 con 0x563c330ab080 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1aeb50 con 0x563c330aa690 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1ade90 con 0x563c330b01f0 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc16429740 10 client.0 _ll_register_callbacks cb 0x7fbc080044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc11dd3640 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330b01f0 0x563c330ba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc10dd1640 1 -- 172.21.15.7:0/3679119173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330b01f0 msgr2=0x563c330ba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc10dd1640 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330b01f0 0x563c330ba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc10dd1640 1 -- 172.21.15.7:0/3679119173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 msgr2=0x563c330b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc10dd1640 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc10dd1640 1 -- 172.21.15.7:0/3679119173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c330bae20 con 0x563c330ab080 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc115d2640 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:08.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbc10dd1640 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330b9820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800a250 tx=0x7fbc0801b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb021afac2aa235 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.115+0000 7fbbf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/3679119173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0803e060 con 0x563c330ab080 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/3679119173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc0801b9e0 con 0x563c330ab080 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc04003e50 con 0x563c330ab080 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/3679119173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc08035040 con 0x563c330ab080 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbc16429740 10 client.17973 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563c3312d1a0 con 0x563c330ab080 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/3679119173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc04003e50 con 0x563c330ab080 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/3679119173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbc08043070 con 0x563c330ab080 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbc16429740 10 client.17973 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbc16429740 10 client.17973 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbc16429740 20 client.17973 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbc16429740 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x563c3312da90 0x563c3312feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbc16429740 4 client.17973 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:52:08.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.116+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c33017c50 con 0x563c3312da90 2024-03-26T08:52:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.117+0000 7fbc115d2640 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x563c3312da90 0x563c3312feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc115d2640 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x563c3312da90 0x563c3312feb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25fbb3144765aa0a server_cookie=7a6472efc7dbcf2a in_seq=0 out_seq=0 2024-03-26T08:52:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbbf4ff9640 10 client.17973 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:52:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbbf4ff9640 1 -- 172.21.15.7:0/3679119173 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c33017c50 con 0x563c3312da90 2024-03-26T08:52:08.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbbf4ff9640 10 client.17973 handle_command_reply: tid=0 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc16429740 1 client.17973 shutdown 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 msgr2=0x563c330b9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc16429740 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330b9820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800a250 tx=0x7fbc0801b7a0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 shutdown_connections 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc16429740 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x563c3312da90 0x563c3312feb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc16429740 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563c330b01f0 0x563c330ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc16429740 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563c330ab080 0x563c330b9820 secure :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800a250 tx=0x7fbc0801b7a0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc16429740 1 --2- 172.21.15.7:0/3679119173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563c330aa690 0x563c330b92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 >> 172.21.15.7:0/3679119173 conn(0x563c330a5c00 msgr2=0x563c330b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.118+0000 7fbbf4ff9640 0 client.17973 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.119+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 shutdown_connections 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.119+0000 7fbc16429740 1 -- 172.21.15.7:0/3679119173 wait complete. 2024-03-26T08:52:08.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.119+0000 7fbc16429740 20 client.17973 trim_cache size 0 max 16384 2024-03-26T08:52:08.119 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.119+0000 7fbc13836640 1 -- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbf80a29d0 msgr2=0x7fbbf80a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.119+0000 7fbc13836640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbf80a29d0 0x7fbbf80a4e90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c10c260 tx=0x7fbc08005f90 comp rx=0 tx=0).stop 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.119+0000 7fbc13836640 1 -- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 msgr2=0x7fbc0c10c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.120+0000 7fbc13836640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 0x7fbc0c10c4f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009230 tx=0x7fbbfc02c650 comp rx=0 tx=0).stop 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.120+0000 7fbc13836640 1 -- 172.21.15.7:0/1311842945 shutdown_connections 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.120+0000 7fbc13836640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbbf80a29d0 0x7fbbf80a4e90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.120+0000 7fbc13836640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c10a0f0 0x7fbc0c10b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.120+0000 7fbc13836640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbc0c109750 0x7fbc0c10ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.120+0000 7fbc13836640 1 --2- 172.21.15.7:0/1311842945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc0c108a90 0x7fbc0c10c4f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.120+0000 7fbc13836640 1 -- 172.21.15.7:0/1311842945 >> 172.21.15.7:0/1311842945 conn(0x7fbc0c06e640 msgr2=0x7fbc0c10ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.120+0000 7fbc13836640 1 -- 172.21.15.7:0/1311842945 shutdown_connections 2024-03-26T08:52:08.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.120+0000 7fbc13836640 1 -- 172.21.15.7:0/1311842945 wait complete. 2024-03-26T08:52:08.129 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:52:08.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:52:08.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 -- 172.21.15.7:0/3355727700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 msgr2=0x7f0ec4108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/3355727700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 0x7f0ec4108130 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009ca0 tx=0x7f0ebc02c140 comp rx=0 tx=0).stop 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 -- 172.21.15.7:0/3355727700 shutdown_connections 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/3355727700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 0x7f0ec4108130 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/3355727700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4100ee0 0x7f0ec4101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/3355727700 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4100040 0x7f0ec4100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 -- 172.21.15.7:0/3355727700 >> 172.21.15.7:0/3355727700 conn(0x7f0ec40fae50 msgr2=0x7f0ec40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 -- 172.21.15.7:0/3355727700 shutdown_connections 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 -- 172.21.15.7:0/3355727700 wait complete. 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 Processor -- start 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 -- start start 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4100040 0x7f0ec41979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4100ee0 0x7f0ec4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 0x7f0ec419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec410b150 con 0x7f0ec41018a0 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec410afd0 con 0x7f0ec4100ee0 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.276+0000 7f0ec9c80640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec410b2d0 con 0x7f0ec4100040 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 0x7f0ec419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 0x7f0ec419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40946/0 (socket says 172.21.15.7:40946) 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec3fff640 1 -- 172.21.15.7:0/1040568659 learned_addr learned my addr 172.21.15.7:0/1040568659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4100ee0 0x7f0ec4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec37fe640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4100040 0x7f0ec41979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec3fff640 1 -- 172.21.15.7:0/1040568659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4100040 msgr2=0x7f0ec41979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec3fff640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4100040 0x7f0ec41979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec3fff640 1 -- 172.21.15.7:0/1040568659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4100ee0 msgr2=0x7f0ec4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec3fff640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4100ee0 0x7f0ec4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec3fff640 1 -- 172.21.15.7:0/1040568659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec419f610 con 0x7f0ec41018a0 2024-03-26T08:52:08.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec3fff640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 0x7f0ec419efa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009bf0 tx=0x7f0ebc035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec17fa640 1 -- 172.21.15.7:0/1040568659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc034770 con 0x7f0ec41018a0 2024-03-26T08:52:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec17fa640 1 -- 172.21.15.7:0/1040568659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ebc032880 con 0x7f0ec41018a0 2024-03-26T08:52:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec9c80640 1 -- 172.21.15.7:0/1040568659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec4102100 con 0x7f0ec41018a0 2024-03-26T08:52:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec17fa640 1 -- 172.21.15.7:0/1040568659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc004280 con 0x7f0ec41018a0 2024-03-26T08:52:08.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.277+0000 7f0ec9c80640 1 -- 172.21.15.7:0/1040568659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec41025e0 con 0x7f0ec41018a0 2024-03-26T08:52:08.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.281+0000 7f0ecc860740 1 Processor -- start 2024-03-26T08:52:08.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.281+0000 7f0ec17fa640 1 -- 172.21.15.7:0/1040568659 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0ebc003970 con 0x7f0ec41018a0 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.281+0000 7f0ecc860740 1 -- start start 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.281+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0e690 0x55d732c0eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.281+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0f080 0x55d732c13b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.281+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c141f0 0x55d732c16680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.281+0000 7f0ecc860740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d732bd1610 con 0x55d732c0e690 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.281+0000 7f0ecc860740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d732b743c0 con 0x55d732c0f080 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.281+0000 7f0ecc860740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d732b1cd60 con 0x55d732c141f0 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ec37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0f080 0x55d732c13b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c141f0 0x55d732c16680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ec37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0f080 0x55d732c13b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44250/0 (socket says 172.21.15.7:44250) 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ec37fe640 1 -- 172.21.15.7:0/1341470069 learned_addr learned my addr 172.21.15.7:0/1341470069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ec17fa640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0eb00a2a70 0x7f0eb00a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/1341470069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0e690 0x55d732c0eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ebbfff640 1 -- 172.21.15.7:0/1341470069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0f080 msgr2=0x55d732c13b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/1341470069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0f080 0x55d732c13b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ebbfff640 1 -- 172.21.15.7:0/1341470069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0e690 msgr2=0x55d732c0eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/1341470069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0e690 0x55d732c0eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ebbfff640 1 -- 172.21.15.7:0/1341470069 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7328883f0 con 0x55d732c141f0 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ec17fa640 1 -- 172.21.15.7:0/1040568659 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0ebc0f8fe0 con 0x7f0ec41018a0 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ec3fff640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0eb00a2a70 0x7f0eb00a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.282+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/1341470069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c141f0 0x55d732c16680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4198f00 tx=0x7f0eac012040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9b671394ab67ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0eb9ffb640 1 -- 172.21.15.7:0/1341470069 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac004030 con 0x55d732c141f0 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0eb9ffb640 1 -- 172.21.15.7:0/1341470069 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eac005500 con 0x55d732c141f0 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0eb9ffb640 1 -- 172.21.15.7:0/1341470069 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac004c40 con 0x55d732c141f0 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ecc860740 1 -- 172.21.15.7:0/1341470069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c141f0 msgr2=0x55d732c16680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ec3fff640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0eb00a2a70 0x7f0eb00a4f30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc032ba0 tx=0x7f0ebc02c140 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ecc860740 1 --2- 172.21.15.7:0/1341470069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c141f0 0x55d732c16680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4198f00 tx=0x7f0eac012040 comp rx=0 tx=0).stop 2024-03-26T08:52:08.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ecc860740 1 -- 172.21.15.7:0/1341470069 shutdown_connections 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ecc860740 1 --2- 172.21.15.7:0/1341470069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c141f0 0x55d732c16680 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ecc860740 1 --2- 172.21.15.7:0/1341470069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0f080 0x55d732c13b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ecc860740 1 --2- 172.21.15.7:0/1341470069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0e690 0x55d732c0eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ecc860740 1 -- 172.21.15.7:0/1341470069 >> 172.21.15.7:0/1341470069 conn(0x55d732c09c00 msgr2=0x55d732c0a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ecc860740 1 -- 172.21.15.7:0/1341470069 shutdown_connections 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.283+0000 7f0ecc860740 1 -- 172.21.15.7:0/1341470069 wait complete. 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ecc860740 1 -- start start 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ecc860740 1 Processor -- start 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c18a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 0x55d732c18f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 0x55d732c1ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ec37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 0x55d732c18f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c18a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ecc860740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d732b30240 con 0x55d732c0f080 2024-03-26T08:52:08.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ecc860740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d732b70460 con 0x55d732c141f0 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ec37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 0x55d732c18f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40960/0 (socket says 172.21.15.7:40960) 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ecc860740 1 -- 172.21.15.7:0/2937024527 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d732c171a0 con 0x55d732c0e690 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ec37fe640 1 -- 172.21.15.7:0/2937024527 learned_addr learned my addr 172.21.15.7:0/2937024527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ecc860740 10 client.0 _ll_register_callbacks cb 0x55d732b2fc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.284+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 0x55d732c1ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ec37fe640 1 -- 172.21.15.7:0/2937024527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 msgr2=0x55d732c18a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ec37fe640 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c18a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ec37fe640 1 -- 172.21.15.7:0/2937024527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 msgr2=0x55d732c1ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ec37fe640 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 0x55d732c1ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ec37fe640 1 -- 172.21.15.7:0/2937024527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7329527d0 con 0x55d732c0f080 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ec37fe640 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 0x55d732c18f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e6e0 tx=0x7f0eb400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8890045d2bdf3181 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/2937024527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb401e060 con 0x55d732c0f080 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/2937024527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0eb4004260 con 0x55d732c0f080 2024-03-26T08:52:08.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.285+0000 7f0ecc860740 1 -- 172.21.15.7:0/2937024527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0eac005500 con 0x55d732c0f080 2024-03-26T08:52:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.286+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/2937024527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eb4004460 con 0x55d732c0f080 2024-03-26T08:52:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.286+0000 7f0ecc860740 10 client.17985 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.287+0000 7f0ecc860740 1 -- 172.21.15.7:0/2937024527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d732c233b0 con 0x55d732c0f080 2024-03-26T08:52:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.287+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/2937024527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eac005500 con 0x55d732c0f080 2024-03-26T08:52:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.287+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/2937024527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0eb4023070 con 0x55d732c0f080 2024-03-26T08:52:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.287+0000 7f0ecc860740 10 client.17985 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.287+0000 7f0ecc860740 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.287+0000 7f0ecc860740 20 client.17985 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:08.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.287+0000 7f0ecc860740 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d732c24ab0 0x55d732c26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.288+0000 7f0ecc860740 4 client.17985 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.288+0000 7f0ecc860740 1 -- 172.21.15.7:0/2937024527 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d732b81900 con 0x55d732c24ab0 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.288+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d732c24ab0 0x55d732c26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.288+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d732c24ab0 0x55d732c26f10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=172ae712a6a5dde9 server_cookie=d3c5c5366b194ef8 in_seq=0 out_seq=0 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.288+0000 7f0ea2ffd640 10 client.17985 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/2937024527 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d732b81900 con 0x55d732c24ab0 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ea2ffd640 10 client.17985 handle_command_reply: tid=0 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 client.17985 shutdown 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 -- 172.21.15.7:0/2937024527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 msgr2=0x55d732c18f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 0x55d732c18f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e6e0 tx=0x7f0eb400eba0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 -- 172.21.15.7:0/2937024527 shutdown_connections 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d732c24ab0 0x55d732c26f10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ea2ffd640 0 client.17985 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 0x55d732c1ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 0x55d732c18f70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 --2- 172.21.15.7:0/2937024527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c18a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 -- 172.21.15.7:0/2937024527 >> 172.21.15.7:0/2937024527 conn(0x55d732c09c00 msgr2=0x55d732c08e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 -- 172.21.15.7:0/2937024527 shutdown_connections 2024-03-26T08:52:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.289+0000 7f0ecc860740 1 -- 172.21.15.7:0/2937024527 wait complete. 2024-03-26T08:52:08.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.290+0000 7f0ecc860740 20 client.17985 trim_cache size 0 max 16384 2024-03-26T08:52:08.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.305+0000 7f0ecc860740 1 Processor -- start 2024-03-26T08:52:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ecc860740 1 -- start start 2024-03-26T08:52:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c08590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 0x55d732c1ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 0x55d732c08ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ecc860740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41a8de0 con 0x55d732c0f080 2024-03-26T08:52:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c08590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ecc860740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41a68e0 con 0x55d732c141f0 2024-03-26T08:52:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ecc860740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41a4960 con 0x55d732c0e690 2024-03-26T08:52:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c08590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49708/0 (socket says 172.21.15.7:49708) 2024-03-26T08:52:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ebbfff640 1 -- 172.21.15.7:0/1089503122 learned_addr learned my addr 172.21.15.7:0/1089503122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.306+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/1089503122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 0x55d732c08ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.307+0000 7f0ebbfff640 1 -- 172.21.15.7:0/1089503122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 msgr2=0x55d732c08ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.307+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/1089503122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 0x55d732c08ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.307+0000 7f0ebbfff640 1 -- 172.21.15.7:0/1089503122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 msgr2=0x55d732c1ff70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.307+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/1089503122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 0x55d732c1ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.307+0000 7f0ebbfff640 1 -- 172.21.15.7:0/1089503122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d732c05480 con 0x55d732c0e690 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.307+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/1089503122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c08590 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009230 tx=0x7f0eac010480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3d2c25ed465921b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.308+0000 7f0eb97fa640 1 -- 172.21.15.7:0/1089503122 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac037020 con 0x55d732c0e690 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.308+0000 7f0eb97fa640 1 -- 172.21.15.7:0/1089503122 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eac005480 con 0x55d732c0e690 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.308+0000 7f0eb97fa640 1 -- 172.21.15.7:0/1089503122 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac027040 con 0x55d732c0e690 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.308+0000 7f0ecc860740 1 -- 172.21.15.7:0/1089503122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 msgr2=0x55d732c08590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.308+0000 7f0ecc860740 1 --2- 172.21.15.7:0/1089503122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c08590 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009230 tx=0x7f0eac010480 comp rx=0 tx=0).stop 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.308+0000 7f0ecc860740 1 -- 172.21.15.7:0/1089503122 shutdown_connections 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.308+0000 7f0ecc860740 1 --2- 172.21.15.7:0/1089503122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c141f0 0x55d732c08ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.308+0000 7f0ecc860740 1 --2- 172.21.15.7:0/1089503122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c0f080 0x55d732c1ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.309+0000 7f0ecc860740 1 --2- 172.21.15.7:0/1089503122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0e690 0x55d732c08590 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.309+0000 7f0ecc860740 1 -- 172.21.15.7:0/1089503122 >> 172.21.15.7:0/1089503122 conn(0x55d732c09c00 msgr2=0x55d732c0f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.309+0000 7f0ecc860740 1 -- 172.21.15.7:0/1089503122 shutdown_connections 2024-03-26T08:52:08.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.309+0000 7f0ecc860740 1 -- 172.21.15.7:0/1089503122 wait complete. 2024-03-26T08:52:08.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.309+0000 7f0ecc860740 1 -- start start 2024-03-26T08:52:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.309+0000 7f0ecc860740 1 Processor -- start 2024-03-26T08:52:08.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.309+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0e690 0x55d732c1daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.309+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0f080 0x55d732c1dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ecc860740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c141f0 0x55d732c1eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ecc860740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41a4f60 con 0x55d732c141f0 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ecc860740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41a6280 con 0x55d732c0e690 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ecc860740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41a55c0 con 0x55d732c0f080 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ecc860740 10 client.0 _ll_register_callbacks cb 0x7f0eb400aea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ec37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0f080 0x55d732c1dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0e690 0x55d732c1daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ec37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0f080 0x55d732c1dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49710/0 (socket says 172.21.15.7:49710) 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ec37fe640 1 -- 172.21.15.7:0/471686305 learned_addr learned my addr 172.21.15.7:0/471686305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0e690 0x55d732c1daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44272/0 (socket says 172.21.15.7:44272) 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c141f0 0x55d732c1eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ec37fe640 1 -- 172.21.15.7:0/471686305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0e690 msgr2=0x55d732c1daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ec37fe640 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0e690 0x55d732c1daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ec37fe640 1 -- 172.21.15.7:0/471686305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c141f0 msgr2=0x55d732c1eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ec37fe640 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c141f0 0x55d732c1eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ec37fe640 1 -- 172.21.15.7:0/471686305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d732c1f5e0 con 0x55d732c0f080 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ec37fe640 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0f080 0x55d732c1dfe0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000c00 tx=0x7f0eb4003ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc59b42fd9257444 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.310+0000 7f0ea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.311+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/471686305 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4029040 con 0x55d732c0f080 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.311+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/471686305 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0eb4005040 con 0x55d732c0f080 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.311+0000 7f0ecc860740 1 -- 172.21.15.7:0/471686305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0eac005480 con 0x55d732c0f080 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.311+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/471686305 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eb4005240 con 0x55d732c0f080 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.311+0000 7f0ecc860740 10 client.27820 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.311+0000 7f0ecc860740 1 -- 172.21.15.7:0/471686305 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d732c80750 con 0x55d732c0f080 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.311+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/471686305 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0eac005480 con 0x55d732c0f080 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.312+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/471686305 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0eb4023030 con 0x55d732c0f080 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.312+0000 7f0ecc860740 10 client.27820 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:08.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.312+0000 7f0ecc860740 10 client.27820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.312+0000 7f0ecc860740 20 client.27820 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.312+0000 7f0ecc860740 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d732c24ab0 0x55d732c83110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.312+0000 7f0ecc860740 4 client.27820 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:52:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.312+0000 7f0ecc860740 1 -- 172.21.15.7:0/471686305 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d732c13f60 con 0x55d732c24ab0 2024-03-26T08:52:08.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.312+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d732c24ab0 0x55d732c83110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ea2ffd640 10 client.27820 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ebbfff640 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d732c24ab0 0x55d732c83110 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f94c00a3f7911a server_cookie=90b4e01162f41989 in_seq=0 out_seq=0 2024-03-26T08:52:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ea2ffd640 1 -- 172.21.15.7:0/471686305 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d732c13f60 con 0x55d732c24ab0 2024-03-26T08:52:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ea2ffd640 10 client.27820 handle_command_reply: tid=0 2024-03-26T08:52:08.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ecc860740 1 client.27820 shutdown 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ecc860740 1 -- 172.21.15.7:0/471686305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0f080 msgr2=0x55d732c1dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ecc860740 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0f080 0x55d732c1dfe0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000c00 tx=0x7f0eb4003ac0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ecc860740 1 -- 172.21.15.7:0/471686305 shutdown_connections 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ecc860740 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d732c24ab0 0x55d732c83110 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ecc860740 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d732c141f0 0x55d732c1eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ea2ffd640 0 client.27820 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.314+0000 7f0ecc860740 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d732c0f080 0x55d732c1dfe0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ecc860740 1 --2- 172.21.15.7:0/471686305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d732c0e690 0x55d732c1daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ecc860740 1 -- 172.21.15.7:0/471686305 >> 172.21.15.7:0/471686305 conn(0x55d732c09c00 msgr2=0x55d732c0ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ecc860740 1 -- 172.21.15.7:0/471686305 shutdown_connections 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ecc860740 1 -- 172.21.15.7:0/471686305 wait complete. 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ecc860740 20 client.27820 trim_cache size 0 max 16384 2024-03-26T08:52:08.313 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:52:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 -- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0eb00a2a70 msgr2=0x7f0eb00a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0eb00a2a70 0x7f0eb00a4f30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc032ba0 tx=0x7f0ebc02c140 comp rx=0 tx=0).stop 2024-03-26T08:52:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 -- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 msgr2=0x7f0ec419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 0x7f0ec419efa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009bf0 tx=0x7f0ebc035d10 comp rx=0 tx=0).stop 2024-03-26T08:52:08.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 -- 172.21.15.7:0/1040568659 shutdown_connections 2024-03-26T08:52:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0eb00a2a70 0x7f0eb00a4f30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ec41018a0 0x7f0ec419efa0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4100ee0 0x7f0ec4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 --2- 172.21.15.7:0/1040568659 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0ec4100040 0x7f0ec41979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 -- 172.21.15.7:0/1040568659 >> 172.21.15.7:0/1040568659 conn(0x7f0ec40fae50 msgr2=0x7f0ec4106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 -- 172.21.15.7:0/1040568659 shutdown_connections 2024-03-26T08:52:08.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:08.315+0000 7f0ec9c80640 1 -- 172.21.15.7:0/1040568659 wait complete. 2024-03-26T08:52:08.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:08 smithi007 ceph-mon[32462]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 651 op/s 2024-03-26T08:52:08.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:08 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2173674221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:52:08.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:08 smithi016 ceph-mon[35851]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 651 op/s 2024-03-26T08:52:08.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:08 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2173674221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:52:08.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:08 smithi044 ceph-mon[36461]: pgmap v651: 97 pgs: 97 active+clean; 1.6 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 651 op/s 2024-03-26T08:52:08.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:08 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2173674221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:52:09.326 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:52:09.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:52:09.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:52:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.473+0000 7fe6cb2de640 1 -- 172.21.15.7:0/2106294051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41028c0 msgr2=0x7fe6c4102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.473+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/2106294051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41028c0 0x7fe6c4102d40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009900 tx=0x7fe6b402bc30 comp rx=0 tx=0).stop 2024-03-26T08:52:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.473+0000 7fe6cb2de640 1 -- 172.21.15.7:0/2106294051 shutdown_connections 2024-03-26T08:52:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.473+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/2106294051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4103280 0x7fe6c4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.473+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/2106294051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41028c0 0x7fe6c4102d40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.473+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/2106294051 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c41016d0 0x7fe6c4101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.473+0000 7fe6cb2de640 1 -- 172.21.15.7:0/2106294051 >> 172.21.15.7:0/2106294051 conn(0x7fe6c40fcc60 msgr2=0x7fe6c40ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.474+0000 7fe6cb2de640 1 -- 172.21.15.7:0/2106294051 shutdown_connections 2024-03-26T08:52:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.474+0000 7fe6cb2de640 1 -- 172.21.15.7:0/2106294051 wait complete. 2024-03-26T08:52:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.474+0000 7fe6cb2de640 1 Processor -- start 2024-03-26T08:52:09.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.474+0000 7fe6cb2de640 1 -- start start 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6cb2de640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41016d0 0x7fe6c4070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6cb2de640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c41028c0 0x7fe6c40705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6cb2de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4103280 0x7fe6c41a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6cb2de640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c410d380 con 0x7fe6c41016d0 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6cb2de640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c410d200 con 0x7fe6c4103280 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6cb2de640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c410d500 con 0x7fe6c41028c0 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c907a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41016d0 0x7fe6c4070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c907a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41016d0 0x7fe6c4070080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40986/0 (socket says 172.21.15.7:40986) 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c907a640 1 -- 172.21.15.7:0/280579374 learned_addr learned my addr 172.21.15.7:0/280579374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c987b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4103280 0x7fe6c41a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c8879640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c41028c0 0x7fe6c40705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c987b640 1 -- 172.21.15.7:0/280579374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c41028c0 msgr2=0x7fe6c40705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c987b640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c41028c0 0x7fe6c40705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c987b640 1 -- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41016d0 msgr2=0x7fe6c4070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c987b640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41016d0 0x7fe6c4070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c987b640 1 -- 172.21.15.7:0/280579374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c41a7fd0 con 0x7fe6c4103280 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c907a640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41016d0 0x7fe6c4070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c8879640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c41028c0 0x7fe6c40705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.475+0000 7fe6c987b640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4103280 0x7fe6c41a7a90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009fd0 tx=0x7fe6bc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.476+0000 7fe6ba7fc640 1 -- 172.21.15.7:0/280579374 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc00ce20 con 0x7fe6c4103280 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.476+0000 7fe6cb2de640 1 -- 172.21.15.7:0/280579374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c41a81a0 con 0x7fe6c4103280 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.476+0000 7fe6ba7fc640 1 -- 172.21.15.7:0/280579374 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc005020 con 0x7fe6c4103280 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.476+0000 7fe6cb2de640 1 -- 172.21.15.7:0/280579374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c41a8660 con 0x7fe6c4103280 2024-03-26T08:52:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.476+0000 7fe6ba7fc640 1 -- 172.21.15.7:0/280579374 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc010720 con 0x7fe6c4103280 2024-03-26T08:52:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.479+0000 7fe6cdf9b740 1 Processor -- start 2024-03-26T08:52:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.479+0000 7fe6ba7fc640 1 -- 172.21.15.7:0/280579374 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe6bc0109c0 con 0x7fe6c4103280 2024-03-26T08:52:09.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6cdf9b740 1 -- start start 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7df970 0x55a91d7e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e22a0 0x55a91d7e4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 0x55a91d7e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91d716c90 con 0x55a91d7e4d00 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91d73c070 con 0x55a91d7e22a0 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91d6e4c80 con 0x55a91d7df970 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6c907a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e22a0 0x55a91d7e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6c8879640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 0x55a91d7e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6c907a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e22a0 0x55a91d7e4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44306/0 (socket says 172.21.15.7:44306) 2024-03-26T08:52:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6c907a640 1 -- 172.21.15.7:0/1333078151 learned_addr learned my addr 172.21.15.7:0/1333078151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.480+0000 7fe6c8879640 1 --2- 172.21.15.7:0/1333078151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7df970 0x55a91d7e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6c8879640 1 -- 172.21.15.7:0/1333078151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7df970 msgr2=0x55a91d7e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6c8879640 1 --2- 172.21.15.7:0/1333078151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7df970 0x55a91d7e1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6c8879640 1 -- 172.21.15.7:0/1333078151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e22a0 msgr2=0x55a91d7e4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6c8879640 1 --2- 172.21.15.7:0/1333078151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e22a0 0x55a91d7e4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6c8879640 1 -- 172.21.15.7:0/1333078151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a91d6dcd10 con 0x55a91d7e4d00 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6ba7fc640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6b00a2900 0x7fe6b00a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6c907a640 1 --2- 172.21.15.7:0/1333078151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e22a0 0x55a91d7e4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6c8879640 1 --2- 172.21.15.7:0/1333078151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 0x55a91d7e7150 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4036040 tx=0x7fe6b4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6622d7c35e0e07 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6ba7fc640 1 -- 172.21.15.7:0/280579374 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6bc0cf0b0 con 0x7fe6c4103280 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6c907a640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6b00a2900 0x7fe6b00a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6af7fe640 1 -- 172.21.15.7:0/1333078151 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4004240 con 0x55a91d7e4d00 2024-03-26T08:52:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6af7fe640 1 -- 172.21.15.7:0/1333078151 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6b40043e0 con 0x55a91d7e4d00 2024-03-26T08:52:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.481+0000 7fe6af7fe640 1 -- 172.21.15.7:0/1333078151 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4003cd0 con 0x55a91d7e4d00 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6c907a640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6b00a2900 0x7fe6b00a4dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55a91d7d5d20 tx=0x7fe6c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/1333078151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 msgr2=0x55a91d7e7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/1333078151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 0x55a91d7e7150 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4036040 tx=0x7fe6b4005b50 comp rx=0 tx=0).stop 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/1333078151 shutdown_connections 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/1333078151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 0x55a91d7e7150 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/1333078151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e22a0 0x55a91d7e4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/1333078151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7df970 0x55a91d7e1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/1333078151 >> 172.21.15.7:0/1333078151 conn(0x55a91d79f1a0 msgr2=0x55a91d7d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/1333078151 shutdown_connections 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.482+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/1333078151 wait complete. 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.483+0000 7fe6cdf9b740 1 -- start start 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.483+0000 7fe6cdf9b740 1 Processor -- start 2024-03-26T08:52:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7e9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 0x55a91d7e9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 0x55a91d7f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91d6f80d0 con 0x55a91d7e4d00 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91d738110 con 0x55a91d7df970 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91d7e7c10 con 0x55a91d7e22a0 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c907a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7e9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c907a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7e9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44308/0 (socket says 172.21.15.7:44308) 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c907a640 1 -- 172.21.15.7:0/470384061 learned_addr learned my addr 172.21.15.7:0/470384061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c8879640 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 0x55a91d7e9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6cdf9b740 10 client.0 _ll_register_callbacks cb 0x55a91d733e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c987b640 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 0x55a91d7f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c8879640 1 -- 172.21.15.7:0/470384061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 msgr2=0x55a91d7e9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c8879640 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7e9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c8879640 1 -- 172.21.15.7:0/470384061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 msgr2=0x55a91d7f0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c8879640 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 0x55a91d7f0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c8879640 1 -- 172.21.15.7:0/470384061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a91d5187e0 con 0x55a91d7e22a0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.484+0000 7fe6c907a640 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7e9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6c8879640 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 0x55a91d7e9a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe6b402c7b0 tx=0x7fe6b402c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f23a17d3bf0485f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6acff9640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6acff9640 1 -- 172.21.15.7:0/470384061 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4051020 con 0x55a91d7e22a0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/470384061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6b40043e0 con 0x55a91d7e22a0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6acff9640 1 -- 172.21.15.7:0/470384061 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6b40046c0 con 0x55a91d7e22a0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6acff9640 1 -- 172.21.15.7:0/470384061 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b4003e60 con 0x55a91d7e22a0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6cdf9b740 10 client.27832 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/470384061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a91d578f70 con 0x55a91d7e22a0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6acff9640 1 -- 172.21.15.7:0/470384061 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6b40043e0 con 0x55a91d7e22a0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6acff9640 1 -- 172.21.15.7:0/470384061 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe6b4056070 con 0x55a91d7e22a0 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6cdf9b740 10 client.27832 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6cdf9b740 10 client.27832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6cdf9b740 20 client.27832 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a91d7f4f50 0x55a91d7f7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6cdf9b740 4 client.27832 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.485+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/470384061 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a91d7495b0 con 0x55a91d7f4f50 2024-03-26T08:52:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.486+0000 7fe6c907a640 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a91d7f4f50 0x55a91d7f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.486+0000 7fe6acff9640 10 client.27832 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.486+0000 7fe6c907a640 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a91d7f4f50 0x55a91d7f7340 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1951854688c3f580 server_cookie=ec3a5bd725e15598 in_seq=0 out_seq=0 2024-03-26T08:52:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6acff9640 1 -- 172.21.15.7:0/470384061 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a91d7495b0 con 0x55a91d7f4f50 2024-03-26T08:52:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6acff9640 10 client.27832 handle_command_reply: tid=0 2024-03-26T08:52:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 client.27832 shutdown 2024-03-26T08:52:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/470384061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 msgr2=0x55a91d7e9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 0x55a91d7e9a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe6b402c7b0 tx=0x7fe6b402c8a0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/470384061 shutdown_connections 2024-03-26T08:52:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a91d7f4f50 0x55a91d7f7340 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e4d00 0x55a91d7f0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 0x55a91d7e9a40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/470384061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7e9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6acff9640 0 client.27832 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/470384061 >> 172.21.15.7:0/470384061 conn(0x55a91d79f1a0 msgr2=0x55a91d7df1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/470384061 shutdown_connections 2024-03-26T08:52:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/470384061 wait complete. 2024-03-26T08:52:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.487+0000 7fe6cdf9b740 20 client.27832 trim_cache size 0 max 16384 2024-03-26T08:52:09.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.521+0000 7fe6cdf9b740 1 Processor -- start 2024-03-26T08:52:09.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.521+0000 7fe6cdf9b740 1 -- start start 2024-03-26T08:52:09.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.522+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7df970 0x55a91d7df070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.522+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 0x55a91d7f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.522+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e4d00 0x55a91d7fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.522+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a8010890 con 0x55a91d7df970 2024-03-26T08:52:09.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.522+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800e390 con 0x55a91d7e4d00 2024-03-26T08:52:09.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.522+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800c410 con 0x55a91d7e22a0 2024-03-26T08:52:09.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.523+0000 7fe6c987b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e4d00 0x55a91d7fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.523+0000 7fe6c987b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e4d00 0x55a91d7fbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44314/0 (socket says 172.21.15.7:44314) 2024-03-26T08:52:09.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.523+0000 7fe6c987b640 1 -- 172.21.15.7:0/81284636 learned_addr learned my addr 172.21.15.7:0/81284636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.525+0000 7fe6c8879640 1 --2- 172.21.15.7:0/81284636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 0x55a91d7f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.525+0000 7fe6c907a640 1 --2- 172.21.15.7:0/81284636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7df970 0x55a91d7df070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.525+0000 7fe6c987b640 1 -- 172.21.15.7:0/81284636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 msgr2=0x55a91d7f0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.525+0000 7fe6c987b640 1 --2- 172.21.15.7:0/81284636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 0x55a91d7f0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.525+0000 7fe6c987b640 1 -- 172.21.15.7:0/81284636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7df970 msgr2=0x55a91d7df070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.525+0000 7fe6c987b640 1 --2- 172.21.15.7:0/81284636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7df970 0x55a91d7df070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.525+0000 7fe6c987b640 1 -- 172.21.15.7:0/81284636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a91d410170 con 0x55a91d7e4d00 2024-03-26T08:52:09.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.533+0000 7fe6c987b640 1 --2- 172.21.15.7:0/81284636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e4d00 0x55a91d7fbac0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc005fd0 tx=0x7fe6bc09c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53ec83124971b788 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.536+0000 7fe6aeffd640 1 -- 172.21.15.7:0/81284636 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0b8020 con 0x55a91d7e4d00 2024-03-26T08:52:09.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.536+0000 7fe6aeffd640 1 -- 172.21.15.7:0/81284636 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc0a46b0 con 0x55a91d7e4d00 2024-03-26T08:52:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.537+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/81284636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e4d00 msgr2=0x55a91d7fbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.537+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/81284636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e4d00 0x55a91d7fbac0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc005fd0 tx=0x7fe6bc09c6d0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.538+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/81284636 shutdown_connections 2024-03-26T08:52:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.538+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/81284636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7e4d00 0x55a91d7fbac0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.538+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/81284636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e22a0 0x55a91d7f0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.538+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/81284636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7df970 0x55a91d7df070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.538+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/81284636 >> 172.21.15.7:0/81284636 conn(0x55a91d79f1a0 msgr2=0x55a91d7de960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.547+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/81284636 shutdown_connections 2024-03-26T08:52:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.551+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/81284636 wait complete. 2024-03-26T08:52:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.551+0000 7fe6cdf9b740 1 -- start start 2024-03-26T08:52:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.551+0000 7fe6cdf9b740 1 Processor -- start 2024-03-26T08:52:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.552+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7ee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.552+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e22a0 0x55a91d7eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.552+0000 7fe6cdf9b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e4d00 0x55a91d7efe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.552+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800ca10 con 0x55a91d7e22a0 2024-03-26T08:52:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.552+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800dd30 con 0x55a91d7df970 2024-03-26T08:52:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.552+0000 7fe6cdf9b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800d070 con 0x55a91d7e4d00 2024-03-26T08:52:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.552+0000 7fe6c987b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e4d00 0x55a91d7efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.552+0000 7fe6c987b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e4d00 0x55a91d7efe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49760/0 (socket says 172.21.15.7:49760) 2024-03-26T08:52:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.552+0000 7fe6c987b640 1 -- 172.21.15.7:0/4285972545 learned_addr learned my addr 172.21.15.7:0/4285972545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.553+0000 7fe6cdf9b740 10 client.0 _ll_register_callbacks cb 0x7fe6c0009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.553+0000 7fe6c8879640 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e22a0 0x55a91d7eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.553+0000 7fe6c907a640 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7ee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.553+0000 7fe6c8879640 1 -- 172.21.15.7:0/4285972545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e4d00 msgr2=0x55a91d7efe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.553+0000 7fe6c8879640 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e4d00 0x55a91d7efe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.554+0000 7fe6c8879640 1 -- 172.21.15.7:0/4285972545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 msgr2=0x55a91d7ee9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.554+0000 7fe6c8879640 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7ee9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.554+0000 7fe6c8879640 1 -- 172.21.15.7:0/4285972545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a91d7f0430 con 0x55a91d7e22a0 2024-03-26T08:52:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.554+0000 7fe6c907a640 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7ee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.554+0000 7fe6c8879640 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e22a0 0x55a91d7eef20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe6b402c170 tx=0x7fe6b40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e151abb5efb883 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.554+0000 7fe6acff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.555+0000 7fe6acff9640 1 -- 172.21.15.7:0/4285972545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4051070 con 0x55a91d7e22a0 2024-03-26T08:52:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.555+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/4285972545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6bc0a46b0 con 0x55a91d7e22a0 2024-03-26T08:52:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.555+0000 7fe6acff9640 1 -- 172.21.15.7:0/4285972545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6b4004ed0 con 0x55a91d7e22a0 2024-03-26T08:52:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.559+0000 7fe6acff9640 1 -- 172.21.15.7:0/4285972545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b405b030 con 0x55a91d7e22a0 2024-03-26T08:52:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.562+0000 7fe6cdf9b740 10 client.18006 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.562+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/4285972545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a91d850e70 con 0x55a91d7e22a0 2024-03-26T08:52:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.563+0000 7fe6acff9640 1 -- 172.21.15.7:0/4285972545 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc0a46b0 con 0x55a91d7e22a0 2024-03-26T08:52:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.563+0000 7fe6acff9640 1 -- 172.21.15.7:0/4285972545 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe6b4056070 con 0x55a91d7e22a0 2024-03-26T08:52:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.564+0000 7fe6cdf9b740 10 client.18006 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.564+0000 7fe6cdf9b740 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.564+0000 7fe6cdf9b740 20 client.18006 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.564+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a91d851760 0x55a91d853b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.564+0000 7fe6cdf9b740 4 client.18006 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:52:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.564+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/4285972545 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a91d743900 con 0x55a91d851760 2024-03-26T08:52:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.568+0000 7fe6c907a640 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a91d851760 0x55a91d853b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.570+0000 7fe6c907a640 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a91d851760 0x55a91d853b80 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c63c5b543046c55 server_cookie=404e433b2307e89d in_seq=0 out_seq=0 2024-03-26T08:52:09.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.570+0000 7fe6acff9640 10 client.18006 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.572+0000 7fe6acff9640 1 -- 172.21.15.7:0/4285972545 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a91d743900 con 0x55a91d851760 2024-03-26T08:52:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.572+0000 7fe6acff9640 10 client.18006 handle_command_reply: tid=0 2024-03-26T08:52:09.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.573+0000 7fe6cdf9b740 1 client.18006 shutdown 2024-03-26T08:52:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.575+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/4285972545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e22a0 msgr2=0x55a91d7eef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.575+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e22a0 0x55a91d7eef20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe6b402c170 tx=0x7fe6b40057c0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/4285972545 shutdown_connections 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a91d851760 0x55a91d853b80 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a91d7e4d00 0x55a91d7efe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a91d7e22a0 0x55a91d7eef20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6cdf9b740 1 --2- 172.21.15.7:0/4285972545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a91d7df970 0x55a91d7ee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/4285972545 >> 172.21.15.7:0/4285972545 conn(0x55a91d79f1a0 msgr2=0x55a91d7df1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6acff9640 0 client.18006 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/4285972545 shutdown_connections 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6cdf9b740 1 -- 172.21.15.7:0/4285972545 wait complete. 2024-03-26T08:52:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.576+0000 7fe6cdf9b740 20 client.18006 trim_cache size 0 max 16384 2024-03-26T08:52:09.573 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:52:09.573 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:52:09.573 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "4664724c-954e-4494-95bd-2e3128f5e8b2", 2024-03-26T08:52:09.573 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:52:09.573 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:52:09.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.577+0000 7fe6cb2de640 1 -- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6b00a2900 msgr2=0x7fe6b00a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.577+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6b00a2900 0x7fe6b00a4dc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55a91d7d5d20 tx=0x7fe6c0005f90 comp rx=0 tx=0).stop 2024-03-26T08:52:09.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.577+0000 7fe6cb2de640 1 -- 172.21.15.7:0/280579374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4103280 msgr2=0x7fe6c41a7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.577+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4103280 0x7fe6c41a7a90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009fd0 tx=0x7fe6bc00ef10 comp rx=0 tx=0).stop 2024-03-26T08:52:09.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.578+0000 7fe6cb2de640 1 -- 172.21.15.7:0/280579374 shutdown_connections 2024-03-26T08:52:09.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.578+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6b00a2900 0x7fe6b00a4dc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.578+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6c4103280 0x7fe6c41a7a90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.578+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe6c41028c0 0x7fe6c40705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.578+0000 7fe6cb2de640 1 --2- 172.21.15.7:0/280579374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6c41016d0 0x7fe6c4070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.578+0000 7fe6cb2de640 1 -- 172.21.15.7:0/280579374 >> 172.21.15.7:0/280579374 conn(0x7fe6c40fcc60 msgr2=0x7fe6c4105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.578+0000 7fe6cb2de640 1 -- 172.21.15.7:0/280579374 shutdown_connections 2024-03-26T08:52:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.578+0000 7fe6cb2de640 1 -- 172.21.15.7:0/280579374 wait complete. 2024-03-26T08:52:09.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:52:09.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:52:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.745+0000 7f6d17db9640 1 -- 172.21.15.7:0/1176160084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d1006d5e0 msgr2=0x7f6d1006da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.745+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1176160084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d1006d5e0 0x7f6d1006da00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c01a910 tx=0x7f6d0c03cca0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.746+0000 7f6d17db9640 1 -- 172.21.15.7:0/1176160084 shutdown_connections 2024-03-26T08:52:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.746+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1176160084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d1006f1c0 0x7f6d1006fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.746+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1176160084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1006e820 0x7f6d1006ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.746+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1176160084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d1006d5e0 0x7f6d1006da00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.746+0000 7f6d17db9640 1 -- 172.21.15.7:0/1176160084 >> 172.21.15.7:0/1176160084 conn(0x7f6d1006c170 msgr2=0x7f6d1006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.746+0000 7f6d17db9640 1 -- 172.21.15.7:0/1176160084 shutdown_connections 2024-03-26T08:52:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.746+0000 7f6d17db9640 1 -- 172.21.15.7:0/1176160084 wait complete. 2024-03-26T08:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.747+0000 7f6d17db9640 1 Processor -- start 2024-03-26T08:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.747+0000 7f6d17db9640 1 -- start start 2024-03-26T08:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.747+0000 7f6d17db9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d1006d5e0 0x7f6d1019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d17db9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1006e820 0x7f6d1019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d17db9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d1006f1c0 0x7f6d101a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d17db9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1010da50 con 0x7f6d1006d5e0 2024-03-26T08:52:09.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d17db9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1010d8d0 con 0x7f6d1006e820 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d17db9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1010dbd0 con 0x7f6d1006f1c0 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15b55640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d1006d5e0 0x7f6d1019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1006e820 0x7f6d1019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15354640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1006e820 0x7f6d1019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44360/0 (socket says 172.21.15.7:44360) 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15b55640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d1006d5e0 0x7f6d1019e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41046/0 (socket says 172.21.15.7:41046) 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15354640 1 -- 172.21.15.7:0/1958717265 learned_addr learned my addr 172.21.15.7:0/1958717265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d16356640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d1006f1c0 0x7f6d101a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15354640 1 -- 172.21.15.7:0/1958717265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d1006f1c0 msgr2=0x7f6d101a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15354640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d1006f1c0 0x7f6d101a5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15354640 1 -- 172.21.15.7:0/1958717265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d1006d5e0 msgr2=0x7f6d1019e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15354640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d1006d5e0 0x7f6d1019e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.748+0000 7f6d15354640 1 -- 172.21.15.7:0/1958717265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d101a5de0 con 0x7f6d1006e820 2024-03-26T08:52:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.749+0000 7f6d15354640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1006e820 0x7f6d1019e700 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b450 tx=0x7f6d0000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.749+0000 7f6d06ffd640 1 -- 172.21.15.7:0/1958717265 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00015020 con 0x7f6d1006e820 2024-03-26T08:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.749+0000 7f6d17db9640 1 -- 172.21.15.7:0/1958717265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d101a60d0 con 0x7f6d1006e820 2024-03-26T08:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.749+0000 7f6d06ffd640 1 -- 172.21.15.7:0/1958717265 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d00011d70 con 0x7f6d1006e820 2024-03-26T08:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.749+0000 7f6d17db9640 1 -- 172.21.15.7:0/1958717265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d101a66f0 con 0x7f6d1006e820 2024-03-26T08:52:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.750+0000 7f6d06ffd640 1 -- 172.21.15.7:0/1958717265 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0000f340 con 0x7f6d1006e820 2024-03-26T08:52:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.753+0000 7f6d1a9a5740 1 Processor -- start 2024-03-26T08:52:09.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.754+0000 7f6d06ffd640 1 -- 172.21.15.7:0/1958717265 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6d00018020 con 0x7f6d1006e820 2024-03-26T08:52:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d1a9a5740 1 -- start start 2024-03-26T08:52:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d06ffd640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cfc0a2950 0x7f6cfc0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 0x55cc549311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54931790 0x55cc54936290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d06ffd640 1 -- 172.21.15.7:0/1958717265 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6d000ce420 con 0x7f6d1006e820 2024-03-26T08:52:09.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54936900 0x55cc54938d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d1a9a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc548963d0 con 0x55cc54936900 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d1a9a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc5483ead0 con 0x55cc54930da0 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d16356640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cfc0a2950 0x7f6cfc0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d1a9a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc54892470 con 0x55cc54931790 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d16356640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54936900 0x55cc54938d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d16356640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54936900 0x55cc54938d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41056/0 (socket says 172.21.15.7:41056) 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d16356640 1 -- 172.21.15.7:0/1817112299 learned_addr learned my addr 172.21.15.7:0/1817112299 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d15354640 1 --2- 172.21.15.7:0/1817112299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54931790 0x55cc54936290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d15b55640 1 --2- 172.21.15.7:0/1817112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 0x55cc549311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d15354640 1 -- 172.21.15.7:0/1817112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 msgr2=0x55cc549311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d15354640 1 --2- 172.21.15.7:0/1817112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 0x55cc549311c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d15354640 1 -- 172.21.15.7:0/1817112299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54936900 msgr2=0x55cc54938d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.755+0000 7f6d15354640 1 --2- 172.21.15.7:0/1817112299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54936900 0x55cc54938d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.756+0000 7f6d15354640 1 -- 172.21.15.7:0/1817112299 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc546747d0 con 0x55cc54931790 2024-03-26T08:52:09.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.756+0000 7f6d15354640 1 --2- 172.21.15.7:0/1817112299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54931790 0x55cc54936290 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6d0009ad10 tx=0x7f6d0009a980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5b149fcde234fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.756+0000 7f6d16356640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cfc0a2950 0x7f6cfc0a4e10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009fd0 tx=0x7f6d08009f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.756+0000 7f6cfbfff640 1 -- 172.21.15.7:0/1817112299 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0009eed0 con 0x55cc54931790 2024-03-26T08:52:09.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.756+0000 7f6cfbfff640 1 -- 172.21.15.7:0/1817112299 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d000a3520 con 0x55cc54931790 2024-03-26T08:52:09.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.756+0000 7f6cfbfff640 1 -- 172.21.15.7:0/1817112299 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0009a430 con 0x55cc54931790 2024-03-26T08:52:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.757+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1817112299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54931790 msgr2=0x55cc54936290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.757+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1817112299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54931790 0x55cc54936290 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6d0009ad10 tx=0x7f6d0009a980 comp rx=0 tx=0).stop 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.757+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1817112299 shutdown_connections 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.757+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1817112299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54936900 0x55cc54938d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.757+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1817112299 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54931790 0x55cc54936290 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.757+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1817112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 0x55cc549311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.757+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1817112299 >> 172.21.15.7:0/1817112299 conn(0x55cc5492c310 msgr2=0x55cc5492c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.757+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1817112299 shutdown_connections 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.757+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1817112299 wait complete. 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d1a9a5740 1 -- start start 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d1a9a5740 1 Processor -- start 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 0x55cc5493b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54931790 0x55cc5493b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54936900 0x55cc549427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d1a9a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc549398b0 con 0x55cc54931790 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d1a9a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc54852060 con 0x55cc54930da0 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d1a9a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc54939a30 con 0x55cc54936900 2024-03-26T08:52:09.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d1a9a5740 10 client.0 _ll_register_callbacks cb 0x55cc548346a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d15354640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54931790 0x55cc5493b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d15b55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 0x55cc5493b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d15b55640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 0x55cc5493b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44372/0 (socket says 172.21.15.7:44372) 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d15b55640 1 -- 172.21.15.7:0/1324612210 learned_addr learned my addr 172.21.15.7:0/1324612210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.758+0000 7f6d16356640 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54936900 0x55cc549427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6d15354640 1 -- 172.21.15.7:0/1324612210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54936900 msgr2=0x55cc549427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6d15354640 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54936900 0x55cc549427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6d15354640 1 -- 172.21.15.7:0/1324612210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 msgr2=0x55cc5493b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6d15354640 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 0x55cc5493b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6d15354640 1 -- 172.21.15.7:0/1324612210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc545aa3f0 con 0x55cc54931790 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6d15354640 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54931790 0x55cc5493b780 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6d0009ad10 tx=0x7f6d00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b13b9f20bfeeaa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6cf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1324612210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0007d020 con 0x55cc54931790 2024-03-26T08:52:09.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1324612210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6d000cbf60 con 0x55cc54931790 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1324612210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d000a3520 con 0x55cc54931790 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1324612210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d0009eed0 con 0x55cc54931790 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6d1a9a5740 10 client.18018 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.759+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1324612210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cc5456a0f0 con 0x55cc54931790 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.760+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1324612210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d000a3520 con 0x55cc54931790 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.760+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1324612210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6d00096b60 con 0x55cc54931790 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.761+0000 7f6d1a9a5740 10 client.18018 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.761+0000 7f6d1a9a5740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.761+0000 7f6d1a9a5740 20 client.18018 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.761+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc54946920 0x55cc54948d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.761+0000 7f6d1a9a5740 4 client.18018 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:09.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.761+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1324612210 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc548a3910 con 0x55cc54946920 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.762+0000 7f6d15b55640 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc54946920 0x55cc54948d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.762+0000 7f6d15b55640 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc54946920 0x55cc54948d10 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c93b8cfda45fd8ae server_cookie=78b5f3da0062cf14 in_seq=0 out_seq=0 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.762+0000 7f6cf97fa640 10 client.18018 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.763+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1324612210 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cc548a3910 con 0x55cc54946920 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.763+0000 7f6cf97fa640 10 client.18018 handle_command_reply: tid=0 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.763+0000 7f6d1a9a5740 1 client.18018 shutdown 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.763+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1324612210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54931790 msgr2=0x55cc5493b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.763+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54931790 0x55cc5493b780 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6d0009ad10 tx=0x7f6d00004750 comp rx=0 tx=0).stop 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1324612210 shutdown_connections 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc54946920 0x55cc54948d10 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54936900 0x55cc549427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54931790 0x55cc5493b780 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1324612210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54930da0 0x55cc5493b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1324612210 >> 172.21.15.7:0/1324612210 conn(0x55cc5492c310 msgr2=0x55cc5492b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6cf97fa640 0 client.18018 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:09.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1324612210 shutdown_connections 2024-03-26T08:52:09.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1324612210 wait complete. 2024-03-26T08:52:09.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.764+0000 7f6d1a9a5740 20 client.18018 trim_cache size 0 max 16384 2024-03-26T08:52:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.786+0000 7f6d1a9a5740 1 Processor -- start 2024-03-26T08:52:09.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.786+0000 7f6d1a9a5740 1 -- start start 2024-03-26T08:52:09.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.786+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc5492c310 0x55cc54927390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.786+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54930da0 0x55cc54942780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc5492aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15354640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54930da0 0x55cc54942780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15354640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54930da0 0x55cc54942780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41072/0 (socket says 172.21.15.7:41072) 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15b55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc5492c310 0x55cc54927390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15354640 1 -- 172.21.15.7:0/305051142 learned_addr learned my addr 172.21.15.7:0/305051142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/305051142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d101b17a0 con 0x55cc54930da0 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/305051142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d101af2a0 con 0x55cc54931790 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/305051142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d101ad320 con 0x55cc5492c310 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15b55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc5492c310 0x55cc54927390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49824/0 (socket says 172.21.15.7:49824) 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d16356640 1 --2- 172.21.15.7:0/305051142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc5492aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15354640 1 -- 172.21.15.7:0/305051142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc5492c310 msgr2=0x55cc54927390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15354640 1 --2- 172.21.15.7:0/305051142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc5492c310 0x55cc54927390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15354640 1 -- 172.21.15.7:0/305051142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 msgr2=0x55cc5492aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15354640 1 --2- 172.21.15.7:0/305051142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc5492aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.787+0000 7f6d15354640 1 -- 172.21.15.7:0/305051142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc54927b60 con 0x55cc54930da0 2024-03-26T08:52:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.788+0000 7f6d15354640 1 --2- 172.21.15.7:0/305051142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54930da0 0x55cc54942780 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6d000a5040 tx=0x7f6d00010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bcd6eb3691ad02 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.788+0000 7f6cfb7fe640 1 -- 172.21.15.7:0/305051142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0007d070 con 0x55cc54930da0 2024-03-26T08:52:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.788+0000 7f6cfb7fe640 1 -- 172.21.15.7:0/305051142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d000a3520 con 0x55cc54930da0 2024-03-26T08:52:09.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.788+0000 7f6cfb7fe640 1 -- 172.21.15.7:0/305051142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d000b92f0 con 0x55cc54930da0 2024-03-26T08:52:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/305051142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54930da0 msgr2=0x55cc54942780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/305051142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54930da0 0x55cc54942780 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6d000a5040 tx=0x7f6d00010990 comp rx=0 tx=0).stop 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/305051142 shutdown_connections 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/305051142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc5492aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/305051142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc54930da0 0x55cc54942780 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/305051142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc5492c310 0x55cc54927390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/305051142 >> 172.21.15.7:0/305051142 conn(0x55cc54932570 msgr2=0x55cc54932c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/305051142 shutdown_connections 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/305051142 wait complete. 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.789+0000 7f6d1a9a5740 1 -- start start 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d1a9a5740 1 Processor -- start 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc5492c310 0x55cc549407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54930da0 0x55cc54940cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d1a9a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc54941d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d1a9a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d101ad920 con 0x55cc5492c310 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d1a9a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d101aec40 con 0x55cc54931790 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d1a9a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d101adf80 con 0x55cc54930da0 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d1a9a5740 10 client.0 _ll_register_callbacks cb 0x7f6d0800a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:09.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d15b55640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc5492c310 0x55cc549407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d16356640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc54941d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d16356640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc54941d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44394/0 (socket says 172.21.15.7:44394) 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d16356640 1 -- 172.21.15.7:0/1366712141 learned_addr learned my addr 172.21.15.7:0/1366712141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d15b55640 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc5492c310 0x55cc549407b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41074/0 (socket says 172.21.15.7:41074) 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.790+0000 7f6d15354640 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54930da0 0x55cc54940cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.791+0000 7f6d16356640 1 -- 172.21.15.7:0/1366712141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54930da0 msgr2=0x55cc54940cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.791+0000 7f6d16356640 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54930da0 0x55cc54940cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.791+0000 7f6d16356640 1 -- 172.21.15.7:0/1366712141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc5492c310 msgr2=0x55cc549407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.791+0000 7f6d16356640 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc5492c310 0x55cc549407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.791+0000 7f6d16356640 1 -- 172.21.15.7:0/1366712141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc549422f0 con 0x55cc54931790 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.791+0000 7f6d16356640 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc54941d00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6d0801a6a0 tx=0x7f6d08004a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611402a44d245b3a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:09.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.791+0000 7f6cf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.791+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1366712141 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0802b020 con 0x55cc54931790 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.792+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1366712141 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6d08027730 con 0x55cc54931790 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.792+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1366712141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d000a3520 con 0x55cc54931790 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.792+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1366712141 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d08014040 con 0x55cc54931790 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.792+0000 7f6d1a9a5740 10 client.27836 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.792+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1366712141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55cc549a28c0 con 0x55cc54931790 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.793+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1366712141 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d000a3520 con 0x55cc54931790 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.793+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1366712141 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6d08038020 con 0x55cc54931790 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.793+0000 7f6d1a9a5740 10 client.27836 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:09.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.793+0000 7f6d1a9a5740 10 client.27836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.793+0000 7f6d1a9a5740 20 client.27836 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.793+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc54936900 0x55cc549a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.793+0000 7f6d1a9a5740 4 client.27836 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.793+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1366712141 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cc5489dc60 con 0x55cc54936900 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.793+0000 7f6d15b55640 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc54936900 0x55cc549a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6cf97fa640 10 client.27836 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d15b55640 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc54936900 0x55cc549a52b0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128b3140967092d1 server_cookie=6fe909ef1973e71f in_seq=0 out_seq=0 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6cf97fa640 1 -- 172.21.15.7:0/1366712141 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cc5489dc60 con 0x55cc54936900 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6cf97fa640 10 client.27836 handle_command_reply: tid=0 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 client.27836 shutdown 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1366712141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 msgr2=0x55cc54941d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc54941d00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6d0801a6a0 tx=0x7f6d08004a60 comp rx=0 tx=0).stop 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1366712141 shutdown_connections 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55cc54936900 0x55cc549a52b0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc54931790 0x55cc54941d00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55cc54930da0 0x55cc54940cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 --2- 172.21.15.7:0/1366712141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc5492c310 0x55cc549407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6cf97fa640 0 client.27836 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1366712141 >> 172.21.15.7:0/1366712141 conn(0x55cc54932570 msgr2=0x55cc5492f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1366712141 shutdown_connections 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 1 -- 172.21.15.7:0/1366712141 wait complete. 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.794+0000 7f6d1a9a5740 20 client.27836 trim_cache size 0 max 16384 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stdout: "4664724c-954e-4494-95bd-2e3128f5e8b2": { 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "4664724c-954e-4494-95bd-2e3128f5e8b2", 2024-03-26T08:52:09.793 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:52:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:52:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:52:09.794 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 -- 172.21.15.7:0/1958717265 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cfc0a2950 msgr2=0x7f6cfc0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cfc0a2950 0x7f6cfc0a4e10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009fd0 tx=0x7f6d08009f40 comp rx=0 tx=0).stop 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 -- 172.21.15.7:0/1958717265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1006e820 msgr2=0x7f6d1019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1006e820 0x7f6d1019e700 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b450 tx=0x7f6d0000b910 comp rx=0 tx=0).stop 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 -- 172.21.15.7:0/1958717265 shutdown_connections 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6cfc0a2950 0x7f6cfc0a4e10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6d1006f1c0 0x7f6d101a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1006e820 0x7f6d1019e700 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 --2- 172.21.15.7:0/1958717265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d1006d5e0 0x7f6d1019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 -- 172.21.15.7:0/1958717265 >> 172.21.15.7:0/1958717265 conn(0x7f6d1006c170 msgr2=0x7f6d10109530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 -- 172.21.15.7:0/1958717265 shutdown_connections 2024-03-26T08:52:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:09.795+0000 7f6d17db9640 1 -- 172.21.15.7:0/1958717265 wait complete. 2024-03-26T08:52:09.806 INFO:tasks.cephfs.filesystem:scrub status for tag:4664724c-954e-4494-95bd-2e3128f5e8b2 - {'path': '/', 'tag': '4664724c-954e-4494-95bd-2e3128f5e8b2', 'options': 'recursive,force'} 2024-03-26T08:52:10.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:10 smithi016 ceph-mon[35851]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 MiB/s wr, 647 op/s 2024-03-26T08:52:10.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:10 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:52:10.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:10 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:10.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:10 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:52:10.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:10 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:10.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:10 smithi007 ceph-mon[32462]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 MiB/s wr, 647 op/s 2024-03-26T08:52:11.522 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:10 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:52:11.523 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:10 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:11.523 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:10 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:52:11.523 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:10 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:11.524 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:10 smithi044 ceph-mon[36461]: pgmap v652: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 MiB/s wr, 647 op/s 2024-03-26T08:52:11.524 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:10 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:52:11.524 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:10 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:11.524 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:10 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:52:11.524 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:10 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:11.524 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 580.0 s, 2056.6 tps, lat 48.177 ms stddev 54.417 2024-03-26T08:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.296+0000 7f9bdc977640 1 -- 172.21.15.7:0/1079539117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd81021b0 msgr2=0x7f9bd8108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.296+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1079539117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd81021b0 0x7f9bd8108a40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009ca0 tx=0x7f9bd002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.296+0000 7f9bdc977640 1 -- 172.21.15.7:0/1079539117 shutdown_connections 2024-03-26T08:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.296+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1079539117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd81021b0 0x7f9bd8108a40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.296+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1079539117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd81017f0 0x7f9bd8101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.296+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1079539117 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd8100600 0x7f9bd8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.296+0000 7f9bdc977640 1 -- 172.21.15.7:0/1079539117 >> 172.21.15.7:0/1079539117 conn(0x7f9bd80fbb70 msgr2=0x7f9bd80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.296+0000 7f9bdc977640 1 -- 172.21.15.7:0/1079539117 shutdown_connections 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.296+0000 7f9bdc977640 1 -- 172.21.15.7:0/1079539117 wait complete. 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.297+0000 7f9bdc977640 1 Processor -- start 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.297+0000 7f9bdc977640 1 -- start start 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.297+0000 7f9bdc977640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd8100600 0x7f9bd81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.297+0000 7f9bdc977640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd81017f0 0x7f9bd8199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.297+0000 7f9bdc977640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd81021b0 0x7f9bd81a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bdc977640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd810b1d0 con 0x7f9bd81021b0 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bdc977640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd806a790 con 0x7f9bd8100600 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bdc977640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd810b350 con 0x7f9bd81017f0 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd8100600 0x7f9bd81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd81017f0 0x7f9bd8199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd81021b0 0x7f9bd81a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd8100600 0x7f9bd81999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44418/0 (socket says 172.21.15.7:44418) 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bd659c640 1 -- 172.21.15.7:0/1170163981 learned_addr learned my addr 172.21.15.7:0/1170163981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bcffff640 1 -- 172.21.15.7:0/1170163981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd8100600 msgr2=0x7f9bd81999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bcffff640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd8100600 0x7f9bd81999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bcffff640 1 -- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd81021b0 msgr2=0x7f9bd81a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bcffff640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd81021b0 0x7f9bd81a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bcffff640 1 -- 172.21.15.7:0/1170163981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd81a15e0 con 0x7f9bd81017f0 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd81021b0 0x7f9bd81a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bcffff640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd81017f0 0x7f9bd8199f00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e9f0 tx=0x7f9bc000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bcf7fe640 1 -- 172.21.15.7:0/1170163981 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000ccc0 con 0x7f9bd81017f0 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.298+0000 7f9bdc977640 1 -- 172.21.15.7:0/1170163981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd81a18d0 con 0x7f9bd81017f0 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.299+0000 7f9bcf7fe640 1 -- 172.21.15.7:0/1170163981 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bc0010910 con 0x7f9bd81017f0 2024-03-26T08:52:11.538 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.299+0000 7f9bdc977640 1 -- 172.21.15.7:0/1170163981 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd81a1e70 con 0x7f9bd81017f0 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.299+0000 7f9bcf7fe640 1 -- 172.21.15.7:0/1170163981 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0018960 con 0x7f9bd81017f0 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.301+0000 7f9bdf565740 1 Processor -- start 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.302+0000 7f9bdf565740 1 -- start start 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.302+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae287dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.302+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae2882e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.302+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae2883480 0x557ae2885910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.302+0000 7f9bdf565740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae27e2b10 con 0x557ae287d920 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.302+0000 7f9bdf565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae278b900 con 0x557ae2883480 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.302+0000 7f9bdf565740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae27debb0 con 0x557ae287e310 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.302+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae2882e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.302+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae2882e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49872/0 (socket says 172.21.15.7:49872) 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bd659c640 1 -- 172.21.15.7:0/630226265 learned_addr learned my addr 172.21.15.7:0/630226265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/630226265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae287dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bd659c640 1 -- 172.21.15.7:0/630226265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae2883480 msgr2=0x557ae2885910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bd659c640 1 --2- 172.21.15.7:0/630226265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae2883480 0x557ae2885910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bd659c640 1 -- 172.21.15.7:0/630226265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 msgr2=0x557ae287dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bd659c640 1 --2- 172.21.15.7:0/630226265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae287dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bd659c640 1 -- 172.21.15.7:0/630226265 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ae24f7210 con 0x557ae287e310 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bcf7fe640 1 -- 172.21.15.7:0/1170163981 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9bc0010ac0 con 0x7f9bd81017f0 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bd659c640 1 --2- 172.21.15.7:0/630226265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae2882e10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d70 tx=0x7f9bc8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ea091618504e45 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.303+0000 7f9bccff9640 1 -- 172.21.15.7:0/630226265 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8004220 con 0x557ae287e310 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.304+0000 7f9bccff9640 1 -- 172.21.15.7:0/630226265 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bc80043c0 con 0x557ae287e310 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.304+0000 7f9bccff9640 1 -- 172.21.15.7:0/630226265 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80198f0 con 0x557ae287e310 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.304+0000 7f9bcf7fe640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bc40a27f0 0x7f9bc40a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.304+0000 7f9bcf7fe640 1 -- 172.21.15.7:0/1170163981 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9bc00cee10 con 0x7f9bd81017f0 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.304+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bc40a27f0 0x7f9bc40a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.304+0000 7f9bdf565740 1 -- 172.21.15.7:0/630226265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 msgr2=0x557ae2882e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.304+0000 7f9bdf565740 1 --2- 172.21.15.7:0/630226265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae2882e10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d70 tx=0x7f9bc8011810 comp rx=0 tx=0).stop 2024-03-26T08:52:11.540 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 -- 172.21.15.7:0/630226265 shutdown_connections 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 --2- 172.21.15.7:0/630226265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae2883480 0x557ae2885910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 --2- 172.21.15.7:0/630226265 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae2882e10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 --2- 172.21.15.7:0/630226265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae287dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 -- 172.21.15.7:0/630226265 >> 172.21.15.7:0/630226265 conn(0x557ae2878e90 msgr2=0x557ae2879290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bc40a27f0 0x7f9bc40a4cb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0005e30 tx=0x7f9bd0005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 -- 172.21.15.7:0/630226265 shutdown_connections 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 -- 172.21.15.7:0/630226265 wait complete. 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 -- start start 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 Processor -- start 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae2887b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 0x557ae28880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 0x557ae288f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae2886430 con 0x557ae287d920 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae279ed50 con 0x557ae287e310 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.305+0000 7f9bdf565740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae28865b0 con 0x557ae2883480 2024-03-26T08:52:11.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae2887b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 0x557ae288f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae2887b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41112/0 (socket says 172.21.15.7:41112) 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bcffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 0x557ae28880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bd659c640 1 -- 172.21.15.7:0/3242457112 learned_addr learned my addr 172.21.15.7:0/3242457112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bdf565740 10 client.0 _ll_register_callbacks cb 0x557ae279e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bd6d9d640 1 -- 172.21.15.7:0/3242457112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 msgr2=0x557ae28880d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 0x557ae28880d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bd6d9d640 1 -- 172.21.15.7:0/3242457112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 msgr2=0x557ae2887b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae2887b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bd6d9d640 1 -- 172.21.15.7:0/3242457112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ae25c3620 con 0x557ae2883480 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.306+0000 7f9bcffff640 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 0x557ae28880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 0x557ae288f110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009bf0 tx=0x7f9bd0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8349b6baac101f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:11.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bbdffb640 1 -- 172.21.15.7:0/3242457112 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0051020 con 0x557ae2883480 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bbdffb640 1 -- 172.21.15.7:0/3242457112 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9bd00349f0 con 0x557ae2883480 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bdf565740 1 -- 172.21.15.7:0/3242457112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bc80043c0 con 0x557ae2883480 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bbdffb640 1 -- 172.21.15.7:0/3242457112 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bd0039040 con 0x557ae2883480 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bdf565740 10 client.27862 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bdf565740 1 -- 172.21.15.7:0/3242457112 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557ae2892520 con 0x557ae2883480 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bbdffb640 1 -- 172.21.15.7:0/3242457112 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd0034bf0 con 0x557ae2883480 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.307+0000 7f9bbdffb640 1 -- 172.21.15.7:0/3242457112 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9bd0040430 con 0x557ae2883480 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.308+0000 7f9bdf565740 10 client.27862 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.308+0000 7f9bdf565740 10 client.27862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.308+0000 7f9bdf565740 20 client.27862 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.308+0000 7f9bdf565740 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557ae2893b50 0x557ae2895f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.543 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.308+0000 7f9bdf565740 4 client.27862 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.308+0000 7f9bdf565740 1 -- 172.21.15.7:0/3242457112 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9bc80043c0 con 0x557ae2893b50 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.308+0000 7f9bd659c640 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557ae2893b50 0x557ae2895f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.308+0000 7f9bd659c640 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557ae2893b50 0x557ae2895f40 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=410c1af3ff4cf68 server_cookie=af698d7137e676c8 in_seq=0 out_seq=0 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.308+0000 7f9bbdffb640 10 client.27862 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bbdffb640 1 -- 172.21.15.7:0/3242457112 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f9bc80043c0 con 0x557ae2893b50 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bbdffb640 10 client.27862 handle_command_reply: tid=0 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 client.27862 shutdown 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 -- 172.21.15.7:0/3242457112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 msgr2=0x557ae288f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 0x557ae288f110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009bf0 tx=0x7f9bd0005fd0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 -- 172.21.15.7:0/3242457112 shutdown_connections 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557ae2893b50 0x557ae2895f40 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 0x557ae288f110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 0x557ae28880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 --2- 172.21.15.7:0/3242457112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae2887b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 -- 172.21.15.7:0/3242457112 >> 172.21.15.7:0/3242457112 conn(0x557ae2878e90 msgr2=0x557ae2878020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bbdffb640 0 client.27862 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 -- 172.21.15.7:0/3242457112 shutdown_connections 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 1 -- 172.21.15.7:0/3242457112 wait complete. 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.309+0000 7f9bdf565740 20 client.27862 trim_cache size 0 max 16384 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.328+0000 7f9bdf565740 1 Processor -- start 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.328+0000 7f9bdf565740 1 -- start start 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287d920 0x557ae2882bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae288f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae2883480 0x557ae2887b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bdf565740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81ad0a0 con 0x557ae2883480 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bdf565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81aaba0 con 0x557ae287d920 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bdf565740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81a8c20 con 0x557ae287e310 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae2883480 0x557ae2887b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287d920 0x557ae2882bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd6d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae2883480 0x557ae2887b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41114/0 (socket says 172.21.15.7:41114) 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd6d9d640 1 -- 172.21.15.7:0/1276622588 learned_addr learned my addr 172.21.15.7:0/1276622588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bcffff640 1 --2- 172.21.15.7:0/1276622588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae288f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd6d9d640 1 -- 172.21.15.7:0/1276622588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 msgr2=0x557ae288f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/1276622588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae288f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd6d9d640 1 -- 172.21.15.7:0/1276622588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287d920 msgr2=0x557ae2882bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/1276622588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287d920 0x557ae2882bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd6d9d640 1 -- 172.21.15.7:0/1276622588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ae287f080 con 0x557ae2883480 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/1276622588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae2883480 0x557ae2887b70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009ca0 tx=0x7f9bd0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b032099a8b1228 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.329+0000 7f9bbffff640 1 -- 172.21.15.7:0/1276622588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0032a50 con 0x557ae2883480 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.330+0000 7f9bbffff640 1 -- 172.21.15.7:0/1276622588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd0040630 con 0x557ae2883480 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.330+0000 7f9bbffff640 1 -- 172.21.15.7:0/1276622588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd004d9c0 con 0x557ae2883480 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.330+0000 7f9bdf565740 1 -- 172.21.15.7:0/1276622588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae2883480 msgr2=0x557ae2887b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 --2- 172.21.15.7:0/1276622588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae2883480 0x557ae2887b70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009ca0 tx=0x7f9bd0005ef0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 -- 172.21.15.7:0/1276622588 shutdown_connections 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 --2- 172.21.15.7:0/1276622588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae2883480 0x557ae2887b70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 --2- 172.21.15.7:0/1276622588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae287e310 0x557ae288f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 --2- 172.21.15.7:0/1276622588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287d920 0x557ae2882bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 -- 172.21.15.7:0/1276622588 >> 172.21.15.7:0/1276622588 conn(0x557ae2878e90 msgr2=0x557ae2873eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 -- 172.21.15.7:0/1276622588 shutdown_connections 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 -- 172.21.15.7:0/1276622588 wait complete. 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 -- start start 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 Processor -- start 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae288f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 0x557ae288bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 0x557ae288c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81a9220 con 0x557ae287d920 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae288f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81aa540 con 0x557ae287e310 2024-03-26T08:52:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bdf565740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81a9880 con 0x557ae2883480 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.331+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae288f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41120/0 (socket says 172.21.15.7:41120) 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bd659c640 1 -- 172.21.15.7:0/2467824963 learned_addr learned my addr 172.21.15.7:0/2467824963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bdf565740 10 client.0 _ll_register_callbacks cb 0x7f9bc8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bcffff640 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 0x557ae288bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bd6d9d640 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 0x557ae288c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bd659c640 1 -- 172.21.15.7:0/2467824963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 msgr2=0x557ae288c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bd659c640 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 0x557ae288c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bd659c640 1 -- 172.21.15.7:0/2467824963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 msgr2=0x557ae288bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bd659c640 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 0x557ae288bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bd659c640 1 -- 172.21.15.7:0/2467824963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ae28ed480 con 0x557ae287d920 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bd659c640 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae288f070 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80058c0 tx=0x7f9bc8009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db17560c45cab656 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bbdffb640 1 -- 172.21.15.7:0/2467824963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8005390 con 0x557ae287d920 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bbdffb640 1 -- 172.21.15.7:0/2467824963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9bc8005530 con 0x557ae287d920 2024-03-26T08:52:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bdf565740 1 -- 172.21.15.7:0/2467824963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bd0040630 con 0x557ae287d920 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bbdffb640 1 -- 172.21.15.7:0/2467824963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bc80035f0 con 0x557ae287d920 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bdf565740 10 client.18042 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.332+0000 7f9bdf565740 1 -- 172.21.15.7:0/2467824963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557ae28ee7e0 con 0x557ae287d920 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.333+0000 7f9bbdffb640 1 -- 172.21.15.7:0/2467824963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9bd0040630 con 0x557ae287d920 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.333+0000 7f9bbdffb640 1 -- 172.21.15.7:0/2467824963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9bc8034030 con 0x557ae287d920 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.333+0000 7f9bdf565740 10 client.18042 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.333+0000 7f9bdf565740 10 client.18042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.333+0000 7f9bdf565740 20 client.18042 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.333+0000 7f9bdf565740 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557ae28eef40 0x557ae28f1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.333+0000 7f9bdf565740 4 client.18042 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.333+0000 7f9bdf565740 1 -- 172.21.15.7:0/2467824963 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557ae27f0050 con 0x557ae28eef40 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.334+0000 7f9bcffff640 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557ae28eef40 0x557ae28f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.334+0000 7f9bcffff640 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557ae28eef40 0x557ae28f1330 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9242959a5a01f2e server_cookie=4c91ad71adde8c72 in_seq=0 out_seq=0 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.334+0000 7f9bbdffb640 10 client.18042 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bbdffb640 1 -- 172.21.15.7:0/2467824963 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557ae27f0050 con 0x557ae28eef40 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bbdffb640 10 client.18042 handle_command_reply: tid=0 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 client.18042 shutdown 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 -- 172.21.15.7:0/2467824963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 msgr2=0x557ae288f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae288f070 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80058c0 tx=0x7f9bc8009730 comp rx=0 tx=0).stop 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 -- 172.21.15.7:0/2467824963 shutdown_connections 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557ae28eef40 0x557ae28f1330 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557ae2883480 0x557ae288c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ae287e310 0x557ae288bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 --2- 172.21.15.7:0/2467824963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557ae287d920 0x557ae288f070 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bbdffb640 0 client.18042 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 -- 172.21.15.7:0/2467824963 >> 172.21.15.7:0/2467824963 conn(0x557ae2878e90 msgr2=0x557ae287f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 -- 172.21.15.7:0/2467824963 shutdown_connections 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 1 -- 172.21.15.7:0/2467824963 wait complete. 2024-03-26T08:52:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.352+0000 7f9bdf565740 20 client.18042 trim_cache size 0 max 16384 2024-03-26T08:52:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:52:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:52:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:52:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:52:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 -- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bc40a27f0 msgr2=0x7f9bc40a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:11.551 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bc40a27f0 0x7f9bc40a4cb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0005e30 tx=0x7f9bd0005d80 comp rx=0 tx=0).stop 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 -- 172.21.15.7:0/1170163981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd81017f0 msgr2=0x7f9bd8199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd81017f0 0x7f9bd8199f00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e9f0 tx=0x7f9bc000eeb0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 -- 172.21.15.7:0/1170163981 shutdown_connections 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9bc40a27f0 0x7f9bc40a4cb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9bd81021b0 0x7f9bd81a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9bd81017f0 0x7f9bd8199f00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 --2- 172.21.15.7:0/1170163981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9bd8100600 0x7f9bd81999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 -- 172.21.15.7:0/1170163981 >> 172.21.15.7:0/1170163981 conn(0x7f9bd80fbb70 msgr2=0x7f9bd80fc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 -- 172.21.15.7:0/1170163981 shutdown_connections 2024-03-26T08:52:11.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:11.353+0000 7f9bdc977640 1 -- 172.21.15.7:0/1170163981 wait complete. 2024-03-26T08:52:11.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:11 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:52:11.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:11 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:11.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:11 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:52:11.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:11 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:11.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:11 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:52:11.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:11 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:12.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:12 smithi016 ceph-mon[35851]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 643 op/s 2024-03-26T08:52:12.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:12 smithi007 ceph-mon[32462]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 643 op/s 2024-03-26T08:52:12.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:12 smithi044 ceph-mon[36461]: pgmap v653: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 643 op/s 2024-03-26T08:52:14.621 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:14 smithi044 ceph-mon[36461]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 692 op/s 2024-03-26T08:52:14.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:14 smithi016 ceph-mon[35851]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 692 op/s 2024-03-26T08:52:14.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:14 smithi007 ceph-mon[32462]: pgmap v654: 97 pgs: 97 active+clean; 1.6 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 692 op/s 2024-03-26T08:52:16.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 585.0 s, 1607.2 tps, lat 62.261 ms stddev 68.618 2024-03-26T08:52:16.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:16 smithi016 ceph-mon[35851]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.1 MiB/s wr, 729 op/s 2024-03-26T08:52:16.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:16 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:52:16.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:16 smithi007 ceph-mon[32462]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.1 MiB/s wr, 729 op/s 2024-03-26T08:52:16.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:16 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:52:16.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:16 smithi044 ceph-mon[36461]: pgmap v655: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.1 MiB/s wr, 729 op/s 2024-03-26T08:52:16.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:16 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:52:17.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:17 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:52:17.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:17 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:52:17.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:17 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:52:18.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:18 smithi016 ceph-mon[35851]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.1 MiB/s wr, 721 op/s 2024-03-26T08:52:18.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:18 smithi007 ceph-mon[32462]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.1 MiB/s wr, 721 op/s 2024-03-26T08:52:18.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:18 smithi044 ceph-mon[36461]: pgmap v656: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 8.1 MiB/s wr, 721 op/s 2024-03-26T08:52:20.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:20 smithi016 ceph-mon[35851]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.8 MiB/s wr, 694 op/s 2024-03-26T08:52:20.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:20 smithi007 ceph-mon[32462]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.8 MiB/s wr, 694 op/s 2024-03-26T08:52:20.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:20 smithi044 ceph-mon[36461]: pgmap v657: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.8 MiB/s wr, 694 op/s 2024-03-26T08:52:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 590.0 s, 1527.8 tps, lat 65.847 ms stddev 74.839 2024-03-26T08:52:22.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:22 smithi016 ceph-mon[35851]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.6 MiB/s wr, 761 op/s 2024-03-26T08:52:22.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:22 smithi007 ceph-mon[32462]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.6 MiB/s wr, 761 op/s 2024-03-26T08:52:22.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:22 smithi044 ceph-mon[36461]: pgmap v658: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 8.6 MiB/s wr, 761 op/s 2024-03-26T08:52:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.479+0000 7f112c5c4640 1 -- 172.21.15.7:0/430207023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112410d9c0 msgr2=0x7f112410de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.479+0000 7f112c5c4640 1 --2- 172.21.15.7:0/430207023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112410d9c0 0x7f112410de40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f111c009ca0 tx=0x7f111c02c120 comp rx=0 tx=0).stop 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.479+0000 7f112c5c4640 1 -- 172.21.15.7:0/430207023 shutdown_connections 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.479+0000 7f112c5c4640 1 --2- 172.21.15.7:0/430207023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f112410d9c0 0x7f112410de40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.479+0000 7f112c5c4640 1 --2- 172.21.15.7:0/430207023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f112410b010 0x7f112410d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.479+0000 7f112c5c4640 1 --2- 172.21.15.7:0/430207023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11241086a0 0x7f112410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.479+0000 7f112c5c4640 1 -- 172.21.15.7:0/430207023 >> 172.21.15.7:0/430207023 conn(0x7f112406e6b0 msgr2=0x7f1124070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.479+0000 7f112c5c4640 1 -- 172.21.15.7:0/430207023 shutdown_connections 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.479+0000 7f112c5c4640 1 -- 172.21.15.7:0/430207023 wait complete. 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112c5c4640 1 Processor -- start 2024-03-26T08:52:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112c5c4640 1 -- start start 2024-03-26T08:52:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112c5c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11241086a0 0x7f11241a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112c5c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112410b010 0x7f11241a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112c5c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f112410d9c0 0x7f11241adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112c5c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112411c230 con 0x7f11241086a0 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112c5c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112411c0b0 con 0x7f112410b010 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112c5c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112411c3b0 con 0x7f112410d9c0 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112ab61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f112410d9c0 0x7f11241adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112a360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11241086a0 0x7f11241a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112ab61640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f112410d9c0 0x7f11241adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47356/0 (socket says 172.21.15.7:47356) 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112ab61640 1 -- 172.21.15.7:0/1531736589 learned_addr learned my addr 172.21.15.7:0/1531736589 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f112a360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11241086a0 0x7f11241a6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34662/0 (socket says 172.21.15.7:34662) 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.480+0000 7f1129b5f640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112410b010 0x7f11241a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.481+0000 7f112a360640 1 -- 172.21.15.7:0/1531736589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f112410d9c0 msgr2=0x7f11241adef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.481+0000 7f112a360640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f112410d9c0 0x7f11241adef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.481+0000 7f112a360640 1 -- 172.21.15.7:0/1531736589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112410b010 msgr2=0x7f11241a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.481+0000 7f112a360640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112410b010 0x7f11241a6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.481+0000 7f112a360640 1 -- 172.21.15.7:0/1531736589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11241ae560 con 0x7f11241086a0 2024-03-26T08:52:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.481+0000 7f112a360640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11241086a0 0x7f11241a6940 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1114009fd0 tx=0x7f111400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.482+0000 7f111b7fe640 1 -- 172.21.15.7:0/1531736589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114015020 con 0x7f11241086a0 2024-03-26T08:52:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.482+0000 7f112c5c4640 1 -- 172.21.15.7:0/1531736589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11241ae850 con 0x7f11241086a0 2024-03-26T08:52:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.482+0000 7f112c5c4640 1 -- 172.21.15.7:0/1531736589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11241aee20 con 0x7f11241086a0 2024-03-26T08:52:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.482+0000 7f111b7fe640 1 -- 172.21.15.7:0/1531736589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1114011d10 con 0x7f11241086a0 2024-03-26T08:52:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.483+0000 7f111b7fe640 1 -- 172.21.15.7:0/1531736589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400f2e0 con 0x7f11241086a0 2024-03-26T08:52:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.485+0000 7f112f1ab740 1 Processor -- start 2024-03-26T08:52:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.485+0000 7f111b7fe640 1 -- 172.21.15.7:0/1531736589 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1114018020 con 0x7f11241086a0 2024-03-26T08:52:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.486+0000 7f112f1ab740 1 -- start start 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.486+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f1cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.486+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1d310 0x55e5a6f21e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.486+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 0x55e5a6f24910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.486+0000 7f112f1ab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a6e81b10 con 0x55e5a6f1c920 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.486+0000 7f112f1ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a6e2a900 con 0x55e5a6f22480 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.486+0000 7f112f1ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a6e7dbb0 con 0x55e5a6f1d310 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.486+0000 7f111b7fe640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f11100a2980 0x7f11100a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112a360640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1d310 0x55e5a6f21e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112ab61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f1cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112ab61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f1cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34668/0 (socket says 172.21.15.7:34668) 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112a360640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1d310 0x55e5a6f21e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47364/0 (socket says 172.21.15.7:47364) 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112ab61640 1 -- 172.21.15.7:0/258830101 learned_addr learned my addr 172.21.15.7:0/258830101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:24.484 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f1129b5f640 1 --2- 172.21.15.7:0/258830101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 0x55e5a6f24910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112ab61640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f11100a2980 0x7f11100a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f111b7fe640 1 -- 172.21.15.7:0/1531736589 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f11140ce0b0 con 0x7f11241086a0 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112a360640 1 -- 172.21.15.7:0/258830101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 msgr2=0x55e5a6f24910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112a360640 1 --2- 172.21.15.7:0/258830101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 0x55e5a6f24910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112a360640 1 -- 172.21.15.7:0/258830101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 msgr2=0x55e5a6f1cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112a360640 1 --2- 172.21.15.7:0/258830101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f1cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112a360640 1 -- 172.21.15.7:0/258830101 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5a6b96210 con 0x55e5a6f1d310 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112ab61640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f11100a2980 0x7f11100a4e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f111c032490 tx=0x7f111c032400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.487+0000 7f112a360640 1 --2- 172.21.15.7:0/258830101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1d310 0x55e5a6f21e10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f111409a750 tx=0x7f111406c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=976d39bfaa22585f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.488+0000 7f1118bfe640 1 -- 172.21.15.7:0/258830101 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111409b150 con 0x55e5a6f1d310 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.488+0000 7f1118bfe640 1 -- 172.21.15.7:0/258830101 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1114065cf0 con 0x55e5a6f1d310 2024-03-26T08:52:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.488+0000 7f1118bfe640 1 -- 172.21.15.7:0/258830101 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114010380 con 0x55e5a6f1d310 2024-03-26T08:52:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.489+0000 7f112f1ab740 1 -- 172.21.15.7:0/258830101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1d310 msgr2=0x55e5a6f21e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.489+0000 7f112f1ab740 1 --2- 172.21.15.7:0/258830101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1d310 0x55e5a6f21e10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f111409a750 tx=0x7f111406c750 comp rx=0 tx=0).stop 2024-03-26T08:52:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.489+0000 7f112f1ab740 1 -- 172.21.15.7:0/258830101 shutdown_connections 2024-03-26T08:52:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.489+0000 7f112f1ab740 1 --2- 172.21.15.7:0/258830101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 0x55e5a6f24910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.489+0000 7f112f1ab740 1 --2- 172.21.15.7:0/258830101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1d310 0x55e5a6f21e10 secure :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f111409a750 tx=0x7f111406c750 comp rx=0 tx=0).stop 2024-03-26T08:52:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.489+0000 7f112f1ab740 1 --2- 172.21.15.7:0/258830101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f1cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.489+0000 7f112f1ab740 1 -- 172.21.15.7:0/258830101 >> 172.21.15.7:0/258830101 conn(0x55e5a6f17e90 msgr2=0x55e5a6f18290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.491+0000 7f112f1ab740 1 -- 172.21.15.7:0/258830101 shutdown_connections 2024-03-26T08:52:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.491+0000 7f112f1ab740 1 -- 172.21.15.7:0/258830101 wait complete. 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.491+0000 7f112f1ab740 1 -- start start 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.491+0000 7f112f1ab740 1 Processor -- start 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.491+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f26d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 0x55e5a6f272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f2e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112f1ab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a6f25430 con 0x55e5a6f2c2e0 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112f1ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a6e3dd50 con 0x55e5a6f22480 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112f1ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5a6f255b0 con 0x55e5a6f1c920 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112f1ab740 10 client.0 _ll_register_callbacks cb 0x55e5a6e3d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112a360640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f26d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112ab61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f2e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112a360640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f26d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47368/0 (socket says 172.21.15.7:47368) 2024-03-26T08:52:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112a360640 1 -- 172.21.15.7:0/127625164 learned_addr learned my addr 172.21.15.7:0/127625164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f1129b5f640 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 0x55e5a6f272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f1129b5f640 1 -- 172.21.15.7:0/127625164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 msgr2=0x55e5a6f26d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f1129b5f640 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f26d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f1129b5f640 1 -- 172.21.15.7:0/127625164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f2c2e0 msgr2=0x55e5a6f2e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f1129b5f640 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f2e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f1129b5f640 1 -- 172.21.15.7:0/127625164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5a6c62620 con 0x55e5a6f22480 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.492+0000 7f112a360640 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f26d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.493+0000 7f1129b5f640 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 0x55e5a6f272b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f11241a7e50 tx=0x7f11200119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d522d4766f1589 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.493+0000 7f110dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.493+0000 7f110dffb640 1 -- 172.21.15.7:0/127625164 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120013670 con 0x55e5a6f22480 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.493+0000 7f112f1ab740 1 -- 172.21.15.7:0/127625164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1114065cf0 con 0x55e5a6f22480 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.493+0000 7f110dffb640 1 -- 172.21.15.7:0/127625164 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1120013810 con 0x55e5a6f22480 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.493+0000 7f110dffb640 1 -- 172.21.15.7:0/127625164 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1120013a10 con 0x55e5a6f22480 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.493+0000 7f112f1ab740 10 client.27860 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.493+0000 7f112f1ab740 1 -- 172.21.15.7:0/127625164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e5a6f31b40 con 0x55e5a6f22480 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.493+0000 7f110dffb640 1 -- 172.21.15.7:0/127625164 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1120013bb0 con 0x55e5a6f22480 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f110dffb640 1 -- 172.21.15.7:0/127625164 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1120028400 con 0x55e5a6f22480 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f112f1ab740 10 client.27860 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f112f1ab740 10 client.27860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f112f1ab740 20 client.27860 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f112f1ab740 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e5a6f33650 0x55e5a6f35ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f112f1ab740 4 client.27860 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f112f1ab740 1 -- 172.21.15.7:0/127625164 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1114065cf0 con 0x55e5a6f33650 2024-03-26T08:52:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f112a360640 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e5a6f33650 0x55e5a6f35ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f110dffb640 10 client.27860 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.494+0000 7f112a360640 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e5a6f33650 0x55e5a6f35ab0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e429c4ef53e4e96 server_cookie=4f7142a76bda5301 in_seq=0 out_seq=0 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.495+0000 7f110dffb640 1 -- 172.21.15.7:0/127625164 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f1114065cf0 con 0x55e5a6f33650 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.495+0000 7f110dffb640 10 client.27860 handle_command_reply: tid=0 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.495+0000 7f112f1ab740 1 client.27860 shutdown 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.495+0000 7f112f1ab740 1 -- 172.21.15.7:0/127625164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 msgr2=0x55e5a6f272b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.495+0000 7f112f1ab740 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 0x55e5a6f272b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f11241a7e50 tx=0x7f11200119d0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f112f1ab740 1 -- 172.21.15.7:0/127625164 shutdown_connections 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f112f1ab740 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e5a6f33650 0x55e5a6f35ab0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f112f1ab740 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f2e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f112f1ab740 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f22480 0x55e5a6f272b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f112f1ab740 1 --2- 172.21.15.7:0/127625164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f26d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f112f1ab740 1 -- 172.21.15.7:0/127625164 >> 172.21.15.7:0/127625164 conn(0x55e5a6f17e90 msgr2=0x55e5a6f17020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f110dffb640 0 client.27860 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f112f1ab740 1 -- 172.21.15.7:0/127625164 shutdown_connections 2024-03-26T08:52:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f112f1ab740 1 -- 172.21.15.7:0/127625164 wait complete. 2024-03-26T08:52:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.496+0000 7f112f1ab740 20 client.27860 trim_cache size 0 max 16384 2024-03-26T08:52:24.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.533+0000 7f112f1ab740 1 Processor -- start 2024-03-26T08:52:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.537+0000 7f112f1ab740 1 -- start start 2024-03-26T08:52:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.537+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f24870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.537+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f22480 0x55e5a6f1e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.537+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f1ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.537+0000 7f112f1ab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1108010980 con 0x55e5a6f1c920 2024-03-26T08:52:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.537+0000 7f112f1ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110800e480 con 0x55e5a6f2c2e0 2024-03-26T08:52:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.537+0000 7f112f1ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110800c500 con 0x55e5a6f22480 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.539+0000 7f1129b5f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f22480 0x55e5a6f1e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.539+0000 7f1129b5f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f22480 0x55e5a6f1e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:47380/0 (socket says 172.21.15.7:47380) 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.539+0000 7f1129b5f640 1 -- 172.21.15.7:0/2880945452 learned_addr learned my addr 172.21.15.7:0/2880945452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.539+0000 7f112ab61640 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f1ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.539+0000 7f112a360640 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f24870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.540+0000 7f1129b5f640 1 -- 172.21.15.7:0/2880945452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 msgr2=0x55e5a6f1ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.540+0000 7f1129b5f640 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f1ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.540+0000 7f1129b5f640 1 -- 172.21.15.7:0/2880945452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 msgr2=0x55e5a6f24870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.540+0000 7f1129b5f640 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f24870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.540+0000 7f1129b5f640 1 -- 172.21.15.7:0/2880945452 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5a6f219e0 con 0x55e5a6f22480 2024-03-26T08:52:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.540+0000 7f112a360640 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f24870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:52:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.540+0000 7f1129b5f640 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f22480 0x55e5a6f1e710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f11200288a0 tx=0x7f11200047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64397a43e22f910f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.541+0000 7f110ffff640 1 -- 172.21.15.7:0/2880945452 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120013870 con 0x55e5a6f22480 2024-03-26T08:52:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.541+0000 7f110ffff640 1 -- 172.21.15.7:0/2880945452 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1120028400 con 0x55e5a6f22480 2024-03-26T08:52:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.541+0000 7f110ffff640 1 -- 172.21.15.7:0/2880945452 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120014de0 con 0x55e5a6f22480 2024-03-26T08:52:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.543+0000 7f112f1ab740 1 -- 172.21.15.7:0/2880945452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f22480 msgr2=0x55e5a6f1e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.543+0000 7f112f1ab740 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f22480 0x55e5a6f1e710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f11200288a0 tx=0x7f11200047d0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.543+0000 7f112f1ab740 1 -- 172.21.15.7:0/2880945452 shutdown_connections 2024-03-26T08:52:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.543+0000 7f112f1ab740 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f1ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.543+0000 7f112f1ab740 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f22480 0x55e5a6f1e710 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.543+0000 7f112f1ab740 1 --2- 172.21.15.7:0/2880945452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f1c920 0x55e5a6f24870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.543+0000 7f112f1ab740 1 -- 172.21.15.7:0/2880945452 >> 172.21.15.7:0/2880945452 conn(0x55e5a6f17e90 msgr2=0x55e5a6f18270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.546+0000 7f112f1ab740 1 -- 172.21.15.7:0/2880945452 shutdown_connections 2024-03-26T08:52:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.546+0000 7f112f1ab740 1 -- 172.21.15.7:0/2880945452 wait complete. 2024-03-26T08:52:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.546+0000 7f112f1ab740 1 -- start start 2024-03-26T08:52:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.546+0000 7f112f1ab740 1 Processor -- start 2024-03-26T08:52:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f43240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f22480 0x55e5a6f43780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f112f1ab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f112f1ab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110800cb00 con 0x55e5a6f22480 2024-03-26T08:52:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f112f1ab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110800de20 con 0x55e5a6f2c2e0 2024-03-26T08:52:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f112f1ab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110800d160 con 0x55e5a6f1c920 2024-03-26T08:52:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f112f1ab740 10 client.0 _ll_register_callbacks cb 0x7f1120009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f112a360640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f43240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f1129b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f22480 0x55e5a6f43780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f1129b5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f22480 0x55e5a6f43780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34696/0 (socket says 172.21.15.7:34696) 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f1129b5f640 1 -- 172.21.15.7:0/468064833 learned_addr learned my addr 172.21.15.7:0/468064833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.548+0000 7f112ab61640 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.549+0000 7f1129b5f640 1 -- 172.21.15.7:0/468064833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 msgr2=0x55e5a6f43240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.549+0000 7f1129b5f640 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f43240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.549+0000 7f1129b5f640 1 -- 172.21.15.7:0/468064833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 msgr2=0x55e5a6f446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.549+0000 7f1129b5f640 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.549+0000 7f1129b5f640 1 -- 172.21.15.7:0/468064833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5a6f44d20 con 0x55e5a6f22480 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.550+0000 7f1129b5f640 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f22480 0x55e5a6f43780 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1120011510 tx=0x7f1120011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9d13691259c815 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.550+0000 7f110dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.553+0000 7f110dffb640 1 -- 172.21.15.7:0/468064833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120013bb0 con 0x55e5a6f22480 2024-03-26T08:52:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.553+0000 7f112f1ab740 1 -- 172.21.15.7:0/468064833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1120028400 con 0x55e5a6f22480 2024-03-26T08:52:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.553+0000 7f110dffb640 1 -- 172.21.15.7:0/468064833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1120014440 con 0x55e5a6f22480 2024-03-26T08:52:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.554+0000 7f110dffb640 1 -- 172.21.15.7:0/468064833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1120014640 con 0x55e5a6f22480 2024-03-26T08:52:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.554+0000 7f112f1ab740 10 client.18060 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.554+0000 7f112f1ab740 1 -- 172.21.15.7:0/468064833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e5a6f9f400 con 0x55e5a6f22480 2024-03-26T08:52:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.559+0000 7f110dffb640 1 -- 172.21.15.7:0/468064833 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1120028400 con 0x55e5a6f22480 2024-03-26T08:52:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.559+0000 7f110dffb640 1 -- 172.21.15.7:0/468064833 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1120038030 con 0x55e5a6f22480 2024-03-26T08:52:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.560+0000 7f112f1ab740 10 client.18060 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.560+0000 7f112f1ab740 10 client.18060 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.560+0000 7f112f1ab740 20 client.18060 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.560+0000 7f112f1ab740 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e5a6f9fa80 0x55e5a6fa1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.560+0000 7f112f1ab740 4 client.18060 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:52:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.560+0000 7f112f1ab740 1 -- 172.21.15.7:0/468064833 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e5a6e8f050 con 0x55e5a6f9fa80 2024-03-26T08:52:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.560+0000 7f112a360640 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e5a6f9fa80 0x55e5a6fa1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.561+0000 7f112a360640 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e5a6f9fa80 0x55e5a6fa1ea0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a096bd018faa1d server_cookie=62fcd4ddfe04cff4 in_seq=0 out_seq=0 2024-03-26T08:52:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.561+0000 7f110dffb640 10 client.18060 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.597+0000 7f110dffb640 1 -- 172.21.15.7:0/468064833 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e5a6e8f050 con 0x55e5a6f9fa80 2024-03-26T08:52:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.597+0000 7f110dffb640 10 client.18060 handle_command_reply: tid=0 2024-03-26T08:52:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.597+0000 7f112f1ab740 1 client.18060 shutdown 2024-03-26T08:52:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 -- 172.21.15.7:0/468064833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f22480 msgr2=0x55e5a6f43780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f22480 0x55e5a6f43780 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1120011510 tx=0x7f1120011ed0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 -- 172.21.15.7:0/468064833 shutdown_connections 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e5a6f9fa80 0x55e5a6fa1ea0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5a6f2c2e0 0x55e5a6f446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5a6f22480 0x55e5a6f43780 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 --2- 172.21.15.7:0/468064833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e5a6f1c920 0x55e5a6f43240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 -- 172.21.15.7:0/468064833 >> 172.21.15.7:0/468064833 conn(0x55e5a6f17e90 msgr2=0x55e5a6f26d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f110dffb640 0 client.18060 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 -- 172.21.15.7:0/468064833 shutdown_connections 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.598+0000 7f112f1ab740 1 -- 172.21.15.7:0/468064833 wait complete. 2024-03-26T08:52:24.595 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.599+0000 7f112f1ab740 20 client.18060 trim_cache size 0 max 16384 2024-03-26T08:52:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:52:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:52:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:52:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:52:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.600+0000 7f112c5c4640 1 -- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f11100a2980 msgr2=0x7f11100a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.600+0000 7f112c5c4640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f11100a2980 0x7f11100a4e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f111c032490 tx=0x7f111c032400 comp rx=0 tx=0).stop 2024-03-26T08:52:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.600+0000 7f112c5c4640 1 -- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11241086a0 msgr2=0x7f11241a6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.600+0000 7f112c5c4640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11241086a0 0x7f11241a6940 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1114009fd0 tx=0x7f111400b7b0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.601+0000 7f112c5c4640 1 -- 172.21.15.7:0/1531736589 shutdown_connections 2024-03-26T08:52:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.601+0000 7f112c5c4640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f11100a2980 0x7f11100a4e40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.601+0000 7f112c5c4640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f112410d9c0 0x7f11241adef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.601+0000 7f112c5c4640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f112410b010 0x7f11241a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.601+0000 7f112c5c4640 1 --2- 172.21.15.7:0/1531736589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11241086a0 0x7f11241a6940 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.601+0000 7f112c5c4640 1 -- 172.21.15.7:0/1531736589 >> 172.21.15.7:0/1531736589 conn(0x7f112406e6b0 msgr2=0x7f1124108ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.601+0000 7f112c5c4640 1 -- 172.21.15.7:0/1531736589 shutdown_connections 2024-03-26T08:52:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:24.601+0000 7f112c5c4640 1 -- 172.21.15.7:0/1531736589 wait complete. 2024-03-26T08:52:24.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:24 smithi016 ceph-mon[35851]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.1 MiB/s wr, 799 op/s 2024-03-26T08:52:24.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:24 smithi007 ceph-mon[32462]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.1 MiB/s wr, 799 op/s 2024-03-26T08:52:24.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:24 smithi044 ceph-mon[36461]: pgmap v659: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.1 MiB/s wr, 799 op/s 2024-03-26T08:52:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 595.0 s, 2289.4 tps, lat 43.139 ms stddev 50.703 2024-03-26T08:52:26.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:26 smithi016 ceph-mon[35851]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.6 MiB/s wr, 663 op/s 2024-03-26T08:52:26.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:26 smithi007 ceph-mon[32462]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.6 MiB/s wr, 663 op/s 2024-03-26T08:52:26.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:26 smithi044 ceph-mon[36461]: pgmap v660: 97 pgs: 97 active+clean; 1.6 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.6 MiB/s wr, 663 op/s 2024-03-26T08:52:28.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:28 smithi016 ceph-mon[35851]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 657 op/s 2024-03-26T08:52:28.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:28 smithi007 ceph-mon[32462]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 657 op/s 2024-03-26T08:52:28.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:28 smithi044 ceph-mon[36461]: pgmap v661: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 657 op/s 2024-03-26T08:52:30.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:30 smithi016 ceph-mon[35851]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 638 op/s 2024-03-26T08:52:30.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:30 smithi007 ceph-mon[32462]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 638 op/s 2024-03-26T08:52:30.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:30 smithi044 ceph-mon[36461]: pgmap v662: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 638 op/s 2024-03-26T08:52:31.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 600.0 s, 2412.4 tps, lat 42.027 ms stddev 50.047 2024-03-26T08:52:32.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:32 smithi016 ceph-mon[35851]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 671 op/s 2024-03-26T08:52:32.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:32 smithi007 ceph-mon[32462]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 671 op/s 2024-03-26T08:52:32.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:32 smithi044 ceph-mon[36461]: pgmap v663: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.4 MiB/s wr, 671 op/s 2024-03-26T08:52:34.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:34 smithi016 ceph-mon[35851]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-03-26T08:52:34.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:34 smithi007 ceph-mon[32462]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-03-26T08:52:34.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:34 smithi044 ceph-mon[36461]: pgmap v664: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.3 MiB/s wr, 675 op/s 2024-03-26T08:52:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 605.0 s, 2121.4 tps, lat 47.101 ms stddev 55.952 2024-03-26T08:52:36.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:36 smithi016 ceph-mon[35851]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 622 op/s 2024-03-26T08:52:36.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:36 smithi007 ceph-mon[32462]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 622 op/s 2024-03-26T08:52:36.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:36 smithi044 ceph-mon[36461]: pgmap v665: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 622 op/s 2024-03-26T08:52:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.750+0000 7fd921962640 1 -- 172.21.15.7:0/3951056443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd91c10d9c0 msgr2=0x7fd91c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.750+0000 7fd921962640 1 --2- 172.21.15.7:0/3951056443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd91c10d9c0 0x7fd91c10de40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd908009900 tx=0x7fd90802bc80 comp rx=0 tx=0).stop 2024-03-26T08:52:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.750+0000 7fd921962640 1 -- 172.21.15.7:0/3951056443 shutdown_connections 2024-03-26T08:52:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.750+0000 7fd921962640 1 --2- 172.21.15.7:0/3951056443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd91c10d9c0 0x7fd91c10de40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.750+0000 7fd921962640 1 --2- 172.21.15.7:0/3951056443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd91c10b010 0x7fd91c10d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.750+0000 7fd921962640 1 --2- 172.21.15.7:0/3951056443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91c1086a0 0x7fd91c10aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.750+0000 7fd921962640 1 -- 172.21.15.7:0/3951056443 >> 172.21.15.7:0/3951056443 conn(0x7fd91c06e6b0 msgr2=0x7fd91c070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.750+0000 7fd921962640 1 -- 172.21.15.7:0/3951056443 shutdown_connections 2024-03-26T08:52:37.747 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.751+0000 7fd921962640 1 -- 172.21.15.7:0/3951056443 wait complete. 2024-03-26T08:52:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.751+0000 7fd921962640 1 Processor -- start 2024-03-26T08:52:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd921962640 1 -- start start 2024-03-26T08:52:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd921962640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91c1086a0 0x7fd91c114d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd921962640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd91c10b010 0x7fd91c115250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd921962640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd91c10d9c0 0x7fd91c1b6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd921962640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c11c200 con 0x7fd91c10d9c0 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd921962640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c11c080 con 0x7fd91c1086a0 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd921962640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c11c380 con 0x7fd91c10b010 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd91affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91c1086a0 0x7fd91c114d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd91affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91c1086a0 0x7fd91c114d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53728/0 (socket says 172.21.15.7:53728) 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.752+0000 7fd91affd640 1 -- 172.21.15.7:0/622119803 learned_addr learned my addr 172.21.15.7:0/622119803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.753+0000 7fd91b7fe640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd91c10d9c0 0x7fd91c1b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.753+0000 7fd91affd640 1 -- 172.21.15.7:0/622119803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd91c10b010 msgr2=0x7fd91c115250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.753+0000 7fd91affd640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd91c10b010 0x7fd91c115250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.753+0000 7fd91affd640 1 -- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd91c10d9c0 msgr2=0x7fd91c1b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.753+0000 7fd91affd640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd91c10d9c0 0x7fd91c1b6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.750 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.753+0000 7fd91affd640 1 -- 172.21.15.7:0/622119803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c1158f0 con 0x7fd91c1086a0 2024-03-26T08:52:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.753+0000 7fd91affd640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91c1086a0 0x7fd91c114d10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00b450 tx=0x7fd90c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.754+0000 7fd920960640 1 -- 172.21.15.7:0/622119803 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c015020 con 0x7fd91c1086a0 2024-03-26T08:52:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.754+0000 7fd920960640 1 -- 172.21.15.7:0/622119803 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90c011d70 con 0x7fd91c1086a0 2024-03-26T08:52:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.755+0000 7fd920960640 1 -- 172.21.15.7:0/622119803 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00f340 con 0x7fd91c1086a0 2024-03-26T08:52:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.756+0000 7fd921962640 1 -- 172.21.15.7:0/622119803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c1b6f70 con 0x7fd91c1086a0 2024-03-26T08:52:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.756+0000 7fd921962640 1 -- 172.21.15.7:0/622119803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c1b7320 con 0x7fd91c1086a0 2024-03-26T08:52:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.760+0000 7fd92462d740 1 Processor -- start 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.761+0000 7fd92462d740 1 -- start start 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.761+0000 7fd92462d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3daad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.761+0000 7fd92462d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3dab310 0x5580f3dafe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.761+0000 7fd92462d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 0x5580f3db2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.761+0000 7fd92462d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f3d0fb10 con 0x5580f3daa920 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.761+0000 7fd92462d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f3cb8900 con 0x5580f3db0480 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.761+0000 7fd92462d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f3d0bbb0 con 0x5580f3dab310 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.761+0000 7fd91b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3daad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd91b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3daad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57746/0 (socket says 172.21.15.7:57746) 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd91b7fe640 1 -- 172.21.15.7:0/4253079599 learned_addr learned my addr 172.21.15.7:0/4253079599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/4253079599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 0x5580f3db2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd91a7fc640 1 -- 172.21.15.7:0/4253079599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3dab310 msgr2=0x5580f3dafe10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:52:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/4253079599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3dab310 0x5580f3dafe10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd91a7fc640 1 -- 172.21.15.7:0/4253079599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 msgr2=0x5580f3daad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/4253079599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3daad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd91a7fc640 1 -- 172.21.15.7:0/4253079599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580f3a24210 con 0x5580f3db0480 2024-03-26T08:52:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd920960640 1 -- 172.21.15.7:0/622119803 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd90c018020 con 0x7fd91c1086a0 2024-03-26T08:52:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.762+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/4253079599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 0x5580f3db2910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd914005fd0 tx=0x7fd91400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6afaafb33496b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.763+0000 7fd911ffb640 1 -- 172.21.15.7:0/4253079599 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914016070 con 0x5580f3db0480 2024-03-26T08:52:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.763+0000 7fd911ffb640 1 -- 172.21.15.7:0/4253079599 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd914003970 con 0x5580f3db0480 2024-03-26T08:52:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.763+0000 7fd911ffb640 1 -- 172.21.15.7:0/4253079599 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914004630 con 0x5580f3db0480 2024-03-26T08:52:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd920960640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9000a27c0 0x7fd9000a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd92462d740 1 -- 172.21.15.7:0/4253079599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 msgr2=0x5580f3db2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd92462d740 1 --2- 172.21.15.7:0/4253079599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 0x5580f3db2910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd914005fd0 tx=0x7fd91400e6c0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd920960640 1 -- 172.21.15.7:0/622119803 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd90c0ce3e0 con 0x7fd91c1086a0 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd92462d740 1 -- 172.21.15.7:0/4253079599 shutdown_connections 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd92462d740 1 --2- 172.21.15.7:0/4253079599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 0x5580f3db2910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd92462d740 1 --2- 172.21.15.7:0/4253079599 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3dab310 0x5580f3dafe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd92462d740 1 --2- 172.21.15.7:0/4253079599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3daad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd92462d740 1 -- 172.21.15.7:0/4253079599 >> 172.21.15.7:0/4253079599 conn(0x5580f3da5e90 msgr2=0x5580f3da6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.764+0000 7fd91b7fe640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9000a27c0 0x7fd9000a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.765+0000 7fd92462d740 1 -- 172.21.15.7:0/4253079599 shutdown_connections 2024-03-26T08:52:37.761 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.765+0000 7fd92462d740 1 -- 172.21.15.7:0/4253079599 wait complete. 2024-03-26T08:52:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.765+0000 7fd91b7fe640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9000a27c0 0x7fd9000a4c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd908005ec0 tx=0x7fd908005e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.765+0000 7fd92462d740 1 -- start start 2024-03-26T08:52:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.765+0000 7fd92462d740 1 Processor -- start 2024-03-26T08:52:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd92462d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3daa920 0x5580f3db4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd92462d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3db5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd92462d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3db0480 0x5580f3dbc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd92462d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f3db3480 con 0x5580f3db0480 2024-03-26T08:52:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd92462d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f3ccbd50 con 0x5580f3dab310 2024-03-26T08:52:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd92462d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580f3db3600 con 0x5580f3daa920 2024-03-26T08:52:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd92462d740 10 client.0 _ll_register_callbacks cb 0x5580f3ccb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd91affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3daa920 0x5580f3db4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd91a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3db5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd91a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3db5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53754/0 (socket says 172.21.15.7:53754) 2024-03-26T08:52:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd91a7fc640 1 -- 172.21.15.7:0/3983549437 learned_addr learned my addr 172.21.15.7:0/3983549437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.766+0000 7fd91b7fe640 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3db0480 0x5580f3dbc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd91a7fc640 1 -- 172.21.15.7:0/3983549437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3daa920 msgr2=0x5580f3db4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3daa920 0x5580f3db4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd91a7fc640 1 -- 172.21.15.7:0/3983549437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3db0480 msgr2=0x5580f3dbc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3db0480 0x5580f3dbc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd91a7fc640 1 -- 172.21.15.7:0/3983549437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580f3af0620 con 0x5580f3dab310 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3db5190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd914005cc0 tx=0x7fd914003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488df3b21cbdd879 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd8feffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd8feffd640 1 -- 172.21.15.7:0/3983549437 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd914016070 con 0x5580f3dab310 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd8feffd640 1 -- 172.21.15.7:0/3983549437 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd914004570 con 0x5580f3dab310 2024-03-26T08:52:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.767+0000 7fd92462d740 1 -- 172.21.15.7:0/3983549437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd914003970 con 0x5580f3dab310 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd8feffd640 1 -- 172.21.15.7:0/3983549437 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd914004770 con 0x5580f3dab310 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd92462d740 10 client.27875 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd92462d740 1 -- 172.21.15.7:0/3983549437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5580f3dbf540 con 0x5580f3dab310 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd8feffd640 1 -- 172.21.15.7:0/3983549437 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd914003970 con 0x5580f3dab310 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd8feffd640 1 -- 172.21.15.7:0/3983549437 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd914021070 con 0x5580f3dab310 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd92462d740 10 client.27875 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd92462d740 10 client.27875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd92462d740 20 client.27875 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd92462d740 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5580f3dc0ec0 0x5580f3dc3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd92462d740 4 client.27875 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.768+0000 7fd92462d740 1 -- 172.21.15.7:0/3983549437 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580f3d1d050 con 0x5580f3dc0ec0 2024-03-26T08:52:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.769+0000 7fd91affd640 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5580f3dc0ec0 0x5580f3dc3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.770+0000 7fd91affd640 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5580f3dc0ec0 0x5580f3dc3300 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42fa7900885386f2 server_cookie=2e9e73180eb3400e in_seq=0 out_seq=0 2024-03-26T08:52:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.771+0000 7fd8feffd640 10 client.27875 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.772+0000 7fd8feffd640 1 -- 172.21.15.7:0/3983549437 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5580f3d1d050 con 0x5580f3dc0ec0 2024-03-26T08:52:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.773+0000 7fd8feffd640 10 client.27875 handle_command_reply: tid=0 2024-03-26T08:52:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.773+0000 7fd92462d740 1 client.27875 shutdown 2024-03-26T08:52:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.773+0000 7fd92462d740 1 -- 172.21.15.7:0/3983549437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 msgr2=0x5580f3db5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:37.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.773+0000 7fd92462d740 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3db5190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd914005cc0 tx=0x7fd914003b20 comp rx=0 tx=0).stop 2024-03-26T08:52:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd92462d740 1 -- 172.21.15.7:0/3983549437 shutdown_connections 2024-03-26T08:52:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd92462d740 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5580f3dc0ec0 0x5580f3dc3300 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd92462d740 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3db0480 0x5580f3dbc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd92462d740 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3db5190 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd92462d740 1 --2- 172.21.15.7:0/3983549437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3daa920 0x5580f3db4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd92462d740 1 -- 172.21.15.7:0/3983549437 >> 172.21.15.7:0/3983549437 conn(0x5580f3da5e90 msgr2=0x5580f3da5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd8feffd640 0 client.27875 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd92462d740 1 -- 172.21.15.7:0/3983549437 shutdown_connections 2024-03-26T08:52:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd92462d740 1 -- 172.21.15.7:0/3983549437 wait complete. 2024-03-26T08:52:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.774+0000 7fd92462d740 20 client.27875 trim_cache size 0 max 16384 2024-03-26T08:52:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd92462d740 1 Processor -- start 2024-03-26T08:52:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd92462d740 1 -- start start 2024-03-26T08:52:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd92462d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3da0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd92462d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3dbc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd92462d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3db0480 0x5580f3dac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd92462d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1c25f0 con 0x5580f3daa920 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd91affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3da0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd92462d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1c00f0 con 0x5580f3dab310 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd91affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3da0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57766/0 (socket says 172.21.15.7:57766) 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd91affd640 1 -- 172.21.15.7:0/3773052213 learned_addr learned my addr 172.21.15.7:0/3773052213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.789+0000 7fd92462d740 1 -- 172.21.15.7:0/3773052213 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1be170 con 0x5580f3db0480 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/3773052213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3dbc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd91b7fe640 1 --2- 172.21.15.7:0/3773052213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3db0480 0x5580f3dac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd91affd640 1 -- 172.21.15.7:0/3773052213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3db0480 msgr2=0x5580f3dac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd91affd640 1 --2- 172.21.15.7:0/3773052213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3db0480 0x5580f3dac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd91affd640 1 -- 172.21.15.7:0/3773052213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 msgr2=0x5580f3dbc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd91affd640 1 --2- 172.21.15.7:0/3773052213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3dbc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd91affd640 1 -- 172.21.15.7:0/3773052213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580f3dac120 con 0x5580f3daa920 2024-03-26T08:52:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd91affd640 1 --2- 172.21.15.7:0/3773052213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3da0eb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009fd0 tx=0x7fd90c097110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9740ba51d81180b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd9117fa640 1 -- 172.21.15.7:0/3773052213 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c098de0 con 0x5580f3daa920 2024-03-26T08:52:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd9117fa640 1 -- 172.21.15.7:0/3773052213 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90c099040 con 0x5580f3daa920 2024-03-26T08:52:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.790+0000 7fd9117fa640 1 -- 172.21.15.7:0/3773052213 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c09a190 con 0x5580f3daa920 2024-03-26T08:52:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.791+0000 7fd92462d740 1 -- 172.21.15.7:0/3773052213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 msgr2=0x5580f3da0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.791+0000 7fd92462d740 1 --2- 172.21.15.7:0/3773052213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3da0eb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009fd0 tx=0x7fd90c097110 comp rx=0 tx=0).stop 2024-03-26T08:52:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.791+0000 7fd92462d740 1 -- 172.21.15.7:0/3773052213 shutdown_connections 2024-03-26T08:52:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.791+0000 7fd92462d740 1 --2- 172.21.15.7:0/3773052213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3db0480 0x5580f3dac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.791+0000 7fd92462d740 1 --2- 172.21.15.7:0/3773052213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3dab310 0x5580f3dbc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.791+0000 7fd92462d740 1 --2- 172.21.15.7:0/3773052213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3da0eb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.791+0000 7fd92462d740 1 -- 172.21.15.7:0/3773052213 >> 172.21.15.7:0/3773052213 conn(0x5580f3da5e90 msgr2=0x5580f3da6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.791+0000 7fd92462d740 1 -- 172.21.15.7:0/3773052213 shutdown_connections 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.791+0000 7fd92462d740 1 -- 172.21.15.7:0/3773052213 wait complete. 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.792+0000 7fd92462d740 1 -- start start 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.792+0000 7fd92462d740 1 Processor -- start 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.792+0000 7fd92462d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3db9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.792+0000 7fd92462d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3dab310 0x5580f3db9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.792+0000 7fd92462d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 0x5580f3e1a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3db9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd92462d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1be770 con 0x5580f3daa920 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd92462d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1bfa90 con 0x5580f3db0480 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd92462d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c1bedd0 con 0x5580f3dab310 2024-03-26T08:52:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3db9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57768/0 (socket says 172.21.15.7:57768) 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91affd640 1 -- 172.21.15.7:0/3856305478 learned_addr learned my addr 172.21.15.7:0/3856305478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd92462d740 10 client.0 _ll_register_callbacks cb 0x7fd914004e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3dab310 0x5580f3db9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91b7fe640 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 0x5580f3e1a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91affd640 1 -- 172.21.15.7:0/3856305478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3dab310 msgr2=0x5580f3db9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91affd640 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3dab310 0x5580f3db9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91affd640 1 -- 172.21.15.7:0/3856305478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 msgr2=0x5580f3e1a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91affd640 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 0x5580f3e1a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91affd640 1 -- 172.21.15.7:0/3856305478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580f3e1ad00 con 0x5580f3daa920 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd91affd640 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3db9410 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd90c010a20 tx=0x7fd90c09ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8e871a44580b58 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd8feffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd8feffd640 1 -- 172.21.15.7:0/3856305478 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0a31c0 con 0x5580f3daa920 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd8feffd640 1 -- 172.21.15.7:0/3856305478 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd90c0973b0 con 0x5580f3daa920 2024-03-26T08:52:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd92462d740 1 -- 172.21.15.7:0/3856305478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd90c099040 con 0x5580f3daa920 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.793+0000 7fd8feffd640 1 -- 172.21.15.7:0/3856305478 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd90c0aa050 con 0x5580f3daa920 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd92462d740 10 client.18078 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd92462d740 1 -- 172.21.15.7:0/3856305478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5580f3e1bed0 con 0x5580f3daa920 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd8feffd640 1 -- 172.21.15.7:0/3856305478 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd90c099040 con 0x5580f3daa920 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd8feffd640 1 -- 172.21.15.7:0/3856305478 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd90c022060 con 0x5580f3daa920 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd92462d740 10 client.18078 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd92462d740 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd92462d740 20 client.18078 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd92462d740 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5580f3e1c7b0 0x5580f3e1ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd92462d740 4 client.18078 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd92462d740 1 -- 172.21.15.7:0/3856305478 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5580f3d173a0 con 0x5580f3e1c7b0 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.794+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5580f3e1c7b0 0x5580f3e1ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.795+0000 7fd91a7fc640 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5580f3e1c7b0 0x5580f3e1ebd0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e46b90af1248c13 server_cookie=43bd17fe57f13580 in_seq=0 out_seq=0 2024-03-26T08:52:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.795+0000 7fd8feffd640 10 client.18078 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.823+0000 7fd8feffd640 1 -- 172.21.15.7:0/3856305478 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5580f3d173a0 con 0x5580f3e1c7b0 2024-03-26T08:52:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.823+0000 7fd8feffd640 10 client.18078 handle_command_reply: tid=0 2024-03-26T08:52:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.823+0000 7fd92462d740 1 client.18078 shutdown 2024-03-26T08:52:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd92462d740 1 -- 172.21.15.7:0/3856305478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 msgr2=0x5580f3db9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd92462d740 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3db9410 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd90c010a20 tx=0x7fd90c09ada0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd92462d740 1 -- 172.21.15.7:0/3856305478 shutdown_connections 2024-03-26T08:52:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd92462d740 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5580f3e1c7b0 0x5580f3e1ebd0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd92462d740 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5580f3db0480 0x5580f3e1a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd92462d740 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5580f3dab310 0x5580f3db9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd92462d740 1 --2- 172.21.15.7:0/3856305478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5580f3daa920 0x5580f3db9410 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd92462d740 1 -- 172.21.15.7:0/3856305478 >> 172.21.15.7:0/3856305478 conn(0x5580f3da5e90 msgr2=0x5580f3da0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd8feffd640 0 client.18078 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.824+0000 7fd92462d740 1 -- 172.21.15.7:0/3856305478 shutdown_connections 2024-03-26T08:52:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.826+0000 7fd92462d740 1 -- 172.21.15.7:0/3856305478 wait complete. 2024-03-26T08:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.826+0000 7fd92462d740 20 client.18078 trim_cache size 0 max 16384 2024-03-26T08:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:52:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 -- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9000a27c0 msgr2=0x7fd9000a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9000a27c0 0x7fd9000a4c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd908005ec0 tx=0x7fd908005e10 comp rx=0 tx=0).stop 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 -- 172.21.15.7:0/622119803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91c1086a0 msgr2=0x7fd91c114d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91c1086a0 0x7fd91c114d10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00b450 tx=0x7fd90c00b910 comp rx=0 tx=0).stop 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 -- 172.21.15.7:0/622119803 shutdown_connections 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd9000a27c0 0x7fd9000a4c80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd91c10d9c0 0x7fd91c1b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd91c10b010 0x7fd91c115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 --2- 172.21.15.7:0/622119803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91c1086a0 0x7fd91c114d10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 -- 172.21.15.7:0/622119803 >> 172.21.15.7:0/622119803 conn(0x7fd91c06e6b0 msgr2=0x7fd91c10a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 -- 172.21.15.7:0/622119803 shutdown_connections 2024-03-26T08:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:37.827+0000 7fd921962640 1 -- 172.21.15.7:0/622119803 wait complete. 2024-03-26T08:52:38.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:38 smithi016 ceph-mon[35851]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 664 op/s 2024-03-26T08:52:38.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:38 smithi007 ceph-mon[32462]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 664 op/s 2024-03-26T08:52:38.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:38 smithi044 ceph-mon[36461]: pgmap v666: 97 pgs: 97 active+clean; 1.6 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 664 op/s 2024-03-26T08:52:39.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:52:39.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:52:39.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.959+0000 7f8338d5b640 1 -- 172.21.15.7:0/3353400667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334104ad0 msgr2=0x7f8334106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:39.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.959+0000 7f8338d5b640 1 --2- 172.21.15.7:0/3353400667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334104ad0 0x7f8334106ec0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8328009860 tx=0x7f832802bb70 comp rx=0 tx=0).stop 2024-03-26T08:52:39.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.961+0000 7f8338d5b640 1 -- 172.21.15.7:0/3353400667 shutdown_connections 2024-03-26T08:52:39.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.961+0000 7f8338d5b640 1 --2- 172.21.15.7:0/3353400667 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8334107400 0x7f83341097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.961+0000 7f8338d5b640 1 --2- 172.21.15.7:0/3353400667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8334104ad0 0x7f8334106ec0 secure :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8328009860 tx=0x7f832802bb70 comp rx=0 tx=0).stop 2024-03-26T08:52:39.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.961+0000 7f8338d5b640 1 --2- 172.21.15.7:0/3353400667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341021a0 0x7f8334104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.961+0000 7f8338d5b640 1 -- 172.21.15.7:0/3353400667 >> 172.21.15.7:0/3353400667 conn(0x7f83340fbb30 msgr2=0x7f83340fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:39.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.962+0000 7f8338d5b640 1 -- 172.21.15.7:0/3353400667 shutdown_connections 2024-03-26T08:52:39.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.962+0000 7f8338d5b640 1 -- 172.21.15.7:0/3353400667 wait complete. 2024-03-26T08:52:39.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8338d5b640 1 Processor -- start 2024-03-26T08:52:39.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8338d5b640 1 -- start start 2024-03-26T08:52:39.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8338d5b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341021a0 0x7f8334197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8338d5b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8334107400 0x7f8334198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f833259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341021a0 0x7f8334197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8338d5b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833419d160 0x7f833419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8338d5b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833410d310 con 0x7f83341021a0 2024-03-26T08:52:39.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8334107400 0x7f8334198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8338d5b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833410d190 con 0x7f833419d160 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8338d5b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833410d490 con 0x7f8334107400 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8334107400 0x7f8334198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59946/0 (socket says 172.21.15.7:59946) 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8331d9b640 1 -- 172.21.15.7:0/2417885744 learned_addr learned my addr 172.21.15.7:0/2417885744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f833259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341021a0 0x7f8334197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57792/0 (socket says 172.21.15.7:57792) 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.963+0000 7f8332d9d640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833419d160 0x7f833419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f8331d9b640 1 -- 172.21.15.7:0/2417885744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833419d160 msgr2=0x7f833419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f8331d9b640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833419d160 0x7f833419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f8331d9b640 1 -- 172.21.15.7:0/2417885744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341021a0 msgr2=0x7f8334197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f8331d9b640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341021a0 0x7f8334197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f8331d9b640 1 -- 172.21.15.7:0/2417885744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833419fbc0 con 0x7f8334107400 2024-03-26T08:52:39.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f8331d9b640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8334107400 0x7f8334198100 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8328000c00 tx=0x7f8328005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:39.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f83237fe640 1 -- 172.21.15.7:0/2417885744 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328039050 con 0x7f8334107400 2024-03-26T08:52:39.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f83237fe640 1 -- 172.21.15.7:0/2417885744 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8328042070 con 0x7f8334107400 2024-03-26T08:52:39.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f83237fe640 1 -- 172.21.15.7:0/2417885744 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328003850 con 0x7f8334107400 2024-03-26T08:52:39.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f8338d5b640 1 -- 172.21.15.7:0/2417885744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83340fff90 con 0x7f8334107400 2024-03-26T08:52:39.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.964+0000 7f8338d5b640 1 -- 172.21.15.7:0/2417885744 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83341003a0 con 0x7f8334107400 2024-03-26T08:52:39.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.967+0000 7f833ba24740 1 Processor -- start 2024-03-26T08:52:39.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.967+0000 7f833ba24740 1 -- start start 2024-03-26T08:52:39.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.967+0000 7f83237fe640 1 -- 172.21.15.7:0/2417885744 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f83280039f0 con 0x7f8334107400 2024-03-26T08:52:39.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.968+0000 7f833ba24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.968+0000 7f833ba24740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 0x557a81b6a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.968+0000 7f833ba24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b6cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.968+0000 7f833ba24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81aca3d0 con 0x557a81b6a900 2024-03-26T08:52:39.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.968+0000 7f833ba24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81a72ad0 con 0x557a81b64da0 2024-03-26T08:52:39.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.968+0000 7f833ba24740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81ac6470 con 0x557a81b65790 2024-03-26T08:52:39.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.968+0000 7f83237fe640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f831c0a2800 0x7f831c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.968+0000 7f83237fe640 1 -- 172.21.15.7:0/2417885744 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f83280efef0 con 0x7f8334107400 2024-03-26T08:52:39.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.969+0000 7f8331d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b6cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.969+0000 7f8331d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b6cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57808/0 (socket says 172.21.15.7:57808) 2024-03-26T08:52:39.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.969+0000 7f8331d9b640 1 -- 172.21.15.7:0/2892175992 learned_addr learned my addr 172.21.15.7:0/2892175992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:39.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.969+0000 7f833259c640 1 --2- 172.21.15.7:0/2892175992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 0x557a81b6a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.970+0000 7f8332d9d640 1 --2- 172.21.15.7:0/2892175992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.970+0000 7f8332d9d640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f831c0a2800 0x7f831c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.971+0000 7f8332d9d640 1 -- 172.21.15.7:0/2892175992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 msgr2=0x557a81b6a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.971+0000 7f8332d9d640 1 --2- 172.21.15.7:0/2892175992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 0x557a81b6a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.971+0000 7f8332d9d640 1 -- 172.21.15.7:0/2892175992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 msgr2=0x557a81b6cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.971+0000 7f8332d9d640 1 --2- 172.21.15.7:0/2892175992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b6cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.971+0000 7f8332d9d640 1 -- 172.21.15.7:0/2892175992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a818a87d0 con 0x557a81b64da0 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.971+0000 7f8332d9d640 1 --2- 172.21.15.7:0/2892175992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b651c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f832c011d30 tx=0x7f832c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9723e3de0dfcdebf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.971+0000 7f8332d9d640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f831c0a2800 0x7f831c0a4cc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f832c00fc90 tx=0x7f832c018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.972+0000 7f8320ff9640 1 -- 172.21.15.7:0/2892175992 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c011570 con 0x557a81b64da0 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.972+0000 7f8320ff9640 1 -- 172.21.15.7:0/2892175992 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f832c009d30 con 0x557a81b64da0 2024-03-26T08:52:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.972+0000 7f8320ff9640 1 -- 172.21.15.7:0/2892175992 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c005940 con 0x557a81b64da0 2024-03-26T08:52:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 -- 172.21.15.7:0/2892175992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 msgr2=0x557a81b651c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 --2- 172.21.15.7:0/2892175992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b651c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f832c011d30 tx=0x7f832c00f790 comp rx=0 tx=0).stop 2024-03-26T08:52:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 -- 172.21.15.7:0/2892175992 shutdown_connections 2024-03-26T08:52:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 --2- 172.21.15.7:0/2892175992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b6cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 --2- 172.21.15.7:0/2892175992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 0x557a81b6a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 --2- 172.21.15.7:0/2892175992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b651c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 -- 172.21.15.7:0/2892175992 >> 172.21.15.7:0/2892175992 conn(0x557a81b60310 msgr2=0x557a81b60710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 -- 172.21.15.7:0/2892175992 shutdown_connections 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 -- 172.21.15.7:0/2892175992 wait complete. 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 -- start start 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.973+0000 7f833ba24740 1 Processor -- start 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833ba24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b6f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833ba24740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 0x557a81b6f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833ba24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833ba24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81b6d7f0 con 0x557a81b6a900 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833ba24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81a86060 con 0x557a81b64da0 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833ba24740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a81b6d970 con 0x557a81b65790 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b6f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b6f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53816/0 (socket says 172.21.15.7:53816) 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833259c640 1 -- 172.21.15.7:0/4226766003 learned_addr learned my addr 172.21.15.7:0/4226766003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f8331d9b640 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 0x557a81b6f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833ba24740 10 client.0 _ll_register_callbacks cb 0x557a81a686a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f8332d9d640 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833259c640 1 -- 172.21.15.7:0/4226766003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 msgr2=0x557a81b6f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833259c640 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 0x557a81b6f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833259c640 1 -- 172.21.15.7:0/4226766003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 msgr2=0x557a81b767b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833259c640 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b767b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833259c640 1 -- 172.21.15.7:0/4226766003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a817de3f0 con 0x557a81b64da0 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f8332d9d640 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b767b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f833259c640 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b6f230 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f83240116f0 tx=0x7f8324011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd752c88b9abca21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.974+0000 7f8319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f8319ffb640 1 -- 172.21.15.7:0/4226766003 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832401e020 con 0x557a81b64da0 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f8319ffb640 1 -- 172.21.15.7:0/4226766003 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f832400f940 con 0x557a81b64da0 2024-03-26T08:52:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f833ba24740 1 -- 172.21.15.7:0/4226766003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f832c009d30 con 0x557a81b64da0 2024-03-26T08:52:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f8319ffb640 1 -- 172.21.15.7:0/4226766003 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f832400fb40 con 0x557a81b64da0 2024-03-26T08:52:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f833ba24740 10 client.27893 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f833ba24740 1 -- 172.21.15.7:0/4226766003 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557a8179e0f0 con 0x557a81b64da0 2024-03-26T08:52:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f8319ffb640 1 -- 172.21.15.7:0/4226766003 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8324005500 con 0x557a81b64da0 2024-03-26T08:52:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f8319ffb640 1 -- 172.21.15.7:0/4226766003 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8324019070 con 0x557a81b64da0 2024-03-26T08:52:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f833ba24740 10 client.27893 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f833ba24740 10 client.27893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:39.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f833ba24740 20 client.27893 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:39.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f833ba24740 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557a81b7ac60 0x557a81b7d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f833ba24740 4 client.27893 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:39.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f833ba24740 1 -- 172.21.15.7:0/4226766003 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f832c009d30 con 0x557a81b7ac60 2024-03-26T08:52:39.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.975+0000 7f8331d9b640 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557a81b7ac60 0x557a81b7d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.976+0000 7f8319ffb640 10 client.27893 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:39.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.976+0000 7f8331d9b640 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557a81b7ac60 0x557a81b7d0e0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17ddb0355772ed4 server_cookie=4e44f43de8ebaace in_seq=0 out_seq=0 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f8319ffb640 1 -- 172.21.15.7:0/4226766003 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f832c009d30 con 0x557a81b7ac60 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f8319ffb640 10 client.27893 handle_command_reply: tid=0 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 client.27893 shutdown 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 -- 172.21.15.7:0/4226766003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 msgr2=0x557a81b6f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b6f230 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f83240116f0 tx=0x7f8324011af0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 -- 172.21.15.7:0/4226766003 shutdown_connections 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557a81b7ac60 0x557a81b7d0e0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b767b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b65790 0x557a81b6f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f8319ffb640 0 client.27893 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 --2- 172.21.15.7:0/4226766003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b6f230 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:39.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 -- 172.21.15.7:0/4226766003 >> 172.21.15.7:0/4226766003 conn(0x557a81b60310 msgr2=0x557a81b5f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:39.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.979+0000 7f833ba24740 1 -- 172.21.15.7:0/4226766003 shutdown_connections 2024-03-26T08:52:39.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.980+0000 7f833ba24740 1 -- 172.21.15.7:0/4226766003 wait complete. 2024-03-26T08:52:39.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:39.980+0000 7f833ba24740 20 client.27893 trim_cache size 0 max 16384 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833ba24740 1 Processor -- start 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833ba24740 1 -- start start 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833ba24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b819a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833ba24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b65790 0x557a81b76770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833ba24740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b6a900 0x557a81b81ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833ba24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83341b1bf0 con 0x557a81b65790 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833ba24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83341af6f0 con 0x557a81b64da0 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833ba24740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83341ad770 con 0x557a81b6a900 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b819a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b819a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53818/0 (socket says 172.21.15.7:53818) 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f833259c640 1 -- 172.21.15.7:0/1111121989 learned_addr learned my addr 172.21.15.7:0/1111121989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.002+0000 7f8332d9d640 1 --2- 172.21.15.7:0/1111121989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b6a900 0x557a81b81ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.003+0000 7f833259c640 1 -- 172.21.15.7:0/1111121989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b6a900 msgr2=0x557a81b81ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.003+0000 7f833259c640 1 --2- 172.21.15.7:0/1111121989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b6a900 0x557a81b81ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.003+0000 7f833259c640 1 -- 172.21.15.7:0/1111121989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b65790 msgr2=0x557a81b76770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.003+0000 7f833259c640 1 --2- 172.21.15.7:0/1111121989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b65790 0x557a81b76770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.003+0000 7f833259c640 1 -- 172.21.15.7:0/1111121989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a81b61080 con 0x557a81b64da0 2024-03-26T08:52:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.003+0000 7f833259c640 1 --2- 172.21.15.7:0/1111121989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b819a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8324009970 tx=0x7f832400fad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c44a7356d6a6d44 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.003+0000 7f831bfff640 1 -- 172.21.15.7:0/1111121989 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832401e030 con 0x557a81b64da0 2024-03-26T08:52:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.003+0000 7f831bfff640 1 -- 172.21.15.7:0/1111121989 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8324025d90 con 0x557a81b64da0 2024-03-26T08:52:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.003+0000 7f831bfff640 1 -- 172.21.15.7:0/1111121989 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324003830 con 0x557a81b64da0 2024-03-26T08:52:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.004+0000 7f833ba24740 1 -- 172.21.15.7:0/1111121989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 msgr2=0x557a81b819a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.004+0000 7f833ba24740 1 --2- 172.21.15.7:0/1111121989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b819a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8324009970 tx=0x7f832400fad0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.004+0000 7f833ba24740 1 -- 172.21.15.7:0/1111121989 shutdown_connections 2024-03-26T08:52:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.004+0000 7f833ba24740 1 --2- 172.21.15.7:0/1111121989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b6a900 0x557a81b81ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.004+0000 7f833ba24740 1 --2- 172.21.15.7:0/1111121989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b65790 0x557a81b76770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.004+0000 7f833ba24740 1 --2- 172.21.15.7:0/1111121989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b64da0 0x557a81b819a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.004+0000 7f833ba24740 1 -- 172.21.15.7:0/1111121989 >> 172.21.15.7:0/1111121989 conn(0x557a81b60310 msgr2=0x557a81b63570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.005+0000 7f833ba24740 1 -- 172.21.15.7:0/1111121989 shutdown_connections 2024-03-26T08:52:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.005+0000 7f833ba24740 1 -- 172.21.15.7:0/1111121989 wait complete. 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f833ba24740 1 -- start start 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f833ba24740 1 Processor -- start 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f833ba24740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b64da0 0x557a81b76390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f833ba24740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b65790 0x557a81b749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f833ba24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b74ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f833ba24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83341add70 con 0x557a81b6a900 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f833259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b64da0 0x557a81b76390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f833ba24740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83341af090 con 0x557a81b65790 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f833ba24740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83341ae3d0 con 0x557a81b64da0 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b64da0 0x557a81b76390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59992/0 (socket says 172.21.15.7:59992) 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833259c640 1 -- 172.21.15.7:0/4166715158 learned_addr learned my addr 172.21.15.7:0/4166715158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833ba24740 10 client.0 _ll_register_callbacks cb 0x7f8324009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.006+0000 7f8332d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b74ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f8331d9b640 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b65790 0x557a81b749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833259c640 1 -- 172.21.15.7:0/4166715158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b65790 msgr2=0x557a81b749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833259c640 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b65790 0x557a81b749a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833259c640 1 -- 172.21.15.7:0/4166715158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 msgr2=0x557a81b74ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833259c640 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b74ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833259c640 1 -- 172.21.15.7:0/4166715158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a81b75420 con 0x557a81b64da0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833259c640 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b64da0 0x557a81b76390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8324004ed0 tx=0x7f8324025ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef0d2492d38792d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f8319ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f8319ffb640 1 -- 172.21.15.7:0/4166715158 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832401e040 con 0x557a81b64da0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f8319ffb640 1 -- 172.21.15.7:0/4166715158 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8324003800 con 0x557a81b64da0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.007+0000 7f833ba24740 1 -- 172.21.15.7:0/4166715158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8324025d90 con 0x557a81b64da0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f8319ffb640 1 -- 172.21.15.7:0/4166715158 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8324005c50 con 0x557a81b64da0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f833ba24740 10 client.27913 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f833ba24740 1 -- 172.21.15.7:0/4166715158 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557a81bd1200 con 0x557a81b64da0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f8319ffb640 1 -- 172.21.15.7:0/4166715158 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f832402e480 con 0x557a81b64da0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f8319ffb640 1 -- 172.21.15.7:0/4166715158 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8324019040 con 0x557a81b64da0 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f833ba24740 10 client.27913 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:40.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f833ba24740 10 client.27913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f833ba24740 20 client.27913 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f833ba24740 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557a81bd1ab0 0x557a81bd3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f833ba24740 4 client.27913 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.008+0000 7f833ba24740 1 -- 172.21.15.7:0/4166715158 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f8324025d90 con 0x557a81bd1ab0 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.009+0000 7f8331d9b640 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557a81bd1ab0 0x557a81bd3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.009+0000 7f8331d9b640 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557a81bd1ab0 0x557a81bd3ed0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c41541f591b4c4 server_cookie=17211b6ca09c563 in_seq=0 out_seq=0 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.009+0000 7f8319ffb640 10 client.27913 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.010+0000 7f8319ffb640 1 -- 172.21.15.7:0/4166715158 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f8324025d90 con 0x557a81bd1ab0 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.010+0000 7f8319ffb640 10 client.27913 handle_command_reply: tid=0 2024-03-26T08:52:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.010+0000 7f833ba24740 1 client.27913 shutdown 2024-03-26T08:52:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 -- 172.21.15.7:0/4166715158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b64da0 msgr2=0x557a81b76390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b64da0 0x557a81b76390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8324004ed0 tx=0x7f8324025ae0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 -- 172.21.15.7:0/4166715158 shutdown_connections 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557a81bd1ab0 0x557a81bd3ed0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557a81b6a900 0x557a81b74ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557a81b65790 0x557a81b749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 --2- 172.21.15.7:0/4166715158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557a81b64da0 0x557a81b76390 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f8319ffb640 0 client.27913 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 -- 172.21.15.7:0/4166715158 >> 172.21.15.7:0/4166715158 conn(0x557a81b60310 msgr2=0x557a81b5eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 -- 172.21.15.7:0/4166715158 shutdown_connections 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 1 -- 172.21.15.7:0/4166715158 wait complete. 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.012+0000 7f833ba24740 20 client.27913 trim_cache size 0 max 16384 2024-03-26T08:52:40.010 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:52:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:52:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:52:40.011 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:52:40.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.013+0000 7f8338d5b640 1 -- 172.21.15.7:0/2417885744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f831c0a2800 msgr2=0x7f831c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.013+0000 7f8338d5b640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f831c0a2800 0x7f831c0a4cc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f832c00fc90 tx=0x7f832c018000 comp rx=0 tx=0).stop 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.013+0000 7f8338d5b640 1 -- 172.21.15.7:0/2417885744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8334107400 msgr2=0x7f8334198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.013+0000 7f8338d5b640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8334107400 0x7f8334198100 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8328000c00 tx=0x7f8328005d80 comp rx=0 tx=0).stop 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.013+0000 7f8338d5b640 1 -- 172.21.15.7:0/2417885744 shutdown_connections 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.014+0000 7f8338d5b640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f831c0a2800 0x7f831c0a4cc0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.014+0000 7f8338d5b640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f833419d160 0x7f833419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.014+0000 7f8338d5b640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8334107400 0x7f8334198100 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.014+0000 7f8338d5b640 1 --2- 172.21.15.7:0/2417885744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341021a0 0x7f8334197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.014+0000 7f8338d5b640 1 -- 172.21.15.7:0/2417885744 >> 172.21.15.7:0/2417885744 conn(0x7f83340fbb30 msgr2=0x7f83341055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.015+0000 7f8338d5b640 1 -- 172.21.15.7:0/2417885744 shutdown_connections 2024-03-26T08:52:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.015+0000 7f8338d5b640 1 -- 172.21.15.7:0/2417885744 wait complete. 2024-03-26T08:52:40.025 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:52:40.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:52:40.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.206+0000 7febda6ff640 1 -- 172.21.15.7:0/665858378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41017d0 msgr2=0x7febd4101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.206+0000 7febda6ff640 1 --2- 172.21.15.7:0/665858378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41017d0 0x7febd4101c50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7febc8009860 tx=0x7febc802bb70 comp rx=0 tx=0).stop 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.206+0000 7febda6ff640 1 -- 172.21.15.7:0/665858378 shutdown_connections 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.206+0000 7febda6ff640 1 --2- 172.21.15.7:0/665858378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd4102190 0x7febd4108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.206+0000 7febda6ff640 1 --2- 172.21.15.7:0/665858378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41017d0 0x7febd4101c50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.206+0000 7febda6ff640 1 --2- 172.21.15.7:0/665858378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd41005e0 0x7febd4100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.206+0000 7febda6ff640 1 -- 172.21.15.7:0/665858378 >> 172.21.15.7:0/665858378 conn(0x7febd40fbb30 msgr2=0x7febd40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.206+0000 7febda6ff640 1 -- 172.21.15.7:0/665858378 shutdown_connections 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.206+0000 7febda6ff640 1 -- 172.21.15.7:0/665858378 wait complete. 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.207+0000 7febda6ff640 1 Processor -- start 2024-03-26T08:52:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.207+0000 7febda6ff640 1 -- start start 2024-03-26T08:52:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.208+0000 7febda6ff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41005e0 0x7febd4199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febda6ff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41017d0 0x7febd419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febda6ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd4102190 0x7febd41a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febd3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41005e0 0x7febd4199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febd3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41005e0 0x7febd4199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60014/0 (socket says 172.21.15.7:60014) 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febd3fff640 1 -- 172.21.15.7:0/4103567228 learned_addr learned my addr 172.21.15.7:0/4103567228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd410f700 con 0x7febd41017d0 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd410f580 con 0x7febd4102190 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febd8c9c640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd4102190 0x7febd41a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd410f880 con 0x7febd41005e0 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.209+0000 7febd37fe640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41017d0 0x7febd419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.210+0000 7febd8c9c640 1 -- 172.21.15.7:0/4103567228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41005e0 msgr2=0x7febd4199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.210+0000 7febd8c9c640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41005e0 0x7febd4199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.210+0000 7febd8c9c640 1 -- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41017d0 msgr2=0x7febd419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.210+0000 7febd8c9c640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41017d0 0x7febd419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.210+0000 7febd8c9c640 1 -- 172.21.15.7:0/4103567228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd41a1810 con 0x7febd4102190 2024-03-26T08:52:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.210+0000 7febd8c9c640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd4102190 0x7febd41a11a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7febcc00ef00 tx=0x7febcc00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.211+0000 7febd17fa640 1 -- 172.21.15.7:0/4103567228 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc004b20 con 0x7febd4102190 2024-03-26T08:52:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.211+0000 7febd17fa640 1 -- 172.21.15.7:0/4103567228 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febcc003e50 con 0x7febd4102190 2024-03-26T08:52:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.211+0000 7febd17fa640 1 -- 172.21.15.7:0/4103567228 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00b570 con 0x7febd4102190 2024-03-26T08:52:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.211+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd41a1b00 con 0x7febd4102190 2024-03-26T08:52:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.211+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd41a1fd0 con 0x7febd4102190 2024-03-26T08:52:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.213+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd4107900 con 0x7febd4102190 2024-03-26T08:52:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.213+0000 7febd17fa640 1 -- 172.21.15.7:0/4103567228 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7febcc01d020 con 0x7febd4102190 2024-03-26T08:52:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.214+0000 7febd17fa640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febbc0a2950 0x7febbc0a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.214+0000 7febd17fa640 1 -- 172.21.15.7:0/4103567228 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7febcc0ce870 con 0x7febd4102190 2024-03-26T08:52:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.214+0000 7febd3fff640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febbc0a2950 0x7febbc0a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.214+0000 7febd3fff640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febbc0a2950 0x7febbc0a4e10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7febc40058c0 tx=0x7febc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.221+0000 7febd17fa640 1 -- 172.21.15.7:0/4103567228 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7febcc09b910 con 0x7febd4102190 2024-03-26T08:52:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.606+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febd40622a0 con 0x7febd4102190 2024-03-26T08:52:40.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.607+0000 7febd17fa640 1 -- 172.21.15.7:0/4103567228 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7febcc09b210 con 0x7febd4102190 2024-03-26T08:52:40.603 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:52:40.603 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:52:40.604 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:52:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.610+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febbc0a2950 msgr2=0x7febbc0a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.610+0000 7febda6ff640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febbc0a2950 0x7febbc0a4e10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7febc40058c0 tx=0x7febc4005f90 comp rx=0 tx=0).stop 2024-03-26T08:52:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.610+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd4102190 msgr2=0x7febd41a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.610+0000 7febda6ff640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd4102190 0x7febd41a11a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7febcc00ef00 tx=0x7febcc00c740 comp rx=0 tx=0).stop 2024-03-26T08:52:40.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.611+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 shutdown_connections 2024-03-26T08:52:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.611+0000 7febda6ff640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7febbc0a2950 0x7febbc0a4e10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.611+0000 7febda6ff640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd4102190 0x7febd41a11a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.611+0000 7febda6ff640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febd41017d0 0x7febd419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.611+0000 7febda6ff640 1 --2- 172.21.15.7:0/4103567228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febd41005e0 0x7febd4199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.611+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 >> 172.21.15.7:0/4103567228 conn(0x7febd40fbb30 msgr2=0x7febd4104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.612+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 shutdown_connections 2024-03-26T08:52:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.612+0000 7febda6ff640 1 -- 172.21.15.7:0/4103567228 wait complete. 2024-03-26T08:52:40.622 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:52:40.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:52:40.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:40 smithi016 ceph-mon[35851]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 609 op/s 2024-03-26T08:52:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.773+0000 7f4b28219640 1 -- 172.21.15.7:0/1932424637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b201017b0 msgr2=0x7f4b20101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.773+0000 7f4b28219640 1 --2- 172.21.15.7:0/1932424637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b201017b0 0x7f4b20101c30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009860 tx=0x7f4b1002bb70 comp rx=0 tx=0).stop 2024-03-26T08:52:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.773+0000 7f4b28219640 1 -- 172.21.15.7:0/1932424637 shutdown_connections 2024-03-26T08:52:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.773+0000 7f4b28219640 1 --2- 172.21.15.7:0/1932424637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b20102170 0x7f4b20108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.773+0000 7f4b28219640 1 --2- 172.21.15.7:0/1932424637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b201017b0 0x7f4b20101c30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.773+0000 7f4b28219640 1 --2- 172.21.15.7:0/1932424637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b201005c0 0x7f4b201009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.773+0000 7f4b28219640 1 -- 172.21.15.7:0/1932424637 >> 172.21.15.7:0/1932424637 conn(0x7f4b200fbb30 msgr2=0x7f4b200fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.773+0000 7f4b28219640 1 -- 172.21.15.7:0/1932424637 shutdown_connections 2024-03-26T08:52:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.773+0000 7f4b28219640 1 -- 172.21.15.7:0/1932424637 wait complete. 2024-03-26T08:52:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b28219640 1 Processor -- start 2024-03-26T08:52:40.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b28219640 1 -- start start 2024-03-26T08:52:40.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b28219640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b201005c0 0x7f4b2010af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b28219640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b201017b0 0x7f4b20109580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b28219640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b20102170 0x7f4b20109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b28219640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2010d360 con 0x7f4b201017b0 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b28219640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2010d1e0 con 0x7f4b201005c0 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b267b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b20102170 0x7f4b20109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b267b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b20102170 0x7f4b20109ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60034/0 (socket says 172.21.15.7:60034) 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b267b6640 1 -- 172.21.15.7:0/2509909895 learned_addr learned my addr 172.21.15.7:0/2509909895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.774+0000 7f4b28219640 1 -- 172.21.15.7:0/2509909895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2010d4e0 con 0x7f4b20102170 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.775+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b201005c0 0x7f4b2010af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b257b4640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b201017b0 0x7f4b20109580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b267b6640 1 -- 172.21.15.7:0/2509909895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b201005c0 msgr2=0x7f4b2010af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b267b6640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b201005c0 0x7f4b2010af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b267b6640 1 -- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b201017b0 msgr2=0x7f4b20109580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b267b6640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b201017b0 0x7f4b20109580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b267b6640 1 -- 172.21.15.7:0/2509909895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2010a350 con 0x7f4b20102170 2024-03-26T08:52:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b267b6640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b20102170 0x7f4b20109ac0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800efd0 tx=0x7f4b1800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b16ffd640 1 -- 172.21.15.7:0/2509909895 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18003b70 con 0x7f4b20102170 2024-03-26T08:52:40.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b16ffd640 1 -- 172.21.15.7:0/2509909895 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b18003d10 con 0x7f4b20102170 2024-03-26T08:52:40.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.776+0000 7f4b16ffd640 1 -- 172.21.15.7:0/2509909895 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b180108d0 con 0x7f4b20102170 2024-03-26T08:52:40.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.777+0000 7f4b28219640 1 -- 172.21.15.7:0/2509909895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b201a1b50 con 0x7f4b20102170 2024-03-26T08:52:40.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.777+0000 7f4b28219640 1 -- 172.21.15.7:0/2509909895 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b201a2020 con 0x7f4b20102170 2024-03-26T08:52:40.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.780+0000 7f4b16ffd640 1 -- 172.21.15.7:0/2509909895 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4b18004020 con 0x7f4b20102170 2024-03-26T08:52:40.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.780+0000 7f4b2ae08740 1 Processor -- start 2024-03-26T08:52:40.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.780+0000 7f4b2ae08740 1 -- start start 2024-03-26T08:52:40.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.780+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 0x5595997d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57862/0 (socket says 172.21.15.7:57862) 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 -- 172.21.15.7:0/3168616360 learned_addr learned my addr 172.21.15.7:0/3168616360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3168616360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559599790610 con 0x5595997cd690 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3168616360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595997333c0 con 0x5595997ce080 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3168616360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595996dbd60 con 0x5595997d31f0 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b16ffd640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4b0c0a2880 0x7f4b0c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 --2- 172.21.15.7:0/3168616360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/3168616360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 0x5595997d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b267b6640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4b0c0a2880 0x7f4b0c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 -- 172.21.15.7:0/3168616360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 msgr2=0x5595997d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 --2- 172.21.15.7:0/3168616360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 0x5595997d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 -- 172.21.15.7:0/3168616360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 msgr2=0x5595997cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 --2- 172.21.15.7:0/3168616360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 -- 172.21.15.7:0/3168616360 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595994473f0 con 0x5595997d31f0 2024-03-26T08:52:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b16ffd640 1 -- 172.21.15.7:0/2509909895 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b180cee30 con 0x7f4b20102170 2024-03-26T08:52:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.781+0000 7f4b257b4640 1 --2- 172.21.15.7:0/3168616360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997d5680 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4b10032570 tx=0x7f4b1002bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f12dc81824c603c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.782+0000 7f4b0bfff640 1 -- 172.21.15.7:0/3168616360 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1004a070 con 0x5595997d31f0 2024-03-26T08:52:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.782+0000 7f4b0bfff640 1 -- 172.21.15.7:0/3168616360 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b10034640 con 0x5595997d31f0 2024-03-26T08:52:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.782+0000 7f4b0bfff640 1 -- 172.21.15.7:0/3168616360 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10039040 con 0x5595997d31f0 2024-03-26T08:52:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.782+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3168616360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 msgr2=0x5595997d5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.782+0000 7f4b267b6640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4b0c0a2880 0x7f4b0c0a4d40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800cba0 tx=0x7f4b180973b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.782+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3168616360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997d5680 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4b10032570 tx=0x7f4b1002bb70 comp rx=0 tx=0).stop 2024-03-26T08:52:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.783+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3168616360 shutdown_connections 2024-03-26T08:52:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.783+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3168616360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997d5680 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.783+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3168616360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 0x5595997d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.783+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3168616360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.783+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3168616360 >> 172.21.15.7:0/3168616360 conn(0x5595997c8c00 msgr2=0x5595997c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.783+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3168616360 shutdown_connections 2024-03-26T08:52:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.783+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3168616360 wait complete. 2024-03-26T08:52:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.783+0000 7f4b2ae08740 1 -- start start 2024-03-26T08:52:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.783+0000 7f4b2ae08740 1 Processor -- start 2024-03-26T08:52:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.784+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997d7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.784+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997ce080 0x5595997d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b257b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997d7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b257b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997d7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53892/0 (socket says 172.21.15.7:53892) 2024-03-26T08:52:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997d31f0 0x5595997dee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b257b4640 1 -- 172.21.15.7:0/593078209 learned_addr learned my addr 172.21.15.7:0/593078209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595996ef240 con 0x5595997d31f0 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55959972f460 con 0x5595997cd690 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595997d6140 con 0x5595997ce080 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997ce080 0x5595997d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b2ae08740 10 client.0 _ll_register_callbacks cb 0x5595996eec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b257b4640 1 -- 172.21.15.7:0/593078209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997ce080 msgr2=0x5595997d7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b257b4640 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997ce080 0x5595997d7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b257b4640 1 -- 172.21.15.7:0/593078209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997d31f0 msgr2=0x5595997dee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b257b4640 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997d31f0 0x5595997dee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b257b4640 1 -- 172.21.15.7:0/593078209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595995117d0 con 0x5595997cd690 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b257b4640 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997d7900 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4b10037040 tx=0x7f4b10005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09128045172a11b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.785+0000 7f4b097fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.786+0000 7f4b097fa640 1 -- 172.21.15.7:0/593078209 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10053060 con 0x5595997cd690 2024-03-26T08:52:40.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.786+0000 7f4b097fa640 1 -- 172.21.15.7:0/593078209 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b10032850 con 0x5595997cd690 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.786+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b10034640 con 0x5595997cd690 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.786+0000 7f4b097fa640 1 -- 172.21.15.7:0/593078209 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b10039040 con 0x5595997cd690 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.786+0000 7f4b2ae08740 10 client.27914 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.786+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5595997e2140 con 0x5595997cd690 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.786+0000 7f4b097fa640 1 -- 172.21.15.7:0/593078209 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b10034640 con 0x5595997cd690 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b097fa640 1 -- 172.21.15.7:0/593078209 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4b10058050 con 0x5595997cd690 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b2ae08740 10 client.27914 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b2ae08740 10 client.27914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b2ae08740 20 client.27914 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5595997e3490 0x5595997e5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b2ae08740 4 client.27914 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559599740900 con 0x5595997e3490 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5595997e3490 0x5595997e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b097fa640 10 client.27914 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:52:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.787+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5595997e3490 0x5595997e5880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39fe98448b83d016 server_cookie=9f329780e8293d21 in_seq=0 out_seq=0 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b097fa640 1 -- 172.21.15.7:0/593078209 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559599740900 con 0x5595997e3490 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b097fa640 10 client.27914 handle_command_reply: tid=0 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 client.27914 shutdown 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 msgr2=0x5595997d7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997d7900 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4b10037040 tx=0x7f4b10005f70 comp rx=0 tx=0).stop 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 shutdown_connections 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5595997e3490 0x5595997e5880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997d31f0 0x5595997dee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997ce080 0x5595997d7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/593078209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997d7900 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b097fa640 0 client.27914 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 >> 172.21.15.7:0/593078209 conn(0x5595997c8c00 msgr2=0x5595997c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 shutdown_connections 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 1 -- 172.21.15.7:0/593078209 wait complete. 2024-03-26T08:52:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.788+0000 7f4b2ae08740 20 client.27914 trim_cache size 0 max 16384 2024-03-26T08:52:40.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:40 smithi007 ceph-mon[32462]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 609 op/s 2024-03-26T08:52:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.807+0000 7f4b2ae08740 1 Processor -- start 2024-03-26T08:52:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.807+0000 7f4b2ae08740 1 -- start start 2024-03-26T08:52:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.807+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 0x5595997deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b2ae08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201ad170 con 0x5595997cd690 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b257b4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b257b4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997d7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57882/0 (socket says 172.21.15.7:57882) 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b25fb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 0x5595997deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b257b4640 1 -- 172.21.15.7:0/1475509423 learned_addr learned my addr 172.21.15.7:0/1475509423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b2ae08740 1 -- 172.21.15.7:0/1475509423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201aac70 con 0x5595997ce080 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b2ae08740 1 -- 172.21.15.7:0/1475509423 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201a8cf0 con 0x5595997d31f0 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b257b4640 1 --2- 172.21.15.7:0/1475509423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b25fb5640 1 -- 172.21.15.7:0/1475509423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 msgr2=0x5595997cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/1475509423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b25fb5640 1 -- 172.21.15.7:0/1475509423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 msgr2=0x5595997d7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/1475509423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997d7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b25fb5640 1 -- 172.21.15.7:0/1475509423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595997c9970 con 0x5595997ce080 2024-03-26T08:52:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.808+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/1475509423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 0x5595997deda0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x5595997c9b70 tx=0x7f4b1c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4e5f1a3b5223047 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.809+0000 7f4b0b7fe640 1 -- 172.21.15.7:0/1475509423 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c024070 con 0x5595997ce080 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.809+0000 7f4b0b7fe640 1 -- 172.21.15.7:0/1475509423 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4b1c0202f0 con 0x5595997ce080 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.809+0000 7f4b0b7fe640 1 -- 172.21.15.7:0/1475509423 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c020770 con 0x5595997ce080 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.809+0000 7f4b2ae08740 1 -- 172.21.15.7:0/1475509423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 msgr2=0x5595997deda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.809+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/1475509423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 0x5595997deda0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x5595997c9b70 tx=0x7f4b1c009260 comp rx=0 tx=0).stop 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.810+0000 7f4b2ae08740 1 -- 172.21.15.7:0/1475509423 shutdown_connections 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.810+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/1475509423 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.810+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/1475509423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997ce080 0x5595997deda0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.810+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/1475509423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997cd690 0x5595997d7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.810+0000 7f4b2ae08740 1 -- 172.21.15.7:0/1475509423 >> 172.21.15.7:0/1475509423 conn(0x5595997c8c00 msgr2=0x5595997cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.810+0000 7f4b2ae08740 1 -- 172.21.15.7:0/1475509423 shutdown_connections 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.810+0000 7f4b2ae08740 1 -- 172.21.15.7:0/1475509423 wait complete. 2024-03-26T08:52:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.810+0000 7f4b2ae08740 1 -- start start 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.810+0000 7f4b2ae08740 1 Processor -- start 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997ce080 0x5595997dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b2ae08740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b25fb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997ce080 0x5595997dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b2ae08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201a92f0 con 0x5595997ce080 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b257b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b25fb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997ce080 0x5595997dd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57884/0 (socket says 172.21.15.7:57884) 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b25fb5640 1 -- 172.21.15.7:0/3508221076 learned_addr learned my addr 172.21.15.7:0/3508221076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201aa610 con 0x5595997cd690 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201a9950 con 0x5595997d31f0 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b2ae08740 10 client.0 _ll_register_callbacks cb 0x7f4b100096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b257b4640 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b257b4640 1 -- 172.21.15.7:0/3508221076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 msgr2=0x5595997de2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b257b4640 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997de2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b257b4640 1 -- 172.21.15.7:0/3508221076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997ce080 msgr2=0x5595997dd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b257b4640 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997ce080 0x5595997dd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b257b4640 1 -- 172.21.15.7:0/3508221076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595997de8d0 con 0x5595997cd690 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997ce080 0x5595997dd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b257b4640 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997dcd90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x5595996dbe60 tx=0x7f4b1002bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fa937af7e6ab26c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.811+0000 7f4b097fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b097fa640 1 -- 172.21.15.7:0/3508221076 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10053070 con 0x5595997cd690 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b097fa640 1 -- 172.21.15.7:0/3508221076 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b10034440 con 0x5595997cd690 2024-03-26T08:52:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55959973ac50 con 0x5595997cd690 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b097fa640 1 -- 172.21.15.7:0/3508221076 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b1006c050 con 0x5595997cd690 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b2ae08740 10 client.27920 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55959983f420 con 0x5595997cd690 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b097fa640 1 -- 172.21.15.7:0/3508221076 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55959973ac50 con 0x5595997cd690 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b097fa640 1 -- 172.21.15.7:0/3508221076 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4b10058040 con 0x5595997cd690 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b2ae08740 10 client.27920 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b2ae08740 10 client.27920 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b2ae08740 20 client.27920 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55959983fc70 0x559599842090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b2ae08740 4 client.27920 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5595996b8640 con 0x55959983fc70 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.812+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55959983fc70 0x559599842090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b097fa640 10 client.27920 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b25fb5640 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55959983fc70 0x559599842090 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a36e0ccf0e6055aa server_cookie=a7b1583e396df51f in_seq=0 out_seq=0 2024-03-26T08:52:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b097fa640 1 -- 172.21.15.7:0/3508221076 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5595996b8640 con 0x55959983fc70 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b097fa640 10 client.27920 handle_command_reply: tid=0 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b2ae08740 1 client.27920 shutdown 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 msgr2=0x5595997dcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997dcd90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x5595996dbe60 tx=0x7f4b1002bb70 comp rx=0 tx=0).stop 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 shutdown_connections 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55959983fc70 0x559599842090 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595997d31f0 0x5595997de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595997ce080 0x5595997dd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b2ae08740 1 --2- 172.21.15.7:0/3508221076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595997cd690 0x5595997dcd90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b097fa640 0 client.27920 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.813+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 >> 172.21.15.7:0/3508221076 conn(0x5595997c8c00 msgr2=0x5595997cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.814+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 shutdown_connections 2024-03-26T08:52:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.815+0000 7f4b2ae08740 1 -- 172.21.15.7:0/3508221076 wait complete. 2024-03-26T08:52:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.815+0000 7f4b2ae08740 20 client.27920 trim_cache size 0 max 16384 2024-03-26T08:52:40.815 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.816+0000 7f4b28219640 1 -- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4b0c0a2880 msgr2=0x7f4b0c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.816+0000 7f4b28219640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4b0c0a2880 0x7f4b0c0a4d40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800cba0 tx=0x7f4b180973b0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.816+0000 7f4b28219640 1 -- 172.21.15.7:0/2509909895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b20102170 msgr2=0x7f4b20109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.817+0000 7f4b28219640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b20102170 0x7f4b20109ac0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800efd0 tx=0x7f4b1800c6a0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.817+0000 7f4b28219640 1 -- 172.21.15.7:0/2509909895 shutdown_connections 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.817+0000 7f4b28219640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4b0c0a2880 0x7f4b0c0a4d40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.817+0000 7f4b28219640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4b20102170 0x7f4b20109ac0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.817+0000 7f4b28219640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b201017b0 0x7f4b20109580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.817+0000 7f4b28219640 1 --2- 172.21.15.7:0/2509909895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b201005c0 0x7f4b2010af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.817+0000 7f4b28219640 1 -- 172.21.15.7:0/2509909895 >> 172.21.15.7:0/2509909895 conn(0x7f4b200fbb30 msgr2=0x7f4b20104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.817+0000 7f4b28219640 1 -- 172.21.15.7:0/2509909895 shutdown_connections 2024-03-26T08:52:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.817+0000 7f4b28219640 1 -- 172.21.15.7:0/2509909895 wait complete. 2024-03-26T08:52:40.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:40 smithi044 ceph-mon[36461]: pgmap v667: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 MiB/s wr, 609 op/s 2024-03-26T08:52:40.828 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:52:40.828 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:52:40.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.996+0000 7f077ffff640 1 -- 172.21.15.7:0/1961442705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 msgr2=0x7f077809d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.996+0000 7f077ffff640 1 --2- 172.21.15.7:0/1961442705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 0x7f077809d0b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f07800674e0 tx=0x7f0780097650 comp rx=0 tx=0).stop 2024-03-26T08:52:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.996+0000 7f077ffff640 1 -- 172.21.15.7:0/1961442705 shutdown_connections 2024-03-26T08:52:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.996+0000 7f077ffff640 1 --2- 172.21.15.7:0/1961442705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 0x7f077809d0b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.996+0000 7f077ffff640 1 --2- 172.21.15.7:0/1961442705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778095e60 0x7f07780962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.996+0000 7f077ffff640 1 --2- 172.21.15.7:0/1961442705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0778094c70 0x7f0778095090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.996+0000 7f077ffff640 1 -- 172.21.15.7:0/1961442705 >> 172.21.15.7:0/1961442705 conn(0x7f07780901c0 msgr2=0x7f0778092620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.997+0000 7f077ffff640 1 -- 172.21.15.7:0/1961442705 shutdown_connections 2024-03-26T08:52:40.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.997+0000 7f077ffff640 1 -- 172.21.15.7:0/1961442705 wait complete. 2024-03-26T08:52:40.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.997+0000 7f077ffff640 1 Processor -- start 2024-03-26T08:52:40.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077ffff640 1 -- start start 2024-03-26T08:52:40.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0778094c70 0x7f07780052a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778095e60 0x7f0778143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 0x7f0778146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077ffff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780a0800 con 0x7f0778094c70 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077ffff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780a0680 con 0x7f0778096820 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077ffff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07780a0980 con 0x7f0778095e60 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0778094c70 0x7f07780052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778095e60 0x7f0778143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 0x7f0778146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 0x7f0778146500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53932/0 (socket says 172.21.15.7:53932) 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0778094c70 0x7f07780052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57904/0 (socket says 172.21.15.7:57904) 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778095e60 0x7f0778143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60100/0 (socket says 172.21.15.7:60100) 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077f7fe640 1 -- 172.21.15.7:0/3890824929 learned_addr learned my addr 172.21.15.7:0/3890824929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077f7fe640 1 -- 172.21.15.7:0/3890824929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778095e60 msgr2=0x7f0778143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077f7fe640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778095e60 0x7f0778143fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077f7fe640 1 -- 172.21.15.7:0/3890824929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0778094c70 msgr2=0x7f07780052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077f7fe640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0778094c70 0x7f07780052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.998+0000 7f077f7fe640 1 -- 172.21.15.7:0/3890824929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778146cf0 con 0x7f0778096820 2024-03-26T08:52:40.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.999+0000 7f077f7fe640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 0x7f0778146500 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f07800501b0 tx=0x7f078009e630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:40.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.999+0000 7f0767fff640 1 -- 172.21.15.7:0/3890824929 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07800a3070 con 0x7f0778096820 2024-03-26T08:52:40.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.999+0000 7f0767fff640 1 -- 172.21.15.7:0/3890824929 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0780073710 con 0x7f0778096820 2024-03-26T08:52:40.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.999+0000 7f0767fff640 1 -- 172.21.15.7:0/3890824929 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078006e070 con 0x7f0778096820 2024-03-26T08:52:40.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.999+0000 7f077ffff640 1 -- 172.21.15.7:0/3890824929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0778146f20 con 0x7f0778096820 2024-03-26T08:52:40.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:40.999+0000 7f077ffff640 1 -- 172.21.15.7:0/3890824929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0778147330 con 0x7f0778096820 2024-03-26T08:52:40.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.001+0000 7f0767fff640 1 -- 172.21.15.7:0/3890824929 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0780073230 con 0x7f0778096820 2024-03-26T08:52:40.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0767fff640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07680a2860 0x7f07680a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:40.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0767fff640 1 -- 172.21.15.7:0/3890824929 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0780153920 con 0x7f0778096820 2024-03-26T08:52:40.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0787fd3740 1 Processor -- start 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f077effd640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07680a2860 0x7f07680a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0787fd3740 1 -- start start 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06eaa0 0x559d2f0735a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f073c10 0x559d2f0760a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0787fd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2f028610 con 0x559d2f06e0b0 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0787fd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2efcb3c0 con 0x559d2f06eaa0 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.002+0000 7f0787fd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2ef73d60 con 0x559d2f073c10 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06eaa0 0x559d2f0735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f073c10 0x559d2f0760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06eaa0 0x559d2f0735a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53940/0 (socket says 172.21.15.7:53940) 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077e7fc640 1 -- 172.21.15.7:0/2648654614 learned_addr learned my addr 172.21.15.7:0/2648654614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077effd640 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077effd640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07680a2860 0x7f07680a4d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0778095cc0 tx=0x7f0770011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077f7fe640 1 -- 172.21.15.7:0/2648654614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06eaa0 msgr2=0x559d2f0735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077f7fe640 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06eaa0 0x559d2f0735a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077f7fe640 1 -- 172.21.15.7:0/2648654614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 msgr2=0x559d2f06e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077f7fe640 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f06e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077f7fe640 1 -- 172.21.15.7:0/2648654614 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d2ecdf3f0 con 0x559d2f073c10 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077e7fc640 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06eaa0 0x559d2f0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077effd640 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f06e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f077f7fe640 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f073c10 0x559d2f0760a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0780151420 tx=0x7f078009dad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec835029902acd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f07657fa640 1 -- 172.21.15.7:0/2648654614 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07800ac070 con 0x559d2f073c10 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f07657fa640 1 -- 172.21.15.7:0/2648654614 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0780124410 con 0x559d2f073c10 2024-03-26T08:52:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.003+0000 7f07657fa640 1 -- 172.21.15.7:0/2648654614 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078011e670 con 0x559d2f073c10 2024-03-26T08:52:41.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.004+0000 7f0787fd3740 1 -- 172.21.15.7:0/2648654614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f073c10 msgr2=0x559d2f0760a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.004+0000 7f0787fd3740 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f073c10 0x559d2f0760a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0780151420 tx=0x7f078009dad0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.004+0000 7f0787fd3740 1 -- 172.21.15.7:0/2648654614 shutdown_connections 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.004+0000 7f0787fd3740 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f073c10 0x559d2f0760a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.004+0000 7f0787fd3740 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06eaa0 0x559d2f0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.004+0000 7f0787fd3740 1 --2- 172.21.15.7:0/2648654614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f06e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.004+0000 7f0787fd3740 1 -- 172.21.15.7:0/2648654614 >> 172.21.15.7:0/2648654614 conn(0x559d2f069620 msgr2=0x559d2f069a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.004+0000 7f0787fd3740 1 -- 172.21.15.7:0/2648654614 shutdown_connections 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.004+0000 7f0787fd3740 1 -- 172.21.15.7:0/2648654614 wait complete. 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f0787fd3740 1 -- start start 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f0787fd3740 1 Processor -- start 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f078120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 0x559d2f078660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 0x559d2f07f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f0787fd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2ef87240 con 0x559d2f06e0b0 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f0787fd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2efc7460 con 0x559d2f073c10 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f0787fd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d2f076980 con 0x559d2f06eaa0 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f0787fd3740 10 client.0 _ll_register_callbacks cb 0x559d2ef86c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f077effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f078120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f077effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f078120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57926/0 (socket says 172.21.15.7:57926) 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f077effd640 1 -- 172.21.15.7:0/3137901892 learned_addr learned my addr 172.21.15.7:0/3137901892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f077f7fe640 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 0x559d2f07f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.005+0000 7f077e7fc640 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 0x559d2f078660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.006+0000 7f077effd640 1 -- 172.21.15.7:0/3137901892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 msgr2=0x559d2f078660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.006+0000 7f077effd640 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 0x559d2f078660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.006+0000 7f077effd640 1 -- 172.21.15.7:0/3137901892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 msgr2=0x559d2f07f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.006+0000 7f077effd640 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 0x559d2f07f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.006+0000 7f077effd640 1 -- 172.21.15.7:0/3137901892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d2eda97d0 con 0x559d2f06e0b0 2024-03-26T08:52:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.006+0000 7f077f7fe640 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 0x559d2f07f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.007+0000 7f077effd640 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f078120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f077001e990 tx=0x7f0770004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9c1e17bc89692d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.007+0000 7f0762ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.007+0000 7f0762ff5640 1 -- 172.21.15.7:0/3137901892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770017040 con 0x559d2f06e0b0 2024-03-26T08:52:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.007+0000 7f0762ff5640 1 -- 172.21.15.7:0/3137901892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f077001e020 con 0x559d2f06e0b0 2024-03-26T08:52:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.007+0000 7f0787fd3740 1 -- 172.21.15.7:0/3137901892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0780124410 con 0x559d2f06e0b0 2024-03-26T08:52:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.007+0000 7f0762ff5640 1 -- 172.21.15.7:0/3137901892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f077001e220 con 0x559d2f06e0b0 2024-03-26T08:52:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.007+0000 7f0787fd3740 10 client.18126 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.007+0000 7f0787fd3740 1 -- 172.21.15.7:0/3137901892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559d2f082b00 con 0x559d2f06e0b0 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.008+0000 7f0762ff5640 1 -- 172.21.15.7:0/3137901892 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0770019050 con 0x559d2f06e0b0 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.008+0000 7f0762ff5640 1 -- 172.21.15.7:0/3137901892 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0770025070 con 0x559d2f06e0b0 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.008+0000 7f0787fd3740 10 client.18126 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.008+0000 7f0787fd3740 10 client.18126 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.008+0000 7f0787fd3740 20 client.18126 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.008+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x559d2f0841d0 0x559d2f0865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.008+0000 7f0787fd3740 4 client.18126 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.008+0000 7f0787fd3740 1 -- 172.21.15.7:0/3137901892 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0780124410 con 0x559d2f0841d0 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.009+0000 7f077e7fc640 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x559d2f0841d0 0x559d2f0865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.009+0000 7f077e7fc640 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x559d2f0841d0 0x559d2f0865f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a247a7f193a9d3f5 server_cookie=3434d4a6a017647a in_seq=0 out_seq=0 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.009+0000 7f0762ff5640 10 client.18126 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:52:41.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0762ff5640 1 -- 172.21.15.7:0/3137901892 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f0780124410 con 0x559d2f0841d0 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0762ff5640 10 client.18126 handle_command_reply: tid=0 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 client.18126 shutdown 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 -- 172.21.15.7:0/3137901892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 msgr2=0x559d2f078120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f078120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f077001e990 tx=0x7f0770004730 comp rx=0 tx=0).stop 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 -- 172.21.15.7:0/3137901892 shutdown_connections 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 0x559d2f07f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x559d2f0841d0 0x559d2f0865f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 0x559d2f078660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0762ff5640 0 client.18126 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3137901892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06e0b0 0x559d2f078120 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 -- 172.21.15.7:0/3137901892 >> 172.21.15.7:0/3137901892 conn(0x559d2f069620 msgr2=0x559d2f060070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 -- 172.21.15.7:0/3137901892 shutdown_connections 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.010+0000 7f0787fd3740 1 -- 172.21.15.7:0/3137901892 wait complete. 2024-03-26T08:52:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.011+0000 7f0787fd3740 20 client.18126 trim_cache size 0 max 16384 2024-03-26T08:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.027+0000 7f0787fd3740 1 Processor -- start 2024-03-26T08:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f0787fd3740 1 -- start start 2024-03-26T08:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06e0b0 0x559d2f077740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 0x559d2f07f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f073c10 0x559d2f0780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f0787fd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077814e2a0 con 0x559d2f073c10 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f0787fd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077814bda0 con 0x559d2f06e0b0 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f0787fd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0778149e20 con 0x559d2f06eaa0 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06e0b0 0x559d2f077740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06e0b0 0x559d2f077740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53966/0 (socket says 172.21.15.7:53966) 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 0x559d2f07f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077effd640 1 -- 172.21.15.7:0/3933818755 learned_addr learned my addr 172.21.15.7:0/3933818755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077f7fe640 1 --2- 172.21.15.7:0/3933818755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f073c10 0x559d2f0780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077effd640 1 -- 172.21.15.7:0/3933818755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 msgr2=0x559d2f07f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077effd640 1 --2- 172.21.15.7:0/3933818755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 0x559d2f07f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077effd640 1 -- 172.21.15.7:0/3933818755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f073c10 msgr2=0x559d2f0780e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077effd640 1 --2- 172.21.15.7:0/3933818755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f073c10 0x559d2f0780e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077effd640 1 -- 172.21.15.7:0/3933818755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d2f082920 con 0x559d2f06e0b0 2024-03-26T08:52:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.028+0000 7f077effd640 1 --2- 172.21.15.7:0/3933818755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06e0b0 0x559d2f077740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0770006000 tx=0x7f077001e7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34fa957c0aee637c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.029+0000 7f0764ff9640 1 -- 172.21.15.7:0/3933818755 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0770017030 con 0x559d2f06e0b0 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.029+0000 7f0764ff9640 1 -- 172.21.15.7:0/3933818755 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f077001e020 con 0x559d2f06e0b0 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.029+0000 7f0764ff9640 1 -- 172.21.15.7:0/3933818755 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077001d490 con 0x559d2f06e0b0 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 -- 172.21.15.7:0/3933818755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06e0b0 msgr2=0x559d2f077740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3933818755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06e0b0 0x559d2f077740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0770006000 tx=0x7f077001e7a0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 -- 172.21.15.7:0/3933818755 shutdown_connections 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3933818755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f073c10 0x559d2f0780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3933818755 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06eaa0 0x559d2f07f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 --2- 172.21.15.7:0/3933818755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f06e0b0 0x559d2f077740 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 -- 172.21.15.7:0/3933818755 >> 172.21.15.7:0/3933818755 conn(0x559d2f069620 msgr2=0x559d2f06f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 -- 172.21.15.7:0/3933818755 shutdown_connections 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 -- 172.21.15.7:0/3933818755 wait complete. 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 -- start start 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 Processor -- start 2024-03-26T08:52:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.030+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06e0b0 0x559d2f07cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06eaa0 0x559d2f07d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f0787fd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 0x559d2f07e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f0787fd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077814a420 con 0x559d2f06eaa0 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f0787fd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077814b740 con 0x559d2f073c10 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f0787fd3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077814aa80 con 0x559d2f06e0b0 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f0787fd3740 10 client.0 _ll_register_callbacks cb 0x7f077001e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077effd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06e0b0 0x559d2f07cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06eaa0 0x559d2f07d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077e7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06eaa0 0x559d2f07d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57942/0 (socket says 172.21.15.7:57942) 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077e7fc640 1 -- 172.21.15.7:0/1110208101 learned_addr learned my addr 172.21.15.7:0/1110208101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077f7fe640 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 0x559d2f07e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077e7fc640 1 -- 172.21.15.7:0/1110208101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06e0b0 msgr2=0x559d2f07cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077e7fc640 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06e0b0 0x559d2f07cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077e7fc640 1 -- 172.21.15.7:0/1110208101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 msgr2=0x559d2f07e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077e7fc640 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 0x559d2f07e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077e7fc640 1 -- 172.21.15.7:0/1110208101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d2f07eaf0 con 0x559d2f06eaa0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f077e7fc640 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06eaa0 0x559d2f07d4f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0774009d70 tx=0x7f0774020b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dbfc8723d3e259 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.031+0000 7f0762ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0762ff5640 1 -- 172.21.15.7:0/1110208101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774011ca0 con 0x559d2f06eaa0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0787fd3740 1 -- 172.21.15.7:0/1110208101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f077001e020 con 0x559d2f06eaa0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0762ff5640 1 -- 172.21.15.7:0/1110208101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0774004020 con 0x559d2f06eaa0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0762ff5640 1 -- 172.21.15.7:0/1110208101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0774011e40 con 0x559d2f06eaa0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0787fd3740 10 client.18132 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0787fd3740 1 -- 172.21.15.7:0/1110208101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559d2f0dfe00 con 0x559d2f06eaa0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0762ff5640 1 -- 172.21.15.7:0/1110208101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f077001e020 con 0x559d2f06eaa0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0762ff5640 1 -- 172.21.15.7:0/1110208101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f07740047a0 con 0x559d2f06eaa0 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0787fd3740 10 client.18132 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0787fd3740 10 client.18132 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0787fd3740 20 client.18132 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0787fd3740 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x559d2f0841d0 0x559d2f0e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0787fd3740 4 client.18132 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f0787fd3740 1 -- 172.21.15.7:0/1110208101 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d2efd8900 con 0x559d2f0841d0 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.032+0000 7f077effd640 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x559d2f0841d0 0x559d2f0e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f077effd640 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x559d2f0841d0 0x559d2f0e2870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7aa9dfe4e2aead43 server_cookie=bad7d57b74a4b50b in_seq=0 out_seq=0 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0762ff5640 10 client.18132 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0762ff5640 1 -- 172.21.15.7:0/1110208101 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d2efd8900 con 0x559d2f0841d0 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0762ff5640 10 client.18132 handle_command_reply: tid=0 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 client.18132 shutdown 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 -- 172.21.15.7:0/1110208101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06eaa0 msgr2=0x559d2f07d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06eaa0 0x559d2f07d4f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0774009d70 tx=0x7f0774020b60 comp rx=0 tx=0).stop 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 -- 172.21.15.7:0/1110208101 shutdown_connections 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559d2f073c10 0x559d2f07e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559d2f06eaa0 0x559d2f07d4f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x559d2f0841d0 0x559d2f0e2870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 --2- 172.21.15.7:0/1110208101 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559d2f06e0b0 0x559d2f07cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 -- 172.21.15.7:0/1110208101 >> 172.21.15.7:0/1110208101 conn(0x559d2f069620 msgr2=0x559d2f0780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0762ff5640 0 client.18132 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:52:41.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.033+0000 7f0787fd3740 1 -- 172.21.15.7:0/1110208101 shutdown_connections 2024-03-26T08:52:41.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.034+0000 7f0787fd3740 1 -- 172.21.15.7:0/1110208101 wait complete. 2024-03-26T08:52:41.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.034+0000 7f0787fd3740 20 client.18132 trim_cache size 0 max 16384 2024-03-26T08:52:41.035 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:52:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.036+0000 7f077ffff640 1 -- 172.21.15.7:0/3890824929 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07680a2860 msgr2=0x7f07680a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.036+0000 7f077ffff640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07680a2860 0x7f07680a4d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0778095cc0 tx=0x7f0770011440 comp rx=0 tx=0).stop 2024-03-26T08:52:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.036+0000 7f077ffff640 1 -- 172.21.15.7:0/3890824929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 msgr2=0x7f0778146500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.036+0000 7f077ffff640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 0x7f0778146500 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f07800501b0 tx=0x7f078009e630 comp rx=0 tx=0).stop 2024-03-26T08:52:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.037+0000 7f077ffff640 1 -- 172.21.15.7:0/3890824929 shutdown_connections 2024-03-26T08:52:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.037+0000 7f077ffff640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f07680a2860 0x7f07680a4d20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.037+0000 7f077ffff640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0778096820 0x7f0778146500 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.037+0000 7f077ffff640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0778095e60 0x7f0778143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.037+0000 7f077ffff640 1 --2- 172.21.15.7:0/3890824929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0778094c70 0x7f07780052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.037+0000 7f077ffff640 1 -- 172.21.15.7:0/3890824929 >> 172.21.15.7:0/3890824929 conn(0x7f07780901c0 msgr2=0x7f0778099090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.038+0000 7f077ffff640 1 -- 172.21.15.7:0/3890824929 shutdown_connections 2024-03-26T08:52:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.038+0000 7f077ffff640 1 -- 172.21.15.7:0/3890824929 wait complete. 2024-03-26T08:52:41.048 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:52:41.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:52:41.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 610.0 s, 2091.0 tps, lat 47.835 ms stddev 52.534 2024-03-26T08:52:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.210+0000 7f09d4068640 1 -- 172.21.15.7:0/1875086134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09cc0fff30 msgr2=0x7f09cc102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.210+0000 7f09d4068640 1 --2- 172.21.15.7:0/1875086134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09cc0fff30 0x7f09cc102360 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009be0 tx=0x7f09c402bef0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.210+0000 7f09d4068640 1 -- 172.21.15.7:0/1875086134 shutdown_connections 2024-03-26T08:52:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.210+0000 7f09d4068640 1 --2- 172.21.15.7:0/1875086134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc105250 0x7f09cc1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.210+0000 7f09d4068640 1 --2- 172.21.15.7:0/1875086134 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09cc1028a0 0x7f09cc104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.210+0000 7f09d4068640 1 --2- 172.21.15.7:0/1875086134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09cc0fff30 0x7f09cc102360 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.210+0000 7f09d4068640 1 -- 172.21.15.7:0/1875086134 >> 172.21.15.7:0/1875086134 conn(0x7f09cc0fbb30 msgr2=0x7f09cc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.210+0000 7f09d4068640 1 -- 172.21.15.7:0/1875086134 shutdown_connections 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.210+0000 7f09d4068640 1 -- 172.21.15.7:0/1875086134 wait complete. 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d4068640 1 Processor -- start 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d4068640 1 -- start start 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d4068640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc0fff30 0x7f09cc19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d4068640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09cc1028a0 0x7f09cc19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d4068640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09cc105250 0x7f09cc1a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d4068640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc113910 con 0x7f09cc105250 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d4068640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc113790 con 0x7f09cc0fff30 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d4068640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc113a90 con 0x7f09cc1028a0 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d1e04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc0fff30 0x7f09cc19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.211+0000 7f09d1e04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc0fff30 0x7f09cc19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54000/0 (socket says 172.21.15.7:54000) 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d1e04640 1 -- 172.21.15.7:0/242345050 learned_addr learned my addr 172.21.15.7:0/242345050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d1603640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09cc1028a0 0x7f09cc19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d2605640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09cc105250 0x7f09cc1a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d1e04640 1 -- 172.21.15.7:0/242345050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09cc1028a0 msgr2=0x7f09cc19e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d1e04640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09cc1028a0 0x7f09cc19e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d1e04640 1 -- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09cc105250 msgr2=0x7f09cc1a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d1e04640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09cc105250 0x7f09cc1a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d1e04640 1 -- 172.21.15.7:0/242345050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc1a5bd0 con 0x7f09cc0fff30 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d1603640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09cc1028a0 0x7f09cc19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d1e04640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc0fff30 0x7f09cc19dfb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f09c4006010 tx=0x7f09c4033830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09beffd640 1 -- 172.21.15.7:0/242345050 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4043020 con 0x7f09cc0fff30 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09beffd640 1 -- 172.21.15.7:0/242345050 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09c402ca60 con 0x7f09cc0fff30 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.212+0000 7f09d4068640 1 -- 172.21.15.7:0/242345050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc1a5e60 con 0x7f09cc0fff30 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.213+0000 7f09beffd640 1 -- 172.21.15.7:0/242345050 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4034690 con 0x7f09cc0fff30 2024-03-26T08:52:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.213+0000 7f09d4068640 1 -- 172.21.15.7:0/242345050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc1a6320 con 0x7f09cc0fff30 2024-03-26T08:52:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.215+0000 7f09beffd640 1 -- 172.21.15.7:0/242345050 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f09c4046020 con 0x7f09cc0fff30 2024-03-26T08:52:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.216+0000 7f09beffd640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09b80a2880 0x7f09b80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.216+0000 7f09d1603640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09b80a2880 0x7f09b80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.216+0000 7f09beffd640 1 -- 172.21.15.7:0/242345050 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f09c40f1130 con 0x7f09cc0fff30 2024-03-26T08:52:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.216+0000 7f09d6c4f740 1 Processor -- start 2024-03-26T08:52:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.217+0000 7f09d1603640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09b80a2880 0x7f09b80a4d40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f09cc19f4c0 tx=0x7f09c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.217+0000 7f09d6c4f740 1 -- start start 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.217+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 0x559eaecc1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.217+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc2080 0x559eaecc6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.218+0000 7f09d1603640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc2080 0x559eaecc6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.218+0000 7f09d1603640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc2080 0x559eaecc6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60172/0 (socket says 172.21.15.7:60172) 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.218+0000 7f09d1603640 1 -- 172.21.15.7:0/3341242044 learned_addr learned my addr 172.21.15.7:0/3341242044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.218+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3341242044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 0x559eaecc9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.218+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3341242044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eaec84610 con 0x559eaecc1690 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.218+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3341242044 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eaec273c0 con 0x559eaecc71f0 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.218+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3341242044 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eaebcfd60 con 0x559eaecc2080 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.218+0000 7f09d2605640 1 --2- 172.21.15.7:0/3341242044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 0x559eaecc9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.219+0000 7f09d1603640 1 -- 172.21.15.7:0/3341242044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 msgr2=0x559eaecc9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.219+0000 7f09d1603640 1 --2- 172.21.15.7:0/3341242044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 0x559eaecc9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.219+0000 7f09d1603640 1 -- 172.21.15.7:0/3341242044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 msgr2=0x559eaecc1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.219+0000 7f09d1603640 1 --2- 172.21.15.7:0/3341242044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 0x559eaecc1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.219+0000 7f09d1603640 1 -- 172.21.15.7:0/3341242044 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eae93b3f0 con 0x559eaecc2080 2024-03-26T08:52:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.219+0000 7f09d1603640 1 --2- 172.21.15.7:0/3341242044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc2080 0x559eaecc6b80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009ba0 tx=0x7f09c0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57449db4568a53ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.219+0000 7f09b7fff640 1 -- 172.21.15.7:0/3341242044 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000f480 con 0x559eaecc2080 2024-03-26T08:52:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.219+0000 7f09b7fff640 1 -- 172.21.15.7:0/3341242044 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09c000fb00 con 0x559eaecc2080 2024-03-26T08:52:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.220+0000 7f09b7fff640 1 -- 172.21.15.7:0/3341242044 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c001cbb0 con 0x559eaecc2080 2024-03-26T08:52:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.220+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3341242044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc2080 msgr2=0x559eaecc6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.220+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3341242044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc2080 0x559eaecc6b80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009ba0 tx=0x7f09c0014000 comp rx=0 tx=0).stop 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.220+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3341242044 shutdown_connections 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.220+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3341242044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 0x559eaecc9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.220+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3341242044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc2080 0x559eaecc6b80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.220+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3341242044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 0x559eaecc1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.220+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3341242044 >> 172.21.15.7:0/3341242044 conn(0x559eaecbcc00 msgr2=0x559eaecbd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.221+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3341242044 shutdown_connections 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.221+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3341242044 wait complete. 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.221+0000 7f09d6c4f740 1 -- start start 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.221+0000 7f09d6c4f740 1 Processor -- start 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 0x559eaeccb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc2080 0x559eaeccbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 0x559eaecd2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d6c4f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eaebe3240 con 0x559eaecc1690 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d1e04640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 0x559eaeccb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d1603640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc2080 0x559eaeccbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d1e04640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 0x559eaeccb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57960/0 (socket says 172.21.15.7:57960) 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d1e04640 1 -- 172.21.15.7:0/3456883628 learned_addr learned my addr 172.21.15.7:0/3456883628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d2605640 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 0x559eaecd2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eaec23460 con 0x559eaecc2080 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eaecca1f0 con 0x559eaecc71f0 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.222+0000 7f09d6c4f740 10 client.0 _ll_register_callbacks cb 0x559eaebe2c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09d1e04640 1 -- 172.21.15.7:0/3456883628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 msgr2=0x559eaecd2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09d1e04640 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 0x559eaecd2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09d1e04640 1 -- 172.21.15.7:0/3456883628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc2080 msgr2=0x559eaeccbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09d1e04640 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc2080 0x559eaeccbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09d1e04640 1 -- 172.21.15.7:0/3456883628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559eaea057d0 con 0x559eaecc1690 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09d2605640 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 0x559eaecd2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09d1603640 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc2080 0x559eaeccbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09d1e04640 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 0x559eaeccb930 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f09c4000f80 tx=0x7f09c40bd690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bdbd119899e597 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09b57fa640 1 -- 172.21.15.7:0/3456883628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c40bda90 con 0x559eaecc1690 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09b57fa640 1 -- 172.21.15.7:0/3456883628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f09c40c5da0 con 0x559eaecc1690 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.223+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09c000fb00 con 0x559eaecc1690 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.224+0000 7f09b57fa640 1 -- 172.21.15.7:0/3456883628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c40c1be0 con 0x559eaecc1690 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.224+0000 7f09d6c4f740 10 client.18141 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.224+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559eaecd6200 con 0x559eaecc1690 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.224+0000 7f09b57fa640 1 -- 172.21.15.7:0/3456883628 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09c000fb00 con 0x559eaecc1690 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.224+0000 7f09b57fa640 1 -- 172.21.15.7:0/3456883628 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f09c4004b40 con 0x559eaecc1690 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.224+0000 7f09d6c4f740 10 client.18141 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.224+0000 7f09d6c4f740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.224+0000 7f09d6c4f740 20 client.18141 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.225+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559eaecd78b0 0x559eaecd9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.225+0000 7f09d6c4f740 4 client.18141 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.225+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559eaec34900 con 0x559eaecd78b0 2024-03-26T08:52:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.225+0000 7f09d1603640 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559eaecd78b0 0x559eaecd9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.225+0000 7f09b57fa640 10 client.18141 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.225+0000 7f09d1603640 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559eaecd78b0 0x559eaecd9cc0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1ea719645fc103 server_cookie=4762f91248118506 in_seq=0 out_seq=0 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09b57fa640 1 -- 172.21.15.7:0/3456883628 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559eaec34900 con 0x559eaecd78b0 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09b57fa640 10 client.18141 handle_command_reply: tid=0 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 client.18141 shutdown 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 msgr2=0x559eaeccb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 0x559eaeccb930 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f09c4000f80 tx=0x7f09c40bd690 comp rx=0 tx=0).stop 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 shutdown_connections 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559eaecd78b0 0x559eaecd9cc0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 0x559eaecd2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc2080 0x559eaeccbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3456883628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc1690 0x559eaeccb930 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 >> 172.21.15.7:0/3456883628 conn(0x559eaecbcc00 msgr2=0x559eaecbbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09b57fa640 0 client.18141 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:41.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 shutdown_connections 2024-03-26T08:52:41.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.226+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3456883628 wait complete. 2024-03-26T08:52:41.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.227+0000 7f09d6c4f740 20 client.18141 trim_cache size 0 max 16384 2024-03-26T08:52:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.246+0000 7f09d6c4f740 1 Processor -- start 2024-03-26T08:52:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.246+0000 7f09d6c4f740 1 -- start start 2024-03-26T08:52:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc1690 0x559eaecc0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 0x559eaecd2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 0x559eaeccb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d6c4f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1b15c0 con 0x559eaecc2080 2024-03-26T08:52:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d6c4f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1af0c0 con 0x559eaecc1690 2024-03-26T08:52:41.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d6c4f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1ad140 con 0x559eaecc71f0 2024-03-26T08:52:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d1e04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc1690 0x559eaecc0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d1e04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc1690 0x559eaecc0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54034/0 (socket says 172.21.15.7:54034) 2024-03-26T08:52:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d1e04640 1 -- 172.21.15.7:0/3617387369 learned_addr learned my addr 172.21.15.7:0/3617387369 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d1603640 1 --2- 172.21.15.7:0/3617387369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 0x559eaecd2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d1e04640 1 -- 172.21.15.7:0/3617387369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 msgr2=0x559eaeccb540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:52:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d1e04640 1 --2- 172.21.15.7:0/3617387369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 0x559eaeccb540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d1e04640 1 -- 172.21.15.7:0/3617387369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 msgr2=0x559eaecd2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d1e04640 1 --2- 172.21.15.7:0/3617387369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 0x559eaecd2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.247+0000 7f09d1e04640 1 -- 172.21.15.7:0/3617387369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eaecc12d0 con 0x559eaecc1690 2024-03-26T08:52:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.248+0000 7f09d1e04640 1 --2- 172.21.15.7:0/3617387369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc1690 0x559eaecc0570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005ee0 tx=0x7f09c40bd700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1858da8881cec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.248+0000 7f09b77fe640 1 -- 172.21.15.7:0/3617387369 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c40bda90 con 0x559eaecc1690 2024-03-26T08:52:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.248+0000 7f09b77fe640 1 -- 172.21.15.7:0/3617387369 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09c40c6230 con 0x559eaecc1690 2024-03-26T08:52:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.248+0000 7f09b77fe640 1 -- 172.21.15.7:0/3617387369 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c40d6b50 con 0x559eaecc1690 2024-03-26T08:52:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3617387369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc1690 msgr2=0x559eaecc0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3617387369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc1690 0x559eaecc0570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005ee0 tx=0x7f09c40bd700 comp rx=0 tx=0).stop 2024-03-26T08:52:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3617387369 shutdown_connections 2024-03-26T08:52:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3617387369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc71f0 0x559eaeccb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3617387369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 0x559eaecd2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/3617387369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc1690 0x559eaecc0570 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3617387369 >> 172.21.15.7:0/3617387369 conn(0x559eaecbcc00 msgr2=0x559eaecbfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3617387369 shutdown_connections 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 -- 172.21.15.7:0/3617387369 wait complete. 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 -- start start 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.249+0000 7f09d6c4f740 1 Processor -- start 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc1690 0x559eaecd0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 0x559eaecd0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d6c4f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 0x559eaecd1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d6c4f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1ad740 con 0x559eaecc2080 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d6c4f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1aea60 con 0x559eaecc71f0 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d6c4f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1adda0 con 0x559eaecc1690 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d2605640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 0x559eaecd1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d2605640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 0x559eaecd1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54050/0 (socket says 172.21.15.7:54050) 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d2605640 1 -- 172.21.15.7:0/2154545754 learned_addr learned my addr 172.21.15.7:0/2154545754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d1e04640 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc1690 0x559eaecd0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d6c4f740 10 client.0 _ll_register_callbacks cb 0x7f09c40bd860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d1603640 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 0x559eaecd0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d1e04640 1 -- 172.21.15.7:0/2154545754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 msgr2=0x559eaecd1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d1e04640 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 0x559eaecd1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d1e04640 1 -- 172.21.15.7:0/2154545754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 msgr2=0x559eaecd0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d1e04640 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 0x559eaecd0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d1e04640 1 -- 172.21.15.7:0/2154545754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559eaecd2460 con 0x559eaecc1690 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.250+0000 7f09d1603640 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 0x559eaecd0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.251+0000 7f09d1e04640 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc1690 0x559eaecd0920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x559eaecc2790 tx=0x7f09c402bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7fdc2ddea7035 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.251+0000 7f09b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.251+0000 7f09b57fa640 1 -- 172.21.15.7:0/2154545754 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c40c5fa0 con 0x559eaecc1690 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.251+0000 7f09b57fa640 1 -- 172.21.15.7:0/2154545754 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f09c40d6b50 con 0x559eaecc1690 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.251+0000 7f09d6c4f740 1 -- 172.21.15.7:0/2154545754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09c40c6230 con 0x559eaecc1690 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.251+0000 7f09b57fa640 1 -- 172.21.15.7:0/2154545754 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c40bda90 con 0x559eaecc1690 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.251+0000 7f09d6c4f740 10 client.27970 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.251+0000 7f09d6c4f740 1 -- 172.21.15.7:0/2154545754 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559eaecd2460 con 0x559eaecc1690 2024-03-26T08:52:41.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.251+0000 7f09b57fa640 1 -- 172.21.15.7:0/2154545754 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09c40c6230 con 0x559eaecc1690 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09b57fa640 1 -- 172.21.15.7:0/2154545754 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f09c40cf050 con 0x559eaecc1690 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09d6c4f740 10 client.27970 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09d6c4f740 10 client.27970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09d6c4f740 20 client.27970 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559eaed33990 0x559eaed35d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09d6c4f740 4 client.27970 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09d6c4f740 1 -- 172.21.15.7:0/2154545754 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559eaec2ec50 con 0x559eaed33990 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09d1603640 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559eaed33990 0x559eaed35d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09b57fa640 10 client.27970 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.252+0000 7f09d1603640 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559eaed33990 0x559eaed35d80 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e380b7522107af33 server_cookie=857b3ca525aa4b28 in_seq=0 out_seq=0 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09b57fa640 1 -- 172.21.15.7:0/2154545754 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559eaec2ec50 con 0x559eaed33990 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09b57fa640 10 client.27970 handle_command_reply: tid=0 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 client.27970 shutdown 2024-03-26T08:52:41.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 -- 172.21.15.7:0/2154545754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc1690 msgr2=0x559eaecd0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc1690 0x559eaecd0920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x559eaecc2790 tx=0x7f09c402bef0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 -- 172.21.15.7:0/2154545754 shutdown_connections 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559eaed33990 0x559eaed35d80 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559eaecc71f0 0x559eaecd1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09b57fa640 0 client.27970 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559eaecc2080 0x559eaecd0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 --2- 172.21.15.7:0/2154545754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559eaecc1690 0x559eaecd0920 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 -- 172.21.15.7:0/2154545754 >> 172.21.15.7:0/2154545754 conn(0x559eaecbcc00 msgr2=0x559eaeccb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 -- 172.21.15.7:0/2154545754 shutdown_connections 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 1 -- 172.21.15.7:0/2154545754 wait complete. 2024-03-26T08:52:41.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.253+0000 7f09d6c4f740 20 client.27970 trim_cache size 0 max 16384 2024-03-26T08:52:41.255 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:52:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 -- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09b80a2880 msgr2=0x7f09b80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09b80a2880 0x7f09b80a4d40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f09cc19f4c0 tx=0x7f09c0005f90 comp rx=0 tx=0).stop 2024-03-26T08:52:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 -- 172.21.15.7:0/242345050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc0fff30 msgr2=0x7f09cc19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:41.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc0fff30 0x7f09cc19dfb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f09c4006010 tx=0x7f09c4033830 comp rx=0 tx=0).stop 2024-03-26T08:52:41.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 -- 172.21.15.7:0/242345050 shutdown_connections 2024-03-26T08:52:41.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f09b80a2880 0x7f09b80a4d40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09cc105250 0x7f09cc1a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f09cc1028a0 0x7f09cc19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 --2- 172.21.15.7:0/242345050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09cc0fff30 0x7f09cc19dfb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:41.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 -- 172.21.15.7:0/242345050 >> 172.21.15.7:0/242345050 conn(0x7f09cc0fbb30 msgr2=0x7f09cc103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:41.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 -- 172.21.15.7:0/242345050 shutdown_connections 2024-03-26T08:52:41.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:41.254+0000 7f09d4068640 1 -- 172.21.15.7:0/242345050 wait complete. 2024-03-26T08:52:41.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:41 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4103567228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:52:41.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:41 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4103567228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:52:41.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:41 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4103567228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:52:42.266 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:52:42.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:52:42.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:52:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.436+0000 7f264b59e640 1 -- 172.21.15.7:0/3170379357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c109680 msgr2=0x7f264c10df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.436+0000 7f264b59e640 1 --2- 172.21.15.7:0/3170379357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c109680 0x7f264c10df20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2644009ca0 tx=0x7f264402c120 comp rx=0 tx=0).stop 2024-03-26T08:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.437+0000 7f264b59e640 1 -- 172.21.15.7:0/3170379357 shutdown_connections 2024-03-26T08:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.437+0000 7f264b59e640 1 --2- 172.21.15.7:0/3170379357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c109680 0x7f264c10df20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.437+0000 7f264b59e640 1 --2- 172.21.15.7:0/3170379357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f264c108cc0 0x7f264c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.437+0000 7f264b59e640 1 --2- 172.21.15.7:0/3170379357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c072a10 0x7f264c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.437+0000 7f264b59e640 1 -- 172.21.15.7:0/3170379357 >> 172.21.15.7:0/3170379357 conn(0x7f264c06e4e0 msgr2=0x7f264c070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.437+0000 7f264b59e640 1 -- 172.21.15.7:0/3170379357 shutdown_connections 2024-03-26T08:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.437+0000 7f264b59e640 1 -- 172.21.15.7:0/3170379357 wait complete. 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.438+0000 7f264b59e640 1 Processor -- start 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.438+0000 7f264b59e640 1 -- start start 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.438+0000 7f264b59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c072a10 0x7f264c119930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.438+0000 7f264b59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c108cc0 0x7f264c119e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.438+0000 7f264b59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f264c109680 0x7f264c117f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.438+0000 7f264b59e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c110820 con 0x7f264c108cc0 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.438+0000 7f264b59e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1106a0 con 0x7f264c072a10 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.438+0000 7f264b59e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1109a0 con 0x7f264c109680 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c072a10 0x7f264c119930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c072a10 0x7f264c119930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54056/0 (socket says 172.21.15.7:54056) 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264a59c640 1 -- 172.21.15.7:0/461298234 learned_addr learned my addr 172.21.15.7:0/461298234 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f2649d9b640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c108cc0 0x7f264c119e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264ad9d640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f264c109680 0x7f264c117f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264a59c640 1 -- 172.21.15.7:0/461298234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f264c109680 msgr2=0x7f264c117f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264a59c640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f264c109680 0x7f264c117f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264a59c640 1 -- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c108cc0 msgr2=0x7f264c119e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264a59c640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c108cc0 0x7f264c119e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264a59c640 1 -- 172.21.15.7:0/461298234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c118710 con 0x7f264c072a10 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.439+0000 7f264a59c640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c072a10 0x7f264c119930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d950 tx=0x7f263c00dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.440+0000 7f26437fe640 1 -- 172.21.15.7:0/461298234 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c017070 con 0x7f264c072a10 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.440+0000 7f26437fe640 1 -- 172.21.15.7:0/461298234 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f263c00de80 con 0x7f264c072a10 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.441+0000 7f26437fe640 1 -- 172.21.15.7:0/461298234 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c00f620 con 0x7f264c072a10 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.441+0000 7f264b59e640 1 -- 172.21.15.7:0/461298234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f264c118a00 con 0x7f264c072a10 2024-03-26T08:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.441+0000 7f264b59e640 1 -- 172.21.15.7:0/461298234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264c1b7360 con 0x7f264c072a10 2024-03-26T08:52:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.444+0000 7f2653831740 1 Processor -- start 2024-03-26T08:52:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.444+0000 7f26437fe640 1 -- 172.21.15.7:0/461298234 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f263c009c30 con 0x7f264c072a10 2024-03-26T08:52:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.445+0000 7f2653831740 1 -- start start 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.445+0000 7f2653831740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca52fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.445+0000 7f26437fe640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26380a27a0 0x7f26380a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.445+0000 7f2653831740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5303020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2653831740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca5305b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f264ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca52fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f264ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca52fdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57998/0 (socket says 172.21.15.7:57998) 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f264ad9d640 1 -- 172.21.15.7:0/3582214235 learned_addr learned my addr 172.21.15.7:0/3582214235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2653831740 1 -- 172.21.15.7:0/3582214235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca523dc90 con 0x55bca52fdb30 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2653831740 1 -- 172.21.15.7:0/3582214235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca5263070 con 0x55bca52fe520 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2653831740 1 -- 172.21.15.7:0/3582214235 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca520bc80 con 0x55bca5303690 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2649d9b640 1 --2- 172.21.15.7:0/3582214235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5303020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f264ad9d640 1 --2- 172.21.15.7:0/3582214235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca5305b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2649d9b640 1 -- 172.21.15.7:0/3582214235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 msgr2=0x55bca5305b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2649d9b640 1 --2- 172.21.15.7:0/3582214235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca5305b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2649d9b640 1 -- 172.21.15.7:0/3582214235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 msgr2=0x55bca52fdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2649d9b640 1 --2- 172.21.15.7:0/3582214235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca52fdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f2649d9b640 1 -- 172.21.15.7:0/3582214235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bca5203d10 con 0x55bca52fe520 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.446+0000 7f264a59c640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26380a27a0 0x7f26380a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.447+0000 7f2649d9b640 1 --2- 172.21.15.7:0/3582214235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5303020 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f264c072730 tx=0x7f2634009e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7424eb5df71ee28d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.447+0000 7f2640ff9640 1 -- 172.21.15.7:0/3582214235 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634017070 con 0x55bca52fe520 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.447+0000 7f2640ff9640 1 -- 172.21.15.7:0/3582214235 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2634013470 con 0x55bca52fe520 2024-03-26T08:52:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.447+0000 7f26437fe640 1 -- 172.21.15.7:0/461298234 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f263c0ce6b0 con 0x7f264c072a10 2024-03-26T08:52:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.448+0000 7f2640ff9640 1 -- 172.21.15.7:0/3582214235 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634012630 con 0x55bca52fe520 2024-03-26T08:52:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.448+0000 7f264a59c640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26380a27a0 0x7f26380a4c60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f263c0d1fd0 tx=0x7f263c0d1d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.448+0000 7f2653831740 1 -- 172.21.15.7:0/3582214235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 msgr2=0x55bca5303020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.448+0000 7f2653831740 1 --2- 172.21.15.7:0/3582214235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5303020 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f264c072730 tx=0x7f2634009e00 comp rx=0 tx=0).stop 2024-03-26T08:52:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.448+0000 7f2653831740 1 -- 172.21.15.7:0/3582214235 shutdown_connections 2024-03-26T08:52:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.448+0000 7f2653831740 1 --2- 172.21.15.7:0/3582214235 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca5305b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.449+0000 7f2653831740 1 --2- 172.21.15.7:0/3582214235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5303020 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.449+0000 7f2653831740 1 --2- 172.21.15.7:0/3582214235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca52fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.449+0000 7f2653831740 1 -- 172.21.15.7:0/3582214235 >> 172.21.15.7:0/3582214235 conn(0x55bca52c61a0 msgr2=0x55bca52f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.449+0000 7f2653831740 1 -- 172.21.15.7:0/3582214235 shutdown_connections 2024-03-26T08:52:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.449+0000 7f2653831740 1 -- 172.21.15.7:0/3582214235 wait complete. 2024-03-26T08:52:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.450+0000 7f2653831740 1 -- start start 2024-03-26T08:52:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.450+0000 7f2653831740 1 Processor -- start 2024-03-26T08:52:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.450+0000 7f2653831740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca5307cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.450+0000 7f2653831740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5308230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.450+0000 7f264ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca5307cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.450+0000 7f264ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca5307cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58010/0 (socket says 172.21.15.7:58010) 2024-03-26T08:52:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.451+0000 7f264ad9d640 1 -- 172.21.15.7:0/255817537 learned_addr learned my addr 172.21.15.7:0/255817537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.451+0000 7f2653831740 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca530f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.451+0000 7f2649d9b640 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5308230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.451+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca521f0d0 con 0x55bca52fdb30 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.451+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca525f110 con 0x55bca52fe520 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.451+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bca53066e0 con 0x55bca5303690 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.451+0000 7f2653831740 10 client.0 _ll_register_callbacks cb 0x55bca525ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.452+0000 7f264ad9d640 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca530f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.452+0000 7f2649d9b640 1 -- 172.21.15.7:0/255817537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 msgr2=0x55bca530f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.452+0000 7f2649d9b640 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca530f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.452+0000 7f2649d9b640 1 -- 172.21.15.7:0/255817537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 msgr2=0x55bca5307cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.452+0000 7f2649d9b640 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca5307cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.452+0000 7f2649d9b640 1 -- 172.21.15.7:0/255817537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bca503f7e0 con 0x55bca52fe520 2024-03-26T08:52:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.452+0000 7f264ad9d640 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca5307cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.453+0000 7f2649d9b640 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5308230 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55bca52fa3f0 tx=0x7f263400ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18f95c1fd2395c85 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.453+0000 7f2631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.456+0000 7f2631ffb640 1 -- 172.21.15.7:0/255817537 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634017050 con 0x55bca52fe520 2024-03-26T08:52:42.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.456+0000 7f2631ffb640 1 -- 172.21.15.7:0/255817537 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f263401c860 con 0x55bca52fe520 2024-03-26T08:52:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.456+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2634013470 con 0x55bca52fe520 2024-03-26T08:52:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.456+0000 7f2631ffb640 1 -- 172.21.15.7:0/255817537 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f263402e050 con 0x55bca52fe520 2024-03-26T08:52:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2653831740 10 client.27968 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bca509ff70 con 0x55bca52fe520 2024-03-26T08:52:42.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2631ffb640 1 -- 172.21.15.7:0/255817537 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2634013470 con 0x55bca52fe520 2024-03-26T08:52:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2631ffb640 1 -- 172.21.15.7:0/255817537 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f26340257c0 con 0x55bca52fe520 2024-03-26T08:52:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2653831740 10 client.27968 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2653831740 10 client.27968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2653831740 20 client.27968 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2653831740 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bca5313d80 0x55bca53161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2653831740 4 client.27968 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.457+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bca52705b0 con 0x55bca5313d80 2024-03-26T08:52:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.458+0000 7f264ad9d640 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bca5313d80 0x55bca53161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.458+0000 7f2631ffb640 10 client.27968 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.458+0000 7f264ad9d640 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bca5313d80 0x55bca53161c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=423cedd353510288 server_cookie=a38718bf66b99923 in_seq=0 out_seq=0 2024-03-26T08:52:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2631ffb640 1 -- 172.21.15.7:0/255817537 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bca52705b0 con 0x55bca5313d80 2024-03-26T08:52:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2631ffb640 10 client.27968 handle_command_reply: tid=0 2024-03-26T08:52:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 client.27968 shutdown 2024-03-26T08:52:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 msgr2=0x55bca5308230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5308230 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55bca52fa3f0 tx=0x7f263400ef00 comp rx=0 tx=0).stop 2024-03-26T08:52:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 shutdown_connections 2024-03-26T08:52:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bca5313d80 0x55bca53161c0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca530f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fe520 0x55bca5308230 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2631ffb640 0 client.27968 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 --2- 172.21.15.7:0/255817537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fdb30 0x55bca5307cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 >> 172.21.15.7:0/255817537 conn(0x55bca52c61a0 msgr2=0x55bca52f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.459+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 shutdown_connections 2024-03-26T08:52:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.460+0000 7f2653831740 1 -- 172.21.15.7:0/255817537 wait complete. 2024-03-26T08:52:42.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.460+0000 7f2653831740 20 client.27968 trim_cache size 0 max 16384 2024-03-26T08:52:42.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.479+0000 7f2653831740 1 Processor -- start 2024-03-26T08:52:42.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.479+0000 7f2653831740 1 -- start start 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f2653831740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca52f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f2653831740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f2653831740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca5305a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f2653831740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1c2490 con 0x55bca52fe520 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f2653831740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1bff90 con 0x55bca52fdb30 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f2653831740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1be010 con 0x55bca5303690 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca52f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f2649d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca52f96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54072/0 (socket says 172.21.15.7:54072) 2024-03-26T08:52:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 -- 172.21.15.7:0/1385461205 learned_addr learned my addr 172.21.15.7:0/1385461205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 --2- 172.21.15.7:0/1385461205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca5305a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 -- 172.21.15.7:0/1385461205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 msgr2=0x55bca5305a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 --2- 172.21.15.7:0/1385461205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca5305a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 -- 172.21.15.7:0/1385461205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 msgr2=0x55bca530f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 --2- 172.21.15.7:0/1385461205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 -- 172.21.15.7:0/1385461205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bca4f37170 con 0x55bca52fdb30 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f2649d9b640 1 --2- 172.21.15.7:0/1385461205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.480+0000 7f264ad9d640 1 --2- 172.21.15.7:0/1385461205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca52f96d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f264402e010 tx=0x7f2644032340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa0107a6cb6d85f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2633fff640 1 -- 172.21.15.7:0/1385461205 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644039040 con 0x55bca52fdb30 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2633fff640 1 -- 172.21.15.7:0/1385461205 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2644052070 con 0x55bca52fdb30 2024-03-26T08:52:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2633fff640 1 -- 172.21.15.7:0/1385461205 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26440346a0 con 0x55bca52fdb30 2024-03-26T08:52:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2653831740 1 -- 172.21.15.7:0/1385461205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 msgr2=0x55bca52f96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2653831740 1 --2- 172.21.15.7:0/1385461205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca52f96d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f264402e010 tx=0x7f2644032340 comp rx=0 tx=0).stop 2024-03-26T08:52:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2653831740 1 -- 172.21.15.7:0/1385461205 shutdown_connections 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2653831740 1 --2- 172.21.15.7:0/1385461205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca5305a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2653831740 1 --2- 172.21.15.7:0/1385461205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2653831740 1 --2- 172.21.15.7:0/1385461205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca52f96d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2653831740 1 -- 172.21.15.7:0/1385461205 >> 172.21.15.7:0/1385461205 conn(0x55bca52c61a0 msgr2=0x55bca52f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.481+0000 7f2653831740 1 -- 172.21.15.7:0/1385461205 shutdown_connections 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 1 -- 172.21.15.7:0/1385461205 wait complete. 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 1 -- start start 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 1 Processor -- start 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca530cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca530e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1be610 con 0x55bca52fe520 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1bf930 con 0x55bca52fdb30 2024-03-26T08:52:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c1bec70 con 0x55bca5303690 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2653831740 10 client.0 _ll_register_callbacks cb 0x7f2644037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2649d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2649d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58020/0 (socket says 172.21.15.7:58020) 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f2649d9b640 1 -- 172.21.15.7:0/4087749579 learned_addr learned my addr 172.21.15.7:0/4087749579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f264ad9d640 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca530cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.482+0000 7f264ad9d640 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca530e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2649d9b640 1 -- 172.21.15.7:0/4087749579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 msgr2=0x55bca530e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2649d9b640 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca530e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2649d9b640 1 -- 172.21.15.7:0/4087749579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 msgr2=0x55bca530cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2649d9b640 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca530cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2649d9b640 1 -- 172.21.15.7:0/4087749579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bca530e740 con 0x55bca52fe520 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f264ad9d640 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca530cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2649d9b640 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530d140 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x55bca53091d0 tx=0x7f2634013c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffd907d403fd57f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2631ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2631ffb640 1 -- 172.21.15.7:0/4087749579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2634037060 con 0x55bca52fe520 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2631ffb640 1 -- 172.21.15.7:0/4087749579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f26340260b0 con 0x55bca52fe520 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2653831740 1 -- 172.21.15.7:0/4087749579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2644052070 con 0x55bca52fe520 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2631ffb640 1 -- 172.21.15.7:0/4087749579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2634017040 con 0x55bca52fe520 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2653831740 10 client.18162 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2653831740 1 -- 172.21.15.7:0/4087749579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55bca536f950 con 0x55bca52fe520 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2631ffb640 1 -- 172.21.15.7:0/4087749579 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2644052070 con 0x55bca52fe520 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2631ffb640 1 -- 172.21.15.7:0/4087749579 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f263403c070 con 0x55bca52fe520 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2653831740 10 client.18162 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2653831740 10 client.18162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2653831740 20 client.18162 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2653831740 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bca5370210 0x55bca5372630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2653831740 4 client.18162 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.483+0000 7f2653831740 1 -- 172.21.15.7:0/4087749579 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bca526a900 con 0x55bca5370210 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.484+0000 7f264ad9d640 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bca5370210 0x55bca5372630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.484+0000 7f2631ffb640 10 client.18162 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:42.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.484+0000 7f264ad9d640 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bca5370210 0x55bca5372630 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc3b7d835ab18b server_cookie=311bfbd503f5803e in_seq=0 out_seq=0 2024-03-26T08:52:42.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.485+0000 7f2631ffb640 1 -- 172.21.15.7:0/4087749579 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bca526a900 con 0x55bca5370210 2024-03-26T08:52:42.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.485+0000 7f2631ffb640 10 client.18162 handle_command_reply: tid=0 2024-03-26T08:52:42.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.485+0000 7f2653831740 1 client.18162 shutdown 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 -- 172.21.15.7:0/4087749579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 msgr2=0x55bca530d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530d140 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x55bca53091d0 tx=0x7f2634013c80 comp rx=0 tx=0).stop 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 -- 172.21.15.7:0/4087749579 shutdown_connections 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55bca5370210 0x55bca5372630 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55bca5303690 0x55bca530e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2631ffb640 0 client.18162 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bca52fe520 0x55bca530d140 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 --2- 172.21.15.7:0/4087749579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bca52fdb30 0x55bca530cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 -- 172.21.15.7:0/4087749579 >> 172.21.15.7:0/4087749579 conn(0x55bca52c61a0 msgr2=0x55bca52f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 -- 172.21.15.7:0/4087749579 shutdown_connections 2024-03-26T08:52:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 1 -- 172.21.15.7:0/4087749579 wait complete. 2024-03-26T08:52:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.486+0000 7f2653831740 20 client.18162 trim_cache size 0 max 16384 2024-03-26T08:52:42.487 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:52:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:52:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "f08f9919-dc06-4cd6-8083-367998e12332", 2024-03-26T08:52:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:52:42.487 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:52:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 -- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26380a27a0 msgr2=0x7f26380a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26380a27a0 0x7f26380a4c60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f263c0d1fd0 tx=0x7f263c0d1d80 comp rx=0 tx=0).stop 2024-03-26T08:52:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 -- 172.21.15.7:0/461298234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c072a10 msgr2=0x7f264c119930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c072a10 0x7f264c119930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d950 tx=0x7f263c00dd50 comp rx=0 tx=0).stop 2024-03-26T08:52:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 -- 172.21.15.7:0/461298234 shutdown_connections 2024-03-26T08:52:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f26380a27a0 0x7f26380a4c60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f264c109680 0x7f264c117f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f264c108cc0 0x7f264c119e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 --2- 172.21.15.7:0/461298234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f264c072a10 0x7f264c119930 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 -- 172.21.15.7:0/461298234 >> 172.21.15.7:0/461298234 conn(0x7f264c06e4e0 msgr2=0x7f264c10bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 -- 172.21.15.7:0/461298234 shutdown_connections 2024-03-26T08:52:42.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.487+0000 7f264b59e640 1 -- 172.21.15.7:0/461298234 wait complete. 2024-03-26T08:52:42.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:52:42.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:52:42.546 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:42 smithi007 ceph-mon[32462]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 MiB/s wr, 651 op/s 2024-03-26T08:52:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.647+0000 7f43bcb09640 1 -- 172.21.15.7:0/1079709738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b8107cc0 msgr2=0x7f43b81080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.647+0000 7f43bcb09640 1 --2- 172.21.15.7:0/1079709738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b8107cc0 0x7f43b81080e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009900 tx=0x7f43a802bc60 comp rx=0 tx=0).stop 2024-03-26T08:52:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.647+0000 7f43bcb09640 1 -- 172.21.15.7:0/1079709738 shutdown_connections 2024-03-26T08:52:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.647+0000 7f43bcb09640 1 --2- 172.21.15.7:0/1079709738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43b8109870 0x7f43b8110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.647+0000 7f43bcb09640 1 --2- 172.21.15.7:0/1079709738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108eb0 0x7f43b8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.647+0000 7f43bcb09640 1 --2- 172.21.15.7:0/1079709738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b8107cc0 0x7f43b81080e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.647+0000 7f43bcb09640 1 -- 172.21.15.7:0/1079709738 >> 172.21.15.7:0/1079709738 conn(0x7f43b8076c60 msgr2=0x7f43b80790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.647+0000 7f43bcb09640 1 -- 172.21.15.7:0/1079709738 shutdown_connections 2024-03-26T08:52:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.647+0000 7f43bcb09640 1 -- 172.21.15.7:0/1079709738 wait complete. 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43bcb09640 1 Processor -- start 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43bcb09640 1 -- start start 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43bcb09640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b8107cc0 0x7f43b819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43bcb09640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43b8108eb0 0x7f43b819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43bcb09640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8109870 0x7f43b81a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43bcb09640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8113960 con 0x7f43b8107cc0 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43bcb09640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b81137e0 con 0x7f43b8109870 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43bcb09640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8113ae0 con 0x7f43b8108eb0 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b8107cc0 0x7f43b819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43b659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b8107cc0 0x7f43b819e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58044/0 (socket says 172.21.15.7:58044) 2024-03-26T08:52:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43b659c640 1 -- 172.21.15.7:0/2414899221 learned_addr learned my addr 172.21.15.7:0/2414899221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.649+0000 7f43b6d9d640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8109870 0x7f43b81a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.650+0000 7f43b6d9d640 1 -- 172.21.15.7:0/2414899221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43b8108eb0 msgr2=0x7f43b819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.650+0000 7f43b6d9d640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43b8108eb0 0x7f43b819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.650+0000 7f43b6d9d640 1 -- 172.21.15.7:0/2414899221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b8107cc0 msgr2=0x7f43b819e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.650+0000 7f43b6d9d640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b8107cc0 0x7f43b819e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.650+0000 7f43b6d9d640 1 -- 172.21.15.7:0/2414899221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b81a5c50 con 0x7f43b8109870 2024-03-26T08:52:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.651+0000 7f43b6d9d640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8109870 0x7f43b81a55e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f43b000db20 tx=0x7f43b000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.651+0000 7f43af7fe640 1 -- 172.21.15.7:0/2414899221 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0017070 con 0x7f43b8109870 2024-03-26T08:52:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.651+0000 7f43af7fe640 1 -- 172.21.15.7:0/2414899221 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b000be40 con 0x7f43b8109870 2024-03-26T08:52:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.651+0000 7f43af7fe640 1 -- 172.21.15.7:0/2414899221 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b00106c0 con 0x7f43b8109870 2024-03-26T08:52:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.651+0000 7f43bcb09640 1 -- 172.21.15.7:0/2414899221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b81a5f40 con 0x7f43b8109870 2024-03-26T08:52:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.651+0000 7f43bcb09640 1 -- 172.21.15.7:0/2414899221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b81a6350 con 0x7f43b8109870 2024-03-26T08:52:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.655+0000 7f43bf7d4740 1 Processor -- start 2024-03-26T08:52:42.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.656+0000 7f43af7fe640 1 -- 172.21.15.7:0/2414899221 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f43b0004140 con 0x7f43b8109870 2024-03-26T08:52:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.656+0000 7f43bf7d4740 1 -- start start 2024-03-26T08:52:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc472dc0 0x5615bc4731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc4737b0 0x5615bc4782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc478920 0x5615bc47adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43bf7d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615bc3d83d0 con 0x5615bc478920 2024-03-26T08:52:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43bf7d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615bc380ad0 con 0x5615bc472dc0 2024-03-26T08:52:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43bf7d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615bc3d4470 con 0x5615bc4737b0 2024-03-26T08:52:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43af7fe640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43a40a2980 0x7f43a40a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc478920 0x5615bc47adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc478920 0x5615bc47adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58046/0 (socket says 172.21.15.7:58046) 2024-03-26T08:52:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43affff640 1 -- 172.21.15.7:0/3168063832 learned_addr learned my addr 172.21.15.7:0/3168063832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43affff640 1 --2- 172.21.15.7:0/3168063832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc472dc0 0x5615bc4731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43b6d9d640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43a40a2980 0x7f43a40a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.657+0000 7f43af7fe640 1 -- 172.21.15.7:0/2414899221 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f43b00ce620 con 0x7f43b8109870 2024-03-26T08:52:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.658+0000 7f43affff640 1 -- 172.21.15.7:0/3168063832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc4737b0 msgr2=0x5615bc4782b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.658+0000 7f43affff640 1 --2- 172.21.15.7:0/3168063832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc4737b0 0x5615bc4782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.658+0000 7f43affff640 1 -- 172.21.15.7:0/3168063832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc478920 msgr2=0x5615bc47adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.658+0000 7f43affff640 1 --2- 172.21.15.7:0/3168063832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc478920 0x5615bc47adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.658+0000 7f43affff640 1 -- 172.21.15.7:0/3168063832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615bc1b67d0 con 0x5615bc472dc0 2024-03-26T08:52:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.658+0000 7f43affff640 1 --2- 172.21.15.7:0/3168063832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc472dc0 0x5615bc4731e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f43a0013760 tx=0x7f43a0005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541d9b12f6eae95f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.658+0000 7f43acff9640 1 -- 172.21.15.7:0/3168063832 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0013da0 con 0x5615bc472dc0 2024-03-26T08:52:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.659+0000 7f43acff9640 1 -- 172.21.15.7:0/3168063832 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43a0017070 con 0x5615bc472dc0 2024-03-26T08:52:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.659+0000 7f43acff9640 1 -- 172.21.15.7:0/3168063832 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a000f840 con 0x5615bc472dc0 2024-03-26T08:52:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3168063832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc472dc0 msgr2=0x5615bc4731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3168063832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc472dc0 0x5615bc4731e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f43a0013760 tx=0x7f43a0005ac0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43b6d9d640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43a40a2980 0x7f43a40a4e40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f43b0005fd0 tx=0x7f43b00044e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3168063832 shutdown_connections 2024-03-26T08:52:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3168063832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc478920 0x5615bc47adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3168063832 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc4737b0 0x5615bc4782b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3168063832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc472dc0 0x5615bc4731e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3168063832 >> 172.21.15.7:0/3168063832 conn(0x5615bc46e330 msgr2=0x5615bc46e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3168063832 shutdown_connections 2024-03-26T08:52:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.660+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3168063832 wait complete. 2024-03-26T08:52:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.661+0000 7f43bf7d4740 1 -- start start 2024-03-26T08:52:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.661+0000 7f43bf7d4740 1 Processor -- start 2024-03-26T08:52:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.661+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc47d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.661+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc47d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.661+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc478920 0x5615bc4847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.661+0000 7f43bf7d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615bc47b920 con 0x5615bc472dc0 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.661+0000 7f43bf7d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615bc394060 con 0x5615bc4737b0 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.661+0000 7f43bf7d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615bc47baa0 con 0x5615bc478920 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.661+0000 7f43bf7d4740 10 client.0 _ll_register_callbacks cb 0x5615bc3766a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc47d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc47d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58056/0 (socket says 172.21.15.7:58056) 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43affff640 1 -- 172.21.15.7:0/2627562993 learned_addr learned my addr 172.21.15.7:0/2627562993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43b659c640 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc47d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43affff640 1 -- 172.21.15.7:0/2627562993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc478920 msgr2=0x5615bc4847d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43affff640 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc478920 0x5615bc4847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43affff640 1 -- 172.21.15.7:0/2627562993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 msgr2=0x5615bc47d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43affff640 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc47d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43affff640 1 -- 172.21.15.7:0/2627562993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615bc0ec3f0 con 0x5615bc472dc0 2024-03-26T08:52:42.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f43affff640 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc47d250 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f43a000ce80 tx=0x7f43a000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1a057aa454fe17 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.662+0000 7f439dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.663+0000 7f439dffb640 1 -- 172.21.15.7:0/2627562993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0029060 con 0x5615bc472dc0 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.663+0000 7f439dffb640 1 -- 172.21.15.7:0/2627562993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f43a0009040 con 0x5615bc472dc0 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.663+0000 7f43bf7d4740 1 -- 172.21.15.7:0/2627562993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43a0017070 con 0x5615bc472dc0 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.663+0000 7f439dffb640 1 -- 172.21.15.7:0/2627562993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43a0011960 con 0x5615bc472dc0 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.663+0000 7f43bf7d4740 10 client.18174 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.663+0000 7f43bf7d4740 1 -- 172.21.15.7:0/2627562993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5615bc0ac0f0 con 0x5615bc472dc0 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f439dffb640 1 -- 172.21.15.7:0/2627562993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43a0017070 con 0x5615bc472dc0 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f439dffb640 1 -- 172.21.15.7:0/2627562993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f43a000f040 con 0x5615bc472dc0 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f43bf7d4740 10 client.18174 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f43bf7d4740 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f43bf7d4740 20 client.18174 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5615bc499a50 0x5615bc49be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f43bf7d4740 4 client.18174 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f43bf7d4740 1 -- 172.21.15.7:0/2627562993 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615bc3e5910 con 0x5615bc499a50 2024-03-26T08:52:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f43b659c640 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5615bc499a50 0x5615bc49be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f439dffb640 10 client.18174 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:42.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.664+0000 7f43b659c640 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5615bc499a50 0x5615bc49be40 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a4241f15f20aa0 server_cookie=26caa242679b99d0 in_seq=0 out_seq=0 2024-03-26T08:52:42.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.665+0000 7f439dffb640 1 -- 172.21.15.7:0/2627562993 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5615bc3e5910 con 0x5615bc499a50 2024-03-26T08:52:42.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.665+0000 7f439dffb640 10 client.18174 handle_command_reply: tid=0 2024-03-26T08:52:42.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.665+0000 7f43bf7d4740 1 client.18174 shutdown 2024-03-26T08:52:42.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 -- 172.21.15.7:0/2627562993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 msgr2=0x5615bc47d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc47d250 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f43a000ce80 tx=0x7f43a000ceb0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 -- 172.21.15.7:0/2627562993 shutdown_connections 2024-03-26T08:52:42.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5615bc499a50 0x5615bc49be40 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc478920 0x5615bc4847d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc47d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/2627562993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc47d250 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f439dffb640 0 client.18174 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:42.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 -- 172.21.15.7:0/2627562993 >> 172.21.15.7:0/2627562993 conn(0x5615bc46e330 msgr2=0x5615bc46d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 -- 172.21.15.7:0/2627562993 shutdown_connections 2024-03-26T08:52:42.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 1 -- 172.21.15.7:0/2627562993 wait complete. 2024-03-26T08:52:42.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.666+0000 7f43bf7d4740 20 client.18174 trim_cache size 0 max 16384 2024-03-26T08:52:42.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.687+0000 7f43bf7d4740 1 Processor -- start 2024-03-26T08:52:42.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.687+0000 7f43bf7d4740 1 -- start start 2024-03-26T08:52:42.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.687+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc474b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.687+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc484790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc478920 0x5615bc4750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43bf7d4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b81a8e80 con 0x5615bc472dc0 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43bf7d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b81a6980 con 0x5615bc4737b0 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43bf7d4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8080b20 con 0x5615bc478920 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43b659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc484790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43b659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc484790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54116/0 (socket says 172.21.15.7:54116) 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43b659c640 1 -- 172.21.15.7:0/3391711107 learned_addr learned my addr 172.21.15.7:0/3391711107 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43affff640 1 --2- 172.21.15.7:0/3391711107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc474b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43b659c640 1 -- 172.21.15.7:0/3391711107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc478920 msgr2=0x5615bc4750b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43b659c640 1 --2- 172.21.15.7:0/3391711107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc478920 0x5615bc4750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43b659c640 1 -- 172.21.15.7:0/3391711107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 msgr2=0x5615bc474b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43b659c640 1 --2- 172.21.15.7:0/3391711107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc474b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.688+0000 7f43b659c640 1 -- 172.21.15.7:0/3391711107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615bc469da0 con 0x5615bc4737b0 2024-03-26T08:52:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.689+0000 7f43b659c640 1 --2- 172.21.15.7:0/3391711107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc484790 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f43a8000c00 tx=0x7f43a802cb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fee77e9624bda76 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.689+0000 7f439ffff640 1 -- 172.21.15.7:0/3391711107 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8045070 con 0x5615bc4737b0 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.689+0000 7f439ffff640 1 -- 172.21.15.7:0/3391711107 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43a802cc90 con 0x5615bc4737b0 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.689+0000 7f439ffff640 1 -- 172.21.15.7:0/3391711107 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8004920 con 0x5615bc4737b0 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3391711107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 msgr2=0x5615bc484790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3391711107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc484790 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f43a8000c00 tx=0x7f43a802cb20 comp rx=0 tx=0).stop 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3391711107 shutdown_connections 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3391711107 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc478920 0x5615bc4750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3391711107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc4737b0 0x5615bc484790 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3391711107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc474b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3391711107 >> 172.21.15.7:0/3391711107 conn(0x5615bc46e330 msgr2=0x5615bc49ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3391711107 shutdown_connections 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3391711107 wait complete. 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 -- start start 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.690+0000 7f43bf7d4740 1 Processor -- start 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc481940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43bf7d4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc4737b0 0x5615bc481e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc481940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43affff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc481940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58074/0 (socket says 172.21.15.7:58074) 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43affff640 1 -- 172.21.15.7:0/3238342136 learned_addr learned my addr 172.21.15.7:0/3238342136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc478920 0x5615bc482e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8081120 con 0x5615bc472dc0 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8082440 con 0x5615bc478920 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8081780 con 0x5615bc4737b0 2024-03-26T08:52:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43b659c640 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc4737b0 0x5615bc481e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43bf7d4740 10 client.0 _ll_register_callbacks cb 0x7f43a000c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.691+0000 7f43affff640 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc478920 0x5615bc482e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f43affff640 1 -- 172.21.15.7:0/3238342136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc4737b0 msgr2=0x5615bc481e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f43affff640 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc4737b0 0x5615bc481e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f43affff640 1 -- 172.21.15.7:0/3238342136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc478920 msgr2=0x5615bc482e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f43affff640 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc478920 0x5615bc482e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f43affff640 1 -- 172.21.15.7:0/3238342136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615bc483480 con 0x5615bc472dc0 2024-03-26T08:52:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f43affff640 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc481940 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f43a000c9d0 tx=0x7f43a000c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ace00b7c9f4cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f439dffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:52:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f439dffb640 1 -- 172.21.15.7:0/3238342136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0029030 con 0x5615bc472dc0 2024-03-26T08:52:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f439dffb640 1 -- 172.21.15.7:0/3238342136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f43a0021070 con 0x5615bc472dc0 2024-03-26T08:52:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.692+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43a802cc90 con 0x5615bc472dc0 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.693+0000 7f439dffb640 1 -- 172.21.15.7:0/3238342136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43a0013260 con 0x5615bc472dc0 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.693+0000 7f43bf7d4740 10 client.18180 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.693+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5615bc484770 con 0x5615bc472dc0 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.693+0000 7f439dffb640 1 -- 172.21.15.7:0/3238342136 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43a802cc90 con 0x5615bc472dc0 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.693+0000 7f439dffb640 1 -- 172.21.15.7:0/3238342136 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f43a000c020 con 0x5615bc472dc0 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f43bf7d4740 10 client.18180 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f43bf7d4740 10 client.18180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f43bf7d4740 20 client.18180 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5615bc4f5410 0x5615bc4f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f43bf7d4740 4 client.18180 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5615bc3dfc60 con 0x5615bc4f5410 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f43b659c640 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5615bc4f5410 0x5615bc4f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f439dffb640 10 client.18180 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f43b659c640 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5615bc4f5410 0x5615bc4f7800 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84691b858f799874 server_cookie=725908fd1d57cdab in_seq=0 out_seq=0 2024-03-26T08:52:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f439dffb640 1 -- 172.21.15.7:0/3238342136 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5615bc3dfc60 con 0x5615bc4f5410 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f439dffb640 10 client.18180 handle_command_reply: tid=0 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.694+0000 7f43bf7d4740 1 client.18180 shutdown 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 msgr2=0x5615bc481940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc481940 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f43a000c9d0 tx=0x7f43a000c340 comp rx=0 tx=0).stop 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 shutdown_connections 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5615bc4f5410 0x5615bc4f7800 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615bc478920 0x5615bc482e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615bc4737b0 0x5615bc481e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 --2- 172.21.15.7:0/3238342136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5615bc472dc0 0x5615bc481940 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 >> 172.21.15.7:0/3238342136 conn(0x5615bc46e330 msgr2=0x5615bc474b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f439dffb640 0 client.18180 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 shutdown_connections 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 1 -- 172.21.15.7:0/3238342136 wait complete. 2024-03-26T08:52:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.695+0000 7f43bf7d4740 20 client.18180 trim_cache size 0 max 16384 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (551 inodes in the stack)", 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout: "f08f9919-dc06-4cd6-8083-367998e12332": { 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "f08f9919-dc06-4cd6-8083-367998e12332", 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:52:42.695 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:52:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 -- 172.21.15.7:0/2414899221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43a40a2980 msgr2=0x7f43a40a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43a40a2980 0x7f43a40a4e40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f43b0005fd0 tx=0x7f43b00044e0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 -- 172.21.15.7:0/2414899221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8109870 msgr2=0x7f43b81a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8109870 0x7f43b81a55e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f43b000db20 tx=0x7f43b000b260 comp rx=0 tx=0).stop 2024-03-26T08:52:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 -- 172.21.15.7:0/2414899221 shutdown_connections 2024-03-26T08:52:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f43a40a2980 0x7f43a40a4e40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8109870 0x7f43b81a55e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f43b8108eb0 0x7f43b819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 --2- 172.21.15.7:0/2414899221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b8107cc0 0x7f43b819e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 -- 172.21.15.7:0/2414899221 >> 172.21.15.7:0/2414899221 conn(0x7f43b8076c60 msgr2=0x7f43b810c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 -- 172.21.15.7:0/2414899221 shutdown_connections 2024-03-26T08:52:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:52:42.696+0000 7f43bcb09640 1 -- 172.21.15.7:0/2414899221 wait complete. 2024-03-26T08:52:42.706 INFO:tasks.cephfs.filesystem:scrub status for tag:f08f9919-dc06-4cd6-8083-367998e12332 - {'path': '/', 'tag': 'f08f9919-dc06-4cd6-8083-367998e12332', 'options': 'recursive,force'} 2024-03-26T08:52:42.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:42 smithi016 ceph-mon[35851]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 MiB/s wr, 651 op/s 2024-03-26T08:52:42.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:42 smithi044 ceph-mon[36461]: pgmap v668: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 MiB/s wr, 651 op/s 2024-03-26T08:52:43.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:43 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:52:43.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:43 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:43.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:43 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:52:43.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:43 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:52:43.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:43 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:43.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:43 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:43.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:43 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:52:43.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:43 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:43.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:43 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:52:43.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:43 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:52:43.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:43 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:43.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:43 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:43.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:43 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:52:43.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:43 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:43.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:43 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:52:43.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:43 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:52:43.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:43 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:43.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:43 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:52:44.721 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:44 smithi044 ceph-mon[36461]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 697 op/s 2024-03-26T08:52:44.721 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:44 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:52:44.738 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:44 smithi007 ceph-mon[32462]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 697 op/s 2024-03-26T08:52:44.739 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:44 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:52:44.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:44 smithi016 ceph-mon[35851]: pgmap v669: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 MiB/s wr, 697 op/s 2024-03-26T08:52:44.742 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:44 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:52:46.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 615.0 s, 1290.4 tps, lat 76.017 ms stddev 81.252 2024-03-26T08:52:46.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:46 smithi016 ceph-mon[35851]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.5 MiB/s wr, 679 op/s 2024-03-26T08:52:46.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:46 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:46 smithi007 ceph-mon[32462]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.5 MiB/s wr, 679 op/s 2024-03-26T08:52:46.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:46 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:46 smithi044 ceph-mon[36461]: pgmap v670: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.5 MiB/s wr, 679 op/s 2024-03-26T08:52:46.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:46.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:46 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:47.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:47 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:52:47.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:47 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:52:47.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:47 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:47.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:47 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:52:47.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:47 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:52:47.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:47 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:47.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:47 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:52:47.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:47 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:52:47.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:47 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:52:48.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:48 smithi016 ceph-mon[35851]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.9 MiB/s wr, 734 op/s 2024-03-26T08:52:48.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:48 smithi007 ceph-mon[32462]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.9 MiB/s wr, 734 op/s 2024-03-26T08:52:48.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:48 smithi044 ceph-mon[36461]: pgmap v671: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 6.9 MiB/s wr, 734 op/s 2024-03-26T08:52:49.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:49 smithi016 ceph-mon[35851]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.3 MiB/s wr, 707 op/s 2024-03-26T08:52:49.796 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:49 smithi007 ceph-mon[32462]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.3 MiB/s wr, 707 op/s 2024-03-26T08:52:49.820 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:49 smithi044 ceph-mon[36461]: pgmap v672: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.3 MiB/s wr, 707 op/s 2024-03-26T08:52:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.945+0000 7faac70c0640 1 -- 172.21.15.7:0/3566538775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 msgr2=0x7faac0101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.945+0000 7faac70c0640 1 --2- 172.21.15.7:0/3566538775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 0x7faac0101b20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7faab4009900 tx=0x7faab402bc50 comp rx=0 tx=0).stop 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.945+0000 7faac70c0640 1 -- 172.21.15.7:0/3566538775 shutdown_connections 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.945+0000 7faac70c0640 1 --2- 172.21.15.7:0/3566538775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac01032b0 0x7faac0109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.945+0000 7faac70c0640 1 --2- 172.21.15.7:0/3566538775 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01028f0 0x7faac0102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.945+0000 7faac70c0640 1 --2- 172.21.15.7:0/3566538775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 0x7faac0101b20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.945+0000 7faac70c0640 1 -- 172.21.15.7:0/3566538775 >> 172.21.15.7:0/3566538775 conn(0x7faac00fcc90 msgr2=0x7faac00ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.945+0000 7faac70c0640 1 -- 172.21.15.7:0/3566538775 shutdown_connections 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.945+0000 7faac70c0640 1 -- 172.21.15.7:0/3566538775 wait complete. 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac70c0640 1 Processor -- start 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac70c0640 1 -- start start 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac70c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 0x7faac0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac70c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01028f0 0x7faac0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac70c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac01032b0 0x7faac019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac70c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac010d3f0 con 0x7faac0101700 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac70c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac010d270 con 0x7faac01032b0 2024-03-26T08:52:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac70c0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac010d570 con 0x7faac01028f0 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac4e5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 0x7faac0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac4e5c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 0x7faac0197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35354/0 (socket says 172.21.15.7:35354) 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac4e5c640 1 -- 172.21.15.7:0/4113970640 learned_addr learned my addr 172.21.15.7:0/4113970640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faabffff640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01028f0 0x7faac0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac565d640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac01032b0 0x7faac019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac4e5c640 1 -- 172.21.15.7:0/4113970640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01028f0 msgr2=0x7faac0198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac4e5c640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01028f0 0x7faac0198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac4e5c640 1 -- 172.21.15.7:0/4113970640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac01032b0 msgr2=0x7faac019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac4e5c640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac01032b0 0x7faac019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faac4e5c640 1 -- 172.21.15.7:0/4113970640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac019f750 con 0x7faac0101700 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.946+0000 7faabffff640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01028f0 0x7faac0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.947+0000 7faac4e5c640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 0x7faac0197b30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7faab4000c00 tx=0x7faab4005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.947+0000 7faabdffb640 1 -- 172.21.15.7:0/4113970640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4042020 con 0x7faac0101700 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.947+0000 7faac70c0640 1 -- 172.21.15.7:0/4113970640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac019f9e0 con 0x7faac0101700 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.947+0000 7faabdffb640 1 -- 172.21.15.7:0/4113970640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faab40044d0 con 0x7faac0101700 2024-03-26T08:52:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.947+0000 7faabdffb640 1 -- 172.21.15.7:0/4113970640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab403e460 con 0x7faac0101700 2024-03-26T08:52:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.947+0000 7faac70c0640 1 -- 172.21.15.7:0/4113970640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac019ff30 con 0x7faac0101700 2024-03-26T08:52:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac9d87740 1 Processor -- start 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac9d87740 1 -- start start 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac9d87740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac9d87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 0x56370ffd2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac9d87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffd4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac9d87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370ff2db10 con 0x56370ffcd720 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac9d87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370fed6900 con 0x56370ffd2890 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac9d87740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370ff29bb0 con 0x56370ffccd30 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faabdffb640 1 -- 172.21.15.7:0/4113970640 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7faab4045020 con 0x7faac0101700 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac565d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faabffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffd4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac565d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38400/0 (socket says 172.21.15.7:38400) 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac565d640 1 -- 172.21.15.7:0/433918025 learned_addr learned my addr 172.21.15.7:0/433918025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.950+0000 7faac4e5c640 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 0x56370ffd2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faac565d640 1 -- 172.21.15.7:0/433918025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 msgr2=0x56370ffd4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faac565d640 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffd4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faac565d640 1 -- 172.21.15.7:0/433918025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 msgr2=0x56370ffd2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faac565d640 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 0x56370ffd2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faac565d640 1 -- 172.21.15.7:0/433918025 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56370fc42210 con 0x56370ffccd30 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faabffff640 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffd4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faac4e5c640 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 0x56370ffd2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faac565d640 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcd150 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faab800a4d0 tx=0x7faab8009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db144c753701abad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faaaaffd640 1 -- 172.21.15.7:0/433918025 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8018020 con 0x56370ffccd30 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faaaaffd640 1 -- 172.21.15.7:0/433918025 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faab8014cf0 con 0x56370ffccd30 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.951+0000 7faaaaffd640 1 -- 172.21.15.7:0/433918025 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8013dd0 con 0x56370ffccd30 2024-03-26T08:52:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.952+0000 7faabdffb640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faab00a27f0 0x7faab00a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.952+0000 7faac4e5c640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faab00a27f0 0x7faab00a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.952+0000 7faabdffb640 1 -- 172.21.15.7:0/4113970640 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faab40f0a20 con 0x7faac0101700 2024-03-26T08:52:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.952+0000 7faac9d87740 1 -- 172.21.15.7:0/433918025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 msgr2=0x56370ffcd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.952+0000 7faac9d87740 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcd150 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faab800a4d0 tx=0x7faab8009650 comp rx=0 tx=0).stop 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.953+0000 7faac9d87740 1 -- 172.21.15.7:0/433918025 shutdown_connections 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.953+0000 7faac9d87740 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffd4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.953+0000 7faac9d87740 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 0x56370ffd2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.953+0000 7faac9d87740 1 --2- 172.21.15.7:0/433918025 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcd150 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.953+0000 7faac9d87740 1 -- 172.21.15.7:0/433918025 >> 172.21.15.7:0/433918025 conn(0x56370ffc3e90 msgr2=0x56370ffc4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.953+0000 7faac9d87740 1 -- 172.21.15.7:0/433918025 shutdown_connections 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.953+0000 7faac9d87740 1 -- 172.21.15.7:0/433918025 wait complete. 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.953+0000 7faac4e5c640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faab00a27f0 0x7faab00a4cb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x56370ffcdb20 tx=0x7faab402bc50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.954+0000 7faac9d87740 1 -- start start 2024-03-26T08:52:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.954+0000 7faac9d87740 1 Processor -- start 2024-03-26T08:52:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac9d87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffccd30 0x56370ffd7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac9d87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 0x56370ffd7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac9d87740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffd2890 0x56370ffde5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac9d87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370ffd5840 con 0x56370ffcd720 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac9d87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370fee9d50 con 0x56370ffccd30 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac9d87740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370ffd59c0 con 0x56370ffd2890 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac9d87740 10 client.0 _ll_register_callbacks cb 0x56370fee9720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac565d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffccd30 0x56370ffd7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac565d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffccd30 0x56370ffd7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54232/0 (socket says 172.21.15.7:54232) 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac565d640 1 -- 172.21.15.7:0/2394277217 learned_addr learned my addr 172.21.15.7:0/2394277217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac565d640 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffd2890 0x56370ffde5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faabffff640 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 0x56370ffd7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.954 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac565d640 1 -- 172.21.15.7:0/2394277217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffd2890 msgr2=0x56370ffde5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac565d640 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffd2890 0x56370ffde5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac565d640 1 -- 172.21.15.7:0/2394277217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 msgr2=0x56370ffd7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac565d640 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 0x56370ffd7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.955+0000 7faac565d640 1 -- 172.21.15.7:0/2394277217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56370fd0e620 con 0x56370ffccd30 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.956+0000 7faac565d640 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffccd30 0x56370ffd7050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7faab80093b0 tx=0x7faab8009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa8196a539900f6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.956+0000 7faa9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.956+0000 7faa9ffff640 1 -- 172.21.15.7:0/2394277217 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8018070 con 0x56370ffccd30 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.956+0000 7faac9d87740 1 -- 172.21.15.7:0/2394277217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faab8014cf0 con 0x56370ffccd30 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.956+0000 7faa9ffff640 1 -- 172.21.15.7:0/2394277217 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faab8014820 con 0x56370ffccd30 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.956+0000 7faa9ffff640 1 -- 172.21.15.7:0/2394277217 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab8014a20 con 0x56370ffccd30 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.956+0000 7faac9d87740 10 client.27998 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.956+0000 7faac9d87740 1 -- 172.21.15.7:0/2394277217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56370ffe1990 con 0x56370ffccd30 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.956+0000 7faa9ffff640 1 -- 172.21.15.7:0/2394277217 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faab8014cf0 con 0x56370ffccd30 2024-03-26T08:52:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.957+0000 7faa9ffff640 1 -- 172.21.15.7:0/2394277217 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7faab801c3b0 con 0x56370ffccd30 2024-03-26T08:52:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.957+0000 7faac9d87740 10 client.27998 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.957+0000 7faac9d87740 10 client.27998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.957+0000 7faac9d87740 20 client.27998 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.957+0000 7faac9d87740 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56370ffe2e10 0x56370ffe5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.957+0000 7faac9d87740 4 client.27998 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:52:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.957+0000 7faac9d87740 1 -- 172.21.15.7:0/2394277217 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56370ff3b050 con 0x56370ffe2e10 2024-03-26T08:52:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.957+0000 7faabffff640 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56370ffe2e10 0x56370ffe5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.958+0000 7faabffff640 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56370ffe2e10 0x56370ffe5200 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4254be4a8a0a386 server_cookie=948fa70373ff7571 in_seq=0 out_seq=0 2024-03-26T08:52:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.958+0000 7faa9ffff640 10 client.27998 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faa9ffff640 1 -- 172.21.15.7:0/2394277217 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56370ff3b050 con 0x56370ffe2e10 2024-03-26T08:52:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faa9ffff640 10 client.27998 handle_command_reply: tid=0 2024-03-26T08:52:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 client.27998 shutdown 2024-03-26T08:52:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 -- 172.21.15.7:0/2394277217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffccd30 msgr2=0x56370ffd7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffccd30 0x56370ffd7050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7faab80093b0 tx=0x7faab8009c60 comp rx=0 tx=0).stop 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 -- 172.21.15.7:0/2394277217 shutdown_connections 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56370ffe2e10 0x56370ffe5200 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffd2890 0x56370ffde5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffcd720 0x56370ffd7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 --2- 172.21.15.7:0/2394277217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffccd30 0x56370ffd7050 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 -- 172.21.15.7:0/2394277217 >> 172.21.15.7:0/2394277217 conn(0x56370ffc3e90 msgr2=0x56370ffc3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faa9ffff640 0 client.27998 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 -- 172.21.15.7:0/2394277217 shutdown_connections 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 1 -- 172.21.15.7:0/2394277217 wait complete. 2024-03-26T08:52:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.959+0000 7faac9d87740 20 client.27998 trim_cache size 0 max 16384 2024-03-26T08:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.979+0000 7faac9d87740 1 Processor -- start 2024-03-26T08:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.979+0000 7faac9d87740 1 -- start start 2024-03-26T08:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.979+0000 7faac9d87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffccd30 0x56370ffcea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.979+0000 7faac9d87740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffcd720 0x56370ffde590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.979+0000 7faac9d87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffcef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.980+0000 7faac9d87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01a8f10 con 0x56370ffccd30 2024-03-26T08:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.980+0000 7faac9d87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01a6a10 con 0x56370ffd2890 2024-03-26T08:52:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.980+0000 7faac9d87740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01a4a90 con 0x56370ffcd720 2024-03-26T08:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.984+0000 7faac565d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffcef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.984+0000 7faac565d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffccd30 0x56370ffcea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.984+0000 7faac565d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffccd30 0x56370ffcea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35390/0 (socket says 172.21.15.7:35390) 2024-03-26T08:52:50.981 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.984+0000 7faac565d640 1 -- 172.21.15.7:0/88573994 learned_addr learned my addr 172.21.15.7:0/88573994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.986+0000 7faabffff640 1 --2- 172.21.15.7:0/88573994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffcd720 0x56370ffde590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.987+0000 7faac565d640 1 -- 172.21.15.7:0/88573994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffcd720 msgr2=0x56370ffde590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.987+0000 7faac565d640 1 --2- 172.21.15.7:0/88573994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffcd720 0x56370ffde590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.987+0000 7faac565d640 1 -- 172.21.15.7:0/88573994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffccd30 msgr2=0x56370ffcea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.987+0000 7faac565d640 1 --2- 172.21.15.7:0/88573994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffccd30 0x56370ffcea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.987+0000 7faac565d640 1 -- 172.21.15.7:0/88573994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56370ffcd110 con 0x56370ffd2890 2024-03-26T08:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:50.996+0000 7faac565d640 1 --2- 172.21.15.7:0/88573994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffcef80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faab8001220 tx=0x7faab8000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc730037d25d86ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.000+0000 7faaaa7fc640 1 -- 172.21.15.7:0/88573994 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8018040 con 0x56370ffd2890 2024-03-26T08:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.000+0000 7faaaa7fc640 1 -- 172.21.15.7:0/88573994 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faab801c3b0 con 0x56370ffd2890 2024-03-26T08:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.001+0000 7faac9d87740 1 -- 172.21.15.7:0/88573994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 msgr2=0x56370ffcef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.001+0000 7faac9d87740 1 --2- 172.21.15.7:0/88573994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffcef80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7faab8001220 tx=0x7faab8000f40 comp rx=0 tx=0).stop 2024-03-26T08:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.009+0000 7faac9d87740 1 -- 172.21.15.7:0/88573994 shutdown_connections 2024-03-26T08:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.009+0000 7faac9d87740 1 --2- 172.21.15.7:0/88573994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffd2890 0x56370ffcef80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.009+0000 7faac9d87740 1 --2- 172.21.15.7:0/88573994 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffcd720 0x56370ffde590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.009+0000 7faac9d87740 1 --2- 172.21.15.7:0/88573994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffccd30 0x56370ffcea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.009+0000 7faac9d87740 1 -- 172.21.15.7:0/88573994 >> 172.21.15.7:0/88573994 conn(0x56370ffc3e90 msgr2=0x56370ffc4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.018+0000 7faac9d87740 1 -- 172.21.15.7:0/88573994 shutdown_connections 2024-03-26T08:52:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.021+0000 7faac9d87740 1 -- 172.21.15.7:0/88573994 wait complete. 2024-03-26T08:52:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.021+0000 7faac9d87740 1 -- start start 2024-03-26T08:52:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.022+0000 7faac9d87740 1 Processor -- start 2024-03-26T08:52:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.026+0000 7faac9d87740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.026+0000 7faac9d87740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffcd720 0x56370ffcb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.026+0000 7faac9d87740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffd2890 0x56370ffde590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.026+0000 7faac9d87740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01a5090 con 0x56370ffd2890 2024-03-26T08:52:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.026+0000 7faac9d87740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01a63b0 con 0x56370ffcd720 2024-03-26T08:52:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.026+0000 7faac9d87740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01a56f0 con 0x56370ffccd30 2024-03-26T08:52:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.026+0000 7faac9d87740 10 client.0 _ll_register_callbacks cb 0x7faab80137b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:52:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.030+0000 7faac565d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.030+0000 7faabffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffcd720 0x56370ffcb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.030+0000 7faac565d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38434/0 (socket says 172.21.15.7:38434) 2024-03-26T08:52:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.030+0000 7faac565d640 1 -- 172.21.15.7:0/1114255438 learned_addr learned my addr 172.21.15.7:0/1114255438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:52:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.030+0000 7faac565d640 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffd2890 0x56370ffde590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.031+0000 7faac565d640 1 -- 172.21.15.7:0/1114255438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffcd720 msgr2=0x56370ffcb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.031+0000 7faac565d640 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffcd720 0x56370ffcb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.031+0000 7faac565d640 1 -- 172.21.15.7:0/1114255438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffd2890 msgr2=0x56370ffde590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:52:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.031+0000 7faac565d640 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffd2890 0x56370ffde590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.031+0000 7faac565d640 1 -- 172.21.15.7:0/1114255438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563710049630 con 0x56370ffccd30 2024-03-26T08:52:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.032+0000 7faac565d640 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcb030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faab8007060 tx=0x7faab8013860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22b6866aef8c0130 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:52:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.033+0000 7faa9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:52:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.033+0000 7faa9ffff640 1 -- 172.21.15.7:0/1114255438 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab804a060 con 0x56370ffccd30 2024-03-26T08:52:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.033+0000 7faac9d87740 1 -- 172.21.15.7:0/1114255438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faab801c3b0 con 0x56370ffccd30 2024-03-26T08:52:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.033+0000 7faa9ffff640 1 -- 172.21.15.7:0/1114255438 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faab801cbc0 con 0x56370ffccd30 2024-03-26T08:52:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.034+0000 7faa9ffff640 1 -- 172.21.15.7:0/1114255438 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab8014dc0 con 0x56370ffccd30 2024-03-26T08:52:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.038+0000 7faac9d87740 10 client.28009 fetch_fsmap learned FSMap version 25 2024-03-26T08:52:51.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.038+0000 7faac9d87740 1 -- 172.21.15.7:0/1114255438 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56371004a790 con 0x56370ffccd30 2024-03-26T08:52:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.040+0000 7faa9ffff640 1 -- 172.21.15.7:0/1114255438 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faab801c3b0 con 0x56370ffccd30 2024-03-26T08:52:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.040+0000 7faa9ffff640 1 -- 172.21.15.7:0/1114255438 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7faab8050460 con 0x56370ffccd30 2024-03-26T08:52:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.040+0000 7faac9d87740 10 client.28009 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:52:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.040+0000 7faac9d87740 10 client.28009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:52:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.040+0000 7faac9d87740 20 client.28009 populate_metadata read hostname 'smithi007' 2024-03-26T08:52:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.040+0000 7faac9d87740 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56371004b070 0x56371004d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:52:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.040+0000 7faac9d87740 4 client.28009 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:52:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.040+0000 7faac9d87740 1 -- 172.21.15.7:0/1114255438 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56370ff353a0 con 0x56371004b070 2024-03-26T08:52:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.040+0000 7faabffff640 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56371004b070 0x56371004d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:52:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.041+0000 7faa9ffff640 10 client.28009 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.041+0000 7faabffff640 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56371004b070 0x56371004d490 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cac18e4da7dc6da server_cookie=283f343179fb1764 in_seq=0 out_seq=0 2024-03-26T08:52:51.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 620.0 s, 1769.6 tps, lat 56.873 ms stddev 67.278 2024-03-26T08:52:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.089+0000 7faa9ffff640 1 -- 172.21.15.7:0/1114255438 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56370ff353a0 con 0x56371004b070 2024-03-26T08:52:51.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.089+0000 7faa9ffff640 10 client.28009 handle_command_reply: tid=0 2024-03-26T08:52:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.089+0000 7faac9d87740 1 client.28009 shutdown 2024-03-26T08:52:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.089+0000 7faac9d87740 1 -- 172.21.15.7:0/1114255438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 msgr2=0x56370ffcb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.089+0000 7faac9d87740 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcb030 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7faab8007060 tx=0x7faab8013860 comp rx=0 tx=0).stop 2024-03-26T08:52:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac9d87740 1 -- 172.21.15.7:0/1114255438 shutdown_connections 2024-03-26T08:52:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac9d87740 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56371004b070 0x56371004d490 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac9d87740 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56370ffd2890 0x56370ffde590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac9d87740 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56370ffcd720 0x56370ffcb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac9d87740 1 --2- 172.21.15.7:0/1114255438 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56370ffccd30 0x56370ffcb030 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac9d87740 1 -- 172.21.15.7:0/1114255438 >> 172.21.15.7:0/1114255438 conn(0x56370ffc3e90 msgr2=0x56370ffcea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faa9ffff640 0 client.28009 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:52:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac9d87740 1 -- 172.21.15.7:0/1114255438 shutdown_connections 2024-03-26T08:52:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac9d87740 1 -- 172.21.15.7:0/1114255438 wait complete. 2024-03-26T08:52:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac9d87740 20 client.28009 trim_cache size 0 max 16384 2024-03-26T08:52:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:52:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:52:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:52:51.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:52:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 -- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faab00a27f0 msgr2=0x7faab00a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faab00a27f0 0x7faab00a4cb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x56370ffcdb20 tx=0x7faab402bc50 comp rx=0 tx=0).stop 2024-03-26T08:52:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 -- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 msgr2=0x7faac0197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:52:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 0x7faac0197b30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7faab4000c00 tx=0x7faab4005d10 comp rx=0 tx=0).stop 2024-03-26T08:52:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 -- 172.21.15.7:0/4113970640 shutdown_connections 2024-03-26T08:52:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faab00a27f0 0x7faab00a4cb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac01032b0 0x7faac019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.088 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faac01028f0 0x7faac0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 --2- 172.21.15.7:0/4113970640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac0101700 0x7faac0197b30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:52:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.090+0000 7faac70c0640 1 -- 172.21.15.7:0/4113970640 >> 172.21.15.7:0/4113970640 conn(0x7faac00fcc90 msgr2=0x7faac0105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:52:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.091+0000 7faac70c0640 1 -- 172.21.15.7:0/4113970640 shutdown_connections 2024-03-26T08:52:51.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:52:51.091+0000 7faac70c0640 1 -- 172.21.15.7:0/4113970640 wait complete. 2024-03-26T08:52:51.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:51 smithi016 ceph-mon[35851]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.8 MiB/s wr, 741 op/s 2024-03-26T08:52:51.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:51 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:52:51.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:51 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:52:52.050 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:51 smithi007 ceph-mon[32462]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.8 MiB/s wr, 741 op/s 2024-03-26T08:52:52.050 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:51 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:52:52.050 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:51 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:52:52.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:51 smithi044 ceph-mon[36461]: pgmap v673: 97 pgs: 97 active+clean; 1.6 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.8 MiB/s wr, 741 op/s 2024-03-26T08:52:52.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:51 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:52:52.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:51 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:52:53.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:53 smithi016 ceph-mon[35851]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.5 MiB/s wr, 800 op/s 2024-03-26T08:52:54.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:53 smithi007 ceph-mon[32462]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.5 MiB/s wr, 800 op/s 2024-03-26T08:52:54.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:53 smithi044 ceph-mon[36461]: pgmap v674: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.5 MiB/s wr, 800 op/s 2024-03-26T08:52:55.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:55 smithi016 ceph-mon[35851]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.2 MiB/s wr, 766 op/s 2024-03-26T08:52:56.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:55 smithi007 ceph-mon[32462]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.2 MiB/s wr, 766 op/s 2024-03-26T08:52:56.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:55 smithi044 ceph-mon[36461]: pgmap v675: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.2 MiB/s wr, 766 op/s 2024-03-26T08:52:56.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 625.0 s, 2386.8 tps, lat 42.267 ms stddev 51.971 2024-03-26T08:52:57.848 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:57 smithi007 ceph-mon[32462]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.9 MiB/s wr, 723 op/s 2024-03-26T08:52:57.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:57 smithi016 ceph-mon[35851]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.9 MiB/s wr, 723 op/s 2024-03-26T08:52:58.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:57 smithi044 ceph-mon[36461]: pgmap v676: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 6.9 MiB/s wr, 723 op/s 2024-03-26T08:52:59.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:52:59 smithi016 ceph-mon[35851]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 626 op/s 2024-03-26T08:53:00.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:52:59 smithi007 ceph-mon[32462]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 626 op/s 2024-03-26T08:53:00.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:52:59 smithi044 ceph-mon[36461]: pgmap v677: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 626 op/s 2024-03-26T08:53:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 630.0 s, 1672.8 tps, lat 59.579 ms stddev 62.761 2024-03-26T08:53:01.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:01 smithi016 ceph-mon[35851]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 MiB/s wr, 690 op/s 2024-03-26T08:53:02.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:01 smithi007 ceph-mon[32462]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 MiB/s wr, 690 op/s 2024-03-26T08:53:02.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:01 smithi044 ceph-mon[36461]: pgmap v678: 97 pgs: 97 active+clean; 1.6 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 MiB/s wr, 690 op/s 2024-03-26T08:53:03.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:03 smithi016 ceph-mon[35851]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 674 op/s 2024-03-26T08:53:04.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:03 smithi007 ceph-mon[32462]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 674 op/s 2024-03-26T08:53:04.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:03 smithi044 ceph-mon[36461]: pgmap v679: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 MiB/s wr, 674 op/s 2024-03-26T08:53:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.237+0000 7f631da27640 1 -- 172.21.15.7:0/1824031838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181086a0 msgr2=0x7f631810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.237+0000 7f631da27640 1 --2- 172.21.15.7:0/1824031838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181086a0 0x7f631810aad0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6300009900 tx=0x7f630002bc60 comp rx=0 tx=0).stop 2024-03-26T08:53:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.243+0000 7f631da27640 1 -- 172.21.15.7:0/1824031838 shutdown_connections 2024-03-26T08:53:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.243+0000 7f631da27640 1 --2- 172.21.15.7:0/1824031838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f631810d9c0 0x7f631810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.243+0000 7f631da27640 1 --2- 172.21.15.7:0/1824031838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631810b010 0x7f631810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.243+0000 7f631da27640 1 --2- 172.21.15.7:0/1824031838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f63181086a0 0x7f631810aad0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.243+0000 7f631da27640 1 -- 172.21.15.7:0/1824031838 >> 172.21.15.7:0/1824031838 conn(0x7f631806e6b0 msgr2=0x7f6318070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:04.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.248+0000 7f631da27640 1 -- 172.21.15.7:0/1824031838 shutdown_connections 2024-03-26T08:53:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.248+0000 7f631da27640 1 -- 172.21.15.7:0/1824031838 wait complete. 2024-03-26T08:53:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.248+0000 7f631da27640 1 Processor -- start 2024-03-26T08:53:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.249+0000 7f631da27640 1 -- start start 2024-03-26T08:53:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.249+0000 7f631da27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63181086a0 0x7f6318116da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.249+0000 7f631da27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f631810b010 0x7f63181172e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.249+0000 7f631da27640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631810d9c0 0x7f63181b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.249+0000 7f631da27640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631811c1e0 con 0x7f631810d9c0 2024-03-26T08:53:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.249+0000 7f631da27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631811c060 con 0x7f63181086a0 2024-03-26T08:53:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.249+0000 7f631da27640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631811c360 con 0x7f631810b010 2024-03-26T08:53:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.250+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63181086a0 0x7f6318116da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.250+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63181086a0 0x7f6318116da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55934/0 (socket says 172.21.15.7:55934) 2024-03-26T08:53:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.250+0000 7f6316ffd640 1 -- 172.21.15.7:0/766117803 learned_addr learned my addr 172.21.15.7:0/766117803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.251+0000 7f63167fc640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f631810b010 0x7f63181172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.252+0000 7f6316ffd640 1 -- 172.21.15.7:0/766117803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f631810b010 msgr2=0x7f63181172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.252+0000 7f6316ffd640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f631810b010 0x7f63181172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.252+0000 7f6316ffd640 1 -- 172.21.15.7:0/766117803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631810d9c0 msgr2=0x7f63181b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.252+0000 7f6316ffd640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631810d9c0 0x7f63181b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.250 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.252+0000 7f6316ffd640 1 -- 172.21.15.7:0/766117803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63181b6e30 con 0x7f63181086a0 2024-03-26T08:53:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.258+0000 7f6316ffd640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63181086a0 0x7f6318116da0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f630002efd0 tx=0x7f6300004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.259+0000 7f631ca25640 1 -- 172.21.15.7:0/766117803 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300041070 con 0x7f63181086a0 2024-03-26T08:53:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.259+0000 7f631da27640 1 -- 172.21.15.7:0/766117803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63181b7000 con 0x7f63181086a0 2024-03-26T08:53:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.259+0000 7f631da27640 1 -- 172.21.15.7:0/766117803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63181b7400 con 0x7f63181086a0 2024-03-26T08:53:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.259+0000 7f631ca25640 1 -- 172.21.15.7:0/766117803 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63000054e0 con 0x7f63181086a0 2024-03-26T08:53:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.259+0000 7f631ca25640 1 -- 172.21.15.7:0/766117803 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300034440 con 0x7f63181086a0 2024-03-26T08:53:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.267+0000 7f631ca25640 1 -- 172.21.15.7:0/766117803 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6300044070 con 0x7f63181086a0 2024-03-26T08:53:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.267+0000 7f631ca25640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f63040a2a00 0x7f63040a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.267+0000 7f631ca25640 1 -- 172.21.15.7:0/766117803 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f63000f0bb0 con 0x7f63181086a0 2024-03-26T08:53:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.269+0000 7f63167fc640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f63040a2a00 0x7f63040a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.269+0000 7f6320612740 1 Processor -- start 2024-03-26T08:53:04.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.269+0000 7f6320612740 1 -- start start 2024-03-26T08:53:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.269+0000 7f6320612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdfe920 0x55b32fdfed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.269+0000 7f6320612740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdff310 0x55b32fe03e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.269+0000 7f6320612740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fe04480 0x55b32fe06910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.270+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdfe920 0x55b32fdfed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.270+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdfe920 0x55b32fdfed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55946/0 (socket says 172.21.15.7:55946) 2024-03-26T08:53:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.270+0000 7f6316ffd640 1 -- 172.21.15.7:0/4031025061 learned_addr learned my addr 172.21.15.7:0/4031025061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.270+0000 7f63177fe640 1 --2- 172.21.15.7:0/4031025061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fe04480 0x55b32fe06910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.270+0000 7f6320612740 1 -- 172.21.15.7:0/4031025061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32fd63b10 con 0x55b32fdff310 2024-03-26T08:53:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.270+0000 7f6320612740 1 -- 172.21.15.7:0/4031025061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32fd0c900 con 0x55b32fdfe920 2024-03-26T08:53:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.270+0000 7f6320612740 1 -- 172.21.15.7:0/4031025061 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32fd5fbb0 con 0x55b32fe04480 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f6316ffd640 1 -- 172.21.15.7:0/4031025061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fe04480 msgr2=0x55b32fe06910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f6316ffd640 1 --2- 172.21.15.7:0/4031025061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fe04480 0x55b32fe06910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f6316ffd640 1 -- 172.21.15.7:0/4031025061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdff310 msgr2=0x55b32fe03e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f6316ffd640 1 --2- 172.21.15.7:0/4031025061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdff310 0x55b32fe03e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f6316ffd640 1 -- 172.21.15.7:0/4031025061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b32fa78210 con 0x55b32fdfe920 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f6316ffd640 1 --2- 172.21.15.7:0/4031025061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdfe920 0x55b32fdfed40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f630002c910 tx=0x7f63000bd050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3480c5d89277b33c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f63167fc640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f63040a2a00 0x7f63040a4ec0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f63181182b0 tx=0x7f6310009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f630dffb640 1 -- 172.21.15.7:0/4031025061 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630004b070 con 0x55b32fdfe920 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f630dffb640 1 -- 172.21.15.7:0/4031025061 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6300005690 con 0x55b32fdfe920 2024-03-26T08:53:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.271+0000 7f630dffb640 1 -- 172.21.15.7:0/4031025061 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63000c5cb0 con 0x55b32fdfe920 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.272+0000 7f6320612740 1 -- 172.21.15.7:0/4031025061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdfe920 msgr2=0x55b32fdfed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.272+0000 7f6320612740 1 --2- 172.21.15.7:0/4031025061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdfe920 0x55b32fdfed40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f630002c910 tx=0x7f63000bd050 comp rx=0 tx=0).stop 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.273+0000 7f6320612740 1 -- 172.21.15.7:0/4031025061 shutdown_connections 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.273+0000 7f6320612740 1 --2- 172.21.15.7:0/4031025061 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fe04480 0x55b32fe06910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.273+0000 7f6320612740 1 --2- 172.21.15.7:0/4031025061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdff310 0x55b32fe03e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.273+0000 7f6320612740 1 --2- 172.21.15.7:0/4031025061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdfe920 0x55b32fdfed40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.273+0000 7f6320612740 1 -- 172.21.15.7:0/4031025061 >> 172.21.15.7:0/4031025061 conn(0x55b32fdf9e90 msgr2=0x55b32fdfa290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.273+0000 7f6320612740 1 -- 172.21.15.7:0/4031025061 shutdown_connections 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.273+0000 7f6320612740 1 -- 172.21.15.7:0/4031025061 wait complete. 2024-03-26T08:53:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.274+0000 7f6320612740 1 -- start start 2024-03-26T08:53:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.274+0000 7f6320612740 1 Processor -- start 2024-03-26T08:53:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.274+0000 7f6320612740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdfe920 0x55b32fe08940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.275+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdfe920 0x55b32fe08940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.275+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdfe920 0x55b32fe08940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49734/0 (socket says 172.21.15.7:49734) 2024-03-26T08:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.275+0000 7f6316ffd640 1 -- 172.21.15.7:0/1988343523 learned_addr learned my addr 172.21.15.7:0/1988343523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.275+0000 7f6320612740 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdff310 0x55b32fe08e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.275+0000 7f6320612740 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fe04480 0x55b32fe0fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.275+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32fe071f0 con 0x55b32fdfe920 2024-03-26T08:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.276+0000 7f63177fe640 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fe04480 0x55b32fe0fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.276+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32fd1fd50 con 0x55b32fe04480 2024-03-26T08:53:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.276+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32fe07370 con 0x55b32fdff310 2024-03-26T08:53:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.276+0000 7f6320612740 10 client.0 _ll_register_callbacks cb 0x55b32fd1f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.276+0000 7f63177fe640 1 -- 172.21.15.7:0/1988343523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdff310 msgr2=0x55b32fe08e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:53:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.276+0000 7f63177fe640 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdff310 0x55b32fe08e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.276+0000 7f63177fe640 1 -- 172.21.15.7:0/1988343523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdfe920 msgr2=0x55b32fe08940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.276+0000 7f63177fe640 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdfe920 0x55b32fe08940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.276+0000 7f63177fe640 1 -- 172.21.15.7:0/1988343523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b32fb44620 con 0x55b32fe04480 2024-03-26T08:53:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.277+0000 7f63177fe640 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fe04480 0x55b32fe0fec0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f630800f9b0 tx=0x7f630800fe70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c04af2d9f31df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.278+0000 7f62f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:53:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.278+0000 7f62f6ffd640 1 -- 172.21.15.7:0/1988343523 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630800dc50 con 0x55b32fe04480 2024-03-26T08:53:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.278+0000 7f62f6ffd640 1 -- 172.21.15.7:0/1988343523 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f630800ddf0 con 0x55b32fe04480 2024-03-26T08:53:04.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.278+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b32fd71050 con 0x55b32fe04480 2024-03-26T08:53:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.280+0000 7f62f6ffd640 1 -- 172.21.15.7:0/1988343523 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6308003970 con 0x55b32fe04480 2024-03-26T08:53:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.280+0000 7f6320612740 10 client.28010 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:04.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.280+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b32fe13080 con 0x55b32fe04480 2024-03-26T08:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.280+0000 7f62f6ffd640 1 -- 172.21.15.7:0/1988343523 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6308004020 con 0x55b32fe04480 2024-03-26T08:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f62f6ffd640 1 -- 172.21.15.7:0/1988343523 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6308010510 con 0x55b32fe04480 2024-03-26T08:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f6320612740 10 client.28010 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f6320612740 10 client.28010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f6320612740 20 client.28010 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f6320612740 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b32fe14710 0x55b32fe16b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f6320612740 4 client.28010 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b32fd71050 con 0x55b32fe14710 2024-03-26T08:53:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f6316ffd640 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b32fe14710 0x55b32fe16b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f6316ffd640 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b32fe14710 0x55b32fe16b30 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3c42bc9f947dd9 server_cookie=3327d4a92b56ab9f in_seq=0 out_seq=0 2024-03-26T08:53:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.281+0000 7f62f6ffd640 10 client.28010 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.283+0000 7f62f6ffd640 1 -- 172.21.15.7:0/1988343523 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b32fd71050 con 0x55b32fe14710 2024-03-26T08:53:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.284+0000 7f62f6ffd640 10 client.28010 handle_command_reply: tid=0 2024-03-26T08:53:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.284+0000 7f6320612740 1 client.28010 shutdown 2024-03-26T08:53:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.284+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fe04480 msgr2=0x55b32fe0fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.285+0000 7f6320612740 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fe04480 0x55b32fe0fec0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f630800f9b0 tx=0x7f630800fe70 comp rx=0 tx=0).stop 2024-03-26T08:53:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.285+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 shutdown_connections 2024-03-26T08:53:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.285+0000 7f6320612740 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b32fe14710 0x55b32fe16b30 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.285+0000 7f6320612740 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fe04480 0x55b32fe0fec0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.285+0000 7f6320612740 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdff310 0x55b32fe08e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.285+0000 7f6320612740 1 --2- 172.21.15.7:0/1988343523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdfe920 0x55b32fe08940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.285+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 >> 172.21.15.7:0/1988343523 conn(0x55b32fdf9e90 msgr2=0x55b32fdf9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.285+0000 7f62f6ffd640 0 client.28010 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.285+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 shutdown_connections 2024-03-26T08:53:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.287+0000 7f6320612740 1 -- 172.21.15.7:0/1988343523 wait complete. 2024-03-26T08:53:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.287+0000 7f6320612740 20 client.28010 trim_cache size 0 max 16384 2024-03-26T08:53:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.314+0000 7f6320612740 1 Processor -- start 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.314+0000 7f6320612740 1 -- start start 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6320612740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fdfed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6320612740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdfe920 0x55b32fe0fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6320612740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 0x55b32fe00590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6320612740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f8010980 con 0x55b32fdf9e90 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6320612740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f800e480 con 0x55b32fdff310 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6320612740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f800c500 con 0x55b32fdfe920 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fdfed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fdfed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49738/0 (socket says 172.21.15.7:49738) 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6316ffd640 1 -- 172.21.15.7:0/1711953630 learned_addr learned my addr 172.21.15.7:0/1711953630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f63177fe640 1 --2- 172.21.15.7:0/1711953630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 0x55b32fe00590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6316ffd640 1 -- 172.21.15.7:0/1711953630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdfe920 msgr2=0x55b32fe0fe80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6316ffd640 1 --2- 172.21.15.7:0/1711953630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdfe920 0x55b32fe0fe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6316ffd640 1 -- 172.21.15.7:0/1711953630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 msgr2=0x55b32fe00590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6316ffd640 1 --2- 172.21.15.7:0/1711953630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 0x55b32fe00590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.315+0000 7f6316ffd640 1 -- 172.21.15.7:0/1711953630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b32fe03860 con 0x55b32fdf9e90 2024-03-26T08:53:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.316+0000 7f6316ffd640 1 --2- 172.21.15.7:0/1711953630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fdfed20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6300004d80 tx=0x7f63000bcf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58422d97e0775711 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.316+0000 7f630d7fa640 1 -- 172.21.15.7:0/1711953630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63000da070 con 0x55b32fdf9e90 2024-03-26T08:53:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.316+0000 7f630d7fa640 1 -- 172.21.15.7:0/1711953630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63000b9af0 con 0x55b32fdf9e90 2024-03-26T08:53:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.316+0000 7f630d7fa640 1 -- 172.21.15.7:0/1711953630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630004b040 con 0x55b32fdf9e90 2024-03-26T08:53:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.317+0000 7f6320612740 1 -- 172.21.15.7:0/1711953630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 msgr2=0x55b32fdfed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.317+0000 7f6320612740 1 --2- 172.21.15.7:0/1711953630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fdfed20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6300004d80 tx=0x7f63000bcf70 comp rx=0 tx=0).stop 2024-03-26T08:53:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.317+0000 7f6320612740 1 -- 172.21.15.7:0/1711953630 shutdown_connections 2024-03-26T08:53:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.317+0000 7f6320612740 1 --2- 172.21.15.7:0/1711953630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 0x55b32fe00590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.317+0000 7f6320612740 1 --2- 172.21.15.7:0/1711953630 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdfe920 0x55b32fe0fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.317+0000 7f6320612740 1 --2- 172.21.15.7:0/1711953630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fdfed20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.317+0000 7f6320612740 1 -- 172.21.15.7:0/1711953630 >> 172.21.15.7:0/1711953630 conn(0x55b32fe03f50 msgr2=0x55b32fdfa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.317+0000 7f6320612740 1 -- 172.21.15.7:0/1711953630 shutdown_connections 2024-03-26T08:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.318+0000 7f6320612740 1 -- 172.21.15.7:0/1711953630 wait complete. 2024-03-26T08:53:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.318+0000 7f6320612740 1 -- start start 2024-03-26T08:53:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.318+0000 7f6320612740 1 Processor -- start 2024-03-26T08:53:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6320612740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fe0d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fe0d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6316ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fe0d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49752/0 (socket says 172.21.15.7:49752) 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6320612740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdfe920 0x55b32fe0d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6316ffd640 1 -- 172.21.15.7:0/3485480819 learned_addr learned my addr 172.21.15.7:0/3485480819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6320612740 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 0x55b32fe0ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f800cb00 con 0x55b32fdf9e90 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f800de20 con 0x55b32fdff310 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f800d160 con 0x55b32fdfe920 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f63167fc640 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdfe920 0x55b32fe0d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f63177fe640 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 0x55b32fe0ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.319+0000 7f6320612740 10 client.0 _ll_register_callbacks cb 0x7f6308007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f6316ffd640 1 -- 172.21.15.7:0/3485480819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdfe920 msgr2=0x55b32fe0d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f6316ffd640 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdfe920 0x55b32fe0d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f6316ffd640 1 -- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 msgr2=0x55b32fe0ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f6316ffd640 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 0x55b32fe0ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f6316ffd640 1 -- 172.21.15.7:0/3485480819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b32fe0eff0 con 0x55b32fdf9e90 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f63177fe640 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 0x55b32fe0ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f6316ffd640 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fe0d4b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f63000b9e50 tx=0x7f63000b9060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb77f83834b5c962 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f62f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f62f6ffd640 1 -- 172.21.15.7:0/3485480819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63000da050 con 0x55b32fdf9e90 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f62f6ffd640 1 -- 172.21.15.7:0/3485480819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f63000035f0 con 0x55b32fdf9e90 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.320+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63000b9af0 con 0x55b32fdf9e90 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f62f6ffd640 1 -- 172.21.15.7:0/3485480819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63000bdc10 con 0x55b32fdf9e90 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f6320612740 10 client.18207 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b32fe102a0 con 0x55b32fdf9e90 2024-03-26T08:53:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f62f6ffd640 1 -- 172.21.15.7:0/3485480819 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f63000b9af0 con 0x55b32fdf9e90 2024-03-26T08:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f62f6ffd640 1 -- 172.21.15.7:0/3485480819 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f63000bc960 con 0x55b32fdf9e90 2024-03-26T08:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f6320612740 10 client.18207 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f6320612740 10 client.18207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f6320612740 20 client.18207 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f6320612740 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b32fe04480 0x55b32fe72150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f6320612740 4 client.18207 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b32fd6b3a0 con 0x55b32fe04480 2024-03-26T08:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.321+0000 7f63167fc640 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b32fe04480 0x55b32fe72150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.322+0000 7f62f6ffd640 10 client.18207 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.322+0000 7f63167fc640 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b32fe04480 0x55b32fe72150 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e5059fdc14812b4 server_cookie=8bd2c39bc507786a in_seq=0 out_seq=0 2024-03-26T08:53:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.338+0000 7f62f6ffd640 1 -- 172.21.15.7:0/3485480819 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b32fd6b3a0 con 0x55b32fe04480 2024-03-26T08:53:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.338+0000 7f62f6ffd640 10 client.18207 handle_command_reply: tid=0 2024-03-26T08:53:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.338+0000 7f6320612740 1 client.18207 shutdown 2024-03-26T08:53:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:53:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:53:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:53:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:53:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.338+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 msgr2=0x55b32fe0d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:04.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.338+0000 7f6320612740 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fe0d4b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f63000b9e50 tx=0x7f63000b9060 comp rx=0 tx=0).stop 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.339+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 shutdown_connections 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.339+0000 7f6320612740 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b32fe04480 0x55b32fe72150 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.339+0000 7f6320612740 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b32fdff310 0x55b32fe0ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.339+0000 7f6320612740 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b32fdfe920 0x55b32fe0d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.339+0000 7f6320612740 1 --2- 172.21.15.7:0/3485480819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b32fdf9e90 0x55b32fe0d4b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.339+0000 7f62f6ffd640 0 client.18207 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.339+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 >> 172.21.15.7:0/3485480819 conn(0x55b32fe03f50 msgr2=0x55b32fe00590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.339+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 shutdown_connections 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.339+0000 7f6320612740 1 -- 172.21.15.7:0/3485480819 wait complete. 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.340+0000 7f6320612740 20 client.18207 trim_cache size 0 max 16384 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 -- 172.21.15.7:0/766117803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f63040a2a00 msgr2=0x7f63040a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f63040a2a00 0x7f63040a4ec0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f63181182b0 tx=0x7f6310009110 comp rx=0 tx=0).stop 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 -- 172.21.15.7:0/766117803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63181086a0 msgr2=0x7f6318116da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63181086a0 0x7f6318116da0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f630002efd0 tx=0x7f6300004040 comp rx=0 tx=0).stop 2024-03-26T08:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 -- 172.21.15.7:0/766117803 shutdown_connections 2024-03-26T08:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f63040a2a00 0x7f63040a4ec0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631810d9c0 0x7f63181b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f631810b010 0x7f63181172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 --2- 172.21.15.7:0/766117803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63181086a0 0x7f6318116da0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 -- 172.21.15.7:0/766117803 >> 172.21.15.7:0/766117803 conn(0x7f631806e6b0 msgr2=0x7f6318108e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 -- 172.21.15.7:0/766117803 shutdown_connections 2024-03-26T08:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:04.341+0000 7f631da27640 1 -- 172.21.15.7:0/766117803 wait complete. 2024-03-26T08:53:05.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:05 smithi016 ceph-mon[35851]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 MiB/s wr, 574 op/s 2024-03-26T08:53:06.046 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:05 smithi007 ceph-mon[32462]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 MiB/s wr, 574 op/s 2024-03-26T08:53:06.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:05 smithi044 ceph-mon[36461]: pgmap v680: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 MiB/s wr, 574 op/s 2024-03-26T08:53:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 635.0 s, 1599.6 tps, lat 62.615 ms stddev 63.863 2024-03-26T08:53:07.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:07 smithi016 ceph-mon[35851]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.5 MiB/s wr, 589 op/s 2024-03-26T08:53:08.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:07 smithi007 ceph-mon[32462]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.5 MiB/s wr, 589 op/s 2024-03-26T08:53:08.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:07 smithi044 ceph-mon[36461]: pgmap v681: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.5 MiB/s wr, 589 op/s 2024-03-26T08:53:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:09 smithi016 ceph-mon[35851]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.3 MiB/s wr, 581 op/s 2024-03-26T08:53:10.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:09 smithi007 ceph-mon[32462]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.3 MiB/s wr, 581 op/s 2024-03-26T08:53:10.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:09 smithi044 ceph-mon[36461]: pgmap v682: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.3 MiB/s wr, 581 op/s 2024-03-26T08:53:11.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 640.0 s, 1444.6 tps, lat 68.743 ms stddev 71.703 2024-03-26T08:53:11.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:11 smithi016 ceph-mon[35851]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 MiB/s wr, 623 op/s 2024-03-26T08:53:12.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:11 smithi007 ceph-mon[32462]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 MiB/s wr, 623 op/s 2024-03-26T08:53:12.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:11 smithi044 ceph-mon[36461]: pgmap v683: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 MiB/s wr, 623 op/s 2024-03-26T08:53:12.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:53:12.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.863+0000 7ff033122640 1 -- 172.21.15.7:0/2533714196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c10b010 msgr2=0x7ff02c10d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.863+0000 7ff033122640 1 --2- 172.21.15.7:0/2533714196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c10b010 0x7ff02c10d480 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009860 tx=0x7ff01c02bbb0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.863+0000 7ff033122640 1 -- 172.21.15.7:0/2533714196 shutdown_connections 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.863+0000 7ff033122640 1 --2- 172.21.15.7:0/2533714196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff02c10d9c0 0x7ff02c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.863+0000 7ff033122640 1 --2- 172.21.15.7:0/2533714196 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c10b010 0x7ff02c10d480 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.863+0000 7ff033122640 1 --2- 172.21.15.7:0/2533714196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1086a0 0x7ff02c10aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.863+0000 7ff033122640 1 -- 172.21.15.7:0/2533714196 >> 172.21.15.7:0/2533714196 conn(0x7ff02c06e640 msgr2=0x7ff02c070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.864+0000 7ff033122640 1 -- 172.21.15.7:0/2533714196 shutdown_connections 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.864+0000 7ff033122640 1 -- 172.21.15.7:0/2533714196 wait complete. 2024-03-26T08:53:12.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.865+0000 7ff033122640 1 Processor -- start 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.865+0000 7ff033122640 1 -- start start 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.865+0000 7ff033122640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1086a0 0x7ff02c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.865+0000 7ff033122640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff02c10b010 0x7ff02c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.865+0000 7ff033122640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c10d9c0 0x7ff02c1a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.865+0000 7ff033122640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c111900 con 0x7ff02c10b010 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.865+0000 7ff033122640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c111780 con 0x7ff02c1086a0 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff0316bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c10d9c0 0x7ff02c1a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff0316bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c10d9c0 0x7ff02c1a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45730/0 (socket says 172.21.15.7:45730) 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff0316bf640 1 -- 172.21.15.7:0/2473672679 learned_addr learned my addr 172.21.15.7:0/2473672679 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff033122640 1 -- 172.21.15.7:0/2473672679 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c111a80 con 0x7ff02c10d9c0 2024-03-26T08:53:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff02bfff640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff02c10b010 0x7ff02c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff02bfff640 1 -- 172.21.15.7:0/2473672679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c10d9c0 msgr2=0x7ff02c1a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff02bfff640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c10d9c0 0x7ff02c1a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff02bfff640 1 -- 172.21.15.7:0/2473672679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1086a0 msgr2=0x7ff02c19ddc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:53:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff02bfff640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1086a0 0x7ff02c19ddc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.866+0000 7ff02bfff640 1 -- 172.21.15.7:0/2473672679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c1a59d0 con 0x7ff02c10b010 2024-03-26T08:53:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.867+0000 7ff02bfff640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff02c10b010 0x7ff02c19e300 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009780 tx=0x7ff01c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.867+0000 7ff029ffb640 1 -- 172.21.15.7:0/2473672679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c039040 con 0x7ff02c10b010 2024-03-26T08:53:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.867+0000 7ff033122640 1 -- 172.21.15.7:0/2473672679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c1a5c60 con 0x7ff02c10b010 2024-03-26T08:53:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.867+0000 7ff029ffb640 1 -- 172.21.15.7:0/2473672679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff01c044070 con 0x7ff02c10b010 2024-03-26T08:53:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.867+0000 7ff029ffb640 1 -- 172.21.15.7:0/2473672679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c032700 con 0x7ff02c10b010 2024-03-26T08:53:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.867+0000 7ff033122640 1 -- 172.21.15.7:0/2473672679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c1a6170 con 0x7ff02c10b010 2024-03-26T08:53:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff035ded740 1 Processor -- start 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff035ded740 1 -- start start 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff035ded740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d89e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff035ded740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff035ded740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 0x55de7d8a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff035ded740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7d8033d0 con 0x55de7d89e790 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff035ded740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7d7abad0 con 0x55de7d8a3900 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff035ded740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7d7ff470 con 0x55de7d89dda0 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff029ffb640 1 -- 172.21.15.7:0/2473672679 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff01c02c930 con 0x7ff02c10b010 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.871+0000 7ff0316bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d89e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.872+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 0x55de7d8a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.872+0000 7ff0316bf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d89e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45736/0 (socket says 172.21.15.7:45736) 2024-03-26T08:53:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.872+0000 7ff0316bf640 1 -- 172.21.15.7:0/471996472 learned_addr learned my addr 172.21.15.7:0/471996472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.872+0000 7ff030ebe640 1 --2- 172.21.15.7:0/471996472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.872+0000 7ff0316bf640 1 -- 172.21.15.7:0/471996472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 msgr2=0x55de7d8a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.872+0000 7ff0316bf640 1 --2- 172.21.15.7:0/471996472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 0x55de7d8a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.872+0000 7ff0316bf640 1 -- 172.21.15.7:0/471996472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 msgr2=0x55de7d8a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.872+0000 7ff0316bf640 1 --2- 172.21.15.7:0/471996472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.872+0000 7ff0316bf640 1 -- 172.21.15.7:0/471996472 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7d5e17d0 con 0x55de7d89dda0 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.873+0000 7ff029ffb640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff0140a2880 0x7ff0140a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.873+0000 7ff0316bf640 1 --2- 172.21.15.7:0/471996472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d89e1c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff02001cd60 tx=0x7ff0200217b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89bd301fa052606f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.873+0000 7ff030ebe640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff0140a2880 0x7ff0140a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.873+0000 7ff029ffb640 1 -- 172.21.15.7:0/2473672679 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff01c0f0b60 con 0x7ff02c10b010 2024-03-26T08:53:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.873+0000 7ff012ffd640 1 -- 172.21.15.7:0/471996472 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02001ce50 con 0x55de7d89dda0 2024-03-26T08:53:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.873+0000 7ff012ffd640 1 -- 172.21.15.7:0/471996472 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff020022b40 con 0x55de7d89dda0 2024-03-26T08:53:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.873+0000 7ff012ffd640 1 -- 172.21.15.7:0/471996472 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02002dd80 con 0x55de7d89dda0 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff035ded740 1 -- 172.21.15.7:0/471996472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 msgr2=0x55de7d89e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff035ded740 1 --2- 172.21.15.7:0/471996472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d89e1c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff02001cd60 tx=0x7ff0200217b0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff030ebe640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff0140a2880 0x7ff0140a4d40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55de7d89eb90 tx=0x7ff02400a4f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff035ded740 1 -- 172.21.15.7:0/471996472 shutdown_connections 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff035ded740 1 --2- 172.21.15.7:0/471996472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 0x55de7d8a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff035ded740 1 --2- 172.21.15.7:0/471996472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff035ded740 1 --2- 172.21.15.7:0/471996472 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d89e1c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff035ded740 1 -- 172.21.15.7:0/471996472 >> 172.21.15.7:0/471996472 conn(0x55de7d899310 msgr2=0x55de7d899710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff035ded740 1 -- 172.21.15.7:0/471996472 shutdown_connections 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.874+0000 7ff035ded740 1 -- 172.21.15.7:0/471996472 wait complete. 2024-03-26T08:53:12.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff035ded740 1 -- start start 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff035ded740 1 Processor -- start 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff035ded740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89dda0 0x55de7d8a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff035ded740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff035ded740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d8a3900 0x55de7d8af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff035ded740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7d8a68b0 con 0x55de7d89e790 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff035ded740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7d7bf060 con 0x55de7d89dda0 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff035ded740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de7d8a6a30 con 0x55de7d8a3900 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff035ded740 10 client.0 _ll_register_callbacks cb 0x55de7d7a16a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39862/0 (socket says 172.21.15.7:39862) 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff02bfff640 1 -- 172.21.15.7:0/1025659229 learned_addr learned my addr 172.21.15.7:0/1025659229 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.875+0000 7ff030ebe640 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89dda0 0x55de7d8a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff0316bf640 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d8a3900 0x55de7d8af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff02bfff640 1 -- 172.21.15.7:0/1025659229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d8a3900 msgr2=0x55de7d8af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff02bfff640 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d8a3900 0x55de7d8af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff02bfff640 1 -- 172.21.15.7:0/1025659229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89dda0 msgr2=0x55de7d8a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff02bfff640 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89dda0 0x55de7d8a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff02bfff640 1 -- 172.21.15.7:0/1025659229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de7d5173f0 con 0x55de7d89e790 2024-03-26T08:53:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff02bfff640 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a87e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0c5770 tx=0x7ff01c0f4110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473459445ed1eff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff00bfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff00bfff640 1 -- 172.21.15.7:0/1025659229 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0b9710 con 0x55de7d89e790 2024-03-26T08:53:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff035ded740 1 -- 172.21.15.7:0/1025659229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff020022b40 con 0x55de7d89e790 2024-03-26T08:53:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff00bfff640 1 -- 172.21.15.7:0/1025659229 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff01c0c5810 con 0x55de7d89e790 2024-03-26T08:53:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff00bfff640 1 -- 172.21.15.7:0/1025659229 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff01c0bb640 con 0x55de7d89e790 2024-03-26T08:53:12.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.876+0000 7ff035ded740 10 client.18219 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff035ded740 1 -- 172.21.15.7:0/1025659229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de7d4d70f0 con 0x55de7d89e790 2024-03-26T08:53:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff00bfff640 1 -- 172.21.15.7:0/1025659229 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff020022b40 con 0x55de7d89e790 2024-03-26T08:53:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff00bfff640 1 -- 172.21.15.7:0/1025659229 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff01c03c030 con 0x55de7d89e790 2024-03-26T08:53:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff035ded740 10 client.18219 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff035ded740 10 client.18219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff035ded740 20 client.18219 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff035ded740 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de7d8b3ca0 0x55de7d8b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff035ded740 4 client.18219 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff035ded740 1 -- 172.21.15.7:0/1025659229 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de7d810910 con 0x55de7d8b3ca0 2024-03-26T08:53:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.877+0000 7ff030ebe640 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de7d8b3ca0 0x55de7d8b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.878+0000 7ff00bfff640 10 client.18219 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.878+0000 7ff030ebe640 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de7d8b3ca0 0x55de7d8b60c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c597d8ebd1e891be server_cookie=c1665c9cc8fe319b in_seq=0 out_seq=0 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff00bfff640 1 -- 172.21.15.7:0/1025659229 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55de7d810910 con 0x55de7d8b3ca0 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff00bfff640 10 client.18219 handle_command_reply: tid=0 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 client.18219 shutdown 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 -- 172.21.15.7:0/1025659229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 msgr2=0x55de7d8a87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a87e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0c5770 tx=0x7ff01c0f4110 comp rx=0 tx=0).stop 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 -- 172.21.15.7:0/1025659229 shutdown_connections 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de7d8b3ca0 0x55de7d8b60c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d8a3900 0x55de7d8af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89e790 0x55de7d8a87e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 --2- 172.21.15.7:0/1025659229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89dda0 0x55de7d8a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff00bfff640 0 client.18219 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 -- 172.21.15.7:0/1025659229 >> 172.21.15.7:0/1025659229 conn(0x55de7d899310 msgr2=0x55de7d898530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 -- 172.21.15.7:0/1025659229 shutdown_connections 2024-03-26T08:53:12.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 1 -- 172.21.15.7:0/1025659229 wait complete. 2024-03-26T08:53:12.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.879+0000 7ff035ded740 20 client.18219 trim_cache size 0 max 16384 2024-03-26T08:53:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.901+0000 7ff035ded740 1 Processor -- start 2024-03-26T08:53:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.901+0000 7ff035ded740 1 -- start start 2024-03-26T08:53:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.901+0000 7ff035ded740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89dda0 0x55de7d8a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.901+0000 7ff035ded740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89e790 0x55de7d8af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff035ded740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 0x55de7d8ba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff035ded740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c1b1430 con 0x55de7d89dda0 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff035ded740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c1aef30 con 0x55de7d8a3900 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff035ded740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c1acfb0 con 0x55de7d89e790 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff030ebe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89dda0 0x55de7d8a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff030ebe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89dda0 0x55de7d8a78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39870/0 (socket says 172.21.15.7:39870) 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff030ebe640 1 -- 172.21.15.7:0/1315075741 learned_addr learned my addr 172.21.15.7:0/1315075741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff02bfff640 1 --2- 172.21.15.7:0/1315075741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89e790 0x55de7d8af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff0316bf640 1 --2- 172.21.15.7:0/1315075741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 0x55de7d8ba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff030ebe640 1 -- 172.21.15.7:0/1315075741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89e790 msgr2=0x55de7d8af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff030ebe640 1 --2- 172.21.15.7:0/1315075741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89e790 0x55de7d8af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff030ebe640 1 -- 172.21.15.7:0/1315075741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 msgr2=0x55de7d8ba9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff030ebe640 1 --2- 172.21.15.7:0/1315075741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 0x55de7d8ba9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff030ebe640 1 -- 172.21.15.7:0/1315075741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de7d8a2f40 con 0x55de7d89dda0 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff030ebe640 1 --2- 172.21.15.7:0/1315075741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89dda0 0x55de7d8a78c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55de7d89f200 tx=0x7ff024012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838fc9f41bea8f06 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff0127fc640 1 -- 172.21.15.7:0/1315075741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024016070 con 0x55de7d89dda0 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff0127fc640 1 -- 172.21.15.7:0/1315075741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff02402d070 con 0x55de7d89dda0 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.902+0000 7ff0127fc640 1 -- 172.21.15.7:0/1315075741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02401ed20 con 0x55de7d89dda0 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.903+0000 7ff035ded740 1 -- 172.21.15.7:0/1315075741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89dda0 msgr2=0x55de7d8a78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.903+0000 7ff035ded740 1 --2- 172.21.15.7:0/1315075741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89dda0 0x55de7d8a78c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55de7d89f200 tx=0x7ff024012000 comp rx=0 tx=0).stop 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.903+0000 7ff035ded740 1 -- 172.21.15.7:0/1315075741 shutdown_connections 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.903+0000 7ff035ded740 1 --2- 172.21.15.7:0/1315075741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d8a3900 0x55de7d8ba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.903+0000 7ff035ded740 1 --2- 172.21.15.7:0/1315075741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89e790 0x55de7d8af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.903+0000 7ff035ded740 1 --2- 172.21.15.7:0/1315075741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d89dda0 0x55de7d8a78c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.903+0000 7ff035ded740 1 -- 172.21.15.7:0/1315075741 >> 172.21.15.7:0/1315075741 conn(0x55de7d899310 msgr2=0x55de7d89fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.903+0000 7ff035ded740 1 -- 172.21.15.7:0/1315075741 shutdown_connections 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.903+0000 7ff035ded740 1 -- 172.21.15.7:0/1315075741 wait complete. 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.904+0000 7ff035ded740 1 -- start start 2024-03-26T08:53:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.904+0000 7ff035ded740 1 Processor -- start 2024-03-26T08:53:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.904+0000 7ff035ded740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d8ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.904+0000 7ff035ded740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89e790 0x55de7d8addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.904+0000 7ff035ded740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d8a3900 0x55de7d8aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.904+0000 7ff030ebe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d8ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff030ebe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d8ad880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45756/0 (socket says 172.21.15.7:45756) 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff030ebe640 1 -- 172.21.15.7:0/3423856175 learned_addr learned my addr 172.21.15.7:0/3423856175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89e790 0x55de7d8addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff0316bf640 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d8a3900 0x55de7d8aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.904+0000 7ff035ded740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c1ad5b0 con 0x55de7d8a3900 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c1ae8d0 con 0x55de7d89e790 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c1adc10 con 0x55de7d89dda0 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff035ded740 10 client.0 _ll_register_callbacks cb 0x7ff01c0f42d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff0316bf640 1 -- 172.21.15.7:0/3423856175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 msgr2=0x55de7d8ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff0316bf640 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d8ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff0316bf640 1 -- 172.21.15.7:0/3423856175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89e790 msgr2=0x55de7d8addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff0316bf640 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89e790 0x55de7d8addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff0316bf640 1 -- 172.21.15.7:0/3423856175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de7d8af370 con 0x55de7d8a3900 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff0316bf640 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d8a3900 0x55de7d8aed80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff02001cd30 tx=0x7ff020005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce049c65696089a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.905+0000 7ff00bfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.906+0000 7ff00bfff640 1 -- 172.21.15.7:0/3423856175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02003c060 con 0x55de7d8a3900 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.906+0000 7ff00bfff640 1 -- 172.21.15.7:0/3423856175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff0200055f0 con 0x55de7d8a3900 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.906+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff02402d070 con 0x55de7d8a3900 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff00bfff640 1 -- 172.21.15.7:0/3423856175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff020004680 con 0x55de7d8a3900 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff035ded740 10 client.18225 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de7d90fe70 con 0x55de7d8a3900 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff00bfff640 1 -- 172.21.15.7:0/3423856175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff02402d070 con 0x55de7d8a3900 2024-03-26T08:53:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff00bfff640 1 -- 172.21.15.7:0/3423856175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff02001f040 con 0x55de7d8a3900 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff035ded740 10 client.18225 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff035ded740 10 client.18225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff035ded740 20 client.18225 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff035ded740 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de7d910360 0x55de7d9127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff035ded740 4 client.18225 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de7d8a8090 con 0x55de7d910360 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.907+0000 7ff030ebe640 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de7d910360 0x55de7d9127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff00bfff640 10 client.18225 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff030ebe640 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de7d910360 0x55de7d9127e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24ee0c24fd071ae0 server_cookie=8f95d1dd2bb51fb5 in_seq=0 out_seq=0 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff00bfff640 1 -- 172.21.15.7:0/3423856175 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55de7d8a8090 con 0x55de7d910360 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff00bfff640 10 client.18225 handle_command_reply: tid=0 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 client.18225 shutdown 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d8a3900 msgr2=0x55de7d8aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d8a3900 0x55de7d8aed80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff02001cd30 tx=0x7ff020005270 comp rx=0 tx=0).stop 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 shutdown_connections 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de7d910360 0x55de7d9127e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de7d8a3900 0x55de7d8aed80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de7d89e790 0x55de7d8addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff00bfff640 0 client.18225 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 --2- 172.21.15.7:0/3423856175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de7d89dda0 0x55de7d8ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 >> 172.21.15.7:0/3423856175 conn(0x55de7d899310 msgr2=0x55de7d89fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 shutdown_connections 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 1 -- 172.21.15.7:0/3423856175 wait complete. 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.908+0000 7ff035ded740 20 client.18225 trim_cache size 0 max 16384 2024-03-26T08:53:12.909 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:53:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:53:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:53:12.910 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 -- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff0140a2880 msgr2=0x7ff0140a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff0140a2880 0x7ff0140a4d40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x55de7d89eb90 tx=0x7ff02400a4f0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 -- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff02c10b010 msgr2=0x7ff02c19e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff02c10b010 0x7ff02c19e300 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009780 tx=0x7ff01c004430 comp rx=0 tx=0).stop 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 -- 172.21.15.7:0/2473672679 shutdown_connections 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff0140a2880 0x7ff0140a4d40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff02c10d9c0 0x7ff02c1a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff02c10b010 0x7ff02c19e300 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 --2- 172.21.15.7:0/2473672679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1086a0 0x7ff02c19ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 -- 172.21.15.7:0/2473672679 >> 172.21.15.7:0/2473672679 conn(0x7ff02c06e640 msgr2=0x7ff02c10b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 -- 172.21.15.7:0/2473672679 shutdown_connections 2024-03-26T08:53:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:12.909+0000 7ff033122640 1 -- 172.21.15.7:0/2473672679 wait complete. 2024-03-26T08:53:12.918 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:53:12.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:53:13.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.062+0000 7f71d41f3640 1 -- 172.21.15.7:0/479799729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71cc108910 msgr2=0x7f71cc108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.062+0000 7f71d41f3640 1 --2- 172.21.15.7:0/479799729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71cc108910 0x7f71cc108d30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009900 tx=0x7f71c802bcc0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.063+0000 7f71d41f3640 1 -- 172.21.15.7:0/479799729 shutdown_connections 2024-03-26T08:53:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.063+0000 7f71d41f3640 1 --2- 172.21.15.7:0/479799729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc10e030 0x7f71cc110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.063+0000 7f71d41f3640 1 --2- 172.21.15.7:0/479799729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71cc10b700 0x7f71cc10daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.063+0000 7f71d41f3640 1 --2- 172.21.15.7:0/479799729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71cc108910 0x7f71cc108d30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.063+0000 7f71d41f3640 1 -- 172.21.15.7:0/479799729 >> 172.21.15.7:0/479799729 conn(0x7f71cc06e8f0 msgr2=0x7f71cc070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.063+0000 7f71d41f3640 1 -- 172.21.15.7:0/479799729 shutdown_connections 2024-03-26T08:53:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.063+0000 7f71d41f3640 1 -- 172.21.15.7:0/479799729 wait complete. 2024-03-26T08:53:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.064+0000 7f71d41f3640 1 Processor -- start 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.064+0000 7f71d41f3640 1 -- start start 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.064+0000 7f71d41f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71cc108910 0x7f71cc19e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.064+0000 7f71d41f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71cc10b700 0x7f71cc19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.064+0000 7f71d41f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc10e030 0x7f71cc1a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.064+0000 7f71d41f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc1139d0 con 0x7f71cc10e030 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.064+0000 7f71d41f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc113850 con 0x7f71cc10b700 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.064+0000 7f71d41f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc113b50 con 0x7f71cc108910 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d2790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc10e030 0x7f71cc1a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d2790640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc10e030 0x7f71cc1a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39888/0 (socket says 172.21.15.7:39888) 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d2790640 1 -- 172.21.15.7:0/518961158 learned_addr learned my addr 172.21.15.7:0/518961158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d178e640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71cc10b700 0x7f71cc19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d2790640 1 -- 172.21.15.7:0/518961158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71cc108910 msgr2=0x7f71cc19e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d2790640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71cc108910 0x7f71cc19e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d2790640 1 -- 172.21.15.7:0/518961158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71cc10b700 msgr2=0x7f71cc19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d2790640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71cc10b700 0x7f71cc19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d2790640 1 -- 172.21.15.7:0/518961158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc1b6ce0 con 0x7f71cc10e030 2024-03-26T08:53:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.065+0000 7f71d2790640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc10e030 0x7f71cc1a55f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f71c400b970 tx=0x7f71c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.066+0000 7f71c2ffd640 1 -- 172.21.15.7:0/518961158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4019070 con 0x7f71cc10e030 2024-03-26T08:53:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.066+0000 7f71c2ffd640 1 -- 172.21.15.7:0/518961158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71c400cb40 con 0x7f71cc10e030 2024-03-26T08:53:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.066+0000 7f71c2ffd640 1 -- 172.21.15.7:0/518961158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40076a0 con 0x7f71cc10e030 2024-03-26T08:53:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.068+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc1b6f70 con 0x7f71cc10e030 2024-03-26T08:53:13.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.068+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc1b7380 con 0x7f71cc10e030 2024-03-26T08:53:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.070+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc0703e0 con 0x7f71cc10e030 2024-03-26T08:53:13.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.073+0000 7f71c2ffd640 1 -- 172.21.15.7:0/518961158 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f71c40149b0 con 0x7f71cc10e030 2024-03-26T08:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.074+0000 7f71c2ffd640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b80a2a00 0x7f71b80a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.074+0000 7f71c2ffd640 1 -- 172.21.15.7:0/518961158 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f71c40cdf80 con 0x7f71cc10e030 2024-03-26T08:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.076+0000 7f71d1f8f640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b80a2a00 0x7f71b80a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.077+0000 7f71d1f8f640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b80a2a00 0x7f71b80a4ec0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005ab0 tx=0x7f71c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.080+0000 7f71c2ffd640 1 -- 172.21.15.7:0/518961158 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71c409afe0 con 0x7f71cc10e030 2024-03-26T08:53:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.374+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71cc02d070 con 0x7f71cc10e030 2024-03-26T08:53:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.374+0000 7f71c2ffd640 1 -- 172.21.15.7:0/518961158 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f71c409a8e0 con 0x7f71cc10e030 2024-03-26T08:53:13.371 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:53:13.371 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:53:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:53:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.377+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b80a2a00 msgr2=0x7f71b80a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.377+0000 7f71d41f3640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b80a2a00 0x7f71b80a4ec0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005ab0 tx=0x7f71c8000bc0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.377+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc10e030 msgr2=0x7f71cc1a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.377+0000 7f71d41f3640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc10e030 0x7f71cc1a55f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f71c400b970 tx=0x7f71c400be30 comp rx=0 tx=0).stop 2024-03-26T08:53:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.378+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 shutdown_connections 2024-03-26T08:53:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.378+0000 7f71d41f3640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71b80a2a00 0x7f71b80a4ec0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.378+0000 7f71d41f3640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc10e030 0x7f71cc1a55f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.378+0000 7f71d41f3640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71cc10b700 0x7f71cc19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.378+0000 7f71d41f3640 1 --2- 172.21.15.7:0/518961158 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f71cc108910 0x7f71cc19e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.378+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 >> 172.21.15.7:0/518961158 conn(0x7f71cc06e8f0 msgr2=0x7f71cc10bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.378+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 shutdown_connections 2024-03-26T08:53:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.378+0000 7f71d41f3640 1 -- 172.21.15.7:0/518961158 wait complete. 2024-03-26T08:53:13.386 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:53:13.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:53:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 -- 172.21.15.7:0/1093019326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c1073c0 msgr2=0x7fda8c109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 --2- 172.21.15.7:0/1093019326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c1073c0 0x7fda8c109810 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fda84009be0 tx=0x7fda8402bef0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 -- 172.21.15.7:0/1093019326 shutdown_connections 2024-03-26T08:53:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 --2- 172.21.15.7:0/1093019326 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c1073c0 0x7fda8c109810 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 --2- 172.21.15.7:0/1093019326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c104a90 0x7fda8c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 --2- 172.21.15.7:0/1093019326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda8c102160 0x7fda8c104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 -- 172.21.15.7:0/1093019326 >> 172.21.15.7:0/1093019326 conn(0x7fda8c0fbb70 msgr2=0x7fda8c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 -- 172.21.15.7:0/1093019326 shutdown_connections 2024-03-26T08:53:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 -- 172.21.15.7:0/1093019326 wait complete. 2024-03-26T08:53:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.541+0000 7fda936b2640 1 Processor -- start 2024-03-26T08:53:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda936b2640 1 -- start start 2024-03-26T08:53:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda936b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c102160 0x7fda8c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda936b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c104a90 0x7fda8c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda936b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda8c1073c0 0x7fda8c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda936b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c10d380 con 0x7fda8c1073c0 2024-03-26T08:53:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda936b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c10d200 con 0x7fda8c102160 2024-03-26T08:53:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda936b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c10d500 con 0x7fda8c104a90 2024-03-26T08:53:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c104a90 0x7fda8c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c104a90 0x7fda8c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45794/0 (socket says 172.21.15.7:45794) 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.542+0000 7fda90c4d640 1 -- 172.21.15.7:0/1499782834 learned_addr learned my addr 172.21.15.7:0/1499782834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.543+0000 7fda9144e640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c102160 0x7fda8c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.543+0000 7fda90c4d640 1 -- 172.21.15.7:0/1499782834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c102160 msgr2=0x7fda8c197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.543+0000 7fda90c4d640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c102160 0x7fda8c197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.543+0000 7fda90c4d640 1 -- 172.21.15.7:0/1499782834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda8c1073c0 msgr2=0x7fda8c19ef40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.543+0000 7fda90c4d640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda8c1073c0 0x7fda8c19ef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.543+0000 7fda90c4d640 1 -- 172.21.15.7:0/1499782834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c19f5b0 con 0x7fda8c104a90 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.543+0000 7fda90c4d640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c104a90 0x7fda8c197ed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00e9f0 tx=0x7fda7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.544+0000 7fda827fc640 1 -- 172.21.15.7:0/1499782834 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c00ccc0 con 0x7fda8c104a90 2024-03-26T08:53:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.544+0000 7fda827fc640 1 -- 172.21.15.7:0/1499782834 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda7c005500 con 0x7fda8c104a90 2024-03-26T08:53:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.544+0000 7fda827fc640 1 -- 172.21.15.7:0/1499782834 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c010690 con 0x7fda8c104a90 2024-03-26T08:53:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.545+0000 7fda936b2640 1 -- 172.21.15.7:0/1499782834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8c19f8a0 con 0x7fda8c104a90 2024-03-26T08:53:13.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.545+0000 7fda936b2640 1 -- 172.21.15.7:0/1499782834 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8c19fda0 con 0x7fda8c104a90 2024-03-26T08:53:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.547+0000 7fda827fc640 1 -- 172.21.15.7:0/1499782834 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fda7c0046c0 con 0x7fda8c104a90 2024-03-26T08:53:13.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.547+0000 7fda9637a740 1 Processor -- start 2024-03-26T08:53:13.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda9637a740 1 -- start start 2024-03-26T08:53:13.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda9637a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb2690 0x5626b2fb2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda9637a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fb7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda9637a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb81f0 0x5626b2fba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda9637a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b2f75610 con 0x5626b2fb3080 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda9637a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b2f183c0 con 0x5626b2fb2690 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda9637a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b2ec0d60 con 0x5626b2fb81f0 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda91c4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb2690 0x5626b2fb2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda9144e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fb7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb81f0 0x5626b2fba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda91c4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb2690 0x5626b2fb2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40852/0 (socket says 172.21.15.7:40852) 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb81f0 0x5626b2fba680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45806/0 (socket says 172.21.15.7:45806) 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda91c4f640 1 -- 172.21.15.7:0/144070672 learned_addr learned my addr 172.21.15.7:0/144070672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda827fc640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fda780a2950 0x7fda780a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda91c4f640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fda780a2950 0x7fda780a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda827fc640 1 -- 172.21.15.7:0/1499782834 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fda7c0cee00 con 0x7fda8c104a90 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda90c4d640 1 -- 172.21.15.7:0/144070672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb2690 msgr2=0x5626b2fb2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda90c4d640 1 --2- 172.21.15.7:0/144070672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb2690 0x5626b2fb2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda90c4d640 1 -- 172.21.15.7:0/144070672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 msgr2=0x5626b2fb7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda90c4d640 1 --2- 172.21.15.7:0/144070672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fb7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda90c4d640 1 -- 172.21.15.7:0/144070672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b2c2c3f0 con 0x5626b2fb81f0 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.548+0000 7fda9144e640 1 --2- 172.21.15.7:0/144070672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fb7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.549+0000 7fda90c4d640 1 --2- 172.21.15.7:0/144070672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb81f0 0x5626b2fba680 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fda7c09b460 tx=0x7fda7c004b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2360370a75faf01f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.549+0000 7fda91c4f640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fda780a2950 0x7fda780a4e10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fda84034fd0 tx=0x7fda84009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.549+0000 7fda777fe640 1 -- 172.21.15.7:0/144070672 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c09b0c0 con 0x5626b2fb81f0 2024-03-26T08:53:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.549+0000 7fda777fe640 1 -- 172.21.15.7:0/144070672 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda7c09be60 con 0x5626b2fb81f0 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.549+0000 7fda777fe640 1 -- 172.21.15.7:0/144070672 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0056b0 con 0x5626b2fb81f0 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 -- 172.21.15.7:0/144070672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb81f0 msgr2=0x5626b2fba680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 --2- 172.21.15.7:0/144070672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb81f0 0x5626b2fba680 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fda7c09b460 tx=0x7fda7c004b70 comp rx=0 tx=0).stop 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 -- 172.21.15.7:0/144070672 shutdown_connections 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 --2- 172.21.15.7:0/144070672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb81f0 0x5626b2fba680 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 --2- 172.21.15.7:0/144070672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fb7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 --2- 172.21.15.7:0/144070672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb2690 0x5626b2fb2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 -- 172.21.15.7:0/144070672 >> 172.21.15.7:0/144070672 conn(0x5626b2fadc00 msgr2=0x5626b2fae000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 -- 172.21.15.7:0/144070672 shutdown_connections 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 -- 172.21.15.7:0/144070672 wait complete. 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 -- start start 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.550+0000 7fda9637a740 1 Processor -- start 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b2ed4240 con 0x5626b2fb3080 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b2f14460 con 0x5626b2fb81f0 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626b2fbf4f0 con 0x5626b2fb2690 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39914/0 (socket says 172.21.15.7:39914) 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda90c4d640 1 -- 172.21.15.7:0/3829009 learned_addr learned my addr 172.21.15.7:0/3829009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda91c4f640 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9144e640 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda90c4d640 1 -- 172.21.15.7:0/3829009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 msgr2=0x5626b2fc0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda90c4d640 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda90c4d640 1 -- 172.21.15.7:0/3829009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 msgr2=0x5626b2fc83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda90c4d640 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda90c4d640 1 -- 172.21.15.7:0/3829009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626b2cf67d0 con 0x5626b2fb3080 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 10 client.0 _ll_register_callbacks cb 0x5626b2ed3c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda91c4f640 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 1 -- 172.21.15.7:0/3829009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda7c09be60 con 0x5626b2fb3080 2024-03-26T08:53:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda90c4d640 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc1380 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fda7c097bc0 tx=0x7fda7c09f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60ebcf90af9c918 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda74ff9640 10 client.18237 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda74ff9640 1 -- 172.21.15.7:0/3829009 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0b3020 con 0x5626b2fb3080 2024-03-26T08:53:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda74ff9640 1 -- 172.21.15.7:0/3829009 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fda7c0afd50 con 0x5626b2fb3080 2024-03-26T08:53:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda74ff9640 1 -- 172.21.15.7:0/3829009 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda7c09b0c0 con 0x5626b2fb3080 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 10 client.18237 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.551+0000 7fda9637a740 1 -- 172.21.15.7:0/3829009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x5626b2fcb590 con 0x5626b2fb3080 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda74ff9640 1 -- 172.21.15.7:0/3829009 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fda7c09abb0 con 0x5626b2fb3080 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda9637a740 10 client.18237 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda9637a740 10 client.18237 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda9637a740 20 client.18237 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda9637a740 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5626b2fcc8e0 0x5626b2fcecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda9637a740 4 client.18237 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda9637a740 1 -- 172.21.15.7:0/3829009 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626b2f25900 con 0x5626b2fcc8e0 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda9144e640 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5626b2fcc8e0 0x5626b2fcecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda9144e640 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5626b2fcc8e0 0x5626b2fcecd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b33ed5ecf7c5325 server_cookie=2283cc226b8f0e87 in_seq=0 out_seq=0 2024-03-26T08:53:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.552+0000 7fda74ff9640 10 client.18237 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:53:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.553+0000 7fda74ff9640 1 -- 172.21.15.7:0/3829009 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5626b2f25900 con 0x5626b2fcc8e0 2024-03-26T08:53:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.553+0000 7fda74ff9640 10 client.18237 handle_command_reply: tid=0 2024-03-26T08:53:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.553+0000 7fda9637a740 1 client.18237 shutdown 2024-03-26T08:53:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.553+0000 7fda9637a740 1 -- 172.21.15.7:0/3829009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 msgr2=0x5626b2fc1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.553+0000 7fda9637a740 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc1380 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fda7c097bc0 tx=0x7fda7c09f8b0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.554+0000 7fda9637a740 1 -- 172.21.15.7:0/3829009 shutdown_connections 2024-03-26T08:53:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.554+0000 7fda9637a740 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5626b2fcc8e0 0x5626b2fcecd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.554+0000 7fda9637a740 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.554+0000 7fda9637a740 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc1380 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.554+0000 7fda9637a740 1 --2- 172.21.15.7:0/3829009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.554+0000 7fda9637a740 1 -- 172.21.15.7:0/3829009 >> 172.21.15.7:0/3829009 conn(0x5626b2fadc00 msgr2=0x5626b2face20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.554+0000 7fda74ff9640 0 client.18237 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:53:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.554+0000 7fda9637a740 1 -- 172.21.15.7:0/3829009 shutdown_connections 2024-03-26T08:53:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.554+0000 7fda9637a740 1 -- 172.21.15.7:0/3829009 wait complete. 2024-03-26T08:53:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.555+0000 7fda9637a740 20 client.18237 trim_cache size 0 max 16384 2024-03-26T08:53:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.571+0000 7fda9637a740 1 Processor -- start 2024-03-26T08:53:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.571+0000 7fda9637a740 1 -- start start 2024-03-26T08:53:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda9637a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda9637a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda9637a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda9637a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1a8d60 con 0x5626b2fb3080 2024-03-26T08:53:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda9637a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1a6860 con 0x5626b2fb81f0 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda9637a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1a48e0 con 0x5626b2fb2690 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39926/0 (socket says 172.21.15.7:39926) 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda90c4d640 1 -- 172.21.15.7:0/1720362198 learned_addr learned my addr 172.21.15.7:0/1720362198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda9144e640 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda91c4f640 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda90c4d640 1 -- 172.21.15.7:0/1720362198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 msgr2=0x5626b2fc0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda90c4d640 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda90c4d640 1 -- 172.21.15.7:0/1720362198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 msgr2=0x5626b2fc1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda90c4d640 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda90c4d640 1 -- 172.21.15.7:0/1720362198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626b2fc1ce0 con 0x5626b2fb3080 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda9144e640 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda90c4d640 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc8380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0cf0f0 tx=0x7fda7c0b0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becaeb0ffcf3f385 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda76ffd640 1 -- 172.21.15.7:0/1720362198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0b3070 con 0x5626b2fb3080 2024-03-26T08:53:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda76ffd640 1 -- 172.21.15.7:0/1720362198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda7c09be60 con 0x5626b2fb3080 2024-03-26T08:53:13.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.572+0000 7fda76ffd640 1 -- 172.21.15.7:0/1720362198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c09a620 con 0x5626b2fb3080 2024-03-26T08:53:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.573+0000 7fda9637a740 1 -- 172.21.15.7:0/1720362198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 msgr2=0x5626b2fc8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.573+0000 7fda9637a740 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc8380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0cf0f0 tx=0x7fda7c0b0000 comp rx=0 tx=0).stop 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.573+0000 7fda9637a740 1 -- 172.21.15.7:0/1720362198 shutdown_connections 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.573+0000 7fda9637a740 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.573+0000 7fda9637a740 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc8380 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.573+0000 7fda9637a740 1 --2- 172.21.15.7:0/1720362198 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.573+0000 7fda9637a740 1 -- 172.21.15.7:0/1720362198 >> 172.21.15.7:0/1720362198 conn(0x5626b2fadc00 msgr2=0x5626b2fbf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.573+0000 7fda9637a740 1 -- 172.21.15.7:0/1720362198 shutdown_connections 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.573+0000 7fda9637a740 1 -- 172.21.15.7:0/1720362198 wait complete. 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9637a740 1 -- start start 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9637a740 1 Processor -- start 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9637a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9637a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9637a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9637a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1a4ee0 con 0x5626b2fb3080 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9637a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1a6200 con 0x5626b2fb81f0 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9637a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8c1a5540 con 0x5626b2fb2690 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9637a740 10 client.0 _ll_register_callbacks cb 0x7fda7c004a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda9144e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda90c4d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39940/0 (socket says 172.21.15.7:39940) 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda90c4d640 1 -- 172.21.15.7:0/3436590172 learned_addr learned my addr 172.21.15.7:0/3436590172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda91c4f640 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda90c4d640 1 -- 172.21.15.7:0/3436590172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 msgr2=0x5626b2fc5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda90c4d640 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda90c4d640 1 -- 172.21.15.7:0/3436590172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 msgr2=0x5626b2fc7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda90c4d640 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.574+0000 7fda90c4d640 1 -- 172.21.15.7:0/3436590172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626b2fc7780 con 0x5626b2fb3080 2024-03-26T08:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.575+0000 7fda90c4d640 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc61d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda7c097bc0 tx=0x7fda7c004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f54aa5e81d4ed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.575+0000 7fda74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.575+0000 7fda74ff9640 1 -- 172.21.15.7:0/3436590172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0a9020 con 0x5626b2fb3080 2024-03-26T08:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.576+0000 7fda74ff9640 1 -- 172.21.15.7:0/3436590172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fda7c09a830 con 0x5626b2fb3080 2024-03-26T08:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.576+0000 7fda9637a740 1 -- 172.21.15.7:0/3436590172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda7c09be60 con 0x5626b2fb3080 2024-03-26T08:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.577+0000 7fda74ff9640 1 -- 172.21.15.7:0/3436590172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda7c0b3070 con 0x5626b2fb3080 2024-03-26T08:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.577+0000 7fda9637a740 10 client.18243 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.577+0000 7fda9637a740 1 -- 172.21.15.7:0/3436590172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5626b2fc7780 con 0x5626b2fb3080 2024-03-26T08:53:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.577+0000 7fda74ff9640 1 -- 172.21.15.7:0/3436590172 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fda7c0a3f00 con 0x5626b2fb3080 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda74ff9640 1 -- 172.21.15.7:0/3436590172 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fda7c0ae070 con 0x5626b2fb3080 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda9637a740 10 client.18243 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda9637a740 10 client.18243 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda9637a740 20 client.18243 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda9637a740 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5626b30286c0 0x5626b302aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda9637a740 4 client.18243 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda9637a740 1 -- 172.21.15.7:0/3436590172 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fda7c09be60 con 0x5626b30286c0 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda9144e640 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5626b30286c0 0x5626b302aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda9144e640 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5626b30286c0 0x5626b302aae0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55c17a1c4b6b404d server_cookie=eef11b20f29781c5 in_seq=0 out_seq=0 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.578+0000 7fda74ff9640 10 client.18243 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:53:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda74ff9640 1 -- 172.21.15.7:0/3436590172 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fda7c09be60 con 0x5626b30286c0 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda74ff9640 10 client.18243 handle_command_reply: tid=0 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 client.18243 shutdown 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 -- 172.21.15.7:0/3436590172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 msgr2=0x5626b2fc61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc61d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda7c097bc0 tx=0x7fda7c004b70 comp rx=0 tx=0).stop 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 -- 172.21.15.7:0/3436590172 shutdown_connections 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5626b30286c0 0x5626b302aae0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626b2fb81f0 0x5626b2fc7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b2fb3080 0x5626b2fc61d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 --2- 172.21.15.7:0/3436590172 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626b2fb2690 0x5626b2fc5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 -- 172.21.15.7:0/3436590172 >> 172.21.15.7:0/3436590172 conn(0x5626b2fadc00 msgr2=0x5626b2fc0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda74ff9640 0 client.18243 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 -- 172.21.15.7:0/3436590172 shutdown_connections 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 1 -- 172.21.15.7:0/3436590172 wait complete. 2024-03-26T08:53:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.579+0000 7fda9637a740 20 client.18243 trim_cache size 0 max 16384 2024-03-26T08:53:13.580 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 -- 172.21.15.7:0/1499782834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fda780a2950 msgr2=0x7fda780a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fda780a2950 0x7fda780a4e10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fda84034fd0 tx=0x7fda84009500 comp rx=0 tx=0).stop 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 -- 172.21.15.7:0/1499782834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c104a90 msgr2=0x7fda8c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c104a90 0x7fda8c197ed0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fda7c00e9f0 tx=0x7fda7c00eeb0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 -- 172.21.15.7:0/1499782834 shutdown_connections 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fda780a2950 0x7fda780a4e10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda8c1073c0 0x7fda8c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fda8c104a90 0x7fda8c197ed0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 --2- 172.21.15.7:0/1499782834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8c102160 0x7fda8c197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 -- 172.21.15.7:0/1499782834 >> 172.21.15.7:0/1499782834 conn(0x7fda8c0fbb70 msgr2=0x7fda8c102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 -- 172.21.15.7:0/1499782834 shutdown_connections 2024-03-26T08:53:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.580+0000 7fda936b2640 1 -- 172.21.15.7:0/1499782834 wait complete. 2024-03-26T08:53:13.588 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:53:13.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 -- 172.21.15.7:0/2283379740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f810c0e0 msgr2=0x7f23f810e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 --2- 172.21.15.7:0/2283379740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f810c0e0 0x7f23f810e4d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009900 tx=0x7f23e802bc30 comp rx=0 tx=0).stop 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 -- 172.21.15.7:0/2283379740 shutdown_connections 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 --2- 172.21.15.7:0/2283379740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f810eaa0 0x7f23f8110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 --2- 172.21.15.7:0/2283379740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f810c0e0 0x7f23f810e4d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 --2- 172.21.15.7:0/2283379740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f81097b0 0x7f23f810bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 -- 172.21.15.7:0/2283379740 >> 172.21.15.7:0/2283379740 conn(0x7f23f806e880 msgr2=0x7f23f8070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 -- 172.21.15.7:0/2283379740 shutdown_connections 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 -- 172.21.15.7:0/2283379740 wait complete. 2024-03-26T08:53:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.732+0000 7f23fdebf640 1 Processor -- start 2024-03-26T08:53:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23fdebf640 1 -- start start 2024-03-26T08:53:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23fdebf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81097b0 0x7f23f8109210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23fdebf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f810c0e0 0x7f23f81a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23fdebf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f810eaa0 0x7f23f81a75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23fdebf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f8114a20 con 0x7f23f81097b0 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23fdebf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81148a0 con 0x7f23f810eaa0 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23fdebf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f8114ba0 con 0x7f23f810c0e0 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81097b0 0x7f23f8109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81097b0 0x7f23f8109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39952/0 (socket says 172.21.15.7:39952) 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f77fe640 1 -- 172.21.15.7:0/1819077445 learned_addr learned my addr 172.21.15.7:0/1819077445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f810eaa0 0x7f23f81a75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f6ffd640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f810c0e0 0x7f23f81a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f77fe640 1 -- 172.21.15.7:0/1819077445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f810c0e0 msgr2=0x7f23f81a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f77fe640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f810c0e0 0x7f23f81a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f77fe640 1 -- 172.21.15.7:0/1819077445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f810eaa0 msgr2=0x7f23f81a75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f77fe640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f810eaa0 0x7f23f81a75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.733+0000 7f23f77fe640 1 -- 172.21.15.7:0/1819077445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f81a7ec0 con 0x7f23f81097b0 2024-03-26T08:53:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.734+0000 7f23f77fe640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81097b0 0x7f23f8109210 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f23f000eb90 tx=0x7f23f000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.734+0000 7f23f4ff9640 1 -- 172.21.15.7:0/1819077445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0010430 con 0x7f23f81097b0 2024-03-26T08:53:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.734+0000 7f23f4ff9640 1 -- 172.21.15.7:0/1819077445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23f0010ab0 con 0x7f23f81097b0 2024-03-26T08:53:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.734+0000 7f23f4ff9640 1 -- 172.21.15.7:0/1819077445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0018af0 con 0x7f23f81097b0 2024-03-26T08:53:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.734+0000 7f23fdebf640 1 -- 172.21.15.7:0/1819077445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f81a8150 con 0x7f23f81097b0 2024-03-26T08:53:13.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.734+0000 7f23fdebf640 1 -- 172.21.15.7:0/1819077445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f81a8620 con 0x7f23f81097b0 2024-03-26T08:53:13.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.737+0000 7f2400b88740 1 Processor -- start 2024-03-26T08:53:13.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f2400b88740 1 -- start start 2024-03-26T08:53:13.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f2400b88740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f2400b88740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088e80 0x564e3208d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f2400b88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e3208fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f2400b88740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e32042610 con 0x564e32088490 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f2400b88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e31fe53c0 con 0x564e3208da80 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f2400b88740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e31f8dd60 con 0x564e32088e80 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e3208fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e3208fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40908/0 (socket says 172.21.15.7:40908) 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320888b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39958/0 (socket says 172.21.15.7:39958) 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f6ffd640 1 -- 172.21.15.7:0/1122591802 learned_addr learned my addr 172.21.15.7:0/1122591802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f6ffd640 1 -- 172.21.15.7:0/1122591802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088e80 msgr2=0x564e3208d410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f6ffd640 1 --2- 172.21.15.7:0/1122591802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088e80 0x564e3208d410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f6ffd640 1 -- 172.21.15.7:0/1122591802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 msgr2=0x564e320888b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f6ffd640 1 --2- 172.21.15.7:0/1122591802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320888b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f6ffd640 1 -- 172.21.15.7:0/1122591802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e31cf93f0 con 0x564e3208da80 2024-03-26T08:53:13.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.738+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1122591802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:13.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.739+0000 7f23f4ff9640 1 -- 172.21.15.7:0/1819077445 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f23f0018c90 con 0x7f23f81097b0 2024-03-26T08:53:13.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.739+0000 7f23f6ffd640 1 --2- 172.21.15.7:0/1122591802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e3208fef0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23f80730a0 tx=0x7f23e8009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0b109be4b55ec2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.739+0000 7f23ddffb640 1 -- 172.21.15.7:0/1122591802 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e802cb80 con 0x564e3208da80 2024-03-26T08:53:13.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.739+0000 7f23ddffb640 1 -- 172.21.15.7:0/1122591802 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23e8005040 con 0x564e3208da80 2024-03-26T08:53:13.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.739+0000 7f23ddffb640 1 -- 172.21.15.7:0/1122591802 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e80048e0 con 0x564e3208da80 2024-03-26T08:53:13.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.739+0000 7f23f4ff9640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23e00a2800 0x7f23e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.739+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23e00a2800 0x7f23e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.739+0000 7f23f4ff9640 1 -- 172.21.15.7:0/1819077445 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23f00cf280 con 0x7f23f81097b0 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 -- 172.21.15.7:0/1122591802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 msgr2=0x564e3208fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 --2- 172.21.15.7:0/1122591802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e3208fef0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23f80730a0 tx=0x7f23e8009560 comp rx=0 tx=0).stop 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 -- 172.21.15.7:0/1122591802 shutdown_connections 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 --2- 172.21.15.7:0/1122591802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e3208fef0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 --2- 172.21.15.7:0/1122591802 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088e80 0x564e3208d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 --2- 172.21.15.7:0/1122591802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 -- 172.21.15.7:0/1122591802 >> 172.21.15.7:0/1122591802 conn(0x564e3207ab80 msgr2=0x564e3207af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 -- 172.21.15.7:0/1122591802 shutdown_connections 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 -- 172.21.15.7:0/1122591802 wait complete. 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23e00a2800 0x7f23e00a4cc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x564e3207e840 tx=0x7f23ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 -- start start 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 Processor -- start 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e32092250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088e80 0x564e32092790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.740+0000 7f2400b88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e320997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f2400b88740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e31fa1240 con 0x564e32088490 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f2400b88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e31fe1460 con 0x564e3208da80 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f2400b88740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e320909b0 con 0x564e32088e80 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f2400b88740 10 client.0 _ll_register_callbacks cb 0x564e31fa0c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e32092250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e320997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e32092250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39968/0 (socket says 172.21.15.7:39968) 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f77fe640 1 -- 172.21.15.7:0/1953422241 learned_addr learned my addr 172.21.15.7:0/1953422241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f6ffd640 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088e80 0x564e32092790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f7fff640 1 -- 172.21.15.7:0/1953422241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088e80 msgr2=0x564e32092790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088e80 0x564e32092790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f7fff640 1 -- 172.21.15.7:0/1953422241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 msgr2=0x564e32092250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e32092250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f7fff640 1 -- 172.21.15.7:0/1953422241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e31dc37d0 con 0x564e3208da80 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f77fe640 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e32092250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e320997d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0045f0 tx=0x7f23ec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85910e80d65c4731 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.741+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1953422241 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec029440 con 0x564e3208da80 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f2400b88740 1 -- 172.21.15.7:0/1953422241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23e8005040 con 0x564e3208da80 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1953422241 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23ec01f430 con 0x564e3208da80 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1953422241 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ec01f630 con 0x564e3208da80 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f2400b88740 10 client.28064 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f2400b88740 1 -- 172.21.15.7:0/1953422241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e3209cc00 con 0x564e3208da80 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1953422241 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23e8005040 con 0x564e3208da80 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1953422241 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f23ec027070 con 0x564e3208da80 2024-03-26T08:53:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f2400b88740 10 client.28064 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f2400b88740 10 client.28064 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f2400b88740 20 client.28064 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f2400b88740 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564e3209e220 0x564e320a0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f2400b88740 4 client.28064 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f2400b88740 1 -- 172.21.15.7:0/1953422241 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e31ff2900 con 0x564e3209e220 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.742+0000 7f23f77fe640 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564e3209e220 0x564e320a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.743+0000 7f23f77fe640 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564e3209e220 0x564e320a0660 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea9670674b04f589 server_cookie=5fb8201363a4a8eb in_seq=0 out_seq=0 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.743+0000 7f23d6ffd640 10 client.28064 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1953422241 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564e31ff2900 con 0x564e3209e220 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f23d6ffd640 10 client.28064 handle_command_reply: tid=0 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 client.28064 shutdown 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 -- 172.21.15.7:0/1953422241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 msgr2=0x564e320997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e320997d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0045f0 tx=0x7f23ec004730 comp rx=0 tx=0).stop 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 -- 172.21.15.7:0/1953422241 shutdown_connections 2024-03-26T08:53:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e320997d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564e3209e220 0x564e320a0660 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088e80 0x564e32092790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f23d6ffd640 0 client.28064 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:53:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 --2- 172.21.15.7:0/1953422241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e32092250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 -- 172.21.15.7:0/1953422241 >> 172.21.15.7:0/1953422241 conn(0x564e3207ab80 msgr2=0x564e32087b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 -- 172.21.15.7:0/1953422241 shutdown_connections 2024-03-26T08:53:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 1 -- 172.21.15.7:0/1953422241 wait complete. 2024-03-26T08:53:13.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.744+0000 7f2400b88740 20 client.28064 trim_cache size 0 max 16384 2024-03-26T08:53:13.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f2400b88740 1 Processor -- start 2024-03-26T08:53:13.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f2400b88740 1 -- start start 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f2400b88740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088490 0x564e32091450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f2400b88740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088e80 0x564e32099790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f2400b88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e32079590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f2400b88740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81b15e0 con 0x564e32088e80 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f2400b88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81af0e0 con 0x564e3208da80 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f2400b88740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81ad160 con 0x564e32088490 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f23f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e32079590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f23f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e32079590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40926/0 (socket says 172.21.15.7:40926) 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f23f7fff640 1 -- 172.21.15.7:0/2706080470 learned_addr learned my addr 172.21.15.7:0/2706080470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.760+0000 7f23f6ffd640 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088e80 0x564e32099790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.761+0000 7f23f77fe640 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088490 0x564e32091450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.761+0000 7f23f7fff640 1 -- 172.21.15.7:0/2706080470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088490 msgr2=0x564e32091450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.761+0000 7f23f7fff640 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088490 0x564e32091450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.761+0000 7f23f7fff640 1 -- 172.21.15.7:0/2706080470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088e80 msgr2=0x564e32099790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.761+0000 7f23f7fff640 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088e80 0x564e32099790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.761+0000 7f23f7fff640 1 -- 172.21.15.7:0/2706080470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e3207b8f0 con 0x564e3208da80 2024-03-26T08:53:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.761+0000 7f23f6ffd640 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088e80 0x564e32099790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.761+0000 7f23f7fff640 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e32079590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f23ec019040 tx=0x7f23ec01f430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8406a1610e819c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.762+0000 7f23dd7fa640 1 -- 172.21.15.7:0/2706080470 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec029640 con 0x564e3208da80 2024-03-26T08:53:13.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.762+0000 7f23dd7fa640 1 -- 172.21.15.7:0/2706080470 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23ec01fcb0 con 0x564e3208da80 2024-03-26T08:53:13.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.762+0000 7f23dd7fa640 1 -- 172.21.15.7:0/2706080470 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0339e0 con 0x564e3208da80 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 -- 172.21.15.7:0/2706080470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 msgr2=0x564e32079590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e32079590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f23ec019040 tx=0x7f23ec01f430 comp rx=0 tx=0).stop 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 -- 172.21.15.7:0/2706080470 shutdown_connections 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e3208da80 0x564e32079590 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088e80 0x564e32099790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 --2- 172.21.15.7:0/2706080470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e32088490 0x564e32091450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 -- 172.21.15.7:0/2706080470 >> 172.21.15.7:0/2706080470 conn(0x564e3207ab80 msgr2=0x564e32090d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 -- 172.21.15.7:0/2706080470 shutdown_connections 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 -- 172.21.15.7:0/2706080470 wait complete. 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 -- start start 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.763+0000 7f2400b88740 1 Processor -- start 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f2400b88740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f2400b88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e32088e80 0x564e32097c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f2400b88740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e3208da80 0x564e32098c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f2400b88740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81ad760 con 0x564e32088490 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f2400b88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81aea80 con 0x564e32088e80 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f2400b88740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81addc0 con 0x564e3208da80 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f2400b88740 10 client.0 _ll_register_callbacks cb 0x7f23ec0043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e3208da80 0x564e32098c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39984/0 (socket says 172.21.15.7:39984) 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e32088e80 0x564e32097c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f77fe640 1 -- 172.21.15.7:0/1047013840 learned_addr learned my addr 172.21.15.7:0/1047013840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f7fff640 1 -- 172.21.15.7:0/1047013840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e32088e80 msgr2=0x564e32097c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e32088e80 0x564e32097c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f7fff640 1 -- 172.21.15.7:0/1047013840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 msgr2=0x564e320976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f7fff640 1 -- 172.21.15.7:0/1047013840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e32099200 con 0x564e3208da80 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f77fe640 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23f7fff640 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e3208da80 0x564e32098c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f23ec013010 tx=0x7f23ec01fa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ade130c5ff102ac8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.764+0000 7f23d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1047013840 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec040070 con 0x564e3208da80 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f2400b88740 1 -- 172.21.15.7:0/1047013840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23ec01fcb0 con 0x564e3208da80 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1047013840 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23ec005450 con 0x564e3208da80 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1047013840 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ec04c050 con 0x564e3208da80 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f2400b88740 10 client.28075 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f2400b88740 1 -- 172.21.15.7:0/1047013840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564e320fa330 con 0x564e3208da80 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1047013840 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23ec01fcb0 con 0x564e3208da80 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1047013840 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f23ec027040 con 0x564e3208da80 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f2400b88740 10 client.28075 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f2400b88740 10 client.28075 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:53:13.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f2400b88740 20 client.28075 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f2400b88740 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564e3209e220 0x564e320fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f2400b88740 4 client.28075 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:53:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.765+0000 7f2400b88740 1 -- 172.21.15.7:0/1047013840 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e31fecc50 con 0x564e3209e220 2024-03-26T08:53:13.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f23f77fe640 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564e3209e220 0x564e320fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f23f77fe640 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564e3209e220 0x564e320fcb80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee86b31ee09c6ac4 server_cookie=205396c89e52d3ea in_seq=0 out_seq=0 2024-03-26T08:53:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f23d6ffd640 10 client.28075 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:53:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f23d6ffd640 1 -- 172.21.15.7:0/1047013840 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e31fecc50 con 0x564e3209e220 2024-03-26T08:53:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f23d6ffd640 10 client.28075 handle_command_reply: tid=0 2024-03-26T08:53:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 client.28075 shutdown 2024-03-26T08:53:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 -- 172.21.15.7:0/1047013840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e3208da80 msgr2=0x564e32098c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e3208da80 0x564e32098c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f23ec013010 tx=0x7f23ec01fa80 comp rx=0 tx=0).stop 2024-03-26T08:53:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 -- 172.21.15.7:0/1047013840 shutdown_connections 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564e3209e220 0x564e320fcb80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564e3208da80 0x564e32098c10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f23d6ffd640 0 client.28075 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564e32088e80 0x564e32097c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 --2- 172.21.15.7:0/1047013840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e32088490 0x564e320976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 -- 172.21.15.7:0/1047013840 >> 172.21.15.7:0/1047013840 conn(0x564e3207ab80 msgr2=0x564e320915d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 -- 172.21.15.7:0/1047013840 shutdown_connections 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 1 -- 172.21.15.7:0/1047013840 wait complete. 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.766+0000 7f2400b88740 20 client.28075 trim_cache size 0 max 16384 2024-03-26T08:53:13.767 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:53:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 -- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23e00a2800 msgr2=0x7f23e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23e00a2800 0x7f23e00a4cc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x564e3207e840 tx=0x7f23ec005f90 comp rx=0 tx=0).stop 2024-03-26T08:53:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 -- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81097b0 msgr2=0x7f23f8109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81097b0 0x7f23f8109210 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f23f000eb90 tx=0x7f23f000ef90 comp rx=0 tx=0).stop 2024-03-26T08:53:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 -- 172.21.15.7:0/1819077445 shutdown_connections 2024-03-26T08:53:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23e00a2800 0x7f23e00a4cc0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f810eaa0 0x7f23f81a75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23f810c0e0 0x7f23f81a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 --2- 172.21.15.7:0/1819077445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81097b0 0x7f23f8109210 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 -- 172.21.15.7:0/1819077445 >> 172.21.15.7:0/1819077445 conn(0x7f23f806e880 msgr2=0x7f23f810a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 -- 172.21.15.7:0/1819077445 shutdown_connections 2024-03-26T08:53:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.767+0000 7f23fdebf640 1 -- 172.21.15.7:0/1819077445 wait complete. 2024-03-26T08:53:13.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:13 smithi007 ceph-mon[32462]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 619 op/s 2024-03-26T08:53:13.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:13 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/518961158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:53:13.777 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:53:13.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:53:13.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.942+0000 7fe7e5844640 1 -- 172.21.15.7:0/329635799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7e0100680 msgr2=0x7fe7e0100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.942+0000 7fe7e5844640 1 --2- 172.21.15.7:0/329635799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7e0100680 0x7fe7e0100ae0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009860 tx=0x7fe7d802bb70 comp rx=0 tx=0).stop 2024-03-26T08:53:13.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.948+0000 7fe7e5844640 1 -- 172.21.15.7:0/329635799 shutdown_connections 2024-03-26T08:53:13.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.948+0000 7fe7e5844640 1 --2- 172.21.15.7:0/329635799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0101020 0x7fe7e01058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.948+0000 7fe7e5844640 1 --2- 172.21.15.7:0/329635799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7e0100680 0x7fe7e0100ae0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.948+0000 7fe7e5844640 1 --2- 172.21.15.7:0/329635799 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e00ff550 0x7fe7e00ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.948+0000 7fe7e5844640 1 -- 172.21.15.7:0/329635799 >> 172.21.15.7:0/329635799 conn(0x7fe7e00fae50 msgr2=0x7fe7e00fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.950+0000 7fe7e5844640 1 -- 172.21.15.7:0/329635799 shutdown_connections 2024-03-26T08:53:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.950+0000 7fe7e5844640 1 -- 172.21.15.7:0/329635799 wait complete. 2024-03-26T08:53:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.951+0000 7fe7e5844640 1 Processor -- start 2024-03-26T08:53:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.951+0000 7fe7e5844640 1 -- start start 2024-03-26T08:53:13.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.952+0000 7fe7e5844640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7e00ff550 0x7fe7e019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.952+0000 7fe7e5844640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100680 0x7fe7e019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.952+0000 7fe7e5844640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101020 0x7fe7e01a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.952+0000 7fe7e5844640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0113990 con 0x7fe7e0100680 2024-03-26T08:53:13.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.952+0000 7fe7e5844640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0113810 con 0x7fe7e00ff550 2024-03-26T08:53:13.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.952+0000 7fe7e5844640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e0113b10 con 0x7fe7e0101020 2024-03-26T08:53:13.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.953+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101020 0x7fe7e01a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.953+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101020 0x7fe7e01a5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45906/0 (socket says 172.21.15.7:45906) 2024-03-26T08:53:13.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.953+0000 7fe7de7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100680 0x7fe7e019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.953+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7e00ff550 0x7fe7e019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.953+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7e00ff550 0x7fe7e019df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40958/0 (socket says 172.21.15.7:40958) 2024-03-26T08:53:13.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.953+0000 7fe7deffd640 1 -- 172.21.15.7:0/761942006 learned_addr learned my addr 172.21.15.7:0/761942006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.954+0000 7fe7de7fc640 1 -- 172.21.15.7:0/761942006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101020 msgr2=0x7fe7e01a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.954+0000 7fe7de7fc640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101020 0x7fe7e01a5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.954+0000 7fe7de7fc640 1 -- 172.21.15.7:0/761942006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7e00ff550 msgr2=0x7fe7e019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.954+0000 7fe7de7fc640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7e00ff550 0x7fe7e019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.954+0000 7fe7de7fc640 1 -- 172.21.15.7:0/761942006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e01a5ba0 con 0x7fe7e0100680 2024-03-26T08:53:13.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.955+0000 7fe7de7fc640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100680 0x7fe7e019e4c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009780 tx=0x7fe7d8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.955+0000 7fe7e4842640 1 -- 172.21.15.7:0/761942006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8039040 con 0x7fe7e0100680 2024-03-26T08:53:13.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.956+0000 7fe7e4842640 1 -- 172.21.15.7:0/761942006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7d8005050 con 0x7fe7e0100680 2024-03-26T08:53:13.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.956+0000 7fe7e4842640 1 -- 172.21.15.7:0/761942006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8004b20 con 0x7fe7e0100680 2024-03-26T08:53:13.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.956+0000 7fe7e5844640 1 -- 172.21.15.7:0/761942006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e01a5e30 con 0x7fe7e0100680 2024-03-26T08:53:13.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.956+0000 7fe7e5844640 1 -- 172.21.15.7:0/761942006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e01a6240 con 0x7fe7e0100680 2024-03-26T08:53:13.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.961+0000 7fe7e842d740 1 Processor -- start 2024-03-26T08:53:13.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.961+0000 7fe7e842d740 1 -- start start 2024-03-26T08:53:13.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.961+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab86ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.961+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.961+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 0x559caab8e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.961+0000 7fe7e842d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559caab49610 con 0x559caab8c1f0 2024-03-26T08:53:13.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.961+0000 7fe7e842d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559caaaec3c0 con 0x559caab87080 2024-03-26T08:53:13.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.961+0000 7fe7e842d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559caaa94d60 con 0x559caab86690 2024-03-26T08:53:13.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab86ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab86ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45916/0 (socket says 172.21.15.7:45916) 2024-03-26T08:53:13.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7df7fe640 1 -- 172.21.15.7:0/517529398 learned_addr learned my addr 172.21.15.7:0/517529398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7deffd640 1 --2- 172.21.15.7:0/517529398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7df7fe640 1 -- 172.21.15.7:0/517529398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 msgr2=0x559caab8bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/517529398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab8bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7e4842640 1 -- 172.21.15.7:0/761942006 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe7d8044030 con 0x7fe7e0100680 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7df7fe640 1 -- 172.21.15.7:0/517529398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 msgr2=0x559caab8e680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/517529398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 0x559caab8e680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.962+0000 7fe7df7fe640 1 -- 172.21.15.7:0/517529398 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559caa8003f0 con 0x559caab86690 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.963+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/517529398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab86ab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000f480 tx=0x7fe7d000f940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff9a2af4c68fa224 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.963+0000 7fe7d5ffb640 1 -- 172.21.15.7:0/517529398 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d001d070 con 0x559caab86690 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.963+0000 7fe7d5ffb640 1 -- 172.21.15.7:0/517529398 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7d000cb80 con 0x559caab86690 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.963+0000 7fe7e4842640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe7cc0a2990 0x7fe7cc0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.963+0000 7fe7e4842640 1 -- 172.21.15.7:0/761942006 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe7d80efa90 con 0x7fe7e0100680 2024-03-26T08:53:13.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.963+0000 7fe7d5ffb640 1 -- 172.21.15.7:0/517529398 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0018d50 con 0x559caab86690 2024-03-26T08:53:13.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7deffd640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe7cc0a2990 0x7fe7cc0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7e842d740 1 -- 172.21.15.7:0/517529398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 msgr2=0x559caab86ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7e842d740 1 --2- 172.21.15.7:0/517529398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab86ab0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000f480 tx=0x7fe7d000f940 comp rx=0 tx=0).stop 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7e842d740 1 -- 172.21.15.7:0/517529398 shutdown_connections 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7e842d740 1 --2- 172.21.15.7:0/517529398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 0x559caab8e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7e842d740 1 --2- 172.21.15.7:0/517529398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab8bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7e842d740 1 --2- 172.21.15.7:0/517529398 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab86ab0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7e842d740 1 -- 172.21.15.7:0/517529398 >> 172.21.15.7:0/517529398 conn(0x559caab81c00 msgr2=0x559caab82000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7deffd640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe7cc0a2990 0x7fe7cc0a4e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x559caab87480 tx=0x7fe7c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.964+0000 7fe7e842d740 1 -- 172.21.15.7:0/517529398 shutdown_connections 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.965+0000 7fe7e842d740 1 -- 172.21.15.7:0/517529398 wait complete. 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.965+0000 7fe7e842d740 1 -- start start 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.965+0000 7fe7e842d740 1 Processor -- start 2024-03-26T08:53:13.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.965+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab86690 0x559caab90bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.965+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab87080 0x559caab910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.965+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab98130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.965+0000 7fe7e842d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x559caaaa8240 con 0x559caab87080 2024-03-26T08:53:13.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.965+0000 7fe7e842d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x559caaae8460 con 0x559caab86690 2024-03-26T08:53:13.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.965+0000 7fe7e842d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x559caab8f240 con 0x559caab8c1f0 2024-03-26T08:53:13.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7e842d740 10 client.0 _ll_register_callbacks cb 0x559caaaa7c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:13.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab86690 0x559caab90bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab98130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab98130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45920/0 (socket says 172.21.15.7:45920) 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7df7fe640 1 -- 172.21.15.7:0/1499635929 learned_addr learned my addr 172.21.15.7:0/1499635929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7df7fe640 1 -- 172.21.15.7:0/1499635929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab86690 msgr2=0x559caab90bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab86690 0x559caab90bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7df7fe640 1 -- 172.21.15.7:0/1499635929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab87080 msgr2=0x559caab910f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab87080 0x559caab910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.966+0000 7fe7df7fe640 1 -- 172.21.15.7:0/1499635929 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559caa8ca7d0 con 0x559caab8c1f0 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.967+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab98130 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006010 tx=0x7fe7d000c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58e14e347c26da71 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.967+0000 7fe7beffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:53:13.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.967+0000 7fe7beffd640 1 -- 172.21.15.7:0/1499635929 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d001d040 con 0x559caab8c1f0 2024-03-26T08:53:13.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.967+0000 7fe7beffd640 1 -- 172.21.15.7:0/1499635929 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe7d0018a50 con 0x559caab8c1f0 2024-03-26T08:53:13.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.967+0000 7fe7e842d740 1 -- 172.21.15.7:0/1499635929 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7d000cb80 con 0x559caab8c1f0 2024-03-26T08:53:13.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.967+0000 7fe7beffd640 1 -- 172.21.15.7:0/1499635929 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d0018c50 con 0x559caab8c1f0 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7e842d740 10 client.28087 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7e842d740 1 -- 172.21.15.7:0/1499635929 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559caab9b4f0 con 0x559caab8c1f0 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7beffd640 1 -- 172.21.15.7:0/1499635929 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7d000cb80 con 0x559caab8c1f0 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7beffd640 1 -- 172.21.15.7:0/1499635929 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe7d000b040 con 0x559caab8c1f0 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7e842d740 10 client.28087 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7e842d740 10 client.28087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7e842d740 20 client.28087 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7e842d740 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559caab9c8f0 0x559caab9ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7e842d740 4 client.28087 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7e842d740 1 -- 172.21.15.7:0/1499635929 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559caaaf9900 con 0x559caab9c8f0 2024-03-26T08:53:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.970+0000 7fe7deffd640 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559caab9c8f0 0x559caab9ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.972+0000 7fe7beffd640 10 client.28087 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:13.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.972+0000 7fe7deffd640 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559caab9c8f0 0x559caab9ece0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8944a033ce62fb server_cookie=c2fba4c622124bba in_seq=0 out_seq=0 2024-03-26T08:53:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7beffd640 1 -- 172.21.15.7:0/1499635929 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559caaaf9900 con 0x559caab9c8f0 2024-03-26T08:53:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7beffd640 10 client.28087 handle_command_reply: tid=0 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 client.28087 shutdown 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 -- 172.21.15.7:0/1499635929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 msgr2=0x559caab98130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab98130 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0006010 tx=0x7fe7d000c950 comp rx=0 tx=0).stop 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 -- 172.21.15.7:0/1499635929 shutdown_connections 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559caab9c8f0 0x559caab9ece0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab98130 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab87080 0x559caab910f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7beffd640 0 client.28087 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 --2- 172.21.15.7:0/1499635929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab86690 0x559caab90bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 -- 172.21.15.7:0/1499635929 >> 172.21.15.7:0/1499635929 conn(0x559caab81c00 msgr2=0x559caab80e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 -- 172.21.15.7:0/1499635929 shutdown_connections 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 1 -- 172.21.15.7:0/1499635929 wait complete. 2024-03-26T08:53:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.973+0000 7fe7e842d740 20 client.28087 trim_cache size 0 max 16384 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.989+0000 7fe7e842d740 1 Processor -- start 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7e842d740 1 -- start start 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab86690 0x559caab8f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab8fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7e842d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01b1660 con 0x559caab86690 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7e842d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01af160 con 0x559caab87080 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7e842d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01ad1e0 con 0x559caab8c1f0 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab86690 0x559caab8f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab86690 0x559caab8f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40024/0 (socket says 172.21.15.7:40024) 2024-03-26T08:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7deffd640 1 -- 172.21.15.7:0/86923714 learned_addr learned my addr 172.21.15.7:0/86923714 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab8fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7de7fc640 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7deffd640 1 -- 172.21.15.7:0/86923714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 msgr2=0x559caab8fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7deffd640 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab8fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7deffd640 1 -- 172.21.15.7:0/86923714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 msgr2=0x559caab980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7deffd640 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7deffd640 1 -- 172.21.15.7:0/86923714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559caab7d480 con 0x559caab86690 2024-03-26T08:53:13.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.990+0000 7fe7de7fc640 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:53:13.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.991+0000 7fe7deffd640 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab86690 0x559caab8f610 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8004000 tx=0x7fe7c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de6688d6c1bf7e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.991+0000 7fe7d57fa640 1 -- 172.21.15.7:0/86923714 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c802cad0 con 0x559caab86690 2024-03-26T08:53:13.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.992+0000 7fe7d57fa640 1 -- 172.21.15.7:0/86923714 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7c8032070 con 0x559caab86690 2024-03-26T08:53:13.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.992+0000 7fe7d57fa640 1 -- 172.21.15.7:0/86923714 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8021670 con 0x559caab86690 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.993+0000 7fe7e842d740 1 -- 172.21.15.7:0/86923714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab86690 msgr2=0x559caab8f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.993+0000 7fe7e842d740 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab86690 0x559caab8f610 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8004000 tx=0x7fe7c8017000 comp rx=0 tx=0).stop 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.993+0000 7fe7e842d740 1 -- 172.21.15.7:0/86923714 shutdown_connections 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.993+0000 7fe7e842d740 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab8c1f0 0x559caab8fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.993+0000 7fe7e842d740 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.993+0000 7fe7e842d740 1 --2- 172.21.15.7:0/86923714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab86690 0x559caab8f610 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.993+0000 7fe7e842d740 1 -- 172.21.15.7:0/86923714 >> 172.21.15.7:0/86923714 conn(0x559caab81c00 msgr2=0x559caab90bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.993+0000 7fe7e842d740 1 -- 172.21.15.7:0/86923714 shutdown_connections 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.993+0000 7fe7e842d740 1 -- 172.21.15.7:0/86923714 wait complete. 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7e842d740 1 -- start start 2024-03-26T08:53:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7e842d740 1 Processor -- start 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab957c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab95d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7e842d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 0x559caab96d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7e842d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01ad7e0 con 0x559caab8c1f0 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7e842d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01aeb00 con 0x559caab87080 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7e842d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e01ade40 con 0x559caab86690 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7e842d740 10 client.0 _ll_register_callbacks cb 0x7fe7d0009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 0x559caab96d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7de7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab95d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7df7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 0x559caab96d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40034/0 (socket says 172.21.15.7:40034) 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7df7fe640 1 -- 172.21.15.7:0/2812235743 learned_addr learned my addr 172.21.15.7:0/2812235743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7deffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45940/0 (socket says 172.21.15.7:45940) 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.994+0000 7fe7de7fc640 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab95d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40990/0 (socket says 172.21.15.7:40990) 2024-03-26T08:53:13.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7df7fe640 1 -- 172.21.15.7:0/2812235743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 msgr2=0x559caab957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7df7fe640 1 -- 172.21.15.7:0/2812235743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 msgr2=0x559caab95d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab95d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7df7fe640 1 -- 172.21.15.7:0/2812235743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559caab97300 con 0x559caab8c1f0 2024-03-26T08:53:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7deffd640 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7de7fc640 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab95d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:13.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7df7fe640 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 0x559caab96d10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0018400 tx=0x7fe7d0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96f36d6e3389a23 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7beffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.995+0000 7fe7beffd640 1 -- 172.21.15.7:0/2812235743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d001d040 con 0x559caab8c1f0 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7e842d740 1 -- 172.21.15.7:0/2812235743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7c8032070 con 0x559caab8c1f0 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7beffd640 1 -- 172.21.15.7:0/2812235743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe7d0015040 con 0x559caab8c1f0 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7beffd640 1 -- 172.21.15.7:0/2812235743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d0005250 con 0x559caab8c1f0 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7e842d740 10 client.18273 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7e842d740 1 -- 172.21.15.7:0/2812235743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559caabf8060 con 0x559caab8c1f0 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7beffd640 1 -- 172.21.15.7:0/2812235743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7d0026070 con 0x559caab8c1f0 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7beffd640 1 -- 172.21.15.7:0/2812235743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe7d000b040 con 0x559caab8c1f0 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7e842d740 10 client.18273 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7e842d740 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7e842d740 20 client.18273 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7e842d740 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559caabf89a0 0x559caabfadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7e842d740 4 client.18273 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:53:13.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.996+0000 7fe7e842d740 1 -- 172.21.15.7:0/2812235743 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe7c8032070 con 0x559caabf89a0 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.997+0000 7fe7deffd640 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559caabf89a0 0x559caabfadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.997+0000 7fe7beffd640 10 client.18273 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.997+0000 7fe7deffd640 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559caabf89a0 0x559caabfadc0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=812db82f3c799aa2 server_cookie=9a789ab9de0c51c2 in_seq=0 out_seq=0 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.997+0000 7fe7beffd640 1 -- 172.21.15.7:0/2812235743 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe7c8032070 con 0x559caabf89a0 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.997+0000 7fe7beffd640 10 client.18273 handle_command_reply: tid=0 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.997+0000 7fe7e842d740 1 client.18273 shutdown 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.997+0000 7fe7e842d740 1 -- 172.21.15.7:0/2812235743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 msgr2=0x559caab96d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.997+0000 7fe7e842d740 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 0x559caab96d10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0018400 tx=0x7fe7d0009ae0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e842d740 1 -- 172.21.15.7:0/2812235743 shutdown_connections 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e842d740 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x559caabf89a0 0x559caabfadc0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e842d740 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559caab8c1f0 0x559caab96d10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e842d740 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559caab87080 0x559caab95d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e842d740 1 --2- 172.21.15.7:0/2812235743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x559caab86690 0x559caab957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e842d740 1 -- 172.21.15.7:0/2812235743 >> 172.21.15.7:0/2812235743 conn(0x559caab81c00 msgr2=0x559caab90bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:13.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7beffd640 0 client.18273 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:13.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e842d740 1 -- 172.21.15.7:0/2812235743 shutdown_connections 2024-03-26T08:53:13.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e842d740 1 -- 172.21.15.7:0/2812235743 wait complete. 2024-03-26T08:53:13.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e842d740 20 client.18273 trim_cache size 0 max 16384 2024-03-26T08:53:13.998 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:53:13.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:13 smithi016 ceph-mon[35851]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 619 op/s 2024-03-26T08:53:13.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:13 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/518961158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:53:13.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 -- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe7cc0a2990 msgr2=0x7fe7cc0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:13.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe7cc0a2990 0x7fe7cc0a4e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x559caab87480 tx=0x7fe7c8005f90 comp rx=0 tx=0).stop 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 -- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100680 msgr2=0x7fe7e019e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100680 0x7fe7e019e4c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009780 tx=0x7fe7d8004530 comp rx=0 tx=0).stop 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 -- 172.21.15.7:0/761942006 shutdown_connections 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe7cc0a2990 0x7fe7cc0a4e50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe7e0101020 0x7fe7e01a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7e0100680 0x7fe7e019e4c0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 --2- 172.21.15.7:0/761942006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7e00ff550 0x7fe7e019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 -- 172.21.15.7:0/761942006 >> 172.21.15.7:0/761942006 conn(0x7fe7e00fae50 msgr2=0x7fe7e01018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 -- 172.21.15.7:0/761942006 shutdown_connections 2024-03-26T08:53:14.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:13.998+0000 7fe7e5844640 1 -- 172.21.15.7:0/761942006 wait complete. 2024-03-26T08:53:14.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:13 smithi044 ceph-mon[36461]: pgmap v684: 97 pgs: 97 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.5 MiB/s wr, 619 op/s 2024-03-26T08:53:14.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:13 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/518961158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:53:15.008 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:53:15.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:53:15.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:53:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 -- 172.21.15.7:0/3623291430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e000ff990 msgr2=0x7f1e000ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 --2- 172.21.15.7:0/3623291430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e000ff990 0x7f1e000ffdb0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009900 tx=0x7f1df002bc80 comp rx=0 tx=0).stop 2024-03-26T08:53:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 -- 172.21.15.7:0/3623291430 shutdown_connections 2024-03-26T08:53:15.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 --2- 172.21.15.7:0/3623291430 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e001033c0 0x7f1e00109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 --2- 172.21.15.7:0/3623291430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e00102a00 0x7f1e00102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 --2- 172.21.15.7:0/3623291430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e000ff990 0x7f1e000ffdb0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 -- 172.21.15.7:0/3623291430 >> 172.21.15.7:0/3623291430 conn(0x7f1e000fb2c0 msgr2=0x7f1e000fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 -- 172.21.15.7:0/3623291430 shutdown_connections 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 -- 172.21.15.7:0/3623291430 wait complete. 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.140+0000 7f1e05c44640 1 Processor -- start 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1e05c44640 1 -- start start 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1e05c44640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000ff990 0x7f1e00197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1e05c44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e00102a00 0x7f1e00197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1e05c44640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e001033c0 0x7f1e0019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1e05c44640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0010d3b0 con 0x7f1e001033c0 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1e05c44640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0010d230 con 0x7f1e00102a00 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1e05c44640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0010d530 con 0x7f1e000ff990 2024-03-26T08:53:15.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dfffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e001033c0 0x7f1e0019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000ff990 0x7f1e00197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dfffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e001033c0 0x7f1e0019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40062/0 (socket says 172.21.15.7:40062) 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000ff990 0x7f1e00197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45948/0 (socket says 172.21.15.7:45948) 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dfffff640 1 -- 172.21.15.7:0/128836440 learned_addr learned my addr 172.21.15.7:0/128836440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e00102a00 0x7f1e00197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dfffff640 1 -- 172.21.15.7:0/128836440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000ff990 msgr2=0x7f1e00197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dfffff640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000ff990 0x7f1e00197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dfffff640 1 -- 172.21.15.7:0/128836440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e00102a00 msgr2=0x7f1e00197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dfffff640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e00102a00 0x7f1e00197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.141+0000 7f1dfffff640 1 -- 172.21.15.7:0/128836440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0019f640 con 0x7f1e001033c0 2024-03-26T08:53:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.142+0000 7f1dfffff640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e001033c0 0x7f1e0019efd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1df800eb10 tx=0x7f1df800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.142+0000 7f1dfcff9640 1 -- 172.21.15.7:0/128836440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8004b20 con 0x7f1e001033c0 2024-03-26T08:53:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.143+0000 7f1dfcff9640 1 -- 172.21.15.7:0/128836440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df8004cc0 con 0x7f1e001033c0 2024-03-26T08:53:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.143+0000 7f1dfcff9640 1 -- 172.21.15.7:0/128836440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8010760 con 0x7f1e001033c0 2024-03-26T08:53:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.143+0000 7f1e05c44640 1 -- 172.21.15.7:0/128836440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0019f930 con 0x7f1e001033c0 2024-03-26T08:53:15.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.143+0000 7f1e05c44640 1 -- 172.21.15.7:0/128836440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0019fdd0 con 0x7f1e001033c0 2024-03-26T08:53:15.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.145+0000 7f1dfcff9640 1 -- 172.21.15.7:0/128836440 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1df8005020 con 0x7f1e001033c0 2024-03-26T08:53:15.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.146+0000 7f1dfcff9640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1de80a28d0 0x7f1de80a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.146+0000 7f1e0882e740 1 Processor -- start 2024-03-26T08:53:15.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.147+0000 7f1dfcff9640 1 -- 172.21.15.7:0/128836440 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1df80cf0b0 con 0x7f1e001033c0 2024-03-26T08:53:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.151+0000 7f1e0882e740 1 -- start start 2024-03-26T08:53:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.151+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e905d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.151+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e90ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.152+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e90d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.152+0000 7f1e0882e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674e845c90 con 0x55674e906350 2024-03-26T08:53:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.152+0000 7f1e0882e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674e86b070 con 0x55674e90b4c0 2024-03-26T08:53:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.152+0000 7f1e0882e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674e813c80 con 0x55674e905960 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.152+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1de80a28d0 0x7f1de80a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.153+0000 7f1dff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e905d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.153+0000 7f1dff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e905d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45958/0 (socket says 172.21.15.7:45958) 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.153+0000 7f1dff7fe640 1 -- 172.21.15.7:0/3354671347 learned_addr learned my addr 172.21.15.7:0/3354671347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.153+0000 7f1dff7fe640 1 -- 172.21.15.7:0/3354671347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 msgr2=0x55674e90d950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.153+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/3354671347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e90d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.153+0000 7f1dff7fe640 1 -- 172.21.15.7:0/3354671347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 msgr2=0x55674e90ae50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.153+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/3354671347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e90ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.153+0000 7f1dff7fe640 1 -- 172.21.15.7:0/3354671347 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55674e80bd10 con 0x55674e905960 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.154+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1de80a28d0 0x7f1de80a4d90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009900 tx=0x7f1df002c690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.154+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/3354671347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e905d80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e00069aa0 tx=0x7f1df004b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10b5f67ef1bc9081 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.156+0000 7f1de5ffb640 1 -- 172.21.15.7:0/3354671347 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df00044d0 con 0x55674e905960 2024-03-26T08:53:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.156+0000 7f1de5ffb640 1 -- 172.21.15.7:0/3354671347 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df0005510 con 0x55674e905960 2024-03-26T08:53:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.156+0000 7f1de5ffb640 1 -- 172.21.15.7:0/3354671347 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0003850 con 0x55674e905960 2024-03-26T08:53:15.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.159+0000 7f1e0882e740 1 -- 172.21.15.7:0/3354671347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 msgr2=0x55674e905d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.159+0000 7f1e0882e740 1 --2- 172.21.15.7:0/3354671347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e905d80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1e00069aa0 tx=0x7f1df004b000 comp rx=0 tx=0).stop 2024-03-26T08:53:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.163+0000 7f1e0882e740 1 -- 172.21.15.7:0/3354671347 shutdown_connections 2024-03-26T08:53:15.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.163+0000 7f1e0882e740 1 --2- 172.21.15.7:0/3354671347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e90d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.163+0000 7f1e0882e740 1 --2- 172.21.15.7:0/3354671347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e90ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.163+0000 7f1e0882e740 1 --2- 172.21.15.7:0/3354671347 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e905d80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.163+0000 7f1e0882e740 1 -- 172.21.15.7:0/3354671347 >> 172.21.15.7:0/3354671347 conn(0x55674e8ce1a0 msgr2=0x55674e9012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.165+0000 7f1e0882e740 1 -- 172.21.15.7:0/3354671347 shutdown_connections 2024-03-26T08:53:15.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.168+0000 7f1e0882e740 1 -- 172.21.15.7:0/3354671347 wait complete. 2024-03-26T08:53:15.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.168+0000 7f1e0882e740 1 -- start start 2024-03-26T08:53:15.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.168+0000 7f1e0882e740 1 Processor -- start 2024-03-26T08:53:15.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.169+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e905960 0x55674e920dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.169+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e906350 0x55674e921300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.169+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e928340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.169+0000 7f1e0882e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674e8270d0 con 0x55674e905960 2024-03-26T08:53:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.169+0000 7f1e0882e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674e867110 con 0x55674e90b4c0 2024-03-26T08:53:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.169+0000 7f1e0882e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55674e91f420 con 0x55674e906350 2024-03-26T08:53:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.169+0000 7f1e0882e740 10 client.0 _ll_register_callbacks cb 0x55674e862e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dfeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e906350 0x55674e921300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dfeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e906350 0x55674e921300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45970/0 (socket says 172.21.15.7:45970) 2024-03-26T08:53:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dfeffd640 1 -- 172.21.15.7:0/1496200329 learned_addr learned my addr 172.21.15.7:0/1496200329 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e905960 0x55674e920dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dfffff640 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e928340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dfeffd640 1 -- 172.21.15.7:0/1496200329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 msgr2=0x55674e928340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e928340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dfeffd640 1 -- 172.21.15.7:0/1496200329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e905960 msgr2=0x55674e920dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e905960 0x55674e920dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.170+0000 7f1dfeffd640 1 -- 172.21.15.7:0/1496200329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55674e6477e0 con 0x55674e906350 2024-03-26T08:53:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.171+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e906350 0x55674e921300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1df40116f0 tx=0x7f1df4011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be87b11682dd56d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.171+0000 7f1ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:53:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.171+0000 7f1ddeffd640 1 -- 172.21.15.7:0/1496200329 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4004440 con 0x55674e906350 2024-03-26T08:53:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.171+0000 7f1e0882e740 1 -- 172.21.15.7:0/1496200329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1df0005510 con 0x55674e906350 2024-03-26T08:53:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.171+0000 7f1ddeffd640 1 -- 172.21.15.7:0/1496200329 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1df400f8c0 con 0x55674e906350 2024-03-26T08:53:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.171+0000 7f1ddeffd640 1 -- 172.21.15.7:0/1496200329 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df400fac0 con 0x55674e906350 2024-03-26T08:53:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.172+0000 7f1e0882e740 10 client.28105 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.172+0000 7f1e0882e740 1 -- 172.21.15.7:0/1496200329 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55674e6a7f70 con 0x55674e906350 2024-03-26T08:53:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.173+0000 7f1ddeffd640 1 -- 172.21.15.7:0/1496200329 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df0005510 con 0x55674e906350 2024-03-26T08:53:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.173+0000 7f1ddeffd640 1 -- 172.21.15.7:0/1496200329 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1df4018070 con 0x55674e906350 2024-03-26T08:53:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.173+0000 7f1e0882e740 10 client.28105 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.173+0000 7f1e0882e740 10 client.28105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.173+0000 7f1e0882e740 20 client.28105 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.173+0000 7f1e0882e740 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55674e92d010 0x55674e92f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.173+0000 7f1e0882e740 4 client.28105 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.173+0000 7f1e0882e740 1 -- 172.21.15.7:0/1496200329 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55674e8785b0 con 0x55674e92d010 2024-03-26T08:53:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.173+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55674e92d010 0x55674e92f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.175+0000 7f1ddeffd640 10 client.28105 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:15.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.175+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55674e92d010 0x55674e92f450 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa0c592a99520a7 server_cookie=1a6119b4d0396a08 in_seq=0 out_seq=0 2024-03-26T08:53:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.176+0000 7f1ddeffd640 1 -- 172.21.15.7:0/1496200329 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55674e8785b0 con 0x55674e92d010 2024-03-26T08:53:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.176+0000 7f1ddeffd640 10 client.28105 handle_command_reply: tid=0 2024-03-26T08:53:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.176+0000 7f1e0882e740 1 client.28105 shutdown 2024-03-26T08:53:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.177+0000 7f1e0882e740 1 -- 172.21.15.7:0/1496200329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e906350 msgr2=0x55674e921300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.177+0000 7f1e0882e740 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e906350 0x55674e921300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1df40116f0 tx=0x7f1df4011bb0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.177+0000 7f1e0882e740 1 -- 172.21.15.7:0/1496200329 shutdown_connections 2024-03-26T08:53:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.177+0000 7f1e0882e740 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55674e92d010 0x55674e92f450 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.177+0000 7f1e0882e740 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e928340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.177+0000 7f1e0882e740 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e906350 0x55674e921300 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.177+0000 7f1e0882e740 1 --2- 172.21.15.7:0/1496200329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e905960 0x55674e920dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.177+0000 7f1e0882e740 1 -- 172.21.15.7:0/1496200329 >> 172.21.15.7:0/1496200329 conn(0x55674e8ce1a0 msgr2=0x55674e900530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.178+0000 7f1ddeffd640 0 client.28105 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.179+0000 7f1e0882e740 1 -- 172.21.15.7:0/1496200329 shutdown_connections 2024-03-26T08:53:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.179+0000 7f1e0882e740 1 -- 172.21.15.7:0/1496200329 wait complete. 2024-03-26T08:53:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.179+0000 7f1e0882e740 20 client.28105 trim_cache size 0 max 16384 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1e0882e740 1 Processor -- start 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1e0882e740 1 -- start start 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e904510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e928300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e904a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1e0882e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e001a0670 con 0x55674e906350 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1e0882e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000719b0 con 0x55674e90b4c0 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1e0882e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0006fa30 con 0x55674e905960 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1dfeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e928300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1dfeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e928300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40088/0 (socket says 172.21.15.7:40088) 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1dfeffd640 1 -- 172.21.15.7:0/2423924459 learned_addr learned my addr 172.21.15.7:0/2423924459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/2423924459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e904510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.199+0000 7f1dfffff640 1 --2- 172.21.15.7:0/2423924459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e904a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.200+0000 7f1dfeffd640 1 -- 172.21.15.7:0/2423924459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 msgr2=0x55674e904510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.200+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/2423924459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e904510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.200+0000 7f1dfeffd640 1 -- 172.21.15.7:0/2423924459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 msgr2=0x55674e904a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.200+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/2423924459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e904a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.200+0000 7f1dfeffd640 1 -- 172.21.15.7:0/2423924459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55674e53f170 con 0x55674e906350 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.200+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/2423924459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e928300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1df4001eb0 tx=0x7f1df400f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6374f8391ad699bd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.200+0000 7f1de57fa640 1 -- 172.21.15.7:0/2423924459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4000e60 con 0x55674e906350 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.200+0000 7f1de57fa640 1 -- 172.21.15.7:0/2423924459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df400fc60 con 0x55674e906350 2024-03-26T08:53:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.200+0000 7f1de57fa640 1 -- 172.21.15.7:0/2423924459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4003970 con 0x55674e906350 2024-03-26T08:53:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 -- 172.21.15.7:0/2423924459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 msgr2=0x55674e928300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 --2- 172.21.15.7:0/2423924459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e928300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1df4001eb0 tx=0x7f1df400f9c0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 -- 172.21.15.7:0/2423924459 shutdown_connections 2024-03-26T08:53:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 --2- 172.21.15.7:0/2423924459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e904a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 --2- 172.21.15.7:0/2423924459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e928300 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 --2- 172.21.15.7:0/2423924459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e904510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 -- 172.21.15.7:0/2423924459 >> 172.21.15.7:0/2423924459 conn(0x55674e8ce1a0 msgr2=0x55674e907920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 -- 172.21.15.7:0/2423924459 shutdown_connections 2024-03-26T08:53:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 -- 172.21.15.7:0/2423924459 wait complete. 2024-03-26T08:53:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.201+0000 7f1e0882e740 1 -- start start 2024-03-26T08:53:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.202+0000 7f1e0882e740 1 Processor -- start 2024-03-26T08:53:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.203+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e9256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e925c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1e0882e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e926bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e9256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1e0882e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00070030 con 0x55674e906350 2024-03-26T08:53:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1e0882e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00071350 con 0x55674e90b4c0 2024-03-26T08:53:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1e0882e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00070690 con 0x55674e905960 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dff7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e9256d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:45988/0 (socket says 172.21.15.7:45988) 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dff7fe640 1 -- 172.21.15.7:0/661223036 learned_addr learned my addr 172.21.15.7:0/661223036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e925c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1e0882e740 10 client.0 _ll_register_callbacks cb 0x7f1df40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dfffff640 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e926bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dff7fe640 1 -- 172.21.15.7:0/661223036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 msgr2=0x55674e926bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e926bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dff7fe640 1 -- 172.21.15.7:0/661223036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 msgr2=0x55674e925c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e925c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dff7fe640 1 -- 172.21.15.7:0/661223036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55674e9271c0 con 0x55674e905960 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e925c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.204+0000 7f1dfffff640 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e926bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.205+0000 7f1dff7fe640 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e9256d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e00069aa0 tx=0x7f1df0005170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f527b64030ed04 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.205+0000 7f1ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:53:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.205+0000 7f1ddeffd640 1 -- 172.21.15.7:0/661223036 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df005b070 con 0x55674e905960 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.205+0000 7f1ddeffd640 1 -- 172.21.15.7:0/661223036 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1df00574b0 con 0x55674e905960 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.205+0000 7f1e0882e740 1 -- 172.21.15.7:0/661223036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1df400fc60 con 0x55674e905960 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.205+0000 7f1ddeffd640 1 -- 172.21.15.7:0/661223036 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df00052a0 con 0x55674e905960 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.205+0000 7f1e0882e740 10 client.28111 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.205+0000 7f1e0882e740 1 -- 172.21.15.7:0/661223036 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55674e928550 con 0x55674e905960 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1ddeffd640 1 -- 172.21.15.7:0/661223036 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1df400fc60 con 0x55674e905960 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1ddeffd640 1 -- 172.21.15.7:0/661223036 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1df005e030 con 0x55674e905960 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1e0882e740 10 client.28111 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1e0882e740 10 client.28111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1e0882e740 20 client.28111 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1e0882e740 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55674e988cb0 0x55674e98b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1e0882e740 4 client.28111 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1e0882e740 1 -- 172.21.15.7:0/661223036 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55674e872900 con 0x55674e988cb0 2024-03-26T08:53:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55674e988cb0 0x55674e98b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1ddeffd640 10 client.28111 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.206+0000 7f1dfeffd640 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55674e988cb0 0x55674e98b0a0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f0833dbeb173ca server_cookie=de41ea8cfa17683a in_seq=0 out_seq=0 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.207+0000 7f1ddeffd640 1 -- 172.21.15.7:0/661223036 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55674e872900 con 0x55674e988cb0 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.207+0000 7f1ddeffd640 10 client.28111 handle_command_reply: tid=0 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.207+0000 7f1e0882e740 1 client.28111 shutdown 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 -- 172.21.15.7:0/661223036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 msgr2=0x55674e9256d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e9256d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e00069aa0 tx=0x7f1df0005170 comp rx=0 tx=0).stop 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 -- 172.21.15.7:0/661223036 shutdown_connections 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55674e988cb0 0x55674e98b0a0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55674e90b4c0 0x55674e926bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55674e906350 0x55674e925c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 --2- 172.21.15.7:0/661223036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55674e905960 0x55674e9256d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1ddeffd640 0 client.28111 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 -- 172.21.15.7:0/661223036 >> 172.21.15.7:0/661223036 conn(0x55674e8ce1a0 msgr2=0x55674e904510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 -- 172.21.15.7:0/661223036 shutdown_connections 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 1 -- 172.21.15.7:0/661223036 wait complete. 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e0882e740 20 client.28111 trim_cache size 0 max 16384 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "2e2f2173-61ac-4fc9-b11b-9745287fe8a6", 2024-03-26T08:53:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:53:15.208 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:53:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e05c44640 1 -- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1de80a28d0 msgr2=0x7f1de80a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e05c44640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1de80a28d0 0x7f1de80a4d90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009900 tx=0x7f1df002c690 comp rx=0 tx=0).stop 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e05c44640 1 -- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e001033c0 msgr2=0x7f1e0019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.208+0000 7f1e05c44640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e001033c0 0x7f1e0019efd0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1df800eb10 tx=0x7f1df800c2d0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.209+0000 7f1e05c44640 1 -- 172.21.15.7:0/128836440 shutdown_connections 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.209+0000 7f1e05c44640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1de80a28d0 0x7f1de80a4d90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.209+0000 7f1e05c44640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e001033c0 0x7f1e0019efd0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.209+0000 7f1e05c44640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e00102a00 0x7f1e00197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.209+0000 7f1e05c44640 1 --2- 172.21.15.7:0/128836440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1e000ff990 0x7f1e00197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.209+0000 7f1e05c44640 1 -- 172.21.15.7:0/128836440 >> 172.21.15.7:0/128836440 conn(0x7f1e000fb2c0 msgr2=0x7f1e00105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.209+0000 7f1e05c44640 1 -- 172.21.15.7:0/128836440 shutdown_connections 2024-03-26T08:53:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.209+0000 7f1e05c44640 1 -- 172.21.15.7:0/128836440 wait complete. 2024-03-26T08:53:15.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:53:15.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:53:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.380+0000 7f5abd8ea640 1 -- 172.21.15.7:0/2260447669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab810d9c0 msgr2=0x7f5ab810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.380+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/2260447669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab810d9c0 0x7f5ab810de40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009900 tx=0x7f5aa802bc60 comp rx=0 tx=0).stop 2024-03-26T08:53:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.387+0000 7f5abd8ea640 1 -- 172.21.15.7:0/2260447669 shutdown_connections 2024-03-26T08:53:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.387+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/2260447669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab810d9c0 0x7f5ab810de40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.387+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/2260447669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab810b010 0x7f5ab810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.387+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/2260447669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab81086a0 0x7f5ab810aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.387+0000 7f5abd8ea640 1 -- 172.21.15.7:0/2260447669 >> 172.21.15.7:0/2260447669 conn(0x7f5ab806e640 msgr2=0x7f5ab8070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.390+0000 7f5abd8ea640 1 -- 172.21.15.7:0/2260447669 shutdown_connections 2024-03-26T08:53:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.394+0000 7f5abd8ea640 1 -- 172.21.15.7:0/2260447669 wait complete. 2024-03-26T08:53:15.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.395+0000 7f5abd8ea640 1 Processor -- start 2024-03-26T08:53:15.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.398+0000 7f5abd8ea640 1 -- start start 2024-03-26T08:53:15.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.398+0000 7f5abd8ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab81086a0 0x7f5ab8114d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.398+0000 7f5abd8ea640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab810b010 0x7f5ab8115280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.398+0000 7f5abd8ea640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab810d9c0 0x7f5ab81b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.399+0000 7f5abd8ea640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab811c160 con 0x7f5ab810b010 2024-03-26T08:53:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.399+0000 7f5abd8ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab811bfe0 con 0x7f5ab81086a0 2024-03-26T08:53:15.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.399+0000 7f5abd8ea640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab811c2e0 con 0x7f5ab810d9c0 2024-03-26T08:53:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.401+0000 7f5ab67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab810b010 0x7f5ab8115280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.401+0000 7f5ab67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab810b010 0x7f5ab8115280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40112/0 (socket says 172.21.15.7:40112) 2024-03-26T08:53:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.401+0000 7f5ab67fc640 1 -- 172.21.15.7:0/4236858234 learned_addr learned my addr 172.21.15.7:0/4236858234 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.401+0000 7f5ab77fe640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab810d9c0 0x7f5ab81b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.403+0000 7f5ab67fc640 1 -- 172.21.15.7:0/4236858234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab810d9c0 msgr2=0x7f5ab81b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.403+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab810d9c0 0x7f5ab81b6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.403+0000 7f5ab67fc640 1 -- 172.21.15.7:0/4236858234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab81086a0 msgr2=0x7f5ab8114d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.404+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab81086a0 0x7f5ab8114d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.404+0000 7f5ab67fc640 1 -- 172.21.15.7:0/4236858234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab81b6db0 con 0x7f5ab810b010 2024-03-26T08:53:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.404+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab810b010 0x7f5ab8115280 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000eab0 tx=0x7f5ab000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.406+0000 7f5abc8e8640 1 -- 172.21.15.7:0/4236858234 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000ce40 con 0x7f5ab810b010 2024-03-26T08:53:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.406+0000 7f5abc8e8640 1 -- 172.21.15.7:0/4236858234 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab0011950 con 0x7f5ab810b010 2024-03-26T08:53:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.406+0000 7f5abc8e8640 1 -- 172.21.15.7:0/4236858234 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0010b40 con 0x7f5ab810b010 2024-03-26T08:53:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.406+0000 7f5abd8ea640 1 -- 172.21.15.7:0/4236858234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab81b6f80 con 0x7f5ab810b010 2024-03-26T08:53:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.406+0000 7f5abd8ea640 1 -- 172.21.15.7:0/4236858234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab81b7330 con 0x7f5ab810b010 2024-03-26T08:53:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.411+0000 7f5ac05b1740 1 Processor -- start 2024-03-26T08:53:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.411+0000 7f5ac05b1740 1 -- start start 2024-03-26T08:53:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.411+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.411+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4d7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.411+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d7900 0x55ebba4d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.411+0000 7f5ac05b1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebba4373d0 con 0x55ebba4d2790 2024-03-26T08:53:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.411+0000 7f5ac05b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebba3dfad0 con 0x55ebba4d1da0 2024-03-26T08:53:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.411+0000 7f5ac05b1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebba433470 con 0x55ebba4d7900 2024-03-26T08:53:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.412+0000 7f5ab77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.412+0000 7f5ab77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4d21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:41092/0 (socket says 172.21.15.7:41092) 2024-03-26T08:53:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.412+0000 7f5ab77fe640 1 -- 172.21.15.7:0/1007042140 learned_addr learned my addr 172.21.15.7:0/1007042140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.412+0000 7f5ab6ffd640 1 --2- 172.21.15.7:0/1007042140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.412+0000 7f5ab77fe640 1 -- 172.21.15.7:0/1007042140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d7900 msgr2=0x55ebba4d9d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.412+0000 7f5ab77fe640 1 --2- 172.21.15.7:0/1007042140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d7900 0x55ebba4d9d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.412+0000 7f5ab77fe640 1 -- 172.21.15.7:0/1007042140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 msgr2=0x55ebba4d7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.412+0000 7f5ab77fe640 1 --2- 172.21.15.7:0/1007042140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4d7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.412+0000 7f5ab77fe640 1 -- 172.21.15.7:0/1007042140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebba2157d0 con 0x55ebba4d1da0 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.413+0000 7f5abc8e8640 1 -- 172.21.15.7:0/4236858234 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5ab0010ce0 con 0x7f5ab810b010 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.413+0000 7f5ab77fe640 1 --2- 172.21.15.7:0/1007042140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4d21c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5aa802c910 tx=0x7f5aa802c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a335915b96f552 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.413+0000 7f5a9dffb640 1 -- 172.21.15.7:0/1007042140 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8042070 con 0x55ebba4d1da0 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.413+0000 7f5a9dffb640 1 -- 172.21.15.7:0/1007042140 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5aa802cbb0 con 0x55ebba4d1da0 2024-03-26T08:53:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.413+0000 7f5a9dffb640 1 -- 172.21.15.7:0/1007042140 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80046c0 con 0x55ebba4d1da0 2024-03-26T08:53:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.413+0000 7f5abc8e8640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa00a2a00 0x7f5aa00a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.414+0000 7f5abc8e8640 1 -- 172.21.15.7:0/4236858234 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5ab00cec70 con 0x7f5ab810b010 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.414+0000 7f5ab6ffd640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa00a2a00 0x7f5aa00a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.414+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1007042140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 msgr2=0x55ebba4d21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.414+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/1007042140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4d21c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5aa802c910 tx=0x7f5aa802c6b0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.414+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1007042140 shutdown_connections 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.414+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/1007042140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d7900 0x55ebba4d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.414+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/1007042140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.414+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/1007042140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4d21c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.414+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1007042140 >> 172.21.15.7:0/1007042140 conn(0x55ebba4cd310 msgr2=0x55ebba4cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.415+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1007042140 shutdown_connections 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.415+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1007042140 wait complete. 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.415+0000 7f5ac05b1740 1 -- start start 2024-03-26T08:53:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.415+0000 7f5ab6ffd640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa00a2a00 0x7f5aa00a4ec0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55ebba4d2b90 tx=0x7f5aac009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.416+0000 7f5ac05b1740 1 Processor -- start 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.416+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.416+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4dc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.416+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d7900 0x55ebba4e35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.417+0000 7f5ab67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4dc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.417+0000 7f5ab67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4dc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40124/0 (socket says 172.21.15.7:40124) 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.417+0000 7f5ab67fc640 1 -- 172.21.15.7:0/2940846794 learned_addr learned my addr 172.21.15.7:0/2940846794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.417+0000 7f5ab6ffd640 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.417+0000 7f5ab77fe640 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d7900 0x55ebba4e35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.417+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebba4da850 con 0x55ebba4d2790 2024-03-26T08:53:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.417+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebba3f3060 con 0x55ebba4d1da0 2024-03-26T08:53:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.417+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebba4da9d0 con 0x55ebba4d7900 2024-03-26T08:53:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.417+0000 7f5ac05b1740 10 client.0 _ll_register_callbacks cb 0x55ebba3d56a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5ab67fc640 1 -- 172.21.15.7:0/2940846794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d7900 msgr2=0x55ebba4e35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d7900 0x55ebba4e35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5ab67fc640 1 -- 172.21.15.7:0/2940846794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 msgr2=0x55ebba4dc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4dc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5ab67fc640 1 -- 172.21.15.7:0/2940846794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebba14b3f0 con 0x55ebba4d2790 2024-03-26T08:53:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4dc570 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0097a30 tx=0x7f5ab009b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cdfbd40b6c041d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2940846794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00b3030 con 0x55ebba4d2790 2024-03-26T08:53:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2940846794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5ab00b4270 con 0x55ebba4d2790 2024-03-26T08:53:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.418+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5aa802cbb0 con 0x55ebba4d2790 2024-03-26T08:53:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.419+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2940846794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ab00b4470 con 0x55ebba4d2790 2024-03-26T08:53:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.419+0000 7f5ac05b1740 10 client.18297 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.419+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ebba10b0f0 con 0x55ebba4d2790 2024-03-26T08:53:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.419+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2940846794 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab009f720 con 0x55ebba4d2790 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.420+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2940846794 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5ab0021020 con 0x55ebba4d2790 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.420+0000 7f5ac05b1740 10 client.18297 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.420+0000 7f5ac05b1740 10 client.18297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.420+0000 7f5ac05b1740 20 client.18297 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.420+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebba4e79f0 0x55ebba4e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.420+0000 7f5ac05b1740 4 client.18297 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.420+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5aa802cbb0 con 0x55ebba4e79f0 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.421+0000 7f5ab6ffd640 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebba4e79f0 0x55ebba4e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.421+0000 7f5a96ffd640 10 client.18297 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.421+0000 7f5ab6ffd640 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebba4e79f0 0x55ebba4e9e10 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215ed5f6ace7a248 server_cookie=a082d49d58a07e37 in_seq=0 out_seq=0 2024-03-26T08:53:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.423+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2940846794 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f5aa802cbb0 con 0x55ebba4e79f0 2024-03-26T08:53:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.423+0000 7f5a96ffd640 10 client.18297 handle_command_reply: tid=0 2024-03-26T08:53:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.423+0000 7f5ac05b1740 1 client.18297 shutdown 2024-03-26T08:53:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 msgr2=0x55ebba4dc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4dc570 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0097a30 tx=0x7f5ab009b060 comp rx=0 tx=0).stop 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 shutdown_connections 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebba4e79f0 0x55ebba4e9e10 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5a96ffd640 0 client.18297 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d7900 0x55ebba4e35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4dc570 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2940846794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d1da0 0x55ebba4dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 >> 172.21.15.7:0/2940846794 conn(0x55ebba4cd310 msgr2=0x55ebba4cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 shutdown_connections 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2940846794 wait complete. 2024-03-26T08:53:15.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.424+0000 7f5ac05b1740 20 client.18297 trim_cache size 0 max 16384 2024-03-26T08:53:15.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ac05b1740 1 Processor -- start 2024-03-26T08:53:15.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ac05b1740 1 -- start start 2024-03-26T08:53:15.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d1da0 0x55ebba4cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 0x55ebba4ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ac05b1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa40108c0 con 0x55ebba4d2790 2024-03-26T08:53:15.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ac05b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa400e3c0 con 0x55ebba4d7900 2024-03-26T08:53:15.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ac05b1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa400c440 con 0x55ebba4d1da0 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ab67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d1da0 0x55ebba4cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d1da0 0x55ebba4cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:46036/0 (socket says 172.21.15.7:46036) 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ab67fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4e3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40138/0 (socket says 172.21.15.7:40138) 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ab6ffd640 1 -- 172.21.15.7:0/1059811091 learned_addr learned my addr 172.21.15.7:0/1059811091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.447+0000 7f5ab77fe640 1 --2- 172.21.15.7:0/1059811091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 0x55ebba4ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.448+0000 7f5ab67fc640 1 -- 172.21.15.7:0/1059811091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d1da0 msgr2=0x55ebba4cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.448+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/1059811091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d1da0 0x55ebba4cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.448+0000 7f5ab67fc640 1 -- 172.21.15.7:0/1059811091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 msgr2=0x55ebba4ee6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.448+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/1059811091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 0x55ebba4ee6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.448+0000 7f5ab67fc640 1 -- 172.21.15.7:0/1059811091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebba4ccc60 con 0x55ebba4d2790 2024-03-26T08:53:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.448+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/1059811091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4e3570 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5ab009abe0 tx=0x7f5ab009b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9648bb8aab5473 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.448+0000 7f5a9d7fa640 1 -- 172.21.15.7:0/1059811091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab00b3070 con 0x55ebba4d2790 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.448+0000 7f5a9d7fa640 1 -- 172.21.15.7:0/1059811091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab009f720 con 0x55ebba4d2790 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.448+0000 7f5a9d7fa640 1 -- 172.21.15.7:0/1059811091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab009a6f0 con 0x55ebba4d2790 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.449+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1059811091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 msgr2=0x55ebba4e3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.449+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/1059811091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4e3570 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5ab009abe0 tx=0x7f5ab009b750 comp rx=0 tx=0).stop 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.449+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1059811091 shutdown_connections 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.449+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/1059811091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 0x55ebba4ee6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.449+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/1059811091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d2790 0x55ebba4e3570 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.449+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/1059811091 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d1da0 0x55ebba4cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.449+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1059811091 >> 172.21.15.7:0/1059811091 conn(0x55ebba4cd310 msgr2=0x55ebba4cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.449+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1059811091 shutdown_connections 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.449+0000 7f5ac05b1740 1 -- 172.21.15.7:0/1059811091 wait complete. 2024-03-26T08:53:15.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ac05b1740 1 -- start start 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ac05b1740 1 Processor -- start 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d1da0 0x55ebba4e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d2790 0x55ebba4e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d1da0 0x55ebba4e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ab6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d1da0 0x55ebba4e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40148/0 (socket says 172.21.15.7:40148) 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ac05b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 0x55ebba4e15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ac05b1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa400ca40 con 0x55ebba4d1da0 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ac05b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa400dd60 con 0x55ebba4d7900 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ac05b1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa400d0a0 con 0x55ebba4d2790 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ac05b1740 10 client.0 _ll_register_callbacks cb 0x7f5ab009b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.450+0000 7f5ab6ffd640 1 -- 172.21.15.7:0/2416604928 learned_addr learned my addr 172.21.15.7:0/2416604928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.451+0000 7f5ab77fe640 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 0x55ebba4e15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.451+0000 7f5ab6ffd640 1 -- 172.21.15.7:0/2416604928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d2790 msgr2=0x55ebba4e34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.451+0000 7f5ab6ffd640 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d2790 0x55ebba4e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.451+0000 7f5ab6ffd640 1 -- 172.21.15.7:0/2416604928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 msgr2=0x55ebba4e15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.451+0000 7f5ab6ffd640 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 0x55ebba4e15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.451+0000 7f5ab6ffd640 1 -- 172.21.15.7:0/2416604928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebba4e1e20 con 0x55ebba4d1da0 2024-03-26T08:53:15.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.451+0000 7f5ab77fe640 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 0x55ebba4e15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.453+0000 7f5a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:15.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.453+0000 7f5ab6ffd640 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d1da0 0x55ebba4e2f90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5aac026330 tx=0x7f5aac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab40e619b6a1470 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.454+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2416604928 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac035060 con 0x55ebba4d1da0 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.454+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2416604928 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5aac00f040 con 0x55ebba4d1da0 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.454+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2416604928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ab009f720 con 0x55ebba4d1da0 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.454+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2416604928 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aac01b040 con 0x55ebba4d1da0 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.454+0000 7f5ac05b1740 10 client.18303 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.454+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2416604928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ebba543ea0 con 0x55ebba4d1da0 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.455+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2416604928 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5ab009f720 con 0x55ebba4d1da0 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.455+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2416604928 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5aac022660 con 0x55ebba4d1da0 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.455+0000 7f5ac05b1740 10 client.18303 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.455+0000 7f5ac05b1740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.455+0000 7f5ac05b1740 20 client.18303 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.455+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebba544710 0x55ebba546bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.455+0000 7f5ac05b1740 4 client.18303 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.455+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2416604928 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ebba444910 con 0x55ebba544710 2024-03-26T08:53:15.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.455+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebba544710 0x55ebba546bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.456+0000 7f5a96ffd640 10 client.18303 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.456+0000 7f5ab67fc640 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebba544710 0x55ebba546bd0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c6ebb6f6201612 server_cookie=8e21dbb9e93c0f1a in_seq=0 out_seq=0 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.456+0000 7f5a96ffd640 1 -- 172.21.15.7:0/2416604928 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55ebba444910 con 0x55ebba544710 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.456+0000 7f5a96ffd640 10 client.18303 handle_command_reply: tid=0 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.456+0000 7f5ac05b1740 1 client.18303 shutdown 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.456+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2416604928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d1da0 msgr2=0x55ebba4e2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.456+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d1da0 0x55ebba4e2f90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5aac026330 tx=0x7f5aac015000 comp rx=0 tx=0).stop 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.456+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2416604928 shutdown_connections 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.456+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebba544710 0x55ebba546bd0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebba4d7900 0x55ebba4e15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5a96ffd640 0 client.18303 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:15.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebba4d2790 0x55ebba4e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5ac05b1740 1 --2- 172.21.15.7:0/2416604928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebba4d1da0 0x55ebba4e2f90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2416604928 >> 172.21.15.7:0/2416604928 conn(0x55ebba4cd310 msgr2=0x55ebba4cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2416604928 shutdown_connections 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5ac05b1740 1 -- 172.21.15.7:0/2416604928 wait complete. 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5ac05b1740 20 client.18303 trim_cache size 0 max 16384 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (140 inodes in the stack)", 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "2e2f2173-61ac-4fc9-b11b-9745287fe8a6": { 2024-03-26T08:53:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:53:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "2e2f2173-61ac-4fc9-b11b-9745287fe8a6", 2024-03-26T08:53:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:53:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:53:15.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:53:15.456 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5abd8ea640 1 -- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa00a2a00 msgr2=0x7f5aa00a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa00a2a00 0x7f5aa00a4ec0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55ebba4d2b90 tx=0x7f5aac009110 comp rx=0 tx=0).stop 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5abd8ea640 1 -- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab810b010 msgr2=0x7f5ab8115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab810b010 0x7f5ab8115280 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000eab0 tx=0x7f5ab000ef70 comp rx=0 tx=0).stop 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5abd8ea640 1 -- 172.21.15.7:0/4236858234 shutdown_connections 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5aa00a2a00 0x7f5aa00a4ec0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5ab810d9c0 0x7f5ab81b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.457+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ab810b010 0x7f5ab8115280 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.458+0000 7f5abd8ea640 1 --2- 172.21.15.7:0/4236858234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ab81086a0 0x7f5ab8114d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.458+0000 7f5abd8ea640 1 -- 172.21.15.7:0/4236858234 >> 172.21.15.7:0/4236858234 conn(0x7f5ab806e640 msgr2=0x7f5ab810d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.458+0000 7f5abd8ea640 1 -- 172.21.15.7:0/4236858234 shutdown_connections 2024-03-26T08:53:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:15.458+0000 7f5abd8ea640 1 -- 172.21.15.7:0/4236858234 wait complete. 2024-03-26T08:53:15.467 INFO:tasks.cephfs.filesystem:scrub status for tag:2e2f2173-61ac-4fc9-b11b-9745287fe8a6 - {'path': '/', 'tag': '2e2f2173-61ac-4fc9-b11b-9745287fe8a6', 'options': 'recursive,force'} 2024-03-26T08:53:15.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:15 smithi007 ceph-mon[32462]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 MiB/s wr, 618 op/s 2024-03-26T08:53:15.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:15 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:53:15.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:15 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:15.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:15 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:53:15.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:15 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:53:15.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:15 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:15.872 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:15 smithi044 ceph-mon[36461]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 MiB/s wr, 618 op/s 2024-03-26T08:53:15.872 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:15 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:53:15.872 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:15 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:15.872 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:15 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:53:15.872 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:15 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:53:15.872 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:15 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:15.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:15 smithi016 ceph-mon[35851]: pgmap v685: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.0 MiB/s wr, 618 op/s 2024-03-26T08:53:15.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:15 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:53:15.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:15 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:15.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:15 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:53:15.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:15 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:53:15.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:15 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:16.075 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 645.0 s, 1636.4 tps, lat 61.582 ms stddev 73.201 2024-03-26T08:53:16.898 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:16 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:17.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:16 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:17.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:16 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.455+0000 7f807f12f640 1 -- 172.21.15.7:0/1269107285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8078101700 msgr2=0x7f8078101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.455+0000 7f807f12f640 1 --2- 172.21.15.7:0/1269107285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8078101700 0x7f8078101b20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f806c009900 tx=0x7f806c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:53:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.456+0000 7f807f12f640 1 -- 172.21.15.7:0/1269107285 shutdown_connections 2024-03-26T08:53:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.456+0000 7f807f12f640 1 --2- 172.21.15.7:0/1269107285 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80781032b0 0x7f8078109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.456+0000 7f807f12f640 1 --2- 172.21.15.7:0/1269107285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80781028f0 0x7f8078102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.456+0000 7f807f12f640 1 --2- 172.21.15.7:0/1269107285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8078101700 0x7f8078101b20 secure :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f806c009900 tx=0x7f806c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:53:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.456+0000 7f807f12f640 1 -- 172.21.15.7:0/1269107285 >> 172.21.15.7:0/1269107285 conn(0x7f80780fcc70 msgr2=0x7f80780ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.457+0000 7f807f12f640 1 -- 172.21.15.7:0/1269107285 shutdown_connections 2024-03-26T08:53:17.454 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.457+0000 7f807f12f640 1 -- 172.21.15.7:0/1269107285 wait complete. 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807f12f640 1 Processor -- start 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807f12f640 1 -- start start 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807f12f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80781028f0 0x7f8078197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807f12f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80781032b0 0x7f8078198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807f12f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807819d180 0x7f807819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807f12f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807810d380 con 0x7f807819d180 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807f12f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807810d200 con 0x7f80781032b0 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807f12f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807810d500 con 0x7f80781028f0 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807d6cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807819d180 0x7f807819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807d6cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807819d180 0x7f807819f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52982/0 (socket says 172.21.15.7:52982) 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807d6cc640 1 -- 172.21.15.7:0/1012062266 learned_addr learned my addr 172.21.15.7:0/1012062266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f807cecb640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80781028f0 0x7f8078197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.458+0000 7f8077fff640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80781032b0 0x7f8078198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f807d6cc640 1 -- 172.21.15.7:0/1012062266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80781028f0 msgr2=0x7f8078197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f807d6cc640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80781028f0 0x7f8078197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f807d6cc640 1 -- 172.21.15.7:0/1012062266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80781032b0 msgr2=0x7f8078198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f807d6cc640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80781032b0 0x7f8078198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f807d6cc640 1 -- 172.21.15.7:0/1012062266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807819fc10 con 0x7f807819d180 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f8077fff640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80781032b0 0x7f8078198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f807cecb640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80781028f0 0x7f8078197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f807d6cc640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807819d180 0x7f807819f570 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f807000eb10 tx=0x7f807000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f8075ffb640 1 -- 172.21.15.7:0/1012062266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070004b20 con 0x7f807819d180 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f8075ffb640 1 -- 172.21.15.7:0/1012062266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8070004cc0 con 0x7f807819d180 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f807f12f640 1 -- 172.21.15.7:0/1012062266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807819ff00 con 0x7f807819d180 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f8075ffb640 1 -- 172.21.15.7:0/1012062266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070010790 con 0x7f807819d180 2024-03-26T08:53:17.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.459+0000 7f807f12f640 1 -- 172.21.15.7:0/1012062266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80781a0360 con 0x7f807819d180 2024-03-26T08:53:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.461+0000 7f8075ffb640 1 -- 172.21.15.7:0/1012062266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8070005020 con 0x7f807819d180 2024-03-26T08:53:17.458 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.462+0000 7f8081df1740 1 Processor -- start 2024-03-26T08:53:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.462+0000 7f8075ffb640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f80680a27f0 0x7f80680a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.463+0000 7f8075ffb640 1 -- 172.21.15.7:0/1012062266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f80700cefc0 con 0x7f807819d180 2024-03-26T08:53:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.464+0000 7f8081df1740 1 -- start start 2024-03-26T08:53:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.464+0000 7f8077fff640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f80680a27f0 0x7f80680a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f8081df1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd0920 0x564699bd0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f8081df1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bd5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f8081df1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699bd8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f8081df1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564699b35b10 con 0x564699bd1310 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f8081df1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564699ade900 con 0x564699bd6480 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f8081df1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564699b31bb0 con 0x564699bd0920 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f807cecb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd0920 0x564699bd0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f807cecb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd0920 0x564699bd0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35274/0 (socket says 172.21.15.7:35274) 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f807cecb640 1 -- 172.21.15.7:0/382829520 learned_addr learned my addr 172.21.15.7:0/382829520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f8077fff640 1 --2- 172.21.15.7:0/382829520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bd5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f8077fff640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f80680a27f0 0x7f80680a4cb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8064005d00 tx=0x7f806400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f807d6cc640 1 --2- 172.21.15.7:0/382829520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699bd8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f807cecb640 1 -- 172.21.15.7:0/382829520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 msgr2=0x564699bd8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f807cecb640 1 --2- 172.21.15.7:0/382829520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699bd8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f807cecb640 1 -- 172.21.15.7:0/382829520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 msgr2=0x564699bd5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f807cecb640 1 --2- 172.21.15.7:0/382829520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bd5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.465+0000 7f807cecb640 1 -- 172.21.15.7:0/382829520 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56469984a210 con 0x564699bd0920 2024-03-26T08:53:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.466+0000 7f807cecb640 1 --2- 172.21.15.7:0/382829520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd0920 0x564699bd0d40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f806c009230 tx=0x7f806c02cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1423e8d232c055d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.466+0000 7f8062ffd640 1 -- 172.21.15.7:0/382829520 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c042070 con 0x564699bd0920 2024-03-26T08:53:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.466+0000 7f8062ffd640 1 -- 172.21.15.7:0/382829520 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f806c004ba0 con 0x564699bd0920 2024-03-26T08:53:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.466+0000 7f8062ffd640 1 -- 172.21.15.7:0/382829520 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c0052b0 con 0x564699bd0920 2024-03-26T08:53:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 -- 172.21.15.7:0/382829520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd0920 msgr2=0x564699bd0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 --2- 172.21.15.7:0/382829520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd0920 0x564699bd0d40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f806c009230 tx=0x7f806c02cde0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 -- 172.21.15.7:0/382829520 shutdown_connections 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 --2- 172.21.15.7:0/382829520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699bd8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 --2- 172.21.15.7:0/382829520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bd5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 --2- 172.21.15.7:0/382829520 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd0920 0x564699bd0d40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 -- 172.21.15.7:0/382829520 >> 172.21.15.7:0/382829520 conn(0x564699bcbe90 msgr2=0x564699bcc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 -- 172.21.15.7:0/382829520 shutdown_connections 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 -- 172.21.15.7:0/382829520 wait complete. 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 -- start start 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.467+0000 7f8081df1740 1 Processor -- start 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8081df1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd0920 0x564699bdac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8081df1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bdb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8081df1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd6480 0x564699be21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8081df1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564699bd93d0 con 0x564699bd1310 2024-03-26T08:53:17.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8081df1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564699af1d50 con 0x564699bd0920 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8081df1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564699bd9550 con 0x564699bd6480 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8081df1740 10 client.0 _ll_register_callbacks cb 0x564699af1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8077fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bdb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f807cecb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd0920 0x564699bdac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f807d6cc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd6480 0x564699be21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8077fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bdb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53000/0 (socket says 172.21.15.7:53000) 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8077fff640 1 -- 172.21.15.7:0/3050660290 learned_addr learned my addr 172.21.15.7:0/3050660290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f807cecb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd0920 0x564699bdac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38178/0 (socket says 172.21.15.7:38178) 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8077fff640 1 -- 172.21.15.7:0/3050660290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd6480 msgr2=0x564699be21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8077fff640 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd6480 0x564699be21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8077fff640 1 -- 172.21.15.7:0/3050660290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd0920 msgr2=0x564699bdac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8077fff640 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd0920 0x564699bdac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8077fff640 1 -- 172.21.15.7:0/3050660290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564699916620 con 0x564699bd1310 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f807d6cc640 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd6480 0x564699be21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f807cecb640 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd0920 0x564699bdac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8077fff640 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bdb180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8064011bb0 tx=0x7f8064004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee9362686226d4b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.468+0000 7f8057fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.469+0000 7f8057fff640 1 -- 172.21.15.7:0/3050660290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806401ecc0 con 0x564699bd1310 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.469+0000 7f8057fff640 1 -- 172.21.15.7:0/3050660290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8064004020 con 0x564699bd1310 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.469+0000 7f8081df1740 1 -- 172.21.15.7:0/3050660290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f806c004ba0 con 0x564699bd1310 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.469+0000 7f8057fff640 1 -- 172.21.15.7:0/3050660290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f806401ee60 con 0x564699bd1310 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.469+0000 7f8081df1740 10 client.18315 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.469+0000 7f8081df1740 1 -- 172.21.15.7:0/3050660290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564699be5600 con 0x564699bd1310 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.469+0000 7f8057fff640 1 -- 172.21.15.7:0/3050660290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f806c004ba0 con 0x564699bd1310 2024-03-26T08:53:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.469+0000 7f8057fff640 1 -- 172.21.15.7:0/3050660290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f80640285a0 con 0x564699bd1310 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.470+0000 7f8081df1740 10 client.18315 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.470+0000 7f8081df1740 10 client.18315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.470+0000 7f8081df1740 20 client.18315 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.470+0000 7f8081df1740 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564699be6d20 0x564699be9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.470+0000 7f8081df1740 4 client.18315 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.470+0000 7f8081df1740 1 -- 172.21.15.7:0/3050660290 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564699b43050 con 0x564699be6d20 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.471+0000 7f807cecb640 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564699be6d20 0x564699be9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.471+0000 7f807cecb640 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564699be6d20 0x564699be9110 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c1f13f7c838477 server_cookie=4a326ec4c143e501 in_seq=0 out_seq=0 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.471+0000 7f8057fff640 10 client.18315 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8057fff640 1 -- 172.21.15.7:0/3050660290 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564699b43050 con 0x564699be6d20 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8057fff640 10 client.18315 handle_command_reply: tid=0 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 client.18315 shutdown 2024-03-26T08:53:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 -- 172.21.15.7:0/3050660290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 msgr2=0x564699bdb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bdb180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8064011bb0 tx=0x7f8064004730 comp rx=0 tx=0).stop 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 -- 172.21.15.7:0/3050660290 shutdown_connections 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564699be6d20 0x564699be9110 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8057fff640 0 client.18315 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd6480 0x564699be21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd1310 0x564699bdb180 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 --2- 172.21.15.7:0/3050660290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd0920 0x564699bdac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 -- 172.21.15.7:0/3050660290 >> 172.21.15.7:0/3050660290 conn(0x564699bcbe90 msgr2=0x564699bcb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 -- 172.21.15.7:0/3050660290 shutdown_connections 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 1 -- 172.21.15.7:0/3050660290 wait complete. 2024-03-26T08:53:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.472+0000 7f8081df1740 20 client.18315 trim_cache size 0 max 16384 2024-03-26T08:53:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.488+0000 7f8081df1740 1 Processor -- start 2024-03-26T08:53:17.485 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.488+0000 7f8081df1740 1 -- start start 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8081df1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bd26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8081df1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699be2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8081df1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699bd2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8081df1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781ab530 con 0x564699bd0920 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8081df1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781a9030 con 0x564699bd6480 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8081df1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781a70b0 con 0x564699bd1310 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8077fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699be2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f807cecb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bd26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8077fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699be2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:35294/0 (socket says 172.21.15.7:35294) 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8077fff640 1 -- 172.21.15.7:0/618906784 learned_addr learned my addr 172.21.15.7:0/618906784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f807d6cc640 1 --2- 172.21.15.7:0/618906784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699bd2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8077fff640 1 -- 172.21.15.7:0/618906784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 msgr2=0x564699bd2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8077fff640 1 --2- 172.21.15.7:0/618906784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699bd2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.487 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8077fff640 1 -- 172.21.15.7:0/618906784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 msgr2=0x564699bd26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8077fff640 1 --2- 172.21.15.7:0/618906784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bd26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.489+0000 7f8077fff640 1 -- 172.21.15.7:0/618906784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564699bccc00 con 0x564699bd1310 2024-03-26T08:53:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.490+0000 7f807cecb640 1 --2- 172.21.15.7:0/618906784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bd26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.490+0000 7f8077fff640 1 --2- 172.21.15.7:0/618906784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699be2180 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8064006010 tx=0x7f8064004660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61222cac90bedb3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.490+0000 7f80627fc640 1 -- 172.21.15.7:0/618906784 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8064038070 con 0x564699bd1310 2024-03-26T08:53:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.490+0000 7f80627fc640 1 -- 172.21.15.7:0/618906784 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f806401e850 con 0x564699bd1310 2024-03-26T08:53:17.488 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.490+0000 7f80627fc640 1 -- 172.21.15.7:0/618906784 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806401fb70 con 0x564699bd1310 2024-03-26T08:53:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 -- 172.21.15.7:0/618906784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 msgr2=0x564699be2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:17.490 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 --2- 172.21.15.7:0/618906784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699be2180 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8064006010 tx=0x7f8064004660 comp rx=0 tx=0).stop 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 -- 172.21.15.7:0/618906784 shutdown_connections 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 --2- 172.21.15.7:0/618906784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699bd2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 --2- 172.21.15.7:0/618906784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699be2180 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 --2- 172.21.15.7:0/618906784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bd26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 -- 172.21.15.7:0/618906784 >> 172.21.15.7:0/618906784 conn(0x564699bcbe90 msgr2=0x564699bcc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 -- 172.21.15.7:0/618906784 shutdown_connections 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 -- 172.21.15.7:0/618906784 wait complete. 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 -- start start 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.491+0000 7f8081df1740 1 Processor -- start 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f8081df1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bdf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f8081df1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699bdf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f8081df1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699be0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f8081df1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781a76b0 con 0x564699bd0920 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f8081df1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781a89d0 con 0x564699bd6480 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f8081df1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781a7d10 con 0x564699bd1310 2024-03-26T08:53:17.491 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f8081df1740 10 client.0 _ll_register_callbacks cb 0x7f8064011500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807cecb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bdf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807cecb640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bdf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53018/0 (socket says 172.21.15.7:53018) 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807cecb640 1 -- 172.21.15.7:0/1291402289 learned_addr learned my addr 172.21.15.7:0/1291402289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f8077fff640 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699bdf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807d6cc640 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699be0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807cecb640 1 -- 172.21.15.7:0/1291402289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 msgr2=0x564699bdf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807cecb640 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699bdf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807cecb640 1 -- 172.21.15.7:0/1291402289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 msgr2=0x564699be0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807cecb640 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699be0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807cecb640 1 -- 172.21.15.7:0/1291402289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564699be0f80 con 0x564699bd0920 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.492+0000 7f807d6cc640 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699be0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.493+0000 7f807cecb640 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bdf440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f806c005920 tx=0x7f806c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6811eee1426de308 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.493+0000 7f8057fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.493+0000 7f8057fff640 1 -- 172.21.15.7:0/1291402289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c059060 con 0x564699bd0920 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.493+0000 7f8057fff640 1 -- 172.21.15.7:0/1291402289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f806c0530d0 con 0x564699bd0920 2024-03-26T08:53:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.493+0000 7f8081df1740 1 -- 172.21.15.7:0/1291402289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f806401e850 con 0x564699bd0920 2024-03-26T08:53:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.493+0000 7f8057fff640 1 -- 172.21.15.7:0/1291402289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f806c042040 con 0x564699bd0920 2024-03-26T08:53:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.493+0000 7f8081df1740 10 client.18321 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.493+0000 7f8081df1740 1 -- 172.21.15.7:0/1291402289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564699be2300 con 0x564699bd0920 2024-03-26T08:53:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.493+0000 7f8057fff640 1 -- 172.21.15.7:0/1291402289 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f806401e850 con 0x564699bd0920 2024-03-26T08:53:17.493 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.494+0000 7f8057fff640 1 -- 172.21.15.7:0/1291402289 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f806c05e070 con 0x564699bd0920 2024-03-26T08:53:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.498+0000 7f8081df1740 10 client.18321 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.498+0000 7f8081df1740 10 client.18321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.498+0000 7f8081df1740 20 client.18321 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.498+0000 7f8081df1740 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564699be6d20 0x564699c447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.498+0000 7f8081df1740 4 client.18321 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:53:17.494 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.498+0000 7f8081df1740 1 -- 172.21.15.7:0/1291402289 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564699b3d3a0 con 0x564699be6d20 2024-03-26T08:53:17.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.521+0000 7f8077fff640 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564699be6d20 0x564699c447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.523+0000 7f8077fff640 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564699be6d20 0x564699c447e0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7f6415a70a3a1ac server_cookie=5f230fa0f1234fb9 in_seq=0 out_seq=0 2024-03-26T08:53:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.523+0000 7f8057fff640 10 client.18321 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8057fff640 1 -- 172.21.15.7:0/1291402289 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564699b3d3a0 con 0x564699be6d20 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8057fff640 10 client.18321 handle_command_reply: tid=0 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 client.18321 shutdown 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 -- 172.21.15.7:0/1291402289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 msgr2=0x564699bdf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bdf440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f806c005920 tx=0x7f806c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 -- 172.21.15.7:0/1291402289 shutdown_connections 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564699be6d20 0x564699c447e0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564699bd6480 0x564699be0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564699bd1310 0x564699bdf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 --2- 172.21.15.7:0/1291402289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564699bd0920 0x564699bdf440 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8057fff640 0 client.18321 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 -- 172.21.15.7:0/1291402289 >> 172.21.15.7:0/1291402289 conn(0x564699bcbe90 msgr2=0x564699bd26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 -- 172.21.15.7:0/1291402289 shutdown_connections 2024-03-26T08:53:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 1 -- 172.21.15.7:0/1291402289 wait complete. 2024-03-26T08:53:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.556+0000 7f8081df1740 20 client.18321 trim_cache size 0 max 16384 2024-03-26T08:53:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:53:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:53:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:53:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 -- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f80680a27f0 msgr2=0x7f80680a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f80680a27f0 0x7f80680a4cb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8064005d00 tx=0x7f806400b410 comp rx=0 tx=0).stop 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 -- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807819d180 msgr2=0x7f807819f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807819d180 0x7f807819f570 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f807000eb10 tx=0x7f807000c2d0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 -- 172.21.15.7:0/1012062266 shutdown_connections 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f80680a27f0 0x7f80680a4cb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807819d180 0x7f807819f570 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80781032b0 0x7f8078198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 --2- 172.21.15.7:0/1012062266 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80781028f0 0x7f8078197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 -- 172.21.15.7:0/1012062266 >> 172.21.15.7:0/1012062266 conn(0x7f80780fcc70 msgr2=0x7f8078105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 -- 172.21.15.7:0/1012062266 shutdown_connections 2024-03-26T08:53:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:17.557+0000 7f807f12f640 1 -- 172.21.15.7:0/1012062266 wait complete. 2024-03-26T08:53:17.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:17 smithi016 ceph-mon[35851]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.4 MiB/s wr, 666 op/s 2024-03-26T08:53:18.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:17 smithi007 ceph-mon[32462]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.4 MiB/s wr, 666 op/s 2024-03-26T08:53:18.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:17 smithi044 ceph-mon[36461]: pgmap v686: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.4 MiB/s wr, 666 op/s 2024-03-26T08:53:19.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:19 smithi016 ceph-mon[35851]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 654 op/s 2024-03-26T08:53:20.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:19 smithi007 ceph-mon[32462]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 654 op/s 2024-03-26T08:53:20.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:19 smithi044 ceph-mon[36461]: pgmap v687: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.7 MiB/s wr, 654 op/s 2024-03-26T08:53:21.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 650.0 s, 2290.8 tps, lat 43.870 ms stddev 48.589 2024-03-26T08:53:21.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:21 smithi016 ceph-mon[35851]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.3 MiB/s wr, 718 op/s 2024-03-26T08:53:21.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:21 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:53:21.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:21 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:53:22.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:21 smithi007 ceph-mon[32462]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.3 MiB/s wr, 718 op/s 2024-03-26T08:53:22.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:21 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:53:22.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:21 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:53:22.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:21 smithi044 ceph-mon[36461]: pgmap v688: 97 pgs: 97 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.3 MiB/s wr, 718 op/s 2024-03-26T08:53:22.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:21 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:53:22.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:21 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:53:23.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:23 smithi016 ceph-mon[35851]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.2 MiB/s wr, 794 op/s 2024-03-26T08:53:24.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:23 smithi007 ceph-mon[32462]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.2 MiB/s wr, 794 op/s 2024-03-26T08:53:24.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:23 smithi044 ceph-mon[36461]: pgmap v689: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.2 MiB/s wr, 794 op/s 2024-03-26T08:53:25.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:25 smithi016 ceph-mon[35851]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.4 MiB/s wr, 719 op/s 2024-03-26T08:53:26.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:25 smithi007 ceph-mon[32462]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.4 MiB/s wr, 719 op/s 2024-03-26T08:53:26.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:25 smithi044 ceph-mon[36461]: pgmap v690: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.4 MiB/s wr, 719 op/s 2024-03-26T08:53:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 655.0 s, 2223.6 tps, lat 44.463 ms stddev 51.854 2024-03-26T08:53:27.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:27 smithi016 ceph-mon[35851]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.7 MiB/s wr, 747 op/s 2024-03-26T08:53:28.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:27 smithi007 ceph-mon[32462]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.7 MiB/s wr, 747 op/s 2024-03-26T08:53:28.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:27 smithi044 ceph-mon[36461]: pgmap v691: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.7 MiB/s wr, 747 op/s 2024-03-26T08:53:29.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:29 smithi016 ceph-mon[35851]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.4 MiB/s wr, 714 op/s 2024-03-26T08:53:30.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:29 smithi007 ceph-mon[32462]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.4 MiB/s wr, 714 op/s 2024-03-26T08:53:30.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:29 smithi044 ceph-mon[36461]: pgmap v692: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.4 MiB/s wr, 714 op/s 2024-03-26T08:53:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.667+0000 7fe0319b2640 1 -- 172.21.15.7:0/3566411563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1017f0 msgr2=0x7fe02c101c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.667+0000 7fe0319b2640 1 --2- 172.21.15.7:0/3566411563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1017f0 0x7fe02c101c70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe014009900 tx=0x7fe01402bc30 comp rx=0 tx=0).stop 2024-03-26T08:53:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.667+0000 7fe0319b2640 1 -- 172.21.15.7:0/3566411563 shutdown_connections 2024-03-26T08:53:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.667+0000 7fe0319b2640 1 --2- 172.21.15.7:0/3566411563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02c1021b0 0x7fe02c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.667+0000 7fe0319b2640 1 --2- 172.21.15.7:0/3566411563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1017f0 0x7fe02c101c70 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.667+0000 7fe0319b2640 1 --2- 172.21.15.7:0/3566411563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02c100600 0x7fe02c100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.667+0000 7fe0319b2640 1 -- 172.21.15.7:0/3566411563 >> 172.21.15.7:0/3566411563 conn(0x7fe02c0fbb70 msgr2=0x7fe02c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.667+0000 7fe0319b2640 1 -- 172.21.15.7:0/3566411563 shutdown_connections 2024-03-26T08:53:30.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.667+0000 7fe0319b2640 1 -- 172.21.15.7:0/3566411563 wait complete. 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0319b2640 1 Processor -- start 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0319b2640 1 -- start start 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0319b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02c100600 0x7fe02c197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0319b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1021b0 0x7fe02c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0319b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02c19d190 0x7fe02c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0319b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c10c2c0 con 0x7fe02c1021b0 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0319b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c10c140 con 0x7fe02c100600 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0319b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c10c440 con 0x7fe02c19d190 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0227fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1021b0 0x7fe02c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0227fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1021b0 0x7fe02c198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57698/0 (socket says 172.21.15.7:57698) 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe0227fc640 1 -- 172.21.15.7:0/2220255208 learned_addr learned my addr 172.21.15.7:0/2220255208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe02affd640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02c100600 0x7fe02c197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.668+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02c19d190 0x7fe02c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0227fc640 1 -- 172.21.15.7:0/2220255208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02c19d190 msgr2=0x7fe02c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0227fc640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02c19d190 0x7fe02c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0227fc640 1 -- 172.21.15.7:0/2220255208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02c100600 msgr2=0x7fe02c197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0227fc640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02c100600 0x7fe02c197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0227fc640 1 -- 172.21.15.7:0/2220255208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c19fbf0 con 0x7fe02c1021b0 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe02affd640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02c100600 0x7fe02c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0227fc640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1021b0 0x7fe02c198130 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe01402c240 tx=0x7fe01402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0309b0640 1 -- 172.21.15.7:0/2220255208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014043020 con 0x7fe02c1021b0 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0319b2640 1 -- 172.21.15.7:0/2220255208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c19fe80 con 0x7fe02c1021b0 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0309b0640 1 -- 172.21.15.7:0/2220255208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe0140054e0 con 0x7fe02c1021b0 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0309b0640 1 -- 172.21.15.7:0/2220255208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe014003b50 con 0x7fe02c1021b0 2024-03-26T08:53:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.669+0000 7fe0319b2640 1 -- 172.21.15.7:0/2220255208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c1a02f0 con 0x7fe02c1021b0 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe03459f740 1 Processor -- start 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe03459f740 1 -- start start 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe03459f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55ca920 0x55a1b55cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe03459f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe03459f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55d0480 0x55a1b55d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe03459f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b552fb10 con 0x55a1b55ca920 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe03459f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b54d8900 con 0x55a1b55cb310 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe03459f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b552bbb0 con 0x55a1b55d0480 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55ca920 0x55a1b55cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55ca920 0x55a1b55cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57702/0 (socket says 172.21.15.7:57702) 2024-03-26T08:53:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55cfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53316/0 (socket says 172.21.15.7:53316) 2024-03-26T08:53:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02b7fe640 1 -- 172.21.15.7:0/3764103432 learned_addr learned my addr 172.21.15.7:0/3764103432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe0309b0640 1 -- 172.21.15.7:0/2220255208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe014046020 con 0x7fe02c1021b0 2024-03-26T08:53:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02affd640 1 -- 172.21.15.7:0/3764103432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55d0480 msgr2=0x55a1b55d2910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02affd640 1 --2- 172.21.15.7:0/3764103432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55d0480 0x55a1b55d2910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02affd640 1 -- 172.21.15.7:0/3764103432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55ca920 msgr2=0x55a1b55cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02affd640 1 --2- 172.21.15.7:0/3764103432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55ca920 0x55a1b55cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.673+0000 7fe02affd640 1 -- 172.21.15.7:0/3764103432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1b5244210 con 0x55a1b55cb310 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.674+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/3764103432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55ca920 0x55a1b55cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.674+0000 7fe02affd640 1 --2- 172.21.15.7:0/3764103432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55cfe10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009fa0 tx=0x7fe01c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=531a53987e648f6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.674+0000 7fe021ffb640 1 -- 172.21.15.7:0/3764103432 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c004310 con 0x55a1b55cb310 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.674+0000 7fe0309b0640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0180a27f0 0x7fe0180a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.674+0000 7fe021ffb640 1 -- 172.21.15.7:0/3764103432 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe01c01a070 con 0x55a1b55cb310 2024-03-26T08:53:30.672 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.674+0000 7fe021ffb640 1 -- 172.21.15.7:0/3764103432 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c004e40 con 0x55a1b55cb310 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.674+0000 7fe0309b0640 1 -- 172.21.15.7:0/2220255208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe0140f0790 con 0x7fe02c1021b0 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.674+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0180a27f0 0x7fe0180a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.675+0000 7fe03459f740 1 -- 172.21.15.7:0/3764103432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 msgr2=0x55a1b55cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.675+0000 7fe03459f740 1 --2- 172.21.15.7:0/3764103432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55cfe10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009fa0 tx=0x7fe01c0118d0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.676+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0180a27f0 0x7fe0180a4cb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55a1b55cbb70 tx=0x7fe024009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.676+0000 7fe03459f740 1 -- 172.21.15.7:0/3764103432 shutdown_connections 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.676+0000 7fe03459f740 1 --2- 172.21.15.7:0/3764103432 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55d0480 0x55a1b55d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.676+0000 7fe03459f740 1 --2- 172.21.15.7:0/3764103432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55cfe10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.676+0000 7fe03459f740 1 --2- 172.21.15.7:0/3764103432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55ca920 0x55a1b55cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.676+0000 7fe03459f740 1 -- 172.21.15.7:0/3764103432 >> 172.21.15.7:0/3764103432 conn(0x55a1b55c5e90 msgr2=0x55a1b55c6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.676+0000 7fe03459f740 1 -- 172.21.15.7:0/3764103432 shutdown_connections 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.676+0000 7fe03459f740 1 -- 172.21.15.7:0/3764103432 wait complete. 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe03459f740 1 -- start start 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe03459f740 1 Processor -- start 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe03459f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 0x55a1b55d4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe03459f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe03459f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe03459f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b55d3430 con 0x55a1b55d0480 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe03459f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b54ebd50 con 0x55a1b55cb310 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe03459f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1b55d35b0 con 0x55a1b55ca920 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe03459f740 10 client.0 _ll_register_callbacks cb 0x55a1b54eb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe02affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 0x55a1b55d4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe02b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe0227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe02affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 0x55a1b55d4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58722/0 (socket says 172.21.15.7:58722) 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.677+0000 7fe02affd640 1 -- 172.21.15.7:0/3198395696 learned_addr learned my addr 172.21.15.7:0/3198395696 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe02b7fe640 1 -- 172.21.15.7:0/3198395696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 msgr2=0x55a1b55d4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 0x55a1b55d4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe02b7fe640 1 -- 172.21.15.7:0/3198395696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 msgr2=0x55a1b55d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe02b7fe640 1 -- 172.21.15.7:0/3198395696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1b5310620 con 0x55a1b55d0480 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe02affd640 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 0x55a1b55d4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe0227fc640 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dc260 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe024009fd0 tx=0x7fe024004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0a0da57fe67069 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe00affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe00affd640 1 -- 172.21.15.7:0/3198395696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0240043d0 con 0x55a1b55d0480 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe00affd640 1 -- 172.21.15.7:0/3198395696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe024004570 con 0x55a1b55d0480 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe03459f740 1 -- 172.21.15.7:0/3198395696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe01c01a070 con 0x55a1b55d0480 2024-03-26T08:53:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.678+0000 7fe00affd640 1 -- 172.21.15.7:0/3198395696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe024004770 con 0x55a1b55d0480 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.679+0000 7fe03459f740 10 client.18333 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.679+0000 7fe03459f740 1 -- 172.21.15.7:0/3198395696 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a1b55df6a0 con 0x55a1b55d0480 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.679+0000 7fe00affd640 1 -- 172.21.15.7:0/3198395696 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe01c01a070 con 0x55a1b55d0480 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.679+0000 7fe00affd640 1 -- 172.21.15.7:0/3198395696 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe024026070 con 0x55a1b55d0480 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.680+0000 7fe03459f740 10 client.18333 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.680+0000 7fe03459f740 10 client.18333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.680+0000 7fe03459f740 20 client.18333 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.680+0000 7fe03459f740 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1b55e0d90 0x55a1b55e3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.680+0000 7fe03459f740 4 client.18333 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.680+0000 7fe03459f740 1 -- 172.21.15.7:0/3198395696 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1b553d050 con 0x55a1b55e0d90 2024-03-26T08:53:30.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.680+0000 7fe02affd640 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1b55e0d90 0x55a1b55e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.681+0000 7fe02affd640 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1b55e0d90 0x55a1b55e3180 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566085b5dc5427cf server_cookie=48a1957c2e34faf1 in_seq=0 out_seq=0 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.681+0000 7fe00affd640 10 client.18333 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe00affd640 1 -- 172.21.15.7:0/3198395696 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a1b553d050 con 0x55a1b55e0d90 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe00affd640 10 client.18333 handle_command_reply: tid=0 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 client.18333 shutdown 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 -- 172.21.15.7:0/3198395696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 msgr2=0x55a1b55dc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dc260 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe024009fd0 tx=0x7fe024004ab0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 -- 172.21.15.7:0/3198395696 shutdown_connections 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1b55e0d90 0x55a1b55e3180 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dc260 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe00affd640 0 client.18333 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 --2- 172.21.15.7:0/3198395696 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 0x55a1b55d4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 -- 172.21.15.7:0/3198395696 >> 172.21.15.7:0/3198395696 conn(0x55a1b55c5e90 msgr2=0x55a1b55c5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:30.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 -- 172.21.15.7:0/3198395696 shutdown_connections 2024-03-26T08:53:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 1 -- 172.21.15.7:0/3198395696 wait complete. 2024-03-26T08:53:30.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.682+0000 7fe03459f740 20 client.18333 trim_cache size 0 max 16384 2024-03-26T08:53:30.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.713+0000 7fe03459f740 1 Processor -- start 2024-03-26T08:53:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.720+0000 7fe03459f740 1 -- start start 2024-03-26T08:53:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.720+0000 7fe03459f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 0x55a1b55c6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.720+0000 7fe03459f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.720+0000 7fe03459f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.720+0000 7fe03459f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1a93e0 con 0x55a1b55d0480 2024-03-26T08:53:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.720+0000 7fe03459f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1a6ee0 con 0x55a1b55cb310 2024-03-26T08:53:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.720+0000 7fe03459f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1a4f60 con 0x55a1b55ca920 2024-03-26T08:53:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.721+0000 7fe0227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.721+0000 7fe0227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55dc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53340/0 (socket says 172.21.15.7:53340) 2024-03-26T08:53:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.721+0000 7fe0227fc640 1 -- 172.21.15.7:0/4106608905 learned_addr learned my addr 172.21.15.7:0/4106608905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.723+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/4106608905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.723+0000 7fe02b7fe640 1 -- 172.21.15.7:0/4106608905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 msgr2=0x55a1b55c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.723+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/4106608905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 0x55a1b55c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.723+0000 7fe02b7fe640 1 -- 172.21.15.7:0/4106608905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 msgr2=0x55a1b55dc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.723+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/4106608905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55dc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.723+0000 7fe02b7fe640 1 -- 172.21.15.7:0/4106608905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1b55cb6f0 con 0x55a1b55d0480 2024-03-26T08:53:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.724+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/4106608905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55cc710 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe024000f80 tx=0x7fe024003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bb3d85989b7108 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.728+0000 7fe0217fa640 1 -- 172.21.15.7:0/4106608905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024037030 con 0x55a1b55d0480 2024-03-26T08:53:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.728+0000 7fe0217fa640 1 -- 172.21.15.7:0/4106608905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe024009570 con 0x55a1b55d0480 2024-03-26T08:53:30.725 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.728+0000 7fe0217fa640 1 -- 172.21.15.7:0/4106608905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024028860 con 0x55a1b55d0480 2024-03-26T08:53:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.729+0000 7fe03459f740 1 -- 172.21.15.7:0/4106608905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 msgr2=0x55a1b55cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.729+0000 7fe03459f740 1 --2- 172.21.15.7:0/4106608905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55cc710 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe024000f80 tx=0x7fe024003f50 comp rx=0 tx=0).stop 2024-03-26T08:53:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.730+0000 7fe03459f740 1 -- 172.21.15.7:0/4106608905 shutdown_connections 2024-03-26T08:53:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.730+0000 7fe03459f740 1 --2- 172.21.15.7:0/4106608905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55cc710 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.730+0000 7fe03459f740 1 --2- 172.21.15.7:0/4106608905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55cb310 0x55a1b55dc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.730+0000 7fe03459f740 1 --2- 172.21.15.7:0/4106608905 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55ca920 0x55a1b55c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.730+0000 7fe03459f740 1 -- 172.21.15.7:0/4106608905 >> 172.21.15.7:0/4106608905 conn(0x55a1b55c5e90 msgr2=0x55a1b55c0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:30.729 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.733+0000 7fe03459f740 1 -- 172.21.15.7:0/4106608905 shutdown_connections 2024-03-26T08:53:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.734+0000 7fe03459f740 1 -- 172.21.15.7:0/4106608905 wait complete. 2024-03-26T08:53:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.734+0000 7fe03459f740 1 -- start start 2024-03-26T08:53:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.734+0000 7fe03459f740 1 Processor -- start 2024-03-26T08:53:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.735+0000 7fe03459f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55ca920 0x55a1b55d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.735+0000 7fe03459f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55cb310 0x55a1b55d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.735+0000 7fe03459f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.735+0000 7fe03459f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1a5560 con 0x55a1b55d0480 2024-03-26T08:53:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.735+0000 7fe03459f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1a6880 con 0x55a1b55ca920 2024-03-26T08:53:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.735+0000 7fe03459f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1a5bc0 con 0x55a1b55cb310 2024-03-26T08:53:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.735+0000 7fe03459f740 10 client.0 _ll_register_callbacks cb 0x7fe024003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.736+0000 7fe0227fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55cb310 0x55a1b55d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.736+0000 7fe02affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55ca920 0x55a1b55d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.736+0000 7fe02affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55ca920 0x55a1b55d7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53356/0 (socket says 172.21.15.7:53356) 2024-03-26T08:53:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.736+0000 7fe02affd640 1 -- 172.21.15.7:0/3274271831 learned_addr learned my addr 172.21.15.7:0/3274271831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.738+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.738+0000 7fe02affd640 1 -- 172.21.15.7:0/3274271831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55cb310 msgr2=0x55a1b55d7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.738+0000 7fe02affd640 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55cb310 0x55a1b55d7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.738+0000 7fe02affd640 1 -- 172.21.15.7:0/3274271831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 msgr2=0x55a1b55dae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.738+0000 7fe02affd640 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.738+0000 7fe02affd640 1 -- 172.21.15.7:0/3274271831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1b55db460 con 0x55a1b55ca920 2024-03-26T08:53:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.738+0000 7fe02b7fe640 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:53:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.738+0000 7fe02affd640 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55ca920 0x55a1b55d7910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe01c005890 tx=0x7fe01c009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=280e529ae86beb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.739+0000 7fe00affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:53:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.739+0000 7fe00affd640 1 -- 172.21.15.7:0/3274271831 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c023040 con 0x55a1b55ca920 2024-03-26T08:53:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.740+0000 7fe00affd640 1 -- 172.21.15.7:0/3274271831 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe01c030930 con 0x55a1b55ca920 2024-03-26T08:53:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.741+0000 7fe03459f740 1 -- 172.21.15.7:0/3274271831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe024009570 con 0x55a1b55ca920 2024-03-26T08:53:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.743+0000 7fe00affd640 1 -- 172.21.15.7:0/3274271831 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe01c037050 con 0x55a1b55ca920 2024-03-26T08:53:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.744+0000 7fe03459f740 10 client.28154 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.744+0000 7fe03459f740 1 -- 172.21.15.7:0/3274271831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a1b5655e90 con 0x55a1b55ca920 2024-03-26T08:53:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.744+0000 7fe00affd640 1 -- 172.21.15.7:0/3274271831 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe024009570 con 0x55a1b55ca920 2024-03-26T08:53:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.745+0000 7fe00affd640 1 -- 172.21.15.7:0/3274271831 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe01c029040 con 0x55a1b55ca920 2024-03-26T08:53:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.745+0000 7fe03459f740 10 client.28154 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.745+0000 7fe03459f740 10 client.28154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.745+0000 7fe03459f740 20 client.28154 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.745+0000 7fe03459f740 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1b5656580 0x55a1b56589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.745+0000 7fe03459f740 4 client.28154 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:53:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.745+0000 7fe03459f740 1 -- 172.21.15.7:0/3274271831 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a1b55373a0 con 0x55a1b5656580 2024-03-26T08:53:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.745+0000 7fe0227fc640 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1b5656580 0x55a1b56589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.746+0000 7fe0227fc640 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1b5656580 0x55a1b56589a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2886a69e9e5c4f6 server_cookie=b55bc8ff64659f93 in_seq=0 out_seq=0 2024-03-26T08:53:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.746+0000 7fe00affd640 10 client.28154 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.772+0000 7fe00affd640 1 -- 172.21.15.7:0/3274271831 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a1b55373a0 con 0x55a1b5656580 2024-03-26T08:53:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.772+0000 7fe00affd640 10 client.28154 handle_command_reply: tid=0 2024-03-26T08:53:30.769 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.772+0000 7fe03459f740 1 client.28154 shutdown 2024-03-26T08:53:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 -- 172.21.15.7:0/3274271831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55ca920 msgr2=0x55a1b55d7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55ca920 0x55a1b55d7910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe01c005890 tx=0x7fe01c009c40 comp rx=0 tx=0).stop 2024-03-26T08:53:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 -- 172.21.15.7:0/3274271831 shutdown_connections 2024-03-26T08:53:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55a1b5656580 0x55a1b56589a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a1b55d0480 0x55a1b55dae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55a1b55cb310 0x55a1b55d7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 --2- 172.21.15.7:0/3274271831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a1b55ca920 0x55a1b55d7910 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 -- 172.21.15.7:0/3274271831 >> 172.21.15.7:0/3274271831 conn(0x55a1b55c5e90 msgr2=0x55a1b55c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe00affd640 0 client.28154 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 -- 172.21.15.7:0/3274271831 shutdown_connections 2024-03-26T08:53:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 1 -- 172.21.15.7:0/3274271831 wait complete. 2024-03-26T08:53:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.773+0000 7fe03459f740 20 client.28154 trim_cache size 0 max 16384 2024-03-26T08:53:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:53:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:53:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:53:30.771 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:53:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 -- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0180a27f0 msgr2=0x7fe0180a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0180a27f0 0x7fe0180a4cb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55a1b55cbb70 tx=0x7fe024009500 comp rx=0 tx=0).stop 2024-03-26T08:53:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 -- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1021b0 msgr2=0x7fe02c198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1021b0 0x7fe02c198130 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe01402c240 tx=0x7fe01402c6c0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.772 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 -- 172.21.15.7:0/2220255208 shutdown_connections 2024-03-26T08:53:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe0180a27f0 0x7fe0180a4cb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe02c19d190 0x7fe02c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe02c1021b0 0x7fe02c198130 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 --2- 172.21.15.7:0/2220255208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02c100600 0x7fe02c197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 -- 172.21.15.7:0/2220255208 >> 172.21.15.7:0/2220255208 conn(0x7fe02c0fbb70 msgr2=0x7fe02c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 -- 172.21.15.7:0/2220255208 shutdown_connections 2024-03-26T08:53:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:30.774+0000 7fe0319b2640 1 -- 172.21.15.7:0/2220255208 wait complete. 2024-03-26T08:53:31.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 660.0 s, 2357.6 tps, lat 42.869 ms stddev 50.750 2024-03-26T08:53:31.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:31 smithi016 ceph-mon[35851]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.3 MiB/s wr, 684 op/s 2024-03-26T08:53:32.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:31 smithi007 ceph-mon[32462]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.3 MiB/s wr, 684 op/s 2024-03-26T08:53:32.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:31 smithi044 ceph-mon[36461]: pgmap v693: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.3 MiB/s wr, 684 op/s 2024-03-26T08:53:33.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:33 smithi016 ceph-mon[35851]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 734 op/s 2024-03-26T08:53:34.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:33 smithi007 ceph-mon[32462]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 734 op/s 2024-03-26T08:53:34.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:33 smithi044 ceph-mon[36461]: pgmap v694: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 734 op/s 2024-03-26T08:53:35.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:35 smithi016 ceph-mon[35851]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 628 op/s 2024-03-26T08:53:36.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:35 smithi007 ceph-mon[32462]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 628 op/s 2024-03-26T08:53:36.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:35 smithi044 ceph-mon[36461]: pgmap v695: 97 pgs: 97 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 628 op/s 2024-03-26T08:53:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 665.0 s, 1792.2 tps, lat 55.272 ms stddev 61.455 2024-03-26T08:53:37.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:37 smithi016 ceph-mon[35851]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 681 op/s 2024-03-26T08:53:38.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:37 smithi007 ceph-mon[32462]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 681 op/s 2024-03-26T08:53:38.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:37 smithi044 ceph-mon[36461]: pgmap v696: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 681 op/s 2024-03-26T08:53:39.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:39 smithi016 ceph-mon[35851]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 616 op/s 2024-03-26T08:53:40.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:39 smithi007 ceph-mon[32462]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 616 op/s 2024-03-26T08:53:40.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:39 smithi044 ceph-mon[36461]: pgmap v697: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.6 MiB/s wr, 616 op/s 2024-03-26T08:53:41.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 670.0 s, 1750.6 tps, lat 57.484 ms stddev 66.484 2024-03-26T08:53:41.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:41 smithi016 ceph-mon[35851]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 624 op/s 2024-03-26T08:53:42.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:41 smithi007 ceph-mon[32462]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 624 op/s 2024-03-26T08:53:42.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:41 smithi044 ceph-mon[36461]: pgmap v698: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.6 MiB/s wr, 624 op/s 2024-03-26T08:53:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.901+0000 7f76b75f3640 1 -- 172.21.15.7:0/2057189600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76b0102180 msgr2=0x7f76b0068e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.901+0000 7f76b75f3640 1 --2- 172.21.15.7:0/2057189600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76b0102180 0x7f76b0068e90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009ca0 tx=0x7f76a802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.901+0000 7f76b75f3640 1 -- 172.21.15.7:0/2057189600 shutdown_connections 2024-03-26T08:53:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.901+0000 7f76b75f3640 1 --2- 172.21.15.7:0/2057189600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76b0102180 0x7f76b0068e90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.901+0000 7f76b75f3640 1 --2- 172.21.15.7:0/2057189600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b01017c0 0x7f76b0101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.901+0000 7f76b75f3640 1 --2- 172.21.15.7:0/2057189600 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b01005d0 0x7f76b01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.901+0000 7f76b75f3640 1 -- 172.21.15.7:0/2057189600 >> 172.21.15.7:0/2057189600 conn(0x7f76b00fbb40 msgr2=0x7f76b00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.901+0000 7f76b75f3640 1 -- 172.21.15.7:0/2057189600 shutdown_connections 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.901+0000 7f76b75f3640 1 -- 172.21.15.7:0/2057189600 wait complete. 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b75f3640 1 Processor -- start 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b75f3640 1 -- start start 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b75f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b01005d0 0x7f76b0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b75f3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76b01017c0 0x7f76b0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b75f3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b0102180 0x7f76b019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b75f3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b010d350 con 0x7f76b01017c0 2024-03-26T08:53:43.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b75f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b010d1d0 con 0x7f76b01005d0 2024-03-26T08:53:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b75f3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b010d4d0 con 0x7f76b0102180 2024-03-26T08:53:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b4b8e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76b01017c0 0x7f76b0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b538f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b01005d0 0x7f76b0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b538f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b01005d0 0x7f76b0197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50322/0 (socket says 172.21.15.7:50322) 2024-03-26T08:53:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b538f640 1 -- 172.21.15.7:0/72019891 learned_addr learned my addr 172.21.15.7:0/72019891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:43.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b5b90640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b0102180 0x7f76b019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b5b90640 1 -- 172.21.15.7:0/72019891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b01005d0 msgr2=0x7f76b0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b5b90640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b01005d0 0x7f76b0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b5b90640 1 -- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76b01017c0 msgr2=0x7f76b0197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b5b90640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76b01017c0 0x7f76b0197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.902+0000 7f76b5b90640 1 -- 172.21.15.7:0/72019891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b019f590 con 0x7f76b0102180 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.903+0000 7f76b5b90640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b0102180 0x7f76b019ef20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f76a802c630 tx=0x7f76a8032f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.903+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76b01017c0 0x7f76b0197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.903+0000 7f76a67fc640 1 -- 172.21.15.7:0/72019891 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8041020 con 0x7f76b0102180 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.903+0000 7f76a67fc640 1 -- 172.21.15.7:0/72019891 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76a8034cd0 con 0x7f76b0102180 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.903+0000 7f76a67fc640 1 -- 172.21.15.7:0/72019891 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8039040 con 0x7f76b0102180 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.904+0000 7f76b75f3640 1 -- 172.21.15.7:0/72019891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b01029e0 con 0x7f76b0102180 2024-03-26T08:53:43.903 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.904+0000 7f76b75f3640 1 -- 172.21.15.7:0/72019891 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b0102e50 con 0x7f76b0102180 2024-03-26T08:53:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.907+0000 7f76a67fc640 1 -- 172.21.15.7:0/72019891 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f76a8044020 con 0x7f76b0102180 2024-03-26T08:53:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.907+0000 7f76a67fc640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f769c0a6c80 0x7f769c0a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.907+0000 7f76a67fc640 1 -- 172.21.15.7:0/72019891 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76a80f0890 con 0x7f76b0102180 2024-03-26T08:53:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76b538f640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f769c0a6c80 0x7f769c0a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76ba1dc740 1 Processor -- start 2024-03-26T08:53:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76ba1dc740 1 -- start start 2024-03-26T08:53:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c5920 0x55d25f5c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 0x55d25f5cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25f52ab10 con 0x55d25f5c6310 2024-03-26T08:53:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25f4d3900 con 0x55d25f5cb480 2024-03-26T08:53:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25f526bb0 con 0x55d25f5c5920 2024-03-26T08:53:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76b4b8e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76b4b8e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40454/0 (socket says 172.21.15.7:40454) 2024-03-26T08:53:43.906 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.908+0000 7f76b4b8e640 1 -- 172.21.15.7:0/1417896527 learned_addr learned my addr 172.21.15.7:0/1417896527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:43.907 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.909+0000 7f76b4b8e640 1 -- 172.21.15.7:0/1417896527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c5920 msgr2=0x55d25f5c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.909+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/1417896527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c5920 0x55d25f5c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.909+0000 7f76b4b8e640 1 -- 172.21.15.7:0/1417896527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 msgr2=0x55d25f5cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.909+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/1417896527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 0x55d25f5cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.909+0000 7f76b538f640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f769c0a6c80 0x7f769c0a9140 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f76a00058c0 tx=0x7f76a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.909+0000 7f76b4b8e640 1 -- 172.21.15.7:0/1417896527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d25f23f210 con 0x55d25f5c6310 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.909+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/1417896527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5cae10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009cf0 tx=0x7f76ac0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbf337f70a18eb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.909+0000 7f769b7fe640 1 -- 172.21.15.7:0/1417896527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac012450 con 0x55d25f5c6310 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.910+0000 7f769b7fe640 1 -- 172.21.15.7:0/1417896527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76ac012ad0 con 0x55d25f5c6310 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.910+0000 7f769b7fe640 1 -- 172.21.15.7:0/1417896527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00fca0 con 0x55d25f5c6310 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.910+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1417896527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 msgr2=0x55d25f5cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.910+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1417896527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5cae10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009cf0 tx=0x7f76ac0117b0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.910+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1417896527 shutdown_connections 2024-03-26T08:53:43.908 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.910+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1417896527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 0x55d25f5cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.910+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1417896527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5cae10 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.910+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1417896527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c5920 0x55d25f5c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.910+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1417896527 >> 172.21.15.7:0/1417896527 conn(0x55d25f5c0e90 msgr2=0x55d25f5c1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.911+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1417896527 shutdown_connections 2024-03-26T08:53:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.911+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1417896527 wait complete. 2024-03-26T08:53:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.911+0000 7f76ba1dc740 1 -- start start 2024-03-26T08:53:43.909 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.911+0000 7f76ba1dc740 1 Processor -- start 2024-03-26T08:53:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c6310 0x55d25f5d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f5d7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25f5ce430 con 0x55d25f5c5920 2024-03-26T08:53:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b538f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25f4e6d50 con 0x55d25f5c6310 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d25f5ce5b0 con 0x55d25f5cb480 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b538f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40460/0 (socket says 172.21.15.7:40460) 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b538f640 1 -- 172.21.15.7:0/3461406464 learned_addr learned my addr 172.21.15.7:0/3461406464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c6310 0x55d25f5d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b5b90640 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f5d7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76ba1dc740 10 client.0 _ll_register_callbacks cb 0x55d25f4e6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b538f640 1 -- 172.21.15.7:0/3461406464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 msgr2=0x55d25f5d7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b538f640 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f5d7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b538f640 1 -- 172.21.15.7:0/3461406464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c6310 msgr2=0x55d25f5d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b538f640 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c6310 0x55d25f5d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b538f640 1 -- 172.21.15.7:0/3461406464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d25f30b620 con 0x55d25f5c5920 2024-03-26T08:53:43.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b5b90640 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f5d7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.912+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c6310 0x55d25f5d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.913+0000 7f76b538f640 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5cfcd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f76a0012c10 tx=0x7f76a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39337d59e20807fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.913+0000 7f7698ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.914+0000 7f7698ff9640 1 -- 172.21.15.7:0/3461406464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0012e40 con 0x55d25f5c5920 2024-03-26T08:53:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.915+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3461406464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76ac012ad0 con 0x55d25f5c5920 2024-03-26T08:53:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.915+0000 7f7698ff9640 1 -- 172.21.15.7:0/3461406464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76a0012c40 con 0x55d25f5c5920 2024-03-26T08:53:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.915+0000 7f7698ff9640 1 -- 172.21.15.7:0/3461406464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76a0012e40 con 0x55d25f5c5920 2024-03-26T08:53:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.915+0000 7f76ba1dc740 10 client.18351 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.915+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3461406464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d25f5da6e0 con 0x55d25f5c5920 2024-03-26T08:53:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.915+0000 7f7698ff9640 1 -- 172.21.15.7:0/3461406464 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76ac012ad0 con 0x55d25f5c5920 2024-03-26T08:53:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.915+0000 7f7698ff9640 1 -- 172.21.15.7:0/3461406464 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f76a0027570 con 0x55d25f5c5920 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.916+0000 7f76ba1dc740 10 client.18351 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.916+0000 7f76ba1dc740 10 client.18351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.916+0000 7f76ba1dc740 20 client.18351 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.916+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d25f5dbd60 0x55d25f5de190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.916+0000 7f76ba1dc740 4 client.18351 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.916+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3461406464 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d25f538050 con 0x55d25f5dbd60 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.916+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d25f5dbd60 0x55d25f5de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.916+0000 7f7698ff9640 10 client.18351 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.916+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d25f5dbd60 0x55d25f5de190 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ada12319a59981f server_cookie=b06552d5d6b035fd in_seq=0 out_seq=0 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f7698ff9640 1 -- 172.21.15.7:0/3461406464 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d25f538050 con 0x55d25f5dbd60 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f7698ff9640 10 client.18351 handle_command_reply: tid=0 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 client.18351 shutdown 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3461406464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 msgr2=0x55d25f5cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5cfcd0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f76a0012c10 tx=0x7f76a00046c0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3461406464 shutdown_connections 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d25f5dbd60 0x55d25f5de190 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f5d7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c6310 0x55d25f5d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3461406464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5cfcd0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3461406464 >> 172.21.15.7:0/3461406464 conn(0x55d25f5c0e90 msgr2=0x55d25f5c0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f7698ff9640 0 client.18351 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3461406464 shutdown_connections 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3461406464 wait complete. 2024-03-26T08:53:43.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.917+0000 7f76ba1dc740 20 client.18351 trim_cache size 0 max 16384 2024-03-26T08:53:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.935+0000 7f76ba1dc740 1 Processor -- start 2024-03-26T08:53:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.935+0000 7f76ba1dc740 1 -- start start 2024-03-26T08:53:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.935+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c6310 0x55d25f5d7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 0x55d25f5cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b01a8d60 con 0x55d25f5c5920 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b01a6860 con 0x55d25f5cb480 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b01a48e0 con 0x55d25f5c6310 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b4b8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c6310 0x55d25f5d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b4b8e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c6310 0x55d25f5d7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60176/0 (socket says 172.21.15.7:60176) 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b4b8e640 1 -- 172.21.15.7:0/3031016018 learned_addr learned my addr 172.21.15.7:0/3031016018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b5b90640 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 0x55d25f5cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b538f640 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b4b8e640 1 -- 172.21.15.7:0/3031016018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 msgr2=0x55d25f5cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 0x55d25f5cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b4b8e640 1 -- 172.21.15.7:0/3031016018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 msgr2=0x55d25f5d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:43.934 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b4b8e640 1 -- 172.21.15.7:0/3031016018 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d25f5c1c00 con 0x55d25f5c6310 2024-03-26T08:53:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b5b90640 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 0x55d25f5cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b538f640 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.936+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c6310 0x55d25f5d7210 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f76ac01c040 tx=0x7f76ac009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e094ac89ff483664 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f769affd640 1 -- 172.21.15.7:0/3031016018 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac02a1e0 con 0x55d25f5c6310 2024-03-26T08:53:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f769affd640 1 -- 172.21.15.7:0/3031016018 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76ac02ad60 con 0x55d25f5c6310 2024-03-26T08:53:43.935 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f769affd640 1 -- 172.21.15.7:0/3031016018 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac012450 con 0x55d25f5c6310 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3031016018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c6310 msgr2=0x55d25f5d7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c6310 0x55d25f5d7210 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f76ac01c040 tx=0x7f76ac009f00 comp rx=0 tx=0).stop 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3031016018 shutdown_connections 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5cb480 0x55d25f5cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5c6310 0x55d25f5d7210 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/3031016018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c5920 0x55d25f5d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.937+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3031016018 >> 172.21.15.7:0/3031016018 conn(0x55d25f5c0e90 msgr2=0x55d25f5ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3031016018 shutdown_connections 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 -- 172.21.15.7:0/3031016018 wait complete. 2024-03-26T08:53:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 -- start start 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 Processor -- start 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c5920 0x55d25f5d4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f6356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b01a4ee0 con 0x55d25f5c6310 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b01a6200 con 0x55d25f5c5920 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b01a5540 con 0x55d25f5cb480 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.938+0000 7f76ba1dc740 10 client.0 _ll_register_callbacks cb 0x7f76a00128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b538f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c5920 0x55d25f5d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b5b90640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f6356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b538f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c5920 0x55d25f5d4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50372/0 (socket says 172.21.15.7:50372) 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b5b90640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f6356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60186/0 (socket says 172.21.15.7:60186) 2024-03-26T08:53:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b538f640 1 -- 172.21.15.7:0/1182083480 learned_addr learned my addr 172.21.15.7:0/1182083480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b538f640 1 -- 172.21.15.7:0/1182083480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 msgr2=0x55d25f6356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b538f640 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f6356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b538f640 1 -- 172.21.15.7:0/1182083480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 msgr2=0x55d25f5d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b538f640 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b538f640 1 -- 172.21.15.7:0/1182083480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d25f5d4bd0 con 0x55d25f5c5920 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f7698ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.939+0000 7f76b538f640 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c5920 0x55d25f5d4020 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c00 tx=0x7f76a0027630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2b9c8e21b837207 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.940+0000 7f7698ff9640 1 -- 172.21.15.7:0/1182083480 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0012c40 con 0x55d25f5c5920 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.940+0000 7f7698ff9640 1 -- 172.21.15.7:0/1182083480 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76a0005020 con 0x55d25f5c5920 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.940+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1182083480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76ac02ad60 con 0x55d25f5c5920 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.940+0000 7f7698ff9640 1 -- 172.21.15.7:0/1182083480 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76a0005220 con 0x55d25f5c5920 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.940+0000 7f76ba1dc740 10 client.28169 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.940+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1182083480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d25f636d50 con 0x55d25f5c5920 2024-03-26T08:53:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.940+0000 7f7698ff9640 1 -- 172.21.15.7:0/1182083480 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76ac02ad60 con 0x55d25f5c5920 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f7698ff9640 1 -- 172.21.15.7:0/1182083480 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f76a0032670 con 0x55d25f5c5920 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f76ba1dc740 10 client.28169 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f76ba1dc740 10 client.28169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f76ba1dc740 20 client.28169 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d25f637680 0x55d25f64aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f76ba1dc740 4 client.28169 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1182083480 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d25f5323a0 con 0x55d25f637680 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d25f637680 0x55d25f64aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f7698ff9640 10 client.28169 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.941+0000 7f76b4b8e640 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d25f637680 0x55d25f64aea0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63cf4dd1eef4b20 server_cookie=c013ca11a623781 in_seq=0 out_seq=0 2024-03-26T08:53:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f7698ff9640 1 -- 172.21.15.7:0/1182083480 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d25f5323a0 con 0x55d25f637680 2024-03-26T08:53:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f7698ff9640 10 client.28169 handle_command_reply: tid=0 2024-03-26T08:53:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 client.28169 shutdown 2024-03-26T08:53:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1182083480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c5920 msgr2=0x55d25f5d4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c5920 0x55d25f5d4020 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c00 tx=0x7f76a0027630 comp rx=0 tx=0).stop 2024-03-26T08:53:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1182083480 shutdown_connections 2024-03-26T08:53:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d25f637680 0x55d25f64aea0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d25f5cb480 0x55d25f6356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d25f5c6310 0x55d25f5d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f7698ff9640 0 client.28169 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 --2- 172.21.15.7:0/1182083480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d25f5c5920 0x55d25f5d4020 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1182083480 >> 172.21.15.7:0/1182083480 conn(0x55d25f5c0e90 msgr2=0x55d25f5cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1182083480 shutdown_connections 2024-03-26T08:53:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 1 -- 172.21.15.7:0/1182083480 wait complete. 2024-03-26T08:53:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.957+0000 7f76ba1dc740 20 client.28169 trim_cache size 0 max 16384 2024-03-26T08:53:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:53:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:53:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:53:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:53:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 -- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f769c0a6c80 msgr2=0x7f769c0a9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f769c0a6c80 0x7f769c0a9140 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f76a00058c0 tx=0x7f76a0005f90 comp rx=0 tx=0).stop 2024-03-26T08:53:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 -- 172.21.15.7:0/72019891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b0102180 msgr2=0x7f76b019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b0102180 0x7f76b019ef20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f76a802c630 tx=0x7f76a8032f20 comp rx=0 tx=0).stop 2024-03-26T08:53:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 -- 172.21.15.7:0/72019891 shutdown_connections 2024-03-26T08:53:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f769c0a6c80 0x7f769c0a9140 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76b0102180 0x7f76b019ef20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.962 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76b01017c0 0x7f76b0197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 --2- 172.21.15.7:0/72019891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b01005d0 0x7f76b0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 -- 172.21.15.7:0/72019891 >> 172.21.15.7:0/72019891 conn(0x7f76b00fbb40 msgr2=0x7f76b00fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 -- 172.21.15.7:0/72019891 shutdown_connections 2024-03-26T08:53:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:43.958+0000 7f76b75f3640 1 -- 172.21.15.7:0/72019891 wait complete. 2024-03-26T08:53:43.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:43 smithi016 ceph-mon[35851]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 MiB/s wr, 688 op/s 2024-03-26T08:53:44.047 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:43 smithi007 ceph-mon[32462]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 MiB/s wr, 688 op/s 2024-03-26T08:53:44.070 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:43 smithi044 ceph-mon[36461]: pgmap v699: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 MiB/s wr, 688 op/s 2024-03-26T08:53:45.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:53:45.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:53:45.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.630+0000 7fa2407cd640 1 -- 172.21.15.7:0/1810363831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa238102170 msgr2=0x7fa238108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.630+0000 7fa2407cd640 1 --2- 172.21.15.7:0/1810363831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa238102170 0x7fa238108a00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa230009ca0 tx=0x7fa23002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.630+0000 7fa2407cd640 1 -- 172.21.15.7:0/1810363831 shutdown_connections 2024-03-26T08:53:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.630+0000 7fa2407cd640 1 --2- 172.21.15.7:0/1810363831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa238102170 0x7fa238108a00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.630+0000 7fa2407cd640 1 --2- 172.21.15.7:0/1810363831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2381017b0 0x7fa238101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.630+0000 7fa2407cd640 1 --2- 172.21.15.7:0/1810363831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2381005c0 0x7fa2381009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.630+0000 7fa2407cd640 1 -- 172.21.15.7:0/1810363831 >> 172.21.15.7:0/1810363831 conn(0x7fa2380fbb30 msgr2=0x7fa2380fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.630+0000 7fa2407cd640 1 -- 172.21.15.7:0/1810363831 shutdown_connections 2024-03-26T08:53:45.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.630+0000 7fa2407cd640 1 -- 172.21.15.7:0/1810363831 wait complete. 2024-03-26T08:53:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.631+0000 7fa2407cd640 1 Processor -- start 2024-03-26T08:53:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.631+0000 7fa2407cd640 1 -- start start 2024-03-26T08:53:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa2407cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2381005c0 0x7fa23806a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa2407cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2381017b0 0x7fa23819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa2407cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa238102170 0x7fa23819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa2407cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23810c220 con 0x7fa238102170 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa2407cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23810c0a0 con 0x7fa2381005c0 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa2407cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23810c3a0 con 0x7fa2381017b0 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa23dd68640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2381017b0 0x7fa23819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa23e569640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2381005c0 0x7fa23806a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa23dd68640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2381017b0 0x7fa23819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60206/0 (socket says 172.21.15.7:60206) 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa23e569640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2381005c0 0x7fa23806a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50382/0 (socket says 172.21.15.7:50382) 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa23dd68640 1 -- 172.21.15.7:0/503617510 learned_addr learned my addr 172.21.15.7:0/503617510 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.632+0000 7fa23ed6a640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa238102170 0x7fa23819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa23e569640 1 -- 172.21.15.7:0/503617510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2381017b0 msgr2=0x7fa23819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa23e569640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2381017b0 0x7fa23819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa23e569640 1 -- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa238102170 msgr2=0x7fa23819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa23e569640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa238102170 0x7fa23819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa23e569640 1 -- 172.21.15.7:0/503617510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23819b380 con 0x7fa2381005c0 2024-03-26T08:53:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa23e569640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2381005c0 0x7fa23806a870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa23400e9c0 tx=0x7fa23400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:45.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa22f7fe640 1 -- 172.21.15.7:0/503617510 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23400ccc0 con 0x7fa2381005c0 2024-03-26T08:53:45.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa22f7fe640 1 -- 172.21.15.7:0/503617510 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2340035f0 con 0x7fa2381005c0 2024-03-26T08:53:45.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa2407cd640 1 -- 172.21.15.7:0/503617510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23819b670 con 0x7fa2381005c0 2024-03-26T08:53:45.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa2407cd640 1 -- 172.21.15.7:0/503617510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23819bc10 con 0x7fa2381005c0 2024-03-26T08:53:45.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.633+0000 7fa22f7fe640 1 -- 172.21.15.7:0/503617510 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23400ce60 con 0x7fa2381005c0 2024-03-26T08:53:45.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.636+0000 7fa22f7fe640 1 -- 172.21.15.7:0/503617510 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa2340037a0 con 0x7fa2381005c0 2024-03-26T08:53:45.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.636+0000 7fa22f7fe640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2240a2880 0x7fa2240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.636+0000 7fa22f7fe640 1 -- 172.21.15.7:0/503617510 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa2340cedd0 con 0x7fa2381005c0 2024-03-26T08:53:45.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.636+0000 7fa23dd68640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2240a2880 0x7fa2240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.637+0000 7fa2433ba740 1 Processor -- start 2024-03-26T08:53:45.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.637+0000 7fa2433ba740 1 -- start start 2024-03-26T08:53:45.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.637+0000 7fa2433ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484dfe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.638+0000 7fa23e569640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484dfe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.638+0000 7fa23e569640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484dfe1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50388/0 (socket says 172.21.15.7:50388) 2024-03-26T08:53:45.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.638+0000 7fa23e569640 1 -- 172.21.15.7:0/1961830681 learned_addr learned my addr 172.21.15.7:0/1961830681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.638+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfe790 0x55b484e03290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.638+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 0x55b484e05d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.638+0000 7fa2433ba740 1 -- 172.21.15.7:0/1961830681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b484d633d0 con 0x55b484dfe790 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.638+0000 7fa2433ba740 1 -- 172.21.15.7:0/1961830681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b484d0bad0 con 0x55b484dfdda0 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.638+0000 7fa23dd68640 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfe790 0x55b484e03290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.638+0000 7fa2433ba740 1 -- 172.21.15.7:0/1961830681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b484d5f470 con 0x55b484e03900 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa23ed6a640 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 0x55b484e05d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa23dd68640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2240a2880 0x7fa2240a4d40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa23806a5e0 tx=0x7fa228005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa23e569640 1 -- 172.21.15.7:0/1961830681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 msgr2=0x55b484e05d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa23e569640 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 0x55b484e05d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa23e569640 1 -- 172.21.15.7:0/1961830681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfe790 msgr2=0x55b484e03290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa23e569640 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfe790 0x55b484e03290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa23e569640 1 -- 172.21.15.7:0/1961830681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b484b417d0 con 0x55b484dfdda0 2024-03-26T08:53:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa23e569640 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484dfe1c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa234097bd0 tx=0x7fa23409be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70979b042064e0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa22cff9640 1 -- 172.21.15.7:0/1961830681 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2340a3f10 con 0x55b484dfdda0 2024-03-26T08:53:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa22cff9640 1 -- 172.21.15.7:0/1961830681 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa234099a70 con 0x55b484dfdda0 2024-03-26T08:53:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.639+0000 7fa22cff9640 1 -- 172.21.15.7:0/1961830681 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2340a3ce0 con 0x55b484dfdda0 2024-03-26T08:53:45.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 -- 172.21.15.7:0/1961830681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 msgr2=0x55b484dfe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484dfe1c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa234097bd0 tx=0x7fa23409be70 comp rx=0 tx=0).stop 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 -- 172.21.15.7:0/1961830681 shutdown_connections 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 0x55b484e05d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfe790 0x55b484e03290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1961830681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484dfe1c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 -- 172.21.15.7:0/1961830681 >> 172.21.15.7:0/1961830681 conn(0x55b484df9310 msgr2=0x55b484df9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 -- 172.21.15.7:0/1961830681 shutdown_connections 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 -- 172.21.15.7:0/1961830681 wait complete. 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.640+0000 7fa2433ba740 1 -- start start 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa2433ba740 1 Processor -- start 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa2433ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa2433ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfe790 0x55b484e08620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa23e569640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa23e569640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40530/0 (socket says 172.21.15.7:40530) 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa23e569640 1 -- 172.21.15.7:0/591577033 learned_addr learned my addr 172.21.15.7:0/591577033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa2433ba740 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 0x55b484e0f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b484e067f0 con 0x55b484dfdda0 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b484d1f060 con 0x55b484dfe790 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b484e06970 con 0x55b484e03900 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa23dd68640 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfe790 0x55b484e08620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa23ed6a640 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 0x55b484e0f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.641+0000 7fa2433ba740 10 client.0 _ll_register_callbacks cb 0x55b484d016a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa23dd68640 1 -- 172.21.15.7:0/591577033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 msgr2=0x55b484e0f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa23dd68640 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 0x55b484e0f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa23dd68640 1 -- 172.21.15.7:0/591577033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 msgr2=0x55b484e080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa23dd68640 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa23dd68640 1 -- 172.21.15.7:0/591577033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b484a773f0 con 0x55b484dfe790 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa23dd68640 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfe790 0x55b484e08620 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55b484dff200 tx=0x7fa228004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=705b7db4cbc7b0d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa221ffb640 1 -- 172.21.15.7:0/591577033 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22801fe60 con 0x55b484dfe790 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa221ffb640 1 -- 172.21.15.7:0/591577033 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa2280273e0 con 0x55b484dfe790 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.642+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa234099a70 con 0x55b484dfe790 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa221ffb640 1 -- 172.21.15.7:0/591577033 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2280275e0 con 0x55b484dfe790 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa2433ba740 10 client.28181 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b484a370f0 con 0x55b484dfe790 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa221ffb640 1 -- 172.21.15.7:0/591577033 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa228027c60 con 0x55b484dfe790 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa221ffb640 1 -- 172.21.15.7:0/591577033 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa228026070 con 0x55b484dfe790 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa2433ba740 10 client.28181 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa2433ba740 10 client.28181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:45.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa2433ba740 20 client.28181 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa2433ba740 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b484e13ac0 0x55b484e15f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.643+0000 7fa2433ba740 4 client.28181 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.644+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa234099a70 con 0x55b484e13ac0 2024-03-26T08:53:45.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.644+0000 7fa23e569640 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b484e13ac0 0x55b484e15f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.644+0000 7fa221ffb640 10 client.28181 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.644+0000 7fa23e569640 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b484e13ac0 0x55b484e15f40 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aace88d18fb97d74 server_cookie=b24d8e53ba42e45e in_seq=0 out_seq=0 2024-03-26T08:53:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa221ffb640 1 -- 172.21.15.7:0/591577033 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fa234099a70 con 0x55b484e13ac0 2024-03-26T08:53:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa221ffb640 10 client.28181 handle_command_reply: tid=0 2024-03-26T08:53:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 client.28181 shutdown 2024-03-26T08:53:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfe790 msgr2=0x55b484e08620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfe790 0x55b484e08620 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55b484dff200 tx=0x7fa228004730 comp rx=0 tx=0).stop 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 shutdown_connections 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b484e13ac0 0x55b484e15f40 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484e03900 0x55b484e0f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfe790 0x55b484e08620 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa221ffb640 0 client.28181 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 --2- 172.21.15.7:0/591577033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 >> 172.21.15.7:0/591577033 conn(0x55b484df9310 msgr2=0x55b484df8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 shutdown_connections 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 1 -- 172.21.15.7:0/591577033 wait complete. 2024-03-26T08:53:45.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.645+0000 7fa2433ba740 20 client.28181 trim_cache size 0 max 16384 2024-03-26T08:53:45.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.668+0000 7fa2433ba740 1 Processor -- start 2024-03-26T08:53:45.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.668+0000 7fa2433ba740 1 -- start start 2024-03-26T08:53:45.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa2433ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484df7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa2433ba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484dfe790 0x55b484e0f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa2433ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484e03900 0x55b484df81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23dd68640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484dfe790 0x55b484e0f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23e569640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484df7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23ed6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484e03900 0x55b484df81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23e569640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484df7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50400/0 (socket says 172.21.15.7:50400) 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23e569640 1 -- 172.21.15.7:0/1761208141 learned_addr learned my addr 172.21.15.7:0/1761208141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa2433ba740 1 -- 172.21.15.7:0/1761208141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2381b16f0 con 0x55b484e03900 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa2433ba740 1 -- 172.21.15.7:0/1761208141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2381af1f0 con 0x55b484dfdda0 2024-03-26T08:53:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa2433ba740 1 -- 172.21.15.7:0/1761208141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2381ad270 con 0x55b484dfe790 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23e569640 1 -- 172.21.15.7:0/1761208141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484dfe790 msgr2=0x55b484e0f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23e569640 1 --2- 172.21.15.7:0/1761208141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484dfe790 0x55b484e0f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23e569640 1 -- 172.21.15.7:0/1761208141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484e03900 msgr2=0x55b484df81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23e569640 1 --2- 172.21.15.7:0/1761208141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484e03900 0x55b484df81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.669+0000 7fa23e569640 1 -- 172.21.15.7:0/1761208141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b484df8c60 con 0x55b484dfdda0 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa23e569640 1 --2- 172.21.15.7:0/1761208141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484df7ca0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa23409c020 tx=0x7fa234097b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86480957ad9ccf13 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa223fff640 1 -- 172.21.15.7:0/1761208141 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234097de0 con 0x55b484dfdda0 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa223fff640 1 -- 172.21.15.7:0/1761208141 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2340b9070 con 0x55b484dfdda0 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa223fff640 1 -- 172.21.15.7:0/1761208141 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2340afb50 con 0x55b484dfdda0 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa2433ba740 1 -- 172.21.15.7:0/1761208141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 msgr2=0x55b484df7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1761208141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484df7ca0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa23409c020 tx=0x7fa234097b00 comp rx=0 tx=0).stop 2024-03-26T08:53:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa2433ba740 1 -- 172.21.15.7:0/1761208141 shutdown_connections 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1761208141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484e03900 0x55b484df81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1761208141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484dfe790 0x55b484e0f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa2433ba740 1 --2- 172.21.15.7:0/1761208141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484dfdda0 0x55b484df7ca0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.670+0000 7fa2433ba740 1 -- 172.21.15.7:0/1761208141 >> 172.21.15.7:0/1761208141 conn(0x55b484df9310 msgr2=0x55b484df96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 -- 172.21.15.7:0/1761208141 shutdown_connections 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 -- 172.21.15.7:0/1761208141 wait complete. 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 -- start start 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 Processor -- start 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e0f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484dfe790 0x55b484e0d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484e03900 0x55b484e0dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2381ad870 con 0x55b484dfdda0 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2381aeb90 con 0x55b484e03900 2024-03-26T08:53:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2381aded0 con 0x55b484dfe790 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa2433ba740 10 client.0 _ll_register_callbacks cb 0x7fa228011d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa23e569640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e0f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa23e569640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e0f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40548/0 (socket says 172.21.15.7:40548) 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.671+0000 7fa23e569640 1 -- 172.21.15.7:0/2555699883 learned_addr learned my addr 172.21.15.7:0/2555699883 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.672+0000 7fa23ed6a640 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484e03900 0x55b484e0dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.672+0000 7fa23e569640 1 -- 172.21.15.7:0/2555699883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484dfe790 msgr2=0x55b484e0d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.672+0000 7fa23e569640 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484dfe790 0x55b484e0d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.672+0000 7fa23e569640 1 -- 172.21.15.7:0/2555699883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484e03900 msgr2=0x55b484e0dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.672+0000 7fa23e569640 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484e03900 0x55b484e0dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.672+0000 7fa23e569640 1 -- 172.21.15.7:0/2555699883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b484e0e4e0 con 0x55b484dfdda0 2024-03-26T08:53:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.672+0000 7fa23e569640 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e0f350 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa234086330 tx=0x7fa23409b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c7c0028b0227b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.673+0000 7fa221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:45.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.673+0000 7fa221ffb640 1 -- 172.21.15.7:0/2555699883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234086050 con 0x55b484dfdda0 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.673+0000 7fa221ffb640 1 -- 172.21.15.7:0/2555699883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa234099a70 con 0x55b484dfdda0 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.673+0000 7fa2433ba740 1 -- 172.21.15.7:0/2555699883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2340b9070 con 0x55b484dfdda0 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.673+0000 7fa221ffb640 1 -- 172.21.15.7:0/2555699883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa234097de0 con 0x55b484dfdda0 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.673+0000 7fa2433ba740 10 client.18375 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.673+0000 7fa2433ba740 1 -- 172.21.15.7:0/2555699883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b484e70230 con 0x55b484dfdda0 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa221ffb640 1 -- 172.21.15.7:0/2555699883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2340b9070 con 0x55b484dfdda0 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa221ffb640 1 -- 172.21.15.7:0/2555699883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa234083020 con 0x55b484dfdda0 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa2433ba740 10 client.18375 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa2433ba740 10 client.18375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa2433ba740 20 client.18375 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa2433ba740 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b484e70710 0x55b484e72b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa2433ba740 4 client.18375 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:53:45.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa2433ba740 1 -- 172.21.15.7:0/2555699883 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b484d70910 con 0x55b484e70710 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa23dd68640 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b484e70710 0x55b484e72b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.674+0000 7fa221ffb640 10 client.18375 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa23dd68640 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b484e70710 0x55b484e72b30 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589e347a92973c68 server_cookie=702cf0276be6711 in_seq=0 out_seq=0 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa221ffb640 1 -- 172.21.15.7:0/2555699883 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b484d70910 con 0x55b484e70710 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa221ffb640 10 client.18375 handle_command_reply: tid=0 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 client.18375 shutdown 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 -- 172.21.15.7:0/2555699883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 msgr2=0x55b484e0f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e0f350 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa234086330 tx=0x7fa23409b670 comp rx=0 tx=0).stop 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 -- 172.21.15.7:0/2555699883 shutdown_connections 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55b484e70710 0x55b484e72b30 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b484e03900 0x55b484e0dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b484dfe790 0x55b484e0d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 --2- 172.21.15.7:0/2555699883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b484dfdda0 0x55b484e0f350 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 -- 172.21.15.7:0/2555699883 >> 172.21.15.7:0/2555699883 conn(0x55b484df9310 msgr2=0x55b484df7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa221ffb640 0 client.18375 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 -- 172.21.15.7:0/2555699883 shutdown_connections 2024-03-26T08:53:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 1 -- 172.21.15.7:0/2555699883 wait complete. 2024-03-26T08:53:45.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.675+0000 7fa2433ba740 20 client.18375 trim_cache size 0 max 16384 2024-03-26T08:53:45.676 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:53:45.676 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:53:45.676 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:53:45.676 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:53:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 -- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2240a2880 msgr2=0x7fa2240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2240a2880 0x7fa2240a4d40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa23806a5e0 tx=0x7fa228005f90 comp rx=0 tx=0).stop 2024-03-26T08:53:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 -- 172.21.15.7:0/503617510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2381005c0 msgr2=0x7fa23806a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2381005c0 0x7fa23806a870 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa23400e9c0 tx=0x7fa23400ee80 comp rx=0 tx=0).stop 2024-03-26T08:53:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 -- 172.21.15.7:0/503617510 shutdown_connections 2024-03-26T08:53:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa2240a2880 0x7fa2240a4d40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa238102170 0x7fa23819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2381017b0 0x7fa23819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 --2- 172.21.15.7:0/503617510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2381005c0 0x7fa23806a870 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 -- 172.21.15.7:0/503617510 >> 172.21.15.7:0/503617510 conn(0x7fa2380fbb30 msgr2=0x7fa238104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 -- 172.21.15.7:0/503617510 shutdown_connections 2024-03-26T08:53:45.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.676+0000 7fa2407cd640 1 -- 172.21.15.7:0/503617510 wait complete. 2024-03-26T08:53:45.687 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:53:45.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:53:45.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:45 smithi007 ceph-mon[32462]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.1 MiB/s wr, 568 op/s 2024-03-26T08:53:45.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.830+0000 7fa8fd2e8640 1 -- 172.21.15.7:0/5679085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 msgr2=0x7fa8f8108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.830+0000 7fa8fd2e8640 1 --2- 172.21.15.7:0/5679085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 0x7fa8f8108a80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009ca0 tx=0x7fa8f002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.831+0000 7fa8fd2e8640 1 -- 172.21.15.7:0/5679085 shutdown_connections 2024-03-26T08:53:45.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.831+0000 7fa8fd2e8640 1 --2- 172.21.15.7:0/5679085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 0x7fa8f8108a80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.831+0000 7fa8fd2e8640 1 --2- 172.21.15.7:0/5679085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8f8101830 0x7fa8f8101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.831+0000 7fa8fd2e8640 1 --2- 172.21.15.7:0/5679085 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8100640 0x7fa8f8100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.831+0000 7fa8fd2e8640 1 -- 172.21.15.7:0/5679085 >> 172.21.15.7:0/5679085 conn(0x7fa8f80fbbb0 msgr2=0x7fa8f80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.831+0000 7fa8fd2e8640 1 -- 172.21.15.7:0/5679085 shutdown_connections 2024-03-26T08:53:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.831+0000 7fa8fd2e8640 1 -- 172.21.15.7:0/5679085 wait complete. 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.832+0000 7fa8fd2e8640 1 Processor -- start 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.832+0000 7fa8fd2e8640 1 -- start start 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.832+0000 7fa8fd2e8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8100640 0x7fa8f806a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.832+0000 7fa8fd2e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8f8101830 0x7fa8f819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.832+0000 7fa8fd2e8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 0x7fa8f819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.832+0000 7fa8fd2e8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f810c250 con 0x7fa8f81021f0 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.832+0000 7fa8fd2e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f810c0d0 con 0x7fa8f8101830 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.832+0000 7fa8fd2e8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f810c3d0 con 0x7fa8f8100640 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 0x7fa8f819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8100640 0x7fa8f806a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 0x7fa8f819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40560/0 (socket says 172.21.15.7:40560) 2024-03-26T08:53:45.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f77fe640 1 -- 172.21.15.7:0/108984969 learned_addr learned my addr 172.21.15.7:0/108984969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8f8101830 0x7fa8f819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f6ffd640 1 -- 172.21.15.7:0/108984969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8f8101830 msgr2=0x7fa8f819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f6ffd640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8f8101830 0x7fa8f819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f6ffd640 1 -- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 msgr2=0x7fa8f819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f6ffd640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 0x7fa8f819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f6ffd640 1 -- 172.21.15.7:0/108984969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f819f5e0 con 0x7fa8f8100640 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f67fc640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8f8101830 0x7fa8f819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f77fe640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 0x7fa8f819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8f6ffd640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8100640 0x7fa8f806a890 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e9c0 tx=0x7fa8e800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8dffff640 1 -- 172.21.15.7:0/108984969 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e800ccc0 con 0x7fa8f8100640 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8dffff640 1 -- 172.21.15.7:0/108984969 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8e80035f0 con 0x7fa8f8100640 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8fd2e8640 1 -- 172.21.15.7:0/108984969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f819f870 con 0x7fa8f8100640 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8dffff640 1 -- 172.21.15.7:0/108984969 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e800ce60 con 0x7fa8f8100640 2024-03-26T08:53:45.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.833+0000 7fa8fd2e8640 1 -- 172.21.15.7:0/108984969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f819fdc0 con 0x7fa8f8100640 2024-03-26T08:53:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.835+0000 7fa8ddffb640 1 -- 172.21.15.7:0/108984969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f80622a0 con 0x7fa8f8100640 2024-03-26T08:53:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.836+0000 7fa8dffff640 1 -- 172.21.15.7:0/108984969 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa8e80037a0 con 0x7fa8f8100640 2024-03-26T08:53:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.837+0000 7fa8dffff640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8e00a2880 0x7fa8e00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.837+0000 7fa8f67fc640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8e00a2880 0x7fa8e00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:45.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.837+0000 7fa8dffff640 1 -- 172.21.15.7:0/108984969 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa8e80cef50 con 0x7fa8f8100640 2024-03-26T08:53:45.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.837+0000 7fa8f67fc640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8e00a2880 0x7fa8e00a4d40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa8f806a600 tx=0x7fa8ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:45.843+0000 7fa8dffff640 1 -- 172.21.15.7:0/108984969 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa8e809bff0 con 0x7fa8f8100640 2024-03-26T08:53:45.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:45 smithi016 ceph-mon[35851]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.1 MiB/s wr, 568 op/s 2024-03-26T08:53:46.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:45 smithi044 ceph-mon[36461]: pgmap v700: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5.1 MiB/s wr, 568 op/s 2024-03-26T08:53:46.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 675.0 s, 2139.6 tps, lat 46.418 ms stddev 55.464 2024-03-26T08:53:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.195+0000 7fa8ddffb640 1 -- 172.21.15.7:0/108984969 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8f8062470 con 0x7fa8f8100640 2024-03-26T08:53:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.196+0000 7fa8dffff640 1 -- 172.21.15.7:0/108984969 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fa8e809b8f0 con 0x7fa8f8100640 2024-03-26T08:53:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:53:46.193 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:53:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.199+0000 7fa8ddffb640 1 -- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8e00a2880 msgr2=0x7fa8e00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8e00a2880 0x7fa8e00a4d40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa8f806a600 tx=0x7fa8ec005f90 comp rx=0 tx=0).stop 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 -- 172.21.15.7:0/108984969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8100640 msgr2=0x7fa8f806a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8100640 0x7fa8f806a890 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800e9c0 tx=0x7fa8e800ee80 comp rx=0 tx=0).stop 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 -- 172.21.15.7:0/108984969 shutdown_connections 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa8e00a2880 0x7fa8e00a4d40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8f81021f0 0x7fa8f819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8f8101830 0x7fa8f819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 --2- 172.21.15.7:0/108984969 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8f8100640 0x7fa8f806a890 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 -- 172.21.15.7:0/108984969 >> 172.21.15.7:0/108984969 conn(0x7fa8f80fbbb0 msgr2=0x7fa8f8104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 -- 172.21.15.7:0/108984969 shutdown_connections 2024-03-26T08:53:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.200+0000 7fa8ddffb640 1 -- 172.21.15.7:0/108984969 wait complete. 2024-03-26T08:53:46.211 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:53:46.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:53:46.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.350+0000 7f0c2df76640 1 -- 172.21.15.7:0/202798669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c281016f0 msgr2=0x7f0c28101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.350+0000 7f0c2df76640 1 --2- 172.21.15.7:0/202798669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c281016f0 0x7f0c28101b10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009860 tx=0x7f0c1802bb70 comp rx=0 tx=0).stop 2024-03-26T08:53:46.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.350+0000 7f0c2df76640 1 -- 172.21.15.7:0/202798669 shutdown_connections 2024-03-26T08:53:46.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.350+0000 7f0c2df76640 1 --2- 172.21.15.7:0/202798669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c281032a0 0x7f0c28109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.350+0000 7f0c2df76640 1 --2- 172.21.15.7:0/202798669 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c281028e0 0x7f0c28102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.350+0000 7f0c2df76640 1 --2- 172.21.15.7:0/202798669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c281016f0 0x7f0c28101b10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.350+0000 7f0c2df76640 1 -- 172.21.15.7:0/202798669 >> 172.21.15.7:0/202798669 conn(0x7f0c280fcc60 msgr2=0x7f0c280ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.350+0000 7f0c2df76640 1 -- 172.21.15.7:0/202798669 shutdown_connections 2024-03-26T08:53:46.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.350+0000 7f0c2df76640 1 -- 172.21.15.7:0/202798669 wait complete. 2024-03-26T08:53:46.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c2df76640 1 Processor -- start 2024-03-26T08:53:46.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c2df76640 1 -- start start 2024-03-26T08:53:46.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c2df76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c281016f0 0x7f0c28197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c2df76640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c281028e0 0x7f0c28197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c2df76640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c281032a0 0x7f0c2819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c2df76640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2810d340 con 0x7f0c281028e0 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c2df76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2810d1c0 con 0x7f0c281016f0 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c2df76640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2810d4c0 con 0x7f0c281032a0 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c281028e0 0x7f0c28197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c281016f0 0x7f0c28197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c281028e0 0x7f0c28197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40580/0 (socket says 172.21.15.7:40580) 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c281016f0 0x7f0c28197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50436/0 (socket says 172.21.15.7:50436) 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c27fff640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c281032a0 0x7f0c2819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.351+0000 7f0c26ffd640 1 -- 172.21.15.7:0/2259140748 learned_addr learned my addr 172.21.15.7:0/2259140748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.352+0000 7f0c26ffd640 1 -- 172.21.15.7:0/2259140748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c281032a0 msgr2=0x7f0c2819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.352+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c281032a0 0x7f0c2819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.352+0000 7f0c26ffd640 1 -- 172.21.15.7:0/2259140748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c281016f0 msgr2=0x7f0c28197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.352+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c281016f0 0x7f0c28197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.352+0000 7f0c26ffd640 1 -- 172.21.15.7:0/2259140748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2819f640 con 0x7f0c281028e0 2024-03-26T08:53:46.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.352+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c281028e0 0x7f0c28197f60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009f20 tx=0x7f0c1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.353+0000 7f0c24ff9640 1 -- 172.21.15.7:0/2259140748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c020070 con 0x7f0c281028e0 2024-03-26T08:53:46.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.353+0000 7f0c24ff9640 1 -- 172.21.15.7:0/2259140748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c00cd80 con 0x7f0c281028e0 2024-03-26T08:53:46.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.353+0000 7f0c24ff9640 1 -- 172.21.15.7:0/2259140748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c01bbd0 con 0x7f0c281028e0 2024-03-26T08:53:46.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.354+0000 7f0c2df76640 1 -- 172.21.15.7:0/2259140748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c28073930 con 0x7f0c281028e0 2024-03-26T08:53:46.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.354+0000 7f0c2df76640 1 -- 172.21.15.7:0/2259140748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c28073da0 con 0x7f0c281028e0 2024-03-26T08:53:46.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.363+0000 7f0c30c33740 1 Processor -- start 2024-03-26T08:53:46.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.367+0000 7f0c24ff9640 1 -- 172.21.15.7:0/2259140748 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f0c1c0137d0 con 0x7f0c281028e0 2024-03-26T08:53:46.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.367+0000 7f0c30c33740 1 -- start start 2024-03-26T08:53:46.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.368+0000 7f0c24ff9640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c100a29d0 0x7f0c100a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.368+0000 7f0c24ff9640 1 -- 172.21.15.7:0/2259140748 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0c1c0d2ea0 con 0x7f0c281028e0 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.369+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f60690 0x556ae5f60ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.369+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f65b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.369+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f661f0 0x556ae5f68680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.369+0000 7f0c30c33740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5f23610 con 0x556ae5f61080 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.373+0000 7f0c30c33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5ec63c0 con 0x556ae5f661f0 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.373+0000 7f0c30c33740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5e6ed60 con 0x556ae5f60690 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.374+0000 7f0c27fff640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c100a29d0 0x7f0c100a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.370+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.374+0000 7f0c26ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f65b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40584/0 (socket says 172.21.15.7:40584) 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.374+0000 7f0c26ffd640 1 -- 172.21.15.7:0/1942492367 learned_addr learned my addr 172.21.15.7:0/1942492367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.372+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f60690 0x556ae5f60ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.375+0000 7f0c26ffd640 1 -- 172.21.15.7:0/1942492367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f60690 msgr2=0x556ae5f60ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.382+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/1942492367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f60690 0x556ae5f60ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.382+0000 7f0c26ffd640 1 -- 172.21.15.7:0/1942492367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f661f0 msgr2=0x556ae5f68680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:53:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.382+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/1942492367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f661f0 0x556ae5f68680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.382+0000 7f0c26ffd640 1 -- 172.21.15.7:0/1942492367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae5bda3f0 con 0x556ae5f61080 2024-03-26T08:53:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.385+0000 7f0c27fff640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c100a29d0 0x7f0c100a4e90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005fd0 tx=0x7f0c20010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.386+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/1942492367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f65b80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c09f840 tx=0x7f0c1c09f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5a401ff4ea5dd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.388+0000 7f0c0dffb640 1 -- 172.21.15.7:0/1942492367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0b8070 con 0x556ae5f61080 2024-03-26T08:53:46.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.388+0000 7f0c0dffb640 1 -- 172.21.15.7:0/1942492367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c09f8b0 con 0x556ae5f61080 2024-03-26T08:53:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.388+0000 7f0c0dffb640 1 -- 172.21.15.7:0/1942492367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c09c0b0 con 0x556ae5f61080 2024-03-26T08:53:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.389+0000 7f0c30c33740 1 -- 172.21.15.7:0/1942492367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 msgr2=0x556ae5f65b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.389+0000 7f0c30c33740 1 --2- 172.21.15.7:0/1942492367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f65b80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c09f840 tx=0x7f0c1c09f740 comp rx=0 tx=0).stop 2024-03-26T08:53:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.389+0000 7f0c30c33740 1 -- 172.21.15.7:0/1942492367 shutdown_connections 2024-03-26T08:53:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.389+0000 7f0c30c33740 1 --2- 172.21.15.7:0/1942492367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f661f0 0x556ae5f68680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.389+0000 7f0c30c33740 1 --2- 172.21.15.7:0/1942492367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f65b80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.389+0000 7f0c30c33740 1 --2- 172.21.15.7:0/1942492367 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f60690 0x556ae5f60ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.389+0000 7f0c30c33740 1 -- 172.21.15.7:0/1942492367 >> 172.21.15.7:0/1942492367 conn(0x556ae5f5bc00 msgr2=0x556ae5f5c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.390+0000 7f0c30c33740 1 -- 172.21.15.7:0/1942492367 shutdown_connections 2024-03-26T08:53:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.390+0000 7f0c30c33740 1 -- 172.21.15.7:0/1942492367 wait complete. 2024-03-26T08:53:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.390+0000 7f0c30c33740 1 -- start start 2024-03-26T08:53:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.391+0000 7f0c30c33740 1 Processor -- start 2024-03-26T08:53:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.392+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.392+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 0x556ae5f73800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.392+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f661f0 0x556ae5f7a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.392+0000 7f0c30c33740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5e82240 con 0x556ae5f661f0 2024-03-26T08:53:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.392+0000 7f0c30c33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5ec2460 con 0x556ae5f60690 2024-03-26T08:53:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.392+0000 7f0c30c33740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5f719b0 con 0x556ae5f61080 2024-03-26T08:53:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.392+0000 7f0c30c33740 10 client.0 _ll_register_callbacks cb 0x556ae5e81c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.394+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f661f0 0x556ae5f7a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.394+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f661f0 0x556ae5f7a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40598/0 (socket says 172.21.15.7:40598) 2024-03-26T08:53:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.394+0000 7f0c27fff640 1 -- 172.21.15.7:0/2255719238 learned_addr learned my addr 172.21.15.7:0/2255719238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.395+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 0x556ae5f73800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.395+0000 7f0c277fe640 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.397+0000 7f0c27fff640 1 -- 172.21.15.7:0/2255719238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 msgr2=0x556ae5f73800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.397+0000 7f0c27fff640 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 0x556ae5f73800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.397+0000 7f0c27fff640 1 -- 172.21.15.7:0/2255719238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 msgr2=0x556ae5f732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.397+0000 7f0c27fff640 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.397+0000 7f0c27fff640 1 -- 172.21.15.7:0/2255719238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ae5ca47d0 con 0x556ae5f661f0 2024-03-26T08:53:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.398+0000 7f0c27fff640 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f661f0 0x556ae5f7a840 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0c20019e70 tx=0x7f0c20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f5256da30534e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.398+0000 7f0c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.398+0000 7f0c06ffd640 1 -- 172.21.15.7:0/2255719238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20003970 con 0x556ae5f661f0 2024-03-26T08:53:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.398+0000 7f0c06ffd640 1 -- 172.21.15.7:0/2255719238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0c20009020 con 0x556ae5f661f0 2024-03-26T08:53:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.402+0000 7f0c30c33740 1 -- 172.21.15.7:0/2255719238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c1c09f8b0 con 0x556ae5f661f0 2024-03-26T08:53:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.403+0000 7f0c06ffd640 1 -- 172.21.15.7:0/2255719238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c20003b10 con 0x556ae5f661f0 2024-03-26T08:53:46.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.404+0000 7f0c30c33740 10 client.18393 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:46.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.404+0000 7f0c30c33740 1 -- 172.21.15.7:0/2255719238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556ae5f8ec30 con 0x556ae5f661f0 2024-03-26T08:53:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.410+0000 7f0c06ffd640 1 -- 172.21.15.7:0/2255719238 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c09f8b0 con 0x556ae5f661f0 2024-03-26T08:53:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.410+0000 7f0c06ffd640 1 -- 172.21.15.7:0/2255719238 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0c2000b040 con 0x556ae5f661f0 2024-03-26T08:53:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.410+0000 7f0c30c33740 10 client.18393 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.410+0000 7f0c30c33740 10 client.18393 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:53:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.410+0000 7f0c30c33740 20 client.18393 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.410+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x556ae5f90380 0x556ae5f927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.410+0000 7f0c30c33740 4 client.18393 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.410+0000 7f0c30c33740 1 -- 172.21.15.7:0/2255719238 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ae5ed3900 con 0x556ae5f90380 2024-03-26T08:53:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.412+0000 7f0c277fe640 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x556ae5f90380 0x556ae5f927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.423+0000 7f0c277fe640 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x556ae5f90380 0x556ae5f927e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b3cdc95447903c7 server_cookie=15af155ddc8b9f11 in_seq=0 out_seq=0 2024-03-26T08:53:46.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.423+0000 7f0c06ffd640 10 client.18393 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:53:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.424+0000 7f0c06ffd640 1 -- 172.21.15.7:0/2255719238 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556ae5ed3900 con 0x556ae5f90380 2024-03-26T08:53:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.424+0000 7f0c06ffd640 10 client.18393 handle_command_reply: tid=0 2024-03-26T08:53:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.424+0000 7f0c30c33740 1 client.18393 shutdown 2024-03-26T08:53:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.425+0000 7f0c30c33740 1 -- 172.21.15.7:0/2255719238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f661f0 msgr2=0x556ae5f7a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.425+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f661f0 0x556ae5f7a840 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0c20019e70 tx=0x7f0c20004a40 comp rx=0 tx=0).stop 2024-03-26T08:53:46.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.425+0000 7f0c30c33740 1 -- 172.21.15.7:0/2255719238 shutdown_connections 2024-03-26T08:53:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.425+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x556ae5f90380 0x556ae5f927e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.425+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f661f0 0x556ae5f7a840 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.425+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 0x556ae5f73800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.425+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2255719238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.425+0000 7f0c30c33740 1 -- 172.21.15.7:0/2255719238 >> 172.21.15.7:0/2255719238 conn(0x556ae5f5bc00 msgr2=0x556ae5f5ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.426+0000 7f0c06ffd640 0 client.18393 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:53:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.426+0000 7f0c30c33740 1 -- 172.21.15.7:0/2255719238 shutdown_connections 2024-03-26T08:53:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.426+0000 7f0c30c33740 1 -- 172.21.15.7:0/2255719238 wait complete. 2024-03-26T08:53:46.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.426+0000 7f0c30c33740 20 client.18393 trim_cache size 0 max 16384 2024-03-26T08:53:46.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.450+0000 7f0c30c33740 1 Processor -- start 2024-03-26T08:53:46.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.450+0000 7f0c30c33740 1 -- start start 2024-03-26T08:53:46.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.450+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f60690 0x556ae5f5f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.450+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 0x556ae5f7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.450+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f60690 0x556ae5f5f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.450+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f60690 0x556ae5f5f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40614/0 (socket says 172.21.15.7:40614) 2024-03-26T08:53:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.450+0000 7f0c277fe640 1 -- 172.21.15.7:0/2216311212 learned_addr learned my addr 172.21.15.7:0/2216311212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.450+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f661f0 0x556ae5f970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c30c33740 1 -- 172.21.15.7:0/2216311212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281a0750 con 0x556ae5f60690 2024-03-26T08:53:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 0x556ae5f7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c30c33740 1 -- 172.21.15.7:0/2216311212 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28072370 con 0x556ae5f661f0 2024-03-26T08:53:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c30c33740 1 -- 172.21.15.7:0/2216311212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280703f0 con 0x556ae5f61080 2024-03-26T08:53:46.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c27fff640 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f661f0 0x556ae5f970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c26ffd640 1 -- 172.21.15.7:0/2216311212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f661f0 msgr2=0x556ae5f970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f661f0 0x556ae5f970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c26ffd640 1 -- 172.21.15.7:0/2216311212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f60690 msgr2=0x556ae5f5f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f60690 0x556ae5f5f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c26ffd640 1 -- 172.21.15.7:0/2216311212 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae5f71af0 con 0x556ae5f61080 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c277fe640 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f60690 0x556ae5f5f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.451+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 0x556ae5f7a760 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x556ae5f61af0 tx=0x7f0c1c01b590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891fdb18705536c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.452+0000 7f0c0d7fa640 1 -- 172.21.15.7:0/2216311212 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0b8040 con 0x556ae5f61080 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.452+0000 7f0c0d7fa640 1 -- 172.21.15.7:0/2216311212 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c0a7fa0 con 0x556ae5f61080 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.452+0000 7f0c0d7fa640 1 -- 172.21.15.7:0/2216311212 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0b3be0 con 0x556ae5f61080 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 -- 172.21.15.7:0/2216311212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 msgr2=0x556ae5f7a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 0x556ae5f7a760 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x556ae5f61af0 tx=0x7f0c1c01b590 comp rx=0 tx=0).stop 2024-03-26T08:53:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 -- 172.21.15.7:0/2216311212 shutdown_connections 2024-03-26T08:53:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f661f0 0x556ae5f970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f61080 0x556ae5f7a760 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 --2- 172.21.15.7:0/2216311212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f60690 0x556ae5f5f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 -- 172.21.15.7:0/2216311212 >> 172.21.15.7:0/2216311212 conn(0x556ae5f5bc00 msgr2=0x556ae5f5ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 -- 172.21.15.7:0/2216311212 shutdown_connections 2024-03-26T08:53:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 -- 172.21.15.7:0/2216311212 wait complete. 2024-03-26T08:53:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 -- start start 2024-03-26T08:53:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.453+0000 7f0c30c33740 1 Processor -- start 2024-03-26T08:53:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f78a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c30c33740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f661f0 0x556ae5f78f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c30c33740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c280709f0 con 0x556ae5f61080 2024-03-26T08:53:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c30c33740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28071d10 con 0x556ae5f60690 2024-03-26T08:53:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c30c33740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28071050 con 0x556ae5f661f0 2024-03-26T08:53:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c30c33740 10 client.0 _ll_register_callbacks cb 0x7f0c20019810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f7a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f7a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50474/0 (socket says 172.21.15.7:50474) 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c277fe640 1 -- 172.21.15.7:0/4019383676 learned_addr learned my addr 172.21.15.7:0/4019383676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f78a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c27fff640 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f661f0 0x556ae5f78f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c277fe640 1 -- 172.21.15.7:0/4019383676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f661f0 msgr2=0x556ae5f78f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c277fe640 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f661f0 0x556ae5f78f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c277fe640 1 -- 172.21.15.7:0/4019383676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 msgr2=0x556ae5f78a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c277fe640 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f78a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c277fe640 1 -- 172.21.15.7:0/4019383676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ae5f794b0 con 0x556ae5f60690 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c27fff640 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f661f0 0x556ae5f78f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c277fe640 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f7a420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0c18037040 tx=0x7f0c180344d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=294316e1421a6da9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.454+0000 7f0c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c06ffd640 1 -- 172.21.15.7:0/4019383676 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1804f060 con 0x556ae5f60690 2024-03-26T08:53:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c30c33740 1 -- 172.21.15.7:0/4019383676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c1c0a7fa0 con 0x556ae5f60690 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c06ffd640 1 -- 172.21.15.7:0/4019383676 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0c18034680 con 0x556ae5f60690 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c06ffd640 1 -- 172.21.15.7:0/4019383676 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c18039040 con 0x556ae5f60690 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c30c33740 10 client.28208 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c30c33740 1 -- 172.21.15.7:0/4019383676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556ae5fecac0 con 0x556ae5f60690 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c06ffd640 1 -- 172.21.15.7:0/4019383676 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c0a7fa0 con 0x556ae5f60690 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c06ffd640 1 -- 172.21.15.7:0/4019383676 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f0c18054070 con 0x556ae5f60690 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c30c33740 10 client.28208 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c30c33740 10 client.28208 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c30c33740 20 client.28208 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c30c33740 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x556ae5fed3d0 0x556ae5fef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c30c33740 4 client.28208 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c30c33740 1 -- 172.21.15.7:0/4019383676 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556ae5ecdc50 con 0x556ae5fed3d0 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.455+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x556ae5fed3d0 0x556ae5fef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.456+0000 7f0c26ffd640 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x556ae5fed3d0 0x556ae5fef7f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=522755aa60365da7 server_cookie=8ab488b57a26bec9 in_seq=0 out_seq=0 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.456+0000 7f0c06ffd640 10 client.28208 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:53:46.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.456+0000 7f0c06ffd640 1 -- 172.21.15.7:0/4019383676 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556ae5ecdc50 con 0x556ae5fed3d0 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.456+0000 7f0c06ffd640 10 client.28208 handle_command_reply: tid=0 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.456+0000 7f0c30c33740 1 client.28208 shutdown 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.457+0000 7f0c30c33740 1 -- 172.21.15.7:0/4019383676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 msgr2=0x556ae5f7a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.457+0000 7f0c30c33740 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f7a420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0c18037040 tx=0x7f0c180344d0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.457+0000 7f0c30c33740 1 -- 172.21.15.7:0/4019383676 shutdown_connections 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.457+0000 7f0c30c33740 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x556ae5fed3d0 0x556ae5fef7f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.457+0000 7f0c30c33740 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556ae5f661f0 0x556ae5f78f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.457+0000 7f0c30c33740 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ae5f61080 0x556ae5f78a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.457+0000 7f0c30c33740 1 --2- 172.21.15.7:0/4019383676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ae5f60690 0x556ae5f7a420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.457+0000 7f0c30c33740 1 -- 172.21.15.7:0/4019383676 >> 172.21.15.7:0/4019383676 conn(0x556ae5f5bc00 msgr2=0x556ae5f5f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.458+0000 7f0c06ffd640 0 client.28208 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:53:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.459+0000 7f0c30c33740 1 -- 172.21.15.7:0/4019383676 shutdown_connections 2024-03-26T08:53:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.459+0000 7f0c30c33740 1 -- 172.21.15.7:0/4019383676 wait complete. 2024-03-26T08:53:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.459+0000 7f0c30c33740 20 client.28208 trim_cache size 0 max 16384 2024-03-26T08:53:46.458 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:53:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 -- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c100a29d0 msgr2=0x7f0c100a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c100a29d0 0x7f0c100a4e90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005fd0 tx=0x7f0c20010040 comp rx=0 tx=0).stop 2024-03-26T08:53:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 -- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c281028e0 msgr2=0x7f0c28197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c281028e0 0x7f0c28197f60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009f20 tx=0x7f0c1c00ee80 comp rx=0 tx=0).stop 2024-03-26T08:53:46.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 -- 172.21.15.7:0/2259140748 shutdown_connections 2024-03-26T08:53:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f0c100a29d0 0x7f0c100a4e90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0c281032a0 0x7f0c2819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c281028e0 0x7f0c28197f60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 --2- 172.21.15.7:0/2259140748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c281016f0 0x7f0c28197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 -- 172.21.15.7:0/2259140748 >> 172.21.15.7:0/2259140748 conn(0x7f0c280fcc60 msgr2=0x7f0c28105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 -- 172.21.15.7:0/2259140748 shutdown_connections 2024-03-26T08:53:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.460+0000 7f0c2df76640 1 -- 172.21.15.7:0/2259140748 wait complete. 2024-03-26T08:53:46.469 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:53:46.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:53:46.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.616+0000 7ff1709f4640 1 -- 172.21.15.7:0/2946544222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1017b0 msgr2=0x7ff16c101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.616+0000 7ff1709f4640 1 --2- 172.21.15.7:0/2946544222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1017b0 0x7ff16c101c30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff160009900 tx=0x7ff16002bc30 comp rx=0 tx=0).stop 2024-03-26T08:53:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.617+0000 7ff1709f4640 1 -- 172.21.15.7:0/2946544222 shutdown_connections 2024-03-26T08:53:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.617+0000 7ff1709f4640 1 --2- 172.21.15.7:0/2946544222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c102170 0x7ff16c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.617+0000 7ff1709f4640 1 --2- 172.21.15.7:0/2946544222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1017b0 0x7ff16c101c30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.617+0000 7ff1709f4640 1 --2- 172.21.15.7:0/2946544222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c1005c0 0x7ff16c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.617+0000 7ff1709f4640 1 -- 172.21.15.7:0/2946544222 >> 172.21.15.7:0/2946544222 conn(0x7ff16c0fbb30 msgr2=0x7ff16c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.617+0000 7ff1709f4640 1 -- 172.21.15.7:0/2946544222 shutdown_connections 2024-03-26T08:53:46.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.617+0000 7ff1709f4640 1 -- 172.21.15.7:0/2946544222 wait complete. 2024-03-26T08:53:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.617+0000 7ff1709f4640 1 Processor -- start 2024-03-26T08:53:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.617+0000 7ff1709f4640 1 -- start start 2024-03-26T08:53:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff1709f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1005c0 0x7ff16c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff1709f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c1017b0 0x7ff16c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff1709f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c102170 0x7ff16c1a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff1709f4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c10b1d0 con 0x7ff16c1005c0 2024-03-26T08:53:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff1709f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c10b050 con 0x7ff16c1017b0 2024-03-26T08:53:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff1709f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c10b350 con 0x7ff16c102170 2024-03-26T08:53:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1005c0 0x7ff16c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c102170 0x7ff16c1a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1005c0 0x7ff16c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47594/0 (socket says 172.21.15.7:47594) 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16a59c640 1 -- 172.21.15.7:0/4207795674 learned_addr learned my addr 172.21.15.7:0/4207795674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff169d9b640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c1017b0 0x7ff16c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16a59c640 1 -- 172.21.15.7:0/4207795674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c102170 msgr2=0x7ff16c1a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16a59c640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c102170 0x7ff16c1a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16a59c640 1 -- 172.21.15.7:0/4207795674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c1017b0 msgr2=0x7ff16c19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16a59c640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c1017b0 0x7ff16c19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16a59c640 1 -- 172.21.15.7:0/4207795674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c1a1840 con 0x7ff16c1005c0 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.618+0000 7ff16a59c640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1005c0 0x7ff16c199c30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00e9c0 tx=0x7ff15c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.619+0000 7ff15b7fe640 1 -- 172.21.15.7:0/4207795674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00cd80 con 0x7ff16c1005c0 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.619+0000 7ff1709f4640 1 -- 172.21.15.7:0/4207795674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c1a1b30 con 0x7ff16c1005c0 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.619+0000 7ff15b7fe640 1 -- 172.21.15.7:0/4207795674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff15c0035f0 con 0x7ff16c1005c0 2024-03-26T08:53:46.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.619+0000 7ff15b7fe640 1 -- 172.21.15.7:0/4207795674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00cd80 con 0x7ff16c1005c0 2024-03-26T08:53:46.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.619+0000 7ff1709f4640 1 -- 172.21.15.7:0/4207795674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c1a2080 con 0x7ff16c1005c0 2024-03-26T08:53:46.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.622+0000 7ff1735dd740 1 Processor -- start 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.622+0000 7ff1735dd740 1 -- start start 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 0x56210e5da390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5daa00 0x56210e5dce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff1735dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210e58f610 con 0x56210e5daa00 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff1735dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210e5323c0 con 0x56210e5d4ea0 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff1735dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210e4dad60 con 0x56210e5d5890 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff169d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5daa00 0x56210e5dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff16ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff169d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5daa00 0x56210e5dce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47596/0 (socket says 172.21.15.7:47596) 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff16ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52404/0 (socket says 172.21.15.7:52404) 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff169d9b640 1 -- 172.21.15.7:0/1301542122 learned_addr learned my addr 172.21.15.7:0/1301542122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff15b7fe640 1 -- 172.21.15.7:0/4207795674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff15c0037a0 con 0x7ff16c1005c0 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff16a59c640 1 --2- 172.21.15.7:0/1301542122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 0x56210e5da390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff16ad9d640 1 -- 172.21.15.7:0/1301542122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 msgr2=0x56210e5da390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/1301542122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 0x56210e5da390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff16ad9d640 1 -- 172.21.15.7:0/1301542122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5daa00 msgr2=0x56210e5dce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/1301542122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5daa00 0x56210e5dce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff16ad9d640 1 -- 172.21.15.7:0/1301542122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56210e2463f0 con 0x56210e5d4ea0 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.623+0000 7ff169d9b640 1 --2- 172.21.15.7:0/1301542122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5daa00 0x56210e5dce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.624+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/1301542122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d52c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff164009f00 tx=0x7ff164011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0fdfeb5a301457 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.624+0000 7ff158ff9640 1 -- 172.21.15.7:0/1301542122 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164004d20 con 0x56210e5d4ea0 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.624+0000 7ff15b7fe640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff1540a2bf0 0x7ff1540a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.624+0000 7ff158ff9640 1 -- 172.21.15.7:0/1301542122 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff164004030 con 0x56210e5d4ea0 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.624+0000 7ff158ff9640 1 -- 172.21.15.7:0/1301542122 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff164005260 con 0x56210e5d4ea0 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.624+0000 7ff16a59c640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff1540a2bf0 0x7ff1540a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.624+0000 7ff15b7fe640 1 -- 172.21.15.7:0/4207795674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff15c00cb80 con 0x7ff16c1005c0 2024-03-26T08:53:46.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.625+0000 7ff16a59c640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff1540a2bf0 0x7ff1540a50b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff15c09af40 tx=0x7ff15c054050 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.625+0000 7ff1735dd740 1 -- 172.21.15.7:0/1301542122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 msgr2=0x56210e5d52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.625+0000 7ff1735dd740 1 --2- 172.21.15.7:0/1301542122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d52c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff164009f00 tx=0x7ff164011810 comp rx=0 tx=0).stop 2024-03-26T08:53:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.625+0000 7ff1735dd740 1 -- 172.21.15.7:0/1301542122 shutdown_connections 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.625+0000 7ff1735dd740 1 --2- 172.21.15.7:0/1301542122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5daa00 0x56210e5dce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.625+0000 7ff1735dd740 1 --2- 172.21.15.7:0/1301542122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 0x56210e5da390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.625+0000 7ff1735dd740 1 --2- 172.21.15.7:0/1301542122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d52c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.625+0000 7ff1735dd740 1 -- 172.21.15.7:0/1301542122 >> 172.21.15.7:0/1301542122 conn(0x56210e5d0410 msgr2=0x56210e5d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.626+0000 7ff1735dd740 1 -- 172.21.15.7:0/1301542122 shutdown_connections 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.626+0000 7ff1735dd740 1 -- 172.21.15.7:0/1301542122 wait complete. 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.626+0000 7ff1735dd740 1 -- start start 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.626+0000 7ff1735dd740 1 Processor -- start 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d4ea0 0x56210e5df080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 0x56210e5df5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5daa00 0x56210e5e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff1735dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210e4ee240 con 0x56210e5d4ea0 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff1735dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210e52e460 con 0x56210e5daa00 2024-03-26T08:53:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff1735dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210e5dd9b0 con 0x56210e5d5890 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff1735dd740 10 client.0 _ll_register_callbacks cb 0x56210e4edc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff16ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d4ea0 0x56210e5df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff16ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5daa00 0x56210e5e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff16ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5daa00 0x56210e5e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52414/0 (socket says 172.21.15.7:52414) 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff169d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 0x56210e5df5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.627+0000 7ff16ad9d640 1 -- 172.21.15.7:0/4151100075 learned_addr learned my addr 172.21.15.7:0/4151100075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff16ad9d640 1 -- 172.21.15.7:0/4151100075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 msgr2=0x56210e5df5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 0x56210e5df5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff16ad9d640 1 -- 172.21.15.7:0/4151100075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d4ea0 msgr2=0x56210e5df080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d4ea0 0x56210e5df080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff16ad9d640 1 -- 172.21.15.7:0/4151100075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56210e3107d0 con 0x56210e5daa00 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff169d9b640 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 0x56210e5df5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5daa00 0x56210e5e6590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff164011d10 tx=0x7ff164011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=604b1bd7c4f6fdcf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:53:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff151ffb640 1 -- 172.21.15.7:0/4151100075 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16402d060 con 0x56210e5daa00 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff1735dd740 1 -- 172.21.15.7:0/4151100075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff164004030 con 0x56210e5daa00 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff151ffb640 1 -- 172.21.15.7:0/4151100075 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff16400fa20 con 0x56210e5daa00 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff151ffb640 1 -- 172.21.15.7:0/4151100075 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff16401e040 con 0x56210e5daa00 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff1735dd740 10 client.28220 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff1735dd740 1 -- 172.21.15.7:0/4151100075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56210e5e9950 con 0x56210e5daa00 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.628+0000 7ff151ffb640 1 -- 172.21.15.7:0/4151100075 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff164004030 con 0x56210e5daa00 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.629+0000 7ff151ffb640 1 -- 172.21.15.7:0/4151100075 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff164032070 con 0x56210e5daa00 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.629+0000 7ff1735dd740 10 client.28220 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.629+0000 7ff1735dd740 10 client.28220 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.629+0000 7ff1735dd740 20 client.28220 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.629+0000 7ff1735dd740 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56210e5eb380 0x56210e5ed7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.629+0000 7ff1735dd740 4 client.28220 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.629+0000 7ff1735dd740 1 -- 172.21.15.7:0/4151100075 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56210e53f900 con 0x56210e5eb380 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.629+0000 7ff169d9b640 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56210e5eb380 0x56210e5ed7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.630+0000 7ff151ffb640 10 client.28220 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:53:46.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.630+0000 7ff169d9b640 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56210e5eb380 0x56210e5ed7a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a3dd13db7f39fae server_cookie=2a1c70d0b9f751c in_seq=0 out_seq=0 2024-03-26T08:53:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.630+0000 7ff151ffb640 1 -- 172.21.15.7:0/4151100075 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56210e53f900 con 0x56210e5eb380 2024-03-26T08:53:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.630+0000 7ff151ffb640 10 client.28220 handle_command_reply: tid=0 2024-03-26T08:53:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.630+0000 7ff1735dd740 1 client.28220 shutdown 2024-03-26T08:53:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.631+0000 7ff1735dd740 1 -- 172.21.15.7:0/4151100075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5daa00 msgr2=0x56210e5e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.631+0000 7ff1735dd740 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5daa00 0x56210e5e6590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff164011d10 tx=0x7ff164011810 comp rx=0 tx=0).stop 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.631+0000 7ff1735dd740 1 -- 172.21.15.7:0/4151100075 shutdown_connections 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.631+0000 7ff1735dd740 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5daa00 0x56210e5e6590 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.631+0000 7ff1735dd740 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56210e5eb380 0x56210e5ed7a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.631+0000 7ff1735dd740 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d5890 0x56210e5df5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.631+0000 7ff1735dd740 1 --2- 172.21.15.7:0/4151100075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d4ea0 0x56210e5df080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.631+0000 7ff1735dd740 1 -- 172.21.15.7:0/4151100075 >> 172.21.15.7:0/4151100075 conn(0x56210e5d0410 msgr2=0x56210e5c6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.632+0000 7ff151ffb640 0 client.28220 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.632+0000 7ff1735dd740 1 -- 172.21.15.7:0/4151100075 shutdown_connections 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.632+0000 7ff1735dd740 1 -- 172.21.15.7:0/4151100075 wait complete. 2024-03-26T08:53:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.632+0000 7ff1735dd740 20 client.28220 trim_cache size 0 max 16384 2024-03-26T08:53:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.647+0000 7ff1735dd740 1 Processor -- start 2024-03-26T08:53:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.647+0000 7ff1735dd740 1 -- start start 2024-03-26T08:53:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d0410 0x56210e5d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 0x56210e5e6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff1735dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a8d80 con 0x56210e5d5890 2024-03-26T08:53:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff1735dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a6880 con 0x56210e5d4ea0 2024-03-26T08:53:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff1735dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a4900 con 0x56210e5d0410 2024-03-26T08:53:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff16ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d0410 0x56210e5d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff16ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d0410 0x56210e5d6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49470/0 (socket says 172.21.15.7:49470) 2024-03-26T08:53:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff16ad9d640 1 -- 172.21.15.7:0/1217489502 learned_addr learned my addr 172.21.15.7:0/1217489502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.648+0000 7ff169d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/1217489502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 0x56210e5e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff169d9b640 1 -- 172.21.15.7:0/1217489502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d0410 msgr2=0x56210e5d6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff169d9b640 1 --2- 172.21.15.7:0/1217489502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d0410 0x56210e5d6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff169d9b640 1 -- 172.21.15.7:0/1217489502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 msgr2=0x56210e5e6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff169d9b640 1 --2- 172.21.15.7:0/1217489502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 0x56210e5e6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff169d9b640 1 -- 172.21.15.7:0/1217489502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56210e5d52a0 con 0x56210e5d4ea0 2024-03-26T08:53:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/1217489502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 0x56210e5e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:53:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff169d9b640 1 --2- 172.21.15.7:0/1217489502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d70f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff160000c00 tx=0x7ff160009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17cc0a79ca57cf54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff153fff640 1 -- 172.21.15.7:0/1217489502 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160005d50 con 0x56210e5d4ea0 2024-03-26T08:53:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff153fff640 1 -- 172.21.15.7:0/1217489502 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff16004e030 con 0x56210e5d4ea0 2024-03-26T08:53:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.649+0000 7ff153fff640 1 -- 172.21.15.7:0/1217489502 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff160004eb0 con 0x56210e5d4ea0 2024-03-26T08:53:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.650+0000 7ff1735dd740 1 -- 172.21.15.7:0/1217489502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 msgr2=0x56210e5d70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.650+0000 7ff1735dd740 1 --2- 172.21.15.7:0/1217489502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d70f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff160000c00 tx=0x7ff160009260 comp rx=0 tx=0).stop 2024-03-26T08:53:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.650+0000 7ff1735dd740 1 -- 172.21.15.7:0/1217489502 shutdown_connections 2024-03-26T08:53:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.650+0000 7ff1735dd740 1 --2- 172.21.15.7:0/1217489502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 0x56210e5e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.650+0000 7ff1735dd740 1 --2- 172.21.15.7:0/1217489502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d4ea0 0x56210e5d70f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.650+0000 7ff1735dd740 1 --2- 172.21.15.7:0/1217489502 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d0410 0x56210e5d6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.650+0000 7ff1735dd740 1 -- 172.21.15.7:0/1217489502 >> 172.21.15.7:0/1217489502 conn(0x56210e5da540 msgr2=0x56210e5df080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.650+0000 7ff1735dd740 1 -- 172.21.15.7:0/1217489502 shutdown_connections 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.650+0000 7ff1735dd740 1 -- 172.21.15.7:0/1217489502 wait complete. 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff1735dd740 1 -- start start 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff1735dd740 1 Processor -- start 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d0410 0x56210e5e3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d4ea0 0x56210e5e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff1735dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 0x56210e5e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff1735dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a4f00 con 0x56210e5d5890 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff1735dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a6220 con 0x56210e5d0410 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff1735dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c1a5560 con 0x56210e5d4ea0 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff1735dd740 10 client.0 _ll_register_callbacks cb 0x7ff16400f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff169d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d4ea0 0x56210e5e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff169d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d4ea0 0x56210e5e3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49476/0 (socket says 172.21.15.7:49476) 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.651+0000 7ff169d9b640 1 -- 172.21.15.7:0/345668813 learned_addr learned my addr 172.21.15.7:0/345668813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff169d9b640 1 -- 172.21.15.7:0/345668813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d0410 msgr2=0x56210e5e3a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:53:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff169d9b640 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d0410 0x56210e5e3a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff169d9b640 1 -- 172.21.15.7:0/345668813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 msgr2=0x56210e5e4f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 0x56210e5e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff169d9b640 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 0x56210e5e4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff169d9b640 1 -- 172.21.15.7:0/345668813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56210e5e5510 con 0x56210e5d4ea0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff169d9b640 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d4ea0 0x56210e5e3f60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff16002c7b0 tx=0x7ff160005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=269693ba35221d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff151ffb640 1 -- 172.21.15.7:0/345668813 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1600048c0 con 0x56210e5d4ea0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff1735dd740 1 -- 172.21.15.7:0/345668813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff16004e030 con 0x56210e5d4ea0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.652+0000 7ff151ffb640 1 -- 172.21.15.7:0/345668813 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff160004030 con 0x56210e5d4ea0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.653+0000 7ff151ffb640 1 -- 172.21.15.7:0/345668813 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff160062050 con 0x56210e5d4ea0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.653+0000 7ff1735dd740 10 client.28222 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.653+0000 7ff1735dd740 1 -- 172.21.15.7:0/345668813 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56210e5e6780 con 0x56210e5d4ea0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.653+0000 7ff151ffb640 1 -- 172.21.15.7:0/345668813 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff16004e030 con 0x56210e5d4ea0 2024-03-26T08:53:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.653+0000 7ff151ffb640 1 -- 172.21.15.7:0/345668813 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff160004f10 con 0x56210e5d4ea0 2024-03-26T08:53:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.654+0000 7ff1735dd740 10 client.28222 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.654+0000 7ff1735dd740 10 client.28222 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:53:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.654+0000 7ff1735dd740 20 client.28222 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.654+0000 7ff1735dd740 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56210e5daa00 0x56210e6491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.654+0000 7ff1735dd740 4 client.28222 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.654+0000 7ff1735dd740 1 -- 172.21.15.7:0/345668813 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56210e539c50 con 0x56210e5daa00 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56210e5daa00 0x56210e6491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff16ad9d640 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56210e5daa00 0x56210e6491e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=496fc8316dc4d3a server_cookie=8982688f7b360093 in_seq=0 out_seq=0 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff151ffb640 10 client.28222 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff151ffb640 1 -- 172.21.15.7:0/345668813 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56210e539c50 con 0x56210e5daa00 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff151ffb640 10 client.28222 handle_command_reply: tid=0 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 client.28222 shutdown 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 -- 172.21.15.7:0/345668813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d4ea0 msgr2=0x56210e5e3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d4ea0 0x56210e5e3f60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff16002c7b0 tx=0x7ff160005fd0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 -- 172.21.15.7:0/345668813 shutdown_connections 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56210e5d5890 0x56210e5e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56210e5daa00 0x56210e6491e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56210e5d4ea0 0x56210e5e3f60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 --2- 172.21.15.7:0/345668813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56210e5d0410 0x56210e5e3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 -- 172.21.15.7:0/345668813 >> 172.21.15.7:0/345668813 conn(0x56210e5da540 msgr2=0x56210e5df060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff151ffb640 0 client.28222 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:53:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 -- 172.21.15.7:0/345668813 shutdown_connections 2024-03-26T08:53:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.655+0000 7ff1735dd740 1 -- 172.21.15.7:0/345668813 wait complete. 2024-03-26T08:53:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1735dd740 20 client.28222 trim_cache size 0 max 16384 2024-03-26T08:53:46.656 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:53:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 -- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff1540a2bf0 msgr2=0x7ff1540a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff1540a2bf0 0x7ff1540a50b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff15c09af40 tx=0x7ff15c054050 comp rx=0 tx=0).stop 2024-03-26T08:53:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 -- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1005c0 msgr2=0x7ff16c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1005c0 0x7ff16c199c30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00e9c0 tx=0x7ff15c00ee80 comp rx=0 tx=0).stop 2024-03-26T08:53:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 -- 172.21.15.7:0/4207795674 shutdown_connections 2024-03-26T08:53:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff1540a2bf0 0x7ff1540a50b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff16c102170 0x7ff16c1a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff16c1017b0 0x7ff16c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 --2- 172.21.15.7:0/4207795674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c1005c0 0x7ff16c199c30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 -- 172.21.15.7:0/4207795674 >> 172.21.15.7:0/4207795674 conn(0x7ff16c0fbb30 msgr2=0x7ff16c0fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 -- 172.21.15.7:0/4207795674 shutdown_connections 2024-03-26T08:53:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.656+0000 7ff1709f4640 1 -- 172.21.15.7:0/4207795674 wait complete. 2024-03-26T08:53:46.666 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:53:46.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:53:46.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:46 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/108984969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:53:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.835+0000 7f3b10cf9640 1 -- 172.21.15.7:0/2854702161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c0732c0 msgr2=0x7f3b0c108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.835+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/2854702161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c0732c0 0x7f3b0c108ca0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009ca0 tx=0x7f3b0402c140 comp rx=0 tx=0).stop 2024-03-26T08:53:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.835+0000 7f3b10cf9640 1 -- 172.21.15.7:0/2854702161 shutdown_connections 2024-03-26T08:53:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.835+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/2854702161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b0c109360 0x7f3b0c109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.835+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/2854702161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c0732c0 0x7f3b0c108ca0 secure :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009ca0 tx=0x7f3b0402c140 comp rx=0 tx=0).stop 2024-03-26T08:53:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.835+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/2854702161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b0c072960 0x7f3b0c072d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.835+0000 7f3b10cf9640 1 -- 172.21.15.7:0/2854702161 >> 172.21.15.7:0/2854702161 conn(0x7f3b0c06e4e0 msgr2=0x7f3b0c070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.840+0000 7f3b10cf9640 1 -- 172.21.15.7:0/2854702161 shutdown_connections 2024-03-26T08:53:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.844+0000 7f3b10cf9640 1 -- 172.21.15.7:0/2854702161 wait complete. 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.844+0000 7f3b10cf9640 1 Processor -- start 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.844+0000 7f3b10cf9640 1 -- start start 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.845+0000 7f3b10cf9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b0c072960 0x7f3b0c1193d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.845+0000 7f3b10cf9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b0c109360 0x7f3b0c119910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.845+0000 7f3b10cf9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c11c9a0 0x7f3b0c11ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.845+0000 7f3b10cf9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10d760 con 0x7f3b0c109360 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.845+0000 7f3b10cf9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10d5e0 con 0x7f3b0c11c9a0 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.845+0000 7f3b10cf9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10d8e0 con 0x7f3b0c072960 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.845+0000 7f3b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c11c9a0 0x7f3b0c11ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.846+0000 7f3b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c11c9a0 0x7f3b0c11ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52436/0 (socket says 172.21.15.7:52436) 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.846+0000 7f3b0bfff640 1 -- 172.21.15.7:0/1764381322 learned_addr learned my addr 172.21.15.7:0/1764381322 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.847+0000 7f3b0bfff640 1 -- 172.21.15.7:0/1764381322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b0c072960 msgr2=0x7f3b0c1193d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.847+0000 7f3b0bfff640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b0c072960 0x7f3b0c1193d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.847+0000 7f3b0bfff640 1 -- 172.21.15.7:0/1764381322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b0c109360 msgr2=0x7f3b0c119910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.847+0000 7f3b0bfff640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b0c109360 0x7f3b0c119910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.847+0000 7f3b0bfff640 1 -- 172.21.15.7:0/1764381322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c11d410 con 0x7f3b0c11c9a0 2024-03-26T08:53:46.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.848+0000 7f3b0bfff640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c11c9a0 0x7f3b0c11ce50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c970 tx=0x7f3afc00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.853+0000 7f3b08ff9640 1 -- 172.21.15.7:0/1764381322 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc007bb0 con 0x7f3b0c11c9a0 2024-03-26T08:53:46.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.853+0000 7f3b08ff9640 1 -- 172.21.15.7:0/1764381322 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3afc007d50 con 0x7f3b0c11c9a0 2024-03-26T08:53:46.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.853+0000 7f3b08ff9640 1 -- 172.21.15.7:0/1764381322 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00f450 con 0x7f3b0c11c9a0 2024-03-26T08:53:46.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.854+0000 7f3b10cf9640 1 -- 172.21.15.7:0/1764381322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c11d700 con 0x7f3b0c11c9a0 2024-03-26T08:53:46.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.854+0000 7f3b10cf9640 1 -- 172.21.15.7:0/1764381322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c11dc80 con 0x7f3b0c11c9a0 2024-03-26T08:53:46.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.858+0000 7f3b14c21740 1 Processor -- start 2024-03-26T08:53:46.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.858+0000 7f3b14c21740 1 -- start start 2024-03-26T08:53:46.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.858+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f96860 0x562891f96c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.858+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f97250 0x562891fbdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.858+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.858+0000 7f3b14c21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562891f59610 con 0x562891fbe3d0 2024-03-26T08:53:46.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.858+0000 7f3b14c21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562891efc3c0 con 0x562891f96860 2024-03-26T08:53:46.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.858+0000 7f3b14c21740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562891ea4d60 con 0x562891f97250 2024-03-26T08:53:46.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f96860 0x562891f96c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f96860 0x562891f96c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52442/0 (socket says 172.21.15.7:52442) 2024-03-26T08:53:46.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 -- 172.21.15.7:0/3385515397 learned_addr learned my addr 172.21.15.7:0/3385515397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 --2- 172.21.15.7:0/3385515397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 -- 172.21.15.7:0/3385515397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f97250 msgr2=0x562891fbdd60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:53:46.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 --2- 172.21.15.7:0/3385515397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f97250 0x562891fbdd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 -- 172.21.15.7:0/3385515397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 msgr2=0x562891fc0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 --2- 172.21.15.7:0/3385515397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 -- 172.21.15.7:0/3385515397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562891c103f0 con 0x562891f96860 2024-03-26T08:53:46.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.861+0000 7f3b0affd640 1 --2- 172.21.15.7:0/3385515397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f96860 0x562891f96c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3b0402c6d0 tx=0x7f3b040325a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29be886110ddbd28 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.862+0000 7f3af1ffb640 1 -- 172.21.15.7:0/3385515397 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04039040 con 0x562891f96860 2024-03-26T08:53:46.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.863+0000 7f3af1ffb640 1 -- 172.21.15.7:0/3385515397 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b04003970 con 0x562891f96860 2024-03-26T08:53:46.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.863+0000 7f3af1ffb640 1 -- 172.21.15.7:0/3385515397 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04004280 con 0x562891f96860 2024-03-26T08:53:46.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.863+0000 7f3b08ff9640 1 -- 172.21.15.7:0/1764381322 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3afc0046c0 con 0x7f3b0c11c9a0 2024-03-26T08:53:46.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b14c21740 1 -- 172.21.15.7:0/3385515397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f96860 msgr2=0x562891f96c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b14c21740 1 --2- 172.21.15.7:0/3385515397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f96860 0x562891f96c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3b0402c6d0 tx=0x7f3b040325a0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b08ff9640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3af40a2980 0x7f3af40a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b14c21740 1 -- 172.21.15.7:0/3385515397 shutdown_connections 2024-03-26T08:53:46.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b14c21740 1 --2- 172.21.15.7:0/3385515397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b14c21740 1 --2- 172.21.15.7:0/3385515397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f97250 0x562891fbdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b14c21740 1 --2- 172.21.15.7:0/3385515397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f96860 0x562891f96c80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b14c21740 1 -- 172.21.15.7:0/3385515397 >> 172.21.15.7:0/3385515397 conn(0x562891f91dd0 msgr2=0x562891f921d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b08ff9640 1 -- 172.21.15.7:0/1764381322 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3afc0ce8b0 con 0x7f3b0c11c9a0 2024-03-26T08:53:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.864+0000 7f3b0affd640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3af40a2980 0x7f3af40a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.865+0000 7f3b14c21740 1 -- 172.21.15.7:0/3385515397 shutdown_connections 2024-03-26T08:53:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.865+0000 7f3b14c21740 1 -- 172.21.15.7:0/3385515397 wait complete. 2024-03-26T08:53:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.865+0000 7f3b0affd640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3af40a2980 0x7f3af40a4e40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009bf0 tx=0x7f3b04005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.865+0000 7f3b14c21740 1 -- start start 2024-03-26T08:53:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b14c21740 1 Processor -- start 2024-03-26T08:53:46.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 0x562891fc2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f97250 0x562891fc2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b14c21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562891eb8240 con 0x562891fbe3d0 2024-03-26T08:53:46.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b14c21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562891ef8460 con 0x562891f97250 2024-03-26T08:53:46.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b14c21740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562891fc1320 con 0x562891f96860 2024-03-26T08:53:46.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b14c21740 10 client.0 _ll_register_callbacks cb 0x562891eb7c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:46.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47650/0 (socket says 172.21.15.7:47650) 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.866+0000 7f3b0bfff640 1 -- 172.21.15.7:0/2843288836 learned_addr learned my addr 172.21.15.7:0/2843288836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.867+0000 7f3b0bfff640 1 -- 172.21.15.7:0/2843288836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 msgr2=0x562891fc2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.867+0000 7f3b0bfff640 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 0x562891fc2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.867+0000 7f3b0bfff640 1 -- 172.21.15.7:0/2843288836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f97250 msgr2=0x562891fc2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.867+0000 7f3b0bfff640 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f97250 0x562891fc2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.867+0000 7f3b0bfff640 1 -- 172.21.15.7:0/2843288836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562891cda7d0 con 0x562891fbe3d0 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.868+0000 7f3b0bfff640 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc9ee0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0a3540 tx=0x7f3afc09bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962588fd1bf575b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.868+0000 7f3aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.868+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2843288836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0b3020 con 0x562891fbe3d0 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.868+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2843288836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3afc00db60 con 0x562891fbe3d0 2024-03-26T08:53:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.868+0000 7f3b14c21740 1 -- 172.21.15.7:0/2843288836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b04003970 con 0x562891fbe3d0 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.869+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2843288836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3afc00dd60 con 0x562891fbe3d0 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.869+0000 7f3b14c21740 10 client.18417 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.869+0000 7f3b14c21740 1 -- 172.21.15.7:0/2843288836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562891fcd390 con 0x562891fbe3d0 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.869+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2843288836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3afc09a720 con 0x562891fbe3d0 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.869+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2843288836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3afc09a2a0 con 0x562891fbe3d0 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.870+0000 7f3b14c21740 10 client.18417 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.870+0000 7f3b14c21740 10 client.18417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.870+0000 7f3b14c21740 20 client.18417 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.870+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562891fcea70 0x562891fd0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.870+0000 7f3b14c21740 4 client.18417 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.870+0000 7f3b14c21740 1 -- 172.21.15.7:0/2843288836 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b04003970 con 0x562891fcea70 2024-03-26T08:53:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.871+0000 7f3b0b7fe640 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562891fcea70 0x562891fd0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.873+0000 7f3aeaffd640 10 client.18417 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.873+0000 7f3b0b7fe640 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562891fcea70 0x562891fd0f00 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f69d2464330fc1a7 server_cookie=40b77b9d994c9d02 in_seq=0 out_seq=0 2024-03-26T08:53:46.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2843288836 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f3b04003970 con 0x562891fcea70 2024-03-26T08:53:46.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3aeaffd640 10 client.18417 handle_command_reply: tid=0 2024-03-26T08:53:46.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3b14c21740 1 client.18417 shutdown 2024-03-26T08:53:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3b14c21740 1 -- 172.21.15.7:0/2843288836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 msgr2=0x562891fc9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc9ee0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0a3540 tx=0x7f3afc09bb10 comp rx=0 tx=0).stop 2024-03-26T08:53:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3b14c21740 1 -- 172.21.15.7:0/2843288836 shutdown_connections 2024-03-26T08:53:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562891fcea70 0x562891fd0f00 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3aeaffd640 0 client.18417 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc9ee0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f97250 0x562891fc2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2843288836 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 0x562891fc2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.874+0000 7f3b14c21740 1 -- 172.21.15.7:0/2843288836 >> 172.21.15.7:0/2843288836 conn(0x562891f91dd0 msgr2=0x562891f91070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.875+0000 7f3b14c21740 1 -- 172.21.15.7:0/2843288836 shutdown_connections 2024-03-26T08:53:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.875+0000 7f3b14c21740 1 -- 172.21.15.7:0/2843288836 wait complete. 2024-03-26T08:53:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.875+0000 7f3b14c21740 20 client.18417 trim_cache size 0 max 16384 2024-03-26T08:53:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b14c21740 1 Processor -- start 2024-03-26T08:53:46.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b14c21740 1 -- start start 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891f96860 0x562891f95740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f97250 0x562891fc9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891fbe3d0 0x562891fd5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891f96860 0x562891f95740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891f96860 0x562891f95740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47660/0 (socket says 172.21.15.7:47660) 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b0b7fe640 1 -- 172.21.15.7:0/1615992242 learned_addr learned my addr 172.21.15.7:0/1615992242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b14c21740 1 -- 172.21.15.7:0/1615992242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80109d0 con 0x562891f96860 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b14c21740 1 -- 172.21.15.7:0/1615992242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af800e4d0 con 0x562891fbe3d0 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b14c21740 1 -- 172.21.15.7:0/1615992242 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af800c550 con 0x562891f97250 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b0affd640 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f97250 0x562891fc9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.891+0000 7f3b0bfff640 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891fbe3d0 0x562891fd5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3b0b7fe640 1 -- 172.21.15.7:0/1615992242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f97250 msgr2=0x562891fc9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3b0b7fe640 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f97250 0x562891fc9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3b0b7fe640 1 -- 172.21.15.7:0/1615992242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891fbe3d0 msgr2=0x562891fd5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3b0b7fe640 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891fbe3d0 0x562891fd5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3b0b7fe640 1 -- 172.21.15.7:0/1615992242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562891fd1580 con 0x562891f96860 2024-03-26T08:53:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3b0bfff640 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891fbe3d0 0x562891fd5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:53:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3b0b7fe640 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891f96860 0x562891f95740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009510 tx=0x7f3b00016dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e265270f3445a4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3af17fa640 1 -- 172.21.15.7:0/1615992242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00009cd0 con 0x562891f96860 2024-03-26T08:53:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3af17fa640 1 -- 172.21.15.7:0/1615992242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b0001a030 con 0x562891f96860 2024-03-26T08:53:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.892+0000 7f3af17fa640 1 -- 172.21.15.7:0/1615992242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00015e20 con 0x562891f96860 2024-03-26T08:53:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 -- 172.21.15.7:0/1615992242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891f96860 msgr2=0x562891f95740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891f96860 0x562891f95740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009510 tx=0x7f3b00016dd0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 -- 172.21.15.7:0/1615992242 shutdown_connections 2024-03-26T08:53:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891fbe3d0 0x562891fd5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f97250 0x562891fc9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 --2- 172.21.15.7:0/1615992242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891f96860 0x562891f95740 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 -- 172.21.15.7:0/1615992242 >> 172.21.15.7:0/1615992242 conn(0x562891f91dd0 msgr2=0x562891f95030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 -- 172.21.15.7:0/1615992242 shutdown_connections 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 -- 172.21.15.7:0/1615992242 wait complete. 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.894+0000 7f3b14c21740 1 -- start start 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b14c21740 1 Processor -- start 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 0x562891fc9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f97250 0x562891fc9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b14c21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b14c21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af800cb50 con 0x562891fbe3d0 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b14c21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af800de70 con 0x562891f97250 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b14c21740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af800d1b0 con 0x562891f96860 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 0x562891fc9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 0x562891fc9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49516/0 (socket says 172.21.15.7:49516) 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0b7fe640 1 -- 172.21.15.7:0/2940819014 learned_addr learned my addr 172.21.15.7:0/2940819014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0affd640 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f97250 0x562891fc9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b14c21740 10 client.0 _ll_register_callbacks cb 0x7f3afc09acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0b7fe640 1 -- 172.21.15.7:0/2940819014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f97250 msgr2=0x562891fc9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0b7fe640 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f97250 0x562891fc9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0b7fe640 1 -- 172.21.15.7:0/2940819014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 msgr2=0x562891fc81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0b7fe640 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0b7fe640 1 -- 172.21.15.7:0/2940819014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562891fc1a20 con 0x562891f96860 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.895+0000 7f3b0bfff640 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.896+0000 7f3b0b7fe640 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 0x562891fc9a50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3b00016880 tx=0x7f3b000047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34fb57956e798b5c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.896+0000 7f3aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.896+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2940819014 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00009e80 con 0x562891f96860 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.896+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2940819014 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b00015ba0 con 0x562891f96860 2024-03-26T08:53:46.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.896+0000 7f3b14c21740 1 -- 172.21.15.7:0/2940819014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b0001a030 con 0x562891f96860 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.896+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2940819014 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b00015da0 con 0x562891f96860 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.896+0000 7f3b14c21740 10 client.28228 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.896+0000 7f3b14c21740 1 -- 172.21.15.7:0/2940819014 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562891fce850 con 0x562891f96860 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.896+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2940819014 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b0001a030 con 0x562891f96860 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2940819014 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3b00026070 con 0x562891f96860 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3b14c21740 10 client.28228 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3b14c21740 10 client.28228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3b14c21740 20 client.28228 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56289202b530 0x56289202d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3b14c21740 4 client.28228 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3b14c21740 1 -- 172.21.15.7:0/2940819014 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562891f09900 con 0x56289202b530 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3b0affd640 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56289202b530 0x56289202d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3aeaffd640 10 client.28228 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3b0affd640 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56289202b530 0x56289202d950 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5490263a35651fe server_cookie=4281eea036ac2511 in_seq=0 out_seq=0 2024-03-26T08:53:46.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3aeaffd640 1 -- 172.21.15.7:0/2940819014 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562891f09900 con 0x56289202b530 2024-03-26T08:53:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3aeaffd640 10 client.28228 handle_command_reply: tid=0 2024-03-26T08:53:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.897+0000 7f3b14c21740 1 client.28228 shutdown 2024-03-26T08:53:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 -- 172.21.15.7:0/2940819014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 msgr2=0x562891fc9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 0x562891fc9a50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3b00016880 tx=0x7f3b000047b0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 -- 172.21.15.7:0/2940819014 shutdown_connections 2024-03-26T08:53:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56289202b530 0x56289202d950 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562891fbe3d0 0x562891fc81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562891f97250 0x562891fc9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 --2- 172.21.15.7:0/2940819014 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562891f96860 0x562891fc9a50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 -- 172.21.15.7:0/2940819014 >> 172.21.15.7:0/2940819014 conn(0x562891f91dd0 msgr2=0x562891f958c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3aeaffd640 0 client.28228 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:46.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 -- 172.21.15.7:0/2940819014 shutdown_connections 2024-03-26T08:53:46.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.898+0000 7f3b14c21740 1 -- 172.21.15.7:0/2940819014 wait complete. 2024-03-26T08:53:46.899 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:53:46.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.899+0000 7f3b14c21740 20 client.28228 trim_cache size 0 max 16384 2024-03-26T08:53:46.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 -- 172.21.15.7:0/1764381322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3af40a2980 msgr2=0x7f3af40a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3af40a2980 0x7f3af40a4e40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009bf0 tx=0x7f3b04005bd0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 -- 172.21.15.7:0/1764381322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c11c9a0 msgr2=0x7f3b0c11ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c11c9a0 0x7f3b0c11ce50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00c970 tx=0x7f3afc00ce30 comp rx=0 tx=0).stop 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 -- 172.21.15.7:0/1764381322 shutdown_connections 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3af40a2980 0x7f3af40a4e40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b0c11c9a0 0x7f3b0c11ce50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b0c109360 0x7f3b0c119910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 --2- 172.21.15.7:0/1764381322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3b0c072960 0x7f3b0c1193d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 -- 172.21.15.7:0/1764381322 >> 172.21.15.7:0/1764381322 conn(0x7f3b0c06e4e0 msgr2=0x7f3b0c070d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 -- 172.21.15.7:0/1764381322 shutdown_connections 2024-03-26T08:53:46.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:46.901+0000 7f3b10cf9640 1 -- 172.21.15.7:0/1764381322 wait complete. 2024-03-26T08:53:46.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:46 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/108984969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:53:47.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:46 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/108984969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:53:47.911 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:53:47.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:53:47.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:53:47.944 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:47 smithi007 ceph-mon[32462]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.6 MiB/s wr, 661 op/s 2024-03-26T08:53:47.944 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:47 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:53:47.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:47 smithi016 ceph-mon[35851]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.6 MiB/s wr, 661 op/s 2024-03-26T08:53:47.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:47 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:53:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.071+0000 7fc59ec0b640 1 -- 172.21.15.7:0/293922406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59810d9c0 msgr2=0x7fc59810de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.071+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/293922406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59810d9c0 0x7fc59810de40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc59001acb0 tx=0x7fc59003d170 comp rx=0 tx=0).stop 2024-03-26T08:53:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.072+0000 7fc59ec0b640 1 -- 172.21.15.7:0/293922406 shutdown_connections 2024-03-26T08:53:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.072+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/293922406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59810d9c0 0x7fc59810de40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.072+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/293922406 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59810b010 0x7fc59810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.072+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/293922406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5981086a0 0x7fc59810aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.072+0000 7fc59ec0b640 1 -- 172.21.15.7:0/293922406 >> 172.21.15.7:0/293922406 conn(0x7fc59806e640 msgr2=0x7fc598070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:47 smithi044 ceph-mon[36461]: pgmap v701: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.6 MiB/s wr, 661 op/s 2024-03-26T08:53:48.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:47 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:53:48.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.082+0000 7fc59ec0b640 1 -- 172.21.15.7:0/293922406 shutdown_connections 2024-03-26T08:53:48.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.084+0000 7fc59ec0b640 1 -- 172.21.15.7:0/293922406 wait complete. 2024-03-26T08:53:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.084+0000 7fc59ec0b640 1 Processor -- start 2024-03-26T08:53:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.084+0000 7fc59ec0b640 1 -- start start 2024-03-26T08:53:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.085+0000 7fc59ec0b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5981086a0 0x7fc59819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.085+0000 7fc59ec0b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59810b010 0x7fc59819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.085+0000 7fc59ec0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59810d9c0 0x7fc5981a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.085+0000 7fc59ec0b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc598111a00 con 0x7fc5981086a0 2024-03-26T08:53:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.085+0000 7fc59ec0b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc598111880 con 0x7fc59810d9c0 2024-03-26T08:53:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.085+0000 7fc59ec0b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc598111b80 con 0x7fc59810b010 2024-03-26T08:53:48.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.086+0000 7fc597fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59810b010 0x7fc59819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.086+0000 7fc597fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59810b010 0x7fc59819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49528/0 (socket says 172.21.15.7:49528) 2024-03-26T08:53:48.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.086+0000 7fc597fff640 1 -- 172.21.15.7:0/132187435 learned_addr learned my addr 172.21.15.7:0/132187435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.086+0000 7fc59d1a8640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59810d9c0 0x7fc5981a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.087+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5981086a0 0x7fc59819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.088+0000 7fc59c9a7640 1 -- 172.21.15.7:0/132187435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59810b010 msgr2=0x7fc59819e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.088+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59810b010 0x7fc59819e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.088+0000 7fc59c9a7640 1 -- 172.21.15.7:0/132187435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59810d9c0 msgr2=0x7fc5981a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.088+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59810d9c0 0x7fc5981a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.088+0000 7fc59c9a7640 1 -- 172.21.15.7:0/132187435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5981a5d10 con 0x7fc5981086a0 2024-03-26T08:53:48.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.088+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5981086a0 0x7fc59819e0f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009fd0 tx=0x7fc58c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.092+0000 7fc595ffb640 1 -- 172.21.15.7:0/132187435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c015020 con 0x7fc5981086a0 2024-03-26T08:53:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.092+0000 7fc59ec0b640 1 -- 172.21.15.7:0/132187435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5981a5fa0 con 0x7fc5981086a0 2024-03-26T08:53:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.092+0000 7fc59ec0b640 1 -- 172.21.15.7:0/132187435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5981a64e0 con 0x7fc5981086a0 2024-03-26T08:53:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.093+0000 7fc595ffb640 1 -- 172.21.15.7:0/132187435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc58c011d10 con 0x7fc5981086a0 2024-03-26T08:53:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.093+0000 7fc595ffb640 1 -- 172.21.15.7:0/132187435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c00f2e0 con 0x7fc5981086a0 2024-03-26T08:53:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.096+0000 7fc595ffb640 1 -- 172.21.15.7:0/132187435 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc58c018020 con 0x7fc5981086a0 2024-03-26T08:53:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.097+0000 7fc5a17f6740 1 Processor -- start 2024-03-26T08:53:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.097+0000 7fc595ffb640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5800a2950 0x7fc5800a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.098+0000 7fc595ffb640 1 -- 172.21.15.7:0/132187435 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc58c0cf3a0 con 0x7fc5981086a0 2024-03-26T08:53:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.098+0000 7fc59d1a8640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5800a2950 0x7fc5800a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.098+0000 7fc5a17f6740 1 -- start start 2024-03-26T08:53:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.099+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c78d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.099+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c7de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.099+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c80950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.099+0000 7fc5a17f6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f603bb8c90 con 0x55f603c79350 2024-03-26T08:53:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.099+0000 7fc59d1a8640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5800a2950 0x7fc5800a4e10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc590048040 tx=0x7fc590043bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.099+0000 7fc5a17f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f603bde070 con 0x55f603c7e4c0 2024-03-26T08:53:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc5a17f6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f603b86c80 con 0x55f603c78960 2024-03-26T08:53:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.099+0000 7fc597fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c7de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc59d1a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c80950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc597fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c7de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47696/0 (socket says 172.21.15.7:47696) 2024-03-26T08:53:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc597fff640 1 -- 172.21.15.7:0/858678698 learned_addr learned my addr 172.21.15.7:0/858678698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/858678698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c78d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc597fff640 1 -- 172.21.15.7:0/858678698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 msgr2=0x55f603c78d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc597fff640 1 --2- 172.21.15.7:0/858678698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c78d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc597fff640 1 -- 172.21.15.7:0/858678698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 msgr2=0x55f603c80950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc597fff640 1 --2- 172.21.15.7:0/858678698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c80950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.100+0000 7fc597fff640 1 -- 172.21.15.7:0/858678698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f603b7ed10 con 0x55f603c79350 2024-03-26T08:53:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.101+0000 7fc597fff640 1 --2- 172.21.15.7:0/858678698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c7de50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc588007810 tx=0x7fc588007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f84e14b1bb5317f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.102+0000 7fc57effd640 1 -- 172.21.15.7:0/858678698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58800d950 con 0x55f603c79350 2024-03-26T08:53:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.102+0000 7fc57effd640 1 -- 172.21.15.7:0/858678698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc588018070 con 0x55f603c79350 2024-03-26T08:53:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.102+0000 7fc57effd640 1 -- 172.21.15.7:0/858678698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880123e0 con 0x55f603c79350 2024-03-26T08:53:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.102+0000 7fc5a17f6740 1 -- 172.21.15.7:0/858678698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 msgr2=0x55f603c7de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.102+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/858678698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c7de50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc588007810 tx=0x7fc588007cd0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.103+0000 7fc5a17f6740 1 -- 172.21.15.7:0/858678698 shutdown_connections 2024-03-26T08:53:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.103+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/858678698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c80950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.103+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/858678698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c7de50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.103+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/858678698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c78d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.103+0000 7fc5a17f6740 1 -- 172.21.15.7:0/858678698 >> 172.21.15.7:0/858678698 conn(0x55f603c411a0 msgr2=0x55f603c742d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.103+0000 7fc5a17f6740 1 -- 172.21.15.7:0/858678698 shutdown_connections 2024-03-26T08:53:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.103+0000 7fc5a17f6740 1 -- 172.21.15.7:0/858678698 wait complete. 2024-03-26T08:53:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.103+0000 7fc5a17f6740 1 -- start start 2024-03-26T08:53:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.104+0000 7fc5a17f6740 1 Processor -- start 2024-03-26T08:53:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.104+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c82cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.104+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c83230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c8a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc5a17f6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f603b9a0d0 con 0x55f603c79350 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc5a17f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f603bda110 con 0x55f603c7e4c0 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc5a17f6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f603c81470 con 0x55f603c78960 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc5a17f6740 10 client.0 _ll_register_callbacks cb 0x55f603bd5e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc59c9a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c82cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc59c9a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c82cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49554/0 (socket says 172.21.15.7:49554) 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc59c9a7640 1 -- 172.21.15.7:0/4075277023 learned_addr learned my addr 172.21.15.7:0/4075277023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc597fff640 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c83230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.105+0000 7fc59d1a8640 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c8a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.106+0000 7fc597fff640 1 -- 172.21.15.7:0/4075277023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 msgr2=0x55f603c82cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.106+0000 7fc597fff640 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c82cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.106+0000 7fc597fff640 1 -- 172.21.15.7:0/4075277023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 msgr2=0x55f603c8a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.106+0000 7fc597fff640 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c8a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.106+0000 7fc597fff640 1 -- 172.21.15.7:0/4075277023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6039ba7e0 con 0x55f603c79350 2024-03-26T08:53:48.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.106+0000 7fc597fff640 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c83230 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x55f603c75220 tx=0x7fc5880136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d674e7dcd074895a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.106+0000 7fc577fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:48.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.111+0000 7fc577fff640 1 -- 172.21.15.7:0/4075277023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58800dd60 con 0x55f603c79350 2024-03-26T08:53:48.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.111+0000 7fc5a17f6740 1 -- 172.21.15.7:0/4075277023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc588018070 con 0x55f603c79350 2024-03-26T08:53:48.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.111+0000 7fc577fff640 1 -- 172.21.15.7:0/4075277023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc58801c3f0 con 0x55f603c79350 2024-03-26T08:53:48.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.112+0000 7fc577fff640 1 -- 172.21.15.7:0/4075277023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc58801c610 con 0x55f603c79350 2024-03-26T08:53:48.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.112+0000 7fc5a17f6740 10 client.18432 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:48.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.112+0000 7fc5a17f6740 1 -- 172.21.15.7:0/4075277023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f603a1af70 con 0x55f603c79350 2024-03-26T08:53:48.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.113+0000 7fc577fff640 1 -- 172.21.15.7:0/4075277023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc588018070 con 0x55f603c79350 2024-03-26T08:53:48.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.114+0000 7fc577fff640 1 -- 172.21.15.7:0/4075277023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc58802d050 con 0x55f603c79350 2024-03-26T08:53:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.114+0000 7fc5a17f6740 10 client.18432 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.114+0000 7fc5a17f6740 10 client.18432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.114+0000 7fc5a17f6740 20 client.18432 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.114+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f603c8e760 0x55f603c90b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.114+0000 7fc5a17f6740 4 client.18432 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.114+0000 7fc5a17f6740 1 -- 172.21.15.7:0/4075277023 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f603beb5b0 con 0x55f603c8e760 2024-03-26T08:53:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.116+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f603c8e760 0x55f603c90b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.116+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f603c8e760 0x55f603c90b50 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6784c10adb52637 server_cookie=f48701d6c4fb5a1c in_seq=0 out_seq=0 2024-03-26T08:53:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.117+0000 7fc577fff640 10 client.18432 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc577fff640 1 -- 172.21.15.7:0/4075277023 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f603beb5b0 con 0x55f603c8e760 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc577fff640 10 client.18432 handle_command_reply: tid=0 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc5a17f6740 1 client.18432 shutdown 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc5a17f6740 1 -- 172.21.15.7:0/4075277023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 msgr2=0x55f603c83230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c83230 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x55f603c75220 tx=0x7fc5880136b0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc5a17f6740 1 -- 172.21.15.7:0/4075277023 shutdown_connections 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f603c8e760 0x55f603c90b50 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c8a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c83230 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/4075277023 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c82cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc5a17f6740 1 -- 172.21.15.7:0/4075277023 >> 172.21.15.7:0/4075277023 conn(0x55f603c411a0 msgr2=0x55f603c73530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.118+0000 7fc577fff640 0 client.18432 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.119+0000 7fc5a17f6740 1 -- 172.21.15.7:0/4075277023 shutdown_connections 2024-03-26T08:53:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.119+0000 7fc5a17f6740 1 -- 172.21.15.7:0/4075277023 wait complete. 2024-03-26T08:53:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.120+0000 7fc5a17f6740 20 client.18432 trim_cache size 0 max 16384 2024-03-26T08:53:48.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.150+0000 7fc5a17f6740 1 Processor -- start 2024-03-26T08:53:48.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.150+0000 7fc5a17f6740 1 -- start start 2024-03-26T08:53:48.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c77c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c8a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c78160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc597fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c8a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc59c9a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c77c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc597fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c8a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47710/0 (socket says 172.21.15.7:47710) 2024-03-26T08:53:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc59c9a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c77c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49570/0 (socket says 172.21.15.7:49570) 2024-03-26T08:53:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc597fff640 1 -- 172.21.15.7:0/3717540875 learned_addr learned my addr 172.21.15.7:0/3717540875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc5a17f6740 1 -- 172.21.15.7:0/3717540875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc584010980 con 0x55f603c79350 2024-03-26T08:53:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc5a17f6740 1 -- 172.21.15.7:0/3717540875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58400e480 con 0x55f603c7e4c0 2024-03-26T08:53:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc59d1a8640 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c78160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.151+0000 7fc5a17f6740 1 -- 172.21.15.7:0/3717540875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58400c500 con 0x55f603c78960 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc59c9a7640 1 -- 172.21.15.7:0/3717540875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 msgr2=0x55f603c78160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c78160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc59c9a7640 1 -- 172.21.15.7:0/3717540875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 msgr2=0x55f603c8a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c8a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc59c9a7640 1 -- 172.21.15.7:0/3717540875 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f603c90ec0 con 0x55f603c78960 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc59d1a8640 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c78160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc597fff640 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c8a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c77c20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00bd70 tx=0x7fc58c010fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b31fd037115d62 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc57e7fc640 1 -- 172.21.15.7:0/3717540875 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0b8030 con 0x55f603c78960 2024-03-26T08:53:48.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc57e7fc640 1 -- 172.21.15.7:0/3717540875 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc58c099da0 con 0x55f603c78960 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.152+0000 7fc57e7fc640 1 -- 172.21.15.7:0/3717540875 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0a4010 con 0x55f603c78960 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 -- 172.21.15.7:0/3717540875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 msgr2=0x55f603c77c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c77c20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00bd70 tx=0x7fc58c010fa0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 -- 172.21.15.7:0/3717540875 shutdown_connections 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c7e4c0 0x55f603c78160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c79350 0x55f603c8a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/3717540875 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c77c20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 -- 172.21.15.7:0/3717540875 >> 172.21.15.7:0/3717540875 conn(0x55f603c411a0 msgr2=0x55f603c77510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 -- 172.21.15.7:0/3717540875 shutdown_connections 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 -- 172.21.15.7:0/3717540875 wait complete. 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.153+0000 7fc5a17f6740 1 -- start start 2024-03-26T08:53:48.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.154+0000 7fc5a17f6740 1 Processor -- start 2024-03-26T08:53:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c89c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c79350 0x55f603c8a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc5a17f6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c7e4c0 0x55f603c882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc5a17f6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58400cb00 con 0x55f603c7e4c0 2024-03-26T08:53:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc5a17f6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58400de20 con 0x55f603c79350 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc5a17f6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58400d160 con 0x55f603c78960 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc5a17f6740 10 client.0 _ll_register_callbacks cb 0x7fc590048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc59c9a7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c89c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c79350 0x55f603c8a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc597fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c79350 0x55f603c8a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52540/0 (socket says 172.21.15.7:52540) 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.155+0000 7fc597fff640 1 -- 172.21.15.7:0/774736470 learned_addr learned my addr 172.21.15.7:0/774736470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc59c9a7640 1 -- 172.21.15.7:0/774736470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c79350 msgr2=0x55f603c8a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c79350 0x55f603c8a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc59c9a7640 1 -- 172.21.15.7:0/774736470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c7e4c0 msgr2=0x55f603c882d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c7e4c0 0x55f603c882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc59c9a7640 1 -- 172.21.15.7:0/774736470 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6038b2170 con 0x55f603c78960 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc59c9a7640 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c89c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00bd70 tx=0x7fc58c00bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfb0096b1db7df74 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc577fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:53:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc577fff640 1 -- 172.21.15.7:0/774736470 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0b8030 con 0x55f603c78960 2024-03-26T08:53:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc577fff640 1 -- 172.21.15.7:0/774736470 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc58c09a000 con 0x55f603c78960 2024-03-26T08:53:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.156+0000 7fc5a17f6740 1 -- 172.21.15.7:0/774736470 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc58c099da0 con 0x55f603c78960 2024-03-26T08:53:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.157+0000 7fc577fff640 1 -- 172.21.15.7:0/774736470 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc58c0a4010 con 0x55f603c78960 2024-03-26T08:53:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.157+0000 7fc5a17f6740 10 client.28243 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.157+0000 7fc5a17f6740 1 -- 172.21.15.7:0/774736470 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f603cea8d0 con 0x55f603c78960 2024-03-26T08:53:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.157+0000 7fc577fff640 1 -- 172.21.15.7:0/774736470 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc58c099da0 con 0x55f603c78960 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.158+0000 7fc577fff640 1 -- 172.21.15.7:0/774736470 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc58c0ba3b0 con 0x55f603c78960 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.158+0000 7fc5a17f6740 10 client.28243 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.158+0000 7fc5a17f6740 10 client.28243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.158+0000 7fc5a17f6740 20 client.28243 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.158+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f603ceb320 0x55f603ced740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.158+0000 7fc5a17f6740 4 client.28243 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.158+0000 7fc5a17f6740 1 -- 172.21.15.7:0/774736470 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f603be5900 con 0x55f603ceb320 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.158+0000 7fc597fff640 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f603ceb320 0x55f603ced740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.159+0000 7fc577fff640 10 client.28243 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.159+0000 7fc597fff640 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f603ceb320 0x55f603ced740 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=838b739fe27c5d14 server_cookie=39145e866c42e43a in_seq=0 out_seq=0 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.160+0000 7fc577fff640 1 -- 172.21.15.7:0/774736470 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f603be5900 con 0x55f603ceb320 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.160+0000 7fc577fff640 10 client.28243 handle_command_reply: tid=0 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.160+0000 7fc5a17f6740 1 client.28243 shutdown 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.160+0000 7fc5a17f6740 1 -- 172.21.15.7:0/774736470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 msgr2=0x55f603c89c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.160+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c89c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00bd70 tx=0x7fc58c00bf20 comp rx=0 tx=0).stop 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc5a17f6740 1 -- 172.21.15.7:0/774736470 shutdown_connections 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f603ceb320 0x55f603ced740 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f603c7e4c0 0x55f603c882d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f603c79350 0x55f603c8a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc577fff640 0 client.28243 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc5a17f6740 1 --2- 172.21.15.7:0/774736470 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f603c78960 0x55f603c89c30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc5a17f6740 1 -- 172.21.15.7:0/774736470 >> 172.21.15.7:0/774736470 conn(0x55f603c411a0 msgr2=0x55f603c77da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc5a17f6740 1 -- 172.21.15.7:0/774736470 shutdown_connections 2024-03-26T08:53:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc5a17f6740 1 -- 172.21.15.7:0/774736470 wait complete. 2024-03-26T08:53:48.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.161+0000 7fc5a17f6740 20 client.28243 trim_cache size 0 max 16384 2024-03-26T08:53:48.160 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:53:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:53:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "9bf1c067-396c-45dd-aad8-670176ecca7b", 2024-03-26T08:53:48.160 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:53:48.160 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 -- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5800a2950 msgr2=0x7fc5800a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5800a2950 0x7fc5800a4e10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc590048040 tx=0x7fc590043bf0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 -- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5981086a0 msgr2=0x7fc59819e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5981086a0 0x7fc59819e0f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009fd0 tx=0x7fc58c00b870 comp rx=0 tx=0).stop 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 -- 172.21.15.7:0/132187435 shutdown_connections 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5800a2950 0x7fc5800a4e10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59810d9c0 0x7fc5981a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc59810b010 0x7fc59819e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 --2- 172.21.15.7:0/132187435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5981086a0 0x7fc59819e0f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 -- 172.21.15.7:0/132187435 >> 172.21.15.7:0/132187435 conn(0x7fc59806e640 msgr2=0x7fc59810b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 -- 172.21.15.7:0/132187435 shutdown_connections 2024-03-26T08:53:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.162+0000 7fc59ec0b640 1 -- 172.21.15.7:0/132187435 wait complete. 2024-03-26T08:53:48.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:53:48.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:53:48.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c1d9b640 1 -- 172.21.15.7:0/4279466922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b402cc80 con 0x7f23c41005c0 2024-03-26T08:53:48.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c89bc640 1 -- 172.21.15.7:0/4279466922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41005c0 msgr2=0x7f23c41009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c89bc640 1 --2- 172.21.15.7:0/4279466922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41005c0 0x7f23c41009e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009900 tx=0x7f23b402bc30 comp rx=0 tx=0).stop 2024-03-26T08:53:48.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c89bc640 1 -- 172.21.15.7:0/4279466922 shutdown_connections 2024-03-26T08:53:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c89bc640 1 --2- 172.21.15.7:0/4279466922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4102170 0x7f23c4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c89bc640 1 --2- 172.21.15.7:0/4279466922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c41017b0 0x7f23c4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c89bc640 1 --2- 172.21.15.7:0/4279466922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41005c0 0x7f23c41009e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c89bc640 1 -- 172.21.15.7:0/4279466922 >> 172.21.15.7:0/4279466922 conn(0x7f23c40fbb30 msgr2=0x7f23c40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c89bc640 1 -- 172.21.15.7:0/4279466922 shutdown_connections 2024-03-26T08:53:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.390+0000 7f23c89bc640 1 -- 172.21.15.7:0/4279466922 wait complete. 2024-03-26T08:53:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23c89bc640 1 Processor -- start 2024-03-26T08:53:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23c89bc640 1 -- start start 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23c89bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c41005c0 0x7f23c4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23c89bc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41017b0 0x7f23c4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23c89bc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4102170 0x7f23c419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23c89bc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c410c290 con 0x7f23c41017b0 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23c89bc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c410c110 con 0x7f23c41005c0 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23c89bc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c410c410 con 0x7f23c4102170 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23bbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41017b0 0x7f23c4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23bbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41017b0 0x7f23c4197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47720/0 (socket says 172.21.15.7:47720) 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23bbfff640 1 -- 172.21.15.7:0/2381856979 learned_addr learned my addr 172.21.15.7:0/2381856979 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.391+0000 7f23c2d9d640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4102170 0x7f23c419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.392+0000 7f23bbfff640 1 -- 172.21.15.7:0/2381856979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4102170 msgr2=0x7f23c419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.392+0000 7f23bbfff640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4102170 0x7f23c419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.392+0000 7f23bbfff640 1 -- 172.21.15.7:0/2381856979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c41005c0 msgr2=0x7f23c4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.392+0000 7f23bbfff640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c41005c0 0x7f23c4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.392+0000 7f23bbfff640 1 -- 172.21.15.7:0/2381856979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4102b00 con 0x7f23c41017b0 2024-03-26T08:53:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.393+0000 7f23bbfff640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41017b0 0x7f23c4197f70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00c920 tx=0x7f23ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.393+0000 7f23bb7fe640 1 -- 172.21.15.7:0/2381856979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac007c70 con 0x7f23c41017b0 2024-03-26T08:53:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.394+0000 7f23bb7fe640 1 -- 172.21.15.7:0/2381856979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23ac007e10 con 0x7f23c41017b0 2024-03-26T08:53:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.394+0000 7f23c89bc640 1 -- 172.21.15.7:0/2381856979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c4102df0 con 0x7f23c41017b0 2024-03-26T08:53:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.394+0000 7f23bb7fe640 1 -- 172.21.15.7:0/2381856979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0188b0 con 0x7f23c41017b0 2024-03-26T08:53:48.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.394+0000 7f23c89bc640 1 -- 172.21.15.7:0/2381856979 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c41032e0 con 0x7f23c41017b0 2024-03-26T08:53:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.398+0000 7f23bb7fe640 1 -- 172.21.15.7:0/2381856979 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f23ac018b50 con 0x7f23c41017b0 2024-03-26T08:53:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.398+0000 7f23cb59c740 1 Processor -- start 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.398+0000 7f23cb59c740 1 -- start start 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d1301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d130790 0x56318d135290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d135900 0x56318d137d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23cb59c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318d0953d0 con 0x56318d12fda0 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23cb59c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318d03dad0 con 0x56318d130790 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23cb59c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318d091470 con 0x56318d135900 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bb7fe640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23b00a29d0 0x7f23b00a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d135900 0x56318d137d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d135900 0x56318d137d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49606/0 (socket says 172.21.15.7:49606) 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bbfff640 1 -- 172.21.15.7:0/302356483 learned_addr learned my addr 172.21.15.7:0/302356483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23c2d9d640 1 --2- 172.21.15.7:0/302356483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d1301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bb7fe640 1 -- 172.21.15.7:0/2381856979 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23ac0ceaf0 con 0x7f23c41017b0 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23c2d9d640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23b00a29d0 0x7f23b00a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23c259c640 1 --2- 172.21.15.7:0/302356483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d130790 0x56318d135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bbfff640 1 -- 172.21.15.7:0/302356483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d130790 msgr2=0x56318d135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bbfff640 1 --2- 172.21.15.7:0/302356483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d130790 0x56318d135290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bbfff640 1 -- 172.21.15.7:0/302356483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 msgr2=0x56318d1301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bbfff640 1 --2- 172.21.15.7:0/302356483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d1301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.399+0000 7f23bbfff640 1 -- 172.21.15.7:0/302356483 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318ce737d0 con 0x56318d135900 2024-03-26T08:53:48.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.400+0000 7f23bbfff640 1 --2- 172.21.15.7:0/302356483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d135900 0x56318d137d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f23ac007590 tx=0x7f23ac000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e00da600bfe7f59a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.400+0000 7f23c2d9d640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23b00a29d0 0x7f23b00a4e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005c40 tx=0x7f23bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.400+0000 7f23b8ff9640 1 -- 172.21.15.7:0/302356483 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac09bb50 con 0x56318d135900 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.400+0000 7f23b8ff9640 1 -- 172.21.15.7:0/302356483 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23ac09b450 con 0x56318d135900 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.400+0000 7f23b8ff9640 1 -- 172.21.15.7:0/302356483 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac09b6b0 con 0x56318d135900 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.401+0000 7f23cb59c740 1 -- 172.21.15.7:0/302356483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d135900 msgr2=0x56318d137d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.401+0000 7f23cb59c740 1 --2- 172.21.15.7:0/302356483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d135900 0x56318d137d90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f23ac007590 tx=0x7f23ac000bc0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.401+0000 7f23cb59c740 1 -- 172.21.15.7:0/302356483 shutdown_connections 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.401+0000 7f23cb59c740 1 --2- 172.21.15.7:0/302356483 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d135900 0x56318d137d90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.401+0000 7f23cb59c740 1 --2- 172.21.15.7:0/302356483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d130790 0x56318d135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.401+0000 7f23cb59c740 1 --2- 172.21.15.7:0/302356483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d1301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.402+0000 7f23cb59c740 1 -- 172.21.15.7:0/302356483 >> 172.21.15.7:0/302356483 conn(0x56318d12b310 msgr2=0x56318d12b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.402+0000 7f23cb59c740 1 -- 172.21.15.7:0/302356483 shutdown_connections 2024-03-26T08:53:48.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.402+0000 7f23cb59c740 1 -- 172.21.15.7:0/302356483 wait complete. 2024-03-26T08:53:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.402+0000 7f23cb59c740 1 -- start start 2024-03-26T08:53:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.402+0000 7f23cb59c740 1 Processor -- start 2024-03-26T08:53:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.403+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d13a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.403+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 0x56318d13a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.403+0000 7f23c259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d13a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.403+0000 7f23c259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d13a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47740/0 (socket says 172.21.15.7:47740) 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.403+0000 7f23c259c640 1 -- 172.21.15.7:0/2146123795 learned_addr learned my addr 172.21.15.7:0/2146123795 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.403+0000 7f23bbfff640 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 0x56318d13a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.403+0000 7f23cb59c740 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d135900 0x56318d1416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.404+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318d138900 con 0x56318d12fda0 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.404+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318d051060 con 0x56318d135900 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.404+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318d138a80 con 0x56318d130790 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.404+0000 7f23c2d9d640 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d135900 0x56318d1416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.404+0000 7f23c259c640 1 -- 172.21.15.7:0/2146123795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 msgr2=0x56318d13a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.404+0000 7f23c259c640 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 0x56318d13a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.404+0000 7f23c259c640 1 -- 172.21.15.7:0/2146123795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d135900 msgr2=0x56318d1416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.404+0000 7f23c259c640 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d135900 0x56318d1416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.404+0000 7f23c259c640 1 -- 172.21.15.7:0/2146123795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56318cda93f0 con 0x56318d12fda0 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.405+0000 7f23c259c640 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d13a140 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f23b402c8e0 tx=0x7f23b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6235f183fe6d532 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.405+0000 7f23a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.405+0000 7f23a9ffb640 1 -- 172.21.15.7:0/2146123795 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4042070 con 0x56318d12fda0 2024-03-26T08:53:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.405+0000 7f23a9ffb640 1 -- 172.21.15.7:0/2146123795 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23b4005240 con 0x56318d12fda0 2024-03-26T08:53:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.405+0000 7f23a9ffb640 10 client.0 inactive, discarding osd_map(87..87 src has 1..87) v4 2024-03-26T08:53:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.405+0000 7f23cb59c740 10 client.0 _ll_register_callbacks cb 0x56318d0336a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.405+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23ac09b450 con 0x56318d12fda0 2024-03-26T08:53:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.406+0000 7f23a9ffb640 1 -- 172.21.15.7:0/2146123795 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23b404c050 con 0x56318d12fda0 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.406+0000 7f23cb59c740 10 client.18447 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.406+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x56318cd690f0 con 0x56318d12fda0 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.406+0000 7f23a9ffb640 1 -- 172.21.15.7:0/2146123795 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f23b40046c0 con 0x56318d12fda0 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.406+0000 7f23cb59c740 10 client.18447 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.406+0000 7f23cb59c740 10 client.18447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.406+0000 7f23cb59c740 20 client.18447 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.406+0000 7f23cb59c740 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56318d145a50 0x56318d147f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.407+0000 7f23bbfff640 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56318d145a50 0x56318d147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.407+0000 7f23cb59c740 4 client.18447 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:48.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.407+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23ac09b450 con 0x56318d145a50 2024-03-26T08:53:48.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.409+0000 7f23bbfff640 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56318d145a50 0x56318d147f10 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73dcc9577905c74 server_cookie=9fc4d9efea6f3ecd in_seq=0 out_seq=0 2024-03-26T08:53:48.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.409+0000 7f23a9ffb640 10 client.18447 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:48.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23a9ffb640 1 -- 172.21.15.7:0/2146123795 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f23ac09b450 con 0x56318d145a50 2024-03-26T08:53:48.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23a9ffb640 10 client.18447 handle_command_reply: tid=0 2024-03-26T08:53:48.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 client.18447 shutdown 2024-03-26T08:53:48.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 msgr2=0x56318d13a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d13a140 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f23b402c8e0 tx=0x7f23b4004270 comp rx=0 tx=0).stop 2024-03-26T08:53:48.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 shutdown_connections 2024-03-26T08:53:48.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56318d145a50 0x56318d147f10 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d135900 0x56318d1416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 0x56318d13a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 --2- 172.21.15.7:0/2146123795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12fda0 0x56318d13a140 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23a9ffb640 0 client.18447 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:48.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 >> 172.21.15.7:0/2146123795 conn(0x56318d12b310 msgr2=0x56318d12a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 shutdown_connections 2024-03-26T08:53:48.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.410+0000 7f23cb59c740 1 -- 172.21.15.7:0/2146123795 wait complete. 2024-03-26T08:53:48.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.411+0000 7f23cb59c740 20 client.18447 trim_cache size 0 max 16384 2024-03-26T08:53:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.434+0000 7f23cb59c740 1 Processor -- start 2024-03-26T08:53:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.434+0000 7f23cb59c740 1 -- start start 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12b310 0x56318d14c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12fda0 0x56318d141680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 0x56318d14cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23cb59c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4010800 con 0x56318d12b310 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23cb59c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a400e300 con 0x56318d12fda0 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23cb59c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a400c380 con 0x56318d130790 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23c2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 0x56318d14cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12fda0 0x56318d141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23c2d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 0x56318d14cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49618/0 (socket says 172.21.15.7:49618) 2024-03-26T08:53:48.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23c2d9d640 1 -- 172.21.15.7:0/600745455 learned_addr learned my addr 172.21.15.7:0/600745455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.435+0000 7f23c259c640 1 --2- 172.21.15.7:0/600745455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12b310 0x56318d14c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.436+0000 7f23c259c640 1 -- 172.21.15.7:0/600745455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 msgr2=0x56318d14cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.436+0000 7f23c259c640 1 --2- 172.21.15.7:0/600745455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 0x56318d14cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.436+0000 7f23c259c640 1 -- 172.21.15.7:0/600745455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12fda0 msgr2=0x56318d141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.436+0000 7f23c259c640 1 --2- 172.21.15.7:0/600745455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12fda0 0x56318d141680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.436+0000 7f23c259c640 1 -- 172.21.15.7:0/600745455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318d130b70 con 0x56318d12b310 2024-03-26T08:53:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.436+0000 7f23c259c640 1 --2- 172.21.15.7:0/600745455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12b310 0x56318d14c7f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005ab0 tx=0x7f23b403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65ea026435118f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.436+0000 7f23abfff640 1 -- 172.21.15.7:0/600745455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4042070 con 0x56318d12b310 2024-03-26T08:53:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.436+0000 7f23abfff640 1 -- 172.21.15.7:0/600745455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23b4009520 con 0x56318d12b310 2024-03-26T08:53:48.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.436+0000 7f23abfff640 1 -- 172.21.15.7:0/600745455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4000e60 con 0x56318d12b310 2024-03-26T08:53:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.437+0000 7f23cb59c740 1 -- 172.21.15.7:0/600745455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12b310 msgr2=0x56318d14c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.437+0000 7f23cb59c740 1 --2- 172.21.15.7:0/600745455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12b310 0x56318d14c7f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005ab0 tx=0x7f23b403d000 comp rx=0 tx=0).stop 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.437+0000 7f23cb59c740 1 -- 172.21.15.7:0/600745455 shutdown_connections 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.437+0000 7f23cb59c740 1 --2- 172.21.15.7:0/600745455 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d130790 0x56318d14cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.437+0000 7f23cb59c740 1 --2- 172.21.15.7:0/600745455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12fda0 0x56318d141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.437+0000 7f23cb59c740 1 --2- 172.21.15.7:0/600745455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d12b310 0x56318d14c7f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.437+0000 7f23cb59c740 1 -- 172.21.15.7:0/600745455 >> 172.21.15.7:0/600745455 conn(0x56318d131390 msgr2=0x56318d12ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.437+0000 7f23cb59c740 1 -- 172.21.15.7:0/600745455 shutdown_connections 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 1 -- 172.21.15.7:0/600745455 wait complete. 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 1 -- start start 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 1 Processor -- start 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12b310 0x56318d140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d12fda0 0x56318d1414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d130790 0x56318d13f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a400c980 con 0x56318d130790 2024-03-26T08:53:48.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a400dca0 con 0x56318d12b310 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a400cfe0 con 0x56318d12fda0 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23cb59c740 10 client.0 _ll_register_callbacks cb 0x7f23b402cdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.438+0000 7f23bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d12fda0 0x56318d1414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.439+0000 7f23bbfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d12fda0 0x56318d1414e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:49626/0 (socket says 172.21.15.7:49626) 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.439+0000 7f23bbfff640 1 -- 172.21.15.7:0/3229820412 learned_addr learned my addr 172.21.15.7:0/3229820412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.439+0000 7f23c259c640 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12b310 0x56318d140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.439+0000 7f23c2d9d640 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d130790 0x56318d13f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.439+0000 7f23bbfff640 1 -- 172.21.15.7:0/3229820412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12b310 msgr2=0x56318d140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.439+0000 7f23bbfff640 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12b310 0x56318d140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.439+0000 7f23bbfff640 1 -- 172.21.15.7:0/3229820412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d130790 msgr2=0x56318d13f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.439+0000 7f23bbfff640 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d130790 0x56318d13f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.439+0000 7f23bbfff640 1 -- 172.21.15.7:0/3229820412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56318d13fd90 con 0x56318d12fda0 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.440+0000 7f23bbfff640 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d12fda0 0x56318d1414e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f23ac097e20 tx=0x7f23ac09ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82ba08697c16bca2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.440+0000 7f23a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:53:48.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.440+0000 7f23a9ffb640 1 -- 172.21.15.7:0/3229820412 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0b3020 con 0x56318d12fda0 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.440+0000 7f23cb59c740 1 -- 172.21.15.7:0/3229820412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23b4009520 con 0x56318d12fda0 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.440+0000 7f23a9ffb640 1 -- 172.21.15.7:0/3229820412 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23ac0b4250 con 0x56318d12fda0 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.440+0000 7f23a9ffb640 1 -- 172.21.15.7:0/3229820412 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ac09f5a0 con 0x56318d12fda0 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.440+0000 7f23cb59c740 10 client.28261 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.440+0000 7f23cb59c740 1 -- 172.21.15.7:0/3229820412 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56318d1a1e40 con 0x56318d12fda0 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.440+0000 7f23a9ffb640 1 -- 172.21.15.7:0/3229820412 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23b4009520 con 0x56318d12fda0 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.441+0000 7f23a9ffb640 1 -- 172.21.15.7:0/3229820412 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f23ac0a3760 con 0x56318d12fda0 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.441+0000 7f23cb59c740 10 client.28261 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.441+0000 7f23cb59c740 10 client.28261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.441+0000 7f23cb59c740 20 client.28261 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.441+0000 7f23cb59c740 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56318d135900 0x56318d1a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.441+0000 7f23cb59c740 4 client.28261 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.441+0000 7f23cb59c740 1 -- 172.21.15.7:0/3229820412 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56318d0a2910 con 0x56318d135900 2024-03-26T08:53:48.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.441+0000 7f23c259c640 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56318d135900 0x56318d1a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:48.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.442+0000 7f23c259c640 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56318d135900 0x56318d1a42f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec3121ad6a9cd42 server_cookie=615c00de390b44b in_seq=0 out_seq=0 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.442+0000 7f23a9ffb640 10 client.28261 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.442+0000 7f23a9ffb640 1 -- 172.21.15.7:0/3229820412 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x56318d0a2910 con 0x56318d135900 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.442+0000 7f23a9ffb640 10 client.28261 handle_command_reply: tid=0 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.442+0000 7f23cb59c740 1 client.28261 shutdown 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 -- 172.21.15.7:0/3229820412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d12fda0 msgr2=0x56318d1414e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d12fda0 0x56318d1414e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f23ac097e20 tx=0x7f23ac09ae50 comp rx=0 tx=0).stop 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 -- 172.21.15.7:0/3229820412 shutdown_connections 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56318d135900 0x56318d1a42f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56318d130790 0x56318d13f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23a9ffb640 0 client.28261 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56318d12fda0 0x56318d1414e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 --2- 172.21.15.7:0/3229820412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56318d12b310 0x56318d140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 -- 172.21.15.7:0/3229820412 >> 172.21.15.7:0/3229820412 conn(0x56318d131390 msgr2=0x56318d129ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 -- 172.21.15.7:0/3229820412 shutdown_connections 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 1 -- 172.21.15.7:0/3229820412 wait complete. 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (82 inodes in the stack)", 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "9bf1c067-396c-45dd-aad8-670176ecca7b": { 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "9bf1c067-396c-45dd-aad8-670176ecca7b", 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:53:48.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:53:48.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:53:48.444 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:53:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.443+0000 7f23cb59c740 20 client.28261 trim_cache size 0 max 16384 2024-03-26T08:53:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 -- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23b00a29d0 msgr2=0x7f23b00a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23b00a29d0 0x7f23b00a4e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005c40 tx=0x7f23bc009500 comp rx=0 tx=0).stop 2024-03-26T08:53:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 -- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41017b0 msgr2=0x7f23c4197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41017b0 0x7f23c4197f70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00c920 tx=0x7f23ac00cde0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 -- 172.21.15.7:0/2381856979 shutdown_connections 2024-03-26T08:53:48.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23b00a29d0 0x7f23b00a4e90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23c4102170 0x7f23c419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23c41017b0 0x7f23c4197f70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 --2- 172.21.15.7:0/2381856979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23c41005c0 0x7f23c4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:48.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.444+0000 7f23c89bc640 1 -- 172.21.15.7:0/2381856979 >> 172.21.15.7:0/2381856979 conn(0x7f23c40fbb30 msgr2=0x7f23c40fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:48.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.447+0000 7f23c89bc640 1 -- 172.21.15.7:0/2381856979 shutdown_connections 2024-03-26T08:53:48.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:53:48.447+0000 7f23c89bc640 1 -- 172.21.15.7:0/2381856979 wait complete. 2024-03-26T08:53:48.462 INFO:tasks.cephfs.filesystem:scrub status for tag:9bf1c067-396c-45dd-aad8-670176ecca7b - {'path': '/', 'tag': '9bf1c067-396c-45dd-aad8-670176ecca7b', 'options': 'recursive,force'} 2024-03-26T08:53:49.321 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.321 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.321 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:53:49.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:49.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:53:49.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:53:49.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:49 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:53:49.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:53:49.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:49 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:49.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:53:49.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:49 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:50.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:50 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:50.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:50 smithi016 ceph-mon[35851]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.9 MiB/s wr, 621 op/s 2024-03-26T08:53:50.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:53:50.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:53:50.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:50.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:50 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:50.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:50 smithi007 ceph-mon[32462]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.9 MiB/s wr, 621 op/s 2024-03-26T08:53:50.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:53:50.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:53:50.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:50.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:50 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:53:50.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:50 smithi044 ceph-mon[36461]: pgmap v702: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.9 MiB/s wr, 621 op/s 2024-03-26T08:53:50.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:53:50.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:53:50.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:53:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 680.0 s, 1668.4 tps, lat 60.266 ms stddev 67.435 2024-03-26T08:53:52.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:52 smithi016 ceph-mon[35851]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.5 MiB/s wr, 696 op/s 2024-03-26T08:53:52.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:52 smithi007 ceph-mon[32462]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.5 MiB/s wr, 696 op/s 2024-03-26T08:53:52.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:52 smithi044 ceph-mon[36461]: pgmap v703: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.5 MiB/s wr, 696 op/s 2024-03-26T08:53:54.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:54 smithi016 ceph-mon[35851]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.2 MiB/s wr, 791 op/s 2024-03-26T08:53:54.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:54 smithi007 ceph-mon[32462]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.2 MiB/s wr, 791 op/s 2024-03-26T08:53:54.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:54 smithi044 ceph-mon[36461]: pgmap v704: 97 pgs: 97 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.2 MiB/s wr, 791 op/s 2024-03-26T08:53:56.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 685.0 s, 1541.8 tps, lat 64.784 ms stddev 69.872 2024-03-26T08:53:56.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:56 smithi016 ceph-mon[35851]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 739 op/s 2024-03-26T08:53:56.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:56 smithi007 ceph-mon[32462]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 739 op/s 2024-03-26T08:53:56.570 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:56 smithi044 ceph-mon[36461]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.8 MiB/s wr, 739 op/s 2024-03-26T08:53:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.084+0000 7f6db9ab5640 1 -- 172.21.15.7:0/1770288318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6db4107d20 msgr2=0x7f6db4108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.084+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/1770288318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6db4107d20 0x7f6db4108140 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009900 tx=0x7f6da402bc30 comp rx=0 tx=0).stop 2024-03-26T08:53:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.084+0000 7f6db9ab5640 1 -- 172.21.15.7:0/1770288318 shutdown_connections 2024-03-26T08:53:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.084+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/1770288318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db41098d0 0x7f6db4110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.084+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/1770288318 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db4108f10 0x7f6db4109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.084+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/1770288318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6db4107d20 0x7f6db4108140 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.084+0000 7f6db9ab5640 1 -- 172.21.15.7:0/1770288318 >> 172.21.15.7:0/1770288318 conn(0x7f6db4076cf0 msgr2=0x7f6db4079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.084+0000 7f6db9ab5640 1 -- 172.21.15.7:0/1770288318 shutdown_connections 2024-03-26T08:53:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.085+0000 7f6db9ab5640 1 -- 172.21.15.7:0/1770288318 wait complete. 2024-03-26T08:53:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.085+0000 7f6db9ab5640 1 Processor -- start 2024-03-26T08:53:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.085+0000 7f6db9ab5640 1 -- start start 2024-03-26T08:53:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.085+0000 7f6db9ab5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6db4107d20 0x7f6db419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.085+0000 7f6db9ab5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db4108f10 0x7f6db419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db9ab5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db41098d0 0x7f6db41a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db9ab5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4113990 con 0x7f6db4107d20 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db9ab5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4113810 con 0x7f6db41098d0 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db9ab5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4113b10 con 0x7f6db4108f10 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db4108f10 0x7f6db419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db4108f10 0x7f6db419e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:59922/0 (socket says 172.21.15.7:59922) 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db2ffd640 1 -- 172.21.15.7:0/787829100 learned_addr learned my addr 172.21.15.7:0/787829100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db37fe640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6db4107d20 0x7f6db419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db3fff640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db41098d0 0x7f6db41a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db2ffd640 1 -- 172.21.15.7:0/787829100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db41098d0 msgr2=0x7f6db41a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db41098d0 0x7f6db41a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db2ffd640 1 -- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6db4107d20 msgr2=0x7f6db419e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6db4107d20 0x7f6db419e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db2ffd640 1 -- 172.21.15.7:0/787829100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db407a020 con 0x7f6db4108f10 2024-03-26T08:53:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.086+0000 7f6db37fe640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6db4107d20 0x7f6db419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:53:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.087+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db4108f10 0x7f6db419e550 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6da800e9f0 tx=0x7f6da800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.087+0000 7f6db0ff9640 1 -- 172.21.15.7:0/787829100 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800cd80 con 0x7f6db4108f10 2024-03-26T08:53:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.087+0000 7f6db0ff9640 1 -- 172.21.15.7:0/787829100 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6da8005500 con 0x7f6db4108f10 2024-03-26T08:53:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.087+0000 7f6db9ab5640 1 -- 172.21.15.7:0/787829100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db407a310 con 0x7f6db4108f10 2024-03-26T08:53:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.088+0000 7f6db0ff9640 1 -- 172.21.15.7:0/787829100 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8010690 con 0x7f6db4108f10 2024-03-26T08:53:57.086 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.088+0000 7f6db9ab5640 1 -- 172.21.15.7:0/787829100 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db407a860 con 0x7f6db4108f10 2024-03-26T08:53:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.090+0000 7f6db0ff9640 1 -- 172.21.15.7:0/787829100 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6da80046c0 con 0x7f6db4108f10 2024-03-26T08:53:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6dbc776740 1 Processor -- start 2024-03-26T08:53:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6dbc776740 1 -- start start 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6db0ff9640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6d9c0a2870 0x7f6d9c0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0812920 0x563dd0812d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0817e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0818480 0x563dd081a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6dbc776740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd0777b10 con 0x563dd0818480 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6dbc776740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd0720900 con 0x563dd0813310 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6dbc776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd0773bb0 con 0x563dd0812920 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0817e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6db37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0812920 0x563dd0812d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0817e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44624/0 (socket says 172.21.15.7:44624) 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6db2ffd640 1 -- 172.21.15.7:0/2868913170 learned_addr learned my addr 172.21.15.7:0/2868913170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6db3fff640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6d9c0a2870 0x7f6d9c0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6db0ff9640 1 -- 172.21.15.7:0/787829100 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6da80cef60 con 0x7f6db4108f10 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.091+0000 7f6db3fff640 1 --2- 172.21.15.7:0/2868913170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0818480 0x563dd081a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6db2ffd640 1 -- 172.21.15.7:0/2868913170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0812920 msgr2=0x563dd0812d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/2868913170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0812920 0x563dd0812d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6db2ffd640 1 -- 172.21.15.7:0/2868913170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0818480 msgr2=0x563dd081a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/2868913170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0818480 0x563dd081a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6db2ffd640 1 -- 172.21.15.7:0/2868913170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dd048c210 con 0x563dd0813310 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6db3fff640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6d9c0a2870 0x7f6d9c0a4d30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009fd0 tx=0x7f6dac009dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/2868913170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0817e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6da80cca20 tx=0x7f6da8097c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=436d9e8d95d3892a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6d99ffb640 1 -- 172.21.15.7:0/2868913170 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80b2ff0 con 0x563dd0813310 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6d99ffb640 1 -- 172.21.15.7:0/2868913170 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6da8084e50 con 0x563dd0813310 2024-03-26T08:53:57.091 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.092+0000 7f6d99ffb640 1 -- 172.21.15.7:0/2868913170 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80b4980 con 0x563dd0813310 2024-03-26T08:53:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.093+0000 7f6dbc776740 1 -- 172.21.15.7:0/2868913170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 msgr2=0x563dd0817e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.093+0000 7f6dbc776740 1 --2- 172.21.15.7:0/2868913170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0817e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6da80cca20 tx=0x7f6da8097c20 comp rx=0 tx=0).stop 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.094+0000 7f6dbc776740 1 -- 172.21.15.7:0/2868913170 shutdown_connections 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.094+0000 7f6dbc776740 1 --2- 172.21.15.7:0/2868913170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0818480 0x563dd081a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.095+0000 7f6dbc776740 1 --2- 172.21.15.7:0/2868913170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0817e10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.095+0000 7f6dbc776740 1 --2- 172.21.15.7:0/2868913170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0812920 0x563dd0812d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.095+0000 7f6dbc776740 1 -- 172.21.15.7:0/2868913170 >> 172.21.15.7:0/2868913170 conn(0x563dd080de90 msgr2=0x563dd080e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.095+0000 7f6dbc776740 1 -- 172.21.15.7:0/2868913170 shutdown_connections 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.095+0000 7f6dbc776740 1 -- 172.21.15.7:0/2868913170 wait complete. 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.095+0000 7f6dbc776740 1 -- start start 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6dbc776740 1 Processor -- start 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd081ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0813310 0x563dd081d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0818480 0x563dd0824230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.097 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd081ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6dbc776740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd081b430 con 0x563dd0812920 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6dbc776740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd0733d50 con 0x563dd0818480 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6dbc776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dd081b5b0 con 0x563dd0813310 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd081ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44802/0 (socket says 172.21.15.7:44802) 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db37fe640 1 -- 172.21.15.7:0/3937076918 learned_addr learned my addr 172.21.15.7:0/3937076918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6dbc776740 10 client.0 _ll_register_callbacks cb 0x563dd0733720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db3fff640 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0818480 0x563dd0824230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0813310 0x563dd081d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db37fe640 1 -- 172.21.15.7:0/3937076918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0813310 msgr2=0x563dd081d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db37fe640 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0813310 0x563dd081d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db37fe640 1 -- 172.21.15.7:0/3937076918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0818480 msgr2=0x563dd0824230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db37fe640 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0818480 0x563dd0824230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db37fe640 1 -- 172.21.15.7:0/3937076918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563dd0558620 con 0x563dd0812920 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.096+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0813310 0x563dd081d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6db37fe640 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd081ccb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6db4108d70 tx=0x7f6da40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa6a5f3ed1b4a6e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6d92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6d92ffd640 1 -- 172.21.15.7:0/3937076918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4005d50 con 0x563dd0812920 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6d92ffd640 1 -- 172.21.15.7:0/3937076918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6da4005450 con 0x563dd0812920 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6dbc776740 1 -- 172.21.15.7:0/3937076918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6da8084e50 con 0x563dd0812920 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6d92ffd640 1 -- 172.21.15.7:0/3937076918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6da40044e0 con 0x563dd0812920 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6dbc776740 10 client.18465 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6dbc776740 1 -- 172.21.15.7:0/3937076918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563dd08275f0 con 0x563dd0812920 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6d92ffd640 1 -- 172.21.15.7:0/3937076918 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6da8084e50 con 0x563dd0812920 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6d92ffd640 1 -- 172.21.15.7:0/3937076918 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6da404e050 con 0x563dd0812920 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6dbc776740 10 client.18465 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6dbc776740 10 client.18465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6dbc776740 20 client.18465 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6dbc776740 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563dd0828c10 0x563dd082b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6dbc776740 4 client.18465 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.097+0000 7f6dbc776740 1 -- 172.21.15.7:0/3937076918 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563dd0785050 con 0x563dd0828c10 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.098+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563dd0828c10 0x563dd082b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.098+0000 7f6d92ffd640 10 client.18465 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.098+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563dd0828c10 0x563dd082b060 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9fb4a50b9ad6851 server_cookie=54143f362a344473 in_seq=0 out_seq=0 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6d92ffd640 1 -- 172.21.15.7:0/3937076918 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563dd0785050 con 0x563dd0828c10 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6d92ffd640 10 client.18465 handle_command_reply: tid=0 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 client.18465 shutdown 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 -- 172.21.15.7:0/3937076918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 msgr2=0x563dd081ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd081ccb0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6db4108d70 tx=0x7f6da40042f0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 -- 172.21.15.7:0/3937076918 shutdown_connections 2024-03-26T08:53:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563dd0828c10 0x563dd082b060 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0818480 0x563dd0824230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0813310 0x563dd081d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6d92ffd640 0 client.18465 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 --2- 172.21.15.7:0/3937076918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd081ccb0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 -- 172.21.15.7:0/3937076918 >> 172.21.15.7:0/3937076918 conn(0x563dd080de90 msgr2=0x563dd080d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 -- 172.21.15.7:0/3937076918 shutdown_connections 2024-03-26T08:53:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.099+0000 7f6dbc776740 1 -- 172.21.15.7:0/3937076918 wait complete. 2024-03-26T08:53:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.100+0000 7f6dbc776740 20 client.18465 trim_cache size 0 max 16384 2024-03-26T08:53:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.116+0000 7f6dbc776740 1 Processor -- start 2024-03-26T08:53:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.116+0000 7f6dbc776740 1 -- start start 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.116+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd080c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0824150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 0x563dd080ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6dbc776740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db41b9e70 con 0x563dd0812920 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6dbc776740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db41b7970 con 0x563dd0813310 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6dbc776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40815e0 con 0x563dd0818480 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd080c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0824150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0824150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44630/0 (socket says 172.21.15.7:44630) 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db2ffd640 1 -- 172.21.15.7:0/1479338453 learned_addr learned my addr 172.21.15.7:0/1479338453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db3fff640 1 --2- 172.21.15.7:0/1479338453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 0x563dd080ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db2ffd640 1 -- 172.21.15.7:0/1479338453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 msgr2=0x563dd080ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/1479338453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 0x563dd080ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db2ffd640 1 -- 172.21.15.7:0/1479338453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 msgr2=0x563dd080c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/1479338453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd080c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db2ffd640 1 -- 172.21.15.7:0/1479338453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dd081b6b0 con 0x563dd0813310 2024-03-26T08:53:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/1479338453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0824150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c6a0 tx=0x7f6da8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10ec371fc47da243 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6d997fa640 1 -- 172.21.15.7:0/1479338453 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80ba070 con 0x563dd0813310 2024-03-26T08:53:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6d997fa640 1 -- 172.21.15.7:0/1479338453 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6da8084e50 con 0x563dd0813310 2024-03-26T08:53:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.117+0000 7f6d997fa640 1 -- 172.21.15.7:0/1479338453 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80b4b60 con 0x563dd0813310 2024-03-26T08:53:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.118+0000 7f6dbc776740 1 -- 172.21.15.7:0/1479338453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 msgr2=0x563dd0824150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.118+0000 7f6dbc776740 1 --2- 172.21.15.7:0/1479338453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0824150 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6da800c6a0 tx=0x7f6da8004a40 comp rx=0 tx=0).stop 2024-03-26T08:53:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.118+0000 7f6dbc776740 1 -- 172.21.15.7:0/1479338453 shutdown_connections 2024-03-26T08:53:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.118+0000 7f6dbc776740 1 --2- 172.21.15.7:0/1479338453 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 0x563dd080ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.118+0000 7f6dbc776740 1 --2- 172.21.15.7:0/1479338453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0813310 0x563dd0824150 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.118+0000 7f6dbc776740 1 --2- 172.21.15.7:0/1479338453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0812920 0x563dd080c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.118+0000 7f6dbc776740 1 -- 172.21.15.7:0/1479338453 >> 172.21.15.7:0/1479338453 conn(0x563dd080de90 msgr2=0x563dd080e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.118+0000 7f6dbc776740 1 -- 172.21.15.7:0/1479338453 shutdown_connections 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.118+0000 7f6dbc776740 1 -- 172.21.15.7:0/1479338453 wait complete. 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6dbc776740 1 -- start start 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6dbc776740 1 Processor -- start 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0812920 0x563dd0821d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0813310 0x563dd0822240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6dbc776740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 0x563dd0823250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6dbc776740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4081be0 con 0x563dd0813310 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0813310 0x563dd0822240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6dbc776740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db41b7310 con 0x563dd0812920 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6dbc776740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4082240 con 0x563dd0818480 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0813310 0x563dd0822240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44814/0 (socket says 172.21.15.7:44814) 2024-03-26T08:53:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0812920 0x563dd0821d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db2ffd640 1 -- 172.21.15.7:0/172844921 learned_addr learned my addr 172.21.15.7:0/172844921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6dbc776740 10 client.0 _ll_register_callbacks cb 0x7f6da8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db3fff640 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 0x563dd0823250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db2ffd640 1 -- 172.21.15.7:0/172844921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 msgr2=0x563dd0823250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 0x563dd0823250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db2ffd640 1 -- 172.21.15.7:0/172844921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0812920 msgr2=0x563dd0821d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0812920 0x563dd0821d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db2ffd640 1 -- 172.21.15.7:0/172844921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563dd0823840 con 0x563dd0813310 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6db37fe640 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0812920 0x563dd0821d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.119+0000 7f6db3fff640 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 0x563dd0823250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6db2ffd640 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0813310 0x563dd0822240 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6da80097d0 tx=0x7f6da8097560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf4526adad06882 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6d92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:53:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6d92ffd640 1 -- 172.21.15.7:0/172844921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da80ba040 con 0x563dd0813310 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6d92ffd640 1 -- 172.21.15.7:0/172844921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6da8097f30 con 0x563dd0813310 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6dbc776740 1 -- 172.21.15.7:0/172844921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6da8084e50 con 0x563dd0813310 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6d92ffd640 1 -- 172.21.15.7:0/172844921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6da80b4b60 con 0x563dd0813310 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6dbc776740 10 client.18471 fetch_fsmap learned FSMap version 25 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6dbc776740 1 -- 172.21.15.7:0/172844921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563dd08849a0 con 0x563dd0813310 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6d92ffd640 1 -- 172.21.15.7:0/172844921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6da8084e50 con 0x563dd0813310 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.120+0000 7f6d92ffd640 1 -- 172.21.15.7:0/172844921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6da80aa120 con 0x563dd0813310 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.121+0000 7f6dbc776740 10 client.18471 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.121+0000 7f6dbc776740 10 client.18471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.121+0000 7f6dbc776740 20 client.18471 populate_metadata read hostname 'smithi007' 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.121+0000 7f6dbc776740 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563dd0884e20 0x563dd0887240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.121+0000 7f6dbc776740 4 client.18471 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.121+0000 7f6dbc776740 1 -- 172.21.15.7:0/172844921 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563dd077f3a0 con 0x563dd0884e20 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.121+0000 7f6db37fe640 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563dd0884e20 0x563dd0887240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:53:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.121+0000 7f6db37fe640 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563dd0884e20 0x563dd0887240 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa9aaa4395e45a7 server_cookie=90330c59ce15ff79 in_seq=0 out_seq=0 2024-03-26T08:53:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.121+0000 7f6d92ffd640 10 client.18471 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6d92ffd640 1 -- 172.21.15.7:0/172844921 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563dd077f3a0 con 0x563dd0884e20 2024-03-26T08:53:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6d92ffd640 10 client.18471 handle_command_reply: tid=0 2024-03-26T08:53:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 client.18471 shutdown 2024-03-26T08:53:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 -- 172.21.15.7:0/172844921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0813310 msgr2=0x563dd0822240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0813310 0x563dd0822240 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6da80097d0 tx=0x7f6da8097560 comp rx=0 tx=0).stop 2024-03-26T08:53:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 -- 172.21.15.7:0/172844921 shutdown_connections 2024-03-26T08:53:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x563dd0884e20 0x563dd0887240 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6d92ffd640 0 client.18471 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563dd0818480 0x563dd0823250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563dd0813310 0x563dd0822240 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 --2- 172.21.15.7:0/172844921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563dd0812920 0x563dd0821d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 -- 172.21.15.7:0/172844921 >> 172.21.15.7:0/172844921 conn(0x563dd080de90 msgr2=0x563dd080c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 -- 172.21.15.7:0/172844921 shutdown_connections 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 1 -- 172.21.15.7:0/172844921 wait complete. 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.135+0000 7f6dbc776740 20 client.18471 trim_cache size 0 max 16384 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:53:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:53:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 -- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6d9c0a2870 msgr2=0x7f6d9c0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6d9c0a2870 0x7f6d9c0a4d30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009fd0 tx=0x7f6dac009dc0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 -- 172.21.15.7:0/787829100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db4108f10 msgr2=0x7f6db419e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:53:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db4108f10 0x7f6db419e550 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6da800e9f0 tx=0x7f6da800edf0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 -- 172.21.15.7:0/787829100 shutdown_connections 2024-03-26T08:53:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6d9c0a2870 0x7f6d9c0a4d30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db41098d0 0x7f6db41a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6db4108f10 0x7f6db419e550 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 --2- 172.21.15.7:0/787829100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6db4107d20 0x7f6db419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:53:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 -- 172.21.15.7:0/787829100 >> 172.21.15.7:0/787829100 conn(0x7f6db4076cf0 msgr2=0x7f6db410c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:53:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 -- 172.21.15.7:0/787829100 shutdown_connections 2024-03-26T08:53:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:53:57.136+0000 7f6db9ab5640 1 -- 172.21.15.7:0/787829100 wait complete. 2024-03-26T08:53:57.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:57 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:53:57.448 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:57 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:53:57.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:57 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:53:57.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:57 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:53:57.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:57 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:53:57.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:57 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:53:58.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:53:58 smithi016 ceph-mon[35851]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.1 MiB/s wr, 757 op/s 2024-03-26T08:53:58.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:53:58 smithi007 ceph-mon[32462]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.1 MiB/s wr, 757 op/s 2024-03-26T08:53:58.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:53:58 smithi044 ceph-mon[36461]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.1 MiB/s wr, 757 op/s 2024-03-26T08:54:00.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:00 smithi016 ceph-mon[35851]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.6 MiB/s wr, 667 op/s 2024-03-26T08:54:00.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:00 smithi007 ceph-mon[32462]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.6 MiB/s wr, 667 op/s 2024-03-26T08:54:00.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:00 smithi044 ceph-mon[36461]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 5.6 MiB/s wr, 667 op/s 2024-03-26T08:54:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 690.0 s, 1859.4 tps, lat 53.608 ms stddev 66.528 2024-03-26T08:54:02.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:02 smithi016 ceph-mon[35851]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.0 MiB/s wr, 704 op/s 2024-03-26T08:54:02.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:02 smithi007 ceph-mon[32462]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.0 MiB/s wr, 704 op/s 2024-03-26T08:54:02.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:02 smithi044 ceph-mon[36461]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.0 MiB/s wr, 704 op/s 2024-03-26T08:54:04.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:04 smithi016 ceph-mon[35851]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.5 MiB/s wr, 723 op/s 2024-03-26T08:54:04.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:04 smithi007 ceph-mon[32462]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.5 MiB/s wr, 723 op/s 2024-03-26T08:54:04.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:04 smithi044 ceph-mon[36461]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.5 MiB/s wr, 723 op/s 2024-03-26T08:54:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 695.0 s, 1772.2 tps, lat 56.627 ms stddev 66.450 2024-03-26T08:54:06.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:06 smithi016 ceph-mon[35851]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 MiB/s wr, 642 op/s 2024-03-26T08:54:06.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:06 smithi007 ceph-mon[32462]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 MiB/s wr, 642 op/s 2024-03-26T08:54:06.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:06 smithi044 ceph-mon[36461]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 MiB/s wr, 642 op/s 2024-03-26T08:54:08.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:08 smithi016 ceph-mon[35851]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-03-26T08:54:08.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:08 smithi007 ceph-mon[32462]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-03-26T08:54:08.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:08 smithi044 ceph-mon[36461]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.257+0000 7fae99c9a640 1 -- 172.21.15.7:0/2767860275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae94102180 msgr2=0x7fae94108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.257+0000 7fae99c9a640 1 --2- 172.21.15.7:0/2767860275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae94102180 0x7fae94108a10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009be0 tx=0x7fae8c02bef0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.257+0000 7fae99c9a640 1 -- 172.21.15.7:0/2767860275 shutdown_connections 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.257+0000 7fae99c9a640 1 --2- 172.21.15.7:0/2767860275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae94102180 0x7fae94108a10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.257+0000 7fae99c9a640 1 --2- 172.21.15.7:0/2767860275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae941017c0 0x7fae94101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.257+0000 7fae99c9a640 1 --2- 172.21.15.7:0/2767860275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae941005d0 0x7fae941009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.257+0000 7fae99c9a640 1 -- 172.21.15.7:0/2767860275 >> 172.21.15.7:0/2767860275 conn(0x7fae940fbb40 msgr2=0x7fae940fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.257+0000 7fae99c9a640 1 -- 172.21.15.7:0/2767860275 shutdown_connections 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.257+0000 7fae99c9a640 1 -- 172.21.15.7:0/2767860275 wait complete. 2024-03-26T08:54:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.258+0000 7fae99c9a640 1 Processor -- start 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.258+0000 7fae99c9a640 1 -- start start 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.258+0000 7fae99c9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae941005d0 0x7fae94196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.258+0000 7fae99c9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae941017c0 0x7fae94196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.258+0000 7fae99c9a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae94102180 0x7fae9419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.258+0000 7fae99c9a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9410c230 con 0x7fae94102180 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.258+0000 7fae99c9a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9410c0b0 con 0x7fae941005d0 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.258+0000 7fae99c9a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9410c3b0 con 0x7fae941017c0 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae941005d0 0x7fae94196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae93fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae94102180 0x7fae9419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae92ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae941017c0 0x7fae94196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae941005d0 0x7fae94196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34300/0 (socket says 172.21.15.7:34300) 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae937fe640 1 -- 172.21.15.7:0/1177096864 learned_addr learned my addr 172.21.15.7:0/1177096864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae937fe640 1 -- 172.21.15.7:0/1177096864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae941017c0 msgr2=0x7fae94196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae937fe640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae941017c0 0x7fae94196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae937fe640 1 -- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae94102180 msgr2=0x7fae9419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae937fe640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae94102180 0x7fae9419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae937fe640 1 -- 172.21.15.7:0/1177096864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9419e4a0 con 0x7fae941005d0 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae93fff640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae94102180 0x7fae9419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae92ffd640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae941017c0 0x7fae94196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae937fe640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae941005d0 0x7fae94196880 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e9f0 tx=0x7fae7c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae90ff9640 1 -- 172.21.15.7:0/1177096864 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00ccc0 con 0x7fae941005d0 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae99c9a640 1 -- 172.21.15.7:0/1177096864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9419e790 con 0x7fae941005d0 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae90ff9640 1 -- 172.21.15.7:0/1177096864 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae7c0035f0 con 0x7fae941005d0 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae90ff9640 1 -- 172.21.15.7:0/1177096864 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00ce60 con 0x7fae941005d0 2024-03-26T08:54:10.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.259+0000 7fae99c9a640 1 -- 172.21.15.7:0/1177096864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9419ed30 con 0x7fae941005d0 2024-03-26T08:54:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.262+0000 7fae90ff9640 1 -- 172.21.15.7:0/1177096864 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fae7c0037a0 con 0x7fae941005d0 2024-03-26T08:54:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.262+0000 7fae9c966740 1 Processor -- start 2024-03-26T08:54:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae9c966740 1 -- start start 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae9c966740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8cdb30 0x560f2f8cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae9c966740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae9c966740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8d3690 0x560f2f8d5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae9c966740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2f830b10 con 0x560f2f8cdb30 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae9c966740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2f7d9900 con 0x560f2f8d3690 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae9c966740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2f82cbb0 con 0x560f2f8ce520 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae93fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8cdb30 0x560f2f8cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae90ff9640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae800a2870 0x7fae800a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae92ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8d3690 0x560f2f8d5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae92ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8d3690 0x560f2f8d5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34312/0 (socket says 172.21.15.7:34312) 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae92ffd640 1 -- 172.21.15.7:0/590577443 learned_addr learned my addr 172.21.15.7:0/590577443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae937fe640 1 --2- 172.21.15.7:0/590577443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.263+0000 7fae93fff640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae800a2870 0x7fae800a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae92ffd640 1 -- 172.21.15.7:0/590577443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 msgr2=0x560f2f8d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae92ffd640 1 --2- 172.21.15.7:0/590577443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae90ff9640 1 -- 172.21.15.7:0/1177096864 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae7c0ced00 con 0x7fae941005d0 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae92ffd640 1 -- 172.21.15.7:0/590577443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8cdb30 msgr2=0x560f2f8cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae92ffd640 1 --2- 172.21.15.7:0/590577443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8cdb30 0x560f2f8cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae92ffd640 1 -- 172.21.15.7:0/590577443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f2f545210 con 0x560f2f8d3690 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae92ffd640 1 --2- 172.21.15.7:0/590577443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8d3690 0x560f2f8d5b20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fae84010480 tx=0x7fae84010940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf08b52ead68cf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae93fff640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae800a2870 0x7fae800a4d30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fae8c004ab0 tx=0x7fae8c02bef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae89ffb640 1 -- 172.21.15.7:0/590577443 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84011760 con 0x560f2f8d3690 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae89ffb640 1 -- 172.21.15.7:0/590577443 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae84011de0 con 0x560f2f8d3690 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.264+0000 7fae89ffb640 1 -- 172.21.15.7:0/590577443 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400e430 con 0x560f2f8d3690 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 -- 172.21.15.7:0/590577443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8d3690 msgr2=0x560f2f8d5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 --2- 172.21.15.7:0/590577443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8d3690 0x560f2f8d5b20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fae84010480 tx=0x7fae84010940 comp rx=0 tx=0).stop 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 -- 172.21.15.7:0/590577443 shutdown_connections 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 --2- 172.21.15.7:0/590577443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8d3690 0x560f2f8d5b20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 --2- 172.21.15.7:0/590577443 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 --2- 172.21.15.7:0/590577443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8cdb30 0x560f2f8cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 -- 172.21.15.7:0/590577443 >> 172.21.15.7:0/590577443 conn(0x560f2f8c6e90 msgr2=0x560f2f8c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 -- 172.21.15.7:0/590577443 shutdown_connections 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 -- 172.21.15.7:0/590577443 wait complete. 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 -- start start 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.265+0000 7fae9c966740 1 Processor -- start 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae9c966740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae9c966740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8ce520 0x560f2f8d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae9c966740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8d3690 0x560f2f8e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae9c966740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2f8d6640 con 0x560f2f8ce520 2024-03-26T08:54:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae9c966740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2f7ecd50 con 0x560f2f8cdb30 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae9c966740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2f8d67c0 con 0x560f2f8d3690 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34314/0 (socket says 172.21.15.7:34314) 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae937fe640 1 -- 172.21.15.7:0/3313740467 learned_addr learned my addr 172.21.15.7:0/3313740467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae92ffd640 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8ce520 0x560f2f8d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae93fff640 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8d3690 0x560f2f8e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae9c966740 10 client.0 _ll_register_callbacks cb 0x560f2f7ec720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae937fe640 1 -- 172.21.15.7:0/3313740467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8d3690 msgr2=0x560f2f8e58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae937fe640 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8d3690 0x560f2f8e58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae937fe640 1 -- 172.21.15.7:0/3313740467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8ce520 msgr2=0x560f2f8d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae937fe640 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8ce520 0x560f2f8d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae937fe640 1 -- 172.21.15.7:0/3313740467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f2f611620 con 0x560f2f8cdb30 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.266+0000 7fae93fff640 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8d3690 0x560f2f8e58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.267+0000 7fae937fe640 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2510 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009230 tx=0x7fae7c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ed3bfdf44759ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.267+0000 7fae72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:10.270 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.267+0000 7fae72ffd640 1 -- 172.21.15.7:0/3313740467 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0860b0 con 0x560f2f8cdb30 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.267+0000 7fae72ffd640 1 -- 172.21.15.7:0/3313740467 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae7c0a3970 con 0x560f2f8cdb30 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.267+0000 7fae9c966740 1 -- 172.21.15.7:0/3313740467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae84011de0 con 0x560f2f8cdb30 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.267+0000 7fae72ffd640 1 -- 172.21.15.7:0/3313740467 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae7c09bd60 con 0x560f2f8cdb30 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.267+0000 7fae9c966740 10 client.28295 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.267+0000 7fae9c966740 1 -- 172.21.15.7:0/3313740467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560f2f8e8cb0 con 0x560f2f8cdb30 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae72ffd640 1 -- 172.21.15.7:0/3313740467 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae7c09b660 con 0x560f2f8cdb30 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae72ffd640 1 -- 172.21.15.7:0/3313740467 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fae7c003fb0 con 0x560f2f8cdb30 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae9c966740 10 client.28295 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae9c966740 10 client.28295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae9c966740 20 client.28295 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae9c966740 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560f2f8ea250 0x560f2f8ec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae9c966740 4 client.28295 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:10.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae9c966740 1 -- 172.21.15.7:0/3313740467 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae84011de0 con 0x560f2f8ea250 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae92ffd640 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560f2f8ea250 0x560f2f8ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae92ffd640 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560f2f8ea250 0x560f2f8ec660 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc5a0f2287e0255 server_cookie=5b60023a8ae5f8bc in_seq=0 out_seq=0 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.268+0000 7fae72ffd640 10 client.28295 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae72ffd640 1 -- 172.21.15.7:0/3313740467 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fae84011de0 con 0x560f2f8ea250 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae72ffd640 10 client.28295 handle_command_reply: tid=0 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 client.28295 shutdown 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 -- 172.21.15.7:0/3313740467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 msgr2=0x560f2f8d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2510 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009230 tx=0x7fae7c01d000 comp rx=0 tx=0).stop 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 -- 172.21.15.7:0/3313740467 shutdown_connections 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560f2f8ea250 0x560f2f8ec660 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8d3690 0x560f2f8e58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae72ffd640 0 client.28295 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8ce520 0x560f2f8d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 --2- 172.21.15.7:0/3313740467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2510 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 -- 172.21.15.7:0/3313740467 >> 172.21.15.7:0/3313740467 conn(0x560f2f8c6e90 msgr2=0x560f2f8c6020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 -- 172.21.15.7:0/3313740467 shutdown_connections 2024-03-26T08:54:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.269+0000 7fae9c966740 1 -- 172.21.15.7:0/3313740467 wait complete. 2024-03-26T08:54:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.270+0000 7fae9c966740 20 client.28295 trim_cache size 0 max 16384 2024-03-26T08:54:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae9c966740 1 Processor -- start 2024-03-26T08:54:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae9c966740 1 -- start start 2024-03-26T08:54:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae9c966740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae9c966740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae9c966740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae9c966740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae941a8f20 con 0x560f2f8d3690 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae9c966740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae941a6a20 con 0x560f2f8cdb30 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae9c966740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae941a4aa0 con 0x560f2f8ce520 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae93fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae93fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38282/0 (socket says 172.21.15.7:38282) 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.287+0000 7fae93fff640 1 -- 172.21.15.7:0/415349745 learned_addr learned my addr 172.21.15.7:0/415349745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae92ffd640 1 --2- 172.21.15.7:0/415349745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae937fe640 1 --2- 172.21.15.7:0/415349745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae93fff640 1 -- 172.21.15.7:0/415349745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 msgr2=0x560f2f8e57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae93fff640 1 --2- 172.21.15.7:0/415349745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8e57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae93fff640 1 -- 172.21.15.7:0/415349745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 msgr2=0x560f2f8d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae93fff640 1 --2- 172.21.15.7:0/415349745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae93fff640 1 -- 172.21.15.7:0/415349745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f2f8c7c00 con 0x560f2f8d3690 2024-03-26T08:54:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae93fff640 1 --2- 172.21.15.7:0/415349745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8c80e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009be0 tx=0x7fae8c033f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6878892f610ca9a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae897fa640 1 -- 172.21.15.7:0/415349745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c004100 con 0x560f2f8d3690 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae897fa640 1 -- 172.21.15.7:0/415349745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae8c047040 con 0x560f2f8d3690 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.288+0000 7fae897fa640 1 -- 172.21.15.7:0/415349745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c034c80 con 0x560f2f8d3690 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 -- 172.21.15.7:0/415349745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 msgr2=0x560f2f8c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 --2- 172.21.15.7:0/415349745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8c80e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009be0 tx=0x7fae8c033f40 comp rx=0 tx=0).stop 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 -- 172.21.15.7:0/415349745 shutdown_connections 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 --2- 172.21.15.7:0/415349745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8c80e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 --2- 172.21.15.7:0/415349745 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8e57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 --2- 172.21.15.7:0/415349745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 -- 172.21.15.7:0/415349745 >> 172.21.15.7:0/415349745 conn(0x560f2f8c6e90 msgr2=0x560f2f8d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 -- 172.21.15.7:0/415349745 shutdown_connections 2024-03-26T08:54:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 -- 172.21.15.7:0/415349745 wait complete. 2024-03-26T08:54:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.289+0000 7fae9c966740 1 -- start start 2024-03-26T08:54:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.290+0000 7fae9c966740 1 Processor -- start 2024-03-26T08:54:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.290+0000 7fae9c966740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae9c966740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae9c966740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8e35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34342/0 (socket says 172.21.15.7:34342) 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae937fe640 1 -- 172.21.15.7:0/3446953152 learned_addr learned my addr 172.21.15.7:0/3446953152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae941a50a0 con 0x560f2f8d3690 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae941a63c0 con 0x560f2f8cdb30 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae941a5700 con 0x560f2f8ce520 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae92ffd640 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae9c966740 10 client.0 _ll_register_callbacks cb 0x7fae7c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae93fff640 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae937fe640 1 -- 172.21.15.7:0/3446953152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 msgr2=0x560f2f8e3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae937fe640 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8e3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae937fe640 1 -- 172.21.15.7:0/3446953152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 msgr2=0x560f2f8e4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae937fe640 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8e4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae937fe640 1 -- 172.21.15.7:0/3446953152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f2f8e50e0 con 0x560f2f8cdb30 2024-03-26T08:54:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.291+0000 7fae92ffd640 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8e3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae937fe640 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8e35f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fae7c09b5f0 tx=0x7fae7c003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=498eac52c56d5965 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae72ffd640 1 -- 172.21.15.7:0/3446953152 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0860b0 con 0x560f2f8cdb30 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae8c047040 con 0x560f2f8cdb30 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae72ffd640 1 -- 172.21.15.7:0/3446953152 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae7c0a3970 con 0x560f2f8cdb30 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae72ffd640 1 -- 172.21.15.7:0/3446953152 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae7c097900 con 0x560f2f8cdb30 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae9c966740 10 client.28301 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560f2f9461a0 con 0x560f2f8cdb30 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae72ffd640 1 -- 172.21.15.7:0/3446953152 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae8c047040 con 0x560f2f8cdb30 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae72ffd640 1 -- 172.21.15.7:0/3446953152 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fae7c0b4220 con 0x560f2f8cdb30 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae9c966740 10 client.28301 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae9c966740 10 client.28301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:10.294 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae9c966740 20 client.28301 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae9c966740 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560f2f8ea250 0x560f2f9488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae9c966740 4 client.28301 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:54:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.292+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560f2f83e050 con 0x560f2f8ea250 2024-03-26T08:54:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.293+0000 7fae92ffd640 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560f2f8ea250 0x560f2f9488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.293+0000 7fae92ffd640 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560f2f8ea250 0x560f2f9488f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6b2a6aebaf2842e server_cookie=f89d6122e636ce45 in_seq=0 out_seq=0 2024-03-26T08:54:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.293+0000 7fae72ffd640 10 client.28301 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.408+0000 7fae72ffd640 1 -- 172.21.15.7:0/3446953152 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560f2f83e050 con 0x560f2f8ea250 2024-03-26T08:54:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.408+0000 7fae72ffd640 10 client.28301 handle_command_reply: tid=0 2024-03-26T08:54:10.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.408+0000 7fae9c966740 1 client.28301 shutdown 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 msgr2=0x560f2f8e35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8e35f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fae7c09b5f0 tx=0x7fae7c003d40 comp rx=0 tx=0).stop 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 shutdown_connections 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560f2f8ea250 0x560f2f9488f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f2f8d3690 0x560f2f8e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae72ffd640 0 client.28301 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560f2f8ce520 0x560f2f8e3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 --2- 172.21.15.7:0/3446953152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f2f8cdb30 0x560f2f8e35f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 >> 172.21.15.7:0/3446953152 conn(0x560f2f8c6e90 msgr2=0x560f2f8c5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 shutdown_connections 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 1 -- 172.21.15.7:0/3446953152 wait complete. 2024-03-26T08:54:10.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.409+0000 7fae9c966740 20 client.28301 trim_cache size 0 max 16384 2024-03-26T08:54:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:54:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:54:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:54:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:54:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 -- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae800a2870 msgr2=0x7fae800a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae800a2870 0x7fae800a4d30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fae8c004ab0 tx=0x7fae8c02bef0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 -- 172.21.15.7:0/1177096864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae941005d0 msgr2=0x7fae94196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae941005d0 0x7fae94196880 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e9f0 tx=0x7fae7c00eeb0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 -- 172.21.15.7:0/1177096864 shutdown_connections 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fae800a2870 0x7fae800a4d30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae94102180 0x7fae9419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae941017c0 0x7fae94196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 --2- 172.21.15.7:0/1177096864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae941005d0 0x7fae94196880 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.410+0000 7fae99c9a640 1 -- 172.21.15.7:0/1177096864 >> 172.21.15.7:0/1177096864 conn(0x7fae940fbb40 msgr2=0x7fae94104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.411+0000 7fae99c9a640 1 -- 172.21.15.7:0/1177096864 shutdown_connections 2024-03-26T08:54:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:10.411+0000 7fae99c9a640 1 -- 172.21.15.7:0/1177096864 wait complete. 2024-03-26T08:54:10.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:10 smithi016 ceph-mon[35851]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-03-26T08:54:10.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:10 smithi007 ceph-mon[32462]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-03-26T08:54:10.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:10 smithi044 ceph-mon[36461]: pgmap v712: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 609 op/s 2024-03-26T08:54:11.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 700.0 s, 1298.0 tps, lat 77.150 ms stddev 92.849 2024-03-26T08:54:12.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:12 smithi016 ceph-mon[35851]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.5 MiB/s wr, 691 op/s 2024-03-26T08:54:12.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:12 smithi007 ceph-mon[32462]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.5 MiB/s wr, 691 op/s 2024-03-26T08:54:12.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:12 smithi044 ceph-mon[36461]: pgmap v713: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.5 MiB/s wr, 691 op/s 2024-03-26T08:54:14.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:14 smithi016 ceph-mon[35851]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 748 op/s 2024-03-26T08:54:14.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:14 smithi007 ceph-mon[32462]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 748 op/s 2024-03-26T08:54:14.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:14 smithi044 ceph-mon[36461]: pgmap v714: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 748 op/s 2024-03-26T08:54:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 705.0 s, 1518.6 tps, lat 64.621 ms stddev 74.178 2024-03-26T08:54:16.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:16 smithi016 ceph-mon[35851]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 691 op/s 2024-03-26T08:54:16.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:16 smithi007 ceph-mon[32462]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 691 op/s 2024-03-26T08:54:16.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:16 smithi044 ceph-mon[36461]: pgmap v715: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 11 MiB/s wr, 691 op/s 2024-03-26T08:54:18.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:54:18.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:54:18.492 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:18 smithi007 ceph-mon[32462]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 665 op/s 2024-03-26T08:54:18.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:18 smithi016 ceph-mon[35851]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 665 op/s 2024-03-26T08:54:18.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:18 smithi044 ceph-mon[36461]: pgmap v716: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 13 MiB/s wr, 665 op/s 2024-03-26T08:54:18.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.610+0000 7f6a2d962640 1 -- 172.21.15.7:0/3484151795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a280756e0 msgr2=0x7f6a28073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.610+0000 7f6a2d962640 1 --2- 172.21.15.7:0/3484151795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a280756e0 0x7f6a28073b30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009cd0 tx=0x7f6a2002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.610+0000 7f6a2d962640 1 -- 172.21.15.7:0/3484151795 shutdown_connections 2024-03-26T08:54:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.610+0000 7f6a2d962640 1 --2- 172.21.15.7:0/3484151795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a2810dc80 0x7f6a28110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.610+0000 7f6a2d962640 1 --2- 172.21.15.7:0/3484151795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a28074100 0x7f6a28074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.610+0000 7f6a2d962640 1 --2- 172.21.15.7:0/3484151795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a280756e0 0x7f6a28073b30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.610+0000 7f6a2d962640 1 -- 172.21.15.7:0/3484151795 >> 172.21.15.7:0/3484151795 conn(0x7f6a280fcc20 msgr2=0x7f6a280ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.610+0000 7f6a2d962640 1 -- 172.21.15.7:0/3484151795 shutdown_connections 2024-03-26T08:54:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.610+0000 7f6a2d962640 1 -- 172.21.15.7:0/3484151795 wait complete. 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.611+0000 7f6a2d962640 1 Processor -- start 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.611+0000 7f6a2d962640 1 -- start start 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.611+0000 7f6a2d962640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a28074100 0x7f6a2819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.611+0000 7f6a2d962640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280756e0 0x7f6a2819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.611+0000 7f6a2d962640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2810dc80 0x7f6a281a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.611+0000 7f6a2d962640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28113920 con 0x7f6a2810dc80 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.611+0000 7f6a2d962640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a281137a0 con 0x7f6a280756e0 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.611+0000 7f6a2d962640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28113aa0 con 0x7f6a28074100 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2810dc80 0x7f6a281a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280756e0 0x7f6a2819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2810dc80 0x7f6a281a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57130/0 (socket says 172.21.15.7:57130) 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a26ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a28074100 0x7f6a2819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a267fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280756e0 0x7f6a2819e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60548/0 (socket says 172.21.15.7:60548) 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a277fe640 1 -- 172.21.15.7:0/699477982 learned_addr learned my addr 172.21.15.7:0/699477982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a277fe640 1 -- 172.21.15.7:0/699477982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a28074100 msgr2=0x7f6a2819e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a277fe640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a28074100 0x7f6a2819e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a277fe640 1 -- 172.21.15.7:0/699477982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280756e0 msgr2=0x7f6a2819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a277fe640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280756e0 0x7f6a2819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a277fe640 1 -- 172.21.15.7:0/699477982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a281a5c70 con 0x7f6a2810dc80 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a267fc640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280756e0 0x7f6a2819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a28074100 0x7f6a2819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a277fe640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2810dc80 0x7f6a281a5600 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800e970 tx=0x7f6a1800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a2c960640 1 -- 172.21.15.7:0/699477982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1800ccc0 con 0x7f6a2810dc80 2024-03-26T08:54:18.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a2d962640 1 -- 172.21.15.7:0/699477982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a281a5f60 con 0x7f6a2810dc80 2024-03-26T08:54:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.612+0000 7f6a2d962640 1 -- 172.21.15.7:0/699477982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a281a6420 con 0x7f6a2810dc80 2024-03-26T08:54:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.613+0000 7f6a2c960640 1 -- 172.21.15.7:0/699477982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1800ce60 con 0x7f6a2810dc80 2024-03-26T08:54:18.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.613+0000 7f6a2c960640 1 -- 172.21.15.7:0/699477982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18010490 con 0x7f6a2810dc80 2024-03-26T08:54:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.616+0000 7f6a3062e740 1 Processor -- start 2024-03-26T08:54:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a2c960640 1 -- 172.21.15.7:0/699477982 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f6a180046c0 con 0x7f6a2810dc80 2024-03-26T08:54:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a3062e740 1 -- start start 2024-03-26T08:54:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6953da0 0x5578f69541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6954790 0x5578f6959290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 0x5578f695bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a3062e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f68b93d0 con 0x5578f6954790 2024-03-26T08:54:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a3062e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f6861ad0 con 0x5578f6959900 2024-03-26T08:54:18.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a3062e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f68b5470 con 0x5578f6953da0 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a26ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6954790 0x5578f6959290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a267fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6953da0 0x5578f69541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a26ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6954790 0x5578f6959290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57136/0 (socket says 172.21.15.7:57136) 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a26ffd640 1 -- 172.21.15.7:0/3794445705 learned_addr learned my addr 172.21.15.7:0/3794445705 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a267fc640 1 --2- 172.21.15.7:0/3794445705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 0x5578f695bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a2c960640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a100a2880 0x7f6a100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a26ffd640 1 -- 172.21.15.7:0/3794445705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6953da0 msgr2=0x5578f69541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/3794445705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6953da0 0x5578f69541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a26ffd640 1 -- 172.21.15.7:0/3794445705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 msgr2=0x5578f695bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/3794445705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 0x5578f695bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a26ffd640 1 -- 172.21.15.7:0/3794445705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578f66977d0 con 0x5578f6954790 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a267fc640 1 --2- 172.21.15.7:0/3794445705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6953da0 0x5578f69541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.617+0000 7f6a277fe640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a100a2880 0x7f6a100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.618+0000 7f6a2c960640 1 -- 172.21.15.7:0/699477982 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6a180ced20 con 0x7f6a2810dc80 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.618+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/3794445705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6954790 0x5578f6959290 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009ca0 tx=0x7f6a20005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95a678d3722d741 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:18.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.618+0000 7f6a0dffb640 1 -- 172.21.15.7:0/3794445705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20043070 con 0x5578f6954790 2024-03-26T08:54:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.618+0000 7f6a0dffb640 1 -- 172.21.15.7:0/3794445705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a20032c90 con 0x5578f6954790 2024-03-26T08:54:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.618+0000 7f6a0dffb640 1 -- 172.21.15.7:0/3794445705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20039040 con 0x5578f6954790 2024-03-26T08:54:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a277fe640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a100a2880 0x7f6a100a4d40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6a180a3930 tx=0x7f6a18086040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 -- 172.21.15.7:0/3794445705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6954790 msgr2=0x5578f6959290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 --2- 172.21.15.7:0/3794445705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6954790 0x5578f6959290 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009ca0 tx=0x7f6a20005ef0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 -- 172.21.15.7:0/3794445705 shutdown_connections 2024-03-26T08:54:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 --2- 172.21.15.7:0/3794445705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 0x5578f695bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 --2- 172.21.15.7:0/3794445705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6954790 0x5578f6959290 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 --2- 172.21.15.7:0/3794445705 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6953da0 0x5578f69541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 -- 172.21.15.7:0/3794445705 >> 172.21.15.7:0/3794445705 conn(0x5578f694f310 msgr2=0x5578f694f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 -- 172.21.15.7:0/3794445705 shutdown_connections 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 -- 172.21.15.7:0/3794445705 wait complete. 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.619+0000 7f6a3062e740 1 -- start start 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a3062e740 1 Processor -- start 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f695e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 0x5578f6965740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a3062e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f695c850 con 0x5578f6953da0 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a3062e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f6875060 con 0x5578f6954790 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a3062e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578f695c9d0 con 0x5578f6959900 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a3062e740 10 client.0 _ll_register_callbacks cb 0x5578f68576a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a26ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f695e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57142/0 (socket says 172.21.15.7:57142) 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a267fc640 1 -- 172.21.15.7:0/2456732397 learned_addr learned my addr 172.21.15.7:0/2456732397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a267fc640 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 0x5578f6965740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a26ffd640 1 -- 172.21.15.7:0/2456732397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 msgr2=0x5578f6965740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 0x5578f6965740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a26ffd640 1 -- 172.21.15.7:0/2456732397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 msgr2=0x5578f695e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a26ffd640 1 -- 172.21.15.7:0/2456732397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578f65cd3f0 con 0x5578f6954790 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.620+0000 7f6a267fc640 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.621+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f695e700 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5578f6950280 tx=0x7f6a20005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=715af1304db7460a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.621+0000 7f6a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.621+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2456732397 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2004c070 con 0x5578f6954790 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.621+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2456732397 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6a20032890 con 0x5578f6954790 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.621+0000 7f6a3062e740 1 -- 172.21.15.7:0/2456732397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a20032c90 con 0x5578f6954790 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.621+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2456732397 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a20057050 con 0x5578f6954790 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.621+0000 7f6a3062e740 10 client.28313 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:18.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.621+0000 7f6a3062e740 1 -- 172.21.15.7:0/2456732397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5578f658d0f0 con 0x5578f6954790 2024-03-26T08:54:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.621+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2456732397 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a20032c90 con 0x5578f6954790 2024-03-26T08:54:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.622+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2456732397 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6a200043e0 con 0x5578f6954790 2024-03-26T08:54:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.622+0000 7f6a3062e740 10 client.28313 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.622+0000 7f6a3062e740 10 client.28313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.622+0000 7f6a3062e740 20 client.28313 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.622+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5578f69698a0 0x5578f696bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.622+0000 7f6a3062e740 4 client.28313 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.622+0000 7f6a3062e740 1 -- 172.21.15.7:0/2456732397 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578f68c6910 con 0x5578f69698a0 2024-03-26T08:54:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.622+0000 7f6a267fc640 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5578f69698a0 0x5578f696bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.623+0000 7f6a06ffd640 10 client.28313 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:18.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.623+0000 7f6a267fc640 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5578f69698a0 0x5578f696bc90 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc338f25492b662c server_cookie=5a7cdaeb8c1f2dfd in_seq=0 out_seq=0 2024-03-26T08:54:18.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.624+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2456732397 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5578f68c6910 con 0x5578f69698a0 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.624+0000 7f6a06ffd640 10 client.28313 handle_command_reply: tid=0 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.624+0000 7f6a3062e740 1 client.28313 shutdown 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.625+0000 7f6a3062e740 1 -- 172.21.15.7:0/2456732397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 msgr2=0x5578f695e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.625+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f695e700 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x5578f6950280 tx=0x7f6a20005bd0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a3062e740 1 -- 172.21.15.7:0/2456732397 shutdown_connections 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5578f69698a0 0x5578f696bc90 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 0x5578f6965740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f695e700 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a06ffd640 0 client.28313 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2456732397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a3062e740 1 -- 172.21.15.7:0/2456732397 >> 172.21.15.7:0/2456732397 conn(0x5578f694f310 msgr2=0x5578f694e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a3062e740 1 -- 172.21.15.7:0/2456732397 shutdown_connections 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a3062e740 1 -- 172.21.15.7:0/2456732397 wait complete. 2024-03-26T08:54:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.626+0000 7f6a3062e740 20 client.28313 trim_cache size 0 max 16384 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a3062e740 1 Processor -- start 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a3062e740 1 -- start start 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f6965700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 0x5578f695e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a3062e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14010820 con 0x5578f6953da0 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a3062e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1400e320 con 0x5578f6954790 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a3062e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1400c3a0 con 0x5578f6959900 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57148/0 (socket says 172.21.15.7:57148) 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a267fc640 1 -- 172.21.15.7:0/30200381 learned_addr learned my addr 172.21.15.7:0/30200381 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f6965700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.649+0000 7f6a267fc640 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 0x5578f695e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a267fc640 1 -- 172.21.15.7:0/30200381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 msgr2=0x5578f695e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a267fc640 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 0x5578f695e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a267fc640 1 -- 172.21.15.7:0/30200381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 msgr2=0x5578f6965700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a267fc640 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f6965700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a267fc640 1 -- 172.21.15.7:0/30200381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578f6950080 con 0x5578f6953da0 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f6965700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a267fc640 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c001eb0 tx=0x7f6a1c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508cbdbf4dcc869 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a0d7fa640 1 -- 172.21.15.7:0/30200381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0054d0 con 0x5578f6953da0 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a0d7fa640 1 -- 172.21.15.7:0/30200381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c025c90 con 0x5578f6953da0 2024-03-26T08:54:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.650+0000 7f6a0d7fa640 1 -- 172.21.15.7:0/30200381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c004a00 con 0x5578f6953da0 2024-03-26T08:54:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 -- 172.21.15.7:0/30200381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 msgr2=0x5578f695e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c001eb0 tx=0x7f6a1c005f90 comp rx=0 tx=0).stop 2024-03-26T08:54:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 -- 172.21.15.7:0/30200381 shutdown_connections 2024-03-26T08:54:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6959900 0x5578f695e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6954790 0x5578f6965700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 --2- 172.21.15.7:0/30200381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f695e1c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 -- 172.21.15.7:0/30200381 >> 172.21.15.7:0/30200381 conn(0x5578f694f310 msgr2=0x5578f694f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 -- 172.21.15.7:0/30200381 shutdown_connections 2024-03-26T08:54:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 -- 172.21.15.7:0/30200381 wait complete. 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 -- start start 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.651+0000 7f6a3062e740 1 Processor -- start 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f6962ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6954790 0x5578f6963400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a3062e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 0x5578f6964410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a3062e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1400c9a0 con 0x5578f6953da0 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a3062e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1400dcc0 con 0x5578f6959900 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a3062e740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1400d000 con 0x5578f6954790 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a3062e740 10 client.0 _ll_register_callbacks cb 0x7f6a1c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f6962ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f6962ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57154/0 (socket says 172.21.15.7:57154) 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a267fc640 1 -- 172.21.15.7:0/2352404370 learned_addr learned my addr 172.21.15.7:0/2352404370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6954790 0x5578f6963400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.652+0000 7f6a267fc640 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 0x5578f6964410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.654+0000 7f6a26ffd640 1 -- 172.21.15.7:0/2352404370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 msgr2=0x5578f6964410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:18.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.654+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 0x5578f6964410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.654+0000 7f6a26ffd640 1 -- 172.21.15.7:0/2352404370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 msgr2=0x5578f6962ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:18.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.654+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f6962ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.654+0000 7f6a26ffd640 1 -- 172.21.15.7:0/2352404370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578f6964a00 con 0x5578f6954790 2024-03-26T08:54:18.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.654+0000 7f6a26ffd640 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6954790 0x5578f6963400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6a20000f80 tx=0x7f6a200328c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf7ef3e2d1827a3c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:18.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.654+0000 7f6a06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.655+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2352404370 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2004c040 con 0x5578f6954790 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.655+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2352404370 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6a200349f0 con 0x5578f6954790 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.655+0000 7f6a3062e740 1 -- 172.21.15.7:0/2352404370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a1c025c90 con 0x5578f6954790 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.655+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2352404370 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a20039070 con 0x5578f6954790 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.655+0000 7f6a3062e740 10 client.28315 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.655+0000 7f6a3062e740 1 -- 172.21.15.7:0/2352404370 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5578f69c5000 con 0x5578f6954790 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.655+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2352404370 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6a1c025c90 con 0x5578f6954790 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2352404370 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f6a20004280 con 0x5578f6954790 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a3062e740 10 client.28315 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a3062e740 10 client.28315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a3062e740 20 client.28315 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5578f69698a0 0x5578f69c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a3062e740 4 client.28315 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a3062e740 1 -- 172.21.15.7:0/2352404370 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5578f68c0c60 con 0x5578f69698a0 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a267fc640 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5578f69698a0 0x5578f69c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a06ffd640 10 client.28315 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a267fc640 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5578f69698a0 0x5578f69c7930 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2be3ad9c31df9ad server_cookie=ab1322313e09b1d5 in_seq=0 out_seq=0 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a06ffd640 1 -- 172.21.15.7:0/2352404370 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5578f68c0c60 con 0x5578f69698a0 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a06ffd640 10 client.28315 handle_command_reply: tid=0 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.656+0000 7f6a3062e740 1 client.28315 shutdown 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 -- 172.21.15.7:0/2352404370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6954790 msgr2=0x5578f6963400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6954790 0x5578f6963400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6a20000f80 tx=0x7f6a200328c0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 -- 172.21.15.7:0/2352404370 shutdown_connections 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5578f69698a0 0x5578f69c7930 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578f6959900 0x5578f6964410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a06ffd640 0 client.28315 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5578f6954790 0x5578f6963400 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 --2- 172.21.15.7:0/2352404370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578f6953da0 0x5578f6962ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 -- 172.21.15.7:0/2352404370 >> 172.21.15.7:0/2352404370 conn(0x5578f694f310 msgr2=0x5578f695e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 -- 172.21.15.7:0/2352404370 shutdown_connections 2024-03-26T08:54:18.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 1 -- 172.21.15.7:0/2352404370 wait complete. 2024-03-26T08:54:18.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.657+0000 7f6a3062e740 20 client.28315 trim_cache size 0 max 16384 2024-03-26T08:54:18.657 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:54:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:54:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:54:18.657 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 -- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a100a2880 msgr2=0x7f6a100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a100a2880 0x7f6a100a4d40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6a180a3930 tx=0x7f6a18086040 comp rx=0 tx=0).stop 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 -- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2810dc80 msgr2=0x7f6a281a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2810dc80 0x7f6a281a5600 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6a1800e970 tx=0x7f6a1800ed70 comp rx=0 tx=0).stop 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 -- 172.21.15.7:0/699477982 shutdown_connections 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f6a100a2880 0x7f6a100a4d40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a2810dc80 0x7f6a281a5600 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a280756e0 0x7f6a2819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 --2- 172.21.15.7:0/699477982 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6a28074100 0x7f6a2819e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 -- 172.21.15.7:0/699477982 >> 172.21.15.7:0/699477982 conn(0x7f6a280fcc20 msgr2=0x7f6a28101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 -- 172.21.15.7:0/699477982 shutdown_connections 2024-03-26T08:54:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.658+0000 7f6a2d962640 1 -- 172.21.15.7:0/699477982 wait complete. 2024-03-26T08:54:18.668 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:54:18.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:54:18.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.885+0000 7f5832df4640 1 -- 172.21.15.7:0/2703871252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c100900 msgr2=0x7f582c100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.885+0000 7f5832df4640 1 --2- 172.21.15.7:0/2703871252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c100900 0x7f582c100d60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f581c009900 tx=0x7f581c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:54:18.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.885+0000 7f5832df4640 1 -- 172.21.15.7:0/2703871252 shutdown_connections 2024-03-26T08:54:18.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.885+0000 7f5832df4640 1 --2- 172.21.15.7:0/2703871252 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f582c1012a0 0x7f582c101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.885+0000 7f5832df4640 1 --2- 172.21.15.7:0/2703871252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c100900 0x7f582c100d60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.885+0000 7f5832df4640 1 --2- 172.21.15.7:0/2703871252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c0fffa0 0x7f582c1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.885+0000 7f5832df4640 1 -- 172.21.15.7:0/2703871252 >> 172.21.15.7:0/2703871252 conn(0x7f582c0fbb40 msgr2=0x7f582c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.885+0000 7f5832df4640 1 -- 172.21.15.7:0/2703871252 shutdown_connections 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.885+0000 7f5832df4640 1 -- 172.21.15.7:0/2703871252 wait complete. 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5832df4640 1 Processor -- start 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5832df4640 1 -- start start 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5832df4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c0fffa0 0x7f582c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5832df4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f582c100900 0x7f582c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5832df4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c1012a0 0x7f582c1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5832df4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c10f590 con 0x7f582c1012a0 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5832df4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c10f410 con 0x7f582c0fffa0 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5832df4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c10f710 con 0x7f582c100900 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5831391640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c1012a0 0x7f582c1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5831391640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c1012a0 0x7f582c1a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57174/0 (socket says 172.21.15.7:57174) 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5831391640 1 -- 172.21.15.7:0/2115904180 learned_addr learned my addr 172.21.15.7:0/2115904180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5830b90640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c0fffa0 0x7f582c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5823fff640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f582c100900 0x7f582c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5830b90640 1 -- 172.21.15.7:0/2115904180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f582c100900 msgr2=0x7f582c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5830b90640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f582c100900 0x7f582c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5830b90640 1 -- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c1012a0 msgr2=0x7f582c1a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5830b90640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c1012a0 0x7f582c1a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5830b90640 1 -- 172.21.15.7:0/2115904180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c1a1790 con 0x7f582c0fffa0 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.886+0000 7f5831391640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c1012a0 0x7f582c1a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.887+0000 7f5830b90640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c0fffa0 0x7f582c199b70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f582800e9f0 tx=0x7f582800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.887+0000 7f5821ffb640 1 -- 172.21.15.7:0/2115904180 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582800ccc0 con 0x7f582c0fffa0 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.887+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c1a1a80 con 0x7f582c0fffa0 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.887+0000 7f5821ffb640 1 -- 172.21.15.7:0/2115904180 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5828010910 con 0x7f582c0fffa0 2024-03-26T08:54:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.887+0000 7f5821ffb640 1 -- 172.21.15.7:0/2115904180 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828018960 con 0x7f582c0fffa0 2024-03-26T08:54:18.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.887+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c1a1ff0 con 0x7f582c0fffa0 2024-03-26T08:54:18.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.888+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c10ae00 con 0x7f582c0fffa0 2024-03-26T08:54:18.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.889+0000 7f5821ffb640 1 -- 172.21.15.7:0/2115904180 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5828010ac0 con 0x7f582c0fffa0 2024-03-26T08:54:18.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.890+0000 7f5821ffb640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58140a2800 0x7f58140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:18.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.890+0000 7f5821ffb640 1 -- 172.21.15.7:0/2115904180 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f58280cee10 con 0x7f582c0fffa0 2024-03-26T08:54:18.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.891+0000 7f5823fff640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58140a2800 0x7f58140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:18.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.892+0000 7f5823fff640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58140a2800 0x7f58140a4cc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f581c02c240 tx=0x7f581c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:18.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:18.894+0000 7f5821ffb640 1 -- 172.21.15.7:0/2115904180 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f582809b820 con 0x7f582c0fffa0 2024-03-26T08:54:19.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.244+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f582c02d070 con 0x7f582c0fffa0 2024-03-26T08:54:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.245+0000 7f5821ffb640 1 -- 172.21.15.7:0/2115904180 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f582809f8c0 con 0x7f582c0fffa0 2024-03-26T08:54:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:54:19.242 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:54:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:54:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58140a2800 msgr2=0x7f58140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58140a2800 0x7f58140a4cc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f581c02c240 tx=0x7f581c02df90 comp rx=0 tx=0).stop 2024-03-26T08:54:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c0fffa0 msgr2=0x7f582c199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c0fffa0 0x7f582c199b70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f582800e9f0 tx=0x7f582800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 shutdown_connections 2024-03-26T08:54:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f58140a2800 0x7f58140a4cc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f582c1012a0 0x7f582c1a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f582c100900 0x7f582c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 --2- 172.21.15.7:0/2115904180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f582c0fffa0 0x7f582c199b70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 >> 172.21.15.7:0/2115904180 conn(0x7f582c0fbb40 msgr2=0x7f582c0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 shutdown_connections 2024-03-26T08:54:19.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.248+0000 7f5832df4640 1 -- 172.21.15.7:0/2115904180 wait complete. 2024-03-26T08:54:19.261 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:54:19.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:54:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.424+0000 7f8c180b2640 1 -- 172.21.15.7:0/1036513768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c101086a0 msgr2=0x7f8c1010aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.424+0000 7f8c180b2640 1 --2- 172.21.15.7:0/1036513768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c101086a0 0x7f8c1010aad0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ca0 tx=0x7f8c0802c120 comp rx=0 tx=0).stop 2024-03-26T08:54:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.425+0000 7f8c180b2640 1 -- 172.21.15.7:0/1036513768 shutdown_connections 2024-03-26T08:54:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.425+0000 7f8c180b2640 1 --2- 172.21.15.7:0/1036513768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1010d9c0 0x7f8c1010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.425+0000 7f8c180b2640 1 --2- 172.21.15.7:0/1036513768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c1010b010 0x7f8c1010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.425+0000 7f8c180b2640 1 --2- 172.21.15.7:0/1036513768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c101086a0 0x7f8c1010aad0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.425+0000 7f8c180b2640 1 -- 172.21.15.7:0/1036513768 >> 172.21.15.7:0/1036513768 conn(0x7f8c1006e640 msgr2=0x7f8c10070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.425+0000 7f8c180b2640 1 -- 172.21.15.7:0/1036513768 shutdown_connections 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.425+0000 7f8c180b2640 1 -- 172.21.15.7:0/1036513768 wait complete. 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c180b2640 1 Processor -- start 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c180b2640 1 -- start start 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c180b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c101086a0 0x7f8c101a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c180b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1010b010 0x7f8c101a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c180b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c1010d9c0 0x7f8c101ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c180b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1011c180 con 0x7f8c1010b010 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c180b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1011c000 con 0x7f8c101086a0 2024-03-26T08:54:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c180b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1011c300 con 0x7f8c1010d9c0 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c1564d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1010b010 0x7f8c101a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c1664f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c1010d9c0 0x7f8c101ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c1664f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c1010d9c0 0x7f8c101ade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50476/0 (socket says 172.21.15.7:50476) 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c1664f640 1 -- 172.21.15.7:0/4078286482 learned_addr learned my addr 172.21.15.7:0/4078286482 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.426+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c101086a0 0x7f8c101a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.427+0000 7f8c1664f640 1 -- 172.21.15.7:0/4078286482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c101086a0 msgr2=0x7f8c101a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.427+0000 7f8c1664f640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c101086a0 0x7f8c101a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.427+0000 7f8c1664f640 1 -- 172.21.15.7:0/4078286482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1010b010 msgr2=0x7f8c101a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.427+0000 7f8c1664f640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1010b010 0x7f8c101a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.427+0000 7f8c1664f640 1 -- 172.21.15.7:0/4078286482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c101ae4a0 con 0x7f8c1010d9c0 2024-03-26T08:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.427+0000 7f8c1664f640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c1010d9c0 0x7f8c101ade30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000e9b0 tx=0x7f8c0000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.428+0000 7f8c06ffd640 1 -- 172.21.15.7:0/4078286482 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0000cd80 con 0x7f8c1010d9c0 2024-03-26T08:54:19.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.428+0000 7f8c06ffd640 1 -- 172.21.15.7:0/4078286482 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c000035f0 con 0x7f8c1010d9c0 2024-03-26T08:54:19.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.428+0000 7f8c06ffd640 1 -- 172.21.15.7:0/4078286482 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00010670 con 0x7f8c1010d9c0 2024-03-26T08:54:19.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.430+0000 7f8c180b2640 1 -- 172.21.15.7:0/4078286482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c101ae790 con 0x7f8c1010d9c0 2024-03-26T08:54:19.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.430+0000 7f8c180b2640 1 -- 172.21.15.7:0/4078286482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c101aece0 con 0x7f8c1010d9c0 2024-03-26T08:54:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.434+0000 7f8c06ffd640 1 -- 172.21.15.7:0/4078286482 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8c000037a0 con 0x7f8c1010d9c0 2024-03-26T08:54:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.434+0000 7f8c1ad79740 1 Processor -- start 2024-03-26T08:54:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.435+0000 7f8c1ad79740 1 -- start start 2024-03-26T08:54:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.435+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.435+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656e080 0x558686572b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.435+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586865731f0 0x558686575680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.435+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686530610 con 0x55868656d690 2024-03-26T08:54:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.435+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586864d33c0 con 0x5586865731f0 2024-03-26T08:54:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.435+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868647bd60 con 0x55868656e080 2024-03-26T08:54:19.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.435+0000 7f8c06ffd640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8bfc0a29d0 0x7f8bfc0a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.435+0000 7f8c06ffd640 1 -- 172.21.15.7:0/4078286482 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8c000cf060 con 0x7f8c1010d9c0 2024-03-26T08:54:19.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.437+0000 7f8c15e4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656e080 0x558686572b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.437+0000 7f8c1664f640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8bfc0a29d0 0x7f8bfc0a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.437+0000 7f8c15e4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656e080 0x558686572b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50486/0 (socket says 172.21.15.7:50486) 2024-03-26T08:54:19.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.437+0000 7f8c15e4e640 1 -- 172.21.15.7:0/4274330524 learned_addr learned my addr 172.21.15.7:0/4274330524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c1564d640 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586865731f0 0x558686575680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c1564d640 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c15e4e640 1 -- 172.21.15.7:0/4274330524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586865731f0 msgr2=0x558686575680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586865731f0 0x558686575680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c15e4e640 1 -- 172.21.15.7:0/4274330524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 msgr2=0x55868656dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c15e4e640 1 -- 172.21.15.7:0/4274330524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586861e73f0 con 0x55868656e080 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c1564d640 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656e080 0x558686572b80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ca0 tx=0x7f8c080323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92cfd97fcc63c123 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.438+0000 7f8c1664f640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8bfc0a29d0 0x7f8bfc0a4e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000c6a0 tx=0x7f8c00097c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.439+0000 7f8bfbfff640 1 -- 172.21.15.7:0/4274330524 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08043070 con 0x55868656e080 2024-03-26T08:54:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.439+0000 7f8bfbfff640 1 -- 172.21.15.7:0/4274330524 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c08003970 con 0x55868656e080 2024-03-26T08:54:19.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.439+0000 7f8bfbfff640 1 -- 172.21.15.7:0/4274330524 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08039040 con 0x55868656e080 2024-03-26T08:54:19.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.440+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4274330524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656e080 msgr2=0x558686572b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.440+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656e080 0x558686572b80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ca0 tx=0x7f8c080323c0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.440+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4274330524 shutdown_connections 2024-03-26T08:54:19.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.440+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586865731f0 0x558686575680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.440+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656e080 0x558686572b80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.440+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/4274330524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.440+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4274330524 >> 172.21.15.7:0/4274330524 conn(0x558686568c00 msgr2=0x558686569000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.440+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4274330524 shutdown_connections 2024-03-26T08:54:19.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.440+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4274330524 wait complete. 2024-03-26T08:54:19.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1ad79740 1 -- start start 2024-03-26T08:54:19.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1ad79740 1 Processor -- start 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x5586865779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x558686577f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 0x55868657ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55868648f240 con 0x55868656d690 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586864cf460 con 0x55868656e080 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558686576140 con 0x5586865731f0 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1ad79740 10 client.0 _ll_register_callbacks cb 0x55868648ec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c15e4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x558686577f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c1564d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x5586865779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c15e4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x558686577f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60602/0 (socket says 172.21.15.7:60602) 2024-03-26T08:54:19.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.441+0000 7f8c15e4e640 1 -- 172.21.15.7:0/3196033866 learned_addr learned my addr 172.21.15.7:0/3196033866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8c1564d640 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 0x55868657ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8c15e4e640 1 -- 172.21.15.7:0/3196033866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 msgr2=0x55868657ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 0x55868657ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8c15e4e640 1 -- 172.21.15.7:0/3196033866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 msgr2=0x5586865779f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x5586865779f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8c15e4e640 1 -- 172.21.15.7:0/3196033866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586862b17d0 con 0x55868656e080 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8c1564d640 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 0x55868657ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x558686577f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8c08006010 tx=0x7f8c08003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd63dfdd7a7343e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8bf97fa640 1 -- 172.21.15.7:0/3196033866 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0804c030 con 0x55868656e080 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8bf97fa640 1 -- 172.21.15.7:0/3196033866 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8c080329d0 con 0x55868656e080 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.442+0000 7f8c1ad79740 1 -- 172.21.15.7:0/3196033866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c08003970 con 0x55868656e080 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8bf97fa640 1 -- 172.21.15.7:0/3196033866 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c08039070 con 0x55868656e080 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8c1ad79740 10 client.28331 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8c1ad79740 1 -- 172.21.15.7:0/3196033866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558686582250 con 0x55868656e080 2024-03-26T08:54:19.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8bf97fa640 1 -- 172.21.15.7:0/3196033866 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c08003970 con 0x55868656e080 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8bf97fa640 1 -- 172.21.15.7:0/3196033866 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8c080489c0 con 0x55868656e080 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8c1ad79740 10 client.28331 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8c1ad79740 10 client.28331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8c1ad79740 20 client.28331 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558686583aa0 0x558686585f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8c1ad79740 4 client.28331 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.443+0000 7f8c1ad79740 1 -- 172.21.15.7:0/3196033866 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586864e0900 con 0x558686583aa0 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.444+0000 7f8c1564d640 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558686583aa0 0x558686585f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.444+0000 7f8bf97fa640 10 client.28331 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.444+0000 7f8c1564d640 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558686583aa0 0x558686585f40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c62ca34820786694 server_cookie=e3f1a0be891dae84 in_seq=0 out_seq=0 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.445+0000 7f8bf97fa640 1 -- 172.21.15.7:0/3196033866 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5586864e0900 con 0x558686583aa0 2024-03-26T08:54:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.445+0000 7f8bf97fa640 10 client.28331 handle_command_reply: tid=0 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.445+0000 7f8c1ad79740 1 client.28331 shutdown 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 -- 172.21.15.7:0/3196033866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 msgr2=0x558686577f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x558686577f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8c08006010 tx=0x7f8c08003d20 comp rx=0 tx=0).stop 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 -- 172.21.15.7:0/3196033866 shutdown_connections 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x558686583aa0 0x558686585f40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 0x55868657ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x558686577f30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/3196033866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x5586865779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 -- 172.21.15.7:0/3196033866 >> 172.21.15.7:0/3196033866 conn(0x558686568c00 msgr2=0x558686567e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8bf97fa640 0 client.28331 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 -- 172.21.15.7:0/3196033866 shutdown_connections 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 1 -- 172.21.15.7:0/3196033866 wait complete. 2024-03-26T08:54:19.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.446+0000 7f8c1ad79740 20 client.28331 trim_cache size 0 max 16384 2024-03-26T08:54:19.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.463+0000 7f8c1ad79740 1 Processor -- start 2024-03-26T08:54:19.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.463+0000 7f8c1ad79740 1 -- start start 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.463+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868656fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 0x55868657ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c101b9e20 con 0x55868656d690 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c101b7920 con 0x55868656e080 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c101b59a0 con 0x5586865731f0 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c1564d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c15e4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868656fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c15e4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868656fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60608/0 (socket says 172.21.15.7:60608) 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c1564d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57216/0 (socket says 172.21.15.7:57216) 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c15e4e640 1 -- 172.21.15.7:0/4242316371 learned_addr learned my addr 172.21.15.7:0/4242316371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c1564d640 1 --2- 172.21.15.7:0/4242316371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 0x55868657ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c15e4e640 1 -- 172.21.15.7:0/4242316371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 msgr2=0x55868657ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/4242316371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 0x55868657ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c15e4e640 1 -- 172.21.15.7:0/4242316371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 msgr2=0x55868656f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/4242316371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c15e4e640 1 -- 172.21.15.7:0/4242316371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586865865c0 con 0x55868656e080 2024-03-26T08:54:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c1564d640 1 --2- 172.21.15.7:0/4242316371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/4242316371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868656fb00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8c08048e80 tx=0x7f8c08009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4f1b5591dae513b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.464+0000 7f8bfb7fe640 1 -- 172.21.15.7:0/4242316371 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0804c030 con 0x55868656e080 2024-03-26T08:54:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8bfb7fe640 1 -- 172.21.15.7:0/4242316371 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c08003970 con 0x55868656e080 2024-03-26T08:54:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8bfb7fe640 1 -- 172.21.15.7:0/4242316371 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08043040 con 0x55868656e080 2024-03-26T08:54:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4242316371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 msgr2=0x55868656fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/4242316371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868656fb00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8c08048e80 tx=0x7f8c08009540 comp rx=0 tx=0).stop 2024-03-26T08:54:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4242316371 shutdown_connections 2024-03-26T08:54:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/4242316371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5586865731f0 0x55868657ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/4242316371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868656fb00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/4242316371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55868656d690 0x55868656f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4242316371 >> 172.21.15.7:0/4242316371 conn(0x558686568c00 msgr2=0x5586865764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4242316371 shutdown_connections 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.465+0000 7f8c1ad79740 1 -- 172.21.15.7:0/4242316371 wait complete. 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1ad79740 1 -- start start 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1ad79740 1 Processor -- start 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656d690 0x55868657c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868657c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1ad79740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586865731f0 0x55868657d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c101b5fa0 con 0x5586865731f0 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c101b72c0 con 0x55868656e080 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1ad79740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c101b6600 con 0x55868656d690 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1ad79740 10 client.0 _ll_register_callbacks cb 0x7f8c08003b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1564d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586865731f0 0x55868657d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c15e4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868657c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c15e4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868657c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60622/0 (socket says 172.21.15.7:60622) 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c1564d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586865731f0 0x55868657d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57232/0 (socket says 172.21.15.7:57232) 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.466+0000 7f8c15e4e640 1 -- 172.21.15.7:0/586335977 learned_addr learned my addr 172.21.15.7:0/586335977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8c1564d640 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656d690 0x55868657c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8c15e4e640 1 -- 172.21.15.7:0/586335977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656d690 msgr2=0x55868657c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656d690 0x55868657c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8c15e4e640 1 -- 172.21.15.7:0/586335977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586865731f0 msgr2=0x55868657d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586865731f0 0x55868657d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8c15e4e640 1 -- 172.21.15.7:0/586335977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55868657dcf0 con 0x55868656e080 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8c1564d640 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586865731f0 0x55868657d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8c15e4e640 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868657c760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8c08053040 tx=0x7f8c08009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a49b290b7add75dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8bf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8bf97fa640 1 -- 172.21.15.7:0/586335977 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0804c040 con 0x55868656e080 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8c1ad79740 1 -- 172.21.15.7:0/586335977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c08003970 con 0x55868656e080 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.467+0000 7f8bf97fa640 1 -- 172.21.15.7:0/586335977 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8c08048650 con 0x55868656e080 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.468+0000 7f8bf97fa640 1 -- 172.21.15.7:0/586335977 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c08043040 con 0x55868656e080 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.468+0000 7f8c1ad79740 10 client.28337 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.468+0000 7f8c1ad79740 1 -- 172.21.15.7:0/586335977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55868657eff0 con 0x55868656e080 2024-03-26T08:54:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.468+0000 7f8bf97fa640 1 -- 172.21.15.7:0/586335977 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c08003970 con 0x55868656e080 2024-03-26T08:54:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.469+0000 7f8bf97fa640 1 -- 172.21.15.7:0/586335977 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8c0803f540 con 0x55868656e080 2024-03-26T08:54:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.469+0000 7f8c1ad79740 10 client.28337 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.469+0000 7f8c1ad79740 10 client.28337 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:54:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.469+0000 7f8c1ad79740 20 client.28337 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.469+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5586865df750 0x5586865e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.469+0000 7f8c1ad79740 4 client.28337 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:54:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.469+0000 7f8c1ad79740 1 -- 172.21.15.7:0/586335977 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5586864dac50 con 0x5586865df750 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.470+0000 7f8c1564d640 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5586865df750 0x5586865e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.470+0000 7f8c1564d640 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5586865df750 0x5586865e1b70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8270f82803a684e8 server_cookie=682d45f847e5d4ca in_seq=0 out_seq=0 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.470+0000 7f8bf97fa640 10 client.28337 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.470+0000 7f8bf97fa640 1 -- 172.21.15.7:0/586335977 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5586864dac50 con 0x5586865df750 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.470+0000 7f8bf97fa640 10 client.28337 handle_command_reply: tid=0 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.470+0000 7f8c1ad79740 1 client.28337 shutdown 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 -- 172.21.15.7:0/586335977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 msgr2=0x55868657c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868657c760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8c08053040 tx=0x7f8c08009a10 comp rx=0 tx=0).stop 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 -- 172.21.15.7:0/586335977 shutdown_connections 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5586865df750 0x5586865e1b70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5586865731f0 0x55868657d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55868656e080 0x55868657c760 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 --2- 172.21.15.7:0/586335977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55868656d690 0x55868657c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8bf97fa640 0 client.28337 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:54:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 -- 172.21.15.7:0/586335977 >> 172.21.15.7:0/586335977 conn(0x558686568c00 msgr2=0x5586865764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 -- 172.21.15.7:0/586335977 shutdown_connections 2024-03-26T08:54:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 1 -- 172.21.15.7:0/586335977 wait complete. 2024-03-26T08:54:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.471+0000 7f8c1ad79740 20 client.28337 trim_cache size 0 max 16384 2024-03-26T08:54:19.472 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 -- 172.21.15.7:0/4078286482 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8bfc0a29d0 msgr2=0x7f8bfc0a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8bfc0a29d0 0x7f8bfc0a4e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000c6a0 tx=0x7f8c00097c60 comp rx=0 tx=0).stop 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 -- 172.21.15.7:0/4078286482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c1010d9c0 msgr2=0x7f8c101ade30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c1010d9c0 0x7f8c101ade30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000e9b0 tx=0x7f8c0000ee70 comp rx=0 tx=0).stop 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 -- 172.21.15.7:0/4078286482 shutdown_connections 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8bfc0a29d0 0x7f8bfc0a4e90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c1010d9c0 0x7f8c101ade30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c1010b010 0x7f8c101a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 --2- 172.21.15.7:0/4078286482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c101086a0 0x7f8c101a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 -- 172.21.15.7:0/4078286482 >> 172.21.15.7:0/4078286482 conn(0x7f8c1006e640 msgr2=0x7f8c1010b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 -- 172.21.15.7:0/4078286482 shutdown_connections 2024-03-26T08:54:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.472+0000 7f8c180b2640 1 -- 172.21.15.7:0/4078286482 wait complete. 2024-03-26T08:54:19.481 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:54:19.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:54:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.632+0000 7f5d62e54640 1 -- 172.21.15.7:0/664883749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5c102a10 msgr2=0x7f5d5c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.632+0000 7f5d62e54640 1 --2- 172.21.15.7:0/664883749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5c102a10 0x7f5d5c102e90 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009930 tx=0x7f5d4802bc20 comp rx=0 tx=0).stop 2024-03-26T08:54:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.633+0000 7f5d62e54640 1 -- 172.21.15.7:0/664883749 shutdown_connections 2024-03-26T08:54:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.633+0000 7f5d62e54640 1 --2- 172.21.15.7:0/664883749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c1033d0 0x7f5d5c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.633+0000 7f5d62e54640 1 --2- 172.21.15.7:0/664883749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5c102a10 0x7f5d5c102e90 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.633+0000 7f5d62e54640 1 --2- 172.21.15.7:0/664883749 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c0ff9a0 0x7f5d5c0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.633+0000 7f5d62e54640 1 -- 172.21.15.7:0/664883749 >> 172.21.15.7:0/664883749 conn(0x7f5d5c0fb2b0 msgr2=0x7f5d5c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.633+0000 7f5d62e54640 1 -- 172.21.15.7:0/664883749 shutdown_connections 2024-03-26T08:54:19.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.633+0000 7f5d62e54640 1 -- 172.21.15.7:0/664883749 wait complete. 2024-03-26T08:54:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d62e54640 1 Processor -- start 2024-03-26T08:54:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d62e54640 1 -- start start 2024-03-26T08:54:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d62e54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c0ff9a0 0x7f5d5c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d62e54640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c102a10 0x7f5d5c19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d62e54640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5c1033d0 0x7f5d5c1a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d62e54640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c102120 con 0x7f5d5c1033d0 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d62e54640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c101fa0 con 0x7f5d5c0ff9a0 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d62e54640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1022a0 con 0x7f5d5c102a10 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d613f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5c1033d0 0x7f5d5c1a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d613f1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5c1033d0 0x7f5d5c1a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57250/0 (socket says 172.21.15.7:57250) 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d613f1640 1 -- 172.21.15.7:0/3355480604 learned_addr learned my addr 172.21.15.7:0/3355480604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c102a10 0x7f5d5c19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.634+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c0ff9a0 0x7f5d5c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d5bfff640 1 -- 172.21.15.7:0/3355480604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c0ff9a0 msgr2=0x7f5d5c19de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c0ff9a0 0x7f5d5c19de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d5bfff640 1 -- 172.21.15.7:0/3355480604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5c1033d0 msgr2=0x7f5d5c1a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5c1033d0 0x7f5d5c1a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d5bfff640 1 -- 172.21.15.7:0/3355480604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c1a5a60 con 0x7f5d5c102a10 2024-03-26T08:54:19.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c0ff9a0 0x7f5d5c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:54:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c102a10 0x7f5d5c19e380 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009850 tx=0x7f5d48005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d59ffb640 1 -- 172.21.15.7:0/3355480604 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48041070 con 0x7f5d5c102a10 2024-03-26T08:54:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d59ffb640 1 -- 172.21.15.7:0/3355480604 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d480054e0 con 0x7f5d5c102a10 2024-03-26T08:54:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d59ffb640 1 -- 172.21.15.7:0/3355480604 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48034440 con 0x7f5d5c102a10 2024-03-26T08:54:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d62e54640 1 -- 172.21.15.7:0/3355480604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1a5cf0 con 0x7f5d5c102a10 2024-03-26T08:54:19.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.635+0000 7f5d62e54640 1 -- 172.21.15.7:0/3355480604 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1a6160 con 0x7f5d5c102a10 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.638+0000 7f5d59ffb640 1 -- 172.21.15.7:0/3355480604 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5d48044070 con 0x7f5d5c102a10 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.638+0000 7f5d65a3d740 1 Processor -- start 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.638+0000 7f5d65a3d740 1 -- start start 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 0x55be3e39aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e39b080 0x55be3e39fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e3a01f0 0x55be3e3a2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3e35d610 con 0x55be3e39b080 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3e3003c0 con 0x55be3e3a01f0 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3e2a8d60 con 0x55be3e39a690 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e3a01f0 0x55be3e3a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d5bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e3a01f0 0x55be3e3a2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60652/0 (socket says 172.21.15.7:60652) 2024-03-26T08:54:19.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d5bfff640 1 -- 172.21.15.7:0/3465577795 learned_addr learned my addr 172.21.15.7:0/3465577795 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d613f1640 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 0x55be3e39aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e39b080 0x55be3e39fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d59ffb640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5d4c0a2800 0x7f5d4c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.639+0000 7f5d59ffb640 1 -- 172.21.15.7:0/3355480604 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5d480f0980 con 0x7f5d5c102a10 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d5bfff640 1 -- 172.21.15.7:0/3465577795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 msgr2=0x55be3e39aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 0x55be3e39aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d5bfff640 1 -- 172.21.15.7:0/3465577795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e39b080 msgr2=0x55be3e39fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d613f1640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5d4c0a2800 0x7f5d4c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e39b080 0x55be3e39fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d5bfff640 1 -- 172.21.15.7:0/3465577795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be3e0143f0 con 0x55be3e3a01f0 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e39b080 0x55be3e39fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e3a01f0 0x55be3e3a2680 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5d480bd020 tx=0x7f5d480bd220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afed459b45377319 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d46ffd640 1 -- 172.21.15.7:0/3465577795 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4804b030 con 0x55be3e3a01f0 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d46ffd640 1 -- 172.21.15.7:0/3465577795 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d48005690 con 0x55be3e3a01f0 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.640+0000 7f5d46ffd640 1 -- 172.21.15.7:0/3465577795 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d480c1470 con 0x55be3e3a01f0 2024-03-26T08:54:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.641+0000 7f5d613f1640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5d4c0a2800 0x7f5d4c0a4cc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55be3e39b8e0 tx=0x7f5d54004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.641+0000 7f5d65a3d740 1 -- 172.21.15.7:0/3465577795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e3a01f0 msgr2=0x55be3e3a2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.641+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e3a01f0 0x55be3e3a2680 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5d480bd020 tx=0x7f5d480bd220 comp rx=0 tx=0).stop 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.641+0000 7f5d65a3d740 1 -- 172.21.15.7:0/3465577795 shutdown_connections 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.641+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e3a01f0 0x55be3e3a2680 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.641+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e39b080 0x55be3e39fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.641+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/3465577795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 0x55be3e39aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.642+0000 7f5d65a3d740 1 -- 172.21.15.7:0/3465577795 >> 172.21.15.7:0/3465577795 conn(0x55be3e395c00 msgr2=0x55be3e396000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.642+0000 7f5d65a3d740 1 -- 172.21.15.7:0/3465577795 shutdown_connections 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.642+0000 7f5d65a3d740 1 -- 172.21.15.7:0/3465577795 wait complete. 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.642+0000 7f5d65a3d740 1 -- start start 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.642+0000 7f5d65a3d740 1 Processor -- start 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.642+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.642+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 0x55be3e3a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e3a01f0 0x55be3e3abc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3e2bc240 con 0x55be3e3a01f0 2024-03-26T08:54:19.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3e2fc460 con 0x55be3e39a690 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be3e3003c0 con 0x55be3e39b080 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d65a3d740 10 client.0 _ll_register_callbacks cb 0x55be3e2bbc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d60bf0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d60bf0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60666/0 (socket says 172.21.15.7:60666) 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d60bf0640 1 -- 172.21.15.7:0/104900248 learned_addr learned my addr 172.21.15.7:0/104900248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d613f1640 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e3a01f0 0x55be3e3abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 0x55be3e3a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d60bf0640 1 -- 172.21.15.7:0/104900248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 msgr2=0x55be3e3a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 0x55be3e3a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d60bf0640 1 -- 172.21.15.7:0/104900248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e3a01f0 msgr2=0x55be3e3abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e3a01f0 0x55be3e3abc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d60bf0640 1 -- 172.21.15.7:0/104900248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be3e0de7d0 con 0x55be3e39a690 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a46c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5d50013790 tx=0x7f5d50013c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f2bd16b8ee7b60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.643+0000 7f5d3bfff640 1 -- 172.21.15.7:0/104900248 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50011dd0 con 0x55be3e39a690 2024-03-26T08:54:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d3bfff640 1 -- 172.21.15.7:0/104900248 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5d50004030 con 0x55be3e39a690 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d65a3d740 1 -- 172.21.15.7:0/104900248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be3e30d900 con 0x55be3e39a690 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d3bfff640 1 -- 172.21.15.7:0/104900248 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d50004230 con 0x55be3e39a690 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d65a3d740 10 client.28349 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d65a3d740 1 -- 172.21.15.7:0/104900248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be3e3af020 con 0x55be3e39a690 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d3bfff640 1 -- 172.21.15.7:0/104900248 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55be3e30d900 con 0x55be3e39a690 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d3bfff640 1 -- 172.21.15.7:0/104900248 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5d5001a070 con 0x55be3e39a690 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d65a3d740 10 client.28349 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d65a3d740 10 client.28349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d65a3d740 20 client.28349 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55be3e3b0750 0x55be3e3b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d65a3d740 4 client.28349 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.644+0000 7f5d65a3d740 1 -- 172.21.15.7:0/104900248 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be3e307c50 con 0x55be3e3b0750 2024-03-26T08:54:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.645+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55be3e3b0750 0x55be3e3b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.646+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55be3e3b0750 0x55be3e3b2bd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38ea8aa7f8c0ca3e server_cookie=f6c9892477732154 in_seq=0 out_seq=0 2024-03-26T08:54:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.647+0000 7f5d3bfff640 10 client.28349 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.647+0000 7f5d3bfff640 1 -- 172.21.15.7:0/104900248 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55be3e307c50 con 0x55be3e3b0750 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.647+0000 7f5d3bfff640 10 client.28349 handle_command_reply: tid=0 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.647+0000 7f5d65a3d740 1 client.28349 shutdown 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.647+0000 7f5d65a3d740 1 -- 172.21.15.7:0/104900248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 msgr2=0x55be3e3a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.647+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a46c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5d50013790 tx=0x7f5d50013c50 comp rx=0 tx=0).stop 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d65a3d740 1 -- 172.21.15.7:0/104900248 shutdown_connections 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e3a01f0 0x55be3e3abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55be3e3b0750 0x55be3e3b2bd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 0x55be3e3a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d3bfff640 0 client.28349 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/104900248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a46c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d65a3d740 1 -- 172.21.15.7:0/104900248 >> 172.21.15.7:0/104900248 conn(0x55be3e395c00 msgr2=0x55be3e394e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d65a3d740 1 -- 172.21.15.7:0/104900248 shutdown_connections 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d65a3d740 1 -- 172.21.15.7:0/104900248 wait complete. 2024-03-26T08:54:19.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.648+0000 7f5d65a3d740 20 client.28349 trim_cache size 0 max 16384 2024-03-26T08:54:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.662+0000 7f5d65a3d740 1 Processor -- start 2024-03-26T08:54:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.662+0000 7f5d65a3d740 1 -- start start 2024-03-26T08:54:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 0x55be3e3abb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39b080 0x55be3e3b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400108c0 con 0x55be3e395c00 2024-03-26T08:54:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4000e3c0 con 0x55be3e39b080 2024-03-26T08:54:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4000c440 con 0x55be3e39a690 2024-03-26T08:54:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d60bf0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d5bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 0x55be3e3abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d60bf0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3b74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57270/0 (socket says 172.21.15.7:57270) 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d5bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 0x55be3e3abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50556/0 (socket says 172.21.15.7:50556) 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d60bf0640 1 -- 172.21.15.7:0/1858667936 learned_addr learned my addr 172.21.15.7:0/1858667936 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d613f1640 1 --2- 172.21.15.7:0/1858667936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39b080 0x55be3e3b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d60bf0640 1 -- 172.21.15.7:0/1858667936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 msgr2=0x55be3e3abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/1858667936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 0x55be3e3abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d60bf0640 1 -- 172.21.15.7:0/1858667936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39b080 msgr2=0x55be3e3b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/1858667936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39b080 0x55be3e3b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d60bf0640 1 -- 172.21.15.7:0/1858667936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be3e395a30 con 0x55be3e395c00 2024-03-26T08:54:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.663+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/1858667936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3b74d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009c00 tx=0x7f5d50005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec67e21d7ee39d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.664+0000 7f5d467fc640 1 -- 172.21.15.7:0/1858667936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50011bd0 con 0x55be3e395c00 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.664+0000 7f5d467fc640 1 -- 172.21.15.7:0/1858667936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d50011d70 con 0x55be3e395c00 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.664+0000 7f5d467fc640 1 -- 172.21.15.7:0/1858667936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5001cb80 con 0x55be3e395c00 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.665+0000 7f5d65a3d740 1 -- 172.21.15.7:0/1858667936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 msgr2=0x55be3e3b74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.665+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/1858667936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3b74d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009c00 tx=0x7f5d50005950 comp rx=0 tx=0).stop 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.665+0000 7f5d65a3d740 1 -- 172.21.15.7:0/1858667936 shutdown_connections 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.665+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/1858667936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39b080 0x55be3e3b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.665+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/1858667936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39a690 0x55be3e3abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.665+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/1858667936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3b74d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.665+0000 7f5d65a3d740 1 -- 172.21.15.7:0/1858667936 >> 172.21.15.7:0/1858667936 conn(0x55be3e39be60 msgr2=0x55be3e3996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.665+0000 7f5d65a3d740 1 -- 172.21.15.7:0/1858667936 shutdown_connections 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.665+0000 7f5d65a3d740 1 -- 172.21.15.7:0/1858667936 wait complete. 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d65a3d740 1 -- start start 2024-03-26T08:54:19.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d65a3d740 1 Processor -- start 2024-03-26T08:54:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d65a3d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 0x55be3e3aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4000ca40 con 0x55be3e395c00 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4000dd60 con 0x55be3e39a690 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d65a3d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4000d0a0 con 0x55be3e39b080 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d65a3d740 10 client.0 _ll_register_callbacks cb 0x7f5d50009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d60bf0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d60bf0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3a98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57282/0 (socket says 172.21.15.7:57282) 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d60bf0640 1 -- 172.21.15.7:0/217380036 learned_addr learned my addr 172.21.15.7:0/217380036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d613f1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 0x55be3e3aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.666+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d5bfff640 1 -- 172.21.15.7:0/217380036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 msgr2=0x55be3e3aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 0x55be3e3aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d5bfff640 1 -- 172.21.15.7:0/217380036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 msgr2=0x55be3e3a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d5bfff640 1 -- 172.21.15.7:0/217380036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be3e396970 con 0x55be3e39a690 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d613f1640 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 0x55be3e3aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d5bfff640 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a9e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5d480ef000 tx=0x7f5d480bd180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6f08191d3d5a0fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d3bfff640 1 -- 172.21.15.7:0/217380036 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4804b040 con 0x55be3e39a690 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d3bfff640 1 -- 172.21.15.7:0/217380036 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5d480ee500 con 0x55be3e39a690 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d65a3d740 1 -- 172.21.15.7:0/217380036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d50011d70 con 0x55be3e39a690 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d3bfff640 1 -- 172.21.15.7:0/217380036 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d480c5ac0 con 0x55be3e39a690 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d65a3d740 10 client.28355 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.667+0000 7f5d65a3d740 1 -- 172.21.15.7:0/217380036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be3e40c570 con 0x55be3e39a690 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d3bfff640 1 -- 172.21.15.7:0/217380036 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5d50011d70 con 0x55be3e39a690 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d3bfff640 1 -- 172.21.15.7:0/217380036 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5d480dd0b0 con 0x55be3e39a690 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d65a3d740 10 client.28355 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d65a3d740 10 client.28355 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:54:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d65a3d740 20 client.28355 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55be3e3a01f0 0x55be3e40ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d65a3d740 4 client.28355 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d65a3d740 1 -- 172.21.15.7:0/217380036 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be3e285640 con 0x55be3e3a01f0 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55be3e3a01f0 0x55be3e40ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d60bf0640 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55be3e3a01f0 0x55be3e40ed20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4d8d1b239fec263 server_cookie=a07cc4d93ccea389 in_seq=0 out_seq=0 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.668+0000 7f5d3bfff640 10 client.28355 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d3bfff640 1 -- 172.21.15.7:0/217380036 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be3e285640 con 0x55be3e3a01f0 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d3bfff640 10 client.28355 handle_command_reply: tid=0 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 client.28355 shutdown 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 -- 172.21.15.7:0/217380036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 msgr2=0x55be3e3a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a9e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5d480ef000 tx=0x7f5d480bd180 comp rx=0 tx=0).stop 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 -- 172.21.15.7:0/217380036 shutdown_connections 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55be3e3a01f0 0x55be3e40ed20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55be3e39b080 0x55be3e3aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d3bfff640 0 client.28355 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:54:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55be3e39a690 0x55be3e3a9e30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 --2- 172.21.15.7:0/217380036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be3e395c00 0x55be3e3a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 -- 172.21.15.7:0/217380036 >> 172.21.15.7:0/217380036 conn(0x55be3e39be60 msgr2=0x55be3e394590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 -- 172.21.15.7:0/217380036 shutdown_connections 2024-03-26T08:54:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 1 -- 172.21.15.7:0/217380036 wait complete. 2024-03-26T08:54:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.669+0000 7f5d65a3d740 20 client.28355 trim_cache size 0 max 16384 2024-03-26T08:54:19.670 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 -- 172.21.15.7:0/3355480604 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5d4c0a2800 msgr2=0x7f5d4c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5d4c0a2800 0x7f5d4c0a4cc0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55be3e39b8e0 tx=0x7f5d54004ab0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 -- 172.21.15.7:0/3355480604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c102a10 msgr2=0x7f5d5c19e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c102a10 0x7f5d5c19e380 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009850 tx=0x7f5d48005dd0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 -- 172.21.15.7:0/3355480604 shutdown_connections 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5d4c0a2800 0x7f5d4c0a4cc0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d5c1033d0 0x7f5d5c1a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5d5c102a10 0x7f5d5c19e380 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 --2- 172.21.15.7:0/3355480604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5c0ff9a0 0x7f5d5c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 -- 172.21.15.7:0/3355480604 >> 172.21.15.7:0/3355480604 conn(0x7f5d5c0fb2b0 msgr2=0x7f5d5c10dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 -- 172.21.15.7:0/3355480604 shutdown_connections 2024-03-26T08:54:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.670+0000 7f5d62e54640 1 -- 172.21.15.7:0/3355480604 wait complete. 2024-03-26T08:54:19.683 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:54:19.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:54:19.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.852+0000 7f15d53df640 1 -- 172.21.15.7:0/640158132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d010d9c0 msgr2=0x7f15d010de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.852+0000 7f15d53df640 1 --2- 172.21.15.7:0/640158132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d010d9c0 0x7f15d010de40 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009900 tx=0x7f15c002bc80 comp rx=0 tx=0).stop 2024-03-26T08:54:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.853+0000 7f15d53df640 1 -- 172.21.15.7:0/640158132 shutdown_connections 2024-03-26T08:54:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.853+0000 7f15d53df640 1 --2- 172.21.15.7:0/640158132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d010d9c0 0x7f15d010de40 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.853+0000 7f15d53df640 1 --2- 172.21.15.7:0/640158132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15d010b010 0x7f15d010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.853+0000 7f15d53df640 1 --2- 172.21.15.7:0/640158132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d01086a0 0x7f15d010aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.853+0000 7f15d53df640 1 -- 172.21.15.7:0/640158132 >> 172.21.15.7:0/640158132 conn(0x7f15d006e640 msgr2=0x7f15d0070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.853+0000 7f15d53df640 1 -- 172.21.15.7:0/640158132 shutdown_connections 2024-03-26T08:54:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.853+0000 7f15d53df640 1 -- 172.21.15.7:0/640158132 wait complete. 2024-03-26T08:54:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15d53df640 1 Processor -- start 2024-03-26T08:54:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15d53df640 1 -- start start 2024-03-26T08:54:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15d53df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d01086a0 0x7f15d0114d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15d53df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d010b010 0x7f15d0115290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15d53df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15d010d9c0 0x7f15d01b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15d53df640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d011c1e0 con 0x7f15d01086a0 2024-03-26T08:54:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15d53df640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d011c060 con 0x7f15d010b010 2024-03-26T08:54:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15d53df640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d011c360 con 0x7f15d010d9c0 2024-03-26T08:54:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15ceffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d01086a0 0x7f15d0114d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15ceffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d01086a0 0x7f15d0114d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57296/0 (socket says 172.21.15.7:57296) 2024-03-26T08:54:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.854+0000 7f15ceffd640 1 -- 172.21.15.7:0/2832132887 learned_addr learned my addr 172.21.15.7:0/2832132887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.855+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15d010d9c0 0x7f15d01b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.855+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d010b010 0x7f15d0115290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.855+0000 7f15cf7fe640 1 -- 172.21.15.7:0/2832132887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d010b010 msgr2=0x7f15d0115290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.855+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d010b010 0x7f15d0115290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.855+0000 7f15cf7fe640 1 -- 172.21.15.7:0/2832132887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d01086a0 msgr2=0x7f15d0114d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.855+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d01086a0 0x7f15d0114d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.855+0000 7f15cf7fe640 1 -- 172.21.15.7:0/2832132887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0115970 con 0x7f15d010d9c0 2024-03-26T08:54:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.856+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15d010d9c0 0x7f15d01b68f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f15c002efd0 tx=0x7f15c0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.856+0000 7f15b7fff640 1 -- 172.21.15.7:0/2832132887 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c0041070 con 0x7f15d010d9c0 2024-03-26T08:54:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.856+0000 7f15d53df640 1 -- 172.21.15.7:0/2832132887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d01b6ef0 con 0x7f15d010d9c0 2024-03-26T08:54:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.856+0000 7f15d53df640 1 -- 172.21.15.7:0/2832132887 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d01b73b0 con 0x7f15d010d9c0 2024-03-26T08:54:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.856+0000 7f15b7fff640 1 -- 172.21.15.7:0/2832132887 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c0004ac0 con 0x7f15d010d9c0 2024-03-26T08:54:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.856+0000 7f15b7fff640 1 -- 172.21.15.7:0/2832132887 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c0003b90 con 0x7f15d010d9c0 2024-03-26T08:54:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.860+0000 7f15b7fff640 1 -- 172.21.15.7:0/2832132887 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f15c0044070 con 0x7f15d010d9c0 2024-03-26T08:54:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.861+0000 7f15b7fff640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15b80a2950 0x7f15b80a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.861+0000 7f15b7fff640 1 -- 172.21.15.7:0/2832132887 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f15c00f0c80 con 0x7f15d010d9c0 2024-03-26T08:54:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.861+0000 7f15ceffd640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15b80a2950 0x7f15b80a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.862+0000 7f15d7fc7740 1 Processor -- start 2024-03-26T08:54:19.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.862+0000 7f15ceffd640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15b80a2950 0x7f15b80a4e10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009f50 tx=0x7f15c80095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.862+0000 7f15d7fc7740 1 -- start start 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.862+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf280690 0x55c2bf280ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.862+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf285b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf2861f0 0x55c2bf288680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf243610 con 0x55c2bf2861f0 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf1e63c0 con 0x55c2bf280690 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf18ed60 con 0x55c2bf281080 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15cf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf2861f0 0x55c2bf288680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15cf7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf2861f0 0x55c2bf288680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57304/0 (socket says 172.21.15.7:57304) 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15cf7fe640 1 -- 172.21.15.7:0/385004357 learned_addr learned my addr 172.21.15.7:0/385004357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15cf7fe640 1 -- 172.21.15.7:0/385004357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 msgr2=0x55c2bf285b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/385004357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf285b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15cf7fe640 1 -- 172.21.15.7:0/385004357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf280690 msgr2=0x55c2bf280ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/385004357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf280690 0x55c2bf280ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.863+0000 7f15cf7fe640 1 -- 172.21.15.7:0/385004357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2beefa3f0 con 0x55c2bf2861f0 2024-03-26T08:54:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.864+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/385004357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf2861f0 0x55c2bf288680 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f15c00b99b0 tx=0x7f15c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee0da985cde9e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.865+0000 7f15b57fa640 1 -- 172.21.15.7:0/385004357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c004b070 con 0x55c2bf2861f0 2024-03-26T08:54:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.865+0000 7f15b57fa640 1 -- 172.21.15.7:0/385004357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c00bdce0 con 0x55c2bf2861f0 2024-03-26T08:54:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.865+0000 7f15b57fa640 1 -- 172.21.15.7:0/385004357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c0004c70 con 0x55c2bf2861f0 2024-03-26T08:54:19.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.866+0000 7f15d7fc7740 1 -- 172.21.15.7:0/385004357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf2861f0 msgr2=0x55c2bf288680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.866+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/385004357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf2861f0 0x55c2bf288680 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f15c00b99b0 tx=0x7f15c0005fd0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.866+0000 7f15d7fc7740 1 -- 172.21.15.7:0/385004357 shutdown_connections 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.866+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/385004357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf2861f0 0x55c2bf288680 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.866+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/385004357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf285b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.866+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/385004357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf280690 0x55c2bf280ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.866+0000 7f15d7fc7740 1 -- 172.21.15.7:0/385004357 >> 172.21.15.7:0/385004357 conn(0x55c2bf27bc00 msgr2=0x55c2bf27c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.866+0000 7f15d7fc7740 1 -- 172.21.15.7:0/385004357 shutdown_connections 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.866+0000 7f15d7fc7740 1 -- 172.21.15.7:0/385004357 wait complete. 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.867+0000 7f15d7fc7740 1 -- start start 2024-03-26T08:54:19.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.867+0000 7f15d7fc7740 1 Processor -- start 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.867+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf28a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.867+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.867+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf2861f0 0x55c2bf291cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.867+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf1a2240 con 0x55c2bf280690 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.867+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf1e2460 con 0x55c2bf2861f0 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.867+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2bf288f60 con 0x55c2bf281080 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.867+0000 7f15d7fc7740 10 client.0 _ll_register_callbacks cb 0x55c2bf1a1c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.868+0000 7f15ce7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.868+0000 7f15ce7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50594/0 (socket says 172.21.15.7:50594) 2024-03-26T08:54:19.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.868+0000 7f15ce7fc640 1 -- 172.21.15.7:0/295861653 learned_addr learned my addr 172.21.15.7:0/295861653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.868+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf2861f0 0x55c2bf291cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.868+0000 7f15ceffd640 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf28a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15ce7fc640 1 -- 172.21.15.7:0/295861653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf2861f0 msgr2=0x55c2bf291cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf2861f0 0x55c2bf291cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15ce7fc640 1 -- 172.21.15.7:0/295861653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 msgr2=0x55c2bf28a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf28a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15ce7fc640 1 -- 172.21.15.7:0/295861653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2befc47d0 con 0x55c2bf281080 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf2861f0 0x55c2bf291cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28acb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f15c400e450 tx=0x7f15c400e850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43717bb1111a9ff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15b2ff5640 1 -- 172.21.15.7:0/295861653 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4017070 con 0x55c2bf281080 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15b2ff5640 1 -- 172.21.15.7:0/295861653 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f15c4013680 con 0x55c2bf281080 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.869+0000 7f15d7fc7740 1 -- 172.21.15.7:0/295861653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15c00bdce0 con 0x55c2bf281080 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15b2ff5640 1 -- 172.21.15.7:0/295861653 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15c4023050 con 0x55c2bf281080 2024-03-26T08:54:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15d7fc7740 10 client.28360 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:19.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15d7fc7740 1 -- 172.21.15.7:0/295861653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c2bf295160 con 0x55c2bf281080 2024-03-26T08:54:19.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15b2ff5640 1 -- 172.21.15.7:0/295861653 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c00bdce0 con 0x55c2bf281080 2024-03-26T08:54:19.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15b2ff5640 1 -- 172.21.15.7:0/295861653 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f15c4012910 con 0x55c2bf281080 2024-03-26T08:54:19.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15d7fc7740 10 client.28360 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15d7fc7740 10 client.28360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15d7fc7740 20 client.28360 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c2bf296910 0x55c2bf298d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15d7fc7740 4 client.28360 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.870+0000 7f15d7fc7740 1 -- 172.21.15.7:0/295861653 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2bf1f3900 con 0x55c2bf296910 2024-03-26T08:54:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.871+0000 7f15ceffd640 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c2bf296910 0x55c2bf298d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.871+0000 7f15b2ff5640 10 client.28360 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:19.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.871+0000 7f15ceffd640 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c2bf296910 0x55c2bf298d30 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b97233410b3fdf33 server_cookie=b35a70ef1ff1b73f in_seq=0 out_seq=0 2024-03-26T08:54:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.872+0000 7f15b2ff5640 1 -- 172.21.15.7:0/295861653 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c2bf1f3900 con 0x55c2bf296910 2024-03-26T08:54:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.872+0000 7f15b2ff5640 10 client.28360 handle_command_reply: tid=0 2024-03-26T08:54:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.872+0000 7f15d7fc7740 1 client.28360 shutdown 2024-03-26T08:54:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 -- 172.21.15.7:0/295861653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 msgr2=0x55c2bf28acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28acb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f15c400e450 tx=0x7f15c400e850 comp rx=0 tx=0).stop 2024-03-26T08:54:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 -- 172.21.15.7:0/295861653 shutdown_connections 2024-03-26T08:54:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c2bf296910 0x55c2bf298d30 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf2861f0 0x55c2bf291cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28acb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/295861653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf28a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 -- 172.21.15.7:0/295861653 >> 172.21.15.7:0/295861653 conn(0x55c2bf27bc00 msgr2=0x55c2bf27ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15b2ff5640 0 client.28360 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 -- 172.21.15.7:0/295861653 shutdown_connections 2024-03-26T08:54:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 1 -- 172.21.15.7:0/295861653 wait complete. 2024-03-26T08:54:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.873+0000 7f15d7fc7740 20 client.28360 trim_cache size 0 max 16384 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.898+0000 7f15d7fc7740 1 Processor -- start 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.898+0000 7f15d7fc7740 1 -- start start 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf28a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf291cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf27a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0108c0 con 0x55c2bf280690 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf28a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf28a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60732/0 (socket says 172.21.15.7:60732) 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ceffd640 1 -- 172.21.15.7:0/277045739 learned_addr learned my addr 172.21.15.7:0/277045739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf291cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15d7fc7740 1 -- 172.21.15.7:0/277045739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc00e3c0 con 0x55c2bf27bc00 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15d7fc7740 1 -- 172.21.15.7:0/277045739 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc00c440 con 0x55c2bf281080 2024-03-26T08:54:19.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf27a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ce7fc640 1 -- 172.21.15.7:0/277045739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 msgr2=0x55c2bf27a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf27a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ce7fc640 1 -- 172.21.15.7:0/277045739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 msgr2=0x55c2bf28a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf28a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ce7fc640 1 -- 172.21.15.7:0/277045739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2bf299390 con 0x55c2bf280690 2024-03-26T08:54:19.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ceffd640 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf28a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:19.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.899+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf291cb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f15c4012cb0 tx=0x7f15c40094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f167ffea00088166 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.900+0000 7f15b4ff9640 1 -- 172.21.15.7:0/277045739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4017070 con 0x55c2bf280690 2024-03-26T08:54:19.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.900+0000 7f15b4ff9640 1 -- 172.21.15.7:0/277045739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c40123d0 con 0x55c2bf280690 2024-03-26T08:54:19.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.900+0000 7f15b4ff9640 1 -- 172.21.15.7:0/277045739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4013880 con 0x55c2bf280690 2024-03-26T08:54:19.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.901+0000 7f15d7fc7740 1 -- 172.21.15.7:0/277045739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 msgr2=0x55c2bf291cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.901+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf291cb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f15c4012cb0 tx=0x7f15c40094c0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.902+0000 7f15d7fc7740 1 -- 172.21.15.7:0/277045739 shutdown_connections 2024-03-26T08:54:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.902+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf27a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.902+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf291cb0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.902+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/277045739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf28a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.902+0000 7f15d7fc7740 1 -- 172.21.15.7:0/277045739 >> 172.21.15.7:0/277045739 conn(0x55c2bf2899a0 msgr2=0x55c2bf292740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.902+0000 7f15d7fc7740 1 -- 172.21.15.7:0/277045739 shutdown_connections 2024-03-26T08:54:19.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.902+0000 7f15d7fc7740 1 -- 172.21.15.7:0/277045739 wait complete. 2024-03-26T08:54:19.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15d7fc7740 1 -- start start 2024-03-26T08:54:19.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15d7fc7740 1 Processor -- start 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf291770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf291cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15d7fc7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc00ca40 con 0x55c2bf280690 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc00dd60 con 0x55c2bf27bc00 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15d7fc7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc00d0a0 con 0x55c2bf281080 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf291770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15ceffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf291770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60736/0 (socket says 172.21.15.7:60736) 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15ceffd640 1 -- 172.21.15.7:0/2939143708 learned_addr learned my addr 172.21.15.7:0/2939143708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf291cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.903+0000 7f15d7fc7740 10 client.0 _ll_register_callbacks cb 0x7f15c4009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15cf7fe640 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15ceffd640 1 -- 172.21.15.7:0/2939143708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 msgr2=0x55c2bf28fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15ceffd640 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15ceffd640 1 -- 172.21.15.7:0/2939143708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 msgr2=0x55c2bf291cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15ceffd640 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf291cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15ceffd640 1 -- 172.21.15.7:0/2939143708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2bf27c970 con 0x55c2bf27bc00 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15ceffd640 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf291770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f15c8016cc0 tx=0x7f15c8004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e394e06cf7fecbe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15b2ff5640 1 -- 172.21.15.7:0/2939143708 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8023420 con 0x55c2bf27bc00 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15d7fc7740 1 -- 172.21.15.7:0/2939143708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15c40123d0 con 0x55c2bf27bc00 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15b2ff5640 1 -- 172.21.15.7:0/2939143708 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f15c80235c0 con 0x55c2bf27bc00 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15b2ff5640 1 -- 172.21.15.7:0/2939143708 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15c8034050 con 0x55c2bf27bc00 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15d7fc7740 10 client.28364 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15d7fc7740 1 -- 172.21.15.7:0/2939143708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c2bf2f2c60 con 0x55c2bf27bc00 2024-03-26T08:54:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.904+0000 7f15b2ff5640 1 -- 172.21.15.7:0/2939143708 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c40123d0 con 0x55c2bf27bc00 2024-03-26T08:54:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.905+0000 7f15b2ff5640 1 -- 172.21.15.7:0/2939143708 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f15c8008890 con 0x55c2bf27bc00 2024-03-26T08:54:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.906+0000 7f15d7fc7740 10 client.28364 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.906+0000 7f15d7fc7740 10 client.28364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.906+0000 7f15d7fc7740 20 client.28364 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.906+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c2bf2861f0 0x55c2bf2f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.906+0000 7f15d7fc7740 4 client.28364 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.906+0000 7f15d7fc7740 1 -- 172.21.15.7:0/2939143708 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c2bf1edc50 con 0x55c2bf2861f0 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.906+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c2bf2861f0 0x55c2bf2f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15ce7fc640 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c2bf2861f0 0x55c2bf2f55b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa939650d4feba server_cookie=30c73430094a5b35 in_seq=0 out_seq=0 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15b2ff5640 10 client.28364 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15b2ff5640 1 -- 172.21.15.7:0/2939143708 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c2bf1edc50 con 0x55c2bf2861f0 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15b2ff5640 10 client.28364 handle_command_reply: tid=0 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 client.28364 shutdown 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 -- 172.21.15.7:0/2939143708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 msgr2=0x55c2bf291770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf291770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f15c8016cc0 tx=0x7f15c8004b50 comp rx=0 tx=0).stop 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 -- 172.21.15.7:0/2939143708 shutdown_connections 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c2bf2861f0 0x55c2bf2f55b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c2bf281080 0x55c2bf28fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c2bf280690 0x55c2bf291cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 --2- 172.21.15.7:0/2939143708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c2bf27bc00 0x55c2bf291770 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15b2ff5640 0 client.28364 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:19.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 -- 172.21.15.7:0/2939143708 >> 172.21.15.7:0/2939143708 conn(0x55c2bf2899a0 msgr2=0x55c2bf292740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 -- 172.21.15.7:0/2939143708 shutdown_connections 2024-03-26T08:54:19.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.907+0000 7f15d7fc7740 1 -- 172.21.15.7:0/2939143708 wait complete. 2024-03-26T08:54:19.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.908+0000 7f15d7fc7740 20 client.28364 trim_cache size 0 max 16384 2024-03-26T08:54:19.907 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 -- 172.21.15.7:0/2832132887 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15b80a2950 msgr2=0x7f15b80a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15b80a2950 0x7f15b80a4e10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009f50 tx=0x7f15c80095b0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 -- 172.21.15.7:0/2832132887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15d010d9c0 msgr2=0x7f15d01b68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15d010d9c0 0x7f15d01b68f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f15c002efd0 tx=0x7f15c0004270 comp rx=0 tx=0).stop 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 -- 172.21.15.7:0/2832132887 shutdown_connections 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f15b80a2950 0x7f15b80a4e10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f15d010d9c0 0x7f15d01b68f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15d010b010 0x7f15d0115290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 --2- 172.21.15.7:0/2832132887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d01086a0 0x7f15d0114d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 -- 172.21.15.7:0/2832132887 >> 172.21.15.7:0/2832132887 conn(0x7f15d006e640 msgr2=0x7f15d010a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 -- 172.21.15.7:0/2832132887 shutdown_connections 2024-03-26T08:54:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:19.909+0000 7f15d53df640 1 -- 172.21.15.7:0/2832132887 wait complete. 2024-03-26T08:54:20.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:20 smithi016 ceph-mon[35851]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 776 op/s 2024-03-26T08:54:20.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:20 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2115904180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:54:20.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:20 smithi007 ceph-mon[32462]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 776 op/s 2024-03-26T08:54:20.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:20 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2115904180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:54:20.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:20 smithi044 ceph-mon[36461]: pgmap v717: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 MiB/s wr, 776 op/s 2024-03-26T08:54:20.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:20 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2115904180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:54:20.921 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:54:20.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:54:20.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:54:21.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.070+0000 7f47105c4640 1 -- 172.21.15.7:0/1955061413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4708108910 msgr2=0x7f470810ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.070+0000 7f47105c4640 1 --2- 172.21.15.7:0/1955061413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4708108910 0x7f470810ad00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4704008410 tx=0x7f4704005f90 comp rx=0 tx=0).stop 2024-03-26T08:54:21.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.071+0000 7f47105c4640 1 -- 172.21.15.7:0/1955061413 shutdown_connections 2024-03-26T08:54:21.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.071+0000 7f47105c4640 1 --2- 172.21.15.7:0/1955061413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f470810dd50 0x7f4708110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.071+0000 7f47105c4640 1 --2- 172.21.15.7:0/1955061413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f470810b2d0 0x7f470810d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.071+0000 7f47105c4640 1 --2- 172.21.15.7:0/1955061413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4708108910 0x7f470810ad00 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4704008410 tx=0x7f4704005f90 comp rx=0 tx=0).stop 2024-03-26T08:54:21.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.071+0000 7f47105c4640 1 -- 172.21.15.7:0/1955061413 >> 172.21.15.7:0/1955061413 conn(0x7f470806e640 msgr2=0x7f4708070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.074+0000 7f47105c4640 1 -- 172.21.15.7:0/1955061413 shutdown_connections 2024-03-26T08:54:21.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.074+0000 7f47105c4640 1 -- 172.21.15.7:0/1955061413 wait complete. 2024-03-26T08:54:21.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.075+0000 7f47105c4640 1 Processor -- start 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.075+0000 7f47105c4640 1 -- start start 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.075+0000 7f47105c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f470810b2d0 0x7f470819e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.075+0000 7f47105c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f470810dd50 0x7f470819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.075+0000 7f47105c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47081a37e0 0x7f47081a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f47105c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708113980 con 0x7f470810b2d0 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f47105c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708113800 con 0x7f47081a37e0 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f47105c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4708113b00 con 0x7f470810dd50 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f470e360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f470810b2d0 0x7f470819e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f470e360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f470810b2d0 0x7f470819e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57346/0 (socket says 172.21.15.7:57346) 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f470e360640 1 -- 172.21.15.7:0/649543822 learned_addr learned my addr 172.21.15.7:0/649543822 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f470eb61640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47081a37e0 0x7f47081a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f470e360640 1 -- 172.21.15.7:0/649543822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f470810dd50 msgr2=0x7f470819e780 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:54:21.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f470e360640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f470810dd50 0x7f470819e780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f470e360640 1 -- 172.21.15.7:0/649543822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47081a37e0 msgr2=0x7f47081a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f470e360640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47081a37e0 0x7f47081a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.076+0000 7f470e360640 1 -- 172.21.15.7:0/649543822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47081a6240 con 0x7f470810b2d0 2024-03-26T08:54:21.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.077+0000 7f470e360640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f470810b2d0 0x7f470819e240 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4704000c00 tx=0x7f470400a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.078+0000 7f46ff7fe640 1 -- 172.21.15.7:0/649543822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704032540 con 0x7f470810b2d0 2024-03-26T08:54:21.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.078+0000 7f46ff7fe640 1 -- 172.21.15.7:0/649543822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4704059070 con 0x7f470810b2d0 2024-03-26T08:54:21.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.078+0000 7f46ff7fe640 1 -- 172.21.15.7:0/649543822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704004920 con 0x7f470810b2d0 2024-03-26T08:54:21.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 710.0 s, 1592.6 tps, lat 63.643 ms stddev 72.856 2024-03-26T08:54:21.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.081+0000 7f47105c4640 1 -- 172.21.15.7:0/649543822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47080738d0 con 0x7f470810b2d0 2024-03-26T08:54:21.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.081+0000 7f47105c4640 1 -- 172.21.15.7:0/649543822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4708073dd0 con 0x7f470810b2d0 2024-03-26T08:54:21.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.085+0000 7f47131af740 1 Processor -- start 2024-03-26T08:54:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.086+0000 7f47131af740 1 -- start start 2024-03-26T08:54:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.086+0000 7f47131af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 0x55810ca46590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.086+0000 7f47131af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca4b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.086+0000 7f47131af740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca4e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.086+0000 7f47131af740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c97dc90 con 0x55810ca4bcd0 2024-03-26T08:54:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.086+0000 7f47131af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c9a3070 con 0x55810ca46170 2024-03-26T08:54:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.086+0000 7f47131af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c94bc80 con 0x55810ca46b60 2024-03-26T08:54:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.087+0000 7f470db5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca4e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.087+0000 7f470db5f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca4e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57362/0 (socket says 172.21.15.7:57362) 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.087+0000 7f470db5f640 1 -- 172.21.15.7:0/1490300843 learned_addr learned my addr 172.21.15.7:0/1490300843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.087+0000 7f470db5f640 1 -- 172.21.15.7:0/1490300843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 msgr2=0x55810ca4b660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.087+0000 7f470db5f640 1 --2- 172.21.15.7:0/1490300843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca4b660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.087+0000 7f470db5f640 1 -- 172.21.15.7:0/1490300843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 msgr2=0x55810ca46590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.087+0000 7f470db5f640 1 --2- 172.21.15.7:0/1490300843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 0x55810ca46590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.087+0000 7f470db5f640 1 -- 172.21.15.7:0/1490300843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810c943d10 con 0x55810ca4bcd0 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.088+0000 7f470db5f640 1 --2- 172.21.15.7:0/1490300843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca4e160 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f46f800b3c0 tx=0x7f46f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbc0c13288bf4dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.088+0000 7f46ff7fe640 1 -- 172.21.15.7:0/649543822 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4704004030 con 0x7f470810b2d0 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.088+0000 7f46fcbfe640 1 -- 172.21.15.7:0/1490300843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f800bdd0 con 0x55810ca4bcd0 2024-03-26T08:54:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.088+0000 7f46fcbfe640 1 -- 172.21.15.7:0/1490300843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46f8005500 con 0x55810ca4bcd0 2024-03-26T08:54:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.089+0000 7f46ff7fe640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46f40a2a00 0x7f46f40a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.089+0000 7f46ff7fe640 1 -- 172.21.15.7:0/649543822 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4704111710 con 0x7f470810b2d0 2024-03-26T08:54:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.089+0000 7f46fcbfe640 1 -- 172.21.15.7:0/1490300843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8004da0 con 0x55810ca4bcd0 2024-03-26T08:54:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.089+0000 7f47131af740 1 -- 172.21.15.7:0/1490300843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 msgr2=0x55810ca4e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.089+0000 7f47131af740 1 --2- 172.21.15.7:0/1490300843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca4e160 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f46f800b3c0 tx=0x7f46f8005f90 comp rx=0 tx=0).stop 2024-03-26T08:54:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.090+0000 7f470eb61640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46f40a2a00 0x7f46f40a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.090+0000 7f47131af740 1 -- 172.21.15.7:0/1490300843 shutdown_connections 2024-03-26T08:54:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.090+0000 7f47131af740 1 --2- 172.21.15.7:0/1490300843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca4e160 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.090+0000 7f47131af740 1 --2- 172.21.15.7:0/1490300843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca4b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.090+0000 7f47131af740 1 --2- 172.21.15.7:0/1490300843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 0x55810ca46590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.090+0000 7f47131af740 1 -- 172.21.15.7:0/1490300843 >> 172.21.15.7:0/1490300843 conn(0x55810ca061a0 msgr2=0x55810ca392d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.090+0000 7f47131af740 1 -- 172.21.15.7:0/1490300843 shutdown_connections 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.090+0000 7f47131af740 1 -- 172.21.15.7:0/1490300843 wait complete. 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.090+0000 7f470eb61640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46f40a2a00 0x7f46f40a4ec0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55810ca473c0 tx=0x7f4700007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f47131af740 1 -- start start 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f47131af740 1 Processor -- start 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f47131af740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca3f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f47131af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46b60 0x55810ca3f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f47131af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca4bcd0 0x55810ca44970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f47131af740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c95f0d0 con 0x55810ca46170 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f47131af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c99f110 con 0x55810ca46b60 2024-03-26T08:54:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f47131af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810ca3dab0 con 0x55810ca4bcd0 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f47131af740 10 client.0 _ll_register_callbacks cb 0x55810c99ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f470e360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca3f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f470e360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca3f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57372/0 (socket says 172.21.15.7:57372) 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f470e360640 1 -- 172.21.15.7:0/662423120 learned_addr learned my addr 172.21.15.7:0/662423120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f470eb61640 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca4bcd0 0x55810ca44970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.091+0000 7f470db5f640 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46b60 0x55810ca3f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f470e360640 1 -- 172.21.15.7:0/662423120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca4bcd0 msgr2=0x55810ca44970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f470e360640 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca4bcd0 0x55810ca44970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f470e360640 1 -- 172.21.15.7:0/662423120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46b60 msgr2=0x55810ca3f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f470e360640 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46b60 0x55810ca3f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f470e360640 1 -- 172.21.15.7:0/662423120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55810c77f7e0 con 0x55810ca46170 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f470eb61640 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca4bcd0 0x55810ca44970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f470e360640 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca3f470 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f47040daad0 tx=0x7f47040da490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddd7623f578f2b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f46f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f46f1ffb640 1 -- 172.21.15.7:0/662423120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040e4040 con 0x55810ca46170 2024-03-26T08:54:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f47131af740 1 -- 172.21.15.7:0/662423120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46f8005500 con 0x55810ca46170 2024-03-26T08:54:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.092+0000 7f46f1ffb640 1 -- 172.21.15.7:0/662423120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4704004320 con 0x55810ca46170 2024-03-26T08:54:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.093+0000 7f46f1ffb640 1 -- 172.21.15.7:0/662423120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47040de800 con 0x55810ca46170 2024-03-26T08:54:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f47131af740 10 client.18573 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f47131af740 1 -- 172.21.15.7:0/662423120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55810c7dff70 con 0x55810ca46170 2024-03-26T08:54:21.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f46f1ffb640 1 -- 172.21.15.7:0/662423120 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46f8005500 con 0x55810ca46170 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f46f1ffb640 1 -- 172.21.15.7:0/662423120 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f47040fcb70 con 0x55810ca46170 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f47131af740 10 client.18573 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f47131af740 10 client.18573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f47131af740 20 client.18573 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f47131af740 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55810ca64de0 0x55810ca671d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f47131af740 4 client.18573 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f47131af740 1 -- 172.21.15.7:0/662423120 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55810c9b05b0 con 0x55810ca64de0 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.094+0000 7f470db5f640 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55810ca64de0 0x55810ca671d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.095+0000 7f46f1ffb640 10 client.18573 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.095+0000 7f470db5f640 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55810ca64de0 0x55810ca671d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a7b9aa7c7e0c482 server_cookie=3ce8f58cd0ffea3 in_seq=0 out_seq=0 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f46f1ffb640 1 -- 172.21.15.7:0/662423120 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55810c9b05b0 con 0x55810ca64de0 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f46f1ffb640 10 client.18573 handle_command_reply: tid=0 2024-03-26T08:54:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 client.18573 shutdown 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 -- 172.21.15.7:0/662423120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 msgr2=0x55810ca3f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca3f470 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f47040daad0 tx=0x7f47040da490 comp rx=0 tx=0).stop 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 -- 172.21.15.7:0/662423120 shutdown_connections 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55810ca64de0 0x55810ca671d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca4bcd0 0x55810ca44970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46b60 0x55810ca3f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 --2- 172.21.15.7:0/662423120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca3f470 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 -- 172.21.15.7:0/662423120 >> 172.21.15.7:0/662423120 conn(0x55810ca061a0 msgr2=0x55810ca38530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f46f1ffb640 0 client.18573 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.096+0000 7f47131af740 1 -- 172.21.15.7:0/662423120 shutdown_connections 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.097+0000 7f47131af740 1 -- 172.21.15.7:0/662423120 wait complete. 2024-03-26T08:54:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.097+0000 7f47131af740 20 client.18573 trim_cache size 0 max 16384 2024-03-26T08:54:21.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.124+0000 7f47131af740 1 Processor -- start 2024-03-26T08:54:21.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.124+0000 7f47131af740 1 -- start start 2024-03-26T08:54:21.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.124+0000 7f47131af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 0x55810ca3f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f47131af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca6baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f47131af740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca6bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f47131af740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470807eef0 con 0x55810ca4bcd0 2024-03-26T08:54:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f47131af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470807c9f0 con 0x55810ca46170 2024-03-26T08:54:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f47131af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470807aa70 con 0x55810ca46b60 2024-03-26T08:54:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f470eb61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca6bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f470db5f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca6baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f470eb61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca6bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57386/0 (socket says 172.21.15.7:57386) 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f470eb61640 1 -- 172.21.15.7:0/3614127641 learned_addr learned my addr 172.21.15.7:0/3614127641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.125+0000 7f470e360640 1 --2- 172.21.15.7:0/3614127641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 0x55810ca3f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.126+0000 7f470e360640 1 -- 172.21.15.7:0/3614127641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 msgr2=0x55810ca6baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.126+0000 7f470e360640 1 --2- 172.21.15.7:0/3614127641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca6baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.126+0000 7f470e360640 1 -- 172.21.15.7:0/3614127641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 msgr2=0x55810ca6bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.126+0000 7f470e360640 1 --2- 172.21.15.7:0/3614127641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca6bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.126+0000 7f470e360640 1 -- 172.21.15.7:0/3614127641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810c677170 con 0x55810ca46170 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.126+0000 7f470e360640 1 --2- 172.21.15.7:0/3614127641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 0x55810ca3f790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4704032750 tx=0x7f470400c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75046cc616f28382 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.126+0000 7f46f3fff640 1 -- 172.21.15.7:0/3614127641 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040e4040 con 0x55810ca46170 2024-03-26T08:54:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.126+0000 7f46f3fff640 1 -- 172.21.15.7:0/3614127641 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47040f9060 con 0x55810ca46170 2024-03-26T08:54:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.126+0000 7f46f3fff640 1 -- 172.21.15.7:0/3614127641 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704004440 con 0x55810ca46170 2024-03-26T08:54:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.127+0000 7f470eb61640 1 --2- 172.21.15.7:0/3614127641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca6bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:54:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.127+0000 7f47131af740 1 -- 172.21.15.7:0/3614127641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 msgr2=0x55810ca3f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.127+0000 7f47131af740 1 --2- 172.21.15.7:0/3614127641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 0x55810ca3f790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4704032750 tx=0x7f470400c040 comp rx=0 tx=0).stop 2024-03-26T08:54:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.128+0000 7f47131af740 1 -- 172.21.15.7:0/3614127641 shutdown_connections 2024-03-26T08:54:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.128+0000 7f47131af740 1 --2- 172.21.15.7:0/3614127641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca4bcd0 0x55810ca6bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.128+0000 7f47131af740 1 --2- 172.21.15.7:0/3614127641 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca6baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.128+0000 7f47131af740 1 --2- 172.21.15.7:0/3614127641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca46170 0x55810ca3f790 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.128+0000 7f47131af740 1 -- 172.21.15.7:0/3614127641 >> 172.21.15.7:0/3614127641 conn(0x55810ca061a0 msgr2=0x55810ca3f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.128+0000 7f47131af740 1 -- 172.21.15.7:0/3614127641 shutdown_connections 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.128+0000 7f47131af740 1 -- 172.21.15.7:0/3614127641 wait complete. 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f47131af740 1 -- start start 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f47131af740 1 Processor -- start 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f47131af740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f47131af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f47131af740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca4bcd0 0x55810ca75c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f47131af740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470807b070 con 0x55810ca46170 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f47131af740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470807c390 con 0x55810ca4bcd0 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f47131af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470807b6d0 con 0x55810ca46b60 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f47131af740 10 client.0 _ll_register_callbacks cb 0x7f470402bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f470e360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f470e360640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca770e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57388/0 (socket says 172.21.15.7:57388) 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f470e360640 1 -- 172.21.15.7:0/2310106485 learned_addr learned my addr 172.21.15.7:0/2310106485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f470db5f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.129+0000 7f470eb61640 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca4bcd0 0x55810ca75c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f470e360640 1 -- 172.21.15.7:0/2310106485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 msgr2=0x55810ca756f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f470e360640 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca756f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f470e360640 1 -- 172.21.15.7:0/2310106485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca4bcd0 msgr2=0x55810ca75c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f470e360640 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca4bcd0 0x55810ca75c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f470e360640 1 -- 172.21.15.7:0/2310106485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55810ca762a0 con 0x55810ca46170 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f470db5f640 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f470e360640 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca770e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f47040e62e0 tx=0x7f47040dc610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c469f5a0ff55bd80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f46f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f46f1ffb640 1 -- 172.21.15.7:0/2310106485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040da050 con 0x55810ca46170 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f46f1ffb640 1 -- 172.21.15.7:0/2310106485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f47040dcfc0 con 0x55810ca46170 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f47131af740 1 -- 172.21.15.7:0/2310106485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47040f9060 con 0x55810ca46170 2024-03-26T08:54:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.130+0000 7f46f1ffb640 1 -- 172.21.15.7:0/2310106485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4704004420 con 0x55810ca46170 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f47131af740 10 client.18576 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f47131af740 1 -- 172.21.15.7:0/2310106485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55810ca762a0 con 0x55810ca46170 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f46f1ffb640 1 -- 172.21.15.7:0/2310106485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f47040f9060 con 0x55810ca46170 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f46f1ffb640 1 -- 172.21.15.7:0/2310106485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f47041110d0 con 0x55810ca46170 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f47131af740 10 client.18576 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f47131af740 10 client.18576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f47131af740 20 client.18576 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f47131af740 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55810cac0f10 0x55810cac3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f47131af740 4 client.18576 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f47131af740 1 -- 172.21.15.7:0/2310106485 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55810ca34aa0 con 0x55810cac0f10 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.131+0000 7f470db5f640 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55810cac0f10 0x55810cac3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.132+0000 7f46f1ffb640 10 client.18576 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.132+0000 7f470db5f640 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55810cac0f10 0x55810cac3300 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df870e86397c7c27 server_cookie=1de6b067b0f49663 in_seq=0 out_seq=0 2024-03-26T08:54:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f46f1ffb640 1 -- 172.21.15.7:0/2310106485 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55810ca34aa0 con 0x55810cac0f10 2024-03-26T08:54:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f46f1ffb640 10 client.18576 handle_command_reply: tid=0 2024-03-26T08:54:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f47131af740 1 client.18576 shutdown 2024-03-26T08:54:21.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f47131af740 1 -- 172.21.15.7:0/2310106485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 msgr2=0x55810ca770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f47131af740 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca770e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f47040e62e0 tx=0x7f47040dc610 comp rx=0 tx=0).stop 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f47131af740 1 -- 172.21.15.7:0/2310106485 shutdown_connections 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f47131af740 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55810cac0f10 0x55810cac3300 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f46f1ffb640 0 client.18576 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f47131af740 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810ca4bcd0 0x55810ca75c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f47131af740 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810ca46b60 0x55810ca756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f47131af740 1 --2- 172.21.15.7:0/2310106485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810ca46170 0x55810ca770e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.133+0000 7f47131af740 1 -- 172.21.15.7:0/2310106485 >> 172.21.15.7:0/2310106485 conn(0x55810ca061a0 msgr2=0x55810ca3c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47131af740 1 -- 172.21.15.7:0/2310106485 shutdown_connections 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47131af740 1 -- 172.21.15.7:0/2310106485 wait complete. 2024-03-26T08:54:21.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47131af740 20 client.18576 trim_cache size 0 max 16384 2024-03-26T08:54:21.135 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:54:21.135 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:54:21.135 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "11793a4d-6454-49d6-af16-9a8895484b4d", 2024-03-26T08:54:21.135 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:54:21.135 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 -- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46f40a2a00 msgr2=0x7f46f40a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46f40a2a00 0x7f46f40a4ec0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55810ca473c0 tx=0x7f4700007520 comp rx=0 tx=0).stop 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 -- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f470810b2d0 msgr2=0x7f470819e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f470810b2d0 0x7f470819e240 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4704000c00 tx=0x7f470400a300 comp rx=0 tx=0).stop 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 -- 172.21.15.7:0/649543822 shutdown_connections 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46f40a2a00 0x7f46f40a4ec0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47081a37e0 0x7f47081a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f470810dd50 0x7f470819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 --2- 172.21.15.7:0/649543822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f470810b2d0 0x7f470819e240 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 -- 172.21.15.7:0/649543822 >> 172.21.15.7:0/649543822 conn(0x7f470806e640 msgr2=0x7f4708109ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 -- 172.21.15.7:0/649543822 shutdown_connections 2024-03-26T08:54:21.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.134+0000 7f47105c4640 1 -- 172.21.15.7:0/649543822 wait complete. 2024-03-26T08:54:21.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:54:21.145 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:54:21.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.309+0000 7f2854913640 1 -- 172.21.15.7:0/1394732039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850109000 msgr2=0x7f285010f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.309+0000 7f2854913640 1 --2- 172.21.15.7:0/1394732039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850109000 0x7f285010f9e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2844009900 tx=0x7f284402bc80 comp rx=0 tx=0).stop 2024-03-26T08:54:21.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.314+0000 7f2854913640 1 -- 172.21.15.7:0/1394732039 shutdown_connections 2024-03-26T08:54:21.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.314+0000 7f2854913640 1 --2- 172.21.15.7:0/1394732039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285010ff20 0x7f2850112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.314+0000 7f2854913640 1 --2- 172.21.15.7:0/1394732039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850109000 0x7f285010f9e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.314+0000 7f2854913640 1 --2- 172.21.15.7:0/1394732039 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f28501086a0 0x7f2850108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.314+0000 7f2854913640 1 -- 172.21.15.7:0/1394732039 >> 172.21.15.7:0/1394732039 conn(0x7f285006e640 msgr2=0x7f2850070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.318+0000 7f2854913640 1 -- 172.21.15.7:0/1394732039 shutdown_connections 2024-03-26T08:54:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.321+0000 7f2854913640 1 -- 172.21.15.7:0/1394732039 wait complete. 2024-03-26T08:54:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.321+0000 7f2854913640 1 Processor -- start 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.322+0000 7f2854913640 1 -- start start 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.322+0000 7f2854913640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28501086a0 0x7f285019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.322+0000 7f2854913640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2850109000 0x7f285019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.322+0000 7f2854913640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285010ff20 0x7f28501a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.322+0000 7f2854913640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850117e70 con 0x7f285010ff20 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.322+0000 7f2854913640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850117cf0 con 0x7f28501086a0 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.322+0000 7f2854913640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850117ff0 con 0x7f2850109000 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.322+0000 7f284ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285010ff20 0x7f28501a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.323+0000 7f284ed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285010ff20 0x7f28501a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57406/0 (socket says 172.21.15.7:57406) 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.323+0000 7f284ed9d640 1 -- 172.21.15.7:0/4136706427 learned_addr learned my addr 172.21.15.7:0/4136706427 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.323+0000 7f284ed9d640 1 -- 172.21.15.7:0/4136706427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2850109000 msgr2=0x7f285019e5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:54:21.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.323+0000 7f284ed9d640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2850109000 0x7f285019e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.323+0000 7f284ed9d640 1 -- 172.21.15.7:0/4136706427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28501086a0 msgr2=0x7f285019e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.323+0000 7f284ed9d640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28501086a0 0x7f285019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.323+0000 7f284ed9d640 1 -- 172.21.15.7:0/4136706427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28501b6d70 con 0x7f285010ff20 2024-03-26T08:54:21.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.323+0000 7f284ed9d640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285010ff20 0x7f28501a5660 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f284000b3f0 tx=0x7f284000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.324+0000 7f283f7fe640 1 -- 172.21.15.7:0/4136706427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840015020 con 0x7f285010ff20 2024-03-26T08:54:21.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.324+0000 7f283f7fe640 1 -- 172.21.15.7:0/4136706427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2840011d10 con 0x7f285010ff20 2024-03-26T08:54:21.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.324+0000 7f283f7fe640 1 -- 172.21.15.7:0/4136706427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284000f2e0 con 0x7f285010ff20 2024-03-26T08:54:21.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.325+0000 7f2854913640 1 -- 172.21.15.7:0/4136706427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28501b6fa0 con 0x7f285010ff20 2024-03-26T08:54:21.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.325+0000 7f2854913640 1 -- 172.21.15.7:0/4136706427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28501b7410 con 0x7f285010ff20 2024-03-26T08:54:21.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.329+0000 7f283f7fe640 1 -- 172.21.15.7:0/4136706427 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2840018020 con 0x7f285010ff20 2024-03-26T08:54:21.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.329+0000 7f28575d0740 1 Processor -- start 2024-03-26T08:54:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.330+0000 7f28575d0740 1 -- start start 2024-03-26T08:54:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.330+0000 7f28575d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914eada0 0x5640914eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.330+0000 7f28575d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.330+0000 7f28575d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914f0900 0x5640914f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.330+0000 7f28575d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640914503d0 con 0x5640914eb790 2024-03-26T08:54:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.330+0000 7f28575d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640913f8ad0 con 0x5640914f0900 2024-03-26T08:54:21.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.330+0000 7f28575d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409144c470 con 0x5640914eada0 2024-03-26T08:54:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.330+0000 7f283f7fe640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28380a2950 0x7f28380a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284ed9d640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28380a2950 0x7f28380a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57410/0 (socket says 172.21.15.7:57410) 2024-03-26T08:54:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284e59c640 1 -- 172.21.15.7:0/1453033552 learned_addr learned my addr 172.21.15.7:0/1453033552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f283f7fe640 1 -- 172.21.15.7:0/4136706427 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28400cf280 con 0x7f285010ff20 2024-03-26T08:54:21.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284dd9b640 1 --2- 172.21.15.7:0/1453033552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914eada0 0x5640914eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284e59c640 1 -- 172.21.15.7:0/1453033552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914eada0 msgr2=0x5640914eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284e59c640 1 --2- 172.21.15.7:0/1453033552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914eada0 0x5640914eb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284e59c640 1 -- 172.21.15.7:0/1453033552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914f0900 msgr2=0x5640914f2d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:54:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284e59c640 1 --2- 172.21.15.7:0/1453033552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914f0900 0x5640914f2d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.331+0000 7f284e59c640 1 -- 172.21.15.7:0/1453033552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56409122e7d0 con 0x5640914eb790 2024-03-26T08:54:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.332+0000 7f284e59c640 1 --2- 172.21.15.7:0/1453033552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f0290 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2848009d80 tx=0x7f284800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea42f665fbceae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.332+0000 7f283cff9640 1 -- 172.21.15.7:0/1453033552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800bb30 con 0x5640914eb790 2024-03-26T08:54:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.332+0000 7f283cff9640 1 -- 172.21.15.7:0/1453033552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2848017070 con 0x5640914eb790 2024-03-26T08:54:21.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.332+0000 7f283cff9640 1 -- 172.21.15.7:0/1453033552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28480115f0 con 0x5640914eb790 2024-03-26T08:54:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.333+0000 7f28575d0740 1 -- 172.21.15.7:0/1453033552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 msgr2=0x5640914f0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.333+0000 7f28575d0740 1 --2- 172.21.15.7:0/1453033552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f0290 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2848009d80 tx=0x7f284800b420 comp rx=0 tx=0).stop 2024-03-26T08:54:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.333+0000 7f28575d0740 1 -- 172.21.15.7:0/1453033552 shutdown_connections 2024-03-26T08:54:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.333+0000 7f28575d0740 1 --2- 172.21.15.7:0/1453033552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914f0900 0x5640914f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.333+0000 7f28575d0740 1 --2- 172.21.15.7:0/1453033552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f0290 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.333+0000 7f28575d0740 1 --2- 172.21.15.7:0/1453033552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914eada0 0x5640914eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.333+0000 7f28575d0740 1 -- 172.21.15.7:0/1453033552 >> 172.21.15.7:0/1453033552 conn(0x5640914e6310 msgr2=0x5640914e6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.333+0000 7f284ed9d640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28380a2950 0x7f28380a4e10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2840000c00 tx=0x7f284009e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 -- 172.21.15.7:0/1453033552 shutdown_connections 2024-03-26T08:54:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 -- 172.21.15.7:0/1453033552 wait complete. 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 -- start start 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 Processor -- start 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640914f38b0 con 0x5640914eb790 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56409140c060 con 0x5640914eada0 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640914f3a30 con 0x5640914f0900 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.334+0000 7f28575d0740 10 client.0 _ll_register_callbacks cb 0x5640913ee6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57420/0 (socket says 172.21.15.7:57420) 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284e59c640 1 -- 172.21.15.7:0/2511757237 learned_addr learned my addr 172.21.15.7:0/2511757237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284dd9b640 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284e59c640 1 -- 172.21.15.7:0/2511757237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 msgr2=0x5640914fc770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284e59c640 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914fc770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284e59c640 1 -- 172.21.15.7:0/2511757237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 msgr2=0x5640914f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284e59c640 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284e59c640 1 -- 172.21.15.7:0/2511757237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640911643f0 con 0x5640914eb790 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284dd9b640 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f284e59c640 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f5730 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f284800b920 tx=0x7f284800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352e6b40a04dbbfe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.335+0000 7f2835ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f2835ffb640 1 -- 172.21.15.7:0/2511757237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848003a30 con 0x5640914eb790 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f2835ffb640 1 -- 172.21.15.7:0/2511757237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2848003bd0 con 0x5640914eb790 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f28575d0740 1 -- 172.21.15.7:0/2511757237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2848017070 con 0x5640914eb790 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f2835ffb640 1 -- 172.21.15.7:0/2511757237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2848003dd0 con 0x5640914eb790 2024-03-26T08:54:21.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f28575d0740 10 client.18588 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:21.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f28575d0740 1 -- 172.21.15.7:0/2511757237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5640911240f0 con 0x5640914eb790 2024-03-26T08:54:21.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f2835ffb640 1 -- 172.21.15.7:0/2511757237 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2848017070 con 0x5640914eb790 2024-03-26T08:54:21.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f2835ffb640 1 -- 172.21.15.7:0/2511757237 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2848011070 con 0x5640914eb790 2024-03-26T08:54:21.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f28575d0740 10 client.18588 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:21.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f28575d0740 10 client.18588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:21.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f28575d0740 20 client.18588 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:21.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f28575d0740 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564091500ad0 0x564091502f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f28575d0740 4 client.18588 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.336+0000 7f28575d0740 1 -- 172.21.15.7:0/2511757237 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56409145d910 con 0x564091500ad0 2024-03-26T08:54:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.337+0000 7f284dd9b640 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564091500ad0 0x564091502f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.339+0000 7f284dd9b640 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564091500ad0 0x564091502f00 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185046041b08a3bf server_cookie=f85d2b0c77f9a802 in_seq=0 out_seq=0 2024-03-26T08:54:21.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.339+0000 7f2835ffb640 10 client.18588 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f2835ffb640 1 -- 172.21.15.7:0/2511757237 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56409145d910 con 0x564091500ad0 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f2835ffb640 10 client.18588 handle_command_reply: tid=0 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 client.18588 shutdown 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 -- 172.21.15.7:0/2511757237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 msgr2=0x5640914f5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f5730 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f284800b920 tx=0x7f284800bea0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 -- 172.21.15.7:0/2511757237 shutdown_connections 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564091500ad0 0x564091502f00 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914f5730 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 --2- 172.21.15.7:0/2511757237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 -- 172.21.15.7:0/2511757237 >> 172.21.15.7:0/2511757237 conn(0x5640914e6310 msgr2=0x5640914e5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f2835ffb640 0 client.18588 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:21.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 -- 172.21.15.7:0/2511757237 shutdown_connections 2024-03-26T08:54:21.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 1 -- 172.21.15.7:0/2511757237 wait complete. 2024-03-26T08:54:21.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.340+0000 7f28575d0740 20 client.18588 trim_cache size 0 max 16384 2024-03-26T08:54:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.369+0000 7f28575d0740 1 Processor -- start 2024-03-26T08:54:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.369+0000 7f28575d0740 1 -- start start 2024-03-26T08:54:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.369+0000 7f28575d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914e4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f28575d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f28575d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f28575d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501c2670 con 0x5640914eb790 2024-03-26T08:54:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f28575d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501c0170 con 0x5640914eada0 2024-03-26T08:54:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f28575d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501be1f0 con 0x5640914f0900 2024-03-26T08:54:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f284e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f284dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914e4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f284e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57432/0 (socket says 172.21.15.7:57432) 2024-03-26T08:54:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f284e59c640 1 -- 172.21.15.7:0/3877194769 learned_addr learned my addr 172.21.15.7:0/3877194769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.370+0000 7f284dd9b640 1 --2- 172.21.15.7:0/3877194769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.371+0000 7f284e59c640 1 -- 172.21.15.7:0/3877194769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 msgr2=0x5640914e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.371+0000 7f284e59c640 1 --2- 172.21.15.7:0/3877194769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.371+0000 7f284e59c640 1 -- 172.21.15.7:0/3877194769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 msgr2=0x5640914e4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.371+0000 7f284e59c640 1 --2- 172.21.15.7:0/3877194769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914e4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.371+0000 7f284e59c640 1 -- 172.21.15.7:0/3877194769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640914efe40 con 0x5640914eb790 2024-03-26T08:54:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.371+0000 7f284e59c640 1 --2- 172.21.15.7:0/3877194769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc730 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2848009610 tx=0x7f28480124b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8591be58315df2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.372+0000 7f2837fff640 1 -- 172.21.15.7:0/3877194769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848034070 con 0x5640914eb790 2024-03-26T08:54:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.372+0000 7f2837fff640 1 -- 172.21.15.7:0/3877194769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2848017040 con 0x5640914eb790 2024-03-26T08:54:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.372+0000 7f2837fff640 1 -- 172.21.15.7:0/3877194769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848003f70 con 0x5640914eb790 2024-03-26T08:54:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.373+0000 7f28575d0740 1 -- 172.21.15.7:0/3877194769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 msgr2=0x5640914fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.373+0000 7f28575d0740 1 --2- 172.21.15.7:0/3877194769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc730 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2848009610 tx=0x7f28480124b0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.373+0000 7f28575d0740 1 -- 172.21.15.7:0/3877194769 shutdown_connections 2024-03-26T08:54:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.373+0000 7f28575d0740 1 --2- 172.21.15.7:0/3877194769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.373+0000 7f28575d0740 1 --2- 172.21.15.7:0/3877194769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc730 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.373+0000 7f28575d0740 1 --2- 172.21.15.7:0/3877194769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914e4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.373+0000 7f28575d0740 1 -- 172.21.15.7:0/3877194769 >> 172.21.15.7:0/3877194769 conn(0x5640914e6310 msgr2=0x5640914ec0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.373+0000 7f28575d0740 1 -- 172.21.15.7:0/3877194769 shutdown_connections 2024-03-26T08:54:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.374+0000 7f28575d0740 1 -- 172.21.15.7:0/3877194769 wait complete. 2024-03-26T08:54:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.375+0000 7f28575d0740 1 -- start start 2024-03-26T08:54:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.375+0000 7f28575d0740 1 Processor -- start 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.375+0000 7f28575d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914fc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.375+0000 7f28575d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f28575d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f28575d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501be7f0 con 0x5640914eb790 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f284dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f28575d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501bfb10 con 0x5640914eada0 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f284dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914fc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60836/0 (socket says 172.21.15.7:60836) 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f284dd9b640 1 -- 172.21.15.7:0/3336024089 learned_addr learned my addr 172.21.15.7:0/3336024089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f284e59c640 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f28575d0740 1 -- 172.21.15.7:0/3336024089 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28501bee50 con 0x5640914f0900 2024-03-26T08:54:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f284dd9b640 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.376+0000 7f28575d0740 10 client.0 _ll_register_callbacks cb 0x7f2848009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.377+0000 7f284e59c640 1 -- 172.21.15.7:0/3336024089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 msgr2=0x5640914fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.377+0000 7f284e59c640 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914fa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.377+0000 7f284e59c640 1 -- 172.21.15.7:0/3336024089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 msgr2=0x5640914fc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.377+0000 7f284e59c640 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914fc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.377+0000 7f284e59c640 1 -- 172.21.15.7:0/3336024089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640914faf40 con 0x5640914eb790 2024-03-26T08:54:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.377+0000 7f284dd9b640 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.378+0000 7f284e59c640 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc690 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2848025900 tx=0x7f284800bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b034662d3f719396 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:21.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.378+0000 7f2835ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.378+0000 7f2835ffb640 1 -- 172.21.15.7:0/3336024089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284803c030 con 0x5640914eb790 2024-03-26T08:54:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.378+0000 7f2835ffb640 1 -- 172.21.15.7:0/3336024089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f284800f070 con 0x5640914eb790 2024-03-26T08:54:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.378+0000 7f28575d0740 1 -- 172.21.15.7:0/3336024089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2848017040 con 0x5640914eb790 2024-03-26T08:54:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.379+0000 7f2835ffb640 1 -- 172.21.15.7:0/3336024089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2848034050 con 0x5640914eb790 2024-03-26T08:54:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.379+0000 7f28575d0740 10 client.18594 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.379+0000 7f28575d0740 1 -- 172.21.15.7:0/3336024089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56409155ce40 con 0x5640914eb790 2024-03-26T08:54:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.379+0000 7f2835ffb640 1 -- 172.21.15.7:0/3336024089 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28480070c0 con 0x5640914eb790 2024-03-26T08:54:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.380+0000 7f2835ffb640 1 -- 172.21.15.7:0/3336024089 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2848011050 con 0x5640914eb790 2024-03-26T08:54:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.380+0000 7f28575d0740 10 client.18594 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.380+0000 7f28575d0740 10 client.18594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.380+0000 7f28575d0740 20 client.18594 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.380+0000 7f28575d0740 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56409155d370 0x56409155f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.380+0000 7f28575d0740 4 client.18594 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:54:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.380+0000 7f28575d0740 1 -- 172.21.15.7:0/3336024089 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f2848017040 con 0x56409155d370 2024-03-26T08:54:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.380+0000 7f284dd9b640 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56409155d370 0x56409155f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.381+0000 7f284dd9b640 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56409155d370 0x56409155f7f0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2df4ad217d55b4d server_cookie=e571df1f981b5b18 in_seq=0 out_seq=0 2024-03-26T08:54:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.381+0000 7f2835ffb640 10 client.18594 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f2835ffb640 1 -- 172.21.15.7:0/3336024089 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x7f2848017040 con 0x56409155d370 2024-03-26T08:54:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f2835ffb640 10 client.18594 handle_command_reply: tid=0 2024-03-26T08:54:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 client.18594 shutdown 2024-03-26T08:54:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 -- 172.21.15.7:0/3336024089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 msgr2=0x5640914fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc690 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f2848025900 tx=0x7f284800bef0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 -- 172.21.15.7:0/3336024089 shutdown_connections 2024-03-26T08:54:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56409155d370 0x56409155f7f0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5640914f0900 0x5640914fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5640914eb790 0x5640914fc690 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 --2- 172.21.15.7:0/3336024089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640914eada0 0x5640914fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 -- 172.21.15.7:0/3336024089 >> 172.21.15.7:0/3336024089 conn(0x5640914e6310 msgr2=0x5640914e9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f2835ffb640 0 client.18594 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 -- 172.21.15.7:0/3336024089 shutdown_connections 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 1 -- 172.21.15.7:0/3336024089 wait complete. 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.382+0000 7f28575d0740 20 client.18594 trim_cache size 0 max 16384 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (168 inodes in the stack)", 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "11793a4d-6454-49d6-af16-9a8895484b4d": { 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:54:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "11793a4d-6454-49d6-af16-9a8895484b4d", 2024-03-26T08:54:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:54:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:54:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:54:21.381 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 -- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28380a2950 msgr2=0x7f28380a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28380a2950 0x7f28380a4e10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2840000c00 tx=0x7f284009e040 comp rx=0 tx=0).stop 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 -- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285010ff20 msgr2=0x7f28501a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285010ff20 0x7f28501a5660 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f284000b3f0 tx=0x7f284000b8b0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 -- 172.21.15.7:0/4136706427 shutdown_connections 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f28380a2950 0x7f28380a4e10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285010ff20 0x7f28501a5660 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2850109000 0x7f285019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 --2- 172.21.15.7:0/4136706427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28501086a0 0x7f285019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 -- 172.21.15.7:0/4136706427 >> 172.21.15.7:0/4136706427 conn(0x7f285006e640 msgr2=0x7f28500706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 -- 172.21.15.7:0/4136706427 shutdown_connections 2024-03-26T08:54:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:21.386+0000 7f2854913640 1 -- 172.21.15.7:0/4136706427 wait complete. 2024-03-26T08:54:21.400 INFO:tasks.cephfs.filesystem:scrub status for tag:11793a4d-6454-49d6-af16-9a8895484b4d - {'path': '/', 'tag': '11793a4d-6454-49d6-af16-9a8895484b4d', 'options': 'recursive,force'} 2024-03-26T08:54:22.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:22 smithi016 ceph-mon[35851]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 17 MiB/s wr, 763 op/s 2024-03-26T08:54:22.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:22 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:54:22.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:22 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:22.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:22 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:54:22.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:22 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:54:22.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:22 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:22.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:22 smithi007 ceph-mon[32462]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 17 MiB/s wr, 763 op/s 2024-03-26T08:54:22.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:22 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:54:22.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:22 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:22.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:22 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:54:22.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:22 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:54:22.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:22 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:22.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:22 smithi044 ceph-mon[36461]: pgmap v718: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 17 MiB/s wr, 763 op/s 2024-03-26T08:54:22.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:22 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:54:22.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:22 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:22.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:22 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:54:22.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:22 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:54:22.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:22 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:23.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:23 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.519+0000 7f787d8c0640 1 -- 172.21.15.7:0/2443218258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878107410 msgr2=0x7f7878109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.519+0000 7f787d8c0640 1 --2- 172.21.15.7:0/2443218258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878107410 0x7f7878109800 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7870009ca0 tx=0x7f787002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.519+0000 7f787d8c0640 1 -- 172.21.15.7:0/2443218258 shutdown_connections 2024-03-26T08:54:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.519+0000 7f787d8c0640 1 --2- 172.21.15.7:0/2443218258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878107410 0x7f7878109800 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.519+0000 7f787d8c0640 1 --2- 172.21.15.7:0/2443218258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7878104ae0 0x7f7878106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.519+0000 7f787d8c0640 1 --2- 172.21.15.7:0/2443218258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781021b0 0x7f78781045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.519+0000 7f787d8c0640 1 -- 172.21.15.7:0/2443218258 >> 172.21.15.7:0/2443218258 conn(0x7f78780fbb40 msgr2=0x7f78780fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.519+0000 7f787d8c0640 1 -- 172.21.15.7:0/2443218258 shutdown_connections 2024-03-26T08:54:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.519+0000 7f787d8c0640 1 -- 172.21.15.7:0/2443218258 wait complete. 2024-03-26T08:54:23.517 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f787d8c0640 1 Processor -- start 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f787d8c0640 1 -- start start 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f787d8c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781021b0 0x7f7878101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f787d8c0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878104ae0 0x7f7878100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f787d8c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7878107410 0x7f78781006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f787d8c0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787810d370 con 0x7f7878104ae0 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f787d8c0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787810d1f0 con 0x7f78781021b0 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f787d8c0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787810d4f0 con 0x7f7878107410 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781021b0 0x7f7878101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781021b0 0x7f7878101b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60860/0 (socket says 172.21.15.7:60860) 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f7876ffd640 1 -- 172.21.15.7:0/3687700311 learned_addr learned my addr 172.21.15.7:0/3687700311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f78767fc640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878104ae0 0x7f7878100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.520+0000 7f78777fe640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7878107410 0x7f78781006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.521+0000 7f7876ffd640 1 -- 172.21.15.7:0/3687700311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7878107410 msgr2=0x7f78781006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.521+0000 7f7876ffd640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7878107410 0x7f78781006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.521+0000 7f7876ffd640 1 -- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878104ae0 msgr2=0x7f7878100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.521+0000 7f7876ffd640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878104ae0 0x7f7878100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.521+0000 7f7876ffd640 1 -- 172.21.15.7:0/3687700311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787819b980 con 0x7f78781021b0 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.521+0000 7f78777fe640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7878107410 0x7f78781006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.521+0000 7f7876ffd640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781021b0 0x7f7878101b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7868009fd0 tx=0x7f786800edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.522+0000 7f787c8be640 1 -- 172.21.15.7:0/3687700311 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786800cd80 con 0x7f78781021b0 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.522+0000 7f787c8be640 1 -- 172.21.15.7:0/3687700311 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7868032920 con 0x7f78781021b0 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.522+0000 7f787d8c0640 1 -- 172.21.15.7:0/3687700311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7878100c40 con 0x7f78781021b0 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.522+0000 7f787c8be640 1 -- 172.21.15.7:0/3687700311 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786803a8a0 con 0x7f78781021b0 2024-03-26T08:54:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.522+0000 7f787d8c0640 1 -- 172.21.15.7:0/3687700311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787819bdf0 con 0x7f78781021b0 2024-03-26T08:54:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.524+0000 7f787c8be640 1 -- 172.21.15.7:0/3687700311 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7868032440 con 0x7f78781021b0 2024-03-26T08:54:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.524+0000 7f787c8be640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78600a2870 0x7f78600a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.524+0000 7f787c8be640 1 -- 172.21.15.7:0/3687700311 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78680f0de0 con 0x7f78781021b0 2024-03-26T08:54:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.524+0000 7f78767fc640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78600a2870 0x7f78600a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f78767fc640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78600a2870 0x7f78600a4d30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7878101890 tx=0x7f786c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7880584740 1 Processor -- start 2024-03-26T08:54:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7880584740 1 -- start start 2024-03-26T08:54:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7880584740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7880584740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed4310 0x5593c4ed8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7880584740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4edb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7880584740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c4e38b10 con 0x5593c4ed3920 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7880584740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c4de1900 con 0x5593c4ed4310 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7880584740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c4e34bb0 con 0x5593c4ed9480 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f78767fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed4310 0x5593c4ed8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f78777fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4edb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57466/0 (socket says 172.21.15.7:57466) 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f78777fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4edb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:50714/0 (socket says 172.21.15.7:50714) 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.525+0000 7f7876ffd640 1 -- 172.21.15.7:0/1573428084 learned_addr learned my addr 172.21.15.7:0/1573428084 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f78777fe640 1 -- 172.21.15.7:0/1573428084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed4310 msgr2=0x5593c4ed8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f78777fe640 1 --2- 172.21.15.7:0/1573428084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed4310 0x5593c4ed8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f78777fe640 1 -- 172.21.15.7:0/1573428084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 msgr2=0x5593c4ed3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f78777fe640 1 --2- 172.21.15.7:0/1573428084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f78777fe640 1 -- 172.21.15.7:0/1573428084 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593c4b4d210 con 0x5593c4ed9480 2024-03-26T08:54:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f7876ffd640 1 --2- 172.21.15.7:0/1573428084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f78777fe640 1 --2- 172.21.15.7:0/1573428084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4edb910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f787002c630 tx=0x7f7870005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=968a541c4d094bb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f785dffb640 1 -- 172.21.15.7:0/1573428084 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870043020 con 0x5593c4ed9480 2024-03-26T08:54:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f785dffb640 1 -- 172.21.15.7:0/1573428084 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7870032c90 con 0x5593c4ed9480 2024-03-26T08:54:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.526+0000 7f785dffb640 1 -- 172.21.15.7:0/1573428084 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870039040 con 0x5593c4ed9480 2024-03-26T08:54:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.527+0000 7f7880584740 1 -- 172.21.15.7:0/1573428084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 msgr2=0x5593c4edb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.527+0000 7f7880584740 1 --2- 172.21.15.7:0/1573428084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4edb910 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f787002c630 tx=0x7f7870005f70 comp rx=0 tx=0).stop 2024-03-26T08:54:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 -- 172.21.15.7:0/1573428084 shutdown_connections 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 --2- 172.21.15.7:0/1573428084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4edb910 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 --2- 172.21.15.7:0/1573428084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed4310 0x5593c4ed8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 --2- 172.21.15.7:0/1573428084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 -- 172.21.15.7:0/1573428084 >> 172.21.15.7:0/1573428084 conn(0x5593c4ecee90 msgr2=0x5593c4ecf290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 -- 172.21.15.7:0/1573428084 shutdown_connections 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 -- 172.21.15.7:0/1573428084 wait complete. 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 -- start start 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 Processor -- start 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 0x5593c4eddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ede0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.528+0000 7f7880584740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4ee5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7880584740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c4edc3d0 con 0x5593c4ed4310 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7880584740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c4df4d50 con 0x5593c4ed3920 2024-03-26T08:54:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7880584740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c4edc550 con 0x5593c4ed9480 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78767fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ede0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 0x5593c4eddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78767fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ede0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57468/0 (socket says 172.21.15.7:57468) 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78777fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4ee5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78767fc640 1 -- 172.21.15.7:0/1777905741 learned_addr learned my addr 172.21.15.7:0/1777905741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7880584740 10 client.0 _ll_register_callbacks cb 0x5593c4df4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78767fc640 1 -- 172.21.15.7:0/1777905741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 msgr2=0x5593c4ee5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78767fc640 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4ee5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78767fc640 1 -- 172.21.15.7:0/1777905741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 msgr2=0x5593c4eddbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78767fc640 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 0x5593c4eddbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78767fc640 1 -- 172.21.15.7:0/1777905741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593c4c19620 con 0x5593c4ed4310 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7876ffd640 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 0x5593c4eddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f78767fc640 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ede0f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f786c003a40 tx=0x7f786c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab955037015b75a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7856ffd640 1 -- 172.21.15.7:0/1777905741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c005020 con 0x5593c4ed4310 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7856ffd640 1 -- 172.21.15.7:0/1777905741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f786c0051c0 con 0x5593c4ed4310 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7880584740 1 -- 172.21.15.7:0/1777905741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7870032c90 con 0x5593c4ed4310 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.529+0000 7f7856ffd640 1 -- 172.21.15.7:0/1777905741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f786c0053c0 con 0x5593c4ed4310 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7880584740 10 client.18606 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7880584740 1 -- 172.21.15.7:0/1777905741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5593c4ee85c0 con 0x5593c4ed4310 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7856ffd640 1 -- 172.21.15.7:0/1777905741 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7870032c90 con 0x5593c4ed4310 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7856ffd640 1 -- 172.21.15.7:0/1777905741 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f786c026070 con 0x5593c4ed4310 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7880584740 10 client.18606 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7880584740 10 client.18606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7880584740 20 client.18606 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7880584740 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5593c4ee9c60 0x5593c4eec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7880584740 4 client.18606 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7880584740 1 -- 172.21.15.7:0/1777905741 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593c4e46050 con 0x5593c4ee9c60 2024-03-26T08:54:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.530+0000 7f7876ffd640 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5593c4ee9c60 0x5593c4eec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.531+0000 7f7856ffd640 10 client.18606 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.531+0000 7f7876ffd640 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5593c4ee9c60 0x5593c4eec0d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ef2ac45807b530 server_cookie=1044a85c56005294 in_seq=0 out_seq=0 2024-03-26T08:54:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7856ffd640 1 -- 172.21.15.7:0/1777905741 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5593c4e46050 con 0x5593c4ee9c60 2024-03-26T08:54:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7856ffd640 10 client.18606 handle_command_reply: tid=0 2024-03-26T08:54:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 client.18606 shutdown 2024-03-26T08:54:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 -- 172.21.15.7:0/1777905741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 msgr2=0x5593c4ede0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:23.533 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ede0f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f786c003a40 tx=0x7f786c004730 comp rx=0 tx=0).stop 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 -- 172.21.15.7:0/1777905741 shutdown_connections 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5593c4ee9c60 0x5593c4eec0d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4ee5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ede0f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 --2- 172.21.15.7:0/1777905741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 0x5593c4eddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 -- 172.21.15.7:0/1777905741 >> 172.21.15.7:0/1777905741 conn(0x5593c4ecee90 msgr2=0x5593c4ece020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7856ffd640 0 client.18606 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 -- 172.21.15.7:0/1777905741 shutdown_connections 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.532+0000 7f7880584740 1 -- 172.21.15.7:0/1777905741 wait complete. 2024-03-26T08:54:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.533+0000 7f7880584740 20 client.18606 trim_cache size 0 max 16384 2024-03-26T08:54:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7880584740 1 Processor -- start 2024-03-26T08:54:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7880584740 1 -- start start 2024-03-26T08:54:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7880584740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7880584740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed4310 0x5593c4ed5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7880584740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed9480 0x5593c4ee50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7880584740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78781a8df0 con 0x5593c4ed3920 2024-03-26T08:54:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7880584740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78781a68f0 con 0x5593c4ed9480 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7880584740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78781a2950 con 0x5593c4ed4310 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7876ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57482/0 (socket says 172.21.15.7:57482) 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f7876ffd640 1 -- 172.21.15.7:0/1513226674 learned_addr learned my addr 172.21.15.7:0/1513226674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f78767fc640 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed4310 0x5593c4ed5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.549+0000 7f78777fe640 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed9480 0x5593c4ee50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.550+0000 7f7876ffd640 1 -- 172.21.15.7:0/1513226674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed4310 msgr2=0x5593c4ed5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.550+0000 7f7876ffd640 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed4310 0x5593c4ed5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.550+0000 7f7876ffd640 1 -- 172.21.15.7:0/1513226674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed9480 msgr2=0x5593c4ee50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.550+0000 7f7876ffd640 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed9480 0x5593c4ee50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.550+0000 7f7876ffd640 1 -- 172.21.15.7:0/1513226674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593c4ecfc00 con 0x5593c4ed3920 2024-03-26T08:54:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.550+0000 7f78767fc640 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed4310 0x5593c4ed5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:23.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:23 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.550+0000 7f7876ffd640 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed5630 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f786800c6a0 tx=0x7f78680ee7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b39a8079fab018 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.550+0000 7f785d7fa640 1 -- 172.21.15.7:0/1513226674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868032730 con 0x5593c4ed3920 2024-03-26T08:54:23.552 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.551+0000 7f785d7fa640 1 -- 172.21.15.7:0/1513226674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78680bb820 con 0x5593c4ed3920 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.551+0000 7f785d7fa640 1 -- 172.21.15.7:0/1513226674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680d1dc0 con 0x5593c4ed3920 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 -- 172.21.15.7:0/1513226674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 msgr2=0x5593c4ed5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed5630 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f786800c6a0 tx=0x7f78680ee7b0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 -- 172.21.15.7:0/1513226674 shutdown_connections 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed9480 0x5593c4ee50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed4310 0x5593c4ed5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 --2- 172.21.15.7:0/1513226674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed3920 0x5593c4ed5630 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 -- 172.21.15.7:0/1513226674 >> 172.21.15.7:0/1513226674 conn(0x5593c4ecee90 msgr2=0x5593c4ecf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 -- 172.21.15.7:0/1513226674 shutdown_connections 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 -- 172.21.15.7:0/1513226674 wait complete. 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 -- start start 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 Processor -- start 2024-03-26T08:54:23.553 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 0x5593c4ee27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ee2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4ee3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78781a2f50 con 0x5593c4ed4310 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78781a6290 con 0x5593c4ed3920 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.552+0000 7f7880584740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78781a35b0 con 0x5593c4ed9480 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f7880584740 10 client.0 _ll_register_callbacks cb 0x7f786c0036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78767fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ee2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78767fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ee2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57490/0 (socket says 172.21.15.7:57490) 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78767fc640 1 -- 172.21.15.7:0/2636699066 learned_addr learned my addr 172.21.15.7:0/2636699066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78777fe640 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4ee3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f7876ffd640 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 0x5593c4ee27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78767fc640 1 -- 172.21.15.7:0/2636699066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 msgr2=0x5593c4ee3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78767fc640 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4ee3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78767fc640 1 -- 172.21.15.7:0/2636699066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 msgr2=0x5593c4ee27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78767fc640 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 0x5593c4ee27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78767fc640 1 -- 172.21.15.7:0/2636699066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593c4ee4170 con 0x5593c4ed4310 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f78767fc640 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ee2ce0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f786c009710 tx=0x7f786c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fe9e0028770481 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f7856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f7856ffd640 1 -- 172.21.15.7:0/2636699066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c03f060 con 0x5593c4ed4310 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f7856ffd640 1 -- 172.21.15.7:0/2636699066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f786c03a400 con 0x5593c4ed4310 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f7880584740 1 -- 172.21.15.7:0/2636699066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78680bb820 con 0x5593c4ed4310 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f7856ffd640 1 -- 172.21.15.7:0/2636699066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f786c031ac0 con 0x5593c4ed4310 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f7880584740 10 client.18612 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.553+0000 7f7880584740 1 -- 172.21.15.7:0/2636699066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5593c4f455a0 con 0x5593c4ed4310 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.554+0000 7f7856ffd640 1 -- 172.21.15.7:0/2636699066 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78680bb820 con 0x5593c4ed4310 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.554+0000 7f7856ffd640 1 -- 172.21.15.7:0/2636699066 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f786c026050 con 0x5593c4ed4310 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.554+0000 7f7880584740 10 client.18612 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.554+0000 7f7880584740 10 client.18612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.554+0000 7f7880584740 20 client.18612 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:23.555 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.554+0000 7f7880584740 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5593c4f45d40 0x5593c4f48160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.554+0000 7f7880584740 4 client.18612 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:54:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.554+0000 7f7880584740 1 -- 172.21.15.7:0/2636699066 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5593c4e403a0 con 0x5593c4f45d40 2024-03-26T08:54:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.554+0000 7f7876ffd640 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5593c4f45d40 0x5593c4f48160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.555+0000 7f7876ffd640 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5593c4f45d40 0x5593c4f48160 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b80b4e4cfed135 server_cookie=f2170933c4620f7e in_seq=0 out_seq=0 2024-03-26T08:54:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.555+0000 7f7856ffd640 10 client.18612 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7856ffd640 1 -- 172.21.15.7:0/2636699066 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5593c4e403a0 con 0x5593c4f45d40 2024-03-26T08:54:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7856ffd640 10 client.18612 handle_command_reply: tid=0 2024-03-26T08:54:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 client.18612 shutdown 2024-03-26T08:54:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 -- 172.21.15.7:0/2636699066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 msgr2=0x5593c4ee2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ee2ce0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f786c009710 tx=0x7f786c031a50 comp rx=0 tx=0).stop 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 -- 172.21.15.7:0/2636699066 shutdown_connections 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5593c4f45d40 0x5593c4f48160 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593c4ed9480 0x5593c4ee3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593c4ed4310 0x5593c4ee2ce0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7856ffd640 0 client.18612 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 --2- 172.21.15.7:0/2636699066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593c4ed3920 0x5593c4ee27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 -- 172.21.15.7:0/2636699066 >> 172.21.15.7:0/2636699066 conn(0x5593c4ecee90 msgr2=0x5593c4ed8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 -- 172.21.15.7:0/2636699066 shutdown_connections 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 1 -- 172.21.15.7:0/2636699066 wait complete. 2024-03-26T08:54:23.570 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.572+0000 7f7880584740 20 client.18612 trim_cache size 0 max 16384 2024-03-26T08:54:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:54:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:54:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:54:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:54:23.571 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 -- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78600a2870 msgr2=0x7f78600a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78600a2870 0x7f78600a4d30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7878101890 tx=0x7f786c005f90 comp rx=0 tx=0).stop 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 -- 172.21.15.7:0/3687700311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781021b0 msgr2=0x7f7878101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781021b0 0x7f7878101b20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7868009fd0 tx=0x7f786800edd0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 -- 172.21.15.7:0/3687700311 shutdown_connections 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f78600a2870 0x7f78600a4d30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7878107410 0x7f78781006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7878104ae0 0x7f7878100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 --2- 172.21.15.7:0/3687700311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78781021b0 0x7f7878101b20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 -- 172.21.15.7:0/3687700311 >> 172.21.15.7:0/3687700311 conn(0x7f78780fbb40 msgr2=0x7f7878102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 -- 172.21.15.7:0/3687700311 shutdown_connections 2024-03-26T08:54:23.572 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:23.573+0000 7f787d8c0640 1 -- 172.21.15.7:0/3687700311 wait complete. 2024-03-26T08:54:23.573 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:23 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:24.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:24 smithi016 ceph-mon[35851]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 823 op/s 2024-03-26T08:54:24.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:24 smithi007 ceph-mon[32462]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 823 op/s 2024-03-26T08:54:24.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:24 smithi044 ceph-mon[36461]: pgmap v719: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 17 MiB/s wr, 823 op/s 2024-03-26T08:54:26.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 715.0 s, 1594.6 tps, lat 62.780 ms stddev 73.565 2024-03-26T08:54:26.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:26 smithi016 ceph-mon[35851]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 799 op/s 2024-03-26T08:54:26.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:26 smithi007 ceph-mon[32462]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 799 op/s 2024-03-26T08:54:26.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:26 smithi044 ceph-mon[36461]: pgmap v720: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 16 MiB/s wr, 799 op/s 2024-03-26T08:54:28.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:28 smithi016 ceph-mon[35851]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 15 MiB/s wr, 704 op/s 2024-03-26T08:54:28.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:28 smithi007 ceph-mon[32462]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 15 MiB/s wr, 704 op/s 2024-03-26T08:54:28.573 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:28 smithi044 ceph-mon[36461]: pgmap v721: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 15 MiB/s wr, 704 op/s 2024-03-26T08:54:30.297 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:30 smithi007 ceph-mon[32462]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 16 MiB/s wr, 775 op/s 2024-03-26T08:54:30.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:30 smithi016 ceph-mon[35851]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 16 MiB/s wr, 775 op/s 2024-03-26T08:54:30.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:30 smithi044 ceph-mon[36461]: pgmap v722: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 16 MiB/s wr, 775 op/s 2024-03-26T08:54:31.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 720.0 s, 1552.0 tps, lat 64.192 ms stddev 75.908 2024-03-26T08:54:32.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:32 smithi016 ceph-mon[35851]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 14 MiB/s wr, 748 op/s 2024-03-26T08:54:32.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:32 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:54:32.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:32 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:54:32.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:32 smithi007 ceph-mon[32462]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 14 MiB/s wr, 748 op/s 2024-03-26T08:54:32.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:32 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:54:32.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:32 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:54:32.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:32 smithi044 ceph-mon[36461]: pgmap v723: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 14 MiB/s wr, 748 op/s 2024-03-26T08:54:32.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:32 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:54:32.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:32 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:54:34.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:34 smithi016 ceph-mon[35851]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 14 MiB/s wr, 766 op/s 2024-03-26T08:54:34.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:34 smithi007 ceph-mon[32462]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 14 MiB/s wr, 766 op/s 2024-03-26T08:54:34.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:34 smithi044 ceph-mon[36461]: pgmap v724: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 14 MiB/s wr, 766 op/s 2024-03-26T08:54:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 725.0 s, 1812.0 tps, lat 55.691 ms stddev 67.769 2024-03-26T08:54:36.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:36 smithi016 ceph-mon[35851]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 664 op/s 2024-03-26T08:54:36.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:36 smithi007 ceph-mon[32462]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 664 op/s 2024-03-26T08:54:36.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:36 smithi044 ceph-mon[36461]: pgmap v725: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 13 MiB/s wr, 664 op/s 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.680+0000 7fcd745f4640 1 -- 172.21.15.7:0/2057082634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c107430 msgr2=0x7fcd6c109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.680+0000 7fcd745f4640 1 --2- 172.21.15.7:0/2057082634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c107430 0x7fcd6c109820 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009ca0 tx=0x7fcd5c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.680+0000 7fcd745f4640 1 -- 172.21.15.7:0/2057082634 shutdown_connections 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.680+0000 7fcd745f4640 1 --2- 172.21.15.7:0/2057082634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c107430 0x7fcd6c109820 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.680+0000 7fcd745f4640 1 --2- 172.21.15.7:0/2057082634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd6c104b00 0x7fcd6c106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.680+0000 7fcd745f4640 1 --2- 172.21.15.7:0/2057082634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd6c1021d0 0x7fcd6c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.680+0000 7fcd745f4640 1 -- 172.21.15.7:0/2057082634 >> 172.21.15.7:0/2057082634 conn(0x7fcd6c0fbb60 msgr2=0x7fcd6c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.680+0000 7fcd745f4640 1 -- 172.21.15.7:0/2057082634 shutdown_connections 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.680+0000 7fcd745f4640 1 -- 172.21.15.7:0/2057082634 wait complete. 2024-03-26T08:54:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.681+0000 7fcd745f4640 1 Processor -- start 2024-03-26T08:54:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.681+0000 7fcd745f4640 1 -- start start 2024-03-26T08:54:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.681+0000 7fcd745f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd6c1021d0 0x7fcd6c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.681+0000 7fcd745f4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c104b00 0x7fcd6c19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.681+0000 7fcd745f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd6c107430 0x7fcd6c1a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.681+0000 7fcd745f4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c10d390 con 0x7fcd6c104b00 2024-03-26T08:54:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.681+0000 7fcd745f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c10d210 con 0x7fcd6c107430 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.681+0000 7fcd745f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c10d510 con 0x7fcd6c1021d0 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72390640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd6c1021d0 0x7fcd6c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd71b8f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c104b00 0x7fcd6c19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72390640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd6c1021d0 0x7fcd6c199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42660/0 (socket says 172.21.15.7:42660) 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd71b8f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c104b00 0x7fcd6c19a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33144/0 (socket says 172.21.15.7:33144) 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72390640 1 -- 172.21.15.7:0/3405881329 learned_addr learned my addr 172.21.15.7:0/3405881329 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72b91640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd6c107430 0x7fcd6c1a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72b91640 1 -- 172.21.15.7:0/3405881329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd6c1021d0 msgr2=0x7fcd6c199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72b91640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd6c1021d0 0x7fcd6c199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72b91640 1 -- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c104b00 msgr2=0x7fcd6c19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72b91640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c104b00 0x7fcd6c19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72b91640 1 -- 172.21.15.7:0/3405881329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c1a1790 con 0x7fcd6c107430 2024-03-26T08:54:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c104b00 0x7fcd6c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72390640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd6c1021d0 0x7fcd6c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd72b91640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd6c107430 0x7fcd6c1a1120 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c037040 tx=0x7fcd5c032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.682+0000 7fcd6b7fe640 1 -- 172.21.15.7:0/3405881329 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c041020 con 0x7fcd6c107430 2024-03-26T08:54:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.683+0000 7fcd745f4640 1 -- 172.21.15.7:0/3405881329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6c1a1a20 con 0x7fcd6c107430 2024-03-26T08:54:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.683+0000 7fcd6b7fe640 1 -- 172.21.15.7:0/3405881329 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd5c034cd0 con 0x7fcd6c107430 2024-03-26T08:54:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.683+0000 7fcd6b7fe640 1 -- 172.21.15.7:0/3405881329 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c039040 con 0x7fcd6c107430 2024-03-26T08:54:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.683+0000 7fcd745f4640 1 -- 172.21.15.7:0/3405881329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c1a1f30 con 0x7fcd6c107430 2024-03-26T08:54:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.686+0000 7fcd771dd740 1 Processor -- start 2024-03-26T08:54:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.687+0000 7fcd6b7fe640 1 -- 172.21.15.7:0/3405881329 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcd5c044020 con 0x7fcd6c107430 2024-03-26T08:54:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.687+0000 7fcd771dd740 1 -- start start 2024-03-26T08:54:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.687+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbddde130 0x557fbddde550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.687+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.687+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 0x557fbdde6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.687+0000 7fcd771dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbdd3ab10 con 0x557fbdddeb20 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.687+0000 7fcd771dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbdce3900 con 0x557fbddde130 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.687+0000 7fcd771dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbdd36bb0 con 0x557fbdde3c90 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.687+0000 7fcd71b8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbddde130 0x557fbddde550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd71b8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbddde130 0x557fbddde550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38452/0 (socket says 172.21.15.7:38452) 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd71b8f640 1 -- 172.21.15.7:0/732294640 learned_addr learned my addr 172.21.15.7:0/732294640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd72390640 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 0x557fbdde6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd71b8f640 1 -- 172.21.15.7:0/732294640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 msgr2=0x557fbdde6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 0x557fbdde6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd71b8f640 1 -- 172.21.15.7:0/732294640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 msgr2=0x557fbdde3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd71b8f640 1 -- 172.21.15.7:0/732294640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fbda4f210 con 0x557fbddde130 2024-03-26T08:54:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd72390640 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd6b7fe640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd540a27f0 0x7fcd540a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbddde130 0x557fbddde550 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcd60013bf0 tx=0x7fcd60005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=801b9c632904f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd72390640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd540a27f0 0x7fcd540a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd68bfe640 1 -- 172.21.15.7:0/732294640 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60013ce0 con 0x557fbddde130 2024-03-26T08:54:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd68bfe640 1 -- 172.21.15.7:0/732294640 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd60013120 con 0x557fbddde130 2024-03-26T08:54:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd68bfe640 1 -- 172.21.15.7:0/732294640 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60003850 con 0x557fbddde130 2024-03-26T08:54:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.688+0000 7fcd6b7fe640 1 -- 172.21.15.7:0/3405881329 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcd5c0f0850 con 0x7fcd6c107430 2024-03-26T08:54:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd771dd740 1 -- 172.21.15.7:0/732294640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbddde130 msgr2=0x557fbddde550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd72390640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd540a27f0 0x7fcd540a4cb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x557fbdddef20 tx=0x7fcd64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd771dd740 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbddde130 0x557fbddde550 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcd60013bf0 tx=0x7fcd60005f90 comp rx=0 tx=0).stop 2024-03-26T08:54:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd771dd740 1 -- 172.21.15.7:0/732294640 shutdown_connections 2024-03-26T08:54:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd771dd740 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 0x557fbdde6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd771dd740 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd771dd740 1 --2- 172.21.15.7:0/732294640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbddde130 0x557fbddde550 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd771dd740 1 -- 172.21.15.7:0/732294640 >> 172.21.15.7:0/732294640 conn(0x557fbddd96a0 msgr2=0x557fbddd9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd771dd740 1 -- 172.21.15.7:0/732294640 shutdown_connections 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.689+0000 7fcd771dd740 1 -- 172.21.15.7:0/732294640 wait complete. 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd771dd740 1 -- start start 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd771dd740 1 Processor -- start 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 0x557fbddef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd771dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbdde6be0 con 0x557fbdddeb20 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd771dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbdcf6d50 con 0x557fbdde3c90 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd771dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fbdde6d60 con 0x557fbddde130 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd72390640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd72390640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42674/0 (socket says 172.21.15.7:42674) 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd72390640 1 -- 172.21.15.7:0/283639436 learned_addr learned my addr 172.21.15.7:0/283639436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd771dd740 10 client.0 _ll_register_callbacks cb 0x557fbdcf6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.690+0000 7fcd72b91640 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 0x557fbddef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd72390640 1 -- 172.21.15.7:0/283639436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 msgr2=0x557fbddef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd72390640 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 0x557fbddef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd72390640 1 -- 172.21.15.7:0/283639436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 msgr2=0x557fbdde89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd72390640 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd72390640 1 -- 172.21.15.7:0/283639436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fbdb1b620 con 0x557fbddde130 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd72390640 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde8490 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcd64013010 tx=0x7fcd64004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93994fdc9e314e2e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd51ffb640 1 -- 172.21.15.7:0/283639436 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6402e060 con 0x557fbddde130 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd51ffb640 1 -- 172.21.15.7:0/283639436 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcd64018030 con 0x557fbddde130 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.691+0000 7fcd771dd740 1 -- 172.21.15.7:0/283639436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd60013120 con 0x557fbddde130 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd51ffb640 1 -- 172.21.15.7:0/283639436 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd64009cf0 con 0x557fbddde130 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd771dd740 10 client.28417 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd771dd740 1 -- 172.21.15.7:0/283639436 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557fbddd0a90 con 0x557fbddde130 2024-03-26T08:54:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd51ffb640 1 -- 172.21.15.7:0/283639436 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd60013120 con 0x557fbddde130 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd51ffb640 1 -- 172.21.15.7:0/283639436 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcd64033070 con 0x557fbddde130 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd771dd740 10 client.28417 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd771dd740 10 client.28417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd771dd740 20 client.28417 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd771dd740 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fbddd2160 0x557fbddd4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd771dd740 4 client.28417 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd771dd740 1 -- 172.21.15.7:0/283639436 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fbdd48050 con 0x557fbddd2160 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.692+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fbddd2160 0x557fbddd4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fbddd2160 0x557fbddd4600 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bf3fa774db3a67f server_cookie=4ec09e7fd4caa861 in_seq=0 out_seq=0 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd51ffb640 10 client.28417 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd51ffb640 1 -- 172.21.15.7:0/283639436 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557fbdd48050 con 0x557fbddd2160 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd51ffb640 10 client.28417 handle_command_reply: tid=0 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd771dd740 1 client.28417 shutdown 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd771dd740 1 -- 172.21.15.7:0/283639436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 msgr2=0x557fbdde8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd771dd740 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde8490 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcd64013010 tx=0x7fcd64004730 comp rx=0 tx=0).stop 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd771dd740 1 -- 172.21.15.7:0/283639436 shutdown_connections 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd771dd740 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fbddd2160 0x557fbddd4600 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.693+0000 7fcd771dd740 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 0x557fbddef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.694+0000 7fcd771dd740 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.694+0000 7fcd51ffb640 0 client.28417 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.694+0000 7fcd771dd740 1 --2- 172.21.15.7:0/283639436 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde8490 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.694+0000 7fcd771dd740 1 -- 172.21.15.7:0/283639436 >> 172.21.15.7:0/283639436 conn(0x557fbddd96a0 msgr2=0x557fbddd0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.694+0000 7fcd771dd740 1 -- 172.21.15.7:0/283639436 shutdown_connections 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.694+0000 7fcd771dd740 1 -- 172.21.15.7:0/283639436 wait complete. 2024-03-26T08:54:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.694+0000 7fcd771dd740 20 client.28417 trim_cache size 0 max 16384 2024-03-26T08:54:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.712+0000 7fcd771dd740 1 Processor -- start 2024-03-26T08:54:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.712+0000 7fcd771dd740 1 -- start start 2024-03-26T08:54:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 0x557fbddcf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd771dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1a8d90 con 0x557fbdddeb20 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd771dd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1a6890 con 0x557fbdde3c90 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd771dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1a4910 con 0x557fbddde130 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd71b8f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd71b8f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33182/0 (socket says 172.21.15.7:33182) 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd71b8f640 1 -- 172.21.15.7:0/4015674786 learned_addr learned my addr 172.21.15.7:0/4015674786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd72390640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd72b91640 1 --2- 172.21.15.7:0/4015674786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 0x557fbddcf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd71b8f640 1 -- 172.21.15.7:0/4015674786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 msgr2=0x557fbdde80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/4015674786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd71b8f640 1 -- 172.21.15.7:0/4015674786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 msgr2=0x557fbddcf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:36.711 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/4015674786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 0x557fbddcf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd71b8f640 1 -- 172.21.15.7:0/4015674786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fbdde3160 con 0x557fbdddeb20 2024-03-26T08:54:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd72b91640 1 --2- 172.21.15.7:0/4015674786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 0x557fbddcf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.713+0000 7fcd71b8f640 1 --2- 172.21.15.7:0/4015674786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde85e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcd60013bf0 tx=0x7fcd600096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97be84215ab1bd61 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd53fff640 1 -- 172.21.15.7:0/4015674786 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60013120 con 0x557fbdddeb20 2024-03-26T08:54:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd53fff640 1 -- 172.21.15.7:0/4015674786 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd600055a0 con 0x557fbdddeb20 2024-03-26T08:54:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd53fff640 1 -- 172.21.15.7:0/4015674786 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60003aa0 con 0x557fbdddeb20 2024-03-26T08:54:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd771dd740 1 -- 172.21.15.7:0/4015674786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 msgr2=0x557fbdde85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd771dd740 1 --2- 172.21.15.7:0/4015674786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde85e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcd60013bf0 tx=0x7fcd600096c0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd771dd740 1 -- 172.21.15.7:0/4015674786 shutdown_connections 2024-03-26T08:54:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd771dd740 1 --2- 172.21.15.7:0/4015674786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdde3c90 0x557fbddcf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd771dd740 1 --2- 172.21.15.7:0/4015674786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbdddeb20 0x557fbdde85e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd771dd740 1 --2- 172.21.15.7:0/4015674786 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbddde130 0x557fbdde80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd771dd740 1 -- 172.21.15.7:0/4015674786 >> 172.21.15.7:0/4015674786 conn(0x557fbddd96a0 msgr2=0x557fbddd9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd771dd740 1 -- 172.21.15.7:0/4015674786 shutdown_connections 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.714+0000 7fcd771dd740 1 -- 172.21.15.7:0/4015674786 wait complete. 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd771dd740 1 -- start start 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd771dd740 1 Processor -- start 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbddde130 0x557fbde417d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdddeb20 0x557fbde43d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd771dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 0x557fbde46270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd71b8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdddeb20 0x557fbde43d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd71b8f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdddeb20 0x557fbde43d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38488/0 (socket says 172.21.15.7:38488) 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd71b8f640 1 -- 172.21.15.7:0/3487905384 learned_addr learned my addr 172.21.15.7:0/3487905384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1a4f10 con 0x557fbddde130 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1a6230 con 0x557fbdddeb20 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c1a5570 con 0x557fbdde3c90 2024-03-26T08:54:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd72b91640 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 0x557fbde46270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd72390640 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbddde130 0x557fbde417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.715+0000 7fcd771dd740 10 client.0 _ll_register_callbacks cb 0x7fcd6400fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd72b91640 1 -- 172.21.15.7:0/3487905384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdddeb20 msgr2=0x557fbde43d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd72b91640 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdddeb20 0x557fbde43d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd72b91640 1 -- 172.21.15.7:0/3487905384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbddde130 msgr2=0x557fbde417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd72b91640 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbddde130 0x557fbde417d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd72b91640 1 -- 172.21.15.7:0/3487905384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fbde46960 con 0x557fbdde3c90 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd72390640 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbddde130 0x557fbde417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd72b91640 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 0x557fbde46270 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006010 tx=0x7fcd5c0bcba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6d82aef7877c079 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd51ffb640 1 -- 172.21.15.7:0/3487905384 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0c1170 con 0x557fbdde3c90 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd600055a0 con 0x557fbdde3c90 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd51ffb640 1 -- 172.21.15.7:0/3487905384 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcd5c003970 con 0x557fbdde3c90 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd51ffb640 1 -- 172.21.15.7:0/3487905384 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd5c0bd240 con 0x557fbdde3c90 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd771dd740 10 client.28423 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.716+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557fbde47d10 con 0x557fbdde3c90 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd51ffb640 1 -- 172.21.15.7:0/3487905384 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd600055a0 con 0x557fbdde3c90 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd51ffb640 1 -- 172.21.15.7:0/3487905384 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcd5c03c030 con 0x557fbdde3c90 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd771dd740 10 client.28423 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd771dd740 10 client.28423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd771dd740 20 client.28423 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd771dd740 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fbde48560 0x557fbde4a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd771dd740 4 client.28423 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557fbdd423a0 con 0x557fbde48560 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd72390640 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fbde48560 0x557fbde4a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd72390640 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fbde48560 0x557fbde4a9f0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcad74d0ccee8cbc server_cookie=f6352cc7614cd6da in_seq=0 out_seq=0 2024-03-26T08:54:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.717+0000 7fcd51ffb640 10 client.28423 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.773+0000 7fcd51ffb640 1 -- 172.21.15.7:0/3487905384 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557fbdd423a0 con 0x557fbde48560 2024-03-26T08:54:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.773+0000 7fcd51ffb640 10 client.28423 handle_command_reply: tid=0 2024-03-26T08:54:36.773 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.775+0000 7fcd771dd740 1 client.28423 shutdown 2024-03-26T08:54:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.780+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 msgr2=0x557fbde46270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:36.777 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.780+0000 7fcd771dd740 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 0x557fbde46270 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c006010 tx=0x7fcd5c0bcba0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.785+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 shutdown_connections 2024-03-26T08:54:36.782 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.785+0000 7fcd771dd740 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557fbde48560 0x557fbde4a9f0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.785+0000 7fcd771dd740 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557fbdde3c90 0x557fbde46270 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.785+0000 7fcd771dd740 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbdddeb20 0x557fbde43d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.785+0000 7fcd771dd740 1 --2- 172.21.15.7:0/3487905384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557fbddde130 0x557fbde417d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.783 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.785+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 >> 172.21.15.7:0/3487905384 conn(0x557fbddd96a0 msgr2=0x557fbdde8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.787+0000 7fcd51ffb640 0 client.28423 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:36.784 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.787+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 shutdown_connections 2024-03-26T08:54:36.785 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.789+0000 7fcd771dd740 1 -- 172.21.15.7:0/3487905384 wait complete. 2024-03-26T08:54:36.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.793+0000 7fcd771dd740 20 client.28423 trim_cache size 0 max 16384 2024-03-26T08:54:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:54:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:54:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:54:36.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:54:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.796+0000 7fcd745f4640 1 -- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd540a27f0 msgr2=0x7fcd540a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.796+0000 7fcd745f4640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd540a27f0 0x7fcd540a4cb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x557fbdddef20 tx=0x7fcd64005f90 comp rx=0 tx=0).stop 2024-03-26T08:54:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.796+0000 7fcd745f4640 1 -- 172.21.15.7:0/3405881329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd6c107430 msgr2=0x7fcd6c1a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:36.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.796+0000 7fcd745f4640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd6c107430 0x7fcd6c1a1120 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c037040 tx=0x7fcd5c032ea0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.797+0000 7fcd745f4640 1 -- 172.21.15.7:0/3405881329 shutdown_connections 2024-03-26T08:54:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.797+0000 7fcd745f4640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcd540a27f0 0x7fcd540a4cb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.797+0000 7fcd745f4640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd6c107430 0x7fcd6c1a1120 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.797+0000 7fcd745f4640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd6c104b00 0x7fcd6c19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.797+0000 7fcd745f4640 1 --2- 172.21.15.7:0/3405881329 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcd6c1021d0 0x7fcd6c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.797+0000 7fcd745f4640 1 -- 172.21.15.7:0/3405881329 >> 172.21.15.7:0/3405881329 conn(0x7fcd6c0fbb60 msgr2=0x7fcd6c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.797+0000 7fcd745f4640 1 -- 172.21.15.7:0/3405881329 shutdown_connections 2024-03-26T08:54:36.795 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:36.797+0000 7fcd745f4640 1 -- 172.21.15.7:0/3405881329 wait complete. 2024-03-26T08:54:38.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:38 smithi016 ceph-mon[35851]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 523 op/s 2024-03-26T08:54:38.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:38 smithi007 ceph-mon[32462]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 523 op/s 2024-03-26T08:54:38.573 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:38 smithi044 ceph-mon[36461]: pgmap v726: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 523 op/s 2024-03-26T08:54:40.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:40 smithi016 ceph-mon[35851]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 656 op/s 2024-03-26T08:54:40.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:40 smithi007 ceph-mon[32462]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 656 op/s 2024-03-26T08:54:40.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:40 smithi044 ceph-mon[36461]: pgmap v727: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 13 MiB/s wr, 656 op/s 2024-03-26T08:54:41.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 730.0 s, 1669.2 tps, lat 59.055 ms stddev 66.782 2024-03-26T08:54:42.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:42 smithi016 ceph-mon[35851]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 629 op/s 2024-03-26T08:54:42.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:42 smithi007 ceph-mon[32462]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 629 op/s 2024-03-26T08:54:42.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:42 smithi044 ceph-mon[36461]: pgmap v728: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 629 op/s 2024-03-26T08:54:44.498 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:44 smithi016 ceph-mon[35851]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 656 op/s 2024-03-26T08:54:44.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:44 smithi007 ceph-mon[32462]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 656 op/s 2024-03-26T08:54:44.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:44 smithi044 ceph-mon[36461]: pgmap v729: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 12 MiB/s wr, 656 op/s 2024-03-26T08:54:46.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 735.0 s, 1638.0 tps, lat 61.667 ms stddev 73.738 2024-03-26T08:54:46.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:46 smithi007 ceph-mon[32462]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 677 op/s 2024-03-26T08:54:46.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:46 smithi044 ceph-mon[36461]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 677 op/s 2024-03-26T08:54:46.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:46 smithi016 ceph-mon[35851]: pgmap v730: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 MiB/s wr, 677 op/s 2024-03-26T08:54:48.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:48 smithi007 ceph-mon[32462]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 MiB/s wr, 571 op/s 2024-03-26T08:54:48.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:48 smithi044 ceph-mon[36461]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 MiB/s wr, 571 op/s 2024-03-26T08:54:48.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:48 smithi016 ceph-mon[35851]: pgmap v731: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 11 MiB/s wr, 571 op/s 2024-03-26T08:54:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.915+0000 7f3db3fdf640 1 -- 172.21.15.7:0/2067732340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac100600 msgr2=0x7f3dac100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.915+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/2067732340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac100600 0x7f3dac100a20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009900 tx=0x7f3da802bc30 comp rx=0 tx=0).stop 2024-03-26T08:54:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.915+0000 7f3db3fdf640 1 -- 172.21.15.7:0/2067732340 shutdown_connections 2024-03-26T08:54:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.915+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/2067732340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dac1021b0 0x7f3dac108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.915+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/2067732340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac1017f0 0x7f3dac101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.915+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/2067732340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac100600 0x7f3dac100a20 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.915+0000 7f3db3fdf640 1 -- 172.21.15.7:0/2067732340 >> 172.21.15.7:0/2067732340 conn(0x7f3dac0fbb70 msgr2=0x7f3dac0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.915+0000 7f3db3fdf640 1 -- 172.21.15.7:0/2067732340 shutdown_connections 2024-03-26T08:54:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.916+0000 7f3db3fdf640 1 -- 172.21.15.7:0/2067732340 wait complete. 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.917+0000 7f3db3fdf640 1 Processor -- start 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.917+0000 7f3db3fdf640 1 -- start start 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.917+0000 7f3db3fdf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac100600 0x7f3dac1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.917+0000 7f3db3fdf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dac1017f0 0x7f3dac1a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.917+0000 7f3db3fdf640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1021b0 0x7f3dac1a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.917+0000 7f3db3fdf640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac10c270 con 0x7f3dac1021b0 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.917+0000 7f3db3fdf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac10c0f0 con 0x7f3dac100600 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.917+0000 7f3db3fdf640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac10c3f0 con 0x7f3dac1017f0 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db257c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1021b0 0x7f3dac1a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db257c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1021b0 0x7f3dac1a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57040/0 (socket says 172.21.15.7:57040) 2024-03-26T08:54:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db257c640 1 -- 172.21.15.7:0/859670950 learned_addr learned my addr 172.21.15.7:0/859670950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db157a640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dac1017f0 0x7f3dac1a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac100600 0x7f3dac1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.916 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db157a640 1 -- 172.21.15.7:0/859670950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac100600 msgr2=0x7f3dac1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db157a640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac100600 0x7f3dac1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db157a640 1 -- 172.21.15.7:0/859670950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1021b0 msgr2=0x7f3dac1a5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db157a640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1021b0 0x7f3dac1a5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.918+0000 7f3db157a640 1 -- 172.21.15.7:0/859670950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dac195ec0 con 0x7f3dac1017f0 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.919+0000 7f3db157a640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dac1017f0 0x7f3dac1a2cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00e9f0 tx=0x7f3d9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.919+0000 7f3da2ffd640 1 -- 172.21.15.7:0/859670950 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00ccc0 con 0x7f3dac1017f0 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.919+0000 7f3db3fdf640 1 -- 172.21.15.7:0/859670950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dac1961b0 con 0x7f3dac1017f0 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.919+0000 7f3da2ffd640 1 -- 172.21.15.7:0/859670950 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d9c010910 con 0x7f3dac1017f0 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.919+0000 7f3da2ffd640 1 -- 172.21.15.7:0/859670950 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c018960 con 0x7f3dac1017f0 2024-03-26T08:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.919+0000 7f3db3fdf640 1 -- 172.21.15.7:0/859670950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dac196720 con 0x7f3dac1017f0 2024-03-26T08:54:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.922+0000 7f3da2ffd640 1 -- 172.21.15.7:0/859670950 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3d9c010ac0 con 0x7f3dac1017f0 2024-03-26T08:54:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.922+0000 7f3db6bca740 1 Processor -- start 2024-03-26T08:54:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db6bca740 1 -- start start 2024-03-26T08:54:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4726d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 0x560af472be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af472c480 0x560af472e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db6bca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af468bb10 con 0x560af4727310 2024-03-26T08:54:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db6bca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af4634900 con 0x560af472c480 2024-03-26T08:54:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db6bca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af4687bb0 con 0x560af4726920 2024-03-26T08:54:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db1d7b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 0x560af472be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db257c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4726d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db157a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af472c480 0x560af472e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db257c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4726d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58524/0 (socket says 172.21.15.7:58524) 2024-03-26T08:54:49.921 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.923+0000 7f3db257c640 1 -- 172.21.15.7:0/911832309 learned_addr learned my addr 172.21.15.7:0/911832309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3db257c640 1 -- 172.21.15.7:0/911832309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af472c480 msgr2=0x560af472e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3db257c640 1 --2- 172.21.15.7:0/911832309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af472c480 0x560af472e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3db257c640 1 -- 172.21.15.7:0/911832309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 msgr2=0x560af472be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3db257c640 1 --2- 172.21.15.7:0/911832309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 0x560af472be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3db257c640 1 -- 172.21.15.7:0/911832309 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af43a0210 con 0x560af4726920 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3db157a640 1 --2- 172.21.15.7:0/911832309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af472c480 0x560af472e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/911832309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 0x560af472be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3db257c640 1 --2- 172.21.15.7:0/911832309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4726d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3da4011570 tx=0x7f3da4011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e39814349de390 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3da2ffd640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d980a27f0 0x7f3d980a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3d97fff640 1 -- 172.21.15.7:0/911832309 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4004b20 con 0x560af4726920 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3d97fff640 1 -- 172.21.15.7:0/911832309 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3da4004cc0 con 0x560af4726920 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3d97fff640 1 -- 172.21.15.7:0/911832309 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da40054c0 con 0x560af4726920 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.924+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d980a27f0 0x7f3d980a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3da2ffd640 1 -- 172.21.15.7:0/859670950 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3d9c0cee50 con 0x7f3dac1017f0 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db6bca740 1 -- 172.21.15.7:0/911832309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 msgr2=0x560af4726d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d980a27f0 0x7f3d980a4cb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3da8000c00 tx=0x7f3da8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db6bca740 1 --2- 172.21.15.7:0/911832309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4726d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3da4011570 tx=0x7f3da4011a30 comp rx=0 tx=0).stop 2024-03-26T08:54:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db6bca740 1 -- 172.21.15.7:0/911832309 shutdown_connections 2024-03-26T08:54:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db6bca740 1 --2- 172.21.15.7:0/911832309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af472c480 0x560af472e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db6bca740 1 --2- 172.21.15.7:0/911832309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 0x560af472be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db6bca740 1 --2- 172.21.15.7:0/911832309 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4726d40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db6bca740 1 -- 172.21.15.7:0/911832309 >> 172.21.15.7:0/911832309 conn(0x560af4721e90 msgr2=0x560af4722290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db6bca740 1 -- 172.21.15.7:0/911832309 shutdown_connections 2024-03-26T08:54:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.925+0000 7f3db6bca740 1 -- 172.21.15.7:0/911832309 wait complete. 2024-03-26T08:54:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.926+0000 7f3db6bca740 1 -- start start 2024-03-26T08:54:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.926+0000 7f3db6bca740 1 Processor -- start 2024-03-26T08:54:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.927+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4730bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.927+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af4731110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.927+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af472c480 0x560af47380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.927+0000 7f3db6bca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af472f430 con 0x560af472c480 2024-03-26T08:54:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.927+0000 7f3db6bca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af4647d50 con 0x560af4727310 2024-03-26T08:54:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.927+0000 7f3db6bca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af472f5b0 con 0x560af4726920 2024-03-26T08:54:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.927+0000 7f3db6bca740 10 client.0 _ll_register_callbacks cb 0x560af4647720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.928+0000 7f3db157a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af4731110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.928+0000 7f3db157a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af4731110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57004/0 (socket says 172.21.15.7:57004) 2024-03-26T08:54:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.928+0000 7f3db157a640 1 -- 172.21.15.7:0/635024378 learned_addr learned my addr 172.21.15.7:0/635024378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.928+0000 7f3db157a640 1 -- 172.21.15.7:0/635024378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 msgr2=0x560af4730bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:54:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.928+0000 7f3db157a640 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4730bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.928+0000 7f3db157a640 1 -- 172.21.15.7:0/635024378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af472c480 msgr2=0x560af47380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.928+0000 7f3db157a640 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af472c480 0x560af47380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.928+0000 7f3db157a640 1 -- 172.21.15.7:0/635024378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560af446c620 con 0x560af4727310 2024-03-26T08:54:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.929+0000 7f3db157a640 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af4731110 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0a3a20 tx=0x7f3d9c0cc910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc84d1daf5e0d85 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.929+0000 7f3d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.929+0000 7f3d957fa640 1 -- 172.21.15.7:0/635024378 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c09b3a0 con 0x560af4727310 2024-03-26T08:54:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.929+0000 7f3d957fa640 1 -- 172.21.15.7:0/635024378 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3d9c0860b0 con 0x560af4727310 2024-03-26T08:54:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.929+0000 7f3db6bca740 1 -- 172.21.15.7:0/635024378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3da4004cc0 con 0x560af4727310 2024-03-26T08:54:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3d957fa640 1 -- 172.21.15.7:0/635024378 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d9c09b1c0 con 0x560af4727310 2024-03-26T08:54:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3db6bca740 10 client.28433 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3db6bca740 1 -- 172.21.15.7:0/635024378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560af473b4c0 con 0x560af4727310 2024-03-26T08:54:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3d957fa640 1 -- 172.21.15.7:0/635024378 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3da4004cc0 con 0x560af4727310 2024-03-26T08:54:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3d957fa640 1 -- 172.21.15.7:0/635024378 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3d9c09ad40 con 0x560af4727310 2024-03-26T08:54:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3db6bca740 10 client.28433 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3db6bca740 10 client.28433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3db6bca740 20 client.28433 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3db6bca740 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560af473cb30 0x560af473ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3db6bca740 4 client.28433 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.930+0000 7f3db6bca740 1 -- 172.21.15.7:0/635024378 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560af4699050 con 0x560af473cb30 2024-03-26T08:54:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.931+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560af473cb30 0x560af473ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.931+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560af473cb30 0x560af473ef60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91b1a01602713ff server_cookie=d4702c364f1015a7 in_seq=0 out_seq=0 2024-03-26T08:54:49.929 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.931+0000 7f3d957fa640 10 client.28433 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3d957fa640 1 -- 172.21.15.7:0/635024378 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560af4699050 con 0x560af473cb30 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3d957fa640 10 client.28433 handle_command_reply: tid=0 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3db6bca740 1 client.28433 shutdown 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3db6bca740 1 -- 172.21.15.7:0/635024378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 msgr2=0x560af4731110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3db6bca740 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af4731110 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0a3a20 tx=0x7f3d9c0cc910 comp rx=0 tx=0).stop 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3db6bca740 1 -- 172.21.15.7:0/635024378 shutdown_connections 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3db6bca740 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560af473cb30 0x560af473ef60 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3db6bca740 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af472c480 0x560af47380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3db6bca740 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af4731110 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3db6bca740 1 --2- 172.21.15.7:0/635024378 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4730bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3db6bca740 1 -- 172.21.15.7:0/635024378 >> 172.21.15.7:0/635024378 conn(0x560af4721e90 msgr2=0x560af4721020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.934+0000 7f3d957fa640 0 client.28433 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:49.932 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.935+0000 7f3db6bca740 1 -- 172.21.15.7:0/635024378 shutdown_connections 2024-03-26T08:54:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.935+0000 7f3db6bca740 1 -- 172.21.15.7:0/635024378 wait complete. 2024-03-26T08:54:49.933 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.935+0000 7f3db6bca740 20 client.28433 trim_cache size 0 max 16384 2024-03-26T08:54:49.960 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.963+0000 7f3db6bca740 1 Processor -- start 2024-03-26T08:54:49.966 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db6bca740 1 -- start start 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4721e90 0x560af4730bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4731110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 0x560af47380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db6bca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac1b9de0 con 0x560af4727310 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db6bca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac1b78e0 con 0x560af4721e90 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db6bca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac19d440 con 0x560af4726920 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db1d7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4721e90 0x560af4730bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db1d7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4721e90 0x560af4730bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57010/0 (socket says 172.21.15.7:57010) 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db1d7b640 1 -- 172.21.15.7:0/3202388056 learned_addr learned my addr 172.21.15.7:0/3202388056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:49.967 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.969+0000 7f3db157a640 1 --2- 172.21.15.7:0/3202388056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4731110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.970+0000 7f3db257c640 1 --2- 172.21.15.7:0/3202388056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 0x560af47380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.970+0000 7f3db1d7b640 1 -- 172.21.15.7:0/3202388056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 msgr2=0x560af4731110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.970+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/3202388056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4731110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.970+0000 7f3db1d7b640 1 -- 172.21.15.7:0/3202388056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 msgr2=0x560af47380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.970+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/3202388056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 0x560af47380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.970+0000 7f3db1d7b640 1 -- 172.21.15.7:0/3202388056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af4726d00 con 0x560af4721e90 2024-03-26T08:54:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.973+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/3202388056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4721e90 0x560af4730bd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3da802c7b0 tx=0x7f3da8040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cffdda9f86443671 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.975+0000 7f3d977fe640 1 -- 172.21.15.7:0/3202388056 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da80035f0 con 0x560af4721e90 2024-03-26T08:54:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.976+0000 7f3d977fe640 1 -- 172.21.15.7:0/3202388056 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3da8054070 con 0x560af4721e90 2024-03-26T08:54:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.976+0000 7f3db6bca740 1 -- 172.21.15.7:0/3202388056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4721e90 msgr2=0x560af4730bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.976+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3202388056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4721e90 0x560af4730bd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3da802c7b0 tx=0x7f3da8040000 comp rx=0 tx=0).stop 2024-03-26T08:54:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.979+0000 7f3db6bca740 1 -- 172.21.15.7:0/3202388056 shutdown_connections 2024-03-26T08:54:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.979+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3202388056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4727310 0x560af47380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.979+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3202388056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4731110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.979+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3202388056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4721e90 0x560af4730bd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.979+0000 7f3db6bca740 1 -- 172.21.15.7:0/3202388056 >> 172.21.15.7:0/3202388056 conn(0x560af4727f10 msgr2=0x560af4722270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.983+0000 7f3db6bca740 1 -- 172.21.15.7:0/3202388056 shutdown_connections 2024-03-26T08:54:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.984+0000 7f3db6bca740 1 -- 172.21.15.7:0/3202388056 wait complete. 2024-03-26T08:54:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.985+0000 7f3db6bca740 1 -- start start 2024-03-26T08:54:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.985+0000 7f3db6bca740 1 Processor -- start 2024-03-26T08:54:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.985+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4721e90 0x560af4734050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.985+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4734590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.982 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.985+0000 7f3db6bca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af47375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.985+0000 7f3db6bca740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac19da40 con 0x560af4721e90 2024-03-26T08:54:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.985+0000 7f3db6bca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac1b7280 con 0x560af4727310 2024-03-26T08:54:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.985+0000 7f3db6bca740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac19e080 con 0x560af4726920 2024-03-26T08:54:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.985+0000 7f3db6bca740 10 client.0 _ll_register_callbacks cb 0x7f3d9c0d2580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.986+0000 7f3db257c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af47375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.986+0000 7f3db257c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af47375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57022/0 (socket says 172.21.15.7:57022) 2024-03-26T08:54:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.986+0000 7f3db257c640 1 -- 172.21.15.7:0/3822826371 learned_addr learned my addr 172.21.15.7:0/3822826371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:49.983 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.986+0000 7f3db157a640 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4734590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.990+0000 7f3db257c640 1 -- 172.21.15.7:0/3822826371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 msgr2=0x560af4734590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.990+0000 7f3db257c640 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4734590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.990+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4721e90 0x560af4734050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.990+0000 7f3db257c640 1 -- 172.21.15.7:0/3822826371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4721e90 msgr2=0x560af4734050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.990+0000 7f3db257c640 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4721e90 0x560af4734050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.990+0000 7f3db257c640 1 -- 172.21.15.7:0/3822826371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560af4737ba0 con 0x560af4727310 2024-03-26T08:54:49.987 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.990+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4721e90 0x560af4734050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:54:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.991+0000 7f3db257c640 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af47375b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3da401b040 tx=0x7f3da400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7723363574ecce6a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.991+0000 7f3d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.991+0000 7f3d957fa640 1 -- 172.21.15.7:0/3822826371 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4026070 con 0x560af4727310 2024-03-26T08:54:49.988 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.991+0000 7f3db6bca740 1 -- 172.21.15.7:0/3822826371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3da8054070 con 0x560af4727310 2024-03-26T08:54:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.991+0000 7f3d957fa640 1 -- 172.21.15.7:0/3822826371 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3da400f960 con 0x560af4727310 2024-03-26T08:54:49.989 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.992+0000 7f3d957fa640 1 -- 172.21.15.7:0/3822826371 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3da4032050 con 0x560af4727310 2024-03-26T08:54:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.993+0000 7f3db6bca740 10 client.28439 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.993+0000 7f3db6bca740 1 -- 172.21.15.7:0/3822826371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560af47a9dd0 con 0x560af4727310 2024-03-26T08:54:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.994+0000 7f3d957fa640 1 -- 172.21.15.7:0/3822826371 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3da8054070 con 0x560af4727310 2024-03-26T08:54:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.994+0000 7f3d957fa640 1 -- 172.21.15.7:0/3822826371 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3da40057f0 con 0x560af4727310 2024-03-26T08:54:49.991 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.994+0000 7f3db6bca740 10 client.28439 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.994+0000 7f3db6bca740 10 client.28439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.994+0000 7f3db6bca740 20 client.28439 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.994+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560af472c480 0x560af47ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.994+0000 7f3db6bca740 4 client.28439 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:54:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.994+0000 7f3db6bca740 1 -- 172.21.15.7:0/3822826371 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560af46933a0 con 0x560af472c480 2024-03-26T08:54:49.992 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.995+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560af472c480 0x560af47ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.997+0000 7f3db1d7b640 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560af472c480 0x560af47ac760 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1080c48ea9ce0f45 server_cookie=7b43f92c6a29930c in_seq=0 out_seq=0 2024-03-26T08:54:49.994 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:49.997+0000 7f3d957fa640 10 client.28439 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.068+0000 7f3d957fa640 1 -- 172.21.15.7:0/3822826371 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560af46933a0 con 0x560af472c480 2024-03-26T08:54:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.068+0000 7f3d957fa640 10 client.28439 handle_command_reply: tid=0 2024-03-26T08:54:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.068+0000 7f3db6bca740 1 client.28439 shutdown 2024-03-26T08:54:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 -- 172.21.15.7:0/3822826371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 msgr2=0x560af47375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af47375b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3da401b040 tx=0x7f3da400f7f0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 -- 172.21.15.7:0/3822826371 shutdown_connections 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560af472c480 0x560af47ac760 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af4727310 0x560af47375b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3d957fa640 0 client.28439 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560af4726920 0x560af4734590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 --2- 172.21.15.7:0/3822826371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560af4721e90 0x560af4734050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 -- 172.21.15.7:0/3822826371 >> 172.21.15.7:0/3822826371 conn(0x560af4727f10 msgr2=0x560af4730b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 -- 172.21.15.7:0/3822826371 shutdown_connections 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 1 -- 172.21.15.7:0/3822826371 wait complete. 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.069+0000 7f3db6bca740 20 client.28439 trim_cache size 0 max 16384 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:54:50.067 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:54:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:54:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:54:50.068 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 -- 172.21.15.7:0/859670950 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d980a27f0 msgr2=0x7f3d980a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d980a27f0 0x7f3d980a4cb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3da8000c00 tx=0x7f3da8005fd0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 -- 172.21.15.7:0/859670950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dac1017f0 msgr2=0x7f3dac1a2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dac1017f0 0x7f3dac1a2cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00e9f0 tx=0x7f3d9c00eeb0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 -- 172.21.15.7:0/859670950 shutdown_connections 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3d980a27f0 0x7f3d980a4cb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac1021b0 0x7f3dac1a5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3dac1017f0 0x7f3dac1a2cb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 --2- 172.21.15.7:0/859670950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dac100600 0x7f3dac1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 -- 172.21.15.7:0/859670950 >> 172.21.15.7:0/859670950 conn(0x7f3dac0fbb70 msgr2=0x7f3dac0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 -- 172.21.15.7:0/859670950 shutdown_connections 2024-03-26T08:54:50.069 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:54:50.070+0000 7f3db3fdf640 1 -- 172.21.15.7:0/859670950 wait complete. 2024-03-26T08:54:50.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:50 smithi007 ceph-mon[32462]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 738 op/s 2024-03-26T08:54:50.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:54:50.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:50 smithi044 ceph-mon[36461]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 738 op/s 2024-03-26T08:54:50.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:54:50.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:50 smithi016 ceph-mon[35851]: pgmap v732: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 738 op/s 2024-03-26T08:54:50.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:54:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 740.0 s, 1207.6 tps, lat 81.652 ms stddev 135.737 2024-03-26T08:54:51.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:54:51.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:54:51.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:54:51.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:54:51.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:54:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.557+0000 7fc92b6cc640 1 -- 172.21.15.7:0/3666235413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 msgr2=0x7fc924108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.557+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/3666235413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 0x7fc924108a00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc914009ca0 tx=0x7fc91402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.557+0000 7fc92b6cc640 1 -- 172.21.15.7:0/3666235413 shutdown_connections 2024-03-26T08:54:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.557+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/3666235413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 0x7fc924108a00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.557+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/3666235413 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9241017b0 0x7fc924101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.557+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/3666235413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9241005c0 0x7fc9241009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.557+0000 7fc92b6cc640 1 -- 172.21.15.7:0/3666235413 >> 172.21.15.7:0/3666235413 conn(0x7fc9240fbb30 msgr2=0x7fc9240fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.557+0000 7fc92b6cc640 1 -- 172.21.15.7:0/3666235413 shutdown_connections 2024-03-26T08:54:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.557+0000 7fc92b6cc640 1 -- 172.21.15.7:0/3666235413 wait complete. 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc92b6cc640 1 Processor -- start 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc92b6cc640 1 -- start start 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc92b6cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9241005c0 0x7fc9241957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc92b6cc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9241017b0 0x7fc924195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc92b6cc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 0x7fc92419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc92b6cc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92410b140 con 0x7fc924102170 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc92b6cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92410afc0 con 0x7fc9241005c0 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc92b6cc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92410b2c0 con 0x7fc9241017b0 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929c69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 0x7fc92419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929468640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9241005c0 0x7fc9241957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc928c67640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9241017b0 0x7fc924195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929c69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 0x7fc92419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57096/0 (socket says 172.21.15.7:57096) 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929c69640 1 -- 172.21.15.7:0/844005221 learned_addr learned my addr 172.21.15.7:0/844005221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929468640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9241005c0 0x7fc9241957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57046/0 (socket says 172.21.15.7:57046) 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929c69640 1 -- 172.21.15.7:0/844005221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9241017b0 msgr2=0x7fc924195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929c69640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9241017b0 0x7fc924195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929c69640 1 -- 172.21.15.7:0/844005221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9241005c0 msgr2=0x7fc9241957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929c69640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9241005c0 0x7fc9241957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929c69640 1 -- 172.21.15.7:0/844005221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92419d3f0 con 0x7fc924102170 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.558+0000 7fc929468640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9241005c0 0x7fc9241957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.559+0000 7fc929c69640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 0x7fc92419cd80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc914037040 tx=0x7fc914032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.559+0000 7fc9227fc640 1 -- 172.21.15.7:0/844005221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914041020 con 0x7fc924102170 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.559+0000 7fc9227fc640 1 -- 172.21.15.7:0/844005221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc914034cd0 con 0x7fc924102170 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.559+0000 7fc92b6cc640 1 -- 172.21.15.7:0/844005221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92419d680 con 0x7fc924102170 2024-03-26T08:54:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.559+0000 7fc9227fc640 1 -- 172.21.15.7:0/844005221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914039040 con 0x7fc924102170 2024-03-26T08:54:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.559+0000 7fc92b6cc640 1 -- 172.21.15.7:0/844005221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92419db70 con 0x7fc924102170 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc92e38a740 1 Processor -- start 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc9227fc640 1 -- 172.21.15.7:0/844005221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc914044020 con 0x7fc924102170 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc92e38a740 1 -- start start 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f401c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f40790 0x557937f45290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f47d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc92e38a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557937ea53d0 con 0x557937f45900 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc92e38a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557937e4dad0 con 0x557937f3fda0 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc92e38a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557937ea1470 con 0x557937f40790 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc928c67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc928c67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57060/0 (socket says 172.21.15.7:57060) 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc928c67640 1 -- 172.21.15.7:0/1503296624 learned_addr learned my addr 172.21.15.7:0/1503296624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc929468640 1 --2- 172.21.15.7:0/1503296624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f40790 0x557937f45290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.562+0000 7fc928c67640 1 --2- 172.21.15.7:0/1503296624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f47d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc928c67640 1 -- 172.21.15.7:0/1503296624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f40790 msgr2=0x557937f45290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc928c67640 1 --2- 172.21.15.7:0/1503296624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f40790 0x557937f45290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc928c67640 1 -- 172.21.15.7:0/1503296624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 msgr2=0x557937f47d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc928c67640 1 --2- 172.21.15.7:0/1503296624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f47d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc928c67640 1 -- 172.21.15.7:0/1503296624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557937c837d0 con 0x557937f3fda0 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc928c67640 1 --2- 172.21.15.7:0/1503296624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f401c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc918011810 tx=0x7fc9180097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d187fe9329331b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc9227fc640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc90c0a4a90 0x7fc90c0a6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc90b7fe640 1 -- 172.21.15.7:0/1503296624 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918004030 con 0x557937f3fda0 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc929468640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc90c0a4a90 0x7fc90c0a6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc90b7fe640 1 -- 172.21.15.7:0/1503296624 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc918011e50 con 0x557937f3fda0 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc90b7fe640 1 -- 172.21.15.7:0/1503296624 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9180038e0 con 0x557937f3fda0 2024-03-26T08:54:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.563+0000 7fc9227fc640 1 -- 172.21.15.7:0/844005221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc9140f0810 con 0x7fc924102170 2024-03-26T08:54:51.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 -- 172.21.15.7:0/1503296624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 msgr2=0x557937f401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1503296624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f401c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc918011810 tx=0x7fc9180097d0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc929468640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc90c0a4a90 0x7fc90c0a6f50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x557937f40b90 tx=0x7fc91c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 -- 172.21.15.7:0/1503296624 shutdown_connections 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1503296624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f47d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1503296624 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f40790 0x557937f45290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1503296624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f401c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 -- 172.21.15.7:0/1503296624 >> 172.21.15.7:0/1503296624 conn(0x557937f3b310 msgr2=0x557937f3b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 -- 172.21.15.7:0/1503296624 shutdown_connections 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 -- 172.21.15.7:0/1503296624 wait complete. 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 -- start start 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.564+0000 7fc92e38a740 1 Processor -- start 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 0x557937f5b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f5b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f62820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc92e38a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557937f598c0 con 0x557937f45900 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc92e38a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557937e61060 con 0x557937f40790 2024-03-26T08:54:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc92e38a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557937f59a40 con 0x557937f3fda0 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc928c67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f5b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc928c67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f5b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57070/0 (socket says 172.21.15.7:57070) 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc92e38a740 10 client.0 _ll_register_callbacks cb 0x557937e436a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc928c67640 1 -- 172.21.15.7:0/1428956574 learned_addr learned my addr 172.21.15.7:0/1428956574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc929c69640 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f62820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc929468640 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 0x557937f5b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.566+0000 7fc928c67640 1 -- 172.21.15.7:0/1428956574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 msgr2=0x557937f5b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc928c67640 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 0x557937f5b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc928c67640 1 -- 172.21.15.7:0/1428956574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 msgr2=0x557937f62820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc928c67640 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f62820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc928c67640 1 -- 172.21.15.7:0/1428956574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557937bb93f0 con 0x557937f40790 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc929c69640 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f62820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc929468640 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 0x557937f5b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc928c67640 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f5b7e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc918009660 tx=0x7fc918005520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3569360647464aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc908ff9640 1 -- 172.21.15.7:0/1428956574 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91800f9a0 con 0x557937f40790 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc908ff9640 1 -- 172.21.15.7:0/1428956574 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc91800fb40 con 0x557937f40790 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc92e38a740 1 -- 172.21.15.7:0/1428956574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc918011e50 con 0x557937f40790 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc908ff9640 1 -- 172.21.15.7:0/1428956574 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc91800fd40 con 0x557937f40790 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc92e38a740 10 client.28451 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc92e38a740 1 -- 172.21.15.7:0/1428956574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557937b790f0 con 0x557937f40790 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc908ff9640 1 -- 172.21.15.7:0/1428956574 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc918011e50 con 0x557937f40790 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.567+0000 7fc908ff9640 1 -- 172.21.15.7:0/1428956574 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc91801f5b0 con 0x557937f40790 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.568+0000 7fc92e38a740 10 client.28451 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.568+0000 7fc92e38a740 10 client.28451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.568+0000 7fc92e38a740 20 client.28451 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.568+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557937f6aea0 0x557937f6d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.568+0000 7fc92e38a740 4 client.28451 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.568+0000 7fc92e38a740 1 -- 172.21.15.7:0/1428956574 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557937eb2910 con 0x557937f6aea0 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.568+0000 7fc929468640 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557937f6aea0 0x557937f6d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.568+0000 7fc929468640 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557937f6aea0 0x557937f6d2c0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d2acfb116bc1d3 server_cookie=3349db8c1eb2d260 in_seq=0 out_seq=0 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.568+0000 7fc908ff9640 10 client.28451 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc908ff9640 1 -- 172.21.15.7:0/1428956574 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557937eb2910 con 0x557937f6aea0 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc908ff9640 10 client.28451 handle_command_reply: tid=0 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 client.28451 shutdown 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 -- 172.21.15.7:0/1428956574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 msgr2=0x557937f5b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f5b7e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc918009660 tx=0x7fc918005520 comp rx=0 tx=0).stop 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 -- 172.21.15.7:0/1428956574 shutdown_connections 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557937f6aea0 0x557937f6d2c0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f62820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f5b7e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc908ff9640 0 client.28451 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 --2- 172.21.15.7:0/1428956574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 0x557937f5b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 -- 172.21.15.7:0/1428956574 >> 172.21.15.7:0/1428956574 conn(0x557937f3b310 msgr2=0x557937f3a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 -- 172.21.15.7:0/1428956574 shutdown_connections 2024-03-26T08:54:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 1 -- 172.21.15.7:0/1428956574 wait complete. 2024-03-26T08:54:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.569+0000 7fc92e38a740 20 client.28451 trim_cache size 0 max 16384 2024-03-26T08:54:51.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:54:51.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:54:51.571 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:54:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc92e38a740 1 Processor -- start 2024-03-26T08:54:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc92e38a740 1 -- start start 2024-03-26T08:54:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f3ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f40790 0x557937f627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f45900 0x557937f71bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc92e38a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9241a0640 con 0x557937f40790 2024-03-26T08:54:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc92e38a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419e140 con 0x557937f3fda0 2024-03-26T08:54:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc92e38a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc924071cb0 con 0x557937f45900 2024-03-26T08:54:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc928c67640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f40790 0x557937f627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc928c67640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f40790 0x557937f627e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57132/0 (socket says 172.21.15.7:57132) 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc928c67640 1 -- 172.21.15.7:0/4261177894 learned_addr learned my addr 172.21.15.7:0/4261177894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc929468640 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f3ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc929c69640 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f45900 0x557937f71bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc928c67640 1 -- 172.21.15.7:0/4261177894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f45900 msgr2=0x557937f71bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc928c67640 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f45900 0x557937f71bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc928c67640 1 -- 172.21.15.7:0/4261177894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 msgr2=0x557937f3ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc928c67640 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f3ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc928c67640 1 -- 172.21.15.7:0/4261177894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557937f6d940 con 0x557937f40790 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.590+0000 7fc929468640 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f3ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:51.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.591+0000 7fc928c67640 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f40790 0x557937f627e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc918001eb0 tx=0x7fc918011780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd58a49b97de3882 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.591+0000 7fc90affd640 1 -- 172.21.15.7:0/4261177894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918009780 con 0x557937f40790 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.591+0000 7fc90affd640 1 -- 172.21.15.7:0/4261177894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc918011e50 con 0x557937f40790 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.591+0000 7fc90affd640 1 -- 172.21.15.7:0/4261177894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918003920 con 0x557937f40790 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.592+0000 7fc92e38a740 1 -- 172.21.15.7:0/4261177894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f40790 msgr2=0x557937f627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.592+0000 7fc92e38a740 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f40790 0x557937f627e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc918001eb0 tx=0x7fc918011780 comp rx=0 tx=0).stop 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.592+0000 7fc92e38a740 1 -- 172.21.15.7:0/4261177894 shutdown_connections 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.592+0000 7fc92e38a740 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f45900 0x557937f71bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.592+0000 7fc92e38a740 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f40790 0x557937f627e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.592+0000 7fc92e38a740 1 --2- 172.21.15.7:0/4261177894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f3fda0 0x557937f3ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.592+0000 7fc92e38a740 1 -- 172.21.15.7:0/4261177894 >> 172.21.15.7:0/4261177894 conn(0x557937f3b310 msgr2=0x557937f3e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:51.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 -- 172.21.15.7:0/4261177894 shutdown_connections 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 -- 172.21.15.7:0/4261177894 wait complete. 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 -- start start 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 Processor -- start 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 0x557937f60170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240722b0 con 0x557937f45900 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240735d0 con 0x557937f40790 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc924072910 con 0x557937f3fda0 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc92e38a740 10 client.0 _ll_register_callbacks cb 0x7fc918009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc929c69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc929c69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57138/0 (socket says 172.21.15.7:57138) 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc929c69640 1 -- 172.21.15.7:0/508923322 learned_addr learned my addr 172.21.15.7:0/508923322 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc928c67640 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.593+0000 7fc929468640 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 0x557937f60170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc929c69640 1 -- 172.21.15.7:0/508923322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 msgr2=0x557937f60170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc929c69640 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 0x557937f60170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc929c69640 1 -- 172.21.15.7:0/508923322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 msgr2=0x557937f606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc929c69640 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc929c69640 1 -- 172.21.15.7:0/508923322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557937f61cb0 con 0x557937f45900 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc928c67640 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc929c69640 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f616c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc9140bcf00 tx=0x7fc91402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff39de9a63d35ba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc908ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc908ff9640 1 -- 172.21.15.7:0/508923322 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140bd900 con 0x557937f45900 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc908ff9640 1 -- 172.21.15.7:0/508923322 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc914003970 con 0x557937f45900 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc92e38a740 1 -- 172.21.15.7:0/508923322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc918011e50 con 0x557937f45900 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc908ff9640 1 -- 172.21.15.7:0/508923322 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9140bd200 con 0x557937f45900 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc92e38a740 10 client.18660 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc92e38a740 1 -- 172.21.15.7:0/508923322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557937fc6aa0 con 0x557937f45900 2024-03-26T08:54:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.594+0000 7fc908ff9640 1 -- 172.21.15.7:0/508923322 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc918011e50 con 0x557937f45900 2024-03-26T08:54:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.595+0000 7fc908ff9640 1 -- 172.21.15.7:0/508923322 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc91403c030 con 0x557937f45900 2024-03-26T08:54:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.595+0000 7fc92e38a740 10 client.18660 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.595+0000 7fc92e38a740 10 client.18660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.595+0000 7fc92e38a740 20 client.18660 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.595+0000 7fc92e38a740 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557937fc7450 0x557937fc9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.595+0000 7fc92e38a740 4 client.18660 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:54:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.595+0000 7fc92e38a740 1 -- 172.21.15.7:0/508923322 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557937f5b090 con 0x557937fc7450 2024-03-26T08:54:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.595+0000 7fc929468640 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557937fc7450 0x557937fc9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.596+0000 7fc929468640 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557937fc7450 0x557937fc9870 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd6b9e7b734dae4 server_cookie=91a9d4048297f7c2 in_seq=0 out_seq=0 2024-03-26T08:54:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.596+0000 7fc908ff9640 10 client.18660 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.596+0000 7fc908ff9640 1 -- 172.21.15.7:0/508923322 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557937f5b090 con 0x557937fc7450 2024-03-26T08:54:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.596+0000 7fc908ff9640 10 client.18660 handle_command_reply: tid=0 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.596+0000 7fc92e38a740 1 client.18660 shutdown 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.596+0000 7fc92e38a740 1 -- 172.21.15.7:0/508923322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 msgr2=0x557937f616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.596+0000 7fc92e38a740 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f616c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc9140bcf00 tx=0x7fc91402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92e38a740 1 -- 172.21.15.7:0/508923322 shutdown_connections 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92e38a740 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x557937fc7450 0x557937fc9870 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92e38a740 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557937f45900 0x557937f616c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92e38a740 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557937f40790 0x557937f606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc908ff9640 0 client.18660 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92e38a740 1 --2- 172.21.15.7:0/508923322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557937f3fda0 0x557937f60170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92e38a740 1 -- 172.21.15.7:0/508923322 >> 172.21.15.7:0/508923322 conn(0x557937f3b310 msgr2=0x557937f3ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92e38a740 1 -- 172.21.15.7:0/508923322 shutdown_connections 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92e38a740 1 -- 172.21.15.7:0/508923322 wait complete. 2024-03-26T08:54:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92e38a740 20 client.18660 trim_cache size 0 max 16384 2024-03-26T08:54:51.599 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:54:51.599 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:54:51.599 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:54:51.599 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:54:51.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 -- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc90c0a4a90 msgr2=0x7fc90c0a6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc90c0a4a90 0x7fc90c0a6f50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x557937f40b90 tx=0x7fc91c005f90 comp rx=0 tx=0).stop 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 -- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 msgr2=0x7fc92419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 0x7fc92419cd80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc914037040 tx=0x7fc914032ea0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 -- 172.21.15.7:0/844005221 shutdown_connections 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc90c0a4a90 0x7fc90c0a6f50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc924102170 0x7fc92419cd80 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc9241017b0 0x7fc924195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 --2- 172.21.15.7:0/844005221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9241005c0 0x7fc9241957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 -- 172.21.15.7:0/844005221 >> 172.21.15.7:0/844005221 conn(0x7fc9240fbb30 msgr2=0x7fc924104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 -- 172.21.15.7:0/844005221 shutdown_connections 2024-03-26T08:54:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.597+0000 7fc92b6cc640 1 -- 172.21.15.7:0/844005221 wait complete. 2024-03-26T08:54:51.612 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:54:51.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:54:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:54:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:54:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.793+0000 7fe221397640 1 -- 172.21.15.7:0/2254150449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c1017d0 msgr2=0x7fe21c101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.793+0000 7fe221397640 1 --2- 172.21.15.7:0/2254150449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c1017d0 0x7fe21c101c50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe204009860 tx=0x7fe20402bb70 comp rx=0 tx=0).stop 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.793+0000 7fe221397640 1 -- 172.21.15.7:0/2254150449 shutdown_connections 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.793+0000 7fe221397640 1 --2- 172.21.15.7:0/2254150449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe21c102190 0x7fe21c108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.793+0000 7fe221397640 1 --2- 172.21.15.7:0/2254150449 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c1017d0 0x7fe21c101c50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.793+0000 7fe221397640 1 --2- 172.21.15.7:0/2254150449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21c1005e0 0x7fe21c100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.793+0000 7fe221397640 1 -- 172.21.15.7:0/2254150449 >> 172.21.15.7:0/2254150449 conn(0x7fe21c0fbb30 msgr2=0x7fe21c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.793+0000 7fe221397640 1 -- 172.21.15.7:0/2254150449 shutdown_connections 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.793+0000 7fe221397640 1 -- 172.21.15.7:0/2254150449 wait complete. 2024-03-26T08:54:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe221397640 1 Processor -- start 2024-03-26T08:54:51.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe221397640 1 -- start start 2024-03-26T08:54:51.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe221397640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c1005e0 0x7fe21c195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe221397640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21c1017d0 0x7fe21c195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe221397640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe21c102190 0x7fe21c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe221397640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c10b160 con 0x7fe21c1017d0 2024-03-26T08:54:51.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe221397640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c10afe0 con 0x7fe21c102190 2024-03-26T08:54:51.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe221397640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c10b2e0 con 0x7fe21c1005e0 2024-03-26T08:54:51.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21c1017d0 0x7fe21c195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21c1017d0 0x7fe21c195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57150/0 (socket says 172.21.15.7:57150) 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe21affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c1005e0 0x7fe21c195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe21b7fe640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe21c102190 0x7fe21c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.794+0000 7fe21a7fc640 1 -- 172.21.15.7:0/3729471713 learned_addr learned my addr 172.21.15.7:0/3729471713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe21a7fc640 1 -- 172.21.15.7:0/3729471713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c1005e0 msgr2=0x7fe21c195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe21a7fc640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c1005e0 0x7fe21c195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe21a7fc640 1 -- 172.21.15.7:0/3729471713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe21c102190 msgr2=0x7fe21c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe21a7fc640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe21c102190 0x7fe21c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe21a7fc640 1 -- 172.21.15.7:0/3729471713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c19d3b0 con 0x7fe21c1017d0 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe21b7fe640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe21c102190 0x7fe21c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe21a7fc640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21c1017d0 0x7fe21c195cd0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe204005940 tx=0x7fe2040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe213fff640 1 -- 172.21.15.7:0/3729471713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204041020 con 0x7fe21c1017d0 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c19d640 con 0x7fe21c1017d0 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe213fff640 1 -- 172.21.15.7:0/3729471713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2040347b0 con 0x7fe21c1017d0 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe213fff640 1 -- 172.21.15.7:0/3729471713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204039040 con 0x7fe21c1017d0 2024-03-26T08:54:51.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.795+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c19db80 con 0x7fe21c1017d0 2024-03-26T08:54:51.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.798+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21c107900 con 0x7fe21c1017d0 2024-03-26T08:54:51.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.798+0000 7fe213fff640 1 -- 172.21.15.7:0/3729471713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe204044020 con 0x7fe21c1017d0 2024-03-26T08:54:51.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.799+0000 7fe213fff640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe2080a2800 0x7fe2080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:51.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.799+0000 7fe213fff640 1 -- 172.21.15.7:0/3729471713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe2040f0540 con 0x7fe21c1017d0 2024-03-26T08:54:51.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.799+0000 7fe21affd640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe2080a2800 0x7fe2080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:51.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.799+0000 7fe21affd640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe2080a2800 0x7fe2080a4cc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe21c101630 tx=0x7fe21400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:51.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:51.804+0000 7fe213fff640 1 -- 172.21.15.7:0/3729471713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe20404b020 con 0x7fe21c1017d0 2024-03-26T08:54:52.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.130+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe21c0622a0 con 0x7fe21c1017d0 2024-03-26T08:54:52.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.130+0000 7fe213fff640 1 -- 172.21.15.7:0/3729471713 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fe2040bd5e0 con 0x7fe21c1017d0 2024-03-26T08:54:52.128 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:54:52.128 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:54:52.128 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:54:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe2080a2800 msgr2=0x7fe2080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe2080a2800 0x7fe2080a4cc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe21c101630 tx=0x7fe21400b500 comp rx=0 tx=0).stop 2024-03-26T08:54:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21c1017d0 msgr2=0x7fe21c195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21c1017d0 0x7fe21c195cd0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fe204005940 tx=0x7fe2040042f0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 shutdown_connections 2024-03-26T08:54:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe2080a2800 0x7fe2080a4cc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe21c102190 0x7fe21c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21c1017d0 0x7fe21c195cd0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 --2- 172.21.15.7:0/3729471713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe21c1005e0 0x7fe21c195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 >> 172.21.15.7:0/3729471713 conn(0x7fe21c0fbb30 msgr2=0x7fe21c104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 shutdown_connections 2024-03-26T08:54:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.133+0000 7fe221397640 1 -- 172.21.15.7:0/3729471713 wait complete. 2024-03-26T08:54:52.144 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:54:52.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:54:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 -- 172.21.15.7:0/3838113390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 msgr2=0x7f9fd01009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/3838113390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 0x7f9fd01009e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9fc001a910 tx=0x7f9fc003cc70 comp rx=0 tx=0).stop 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 -- 172.21.15.7:0/3838113390 shutdown_connections 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/3838113390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd0102170 0x7f9fd0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/3838113390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd01017b0 0x7f9fd0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/3838113390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 0x7f9fd01009e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 -- 172.21.15.7:0/3838113390 >> 172.21.15.7:0/3838113390 conn(0x7f9fd00fbb30 msgr2=0x7f9fd00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 -- 172.21.15.7:0/3838113390 shutdown_connections 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 -- 172.21.15.7:0/3838113390 wait complete. 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.290+0000 7f9fd6b53640 1 Processor -- start 2024-03-26T08:54:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd6b53640 1 -- start start 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd6b53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 0x7f9fd0107c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd6b53640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd01017b0 0x7f9fd01081c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd6b53640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd0102170 0x7f9fd0194600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd6b53640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd006cc40 con 0x7f9fd0102170 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd6b53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd006cac0 con 0x7f9fd01005c0 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd6b53640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd006cdc0 con 0x7f9fd01017b0 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd48ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 0x7f9fd0107c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd50f0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd0102170 0x7f9fd0194600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd48ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 0x7f9fd0107c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57136/0 (socket says 172.21.15.7:57136) 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fd48ef640 1 -- 172.21.15.7:0/1346855894 learned_addr learned my addr 172.21.15.7:0/1346855894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.291+0000 7f9fcffff640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd01017b0 0x7f9fd01081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fd50f0640 1 -- 172.21.15.7:0/1346855894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd01017b0 msgr2=0x7f9fd01081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fd50f0640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd01017b0 0x7f9fd01081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fd50f0640 1 -- 172.21.15.7:0/1346855894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 msgr2=0x7f9fd0107c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fd50f0640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 0x7f9fd0107c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fd50f0640 1 -- 172.21.15.7:0/1346855894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd0199080 con 0x7f9fd0102170 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 0x7f9fd0107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fd50f0640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd0102170 0x7f9fd0194600 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800e940 tx=0x7f9fc800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fcdffb640 1 -- 172.21.15.7:0/1346855894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8004d20 con 0x7f9fd0102170 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fd6b53640 1 -- 172.21.15.7:0/1346855894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd0199370 con 0x7f9fd0102170 2024-03-26T08:54:52.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.292+0000 7f9fcdffb640 1 -- 172.21.15.7:0/1346855894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc8005020 con 0x7f9fd0102170 2024-03-26T08:54:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.293+0000 7f9fd6b53640 1 -- 172.21.15.7:0/1346855894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd0199990 con 0x7f9fd0102170 2024-03-26T08:54:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.294+0000 7f9fcdffb640 1 -- 172.21.15.7:0/1346855894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80116b0 con 0x7f9fd0102170 2024-03-26T08:54:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.296+0000 7f9fcdffb640 1 -- 172.21.15.7:0/1346855894 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9fc80054d0 con 0x7f9fd0102170 2024-03-26T08:54:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.296+0000 7f9fd981b740 1 Processor -- start 2024-03-26T08:54:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.296+0000 7f9fd981b740 1 -- start start 2024-03-26T08:54:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.296+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.296+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f4eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.296+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f51680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.296+0000 7f9fd981b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d2f0c610 con 0x5593d2f4a080 2024-03-26T08:54:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.296+0000 7f9fd981b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d2eaf3c0 con 0x5593d2f49690 2024-03-26T08:54:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.296+0000 7f9fd981b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d2e57d60 con 0x5593d2f4f1f0 2024-03-26T08:54:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.297+0000 7f9fd48ef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f4eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.297+0000 7f9fd48ef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f4eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57190/0 (socket says 172.21.15.7:57190) 2024-03-26T08:54:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.297+0000 7f9fd48ef640 1 -- 172.21.15.7:0/39767543 learned_addr learned my addr 172.21.15.7:0/39767543 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.297+0000 7f9fd48ef640 1 -- 172.21.15.7:0/39767543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 msgr2=0x5593d2f51680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.297+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/39767543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f51680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.297+0000 7f9fd48ef640 1 -- 172.21.15.7:0/39767543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 msgr2=0x5593d2f49ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.297+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/39767543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f49ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.297+0000 7f9fd48ef640 1 -- 172.21.15.7:0/39767543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593d2bc33f0 con 0x5593d2f4a080 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.297+0000 7f9fcdffb640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fb80a27c0 0x7f9fb80a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.298+0000 7f9fcdffb640 1 -- 172.21.15.7:0/1346855894 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9fc80ced80 con 0x7f9fd0102170 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.298+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/39767543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f4eb80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9fc003d280 tx=0x7f9fc0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b153baea9adcca5e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.298+0000 7f9fcffff640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fb80a27c0 0x7f9fb80a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.298+0000 7f9fb6ffd640 1 -- 172.21.15.7:0/39767543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0004040 con 0x5593d2f4a080 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.298+0000 7f9fb6ffd640 1 -- 172.21.15.7:0/39767543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc00041e0 con 0x5593d2f4a080 2024-03-26T08:54:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.298+0000 7f9fb6ffd640 1 -- 172.21.15.7:0/39767543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00457b0 con 0x5593d2f4a080 2024-03-26T08:54:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.299+0000 7f9fd981b740 1 -- 172.21.15.7:0/39767543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 msgr2=0x5593d2f4eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.299+0000 7f9fcffff640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fb80a27c0 0x7f9fb80a4c80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x5593d2f4a8e0 tx=0x7f9fc4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.299+0000 7f9fd981b740 1 --2- 172.21.15.7:0/39767543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f4eb80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9fc003d280 tx=0x7f9fc0005d50 comp rx=0 tx=0).stop 2024-03-26T08:54:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.299+0000 7f9fd981b740 1 -- 172.21.15.7:0/39767543 shutdown_connections 2024-03-26T08:54:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.299+0000 7f9fd981b740 1 --2- 172.21.15.7:0/39767543 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f51680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.299+0000 7f9fd981b740 1 --2- 172.21.15.7:0/39767543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f4eb80 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.299+0000 7f9fd981b740 1 --2- 172.21.15.7:0/39767543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f49ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.299+0000 7f9fd981b740 1 -- 172.21.15.7:0/39767543 >> 172.21.15.7:0/39767543 conn(0x5593d2f44c00 msgr2=0x5593d2f45000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.300+0000 7f9fd981b740 1 -- 172.21.15.7:0/39767543 shutdown_connections 2024-03-26T08:54:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.300+0000 7f9fd981b740 1 -- 172.21.15.7:0/39767543 wait complete. 2024-03-26T08:54:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.300+0000 7f9fd981b740 1 -- start start 2024-03-26T08:54:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.300+0000 7f9fd981b740 1 Processor -- start 2024-03-26T08:54:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f5c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f5c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f63630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fd981b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d2e6b240 con 0x5593d2f4a080 2024-03-26T08:54:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fd981b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d2eab460 con 0x5593d2f49690 2024-03-26T08:54:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fd981b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d2f52190 con 0x5593d2f4f1f0 2024-03-26T08:54:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f5c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f5c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57200/0 (socket says 172.21.15.7:57200) 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fcffff640 1 -- 172.21.15.7:0/3199777301 learned_addr learned my addr 172.21.15.7:0/3199777301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f5c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fd50f0640 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f63630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fcffff640 1 -- 172.21.15.7:0/3199777301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 msgr2=0x5593d2f63630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fcffff640 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f63630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fcffff640 1 -- 172.21.15.7:0/3199777301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 msgr2=0x5593d2f5c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fcffff640 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f5c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fcffff640 1 -- 172.21.15.7:0/3199777301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593d2c8d7d0 con 0x5593d2f4a080 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.301+0000 7f9fd981b740 10 client.0 _ll_register_callbacks cb 0x5593d2e6ac10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.302+0000 7f9fd981b740 1 -- 172.21.15.7:0/3199777301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fc00041e0 con 0x5593d2f4a080 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.302+0000 7f9fcffff640 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f5c5f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4015700 tx=0x7f9fc40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7041f01030e95004 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.302+0000 7f9faffff640 10 client.18678 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.302+0000 7f9faffff640 1 -- 172.21.15.7:0/3199777301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4023020 con 0x5593d2f4a080 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.302+0000 7f9faffff640 1 -- 172.21.15.7:0/3199777301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9fc4013510 con 0x5593d2f4a080 2024-03-26T08:54:52.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.302+0000 7f9faffff640 1 -- 172.21.15.7:0/3199777301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fc401b040 con 0x5593d2f4a080 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.302+0000 7f9fd981b740 10 client.18678 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.302+0000 7f9fd981b740 1 -- 172.21.15.7:0/3199777301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x5593d2f66920 con 0x5593d2f4a080 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.302+0000 7f9faffff640 1 -- 172.21.15.7:0/3199777301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9fc4030020 con 0x5593d2f4a080 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.303+0000 7f9fd981b740 10 client.18678 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.303+0000 7f9fd981b740 10 client.18678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.303+0000 7f9fd981b740 20 client.18678 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.303+0000 7f9fd981b740 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5593d2f67f50 0x5593d2f6a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.303+0000 7f9fd981b740 4 client.18678 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.303+0000 7f9fd981b740 1 -- 172.21.15.7:0/3199777301 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593d2ebc900 con 0x5593d2f67f50 2024-03-26T08:54:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.303+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5593d2f67f50 0x5593d2f6a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.304+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5593d2f67f50 0x5593d2f6a3e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f503f335589c7b6d server_cookie=3e2d9138ca7ce88e in_seq=0 out_seq=0 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.304+0000 7f9faffff640 10 client.18678 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.304+0000 7f9faffff640 1 -- 172.21.15.7:0/3199777301 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5593d2ebc900 con 0x5593d2f67f50 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9faffff640 10 client.18678 handle_command_reply: tid=0 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 client.18678 shutdown 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 -- 172.21.15.7:0/3199777301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 msgr2=0x5593d2f5c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f5c5f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4015700 tx=0x7f9fc40047d0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 -- 172.21.15.7:0/3199777301 shutdown_connections 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5593d2f67f50 0x5593d2f6a3e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f63630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f5c5f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 --2- 172.21.15.7:0/3199777301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f5c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 -- 172.21.15.7:0/3199777301 >> 172.21.15.7:0/3199777301 conn(0x5593d2f44c00 msgr2=0x5593d2f43e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9faffff640 0 client.18678 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:54:52.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 -- 172.21.15.7:0/3199777301 shutdown_connections 2024-03-26T08:54:52.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 1 -- 172.21.15.7:0/3199777301 wait complete. 2024-03-26T08:54:52.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.305+0000 7f9fd981b740 20 client.18678 trim_cache size 0 max 16384 2024-03-26T08:54:52.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.319+0000 7f9fd981b740 1 Processor -- start 2024-03-26T08:54:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.320+0000 7f9fd981b740 1 -- start start 2024-03-26T08:54:52.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.320+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f49690 0x5593d2f69f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4a080 0x5593d2f635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f4f1f0 0x5593d2f43590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd981b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd01a0540 con 0x5593d2f49690 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd981b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019e040 con 0x5593d2f4f1f0 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd981b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019c0c0 con 0x5593d2f4a080 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd48ef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f49690 0x5593d2f69f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd50f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f4f1f0 0x5593d2f43590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4a080 0x5593d2f635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd48ef640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f49690 0x5593d2f69f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57204/0 (socket says 172.21.15.7:57204) 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd48ef640 1 -- 172.21.15.7:0/301461574 learned_addr learned my addr 172.21.15.7:0/301461574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fcffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4a080 0x5593d2f635f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58626/0 (socket says 172.21.15.7:58626) 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd50f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f4f1f0 0x5593d2f43590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57166/0 (socket says 172.21.15.7:57166) 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd48ef640 1 -- 172.21.15.7:0/301461574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4a080 msgr2=0x5593d2f635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/301461574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4a080 0x5593d2f635f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd48ef640 1 -- 172.21.15.7:0/301461574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f4f1f0 msgr2=0x5593d2f43590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/301461574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f4f1f0 0x5593d2f43590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd48ef640 1 -- 172.21.15.7:0/301461574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593d2f45970 con 0x5593d2f49690 2024-03-26T08:54:52.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/301461574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f49690 0x5593d2f69f60 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0005ec0 tx=0x7f9fc001a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89947b243ff15d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.321+0000 7f9fb67fc640 1 -- 172.21.15.7:0/301461574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00457b0 con 0x5593d2f49690 2024-03-26T08:54:52.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fb67fc640 1 -- 172.21.15.7:0/301461574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc0045e30 con 0x5593d2f49690 2024-03-26T08:54:52.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fb67fc640 1 -- 172.21.15.7:0/301461574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0064a90 con 0x5593d2f49690 2024-03-26T08:54:52.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fd981b740 1 -- 172.21.15.7:0/301461574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f49690 msgr2=0x5593d2f69f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fd981b740 1 --2- 172.21.15.7:0/301461574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f49690 0x5593d2f69f60 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0005ec0 tx=0x7f9fc001a270 comp rx=0 tx=0).stop 2024-03-26T08:54:52.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fd981b740 1 -- 172.21.15.7:0/301461574 shutdown_connections 2024-03-26T08:54:52.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fd981b740 1 --2- 172.21.15.7:0/301461574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f4f1f0 0x5593d2f43590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fd981b740 1 --2- 172.21.15.7:0/301461574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4a080 0x5593d2f635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fd981b740 1 --2- 172.21.15.7:0/301461574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f49690 0x5593d2f69f60 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fd981b740 1 -- 172.21.15.7:0/301461574 >> 172.21.15.7:0/301461574 conn(0x5593d2f44c00 msgr2=0x5593d2f4b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fd981b740 1 -- 172.21.15.7:0/301461574 shutdown_connections 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.322+0000 7f9fd981b740 1 -- 172.21.15.7:0/301461574 wait complete. 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.323+0000 7f9fd981b740 1 -- start start 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.323+0000 7f9fd981b740 1 Processor -- start 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.323+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.323+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f61720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.323+0000 7f9fd981b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f62730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.323+0000 7f9fd981b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019c6c0 con 0x5593d2f4a080 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.323+0000 7f9fd981b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019d9e0 con 0x5593d2f49690 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.323+0000 7f9fd981b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019cd20 con 0x5593d2f4f1f0 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.323+0000 7f9fd981b740 10 client.0 _ll_register_callbacks cb 0x7f9fc400eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd48ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd48ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57172/0 (socket says 172.21.15.7:57172) 2024-03-26T08:54:52.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd50f0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f62730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd48ef640 1 -- 172.21.15.7:0/2090930192 learned_addr learned my addr 172.21.15.7:0/2090930192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd50f0640 1 -- 172.21.15.7:0/2090930192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 msgr2=0x5593d2f611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd50f0640 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd50f0640 1 -- 172.21.15.7:0/2090930192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 msgr2=0x5593d2f61720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd50f0640 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f61720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd50f0640 1 -- 172.21.15.7:0/2090930192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593d2f62d20 con 0x5593d2f4f1f0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd50f0640 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f62730 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8005940 tx=0x7f9fc809b100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2007159ae56d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9faffff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9faffff640 1 -- 172.21.15.7:0/2090930192 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc801f040 con 0x5593d2f4f1f0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9fd981b740 1 -- 172.21.15.7:0/2090930192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fc0045e30 con 0x5593d2f4f1f0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.324+0000 7f9faffff640 1 -- 172.21.15.7:0/2090930192 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9fc80999e0 con 0x5593d2f4f1f0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9faffff640 1 -- 172.21.15.7:0/2090930192 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fc80a3e80 con 0x5593d2f4f1f0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9fd981b740 10 client.28471 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9fd981b740 1 -- 172.21.15.7:0/2090930192 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5593d2fc3d70 con 0x5593d2f4f1f0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9faffff640 1 -- 172.21.15.7:0/2090930192 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc0045e30 con 0x5593d2f4f1f0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9faffff640 1 -- 172.21.15.7:0/2090930192 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9fc800f040 con 0x5593d2f4f1f0 2024-03-26T08:54:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9fd981b740 10 client.28471 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9fd981b740 10 client.28471 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:54:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9fd981b740 20 client.28471 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9fd981b740 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5593d2fc47a0 0x5593d2fc6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9fd981b740 4 client.28471 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:54:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9fd981b740 1 -- 172.21.15.7:0/2090930192 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593d2eb6c50 con 0x5593d2fc47a0 2024-03-26T08:54:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.325+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5593d2fc47a0 0x5593d2fc6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9faffff640 10 client.28471 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:54:52.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd48ef640 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5593d2fc47a0 0x5593d2fc6bc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfecb2e499f9b741 server_cookie=d1a4782b430c669b in_seq=0 out_seq=0 2024-03-26T08:54:52.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9faffff640 1 -- 172.21.15.7:0/2090930192 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593d2eb6c50 con 0x5593d2fc47a0 2024-03-26T08:54:52.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9faffff640 10 client.28471 handle_command_reply: tid=0 2024-03-26T08:54:52.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 client.28471 shutdown 2024-03-26T08:54:52.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 -- 172.21.15.7:0/2090930192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 msgr2=0x5593d2f62730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f62730 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8005940 tx=0x7f9fc809b100 comp rx=0 tx=0).stop 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 -- 172.21.15.7:0/2090930192 shutdown_connections 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5593d2fc47a0 0x5593d2fc6bc0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5593d2f4f1f0 0x5593d2f62730 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5593d2f4a080 0x5593d2f61720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 --2- 172.21.15.7:0/2090930192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5593d2f49690 0x5593d2f611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 -- 172.21.15.7:0/2090930192 >> 172.21.15.7:0/2090930192 conn(0x5593d2f44c00 msgr2=0x5593d2f47e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9faffff640 0 client.28471 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 -- 172.21.15.7:0/2090930192 shutdown_connections 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 1 -- 172.21.15.7:0/2090930192 wait complete. 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.326+0000 7f9fd981b740 20 client.28471 trim_cache size 0 max 16384 2024-03-26T08:54:52.327 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:54:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 -- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fb80a27c0 msgr2=0x7f9fb80a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fb80a27c0 0x7f9fb80a4c80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x5593d2f4a8e0 tx=0x7f9fc4009110 comp rx=0 tx=0).stop 2024-03-26T08:54:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 -- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd0102170 msgr2=0x7f9fd0194600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd0102170 0x7f9fd0194600 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800e940 tx=0x7f9fc800ed40 comp rx=0 tx=0).stop 2024-03-26T08:54:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 -- 172.21.15.7:0/1346855894 shutdown_connections 2024-03-26T08:54:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f9fb80a27c0 0x7f9fb80a4c80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fd0102170 0x7f9fd0194600 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9fd01017b0 0x7f9fd01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 --2- 172.21.15.7:0/1346855894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd01005c0 0x7f9fd0107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 -- 172.21.15.7:0/1346855894 >> 172.21.15.7:0/1346855894 conn(0x7f9fd00fbb30 msgr2=0x7f9fd006f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 -- 172.21.15.7:0/1346855894 shutdown_connections 2024-03-26T08:54:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.327+0000 7f9fd6b53640 1 -- 172.21.15.7:0/1346855894 wait complete. 2024-03-26T08:54:52.338 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:54:52.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:54:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.466+0000 7fd2a7444640 1 -- 172.21.15.7:0/2671177340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2a0104a70 msgr2=0x7fd2a0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.466+0000 7fd2a7444640 1 --2- 172.21.15.7:0/2671177340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2a0104a70 0x7fd2a0106e60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd290009900 tx=0x7fd29002bc30 comp rx=0 tx=0).stop 2024-03-26T08:54:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.466+0000 7fd2a7444640 1 -- 172.21.15.7:0/2671177340 shutdown_connections 2024-03-26T08:54:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.466+0000 7fd2a7444640 1 --2- 172.21.15.7:0/2671177340 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a01073a0 0x7fd2a01097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.466+0000 7fd2a7444640 1 --2- 172.21.15.7:0/2671177340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2a0104a70 0x7fd2a0106e60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.466+0000 7fd2a7444640 1 --2- 172.21.15.7:0/2671177340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0102140 0x7fd2a0104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.466+0000 7fd2a7444640 1 -- 172.21.15.7:0/2671177340 >> 172.21.15.7:0/2671177340 conn(0x7fd2a00fbb30 msgr2=0x7fd2a00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.470+0000 7fd2a7444640 1 -- 172.21.15.7:0/2671177340 shutdown_connections 2024-03-26T08:54:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.471+0000 7fd2a7444640 1 -- 172.21.15.7:0/2671177340 wait complete. 2024-03-26T08:54:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.472+0000 7fd2a7444640 1 Processor -- start 2024-03-26T08:54:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.472+0000 7fd2a7444640 1 -- start start 2024-03-26T08:54:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.472+0000 7fd2a7444640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a0102140 0x7fd2a0072b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.473+0000 7fd2a7444640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0104a70 0x7fd2a00730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.473+0000 7fd2a7444640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2a01073a0 0x7fd2a006f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.473+0000 7fd2a7444640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a0100d30 con 0x7fd2a01073a0 2024-03-26T08:54:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.473+0000 7fd2a7444640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a0100bb0 con 0x7fd2a0104a70 2024-03-26T08:54:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.473+0000 7fd2a7444640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a0100eb0 con 0x7fd2a0102140 2024-03-26T08:54:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.475+0000 7fd2a51e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a0102140 0x7fd2a0072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.475+0000 7fd2a51e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a0102140 0x7fd2a0072b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58648/0 (socket says 172.21.15.7:58648) 2024-03-26T08:54:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.475+0000 7fd2a51e0640 1 -- 172.21.15.7:0/637141186 learned_addr learned my addr 172.21.15.7:0/637141186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.476+0000 7fd2a49df640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0104a70 0x7fd2a00730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.476+0000 7fd2a59e1640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2a01073a0 0x7fd2a006f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.476+0000 7fd2a59e1640 1 -- 172.21.15.7:0/637141186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a0102140 msgr2=0x7fd2a0072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.476+0000 7fd2a59e1640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a0102140 0x7fd2a0072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.477+0000 7fd2a59e1640 1 -- 172.21.15.7:0/637141186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0104a70 msgr2=0x7fd2a00730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.477+0000 7fd2a59e1640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0104a70 0x7fd2a00730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.477+0000 7fd2a59e1640 1 -- 172.21.15.7:0/637141186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a006fd30 con 0x7fd2a01073a0 2024-03-26T08:54:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.477+0000 7fd2a59e1640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2a01073a0 0x7fd2a006f5b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd298009fd0 tx=0x7fd29800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.478+0000 7fd2967fc640 1 -- 172.21.15.7:0/637141186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29800ce20 con 0x7fd2a01073a0 2024-03-26T08:54:52.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.478+0000 7fd2967fc640 1 -- 172.21.15.7:0/637141186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd298005020 con 0x7fd2a01073a0 2024-03-26T08:54:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.479+0000 7fd2967fc640 1 -- 172.21.15.7:0/637141186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298010720 con 0x7fd2a01073a0 2024-03-26T08:54:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.479+0000 7fd2a7444640 1 -- 172.21.15.7:0/637141186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a0070020 con 0x7fd2a01073a0 2024-03-26T08:54:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.479+0000 7fd2a7444640 1 -- 172.21.15.7:0/637141186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a0199860 con 0x7fd2a01073a0 2024-03-26T08:54:52.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.482+0000 7fd2aa02d740 1 Processor -- start 2024-03-26T08:54:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.482+0000 7fd2967fc640 1 -- 172.21.15.7:0/637141186 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd298004a00 con 0x7fd2a01073a0 2024-03-26T08:54:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.483+0000 7fd2aa02d740 1 -- start start 2024-03-26T08:54:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.483+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.483+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9cd890 0x56483d9d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.483+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9d2a00 0x56483d9d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.483+0000 7fd2aa02d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d987610 con 0x56483d9cd890 2024-03-26T08:54:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.483+0000 7fd2aa02d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d92a3c0 con 0x56483d9d2a00 2024-03-26T08:54:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.483+0000 7fd2aa02d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d8d2d60 con 0x56483d9ccea0 2024-03-26T08:54:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.483+0000 7fd2967fc640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd28c0a2830 0x7fd28c0a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.484+0000 7fd2967fc640 1 -- 172.21.15.7:0/637141186 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2980cf160 con 0x7fd2a01073a0 2024-03-26T08:54:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.491+0000 7fd2a59e1640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd28c0a2830 0x7fd28c0a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.492+0000 7fd2a49df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.492+0000 7fd2a49df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9d2a00 0x56483d9d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.492+0000 7fd2a49df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58662/0 (socket says 172.21.15.7:58662) 2024-03-26T08:54:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.492+0000 7fd2a49df640 1 -- 172.21.15.7:0/190905488 learned_addr learned my addr 172.21.15.7:0/190905488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.493+0000 7fd2a51e0640 1 --2- 172.21.15.7:0/190905488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9cd890 0x56483d9d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.493+0000 7fd2a59e1640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd28c0a2830 0x7fd28c0a4cf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd29809ba50 tx=0x7fd298005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.498+0000 7fd2a49df640 1 -- 172.21.15.7:0/190905488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9d2a00 msgr2=0x56483d9d4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.498+0000 7fd2a49df640 1 --2- 172.21.15.7:0/190905488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9d2a00 0x56483d9d4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.499+0000 7fd2a49df640 1 -- 172.21.15.7:0/190905488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9cd890 msgr2=0x56483d9d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.499+0000 7fd2a49df640 1 --2- 172.21.15.7:0/190905488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9cd890 0x56483d9d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.499+0000 7fd2a49df640 1 -- 172.21.15.7:0/190905488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56483d63e3f0 con 0x56483d9ccea0 2024-03-26T08:54:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.500+0000 7fd2a49df640 1 --2- 172.21.15.7:0/190905488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cd2c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd29002c9f0 tx=0x7fd29002c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=962cc52d51d9ad3d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.506+0000 7fd28b7fe640 1 -- 172.21.15.7:0/190905488 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29004a070 con 0x56483d9ccea0 2024-03-26T08:54:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.506+0000 7fd28b7fe640 1 -- 172.21.15.7:0/190905488 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd290005470 con 0x56483d9ccea0 2024-03-26T08:54:52.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.506+0000 7fd28b7fe640 1 -- 172.21.15.7:0/190905488 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290005220 con 0x56483d9ccea0 2024-03-26T08:54:52.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.507+0000 7fd2aa02d740 1 -- 172.21.15.7:0/190905488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 msgr2=0x56483d9cd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.507+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/190905488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cd2c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd29002c9f0 tx=0x7fd29002c7e0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.511+0000 7fd2aa02d740 1 -- 172.21.15.7:0/190905488 shutdown_connections 2024-03-26T08:54:52.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.511+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/190905488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9d2a00 0x56483d9d4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.511+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/190905488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9cd890 0x56483d9d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.511+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/190905488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cd2c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.511+0000 7fd2aa02d740 1 -- 172.21.15.7:0/190905488 >> 172.21.15.7:0/190905488 conn(0x56483d9c8410 msgr2=0x56483d9c8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.519+0000 7fd2aa02d740 1 -- 172.21.15.7:0/190905488 shutdown_connections 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.523+0000 7fd2aa02d740 1 -- 172.21.15.7:0/190905488 wait complete. 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.523+0000 7fd2aa02d740 1 -- start start 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.523+0000 7fd2aa02d740 1 Processor -- start 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.524+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9ccea0 0x56483d9d7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.524+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9cd890 0x56483d9d77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.524+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9d2a00 0x56483d9de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.524+0000 7fd2aa02d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d8e6240 con 0x56483d9d2a00 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.524+0000 7fd2aa02d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d926460 con 0x56483d9ccea0 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.524+0000 7fd2aa02d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56483d9d5950 con 0x56483d9cd890 2024-03-26T08:54:52.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.524+0000 7fd2aa02d740 10 client.0 _ll_register_callbacks cb 0x56483d8e5c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:52.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.526+0000 7fd2a49df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9d2a00 0x56483d9de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.526+0000 7fd2a49df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9d2a00 0x56483d9de800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57264/0 (socket says 172.21.15.7:57264) 2024-03-26T08:54:52.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.526+0000 7fd2a49df640 1 -- 172.21.15.7:0/553684441 learned_addr learned my addr 172.21.15.7:0/553684441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.526+0000 7fd2a49df640 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9ccea0 0x56483d9d7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.527+0000 7fd2a49df640 1 -- 172.21.15.7:0/553684441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9cd890 msgr2=0x56483d9d77c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:54:52.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.527+0000 7fd2a49df640 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9cd890 0x56483d9d77c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.527+0000 7fd2a49df640 1 -- 172.21.15.7:0/553684441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9ccea0 msgr2=0x56483d9d7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.527+0000 7fd2a49df640 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9ccea0 0x56483d9d7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.528+0000 7fd2a49df640 1 -- 172.21.15.7:0/553684441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56483d7087d0 con 0x56483d9d2a00 2024-03-26T08:54:52.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.528+0000 7fd2a49df640 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9d2a00 0x56483d9de800 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd290009a20 tx=0x7fd290005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77d34debc44cfd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.528+0000 7fd288ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:52.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.528+0000 7fd288ff9640 1 -- 172.21.15.7:0/553684441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29004a040 con 0x56483d9d2a00 2024-03-26T08:54:52.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.529+0000 7fd2aa02d740 1 -- 172.21.15.7:0/553684441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd290005470 con 0x56483d9d2a00 2024-03-26T08:54:52.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.529+0000 7fd288ff9640 1 -- 172.21.15.7:0/553684441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd290004230 con 0x56483d9d2a00 2024-03-26T08:54:52.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.529+0000 7fd288ff9640 1 -- 172.21.15.7:0/553684441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2900046c0 con 0x56483d9d2a00 2024-03-26T08:54:52.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.529+0000 7fd2aa02d740 10 client.18693 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:52.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.529+0000 7fd2aa02d740 1 -- 172.21.15.7:0/553684441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56483d9e1b70 con 0x56483d9d2a00 2024-03-26T08:54:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.534+0000 7fd288ff9640 1 -- 172.21.15.7:0/553684441 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd290005470 con 0x56483d9d2a00 2024-03-26T08:54:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.534+0000 7fd288ff9640 1 -- 172.21.15.7:0/553684441 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd29005c050 con 0x56483d9d2a00 2024-03-26T08:54:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.535+0000 7fd2aa02d740 10 client.18693 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.535+0000 7fd2aa02d740 10 client.18693 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:54:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.535+0000 7fd2aa02d740 20 client.18693 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.535+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56483d9e2ec0 0x56483d9e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.535+0000 7fd2aa02d740 4 client.18693 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.535+0000 7fd2aa02d740 1 -- 172.21.15.7:0/553684441 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56483d937900 con 0x56483d9e2ec0 2024-03-26T08:54:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.535+0000 7fd2a51e0640 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56483d9e2ec0 0x56483d9e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.540+0000 7fd2a51e0640 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56483d9e2ec0 0x56483d9e52b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5aab9d92073ea5ce server_cookie=71463a11ae0579fe in_seq=0 out_seq=0 2024-03-26T08:54:52.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.540+0000 7fd288ff9640 10 client.18693 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:54:52.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.541+0000 7fd288ff9640 1 -- 172.21.15.7:0/553684441 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56483d937900 con 0x56483d9e2ec0 2024-03-26T08:54:52.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.541+0000 7fd288ff9640 10 client.18693 handle_command_reply: tid=0 2024-03-26T08:54:52.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.542+0000 7fd2aa02d740 1 client.18693 shutdown 2024-03-26T08:54:52.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.545+0000 7fd2aa02d740 1 -- 172.21.15.7:0/553684441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9d2a00 msgr2=0x56483d9de800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.545+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9d2a00 0x56483d9de800 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd290009a20 tx=0x7fd290005e50 comp rx=0 tx=0).stop 2024-03-26T08:54:52.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.545+0000 7fd2aa02d740 1 -- 172.21.15.7:0/553684441 shutdown_connections 2024-03-26T08:54:52.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.545+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9d2a00 0x56483d9de800 secure :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd290009a20 tx=0x7fd290005e50 comp rx=0 tx=0).stop 2024-03-26T08:54:52.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.545+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56483d9e2ec0 0x56483d9e52b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.545+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9cd890 0x56483d9d77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.545+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/553684441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9ccea0 0x56483d9d7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.545+0000 7fd2aa02d740 1 -- 172.21.15.7:0/553684441 >> 172.21.15.7:0/553684441 conn(0x56483d9c8410 msgr2=0x56483d9bee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.546+0000 7fd288ff9640 0 client.18693 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:54:52.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.546+0000 7fd2aa02d740 1 -- 172.21.15.7:0/553684441 shutdown_connections 2024-03-26T08:54:52.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.546+0000 7fd2aa02d740 1 -- 172.21.15.7:0/553684441 wait complete. 2024-03-26T08:54:52.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.546+0000 7fd2aa02d740 20 client.18693 trim_cache size 0 max 16384 2024-03-26T08:54:52.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:52 smithi007 ceph-mon[32462]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-03-26T08:54:52.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:52 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3729471713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:54:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2aa02d740 1 Processor -- start 2024-03-26T08:54:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2aa02d740 1 -- start start 2024-03-26T08:54:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9cd890 0x56483d9cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9de790 0x56483d9be590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2a49df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2a49df640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58672/0 (socket says 172.21.15.7:58672) 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2a49df640 1 -- 172.21.15.7:0/3350785578 learned_addr learned my addr 172.21.15.7:0/3350785578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2a51e0640 1 --2- 172.21.15.7:0/3350785578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9cd890 0x56483d9cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2aa02d740 1 -- 172.21.15.7:0/3350785578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a01a0530 con 0x56483d9de790 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2aa02d740 1 -- 172.21.15.7:0/3350785578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a019e030 con 0x56483d9cd890 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2aa02d740 1 -- 172.21.15.7:0/3350785578 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a019c0f0 con 0x56483d9ccea0 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.562+0000 7fd2a49df640 1 --2- 172.21.15.7:0/3350785578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9de790 0x56483d9be590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.563+0000 7fd2a49df640 1 -- 172.21.15.7:0/3350785578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 msgr2=0x56483d9cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.563+0000 7fd2a49df640 1 --2- 172.21.15.7:0/3350785578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.563+0000 7fd2a49df640 1 -- 172.21.15.7:0/3350785578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9cd890 msgr2=0x56483d9cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.563+0000 7fd2a49df640 1 --2- 172.21.15.7:0/3350785578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9cd890 0x56483d9cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.563+0000 7fd2a49df640 1 -- 172.21.15.7:0/3350785578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56483d9e5760 con 0x56483d9de790 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.563+0000 7fd2a49df640 1 --2- 172.21.15.7:0/3350785578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9de790 0x56483d9be590 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd29002c9f0 tx=0x7fd29002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541fcf78185dbd93 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.563+0000 7fd28affd640 1 -- 172.21.15.7:0/3350785578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29004a040 con 0x56483d9de790 2024-03-26T08:54:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.563+0000 7fd28affd640 1 -- 172.21.15.7:0/3350785578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd29002c5d0 con 0x56483d9de790 2024-03-26T08:54:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.563+0000 7fd28affd640 1 -- 172.21.15.7:0/3350785578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290034880 con 0x56483d9de790 2024-03-26T08:54:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 -- 172.21.15.7:0/3350785578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9de790 msgr2=0x56483d9be590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/3350785578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9de790 0x56483d9be590 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd29002c9f0 tx=0x7fd29002bc30 comp rx=0 tx=0).stop 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 -- 172.21.15.7:0/3350785578 shutdown_connections 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/3350785578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9de790 0x56483d9be590 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/3350785578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9cd890 0x56483d9cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/3350785578 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9ccea0 0x56483d9cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 -- 172.21.15.7:0/3350785578 >> 172.21.15.7:0/3350785578 conn(0x56483d9c8410 msgr2=0x56483d9d7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 -- 172.21.15.7:0/3350785578 shutdown_connections 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 -- 172.21.15.7:0/3350785578 wait complete. 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 -- start start 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.564+0000 7fd2aa02d740 1 Processor -- start 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9ccea0 0x56483d9dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9cd890 0x56483d9dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2aa02d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9de790 0x56483d9dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2a49df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9ccea0 0x56483d9dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2a49df640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9ccea0 0x56483d9dbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57274/0 (socket says 172.21.15.7:57274) 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2a49df640 1 -- 172.21.15.7:0/2681647348 learned_addr learned my addr 172.21.15.7:0/2681647348 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2a51e0640 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9cd890 0x56483d9dc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2aa02d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a019c6f0 con 0x56483d9ccea0 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a019d9d0 con 0x56483d9de790 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a019cd10 con 0x56483d9cd890 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.565+0000 7fd2a49df640 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9de790 0x56483d9dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.566+0000 7fd2aa02d740 10 client.0 _ll_register_callbacks cb 0x7fd290009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.566+0000 7fd2a49df640 1 -- 172.21.15.7:0/2681647348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9cd890 msgr2=0x56483d9dc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.566+0000 7fd2a49df640 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9cd890 0x56483d9dc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.566+0000 7fd2a49df640 1 -- 172.21.15.7:0/2681647348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9de790 msgr2=0x56483d9dd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.566+0000 7fd2a49df640 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9de790 0x56483d9dd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.566+0000 7fd2a49df640 1 -- 172.21.15.7:0/2681647348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56483d9dd8c0 con 0x56483d9ccea0 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.566+0000 7fd2a49df640 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9ccea0 0x56483d9dbeb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd290004dd0 tx=0x7fd290043550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815de5c3f1b9c540 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.566+0000 7fd288ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.567+0000 7fd288ff9640 1 -- 172.21.15.7:0/2681647348 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290034810 con 0x56483d9ccea0 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.567+0000 7fd288ff9640 1 -- 172.21.15.7:0/2681647348 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2900349b0 con 0x56483d9ccea0 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.567+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd29002c5d0 con 0x56483d9ccea0 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.567+0000 7fd288ff9640 1 -- 172.21.15.7:0/2681647348 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd290003f60 con 0x56483d9ccea0 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.567+0000 7fd2aa02d740 10 client.18699 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.567+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56483da3e9a0 con 0x56483d9ccea0 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.568+0000 7fd288ff9640 1 -- 172.21.15.7:0/2681647348 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2900035f0 con 0x56483d9ccea0 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.568+0000 7fd288ff9640 1 -- 172.21.15.7:0/2681647348 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd290074050 con 0x56483d9ccea0 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.568+0000 7fd2aa02d740 10 client.18699 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.568+0000 7fd2aa02d740 10 client.18699 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:54:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.568+0000 7fd2aa02d740 20 client.18699 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.569+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56483da3f0b0 0x56483da414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.569+0000 7fd2aa02d740 4 client.18699 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:54:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.569+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fd29002c5d0 con 0x56483da3f0b0 2024-03-26T08:54:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.570+0000 7fd2a51e0640 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56483da3f0b0 0x56483da414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.572+0000 7fd288ff9640 10 client.18699 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:54:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.572+0000 7fd2a51e0640 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56483da3f0b0 0x56483da414d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af9589184663ebe1 server_cookie=cd6fe1daa2ed7be0 in_seq=0 out_seq=0 2024-03-26T08:54:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.572+0000 7fd288ff9640 1 -- 172.21.15.7:0/2681647348 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fd29002c5d0 con 0x56483da3f0b0 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.572+0000 7fd288ff9640 10 client.18699 handle_command_reply: tid=0 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.572+0000 7fd2aa02d740 1 client.18699 shutdown 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9ccea0 msgr2=0x56483d9dbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9ccea0 0x56483d9dbeb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fd290004dd0 tx=0x7fd290043550 comp rx=0 tx=0).stop 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 shutdown_connections 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56483d9de790 0x56483d9dd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56483da3f0b0 0x56483da414d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd288ff9640 0 client.18699 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56483d9cd890 0x56483d9dc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 --2- 172.21.15.7:0/2681647348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56483d9ccea0 0x56483d9dbeb0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 >> 172.21.15.7:0/2681647348 conn(0x56483d9c8410 msgr2=0x56483d9d7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 shutdown_connections 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 1 -- 172.21.15.7:0/2681647348 wait complete. 2024-03-26T08:54:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.573+0000 7fd2aa02d740 20 client.18699 trim_cache size 0 max 16384 2024-03-26T08:54:52.572 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:54:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 -- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd28c0a2830 msgr2=0x7fd28c0a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd28c0a2830 0x7fd28c0a4cf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd29809ba50 tx=0x7fd298005f70 comp rx=0 tx=0).stop 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 -- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2a01073a0 msgr2=0x7fd2a006f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2a01073a0 0x7fd2a006f5b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd298009fd0 tx=0x7fd29800ef10 comp rx=0 tx=0).stop 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 -- 172.21.15.7:0/637141186 shutdown_connections 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd28c0a2830 0x7fd28c0a4cf0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2a01073a0 0x7fd2a006f5b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a0104a70 0x7fd2a00730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 --2- 172.21.15.7:0/637141186 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd2a0102140 0x7fd2a0072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 -- 172.21.15.7:0/637141186 >> 172.21.15.7:0/637141186 conn(0x7fd2a00fbb30 msgr2=0x7fd2a01077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 -- 172.21.15.7:0/637141186 shutdown_connections 2024-03-26T08:54:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.574+0000 7fd2a7444640 1 -- 172.21.15.7:0/637141186 wait complete. 2024-03-26T08:54:52.583 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:54:52.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:54:52.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.744+0000 7ff5699b9640 1 -- 172.21.15.7:0/1680800986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff56410b010 msgr2=0x7ff56410d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.744+0000 7ff5699b9640 1 --2- 172.21.15.7:0/1680800986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff56410b010 0x7ff56410d480 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff554009860 tx=0x7ff55402bbb0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.744+0000 7ff5699b9640 1 -- 172.21.15.7:0/1680800986 shutdown_connections 2024-03-26T08:54:52.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.744+0000 7ff5699b9640 1 --2- 172.21.15.7:0/1680800986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff56410d9c0 0x7ff56410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.744+0000 7ff5699b9640 1 --2- 172.21.15.7:0/1680800986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff56410b010 0x7ff56410d480 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.744+0000 7ff5699b9640 1 --2- 172.21.15.7:0/1680800986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5641086a0 0x7ff56410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.744+0000 7ff5699b9640 1 -- 172.21.15.7:0/1680800986 >> 172.21.15.7:0/1680800986 conn(0x7ff56406e640 msgr2=0x7ff564070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.744+0000 7ff5699b9640 1 -- 172.21.15.7:0/1680800986 shutdown_connections 2024-03-26T08:54:52.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.744+0000 7ff5699b9640 1 -- 172.21.15.7:0/1680800986 wait complete. 2024-03-26T08:54:52.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.745+0000 7ff5699b9640 1 Processor -- start 2024-03-26T08:54:52.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.745+0000 7ff5699b9640 1 -- start start 2024-03-26T08:54:52.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.745+0000 7ff5699b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5641086a0 0x7ff56419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.745+0000 7ff5699b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56410b010 0x7ff56419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.745+0000 7ff5699b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff56410d9c0 0x7ff5641a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.745+0000 7ff5699b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff564111950 con 0x7ff5641086a0 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.745+0000 7ff5699b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641117d0 con 0x7ff56410b010 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.745+0000 7ff5699b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff564111ad0 con 0x7ff56410d9c0 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.746+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5641086a0 0x7ff56419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.746+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5641086a0 0x7ff56419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57296/0 (socket says 172.21.15.7:57296) 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.746+0000 7ff562ffd640 1 -- 172.21.15.7:0/511487024 learned_addr learned my addr 172.21.15.7:0/511487024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.746+0000 7ff562ffd640 1 -- 172.21.15.7:0/511487024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff56410d9c0 msgr2=0x7ff5641a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.746+0000 7ff562ffd640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff56410d9c0 0x7ff5641a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.746+0000 7ff562ffd640 1 -- 172.21.15.7:0/511487024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56410b010 msgr2=0x7ff56419e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.746+0000 7ff562ffd640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56410b010 0x7ff56419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.746+0000 7ff562ffd640 1 -- 172.21.15.7:0/511487024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5641a5c00 con 0x7ff5641086a0 2024-03-26T08:54:52.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.746+0000 7ff562ffd640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5641086a0 0x7ff56419dfe0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00cce0 tx=0x7ff55c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.747+0000 7ff5689b7640 1 -- 172.21.15.7:0/511487024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c004b20 con 0x7ff5641086a0 2024-03-26T08:54:52.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.747+0000 7ff5689b7640 1 -- 172.21.15.7:0/511487024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff55c004cc0 con 0x7ff5641086a0 2024-03-26T08:54:52.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.747+0000 7ff5689b7640 1 -- 172.21.15.7:0/511487024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00f7d0 con 0x7ff5641086a0 2024-03-26T08:54:52.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.747+0000 7ff5699b9640 1 -- 172.21.15.7:0/511487024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5641a5ef0 con 0x7ff5641086a0 2024-03-26T08:54:52.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.748+0000 7ff5699b9640 1 -- 172.21.15.7:0/511487024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5641a6300 con 0x7ff5641086a0 2024-03-26T08:54:52.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:52 smithi016 ceph-mon[35851]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-03-26T08:54:52.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:52 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3729471713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:54:52.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.751+0000 7ff5689b7640 1 -- 172.21.15.7:0/511487024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff55c00f970 con 0x7ff5641086a0 2024-03-26T08:54:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.752+0000 7ff5689b7640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff54c0a2a00 0x7ff54c0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.752+0000 7ff5689b7640 1 -- 172.21.15.7:0/511487024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff55c0cead0 con 0x7ff5641086a0 2024-03-26T08:54:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.752+0000 7ff5627fc640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff54c0a2a00 0x7ff54c0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff56c5a5740 1 Processor -- start 2024-03-26T08:54:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff56c5a5740 1 -- start start 2024-03-26T08:54:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345ce080 0x55d5345d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff5627fc640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff54c0a2a00 0x7ff54c0a4ec0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff5540349f0 tx=0x7ff5540056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345d31f0 0x55d5345d5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d534590610 con 0x55d5345cd690 2024-03-26T08:54:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5345333c0 con 0x55d5345ce080 2024-03-26T08:54:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5344dbd60 con 0x55d5345d31f0 2024-03-26T08:54:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff5627fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345ce080 0x55d5345d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57310/0 (socket says 172.21.15.7:57310) 2024-03-26T08:54:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.753+0000 7ff562ffd640 1 -- 172.21.15.7:0/1455016955 learned_addr learned my addr 172.21.15.7:0/1455016955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff5637fe640 1 --2- 172.21.15.7:0/1455016955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345d31f0 0x55d5345d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff562ffd640 1 -- 172.21.15.7:0/1455016955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345d31f0 msgr2=0x55d5345d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff562ffd640 1 --2- 172.21.15.7:0/1455016955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345d31f0 0x55d5345d5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff562ffd640 1 -- 172.21.15.7:0/1455016955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345ce080 msgr2=0x55d5345d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff562ffd640 1 --2- 172.21.15.7:0/1455016955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345ce080 0x55d5345d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff562ffd640 1 -- 172.21.15.7:0/1455016955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5342473f0 con 0x55d5345cd690 2024-03-26T08:54:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff562ffd640 1 --2- 172.21.15.7:0/1455016955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345cdab0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff55c097890 tx=0x7ff55c09ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d36a09b4612280 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff549ffb640 1 -- 172.21.15.7:0/1455016955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0b4070 con 0x55d5345cd690 2024-03-26T08:54:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff549ffb640 1 -- 172.21.15.7:0/1455016955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff55c0a3bd0 con 0x55d5345cd690 2024-03-26T08:54:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.754+0000 7ff549ffb640 1 -- 172.21.15.7:0/1455016955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c01d040 con 0x55d5345cd690 2024-03-26T08:54:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.755+0000 7ff56c5a5740 1 -- 172.21.15.7:0/1455016955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 msgr2=0x55d5345cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.755+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/1455016955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345cdab0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff55c097890 tx=0x7ff55c09ae90 comp rx=0 tx=0).stop 2024-03-26T08:54:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.756+0000 7ff56c5a5740 1 -- 172.21.15.7:0/1455016955 shutdown_connections 2024-03-26T08:54:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.756+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/1455016955 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345d31f0 0x55d5345d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.756+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/1455016955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345ce080 0x55d5345d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.756+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/1455016955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345cdab0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.756+0000 7ff56c5a5740 1 -- 172.21.15.7:0/1455016955 >> 172.21.15.7:0/1455016955 conn(0x55d5345c8c00 msgr2=0x55d5345c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.756+0000 7ff56c5a5740 1 -- 172.21.15.7:0/1455016955 shutdown_connections 2024-03-26T08:54:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.756+0000 7ff56c5a5740 1 -- 172.21.15.7:0/1455016955 wait complete. 2024-03-26T08:54:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.756+0000 7ff56c5a5740 1 -- start start 2024-03-26T08:54:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.756+0000 7ff56c5a5740 1 Processor -- start 2024-03-26T08:54:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345d7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345d7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff5627fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345d7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345d7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff5627fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345d7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57316/0 (socket says 172.21.15.7:57316) 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345d7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58722/0 (socket says 172.21.15.7:58722) 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5344ef240 con 0x55d5345ce080 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53452f460 con 0x55d5345d31f0 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5345d6140 con 0x55d5345cd690 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff56c5a5740 10 client.0 _ll_register_callbacks cb 0x55d5344eec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.757+0000 7ff5627fc640 1 -- 172.21.15.7:0/3373268429 learned_addr learned my addr 172.21.15.7:0/3373268429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff5637fe640 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345dedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff5627fc640 1 -- 172.21.15.7:0/3373268429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 msgr2=0x55d5345d7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff5627fc640 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345d7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff5627fc640 1 -- 172.21.15.7:0/3373268429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 msgr2=0x55d5345dedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff5627fc640 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345dedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff5627fc640 1 -- 172.21.15.7:0/3373268429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5343117d0 con 0x55d5345ce080 2024-03-26T08:54:52.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff562ffd640 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345d7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff5627fc640 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345d7d90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff554032570 tx=0x7ff5540344b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2a2f02e4427c36 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff542ffd640 1 -- 172.21.15.7:0/3373268429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55404f020 con 0x55d5345ce080 2024-03-26T08:54:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.758+0000 7ff542ffd640 1 -- 172.21.15.7:0/3373268429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff554004250 con 0x55d5345ce080 2024-03-26T08:54:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3373268429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff55c0a3bd0 con 0x55d5345ce080 2024-03-26T08:54:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff542ffd640 1 -- 172.21.15.7:0/3373268429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff554039040 con 0x55d5345ce080 2024-03-26T08:54:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff56c5a5740 10 client.18711 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3373268429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d5345e21b0 con 0x55d5345ce080 2024-03-26T08:54:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff542ffd640 1 -- 172.21.15.7:0/3373268429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff55c0a3bd0 con 0x55d5345ce080 2024-03-26T08:54:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff542ffd640 1 -- 172.21.15.7:0/3373268429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff55403c030 con 0x55d5345ce080 2024-03-26T08:54:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff56c5a5740 10 client.18711 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff56c5a5740 10 client.18711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff56c5a5740 20 client.18711 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d5345e3740 0x55d5345e5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff56c5a5740 4 client.18711 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.759+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3373268429 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d534540900 con 0x55d5345e3740 2024-03-26T08:54:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.760+0000 7ff562ffd640 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d5345e3740 0x55d5345e5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.761+0000 7ff562ffd640 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d5345e3740 0x55d5345e5b50 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac53db73398c0632 server_cookie=d3327f4863be4d95 in_seq=0 out_seq=0 2024-03-26T08:54:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.761+0000 7ff542ffd640 10 client.18711 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.762+0000 7ff542ffd640 1 -- 172.21.15.7:0/3373268429 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d534540900 con 0x55d5345e3740 2024-03-26T08:54:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.762+0000 7ff542ffd640 10 client.18711 handle_command_reply: tid=0 2024-03-26T08:54:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.762+0000 7ff56c5a5740 1 client.18711 shutdown 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3373268429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 msgr2=0x55d5345d7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345d7d90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff554032570 tx=0x7ff5540344b0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3373268429 shutdown_connections 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d5345e3740 0x55d5345e5b50 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345dedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345d7d90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3373268429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345d7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3373268429 >> 172.21.15.7:0/3373268429 conn(0x55d5345c8c00 msgr2=0x55d5345c7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff542ffd640 0 client.18711 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3373268429 shutdown_connections 2024-03-26T08:54:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3373268429 wait complete. 2024-03-26T08:54:52.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.763+0000 7ff56c5a5740 20 client.18711 trim_cache size 0 max 16384 2024-03-26T08:54:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff56c5a5740 1 Processor -- start 2024-03-26T08:54:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff56c5a5740 1 -- start start 2024-03-26T08:54:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641b16e0 con 0x55d5345ce080 2024-03-26T08:54:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641af1e0 con 0x55d5345d31f0 2024-03-26T08:54:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641ad260 con 0x55d5345cd690 2024-03-26T08:54:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5627fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5627fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345ded90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57326/0 (socket says 172.21.15.7:57326) 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5627fc640 1 -- 172.21.15.7:0/2410458658 learned_addr learned my addr 172.21.15.7:0/2410458658 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff562ffd640 1 --2- 172.21.15.7:0/2410458658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5637fe640 1 --2- 172.21.15.7:0/2410458658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5627fc640 1 -- 172.21.15.7:0/2410458658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 msgr2=0x55d5345cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5627fc640 1 --2- 172.21.15.7:0/2410458658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5627fc640 1 -- 172.21.15.7:0/2410458658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 msgr2=0x55d5345cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5627fc640 1 --2- 172.21.15.7:0/2410458658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5627fc640 1 -- 172.21.15.7:0/2410458658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5345ce460 con 0x55d5345ce080 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.778+0000 7ff5627fc640 1 --2- 172.21.15.7:0/2410458658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345ded90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff5540095f0 tx=0x7ff554034bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab27530418dbc7e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff5497fa640 1 -- 172.21.15.7:0/2410458658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55404f070 con 0x55d5345ce080 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff5497fa640 1 -- 172.21.15.7:0/2410458658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff554003e50 con 0x55d5345ce080 2024-03-26T08:54:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff5497fa640 1 -- 172.21.15.7:0/2410458658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554032dc0 con 0x55d5345ce080 2024-03-26T08:54:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff56c5a5740 1 -- 172.21.15.7:0/2410458658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 msgr2=0x55d5345ded90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/2410458658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345ded90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff5540095f0 tx=0x7ff554034bc0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff56c5a5740 1 -- 172.21.15.7:0/2410458658 shutdown_connections 2024-03-26T08:54:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/2410458658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/2410458658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345ce080 0x55d5345ded90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/2410458658 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345cd690 0x55d5345cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff56c5a5740 1 -- 172.21.15.7:0/2410458658 >> 172.21.15.7:0/2410458658 conn(0x55d5345c8c00 msgr2=0x55d5345cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.779+0000 7ff56c5a5740 1 -- 172.21.15.7:0/2410458658 shutdown_connections 2024-03-26T08:54:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 1 -- 172.21.15.7:0/2410458658 wait complete. 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 1 -- start start 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 1 Processor -- start 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345ce080 0x55d5345dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345ddc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641ad860 con 0x55d5345cd690 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641aeb80 con 0x55d5345d31f0 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641adec0 con 0x55d5345ce080 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff56c5a5740 10 client.0 _ll_register_callbacks cb 0x7ff554005740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345dc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57332/0 (socket says 172.21.15.7:57332) 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.780+0000 7ff562ffd640 1 -- 172.21.15.7:0/3445076739 learned_addr learned my addr 172.21.15.7:0/3445076739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff5627fc640 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345ce080 0x55d5345dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff5637fe640 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345ddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff562ffd640 1 -- 172.21.15.7:0/3445076739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345ce080 msgr2=0x55d5345dcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff562ffd640 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345ce080 0x55d5345dcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff562ffd640 1 -- 172.21.15.7:0/3445076739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 msgr2=0x55d5345ddc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff562ffd640 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345ddc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff562ffd640 1 -- 172.21.15.7:0/3445076739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5345de210 con 0x55d5345cd690 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff562ffd640 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345dc6d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff55c097890 tx=0x7ff55c00ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1697c7fe336923c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff542ffd640 1 -- 172.21.15.7:0/3445076739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c01d040 con 0x55d5345cd690 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3445076739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff554003e50 con 0x55d5345cd690 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff542ffd640 1 -- 172.21.15.7:0/3445076739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff55c09f170 con 0x55d5345cd690 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff542ffd640 1 -- 172.21.15.7:0/3445076739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff55c0b4040 con 0x55d5345cd690 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff56c5a5740 10 client.18717 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3445076739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d53463f310 con 0x55d5345cd690 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff542ffd640 1 -- 172.21.15.7:0/3445076739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff554003e50 con 0x55d5345cd690 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.781+0000 7ff542ffd640 1 -- 172.21.15.7:0/3445076739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff55c0ac050 con 0x55d5345cd690 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.782+0000 7ff56c5a5740 10 client.18717 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.782+0000 7ff56c5a5740 10 client.18717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.782+0000 7ff56c5a5740 20 client.18717 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.782+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d53463f880 0x55d534641c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.782+0000 7ff56c5a5740 4 client.18717 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.782+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3445076739 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d53453ac50 con 0x55d53463f880 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.782+0000 7ff5627fc640 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d53463f880 0x55d534641c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.782+0000 7ff5627fc640 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d53463f880 0x55d534641c70 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab1772c6f485bf36 server_cookie=a351698bde8d9989 in_seq=0 out_seq=0 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.782+0000 7ff542ffd640 10 client.18717 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff542ffd640 1 -- 172.21.15.7:0/3445076739 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d53453ac50 con 0x55d53463f880 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff542ffd640 10 client.18717 handle_command_reply: tid=0 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 client.18717 shutdown 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3445076739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 msgr2=0x55d5345dc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345dc6d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff55c097890 tx=0x7ff55c00ac50 comp rx=0 tx=0).stop 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3445076739 shutdown_connections 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d53463f880 0x55d534641c70 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff542ffd640 0 client.18717 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5345d31f0 0x55d5345ddc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d5345ce080 0x55d5345dcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 --2- 172.21.15.7:0/3445076739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d5345cd690 0x55d5345dc6d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3445076739 >> 172.21.15.7:0/3445076739 conn(0x55d5345c8c00 msgr2=0x55d5345cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3445076739 shutdown_connections 2024-03-26T08:54:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 1 -- 172.21.15.7:0/3445076739 wait complete. 2024-03-26T08:54:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.783+0000 7ff56c5a5740 20 client.18717 trim_cache size 0 max 16384 2024-03-26T08:54:52.785 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:54:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 -- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff54c0a2a00 msgr2=0x7ff54c0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff54c0a2a00 0x7ff54c0a4ec0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff5540349f0 tx=0x7ff5540056d0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 -- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5641086a0 msgr2=0x7ff56419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5641086a0 0x7ff56419dfe0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00cce0 tx=0x7ff55c00a620 comp rx=0 tx=0).stop 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 -- 172.21.15.7:0/511487024 shutdown_connections 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff54c0a2a00 0x7ff54c0a4ec0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff56410d9c0 0x7ff5641a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56410b010 0x7ff56419e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 --2- 172.21.15.7:0/511487024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5641086a0 0x7ff56419dfe0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 -- 172.21.15.7:0/511487024 >> 172.21.15.7:0/511487024 conn(0x7ff56406e640 msgr2=0x7ff56410b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 -- 172.21.15.7:0/511487024 shutdown_connections 2024-03-26T08:54:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:52.784+0000 7ff5699b9640 1 -- 172.21.15.7:0/511487024 wait complete. 2024-03-26T08:54:52.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:52 smithi044 ceph-mon[36461]: pgmap v733: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-03-26T08:54:52.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:52 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3729471713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:54:53.793 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:54:53.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:54:53.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:54:53.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.935+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3107388383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 msgr2=0x7f67b0101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:53.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.935+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3107388383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 0x7f67b0101c10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009900 tx=0x7f67a402bc30 comp rx=0 tx=0).stop 2024-03-26T08:54:53.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.935+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3107388383 shutdown_connections 2024-03-26T08:54:53.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.935+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3107388383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b0102150 0x7f67b01089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.935+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3107388383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 0x7f67b0101c10 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.935+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3107388383 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b01005a0 0x7f67b01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.935+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3107388383 >> 172.21.15.7:0/3107388383 conn(0x7f67b00fbaf0 msgr2=0x7f67b00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:53.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.935+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3107388383 shutdown_connections 2024-03-26T08:54:53.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.935+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3107388383 wait complete. 2024-03-26T08:54:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67b4cc0640 1 Processor -- start 2024-03-26T08:54:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67b4cc0640 1 -- start start 2024-03-26T08:54:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67b4cc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b01005a0 0x7f67b0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67b4cc0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 0x7f67b0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67b4cc0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b0102150 0x7f67b019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67b4cc0640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b010c270 con 0x7f67b0101790 2024-03-26T08:54:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67b4cc0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b010c0f0 con 0x7f67b01005a0 2024-03-26T08:54:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67b4cc0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b010c3f0 con 0x7f67b0102150 2024-03-26T08:54:53.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67add9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 0x7f67b0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67add9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 0x7f67b0197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57348/0 (socket says 172.21.15.7:57348) 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67add9b640 1 -- 172.21.15.7:0/3888250370 learned_addr learned my addr 172.21.15.7:0/3888250370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67ae59c640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b01005a0 0x7f67b0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.936+0000 7f67aed9d640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b0102150 0x7f67b019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f67add9b640 1 -- 172.21.15.7:0/3888250370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b0102150 msgr2=0x7f67b019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f67add9b640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b0102150 0x7f67b019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f67add9b640 1 -- 172.21.15.7:0/3888250370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b01005a0 msgr2=0x7f67b0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f67add9b640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b01005a0 0x7f67b0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f67add9b640 1 -- 172.21.15.7:0/3888250370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b019f640 con 0x7f67b0101790 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f67aed9d640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b0102150 0x7f67b019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f67add9b640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 0x7f67b0197f60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f67a402c8e0 tx=0x7f67a402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f679f7fe640 1 -- 172.21.15.7:0/3888250370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4043020 con 0x7f67b0101790 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f679f7fe640 1 -- 172.21.15.7:0/3888250370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67a40054e0 con 0x7f67b0101790 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3888250370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b019f8d0 con 0x7f67b0101790 2024-03-26T08:54:53.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f679f7fe640 1 -- 172.21.15.7:0/3888250370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4003c70 con 0x7f67b0101790 2024-03-26T08:54:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.937+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3888250370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b019fe20 con 0x7f67b0101790 2024-03-26T08:54:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.940+0000 7f679f7fe640 1 -- 172.21.15.7:0/3888250370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f67a4046020 con 0x7f67b0101790 2024-03-26T08:54:53.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67b7985740 1 Processor -- start 2024-03-26T08:54:53.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67b7985740 1 -- start start 2024-03-26T08:54:53.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f679f7fe640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f67980a2800 0x7f67980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67b7985740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff163d70 0x55ebff164190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67b7985740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff169260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67b7985740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff1698d0 0x55ebff16bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67b7985740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebff09fc90 con 0x55ebff164760 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67b7985740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebff0c5070 con 0x55ebff163d70 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67b7985740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebff06dc80 con 0x55ebff1698d0 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67aed9d640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f67980a2800 0x7f67980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f679f7fe640 1 -- 172.21.15.7:0/3888250370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67a40f07c0 con 0x7f67b0101790 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67ae59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff163d70 0x55ebff164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67add9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff169260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67add9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff169260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57350/0 (socket says 172.21.15.7:57350) 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67add9b640 1 -- 172.21.15.7:0/2191626213 learned_addr learned my addr 172.21.15.7:0/2191626213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.941+0000 7f67aed9d640 1 --2- 172.21.15.7:0/2191626213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff1698d0 0x55ebff16bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f67add9b640 1 -- 172.21.15.7:0/2191626213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff1698d0 msgr2=0x55ebff16bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f67add9b640 1 --2- 172.21.15.7:0/2191626213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff1698d0 0x55ebff16bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f67add9b640 1 -- 172.21.15.7:0/2191626213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff163d70 msgr2=0x55ebff164190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f67add9b640 1 --2- 172.21.15.7:0/2191626213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff163d70 0x55ebff164190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f67add9b640 1 -- 172.21.15.7:0/2191626213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebff065d10 con 0x55ebff164760 2024-03-26T08:54:53.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f67aed9d640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f67980a2800 0x7f67980a4cc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005fd0 tx=0x7f67a8013660 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f67add9b640 1 --2- 172.21.15.7:0/2191626213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff169260 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f67a40bd0f0 tx=0x7f67a40bd060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3112dee1d782c8f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f679cff9640 1 -- 172.21.15.7:0/2191626213 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4004250 con 0x55ebff164760 2024-03-26T08:54:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f679cff9640 1 -- 172.21.15.7:0/2191626213 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67a4005690 con 0x55ebff164760 2024-03-26T08:54:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.942+0000 7f679cff9640 1 -- 172.21.15.7:0/2191626213 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a40c5900 con 0x55ebff164760 2024-03-26T08:54:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.943+0000 7f67b7985740 1 -- 172.21.15.7:0/2191626213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 msgr2=0x55ebff169260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.943+0000 7f67b7985740 1 --2- 172.21.15.7:0/2191626213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff169260 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f67a40bd0f0 tx=0x7f67a40bd060 comp rx=0 tx=0).stop 2024-03-26T08:54:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.944+0000 7f67b7985740 1 -- 172.21.15.7:0/2191626213 shutdown_connections 2024-03-26T08:54:53.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.944+0000 7f67b7985740 1 --2- 172.21.15.7:0/2191626213 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff1698d0 0x55ebff16bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.944+0000 7f67b7985740 1 --2- 172.21.15.7:0/2191626213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff169260 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.944+0000 7f67b7985740 1 --2- 172.21.15.7:0/2191626213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff163d70 0x55ebff164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.944+0000 7f67b7985740 1 -- 172.21.15.7:0/2191626213 >> 172.21.15.7:0/2191626213 conn(0x55ebff1281a0 msgr2=0x55ebff15b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.951+0000 7f67b7985740 1 -- 172.21.15.7:0/2191626213 shutdown_connections 2024-03-26T08:54:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.951+0000 7f67b7985740 1 -- 172.21.15.7:0/2191626213 wait complete. 2024-03-26T08:54:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.952+0000 7f67b7985740 1 -- start start 2024-03-26T08:54:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.952+0000 7f67b7985740 1 Processor -- start 2024-03-26T08:54:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.952+0000 7f67b7985740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff16de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.952+0000 7f67b7985740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff16e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.952+0000 7f67b7985740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 0x55ebff1753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.953+0000 7f67b7985740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebff0810d0 con 0x55ebff164760 2024-03-26T08:54:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.953+0000 7f67b7985740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebff0c1110 con 0x55ebff1698d0 2024-03-26T08:54:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.953+0000 7f67b7985740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebff16c640 con 0x55ebff163d70 2024-03-26T08:54:53.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.953+0000 7f67b7985740 10 client.0 _ll_register_callbacks cb 0x55ebff0bce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.956+0000 7f67aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 0x55ebff1753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.956+0000 7f67ae59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff16de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.958+0000 7f67add9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff16e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.958+0000 7f67add9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff16e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57358/0 (socket says 172.21.15.7:57358) 2024-03-26T08:54:53.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.958+0000 7f67add9b640 1 -- 172.21.15.7:0/486342797 learned_addr learned my addr 172.21.15.7:0/486342797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:53.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.960+0000 7f67add9b640 1 -- 172.21.15.7:0/486342797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 msgr2=0x55ebff16de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:53.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.960+0000 7f67add9b640 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff16de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.960+0000 7f67add9b640 1 -- 172.21.15.7:0/486342797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 msgr2=0x55ebff1753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:53.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.960+0000 7f67add9b640 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 0x55ebff1753e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.960+0000 7f67add9b640 1 -- 172.21.15.7:0/486342797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebfeea17e0 con 0x55ebff164760 2024-03-26T08:54:53.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.961+0000 7f67add9b640 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff16e3a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f67a40c53d0 tx=0x7f67a4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eee9d8f0b9b61f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:53.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.961+0000 7f6795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:54:53.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.962+0000 7f6795ffb640 1 -- 172.21.15.7:0/486342797 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4004250 con 0x55ebff164760 2024-03-26T08:54:53.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.962+0000 7f6795ffb640 1 -- 172.21.15.7:0/486342797 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67a40a62b0 con 0x55ebff164760 2024-03-26T08:54:53.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.963+0000 7f67b7985740 1 -- 172.21.15.7:0/486342797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebff0d25b0 con 0x55ebff164760 2024-03-26T08:54:53.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.965+0000 7f6795ffb640 1 -- 172.21.15.7:0/486342797 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67a40a64b0 con 0x55ebff164760 2024-03-26T08:54:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.974+0000 7f67b7985740 10 client.18729 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:53.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.974+0000 7f67b7985740 1 -- 172.21.15.7:0/486342797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ebfef01f70 con 0x55ebff164760 2024-03-26T08:54:53.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.976+0000 7f6795ffb640 1 -- 172.21.15.7:0/486342797 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ebff0d25b0 con 0x55ebff164760 2024-03-26T08:54:53.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.976+0000 7f6795ffb640 1 -- 172.21.15.7:0/486342797 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f67a40c5490 con 0x55ebff164760 2024-03-26T08:54:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.978+0000 7f67b7985740 10 client.18729 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.978+0000 7f67b7985740 10 client.18729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.978+0000 7f67b7985740 20 client.18729 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.978+0000 7f67b7985740 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebff179a60 0x55ebff17bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.978+0000 7f67b7985740 4 client.18729 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.978+0000 7f67b7985740 1 -- 172.21.15.7:0/486342797 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebff0cc900 con 0x55ebff179a60 2024-03-26T08:54:53.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.983+0000 7f67ae59c640 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebff179a60 0x55ebff17bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.990+0000 7f67ae59c640 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebff179a60 0x55ebff17bec0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a0c478ad0311eb server_cookie=b475ec2481ef74c in_seq=0 out_seq=0 2024-03-26T08:54:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.990+0000 7f6795ffb640 10 client.18729 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.991+0000 7f6795ffb640 1 -- 172.21.15.7:0/486342797 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ebff0cc900 con 0x55ebff179a60 2024-03-26T08:54:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.991+0000 7f6795ffb640 10 client.18729 handle_command_reply: tid=0 2024-03-26T08:54:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.991+0000 7f67b7985740 1 client.18729 shutdown 2024-03-26T08:54:53.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.997+0000 7f67b7985740 1 -- 172.21.15.7:0/486342797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 msgr2=0x55ebff16e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.997+0000 7f67b7985740 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff16e3a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f67a40c53d0 tx=0x7f67a4005e50 comp rx=0 tx=0).stop 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.998+0000 7f67b7985740 1 -- 172.21.15.7:0/486342797 shutdown_connections 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.998+0000 7f67b7985740 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebff179a60 0x55ebff17bec0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.998+0000 7f67b7985740 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 0x55ebff1753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.998+0000 7f67b7985740 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff16e3a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.998+0000 7f67b7985740 1 --2- 172.21.15.7:0/486342797 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff16de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.998+0000 7f67b7985740 1 -- 172.21.15.7:0/486342797 >> 172.21.15.7:0/486342797 conn(0x55ebff1281a0 msgr2=0x55ebff15a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.998+0000 7f6795ffb640 0 client.18729 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.999+0000 7f67b7985740 1 -- 172.21.15.7:0/486342797 shutdown_connections 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.999+0000 7f67b7985740 1 -- 172.21.15.7:0/486342797 wait complete. 2024-03-26T08:54:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:53.999+0000 7f67b7985740 20 client.18729 trim_cache size 0 max 16384 2024-03-26T08:54:54.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.022+0000 7f67b7985740 1 Processor -- start 2024-03-26T08:54:54.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.022+0000 7f67b7985740 1 -- start start 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.022+0000 7f67b7985740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff16d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.022+0000 7f67b7985740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff164760 0x55ebff1753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.022+0000 7f67b7985740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff1698d0 0x55ebff15e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67b7985740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01a8f00 con 0x55ebff1698d0 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67b7985740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01a6a00 con 0x55ebff164760 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67b7985740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01a4a80 con 0x55ebff163d70 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67aed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff1698d0 0x55ebff15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67aed9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff1698d0 0x55ebff15e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57374/0 (socket says 172.21.15.7:57374) 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67aed9d640 1 -- 172.21.15.7:0/2723793995 learned_addr learned my addr 172.21.15.7:0/2723793995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67ae59c640 1 --2- 172.21.15.7:0/2723793995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff16d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67add9b640 1 --2- 172.21.15.7:0/2723793995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff164760 0x55ebff1753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67aed9d640 1 -- 172.21.15.7:0/2723793995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 msgr2=0x55ebff16d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67aed9d640 1 --2- 172.21.15.7:0/2723793995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff16d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67aed9d640 1 -- 172.21.15.7:0/2723793995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff164760 msgr2=0x55ebff1753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67aed9d640 1 --2- 172.21.15.7:0/2723793995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff164760 0x55ebff1753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67aed9d640 1 -- 172.21.15.7:0/2723793995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebfed99170 con 0x55ebff1698d0 2024-03-26T08:54:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f67aed9d640 1 --2- 172.21.15.7:0/2723793995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff1698d0 0x55ebff15e510 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f67a8013ac0 tx=0x7f67a8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547d6b459a26efda server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f6797fff640 1 -- 172.21.15.7:0/2723793995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a8003e00 con 0x55ebff1698d0 2024-03-26T08:54:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f6797fff640 1 -- 172.21.15.7:0/2723793995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67a8004530 con 0x55ebff1698d0 2024-03-26T08:54:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.023+0000 7f6797fff640 1 -- 172.21.15.7:0/2723793995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a8005960 con 0x55ebff1698d0 2024-03-26T08:54:54.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 -- 172.21.15.7:0/2723793995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff1698d0 msgr2=0x55ebff15e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 --2- 172.21.15.7:0/2723793995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff1698d0 0x55ebff15e510 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f67a8013ac0 tx=0x7f67a8021000 comp rx=0 tx=0).stop 2024-03-26T08:54:54.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 -- 172.21.15.7:0/2723793995 shutdown_connections 2024-03-26T08:54:54.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 --2- 172.21.15.7:0/2723793995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff1698d0 0x55ebff15e510 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 --2- 172.21.15.7:0/2723793995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff164760 0x55ebff1753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 --2- 172.21.15.7:0/2723793995 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff16d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 -- 172.21.15.7:0/2723793995 >> 172.21.15.7:0/2723793995 conn(0x55ebff1281a0 msgr2=0x55ebff16d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 -- 172.21.15.7:0/2723793995 shutdown_connections 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 -- 172.21.15.7:0/2723793995 wait complete. 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.024+0000 7f67b7985740 1 -- start start 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67b7985740 1 Processor -- start 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67b7985740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff173210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67b7985740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff173750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67b7985740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 0x55ebff174760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67b7985740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01a5080 con 0x55ebff164760 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67b7985740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01a63a0 con 0x55ebff1698d0 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67b7985740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b01a56e0 con 0x55ebff163d70 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67ae59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff173210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67aed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 0x55ebff174760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67ae59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff173210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58790/0 (socket says 172.21.15.7:58790) 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67ae59c640 1 -- 172.21.15.7:0/1971509738 learned_addr learned my addr 172.21.15.7:0/1971509738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.025+0000 7f67b7985740 10 client.0 _ll_register_callbacks cb 0x7f67a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f67add9b640 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff173750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f67ae59c640 1 -- 172.21.15.7:0/1971509738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 msgr2=0x55ebff174760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f67ae59c640 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 0x55ebff174760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f67ae59c640 1 -- 172.21.15.7:0/1971509738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 msgr2=0x55ebff173750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f67ae59c640 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff173750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f67ae59c640 1 -- 172.21.15.7:0/1971509738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebff174d50 con 0x55ebff163d70 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f67ae59c640 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff173210 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55ebff164fc0 tx=0x7f67a001ea60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=975e80391cf0a249 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f6795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f6795ffb640 1 -- 172.21.15.7:0/1971509738 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0024060 con 0x55ebff163d70 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f6795ffb640 1 -- 172.21.15.7:0/1971509738 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67a00035f0 con 0x55ebff163d70 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.026+0000 7f67b7985740 1 -- 172.21.15.7:0/1971509738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67a8004530 con 0x55ebff163d70 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.027+0000 7f6795ffb640 1 -- 172.21.15.7:0/1971509738 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67a00037f0 con 0x55ebff163d70 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.027+0000 7f67b7985740 10 client.28519 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.027+0000 7f67b7985740 1 -- 172.21.15.7:0/1971509738 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ebff1d5ba0 con 0x55ebff163d70 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.027+0000 7f6795ffb640 1 -- 172.21.15.7:0/1971509738 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67a00104f0 con 0x55ebff163d70 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.027+0000 7f6795ffb640 1 -- 172.21.15.7:0/1971509738 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f67a0029070 con 0x55ebff163d70 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.027+0000 7f67b7985740 10 client.28519 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.027+0000 7f67b7985740 10 client.28519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.028+0000 7f67b7985740 20 client.28519 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.028+0000 7f67b7985740 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebff1d6580 0x55ebff1d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.028+0000 7f67b7985740 4 client.28519 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:54:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.028+0000 7f67b7985740 1 -- 172.21.15.7:0/1971509738 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f67a8004530 con 0x55ebff1d6580 2024-03-26T08:54:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.028+0000 7f67add9b640 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebff1d6580 0x55ebff1d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.032+0000 7f6795ffb640 10 client.28519 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:54.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.032+0000 7f67add9b640 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebff1d6580 0x55ebff1d89a0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9fa21788171c669 server_cookie=f8b9372d2cca0ceb in_seq=0 out_seq=0 2024-03-26T08:54:54.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.034+0000 7f6795ffb640 1 -- 172.21.15.7:0/1971509738 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f67a8004530 con 0x55ebff1d6580 2024-03-26T08:54:54.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.034+0000 7f6795ffb640 10 client.28519 handle_command_reply: tid=0 2024-03-26T08:54:54.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.034+0000 7f67b7985740 1 client.28519 shutdown 2024-03-26T08:54:54.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 -- 172.21.15.7:0/1971509738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 msgr2=0x55ebff173210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff173210 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55ebff164fc0 tx=0x7f67a001ea60 comp rx=0 tx=0).stop 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 -- 172.21.15.7:0/1971509738 shutdown_connections 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ebff1d6580 0x55ebff1d89a0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ebff1698d0 0x55ebff174760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebff164760 0x55ebff173750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 --2- 172.21.15.7:0/1971509738 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ebff163d70 0x55ebff173210 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 -- 172.21.15.7:0/1971509738 >> 172.21.15.7:0/1971509738 conn(0x55ebff1281a0 msgr2=0x55ebff15e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f6795ffb640 0 client.28519 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 -- 172.21.15.7:0/1971509738 shutdown_connections 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 1 -- 172.21.15.7:0/1971509738 wait complete. 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.035+0000 7f67b7985740 20 client.28519 trim_cache size 0 max 16384 2024-03-26T08:54:54.033 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:54:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:54:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "69301ea9-186a-4ac0-91db-32e142464ffb", 2024-03-26T08:54:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:54:54.034 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:54:54.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f67980a2800 msgr2=0x7f67980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f67980a2800 0x7f67980a4cc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005fd0 tx=0x7f67a8013660 comp rx=0 tx=0).stop 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 msgr2=0x7f67b0197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 0x7f67b0197f60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f67a402c8e0 tx=0x7f67a402c650 comp rx=0 tx=0).stop 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3888250370 shutdown_connections 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f67980a2800 0x7f67980a4cc0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f67b0102150 0x7f67b019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b0101790 0x7f67b0197f60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 --2- 172.21.15.7:0/3888250370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67b01005a0 0x7f67b0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3888250370 >> 172.21.15.7:0/3888250370 conn(0x7f67b00fbaf0 msgr2=0x7f67b0069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3888250370 shutdown_connections 2024-03-26T08:54:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.036+0000 7f67b4cc0640 1 -- 172.21.15.7:0/3888250370 wait complete. 2024-03-26T08:54:54.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:54:54.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:54:54.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.200+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3117338127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1017b0 msgr2=0x7fb0cc101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.200+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3117338127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1017b0 0x7fb0cc101c30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009900 tx=0x7fb0c002bc60 comp rx=0 tx=0).stop 2024-03-26T08:54:54.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.205+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3117338127 shutdown_connections 2024-03-26T08:54:54.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.205+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3117338127 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0cc102170 0x7fb0cc108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.205+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3117338127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1017b0 0x7fb0cc101c30 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.205+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3117338127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0cc1005c0 0x7fb0cc1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.206+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3117338127 >> 172.21.15.7:0/3117338127 conn(0x7fb0cc0fbb30 msgr2=0x7fb0cc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.208+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3117338127 shutdown_connections 2024-03-26T08:54:54.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.208+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3117338127 wait complete. 2024-03-26T08:54:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.209+0000 7fb0d0db3640 1 Processor -- start 2024-03-26T08:54:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.209+0000 7fb0d0db3640 1 -- start start 2024-03-26T08:54:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0d0db3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1005c0 0x7fb0cc110bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0d0db3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0cc1017b0 0x7fb0cc1110f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0d0db3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0cc102170 0x7fb0cc118160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1005c0 0x7fb0cc110bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0cc1017b0 0x7fb0cc1110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0cc1017b0 0x7fb0cc1110f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57348/0 (socket says 172.21.15.7:57348) 2024-03-26T08:54:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1005c0 0x7fb0cc110bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57408/0 (socket says 172.21.15.7:57408) 2024-03-26T08:54:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0cad9d640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0cc102170 0x7fb0cc118160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0c9d9b640 1 -- 172.21.15.7:0/3336028132 learned_addr learned my addr 172.21.15.7:0/3336028132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc10b270 con 0x7fb0cc1005c0 2024-03-26T08:54:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc10b0f0 con 0x7fb0cc1017b0 2024-03-26T08:54:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.210+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc10b3f0 con 0x7fb0cc102170 2024-03-26T08:54:54.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0c9d9b640 1 -- 172.21.15.7:0/3336028132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0cc102170 msgr2=0x7fb0cc118160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0c9d9b640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0cc102170 0x7fb0cc118160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0c9d9b640 1 -- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1005c0 msgr2=0x7fb0cc110bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0c9d9b640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1005c0 0x7fb0cc110bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0c9d9b640 1 -- 172.21.15.7:0/3336028132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc1187d0 con 0x7fb0cc1017b0 2024-03-26T08:54:54.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1005c0 0x7fb0cc110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:54:54.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0c9d9b640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0cc1017b0 0x7fb0cc1110f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb0c002c270 tx=0x7fb0c002c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:54.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0bb7fe640 1 -- 172.21.15.7:0/3336028132 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0041070 con 0x7fb0cc1017b0 2024-03-26T08:54:54.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc118a60 con 0x7fb0cc1017b0 2024-03-26T08:54:54.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc1aede0 con 0x7fb0cc1017b0 2024-03-26T08:54:54.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.211+0000 7fb0bb7fe640 1 -- 172.21.15.7:0/3336028132 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0c0004c40 con 0x7fb0cc1017b0 2024-03-26T08:54:54.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.212+0000 7fb0bb7fe640 1 -- 172.21.15.7:0/3336028132 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0003c20 con 0x7fb0cc1017b0 2024-03-26T08:54:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.214+0000 7fb0d3999740 1 Processor -- start 2024-03-26T08:54:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0d3999740 1 -- start start 2024-03-26T08:54:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b36790 0x5570b7b3b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b3b900 0x5570b7b3dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0d3999740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570b7a9b3d0 con 0x5570b7b3b900 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0d3999740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570b7a43ad0 con 0x5570b7b36790 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0d3999740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570b7a97470 con 0x5570b7b35da0 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0bb7fe640 1 -- 172.21.15.7:0/3336028132 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb0c0044070 con 0x7fb0cc1017b0 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0c9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b3b900 0x5570b7b3dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b36790 0x5570b7b3b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0cad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.215+0000 7fb0ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b36790 0x5570b7b3b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57356/0 (socket says 172.21.15.7:57356) 2024-03-26T08:54:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.216+0000 7fb0ca59c640 1 -- 172.21.15.7:0/4091030633 learned_addr learned my addr 172.21.15.7:0/4091030633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0ca59c640 1 -- 172.21.15.7:0/4091030633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 msgr2=0x5570b7b361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/4091030633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0ca59c640 1 -- 172.21.15.7:0/4091030633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b3b900 msgr2=0x5570b7b3dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/4091030633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b3b900 0x5570b7b3dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0ca59c640 1 -- 172.21.15.7:0/4091030633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570b78797d0 con 0x5570b7b36790 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0bb7fe640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b40a2ac0 0x7fb0b40a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0cad9d640 1 --2- 172.21.15.7:0/4091030633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0bb7fe640 1 -- 172.21.15.7:0/3336028132 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb0c00039c0 con 0x7fb0cc1017b0 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0cad9d640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b40a2ac0 0x7fb0b40a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/4091030633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b36790 0x5570b7b3b290 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400c550 tx=0x7fb0c400ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add930a2e4ae37a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0c9d9b640 1 --2- 172.21.15.7:0/4091030633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b3b900 0x5570b7b3dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0b8ff9640 1 -- 172.21.15.7:0/4091030633 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c401a070 con 0x5570b7b36790 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0b8ff9640 1 -- 172.21.15.7:0/4091030633 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0c4007a10 con 0x5570b7b36790 2024-03-26T08:54:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.217+0000 7fb0b8ff9640 1 -- 172.21.15.7:0/4091030633 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4015b10 con 0x5570b7b36790 2024-03-26T08:54:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0d3999740 1 -- 172.21.15.7:0/4091030633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b36790 msgr2=0x5570b7b3b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0cad9d640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b40a2ac0 0x7fb0b40a4f80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x5570b7b36ff0 tx=0x7fb0bc0090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0d3999740 1 --2- 172.21.15.7:0/4091030633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b36790 0x5570b7b3b290 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400c550 tx=0x7fb0c400ca10 comp rx=0 tx=0).stop 2024-03-26T08:54:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0d3999740 1 -- 172.21.15.7:0/4091030633 shutdown_connections 2024-03-26T08:54:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0d3999740 1 --2- 172.21.15.7:0/4091030633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b3b900 0x5570b7b3dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0d3999740 1 --2- 172.21.15.7:0/4091030633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b36790 0x5570b7b3b290 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0d3999740 1 --2- 172.21.15.7:0/4091030633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0d3999740 1 -- 172.21.15.7:0/4091030633 >> 172.21.15.7:0/4091030633 conn(0x5570b7b31310 msgr2=0x5570b7b31710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0d3999740 1 -- 172.21.15.7:0/4091030633 shutdown_connections 2024-03-26T08:54:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.218+0000 7fb0d3999740 1 -- 172.21.15.7:0/4091030633 wait complete. 2024-03-26T08:54:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0d3999740 1 -- start start 2024-03-26T08:54:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0d3999740 1 Processor -- start 2024-03-26T08:54:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b35da0 0x5570b7b40230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b36790 0x5570b7b40770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 0x5570b7b477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0d3999740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570b7b3e8b0 con 0x5570b7b35da0 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0d3999740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570b7a57060 con 0x5570b7b3b900 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0d3999740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570b7b3ea30 con 0x5570b7b36790 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0d3999740 10 client.0 _ll_register_callbacks cb 0x5570b7a396a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b35da0 0x5570b7b40230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0cad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 0x5570b7b477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0c9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b36790 0x5570b7b40770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0ca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b35da0 0x5570b7b40230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57424/0 (socket says 172.21.15.7:57424) 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0ca59c640 1 -- 172.21.15.7:0/1149536072 learned_addr learned my addr 172.21.15.7:0/1149536072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0c9d9b640 1 -- 172.21.15.7:0/1149536072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 msgr2=0x5570b7b477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.219+0000 7fb0c9d9b640 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 0x5570b7b477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0c9d9b640 1 -- 172.21.15.7:0/1149536072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b35da0 msgr2=0x5570b7b40230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0c9d9b640 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b35da0 0x5570b7b40230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0c9d9b640 1 -- 172.21.15.7:0/1149536072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570b77af3f0 con 0x5570b7b36790 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0c9d9b640 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b36790 0x5570b7b40770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb0c002f020 tx=0x7fb0c00f3b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15e1fb66f6f8b8df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/1149536072 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c004b070 con 0x5570b7b36790 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/1149536072 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb0c00bd2b0 con 0x5570b7b36790 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0d3999740 1 -- 172.21.15.7:0/1149536072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0c4007a10 con 0x5570b7b36790 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/1149536072 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c00bce20 con 0x5570b7b36790 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0d3999740 10 client.28528 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0d3999740 1 -- 172.21.15.7:0/1149536072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5570b776f0f0 con 0x5570b7b36790 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.220+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/1149536072 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0c4007a10 con 0x5570b7b36790 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.221+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/1149536072 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb0c00d2070 con 0x5570b7b36790 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.221+0000 7fb0d3999740 10 client.28528 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.221+0000 7fb0d3999740 10 client.28528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.221+0000 7fb0d3999740 20 client.28528 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.221+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5570b7b4bc50 0x5570b7b4e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.221+0000 7fb0d3999740 4 client.28528 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:54:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.221+0000 7fb0d3999740 1 -- 172.21.15.7:0/1149536072 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570b7aa8910 con 0x5570b7b4bc50 2024-03-26T08:54:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.222+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5570b7b4bc50 0x5570b7b4e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.223+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5570b7b4bc50 0x5570b7b4e0f0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53170a6dcf00a9a0 server_cookie=577ae04b2a4cd8a8 in_seq=0 out_seq=0 2024-03-26T08:54:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.223+0000 7fb0b1ffb640 10 client.28528 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.224+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/1149536072 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5570b7aa8910 con 0x5570b7b4bc50 2024-03-26T08:54:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.224+0000 7fb0b1ffb640 10 client.28528 handle_command_reply: tid=0 2024-03-26T08:54:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 client.28528 shutdown 2024-03-26T08:54:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 -- 172.21.15.7:0/1149536072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b36790 msgr2=0x5570b7b40770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b36790 0x5570b7b40770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb0c002f020 tx=0x7fb0c00f3b50 comp rx=0 tx=0).stop 2024-03-26T08:54:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 -- 172.21.15.7:0/1149536072 shutdown_connections 2024-03-26T08:54:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5570b7b4bc50 0x5570b7b4e0f0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 0x5570b7b477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b36790 0x5570b7b40770 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1149536072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b35da0 0x5570b7b40230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0b1ffb640 0 client.28528 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 -- 172.21.15.7:0/1149536072 >> 172.21.15.7:0/1149536072 conn(0x5570b7b31310 msgr2=0x5570b7b30530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 -- 172.21.15.7:0/1149536072 shutdown_connections 2024-03-26T08:54:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 1 -- 172.21.15.7:0/1149536072 wait complete. 2024-03-26T08:54:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.225+0000 7fb0d3999740 20 client.28528 trim_cache size 0 max 16384 2024-03-26T08:54:54.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0d3999740 1 Processor -- start 2024-03-26T08:54:54.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0d3999740 1 -- start start 2024-03-26T08:54:54.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b2fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b47770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0ca59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b2fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 0x5570b7b301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0ca59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b2fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58820/0 (socket says 172.21.15.7:58820) 2024-03-26T08:54:54.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0c9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b47770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0d3999740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac010870 con 0x5570b7b36790 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0d3999740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac00e370 con 0x5570b7b3b900 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0ca59c640 1 -- 172.21.15.7:0/1408117465 learned_addr learned my addr 172.21.15.7:0/1408117465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0d3999740 1 -- 172.21.15.7:0/1408117465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac00c3f0 con 0x5570b7b35da0 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0c9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b47770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57434/0 (socket says 172.21.15.7:57434) 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.249+0000 7fb0cad9d640 1 --2- 172.21.15.7:0/1408117465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 0x5570b7b301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.250+0000 7fb0ca59c640 1 -- 172.21.15.7:0/1408117465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 msgr2=0x5570b7b301e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.250+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/1408117465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 0x5570b7b301e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.250+0000 7fb0ca59c640 1 -- 172.21.15.7:0/1408117465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 msgr2=0x5570b7b47770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.250+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/1408117465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b47770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.250+0000 7fb0ca59c640 1 -- 172.21.15.7:0/1408117465 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570b7b36b70 con 0x5570b7b35da0 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.250+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/1408117465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b2fca0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400cf10 tx=0x7fb0c40093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56553b4d11fcea69 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.250+0000 7fb0b3fff640 1 -- 172.21.15.7:0/1408117465 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c401a040 con 0x5570b7b35da0 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.250+0000 7fb0b3fff640 1 -- 172.21.15.7:0/1408117465 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0c4026160 con 0x5570b7b35da0 2024-03-26T08:54:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.250+0000 7fb0b3fff640 1 -- 172.21.15.7:0/1408117465 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40155d0 con 0x5570b7b35da0 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.251+0000 7fb0d3999740 1 -- 172.21.15.7:0/1408117465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 msgr2=0x5570b7b2fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.251+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1408117465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b2fca0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400cf10 tx=0x7fb0c40093d0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.251+0000 7fb0d3999740 1 -- 172.21.15.7:0/1408117465 shutdown_connections 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.251+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1408117465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b3b900 0x5570b7b301e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.251+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1408117465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b47770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.251+0000 7fb0d3999740 1 --2- 172.21.15.7:0/1408117465 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b35da0 0x5570b7b2fca0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.251+0000 7fb0d3999740 1 -- 172.21.15.7:0/1408117465 >> 172.21.15.7:0/1408117465 conn(0x5570b7b31310 msgr2=0x5570b7b40230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.252+0000 7fb0d3999740 1 -- 172.21.15.7:0/1408117465 shutdown_connections 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.252+0000 7fb0d3999740 1 -- 172.21.15.7:0/1408117465 wait complete. 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.252+0000 7fb0d3999740 1 -- start start 2024-03-26T08:54:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0d3999740 1 Processor -- start 2024-03-26T08:54:54.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b35da0 0x5570b7b470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0d3999740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b3b900 0x5570b7b456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0d3999740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac00c9f0 con 0x5570b7b36790 2024-03-26T08:54:54.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0d3999740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac00dd10 con 0x5570b7b35da0 2024-03-26T08:54:54.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0d3999740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac00d050 con 0x5570b7b3b900 2024-03-26T08:54:54.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0d3999740 10 client.0 _ll_register_callbacks cb 0x7fb0c0009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:54:54.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0c9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0c9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57436/0 (socket says 172.21.15.7:57436) 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0c9d9b640 1 -- 172.21.15.7:0/635969746 learned_addr learned my addr 172.21.15.7:0/635969746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b35da0 0x5570b7b470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0cad9d640 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b3b900 0x5570b7b456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.253+0000 7fb0cad9d640 1 -- 172.21.15.7:0/635969746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b35da0 msgr2=0x5570b7b470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0cad9d640 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b35da0 0x5570b7b470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0cad9d640 1 -- 172.21.15.7:0/635969746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 msgr2=0x5570b7b475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0cad9d640 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0cad9d640 1 -- 172.21.15.7:0/635969746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570b7b45e40 con 0x5570b7b3b900 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0c9d9b640 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0cad9d640 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b3b900 0x5570b7b456b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc011750 tx=0x7fb0bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=914750f67d0deb51 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/635969746 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc033780 con 0x5570b7b3b900 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0d3999740 1 -- 172.21.15.7:0/635969746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0c4026160 con 0x5570b7b3b900 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/635969746 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb0bc033920 con 0x5570b7b3b900 2024-03-26T08:54:54.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/635969746 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0bc033b20 con 0x5570b7b3b900 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0d3999740 10 client.28534 fetch_fsmap learned FSMap version 25 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.254+0000 7fb0d3999740 1 -- 172.21.15.7:0/635969746 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5570b7ba80d0 con 0x5570b7b3b900 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/635969746 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0c4026160 con 0x5570b7b3b900 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/635969746 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb0bc03b070 con 0x5570b7b3b900 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0d3999740 10 client.28534 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0d3999740 10 client.28534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0d3999740 20 client.28534 populate_metadata read hostname 'smithi007' 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0d3999740 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5570b7ba8a90 0x5570b7baaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0d3999740 4 client.28534 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0d3999740 1 -- 172.21.15.7:0/635969746 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5570b7aa2c60 con 0x5570b7ba8a90 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5570b7ba8a90 0x5570b7baaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0b1ffb640 10 client.28534 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.255+0000 7fb0ca59c640 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5570b7ba8a90 0x5570b7baaeb0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d8e03f4371b53a server_cookie=5f5c63b757bd21f5 in_seq=0 out_seq=0 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0b1ffb640 1 -- 172.21.15.7:0/635969746 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5570b7aa2c60 con 0x5570b7ba8a90 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0b1ffb640 10 client.28534 handle_command_reply: tid=0 2024-03-26T08:54:54.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0d3999740 1 client.28534 shutdown 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0d3999740 1 -- 172.21.15.7:0/635969746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b3b900 msgr2=0x5570b7b456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0d3999740 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b3b900 0x5570b7b456b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc011750 tx=0x7fb0bc0046c0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0d3999740 1 -- 172.21.15.7:0/635969746 shutdown_connections 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0d3999740 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5570b7ba8a90 0x5570b7baaeb0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0d3999740 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5570b7b3b900 0x5570b7b456b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0d3999740 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5570b7b36790 0x5570b7b475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0d3999740 1 --2- 172.21.15.7:0/635969746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570b7b35da0 0x5570b7b470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0d3999740 1 -- 172.21.15.7:0/635969746 >> 172.21.15.7:0/635969746 conn(0x5570b7b31310 msgr2=0x5570b7b34570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.256+0000 7fb0b1ffb640 0 client.28534 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.257+0000 7fb0d3999740 1 -- 172.21.15.7:0/635969746 shutdown_connections 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.257+0000 7fb0d3999740 1 -- 172.21.15.7:0/635969746 wait complete. 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.257+0000 7fb0d3999740 20 client.28534 trim_cache size 0 max 16384 2024-03-26T08:54:54.257 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:54:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (223 inodes in the stack)", 2024-03-26T08:54:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:54:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "69301ea9-186a-4ac0-91db-32e142464ffb": { 2024-03-26T08:54:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:54:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "69301ea9-186a-4ac0-91db-32e142464ffb", 2024-03-26T08:54:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:54:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:54:54.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:54:54.258 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.257+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b40a2ac0 msgr2=0x7fb0b40a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.257+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b40a2ac0 0x7fb0b40a4f80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x5570b7b36ff0 tx=0x7fb0bc0090e0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.257+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0cc1017b0 msgr2=0x7fb0cc1110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.257+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0cc1017b0 0x7fb0cc1110f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb0c002c270 tx=0x7fb0c002c670 comp rx=0 tx=0).stop 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.258+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 shutdown_connections 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.258+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0b40a2ac0 0x7fb0b40a4f80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.258+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb0cc102170 0x7fb0cc118160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.258+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0cc1017b0 0x7fb0cc1110f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.258+0000 7fb0d0db3640 1 --2- 172.21.15.7:0/3336028132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0cc1005c0 0x7fb0cc110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.258+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 >> 172.21.15.7:0/3336028132 conn(0x7fb0cc0fbb30 msgr2=0x7fb0cc104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.259+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 shutdown_connections 2024-03-26T08:54:54.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:54:54.259+0000 7fb0d0db3640 1 -- 172.21.15.7:0/3336028132 wait complete. 2024-03-26T08:54:54.270 INFO:tasks.cephfs.filesystem:scrub status for tag:69301ea9-186a-4ac0-91db-32e142464ffb - {'path': '/', 'tag': '69301ea9-186a-4ac0-91db-32e142464ffb', 'options': 'recursive,force'} 2024-03-26T08:54:54.547 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:54 smithi007 ceph-mon[32462]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 37 MiB/s wr, 1.81k op/s 2024-03-26T08:54:54.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:54 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:54:54.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:54 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:54.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:54 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:54:54.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:54 smithi016 ceph-mon[35851]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 37 MiB/s wr, 1.81k op/s 2024-03-26T08:54:54.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:54 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:54:54.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:54 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:54.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:54 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:54:54.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:54 smithi044 ceph-mon[36461]: pgmap v734: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 37 MiB/s wr, 1.81k op/s 2024-03-26T08:54:54.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:54 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:54:54.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:54 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:54.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:54 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:54:55.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:55 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:54:55.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:55 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:55.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:55 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:55.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:55 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:54:55.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:55 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:55.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:55 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:55.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:55 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:54:55.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:55 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:55.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:55 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:54:56.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 745.0 s, 2032.6 tps, lat 50.043 ms stddev 63.142 2024-03-26T08:54:56.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:56 smithi016 ceph-mon[35851]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 36 MiB/s wr, 1.75k op/s 2024-03-26T08:54:56.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:56 smithi007 ceph-mon[32462]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 36 MiB/s wr, 1.75k op/s 2024-03-26T08:54:56.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:56 smithi044 ceph-mon[36461]: pgmap v735: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 36 MiB/s wr, 1.75k op/s 2024-03-26T08:54:58.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:54:58 smithi016 ceph-mon[35851]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 33 MiB/s wr, 1.66k op/s 2024-03-26T08:54:58.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:54:58 smithi007 ceph-mon[32462]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 33 MiB/s wr, 1.66k op/s 2024-03-26T08:54:58.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:54:58 smithi044 ceph-mon[36461]: pgmap v736: 97 pgs: 97 active+clean; 1.6 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 33 MiB/s wr, 1.66k op/s 2024-03-26T08:55:00.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:00 smithi016 ceph-mon[35851]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 35 MiB/s wr, 1.84k op/s 2024-03-26T08:55:00.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:00 smithi007 ceph-mon[32462]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 35 MiB/s wr, 1.84k op/s 2024-03-26T08:55:00.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:00 smithi044 ceph-mon[36461]: pgmap v737: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 35 MiB/s wr, 1.84k op/s 2024-03-26T08:55:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 750.0 s, 1573.4 tps, lat 63.541 ms stddev 73.521 2024-03-26T08:55:01.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:01 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:55:01.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:01 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:55:01.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:01 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:55:02.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:02 smithi016 ceph-mon[35851]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2024-03-26T08:55:02.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:02 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:55:02.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:02 smithi007 ceph-mon[32462]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2024-03-26T08:55:02.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:02 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:55:02.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:02 smithi044 ceph-mon[36461]: pgmap v738: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 34 MiB/s wr, 1.81k op/s 2024-03-26T08:55:02.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:02 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:55:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.179+0000 7fc62924b640 1 -- 172.21.15.7:0/1243660633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6240ff580 msgr2=0x7fc6240ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.179+0000 7fc62924b640 1 --2- 172.21.15.7:0/1243660633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6240ff580 0x7fc6240ff9a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc614009900 tx=0x7fc61402bc30 comp rx=0 tx=0).stop 2024-03-26T08:55:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.179+0000 7fc62924b640 1 -- 172.21.15.7:0/1243660633 shutdown_connections 2024-03-26T08:55:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.179+0000 7fc62924b640 1 --2- 172.21.15.7:0/1243660633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc624101050 0x7fc624105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.179+0000 7fc62924b640 1 --2- 172.21.15.7:0/1243660633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6241006b0 0x7fc624100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.179+0000 7fc62924b640 1 --2- 172.21.15.7:0/1243660633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6240ff580 0x7fc6240ff9a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.179+0000 7fc62924b640 1 -- 172.21.15.7:0/1243660633 >> 172.21.15.7:0/1243660633 conn(0x7fc6240fae80 msgr2=0x7fc6240fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.179+0000 7fc62924b640 1 -- 172.21.15.7:0/1243660633 shutdown_connections 2024-03-26T08:55:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.180+0000 7fc62924b640 1 -- 172.21.15.7:0/1243660633 wait complete. 2024-03-26T08:55:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.180+0000 7fc62924b640 1 Processor -- start 2024-03-26T08:55:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.180+0000 7fc62924b640 1 -- start start 2024-03-26T08:55:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc62924b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6240ff580 0x7fc62419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc62924b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6241006b0 0x7fc62419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc62924b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624101050 0x7fc6241a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc62924b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc624113960 con 0x7fc6241006b0 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc62924b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241137e0 con 0x7fc624101050 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc62924b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc624113ae0 con 0x7fc6240ff580 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6241006b0 0x7fc62419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6241006b0 0x7fc62419e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56500/0 (socket says 172.21.15.7:56500) 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc622d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6240ff580 0x7fc62419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc61bfff640 1 -- 172.21.15.7:0/3743473642 learned_addr learned my addr 172.21.15.7:0/3743473642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc62359e640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624101050 0x7fc6241a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc61bfff640 1 -- 172.21.15.7:0/3743473642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6240ff580 msgr2=0x7fc62419dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc61bfff640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6240ff580 0x7fc62419dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc61bfff640 1 -- 172.21.15.7:0/3743473642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624101050 msgr2=0x7fc6241a5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc61bfff640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624101050 0x7fc6241a5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc61bfff640 1 -- 172.21.15.7:0/3743473642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6241a5bd0 con 0x7fc6241006b0 2024-03-26T08:55:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc622d9d640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6240ff580 0x7fc62419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.181+0000 7fc61bfff640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6241006b0 0x7fc62419e4f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00d950 tx=0x7fc60c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.182+0000 7fc620d99640 1 -- 172.21.15.7:0/3743473642 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c017030 con 0x7fc6241006b0 2024-03-26T08:55:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.182+0000 7fc620d99640 1 -- 172.21.15.7:0/3743473642 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc60c004020 con 0x7fc6241006b0 2024-03-26T08:55:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.182+0000 7fc62924b640 1 -- 172.21.15.7:0/3743473642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6241a5ec0 con 0x7fc6241006b0 2024-03-26T08:55:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.183+0000 7fc62924b640 1 -- 172.21.15.7:0/3743473642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6241a63d0 con 0x7fc6241006b0 2024-03-26T08:55:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.183+0000 7fc620d99640 1 -- 172.21.15.7:0/3743473642 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0047a0 con 0x7fc6241006b0 2024-03-26T08:55:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.186+0000 7fc62be39740 1 Processor -- start 2024-03-26T08:55:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.186+0000 7fc62be39740 1 -- start start 2024-03-26T08:55:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.186+0000 7fc620d99640 1 -- 172.21.15.7:0/3743473642 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc60c0041d0 con 0x7fc6241006b0 2024-03-26T08:55:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.186+0000 7fc62be39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.186+0000 7fc62be39740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61ae7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.186+0000 7fc62be39740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae8480 0x55ee61aea910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.186+0000 7fc62be39740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee61a47b10 con 0x55ee61ae3310 2024-03-26T08:55:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.186+0000 7fc62be39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee619f0900 con 0x55ee61ae2920 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.186+0000 7fc62be39740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee61a43bb0 con 0x55ee61ae8480 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc622d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61ae7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae8480 0x55ee61aea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc622d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61ae7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56502/0 (socket says 172.21.15.7:56502) 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc622d9d640 1 -- 172.21.15.7:0/3993983968 learned_addr learned my addr 172.21.15.7:0/3993983968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc62359e640 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc622d9d640 1 -- 172.21.15.7:0/3993983968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae8480 msgr2=0x55ee61aea910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc622d9d640 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae8480 0x55ee61aea910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc622d9d640 1 -- 172.21.15.7:0/3993983968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 msgr2=0x55ee61ae2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc622d9d640 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc622d9d640 1 -- 172.21.15.7:0/3993983968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee6175c210 con 0x55ee61ae3310 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc62359e640 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc61bfff640 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae8480 0x55ee61aea910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc622d9d640 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61ae7e10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc614036040 tx=0x7fc61402c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f80a0c54d9ad5c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.187+0000 7fc620d99640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6100a2740 0x7fc6100a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.188+0000 7fc6197fa640 1 -- 172.21.15.7:0/3993983968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614004230 con 0x55ee61ae3310 2024-03-26T08:55:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.188+0000 7fc6197fa640 1 -- 172.21.15.7:0/3993983968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc61402c5d0 con 0x55ee61ae3310 2024-03-26T08:55:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.188+0000 7fc6197fa640 1 -- 172.21.15.7:0/3993983968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614004c30 con 0x55ee61ae3310 2024-03-26T08:55:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.188+0000 7fc620d99640 1 -- 172.21.15.7:0/3743473642 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc60c0ce370 con 0x7fc6241006b0 2024-03-26T08:55:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.188+0000 7fc62359e640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6100a2740 0x7fc6100a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 -- 172.21.15.7:0/3993983968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 msgr2=0x55ee61ae7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61ae7e10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc614036040 tx=0x7fc61402c910 comp rx=0 tx=0).stop 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 -- 172.21.15.7:0/3993983968 shutdown_connections 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae8480 0x55ee61aea910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61ae7e10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 --2- 172.21.15.7:0/3993983968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 -- 172.21.15.7:0/3993983968 >> 172.21.15.7:0/3993983968 conn(0x55ee61adde90 msgr2=0x55ee61ade290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62359e640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6100a2740 0x7fc6100a4c00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009fd0 tx=0x7fc61c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 -- 172.21.15.7:0/3993983968 shutdown_connections 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 -- 172.21.15.7:0/3993983968 wait complete. 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 -- start start 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.189+0000 7fc62be39740 1 Processor -- start 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc62be39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61aecc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc62be39740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61aed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc62be39740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 0x55ee61af4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc62be39740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee61aeb3d0 con 0x55ee61ae8480 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc62be39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee61a03d50 con 0x55ee61ae2920 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc62be39740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee61aeb550 con 0x55ee61ae3310 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc62be39740 10 client.0 _ll_register_callbacks cb 0x55ee61a03720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61aed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc622d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61aecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61aed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:54668/0 (socket says 172.21.15.7:54668) 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc61bfff640 1 -- 172.21.15.7:0/3014987088 learned_addr learned my addr 172.21.15.7:0/3014987088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc61bfff640 1 -- 172.21.15.7:0/3014987088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 msgr2=0x55ee61aecc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc61bfff640 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61aecc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc61bfff640 1 -- 172.21.15.7:0/3014987088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 msgr2=0x55ee61af4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc61bfff640 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 0x55ee61af4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.190+0000 7fc61bfff640 1 -- 172.21.15.7:0/3014987088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee61828620 con 0x55ee61ae3310 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc61bfff640 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61aed180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0a2f40 tx=0x7fc60c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=665c7670e757c0e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc60a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:55:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc60a7fc640 1 -- 172.21.15.7:0/3014987088 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c09a810 con 0x55ee61ae3310 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc60a7fc640 1 -- 172.21.15.7:0/3014987088 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc60c09a120 con 0x55ee61ae3310 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc62be39740 1 -- 172.21.15.7:0/3014987088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc61402c5d0 con 0x55ee61ae3310 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc60a7fc640 1 -- 172.21.15.7:0/3014987088 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc60c09a630 con 0x55ee61ae3310 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc62be39740 10 client.28546 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc62be39740 1 -- 172.21.15.7:0/3014987088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ee61af7580 con 0x55ee61ae3310 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc60a7fc640 1 -- 172.21.15.7:0/3014987088 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc61402c5d0 con 0x55ee61ae3310 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.191+0000 7fc60a7fc640 1 -- 172.21.15.7:0/3014987088 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc60c0cd030 con 0x55ee61ae3310 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.192+0000 7fc62be39740 10 client.28546 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.192+0000 7fc62be39740 10 client.28546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.192+0000 7fc62be39740 20 client.28546 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.192+0000 7fc62be39740 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ee61af8cc0 0x55ee61afb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.192+0000 7fc62be39740 4 client.28546 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.192+0000 7fc62be39740 1 -- 172.21.15.7:0/3014987088 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee61a55050 con 0x55ee61af8cc0 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.192+0000 7fc622d9d640 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ee61af8cc0 0x55ee61afb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.192+0000 7fc622d9d640 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ee61af8cc0 0x55ee61afb0f0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6778216a6208a7b server_cookie=c783ddbca698eb21 in_seq=0 out_seq=0 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.192+0000 7fc60a7fc640 10 client.28546 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc60a7fc640 1 -- 172.21.15.7:0/3014987088 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ee61a55050 con 0x55ee61af8cc0 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc60a7fc640 10 client.28546 handle_command_reply: tid=0 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 client.28546 shutdown 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 -- 172.21.15.7:0/3014987088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 msgr2=0x55ee61aed180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61aed180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0a2f40 tx=0x7fc60c004730 comp rx=0 tx=0).stop 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 -- 172.21.15.7:0/3014987088 shutdown_connections 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ee61af8cc0 0x55ee61afb0f0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 0x55ee61af4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61aed180 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 --2- 172.21.15.7:0/3014987088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61aecc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc60a7fc640 0 client.28546 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 -- 172.21.15.7:0/3014987088 >> 172.21.15.7:0/3014987088 conn(0x55ee61adde90 msgr2=0x55ee61add020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 -- 172.21.15.7:0/3014987088 shutdown_connections 2024-03-26T08:55:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 1 -- 172.21.15.7:0/3014987088 wait complete. 2024-03-26T08:55:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.193+0000 7fc62be39740 20 client.28546 trim_cache size 0 max 16384 2024-03-26T08:55:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc62be39740 1 Processor -- start 2024-03-26T08:55:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc62be39740 1 -- start start 2024-03-26T08:55:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc62be39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc62be39740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61af4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc62be39740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 0x55ee61ae4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc62be39740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241b1390 con 0x55ee61ae8480 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc62be39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241aee90 con 0x55ee61ae2920 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc62be39740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241acf10 con 0x55ee61ae3310 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61af4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc62359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 0x55ee61ae4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61af4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:54670/0 (socket says 172.21.15.7:54670) 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc61bfff640 1 -- 172.21.15.7:0/1664920526 learned_addr learned my addr 172.21.15.7:0/1664920526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.212+0000 7fc622d9d640 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc61bfff640 1 -- 172.21.15.7:0/1664920526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 msgr2=0x55ee61ae3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc61bfff640 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc61bfff640 1 -- 172.21.15.7:0/1664920526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 msgr2=0x55ee61ae4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc61bfff640 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 0x55ee61ae4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc61bfff640 1 -- 172.21.15.7:0/1664920526 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee61ae7a00 con 0x55ee61ae3310 2024-03-26T08:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc622d9d640 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc62359e640 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 0x55ee61ae4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc61bfff640 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61af4110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0976a0 tx=0x7fc60c0a3470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4399a6b12ae29fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc618ff9640 1 -- 172.21.15.7:0/1664920526 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0b3030 con 0x55ee61ae3310 2024-03-26T08:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc618ff9640 1 -- 172.21.15.7:0/1664920526 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc60c09b3d0 con 0x55ee61ae3310 2024-03-26T08:55:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.213+0000 7fc618ff9640 1 -- 172.21.15.7:0/1664920526 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c099eb0 con 0x55ee61ae3310 2024-03-26T08:55:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 -- 172.21.15.7:0/1664920526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 msgr2=0x55ee61af4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61af4110 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0976a0 tx=0x7fc60c0a3470 comp rx=0 tx=0).stop 2024-03-26T08:55:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 -- 172.21.15.7:0/1664920526 shutdown_connections 2024-03-26T08:55:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae8480 0x55ee61ae4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae3310 0x55ee61af4110 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 --2- 172.21.15.7:0/1664920526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae2920 0x55ee61ae3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 -- 172.21.15.7:0/1664920526 >> 172.21.15.7:0/1664920526 conn(0x55ee61adde90 msgr2=0x55ee61ade270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 -- 172.21.15.7:0/1664920526 shutdown_connections 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 -- 172.21.15.7:0/1664920526 wait complete. 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.214+0000 7fc62be39740 1 -- start start 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc62be39740 1 Processor -- start 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc62be39740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae2920 0x55ee61af15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc62be39740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61af1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc62be39740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae8480 0x55ee61af2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc62be39740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241ad510 con 0x55ee61ae3310 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc62be39740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241ae830 con 0x55ee61ae8480 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc62be39740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6241adb70 con 0x55ee61ae2920 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc62be39740 10 client.0 _ll_register_callbacks cb 0x55ee61a03720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc622d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae2920 0x55ee61af15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc61bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61af1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc622d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae2920 0x55ee61af15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:54678/0 (socket says 172.21.15.7:54678) 2024-03-26T08:55:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc622d9d640 1 -- 172.21.15.7:0/4125962429 learned_addr learned my addr 172.21.15.7:0/4125962429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc62359e640 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae8480 0x55ee61af2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc622d9d640 1 -- 172.21.15.7:0/4125962429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae8480 msgr2=0x55ee61af2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc622d9d640 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae8480 0x55ee61af2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc622d9d640 1 -- 172.21.15.7:0/4125962429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 msgr2=0x55ee61af1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc622d9d640 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61af1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc622d9d640 1 -- 172.21.15.7:0/4125962429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee61af3120 con 0x55ee61ae2920 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.215+0000 7fc61bfff640 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61af1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc622d9d640 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae2920 0x55ee61af15e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc61402ce70 tx=0x7fc614005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b57fb1aecf096b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc60a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc60a7fc640 1 -- 172.21.15.7:0/4125962429 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614004340 con 0x55ee61ae2920 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc62be39740 1 -- 172.21.15.7:0/4125962429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc60c09b3d0 con 0x55ee61ae2920 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc60a7fc640 1 -- 172.21.15.7:0/4125962429 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc614005040 con 0x55ee61ae2920 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc60a7fc640 1 -- 172.21.15.7:0/4125962429 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6140044e0 con 0x55ee61ae2920 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc62be39740 10 client.28552 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc62be39740 1 -- 172.21.15.7:0/4125962429 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ee61b545c0 con 0x55ee61ae2920 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc60a7fc640 1 -- 172.21.15.7:0/4125962429 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc60c09b3d0 con 0x55ee61ae2920 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc60a7fc640 1 -- 172.21.15.7:0/4125962429 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc61404e600 con 0x55ee61ae2920 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc62be39740 10 client.28552 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc62be39740 10 client.28552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc62be39740 20 client.28552 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc62be39740 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ee61b54a20 0x55ee61b56e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc62be39740 4 client.28552 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.216+0000 7fc62be39740 1 -- 172.21.15.7:0/4125962429 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee61a4f3a0 con 0x55ee61b54a20 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.217+0000 7fc61bfff640 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ee61b54a20 0x55ee61b56e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.217+0000 7fc61bfff640 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ee61b54a20 0x55ee61b56e40 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84936d3b43851796 server_cookie=23d8ad493d4f1a2b in_seq=0 out_seq=0 2024-03-26T08:55:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.217+0000 7fc60a7fc640 10 client.28552 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.336+0000 7fc60a7fc640 1 -- 172.21.15.7:0/4125962429 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee61a4f3a0 con 0x55ee61b54a20 2024-03-26T08:55:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.336+0000 7fc60a7fc640 10 client.28552 handle_command_reply: tid=0 2024-03-26T08:55:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.336+0000 7fc62be39740 1 client.28552 shutdown 2024-03-26T08:55:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.336+0000 7fc62be39740 1 -- 172.21.15.7:0/4125962429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae2920 msgr2=0x55ee61af15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.336+0000 7fc62be39740 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae2920 0x55ee61af15e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc61402ce70 tx=0x7fc614005d50 comp rx=0 tx=0).stop 2024-03-26T08:55:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.337+0000 7fc62be39740 1 -- 172.21.15.7:0/4125962429 shutdown_connections 2024-03-26T08:55:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.337+0000 7fc62be39740 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55ee61b54a20 0x55ee61b56e40 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.337+0000 7fc60a7fc640 0 client.28552 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.337+0000 7fc62be39740 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee61ae8480 0x55ee61af2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.337+0000 7fc62be39740 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee61ae3310 0x55ee61af1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.337+0000 7fc62be39740 1 --2- 172.21.15.7:0/4125962429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55ee61ae2920 0x55ee61af15e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.337+0000 7fc62be39740 1 -- 172.21.15.7:0/4125962429 >> 172.21.15.7:0/4125962429 conn(0x55ee61adde90 msgr2=0x55ee61aeca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.337+0000 7fc62be39740 1 -- 172.21.15.7:0/4125962429 shutdown_connections 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.337+0000 7fc62be39740 1 -- 172.21.15.7:0/4125962429 wait complete. 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.338+0000 7fc62be39740 20 client.28552 trim_cache size 0 max 16384 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:55:03.336 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:55:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.338+0000 7fc62924b640 1 -- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6100a2740 msgr2=0x7fc6100a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.338+0000 7fc62924b640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6100a2740 0x7fc6100a4c00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009fd0 tx=0x7fc61c009500 comp rx=0 tx=0).stop 2024-03-26T08:55:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.338+0000 7fc62924b640 1 -- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6241006b0 msgr2=0x7fc62419e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.338+0000 7fc62924b640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6241006b0 0x7fc62419e4f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00d950 tx=0x7fc60c00de10 comp rx=0 tx=0).stop 2024-03-26T08:55:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.339+0000 7fc62924b640 1 -- 172.21.15.7:0/3743473642 shutdown_connections 2024-03-26T08:55:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.339+0000 7fc62924b640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6100a2740 0x7fc6100a4c00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.339+0000 7fc62924b640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc624101050 0x7fc6241a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.339+0000 7fc62924b640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6241006b0 0x7fc62419e4f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.339+0000 7fc62924b640 1 --2- 172.21.15.7:0/3743473642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc6240ff580 0x7fc62419dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.339+0000 7fc62924b640 1 -- 172.21.15.7:0/3743473642 >> 172.21.15.7:0/3743473642 conn(0x7fc6240fae80 msgr2=0x7fc6241018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.339+0000 7fc62924b640 1 -- 172.21.15.7:0/3743473642 shutdown_connections 2024-03-26T08:55:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:03.339+0000 7fc62924b640 1 -- 172.21.15.7:0/3743473642 wait complete. 2024-03-26T08:55:04.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:04 smithi016 ceph-mon[35851]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 24 MiB/s wr, 1.36k op/s 2024-03-26T08:55:04.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:04 smithi007 ceph-mon[32462]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 24 MiB/s wr, 1.36k op/s 2024-03-26T08:55:04.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:04 smithi044 ceph-mon[36461]: pgmap v739: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 24 MiB/s wr, 1.36k op/s 2024-03-26T08:55:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 755.0 s, 1925.4 tps, lat 51.423 ms stddev 60.287 2024-03-26T08:55:06.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:06 smithi016 ceph-mon[35851]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.0 MiB/s wr, 689 op/s 2024-03-26T08:55:06.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:06 smithi007 ceph-mon[32462]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.0 MiB/s wr, 689 op/s 2024-03-26T08:55:06.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:06 smithi044 ceph-mon[36461]: pgmap v740: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.0 MiB/s wr, 689 op/s 2024-03-26T08:55:08.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:08 smithi016 ceph-mon[35851]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.6 MiB/s wr, 663 op/s 2024-03-26T08:55:08.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:08 smithi007 ceph-mon[32462]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.6 MiB/s wr, 663 op/s 2024-03-26T08:55:08.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:08 smithi044 ceph-mon[36461]: pgmap v741: 97 pgs: 97 active+clean; 1.6 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.6 MiB/s wr, 663 op/s 2024-03-26T08:55:10.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:10 smithi016 ceph-mon[35851]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 9.4 MiB/s wr, 734 op/s 2024-03-26T08:55:10.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:10 smithi007 ceph-mon[32462]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 9.4 MiB/s wr, 734 op/s 2024-03-26T08:55:10.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:10 smithi044 ceph-mon[36461]: pgmap v742: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 9.4 MiB/s wr, 734 op/s 2024-03-26T08:55:11.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 760.0 s, 1813.0 tps, lat 55.601 ms stddev 64.420 2024-03-26T08:55:12.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:12 smithi016 ceph-mon[35851]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.9 MiB/s wr, 660 op/s 2024-03-26T08:55:12.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:12 smithi007 ceph-mon[32462]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.9 MiB/s wr, 660 op/s 2024-03-26T08:55:12.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:12 smithi044 ceph-mon[36461]: pgmap v743: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.9 MiB/s wr, 660 op/s 2024-03-26T08:55:14.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:14 smithi016 ceph-mon[35851]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 651 op/s 2024-03-26T08:55:14.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:14 smithi007 ceph-mon[32462]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 651 op/s 2024-03-26T08:55:14.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:14 smithi044 ceph-mon[36461]: pgmap v744: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 651 op/s 2024-03-26T08:55:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 765.0 s, 1631.8 tps, lat 61.275 ms stddev 72.518 2024-03-26T08:55:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 -- 172.21.15.7:0/3668987046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541017d0 msgr2=0x7f5c54101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 --2- 172.21.15.7:0/3668987046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541017d0 0x7f5c54101c50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009900 tx=0x7f5c4402bc30 comp rx=0 tx=0).stop 2024-03-26T08:55:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 -- 172.21.15.7:0/3668987046 shutdown_connections 2024-03-26T08:55:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 --2- 172.21.15.7:0/3668987046 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c54102190 0x7f5c54108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.467 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 --2- 172.21.15.7:0/3668987046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541017d0 0x7f5c54101c50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 --2- 172.21.15.7:0/3668987046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c541005e0 0x7f5c54100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 -- 172.21.15.7:0/3668987046 >> 172.21.15.7:0/3668987046 conn(0x7f5c540fbb70 msgr2=0x7f5c540fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 -- 172.21.15.7:0/3668987046 shutdown_connections 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 -- 172.21.15.7:0/3668987046 wait complete. 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 Processor -- start 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.467+0000 7f5c5b461640 1 -- start start 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c5b461640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541005e0 0x7f5c54199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c5b461640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c541017d0 0x7f5c5419a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c5b461640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c54102190 0x7f5c541a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c5b461640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5410b160 con 0x7f5c541005e0 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c5b461640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5410afe0 con 0x7f5c54102190 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c5b461640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5410b2e0 con 0x7f5c541017d0 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c591fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541005e0 0x7f5c54199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c591fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541005e0 0x7f5c54199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50644/0 (socket says 172.21.15.7:50644) 2024-03-26T08:55:16.468 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c591fd640 1 -- 172.21.15.7:0/1622246150 learned_addr learned my addr 172.21.15.7:0/1622246150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c589fc640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c541017d0 0x7f5c5419a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c599fe640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c54102190 0x7f5c541a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c591fd640 1 -- 172.21.15.7:0/1622246150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c541017d0 msgr2=0x7f5c5419a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c591fd640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c541017d0 0x7f5c5419a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c591fd640 1 -- 172.21.15.7:0/1622246150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c54102190 msgr2=0x7f5c541a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c591fd640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c54102190 0x7f5c541a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c591fd640 1 -- 172.21.15.7:0/1622246150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c541a1850 con 0x7f5c541005e0 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c591fd640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541005e0 0x7f5c54199c40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000d770 tx=0x7f5c5000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c4a7fc640 1 -- 172.21.15.7:0/1622246150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50017070 con 0x7f5c541005e0 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c4a7fc640 1 -- 172.21.15.7:0/1622246150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c50009c30 con 0x7f5c541005e0 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c5b461640 1 -- 172.21.15.7:0/1622246150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c541a1b40 con 0x7f5c541005e0 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.468+0000 7f5c4a7fc640 1 -- 172.21.15.7:0/1622246150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50003dd0 con 0x7f5c541005e0 2024-03-26T08:55:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.469+0000 7f5c5b461640 1 -- 172.21.15.7:0/1622246150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c541a20e0 con 0x7f5c541005e0 2024-03-26T08:55:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.471+0000 7f5c4a7fc640 1 -- 172.21.15.7:0/1622246150 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5c50003fb0 con 0x7f5c541005e0 2024-03-26T08:55:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c4a7fc640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5c400a2870 0x7f5c400a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c5e053740 1 Processor -- start 2024-03-26T08:55:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c5e053740 1 -- start start 2024-03-26T08:55:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c589fc640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5c400a2870 0x7f5c400a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c4a7fc640 1 -- 172.21.15.7:0/1622246150 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5c500ce420 con 0x7f5c541005e0 2024-03-26T08:55:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952bc920 0x5564952bcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bd310 0x5564952c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952c2480 0x5564952c4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c5e053740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556495221b10 con 0x5564952bc920 2024-03-26T08:55:16.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c5e053740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564951ca900 con 0x5564952bd310 2024-03-26T08:55:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.472+0000 7f5c5e053740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649521dbb0 con 0x5564952c2480 2024-03-26T08:55:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c591fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952bc920 0x5564952bcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c591fd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952bc920 0x5564952bcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50646/0 (socket says 172.21.15.7:50646) 2024-03-26T08:55:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c591fd640 1 -- 172.21.15.7:0/50989286 learned_addr learned my addr 172.21.15.7:0/50989286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c599fe640 1 --2- 172.21.15.7:0/50989286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952c2480 0x5564952c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c589fc640 1 --2- 172.21.15.7:0/50989286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bd310 0x5564952c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c591fd640 1 -- 172.21.15.7:0/50989286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952c2480 msgr2=0x5564952c4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c591fd640 1 --2- 172.21.15.7:0/50989286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952c2480 0x5564952c4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c591fd640 1 -- 172.21.15.7:0/50989286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bd310 msgr2=0x5564952c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c591fd640 1 --2- 172.21.15.7:0/50989286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bd310 0x5564952c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c591fd640 1 -- 172.21.15.7:0/50989286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556494f36210 con 0x5564952bc920 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c589fc640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5c400a2870 0x7f5c400a4d30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5c4402c7b0 tx=0x7f5c44005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.473+0000 7f5c591fd640 1 --2- 172.21.15.7:0/50989286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952bc920 0x5564952bcd40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5c50097150 tx=0x7f5c50004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac90e53c0738dd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.474+0000 7f5c3f7fe640 1 -- 172.21.15.7:0/50989286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c50020040 con 0x5564952bc920 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.474+0000 7f5c3f7fe640 1 -- 172.21.15.7:0/50989286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c5009b480 con 0x5564952bc920 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.474+0000 7f5c3f7fe640 1 -- 172.21.15.7:0/50989286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5009a1d0 con 0x5564952bc920 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.474+0000 7f5c5e053740 1 -- 172.21.15.7:0/50989286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952bc920 msgr2=0x5564952bcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.474+0000 7f5c5e053740 1 --2- 172.21.15.7:0/50989286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952bc920 0x5564952bcd40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5c50097150 tx=0x7f5c50004590 comp rx=0 tx=0).stop 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.475+0000 7f5c5e053740 1 -- 172.21.15.7:0/50989286 shutdown_connections 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.475+0000 7f5c5e053740 1 --2- 172.21.15.7:0/50989286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952c2480 0x5564952c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.475+0000 7f5c5e053740 1 --2- 172.21.15.7:0/50989286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bd310 0x5564952c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.475+0000 7f5c5e053740 1 --2- 172.21.15.7:0/50989286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952bc920 0x5564952bcd40 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.475+0000 7f5c5e053740 1 -- 172.21.15.7:0/50989286 >> 172.21.15.7:0/50989286 conn(0x5564952b7e90 msgr2=0x5564952b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.475+0000 7f5c5e053740 1 -- 172.21.15.7:0/50989286 shutdown_connections 2024-03-26T08:55:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.475+0000 7f5c5e053740 1 -- 172.21.15.7:0/50989286 wait complete. 2024-03-26T08:55:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.475+0000 7f5c5e053740 1 -- start start 2024-03-26T08:55:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.475+0000 7f5c5e053740 1 Processor -- start 2024-03-26T08:55:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c5e053740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564952c5420 con 0x5564952c2480 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c5e053740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564951ddd50 con 0x5564952bc920 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c5e053740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564952c55a0 con 0x5564952bd310 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c589fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c599fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c589fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:38854/0 (socket says 172.21.15.7:38854) 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c591fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c589fc640 1 -- 172.21.15.7:0/784505967 learned_addr learned my addr 172.21.15.7:0/784505967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c5e053740 10 client.0 _ll_register_callbacks cb 0x5564951dd720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c599fe640 1 -- 172.21.15.7:0/784505967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 msgr2=0x5564952c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c599fe640 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c599fe640 1 -- 172.21.15.7:0/784505967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 msgr2=0x5564952c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:16.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c599fe640 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c599fe640 1 -- 172.21.15.7:0/784505967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556495002620 con 0x5564952c2480 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c589fc640 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c599fe640 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952ce190 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c005fd0 tx=0x7f5c4c00ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b33abbd4e095ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.476+0000 7f5c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c3cff9640 1 -- 172.21.15.7:0/784505967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c004d20 con 0x5564952c2480 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c5e053740 1 -- 172.21.15.7:0/784505967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c5009b480 con 0x5564952c2480 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c3cff9640 1 -- 172.21.15.7:0/784505967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5c4c009a30 con 0x5564952c2480 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c3cff9640 1 -- 172.21.15.7:0/784505967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c4c009c30 con 0x5564952c2480 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c5e053740 10 client.18780 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c5e053740 1 -- 172.21.15.7:0/784505967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5564952d15b0 con 0x5564952c2480 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c3cff9640 1 -- 172.21.15.7:0/784505967 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c5009b480 con 0x5564952c2480 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c3cff9640 1 -- 172.21.15.7:0/784505967 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5c4c016400 con 0x5564952c2480 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c5e053740 10 client.18780 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c5e053740 10 client.18780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c5e053740 20 client.18780 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:16.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c5e053740 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5564952d2d90 0x5564952d51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c5e053740 4 client.18780 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.477+0000 7f5c5e053740 1 -- 172.21.15.7:0/784505967 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55649522f050 con 0x5564952d2d90 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.478+0000 7f5c591fd640 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5564952d2d90 0x5564952d51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.478+0000 7f5c3cff9640 10 client.18780 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.478+0000 7f5c591fd640 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5564952d2d90 0x5564952d51d0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969f54fe128ef177 server_cookie=72148dddfe8b596f in_seq=0 out_seq=0 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c3cff9640 1 -- 172.21.15.7:0/784505967 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55649522f050 con 0x5564952d2d90 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c3cff9640 10 client.18780 handle_command_reply: tid=0 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c5e053740 1 client.18780 shutdown 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c5e053740 1 -- 172.21.15.7:0/784505967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 msgr2=0x5564952ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c5e053740 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952ce190 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c005fd0 tx=0x7f5c4c00ae10 comp rx=0 tx=0).stop 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c5e053740 1 -- 172.21.15.7:0/784505967 shutdown_connections 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c5e053740 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5564952d2d90 0x5564952d51d0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c5e053740 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952ce190 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c3cff9640 0 client.18780 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:16.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c5e053740 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.479+0000 7f5c5e053740 1 --2- 172.21.15.7:0/784505967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.480+0000 7f5c5e053740 1 -- 172.21.15.7:0/784505967 >> 172.21.15.7:0/784505967 conn(0x5564952b7e90 msgr2=0x5564952b7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.480+0000 7f5c5e053740 1 -- 172.21.15.7:0/784505967 shutdown_connections 2024-03-26T08:55:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.480+0000 7f5c5e053740 1 -- 172.21.15.7:0/784505967 wait complete. 2024-03-26T08:55:16.482 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.480+0000 7f5c5e053740 20 client.18780 trim_cache size 0 max 16384 2024-03-26T08:55:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c5e053740 1 Processor -- start 2024-03-26T08:55:16.496 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c5e053740 1 -- start start 2024-03-26T08:55:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.497 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c5e053740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541ad2c0 con 0x5564952c2480 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c5e053740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541aadc0 con 0x5564952bc920 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c5e053740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541a8e40 con 0x5564952bd310 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c599fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c599fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50660/0 (socket says 172.21.15.7:50660) 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.498+0000 7f5c599fe640 1 -- 172.21.15.7:0/2439738826 learned_addr learned my addr 172.21.15.7:0/2439738826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c589fc640 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c591fd640 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c599fe640 1 -- 172.21.15.7:0/2439738826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 msgr2=0x5564952ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c599fe640 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c599fe640 1 -- 172.21.15.7:0/2439738826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 msgr2=0x5564952c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c599fe640 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.498 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c599fe640 1 -- 172.21.15.7:0/2439738826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564952b8c00 con 0x5564952c2480 2024-03-26T08:55:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c591fd640 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:55:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c599fe640 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952c7150 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00bdd0 tx=0x7f5c4c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7eb07324eb836a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c3effd640 1 -- 172.21.15.7:0/2439738826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c02e070 con 0x5564952c2480 2024-03-26T08:55:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c3effd640 1 -- 172.21.15.7:0/2439738826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c016ab0 con 0x5564952c2480 2024-03-26T08:55:16.499 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.499+0000 7f5c3effd640 1 -- 172.21.15.7:0/2439738826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c025040 con 0x5564952c2480 2024-03-26T08:55:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 -- 172.21.15.7:0/2439738826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 msgr2=0x5564952c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952c7150 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00bdd0 tx=0x7f5c4c005c50 comp rx=0 tx=0).stop 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 -- 172.21.15.7:0/2439738826 shutdown_connections 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952c7150 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 --2- 172.21.15.7:0/2439738826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 -- 172.21.15.7:0/2439738826 >> 172.21.15.7:0/2439738826 conn(0x5564952b7e90 msgr2=0x5564952b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 -- 172.21.15.7:0/2439738826 shutdown_connections 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 -- 172.21.15.7:0/2439738826 wait complete. 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.500+0000 7f5c5e053740 1 -- start start 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c5e053740 1 Processor -- start 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c5e053740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c5e053740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541a9440 con 0x5564952c2480 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c5e053740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541aa760 con 0x5564952bc920 2024-03-26T08:55:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c5e053740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c541a9aa0 con 0x5564952bd310 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c5e053740 10 client.0 _ll_register_callbacks cb 0x7f5c4c00a540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c591fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c599fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c589fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c591fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39188/0 (socket says 172.21.15.7:39188) 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c591fd640 1 -- 172.21.15.7:0/767472603 learned_addr learned my addr 172.21.15.7:0/767472603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c599fe640 1 -- 172.21.15.7:0/767472603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 msgr2=0x5564952cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c599fe640 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c599fe640 1 -- 172.21.15.7:0/767472603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 msgr2=0x5564952cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c599fe640 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.502 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.501+0000 7f5c599fe640 1 -- 172.21.15.7:0/767472603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564952cd4a0 con 0x5564952c2480 2024-03-26T08:55:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.502+0000 7f5c599fe640 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952cceb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0168c0 tx=0x7f5c4c004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866b2b41321f1936 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.504+0000 7f5c3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.504+0000 7f5c3cff9640 1 -- 172.21.15.7:0/767472603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c02e070 con 0x5564952c2480 2024-03-26T08:55:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.504+0000 7f5c3cff9640 1 -- 172.21.15.7:0/767472603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5c4c005450 con 0x5564952c2480 2024-03-26T08:55:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.504+0000 7f5c5e053740 1 -- 172.21.15.7:0/767472603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c4c016ab0 con 0x5564952c2480 2024-03-26T08:55:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.504+0000 7f5c3cff9640 1 -- 172.21.15.7:0/767472603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c4c025040 con 0x5564952c2480 2024-03-26T08:55:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.504+0000 7f5c5e053740 10 client.18786 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.504+0000 7f5c5e053740 1 -- 172.21.15.7:0/767472603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55649532e760 con 0x5564952c2480 2024-03-26T08:55:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c3cff9640 1 -- 172.21.15.7:0/767472603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c4c016c60 con 0x5564952c2480 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c3cff9640 1 -- 172.21.15.7:0/767472603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5c4c03c030 con 0x5564952c2480 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c5e053740 10 client.18786 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c5e053740 10 client.18786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c5e053740 20 client.18786 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c5e053740 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5564952d2d90 0x556495330fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c5e053740 4 client.18786 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c5e053740 1 -- 172.21.15.7:0/767472603 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f5c4c016ab0 con 0x5564952d2d90 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c591fd640 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5564952d2d90 0x556495330fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c3cff9640 10 client.18786 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.505+0000 7f5c591fd640 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5564952d2d90 0x556495330fe0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e19baa51e1b26e server_cookie=669b42491bce3598 in_seq=0 out_seq=0 2024-03-26T08:55:16.520 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.521+0000 7f5c3cff9640 1 -- 172.21.15.7:0/767472603 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f5c4c016ab0 con 0x5564952d2d90 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c3cff9640 10 client.18786 handle_command_reply: tid=0 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 client.18786 shutdown 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 -- 172.21.15.7:0/767472603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 msgr2=0x5564952cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952cceb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c0168c0 tx=0x7f5c4c004ef0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 -- 172.21.15.7:0/767472603 shutdown_connections 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5564952d2d90 0x556495330fe0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5564952c2480 0x5564952cceb0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5564952bd310 0x5564952cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 --2- 172.21.15.7:0/767472603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5564952bc920 0x5564952cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 -- 172.21.15.7:0/767472603 >> 172.21.15.7:0/767472603 conn(0x5564952b7e90 msgr2=0x5564952c6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c3cff9640 0 client.18786 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 -- 172.21.15.7:0/767472603 shutdown_connections 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 1 -- 172.21.15.7:0/767472603 wait complete. 2024-03-26T08:55:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.522+0000 7f5c5e053740 20 client.18786 trim_cache size 0 max 16384 2024-03-26T08:55:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:55:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:55:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:55:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 -- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5c400a2870 msgr2=0x7f5c400a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5c400a2870 0x7f5c400a4d30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5c4402c7b0 tx=0x7f5c44005d50 comp rx=0 tx=0).stop 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 -- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541005e0 msgr2=0x7f5c54199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541005e0 0x7f5c54199c40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5c5000d770 tx=0x7f5c5000dc30 comp rx=0 tx=0).stop 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 -- 172.21.15.7:0/1622246150 shutdown_connections 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5c400a2870 0x7f5c400a4d30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c54102190 0x7f5c541a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5c541017d0 0x7f5c5419a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 --2- 172.21.15.7:0/1622246150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c541005e0 0x7f5c54199c40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 -- 172.21.15.7:0/1622246150 >> 172.21.15.7:0/1622246150 conn(0x7f5c540fbb70 msgr2=0x7f5c54190820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 -- 172.21.15.7:0/1622246150 shutdown_connections 2024-03-26T08:55:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:16.523+0000 7f5c5b461640 1 -- 172.21.15.7:0/1622246150 wait complete. 2024-03-26T08:55:16.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:16 smithi016 ceph-mon[35851]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.1 MiB/s wr, 610 op/s 2024-03-26T08:55:16.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:16 smithi007 ceph-mon[32462]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.1 MiB/s wr, 610 op/s 2024-03-26T08:55:16.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:16 smithi044 ceph-mon[36461]: pgmap v745: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.1 MiB/s wr, 610 op/s 2024-03-26T08:55:18.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:18 smithi016 ceph-mon[35851]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-03-26T08:55:18.797 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:18 smithi007 ceph-mon[32462]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-03-26T08:55:18.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:18 smithi044 ceph-mon[36461]: pgmap v746: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-03-26T08:55:20.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:20 smithi016 ceph-mon[35851]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 MiB/s wr, 699 op/s 2024-03-26T08:55:20.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:20 smithi007 ceph-mon[32462]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 MiB/s wr, 699 op/s 2024-03-26T08:55:20.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:20 smithi044 ceph-mon[36461]: pgmap v747: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 MiB/s wr, 699 op/s 2024-03-26T08:55:21.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 770.0 s, 1560.2 tps, lat 63.909 ms stddev 75.027 2024-03-26T08:55:22.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:22 smithi016 ceph-mon[35851]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 MiB/s wr, 707 op/s 2024-03-26T08:55:22.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:22 smithi007 ceph-mon[32462]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 MiB/s wr, 707 op/s 2024-03-26T08:55:22.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:22 smithi044 ceph-mon[36461]: pgmap v748: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.7 MiB/s wr, 707 op/s 2024-03-26T08:55:24.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:55:24.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:55:24.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.419+0000 7fadf2f3b640 1 -- 172.21.15.7:0/2574581444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1016d0 msgr2=0x7fadec101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.419+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/2574581444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1016d0 0x7fadec101af0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fade0009900 tx=0x7fade002bc60 comp rx=0 tx=0).stop 2024-03-26T08:55:24.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.419+0000 7fadf2f3b640 1 -- 172.21.15.7:0/2574581444 shutdown_connections 2024-03-26T08:55:24.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.419+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/2574581444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadec103280 0x7fadec109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.419+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/2574581444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadec1028c0 0x7fadec102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.419+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/2574581444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1016d0 0x7fadec101af0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.419+0000 7fadf2f3b640 1 -- 172.21.15.7:0/2574581444 >> 172.21.15.7:0/2574581444 conn(0x7fadec0fcc60 msgr2=0x7fadec0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:24.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.419+0000 7fadf2f3b640 1 -- 172.21.15.7:0/2574581444 shutdown_connections 2024-03-26T08:55:24.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.419+0000 7fadf2f3b640 1 -- 172.21.15.7:0/2574581444 wait complete. 2024-03-26T08:55:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf2f3b640 1 Processor -- start 2024-03-26T08:55:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf2f3b640 1 -- start start 2024-03-26T08:55:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf2f3b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadec1016d0 0x7fadec197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf2f3b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1028c0 0x7fadec198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf2f3b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadec103280 0x7fadec19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf2f3b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec10d3c0 con 0x7fadec1028c0 2024-03-26T08:55:24.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf2f3b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec10d240 con 0x7fadec103280 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf2f3b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec10d540 con 0x7fadec1016d0 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1028c0 0x7fadec198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1028c0 0x7fadec198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53580/0 (socket says 172.21.15.7:53580) 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadebfff640 1 -- 172.21.15.7:0/4195693706 learned_addr learned my addr 172.21.15.7:0/4195693706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadec1016d0 0x7fadec197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.420+0000 7fadf14d8640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadec103280 0x7fadec19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fadebfff640 1 -- 172.21.15.7:0/4195693706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadec1016d0 msgr2=0x7fadec197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fadebfff640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadec1016d0 0x7fadec197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fadebfff640 1 -- 172.21.15.7:0/4195693706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadec103280 msgr2=0x7fadec19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fadebfff640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadec103280 0x7fadec19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fadebfff640 1 -- 172.21.15.7:0/4195693706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec19f700 con 0x7fadec1028c0 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fadebfff640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1028c0 0x7fadec198020 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fadd800d950 tx=0x7fadd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fade9ffb640 1 -- 172.21.15.7:0/4195693706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8017030 con 0x7fadec1028c0 2024-03-26T08:55:24.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fadf2f3b640 1 -- 172.21.15.7:0/4195693706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec19f9f0 con 0x7fadec1028c0 2024-03-26T08:55:24.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fade9ffb640 1 -- 172.21.15.7:0/4195693706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadd8004020 con 0x7fadec1028c0 2024-03-26T08:55:24.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.421+0000 7fadf2f3b640 1 -- 172.21.15.7:0/4195693706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec19ff40 con 0x7fadec1028c0 2024-03-26T08:55:24.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.422+0000 7fade9ffb640 1 -- 172.21.15.7:0/4195693706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd80047a0 con 0x7fadec1028c0 2024-03-26T08:55:24.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.424+0000 7fade9ffb640 1 -- 172.21.15.7:0/4195693706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fadd8009a30 con 0x7fadec1028c0 2024-03-26T08:55:24.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.425+0000 7fadf5c04740 1 Processor -- start 2024-03-26T08:55:24.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.425+0000 7fadf5c04740 1 -- start start 2024-03-26T08:55:24.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.425+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56213807dba0 0x56213807dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.425+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 0x562138082b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.425+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562138083190 0x562138085600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.425+0000 7fadf5c04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562137fda3d0 con 0x56213807dba0 2024-03-26T08:55:24.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.425+0000 7fadf5c04740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562137f82ad0 con 0x56213807e590 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.425+0000 7fadf5c04740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562137fd6470 con 0x562138083190 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562138083190 0x562138085600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadf0cd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 0x562138082b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadebfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562138083190 0x562138085600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44358/0 (socket says 172.21.15.7:44358) 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadebfff640 1 -- 172.21.15.7:0/3733687555 learned_addr learned my addr 172.21.15.7:0/3733687555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadf0cd7640 1 -- 172.21.15.7:0/3733687555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562138083190 msgr2=0x562138085600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/3733687555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562138083190 0x562138085600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadf0cd7640 1 -- 172.21.15.7:0/3733687555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56213807dba0 msgr2=0x56213807dfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fade9ffb640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faddc0a26e0 0x7faddc0a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/3733687555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56213807dba0 0x56213807dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadf0cd7640 1 -- 172.21.15.7:0/3733687555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562137db87d0 con 0x56213807e590 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.426+0000 7fadebfff640 1 --2- 172.21.15.7:0/3733687555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562138083190 0x562138085600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:24.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.427+0000 7fadebfff640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faddc0a26e0 0x7faddc0a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.427+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/3733687555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 0x562138082b20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fade0036040 tx=0x7fade002c600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212476753d28c21d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:24.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.427+0000 7fade9ffb640 1 -- 172.21.15.7:0/4195693706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fadd80ce450 con 0x7fadec1028c0 2024-03-26T08:55:24.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.427+0000 7fadd6ffd640 1 -- 172.21.15.7:0/3733687555 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade002caf0 con 0x56213807e590 2024-03-26T08:55:24.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.427+0000 7fadd6ffd640 1 -- 172.21.15.7:0/3733687555 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fade002cc90 con 0x56213807e590 2024-03-26T08:55:24.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.427+0000 7fadd6ffd640 1 -- 172.21.15.7:0/3733687555 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0005150 con 0x56213807e590 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadf5c04740 1 -- 172.21.15.7:0/3733687555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 msgr2=0x562138082b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3733687555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 0x562138082b20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fade0036040 tx=0x7fade002c600 comp rx=0 tx=0).stop 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadebfff640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faddc0a26e0 0x7faddc0a4ba0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009610 tx=0x7fadd80d1e10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadf5c04740 1 -- 172.21.15.7:0/3733687555 shutdown_connections 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3733687555 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562138083190 0x562138085600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3733687555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 0x562138082b20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3733687555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56213807dba0 0x56213807dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadf5c04740 1 -- 172.21.15.7:0/3733687555 >> 172.21.15.7:0/3733687555 conn(0x562138070290 msgr2=0x562138070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadf5c04740 1 -- 172.21.15.7:0/3733687555 shutdown_connections 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.428+0000 7fadf5c04740 1 -- 172.21.15.7:0/3733687555 wait complete. 2024-03-26T08:55:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf5c04740 1 -- start start 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf5c04740 1 Processor -- start 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x5621380879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x562138087f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213808ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf5c04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621380860c0 con 0x562138083190 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf5c04740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562137f96060 con 0x56213807dba0 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf5c04740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562138086240 con 0x56213807e590 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf5c04740 10 client.0 _ll_register_callbacks cb 0x562137f786a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf14d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213808ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf0cd7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x562138087f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf14d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x5621380879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf0cd7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x562138087f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44360/0 (socket says 172.21.15.7:44360) 2024-03-26T08:55:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf14d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x5621380879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52654/0 (socket says 172.21.15.7:52654) 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.429+0000 7fadf0cd7640 1 -- 172.21.15.7:0/2087380207 learned_addr learned my addr 172.21.15.7:0/2087380207 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf14d8640 1 -- 172.21.15.7:0/2087380207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 msgr2=0x562138087f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf14d8640 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x562138087f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf14d8640 1 -- 172.21.15.7:0/2087380207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 msgr2=0x56213808ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf14d8640 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213808ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf14d8640 1 -- 172.21.15.7:0/2087380207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562137cee3f0 con 0x56213807dba0 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x562138087f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf14d8640 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x5621380879e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fade4009440 tx=0x7fade4007040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d938b5f6d240ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadcbfff640 1 -- 172.21.15.7:0/2087380207 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4014040 con 0x56213807dba0 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf5c04740 1 -- 172.21.15.7:0/2087380207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fade002cc90 con 0x56213807dba0 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadcbfff640 1 -- 172.21.15.7:0/2087380207 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fade400b250 con 0x56213807dba0 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadcbfff640 1 -- 172.21.15.7:0/2087380207 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fade40155c0 con 0x56213807dba0 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf5c04740 10 client.28583 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.430+0000 7fadf5c04740 1 -- 172.21.15.7:0/2087380207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562137cae0f0 con 0x56213807dba0 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadcbfff640 1 -- 172.21.15.7:0/2087380207 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fade002cc90 con 0x56213807dba0 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadcbfff640 1 -- 172.21.15.7:0/2087380207 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fade401b5c0 con 0x56213807dba0 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadf5c04740 10 client.28583 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadf5c04740 10 client.28583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadf5c04740 20 client.28583 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadf5c04740 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562138093520 0x562138095960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadf5c04740 4 client.28583 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadf5c04740 1 -- 172.21.15.7:0/2087380207 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562137fe7910 con 0x562138093520 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562138093520 0x562138095960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadcbfff640 10 client.28583 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:24.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.431+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562138093520 0x562138095960 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c657c7ca40a51ea server_cookie=c5978985a9333e7 in_seq=0 out_seq=0 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.432+0000 7fadcbfff640 1 -- 172.21.15.7:0/2087380207 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562137fe7910 con 0x562138093520 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.432+0000 7fadcbfff640 10 client.28583 handle_command_reply: tid=0 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 client.28583 shutdown 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 -- 172.21.15.7:0/2087380207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 msgr2=0x5621380879e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x5621380879e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fade4009440 tx=0x7fade4007040 comp rx=0 tx=0).stop 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 -- 172.21.15.7:0/2087380207 shutdown_connections 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562138093520 0x562138095960 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadcbfff640 0 client.28583 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213808ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x562138087f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 --2- 172.21.15.7:0/2087380207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x5621380879e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 -- 172.21.15.7:0/2087380207 >> 172.21.15.7:0/2087380207 conn(0x562138070290 msgr2=0x56213807d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:24.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 -- 172.21.15.7:0/2087380207 shutdown_connections 2024-03-26T08:55:24.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.433+0000 7fadf5c04740 1 -- 172.21.15.7:0/2087380207 wait complete. 2024-03-26T08:55:24.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.434+0000 7fadf5c04740 20 client.28583 trim_cache size 0 max 16384 2024-03-26T08:55:24.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf5c04740 1 Processor -- start 2024-03-26T08:55:24.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf5c04740 1 -- start start 2024-03-26T08:55:24.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x562138087e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x56213808ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf5c04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1a8c50 con 0x562138083190 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf5c04740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1a6750 con 0x56213807dba0 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf5c04740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1a47d0 con 0x56213807e590 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf14d8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf0cd7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x56213808ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf0cd7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x56213808ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44368/0 (socket says 172.21.15.7:44368) 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf14d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x562138087e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf0cd7640 1 -- 172.21.15.7:0/3311659475 learned_addr learned my addr 172.21.15.7:0/3311659475 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf0cd7640 1 -- 172.21.15.7:0/3311659475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 msgr2=0x562138087e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/3311659475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x562138087e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf0cd7640 1 -- 172.21.15.7:0/3311659475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 msgr2=0x56213807d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.458+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/3311659475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213807d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.459+0000 7fadf0cd7640 1 -- 172.21.15.7:0/3311659475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621380738d0 con 0x56213807e590 2024-03-26T08:55:24.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.459+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/3311659475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x56213808ef20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fade0005ab0 tx=0x7fade0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c674d98c291ce286 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:24.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.459+0000 7fadd67fc640 1 -- 172.21.15.7:0/3311659475 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0057070 con 0x56213807e590 2024-03-26T08:55:24.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.459+0000 7fadd67fc640 1 -- 172.21.15.7:0/3311659475 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562137fe7910 con 0x56213807e590 2024-03-26T08:55:24.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.459+0000 7fadd67fc640 1 -- 172.21.15.7:0/3311659475 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0005150 con 0x56213807e590 2024-03-26T08:55:24.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.460+0000 7fadf5c04740 1 -- 172.21.15.7:0/3311659475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 msgr2=0x56213808ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.460+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3311659475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x56213808ef20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fade0005ab0 tx=0x7fade0005fd0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.461+0000 7fadf5c04740 1 -- 172.21.15.7:0/3311659475 shutdown_connections 2024-03-26T08:55:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.461+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3311659475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213807d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.461+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3311659475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807e590 0x56213808ef20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.461+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3311659475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807dba0 0x562138087e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.461+0000 7fadf5c04740 1 -- 172.21.15.7:0/3311659475 >> 172.21.15.7:0/3311659475 conn(0x562138070290 msgr2=0x56213807c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.461+0000 7fadf5c04740 1 -- 172.21.15.7:0/3311659475 shutdown_connections 2024-03-26T08:55:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.461+0000 7fadf5c04740 1 -- 172.21.15.7:0/3311659475 wait complete. 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.462+0000 7fadf5c04740 1 -- start start 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.462+0000 7fadf5c04740 1 Processor -- start 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.462+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807dba0 0x56213808ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.462+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 0x56213808cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.463+0000 7fadf5c04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213808df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.463+0000 7fadf5c04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1a4dd0 con 0x562138083190 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.463+0000 7fadf0cd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 0x56213808cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.463+0000 7fadf5c04740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1a60f0 con 0x56213807e590 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.463+0000 7fadf5c04740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1a5430 con 0x56213807dba0 2024-03-26T08:55:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.463+0000 7fadf5c04740 10 client.0 _ll_register_callbacks cb 0x7fade4009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.467+0000 7fadf14d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807dba0 0x56213808ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.467+0000 7fadf14d8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807dba0 0x56213808ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44378/0 (socket says 172.21.15.7:44378) 2024-03-26T08:55:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.467+0000 7fadf14d8640 1 -- 172.21.15.7:0/3766849795 learned_addr learned my addr 172.21.15.7:0/3766849795 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.467+0000 7fadf14d8640 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213808df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.472+0000 7fadf14d8640 1 -- 172.21.15.7:0/3766849795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807dba0 msgr2=0x56213808ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.472+0000 7fadf14d8640 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807dba0 0x56213808ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.472+0000 7fadf14d8640 1 -- 172.21.15.7:0/3766849795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 msgr2=0x56213808cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.472+0000 7fadf14d8640 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 0x56213808cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.472+0000 7fadf14d8640 1 -- 172.21.15.7:0/3766849795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56213808e550 con 0x562138083190 2024-03-26T08:55:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.476+0000 7fadf14d8640 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213808df60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fade400be50 tx=0x7fade4007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f99954df9ccd715 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.477+0000 7fadcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.482+0000 7fadcbfff640 1 -- 172.21.15.7:0/3766849795 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4036060 con 0x562138083190 2024-03-26T08:55:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.482+0000 7fadcbfff640 1 -- 172.21.15.7:0/3766849795 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fade4015760 con 0x562138083190 2024-03-26T08:55:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.482+0000 7fadf5c04740 1 -- 172.21.15.7:0/3766849795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562137fe7910 con 0x562138083190 2024-03-26T08:55:24.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.486+0000 7fadcbfff640 1 -- 172.21.15.7:0/3766849795 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fade400b770 con 0x562138083190 2024-03-26T08:55:24.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.486+0000 7fadf5c04740 10 client.18801 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:24.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.487+0000 7fadf5c04740 1 -- 172.21.15.7:0/3766849795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5621380ef1b0 con 0x562138083190 2024-03-26T08:55:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.493+0000 7fadcbfff640 1 -- 172.21.15.7:0/3766849795 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562137fe7910 con 0x562138083190 2024-03-26T08:55:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.493+0000 7fadcbfff640 1 -- 172.21.15.7:0/3766849795 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fade403b680 con 0x562138083190 2024-03-26T08:55:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.494+0000 7fadf5c04740 10 client.18801 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:24.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.494+0000 7fadf5c04740 10 client.18801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:24.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.494+0000 7fadf5c04740 20 client.18801 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:24.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.494+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5621380efac0 0x5621380f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.494+0000 7fadf5c04740 4 client.18801 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:55:24.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.494+0000 7fadf5c04740 1 -- 172.21.15.7:0/3766849795 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562137fe1c60 con 0x5621380efac0 2024-03-26T08:55:24.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.494+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5621380efac0 0x5621380f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.495+0000 7fadf0cd7640 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5621380efac0 0x5621380f1ee0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d5421fc47d406a server_cookie=3391d8ba0a108bf6 in_seq=0 out_seq=0 2024-03-26T08:55:24.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.495+0000 7fadcbfff640 10 client.18801 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:24.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.500+0000 7fadcbfff640 1 -- 172.21.15.7:0/3766849795 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562137fe1c60 con 0x5621380efac0 2024-03-26T08:55:24.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.500+0000 7fadcbfff640 10 client.18801 handle_command_reply: tid=0 2024-03-26T08:55:24.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.502+0000 7fadf5c04740 1 client.18801 shutdown 2024-03-26T08:55:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.513+0000 7fadf5c04740 1 -- 172.21.15.7:0/3766849795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 msgr2=0x56213808df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.513+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213808df60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fade400be50 tx=0x7fade4007040 comp rx=0 tx=0).stop 2024-03-26T08:55:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.519+0000 7fadf5c04740 1 -- 172.21.15.7:0/3766849795 shutdown_connections 2024-03-26T08:55:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.519+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5621380efac0 0x5621380f1ee0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.519+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562138083190 0x56213808df60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.519+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56213807e590 0x56213808cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.519+0000 7fadf5c04740 1 --2- 172.21.15.7:0/3766849795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56213807dba0 0x56213808ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.519+0000 7fadf5c04740 1 -- 172.21.15.7:0/3766849795 >> 172.21.15.7:0/3766849795 conn(0x562138070290 msgr2=0x562138087ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.519+0000 7fadcbfff640 0 client.18801 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.523+0000 7fadf5c04740 1 -- 172.21.15.7:0/3766849795 shutdown_connections 2024-03-26T08:55:24.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.527+0000 7fadf5c04740 1 -- 172.21.15.7:0/3766849795 wait complete. 2024-03-26T08:55:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.530+0000 7fadf5c04740 20 client.18801 trim_cache size 0 max 16384 2024-03-26T08:55:24.530 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:55:24.530 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:55:24.530 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:55:24.530 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:55:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.536+0000 7fadf2f3b640 1 -- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faddc0a26e0 msgr2=0x7faddc0a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.536+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faddc0a26e0 0x7faddc0a4ba0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009610 tx=0x7fadd80d1e10 comp rx=0 tx=0).stop 2024-03-26T08:55:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.536+0000 7fadf2f3b640 1 -- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1028c0 msgr2=0x7fadec198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.536+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1028c0 0x7fadec198020 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fadd800d950 tx=0x7fadd800de10 comp rx=0 tx=0).stop 2024-03-26T08:55:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.537+0000 7fadf2f3b640 1 -- 172.21.15.7:0/4195693706 shutdown_connections 2024-03-26T08:55:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.537+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faddc0a26e0 0x7faddc0a4ba0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.537+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadec103280 0x7fadec19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.537+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec1028c0 0x7fadec198020 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.537+0000 7fadf2f3b640 1 --2- 172.21.15.7:0/4195693706 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fadec1016d0 0x7fadec197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.537+0000 7fadf2f3b640 1 -- 172.21.15.7:0/4195693706 >> 172.21.15.7:0/4195693706 conn(0x7fadec0fcc60 msgr2=0x7fadec105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.537+0000 7fadf2f3b640 1 -- 172.21.15.7:0/4195693706 shutdown_connections 2024-03-26T08:55:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.537+0000 7fadf2f3b640 1 -- 172.21.15.7:0/4195693706 wait complete. 2024-03-26T08:55:24.551 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:55:24.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:55:24.578 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:24 smithi007 ceph-mon[32462]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 767 op/s 2024-03-26T08:55:24.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.712+0000 7f46926e9640 1 -- 172.21.15.7:0/475283831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c10d9c0 msgr2=0x7f468c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.712+0000 7f46926e9640 1 --2- 172.21.15.7:0/475283831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c10d9c0 0x7f468c10de40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4674009900 tx=0x7f467402bc40 comp rx=0 tx=0).stop 2024-03-26T08:55:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.716+0000 7f46926e9640 1 -- 172.21.15.7:0/475283831 shutdown_connections 2024-03-26T08:55:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.716+0000 7f46926e9640 1 --2- 172.21.15.7:0/475283831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c10d9c0 0x7f468c10de40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.716+0000 7f46926e9640 1 --2- 172.21.15.7:0/475283831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c10b010 0x7f468c10d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.716+0000 7f46926e9640 1 --2- 172.21.15.7:0/475283831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c1086a0 0x7f468c10aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.716+0000 7f46926e9640 1 -- 172.21.15.7:0/475283831 >> 172.21.15.7:0/475283831 conn(0x7f468c06e680 msgr2=0x7f468c070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.717+0000 7f46926e9640 1 -- 172.21.15.7:0/475283831 shutdown_connections 2024-03-26T08:55:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.717+0000 7f46926e9640 1 -- 172.21.15.7:0/475283831 wait complete. 2024-03-26T08:55:24.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.718+0000 7f46926e9640 1 Processor -- start 2024-03-26T08:55:24.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.718+0000 7f46926e9640 1 -- start start 2024-03-26T08:55:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.718+0000 7f46926e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c1086a0 0x7f468c116db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.718+0000 7f46926e9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c10b010 0x7f468c1172f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.718+0000 7f46926e9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c10d9c0 0x7f468c120bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.718+0000 7f46926e9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c11c200 con 0x7f468c10b010 2024-03-26T08:55:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.718+0000 7f46926e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c11c080 con 0x7f468c1086a0 2024-03-26T08:55:24.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.718+0000 7f46926e9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c11c380 con 0x7f468c10d9c0 2024-03-26T08:55:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.719+0000 7f468bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c1086a0 0x7f468c116db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.719+0000 7f468bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c1086a0 0x7f468c116db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52692/0 (socket says 172.21.15.7:52692) 2024-03-26T08:55:24.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.719+0000 7f468bfff640 1 -- 172.21.15.7:0/1456788945 learned_addr learned my addr 172.21.15.7:0/1456788945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:24.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.720+0000 7f4690c86640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c10d9c0 0x7f468c120bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.722+0000 7f468b7fe640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c10b010 0x7f468c1172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.722+0000 7f468bfff640 1 -- 172.21.15.7:0/1456788945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c10d9c0 msgr2=0x7f468c120bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.722+0000 7f468bfff640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c10d9c0 0x7f468c120bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.722+0000 7f468bfff640 1 -- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c10b010 msgr2=0x7f468c1172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:24.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.722+0000 7f468bfff640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c10b010 0x7f468c1172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:24.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.722+0000 7f468bfff640 1 -- 172.21.15.7:0/1456788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c121130 con 0x7f468c1086a0 2024-03-26T08:55:24.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.722+0000 7f468b7fe640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c10b010 0x7f468c1172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:55:24.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.723+0000 7f468bfff640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c1086a0 0x7f468c116db0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f468400ba50 tx=0x7f468400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:24.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.723+0000 7f46897fa640 1 -- 172.21.15.7:0/1456788945 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684019070 con 0x7f468c1086a0 2024-03-26T08:55:24.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.723+0000 7f46897fa640 1 -- 172.21.15.7:0/1456788945 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f468400cbe0 con 0x7f468c1086a0 2024-03-26T08:55:24.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.723+0000 7f46897fa640 1 -- 172.21.15.7:0/1456788945 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840076e0 con 0x7f468c1086a0 2024-03-26T08:55:24.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.724+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c121420 con 0x7f468c1086a0 2024-03-26T08:55:24.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.724+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c121920 con 0x7f468c1086a0 2024-03-26T08:55:24.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.726+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c0052c0 con 0x7f468c1086a0 2024-03-26T08:55:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.727+0000 7f46897fa640 1 -- 172.21.15.7:0/1456788945 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f468401c070 con 0x7f468c1086a0 2024-03-26T08:55:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.728+0000 7f46897fa640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46780a28d0 0x7f46780a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.728+0000 7f46897fa640 1 -- 172.21.15.7:0/1456788945 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f46840cdf40 con 0x7f468c1086a0 2024-03-26T08:55:24.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.729+0000 7f468b7fe640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46780a28d0 0x7f46780a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:24.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.729+0000 7f468b7fe640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46780a28d0 0x7f46780a4d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f467c006fd0 tx=0x7f467c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:24.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:24.735+0000 7f46897fa640 1 -- 172.21.15.7:0/1456788945 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f468409a950 con 0x7f468c1086a0 2024-03-26T08:55:24.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:24 smithi016 ceph-mon[35851]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 767 op/s 2024-03-26T08:55:24.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:24 smithi044 ceph-mon[36461]: pgmap v749: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 MiB/s wr, 767 op/s 2024-03-26T08:55:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.078+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f466c0055b0 con 0x7f468c1086a0 2024-03-26T08:55:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.078+0000 7f46897fa640 1 -- 172.21.15.7:0/1456788945 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f468409e9f0 con 0x7f468c1086a0 2024-03-26T08:55:25.076 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:55:25.076 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:55:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:55:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46780a28d0 msgr2=0x7f46780a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46780a28d0 0x7f46780a4d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f467c006fd0 tx=0x7f467c008040 comp rx=0 tx=0).stop 2024-03-26T08:55:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c1086a0 msgr2=0x7f468c116db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c1086a0 0x7f468c116db0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f468400ba50 tx=0x7f468400bf10 comp rx=0 tx=0).stop 2024-03-26T08:55:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 shutdown_connections 2024-03-26T08:55:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f46780a28d0 0x7f46780a4d90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f468c10d9c0 0x7f468c120bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f468c10b010 0x7f468c1172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 --2- 172.21.15.7:0/1456788945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c1086a0 0x7f468c116db0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 >> 172.21.15.7:0/1456788945 conn(0x7f468c06e680 msgr2=0x7f468c108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 shutdown_connections 2024-03-26T08:55:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.081+0000 7f46926e9640 1 -- 172.21.15.7:0/1456788945 wait complete. 2024-03-26T08:55:25.093 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:55:25.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:55:25.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.275+0000 7f308aaf8640 1 -- 172.21.15.7:0/2337961831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 msgr2=0x7f308410bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.275+0000 7f308aaf8640 1 --2- 172.21.15.7:0/2337961831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 0x7f308410bba0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f3078009900 tx=0x7f307802bc30 comp rx=0 tx=0).stop 2024-03-26T08:55:25.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.275+0000 7f308aaf8640 1 -- 172.21.15.7:0/2337961831 shutdown_connections 2024-03-26T08:55:25.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.275+0000 7f308aaf8640 1 --2- 172.21.15.7:0/2337961831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308410eaa0 0x7f3084110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.275+0000 7f308aaf8640 1 --2- 172.21.15.7:0/2337961831 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308410c0e0 0x7f308410e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.275+0000 7f308aaf8640 1 --2- 172.21.15.7:0/2337961831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 0x7f308410bba0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.275+0000 7f308aaf8640 1 -- 172.21.15.7:0/2337961831 >> 172.21.15.7:0/2337961831 conn(0x7f308406e640 msgr2=0x7f3084070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.275+0000 7f308aaf8640 1 -- 172.21.15.7:0/2337961831 shutdown_connections 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.275+0000 7f308aaf8640 1 -- 172.21.15.7:0/2337961831 wait complete. 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f308aaf8640 1 Processor -- start 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f308aaf8640 1 -- start start 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f308aaf8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 0x7f3084110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f308aaf8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308410c0e0 0x7f308410f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f308aaf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308410eaa0 0x7f308410f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f308aaf8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084114a50 con 0x7f30841097b0 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f308aaf8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841148d0 con 0x7f308410eaa0 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f308aaf8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084114bd0 con 0x7f308410c0e0 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f3088894640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 0x7f3084110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f3088894640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 0x7f3084110a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53664/0 (socket says 172.21.15.7:53664) 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f3088894640 1 -- 172.21.15.7:0/602983947 learned_addr learned my addr 172.21.15.7:0/602983947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.276+0000 7f3083fff640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308410c0e0 0x7f308410f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.277+0000 7f3089095640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308410eaa0 0x7f308410f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.277+0000 7f3088894640 1 -- 172.21.15.7:0/602983947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308410c0e0 msgr2=0x7f308410f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.277+0000 7f3088894640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308410c0e0 0x7f308410f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.277+0000 7f3088894640 1 -- 172.21.15.7:0/602983947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308410eaa0 msgr2=0x7f308410f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.277+0000 7f3088894640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308410eaa0 0x7f308410f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.277+0000 7f3088894640 1 -- 172.21.15.7:0/602983947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308410fec0 con 0x7f30841097b0 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.277+0000 7f3088894640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 0x7f3084110a30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3078000c00 tx=0x7f307802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.278+0000 7f3081ffb640 1 -- 172.21.15.7:0/602983947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078041070 con 0x7f30841097b0 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.278+0000 7f3081ffb640 1 -- 172.21.15.7:0/602983947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3078004450 con 0x7f30841097b0 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.278+0000 7f308aaf8640 1 -- 172.21.15.7:0/602983947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30841aa340 con 0x7f30841097b0 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.278+0000 7f308aaf8640 1 -- 172.21.15.7:0/602983947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841aa800 con 0x7f30841097b0 2024-03-26T08:55:25.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.279+0000 7f3081ffb640 1 -- 172.21.15.7:0/602983947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078041070 con 0x7f30841097b0 2024-03-26T08:55:25.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.281+0000 7f3081ffb640 1 -- 172.21.15.7:0/602983947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3078044070 con 0x7f30841097b0 2024-03-26T08:55:25.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.282+0000 7f3081ffb640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f30740a2a00 0x7f30740a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.282+0000 7f3081ffb640 1 -- 172.21.15.7:0/602983947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f30780f0180 con 0x7f30841097b0 2024-03-26T08:55:25.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.282+0000 7f3083fff640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f30740a2a00 0x7f30740a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.282+0000 7f308d7be740 1 Processor -- start 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.282+0000 7f308d7be740 1 -- start start 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3083fff640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f30740a2a00 0x7f30740a4ec0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30841107a0 tx=0x7f3070005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f308d7be740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674feeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f308d7be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ff3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f308d7be740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ff6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f308d7be740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564674fb1610 con 0x564674fee690 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f308d7be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564674f543c0 con 0x564674fef080 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f308d7be740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564674efcd60 con 0x564674ff41f0 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3089095640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ff6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3088894640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674feeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3083fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ff3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3089095640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ff6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44422/0 (socket says 172.21.15.7:44422) 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3089095640 1 -- 172.21.15.7:0/1779378634 learned_addr learned my addr 172.21.15.7:0/1779378634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3089095640 1 -- 172.21.15.7:0/1779378634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 msgr2=0x564674ff3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3089095640 1 --2- 172.21.15.7:0/1779378634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ff3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3089095640 1 -- 172.21.15.7:0/1779378634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 msgr2=0x564674feeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3089095640 1 --2- 172.21.15.7:0/1779378634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674feeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3089095640 1 -- 172.21.15.7:0/1779378634 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564674c683f0 con 0x564674ff41f0 2024-03-26T08:55:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.283+0000 7f3089095640 1 --2- 172.21.15.7:0/1779378634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ff6680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f307c009fc0 tx=0x7f307c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845bab8cab4acf6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f306effd640 1 -- 172.21.15.7:0/1779378634 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c003b70 con 0x564674ff41f0 2024-03-26T08:55:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f306effd640 1 -- 172.21.15.7:0/1779378634 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f307c003d10 con 0x564674ff41f0 2024-03-26T08:55:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f306effd640 1 -- 172.21.15.7:0/1779378634 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c011aa0 con 0x564674ff41f0 2024-03-26T08:55:25.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f308d7be740 1 -- 172.21.15.7:0/1779378634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 msgr2=0x564674ff6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f308d7be740 1 --2- 172.21.15.7:0/1779378634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ff6680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f307c009fc0 tx=0x7f307c010860 comp rx=0 tx=0).stop 2024-03-26T08:55:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f308d7be740 1 -- 172.21.15.7:0/1779378634 shutdown_connections 2024-03-26T08:55:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f308d7be740 1 --2- 172.21.15.7:0/1779378634 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ff6680 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f308d7be740 1 --2- 172.21.15.7:0/1779378634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ff3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f308d7be740 1 --2- 172.21.15.7:0/1779378634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674feeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.284+0000 7f308d7be740 1 -- 172.21.15.7:0/1779378634 >> 172.21.15.7:0/1779378634 conn(0x564674fe9c00 msgr2=0x564674fea000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 -- 172.21.15.7:0/1779378634 shutdown_connections 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 -- 172.21.15.7:0/1779378634 wait complete. 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 -- start start 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 Processor -- start 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ff8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ffffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564674f10240 con 0x564674fee690 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564674f50460 con 0x564674fef080 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564674ff71a0 con 0x564674ff41f0 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f308d7be740 10 client.0 _ll_register_callbacks cb 0x564674f0fc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f3088894640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f3083fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ff8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.285+0000 7f3089095640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ffffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.286+0000 7f3089095640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ffffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44430/0 (socket says 172.21.15.7:44430) 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.286+0000 7f3089095640 1 -- 172.21.15.7:0/3734193963 learned_addr learned my addr 172.21.15.7:0/3734193963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.286+0000 7f3083fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ff8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52748/0 (socket says 172.21.15.7:52748) 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.286+0000 7f3089095640 1 -- 172.21.15.7:0/3734193963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 msgr2=0x564674ff8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.286+0000 7f3089095640 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ff8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.286+0000 7f3089095640 1 -- 172.21.15.7:0/3734193963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 msgr2=0x564674ff8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.286+0000 7f3089095640 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.286+0000 7f3089095640 1 -- 172.21.15.7:0/3734193963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564674d327d0 con 0x564674ff41f0 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.286+0000 7f3089095640 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ffffb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f307c004ca0 tx=0x7f307c010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a11cc89a4d22779 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.287+0000 7f3063fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.287+0000 7f3063fff640 1 -- 172.21.15.7:0/3734193963 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307c0119f0 con 0x564674ff41f0 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.287+0000 7f3063fff640 1 -- 172.21.15.7:0/3734193963 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f307c011b90 con 0x564674ff41f0 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.287+0000 7f308d7be740 1 -- 172.21.15.7:0/3734193963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f307c003d10 con 0x564674ff41f0 2024-03-26T08:55:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f3063fff640 1 -- 172.21.15.7:0/3734193963 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f307c011d90 con 0x564674ff41f0 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f308d7be740 10 client.28603 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f308d7be740 1 -- 172.21.15.7:0/3734193963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564675003320 con 0x564674ff41f0 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f3063fff640 1 -- 172.21.15.7:0/3734193963 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f307c003d10 con 0x564674ff41f0 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f3063fff640 1 -- 172.21.15.7:0/3734193963 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f307c01b5c0 con 0x564674ff41f0 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f308d7be740 10 client.28603 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f308d7be740 10 client.28603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f308d7be740 20 client.28603 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f308d7be740 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5646750046c0 0x564675006ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f308d7be740 4 client.28603 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.288+0000 7f308d7be740 1 -- 172.21.15.7:0/3734193963 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564674f61900 con 0x5646750046c0 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.289+0000 7f3088894640 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5646750046c0 0x564675006ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.289+0000 7f3063fff640 10 client.28603 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:55:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.289+0000 7f3088894640 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5646750046c0 0x564675006ab0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15dfdef161cc5e66 server_cookie=da690178de1ec880 in_seq=0 out_seq=0 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f3063fff640 1 -- 172.21.15.7:0/3734193963 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564674f61900 con 0x5646750046c0 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f3063fff640 10 client.28603 handle_command_reply: tid=0 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f308d7be740 1 client.28603 shutdown 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f308d7be740 1 -- 172.21.15.7:0/3734193963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 msgr2=0x564674ffffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f308d7be740 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ffffb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f307c004ca0 tx=0x7f307c010fd0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f308d7be740 1 -- 172.21.15.7:0/3734193963 shutdown_connections 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f308d7be740 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5646750046c0 0x564675006ab0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f308d7be740 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674ff41f0 0x564674ffffb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f3063fff640 0 client.28603 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f308d7be740 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ff8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.290+0000 7f308d7be740 1 --2- 172.21.15.7:0/3734193963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.291+0000 7f308d7be740 1 -- 172.21.15.7:0/3734193963 >> 172.21.15.7:0/3734193963 conn(0x564674fe9c00 msgr2=0x564674fe8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.291+0000 7f308d7be740 1 -- 172.21.15.7:0/3734193963 shutdown_connections 2024-03-26T08:55:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.291+0000 7f308d7be740 1 -- 172.21.15.7:0/3734193963 wait complete. 2024-03-26T08:55:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.291+0000 7f308d7be740 20 client.28603 trim_cache size 0 max 16384 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.308+0000 7f308d7be740 1 Processor -- start 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.308+0000 7f308d7be740 1 -- start start 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.308+0000 7f308d7be740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.308+0000 7f308d7be740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fef080 0x564674ff0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.308+0000 7f308d7be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674ff41f0 0x564674ffff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.308+0000 7f308d7be740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841b5ba0 con 0x564674fee690 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.308+0000 7f308d7be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841b36a0 con 0x564674ff41f0 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.308+0000 7f308d7be740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841b1720 con 0x564674fef080 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3088894640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3083fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fef080 0x564674ff0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3088894640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53682/0 (socket says 172.21.15.7:53682) 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3083fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fef080 0x564674ff0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44438/0 (socket says 172.21.15.7:44438) 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3088894640 1 -- 172.21.15.7:0/3138929241 learned_addr learned my addr 172.21.15.7:0/3138929241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3089095640 1 --2- 172.21.15.7:0/3138929241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674ff41f0 0x564674ffff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3088894640 1 -- 172.21.15.7:0/3138929241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fef080 msgr2=0x564674ff0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3088894640 1 --2- 172.21.15.7:0/3138929241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fef080 0x564674ff0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3088894640 1 -- 172.21.15.7:0/3138929241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674ff41f0 msgr2=0x564674ffff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3088894640 1 --2- 172.21.15.7:0/3138929241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674ff41f0 0x564674ffff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3088894640 1 -- 172.21.15.7:0/3138929241 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564674ff3860 con 0x564674fee690 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3089095640 1 --2- 172.21.15.7:0/3138929241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674ff41f0 0x564674ffff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f3088894640 1 --2- 172.21.15.7:0/3138929241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff0610 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f30780b9580 tx=0x7f3078005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13665b18fc5fcc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f306e7fc640 1 -- 172.21.15.7:0/3138929241 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307804b040 con 0x564674fee690 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f306e7fc640 1 -- 172.21.15.7:0/3138929241 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30780c0d00 con 0x564674fee690 2024-03-26T08:55:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.309+0000 7f306e7fc640 1 -- 172.21.15.7:0/3138929241 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30780035f0 con 0x564674fee690 2024-03-26T08:55:25.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.310+0000 7f308d7be740 1 -- 172.21.15.7:0/3138929241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 msgr2=0x564674ff0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.310+0000 7f308d7be740 1 --2- 172.21.15.7:0/3138929241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff0610 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f30780b9580 tx=0x7f3078005e50 comp rx=0 tx=0).stop 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.310+0000 7f308d7be740 1 -- 172.21.15.7:0/3138929241 shutdown_connections 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.310+0000 7f308d7be740 1 --2- 172.21.15.7:0/3138929241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674ff41f0 0x564674ffff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.310+0000 7f308d7be740 1 --2- 172.21.15.7:0/3138929241 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fef080 0x564674ff0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.310+0000 7f308d7be740 1 --2- 172.21.15.7:0/3138929241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674fee690 0x564674ff0610 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.310+0000 7f308d7be740 1 -- 172.21.15.7:0/3138929241 >> 172.21.15.7:0/3138929241 conn(0x564674fe9c00 msgr2=0x564674fefb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.310+0000 7f308d7be740 1 -- 172.21.15.7:0/3138929241 shutdown_connections 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.310+0000 7f308d7be740 1 -- 172.21.15.7:0/3138929241 wait complete. 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.311+0000 7f308d7be740 1 -- start start 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.311+0000 7f308d7be740 1 Processor -- start 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.311+0000 7f308d7be740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fee690 0x564674ffda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.311+0000 7f308d7be740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ffdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.311+0000 7f308d7be740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674ff41f0 0x564674ffef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.311+0000 7f308d7be740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841b1d20 con 0x564674ff41f0 2024-03-26T08:55:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.311+0000 7f308d7be740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841b3040 con 0x564674fef080 2024-03-26T08:55:25.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.311+0000 7f308d7be740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841b2380 con 0x564674fee690 2024-03-26T08:55:25.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.311+0000 7f308d7be740 10 client.0 _ll_register_callbacks cb 0x7f307c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:25.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.319+0000 7f3088894640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fee690 0x564674ffda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.319+0000 7f3088894640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fee690 0x564674ffda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44448/0 (socket says 172.21.15.7:44448) 2024-03-26T08:55:25.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.319+0000 7f3088894640 1 -- 172.21.15.7:0/486190602 learned_addr learned my addr 172.21.15.7:0/486190602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.319+0000 7f3083fff640 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ffdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.322+0000 7f3088894640 1 -- 172.21.15.7:0/486190602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 msgr2=0x564674ffdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.322+0000 7f3088894640 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ffdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.322+0000 7f3088894640 1 -- 172.21.15.7:0/486190602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674ff41f0 msgr2=0x564674ffef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.322+0000 7f3088894640 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674ff41f0 0x564674ffef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.322+0000 7f3088894640 1 -- 172.21.15.7:0/486190602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564674fff560 con 0x564674fee690 2024-03-26T08:55:25.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.323+0000 7f3088894640 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fee690 0x564674ffda20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f30780c4d90 tx=0x7f30780bcd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82a68774fbad7ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.323+0000 7f3063fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:55:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.326+0000 7f3063fff640 1 -- 172.21.15.7:0/486190602 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307804b040 con 0x564674fee690 2024-03-26T08:55:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.326+0000 7f3063fff640 1 -- 172.21.15.7:0/486190602 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f30780d5be0 con 0x564674fee690 2024-03-26T08:55:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.330+0000 7f308d7be740 1 -- 172.21.15.7:0/486190602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30780c0d00 con 0x564674fee690 2024-03-26T08:55:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.334+0000 7f3063fff640 1 -- 172.21.15.7:0/486190602 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3078003c60 con 0x564674fee690 2024-03-26T08:55:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.334+0000 7f308d7be740 10 client.28609 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.334+0000 7f308d7be740 1 -- 172.21.15.7:0/486190602 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56467505a270 con 0x564674fee690 2024-03-26T08:55:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.335+0000 7f3063fff640 1 -- 172.21.15.7:0/486190602 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30780bac50 con 0x564674fee690 2024-03-26T08:55:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.335+0000 7f3063fff640 1 -- 172.21.15.7:0/486190602 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f30780bc000 con 0x564674fee690 2024-03-26T08:55:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.337+0000 7f308d7be740 10 client.28609 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.337+0000 7f308d7be740 10 client.28609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:55:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.337+0000 7f308d7be740 20 client.28609 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.337+0000 7f308d7be740 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5646750046c0 0x56467505c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.337+0000 7f308d7be740 4 client.28609 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:55:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.337+0000 7f308d7be740 1 -- 172.21.15.7:0/486190602 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f30780c0d00 con 0x5646750046c0 2024-03-26T08:55:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.338+0000 7f3083fff640 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5646750046c0 0x56467505c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f3083fff640 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5646750046c0 0x56467505c920 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ba8df0fa8f771b7 server_cookie=a61d15f866a8f944 in_seq=0 out_seq=0 2024-03-26T08:55:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f3063fff640 10 client.28609 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:55:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f3063fff640 1 -- 172.21.15.7:0/486190602 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f30780c0d00 con 0x5646750046c0 2024-03-26T08:55:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f3063fff640 10 client.28609 handle_command_reply: tid=0 2024-03-26T08:55:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f308d7be740 1 client.28609 shutdown 2024-03-26T08:55:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f308d7be740 1 -- 172.21.15.7:0/486190602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fee690 msgr2=0x564674ffda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f308d7be740 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fee690 0x564674ffda20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f30780c4d90 tx=0x7f30780bcd30 comp rx=0 tx=0).stop 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f308d7be740 1 -- 172.21.15.7:0/486190602 shutdown_connections 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f308d7be740 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5646750046c0 0x56467505c920 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f308d7be740 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564674ff41f0 0x564674ffef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f308d7be740 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564674fef080 0x564674ffdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f308d7be740 1 --2- 172.21.15.7:0/486190602 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564674fee690 0x564674ffda20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f308d7be740 1 -- 172.21.15.7:0/486190602 >> 172.21.15.7:0/486190602 conn(0x564674fe9c00 msgr2=0x564674ff0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.341+0000 7f3063fff640 0 client.28609 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.342+0000 7f308d7be740 1 -- 172.21.15.7:0/486190602 shutdown_connections 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.342+0000 7f308d7be740 1 -- 172.21.15.7:0/486190602 wait complete. 2024-03-26T08:55:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.342+0000 7f308d7be740 20 client.28609 trim_cache size 0 max 16384 2024-03-26T08:55:25.341 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:55:25.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.342+0000 7f308aaf8640 1 -- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f30740a2a00 msgr2=0x7f30740a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.342+0000 7f308aaf8640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f30740a2a00 0x7f30740a4ec0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30841107a0 tx=0x7f3070005f90 comp rx=0 tx=0).stop 2024-03-26T08:55:25.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.342+0000 7f308aaf8640 1 -- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 msgr2=0x7f3084110a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.342+0000 7f308aaf8640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 0x7f3084110a30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3078000c00 tx=0x7f307802c640 comp rx=0 tx=0).stop 2024-03-26T08:55:25.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.343+0000 7f308aaf8640 1 -- 172.21.15.7:0/602983947 shutdown_connections 2024-03-26T08:55:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.343+0000 7f308aaf8640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f30740a2a00 0x7f30740a4ec0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.343+0000 7f308aaf8640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308410eaa0 0x7f308410f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.343+0000 7f308aaf8640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f308410c0e0 0x7f308410f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.343+0000 7f308aaf8640 1 --2- 172.21.15.7:0/602983947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30841097b0 0x7f3084110a30 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.343+0000 7f308aaf8640 1 -- 172.21.15.7:0/602983947 >> 172.21.15.7:0/602983947 conn(0x7f308406e640 msgr2=0x7f3084070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.343+0000 7f308aaf8640 1 -- 172.21.15.7:0/602983947 shutdown_connections 2024-03-26T08:55:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.343+0000 7f308aaf8640 1 -- 172.21.15.7:0/602983947 wait complete. 2024-03-26T08:55:25.353 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:55:25.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:55:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 -- 172.21.15.7:0/3595937711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2384102140 msgr2=0x7f2384104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 --2- 172.21.15.7:0/3595937711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2384102140 0x7f2384104530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2380009860 tx=0x7f238002bb70 comp rx=0 tx=0).stop 2024-03-26T08:55:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 -- 172.21.15.7:0/3595937711 shutdown_connections 2024-03-26T08:55:25.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 --2- 172.21.15.7:0/3595937711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23841073a0 0x7f23841097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 --2- 172.21.15.7:0/3595937711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2384104a70 0x7f2384106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 --2- 172.21.15.7:0/3595937711 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2384102140 0x7f2384104530 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 -- 172.21.15.7:0/3595937711 >> 172.21.15.7:0/3595937711 conn(0x7f23840fbb30 msgr2=0x7f23840fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 -- 172.21.15.7:0/3595937711 shutdown_connections 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 -- 172.21.15.7:0/3595937711 wait complete. 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 Processor -- start 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 -- start start 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2384102140 0x7f2384199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.495+0000 7f238c190640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2384104a70 0x7f238419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f238c190640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23841073a0 0x7f23841a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f238c190640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238410f560 con 0x7f2384102140 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f238c190640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238410f3e0 con 0x7f2384104a70 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f238c190640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238410f6e0 con 0x7f23841073a0 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f2389f2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2384102140 0x7f2384199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f238a72d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23841073a0 0x7f23841a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f2389f2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2384102140 0x7f2384199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53714/0 (socket says 172.21.15.7:53714) 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f2389f2c640 1 -- 172.21.15.7:0/814638142 learned_addr learned my addr 172.21.15.7:0/814638142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f238972b640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2384104a70 0x7f238419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f2389f2c640 1 -- 172.21.15.7:0/814638142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23841073a0 msgr2=0x7f23841a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f2389f2c640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23841073a0 0x7f23841a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f2389f2c640 1 -- 172.21.15.7:0/814638142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2384104a70 msgr2=0x7f238419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f2389f2c640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2384104a70 0x7f238419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f2389f2c640 1 -- 172.21.15.7:0/814638142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23841a1800 con 0x7f2384102140 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f238a72d640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23841073a0 0x7f23841a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f2389f2c640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2384102140 0x7f2384199be0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f23800349e0 tx=0x7f2380032450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f237affd640 1 -- 172.21.15.7:0/814638142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380041030 con 0x7f2384102140 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f238c190640 1 -- 172.21.15.7:0/814638142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23841a1a90 con 0x7f2384102140 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f237affd640 1 -- 172.21.15.7:0/814638142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f238002c510 con 0x7f2384102140 2024-03-26T08:55:25.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.496+0000 7f237affd640 1 -- 172.21.15.7:0/814638142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380039040 con 0x7f2384102140 2024-03-26T08:55:25.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.497+0000 7f238c190640 1 -- 172.21.15.7:0/814638142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23841a1f80 con 0x7f2384102140 2024-03-26T08:55:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f237affd640 1 -- 172.21.15.7:0/814638142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2380044070 con 0x7f2384102140 2024-03-26T08:55:25.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238ed7f740 1 Processor -- start 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238ed7f740 1 -- start start 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9260690 0x55b3c9260ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c9265b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c92661f0 0x55b3c9268680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238ed7f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c9223610 con 0x55b3c9261080 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238ed7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c91c63c0 con 0x55b3c9260690 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238ed7f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c916ed60 con 0x55b3c92661f0 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238a72d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9260690 0x55b3c9260ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f2389f2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c9265b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238972b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c92661f0 0x55b3c9268680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238a72d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9260690 0x55b3c9260ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52802/0 (socket says 172.21.15.7:52802) 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238972b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c92661f0 0x55b3c9268680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44482/0 (socket says 172.21.15.7:44482) 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f238a72d640 1 -- 172.21.15.7:0/3272391482 learned_addr learned my addr 172.21.15.7:0/3272391482 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.500+0000 7f2389f2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c9265b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53726/0 (socket says 172.21.15.7:53726) 2024-03-26T08:55:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.501+0000 7f237affd640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23700a2800 0x7f23700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.501+0000 7f238972b640 1 -- 172.21.15.7:0/3272391482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9260690 msgr2=0x55b3c9260ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.501+0000 7f238972b640 1 --2- 172.21.15.7:0/3272391482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9260690 0x55b3c9260ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.501+0000 7f238972b640 1 -- 172.21.15.7:0/3272391482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 msgr2=0x55b3c9265b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.501+0000 7f238972b640 1 --2- 172.21.15.7:0/3272391482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c9265b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.501+0000 7f238972b640 1 -- 172.21.15.7:0/3272391482 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c8eda3f0 con 0x55b3c92661f0 2024-03-26T08:55:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.501+0000 7f237affd640 1 -- 172.21.15.7:0/814638142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23800eff60 con 0x7f2384102140 2024-03-26T08:55:25.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.501+0000 7f238a72d640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23700a2800 0x7f23700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.502+0000 7f238972b640 1 --2- 172.21.15.7:0/3272391482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c92661f0 0x55b3c9268680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23840694d0 tx=0x7f2374011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e9b21c59b212ad1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.502+0000 7f238a72d640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23700a2800 0x7f23700a4cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55b3c92618e0 tx=0x7f237c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.502+0000 7f236ffff640 1 -- 172.21.15.7:0/3272391482 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374004050 con 0x55b3c92661f0 2024-03-26T08:55:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.502+0000 7f236ffff640 1 -- 172.21.15.7:0/3272391482 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23740041f0 con 0x55b3c92661f0 2024-03-26T08:55:25.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.502+0000 7f236ffff640 1 -- 172.21.15.7:0/3272391482 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740197b0 con 0x55b3c92661f0 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.503+0000 7f238ed7f740 1 -- 172.21.15.7:0/3272391482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c92661f0 msgr2=0x55b3c9268680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.503+0000 7f238ed7f740 1 --2- 172.21.15.7:0/3272391482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c92661f0 0x55b3c9268680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23840694d0 tx=0x7f2374011810 comp rx=0 tx=0).stop 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.503+0000 7f238ed7f740 1 -- 172.21.15.7:0/3272391482 shutdown_connections 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.503+0000 7f238ed7f740 1 --2- 172.21.15.7:0/3272391482 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c92661f0 0x55b3c9268680 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.503+0000 7f238ed7f740 1 --2- 172.21.15.7:0/3272391482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c9265b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.503+0000 7f238ed7f740 1 --2- 172.21.15.7:0/3272391482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9260690 0x55b3c9260ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.503+0000 7f238ed7f740 1 -- 172.21.15.7:0/3272391482 >> 172.21.15.7:0/3272391482 conn(0x55b3c925bc00 msgr2=0x55b3c925c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.503+0000 7f238ed7f740 1 -- 172.21.15.7:0/3272391482 shutdown_connections 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.503+0000 7f238ed7f740 1 -- 172.21.15.7:0/3272391482 wait complete. 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238ed7f740 1 -- start start 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238ed7f740 1 Processor -- start 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c926aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9261080 0x55b3c926b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c92661f0 0x55b3c9272000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238ed7f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c9182240 con 0x55b3c92661f0 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238ed7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c91c2460 con 0x55b3c9261080 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238ed7f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c92691a0 con 0x55b3c9260690 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f2389f2c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c926aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238ed7f740 10 client.0 _ll_register_callbacks cb 0x55b3c9181c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238972b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9261080 0x55b3c926b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f2389f2c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c926aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44488/0 (socket says 172.21.15.7:44488) 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f2389f2c640 1 -- 172.21.15.7:0/1226772244 learned_addr learned my addr 172.21.15.7:0/1226772244 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.504+0000 7f238a72d640 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c92661f0 0x55b3c9272000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.505+0000 7f2389f2c640 1 -- 172.21.15.7:0/1226772244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9261080 msgr2=0x55b3c926b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.505+0000 7f2389f2c640 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9261080 0x55b3c926b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.505+0000 7f2389f2c640 1 -- 172.21.15.7:0/1226772244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c92661f0 msgr2=0x55b3c9272000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.505+0000 7f2389f2c640 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c92661f0 0x55b3c9272000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.505+0000 7f2389f2c640 1 -- 172.21.15.7:0/1226772244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3c8fa47d0 con 0x55b3c9260690 2024-03-26T08:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.505+0000 7f2389f2c640 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c926aaf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f23800b92d0 tx=0x7f23800f3740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709870868daaad4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.505+0000 7f236d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.505+0000 7f236d7fa640 1 -- 172.21.15.7:0/1226772244 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800bd000 con 0x55b3c9260690 2024-03-26T08:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.505+0000 7f236d7fa640 1 -- 172.21.15.7:0/1226772244 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f238002ca30 con 0x55b3c9260690 2024-03-26T08:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f238ed7f740 1 -- 172.21.15.7:0/1226772244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23740041f0 con 0x55b3c9260690 2024-03-26T08:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f236d7fa640 1 -- 172.21.15.7:0/1226772244 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23800bc900 con 0x55b3c9260690 2024-03-26T08:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f238ed7f740 10 client.28621 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:25.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f238ed7f740 1 -- 172.21.15.7:0/1226772244 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b3c9275370 con 0x55b3c9260690 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f236d7fa640 1 -- 172.21.15.7:0/1226772244 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23740041f0 con 0x55b3c9260690 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f236d7fa640 1 -- 172.21.15.7:0/1226772244 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f23800d0ce0 con 0x55b3c9260690 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f238ed7f740 10 client.28621 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f238ed7f740 10 client.28621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f238ed7f740 20 client.28621 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.506+0000 7f238ed7f740 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55b3c92767d0 0x55b3c9278c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.507+0000 7f238ed7f740 4 client.28621 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.507+0000 7f238ed7f740 1 -- 172.21.15.7:0/1226772244 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3c91d3900 con 0x55b3c92767d0 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.507+0000 7f238972b640 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55b3c92767d0 0x55b3c9278c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.507+0000 7f236d7fa640 10 client.28621 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.507+0000 7f238972b640 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55b3c92767d0 0x55b3c9278c40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1d96bd92b0d01f0 server_cookie=cdb19d2d58c91b22 in_seq=0 out_seq=0 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f236d7fa640 1 -- 172.21.15.7:0/1226772244 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b3c91d3900 con 0x55b3c92767d0 2024-03-26T08:55:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f236d7fa640 10 client.28621 handle_command_reply: tid=0 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 client.28621 shutdown 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 -- 172.21.15.7:0/1226772244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 msgr2=0x55b3c926aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c926aaf0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f23800b92d0 tx=0x7f23800f3740 comp rx=0 tx=0).stop 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 -- 172.21.15.7:0/1226772244 shutdown_connections 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c92661f0 0x55b3c9272000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c9261080 0x55b3c926b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55b3c92767d0 0x55b3c9278c40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 --2- 172.21.15.7:0/1226772244 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c926aaf0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f236d7fa640 0 client.28621 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 -- 172.21.15.7:0/1226772244 >> 172.21.15.7:0/1226772244 conn(0x55b3c925bc00 msgr2=0x55b3c925ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 -- 172.21.15.7:0/1226772244 shutdown_connections 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 1 -- 172.21.15.7:0/1226772244 wait complete. 2024-03-26T08:55:25.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.508+0000 7f238ed7f740 20 client.28621 trim_cache size 0 max 16384 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.532+0000 7f238ed7f740 1 Processor -- start 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.534+0000 7f238ed7f740 1 -- start start 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.534+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9260690 0x55b3c926af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.534+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9261080 0x55b3c9271fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.534+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 0x55b3c925f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.534+0000 7f238ed7f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841ad360 con 0x55b3c9260690 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.534+0000 7f238ed7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841aae60 con 0x55b3c92661f0 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.534+0000 7f238ed7f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841a8ee0 con 0x55b3c9261080 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.536+0000 7f238972b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9261080 0x55b3c9271fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.537+0000 7f2389f2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9260690 0x55b3c926af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.537+0000 7f2389f2c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9260690 0x55b3c926af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53740/0 (socket says 172.21.15.7:53740) 2024-03-26T08:55:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.537+0000 7f2389f2c640 1 -- 172.21.15.7:0/4148806552 learned_addr learned my addr 172.21.15.7:0/4148806552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.543+0000 7f238a72d640 1 --2- 172.21.15.7:0/4148806552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 0x55b3c925f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.543+0000 7f2389f2c640 1 -- 172.21.15.7:0/4148806552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9261080 msgr2=0x55b3c9271fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.543+0000 7f2389f2c640 1 --2- 172.21.15.7:0/4148806552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9261080 0x55b3c9271fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.543+0000 7f2389f2c640 1 -- 172.21.15.7:0/4148806552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 msgr2=0x55b3c925f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.543+0000 7f2389f2c640 1 --2- 172.21.15.7:0/4148806552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 0x55b3c925f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.543+0000 7f2389f2c640 1 -- 172.21.15.7:0/4148806552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c9257670 con 0x55b3c9260690 2024-03-26T08:55:25.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.546+0000 7f238972b640 1 --2- 172.21.15.7:0/4148806552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9261080 0x55b3c9271fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:55:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.547+0000 7f2389f2c640 1 --2- 172.21.15.7:0/4148806552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9260690 0x55b3c926af90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f23800a7010 tx=0x7f2380005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67074a3d031a5e8f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.549+0000 7f236f7fe640 1 -- 172.21.15.7:0/4148806552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800bd000 con 0x55b3c9260690 2024-03-26T08:55:25.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.549+0000 7f236f7fe640 1 -- 172.21.15.7:0/4148806552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23800c50a0 con 0x55b3c9260690 2024-03-26T08:55:25.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.549+0000 7f236f7fe640 1 -- 172.21.15.7:0/4148806552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800f22a0 con 0x55b3c9260690 2024-03-26T08:55:25.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.551+0000 7f238ed7f740 1 -- 172.21.15.7:0/4148806552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9260690 msgr2=0x55b3c926af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.551+0000 7f238ed7f740 1 --2- 172.21.15.7:0/4148806552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9260690 0x55b3c926af90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f23800a7010 tx=0x7f2380005f90 comp rx=0 tx=0).stop 2024-03-26T08:55:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.558+0000 7f238ed7f740 1 -- 172.21.15.7:0/4148806552 shutdown_connections 2024-03-26T08:55:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.558+0000 7f238ed7f740 1 --2- 172.21.15.7:0/4148806552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 0x55b3c925f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.558+0000 7f238ed7f740 1 --2- 172.21.15.7:0/4148806552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9261080 0x55b3c9271fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.558+0000 7f238ed7f740 1 --2- 172.21.15.7:0/4148806552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9260690 0x55b3c926af90 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.559+0000 7f238ed7f740 1 -- 172.21.15.7:0/4148806552 >> 172.21.15.7:0/4148806552 conn(0x55b3c925bc00 msgr2=0x55b3c925ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.565+0000 7f238ed7f740 1 -- 172.21.15.7:0/4148806552 shutdown_connections 2024-03-26T08:55:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.569+0000 7f238ed7f740 1 -- 172.21.15.7:0/4148806552 wait complete. 2024-03-26T08:55:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.569+0000 7f238ed7f740 1 -- start start 2024-03-26T08:55:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.569+0000 7f238ed7f740 1 Processor -- start 2024-03-26T08:55:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.572+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c9271bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.572+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c92701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.572+0000 7f238ed7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 0x55b3c9270710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.572+0000 7f238ed7f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841a94e0 con 0x55b3c9261080 2024-03-26T08:55:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.572+0000 7f238ed7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841aa800 con 0x55b3c92661f0 2024-03-26T08:55:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.572+0000 7f238ed7f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23841a9b40 con 0x55b3c9260690 2024-03-26T08:55:25.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.573+0000 7f238ed7f740 10 client.0 _ll_register_callbacks cb 0x7f23800097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.575+0000 7f238972b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c92701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.575+0000 7f238972b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c92701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53754/0 (socket says 172.21.15.7:53754) 2024-03-26T08:55:25.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.575+0000 7f238972b640 1 -- 172.21.15.7:0/2442046103 learned_addr learned my addr 172.21.15.7:0/2442046103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.577+0000 7f238a72d640 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 0x55b3c9270710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.577+0000 7f2389f2c640 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c9271bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.579+0000 7f238972b640 1 -- 172.21.15.7:0/2442046103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 msgr2=0x55b3c9271bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.579+0000 7f238972b640 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c9271bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.580+0000 7f238972b640 1 -- 172.21.15.7:0/2442046103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 msgr2=0x55b3c9270710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.580+0000 7f238972b640 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 0x55b3c9270710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.580+0000 7f238972b640 1 -- 172.21.15.7:0/2442046103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3c9270c50 con 0x55b3c9261080 2024-03-26T08:55:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.581+0000 7f238972b640 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c92701d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f23840694d0 tx=0x7f2374005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbb2f988d778c26 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.581+0000 7f236d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:25.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.582+0000 7f236d7fa640 1 -- 172.21.15.7:0/2442046103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237402c2f0 con 0x55b3c9261080 2024-03-26T08:55:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.582+0000 7f236d7fa640 1 -- 172.21.15.7:0/2442046103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f237401f040 con 0x55b3c9261080 2024-03-26T08:55:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.583+0000 7f238ed7f740 1 -- 172.21.15.7:0/2442046103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23800c50a0 con 0x55b3c9261080 2024-03-26T08:55:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.583+0000 7f236d7fa640 1 -- 172.21.15.7:0/2442046103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2374005040 con 0x55b3c9261080 2024-03-26T08:55:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.583+0000 7f238ed7f740 10 client.18834 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:25.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.583+0000 7f238ed7f740 1 -- 172.21.15.7:0/2442046103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b3c92ec660 con 0x55b3c9261080 2024-03-26T08:55:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.583+0000 7f236d7fa640 1 -- 172.21.15.7:0/2442046103 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23800c50a0 con 0x55b3c9261080 2024-03-26T08:55:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.584+0000 7f236d7fa640 1 -- 172.21.15.7:0/2442046103 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f23740035f0 con 0x55b3c9261080 2024-03-26T08:55:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.584+0000 7f238ed7f740 10 client.18834 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.584+0000 7f238ed7f740 10 client.18834 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:55:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.584+0000 7f238ed7f740 20 client.18834 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.584+0000 7f238ed7f740 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55b3c92ecfa0 0x55b3c92ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.584+0000 7f238ed7f740 4 client.18834 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:55:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.584+0000 7f238ed7f740 1 -- 172.21.15.7:0/2442046103 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b3c91cdc50 con 0x55b3c92ecfa0 2024-03-26T08:55:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.584+0000 7f2389f2c640 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55b3c92ecfa0 0x55b3c92ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.585+0000 7f2389f2c640 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55b3c92ecfa0 0x55b3c92ef3c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c096a773c33fd7ec server_cookie=5bb20d649e53f2da in_seq=0 out_seq=0 2024-03-26T08:55:25.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.585+0000 7f236d7fa640 10 client.18834 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:55:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.585+0000 7f236d7fa640 1 -- 172.21.15.7:0/2442046103 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b3c91cdc50 con 0x55b3c92ecfa0 2024-03-26T08:55:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.585+0000 7f236d7fa640 10 client.18834 handle_command_reply: tid=0 2024-03-26T08:55:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f238ed7f740 1 client.18834 shutdown 2024-03-26T08:55:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f238ed7f740 1 -- 172.21.15.7:0/2442046103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 msgr2=0x55b3c92701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f238ed7f740 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c92701d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f23840694d0 tx=0x7f2374005e80 comp rx=0 tx=0).stop 2024-03-26T08:55:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f238ed7f740 1 -- 172.21.15.7:0/2442046103 shutdown_connections 2024-03-26T08:55:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f238ed7f740 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b3c92661f0 0x55b3c9270710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f238ed7f740 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b3c9261080 0x55b3c92701d0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f238ed7f740 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55b3c92ecfa0 0x55b3c92ef3c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f238ed7f740 1 --2- 172.21.15.7:0/2442046103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55b3c9260690 0x55b3c9271bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f238ed7f740 1 -- 172.21.15.7:0/2442046103 >> 172.21.15.7:0/2442046103 conn(0x55b3c925bc00 msgr2=0x55b3c926adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.586+0000 7f236d7fa640 0 client.18834 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:55:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.587+0000 7f238ed7f740 1 -- 172.21.15.7:0/2442046103 shutdown_connections 2024-03-26T08:55:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.587+0000 7f238ed7f740 1 -- 172.21.15.7:0/2442046103 wait complete. 2024-03-26T08:55:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.587+0000 7f238ed7f740 20 client.18834 trim_cache size 0 max 16384 2024-03-26T08:55:25.585 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:55:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 -- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23700a2800 msgr2=0x7f23700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23700a2800 0x7f23700a4cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55b3c92618e0 tx=0x7f237c004a40 comp rx=0 tx=0).stop 2024-03-26T08:55:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 -- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2384102140 msgr2=0x7f2384199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2384102140 0x7f2384199be0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f23800349e0 tx=0x7f2380032450 comp rx=0 tx=0).stop 2024-03-26T08:55:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 -- 172.21.15.7:0/814638142 shutdown_connections 2024-03-26T08:55:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f23700a2800 0x7f23700a4cc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23841073a0 0x7f23841a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2384104a70 0x7f238419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 --2- 172.21.15.7:0/814638142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2384102140 0x7f2384199be0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 -- 172.21.15.7:0/814638142 >> 172.21.15.7:0/814638142 conn(0x7f23840fbb30 msgr2=0x7f2384102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 -- 172.21.15.7:0/814638142 shutdown_connections 2024-03-26T08:55:25.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.588+0000 7f238c190640 1 -- 172.21.15.7:0/814638142 wait complete. 2024-03-26T08:55:25.600 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:55:25.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:55:25.625 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:25 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1456788945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:55:25.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:25 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1456788945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:55:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.764+0000 7f4dadfc4640 1 -- 172.21.15.7:0/555770446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da81086a0 msgr2=0x7f4da810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.764+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/555770446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da81086a0 0x7f4da810aad0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4da001acb0 tx=0x7f4da003d170 comp rx=0 tx=0).stop 2024-03-26T08:55:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.765+0000 7f4dadfc4640 1 -- 172.21.15.7:0/555770446 shutdown_connections 2024-03-26T08:55:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.765+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/555770446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da810d9c0 0x7f4da810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.765+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/555770446 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da810b010 0x7f4da810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.765+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/555770446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da81086a0 0x7f4da810aad0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.765+0000 7f4dadfc4640 1 -- 172.21.15.7:0/555770446 >> 172.21.15.7:0/555770446 conn(0x7f4da806e640 msgr2=0x7f4da8070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.770+0000 7f4dadfc4640 1 -- 172.21.15.7:0/555770446 shutdown_connections 2024-03-26T08:55:25.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.770+0000 7f4dadfc4640 1 -- 172.21.15.7:0/555770446 wait complete. 2024-03-26T08:55:25.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.770+0000 7f4dadfc4640 1 Processor -- start 2024-03-26T08:55:25.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4dadfc4640 1 -- start start 2024-03-26T08:55:25.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4dadfc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da81086a0 0x7f4da819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4dadfc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da810b010 0x7f4da819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4dadfc4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da810d9c0 0x7f4da81a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4dadfc4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da81119b0 con 0x7f4da810b010 2024-03-26T08:55:25.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4dadfc4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8111830 con 0x7f4da81086a0 2024-03-26T08:55:25.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4dadfc4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da8111b30 con 0x7f4da810d9c0 2024-03-26T08:55:25.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da810d9c0 0x7f4da81a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4da7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da810d9c0 0x7f4da81a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44534/0 (socket says 172.21.15.7:44534) 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.771+0000 7f4da7fff640 1 -- 172.21.15.7:0/2492588552 learned_addr learned my addr 172.21.15.7:0/2492588552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.772+0000 7f4da77fe640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da81086a0 0x7f4da819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.772+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da810b010 0x7f4da819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.772+0000 7f4da7fff640 1 -- 172.21.15.7:0/2492588552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da81086a0 msgr2=0x7f4da819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.772+0000 7f4da7fff640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da81086a0 0x7f4da819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.772+0000 7f4da7fff640 1 -- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da810b010 msgr2=0x7f4da819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.772+0000 7f4da7fff640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da810b010 0x7f4da819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.772+0000 7f4da7fff640 1 -- 172.21.15.7:0/2492588552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da81a5ca0 con 0x7f4da810d9c0 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.772+0000 7f4da7fff640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da810d9c0 0x7f4da81a5630 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800c970 tx=0x7f4d9800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.772+0000 7f4da4ff9640 1 -- 172.21.15.7:0/2492588552 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98007bb0 con 0x7f4da810d9c0 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.773+0000 7f4dadfc4640 1 -- 172.21.15.7:0/2492588552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da81a5f90 con 0x7f4da810d9c0 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.773+0000 7f4dadfc4640 1 -- 172.21.15.7:0/2492588552 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da81a64d0 con 0x7f4da810d9c0 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.773+0000 7f4da4ff9640 1 -- 172.21.15.7:0/2492588552 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d98007d50 con 0x7f4da810d9c0 2024-03-26T08:55:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.773+0000 7f4da4ff9640 1 -- 172.21.15.7:0/2492588552 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9801d060 con 0x7f4da810d9c0 2024-03-26T08:55:25.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.776+0000 7f4da4ff9640 1 -- 172.21.15.7:0/2492588552 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4d980046c0 con 0x7f4da810d9c0 2024-03-26T08:55:25.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.776+0000 7f4db0bab740 1 Processor -- start 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.776+0000 7f4db0bab740 1 -- start start 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da4ff9640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4d8c0a2a40 0x7f4d8c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de655690 0x5582de655ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 0x5582de65ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de65b1f0 0x5582de65d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4db0bab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582de618610 con 0x5582de655690 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4db0bab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582de5bb3c0 con 0x5582de656080 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4db0bab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582de563d60 con 0x5582de65b1f0 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de655690 0x5582de655ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da4ff9640 1 -- 172.21.15.7:0/2492588552 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4d98004c40 con 0x7f4da810d9c0 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de655690 0x5582de655ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53788/0 (socket says 172.21.15.7:53788) 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da77fe640 1 -- 172.21.15.7:0/2070189082 learned_addr learned my addr 172.21.15.7:0/2070189082 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4d8c0a2a40 0x7f4d8c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da7fff640 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de65b1f0 0x5582de65d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 0x5582de65ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da77fe640 1 -- 172.21.15.7:0/2070189082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de65b1f0 msgr2=0x5582de65d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da77fe640 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de65b1f0 0x5582de65d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da77fe640 1 -- 172.21.15.7:0/2070189082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 msgr2=0x5582de65ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da77fe640 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 0x5582de65ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da77fe640 1 -- 172.21.15.7:0/2070189082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582de2cf3f0 con 0x5582de655690 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.777+0000 7f4da7fff640 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de65b1f0 0x5582de65d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.778+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4d8c0a2a40 0x7f4d8c0a4f00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4d94006dc0 tx=0x7f4d94009960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.778+0000 7f4da77fe640 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de655690 0x5582de655ab0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4da001acb0 tx=0x7f4da003ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a66ea7c0bbdfb0c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.778+0000 7f4d9dffb640 1 -- 172.21.15.7:0/2070189082 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0055070 con 0x5582de655690 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.778+0000 7f4d9dffb640 1 -- 172.21.15.7:0/2070189082 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4da0043ab0 con 0x5582de655690 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.778+0000 7f4d9dffb640 1 -- 172.21.15.7:0/2070189082 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da004a040 con 0x5582de655690 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 -- 172.21.15.7:0/2070189082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de655690 msgr2=0x5582de655ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de655690 0x5582de655ab0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4da001acb0 tx=0x7f4da003ddd0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 -- 172.21.15.7:0/2070189082 shutdown_connections 2024-03-26T08:55:25.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de65b1f0 0x5582de65d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 0x5582de65ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 --2- 172.21.15.7:0/2070189082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de655690 0x5582de655ab0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 -- 172.21.15.7:0/2070189082 >> 172.21.15.7:0/2070189082 conn(0x5582de650c00 msgr2=0x5582de651000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 -- 172.21.15.7:0/2070189082 shutdown_connections 2024-03-26T08:55:25.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 -- 172.21.15.7:0/2070189082 wait complete. 2024-03-26T08:55:25.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.779+0000 7f4db0bab740 1 -- start start 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.780+0000 7f4db0bab740 1 Processor -- start 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.780+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de65f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.780+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 0x5582de65fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de666f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de65f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de65f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44554/0 (socket says 172.21.15.7:44554) 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4da6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 0x5582de65fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4da77fe640 1 -- 172.21.15.7:0/1617477906 learned_addr learned my addr 172.21.15.7:0/1617477906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582de577240 con 0x5582de65b1f0 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582de5b7460 con 0x5582de656080 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4da7fff640 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de666f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582de65e140 con 0x5582de655690 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.781+0000 7f4db0bab740 10 client.0 _ll_register_callbacks cb 0x5582de576c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4da77fe640 1 -- 172.21.15.7:0/1617477906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 msgr2=0x5582de65fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4da77fe640 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 0x5582de65fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4da77fe640 1 -- 172.21.15.7:0/1617477906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 msgr2=0x5582de666f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4da77fe640 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de666f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4da77fe640 1 -- 172.21.15.7:0/1617477906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582de3997d0 con 0x5582de655690 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4da7fff640 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de666f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4da77fe640 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de65f9b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4da0045fa0 tx=0x7f4da0003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbcd479154f9f8fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 0x5582de65fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4d8affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4d8affd640 1 -- 172.21.15.7:0/1617477906 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da005d070 con 0x5582de655690 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.782+0000 7f4d8affd640 1 -- 172.21.15.7:0/1617477906 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4da00434b0 con 0x5582de655690 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.783+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4da0043ab0 con 0x5582de655690 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.783+0000 7f4d8affd640 1 -- 172.21.15.7:0/1617477906 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da0069050 con 0x5582de655690 2024-03-26T08:55:25.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.783+0000 7f4db0bab740 10 client.28633 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.783+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5582de66a2a0 con 0x5582de655690 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.783+0000 7f4d8affd640 1 -- 172.21.15.7:0/1617477906 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4da00436b0 con 0x5582de655690 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.783+0000 7f4d8affd640 1 -- 172.21.15.7:0/1617477906 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4da0045600 con 0x5582de655690 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.784+0000 7f4db0bab740 10 client.28633 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.784+0000 7f4db0bab740 10 client.28633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.784+0000 7f4db0bab740 20 client.28633 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.784+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5582de66b5f0 0x5582de66d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.784+0000 7f4db0bab740 4 client.28633 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.784+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4da0043ab0 con 0x5582de66b5f0 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.784+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5582de66b5f0 0x5582de66d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.784+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5582de66b5f0 0x5582de66d9e0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3e08162613f6d5 server_cookie=495aab2d99cf157e in_seq=0 out_seq=0 2024-03-26T08:55:25.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.784+0000 7f4d8affd640 10 client.28633 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:25.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.785+0000 7f4d8affd640 1 -- 172.21.15.7:0/1617477906 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f4da0043ab0 con 0x5582de66b5f0 2024-03-26T08:55:25.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.785+0000 7f4d8affd640 10 client.28633 handle_command_reply: tid=0 2024-03-26T08:55:25.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.785+0000 7f4db0bab740 1 client.28633 shutdown 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 msgr2=0x5582de65f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de65f9b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4da0045fa0 tx=0x7f4da0003970 comp rx=0 tx=0).stop 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 shutdown_connections 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5582de66b5f0 0x5582de66d9e0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de666f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de656080 0x5582de65fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1617477906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de65f9b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 >> 172.21.15.7:0/1617477906 conn(0x5582de650c00 msgr2=0x5582de64fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4d8affd640 0 client.28633 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 shutdown_connections 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 1 -- 172.21.15.7:0/1617477906 wait complete. 2024-03-26T08:55:25.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.786+0000 7f4db0bab740 20 client.28633 trim_cache size 0 max 16384 2024-03-26T08:55:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.804+0000 7f4db0bab740 1 Processor -- start 2024-03-26T08:55:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.804+0000 7f4db0bab740 1 -- start start 2024-03-26T08:55:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.804+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de655690 0x5582de65fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.804+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de656080 0x5582de666ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de64f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4db0bab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d900108c0 con 0x5582de65b1f0 2024-03-26T08:55:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4db0bab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000e3c0 con 0x5582de655690 2024-03-26T08:55:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4db0bab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000c440 con 0x5582de656080 2024-03-26T08:55:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de655690 0x5582de65fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de655690 0x5582de65fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52866/0 (socket says 172.21.15.7:52866) 2024-03-26T08:55:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da77fe640 1 -- 172.21.15.7:0/1611524237 learned_addr learned my addr 172.21.15.7:0/1611524237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de656080 0x5582de666ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da7fff640 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de64f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da6ffd640 1 -- 172.21.15.7:0/1611524237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de655690 msgr2=0x5582de65fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de655690 0x5582de65fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da6ffd640 1 -- 172.21.15.7:0/1611524237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 msgr2=0x5582de64f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de64f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da6ffd640 1 -- 172.21.15.7:0/1611524237 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582de64c480 con 0x5582de656080 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da7fff640 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de64f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da77fe640 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de655690 0x5582de65fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de656080 0x5582de666ec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009820 tx=0x7f4d9401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ee26c640aec8cf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.805+0000 7f4d9d7fa640 1 -- 172.21.15.7:0/1611524237 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9402ad20 con 0x5582de656080 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.806+0000 7f4d9d7fa640 1 -- 172.21.15.7:0/1611524237 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d9402e070 con 0x5582de656080 2024-03-26T08:55:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.806+0000 7f4d9d7fa640 1 -- 172.21.15.7:0/1611524237 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94018d30 con 0x5582de656080 2024-03-26T08:55:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.806+0000 7f4db0bab740 1 -- 172.21.15.7:0/1611524237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de656080 msgr2=0x5582de666ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.806+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de656080 0x5582de666ec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009820 tx=0x7f4d9401a000 comp rx=0 tx=0).stop 2024-03-26T08:55:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.806+0000 7f4db0bab740 1 -- 172.21.15.7:0/1611524237 shutdown_connections 2024-03-26T08:55:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.806+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de65b1f0 0x5582de64f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.806+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de656080 0x5582de666ec0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.806+0000 7f4db0bab740 1 --2- 172.21.15.7:0/1611524237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de655690 0x5582de65fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.807+0000 7f4db0bab740 1 -- 172.21.15.7:0/1611524237 >> 172.21.15.7:0/1611524237 conn(0x5582de650c00 msgr2=0x5582de65f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.807+0000 7f4db0bab740 1 -- 172.21.15.7:0/1611524237 shutdown_connections 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.807+0000 7f4db0bab740 1 -- 172.21.15.7:0/1611524237 wait complete. 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.807+0000 7f4db0bab740 1 -- start start 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.807+0000 7f4db0bab740 1 Processor -- start 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.807+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de6668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.807+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de656080 0x5582de666e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de6668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4da77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de6668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44572/0 (socket says 172.21.15.7:44572) 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4db0bab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de65b1f0 0x5582de664ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4db0bab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000ca40 con 0x5582de656080 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4db0bab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000dd60 con 0x5582de65b1f0 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4db0bab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9000d0a0 con 0x5582de655690 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4db0bab740 10 client.0 _ll_register_callbacks cb 0x7f4da003dd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4da77fe640 1 -- 172.21.15.7:0/3720551511 learned_addr learned my addr 172.21.15.7:0/3720551511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:25.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de656080 0x5582de666e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4da77fe640 1 -- 172.21.15.7:0/3720551511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de65b1f0 msgr2=0x5582de664ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4da77fe640 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de65b1f0 0x5582de664ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4da77fe640 1 -- 172.21.15.7:0/3720551511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de656080 msgr2=0x5582de666e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4da77fe640 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de656080 0x5582de666e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.808+0000 7f4da77fe640 1 -- 172.21.15.7:0/3720551511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582de665720 con 0x5582de655690 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.809+0000 7f4da77fe640 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de6668e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4da00459a0 tx=0x7f4da0045dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c63db7d8e30729d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.809+0000 7f4d8affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.809+0000 7f4d8affd640 1 -- 172.21.15.7:0/3720551511 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0066070 con 0x5582de655690 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.809+0000 7f4d8affd640 1 -- 172.21.15.7:0/3720551511 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4da00436c0 con 0x5582de655690 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.809+0000 7f4db0bab740 1 -- 172.21.15.7:0/3720551511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d9402e070 con 0x5582de655690 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.809+0000 7f4d8affd640 1 -- 172.21.15.7:0/3720551511 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da005d040 con 0x5582de655690 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.809+0000 7f4db0bab740 10 client.28639 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4db0bab740 1 -- 172.21.15.7:0/3720551511 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5582de6c7900 con 0x5582de655690 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4d8affd640 1 -- 172.21.15.7:0/3720551511 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4da0043ab0 con 0x5582de655690 2024-03-26T08:55:25.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4d8affd640 1 -- 172.21.15.7:0/3720551511 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4da0004530 con 0x5582de655690 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4db0bab740 10 client.28639 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4db0bab740 10 client.28639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4db0bab740 20 client.28639 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4db0bab740 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5582de6c8230 0x5582de6ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4db0bab740 4 client.28639 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4db0bab740 1 -- 172.21.15.7:0/3720551511 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f4d9402e070 con 0x5582de6c8230 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.810+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5582de6c8230 0x5582de6ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4da6ffd640 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5582de6c8230 0x5582de6ca650 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac43fba6156f6de0 server_cookie=27d5b87fe1318129 in_seq=0 out_seq=0 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4d8affd640 10 client.28639 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4d8affd640 1 -- 172.21.15.7:0/3720551511 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f4d9402e070 con 0x5582de6c8230 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4d8affd640 10 client.28639 handle_command_reply: tid=0 2024-03-26T08:55:25.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 client.28639 shutdown 2024-03-26T08:55:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 -- 172.21.15.7:0/3720551511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 msgr2=0x5582de6668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de6668e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4da00459a0 tx=0x7f4da0045dc0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 -- 172.21.15.7:0/3720551511 shutdown_connections 2024-03-26T08:55:25.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5582de6c8230 0x5582de6ca650 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4d8affd640 0 client.28639 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582de65b1f0 0x5582de664ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582de656080 0x5582de666e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 --2- 172.21.15.7:0/3720551511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5582de655690 0x5582de6668e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 -- 172.21.15.7:0/3720551511 >> 172.21.15.7:0/3720551511 conn(0x5582de650c00 msgr2=0x5582de653e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 -- 172.21.15.7:0/3720551511 shutdown_connections 2024-03-26T08:55:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 1 -- 172.21.15.7:0/3720551511 wait complete. 2024-03-26T08:55:25.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.811+0000 7f4db0bab740 20 client.28639 trim_cache size 0 max 16384 2024-03-26T08:55:25.813 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:55:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 -- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4d8c0a2a40 msgr2=0x7f4d8c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4d8c0a2a40 0x7f4d8c0a4f00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4d94006dc0 tx=0x7f4d94009960 comp rx=0 tx=0).stop 2024-03-26T08:55:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 -- 172.21.15.7:0/2492588552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da810d9c0 msgr2=0x7f4da81a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:25.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da810d9c0 0x7f4da81a5630 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800c970 tx=0x7f4d9800ce30 comp rx=0 tx=0).stop 2024-03-26T08:55:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 -- 172.21.15.7:0/2492588552 shutdown_connections 2024-03-26T08:55:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4d8c0a2a40 0x7f4d8c0a4f00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4da810d9c0 0x7f4da81a5630 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4da810b010 0x7f4da819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 --2- 172.21.15.7:0/2492588552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da81086a0 0x7f4da819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 -- 172.21.15.7:0/2492588552 >> 172.21.15.7:0/2492588552 conn(0x7f4da806e640 msgr2=0x7f4da8108e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 -- 172.21.15.7:0/2492588552 shutdown_connections 2024-03-26T08:55:25.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:25.813+0000 7f4dadfc4640 1 -- 172.21.15.7:0/2492588552 wait complete. 2024-03-26T08:55:25.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:25 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1456788945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:55:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 775.0 s, 1670.6 tps, lat 60.167 ms stddev 72.344 2024-03-26T08:55:26.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:26 smithi016 ceph-mon[35851]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.7 MiB/s wr, 680 op/s 2024-03-26T08:55:26.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:26 smithi007 ceph-mon[32462]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.7 MiB/s wr, 680 op/s 2024-03-26T08:55:26.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:26 smithi044 ceph-mon[36461]: pgmap v750: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.7 MiB/s wr, 680 op/s 2024-03-26T08:55:26.826 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:55:26.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:55:26.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:55:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.965+0000 7f8cdadcd640 1 -- 172.21.15.7:0/2799321987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd41016f0 msgr2=0x7f8cd4101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.965+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/2799321987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd41016f0 0x7f8cd4101b10 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009900 tx=0x7f8cc802bc30 comp rx=0 tx=0).stop 2024-03-26T08:55:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 -- 172.21.15.7:0/2799321987 shutdown_connections 2024-03-26T08:55:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/2799321987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41032a0 0x7f8cd4109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/2799321987 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd41028e0 0x7f8cd4102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/2799321987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd41016f0 0x7f8cd4101b10 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 -- 172.21.15.7:0/2799321987 >> 172.21.15.7:0/2799321987 conn(0x7f8cd40fcc60 msgr2=0x7f8cd40ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:26.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 -- 172.21.15.7:0/2799321987 shutdown_connections 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 -- 172.21.15.7:0/2799321987 wait complete. 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 Processor -- start 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 -- start start 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd41016f0 0x7f8cd4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41028e0 0x7f8cd4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd41032a0 0x7f8cd419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd410d360 con 0x7f8cd41032a0 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd410d1e0 con 0x7f8cd41028e0 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cdadcd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd410d4e0 con 0x7f8cd41016f0 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cd8b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd41016f0 0x7f8cd4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41028e0 0x7f8cd4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cd8b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd41016f0 0x7f8cd4197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:37790/0 (socket says 172.21.15.7:37790) 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41028e0 0x7f8cd4197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47284/0 (socket says 172.21.15.7:47284) 2024-03-26T08:55:26.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.966+0000 7f8cd8b69640 1 -- 172.21.15.7:0/1295184057 learned_addr learned my addr 172.21.15.7:0/1295184057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.967+0000 7f8cd936a640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd41032a0 0x7f8cd419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.967+0000 7f8cd8b69640 1 -- 172.21.15.7:0/1295184057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41028e0 msgr2=0x7f8cd4197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.967+0000 7f8cd8b69640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41028e0 0x7f8cd4197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.967+0000 7f8cd8b69640 1 -- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd41032a0 msgr2=0x7f8cd419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.967+0000 7f8cd8b69640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd41032a0 0x7f8cd419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.967+0000 7f8cd8b69640 1 -- 172.21.15.7:0/1295184057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd419f630 con 0x7f8cd41016f0 2024-03-26T08:55:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.967+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41028e0 0x7f8cd4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.967+0000 7f8cd8b69640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd41016f0 0x7f8cd4197a10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8000c00 tx=0x7f8cc802c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:26.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.968+0000 7f8cd1ffb640 1 -- 172.21.15.7:0/1295184057 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8041070 con 0x7f8cd41016f0 2024-03-26T08:55:26.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.968+0000 7f8cd1ffb640 1 -- 172.21.15.7:0/1295184057 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc8004450 con 0x7f8cd41016f0 2024-03-26T08:55:26.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.968+0000 7f8cd1ffb640 1 -- 172.21.15.7:0/1295184057 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8034440 con 0x7f8cd41016f0 2024-03-26T08:55:26.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.968+0000 7f8cdadcd640 1 -- 172.21.15.7:0/1295184057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd419f8c0 con 0x7f8cd41016f0 2024-03-26T08:55:26.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.969+0000 7f8cdadcd640 1 -- 172.21.15.7:0/1295184057 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd419fd60 con 0x7f8cd41016f0 2024-03-26T08:55:26.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.971+0000 7f8cd1ffb640 1 -- 172.21.15.7:0/1295184057 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8cc8044070 con 0x7f8cd41016f0 2024-03-26T08:55:26.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cdd9ba740 1 Processor -- start 2024-03-26T08:55:26.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cdd9ba740 1 -- start start 2024-03-26T08:55:26.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 0x5591e792d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e7932260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e7934d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cdd9ba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e7868c90 con 0x5591e792d760 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cdd9ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e788e070 con 0x5591e792cd70 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cdd9ba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e7836c80 con 0x5591e79328d0 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd8b69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e7932260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd936a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 0x5591e792d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd1ffb640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cc40a2950 0x7f8cc40a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e7934d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd936a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 0x5591e792d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47296/0 (socket says 172.21.15.7:47296) 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd936a640 1 -- 172.21.15.7:0/3448389910 learned_addr learned my addr 172.21.15.7:0/3448389910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:26.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd8b69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e7932260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44432/0 (socket says 172.21.15.7:44432) 2024-03-26T08:55:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd936a640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cc40a2950 0x7f8cc40a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd3fff640 1 -- 172.21.15.7:0/3448389910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 msgr2=0x5591e792d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/3448389910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 0x5591e792d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.972+0000 7f8cd3fff640 1 -- 172.21.15.7:0/3448389910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 msgr2=0x5591e7932260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.973+0000 7f8cd1ffb640 1 -- 172.21.15.7:0/1295184057 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8cc80f0460 con 0x7f8cd41016f0 2024-03-26T08:55:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.973+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/3448389910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e7932260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.973+0000 7f8cd3fff640 1 -- 172.21.15.7:0/3448389910 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e782ed10 con 0x5591e79328d0 2024-03-26T08:55:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.973+0000 7f8cd936a640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cc40a2950 0x7f8cc40a4e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5591e792dfc0 tx=0x7f8ccc004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.973+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/3448389910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e7934d60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009fa0 tx=0x7f8cc00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81128a6bcc6a365 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.973+0000 7f8cbeffd640 1 -- 172.21.15.7:0/3448389910 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0004310 con 0x5591e79328d0 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.973+0000 7f8cbeffd640 1 -- 172.21.15.7:0/3448389910 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc00044b0 con 0x5591e79328d0 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.973+0000 7f8cbeffd640 1 -- 172.21.15.7:0/3448389910 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00035f0 con 0x5591e79328d0 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.974+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/3448389910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 msgr2=0x5591e7934d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.974+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/3448389910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e7934d60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009fa0 tx=0x7f8cc00118d0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.974+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/3448389910 shutdown_connections 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.974+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/3448389910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e7934d60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.974+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/3448389910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e7932260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.974+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/3448389910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 0x5591e792d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.974+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/3448389910 >> 172.21.15.7:0/3448389910 conn(0x5591e78f11a0 msgr2=0x5591e79242d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.975+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/3448389910 shutdown_connections 2024-03-26T08:55:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.975+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/3448389910 wait complete. 2024-03-26T08:55:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.975+0000 7f8cdd9ba740 1 -- start start 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.975+0000 7f8cdd9ba740 1 Processor -- start 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792cd70 0x5591e7936f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792d760 0x5591e7937440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd8b69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792cd70 0x5591e7936f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd8b69640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792cd70 0x5591e7936f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44436/0 (socket says 172.21.15.7:44436) 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd8b69640 1 -- 172.21.15.7:0/1525603264 learned_addr learned my addr 172.21.15.7:0/1525603264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e793e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e784a0d0 con 0x5591e792cd70 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e788a110 con 0x5591e792d760 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591e7935880 con 0x5591e79328d0 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cdd9ba740 10 client.0 _ll_register_callbacks cb 0x5591e7885e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792d760 0x5591e7937440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd936a640 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e793e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd8b69640 1 -- 172.21.15.7:0/1525603264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 msgr2=0x5591e793e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd8b69640 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e793e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd8b69640 1 -- 172.21.15.7:0/1525603264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792d760 msgr2=0x5591e7937440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd8b69640 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792d760 0x5591e7937440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd8b69640 1 -- 172.21.15.7:0/1525603264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591e766a7e0 con 0x5591e792cd70 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd936a640 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e793e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cd8b69640 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792cd70 0x5591e7936f00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8cc802c7b0 tx=0x7f8cc8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c66e484bcaf30c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.976+0000 7f8cb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cb3fff640 1 -- 172.21.15.7:0/1525603264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80bc860 con 0x5591e792cd70 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cb3fff640 1 -- 172.21.15.7:0/1525603264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8cc80c3070 con 0x5591e792cd70 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cc00044b0 con 0x5591e792cd70 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cb3fff640 1 -- 172.21.15.7:0/1525603264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc8004c80 con 0x5591e792cd70 2024-03-26T08:55:26.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cdd9ba740 10 client.18864 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5591e76caf70 con 0x5591e792cd70 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cb3fff640 1 -- 172.21.15.7:0/1525603264 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8cc80bf070 con 0x5591e792cd70 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cb3fff640 1 -- 172.21.15.7:0/1525603264 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8cc80c50d0 con 0x5591e792cd70 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cdd9ba740 10 client.18864 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cdd9ba740 10 client.18864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cdd9ba740 20 client.18864 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5591e7942c90 0x5591e79450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cdd9ba740 4 client.18864 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.977+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cc00044b0 con 0x5591e7942c90 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.978+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5591e7942c90 0x5591e79450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.978+0000 7f8cb3fff640 10 client.18864 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.978+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5591e7942c90 0x5591e79450a0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd61915beb998c2a server_cookie=6b55eb6ffe5214b8 in_seq=0 out_seq=0 2024-03-26T08:55:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cb3fff640 1 -- 172.21.15.7:0/1525603264 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f8cc00044b0 con 0x5591e7942c90 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cb3fff640 10 client.18864 handle_command_reply: tid=0 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 client.18864 shutdown 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792cd70 msgr2=0x5591e7936f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792cd70 0x5591e7936f00 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8cc802c7b0 tx=0x7f8cc8009560 comp rx=0 tx=0).stop 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 shutdown_connections 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5591e7942c90 0x5591e79450a0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e793e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792d760 0x5591e7937440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/1525603264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792cd70 0x5591e7936f00 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 >> 172.21.15.7:0/1525603264 conn(0x5591e78f11a0 msgr2=0x5591e7923530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cb3fff640 0 client.18864 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 shutdown_connections 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/1525603264 wait complete. 2024-03-26T08:55:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:26.979+0000 7f8cdd9ba740 20 client.18864 trim_cache size 0 max 16384 2024-03-26T08:55:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.000+0000 7f8cdd9ba740 1 Processor -- start 2024-03-26T08:55:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cdd9ba740 1 -- start start 2024-03-26T08:55:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e792cd70 0x5591e7927c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e79328d0 0x5591e7936b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cdd9ba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a8e00 con 0x5591e792d760 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cdd9ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a6900 con 0x5591e79328d0 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cdd9ba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a4980 con 0x5591e792cd70 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44440/0 (socket says 172.21.15.7:44440) 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd8b69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e792cd70 0x5591e7927c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd3fff640 1 -- 172.21.15.7:0/650435009 learned_addr learned my addr 172.21.15.7:0/650435009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd936a640 1 --2- 172.21.15.7:0/650435009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e79328d0 0x5591e7936b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd936a640 1 -- 172.21.15.7:0/650435009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e792cd70 msgr2=0x5591e7927c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd936a640 1 --2- 172.21.15.7:0/650435009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e792cd70 0x5591e7927c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd936a640 1 -- 172.21.15.7:0/650435009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 msgr2=0x5591e793e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd936a640 1 --2- 172.21.15.7:0/650435009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd936a640 1 -- 172.21.15.7:0/650435009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591e7562170 con 0x5591e79328d0 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.001+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/650435009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.002+0000 7f8cd936a640 1 --2- 172.21.15.7:0/650435009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e79328d0 0x5591e7936b10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc005fd0 tx=0x7f8ccc009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3957318631b93fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.003+0000 7f8cbe7fc640 1 -- 172.21.15.7:0/650435009 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc026070 con 0x5591e79328d0 2024-03-26T08:55:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.003+0000 7f8cbe7fc640 1 -- 172.21.15.7:0/650435009 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ccc022b30 con 0x5591e79328d0 2024-03-26T08:55:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.003+0000 7f8cbe7fc640 1 -- 172.21.15.7:0/650435009 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc010c90 con 0x5591e79328d0 2024-03-26T08:55:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/650435009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e79328d0 msgr2=0x5591e7936b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/650435009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e79328d0 0x5591e7936b10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc005fd0 tx=0x7f8ccc009be0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/650435009 shutdown_connections 2024-03-26T08:55:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/650435009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e79328d0 0x5591e7936b10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/650435009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/650435009 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e792cd70 0x5591e7927c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/650435009 >> 172.21.15.7:0/650435009 conn(0x5591e78f11a0 msgr2=0x5591e7927510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/650435009 shutdown_connections 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/650435009 wait complete. 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.004+0000 7f8cdd9ba740 1 -- start start 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cdd9ba740 1 Processor -- start 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 0x5591e793ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cdd9ba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e793c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cdd9ba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a4f80 con 0x5591e792d760 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cdd9ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a62a0 con 0x5591e792cd70 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cdd9ba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41a55e0 con 0x5591e79328d0 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cdd9ba740 10 client.0 _ll_register_callbacks cb 0x7f8cc8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd936a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e793c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44448/0 (socket says 172.21.15.7:44448) 2024-03-26T08:55:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd3fff640 1 -- 172.21.15.7:0/66923521 learned_addr learned my addr 172.21.15.7:0/66923521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd8b69640 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 0x5591e793ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd3fff640 1 -- 172.21.15.7:0/66923521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 msgr2=0x5591e793c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e793c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd3fff640 1 -- 172.21.15.7:0/66923521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 msgr2=0x5591e793ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 0x5591e793ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.005+0000 7f8cd3fff640 1 -- 172.21.15.7:0/66923521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591e793cc60 con 0x5591e792d760 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cd3fff640 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e330 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009fa0 tx=0x7f8cc0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847c57eb6cf0fe5e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cb3fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cb3fff640 1 -- 172.21.15.7:0/66923521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc001d040 con 0x5591e792d760 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/66923521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ccc022b30 con 0x5591e792d760 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cb3fff640 1 -- 172.21.15.7:0/66923521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8cc002b1a0 con 0x5591e792d760 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cb3fff640 1 -- 172.21.15.7:0/66923521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc00035f0 con 0x5591e792d760 2024-03-26T08:55:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cdd9ba740 10 client.18870 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/66923521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5591e799f130 con 0x5591e792d760 2024-03-26T08:55:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cb3fff640 1 -- 172.21.15.7:0/66923521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ccc022b30 con 0x5591e792d760 2024-03-26T08:55:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.006+0000 7f8cb3fff640 1 -- 172.21.15.7:0/66923521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8cc002b520 con 0x5591e792d760 2024-03-26T08:55:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.007+0000 7f8cdd9ba740 10 client.18870 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.007+0000 7f8cdd9ba740 10 client.18870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.007+0000 7f8cdd9ba740 20 client.18870 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.007+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5591e799f850 0x5591e79a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.007+0000 7f8cdd9ba740 4 client.18870 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:55:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.007+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/66923521 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5591e789b5b0 con 0x5591e799f850 2024-03-26T08:55:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.010+0000 7f8cd8b69640 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5591e799f850 0x5591e79a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.019+0000 7f8cd8b69640 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5591e799f850 0x5591e79a1c70 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb4dcf81f5b1557d server_cookie=5f9786ce2bc9ca4 in_seq=0 out_seq=0 2024-03-26T08:55:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.022+0000 7f8cb3fff640 10 client.18870 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.034+0000 7f8cb3fff640 1 -- 172.21.15.7:0/66923521 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5591e789b5b0 con 0x5591e799f850 2024-03-26T08:55:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.034+0000 7f8cb3fff640 10 client.18870 handle_command_reply: tid=0 2024-03-26T08:55:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.034+0000 7f8cdd9ba740 1 client.18870 shutdown 2024-03-26T08:55:27.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.039+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/66923521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 msgr2=0x5591e793e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.039+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e330 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009fa0 tx=0x7f8cc0009310 comp rx=0 tx=0).stop 2024-03-26T08:55:27.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.040+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/66923521 shutdown_connections 2024-03-26T08:55:27.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.040+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5591e799f850 0x5591e79a1c70 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.040+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5591e79328d0 0x5591e793c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.040+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5591e792d760 0x5591e793e330 secure :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009fa0 tx=0x7f8cc0009310 comp rx=0 tx=0).stop 2024-03-26T08:55:27.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.040+0000 7f8cdd9ba740 1 --2- 172.21.15.7:0/66923521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5591e792cd70 0x5591e793ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.040+0000 7f8cb3fff640 0 client.18870 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:27.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.040+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/66923521 >> 172.21.15.7:0/66923521 conn(0x5591e78f11a0 msgr2=0x5591e7936a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:27.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.044+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/66923521 shutdown_connections 2024-03-26T08:55:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.046+0000 7f8cdd9ba740 1 -- 172.21.15.7:0/66923521 wait complete. 2024-03-26T08:55:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.051+0000 7f8cdd9ba740 20 client.18870 trim_cache size 0 max 16384 2024-03-26T08:55:27.051 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:55:27.051 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:55:27.051 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "3b51a7e0-9a85-4ac5-8f4b-a66f6fd4bb94", 2024-03-26T08:55:27.051 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:55:27.051 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:55:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 -- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cc40a2950 msgr2=0x7f8cc40a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cc40a2950 0x7f8cc40a4e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5591e792dfc0 tx=0x7f8ccc004ab0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 -- 172.21.15.7:0/1295184057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd41016f0 msgr2=0x7f8cd4197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd41016f0 0x7f8cd4197a10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8000c00 tx=0x7f8cc802c640 comp rx=0 tx=0).stop 2024-03-26T08:55:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 -- 172.21.15.7:0/1295184057 shutdown_connections 2024-03-26T08:55:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cc40a2950 0x7f8cc40a4e10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd41032a0 0x7f8cd419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41028e0 0x7f8cd4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 --2- 172.21.15.7:0/1295184057 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8cd41016f0 0x7f8cd4197a10 secure :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8000c00 tx=0x7f8cc802c640 comp rx=0 tx=0).stop 2024-03-26T08:55:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.056+0000 7f8cdadcd640 1 -- 172.21.15.7:0/1295184057 >> 172.21.15.7:0/1295184057 conn(0x7f8cd40fcc60 msgr2=0x7f8cd4105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.059+0000 7f8cdadcd640 1 -- 172.21.15.7:0/1295184057 shutdown_connections 2024-03-26T08:55:27.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.059+0000 7f8cdadcd640 1 -- 172.21.15.7:0/1295184057 wait complete. 2024-03-26T08:55:27.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:55:27.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:55:27.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.243+0000 7f7fd2f89640 1 -- 172.21.15.7:0/1288689672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc0fff30 msgr2=0x7f7fcc102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.243+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/1288689672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc0fff30 0x7f7fcc102360 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009be0 tx=0x7f7fc402bf10 comp rx=0 tx=0).stop 2024-03-26T08:55:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.245+0000 7f7fd2f89640 1 -- 172.21.15.7:0/1288689672 shutdown_connections 2024-03-26T08:55:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.245+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/1288689672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc105250 0x7f7fcc1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.245+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/1288689672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc1028a0 0x7f7fcc104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.245+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/1288689672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc0fff30 0x7f7fcc102360 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.245+0000 7f7fd2f89640 1 -- 172.21.15.7:0/1288689672 >> 172.21.15.7:0/1288689672 conn(0x7f7fcc0fbb30 msgr2=0x7f7fcc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.246+0000 7f7fd2f89640 1 -- 172.21.15.7:0/1288689672 shutdown_connections 2024-03-26T08:55:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.246+0000 7f7fd2f89640 1 -- 172.21.15.7:0/1288689672 wait complete. 2024-03-26T08:55:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.246+0000 7f7fd2f89640 1 Processor -- start 2024-03-26T08:55:27.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.246+0000 7f7fd2f89640 1 -- start start 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.246+0000 7f7fd2f89640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc0fff30 0x7f7fcc10b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.246+0000 7f7fd2f89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc1028a0 0x7f7fcc10b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.246+0000 7f7fd2f89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc105250 0x7f7fcc06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.246+0000 7f7fd2f89640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc113a00 con 0x7f7fcc1028a0 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd2f89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc113880 con 0x7f7fcc105250 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd2f89640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc113b80 con 0x7f7fcc0fff30 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd0d25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc0fff30 0x7f7fcc10b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd0d25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc0fff30 0x7f7fcc10b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:37858/0 (socket says 172.21.15.7:37858) 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd0d25640 1 -- 172.21.15.7:0/2764541077 learned_addr learned my addr 172.21.15.7:0/2764541077 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd1526640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc105250 0x7f7fcc06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fcbfff640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc1028a0 0x7f7fcc10b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd0d25640 1 -- 172.21.15.7:0/2764541077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc105250 msgr2=0x7f7fcc06f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd0d25640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc105250 0x7f7fcc06f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd0d25640 1 -- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc1028a0 msgr2=0x7f7fcc10b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd0d25640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc1028a0 0x7f7fcc10b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.247+0000 7f7fd0d25640 1 -- 172.21.15.7:0/2764541077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc10c060 con 0x7f7fcc0fff30 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.248+0000 7f7fd0d25640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc0fff30 0x7f7fcc10b480 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40334c0 tx=0x7f7fc4003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.248+0000 7f7fc9ffb640 1 -- 172.21.15.7:0/2764541077 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4043020 con 0x7f7fcc0fff30 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.248+0000 7f7fc9ffb640 1 -- 172.21.15.7:0/2764541077 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc40338d0 con 0x7f7fcc0fff30 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.249+0000 7f7fd2f89640 1 -- 172.21.15.7:0/2764541077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc06f6e0 con 0x7f7fcc0fff30 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.249+0000 7f7fd2f89640 1 -- 172.21.15.7:0/2764541077 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc06fcb0 con 0x7f7fcc0fff30 2024-03-26T08:55:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.250+0000 7f7fc9ffb640 1 -- 172.21.15.7:0/2764541077 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4043020 con 0x7f7fcc0fff30 2024-03-26T08:55:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.253+0000 7f7fd5b7b740 1 Processor -- start 2024-03-26T08:55:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.253+0000 7f7fd5b7b740 1 -- start start 2024-03-26T08:55:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.253+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde5da0 0x555dafde61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.253+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde6790 0x555dafdeb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.253+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 0x555dafdedd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.253+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dafd4b3d0 con 0x555dafde5da0 2024-03-26T08:55:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.253+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dafcf3ad0 con 0x555dafde6790 2024-03-26T08:55:27.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.253+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dafd47470 con 0x555dafdeb900 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.253+0000 7f7fd1526640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde5da0 0x555dafde61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fd1526640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde5da0 0x555dafde61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44478/0 (socket says 172.21.15.7:44478) 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fd1526640 1 -- 172.21.15.7:0/1071040026 learned_addr learned my addr 172.21.15.7:0/1071040026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fcbfff640 1 --2- 172.21.15.7:0/1071040026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 0x555dafdedd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fc9ffb640 1 -- 172.21.15.7:0/2764541077 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7fc4046020 con 0x7f7fcc0fff30 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fd1526640 1 -- 172.21.15.7:0/1071040026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 msgr2=0x555dafdedd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fd1526640 1 --2- 172.21.15.7:0/1071040026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 0x555dafdedd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fd1526640 1 -- 172.21.15.7:0/1071040026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde6790 msgr2=0x555dafdeb290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fd1526640 1 --2- 172.21.15.7:0/1071040026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde6790 0x555dafdeb290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fd1526640 1 -- 172.21.15.7:0/1071040026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dafb297d0 con 0x555dafde5da0 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fd1526640 1 --2- 172.21.15.7:0/1071040026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde5da0 0x555dafde61c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00099c0 tx=0x7f7fc0009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7562f06c8fa2c584 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fb2ffd640 1 -- 172.21.15.7:0/1071040026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0017070 con 0x555dafde5da0 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.254+0000 7f7fc9ffb640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2c40 0x7f7fb40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.255+0000 7f7fb2ffd640 1 -- 172.21.15.7:0/1071040026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0013470 con 0x555dafde5da0 2024-03-26T08:55:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.255+0000 7f7fb2ffd640 1 -- 172.21.15.7:0/1071040026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0012630 con 0x555dafde5da0 2024-03-26T08:55:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.255+0000 7f7fd0d25640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2c40 0x7f7fb40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.255+0000 7f7fc9ffb640 1 -- 172.21.15.7:0/2764541077 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fc403a040 con 0x7f7fcc0fff30 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.255+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1071040026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde5da0 msgr2=0x555dafde61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.255+0000 7f7fd0d25640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2c40 0x7f7fb40a5100 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40ee010 tx=0x7f7fc402bf10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.255+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1071040026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde5da0 0x555dafde61c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00099c0 tx=0x7f7fc0009e80 comp rx=0 tx=0).stop 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.255+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1071040026 shutdown_connections 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.255+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1071040026 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 0x555dafdedd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.256+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1071040026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde6790 0x555dafdeb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.256+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1071040026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde5da0 0x555dafde61c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.256+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1071040026 >> 172.21.15.7:0/1071040026 conn(0x555dafde1310 msgr2=0x555dafde1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.256+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1071040026 shutdown_connections 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.256+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1071040026 wait complete. 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.256+0000 7f7fd5b7b740 1 -- start start 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.256+0000 7f7fd5b7b740 1 Processor -- start 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafdf01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde6790 0x555dafdf0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafdeb900 0x555dafdf7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dafdee8b0 con 0x555dafdeb900 2024-03-26T08:55:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dafd07060 con 0x555dafde5da0 2024-03-26T08:55:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dafdeea30 con 0x555dafde6790 2024-03-26T08:55:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd1526640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafdf01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd1526640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafdf01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47366/0 (socket says 172.21.15.7:47366) 2024-03-26T08:55:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd1526640 1 -- 172.21.15.7:0/1374968303 learned_addr learned my addr 172.21.15.7:0/1374968303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd1526640 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafdeb900 0x555dafdf7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fcbfff640 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde6790 0x555dafdf0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd5b7b740 10 client.0 _ll_register_callbacks cb 0x555dafce96a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd1526640 1 -- 172.21.15.7:0/1374968303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde6790 msgr2=0x555dafdf0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd1526640 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde6790 0x555dafdf0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd1526640 1 -- 172.21.15.7:0/1374968303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafdeb900 msgr2=0x555dafdf7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd1526640 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafdeb900 0x555dafdf7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.257+0000 7f7fd1526640 1 -- 172.21.15.7:0/1374968303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dafa5f3f0 con 0x555dafde5da0 2024-03-26T08:55:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.258+0000 7f7fd1526640 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafdf01f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0004790 tx=0x7f7fc00046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b623ccbef821b43b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.258+0000 7f7fabfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:55:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.258+0000 7f7fabfff640 1 -- 172.21.15.7:0/1374968303 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0017040 con 0x555dafde5da0 2024-03-26T08:55:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.258+0000 7f7fabfff640 1 -- 172.21.15.7:0/1374968303 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fc001c450 con 0x555dafde5da0 2024-03-26T08:55:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.258+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1374968303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fc0013470 con 0x555dafde5da0 2024-03-26T08:55:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.259+0000 7f7fabfff640 1 -- 172.21.15.7:0/1374968303 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fc001c650 con 0x555dafde5da0 2024-03-26T08:55:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.259+0000 7f7fd5b7b740 10 client.28664 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.259+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1374968303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555dafa1f0f0 con 0x555dafde5da0 2024-03-26T08:55:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.260+0000 7f7fabfff640 1 -- 172.21.15.7:0/1374968303 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc001ccd0 con 0x555dafde5da0 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.260+0000 7f7fabfff640 1 -- 172.21.15.7:0/1374968303 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7fc0028020 con 0x555dafde5da0 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.260+0000 7f7fd5b7b740 10 client.28664 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.260+0000 7f7fd5b7b740 10 client.28664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.260+0000 7f7fd5b7b740 20 client.28664 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.260+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555dafdfb870 0x555dafdfdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.260+0000 7f7fd5b7b740 4 client.28664 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.260+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1374968303 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fc0013470 con 0x555dafdfb870 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.261+0000 7f7fcbfff640 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555dafdfb870 0x555dafdfdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.262+0000 7f7fabfff640 10 client.28664 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.262+0000 7f7fcbfff640 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555dafdfb870 0x555dafdfdc80 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4297e6dcfce88e server_cookie=a7cefd6ced86c1d4 in_seq=0 out_seq=0 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.263+0000 7f7fabfff640 1 -- 172.21.15.7:0/1374968303 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f7fc0013470 con 0x555dafdfb870 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.263+0000 7f7fabfff640 10 client.28664 handle_command_reply: tid=0 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 client.28664 shutdown 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1374968303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 msgr2=0x555dafdf01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafdf01f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0004790 tx=0x7f7fc00046e0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1374968303 shutdown_connections 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555dafdfb870 0x555dafdfdc80 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafdeb900 0x555dafdf7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fabfff640 0 client.28664 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde6790 0x555dafdf0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/1374968303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafdf01f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1374968303 >> 172.21.15.7:0/1374968303 conn(0x555dafde1310 msgr2=0x555dafde0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1374968303 shutdown_connections 2024-03-26T08:55:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/1374968303 wait complete. 2024-03-26T08:55:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.264+0000 7f7fd5b7b740 20 client.28664 trim_cache size 0 max 16384 2024-03-26T08:55:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.286+0000 7f7fd5b7b740 1 Processor -- start 2024-03-26T08:55:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.286+0000 7f7fd5b7b740 1 -- start start 2024-03-26T08:55:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafddfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 0x555dafde01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1b1450 con 0x555dafde6790 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1aef50 con 0x555dafde5da0 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1ad0a0 con 0x555dafdeb900 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fcbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd1526640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 0x555dafde01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fcbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44494/0 (socket says 172.21.15.7:44494) 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fcbfff640 1 -- 172.21.15.7:0/2330153833 learned_addr learned my addr 172.21.15.7:0/2330153833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd1526640 1 --2- 172.21.15.7:0/2330153833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafddfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd1526640 1 -- 172.21.15.7:0/2330153833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 msgr2=0x555dafddfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd1526640 1 --2- 172.21.15.7:0/2330153833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafddfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd1526640 1 -- 172.21.15.7:0/2330153833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 msgr2=0x555dafdf7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd1526640 1 --2- 172.21.15.7:0/2330153833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fd1526640 1 -- 172.21.15.7:0/2330153833 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dafde2080 con 0x555dafdeb900 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.287+0000 7f7fcbfff640 1 --2- 172.21.15.7:0/2330153833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.288+0000 7f7fd1526640 1 --2- 172.21.15.7:0/2330153833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 0x555dafde01e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0000ea0 tx=0x7f7fc0009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58713772801affea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.288+0000 7f7fb27fc640 1 -- 172.21.15.7:0/2330153833 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0017070 con 0x555dafdeb900 2024-03-26T08:55:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.288+0000 7f7fb27fc640 1 -- 172.21.15.7:0/2330153833 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0013470 con 0x555dafdeb900 2024-03-26T08:55:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.288+0000 7f7fb27fc640 1 -- 172.21.15.7:0/2330153833 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0012750 con 0x555dafdeb900 2024-03-26T08:55:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/2330153833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 msgr2=0x555dafde01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/2330153833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 0x555dafde01e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0000ea0 tx=0x7f7fc0009e80 comp rx=0 tx=0).stop 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/2330153833 shutdown_connections 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/2330153833 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafdeb900 0x555dafde01e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/2330153833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/2330153833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafde5da0 0x555dafddfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/2330153833 >> 172.21.15.7:0/2330153833 conn(0x555dafde1310 msgr2=0x555dafdefe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/2330153833 shutdown_connections 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/2330153833 wait complete. 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 -- start start 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.289+0000 7f7fd5b7b740 1 Processor -- start 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde5da0 0x555dafdf7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fd5b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafdeb900 0x555dafdf56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1ad610 con 0x555dafde6790 2024-03-26T08:55:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1ae8f0 con 0x555dafdeb900 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fd5b7b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1adc30 con 0x555dafde5da0 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fcbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fd1526640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde5da0 0x555dafdf7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fcbfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44510/0 (socket says 172.21.15.7:44510) 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fcbfff640 1 -- 172.21.15.7:0/3036277342 learned_addr learned my addr 172.21.15.7:0/3036277342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fd5b7b740 10 client.0 _ll_register_callbacks cb 0x7f7fc0009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fcbfff640 1 -- 172.21.15.7:0/3036277342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde5da0 msgr2=0x555dafdf7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fcbfff640 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde5da0 0x555dafdf7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fcbfff640 1 -- 172.21.15.7:0/3036277342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafdeb900 msgr2=0x555dafdf56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fcbfff640 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafdeb900 0x555dafdf56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fcbfff640 1 -- 172.21.15.7:0/3036277342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dafdf5e30 con 0x555dafde6790 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fcbfff640 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7550 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00bbc0 tx=0x7f7fbc01da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a6173f3e0bc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.290+0000 7f7fabfff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.291+0000 7f7fabfff640 1 -- 172.21.15.7:0/3036277342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc003a00 con 0x555dafde6790 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.291+0000 7f7fabfff640 1 -- 172.21.15.7:0/3036277342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fbc003ba0 con 0x555dafde6790 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.291+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/3036277342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fc0013470 con 0x555dafde6790 2024-03-26T08:55:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.291+0000 7f7fabfff640 1 -- 172.21.15.7:0/3036277342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fbc003da0 con 0x555dafde6790 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.291+0000 7f7fd5b7b740 10 client.18888 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.291+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/3036277342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555dafe57c40 con 0x555dafde6790 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.291+0000 7f7fabfff640 1 -- 172.21.15.7:0/3036277342 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0013470 con 0x555dafde6790 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.291+0000 7f7fabfff640 1 -- 172.21.15.7:0/3036277342 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7fbc025410 con 0x555dafde6790 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.291+0000 7f7fd5b7b740 10 client.18888 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.292+0000 7f7fd5b7b740 10 client.18888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.292+0000 7f7fd5b7b740 20 client.18888 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.292+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555dafdfb870 0x555dafe5a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.292+0000 7f7fd5b7b740 4 client.18888 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.292+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/3036277342 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555dafd58910 con 0x555dafdfb870 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.292+0000 7f7fd1526640 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555dafdfb870 0x555dafe5a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:27.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.292+0000 7f7fabfff640 10 client.18888 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.293+0000 7f7fd1526640 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555dafdfb870 0x555dafe5a640 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc5089c025dfb93 server_cookie=9b4fcb62ba26de79 in_seq=0 out_seq=0 2024-03-26T08:55:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fabfff640 1 -- 172.21.15.7:0/3036277342 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x555dafd58910 con 0x555dafdfb870 2024-03-26T08:55:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fabfff640 10 client.18888 handle_command_reply: tid=0 2024-03-26T08:55:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 client.18888 shutdown 2024-03-26T08:55:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/3036277342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 msgr2=0x555dafdf7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7550 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00bbc0 tx=0x7f7fbc01da50 comp rx=0 tx=0).stop 2024-03-26T08:55:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/3036277342 shutdown_connections 2024-03-26T08:55:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555dafdfb870 0x555dafe5a640 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555dafdeb900 0x555dafdf56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555dafde6790 0x555dafdf7550 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 --2- 172.21.15.7:0/3036277342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555dafde5da0 0x555dafdf7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/3036277342 >> 172.21.15.7:0/3036277342 conn(0x555dafde1310 msgr2=0x555dafde4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fabfff640 0 client.18888 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/3036277342 shutdown_connections 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.294+0000 7f7fd5b7b740 1 -- 172.21.15.7:0/3036277342 wait complete. 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (247 inodes in the stack)", 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "3b51a7e0-9a85-4ac5-8f4b-a66f6fd4bb94": { 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:55:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "3b51a7e0-9a85-4ac5-8f4b-a66f6fd4bb94", 2024-03-26T08:55:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:55:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:55:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:55:27.295 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd5b7b740 20 client.18888 trim_cache size 0 max 16384 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 -- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2c40 msgr2=0x7f7fb40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2c40 0x7f7fb40a5100 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40ee010 tx=0x7f7fc402bf10 comp rx=0 tx=0).stop 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 -- 172.21.15.7:0/2764541077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc0fff30 msgr2=0x7f7fcc10b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc0fff30 0x7f7fcc10b480 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40334c0 tx=0x7f7fc4003ba0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 -- 172.21.15.7:0/2764541077 shutdown_connections 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2c40 0x7f7fb40a5100 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc105250 0x7f7fcc06f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc1028a0 0x7f7fcc10b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 --2- 172.21.15.7:0/2764541077 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc0fff30 0x7f7fcc10b480 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 -- 172.21.15.7:0/2764541077 >> 172.21.15.7:0/2764541077 conn(0x7f7fcc0fbb30 msgr2=0x7f7fcc103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 -- 172.21.15.7:0/2764541077 shutdown_connections 2024-03-26T08:55:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:27.295+0000 7f7fd2f89640 1 -- 172.21.15.7:0/2764541077 wait complete. 2024-03-26T08:55:27.307 INFO:tasks.cephfs.filesystem:scrub status for tag:3b51a7e0-9a85-4ac5-8f4b-a66f6fd4bb94 - {'path': '/', 'tag': '3b51a7e0-9a85-4ac5-8f4b-a66f6fd4bb94', 'options': 'recursive,force'} 2024-03-26T08:55:27.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:27 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:55:27.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:27 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:55:27.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:27 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:55:27.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:27 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:55:27.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:27 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:55:27.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:27 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:55:27.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:27 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:55:27.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:27 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:55:27.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:27 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:55:28.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:28 smithi016 ceph-mon[35851]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.9 MiB/s wr, 641 op/s 2024-03-26T08:55:28.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:28 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:55:28.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:28 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:55:28.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:28 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:55:28.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:28 smithi007 ceph-mon[32462]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.9 MiB/s wr, 641 op/s 2024-03-26T08:55:28.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:28 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:55:28.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:28 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:55:28.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:28 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:55:28.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:28 smithi044 ceph-mon[36461]: pgmap v751: 97 pgs: 97 active+clean; 1.6 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 6.9 MiB/s wr, 641 op/s 2024-03-26T08:55:28.821 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:28 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:55:28.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:28 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:55:28.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:28 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:55:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.646+0000 7fc70f059640 1 -- 172.21.15.7:0/3924592262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081021a0 msgr2=0x7fc708108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.646+0000 7fc70f059640 1 --2- 172.21.15.7:0/3924592262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081021a0 0x7fc708108a30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc700009ca0 tx=0x7fc70002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.646+0000 7fc70f059640 1 -- 172.21.15.7:0/3924592262 shutdown_connections 2024-03-26T08:55:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.646+0000 7fc70f059640 1 --2- 172.21.15.7:0/3924592262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081021a0 0x7fc708108a30 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.646+0000 7fc70f059640 1 --2- 172.21.15.7:0/3924592262 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7081017e0 0x7fc708101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.646+0000 7fc70f059640 1 --2- 172.21.15.7:0/3924592262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7081005f0 0x7fc708100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.646+0000 7fc70f059640 1 -- 172.21.15.7:0/3924592262 >> 172.21.15.7:0/3924592262 conn(0x7fc7080fbb60 msgr2=0x7fc7080fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.646+0000 7fc70f059640 1 -- 172.21.15.7:0/3924592262 shutdown_connections 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.646+0000 7fc70f059640 1 -- 172.21.15.7:0/3924592262 wait complete. 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70f059640 1 Processor -- start 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70f059640 1 -- start start 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70f059640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081005f0 0x7fc708196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70f059640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7081017e0 0x7fc708196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70f059640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7081021a0 0x7fc70819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70f059640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70810c270 con 0x7fc7081005f0 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70f059640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70810c0f0 con 0x7fc7081017e0 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70f059640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70810c3f0 con 0x7fc7081021a0 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70cdf5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081005f0 0x7fc708196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70cdf5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081005f0 0x7fc708196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44514/0 (socket says 172.21.15.7:44514) 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70cdf5640 1 -- 172.21.15.7:0/742660603 learned_addr learned my addr 172.21.15.7:0/742660603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc707fff640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7081017e0 0x7fc708196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7081021a0 0x7fc70819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70cdf5640 1 -- 172.21.15.7:0/742660603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7081021a0 msgr2=0x7fc70819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70cdf5640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7081021a0 0x7fc70819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70cdf5640 1 -- 172.21.15.7:0/742660603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7081017e0 msgr2=0x7fc708196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70cdf5640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7081017e0 0x7fc708196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70cdf5640 1 -- 172.21.15.7:0/742660603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70819e540 con 0x7fc7081005f0 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.647+0000 7fc70cdf5640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081005f0 0x7fc708196920 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800ead0 tx=0x7fc6f800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.648+0000 7fc705ffb640 1 -- 172.21.15.7:0/742660603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800bbd0 con 0x7fc7081005f0 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.648+0000 7fc70f059640 1 -- 172.21.15.7:0/742660603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc708069d00 con 0x7fc7081005f0 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.648+0000 7fc705ffb640 1 -- 172.21.15.7:0/742660603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f800bd70 con 0x7fc7081005f0 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.648+0000 7fc705ffb640 1 -- 172.21.15.7:0/742660603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8017410 con 0x7fc7081005f0 2024-03-26T08:55:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.648+0000 7fc70f059640 1 -- 172.21.15.7:0/742660603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70806a250 con 0x7fc7081005f0 2024-03-26T08:55:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.651+0000 7fc711c41740 1 Processor -- start 2024-03-26T08:55:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc711c41740 1 -- start start 2024-03-26T08:55:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc711c41740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc711c41740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 0x560e578d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc711c41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d7480 0x560e578d9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc711c41740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e57836b10 con 0x560e578d1920 2024-03-26T08:55:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc711c41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e577df900 con 0x560e578d7480 2024-03-26T08:55:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc711c41740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e57832bb0 con 0x560e578d2310 2024-03-26T08:55:29.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc70d5f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc707fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d7480 0x560e578d9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc705ffb640 1 -- 172.21.15.7:0/742660603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc6f801e020 con 0x7fc7081005f0 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc70d5f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44518/0 (socket says 172.21.15.7:44518) 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc70d5f6640 1 -- 172.21.15.7:0/344242698 learned_addr learned my addr 172.21.15.7:0/344242698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc707fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d7480 0x560e578d9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47396/0 (socket says 172.21.15.7:47396) 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc70cdf5640 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 0x560e578d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.652+0000 7fc70d5f6640 1 -- 172.21.15.7:0/344242698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 msgr2=0x560e578d6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 0x560e578d6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc70d5f6640 1 -- 172.21.15.7:0/344242698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d7480 msgr2=0x560e578d9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d7480 0x560e578d9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc70d5f6640 1 -- 172.21.15.7:0/344242698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e5754b210 con 0x560e578d1920 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc70cdf5640 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 0x560e578d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc707fff640 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d7480 0x560e578d9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578d1d40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc700009ca0 tx=0x7fc7000323d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf756427ca5b348 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc705ffb640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6f40a27f0 0x7fc6f40a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc6feffd640 1 -- 172.21.15.7:0/344242698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700043070 con 0x560e578d1920 2024-03-26T08:55:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc6feffd640 1 -- 172.21.15.7:0/344242698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc70002cca0 con 0x560e578d1920 2024-03-26T08:55:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc6feffd640 1 -- 172.21.15.7:0/344242698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700039040 con 0x560e578d1920 2024-03-26T08:55:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc70cdf5640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6f40a27f0 0x7fc6f40a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.653+0000 7fc705ffb640 1 -- 172.21.15.7:0/742660603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc6f80d00e0 con 0x7fc7081005f0 2024-03-26T08:55:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc70cdf5640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6f40a27f0 0x7fc6f40a4cb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8000c00 tx=0x7fc6f80035f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc711c41740 1 -- 172.21.15.7:0/344242698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 msgr2=0x560e578d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc711c41740 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578d1d40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc700009ca0 tx=0x7fc7000323d0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc711c41740 1 -- 172.21.15.7:0/344242698 shutdown_connections 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc711c41740 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d7480 0x560e578d9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc711c41740 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 0x560e578d6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc711c41740 1 --2- 172.21.15.7:0/344242698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578d1d40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc711c41740 1 -- 172.21.15.7:0/344242698 >> 172.21.15.7:0/344242698 conn(0x560e578cce90 msgr2=0x560e578cd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc711c41740 1 -- 172.21.15.7:0/344242698 shutdown_connections 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.654+0000 7fc711c41740 1 -- 172.21.15.7:0/344242698 wait complete. 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.655+0000 7fc711c41740 1 -- start start 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.655+0000 7fc711c41740 1 Processor -- start 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.655+0000 7fc711c41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d1920 0x560e578dbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.655+0000 7fc711c41740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 0x560e578dc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc711c41740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d1920 0x560e578dbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d1920 0x560e578dbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47412/0 (socket says 172.21.15.7:47412) 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 -- 172.21.15.7:0/1960026219 learned_addr learned my addr 172.21.15.7:0/1960026219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e578da3d0 con 0x560e578d7480 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e577f2d50 con 0x560e578d1920 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e578da550 con 0x560e578d2310 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc707fff640 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 0x560e578dc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 -- 172.21.15.7:0/1960026219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 msgr2=0x560e578dc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 0x560e578dc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 -- 172.21.15.7:0/1960026219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 msgr2=0x560e578e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578e3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 -- 172.21.15.7:0/1960026219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e57617620 con 0x560e578d1920 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc711c41740 10 client.0 _ll_register_callbacks cb 0x560e577f2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d1920 0x560e578dbbf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc700009bf0 tx=0x7fc7000039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a01c77a2d9817b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc6e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc70002cca0 con 0x560e578d1920 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc6e7fff640 1 -- 172.21.15.7:0/1960026219 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700056060 con 0x560e578d1920 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.656+0000 7fc6e7fff640 1 -- 172.21.15.7:0/1960026219 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc700051750 con 0x560e578d1920 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc6e7fff640 1 -- 172.21.15.7:0/1960026219 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc700043050 con 0x560e578d1920 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc711c41740 10 client.28679 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x560e578e6140 con 0x560e578d1920 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc6e7fff640 1 -- 172.21.15.7:0/1960026219 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc700051950 con 0x560e578d1920 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc711c41740 10 client.28679 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc711c41740 10 client.28679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc711c41740 20 client.28679 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc711c41740 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560e578e7490 0x560e578e9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc711c41740 4 client.28679 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e57844050 con 0x560e578e7490 2024-03-26T08:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc707fff640 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560e578e7490 0x560e578e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc6e7fff640 10 client.28679 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.657+0000 7fc707fff640 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560e578e7490 0x560e578e9880 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda4f6acbbedab52 server_cookie=4eec7e3cd4f70c30 in_seq=0 out_seq=0 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.658+0000 7fc6e7fff640 1 -- 172.21.15.7:0/1960026219 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560e57844050 con 0x560e578e7490 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.658+0000 7fc6e7fff640 10 client.28679 handle_command_reply: tid=0 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc711c41740 1 client.28679 shutdown 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d1920 msgr2=0x560e578dbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc711c41740 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d1920 0x560e578dbbf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc700009bf0 tx=0x7fc7000039b0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 shutdown_connections 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc711c41740 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560e578e7490 0x560e578e9880 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc711c41740 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc711c41740 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d2310 0x560e578dc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc711c41740 1 --2- 172.21.15.7:0/1960026219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d1920 0x560e578dbbf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc6e7fff640 0 client.28679 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.659+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 >> 172.21.15.7:0/1960026219 conn(0x560e578cce90 msgr2=0x560e578cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.660+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 shutdown_connections 2024-03-26T08:55:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.660+0000 7fc711c41740 1 -- 172.21.15.7:0/1960026219 wait complete. 2024-03-26T08:55:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.660+0000 7fc711c41740 20 client.28679 trim_cache size 0 max 16384 2024-03-26T08:55:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc711c41740 1 Processor -- start 2024-03-26T08:55:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc711c41740 1 -- start start 2024-03-26T08:55:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc711c41740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d1920 0x560e578cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc711c41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc711c41740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc711c41740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a8dd0 con 0x560e578d7480 2024-03-26T08:55:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc711c41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a68d0 con 0x560e578d2310 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc711c41740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a4950 con 0x560e578d1920 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc70d5f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc707fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc70d5f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44542/0 (socket says 172.21.15.7:44542) 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc70d5f6640 1 -- 172.21.15.7:0/2592006572 learned_addr learned my addr 172.21.15.7:0/2592006572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/2592006572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d1920 0x560e578cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc70d5f6640 1 -- 172.21.15.7:0/2592006572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d1920 msgr2=0x560e578cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/2592006572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d1920 0x560e578cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc70d5f6640 1 -- 172.21.15.7:0/2592006572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 msgr2=0x560e578e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/2592006572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc70d5f6640 1 -- 172.21.15.7:0/2592006572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e578d3120 con 0x560e578d7480 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.677+0000 7fc707fff640 1 --2- 172.21.15.7:0/2592006572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/2592006572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578cbcd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc700009bc0 tx=0x7fc70004a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddbb1218d28074c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc6fe7fc640 1 -- 172.21.15.7:0/2592006572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700039040 con 0x560e578d7480 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc6fe7fc640 1 -- 172.21.15.7:0/2592006572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc700051e50 con 0x560e578d7480 2024-03-26T08:55:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc6fe7fc640 1 -- 172.21.15.7:0/2592006572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700048770 con 0x560e578d7480 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc711c41740 1 -- 172.21.15.7:0/2592006572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 msgr2=0x560e578cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc711c41740 1 --2- 172.21.15.7:0/2592006572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578cbcd0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc700009bc0 tx=0x7fc70004a070 comp rx=0 tx=0).stop 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc711c41740 1 -- 172.21.15.7:0/2592006572 shutdown_connections 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc711c41740 1 --2- 172.21.15.7:0/2592006572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d7480 0x560e578cbcd0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc711c41740 1 --2- 172.21.15.7:0/2592006572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc711c41740 1 --2- 172.21.15.7:0/2592006572 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d1920 0x560e578cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.678+0000 7fc711c41740 1 -- 172.21.15.7:0/2592006572 >> 172.21.15.7:0/2592006572 conn(0x560e578cce90 msgr2=0x560e578cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 -- 172.21.15.7:0/2592006572 shutdown_connections 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 -- 172.21.15.7:0/2592006572 wait complete. 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 -- start start 2024-03-26T08:55:29.678 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 Processor -- start 2024-03-26T08:55:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d7480 0x560e578e1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a4f50 con 0x560e578d1920 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a6270 con 0x560e578d2310 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081a55b0 con 0x560e578d7480 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc711c41740 10 client.0 _ll_register_callbacks cb 0x7fc700032740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc70d5f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc70d5f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578e29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44554/0 (socket says 172.21.15.7:44554) 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc70d5f6640 1 -- 172.21.15.7:0/3556329660 learned_addr learned my addr 172.21.15.7:0/3556329660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc707fff640 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.679+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d7480 0x560e578e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.680+0000 7fc707fff640 1 -- 172.21.15.7:0/3556329660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d7480 msgr2=0x560e578e1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.680+0000 7fc707fff640 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d7480 0x560e578e1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.680+0000 7fc707fff640 1 -- 172.21.15.7:0/3556329660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 msgr2=0x560e578e29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.680+0000 7fc707fff640 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578e29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.680+0000 7fc707fff640 1 -- 172.21.15.7:0/3556329660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e578e17e0 con 0x560e578d2310 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.681+0000 7fc707fff640 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e2f30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x560e578e9ff0 tx=0x7fc6f0010e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a3390cab6007aae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.681+0000 7fc6e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.681+0000 7fc6e7fff640 1 -- 172.21.15.7:0/3556329660 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f000f930 con 0x560e578d2310 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.681+0000 7fc6e7fff640 1 -- 172.21.15.7:0/3556329660 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc6f000fad0 con 0x560e578d2310 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc711c41740 1 -- 172.21.15.7:0/3556329660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc700051e50 con 0x560e578d2310 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc6e7fff640 1 -- 172.21.15.7:0/3556329660 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f0035050 con 0x560e578d2310 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc711c41740 10 client.28685 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc711c41740 1 -- 172.21.15.7:0/3556329660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560e57943720 con 0x560e578d2310 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc6e7fff640 1 -- 172.21.15.7:0/3556329660 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f002b030 con 0x560e578d2310 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc6e7fff640 1 -- 172.21.15.7:0/3556329660 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc700032d10 con 0x560e578d2310 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc711c41740 10 client.28685 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc711c41740 10 client.28685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc711c41740 20 client.28685 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc711c41740 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560e578e7490 0x560e57946070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc711c41740 4 client.28685 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:55:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.682+0000 7fc711c41740 1 -- 172.21.15.7:0/3556329660 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fc700051e50 con 0x560e578e7490 2024-03-26T08:55:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.683+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560e578e7490 0x560e57946070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.683+0000 7fc6e7fff640 10 client.28685 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.683+0000 7fc70d5f6640 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560e578e7490 0x560e57946070 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d943af8c71b8c2 server_cookie=a5f4f6c1d3170fda in_seq=0 out_seq=0 2024-03-26T08:55:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc6e7fff640 1 -- 172.21.15.7:0/3556329660 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fc700051e50 con 0x560e578e7490 2024-03-26T08:55:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc6e7fff640 10 client.28685 handle_command_reply: tid=0 2024-03-26T08:55:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc711c41740 1 client.28685 shutdown 2024-03-26T08:55:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc711c41740 1 -- 172.21.15.7:0/3556329660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 msgr2=0x560e578e2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc711c41740 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e2f30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x560e578e9ff0 tx=0x7fc6f0010e60 comp rx=0 tx=0).stop 2024-03-26T08:55:29.707 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc711c41740 1 -- 172.21.15.7:0/3556329660 shutdown_connections 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc711c41740 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x560e578e7490 0x560e57946070 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc711c41740 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560e578d7480 0x560e578e1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc6e7fff640 0 client.28685 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc711c41740 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e578d2310 0x560e578e2f30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc711c41740 1 --2- 172.21.15.7:0/3556329660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e578d1920 0x560e578e29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.709+0000 7fc711c41740 1 -- 172.21.15.7:0/3556329660 >> 172.21.15.7:0/3556329660 conn(0x560e578cce90 msgr2=0x560e578cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc711c41740 1 -- 172.21.15.7:0/3556329660 shutdown_connections 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc711c41740 1 -- 172.21.15.7:0/3556329660 wait complete. 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc711c41740 20 client.28685 trim_cache size 0 max 16384 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:55:29.708 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:55:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:55:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:55:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 -- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6f40a27f0 msgr2=0x7fc6f40a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6f40a27f0 0x7fc6f40a4cb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8000c00 tx=0x7fc6f80035f0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.709 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 -- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081005f0 msgr2=0x7fc708196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081005f0 0x7fc708196920 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc6f800ead0 tx=0x7fc6f800ef90 comp rx=0 tx=0).stop 2024-03-26T08:55:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 -- 172.21.15.7:0/742660603 shutdown_connections 2024-03-26T08:55:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc6f40a27f0 0x7fc6f40a4cb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc7081021a0 0x7fc70819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7081017e0 0x7fc708196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 --2- 172.21.15.7:0/742660603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7081005f0 0x7fc708196920 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 -- 172.21.15.7:0/742660603 >> 172.21.15.7:0/742660603 conn(0x7fc7080fbb60 msgr2=0x7fc708104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 -- 172.21.15.7:0/742660603 shutdown_connections 2024-03-26T08:55:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:29.710+0000 7fc70f059640 1 -- 172.21.15.7:0/742660603 wait complete. 2024-03-26T08:55:29.798 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:29 smithi007 ceph-mon[32462]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 756 op/s 2024-03-26T08:55:29.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:29 smithi044 ceph-mon[36461]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 756 op/s 2024-03-26T08:55:29.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:29 smithi016 ceph-mon[35851]: pgmap v752: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 756 op/s 2024-03-26T08:55:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 780.0 s, 1556.0 tps, lat 64.019 ms stddev 72.850 2024-03-26T08:55:31.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:31 smithi016 ceph-mon[35851]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.7 MiB/s wr, 755 op/s 2024-03-26T08:55:32.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:31 smithi007 ceph-mon[32462]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.7 MiB/s wr, 755 op/s 2024-03-26T08:55:32.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:31 smithi044 ceph-mon[36461]: pgmap v753: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.7 MiB/s wr, 755 op/s 2024-03-26T08:55:33.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:33 smithi016 ceph-mon[35851]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.2 MiB/s wr, 803 op/s 2024-03-26T08:55:34.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:33 smithi007 ceph-mon[32462]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.2 MiB/s wr, 803 op/s 2024-03-26T08:55:34.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:33 smithi044 ceph-mon[36461]: pgmap v754: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.2 MiB/s wr, 803 op/s 2024-03-26T08:55:35.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:35 smithi016 ceph-mon[35851]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-03-26T08:55:36.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:35 smithi007 ceph-mon[32462]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-03-26T08:55:36.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:35 smithi044 ceph-mon[36461]: pgmap v755: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.5 MiB/s wr, 696 op/s 2024-03-26T08:55:36.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 785.0 s, 1932.8 tps, lat 51.082 ms stddev 76.900 2024-03-26T08:55:36.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:36 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:55:36.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:36 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:55:37.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:36 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:55:37.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:36 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:55:37.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:36 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:55:37.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:36 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:55:37.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:37 smithi016 ceph-mon[35851]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.4 MiB/s wr, 691 op/s 2024-03-26T08:55:38.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:37 smithi007 ceph-mon[32462]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.4 MiB/s wr, 691 op/s 2024-03-26T08:55:38.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:37 smithi044 ceph-mon[36461]: pgmap v756: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.4 MiB/s wr, 691 op/s 2024-03-26T08:55:39.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:39 smithi016 ceph-mon[35851]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.1 MiB/s wr, 758 op/s 2024-03-26T08:55:40.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:39 smithi007 ceph-mon[32462]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.1 MiB/s wr, 758 op/s 2024-03-26T08:55:40.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:39 smithi044 ceph-mon[36461]: pgmap v757: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.1 MiB/s wr, 758 op/s 2024-03-26T08:55:41.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 790.0 s, 1845.4 tps, lat 54.863 ms stddev 71.465 2024-03-26T08:55:41.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:41 smithi016 ceph-mon[35851]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.9 MiB/s wr, 714 op/s 2024-03-26T08:55:42.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:41 smithi007 ceph-mon[32462]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.9 MiB/s wr, 714 op/s 2024-03-26T08:55:42.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:41 smithi044 ceph-mon[36461]: pgmap v758: 97 pgs: 97 active+clean; 1.6 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.9 MiB/s wr, 714 op/s 2024-03-26T08:55:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.832+0000 7efe52a27640 1 -- 172.21.15.7:0/900008391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 msgr2=0x7efe4c109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.832+0000 7efe52a27640 1 --2- 172.21.15.7:0/900008391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 0x7efe4c109b60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7efe44009ca0 tx=0x7efe4402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.832+0000 7efe52a27640 1 -- 172.21.15.7:0/900008391 shutdown_connections 2024-03-26T08:55:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.832+0000 7efe52a27640 1 --2- 172.21.15.7:0/900008391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 0x7efe4c109b60 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.832+0000 7efe52a27640 1 --2- 172.21.15.7:0/900008391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c102910 0x7efe4c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.832+0000 7efe52a27640 1 --2- 172.21.15.7:0/900008391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe4c101720 0x7efe4c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.832+0000 7efe52a27640 1 -- 172.21.15.7:0/900008391 >> 172.21.15.7:0/900008391 conn(0x7efe4c0fcc90 msgr2=0x7efe4c0ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.833+0000 7efe52a27640 1 -- 172.21.15.7:0/900008391 shutdown_connections 2024-03-26T08:55:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.834+0000 7efe52a27640 1 -- 172.21.15.7:0/900008391 wait complete. 2024-03-26T08:55:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.834+0000 7efe52a27640 1 Processor -- start 2024-03-26T08:55:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.834+0000 7efe52a27640 1 -- start start 2024-03-26T08:55:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe52a27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c101720 0x7efe4c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe52a27640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe4c102910 0x7efe4c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe52a27640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 0x7efe4c1a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe52a27640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c10d3a0 con 0x7efe4c1032d0 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe52a27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c10d220 con 0x7efe4c101720 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe52a27640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c10d520 con 0x7efe4c102910 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe50fc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 0x7efe4c1a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c101720 0x7efe4c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe50fc4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 0x7efe4c1a5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43802/0 (socket says 172.21.15.7:43802) 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe50fc4640 1 -- 172.21.15.7:0/3706892880 learned_addr learned my addr 172.21.15.7:0/3706892880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe4b7fe640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe4c102910 0x7efe4c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe4bfff640 1 -- 172.21.15.7:0/3706892880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe4c102910 msgr2=0x7efe4c19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe4bfff640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe4c102910 0x7efe4c19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe4bfff640 1 -- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 msgr2=0x7efe4c1a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe4bfff640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 0x7efe4c1a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe4bfff640 1 -- 172.21.15.7:0/3706892880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c1a5bf0 con 0x7efe4c101720 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe50fc4640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 0x7efe4c1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.835+0000 7efe4bfff640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c101720 0x7efe4c19dfd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00e9c0 tx=0x7efe3c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.836+0000 7efe497fa640 1 -- 172.21.15.7:0/3706892880 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00ccc0 con 0x7efe4c101720 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.836+0000 7efe52a27640 1 -- 172.21.15.7:0/3706892880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c1a5ee0 con 0x7efe4c101720 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.836+0000 7efe497fa640 1 -- 172.21.15.7:0/3706892880 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe3c0035f0 con 0x7efe4c101720 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.836+0000 7efe52a27640 1 -- 172.21.15.7:0/3706892880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c1a6340 con 0x7efe4c101720 2024-03-26T08:55:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.836+0000 7efe497fa640 1 -- 172.21.15.7:0/3706892880 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00ce60 con 0x7efe4c101720 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.838+0000 7efe497fa640 1 -- 172.21.15.7:0/3706892880 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7efe3c010690 con 0x7efe4c101720 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe55614740 1 Processor -- start 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe55614740 1 -- start start 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe55614740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df520d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe55614740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df525e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe55614740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 0x5619df528910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe55614740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619df485b10 con 0x5619df526480 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe55614740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619df42e900 con 0x5619df520920 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe55614740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619df481bb0 con 0x5619df521310 2024-03-26T08:55:42.841 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe50fc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df520d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 0x5619df528910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df525e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe50fc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df520d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50936/0 (socket says 172.21.15.7:50936) 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe50fc4640 1 -- 172.21.15.7:0/280057248 learned_addr learned my addr 172.21.15.7:0/280057248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe497fa640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe380a28f0 0x7efe380a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe497fa640 1 -- 172.21.15.7:0/3706892880 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efe3c0ceec0 con 0x7efe4c101720 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe50fc4640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe380a28f0 0x7efe380a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe4bfff640 1 -- 172.21.15.7:0/280057248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 msgr2=0x5619df520d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe4bfff640 1 --2- 172.21.15.7:0/280057248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df520d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe4bfff640 1 -- 172.21.15.7:0/280057248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 msgr2=0x5619df528910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe4bfff640 1 --2- 172.21.15.7:0/280057248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 0x5619df528910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe4bfff640 1 -- 172.21.15.7:0/280057248 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619df19a210 con 0x5619df521310 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.839+0000 7efe4b7fe640 1 --2- 172.21.15.7:0/280057248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 0x5619df528910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.840+0000 7efe4bfff640 1 --2- 172.21.15.7:0/280057248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df525e10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0cc950 tx=0x7efe3c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec789782107de90 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.840+0000 7efe50fc4640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe380a28f0 0x7efe380a4db0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x5619df521b70 tx=0x7efe4402c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.840+0000 7efe427fc640 1 -- 172.21.15.7:0/280057248 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c09b820 con 0x5619df521310 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.840+0000 7efe427fc640 1 -- 172.21.15.7:0/280057248 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe3c09f970 con 0x5619df521310 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.840+0000 7efe427fc640 1 -- 172.21.15.7:0/280057248 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c098000 con 0x5619df521310 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 -- 172.21.15.7:0/280057248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 msgr2=0x5619df525e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 --2- 172.21.15.7:0/280057248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df525e10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0cc950 tx=0x7efe3c013000 comp rx=0 tx=0).stop 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 -- 172.21.15.7:0/280057248 shutdown_connections 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 --2- 172.21.15.7:0/280057248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 0x5619df528910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 --2- 172.21.15.7:0/280057248 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df525e10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 --2- 172.21.15.7:0/280057248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df520d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 -- 172.21.15.7:0/280057248 >> 172.21.15.7:0/280057248 conn(0x5619df51be90 msgr2=0x5619df51c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 -- 172.21.15.7:0/280057248 shutdown_connections 2024-03-26T08:55:42.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 -- 172.21.15.7:0/280057248 wait complete. 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 -- start start 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 Processor -- start 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.841+0000 7efe55614740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 0x5619df52b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe55614740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 0x5619df532190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe55614740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619df529370 con 0x5619df520920 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe55614740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619df441d50 con 0x5619df521310 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe55614740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619df5294f0 con 0x5619df526480 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43816/0 (socket says 172.21.15.7:43816) 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4bfff640 1 -- 172.21.15.7:0/1281797954 learned_addr learned my addr 172.21.15.7:0/1281797954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4b7fe640 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 0x5619df52b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe50fc4640 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 0x5619df532190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe55614740 10 client.0 _ll_register_callbacks cb 0x5619df441720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4bfff640 1 -- 172.21.15.7:0/1281797954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 msgr2=0x5619df532190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4bfff640 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 0x5619df532190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4bfff640 1 -- 172.21.15.7:0/1281797954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 msgr2=0x5619df52b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4bfff640 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 0x5619df52b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4bfff640 1 -- 172.21.15.7:0/1281797954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619df266620 con 0x5619df520920 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe4bfff640 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52ac10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efe3c09b7b0 tx=0x7efe3c097ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bafb9076c7bb750 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe2b7fe640 1 -- 172.21.15.7:0/1281797954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0b3020 con 0x5619df520920 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe2b7fe640 1 -- 172.21.15.7:0/1281797954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efe3c003a60 con 0x5619df520920 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe55614740 1 -- 172.21.15.7:0/1281797954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe3c09f970 con 0x5619df520920 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe2b7fe640 1 -- 172.21.15.7:0/1281797954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe3c0a3fc0 con 0x5619df520920 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe55614740 10 client.18918 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.842+0000 7efe55614740 1 -- 172.21.15.7:0/1281797954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5619df5354c0 con 0x5619df520920 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe2b7fe640 1 -- 172.21.15.7:0/1281797954 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe3c09f970 con 0x5619df520920 2024-03-26T08:55:42.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe2b7fe640 1 -- 172.21.15.7:0/1281797954 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7efe3c09ae00 con 0x5619df520920 2024-03-26T08:55:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe55614740 10 client.18918 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe55614740 10 client.18918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe55614740 20 client.18918 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe55614740 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619df536810 0x5619df538c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe55614740 4 client.18918 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe55614740 1 -- 172.21.15.7:0/1281797954 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619df493050 con 0x5619df536810 2024-03-26T08:55:42.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe4b7fe640 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619df536810 0x5619df538c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe4b7fe640 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619df536810 0x5619df538c00 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bba16795d4b5bc6 server_cookie=f7063cca98ed018e in_seq=0 out_seq=0 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.843+0000 7efe2b7fe640 10 client.18918 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.844+0000 7efe2b7fe640 1 -- 172.21.15.7:0/1281797954 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5619df493050 con 0x5619df536810 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.844+0000 7efe2b7fe640 10 client.18918 handle_command_reply: tid=0 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.844+0000 7efe55614740 1 client.18918 shutdown 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 -- 172.21.15.7:0/1281797954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 msgr2=0x5619df52ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52ac10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7efe3c09b7b0 tx=0x7efe3c097ba0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 -- 172.21.15.7:0/1281797954 shutdown_connections 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619df536810 0x5619df538c00 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 0x5619df532190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 0x5619df52b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 --2- 172.21.15.7:0/1281797954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52ac10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 -- 172.21.15.7:0/1281797954 >> 172.21.15.7:0/1281797954 conn(0x5619df51be90 msgr2=0x5619df51b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe2b7fe640 0 client.18918 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 -- 172.21.15.7:0/1281797954 shutdown_connections 2024-03-26T08:55:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 1 -- 172.21.15.7:0/1281797954 wait complete. 2024-03-26T08:55:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.845+0000 7efe55614740 20 client.18918 trim_cache size 0 max 16384 2024-03-26T08:55:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.862+0000 7efe55614740 1 Processor -- start 2024-03-26T08:55:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.862+0000 7efe55614740 1 -- start start 2024-03-26T08:55:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe55614740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df538820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe55614740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df538d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe55614740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 0x5619df532120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe55614740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c010870 con 0x5619df526480 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe55614740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00e370 con 0x5619df520920 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe55614740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00c3f0 con 0x5619df521310 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df538820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df538d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df538820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50948/0 (socket says 172.21.15.7:50948) 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4bfff640 1 -- 172.21.15.7:0/3671006980 learned_addr learned my addr 172.21.15.7:0/3671006980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe50fc4640 1 --2- 172.21.15.7:0/3671006980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 0x5619df532120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4bfff640 1 -- 172.21.15.7:0/3671006980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 msgr2=0x5619df538d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4bfff640 1 --2- 172.21.15.7:0/3671006980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df538d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4bfff640 1 -- 172.21.15.7:0/3671006980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 msgr2=0x5619df532120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4bfff640 1 --2- 172.21.15.7:0/3671006980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 0x5619df532120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4bfff640 1 -- 172.21.15.7:0/3671006980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619df525910 con 0x5619df520920 2024-03-26T08:55:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.863+0000 7efe4b7fe640 1 --2- 172.21.15.7:0/3671006980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df538d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.864+0000 7efe4bfff640 1 --2- 172.21.15.7:0/3671006980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df538820 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0a3a90 tx=0x7efe3c09b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7926ffd5fa323a60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.864+0000 7efe41ffb640 1 -- 172.21.15.7:0/3671006980 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0b3070 con 0x5619df520920 2024-03-26T08:55:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.864+0000 7efe41ffb640 1 -- 172.21.15.7:0/3671006980 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe3c09f970 con 0x5619df520920 2024-03-26T08:55:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.864+0000 7efe41ffb640 1 -- 172.21.15.7:0/3671006980 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c09a6e0 con 0x5619df520920 2024-03-26T08:55:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 -- 172.21.15.7:0/3671006980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 msgr2=0x5619df538820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 --2- 172.21.15.7:0/3671006980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df538820 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0a3a90 tx=0x7efe3c09b630 comp rx=0 tx=0).stop 2024-03-26T08:55:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 -- 172.21.15.7:0/3671006980 shutdown_connections 2024-03-26T08:55:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 --2- 172.21.15.7:0/3671006980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df526480 0x5619df532120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 --2- 172.21.15.7:0/3671006980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df521310 0x5619df538d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 --2- 172.21.15.7:0/3671006980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df520920 0x5619df538820 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 -- 172.21.15.7:0/3671006980 >> 172.21.15.7:0/3671006980 conn(0x5619df51be90 msgr2=0x5619df516eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 -- 172.21.15.7:0/3671006980 shutdown_connections 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 -- 172.21.15.7:0/3671006980 wait complete. 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 -- start start 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 Processor -- start 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 0x5619df52fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 0x5619df530b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00c9f0 con 0x5619df520920 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00dd10 con 0x5619df521310 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00d050 con 0x5619df526480 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe55614740 10 client.0 _ll_register_callbacks cb 0x7efe3c097400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43834/0 (socket says 172.21.15.7:43834) 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.865+0000 7efe4bfff640 1 -- 172.21.15.7:0/3558570753 learned_addr learned my addr 172.21.15.7:0/3558570753 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe4b7fe640 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 0x5619df52fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe50fc4640 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 0x5619df530b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe4bfff640 1 -- 172.21.15.7:0/3558570753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 msgr2=0x5619df530b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe4bfff640 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 0x5619df530b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe4bfff640 1 -- 172.21.15.7:0/3558570753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 msgr2=0x5619df52fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe4bfff640 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 0x5619df52fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe4bfff640 1 -- 172.21.15.7:0/3558570753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619df531180 con 0x5619df520920 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe4bfff640 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52f640 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0981f0 tx=0x7efe3c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f042fe55be0cc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe2b7fe640 1 -- 172.21.15.7:0/3558570753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0b3030 con 0x5619df520920 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe2b7fe640 1 -- 172.21.15.7:0/3558570753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efe3c09a6e0 con 0x5619df520920 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe55614740 1 -- 172.21.15.7:0/3558570753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe3c09f970 con 0x5619df520920 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe2b7fe640 1 -- 172.21.15.7:0/3558570753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe3c097ac0 con 0x5619df520920 2024-03-26T08:55:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe55614740 10 client.18924 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe55614740 1 -- 172.21.15.7:0/3558570753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5619df532490 con 0x5619df520920 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.866+0000 7efe2b7fe640 1 -- 172.21.15.7:0/3558570753 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe3c09f970 con 0x5619df520920 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe2b7fe640 1 -- 172.21.15.7:0/3558570753 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7efe3c0bb4a0 con 0x5619df520920 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe55614740 10 client.18924 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe55614740 10 client.18924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe55614740 20 client.18924 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe55614740 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619df592470 0x5619df594860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe55614740 4 client.18924 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe55614740 1 -- 172.21.15.7:0/3558570753 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5619df48d3a0 con 0x5619df592470 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe4b7fe640 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619df592470 0x5619df594860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe2b7fe640 10 client.18924 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.867+0000 7efe4b7fe640 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619df592470 0x5619df594860 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76cffa12451782f7 server_cookie=c87c726522861e55 in_seq=0 out_seq=0 2024-03-26T08:55:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.899+0000 7efe2b7fe640 1 -- 172.21.15.7:0/3558570753 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5619df48d3a0 con 0x5619df592470 2024-03-26T08:55:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.899+0000 7efe2b7fe640 10 client.18924 handle_command_reply: tid=0 2024-03-26T08:55:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.900+0000 7efe55614740 1 client.18924 shutdown 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.900+0000 7efe55614740 1 -- 172.21.15.7:0/3558570753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 msgr2=0x5619df52f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.900+0000 7efe55614740 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52f640 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0981f0 tx=0x7efe3c004440 comp rx=0 tx=0).stop 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe55614740 1 -- 172.21.15.7:0/3558570753 shutdown_connections 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe55614740 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5619df592470 0x5619df594860 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe55614740 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5619df526480 0x5619df530b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe55614740 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5619df521310 0x5619df52fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe55614740 1 --2- 172.21.15.7:0/3558570753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5619df520920 0x5619df52f640 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe55614740 1 -- 172.21.15.7:0/3558570753 >> 172.21.15.7:0/3558570753 conn(0x5619df51be90 msgr2=0x5619df522690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe2b7fe640 0 client.18924 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe55614740 1 -- 172.21.15.7:0/3558570753 shutdown_connections 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe55614740 1 -- 172.21.15.7:0/3558570753 wait complete. 2024-03-26T08:55:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.901+0000 7efe55614740 20 client.18924 trim_cache size 0 max 16384 2024-03-26T08:55:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:55:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:55:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:55:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:55:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 -- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe380a28f0 msgr2=0x7efe380a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe380a28f0 0x7efe380a4db0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x5619df521b70 tx=0x7efe4402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 -- 172.21.15.7:0/3706892880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c101720 msgr2=0x7efe4c19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c101720 0x7efe4c19dfd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00e9c0 tx=0x7efe3c00ee80 comp rx=0 tx=0).stop 2024-03-26T08:55:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 -- 172.21.15.7:0/3706892880 shutdown_connections 2024-03-26T08:55:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efe380a28f0 0x7efe380a4db0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe4c1032d0 0x7efe4c1a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe4c102910 0x7efe4c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 --2- 172.21.15.7:0/3706892880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe4c101720 0x7efe4c19dfd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 -- 172.21.15.7:0/3706892880 >> 172.21.15.7:0/3706892880 conn(0x7efe4c0fcc90 msgr2=0x7efe4c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 -- 172.21.15.7:0/3706892880 shutdown_connections 2024-03-26T08:55:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:42.902+0000 7efe52a27640 1 -- 172.21.15.7:0/3706892880 wait complete. 2024-03-26T08:55:43.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:43 smithi016 ceph-mon[35851]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.5 MiB/s wr, 722 op/s 2024-03-26T08:55:44.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:43 smithi007 ceph-mon[32462]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.5 MiB/s wr, 722 op/s 2024-03-26T08:55:44.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:43 smithi044 ceph-mon[36461]: pgmap v759: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.5 MiB/s wr, 722 op/s 2024-03-26T08:55:45.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:45 smithi016 ceph-mon[35851]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 575 op/s 2024-03-26T08:55:46.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:45 smithi007 ceph-mon[32462]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 575 op/s 2024-03-26T08:55:46.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:45 smithi044 ceph-mon[36461]: pgmap v760: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 MiB/s wr, 575 op/s 2024-03-26T08:55:46.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 795.0 s, 1433.8 tps, lat 69.794 ms stddev 74.663 2024-03-26T08:55:47.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:47 smithi016 ceph-mon[35851]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2024-03-26T08:55:48.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:47 smithi007 ceph-mon[32462]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2024-03-26T08:55:48.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:47 smithi044 ceph-mon[36461]: pgmap v761: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2024-03-26T08:55:49.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:49 smithi016 ceph-mon[35851]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 675 op/s 2024-03-26T08:55:50.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:49 smithi007 ceph-mon[32462]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 675 op/s 2024-03-26T08:55:50.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:49 smithi044 ceph-mon[36461]: pgmap v762: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.6 MiB/s wr, 675 op/s 2024-03-26T08:55:50.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:50 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:55:51.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:50 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:55:51.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:50 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:55:51.121 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 800.0 s, 1315.6 tps, lat 75.177 ms stddev 83.283 2024-03-26T08:55:51.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:51 smithi016 ceph-mon[35851]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 640 op/s 2024-03-26T08:55:51.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:55:51.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:55:51.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:51 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:55:52.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:51 smithi007 ceph-mon[32462]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 640 op/s 2024-03-26T08:55:52.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:55:52.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:55:52.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:51 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:55:52.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:51 smithi044 ceph-mon[36461]: pgmap v763: 97 pgs: 97 active+clean; 1.6 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 MiB/s wr, 640 op/s 2024-03-26T08:55:52.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:55:52.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:55:52.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:51 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:55:53.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:53 smithi016 ceph-mon[35851]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.5 MiB/s wr, 636 op/s 2024-03-26T08:55:54.049 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:53 smithi007 ceph-mon[32462]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.5 MiB/s wr, 636 op/s 2024-03-26T08:55:54.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:53 smithi044 ceph-mon[36461]: pgmap v764: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.5 MiB/s wr, 636 op/s 2024-03-26T08:55:55.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:55 smithi016 ceph-mon[35851]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 581 op/s 2024-03-26T08:55:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.036+0000 7f461f59e640 1 -- 172.21.15.7:0/2711511610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201030c0 msgr2=0x7f4620109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.036+0000 7f461f59e640 1 --2- 172.21.15.7:0/2711511610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201030c0 0x7f4620109950 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4618009be0 tx=0x7f461802bef0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.036+0000 7f461f59e640 1 -- 172.21.15.7:0/2711511610 shutdown_connections 2024-03-26T08:55:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.036+0000 7f461f59e640 1 --2- 172.21.15.7:0/2711511610 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201030c0 0x7f4620109950 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.036+0000 7f461f59e640 1 --2- 172.21.15.7:0/2711511610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102700 0x7f4620102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.036+0000 7f461f59e640 1 --2- 172.21.15.7:0/2711511610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620101510 0x7f4620101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.036+0000 7f461f59e640 1 -- 172.21.15.7:0/2711511610 >> 172.21.15.7:0/2711511610 conn(0x7f46200fca80 msgr2=0x7f46200feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.036+0000 7f461f59e640 1 -- 172.21.15.7:0/2711511610 shutdown_connections 2024-03-26T08:55:56.035 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.036+0000 7f461f59e640 1 -- 172.21.15.7:0/2711511610 wait complete. 2024-03-26T08:55:56.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.037+0000 7f461f59e640 1 Processor -- start 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.037+0000 7f461f59e640 1 -- start start 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461f59e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620101510 0x7f46201955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102700 0x7f4620195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461f59e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201030c0 0x7f462019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620101510 0x7f46201955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461e59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620101510 0x7f46201955b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53786/0 (socket says 172.21.15.7:53786) 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461e59c640 1 -- 172.21.15.7:0/3002106415 learned_addr learned my addr 172.21.15.7:0/3002106415 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462010afb0 con 0x7f4620101510 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462010ae30 con 0x7f4620102700 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462010b130 con 0x7f46201030c0 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.038+0000 7f461ed9d640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201030c0 0x7f462019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.039+0000 7f4617fff640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102700 0x7f4620195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.039+0000 7f461ed9d640 1 -- 172.21.15.7:0/3002106415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102700 msgr2=0x7f4620195af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.039+0000 7f461ed9d640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102700 0x7f4620195af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.039+0000 7f461ed9d640 1 -- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620101510 msgr2=0x7f46201955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.039+0000 7f461ed9d640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620101510 0x7f46201955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.039+0000 7f461ed9d640 1 -- 172.21.15.7:0/3002106415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462019d1d0 con 0x7f46201030c0 2024-03-26T08:55:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.039+0000 7f461e59c640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620101510 0x7f46201955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:55:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.039+0000 7f4617fff640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102700 0x7f4620195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.039+0000 7f461ed9d640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201030c0 0x7f462019cb60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4618009b30 tx=0x7f46180337b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.040+0000 7f46177fe640 1 -- 172.21.15.7:0/3002106415 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618043020 con 0x7f46201030c0 2024-03-26T08:55:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.040+0000 7f46177fe640 1 -- 172.21.15.7:0/3002106415 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46180338a0 con 0x7f46201030c0 2024-03-26T08:55:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.040+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462019d460 con 0x7f46201030c0 2024-03-26T08:55:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.040+0000 7f46177fe640 1 -- 172.21.15.7:0/3002106415 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618034690 con 0x7f46201030c0 2024-03-26T08:55:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.040+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462019da00 con 0x7f46201030c0 2024-03-26T08:55:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.044+0000 7f4627797740 1 Processor -- start 2024-03-26T08:55:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.044+0000 7f46177fe640 1 -- 172.21.15.7:0/3002106415 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4618046020 con 0x7f46201030c0 2024-03-26T08:55:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.044+0000 7f4627797740 1 -- start start 2024-03-26T08:55:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.045+0000 7f4627797740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169330 0x55afbf169750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.045+0000 7f4627797740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 0x55afbf16e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.045+0000 7f4627797740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf16ee90 0x55afbf171320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.045+0000 7f4627797740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbf0c5b10 con 0x55afbf16ee90 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.045+0000 7f4627797740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbf06e900 con 0x55afbf169330 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.045+0000 7f4627797740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbf0c1bb0 con 0x55afbf169d20 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.045+0000 7f46177fe640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0038a0 0x7f460c005d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f4617fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169330 0x55afbf169750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f461e59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 0x55afbf16e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f4617fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169330 0x55afbf169750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36058/0 (socket says 172.21.15.7:36058) 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f4617fff640 1 -- 172.21.15.7:0/2388978977 learned_addr learned my addr 172.21.15.7:0/2388978977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f4617fff640 1 --2- 172.21.15.7:0/2388978977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf16ee90 0x55afbf171320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f46177fe640 1 -- 172.21.15.7:0/3002106415 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f46180f0fd0 con 0x7f46201030c0 2024-03-26T08:55:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f461ed9d640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0038a0 0x7f460c005d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f461e59c640 1 -- 172.21.15.7:0/2388978977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169330 msgr2=0x55afbf169750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f461e59c640 1 --2- 172.21.15.7:0/2388978977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169330 0x55afbf169750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f461e59c640 1 -- 172.21.15.7:0/2388978977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf16ee90 msgr2=0x55afbf171320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f461e59c640 1 --2- 172.21.15.7:0/2388978977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf16ee90 0x55afbf171320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f461e59c640 1 -- 172.21.15.7:0/2388978977 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afbedda210 con 0x55afbf169d20 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f4617fff640 1 --2- 172.21.15.7:0/2388978977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169330 0x55afbf169750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.046+0000 7f461e59c640 1 --2- 172.21.15.7:0/2388978977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 0x55afbf16e820 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4610009c10 tx=0x7f4610010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97eba381fecb30f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.047+0000 7f461ed9d640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0038a0 0x7f460c005d60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46180b9e70 tx=0x7f4618034f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.047+0000 7f4614ff9640 1 -- 172.21.15.7:0/2388978977 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461001b480 con 0x55afbf169d20 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.047+0000 7f4614ff9640 1 -- 172.21.15.7:0/2388978977 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4610012930 con 0x55afbf169d20 2024-03-26T08:55:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.047+0000 7f4614ff9640 1 -- 172.21.15.7:0/2388978977 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100117d0 con 0x55afbf169d20 2024-03-26T08:55:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.048+0000 7f4627797740 1 -- 172.21.15.7:0/2388978977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 msgr2=0x55afbf16e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:56.048 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.048+0000 7f4627797740 1 --2- 172.21.15.7:0/2388978977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 0x55afbf16e820 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4610009c10 tx=0x7f4610010740 comp rx=0 tx=0).stop 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.048+0000 7f4627797740 1 -- 172.21.15.7:0/2388978977 shutdown_connections 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.048+0000 7f4627797740 1 --2- 172.21.15.7:0/2388978977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf16ee90 0x55afbf171320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.048+0000 7f4627797740 1 --2- 172.21.15.7:0/2388978977 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 0x55afbf16e820 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.048+0000 7f4627797740 1 --2- 172.21.15.7:0/2388978977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169330 0x55afbf169750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.048+0000 7f4627797740 1 -- 172.21.15.7:0/2388978977 >> 172.21.15.7:0/2388978977 conn(0x55afbf15c090 msgr2=0x55afbf15c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.048+0000 7f4627797740 1 -- 172.21.15.7:0/2388978977 shutdown_connections 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.048+0000 7f4627797740 1 -- 172.21.15.7:0/2388978977 wait complete. 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.049+0000 7f4627797740 1 -- start start 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.049+0000 7f4627797740 1 Processor -- start 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.049+0000 7f4627797740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf1624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.049+0000 7f4627797740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf162a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4627797740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 0x55afbf167a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4627797740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbf160c20 con 0x55afbf169330 2024-03-26T08:55:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4627797740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbf081d50 con 0x55afbf169d20 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4617fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf1624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4627797740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afbf160da0 con 0x55afbf16ee90 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4617fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf1624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53812/0 (socket says 172.21.15.7:53812) 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4617fff640 1 -- 172.21.15.7:0/690603545 learned_addr learned my addr 172.21.15.7:0/690603545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4617fff640 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 0x55afbf167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f461e59c640 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4627797740 10 client.0 _ll_register_callbacks cb 0x55afbf081720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4617fff640 1 -- 172.21.15.7:0/690603545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 msgr2=0x55afbf167a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4617fff640 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 0x55afbf167a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4617fff640 1 -- 172.21.15.7:0/690603545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 msgr2=0x55afbf162a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4617fff640 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf162a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.050+0000 7f4617fff640 1 -- 172.21.15.7:0/690603545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afbeea6620 con 0x55afbf169330 2024-03-26T08:55:56.051 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:55 smithi007 ceph-mon[32462]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 581 op/s 2024-03-26T08:55:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.051+0000 7f4617fff640 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf1624d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f46080116a0 tx=0x7f4608005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af43c98a59ce110 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.051+0000 7f4605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.051+0000 7f4605ffb640 1 -- 172.21.15.7:0/690603545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608005020 con 0x55afbf169330 2024-03-26T08:55:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.051+0000 7f4605ffb640 1 -- 172.21.15.7:0/690603545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f46080051c0 con 0x55afbf169330 2024-03-26T08:55:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.052+0000 7f4627797740 1 -- 172.21.15.7:0/690603545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4610012930 con 0x55afbf169330 2024-03-26T08:55:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.052+0000 7f4605ffb640 1 -- 172.21.15.7:0/690603545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46080053c0 con 0x55afbf169330 2024-03-26T08:55:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.052+0000 7f4627797740 10 client.18933 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.052+0000 7f4627797740 1 -- 172.21.15.7:0/690603545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55afbf186820 con 0x55afbf169330 2024-03-26T08:55:56.053 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.053+0000 7f4605ffb640 1 -- 172.21.15.7:0/690603545 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4610012930 con 0x55afbf169330 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.053+0000 7f4605ffb640 1 -- 172.21.15.7:0/690603545 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f460802c500 con 0x55afbf169330 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.053+0000 7f4627797740 10 client.18933 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.053+0000 7f4627797740 10 client.18933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.053+0000 7f4627797740 20 client.18933 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.053+0000 7f4627797740 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55afbf187e30 0x55afbf18a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.053+0000 7f4627797740 4 client.18933 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.053+0000 7f4627797740 1 -- 172.21.15.7:0/690603545 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afbf0d3050 con 0x55afbf187e30 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.053+0000 7f461e59c640 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55afbf187e30 0x55afbf18a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.054+0000 7f461e59c640 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55afbf187e30 0x55afbf18a240 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9bc1dc0d39ddd08 server_cookie=6f9f3a4e3ede7f49 in_seq=0 out_seq=0 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.054+0000 7f4605ffb640 10 client.18933 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.054+0000 7f4605ffb640 1 -- 172.21.15.7:0/690603545 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55afbf0d3050 con 0x55afbf187e30 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.054+0000 7f4605ffb640 10 client.18933 handle_command_reply: tid=0 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.054+0000 7f4627797740 1 client.18933 shutdown 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 -- 172.21.15.7:0/690603545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 msgr2=0x55afbf1624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf1624d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f46080116a0 tx=0x7f4608005f90 comp rx=0 tx=0).stop 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 -- 172.21.15.7:0/690603545 shutdown_connections 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55afbf187e30 0x55afbf18a240 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 0x55afbf167a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 --2- 172.21.15.7:0/690603545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf1624d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 -- 172.21.15.7:0/690603545 >> 172.21.15.7:0/690603545 conn(0x55afbf15c090 msgr2=0x55afbf15b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4605ffb640 0 client.18933 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 -- 172.21.15.7:0/690603545 shutdown_connections 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 1 -- 172.21.15.7:0/690603545 wait complete. 2024-03-26T08:55:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.055+0000 7f4627797740 20 client.18933 trim_cache size 0 max 16384 2024-03-26T08:55:56.073 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:55 smithi044 ceph-mon[36461]: pgmap v765: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.0 MiB/s wr, 581 op/s 2024-03-26T08:55:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.078+0000 7f4627797740 1 Processor -- start 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.078+0000 7f4627797740 1 -- start start 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.078+0000 7f4627797740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf16b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.078+0000 7f4627797740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf16b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.078+0000 7f4627797740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 0x55afbf1620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.078+0000 7f4627797740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a8c40 con 0x55afbf169330 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.078+0000 7f4627797740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a6740 con 0x55afbf169d20 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.078+0000 7f4627797740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a47c0 con 0x55afbf16ee90 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f461e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf16b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f461e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf16b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36076/0 (socket says 172.21.15.7:36076) 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f461e59c640 1 -- 172.21.15.7:0/2916195536 learned_addr learned my addr 172.21.15.7:0/2916195536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f4617fff640 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf16b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f4617fff640 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 0x55afbf1620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f461e59c640 1 -- 172.21.15.7:0/2916195536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 msgr2=0x55afbf1620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f461e59c640 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 0x55afbf1620e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f461e59c640 1 -- 172.21.15.7:0/2916195536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 msgr2=0x55afbf16b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f461e59c640 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf16b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f461e59c640 1 -- 172.21.15.7:0/2916195536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afbf169160 con 0x55afbf169d20 2024-03-26T08:55:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f4617fff640 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 0x55afbf1620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:55:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f461e59c640 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf16b5c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f4610005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da23897fbda5af3f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:56.078 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 805.0 s, 2017.6 tps, lat 50.451 ms stddev 62.947 2024-03-26T08:55:56.081 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f4607fff640 1 -- 172.21.15.7:0/2916195536 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610009790 con 0x55afbf169d20 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f4607fff640 1 -- 172.21.15.7:0/2916195536 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4610005520 con 0x55afbf169d20 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.079+0000 7f4607fff640 1 -- 172.21.15.7:0/2916195536 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461002ba60 con 0x55afbf169d20 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.080+0000 7f4627797740 1 -- 172.21.15.7:0/2916195536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 msgr2=0x55afbf16b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.080+0000 7f4627797740 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf16b5c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f4610005e30 comp rx=0 tx=0).stop 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.080+0000 7f4627797740 1 -- 172.21.15.7:0/2916195536 shutdown_connections 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.080+0000 7f4627797740 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf16ee90 0x55afbf1620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.080+0000 7f4627797740 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf169d20 0x55afbf16b5c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.080+0000 7f4627797740 1 --2- 172.21.15.7:0/2916195536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf16b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.080+0000 7f4627797740 1 -- 172.21.15.7:0/2916195536 >> 172.21.15.7:0/2916195536 conn(0x55afbf15c090 msgr2=0x55afbf15c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.080+0000 7f4627797740 1 -- 172.21.15.7:0/2916195536 shutdown_connections 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.080+0000 7f4627797740 1 -- 172.21.15.7:0/2916195536 wait complete. 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.081+0000 7f4627797740 1 -- start start 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.081+0000 7f4627797740 1 Processor -- start 2024-03-26T08:55:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.081+0000 7f4627797740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4627797740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 0x55afbf19a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53834/0 (socket says 172.21.15.7:53834) 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 -- 172.21.15.7:0/3308362490 learned_addr learned my addr 172.21.15.7:0/3308362490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4627797740 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf16ee90 0x55afbf198200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a4dc0 con 0x55afbf169330 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a60e0 con 0x55afbf16ee90 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46201a5420 con 0x55afbf169d20 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f461e59c640 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 0x55afbf19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf16ee90 0x55afbf198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 -- 172.21.15.7:0/3308362490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 msgr2=0x55afbf19a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 0x55afbf19a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 -- 172.21.15.7:0/3308362490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf16ee90 msgr2=0x55afbf198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf16ee90 0x55afbf198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 -- 172.21.15.7:0/3308362490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afbf198810 con 0x55afbf169330 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4617fff640 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf199bf0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f46080114b0 tx=0x7f4608009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec735a7e60aaa8be server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.082+0000 7f4627797740 10 client.0 _ll_register_callbacks cb 0x7f4608011b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4610005520 con 0x55afbf169330 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4605ffb640 1 -- 172.21.15.7:0/3308362490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460802ce60 con 0x55afbf169330 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4605ffb640 1 -- 172.21.15.7:0/3308362490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f460802c670 con 0x55afbf169330 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4605ffb640 1 -- 172.21.15.7:0/3308362490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4608010ea0 con 0x55afbf169330 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4627797740 10 client.18939 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x55afbf1e3300 con 0x55afbf169330 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4605ffb640 1 -- 172.21.15.7:0/3308362490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f460803a430 con 0x55afbf169330 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4627797740 10 client.18939 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4627797740 10 client.18939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4627797740 20 client.18939 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4627797740 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55afbf1e3b60 0x55afbf1e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4627797740 4 client.18939 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:55:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.083+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f4610005520 con 0x55afbf1e3b60 2024-03-26T08:55:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.084+0000 7f461e59c640 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55afbf1e3b60 0x55afbf1e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.085+0000 7f461e59c640 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55afbf1e3b60 0x55afbf1e5f80 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c02680293f48f8 server_cookie=c86b9a50ff156c7c in_seq=0 out_seq=0 2024-03-26T08:55:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.085+0000 7f4605ffb640 10 client.18939 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.101+0000 7f4605ffb640 1 -- 172.21.15.7:0/3308362490 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f4610005520 con 0x55afbf1e3b60 2024-03-26T08:55:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.101+0000 7f4605ffb640 10 client.18939 handle_command_reply: tid=0 2024-03-26T08:55:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.101+0000 7f4627797740 1 client.18939 shutdown 2024-03-26T08:55:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 msgr2=0x55afbf199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf199bf0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f46080114b0 tx=0x7f4608009fd0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 shutdown_connections 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55afbf1e3b60 0x55afbf1e5f80 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afbf16ee90 0x55afbf198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55afbf169d20 0x55afbf19a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 --2- 172.21.15.7:0/3308362490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55afbf169330 0x55afbf199bf0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 >> 172.21.15.7:0/3308362490 conn(0x55afbf15c090 msgr2=0x55afbf16b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4605ffb640 0 client.18939 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 shutdown_connections 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 1 -- 172.21.15.7:0/3308362490 wait complete. 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f4627797740 20 client.18939 trim_cache size 0 max 16384 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:55:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:55:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:55:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:55:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0038a0 msgr2=0x7f460c005d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f461f59e640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0038a0 0x7f460c005d60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46180b9e70 tx=0x7f4618034f90 comp rx=0 tx=0).stop 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201030c0 msgr2=0x7f462019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f461f59e640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201030c0 0x7f462019cb60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4618009b30 tx=0x7f46180337b0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.102+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 shutdown_connections 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.103+0000 7f461f59e640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f460c0038a0 0x7f460c005d60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.103+0000 7f461f59e640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46201030c0 0x7f462019cb60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.103+0000 7f461f59e640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4620102700 0x7f4620195af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.103+0000 7f461f59e640 1 --2- 172.21.15.7:0/3002106415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620101510 0x7f46201955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.103+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 >> 172.21.15.7:0/3002106415 conn(0x7f46200fca80 msgr2=0x7f4620105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.103+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 shutdown_connections 2024-03-26T08:55:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:55:56.103+0000 7f461f59e640 1 -- 172.21.15.7:0/3002106415 wait complete. 2024-03-26T08:55:57.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:55:57.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:55:57.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.443+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1054872946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae41005c0 msgr2=0x7f2ae41009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.443+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1054872946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae41005c0 0x7f2ae41009e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009ca0 tx=0x7f2ad402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.443+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1054872946 shutdown_connections 2024-03-26T08:55:57.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.443+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1054872946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae4102170 0x7f2ae4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.443+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1054872946 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae41017b0 0x7f2ae4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.443+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1054872946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae41005c0 0x7f2ae41009e0 secure :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009ca0 tx=0x7f2ad402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.443+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1054872946 >> 172.21.15.7:0/1054872946 conn(0x7f2ae40fbb30 msgr2=0x7f2ae40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:57.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.443+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1054872946 shutdown_connections 2024-03-26T08:55:57.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.443+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1054872946 wait complete. 2024-03-26T08:55:57.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.444+0000 7f2ae91b9640 1 Processor -- start 2024-03-26T08:55:57.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae91b9640 1 -- start start 2024-03-26T08:55:57.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae91b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae41017b0 0x7f2ae4069540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae91b9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae4102170 0x7f2ae419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae91b9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae419ce90 0x7f2ae419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae91b9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae410c230 con 0x7f2ae419ce90 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae91b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae410c0b0 con 0x7f2ae41017b0 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae91b9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae410c3b0 con 0x7f2ae4102170 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae419ce90 0x7f2ae419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae41017b0 0x7f2ae4069540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae4102170 0x7f2ae419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae359e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae419ce90 0x7f2ae419f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57476/0 (socket says 172.21.15.7:57476) 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae359e640 1 -- 172.21.15.7:0/1462652551 learned_addr learned my addr 172.21.15.7:0/1462652551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae41017b0 0x7f2ae4069540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40284/0 (socket says 172.21.15.7:40284) 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae359e640 1 -- 172.21.15.7:0/1462652551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae4102170 msgr2=0x7f2ae419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae359e640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae4102170 0x7f2ae419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae359e640 1 -- 172.21.15.7:0/1462652551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae41017b0 msgr2=0x7f2ae4069540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae359e640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae41017b0 0x7f2ae4069540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae359e640 1 -- 172.21.15.7:0/1462652551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae419fc10 con 0x7f2ae419ce90 2024-03-26T08:55:57.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.445+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae41017b0 0x7f2ae4069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:57.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.446+0000 7f2ae359e640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae419ce90 0x7f2ae419f350 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00e970 tx=0x7f2adc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:57.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.446+0000 7f2acbfff640 1 -- 172.21.15.7:0/1462652551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00ccc0 con 0x7f2ae419ce90 2024-03-26T08:55:57.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.446+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1462652551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae419ff00 con 0x7f2ae419ce90 2024-03-26T08:55:57.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.446+0000 7f2acbfff640 1 -- 172.21.15.7:0/1462652551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2adc00ce60 con 0x7f2ae419ce90 2024-03-26T08:55:57.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.446+0000 7f2acbfff640 1 -- 172.21.15.7:0/1462652551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc010490 con 0x7f2ae419ce90 2024-03-26T08:55:57.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.446+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1462652551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae41a0370 con 0x7f2ae419ce90 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.449+0000 7f2aebda3740 1 Processor -- start 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.449+0000 7f2acbfff640 1 -- 172.21.15.7:0/1462652551 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f2adc005020 con 0x7f2ae419ce90 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.449+0000 7f2aebda3740 1 -- start start 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d7011c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d701790 0x555e4d706290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d706900 0x555e4d708d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2aebda3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4d6663d0 con 0x555e4d706900 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2aebda3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4d60ead0 con 0x555e4d701790 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2aebda3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4d662470 con 0x555e4d700da0 2024-03-26T08:55:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d7011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d701790 0x555e4d706290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d7011c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36312/0 (socket says 172.21.15.7:36312) 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae259c640 1 -- 172.21.15.7:0/579956106 learned_addr learned my addr 172.21.15.7:0/579956106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae259c640 1 --2- 172.21.15.7:0/579956106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d706900 0x555e4d708d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2acbfff640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2acc0a2800 0x7f2acc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae2d9d640 1 -- 172.21.15.7:0/579956106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 msgr2=0x555e4d7011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/579956106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d7011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae2d9d640 1 -- 172.21.15.7:0/579956106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d706900 msgr2=0x555e4d708d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/579956106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d706900 0x555e4d708d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae2d9d640 1 -- 172.21.15.7:0/579956106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4d4447d0 con 0x555e4d701790 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae259c640 1 --2- 172.21.15.7:0/579956106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d7011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae359e640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2acc0a2800 0x7f2acc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2acbfff640 1 -- 172.21.15.7:0/1462652551 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2adc0ceef0 con 0x7f2ae419ce90 2024-03-26T08:55:57.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.450+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/579956106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d701790 0x555e4d706290 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4100100 tx=0x7f2ad4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d7a7e12d3ff533 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2ac97fa640 1 -- 172.21.15.7:0/579956106 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4043030 con 0x555e4d701790 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2ae359e640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2acc0a2800 0x7f2acc0a4cc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2adc004fd0 tx=0x7f2adc01b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2ac97fa640 1 -- 172.21.15.7:0/579956106 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ad4032c90 con 0x555e4d701790 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2ac97fa640 1 -- 172.21.15.7:0/579956106 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4039040 con 0x555e4d701790 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2aebda3740 1 -- 172.21.15.7:0/579956106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d701790 msgr2=0x555e4d706290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2aebda3740 1 --2- 172.21.15.7:0/579956106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d701790 0x555e4d706290 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4100100 tx=0x7f2ad4005ef0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2aebda3740 1 -- 172.21.15.7:0/579956106 shutdown_connections 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2aebda3740 1 --2- 172.21.15.7:0/579956106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d706900 0x555e4d708d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2aebda3740 1 --2- 172.21.15.7:0/579956106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d701790 0x555e4d706290 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2aebda3740 1 --2- 172.21.15.7:0/579956106 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d7011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2aebda3740 1 -- 172.21.15.7:0/579956106 >> 172.21.15.7:0/579956106 conn(0x555e4d6fc310 msgr2=0x555e4d6fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2aebda3740 1 -- 172.21.15.7:0/579956106 shutdown_connections 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.451+0000 7f2aebda3740 1 -- 172.21.15.7:0/579956106 wait complete. 2024-03-26T08:55:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2aebda3740 1 -- start start 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2aebda3740 1 Processor -- start 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d700da0 0x555e4d70b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d70b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d706900 0x555e4d712710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2aebda3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4d709850 con 0x555e4d701790 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2aebda3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4d622060 con 0x555e4d700da0 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2aebda3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4d7099d0 con 0x555e4d706900 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2aebda3740 10 client.0 _ll_register_callbacks cb 0x555e4d6046a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d70b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d700da0 0x555e4d70b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae2d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d70b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57500/0 (socket says 172.21.15.7:57500) 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae2d9d640 1 -- 172.21.15.7:0/1620560603 learned_addr learned my addr 172.21.15.7:0/1620560603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae259c640 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d706900 0x555e4d712710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae2d9d640 1 -- 172.21.15.7:0/1620560603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d706900 msgr2=0x555e4d712710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d706900 0x555e4d712710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae2d9d640 1 -- 172.21.15.7:0/1620560603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d700da0 msgr2=0x555e4d70b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d700da0 0x555e4d70b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae2d9d640 1 -- 172.21.15.7:0/1620560603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e4d37a3f0 con 0x555e4d701790 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d70b6d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4037040 tx=0x7f2ad40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef67386605b64730 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.452+0000 7f2ac6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/1620560603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad404c030 con 0x555e4d701790 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/1620560603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2ad40328a0 con 0x555e4d701790 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2aebda3740 1 -- 172.21.15.7:0/1620560603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ad4032c90 con 0x555e4d701790 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/1620560603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ad4043070 con 0x555e4d701790 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2aebda3740 10 client.18951 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2aebda3740 1 -- 172.21.15.7:0/1620560603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555e4d33a0f0 con 0x555e4d701790 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/1620560603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ad4032c90 con 0x555e4d701790 2024-03-26T08:55:57.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/1620560603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2ad4048550 con 0x555e4d701790 2024-03-26T08:55:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2aebda3740 10 client.18951 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2aebda3740 10 client.18951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2aebda3740 20 client.18951 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2aebda3740 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555e4d716870 0x555e4d718c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2aebda3740 4 client.18951 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2aebda3740 1 -- 172.21.15.7:0/1620560603 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e4d673910 con 0x555e4d716870 2024-03-26T08:55:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.453+0000 7f2ae259c640 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555e4d716870 0x555e4d718c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.454+0000 7f2ae259c640 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555e4d716870 0x555e4d718c60 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7861f6016d35a4a9 server_cookie=19d1e721dff391f7 in_seq=0 out_seq=0 2024-03-26T08:55:57.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.454+0000 7f2ac6ff5640 10 client.18951 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:57.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/1620560603 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555e4d673910 con 0x555e4d716870 2024-03-26T08:55:57.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2ac6ff5640 10 client.18951 handle_command_reply: tid=0 2024-03-26T08:55:57.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 client.18951 shutdown 2024-03-26T08:55:57.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 -- 172.21.15.7:0/1620560603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 msgr2=0x555e4d70b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d70b6d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4037040 tx=0x7f2ad40095c0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 -- 172.21.15.7:0/1620560603 shutdown_connections 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555e4d716870 0x555e4d718c60 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d706900 0x555e4d712710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2ac6ff5640 0 client.18951 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d70b6d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 --2- 172.21.15.7:0/1620560603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d700da0 0x555e4d70b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 -- 172.21.15.7:0/1620560603 >> 172.21.15.7:0/1620560603 conn(0x555e4d6fc310 msgr2=0x555e4d6fb530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 -- 172.21.15.7:0/1620560603 shutdown_connections 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 1 -- 172.21.15.7:0/1620560603 wait complete. 2024-03-26T08:55:57.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.455+0000 7f2aebda3740 20 client.18951 trim_cache size 0 max 16384 2024-03-26T08:55:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.491+0000 7f2aebda3740 1 Processor -- start 2024-03-26T08:55:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.492+0000 7f2aebda3740 1 -- start start 2024-03-26T08:55:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.492+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d702bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.492+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d703130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.492+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 0x555e4d7126d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.492+0000 7f2aebda3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41ab480 con 0x555e4d701790 2024-03-26T08:55:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.492+0000 7f2aebda3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41a8f80 con 0x555e4d706900 2024-03-26T08:55:57.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.492+0000 7f2aebda3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41a7000 con 0x555e4d700da0 2024-03-26T08:55:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d702bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 0x555e4d7126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d702bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36318/0 (socket says 172.21.15.7:36318) 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae259c640 1 -- 172.21.15.7:0/411536766 learned_addr learned my addr 172.21.15.7:0/411536766 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/411536766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d703130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae259c640 1 -- 172.21.15.7:0/411536766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 msgr2=0x555e4d7126d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae259c640 1 --2- 172.21.15.7:0/411536766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 0x555e4d7126d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae259c640 1 -- 172.21.15.7:0/411536766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 msgr2=0x555e4d703130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae259c640 1 --2- 172.21.15.7:0/411536766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d703130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae259c640 1 -- 172.21.15.7:0/411536766 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4d6fd080 con 0x555e4d700da0 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.494+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/411536766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d703130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.495+0000 7f2ae259c640 1 --2- 172.21.15.7:0/411536766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d702bf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=babf32e1d2512813 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.495+0000 7f2ac8ff9640 1 -- 172.21.15.7:0/411536766 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8027d40 con 0x555e4d700da0 2024-03-26T08:55:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.495+0000 7f2ac8ff9640 1 -- 172.21.15.7:0/411536766 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ad8005500 con 0x555e4d700da0 2024-03-26T08:55:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.495+0000 7f2ac8ff9640 1 -- 172.21.15.7:0/411536766 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad80035f0 con 0x555e4d700da0 2024-03-26T08:55:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.497+0000 7f2aebda3740 1 -- 172.21.15.7:0/411536766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 msgr2=0x555e4d702bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.497+0000 7f2aebda3740 1 --2- 172.21.15.7:0/411536766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d702bf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8000c00 tx=0x7f2ad8004730 comp rx=0 tx=0).stop 2024-03-26T08:55:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.497+0000 7f2aebda3740 1 -- 172.21.15.7:0/411536766 shutdown_connections 2024-03-26T08:55:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.497+0000 7f2aebda3740 1 --2- 172.21.15.7:0/411536766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 0x555e4d7126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.497+0000 7f2aebda3740 1 --2- 172.21.15.7:0/411536766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d703130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.497+0000 7f2aebda3740 1 --2- 172.21.15.7:0/411536766 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d702bf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.497+0000 7f2aebda3740 1 -- 172.21.15.7:0/411536766 >> 172.21.15.7:0/411536766 conn(0x555e4d6fc310 msgr2=0x555e4d6fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.498+0000 7f2aebda3740 1 -- 172.21.15.7:0/411536766 shutdown_connections 2024-03-26T08:55:57.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.498+0000 7f2aebda3740 1 -- 172.21.15.7:0/411536766 wait complete. 2024-03-26T08:55:57.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.498+0000 7f2aebda3740 1 -- start start 2024-03-26T08:55:57.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2aebda3740 1 Processor -- start 2024-03-26T08:55:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d710290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d7107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2aebda3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 0x555e4d7117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2aebda3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41a7600 con 0x555e4d701790 2024-03-26T08:55:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2aebda3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41a8920 con 0x555e4d706900 2024-03-26T08:55:57.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2aebda3740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae41a7c60 con 0x555e4d700da0 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2aebda3740 10 client.0 _ll_register_callbacks cb 0x7f2ad8009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d710290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2ae259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d710290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36330/0 (socket says 172.21.15.7:36330) 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2ae259c640 1 -- 172.21.15.7:0/289695689 learned_addr learned my addr 172.21.15.7:0/289695689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2ae259c640 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 0x555e4d7117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.499+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d7107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2ae259c640 1 -- 172.21.15.7:0/289695689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 msgr2=0x555e4d7117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2ae259c640 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 0x555e4d7117e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2ae259c640 1 -- 172.21.15.7:0/289695689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 msgr2=0x555e4d7107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2ae259c640 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d7107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2ae259c640 1 -- 172.21.15.7:0/289695689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e4d711dd0 con 0x555e4d700da0 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2ae259c640 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d710290 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80059f0 tx=0x7f2ad8029040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d03350be712c878 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2ac6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/289695689 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8011c50 con 0x555e4d700da0 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2aebda3740 1 -- 172.21.15.7:0/289695689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ad8005500 con 0x555e4d700da0 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.500+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/289695689 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2ad80047a0 con 0x555e4d700da0 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/289695689 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ad8004c20 con 0x555e4d700da0 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2aebda3740 10 client.28735 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2aebda3740 1 -- 172.21.15.7:0/289695689 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555e4d772370 con 0x555e4d700da0 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/289695689 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ad8005500 con 0x555e4d700da0 2024-03-26T08:55:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/289695689 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f2ad8003ad0 con 0x555e4d700da0 2024-03-26T08:55:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2aebda3740 10 client.28735 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2aebda3740 10 client.28735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2aebda3740 20 client.28735 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2aebda3740 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555e4d772c90 0x555e4d7730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2aebda3740 4 client.28735 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:55:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2aebda3740 1 -- 172.21.15.7:0/289695689 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e4d66dc60 con 0x555e4d772c90 2024-03-26T08:55:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.501+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555e4d772c90 0x555e4d7730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2ae2d9d640 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555e4d772c90 0x555e4d7730a0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e114fd66cdc06c24 server_cookie=9e2d132949d8344a in_seq=0 out_seq=0 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2ac6ff5640 10 client.28735 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2ac6ff5640 1 -- 172.21.15.7:0/289695689 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555e4d66dc60 con 0x555e4d772c90 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2ac6ff5640 10 client.28735 handle_command_reply: tid=0 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2aebda3740 1 client.28735 shutdown 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2aebda3740 1 -- 172.21.15.7:0/289695689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 msgr2=0x555e4d710290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2aebda3740 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d710290 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80059f0 tx=0x7f2ad8029040 comp rx=0 tx=0).stop 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2aebda3740 1 -- 172.21.15.7:0/289695689 shutdown_connections 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2aebda3740 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x555e4d772c90 0x555e4d7730a0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2aebda3740 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e4d706900 0x555e4d7117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2aebda3740 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555e4d701790 0x555e4d7107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2aebda3740 1 --2- 172.21.15.7:0/289695689 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555e4d700da0 0x555e4d710290 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.504+0000 7f2aebda3740 1 -- 172.21.15.7:0/289695689 >> 172.21.15.7:0/289695689 conn(0x555e4d6fc310 msgr2=0x555e4d702bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.505+0000 7f2ac6ff5640 0 client.28735 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.505+0000 7f2aebda3740 1 -- 172.21.15.7:0/289695689 shutdown_connections 2024-03-26T08:55:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.505+0000 7f2aebda3740 1 -- 172.21.15.7:0/289695689 wait complete. 2024-03-26T08:55:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.505+0000 7f2aebda3740 20 client.28735 trim_cache size 0 max 16384 2024-03-26T08:55:57.504 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:55:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:55:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:55:57.504 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.505+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2acc0a2800 msgr2=0x7f2acc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.505+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2acc0a2800 0x7f2acc0a4cc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2adc004fd0 tx=0x7f2adc01b8f0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae419ce90 msgr2=0x7f2ae419f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae419ce90 0x7f2ae419f350 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00e970 tx=0x7f2adc00ed70 comp rx=0 tx=0).stop 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1462652551 shutdown_connections 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f2acc0a2800 0x7f2acc0a4cc0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ae419ce90 0x7f2ae419f350 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2ae4102170 0x7f2ae419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 --2- 172.21.15.7:0/1462652551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae41017b0 0x7f2ae4069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1462652551 >> 172.21.15.7:0/1462652551 conn(0x7f2ae40fbb30 msgr2=0x7f2ae4104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:57.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1462652551 shutdown_connections 2024-03-26T08:55:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.506+0000 7f2ae91b9640 1 -- 172.21.15.7:0/1462652551 wait complete. 2024-03-26T08:55:57.517 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:55:57.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:55:57.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.638+0000 7f3873da9640 1 -- 172.21.15.7:0/2603401557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c104a80 msgr2=0x7f386c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.638+0000 7f3873da9640 1 --2- 172.21.15.7:0/2603401557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c104a80 0x7f386c106e70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f385c009900 tx=0x7f385c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:55:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.639+0000 7f3873da9640 1 -- 172.21.15.7:0/2603401557 shutdown_connections 2024-03-26T08:55:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.639+0000 7f3873da9640 1 --2- 172.21.15.7:0/2603401557 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386c1073b0 0x7f386c109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.639+0000 7f3873da9640 1 --2- 172.21.15.7:0/2603401557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c104a80 0x7f386c106e70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.639+0000 7f3873da9640 1 --2- 172.21.15.7:0/2603401557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c102150 0x7f386c104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.639+0000 7f3873da9640 1 -- 172.21.15.7:0/2603401557 >> 172.21.15.7:0/2603401557 conn(0x7f386c0fbb40 msgr2=0x7f386c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.639+0000 7f3873da9640 1 -- 172.21.15.7:0/2603401557 shutdown_connections 2024-03-26T08:55:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.639+0000 7f3873da9640 1 -- 172.21.15.7:0/2603401557 wait complete. 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3873da9640 1 Processor -- start 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3873da9640 1 -- start start 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3873da9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c102150 0x7f386c101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3873da9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c104a80 0x7f386c100100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3873da9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386c1073b0 0x7f386c100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3873da9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10d350 con 0x7f386c102150 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3873da9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10d1d0 con 0x7f386c104a80 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3873da9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10d4d0 con 0x7f386c1073b0 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3871b45640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c102150 0x7f386c101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3871b45640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c102150 0x7f386c101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57532/0 (socket says 172.21.15.7:57532) 2024-03-26T08:55:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3871b45640 1 -- 172.21.15.7:0/2031425761 learned_addr learned my addr 172.21.15.7:0/2031425761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3872346640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386c1073b0 0x7f386c100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3871344640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c104a80 0x7f386c100100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3871b45640 1 -- 172.21.15.7:0/2031425761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386c1073b0 msgr2=0x7f386c100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3871b45640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386c1073b0 0x7f386c100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3871b45640 1 -- 172.21.15.7:0/2031425761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c104a80 msgr2=0x7f386c100100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3871b45640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c104a80 0x7f386c100100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.640+0000 7f3871b45640 1 -- 172.21.15.7:0/2031425761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c100ed0 con 0x7f386c102150 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.641+0000 7f3871b45640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c102150 0x7f386c101ac0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f386800e9f0 tx=0x7f386800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.641+0000 7f3862ffd640 1 -- 172.21.15.7:0/2031425761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386800cd80 con 0x7f386c102150 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.641+0000 7f3862ffd640 1 -- 172.21.15.7:0/2031425761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3868010910 con 0x7f386c102150 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.641+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c1a1b20 con 0x7f386c102150 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.641+0000 7f3862ffd640 1 -- 172.21.15.7:0/2031425761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3868018990 con 0x7f386c102150 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.641+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c1a2090 con 0x7f386c102150 2024-03-26T08:55:57.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.642+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c0fd630 con 0x7f386c102150 2024-03-26T08:55:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.643+0000 7f3862ffd640 1 -- 172.21.15.7:0/2031425761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3868010ac0 con 0x7f386c102150 2024-03-26T08:55:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.643+0000 7f3862ffd640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f38580a2880 0x7f38580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:57.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.643+0000 7f3862ffd640 1 -- 172.21.15.7:0/2031425761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f38680ced60 con 0x7f386c102150 2024-03-26T08:55:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.644+0000 7f3871344640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f38580a2880 0x7f38580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:57.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.645+0000 7f3871344640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f38580a2880 0x7f38580a4d40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f385c000c00 tx=0x7f385c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:57.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.649+0000 7f3862ffd640 1 -- 172.21.15.7:0/2031425761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f386809b770 con 0x7f386c102150 2024-03-26T08:55:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.969+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f386c0622d0 con 0x7f386c102150 2024-03-26T08:55:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.971+0000 7f3862ffd640 1 -- 172.21.15.7:0/2031425761 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f386809f810 con 0x7f386c102150 2024-03-26T08:55:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:55:57.969 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:55:57.969 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:55:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f38580a2880 msgr2=0x7f38580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f38580a2880 0x7f38580a4d40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f385c000c00 tx=0x7f385c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c102150 msgr2=0x7f386c101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c102150 0x7f386c101ac0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f386800e9f0 tx=0x7f386800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 shutdown_connections 2024-03-26T08:55:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f38580a2880 0x7f38580a4d40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f386c1073b0 0x7f386c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c104a80 0x7f386c100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 --2- 172.21.15.7:0/2031425761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f386c102150 0x7f386c101ac0 secure :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f386800e9f0 tx=0x7f386800eeb0 comp rx=0 tx=0).stop 2024-03-26T08:55:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.975+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 >> 172.21.15.7:0/2031425761 conn(0x7f386c0fbb40 msgr2=0x7f386c1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:57.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.976+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 shutdown_connections 2024-03-26T08:55:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:57.978+0000 7f3873da9640 1 -- 172.21.15.7:0/2031425761 wait complete. 2024-03-26T08:55:57.994 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:55:57.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:55:57.994 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:57 smithi007 ceph-mon[32462]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 MiB/s wr, 590 op/s 2024-03-26T08:55:57.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:57 smithi016 ceph-mon[35851]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 MiB/s wr, 590 op/s 2024-03-26T08:55:58.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:57 smithi044 ceph-mon[36461]: pgmap v766: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.1 MiB/s wr, 590 op/s 2024-03-26T08:55:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.116+0000 7fece3f89640 1 -- 172.21.15.7:0/2486506088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc1017b0 msgr2=0x7fecdc101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.116+0000 7fece3f89640 1 --2- 172.21.15.7:0/2486506088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc1017b0 0x7fecdc101c30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7feccc009900 tx=0x7feccc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:55:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.116+0000 7fece3f89640 1 -- 172.21.15.7:0/2486506088 shutdown_connections 2024-03-26T08:55:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.116+0000 7fece3f89640 1 --2- 172.21.15.7:0/2486506088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc102170 0x7fecdc108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.116+0000 7fece3f89640 1 --2- 172.21.15.7:0/2486506088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc1017b0 0x7fecdc101c30 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.116+0000 7fece3f89640 1 --2- 172.21.15.7:0/2486506088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1005c0 0x7fecdc1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.116+0000 7fece3f89640 1 -- 172.21.15.7:0/2486506088 >> 172.21.15.7:0/2486506088 conn(0x7fecdc0fbb70 msgr2=0x7fecdc0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.116+0000 7fece3f89640 1 -- 172.21.15.7:0/2486506088 shutdown_connections 2024-03-26T08:55:58.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.116+0000 7fece3f89640 1 -- 172.21.15.7:0/2486506088 wait complete. 2024-03-26T08:55:58.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece3f89640 1 Processor -- start 2024-03-26T08:55:58.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece3f89640 1 -- start start 2024-03-26T08:55:58.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece3f89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc1005c0 0x7fecdc104550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece3f89640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1017b0 0x7fecdc102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece3f89640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc102170 0x7fecdc1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece3f89640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc10c240 con 0x7fecdc102170 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece3f89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc10c0c0 con 0x7fecdc1005c0 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece3f89640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc10c3c0 con 0x7fecdc1017b0 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece2526640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc102170 0x7fecdc1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece2526640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc102170 0x7fecdc1030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57544/0 (socket says 172.21.15.7:57544) 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece2526640 1 -- 172.21.15.7:0/3068908357 learned_addr learned my addr 172.21.15.7:0/3068908357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece1d25640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc1005c0 0x7fecdc104550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece1524640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1017b0 0x7fecdc102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece2526640 1 -- 172.21.15.7:0/3068908357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1017b0 msgr2=0x7fecdc102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece2526640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1017b0 0x7fecdc102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece2526640 1 -- 172.21.15.7:0/3068908357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc1005c0 msgr2=0x7fecdc104550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece2526640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc1005c0 0x7fecdc104550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece2526640 1 -- 172.21.15.7:0/3068908357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc103960 con 0x7fecdc102170 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fece2526640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc102170 0x7fecdc1030d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fecd400eb10 tx=0x7fecd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.117+0000 7fecd2ffd640 1 -- 172.21.15.7:0/3068908357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4004b20 con 0x7fecdc102170 2024-03-26T08:55:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.118+0000 7fecd2ffd640 1 -- 172.21.15.7:0/3068908357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecd4004cc0 con 0x7fecdc102170 2024-03-26T08:55:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.118+0000 7fece3f89640 1 -- 172.21.15.7:0/3068908357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc1a1ac0 con 0x7fecdc102170 2024-03-26T08:55:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.118+0000 7fecd2ffd640 1 -- 172.21.15.7:0/3068908357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4010760 con 0x7fecdc102170 2024-03-26T08:55:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.118+0000 7fece3f89640 1 -- 172.21.15.7:0/3068908357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc1a1f90 con 0x7fecdc102170 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece6b7b740 1 Processor -- start 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fecd2ffd640 1 -- 172.21.15.7:0/3068908357 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fecd4005020 con 0x7fecdc102170 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece6b7b740 1 -- start start 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d5aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 0x557552d5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 0x557552d62680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece6b7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557552d1d610 con 0x557552d5a690 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece6b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557552cc03c0 con 0x557552d601f0 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece6b7b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557552c68d60 con 0x557552d5b080 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece1524640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d5aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece1524640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d5aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57554/0 (socket says 172.21.15.7:57554) 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece1524640 1 -- 172.21.15.7:0/3812765258 learned_addr learned my addr 172.21.15.7:0/3812765258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece1d25640 1 --2- 172.21.15.7:0/3812765258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 0x557552d5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fece1524640 1 --2- 172.21.15.7:0/3812765258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 0x557552d62680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.121+0000 7fecd2ffd640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fecc80a2800 0x7fecc80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fece1d25640 1 -- 172.21.15.7:0/3812765258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 msgr2=0x557552d62680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fece1d25640 1 --2- 172.21.15.7:0/3812765258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 0x557552d62680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fece1d25640 1 -- 172.21.15.7:0/3812765258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 msgr2=0x557552d5aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fece2526640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fecc80a2800 0x7fecc80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fece1d25640 1 --2- 172.21.15.7:0/3812765258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d5aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fece1d25640 1 -- 172.21.15.7:0/3812765258 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575529d43f0 con 0x557552d5b080 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fecd2ffd640 1 -- 172.21.15.7:0/3068908357 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fecd40cefa0 con 0x7fecdc102170 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fece1d25640 1 --2- 172.21.15.7:0/3812765258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 0x557552d5fb80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009bd0 tx=0x7fecd8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd297bb16885530 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fece2526640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fecc80a2800 0x7fecc80a4cc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fecd40a3c50 tx=0x7fecd4090ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fecc7fff640 1 -- 172.21.15.7:0/3812765258 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd801e070 con 0x557552d5b080 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fecc7fff640 1 -- 172.21.15.7:0/3812765258 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecd8009d80 con 0x557552d5b080 2024-03-26T08:55:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.122+0000 7fecc7fff640 1 -- 172.21.15.7:0/3812765258 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8019450 con 0x557552d5b080 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 -- 172.21.15.7:0/3812765258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 msgr2=0x557552d5fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3812765258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 0x557552d5fb80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009bd0 tx=0x7fecd8011810 comp rx=0 tx=0).stop 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 -- 172.21.15.7:0/3812765258 shutdown_connections 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3812765258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 0x557552d62680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3812765258 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 0x557552d5fb80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3812765258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d5aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 -- 172.21.15.7:0/3812765258 >> 172.21.15.7:0/3812765258 conn(0x557552d55c00 msgr2=0x557552d56000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 -- 172.21.15.7:0/3812765258 shutdown_connections 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 -- 172.21.15.7:0/3812765258 wait complete. 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 -- start start 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.123+0000 7fece6b7b740 1 Processor -- start 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5a690 0x557552d64a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5b080 0x557552d64fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 0x557552d6bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece6b7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557552c7c240 con 0x557552d5b080 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece6b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557552cbc460 con 0x557552d601f0 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece6b7b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557552d631a0 con 0x557552d5a690 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece6b7b740 10 client.0 _ll_register_callbacks cb 0x557552c7bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1524640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5a690 0x557552d64a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1d25640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5b080 0x557552d64fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1524640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5a690 0x557552d64a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36390/0 (socket says 172.21.15.7:36390) 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1524640 1 -- 172.21.15.7:0/3882843747 learned_addr learned my addr 172.21.15.7:0/3882843747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1524640 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 0x557552d6bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1d25640 1 -- 172.21.15.7:0/3882843747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5a690 msgr2=0x557552d64a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1d25640 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5a690 0x557552d64a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1d25640 1 -- 172.21.15.7:0/3882843747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 msgr2=0x557552d6bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1d25640 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 0x557552d6bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1d25640 1 -- 172.21.15.7:0/3882843747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557552a9e7d0 con 0x557552d5b080 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece1d25640 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5b080 0x557552d64fa0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x557552d56b70 tx=0x7fecd8003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74974ea72138aaa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fecc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fecc57fa640 1 -- 172.21.15.7:0/3882843747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd801e040 con 0x557552d5b080 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fecc57fa640 1 -- 172.21.15.7:0/3882843747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fecd8005370 con 0x557552d5b080 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.124+0000 7fece6b7b740 1 -- 172.21.15.7:0/3882843747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecd8009d80 con 0x557552d5b080 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fecc57fa640 1 -- 172.21.15.7:0/3882843747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd8005570 con 0x557552d5b080 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fece6b7b740 10 client.18972 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fece6b7b740 1 -- 172.21.15.7:0/3882843747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557552d6f350 con 0x557552d5b080 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fecc57fa640 1 -- 172.21.15.7:0/3882843747 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fecd8009d80 con 0x557552d5b080 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fecc57fa640 1 -- 172.21.15.7:0/3882843747 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fecd8021030 con 0x557552d5b080 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fece6b7b740 10 client.18972 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fece6b7b740 10 client.18972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fece6b7b740 20 client.18972 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557552d70890 0x557552d72ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fece6b7b740 4 client.18972 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fece6b7b740 1 -- 172.21.15.7:0/3882843747 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557552ccd900 con 0x557552d70890 2024-03-26T08:55:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.125+0000 7fece1524640 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557552d70890 0x557552d72ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.126+0000 7fecc57fa640 10 client.18972 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:55:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.126+0000 7fece1524640 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557552d70890 0x557552d72ca0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa1e42d2a8f3c409 server_cookie=ce1520f961fe9880 in_seq=0 out_seq=0 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.127+0000 7fecc57fa640 1 -- 172.21.15.7:0/3882843747 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557552ccd900 con 0x557552d70890 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.127+0000 7fecc57fa640 10 client.18972 handle_command_reply: tid=0 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.127+0000 7fece6b7b740 1 client.18972 shutdown 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 -- 172.21.15.7:0/3882843747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5b080 msgr2=0x557552d64fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5b080 0x557552d64fa0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x557552d56b70 tx=0x7fecd8003a30 comp rx=0 tx=0).stop 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 -- 172.21.15.7:0/3882843747 shutdown_connections 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557552d70890 0x557552d72ca0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d601f0 0x557552d6bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5b080 0x557552d64fa0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fecc57fa640 0 client.18972 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 --2- 172.21.15.7:0/3882843747 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5a690 0x557552d64a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 -- 172.21.15.7:0/3882843747 >> 172.21.15.7:0/3882843747 conn(0x557552d55c00 msgr2=0x557552d54e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 -- 172.21.15.7:0/3882843747 shutdown_connections 2024-03-26T08:55:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 1 -- 172.21.15.7:0/3882843747 wait complete. 2024-03-26T08:55:58.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.128+0000 7fece6b7b740 20 client.18972 trim_cache size 0 max 16384 2024-03-26T08:55:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece6b7b740 1 Processor -- start 2024-03-26T08:55:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece6b7b740 1 -- start start 2024-03-26T08:55:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5a690 0x557552d64080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 0x557552d6bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d601f0 0x557552d59570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece6b7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1ad190 con 0x557552d601f0 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece6b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1aac90 con 0x557552d5a690 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece6b7b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1a8d10 con 0x557552d5b080 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece1524640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d601f0 0x557552d59570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece1d25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 0x557552d6bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece1524640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d601f0 0x557552d59570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57582/0 (socket says 172.21.15.7:57582) 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece1524640 1 -- 172.21.15.7:0/420193469 learned_addr learned my addr 172.21.15.7:0/420193469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.143+0000 7fece1524640 1 --2- 172.21.15.7:0/420193469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5a690 0x557552d64080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.144+0000 7fece1524640 1 -- 172.21.15.7:0/420193469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 msgr2=0x557552d6bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.144+0000 7fece1524640 1 --2- 172.21.15.7:0/420193469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 0x557552d6bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.144+0000 7fece1524640 1 -- 172.21.15.7:0/420193469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5a690 msgr2=0x557552d64080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.144+0000 7fece1524640 1 --2- 172.21.15.7:0/420193469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5a690 0x557552d64080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.144+0000 7fece1524640 1 -- 172.21.15.7:0/420193469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557552d63320 con 0x557552d601f0 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.144+0000 7fece1524640 1 --2- 172.21.15.7:0/420193469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d601f0 0x557552d59570 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7feccc02cec0 tx=0x7feccc043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85be8c8f2b3831c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.144+0000 7fecc77fe640 1 -- 172.21.15.7:0/420193469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc055070 con 0x557552d601f0 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.144+0000 7fecc77fe640 1 -- 172.21.15.7:0/420193469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feccc04e690 con 0x557552d601f0 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.144+0000 7fecc77fe640 1 -- 172.21.15.7:0/420193469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc005730 con 0x557552d601f0 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.145+0000 7fece6b7b740 1 -- 172.21.15.7:0/420193469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d601f0 msgr2=0x557552d59570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.145+0000 7fece6b7b740 1 --2- 172.21.15.7:0/420193469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d601f0 0x557552d59570 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7feccc02cec0 tx=0x7feccc043040 comp rx=0 tx=0).stop 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 -- 172.21.15.7:0/420193469 shutdown_connections 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 --2- 172.21.15.7:0/420193469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d601f0 0x557552d59570 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 --2- 172.21.15.7:0/420193469 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d5b080 0x557552d6bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 --2- 172.21.15.7:0/420193469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5a690 0x557552d64080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 -- 172.21.15.7:0/420193469 >> 172.21.15.7:0/420193469 conn(0x557552d55c00 msgr2=0x557552d58e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 -- 172.21.15.7:0/420193469 shutdown_connections 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 -- 172.21.15.7:0/420193469 wait complete. 2024-03-26T08:55:58.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 -- start start 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 Processor -- start 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d6bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5b080 0x557552d6a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d601f0 0x557552d6a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1a9310 con 0x557552d5a690 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1aa630 con 0x557552d5b080 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1a9970 con 0x557552d601f0 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece6b7b740 10 client.0 _ll_register_callbacks cb 0x7feccc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece1524640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d6bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece1524640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d6bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57584/0 (socket says 172.21.15.7:57584) 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece1524640 1 -- 172.21.15.7:0/4186560596 learned_addr learned my addr 172.21.15.7:0/4186560596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.146+0000 7fece1d25640 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5b080 0x557552d6a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece1524640 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d601f0 0x557552d6a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece1524640 1 -- 172.21.15.7:0/4186560596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d601f0 msgr2=0x557552d6a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece1524640 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d601f0 0x557552d6a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece1524640 1 -- 172.21.15.7:0/4186560596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5b080 msgr2=0x557552d6a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece1524640 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5b080 0x557552d6a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece1524640 1 -- 172.21.15.7:0/4186560596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557552d6ad10 con 0x557552d5a690 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece1d25640 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5b080 0x557552d6a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece1524640 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d6bc30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7feccc02c420 tx=0x7feccc02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77028cb8b4c369b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fecc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fecc57fa640 1 -- 172.21.15.7:0/4186560596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc055070 con 0x557552d5a690 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece6b7b740 1 -- 172.21.15.7:0/4186560596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feccc04e690 con 0x557552d5a690 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fecc57fa640 1 -- 172.21.15.7:0/4186560596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7feccc04ed20 con 0x557552d5a690 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fecc57fa640 1 -- 172.21.15.7:0/4186560596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feccc0035f0 con 0x557552d5a690 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece6b7b740 10 client.18978 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.147+0000 7fece6b7b740 1 -- 172.21.15.7:0/4186560596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557552dcce70 con 0x557552d5a690 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fecc57fa640 1 -- 172.21.15.7:0/4186560596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feccc04e690 con 0x557552d5a690 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fecc57fa640 1 -- 172.21.15.7:0/4186560596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7feccc05b030 con 0x557552d5a690 2024-03-26T08:55:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fece6b7b740 10 client.18978 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fece6b7b740 10 client.18978 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fece6b7b740 20 client.18978 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fece6b7b740 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557552dcd6d0 0x557552dcfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fece6b7b740 4 client.18978 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fece6b7b740 1 -- 172.21.15.7:0/4186560596 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557552d64850 con 0x557552dcd6d0 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fece1d25640 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557552dcd6d0 0x557552dcfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fece1d25640 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557552dcd6d0 0x557552dcfac0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=65ee19e64e159db server_cookie=83b906f5420db47a in_seq=0 out_seq=0 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fecc57fa640 10 client.18978 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fecc57fa640 1 -- 172.21.15.7:0/4186560596 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557552d64850 con 0x557552dcd6d0 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.148+0000 7fecc57fa640 10 client.18978 handle_command_reply: tid=0 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 client.18978 shutdown 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 -- 172.21.15.7:0/4186560596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 msgr2=0x557552d6bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d6bc30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7feccc02c420 tx=0x7feccc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 -- 172.21.15.7:0/4186560596 shutdown_connections 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x557552dcd6d0 0x557552dcfac0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557552d601f0 0x557552d6a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557552d5b080 0x557552d6a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 --2- 172.21.15.7:0/4186560596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557552d5a690 0x557552d6bc30 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fecc57fa640 0 client.18978 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:55:58.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 -- 172.21.15.7:0/4186560596 >> 172.21.15.7:0/4186560596 conn(0x557552d55c00 msgr2=0x557552d596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 -- 172.21.15.7:0/4186560596 shutdown_connections 2024-03-26T08:55:58.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 1 -- 172.21.15.7:0/4186560596 wait complete. 2024-03-26T08:55:58.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece6b7b740 20 client.18978 trim_cache size 0 max 16384 2024-03-26T08:55:58.151 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:55:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 -- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fecc80a2800 msgr2=0x7fecc80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fecc80a2800 0x7fecc80a4cc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fecd40a3c50 tx=0x7fecd4090ec0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 -- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc102170 msgr2=0x7fecdc1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc102170 0x7fecdc1030d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fecd400eb10 tx=0x7fecd400c2d0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 -- 172.21.15.7:0/3068908357 shutdown_connections 2024-03-26T08:55:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fecc80a2800 0x7fecc80a4cc0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecdc102170 0x7fecdc1030d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fecdc1017b0 0x7fecdc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 --2- 172.21.15.7:0/3068908357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecdc1005c0 0x7fecdc104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.149+0000 7fece3f89640 1 -- 172.21.15.7:0/3068908357 >> 172.21.15.7:0/3068908357 conn(0x7fecdc0fbb70 msgr2=0x7fecdc0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.150+0000 7fece3f89640 1 -- 172.21.15.7:0/3068908357 shutdown_connections 2024-03-26T08:55:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.150+0000 7fece3f89640 1 -- 172.21.15.7:0/3068908357 wait complete. 2024-03-26T08:55:58.170 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:55:58.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:55:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.329+0000 7fa920bba640 1 -- 172.21.15.7:0/1016210475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c101c40 msgr2=0x7fa91c102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.329+0000 7fa920bba640 1 --2- 172.21.15.7:0/1016210475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c101c40 0x7fa91c102060 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009900 tx=0x7fa90c02bc80 comp rx=0 tx=0).stop 2024-03-26T08:55:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.329+0000 7fa920bba640 1 -- 172.21.15.7:0/1016210475 shutdown_connections 2024-03-26T08:55:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.329+0000 7fa920bba640 1 --2- 172.21.15.7:0/1016210475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c1011c0 0x7fa91c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.329+0000 7fa920bba640 1 --2- 172.21.15.7:0/1016210475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c100800 0x7fa91c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.329+0000 7fa920bba640 1 --2- 172.21.15.7:0/1016210475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c101c40 0x7fa91c102060 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.329+0000 7fa920bba640 1 -- 172.21.15.7:0/1016210475 >> 172.21.15.7:0/1016210475 conn(0x7fa91c0fb2b0 msgr2=0x7fa91c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.329+0000 7fa920bba640 1 -- 172.21.15.7:0/1016210475 shutdown_connections 2024-03-26T08:55:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.329+0000 7fa920bba640 1 -- 172.21.15.7:0/1016210475 wait complete. 2024-03-26T08:55:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.330+0000 7fa920bba640 1 Processor -- start 2024-03-26T08:55:58.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.330+0000 7fa920bba640 1 -- start start 2024-03-26T08:55:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.330+0000 7fa920bba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c100800 0x7fa91c1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.330+0000 7fa920bba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c1011c0 0x7fa91c196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.330+0000 7fa920bba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c101c40 0x7fa91c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.330+0000 7fa920bba640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c10c260 con 0x7fa91c100800 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.330+0000 7fa920bba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c10c0e0 con 0x7fa91c101c40 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.330+0000 7fa920bba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c10c3e0 con 0x7fa91c1011c0 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c100800 0x7fa91c1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c101c40 0x7fa91c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa919d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c1011c0 0x7fa91c196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c100800 0x7fa91c1968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57602/0 (socket says 172.21.15.7:57602) 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa919d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c1011c0 0x7fa91c196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36426/0 (socket says 172.21.15.7:36426) 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c101c40 0x7fa91c19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40446/0 (socket says 172.21.15.7:40446) 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91a59c640 1 -- 172.21.15.7:0/3489097090 learned_addr learned my addr 172.21.15.7:0/3489097090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91a59c640 1 -- 172.21.15.7:0/3489097090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c1011c0 msgr2=0x7fa91c196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91a59c640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c1011c0 0x7fa91c196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91a59c640 1 -- 172.21.15.7:0/3489097090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c101c40 msgr2=0x7fa91c19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91a59c640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c101c40 0x7fa91c19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa91a59c640 1 -- 172.21.15.7:0/3489097090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c19e4f0 con 0x7fa91c100800 2024-03-26T08:55:58.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.331+0000 7fa919d9b640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c1011c0 0x7fa91c196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.332+0000 7fa91a59c640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c100800 0x7fa91c1968d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa90c005ab0 tx=0x7fa90c02c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.332+0000 7fa90b7fe640 1 -- 172.21.15.7:0/3489097090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c041070 con 0x7fa91c100800 2024-03-26T08:55:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.332+0000 7fa90b7fe640 1 -- 172.21.15.7:0/3489097090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa90c004450 con 0x7fa91c100800 2024-03-26T08:55:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.332+0000 7fa920bba640 1 -- 172.21.15.7:0/3489097090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c19e780 con 0x7fa91c100800 2024-03-26T08:55:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.332+0000 7fa90b7fe640 1 -- 172.21.15.7:0/3489097090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c034440 con 0x7fa91c100800 2024-03-26T08:55:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.332+0000 7fa920bba640 1 -- 172.21.15.7:0/3489097090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c19ecc0 con 0x7fa91c100800 2024-03-26T08:55:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.334+0000 7fa90b7fe640 1 -- 172.21.15.7:0/3489097090 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa90c044070 con 0x7fa91c100800 2024-03-26T08:55:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.334+0000 7fa90b7fe640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9040a2880 0x7fa9040a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.334+0000 7fa92379d740 1 Processor -- start 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa919d9b640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9040a2880 0x7fa9040a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa90b7fe640 1 -- 172.21.15.7:0/3489097090 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa90c0f04b0 con 0x7fa91c100800 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa92379d740 1 -- start start 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa92379d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049dd68a0 0x564049dd6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa92379d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049ddbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa92379d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049ddc400 0x564049dde890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa92379d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564049d97610 con 0x564049ddc400 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa92379d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564049d3a3c0 con 0x564049dd68a0 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa92379d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564049ce2d60 con 0x564049dd7290 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa919d9b640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9040a2880 0x7fa9040a4d40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa91c197de0 tx=0x7fa910005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049dd68a0 0x564049dd6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91ad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049ddc400 0x564049dde890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049dd68a0 0x564049dd6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40462/0 (socket says 172.21.15.7:40462) 2024-03-26T08:55:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91a59c640 1 -- 172.21.15.7:0/1965601597 learned_addr learned my addr 172.21.15.7:0/1965601597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa919d9b640 1 --2- 172.21.15.7:0/1965601597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049ddbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91a59c640 1 -- 172.21.15.7:0/1965601597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 msgr2=0x564049ddbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91a59c640 1 --2- 172.21.15.7:0/1965601597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049ddbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91a59c640 1 -- 172.21.15.7:0/1965601597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049ddc400 msgr2=0x564049dde890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91a59c640 1 --2- 172.21.15.7:0/1965601597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049ddc400 0x564049dde890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91a59c640 1 -- 172.21.15.7:0/1965601597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564049a4e3f0 con 0x564049dd68a0 2024-03-26T08:55:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.335+0000 7fa91ad9d640 1 --2- 172.21.15.7:0/1965601597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049ddc400 0x564049dde890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.336+0000 7fa91a59c640 1 --2- 172.21.15.7:0/1965601597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049dd68a0 0x564049dd6cc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0b9920 tx=0x7fa90c003a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfc701aae564620c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.336+0000 7fa908ff9640 1 -- 172.21.15.7:0/1965601597 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c04b070 con 0x564049dd68a0 2024-03-26T08:55:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.336+0000 7fa908ff9640 1 -- 172.21.15.7:0/1965601597 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa90c0c0ff0 con 0x564049dd68a0 2024-03-26T08:55:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.336+0000 7fa908ff9640 1 -- 172.21.15.7:0/1965601597 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0c30c0 con 0x564049dd68a0 2024-03-26T08:55:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.337+0000 7fa92379d740 1 -- 172.21.15.7:0/1965601597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049dd68a0 msgr2=0x564049dd6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.337+0000 7fa92379d740 1 --2- 172.21.15.7:0/1965601597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049dd68a0 0x564049dd6cc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0b9920 tx=0x7fa90c003a20 comp rx=0 tx=0).stop 2024-03-26T08:55:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.337+0000 7fa92379d740 1 -- 172.21.15.7:0/1965601597 shutdown_connections 2024-03-26T08:55:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.337+0000 7fa92379d740 1 --2- 172.21.15.7:0/1965601597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049ddc400 0x564049dde890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.337+0000 7fa92379d740 1 --2- 172.21.15.7:0/1965601597 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049ddbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.337+0000 7fa92379d740 1 --2- 172.21.15.7:0/1965601597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049dd68a0 0x564049dd6cc0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.337+0000 7fa92379d740 1 -- 172.21.15.7:0/1965601597 >> 172.21.15.7:0/1965601597 conn(0x564049dd1e10 msgr2=0x564049dd2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.337+0000 7fa92379d740 1 -- 172.21.15.7:0/1965601597 shutdown_connections 2024-03-26T08:55:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.337+0000 7fa92379d740 1 -- 172.21.15.7:0/1965601597 wait complete. 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa92379d740 1 -- start start 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa92379d740 1 Processor -- start 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa92379d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 0x564049de5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa92379d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049de79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa92379d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049de9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa92379d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564049cf6240 con 0x564049dd7290 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa92379d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564049d36460 con 0x564049ddc400 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa92379d740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564049dcf770 con 0x564049dd68a0 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa919d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049de79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa92379d740 10 client.0 _ll_register_callbacks cb 0x564049cf5c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa91ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049de9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 0x564049de5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa91ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049de9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40472/0 (socket says 172.21.15.7:40472) 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 0x564049de5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36454/0 (socket says 172.21.15.7:36454) 2024-03-26T08:55:58.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa91ad9d640 1 -- 172.21.15.7:0/3070434223 learned_addr learned my addr 172.21.15.7:0/3070434223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.338+0000 7fa919d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049de79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57628/0 (socket says 172.21.15.7:57628) 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa91ad9d640 1 -- 172.21.15.7:0/3070434223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 msgr2=0x564049de5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa91ad9d640 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 0x564049de5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa91ad9d640 1 -- 172.21.15.7:0/3070434223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 msgr2=0x564049de79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa91ad9d640 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049de79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa91ad9d640 1 -- 172.21.15.7:0/3070434223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564049b187d0 con 0x564049ddc400 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa919d9b640 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049de79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa91ad9d640 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049de9ef0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa914011d10 tx=0x7fa91400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be97540887c446d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa901ffb640 1 -- 172.21.15.7:0/3070434223 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91401e060 con 0x564049ddc400 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa901ffb640 1 -- 172.21.15.7:0/3070434223 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa914003970 con 0x564049ddc400 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa92379d740 1 -- 172.21.15.7:0/3070434223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa90c0c0ff0 con 0x564049ddc400 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa901ffb640 1 -- 172.21.15.7:0/3070434223 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa91400fe60 con 0x564049ddc400 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa92379d740 10 client.28775 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa92379d740 1 -- 172.21.15.7:0/3070434223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564049ded570 con 0x564049ddc400 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.339+0000 7fa901ffb640 1 -- 172.21.15.7:0/3070434223 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa90c0c0ff0 con 0x564049ddc400 2024-03-26T08:55:58.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.340+0000 7fa901ffb640 1 -- 172.21.15.7:0/3070434223 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa914023070 con 0x564049ddc400 2024-03-26T08:55:58.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.340+0000 7fa92379d740 10 client.28775 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:58.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.340+0000 7fa92379d740 10 client.28775 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:55:58.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.340+0000 7fa92379d740 20 client.28775 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:58.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.340+0000 7fa92379d740 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564049deec90 0x564049df10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.340+0000 7fa92379d740 4 client.28775 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:58.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.340+0000 7fa92379d740 1 -- 172.21.15.7:0/3070434223 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564049d47900 con 0x564049deec90 2024-03-26T08:55:58.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.340+0000 7fa91a59c640 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564049deec90 0x564049df10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.341+0000 7fa91a59c640 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564049deec90 0x564049df10b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96005c000b30fd68 server_cookie=94d8aeb9e80e13fd in_seq=0 out_seq=0 2024-03-26T08:55:58.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.341+0000 7fa901ffb640 10 client.28775 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.341+0000 7fa901ffb640 1 -- 172.21.15.7:0/3070434223 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564049d47900 con 0x564049deec90 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.341+0000 7fa901ffb640 10 client.28775 handle_command_reply: tid=0 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.341+0000 7fa92379d740 1 client.28775 shutdown 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 -- 172.21.15.7:0/3070434223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 msgr2=0x564049de9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049de9ef0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa914011d10 tx=0x7fa91400f790 comp rx=0 tx=0).stop 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 -- 172.21.15.7:0/3070434223 shutdown_connections 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049de9ef0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049de79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564049deec90 0x564049df10b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 --2- 172.21.15.7:0/3070434223 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 0x564049de5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa901ffb640 0 client.28775 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 -- 172.21.15.7:0/3070434223 >> 172.21.15.7:0/3070434223 conn(0x564049dd1e10 msgr2=0x564049dcee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 -- 172.21.15.7:0/3070434223 shutdown_connections 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 1 -- 172.21.15.7:0/3070434223 wait complete. 2024-03-26T08:55:58.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.342+0000 7fa92379d740 20 client.28775 trim_cache size 0 max 16384 2024-03-26T08:55:58.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.370+0000 7fa92379d740 1 Processor -- start 2024-03-26T08:55:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.372+0000 7fa92379d740 1 -- start start 2024-03-26T08:55:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.372+0000 7fa92379d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 0x564049ddaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.372+0000 7fa92379d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049dd3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.372+0000 7fa92379d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049dd35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.372+0000 7fa92379d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a8d40 con 0x564049dd7290 2024-03-26T08:55:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.372+0000 7fa92379d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a6840 con 0x564049ddc400 2024-03-26T08:55:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.373+0000 7fa91ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049dd35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.373+0000 7fa91ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049dd35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40478/0 (socket says 172.21.15.7:40478) 2024-03-26T08:55:58.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.373+0000 7fa91ad9d640 1 -- 172.21.15.7:0/555954205 learned_addr learned my addr 172.21.15.7:0/555954205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.373+0000 7fa92379d740 1 -- 172.21.15.7:0/555954205 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a48c0 con 0x564049dd68a0 2024-03-26T08:55:58.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.376+0000 7fa919d9b640 1 --2- 172.21.15.7:0/555954205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049dd3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.376+0000 7fa91ad9d640 1 -- 172.21.15.7:0/555954205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 msgr2=0x564049ddaac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:55:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.376+0000 7fa91ad9d640 1 --2- 172.21.15.7:0/555954205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 0x564049ddaac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.376+0000 7fa91ad9d640 1 -- 172.21.15.7:0/555954205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 msgr2=0x564049dd3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.376+0000 7fa91ad9d640 1 --2- 172.21.15.7:0/555954205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049dd3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.376+0000 7fa91ad9d640 1 -- 172.21.15.7:0/555954205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564049dd2b80 con 0x564049ddc400 2024-03-26T08:55:58.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.376+0000 7fa91ad9d640 1 --2- 172.21.15.7:0/555954205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049dd35a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa91401a540 tx=0x7fa914003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ddf851143b634eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.380+0000 7fa903fff640 1 -- 172.21.15.7:0/555954205 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914032070 con 0x564049ddc400 2024-03-26T08:55:58.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.380+0000 7fa903fff640 1 -- 172.21.15.7:0/555954205 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa91401ac10 con 0x564049ddc400 2024-03-26T08:55:58.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.380+0000 7fa903fff640 1 -- 172.21.15.7:0/555954205 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914029040 con 0x564049ddc400 2024-03-26T08:55:58.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.381+0000 7fa92379d740 1 -- 172.21.15.7:0/555954205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 msgr2=0x564049dd35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.381+0000 7fa92379d740 1 --2- 172.21.15.7:0/555954205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049dd35a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa91401a540 tx=0x7fa914003b80 comp rx=0 tx=0).stop 2024-03-26T08:55:58.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.381+0000 7fa92379d740 1 -- 172.21.15.7:0/555954205 shutdown_connections 2024-03-26T08:55:58.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.381+0000 7fa92379d740 1 --2- 172.21.15.7:0/555954205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049dd35a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.381+0000 7fa92379d740 1 --2- 172.21.15.7:0/555954205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd7290 0x564049dd3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.381+0000 7fa92379d740 1 --2- 172.21.15.7:0/555954205 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd68a0 0x564049ddaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.381+0000 7fa92379d740 1 -- 172.21.15.7:0/555954205 >> 172.21.15.7:0/555954205 conn(0x564049dd1e10 msgr2=0x564049dcee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.383+0000 7fa92379d740 1 -- 172.21.15.7:0/555954205 shutdown_connections 2024-03-26T08:55:58.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.383+0000 7fa92379d740 1 -- 172.21.15.7:0/555954205 wait complete. 2024-03-26T08:55:58.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.383+0000 7fa92379d740 1 -- start start 2024-03-26T08:55:58.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.383+0000 7fa92379d740 1 Processor -- start 2024-03-26T08:55:58.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.384+0000 7fa92379d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd68a0 0x564049de18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.384+0000 7fa92379d740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049de1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.384+0000 7fa92379d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049de8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.384+0000 7fa919d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049de1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa919d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049de1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36484/0 (socket says 172.21.15.7:36484) 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa919d9b640 1 -- 172.21.15.7:0/1371501881 learned_addr learned my addr 172.21.15.7:0/1371501881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a4ec0 con 0x564049dd68a0 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a61e0 con 0x564049ddc400 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1a5520 con 0x564049dd7290 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa92379d740 10 client.0 _ll_register_callbacks cb 0x7fa914011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa91a59c640 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd68a0 0x564049de18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa919d9b640 1 -- 172.21.15.7:0/1371501881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 msgr2=0x564049de8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa919d9b640 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049de8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa919d9b640 1 -- 172.21.15.7:0/1371501881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd68a0 msgr2=0x564049de18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa919d9b640 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd68a0 0x564049de18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa919d9b640 1 -- 172.21.15.7:0/1371501881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564049de9360 con 0x564049dd7290 2024-03-26T08:55:58.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.385+0000 7fa91a59c640 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd68a0 0x564049de18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:55:58.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.386+0000 7fa919d9b640 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049de1df0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa91c1018d0 tx=0x7fa910004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893702e150625054 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.386+0000 7fa901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:55:58.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.387+0000 7fa901ffb640 1 -- 172.21.15.7:0/1371501881 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91002e060 con 0x564049dd7290 2024-03-26T08:55:58.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.387+0000 7fa901ffb640 1 -- 172.21.15.7:0/1371501881 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa91001f430 con 0x564049dd7290 2024-03-26T08:55:58.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.387+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa91401ac10 con 0x564049dd7290 2024-03-26T08:55:58.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.388+0000 7fa901ffb640 1 -- 172.21.15.7:0/1371501881 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa91001f670 con 0x564049dd7290 2024-03-26T08:55:58.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.388+0000 7fa92379d740 10 client.28774 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:58.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.388+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564049e4f020 con 0x564049dd7290 2024-03-26T08:55:58.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.388+0000 7fa901ffb640 1 -- 172.21.15.7:0/1371501881 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa91001fcf0 con 0x564049dd7290 2024-03-26T08:55:58.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.389+0000 7fa901ffb640 1 -- 172.21.15.7:0/1371501881 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa910028030 con 0x564049dd7290 2024-03-26T08:55:58.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.389+0000 7fa92379d740 10 client.28774 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:58.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.389+0000 7fa92379d740 10 client.28774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:55:58.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.389+0000 7fa92379d740 20 client.28774 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:58.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.389+0000 7fa92379d740 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564049e4f9e0 0x564049e51e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.389+0000 7fa92379d740 4 client.28774 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:55:58.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.389+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fa91401ac10 con 0x564049e4f9e0 2024-03-26T08:55:58.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.390+0000 7fa91a59c640 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564049e4f9e0 0x564049e51e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.391+0000 7fa91a59c640 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564049e4f9e0 0x564049e51e00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bf85d0c4d68b9de server_cookie=52849fe5382a0415 in_seq=0 out_seq=0 2024-03-26T08:55:58.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.391+0000 7fa901ffb640 10 client.28774 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa901ffb640 1 -- 172.21.15.7:0/1371501881 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fa91401ac10 con 0x564049e4f9e0 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa901ffb640 10 client.28774 handle_command_reply: tid=0 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 client.28774 shutdown 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 msgr2=0x564049de1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049de1df0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa91c1018d0 tx=0x7fa910004730 comp rx=0 tx=0).stop 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 shutdown_connections 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564049ddc400 0x564049de8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x564049e4f9e0 0x564049e51e00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564049dd7290 0x564049de1df0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 --2- 172.21.15.7:0/1371501881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564049dd68a0 0x564049de18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 >> 172.21.15.7:0/1371501881 conn(0x564049dd1e10 msgr2=0x564049df5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa901ffb640 0 client.28774 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:55:58.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 shutdown_connections 2024-03-26T08:55:58.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 1 -- 172.21.15.7:0/1371501881 wait complete. 2024-03-26T08:55:58.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.392+0000 7fa92379d740 20 client.28774 trim_cache size 0 max 16384 2024-03-26T08:55:58.391 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 -- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9040a2880 msgr2=0x7fa9040a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9040a2880 0x7fa9040a4d40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa91c197de0 tx=0x7fa910005f90 comp rx=0 tx=0).stop 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 -- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c100800 msgr2=0x7fa91c1968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c100800 0x7fa91c1968d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa90c005ab0 tx=0x7fa90c02c630 comp rx=0 tx=0).stop 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 -- 172.21.15.7:0/3489097090 shutdown_connections 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa9040a2880 0x7fa9040a4d40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa91c101c40 0x7fa91c19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa91c1011c0 0x7fa91c196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 --2- 172.21.15.7:0/3489097090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91c100800 0x7fa91c1968d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 -- 172.21.15.7:0/3489097090 >> 172.21.15.7:0/3489097090 conn(0x7fa91c0fb2b0 msgr2=0x7fa91c104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 -- 172.21.15.7:0/3489097090 shutdown_connections 2024-03-26T08:55:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.393+0000 7fa920bba640 1 -- 172.21.15.7:0/3489097090 wait complete. 2024-03-26T08:55:58.401 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:55:58.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:55:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.524+0000 7fd03b7cd640 1 -- 172.21.15.7:0/3028422511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 msgr2=0x7fd034101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.524+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/3028422511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 0x7fd034101c30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd028009900 tx=0x7fd02802bc30 comp rx=0 tx=0).stop 2024-03-26T08:55:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.524+0000 7fd03b7cd640 1 -- 172.21.15.7:0/3028422511 shutdown_connections 2024-03-26T08:55:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.524+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/3028422511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034102170 0x7fd034108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.524+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/3028422511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 0x7fd034101c30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.524+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/3028422511 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0341005c0 0x7fd0341009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.524+0000 7fd03b7cd640 1 -- 172.21.15.7:0/3028422511 >> 172.21.15.7:0/3028422511 conn(0x7fd0340fbb30 msgr2=0x7fd0340fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.525+0000 7fd03b7cd640 1 -- 172.21.15.7:0/3028422511 shutdown_connections 2024-03-26T08:55:58.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.525+0000 7fd03b7cd640 1 -- 172.21.15.7:0/3028422511 wait complete. 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd03b7cd640 1 Processor -- start 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd03b7cd640 1 -- start start 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd03b7cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0341005c0 0x7fd0341a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd03b7cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 0x7fd0341a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd03b7cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034102170 0x7fd0341a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd03b7cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03410c2a0 con 0x7fd0341017b0 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd03b7cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03410c120 con 0x7fd034102170 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd03b7cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03410c420 con 0x7fd0341005c0 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd038d68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 0x7fd0341a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd038d68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 0x7fd0341a2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57664/0 (socket says 172.21.15.7:57664) 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd038d68640 1 -- 172.21.15.7:0/811789322 learned_addr learned my addr 172.21.15.7:0/811789322 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd039569640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0341005c0 0x7fd0341a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd039d6a640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034102170 0x7fd0341a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd038d68640 1 -- 172.21.15.7:0/811789322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0341005c0 msgr2=0x7fd0341a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd038d68640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0341005c0 0x7fd0341a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd038d68640 1 -- 172.21.15.7:0/811789322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034102170 msgr2=0x7fd0341a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd038d68640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034102170 0x7fd0341a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd038d68640 1 -- 172.21.15.7:0/811789322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0341a5c30 con 0x7fd0341017b0 2024-03-26T08:55:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.526+0000 7fd038d68640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 0x7fd0341a2ce0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd028000c00 tx=0x7fd02802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.527+0000 7fd0327fc640 1 -- 172.21.15.7:0/811789322 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028043020 con 0x7fd0341017b0 2024-03-26T08:55:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.527+0000 7fd0327fc640 1 -- 172.21.15.7:0/811789322 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0280054e0 con 0x7fd0341017b0 2024-03-26T08:55:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.527+0000 7fd03b7cd640 1 -- 172.21.15.7:0/811789322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0341a5ec0 con 0x7fd0341017b0 2024-03-26T08:55:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.527+0000 7fd0327fc640 1 -- 172.21.15.7:0/811789322 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028003c40 con 0x7fd0341017b0 2024-03-26T08:55:58.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.527+0000 7fd03b7cd640 1 -- 172.21.15.7:0/811789322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0341a6380 con 0x7fd0341017b0 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd0327fc640 1 -- 172.21.15.7:0/811789322 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd028046020 con 0x7fd0341017b0 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd03e3b0740 1 Processor -- start 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd03e3b0740 1 -- start start 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d4fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fd080 0x561d5d501b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d504680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d5d4bf610 con 0x561d5d5021f0 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d5d4623c0 con 0x561d5d4fc690 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d5d40ad60 con 0x561d5d4fd080 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd038d68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d504680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd039d6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d4fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd039569640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fd080 0x561d5d501b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd038d68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d504680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57674/0 (socket says 172.21.15.7:57674) 2024-03-26T08:55:58.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd038d68640 1 -- 172.21.15.7:0/645195815 learned_addr learned my addr 172.21.15.7:0/645195815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd0327fc640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd01c0a2800 0x7fd01c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd039d6a640 1 -- 172.21.15.7:0/645195815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fd080 msgr2=0x561d5d501b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd039d6a640 1 --2- 172.21.15.7:0/645195815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fd080 0x561d5d501b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd039d6a640 1 -- 172.21.15.7:0/645195815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 msgr2=0x561d5d504680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd039d6a640 1 --2- 172.21.15.7:0/645195815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d504680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd039d6a640 1 -- 172.21.15.7:0/645195815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d5d1763f0 con 0x561d5d4fc690 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd038d68640 1 --2- 172.21.15.7:0/645195815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d504680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.530+0000 7fd0327fc640 1 -- 172.21.15.7:0/811789322 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0280f0aa0 con 0x7fd0341017b0 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.531+0000 7fd039d6a640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd01c0a2800 0x7fd01c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.531+0000 7fd039d6a640 1 --2- 172.21.15.7:0/645195815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d4fcab0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd024009fd0 tx=0x7fd024011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332d77710f441175 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.531+0000 7fd01b7fe640 1 -- 172.21.15.7:0/645195815 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024017070 con 0x561d5d4fc690 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.531+0000 7fd01b7fe640 1 -- 172.21.15.7:0/645195815 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd02400fc20 con 0x561d5d4fc690 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.531+0000 7fd01b7fe640 1 -- 172.21.15.7:0/645195815 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02401ba20 con 0x561d5d4fc690 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 -- 172.21.15.7:0/645195815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 msgr2=0x561d5d4fcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd039d6a640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd01c0a2800 0x7fd01c0a4cc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd024022000 tx=0x7fd024023000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/645195815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d4fcab0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd024009fd0 tx=0x7fd024011920 comp rx=0 tx=0).stop 2024-03-26T08:55:58.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 -- 172.21.15.7:0/645195815 shutdown_connections 2024-03-26T08:55:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/645195815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d504680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/645195815 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fd080 0x561d5d501b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/645195815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d4fcab0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 -- 172.21.15.7:0/645195815 >> 172.21.15.7:0/645195815 conn(0x561d5d4f7c00 msgr2=0x561d5d4f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 -- 172.21.15.7:0/645195815 shutdown_connections 2024-03-26T08:55:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 -- 172.21.15.7:0/645195815 wait complete. 2024-03-26T08:55:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 -- start start 2024-03-26T08:55:58.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.532+0000 7fd03e3b0740 1 Processor -- start 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.533+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d506a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.533+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d506fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d5021f0 0x561d5d50dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d5d41e240 con 0x561d5d4fd080 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d5d45e460 con 0x561d5d4fc690 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d5d5051a0 con 0x561d5d5021f0 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd03e3b0740 10 client.0 _ll_register_callbacks cb 0x561d5d41dc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039569640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d506a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039569640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d506a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40512/0 (socket says 172.21.15.7:40512) 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039d6a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d5021f0 0x561d5d50dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039569640 1 -- 172.21.15.7:0/2348246619 learned_addr learned my addr 172.21.15.7:0/2348246619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd038d68640 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d506fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039569640 1 -- 172.21.15.7:0/2348246619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d5021f0 msgr2=0x561d5d50dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039569640 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d5021f0 0x561d5d50dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039569640 1 -- 172.21.15.7:0/2348246619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 msgr2=0x561d5d506fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039569640 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d506fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039569640 1 -- 172.21.15.7:0/2348246619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d5d2407d0 con 0x561d5d4fc690 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd039569640 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d506a60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd034101610 tx=0x7fd02c012aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40576ebd56f2023a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd018ff9640 1 -- 172.21.15.7:0/2348246619 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c004030 con 0x561d5d4fc690 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd018ff9640 1 -- 172.21.15.7:0/2348246619 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd02c0041d0 con 0x561d5d4fc690 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.534+0000 7fd03e3b0740 1 -- 172.21.15.7:0/2348246619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd02400fc20 con 0x561d5d4fc690 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd018ff9640 1 -- 172.21.15.7:0/2348246619 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd02c0043d0 con 0x561d5d4fc690 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd03e3b0740 10 client.28790 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd03e3b0740 1 -- 172.21.15.7:0/2348246619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561d5d5113c0 con 0x561d5d4fc690 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd018ff9640 1 -- 172.21.15.7:0/2348246619 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd02400fc20 con 0x561d5d4fc690 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd018ff9640 1 -- 172.21.15.7:0/2348246619 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd02c00f8c0 con 0x561d5d4fc690 2024-03-26T08:55:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd03e3b0740 10 client.28790 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd03e3b0740 10 client.28790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd03e3b0740 20 client.28790 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561d5d512a70 0x561d5d514ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd03e3b0740 4 client.28790 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd03e3b0740 1 -- 172.21.15.7:0/2348246619 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d5d46f900 con 0x561d5d512a70 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.535+0000 7fd038d68640 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561d5d512a70 0x561d5d514ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd018ff9640 10 client.28790 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd038d68640 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561d5d512a70 0x561d5d514ed0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=950e2eb0580034f5 server_cookie=29c745908c33bb6c in_seq=0 out_seq=0 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd018ff9640 1 -- 172.21.15.7:0/2348246619 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561d5d46f900 con 0x561d5d512a70 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd018ff9640 10 client.28790 handle_command_reply: tid=0 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd03e3b0740 1 client.28790 shutdown 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd03e3b0740 1 -- 172.21.15.7:0/2348246619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 msgr2=0x561d5d506a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d506a60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd034101610 tx=0x7fd02c012aa0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd03e3b0740 1 -- 172.21.15.7:0/2348246619 shutdown_connections 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561d5d512a70 0x561d5d514ed0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d5021f0 0x561d5d50dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd018ff9640 0 client.28790 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d506fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/2348246619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fc690 0x561d5d506a60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.536+0000 7fd03e3b0740 1 -- 172.21.15.7:0/2348246619 >> 172.21.15.7:0/2348246619 conn(0x561d5d4f7c00 msgr2=0x561d5d4f6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.537+0000 7fd03e3b0740 1 -- 172.21.15.7:0/2348246619 shutdown_connections 2024-03-26T08:55:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.537+0000 7fd03e3b0740 1 -- 172.21.15.7:0/2348246619 wait complete. 2024-03-26T08:55:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.537+0000 7fd03e3b0740 20 client.28790 trim_cache size 0 max 16384 2024-03-26T08:55:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd03e3b0740 1 Processor -- start 2024-03-26T08:55:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd03e3b0740 1 -- start start 2024-03-26T08:55:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d506080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d50dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d5021f0 0x561d5d5065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0341b15e0 con 0x561d5d4fd080 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0341af0e0 con 0x561d5d5021f0 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0341ad160 con 0x561d5d4fc690 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd038d68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d50dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd039d6a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d5021f0 0x561d5d5065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd038d68640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d50dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57690/0 (socket says 172.21.15.7:57690) 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd039569640 1 --2- 172.21.15.7:0/1308376968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d506080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd038d68640 1 -- 172.21.15.7:0/1308376968 learned_addr learned my addr 172.21.15.7:0/1308376968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd038d68640 1 -- 172.21.15.7:0/1308376968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 msgr2=0x561d5d506080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd038d68640 1 --2- 172.21.15.7:0/1308376968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d506080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd038d68640 1 -- 172.21.15.7:0/1308376968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d5021f0 msgr2=0x561d5d5065c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd038d68640 1 --2- 172.21.15.7:0/1308376968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d5021f0 0x561d5d5065c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd038d68640 1 -- 172.21.15.7:0/1308376968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d5d4fc2d0 con 0x561d5d4fd080 2024-03-26T08:55:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.551+0000 7fd039569640 1 --2- 172.21.15.7:0/1308376968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d506080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.552+0000 7fd038d68640 1 --2- 172.21.15.7:0/1308376968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d50dfa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd02802ceb0 tx=0x7fd0280ee470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb4da4155f95cdb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.552+0000 7fd01affd640 1 -- 172.21.15.7:0/1308376968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280bd210 con 0x561d5d4fd080 2024-03-26T08:55:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.552+0000 7fd01affd640 1 -- 172.21.15.7:0/1308376968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd028004250 con 0x561d5d4fd080 2024-03-26T08:55:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.552+0000 7fd01affd640 1 -- 172.21.15.7:0/1308376968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280bcda0 con 0x561d5d4fd080 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.552+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1308376968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 msgr2=0x561d5d50dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.552+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1308376968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d50dfa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd02802ceb0 tx=0x7fd0280ee470 comp rx=0 tx=0).stop 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1308376968 shutdown_connections 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1308376968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d5021f0 0x561d5d5065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1308376968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d4fd080 0x561d5d50dfa0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1308376968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d506080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1308376968 >> 172.21.15.7:0/1308376968 conn(0x561d5d4f7c00 msgr2=0x561d5d4fb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1308376968 shutdown_connections 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1308376968 wait complete. 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 -- start start 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 Processor -- start 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d50dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fd080 0x561d5d50c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d50c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0341ad760 con 0x561d5d5021f0 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0341aea80 con 0x561d5d4fd080 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0341addc0 con 0x561d5d4fc690 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd03e3b0740 10 client.0 _ll_register_callbacks cb 0x7fd02c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd039d6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d50c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd039d6a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d50c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57704/0 (socket says 172.21.15.7:57704) 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd038d68640 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fd080 0x561d5d50c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd039d6a640 1 -- 172.21.15.7:0/1460662698 learned_addr learned my addr 172.21.15.7:0/1460662698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.553+0000 7fd039569640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d50dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.554+0000 7fd039d6a640 1 -- 172.21.15.7:0/1460662698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 msgr2=0x561d5d50dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.554+0000 7fd039d6a640 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d50dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.554+0000 7fd039d6a640 1 -- 172.21.15.7:0/1460662698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fd080 msgr2=0x561d5d50c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.554+0000 7fd039d6a640 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fd080 0x561d5d50c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.554+0000 7fd039d6a640 1 -- 172.21.15.7:0/1460662698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d5d4f8970 con 0x561d5d5021f0 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.554+0000 7fd039569640 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d50dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:55:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.555+0000 7fd039d6a640 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d50c930 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x561d5d4fd8e0 tx=0x7fd02401bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12eeaa059a308e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.555+0000 7fd018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.555+0000 7fd018ff9640 1 -- 172.21.15.7:0/1460662698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024038060 con 0x561d5d5021f0 2024-03-26T08:55:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.555+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1460662698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d5d469c50 con 0x561d5d5021f0 2024-03-26T08:55:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.555+0000 7fd018ff9640 1 -- 172.21.15.7:0/1460662698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd02401b830 con 0x561d5d5021f0 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd018ff9640 1 -- 172.21.15.7:0/1460662698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd024017040 con 0x561d5d5021f0 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd03e3b0740 10 client.19011 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1460662698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561d5d56f0e0 con 0x561d5d5021f0 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd018ff9640 1 -- 172.21.15.7:0/1460662698 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561d5d469c50 con 0x561d5d5021f0 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd018ff9640 1 -- 172.21.15.7:0/1460662698 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd024026440 con 0x561d5d5021f0 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd03e3b0740 10 client.19011 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd03e3b0740 10 client.19011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd03e3b0740 20 client.19011 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561d5d56f890 0x561d5d571cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd03e3b0740 4 client.19011 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1460662698 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561d5d3e7640 con 0x561d5d56f890 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.556+0000 7fd039569640 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561d5d56f890 0x561d5d571cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd018ff9640 10 client.19011 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd039569640 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561d5d56f890 0x561d5d571cb0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d02d7855f1bc30e3 server_cookie=f707ed7ef6d6733d in_seq=0 out_seq=0 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd018ff9640 1 -- 172.21.15.7:0/1460662698 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561d5d3e7640 con 0x561d5d56f890 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd018ff9640 10 client.19011 handle_command_reply: tid=0 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 client.19011 shutdown 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1460662698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 msgr2=0x561d5d50c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d50c930 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x561d5d4fd8e0 tx=0x7fd02401bf20 comp rx=0 tx=0).stop 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1460662698 shutdown_connections 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561d5d56f890 0x561d5d571cb0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561d5d5021f0 0x561d5d50c930 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd018ff9640 0 client.19011 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d5d4fd080 0x561d5d50c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 --2- 172.21.15.7:0/1460662698 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561d5d4fc690 0x561d5d50dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1460662698 >> 172.21.15.7:0/1460662698 conn(0x561d5d4f7c00 msgr2=0x561d5d519790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1460662698 shutdown_connections 2024-03-26T08:55:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 1 -- 172.21.15.7:0/1460662698 wait complete. 2024-03-26T08:55:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.557+0000 7fd03e3b0740 20 client.19011 trim_cache size 0 max 16384 2024-03-26T08:55:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 -- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd01c0a2800 msgr2=0x7fd01c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd01c0a2800 0x7fd01c0a4cc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd024022000 tx=0x7fd024023000 comp rx=0 tx=0).stop 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 -- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 msgr2=0x7fd0341a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 0x7fd0341a2ce0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd028000c00 tx=0x7fd02802c650 comp rx=0 tx=0).stop 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 -- 172.21.15.7:0/811789322 shutdown_connections 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd01c0a2800 0x7fd01c0a4cc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034102170 0x7fd0341a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0341017b0 0x7fd0341a2ce0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 --2- 172.21.15.7:0/811789322 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd0341005c0 0x7fd0341a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 -- 172.21.15.7:0/811789322 >> 172.21.15.7:0/811789322 conn(0x7fd0340fbb30 msgr2=0x7fd0340fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 -- 172.21.15.7:0/811789322 shutdown_connections 2024-03-26T08:55:58.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:58.558+0000 7fd03b7cd640 1 -- 172.21.15.7:0/811789322 wait complete. 2024-03-26T08:55:58.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:58 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2031425761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:55:59.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:58 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2031425761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:55:59.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:58 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2031425761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:55:59.573 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:55:59.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:55:59.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:55:59.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.727+0000 7fe70bfff640 1 -- 172.21.15.7:0/1391480942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 msgr2=0x7fe70c101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.727+0000 7fe70bfff640 1 --2- 172.21.15.7:0/1391480942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 0x7fe70c101740 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe704009ca0 tx=0x7fe70402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 -- 172.21.15.7:0/1391480942 shutdown_connections 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 --2- 172.21.15.7:0/1391480942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 0x7fe70c101740 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 --2- 172.21.15.7:0/1391480942 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe70c100500 0x7fe70c100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 --2- 172.21.15.7:0/1391480942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ff7a0 0x7fe70c0ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 -- 172.21.15.7:0/1391480942 >> 172.21.15.7:0/1391480942 conn(0x7fe70c0fb0d0 msgr2=0x7fe70c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 -- 172.21.15.7:0/1391480942 shutdown_connections 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 -- 172.21.15.7:0/1391480942 wait complete. 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 Processor -- start 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 -- start start 2024-03-26T08:55:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ff7a0 0x7fe70c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe70c100500 0x7fe70c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 0x7fe70c1a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c10f380 con 0x7fe70c100ec0 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c10f200 con 0x7fe70c0ff7a0 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.728+0000 7fe70bfff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c10f500 con 0x7fe70c100500 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 0x7fe70c1a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 0x7fe70c1a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57730/0 (socket says 172.21.15.7:57730) 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70b7fe640 1 -- 172.21.15.7:0/4288412529 learned_addr learned my addr 172.21.15.7:0/4288412529 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe703fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe70c100500 0x7fe70c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70affd640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ff7a0 0x7fe70c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70b7fe640 1 -- 172.21.15.7:0/4288412529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe70c100500 msgr2=0x7fe70c199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe70c100500 0x7fe70c199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70b7fe640 1 -- 172.21.15.7:0/4288412529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ff7a0 msgr2=0x7fe70c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ff7a0 0x7fe70c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70b7fe640 1 -- 172.21.15.7:0/4288412529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c1a15f0 con 0x7fe70c100ec0 2024-03-26T08:55:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 0x7fe70c1a0f80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe70402c700 tx=0x7fe704032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe708ff9640 1 -- 172.21.15.7:0/4288412529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704041020 con 0x7fe70c100ec0 2024-03-26T08:55:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe708ff9640 1 -- 172.21.15.7:0/4288412529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe704034cd0 con 0x7fe70c100ec0 2024-03-26T08:55:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe708ff9640 1 -- 172.21.15.7:0/4288412529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704039040 con 0x7fe70c100ec0 2024-03-26T08:55:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70bfff640 1 -- 172.21.15.7:0/4288412529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c1a1880 con 0x7fe70c100ec0 2024-03-26T08:55:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.729+0000 7fe70bfff640 1 -- 172.21.15.7:0/4288412529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70c1a1d70 con 0x7fe70c100ec0 2024-03-26T08:55:59.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe708ff9640 1 -- 172.21.15.7:0/4288412529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe704044020 con 0x7fe70c100ec0 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe713fea740 1 Processor -- start 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe713fea740 1 -- start start 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe713fea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c130b30 0x556c3c130f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe713fea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c136020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe708ff9640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6f80a2800 0x7fe6f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe713fea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c136690 0x556c3c138b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe713fea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3c070c90 con 0x556c3c136690 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe713fea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3c096070 con 0x556c3c131520 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe713fea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3c03ec80 con 0x556c3c130b30 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe703fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c130b30 0x556c3c130f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe703fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c130b30 0x556c3c130f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36578/0 (socket says 172.21.15.7:36578) 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe703fff640 1 -- 172.21.15.7:0/2109385654 learned_addr learned my addr 172.21.15.7:0/2109385654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe70affd640 1 --2- 172.21.15.7:0/2109385654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c136020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe703fff640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6f80a2800 0x7fe6f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe708ff9640 1 -- 172.21.15.7:0/4288412529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe7040f0920 con 0x7fe70c100ec0 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.732+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/2109385654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c136690 0x556c3c138b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe703fff640 1 -- 172.21.15.7:0/2109385654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 msgr2=0x556c3c136020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe703fff640 1 --2- 172.21.15.7:0/2109385654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c136020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe703fff640 1 -- 172.21.15.7:0/2109385654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c136690 msgr2=0x556c3c138b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe703fff640 1 --2- 172.21.15.7:0/2109385654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c136690 0x556c3c138b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe703fff640 1 -- 172.21.15.7:0/2109385654 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c3c036d10 con 0x556c3c130b30 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe703fff640 1 --2- 172.21.15.7:0/2109385654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c130b30 0x556c3c130f50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4011ab0 tx=0x7fe6f4011eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=409503576bc9c4fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe703fff640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6f80a2800 0x7fe6f80a4cc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400f790 tx=0x7fe6f4018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe7017fa640 1 -- 172.21.15.7:0/2109385654 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4011050 con 0x556c3c130b30 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe7017fa640 1 -- 172.21.15.7:0/2109385654 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f40111f0 con 0x556c3c130b30 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.733+0000 7fe7017fa640 1 -- 172.21.15.7:0/2109385654 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40035f0 con 0x556c3c130b30 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 -- 172.21.15.7:0/2109385654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c130b30 msgr2=0x556c3c130f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 --2- 172.21.15.7:0/2109385654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c130b30 0x556c3c130f50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4011ab0 tx=0x7fe6f4011eb0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 -- 172.21.15.7:0/2109385654 shutdown_connections 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 --2- 172.21.15.7:0/2109385654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c136690 0x556c3c138b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 --2- 172.21.15.7:0/2109385654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c136020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 --2- 172.21.15.7:0/2109385654 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c130b30 0x556c3c130f50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 -- 172.21.15.7:0/2109385654 >> 172.21.15.7:0/2109385654 conn(0x556c3c0f91a0 msgr2=0x556c3c12c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 -- 172.21.15.7:0/2109385654 shutdown_connections 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 -- 172.21.15.7:0/2109385654 wait complete. 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 -- start start 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.734+0000 7fe713fea740 1 Processor -- start 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe713fea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c130b30 0x556c3c13af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe713fea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c131520 0x556c3c13b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe713fea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c1424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe713fea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3c0520d0 con 0x556c3c131520 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe713fea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3c092110 con 0x556c3c130b30 2024-03-26T08:55:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe713fea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c3c1395e0 con 0x556c3c136690 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe713fea740 10 client.0 _ll_register_callbacks cb 0x556c3c08de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe703fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c131520 0x556c3c13b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe703fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c131520 0x556c3c13b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57742/0 (socket says 172.21.15.7:57742) 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe703fff640 1 -- 172.21.15.7:0/3129817644 learned_addr learned my addr 172.21.15.7:0/3129817644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c1424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe70affd640 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c130b30 0x556c3c13af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe703fff640 1 -- 172.21.15.7:0/3129817644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 msgr2=0x556c3c1424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe703fff640 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c1424a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe703fff640 1 -- 172.21.15.7:0/3129817644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c130b30 msgr2=0x556c3c13af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe703fff640 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c130b30 0x556c3c13af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe703fff640 1 -- 172.21.15.7:0/3129817644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c3be727e0 con 0x556c3c131520 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c1424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe703fff640 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c131520 0x556c3c13b460 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009560 tx=0x7fe6f4005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d31ac563b8d28a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.735+0000 7fe6f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe6f27fc640 1 -- 172.21.15.7:0/3129817644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4011050 con 0x556c3c131520 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe713fea740 1 -- 172.21.15.7:0/3129817644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6f40111f0 con 0x556c3c131520 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe6f27fc640 1 -- 172.21.15.7:0/3129817644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6f400ed20 con 0x556c3c131520 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe6f27fc640 1 -- 172.21.15.7:0/3129817644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6f4004030 con 0x556c3c131520 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe713fea740 10 client.19023 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe713fea740 1 -- 172.21.15.7:0/3129817644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556c3bed2f70 con 0x556c3c131520 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe6f27fc640 1 -- 172.21.15.7:0/3129817644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6f40111f0 con 0x556c3c131520 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe6f27fc640 1 -- 172.21.15.7:0/3129817644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe6f4004b60 con 0x556c3c131520 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe713fea740 10 client.19023 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe713fea740 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe713fea740 20 client.19023 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe713fea740 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c3c146c10 0x556c3c149000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe713fea740 4 client.19023 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe713fea740 1 -- 172.21.15.7:0/3129817644 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c3c0a35b0 con 0x556c3c146c10 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.736+0000 7fe70affd640 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c3c146c10 0x556c3c149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.737+0000 7fe70affd640 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c3c146c10 0x556c3c149000 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db2cb3b13ae3371e server_cookie=6dad485ce0203aa1 in_seq=0 out_seq=0 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.737+0000 7fe6f27fc640 10 client.19023 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe6f27fc640 1 -- 172.21.15.7:0/3129817644 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556c3c0a35b0 con 0x556c3c146c10 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe6f27fc640 10 client.19023 handle_command_reply: tid=0 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 client.19023 shutdown 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 -- 172.21.15.7:0/3129817644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c131520 msgr2=0x556c3c13b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c131520 0x556c3c13b460 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009560 tx=0x7fe6f4005540 comp rx=0 tx=0).stop 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 -- 172.21.15.7:0/3129817644 shutdown_connections 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c3c146c10 0x556c3c149000 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c1424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c131520 0x556c3c13b460 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe6f27fc640 0 client.19023 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 --2- 172.21.15.7:0/3129817644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c130b30 0x556c3c13af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 -- 172.21.15.7:0/3129817644 >> 172.21.15.7:0/3129817644 conn(0x556c3c0f91a0 msgr2=0x556c3c12b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 -- 172.21.15.7:0/3129817644 shutdown_connections 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 1 -- 172.21.15.7:0/3129817644 wait complete. 2024-03-26T08:55:59.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.738+0000 7fe713fea740 20 client.19023 trim_cache size 0 max 16384 2024-03-26T08:55:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.757+0000 7fe713fea740 1 Processor -- start 2024-03-26T08:55:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.757+0000 7fe713fea740 1 -- start start 2024-03-26T08:55:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.757+0000 7fe713fea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c12af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.757+0000 7fe713fea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.757+0000 7fe713fea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c12b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.757+0000 7fe713fea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c1acfb0 con 0x556c3c130b30 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.757+0000 7fe713fea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c1aaab0 con 0x556c3c131520 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe713fea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c1a8b30 con 0x556c3c136690 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c12b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c12b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36594/0 (socket says 172.21.15.7:36594) 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70b7fe640 1 -- 172.21.15.7:0/1707483920 learned_addr learned my addr 172.21.15.7:0/1707483920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70affd640 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c12af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe703fff640 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70b7fe640 1 -- 172.21.15.7:0/1707483920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 msgr2=0x556c3c142460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70b7fe640 1 -- 172.21.15.7:0/1707483920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 msgr2=0x556c3c12af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c12af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70b7fe640 1 -- 172.21.15.7:0/1707483920 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c3bd6a170 con 0x556c3c136690 2024-03-26T08:55:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.758+0000 7fe70affd640 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c12af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.759+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c12b490 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe7040b9c40 tx=0x7fe704005c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2deafcec858d591 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.759+0000 7fe700ff9640 1 -- 172.21.15.7:0/1707483920 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040bd2c0 con 0x556c3c136690 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.759+0000 7fe700ff9640 1 -- 172.21.15.7:0/1707483920 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7040c1410 con 0x556c3c136690 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.759+0000 7fe700ff9640 1 -- 172.21.15.7:0/1707483920 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040017c0 con 0x556c3c136690 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 -- 172.21.15.7:0/1707483920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 msgr2=0x556c3c12b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c12b490 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe7040b9c40 tx=0x7fe704005c30 comp rx=0 tx=0).stop 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 -- 172.21.15.7:0/1707483920 shutdown_connections 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c12b490 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 --2- 172.21.15.7:0/1707483920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c12af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 -- 172.21.15.7:0/1707483920 >> 172.21.15.7:0/1707483920 conn(0x556c3c0f91a0 msgr2=0x556c3c12ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 -- 172.21.15.7:0/1707483920 shutdown_connections 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 -- 172.21.15.7:0/1707483920 wait complete. 2024-03-26T08:55:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 -- start start 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 Processor -- start 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c141d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c140320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c1a9130 con 0x556c3c130b30 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c1aa450 con 0x556c3c131520 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c1a9790 con 0x556c3c136690 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.760+0000 7fe713fea740 10 client.0 _ll_register_callbacks cb 0x7fe6fc00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe703fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c141d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57760/0 (socket says 172.21.15.7:57760) 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe703fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40596/0 (socket says 172.21.15.7:40596) 2024-03-26T08:55:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70affd640 1 -- 172.21.15.7:0/828498437 learned_addr learned my addr 172.21.15.7:0/828498437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70affd640 1 -- 172.21.15.7:0/828498437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 msgr2=0x556c3c140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70affd640 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70affd640 1 -- 172.21.15.7:0/828498437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 msgr2=0x556c3c142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70affd640 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70affd640 1 -- 172.21.15.7:0/828498437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c3c140960 con 0x556c3c130b30 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe703fff640 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70b7fe640 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe70affd640 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c141d10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc000c00 tx=0x7fe6fc00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb67164dcc53001 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe6f27fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe6f27fc640 1 -- 172.21.15.7:0/828498437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc0037f0 con 0x556c3c130b30 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe6f27fc640 1 -- 172.21.15.7:0/828498437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6fc003990 con 0x556c3c130b30 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe713fea740 1 -- 172.21.15.7:0/828498437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7040c1410 con 0x556c3c130b30 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe6f27fc640 1 -- 172.21.15.7:0/828498437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6fc0202f0 con 0x556c3c130b30 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe713fea740 10 client.19029 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.761+0000 7fe713fea740 1 -- 172.21.15.7:0/828498437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556c3c1a2f00 con 0x556c3c130b30 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.762+0000 7fe6f27fc640 1 -- 172.21.15.7:0/828498437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7040c1410 con 0x556c3c130b30 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.762+0000 7fe6f27fc640 1 -- 172.21.15.7:0/828498437 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe6fc029420 con 0x556c3c130b30 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.762+0000 7fe713fea740 10 client.19029 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.762+0000 7fe713fea740 10 client.19029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.762+0000 7fe713fea740 20 client.19029 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.762+0000 7fe713fea740 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c3c1a3890 0x556c3c1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.762+0000 7fe713fea740 4 client.19029 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.762+0000 7fe713fea740 1 -- 172.21.15.7:0/828498437 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556c3c09d900 con 0x556c3c1a3890 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.762+0000 7fe703fff640 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c3c1a3890 0x556c3c1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.763+0000 7fe703fff640 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c3c1a3890 0x556c3c1a5cb0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58fd38c74d41e20c server_cookie=3a65b13aeb19388 in_seq=0 out_seq=0 2024-03-26T08:55:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.763+0000 7fe6f27fc640 10 client.19029 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:59.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.764+0000 7fe6f27fc640 1 -- 172.21.15.7:0/828498437 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556c3c09d900 con 0x556c3c1a3890 2024-03-26T08:55:59.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.764+0000 7fe6f27fc640 10 client.19029 handle_command_reply: tid=0 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.764+0000 7fe713fea740 1 client.19029 shutdown 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 -- 172.21.15.7:0/828498437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 msgr2=0x556c3c141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c141d10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc000c00 tx=0x7fe6fc00bfb0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 -- 172.21.15.7:0/828498437 shutdown_connections 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556c3c1a3890 0x556c3c1a5cb0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe6f27fc640 0 client.19029 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556c3c136690 0x556c3c140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c3c131520 0x556c3c142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 --2- 172.21.15.7:0/828498437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556c3c130b30 0x556c3c141d10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 -- 172.21.15.7:0/828498437 >> 172.21.15.7:0/828498437 conn(0x556c3c0f91a0 msgr2=0x556c3c14d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 -- 172.21.15.7:0/828498437 shutdown_connections 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 1 -- 172.21.15.7:0/828498437 wait complete. 2024-03-26T08:55:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe713fea740 20 client.19029 trim_cache size 0 max 16384 2024-03-26T08:55:59.767 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:55:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:55:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "476e5651-c3d2-422d-8803-82645fbb2e1b", 2024-03-26T08:55:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:55:59.767 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe70bfff640 1 -- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6f80a2800 msgr2=0x7fe6f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe70bfff640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6f80a2800 0x7fe6f80a4cc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400f790 tx=0x7fe6f4018000 comp rx=0 tx=0).stop 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe70bfff640 1 -- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 msgr2=0x7fe70c1a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe70bfff640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 0x7fe70c1a0f80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe70402c700 tx=0x7fe704032f20 comp rx=0 tx=0).stop 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe70bfff640 1 -- 172.21.15.7:0/4288412529 shutdown_connections 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe70bfff640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe6f80a2800 0x7fe6f80a4cc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe70bfff640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c100ec0 0x7fe70c1a0f80 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe70bfff640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe70c100500 0x7fe70c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.765+0000 7fe70bfff640 1 --2- 172.21.15.7:0/4288412529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70c0ff7a0 0x7fe70c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.766+0000 7fe70bfff640 1 -- 172.21.15.7:0/4288412529 >> 172.21.15.7:0/4288412529 conn(0x7fe70c0fb0d0 msgr2=0x7fe70c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.766+0000 7fe70bfff640 1 -- 172.21.15.7:0/4288412529 shutdown_connections 2024-03-26T08:55:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.766+0000 7fe70bfff640 1 -- 172.21.15.7:0/4288412529 wait complete. 2024-03-26T08:55:59.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:55:59.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:55:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.927+0000 7f3f67627640 1 -- 172.21.15.7:0/3216996922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60104a70 msgr2=0x7f3f60106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.928+0000 7f3f67627640 1 --2- 172.21.15.7:0/3216996922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60104a70 0x7f3f60106e60 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009900 tx=0x7f3f5002bc30 comp rx=0 tx=0).stop 2024-03-26T08:55:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.929+0000 7f3f67627640 1 -- 172.21.15.7:0/3216996922 shutdown_connections 2024-03-26T08:55:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.929+0000 7f3f67627640 1 --2- 172.21.15.7:0/3216996922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f601073a0 0x7f3f601097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.929+0000 7f3f67627640 1 --2- 172.21.15.7:0/3216996922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60104a70 0x7f3f60106e60 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.929+0000 7f3f67627640 1 --2- 172.21.15.7:0/3216996922 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f60102140 0x7f3f60104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.929+0000 7f3f67627640 1 -- 172.21.15.7:0/3216996922 >> 172.21.15.7:0/3216996922 conn(0x7f3f600fbb30 msgr2=0x7f3f600fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.929+0000 7f3f67627640 1 -- 172.21.15.7:0/3216996922 shutdown_connections 2024-03-26T08:55:59.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.929+0000 7f3f67627640 1 -- 172.21.15.7:0/3216996922 wait complete. 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.929+0000 7f3f67627640 1 Processor -- start 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.929+0000 7f3f67627640 1 -- start start 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f67627640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60102140 0x7f3f601977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f67627640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f60104a70 0x7f3f60197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f67627640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f601073a0 0x7f3f6019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f67627640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6010d330 con 0x7f3f60102140 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f67627640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6010d1b0 con 0x7f3f60104a70 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f67627640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6010d4b0 con 0x7f3f601073a0 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f653c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60102140 0x7f3f601977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f653c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60102140 0x7f3f601977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57778/0 (socket says 172.21.15.7:57778) 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f653c3640 1 -- 172.21.15.7:0/2235922138 learned_addr learned my addr 172.21.15.7:0/2235922138 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f60104a70 0x7f3f60197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f65bc4640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f601073a0 0x7f3f6019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f653c3640 1 -- 172.21.15.7:0/2235922138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f601073a0 msgr2=0x7f3f6019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f653c3640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f601073a0 0x7f3f6019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f653c3640 1 -- 172.21.15.7:0/2235922138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f60104a70 msgr2=0x7f3f60197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f653c3640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f60104a70 0x7f3f60197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f653c3640 1 -- 172.21.15.7:0/2235922138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6019f410 con 0x7f3f60102140 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.930+0000 7f3f653c3640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60102140 0x7f3f601977f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009f50 tx=0x7f3f5c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.931+0000 7f3f567fc640 1 -- 172.21.15.7:0/2235922138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c020070 con 0x7f3f60102140 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.931+0000 7f3f567fc640 1 -- 172.21.15.7:0/2235922138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f5c013d00 con 0x7f3f60102140 2024-03-26T08:55:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.931+0000 7f3f67627640 1 -- 172.21.15.7:0/2235922138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f600fff90 con 0x7f3f60102140 2024-03-26T08:55:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.931+0000 7f3f567fc640 1 -- 172.21.15.7:0/2235922138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c01bbb0 con 0x7f3f60102140 2024-03-26T08:55:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.931+0000 7f3f67627640 1 -- 172.21.15.7:0/2235922138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f601004b0 con 0x7f3f60102140 2024-03-26T08:55:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.934+0000 7f3f567fc640 1 -- 172.21.15.7:0/2235922138 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3f5c023070 con 0x7f3f60102140 2024-03-26T08:55:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.935+0000 7f3f6a20f740 1 Processor -- start 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.935+0000 7f3f6a20f740 1 -- start start 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882ba5b0 0x55c8882ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882c0110 0x55c8882c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8882173d0 con 0x55c8882bafa0 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8881bfad0 con 0x55c8882c0110 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c888213470 con 0x55c8882ba5b0 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f567fc640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f4c0a2800 0x7f3f4c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f64bc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882c0110 0x55c8882c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f653c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f64bc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882c0110 0x55c8882c25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40612/0 (socket says 172.21.15.7:40612) 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f65bc4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882ba5b0 0x55c8882ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f64bc2640 1 -- 172.21.15.7:0/316562358 learned_addr learned my addr 172.21.15.7:0/316562358 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f567fc640 1 -- 172.21.15.7:0/2235922138 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3f5c00f3c0 con 0x7f3f60102140 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f64bc2640 1 -- 172.21.15.7:0/316562358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882ba5b0 msgr2=0x55c8882ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/316562358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882ba5b0 0x55c8882ba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f64bc2640 1 -- 172.21.15.7:0/316562358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 msgr2=0x55c8882bfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/316562358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882bfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f65bc4640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f4c0a2800 0x7f3f4c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.936+0000 7f3f64bc2640 1 -- 172.21.15.7:0/316562358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c887ff57d0 con 0x55c8882c0110 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.937+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/316562358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882c0110 0x55c8882c25a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3f600694d0 tx=0x7f3f50009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bdce6e0b8004eaf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.937+0000 7f3f65bc4640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f4c0a2800 0x7f3f4c0a4cc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55c8882bb800 tx=0x7f3f58004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.937+0000 7f3f4b7fe640 1 -- 172.21.15.7:0/316562358 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50042070 con 0x55c8882c0110 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.937+0000 7f3f4b7fe640 1 -- 172.21.15.7:0/316562358 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f5002cd00 con 0x55c8882c0110 2024-03-26T08:55:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.937+0000 7f3f4b7fe640 1 -- 172.21.15.7:0/316562358 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50005120 con 0x55c8882c0110 2024-03-26T08:55:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 -- 172.21.15.7:0/316562358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882c0110 msgr2=0x55c8882c25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/316562358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882c0110 0x55c8882c25a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3f600694d0 tx=0x7f3f50009560 comp rx=0 tx=0).stop 2024-03-26T08:55:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 -- 172.21.15.7:0/316562358 shutdown_connections 2024-03-26T08:55:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/316562358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882c0110 0x55c8882c25a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/316562358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/316562358 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882ba5b0 0x55c8882ba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 -- 172.21.15.7:0/316562358 >> 172.21.15.7:0/316562358 conn(0x55c8882b5b20 msgr2=0x55c8882b5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 -- 172.21.15.7:0/316562358 shutdown_connections 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 -- 172.21.15.7:0/316562358 wait complete. 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 -- start start 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.938+0000 7f3f6a20f740 1 Processor -- start 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 0x55c8882c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8882c3060 con 0x55c8882ba5b0 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8881d3060 con 0x55c8882bafa0 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8882c31e0 con 0x55c8882c0110 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f6a20f740 10 client.0 _ll_register_callbacks cb 0x55c8881b56a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f653c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f65bc4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f64bc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 0x55c8882c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f65bc4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:36640/0 (socket says 172.21.15.7:36640) 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.939+0000 7f3f65bc4640 1 -- 172.21.15.7:0/2099761243 learned_addr learned my addr 172.21.15.7:0/2099761243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.940+0000 7f3f653c3640 1 -- 172.21.15.7:0/2099761243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 msgr2=0x55c8882cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.940+0000 7f3f653c3640 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.940+0000 7f3f653c3640 1 -- 172.21.15.7:0/2099761243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 msgr2=0x55c8882c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.940+0000 7f3f653c3640 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 0x55c8882c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.940+0000 7f3f653c3640 1 -- 172.21.15.7:0/2099761243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c887f2b3f0 con 0x55c8882ba5b0 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.940+0000 7f3f653c3640 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c49a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x55c8882bba10 tx=0x7f3f5c09f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5618532e559a35da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.940+0000 7f3f48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.940+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2099761243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c09b040 con 0x55c8882ba5b0 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.940+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2099761243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3f5c0a7860 con 0x55c8882ba5b0 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.941+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2099761243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f5002cd00 con 0x55c8882ba5b0 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.941+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2099761243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f5c0ce070 con 0x55c8882ba5b0 2024-03-26T08:55:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.941+0000 7f3f6a20f740 10 client.19041 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.941+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2099761243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c887eeb0f0 con 0x55c8882ba5b0 2024-03-26T08:55:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.941+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2099761243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f5002cd00 con 0x55c8882ba5b0 2024-03-26T08:55:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2099761243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3f5c0b8220 con 0x55c8882ba5b0 2024-03-26T08:55:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f6a20f740 10 client.19041 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f6a20f740 10 client.19041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f6a20f740 20 client.19041 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c8882d0450 0x55c8882d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f6a20f740 4 client.19041 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:55:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2099761243 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c888224910 con 0x55c8882d0450 2024-03-26T08:55:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c8882d0450 0x55c8882d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f48ff9640 10 client.19041 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.942+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c8882d0450 0x55c8882d28c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2577fc973af0f261 server_cookie=3b42a5053bb6442d in_seq=0 out_seq=0 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.943+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2099761243 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c888224910 con 0x55c8882d0450 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.943+0000 7f3f48ff9640 10 client.19041 handle_command_reply: tid=0 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.943+0000 7f3f6a20f740 1 client.19041 shutdown 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2099761243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 msgr2=0x55c8882c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c49a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x55c8882bba10 tx=0x7f3f5c09f3d0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2099761243 shutdown_connections 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c8882d0450 0x55c8882d28c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 0x55c8882c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2099761243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c49a0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2099761243 >> 172.21.15.7:0/2099761243 conn(0x55c8882b5b20 msgr2=0x55c8882ac530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f48ff9640 0 client.19041 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2099761243 shutdown_connections 2024-03-26T08:55:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2099761243 wait complete. 2024-03-26T08:55:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.944+0000 7f3f6a20f740 20 client.19041 trim_cache size 0 max 16384 2024-03-26T08:55:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.965+0000 7f3f6a20f740 1 Processor -- start 2024-03-26T08:55:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.965+0000 7f3f6a20f740 1 -- start start 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.965+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.965+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 0x55c8882cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.965+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.965+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f601a8ab0 con 0x55c8882ba5b0 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.965+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f601a65b0 con 0x55c8882bafa0 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.965+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f601a4630 con 0x55c8882c0110 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.965+0000 7f3f653c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.966+0000 7f3f653c3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57806/0 (socket says 172.21.15.7:57806) 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.966+0000 7f3f65bc4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.966+0000 7f3f653c3640 1 -- 172.21.15.7:0/34416712 learned_addr learned my addr 172.21.15.7:0/34416712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.966+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/34416712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 0x55c8882cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.966+0000 7f3f653c3640 1 -- 172.21.15.7:0/34416712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 msgr2=0x55c8882b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.966+0000 7f3f653c3640 1 --2- 172.21.15.7:0/34416712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.966+0000 7f3f653c3640 1 -- 172.21.15.7:0/34416712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 msgr2=0x55c8882cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.966+0000 7f3f653c3640 1 --2- 172.21.15.7:0/34416712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 0x55c8882cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.966+0000 7f3f653c3640 1 -- 172.21.15.7:0/34416712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8882bb380 con 0x55c8882ba5b0 2024-03-26T08:55:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.967+0000 7f3f653c3640 1 --2- 172.21.15.7:0/34416712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c4e40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c09ba60 tx=0x7f3f5c009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd539601c000c2f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.967+0000 7f3f4affd640 1 -- 172.21.15.7:0/34416712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c09b040 con 0x55c8882ba5b0 2024-03-26T08:55:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.967+0000 7f3f4affd640 1 -- 172.21.15.7:0/34416712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f5c09f600 con 0x55c8882ba5b0 2024-03-26T08:55:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.967+0000 7f3f4affd640 1 -- 172.21.15.7:0/34416712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c0b8080 con 0x55c8882ba5b0 2024-03-26T08:55:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.968+0000 7f3f6a20f740 1 -- 172.21.15.7:0/34416712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 msgr2=0x55c8882c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.968+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/34416712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c4e40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c09ba60 tx=0x7f3f5c009790 comp rx=0 tx=0).stop 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.968+0000 7f3f6a20f740 1 -- 172.21.15.7:0/34416712 shutdown_connections 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.968+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/34416712 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.968+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/34416712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882bafa0 0x55c8882cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.968+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/34416712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882ba5b0 0x55c8882c4e40 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.968+0000 7f3f6a20f740 1 -- 172.21.15.7:0/34416712 >> 172.21.15.7:0/34416712 conn(0x55c8882b5b20 msgr2=0x55c8882b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 -- 172.21.15.7:0/34416712 shutdown_connections 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 -- 172.21.15.7:0/34416712 wait complete. 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 -- start start 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 Processor -- start 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882ba5b0 0x55c8882cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882ca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f601a4c30 con 0x55c8882bafa0 2024-03-26T08:55:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f601a5f50 con 0x55c8882ba5b0 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.969+0000 7f3f6a20f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f601a5290 con 0x55c8882c0110 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f6a20f740 10 client.0 _ll_register_callbacks cb 0x7f3f5c00f5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f64bc2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f653c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882ba5b0 0x55c8882cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f64bc2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57814/0 (socket says 172.21.15.7:57814) 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f64bc2640 1 -- 172.21.15.7:0/2011729360 learned_addr learned my addr 172.21.15.7:0/2011729360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f65bc4640 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f64bc2640 1 -- 172.21.15.7:0/2011729360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 msgr2=0x55c8882ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f64bc2640 1 -- 172.21.15.7:0/2011729360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882ba5b0 msgr2=0x55c8882cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882ba5b0 0x55c8882cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f64bc2640 1 -- 172.21.15.7:0/2011729360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8882cad50 con 0x55c8882bafa0 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f653c3640 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882ba5b0 0x55c8882cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f64bc2640 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882ca250 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009230 tx=0x7f3f50005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382d11cf4d28ea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:55:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2011729360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50042040 con 0x55c8882bafa0 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2011729360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f5c09f600 con 0x55c8882bafa0 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.970+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2011729360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3f50004240 con 0x55c8882bafa0 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2011729360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f50004440 con 0x55c8882bafa0 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f6a20f740 10 client.19047 fetch_fsmap learned FSMap version 25 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2011729360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c88832cb60 con 0x55c8882bafa0 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2011729360 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f5c09f600 con 0x55c8882bafa0 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2011729360 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3f50056030 con 0x55c8882bafa0 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f6a20f740 10 client.19047 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f6a20f740 10 client.19047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f6a20f740 20 client.19047 populate_metadata read hostname 'smithi007' 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c88832d170 0x55c88832f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f6a20f740 4 client.19047 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2011729360 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c88821ec60 con 0x55c88832d170 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f653c3640 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c88832d170 0x55c88832f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:55:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f653c3640 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c88832d170 0x55c88832f560 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae13e9510228a42a server_cookie=16cac594d9b8b51a in_seq=0 out_seq=0 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.971+0000 7f3f48ff9640 10 client.19047 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f48ff9640 1 -- 172.21.15.7:0/2011729360 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c88821ec60 con 0x55c88832d170 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f48ff9640 10 client.19047 handle_command_reply: tid=0 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 client.19047 shutdown 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2011729360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 msgr2=0x55c8882ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882ca250 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009230 tx=0x7f3f50005fd0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2011729360 shutdown_connections 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55c88832d170 0x55c88832f560 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55c8882c0110 0x55c8882ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f48ff9640 0 client.19047 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c8882bafa0 0x55c8882ca250 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 --2- 172.21.15.7:0/2011729360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c8882ba5b0 0x55c8882cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2011729360 >> 172.21.15.7:0/2011729360 conn(0x55c8882b5b20 msgr2=0x55c8882b9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2011729360 shutdown_connections 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 1 -- 172.21.15.7:0/2011729360 wait complete. 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.972+0000 7f3f6a20f740 20 client.19047 trim_cache size 0 max 16384 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stdout: "476e5651-c3d2-422d-8803-82645fbb2e1b": { 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "476e5651-c3d2-422d-8803-82645fbb2e1b", 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:55:59.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:55:59.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:55:59.976 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:55:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 -- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f4c0a2800 msgr2=0x7f3f4c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f4c0a2800 0x7f3f4c0a4cc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55c8882bb800 tx=0x7f3f58004ab0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 -- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60102140 msgr2=0x7f3f601977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60102140 0x7f3f601977f0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009f50 tx=0x7f3f5c00ee50 comp rx=0 tx=0).stop 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 -- 172.21.15.7:0/2235922138 shutdown_connections 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3f4c0a2800 0x7f3f4c0a4cc0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3f601073a0 0x7f3f6019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f60104a70 0x7f3f60197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 --2- 172.21.15.7:0/2235922138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60102140 0x7f3f601977f0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 -- 172.21.15.7:0/2235922138 >> 172.21.15.7:0/2235922138 conn(0x7f3f600fbb30 msgr2=0x7f3f601053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 -- 172.21.15.7:0/2235922138 shutdown_connections 2024-03-26T08:55:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:55:59.973+0000 7f3f67627640 1 -- 172.21.15.7:0/2235922138 wait complete. 2024-03-26T08:55:59.981 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:55:59 smithi007 ceph-mon[32462]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-03-26T08:55:59.982 INFO:tasks.cephfs.filesystem:scrub status for tag:476e5651-c3d2-422d-8803-82645fbb2e1b - {'path': '/', 'tag': '476e5651-c3d2-422d-8803-82645fbb2e1b', 'options': 'recursive,force'} 2024-03-26T08:55:59.998 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:55:59 smithi016 ceph-mon[35851]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-03-26T08:56:00.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:55:59 smithi044 ceph-mon[36461]: pgmap v767: 97 pgs: 97 active+clean; 1.6 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 686 op/s 2024-03-26T08:56:00.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:00 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:56:00.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:00 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:00.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:00 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:56:00.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:00 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:56:00.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:00 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:00.999 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:00 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:01.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:00 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:56:01.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:00 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:01.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:00 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:56:01.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:00 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:56:01.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:00 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:01.048 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:00 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:01.071 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:00 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:56:01.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:00 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:01.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:00 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:56:01.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:00 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:56:01.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:00 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:01.072 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:00 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:01.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 810.0 s, 1681.8 tps, lat 59.315 ms stddev 76.637 2024-03-26T08:56:02.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:01 smithi016 ceph-mon[35851]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.2 MiB/s wr, 631 op/s 2024-03-26T08:56:02.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:01 smithi007 ceph-mon[32462]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.2 MiB/s wr, 631 op/s 2024-03-26T08:56:02.321 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:01 smithi044 ceph-mon[36461]: pgmap v768: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.2 MiB/s wr, 631 op/s 2024-03-26T08:56:04.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:03 smithi016 ceph-mon[35851]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.0 MiB/s wr, 739 op/s 2024-03-26T08:56:04.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:03 smithi007 ceph-mon[32462]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.0 MiB/s wr, 739 op/s 2024-03-26T08:56:04.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:03 smithi044 ceph-mon[36461]: pgmap v769: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.0 MiB/s wr, 739 op/s 2024-03-26T08:56:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 815.0 s, 1364.6 tps, lat 71.883 ms stddev 79.173 2024-03-26T08:56:06.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:05 smithi016 ceph-mon[35851]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.6 MiB/s wr, 710 op/s 2024-03-26T08:56:06.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:05 smithi007 ceph-mon[32462]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.6 MiB/s wr, 710 op/s 2024-03-26T08:56:06.321 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:05 smithi044 ceph-mon[36461]: pgmap v770: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.6 MiB/s wr, 710 op/s 2024-03-26T08:56:07.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:06 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:56:07.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:06 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:56:07.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:06 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:56:07.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:06 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:56:07.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:06 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:56:07.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:06 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:56:08.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:07 smithi016 ceph-mon[35851]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.5 MiB/s wr, 656 op/s 2024-03-26T08:56:08.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:07 smithi007 ceph-mon[32462]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.5 MiB/s wr, 656 op/s 2024-03-26T08:56:08.321 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:07 smithi044 ceph-mon[36461]: pgmap v771: 97 pgs: 97 active+clean; 1.6 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.5 MiB/s wr, 656 op/s 2024-03-26T08:56:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.240+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1541302739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c40fffc0 msgr2=0x7f60c41003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.240+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1541302739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c40fffc0 0x7f60c41003e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009900 tx=0x7f60b402bcd0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.240+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1541302739 shutdown_connections 2024-03-26T08:56:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.240+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1541302739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c41012c0 0x7f60c4101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.240+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1541302739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60c4100920 0x7f60c4100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.240+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1541302739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c40fffc0 0x7f60c41003e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.240+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1541302739 >> 172.21.15.7:0/1541302739 conn(0x7f60c40fbb40 msgr2=0x7f60c40fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.241+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1541302739 shutdown_connections 2024-03-26T08:56:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.241+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1541302739 wait complete. 2024-03-26T08:56:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.241+0000 7f60c9e8a640 1 Processor -- start 2024-03-26T08:56:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c9e8a640 1 -- start start 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c9e8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60c40fffc0 0x7f60c4199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c9e8a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c4100920 0x7f60c419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c9e8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c41012c0 0x7f60c41a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c9e8a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410f520 con 0x7f60c4100920 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c9e8a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410f3a0 con 0x7f60c41012c0 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c9e8a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c410f6a0 con 0x7f60c40fffc0 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c4100920 0x7f60c419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60c40fffc0 0x7f60c4199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c41012c0 0x7f60c41a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c4100920 0x7f60c419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32804/0 (socket says 172.21.15.7:32804) 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c2ffd640 1 -- 172.21.15.7:0/1167090864 learned_addr learned my addr 172.21.15.7:0/1167090864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.242+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c41012c0 0x7f60c41a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56454/0 (socket says 172.21.15.7:56454) 2024-03-26T08:56:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c3fff640 1 -- 172.21.15.7:0/1167090864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60c40fffc0 msgr2=0x7f60c4199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c3fff640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60c40fffc0 0x7f60c4199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c3fff640 1 -- 172.21.15.7:0/1167090864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c4100920 msgr2=0x7f60c419a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c3fff640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c4100920 0x7f60c419a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c3fff640 1 -- 172.21.15.7:0/1167090864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c41a17f0 con 0x7f60c41012c0 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c3fff640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c41012c0 0x7f60c41a1180 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009fd0 tx=0x7f60bc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c0ff9640 1 -- 172.21.15.7:0/1167090864 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc00ce20 con 0x7f60c41012c0 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c0ff9640 1 -- 172.21.15.7:0/1167090864 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60bc005020 con 0x7f60c41012c0 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1167090864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c41a1ae0 con 0x7f60c41012c0 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c0ff9640 1 -- 172.21.15.7:0/1167090864 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc010720 con 0x7f60c41012c0 2024-03-26T08:56:09.243 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.243+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1167090864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c41a1fd0 con 0x7f60c41012c0 2024-03-26T08:56:09.245 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.245+0000 7f60c0ff9640 1 -- 172.21.15.7:0/1167090864 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f60bc0109c0 con 0x7f60c41012c0 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60cca71740 1 Processor -- start 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c0ff9640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f60ac0a2870 0x7f60ac0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60cca71740 1 -- start start 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c0ff9640 1 -- 172.21.15.7:0/1167090864 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f60bc0cf0d0 con 0x7f60c41012c0 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60cca71740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b08fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60cca71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 0x55e2b0901e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60cca71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b0904910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60cca71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b0861b10 con 0x55e2b08fd310 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60cca71740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b080a900 con 0x55e2b08fc920 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60cca71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b085dbb0 con 0x55e2b0902480 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b08fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f60ac0a2870 0x7f60ac0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.246 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b0904910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b08fcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56464/0 (socket says 172.21.15.7:56464) 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c37fe640 1 -- 172.21.15.7:0/3827026433 learned_addr learned my addr 172.21.15.7:0/3827026433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/3827026433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 0x55e2b0901e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c3fff640 1 -- 172.21.15.7:0/3827026433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 msgr2=0x55e2b08fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c3fff640 1 --2- 172.21.15.7:0/3827026433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b08fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c3fff640 1 -- 172.21.15.7:0/3827026433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 msgr2=0x55e2b0901e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c3fff640 1 --2- 172.21.15.7:0/3827026433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 0x55e2b0901e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c3fff640 1 -- 172.21.15.7:0/3827026433 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2b0576210 con 0x55e2b0902480 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c37fe640 1 --2- 172.21.15.7:0/3827026433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b08fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.246+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f60ac0a2870 0x7f60ac0a4d30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f60b80098e0 tx=0x7f60b8009c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.247+0000 7f60c3fff640 1 --2- 172.21.15.7:0/3827026433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b0904910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f60bc09b770 tx=0x7f60bc010cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61321047b6cef71c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.247+0000 7f60a9ffb640 1 -- 172.21.15.7:0/3827026433 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc09ba70 con 0x55e2b0902480 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.247+0000 7f60a9ffb640 1 -- 172.21.15.7:0/3827026433 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60bc0afe50 con 0x55e2b0902480 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.247+0000 7f60a9ffb640 1 -- 172.21.15.7:0/3827026433 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc0a3d80 con 0x55e2b0902480 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.248+0000 7f60cca71740 1 -- 172.21.15.7:0/3827026433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 msgr2=0x55e2b0904910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.248+0000 7f60cca71740 1 --2- 172.21.15.7:0/3827026433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b0904910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f60bc09b770 tx=0x7f60bc010cb0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.247 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.248+0000 7f60cca71740 1 -- 172.21.15.7:0/3827026433 shutdown_connections 2024-03-26T08:56:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.248+0000 7f60cca71740 1 --2- 172.21.15.7:0/3827026433 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b0904910 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.248+0000 7f60cca71740 1 --2- 172.21.15.7:0/3827026433 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 0x55e2b0901e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.248+0000 7f60cca71740 1 --2- 172.21.15.7:0/3827026433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b08fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.248+0000 7f60cca71740 1 -- 172.21.15.7:0/3827026433 >> 172.21.15.7:0/3827026433 conn(0x55e2b08f7e90 msgr2=0x55e2b08f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:09.248 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.248+0000 7f60cca71740 1 -- 172.21.15.7:0/3827026433 shutdown_connections 2024-03-26T08:56:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.249+0000 7f60cca71740 1 -- 172.21.15.7:0/3827026433 wait complete. 2024-03-26T08:56:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.249+0000 7f60cca71740 1 -- start start 2024-03-26T08:56:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60cca71740 1 Processor -- start 2024-03-26T08:56:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60cca71740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b0906ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60cca71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 0x55e2b0907220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60cca71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b090e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b0906ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60cca71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b0905430 con 0x55e2b08fd310 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b0906ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56466/0 (socket says 172.21.15.7:56466) 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60c37fe640 1 -- 172.21.15.7:0/873643323 learned_addr learned my addr 172.21.15.7:0/873643323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 0x55e2b0907220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b081dd50 con 0x55e2b08fc920 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2b09055b0 con 0x55e2b0902480 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60c3fff640 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b090e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.250+0000 7f60cca71740 10 client.0 _ll_register_callbacks cb 0x55e2b081d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60c37fe640 1 -- 172.21.15.7:0/873643323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 msgr2=0x55e2b090e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60c37fe640 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b090e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60c37fe640 1 -- 172.21.15.7:0/873643323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 msgr2=0x55e2b0907220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60c37fe640 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 0x55e2b0907220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60c37fe640 1 -- 172.21.15.7:0/873643323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2b0642620 con 0x55e2b08fc920 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60c3fff640 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b090e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 0x55e2b0907220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60c37fe640 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b0906ce0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f60b402c210 tx=0x7f60b4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a535ed745b81b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60a2ffd640 1 -- 172.21.15.7:0/873643323 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4005d50 con 0x55e2b08fc920 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60bc0afe50 con 0x55e2b08fc920 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60a2ffd640 1 -- 172.21.15.7:0/873643323 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f60b4005230 con 0x55e2b08fc920 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60a2ffd640 1 -- 172.21.15.7:0/873643323 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b4005430 con 0x55e2b08fc920 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60cca71740 10 client.28844 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.251+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e2b0911590 con 0x55e2b08fc920 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60a2ffd640 1 -- 172.21.15.7:0/873643323 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60bc0afe50 con 0x55e2b08fc920 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60a2ffd640 1 -- 172.21.15.7:0/873643323 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f60b4003aa0 con 0x55e2b08fc920 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60cca71740 10 client.28844 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60cca71740 10 client.28844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60cca71740 20 client.28844 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60cca71740 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e2b0912bb0 0x55e2b0915000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60cca71740 4 client.28844 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2b086f050 con 0x55e2b0912bb0 2024-03-26T08:56:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e2b0912bb0 0x55e2b0915000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e2b0912bb0 0x55e2b0915000 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef305921e199f150 server_cookie=edb9368ffe793b9b in_seq=0 out_seq=0 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.252+0000 7f60a2ffd640 10 client.28844 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60a2ffd640 1 -- 172.21.15.7:0/873643323 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e2b086f050 con 0x55e2b0912bb0 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60a2ffd640 10 client.28844 handle_command_reply: tid=0 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 client.28844 shutdown 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 msgr2=0x55e2b0906ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b0906ce0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f60b402c210 tx=0x7f60b4004270 comp rx=0 tx=0).stop 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 shutdown_connections 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e2b0912bb0 0x55e2b0915000 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b0902480 0x55e2b090e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60a2ffd640 0 client.28844 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fd310 0x55e2b0907220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 --2- 172.21.15.7:0/873643323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fc920 0x55e2b0906ce0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 >> 172.21.15.7:0/873643323 conn(0x55e2b08f7e90 msgr2=0x55e2b08f7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 shutdown_connections 2024-03-26T08:56:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.253+0000 7f60cca71740 1 -- 172.21.15.7:0/873643323 wait complete. 2024-03-26T08:56:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.254+0000 7f60cca71740 20 client.28844 trim_cache size 0 max 16384 2024-03-26T08:56:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.271+0000 7f60cca71740 1 Processor -- start 2024-03-26T08:56:09.269 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.272+0000 7f60cca71740 1 -- start start 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.272+0000 7f60cca71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fc920 0x55e2b08fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.272+0000 7f60cca71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fd310 0x55e2b090e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.272+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fc920 0x55e2b08fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.272+0000 7f60cca71740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b0902480 0x55e2b0905950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.272+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fc920 0x55e2b08fddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32832/0 (socket says 172.21.15.7:32832) 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.272+0000 7f60c2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fd310 0x55e2b090e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60c37fe640 1 -- 172.21.15.7:0/3181913672 learned_addr learned my addr 172.21.15.7:0/3181913672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60cca71740 1 -- 172.21.15.7:0/3181913672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41ad1d0 con 0x55e2b08fc920 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60cca71740 1 -- 172.21.15.7:0/3181913672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41aacd0 con 0x55e2b0902480 2024-03-26T08:56:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60cca71740 1 -- 172.21.15.7:0/3181913672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41a8d50 con 0x55e2b08fd310 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60c3fff640 1 --2- 172.21.15.7:0/3181913672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b0902480 0x55e2b0905950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60c2ffd640 1 -- 172.21.15.7:0/3181913672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b0902480 msgr2=0x55e2b0905950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/3181913672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b0902480 0x55e2b0905950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60c2ffd640 1 -- 172.21.15.7:0/3181913672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fc920 msgr2=0x55e2b08fddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/3181913672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fc920 0x55e2b08fddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60c2ffd640 1 -- 172.21.15.7:0/3181913672 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2b08f7c00 con 0x55e2b08fd310 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60c3fff640 1 --2- 172.21.15.7:0/3181913672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b0902480 0x55e2b0905950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/3181913672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fd310 0x55e2b090e1b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55e2b08fdd80 tx=0x7f60b8020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd0d3c0d0bab0a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60a97fa640 1 -- 172.21.15.7:0/3181913672 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b802c110 con 0x55e2b08fd310 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60a97fa640 1 -- 172.21.15.7:0/3181913672 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b802c2b0 con 0x55e2b08fd310 2024-03-26T08:56:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.273+0000 7f60a97fa640 1 -- 172.21.15.7:0/3181913672 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b802cd40 con 0x55e2b08fd310 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 -- 172.21.15.7:0/3181913672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fd310 msgr2=0x55e2b090e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 --2- 172.21.15.7:0/3181913672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fd310 0x55e2b090e1b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55e2b08fdd80 tx=0x7f60b8020000 comp rx=0 tx=0).stop 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 -- 172.21.15.7:0/3181913672 shutdown_connections 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 --2- 172.21.15.7:0/3181913672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b0902480 0x55e2b0905950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 --2- 172.21.15.7:0/3181913672 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fd310 0x55e2b090e1b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 --2- 172.21.15.7:0/3181913672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b08fc920 0x55e2b08fddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 -- 172.21.15.7:0/3181913672 >> 172.21.15.7:0/3181913672 conn(0x55e2b08f7e90 msgr2=0x55e2b08f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 -- 172.21.15.7:0/3181913672 shutdown_connections 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 -- 172.21.15.7:0/3181913672 wait complete. 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.274+0000 7f60cca71740 1 -- start start 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60cca71740 1 Processor -- start 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60cca71740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fc920 0x55e2b090ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60cca71740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fd310 0x55e2b090b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60cca71740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b0902480 0x55e2b096c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60cca71740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41a9350 con 0x55e2b0902480 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60cca71740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41aa670 con 0x55e2b08fd310 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60cca71740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41a99b0 con 0x55e2b08fc920 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60cca71740 10 client.0 _ll_register_callbacks cb 0x7f60b402c670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b0902480 0x55e2b096c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fd310 0x55e2b090b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fc920 0x55e2b090ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b0902480 0x55e2b096c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32842/0 (socket says 172.21.15.7:32842) 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c3fff640 1 -- 172.21.15.7:0/83975250 learned_addr learned my addr 172.21.15.7:0/83975250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c37fe640 1 -- 172.21.15.7:0/83975250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fd310 msgr2=0x55e2b090b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c37fe640 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fd310 0x55e2b090b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c37fe640 1 -- 172.21.15.7:0/83975250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b0902480 msgr2=0x55e2b096c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c37fe640 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b0902480 0x55e2b096c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c37fe640 1 -- 172.21.15.7:0/83975250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2b090e3f0 con 0x55e2b08fc920 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c3fff640 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b0902480 0x55e2b096c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:09.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.275+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fd310 0x55e2b090b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60c37fe640 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fc920 0x55e2b090ae30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f60b402efd0 tx=0x7f60b40479f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5fe72245033f26 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60a2ffd640 1 -- 172.21.15.7:0/83975250 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4005d50 con 0x55e2b08fc920 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60a2ffd640 1 -- 172.21.15.7:0/83975250 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f60b4034440 con 0x55e2b08fc920 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60cca71740 1 -- 172.21.15.7:0/83975250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60b802c2b0 con 0x55e2b08fc920 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60a2ffd640 1 -- 172.21.15.7:0/83975250 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60b4061050 con 0x55e2b08fc920 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60cca71740 10 client.28846 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60cca71740 1 -- 172.21.15.7:0/83975250 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e2b096da80 con 0x55e2b08fc920 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60a2ffd640 1 -- 172.21.15.7:0/83975250 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f60b802c2b0 con 0x55e2b08fc920 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60a2ffd640 1 -- 172.21.15.7:0/83975250 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f60b4034b20 con 0x55e2b08fc920 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60cca71740 10 client.28846 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.276+0000 7f60cca71740 10 client.28846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.277+0000 7f60cca71740 20 client.28846 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:09.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.277+0000 7f60cca71740 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e2b096e370 0x55e2b0970790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.277+0000 7f60cca71740 4 client.28846 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:56:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.277+0000 7f60cca71740 1 -- 172.21.15.7:0/83975250 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e2b08693a0 con 0x55e2b096e370 2024-03-26T08:56:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.277+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e2b096e370 0x55e2b0970790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.277+0000 7f60c2ffd640 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e2b096e370 0x55e2b0970790 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3647fd4c879b7c6 server_cookie=bb2a3f7538b245a4 in_seq=0 out_seq=0 2024-03-26T08:56:09.279 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.277+0000 7f60a2ffd640 10 client.28846 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60a2ffd640 1 -- 172.21.15.7:0/83975250 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e2b08693a0 con 0x55e2b096e370 2024-03-26T08:56:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60a2ffd640 10 client.28846 handle_command_reply: tid=0 2024-03-26T08:56:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60cca71740 1 client.28846 shutdown 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60cca71740 1 -- 172.21.15.7:0/83975250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fc920 msgr2=0x55e2b090ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60cca71740 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fc920 0x55e2b090ae30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f60b402efd0 tx=0x7f60b40479f0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60cca71740 1 -- 172.21.15.7:0/83975250 shutdown_connections 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60cca71740 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55e2b096e370 0x55e2b0970790 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60a2ffd640 0 client.28846 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60cca71740 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2b0902480 0x55e2b096c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60cca71740 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2b08fd310 0x55e2b090b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.292+0000 7f60cca71740 1 --2- 172.21.15.7:0/83975250 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55e2b08fc920 0x55e2b090ae30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.293+0000 7f60cca71740 1 -- 172.21.15.7:0/83975250 >> 172.21.15.7:0/83975250 conn(0x55e2b08f7e90 msgr2=0x55e2b08f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.293+0000 7f60cca71740 1 -- 172.21.15.7:0/83975250 shutdown_connections 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.293+0000 7f60cca71740 1 -- 172.21.15.7:0/83975250 wait complete. 2024-03-26T08:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.293+0000 7f60cca71740 20 client.28846 trim_cache size 0 max 16384 2024-03-26T08:56:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:56:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:56:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:56:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:56:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1167090864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f60ac0a2870 msgr2=0x7f60ac0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f60ac0a2870 0x7f60ac0a4d30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f60b80098e0 tx=0x7f60b8009c20 comp rx=0 tx=0).stop 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1167090864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c41012c0 msgr2=0x7f60c41a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c41012c0 0x7f60c41a1180 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009fd0 tx=0x7f60bc00ef10 comp rx=0 tx=0).stop 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1167090864 shutdown_connections 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f60ac0a2870 0x7f60ac0a4d30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c41012c0 0x7f60c41a1180 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c4100920 0x7f60c419a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 --2- 172.21.15.7:0/1167090864 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f60c40fffc0 0x7f60c4199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1167090864 >> 172.21.15.7:0/1167090864 conn(0x7f60c40fbb40 msgr2=0x7f60c40fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1167090864 shutdown_connections 2024-03-26T08:56:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:09.294+0000 7f60c9e8a640 1 -- 172.21.15.7:0/1167090864 wait complete. 2024-03-26T08:56:10.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:09 smithi016 ceph-mon[35851]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 6.4 MiB/s wr, 733 op/s 2024-03-26T08:56:10.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:09 smithi007 ceph-mon[32462]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 6.4 MiB/s wr, 733 op/s 2024-03-26T08:56:10.321 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:09 smithi044 ceph-mon[36461]: pgmap v772: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 6.4 MiB/s wr, 733 op/s 2024-03-26T08:56:11.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 820.0 s, 1489.8 tps, lat 67.854 ms stddev 72.720 2024-03-26T08:56:12.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:11 smithi016 ceph-mon[35851]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 690 op/s 2024-03-26T08:56:12.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:11 smithi007 ceph-mon[32462]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 690 op/s 2024-03-26T08:56:12.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:11 smithi044 ceph-mon[36461]: pgmap v773: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 690 op/s 2024-03-26T08:56:14.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:13 smithi016 ceph-mon[35851]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 695 op/s 2024-03-26T08:56:14.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:13 smithi007 ceph-mon[32462]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 695 op/s 2024-03-26T08:56:14.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:13 smithi044 ceph-mon[36461]: pgmap v774: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.9 MiB/s wr, 695 op/s 2024-03-26T08:56:16.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 825.0 s, 1999.2 tps, lat 50.088 ms stddev 62.658 2024-03-26T08:56:16.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:15 smithi016 ceph-mon[35851]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.5 MiB/s wr, 598 op/s 2024-03-26T08:56:16.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:15 smithi007 ceph-mon[32462]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.5 MiB/s wr, 598 op/s 2024-03-26T08:56:16.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:15 smithi044 ceph-mon[36461]: pgmap v775: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.5 MiB/s wr, 598 op/s 2024-03-26T08:56:18.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:17 smithi016 ceph-mon[35851]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.0 MiB/s wr, 539 op/s 2024-03-26T08:56:18.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:17 smithi007 ceph-mon[32462]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.0 MiB/s wr, 539 op/s 2024-03-26T08:56:18.321 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:17 smithi044 ceph-mon[36461]: pgmap v776: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.0 MiB/s wr, 539 op/s 2024-03-26T08:56:20.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:19 smithi016 ceph-mon[35851]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.9 MiB/s wr, 669 op/s 2024-03-26T08:56:20.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:19 smithi007 ceph-mon[32462]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.9 MiB/s wr, 669 op/s 2024-03-26T08:56:20.321 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:19 smithi044 ceph-mon[36461]: pgmap v777: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.9 MiB/s wr, 669 op/s 2024-03-26T08:56:21.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 830.0 s, 1488.0 tps, lat 66.594 ms stddev 71.176 2024-03-26T08:56:22.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:21 smithi016 ceph-mon[35851]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 627 op/s 2024-03-26T08:56:22.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:21 smithi007 ceph-mon[32462]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 627 op/s 2024-03-26T08:56:22.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:21 smithi044 ceph-mon[36461]: pgmap v778: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.3 MiB/s wr, 627 op/s 2024-03-26T08:56:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.422+0000 7ff946061640 1 -- 172.21.15.7:0/708995028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401032a0 msgr2=0x7ff940109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.422+0000 7ff946061640 1 --2- 172.21.15.7:0/708995028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401032a0 0x7ff940109b30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff938009ca0 tx=0x7ff93802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.422+0000 7ff946061640 1 -- 172.21.15.7:0/708995028 shutdown_connections 2024-03-26T08:56:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.422+0000 7ff946061640 1 --2- 172.21.15.7:0/708995028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401032a0 0x7ff940109b30 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.422+0000 7ff946061640 1 --2- 172.21.15.7:0/708995028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9401028e0 0x7ff940102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.422+0000 7ff946061640 1 --2- 172.21.15.7:0/708995028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9401016f0 0x7ff940101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.422+0000 7ff946061640 1 -- 172.21.15.7:0/708995028 >> 172.21.15.7:0/708995028 conn(0x7ff9400fcc60 msgr2=0x7ff9400ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.422+0000 7ff946061640 1 -- 172.21.15.7:0/708995028 shutdown_connections 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.422+0000 7ff946061640 1 -- 172.21.15.7:0/708995028 wait complete. 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff946061640 1 Processor -- start 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff946061640 1 -- start start 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff946061640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401016f0 0x7ff94019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff946061640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9401028e0 0x7ff94019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff946061640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9401032a0 0x7ff9401a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff946061640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff940075290 con 0x7ff9401016f0 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff946061640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff940075110 con 0x7ff9401028e0 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401016f0 0x7ff94019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff946061640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff940075410 con 0x7ff9401032a0 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401016f0 0x7ff94019ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59310/0 (socket says 172.21.15.7:59310) 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93f7fe640 1 -- 172.21.15.7:0/1693484124 learned_addr learned my addr 172.21.15.7:0/1693484124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93ffff640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9401032a0 0x7ff9401a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93effd640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9401028e0 0x7ff94019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93f7fe640 1 -- 172.21.15.7:0/1693484124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9401032a0 msgr2=0x7ff9401a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9401032a0 0x7ff9401a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93f7fe640 1 -- 172.21.15.7:0/1693484124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9401028e0 msgr2=0x7ff94019e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9401028e0 0x7ff94019e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.423+0000 7ff93f7fe640 1 -- 172.21.15.7:0/1693484124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9401a59e0 con 0x7ff9401016f0 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.424+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401016f0 0x7ff94019ddc0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff93000e9c0 tx=0x7ff93000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.424+0000 7ff93cff9640 1 -- 172.21.15.7:0/1693484124 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000cd80 con 0x7ff9401016f0 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.424+0000 7ff93cff9640 1 -- 172.21.15.7:0/1693484124 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9300035f0 con 0x7ff9401016f0 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.424+0000 7ff946061640 1 -- 172.21.15.7:0/1693484124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9401a5cd0 con 0x7ff9401016f0 2024-03-26T08:56:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.424+0000 7ff93cff9640 1 -- 172.21.15.7:0/1693484124 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930010690 con 0x7ff9401016f0 2024-03-26T08:56:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.424+0000 7ff946061640 1 -- 172.21.15.7:0/1693484124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401a61d0 con 0x7ff9401016f0 2024-03-26T08:56:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.427+0000 7ff93cff9640 1 -- 172.21.15.7:0/1693484124 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ff930004000 con 0x7ff9401016f0 2024-03-26T08:56:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.429+0000 7ff93cff9640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9280a2740 0x7ff9280a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.429+0000 7ff93cff9640 1 -- 172.21.15.7:0/1693484124 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9300cedb0 con 0x7ff9401016f0 2024-03-26T08:56:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.429+0000 7ff93effd640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9280a2740 0x7ff9280a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff948c51740 1 Processor -- start 2024-03-26T08:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff93effd640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9280a2740 0x7ff9280a4c00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff9340095f0 tx=0x7ff934005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff948c51740 1 -- start start 2024-03-26T08:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff948c51740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9af2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff948c51740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9af7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff948c51740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9afa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff948c51740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d9a57b10 con 0x5626d9af8480 2024-03-26T08:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff948c51740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d9a00900 con 0x5626d9af3310 2024-03-26T08:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff948c51740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d9a53bb0 con 0x5626d9af2920 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff93ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9afa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff93ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9afa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59312/0 (socket says 172.21.15.7:59312) 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff93ffff640 1 -- 172.21.15.7:0/528352541 learned_addr learned my addr 172.21.15.7:0/528352541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9af2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.430+0000 7ff93effd640 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9af7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff93ffff640 1 -- 172.21.15.7:0/528352541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 msgr2=0x5626d9af2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff93ffff640 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9af2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff93ffff640 1 -- 172.21.15.7:0/528352541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 msgr2=0x5626d9af7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff93ffff640 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9af7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff93ffff640 1 -- 172.21.15.7:0/528352541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d976c210 con 0x5626d9af8480 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9af2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff93ffff640 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9afa910 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff93802c700 tx=0x7ff9380095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9f6837a269283a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:22.431 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff925ffb640 1 -- 172.21.15.7:0/528352541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938043070 con 0x5626d9af8480 2024-03-26T08:56:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff925ffb640 1 -- 172.21.15.7:0/528352541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff93802cca0 con 0x5626d9af8480 2024-03-26T08:56:22.432 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.431+0000 7ff925ffb640 1 -- 172.21.15.7:0/528352541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938039040 con 0x5626d9af8480 2024-03-26T08:56:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 -- 172.21.15.7:0/528352541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 msgr2=0x5626d9afa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9afa910 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff93802c700 tx=0x7ff9380095d0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 -- 172.21.15.7:0/528352541 shutdown_connections 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9afa910 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9af7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 --2- 172.21.15.7:0/528352541 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9af2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 -- 172.21.15.7:0/528352541 >> 172.21.15.7:0/528352541 conn(0x5626d9aede90 msgr2=0x5626d9aee290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 -- 172.21.15.7:0/528352541 shutdown_connections 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 -- 172.21.15.7:0/528352541 wait complete. 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.432+0000 7ff948c51740 1 -- start start 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff948c51740 1 Processor -- start 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff948c51740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9afcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff948c51740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9afd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff948c51740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9b040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff948c51740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d9afb3d0 con 0x5626d9af8480 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff948c51740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d9a13d50 con 0x5626d9af3310 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff93f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9afcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff93f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9afcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:32804/0 (socket says 172.21.15.7:32804) 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff93f7fe640 1 -- 172.21.15.7:0/969946320 learned_addr learned my addr 172.21.15.7:0/969946320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff93effd640 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9afd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff948c51740 1 -- 172.21.15.7:0/969946320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626d9afb550 con 0x5626d9af2920 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.433+0000 7ff93ffff640 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9b040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff948c51740 10 client.0 _ll_register_callbacks cb 0x5626d9a13720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff93f7fe640 1 -- 172.21.15.7:0/969946320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 msgr2=0x5626d9afd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9afd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff93f7fe640 1 -- 172.21.15.7:0/969946320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 msgr2=0x5626d9b040c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9b040c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff93f7fe640 1 -- 172.21.15.7:0/969946320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626d9838620 con 0x5626d9af2920 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9afcb40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff93009b6a0 tx=0x7ff93009bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=518634ad3dc15ce0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff91effd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff91effd640 1 -- 172.21.15.7:0/969946320 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9300b3070 con 0x5626d9af2920 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff91effd640 1 -- 172.21.15.7:0/969946320 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9300cc980 con 0x5626d9af2920 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff948c51740 1 -- 172.21.15.7:0/969946320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff93802cca0 con 0x5626d9af2920 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff91effd640 1 -- 172.21.15.7:0/969946320 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff93009ab60 con 0x5626d9af2920 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff948c51740 10 client.28858 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.434+0000 7ff948c51740 1 -- 172.21.15.7:0/969946320 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5626d9b074a0 con 0x5626d9af2920 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff91effd640 1 -- 172.21.15.7:0/969946320 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff93802cca0 con 0x5626d9af2920 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff91effd640 1 -- 172.21.15.7:0/969946320 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff930097460 con 0x5626d9af2920 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff948c51740 10 client.28858 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff948c51740 10 client.28858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff948c51740 20 client.28858 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff948c51740 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5626d9b08b90 0x5626d9b0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff948c51740 4 client.28858 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff948c51740 1 -- 172.21.15.7:0/969946320 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626d9a65050 con 0x5626d9b08b90 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff93effd640 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5626d9b08b90 0x5626d9b0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.435+0000 7ff91effd640 10 client.28858 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.436+0000 7ff93effd640 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5626d9b08b90 0x5626d9b0af80 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed9df016819a2d3 server_cookie=6bc32be405e2d7b in_seq=0 out_seq=0 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.436+0000 7ff91effd640 1 -- 172.21.15.7:0/969946320 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5626d9a65050 con 0x5626d9b08b90 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.436+0000 7ff91effd640 10 client.28858 handle_command_reply: tid=0 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.436+0000 7ff948c51740 1 client.28858 shutdown 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 -- 172.21.15.7:0/969946320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 msgr2=0x5626d9afcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9afcb40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff93009b6a0 tx=0x7ff93009bf50 comp rx=0 tx=0).stop 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 -- 172.21.15.7:0/969946320 shutdown_connections 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5626d9b08b90 0x5626d9b0af80 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af8480 0x5626d9b040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff91effd640 0 client.28858 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9afd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 --2- 172.21.15.7:0/969946320 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af2920 0x5626d9afcb40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 -- 172.21.15.7:0/969946320 >> 172.21.15.7:0/969946320 conn(0x5626d9aede90 msgr2=0x5626d9aed020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 -- 172.21.15.7:0/969946320 shutdown_connections 2024-03-26T08:56:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 1 -- 172.21.15.7:0/969946320 wait complete. 2024-03-26T08:56:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.437+0000 7ff948c51740 20 client.28858 trim_cache size 0 max 16384 2024-03-26T08:56:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff948c51740 1 Processor -- start 2024-03-26T08:56:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff948c51740 1 -- start start 2024-03-26T08:56:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff948c51740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9afc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff948c51740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9b04050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff948c51740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9afcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff948c51740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401b1230 con 0x5626d9af2920 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff948c51740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401aed30 con 0x5626d9af3310 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff948c51740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401acdb0 con 0x5626d9af8480 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff93ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9afcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff93f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9afc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff93ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9afcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:32816/0 (socket says 172.21.15.7:32816) 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff93f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9afc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59326/0 (socket says 172.21.15.7:59326) 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff93ffff640 1 -- 172.21.15.7:0/4085817397 learned_addr learned my addr 172.21.15.7:0/4085817397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.455+0000 7ff93effd640 1 --2- 172.21.15.7:0/4085817397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9b04050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff93ffff640 1 -- 172.21.15.7:0/4085817397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 msgr2=0x5626d9b04050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff93ffff640 1 --2- 172.21.15.7:0/4085817397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9b04050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff93ffff640 1 -- 172.21.15.7:0/4085817397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 msgr2=0x5626d9afc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff93ffff640 1 --2- 172.21.15.7:0/4085817397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9afc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff93ffff640 1 -- 172.21.15.7:0/4085817397 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626d9af4120 con 0x5626d9af8480 2024-03-26T08:56:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/4085817397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9afc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff93ffff640 1 --2- 172.21.15.7:0/4085817397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9afcc90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5626d9aeee00 tx=0x7ff938003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=918fe0a6b391985 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff9257fa640 1 -- 172.21.15.7:0/4085817397 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93804e070 con 0x5626d9af8480 2024-03-26T08:56:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff9257fa640 1 -- 172.21.15.7:0/4085817397 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff938048450 con 0x5626d9af8480 2024-03-26T08:56:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.456+0000 7ff9257fa640 1 -- 172.21.15.7:0/4085817397 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938039040 con 0x5626d9af8480 2024-03-26T08:56:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 -- 172.21.15.7:0/4085817397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 msgr2=0x5626d9afcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 --2- 172.21.15.7:0/4085817397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9afcc90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5626d9aeee00 tx=0x7ff938003970 comp rx=0 tx=0).stop 2024-03-26T08:56:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 -- 172.21.15.7:0/4085817397 shutdown_connections 2024-03-26T08:56:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 --2- 172.21.15.7:0/4085817397 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9afcc90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 --2- 172.21.15.7:0/4085817397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9b04050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 --2- 172.21.15.7:0/4085817397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9afc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 -- 172.21.15.7:0/4085817397 >> 172.21.15.7:0/4085817397 conn(0x5626d9aede90 msgr2=0x5626d9aee270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 -- 172.21.15.7:0/4085817397 shutdown_connections 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 -- 172.21.15.7:0/4085817397 wait complete. 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.457+0000 7ff948c51740 1 -- start start 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff948c51740 1 Processor -- start 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff948c51740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9b03d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff948c51740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9b02380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff948c51740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9b028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff948c51740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401ad3b0 con 0x5626d9af2920 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff948c51740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401ae6d0 con 0x5626d9af3310 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff948c51740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401ada10 con 0x5626d9af8480 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff948c51740 10 client.0 _ll_register_callbacks cb 0x7ff93009b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9b03d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93f7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9b03d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59338/0 (socket says 172.21.15.7:59338) 2024-03-26T08:56:22.461 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93f7fe640 1 -- 172.21.15.7:0/3094308336 learned_addr learned my addr 172.21.15.7:0/3094308336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93effd640 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9b02380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93ffff640 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9b028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93f7fe640 1 -- 172.21.15.7:0/3094308336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 msgr2=0x5626d9b028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9b028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93f7fe640 1 -- 172.21.15.7:0/3094308336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 msgr2=0x5626d9b02380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9b02380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93f7fe640 1 -- 172.21.15.7:0/3094308336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626d9b02f00 con 0x5626d9af2920 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93effd640 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9b02380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff93f7fe640 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9b03d70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff930003f80 tx=0x7ff93009b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5386e1362a3d17f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.458+0000 7ff91effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.459+0000 7ff91effd640 1 -- 172.21.15.7:0/3094308336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9300b3070 con 0x5626d9af2920 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.459+0000 7ff91effd640 1 -- 172.21.15.7:0/3094308336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff930097460 con 0x5626d9af2920 2024-03-26T08:56:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.459+0000 7ff948c51740 1 -- 172.21.15.7:0/3094308336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff938048450 con 0x5626d9af2920 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.459+0000 7ff91effd640 1 -- 172.21.15.7:0/3094308336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff93009a650 con 0x5626d9af2920 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.459+0000 7ff948c51740 10 client.19083 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.459+0000 7ff948c51740 1 -- 172.21.15.7:0/3094308336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5626d9b65340 con 0x5626d9af2920 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.460+0000 7ff91effd640 1 -- 172.21.15.7:0/3094308336 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff938048450 con 0x5626d9af2920 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.460+0000 7ff91effd640 1 -- 172.21.15.7:0/3094308336 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ff9300a0780 con 0x5626d9af2920 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.460+0000 7ff948c51740 10 client.19083 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.460+0000 7ff948c51740 10 client.19083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.460+0000 7ff948c51740 20 client.19083 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.460+0000 7ff948c51740 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5626d9b08b90 0x5626d9b67ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.460+0000 7ff948c51740 4 client.19083 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.460+0000 7ff948c51740 1 -- 172.21.15.7:0/3094308336 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5626d9a5f3a0 con 0x5626d9b08b90 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.461+0000 7ff93effd640 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5626d9b08b90 0x5626d9b67ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.461+0000 7ff91effd640 10 client.19083 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.461+0000 7ff93effd640 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5626d9b08b90 0x5626d9b67ca0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f3a3f83955f9f7 server_cookie=62e351af6883d743 in_seq=0 out_seq=0 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff91effd640 1 -- 172.21.15.7:0/3094308336 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5626d9a5f3a0 con 0x5626d9b08b90 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff91effd640 10 client.19083 handle_command_reply: tid=0 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 client.19083 shutdown 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 -- 172.21.15.7:0/3094308336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 msgr2=0x5626d9b03d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9b03d70 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff930003f80 tx=0x7ff93009b350 comp rx=0 tx=0).stop 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 -- 172.21.15.7:0/3094308336 shutdown_connections 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5626d9b08b90 0x5626d9b67ca0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5626d9af8480 0x5626d9b028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5626d9af3310 0x5626d9b02380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 --2- 172.21.15.7:0/3094308336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626d9af2920 0x5626d9b03d70 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff91effd640 0 client.19083 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 -- 172.21.15.7:0/3094308336 >> 172.21.15.7:0/3094308336 conn(0x5626d9aede90 msgr2=0x5626d9afc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 -- 172.21.15.7:0/3094308336 shutdown_connections 2024-03-26T08:56:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 1 -- 172.21.15.7:0/3094308336 wait complete. 2024-03-26T08:56:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.479+0000 7ff948c51740 20 client.19083 trim_cache size 0 max 16384 2024-03-26T08:56:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:56:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:56:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:56:22.479 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:56:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 -- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9280a2740 msgr2=0x7ff9280a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9280a2740 0x7ff9280a4c00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff9340095f0 tx=0x7ff934005f90 comp rx=0 tx=0).stop 2024-03-26T08:56:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 -- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401016f0 msgr2=0x7ff94019ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401016f0 0x7ff94019ddc0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7ff93000e9c0 tx=0x7ff93000ee80 comp rx=0 tx=0).stop 2024-03-26T08:56:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 -- 172.21.15.7:0/1693484124 shutdown_connections 2024-03-26T08:56:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ff9280a2740 0x7ff9280a4c00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9401032a0 0x7ff9401a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9401028e0 0x7ff94019e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 --2- 172.21.15.7:0/1693484124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9401016f0 0x7ff94019ddc0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 -- 172.21.15.7:0/1693484124 >> 172.21.15.7:0/1693484124 conn(0x7ff9400fcc60 msgr2=0x7ff94010dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 -- 172.21.15.7:0/1693484124 shutdown_connections 2024-03-26T08:56:22.481 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:22.480+0000 7ff946061640 1 -- 172.21.15.7:0/1693484124 wait complete. 2024-03-26T08:56:24.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:24 smithi016 ceph-mon[35851]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 MiB/s wr, 662 op/s 2024-03-26T08:56:24.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:24 smithi007 ceph-mon[32462]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 MiB/s wr, 662 op/s 2024-03-26T08:56:24.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:24 smithi044 ceph-mon[36461]: pgmap v779: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.7 MiB/s wr, 662 op/s 2024-03-26T08:56:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 835.0 s, 1311.0 tps, lat 77.186 ms stddev 83.967 2024-03-26T08:56:26.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:26 smithi007 ceph-mon[32462]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.0 MiB/s wr, 662 op/s 2024-03-26T08:56:26.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:26 smithi044 ceph-mon[36461]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.0 MiB/s wr, 662 op/s 2024-03-26T08:56:26.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:26 smithi016 ceph-mon[35851]: pgmap v780: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.0 MiB/s wr, 662 op/s 2024-03-26T08:56:28.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:28 smithi007 ceph-mon[32462]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 MiB/s wr, 611 op/s 2024-03-26T08:56:28.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:28 smithi044 ceph-mon[36461]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 MiB/s wr, 611 op/s 2024-03-26T08:56:28.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:28 smithi016 ceph-mon[35851]: pgmap v781: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.3 MiB/s wr, 611 op/s 2024-03-26T08:56:29.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:56:29.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:56:30.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.141+0000 7f7fd0e75640 1 -- 172.21.15.7:0/836379545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 msgr2=0x7f7fcc10f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.141+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/836379545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 0x7f7fcc10f9e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009900 tx=0x7f7fc002bc30 comp rx=0 tx=0).stop 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.141+0000 7f7fd0e75640 1 -- 172.21.15.7:0/836379545 shutdown_connections 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.141+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/836379545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc10ff20 0x7f7fcc112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.141+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/836379545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 0x7f7fcc10f9e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.141+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/836379545 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc1086a0 0x7f7fcc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.141+0000 7f7fd0e75640 1 -- 172.21.15.7:0/836379545 >> 172.21.15.7:0/836379545 conn(0x7f7fcc06e640 msgr2=0x7f7fcc070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.141+0000 7f7fd0e75640 1 -- 172.21.15.7:0/836379545 shutdown_connections 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.141+0000 7f7fd0e75640 1 -- 172.21.15.7:0/836379545 wait complete. 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.142+0000 7f7fd0e75640 1 Processor -- start 2024-03-26T08:56:30.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.142+0000 7f7fd0e75640 1 -- start start 2024-03-26T08:56:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.142+0000 7f7fd0e75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc1086a0 0x7f7fcc1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.142+0000 7f7fd0e75640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 0x7f7fcc1a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.142+0000 7f7fd0e75640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc10ff20 0x7f7fcc1a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.142+0000 7f7fd0e75640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc115b00 con 0x7f7fcc109000 2024-03-26T08:56:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.142+0000 7f7fd0e75640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc115980 con 0x7f7fcc1086a0 2024-03-26T08:56:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fc9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 0x7f7fcc1a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fd0e75640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc115c80 con 0x7f7fcc10ff20 2024-03-26T08:56:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fc9d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 0x7f7fcc1a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36904/0 (socket says 172.21.15.7:36904) 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fc9d9b640 1 -- 172.21.15.7:0/569945919 learned_addr learned my addr 172.21.15.7:0/569945919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc10ff20 0x7f7fcc1a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fca59c640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc1086a0 0x7f7fcc1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fc9d9b640 1 -- 172.21.15.7:0/569945919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc10ff20 msgr2=0x7f7fcc1a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fc9d9b640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc10ff20 0x7f7fcc1a7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fc9d9b640 1 -- 172.21.15.7:0/569945919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc1086a0 msgr2=0x7f7fcc1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fc9d9b640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc1086a0 0x7f7fcc1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fc9d9b640 1 -- 172.21.15.7:0/569945919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc1a7df0 con 0x7f7fcc109000 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.143+0000 7f7fc9d9b640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 0x7f7fcc1a0710 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009230 tx=0x7f7fc002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.144+0000 7f7fbb7fe640 1 -- 172.21.15.7:0/569945919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0041070 con 0x7f7fcc109000 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.144+0000 7f7fbb7fe640 1 -- 172.21.15.7:0/569945919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0004450 con 0x7f7fcc109000 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.144+0000 7f7fbb7fe640 1 -- 172.21.15.7:0/569945919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0003e20 con 0x7f7fcc109000 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.144+0000 7f7fd0e75640 1 -- 172.21.15.7:0/569945919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc1a8080 con 0x7f7fcc109000 2024-03-26T08:56:30.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.144+0000 7f7fd0e75640 1 -- 172.21.15.7:0/569945919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc1a85c0 con 0x7f7fcc109000 2024-03-26T08:56:30.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.146+0000 7f7fbb7fe640 1 -- 172.21.15.7:0/569945919 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7fc0044070 con 0x7f7fcc109000 2024-03-26T08:56:30.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.146+0000 7f7fd3a59740 1 Processor -- start 2024-03-26T08:56:30.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.146+0000 7f7fd3a59740 1 -- start start 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50723da0 0x55de507241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50724790 0x55de50729290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50729900 0x55de5072bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fd3a59740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de506893d0 con 0x55de50724790 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fd3a59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de50631ad0 con 0x55de50729900 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fd3a59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de50685470 con 0x55de50723da0 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fbb7fe640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2880 0x7f7fb40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fcad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50723da0 0x55de507241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50729900 0x55de5072bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50724790 0x55de50729290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fcad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50723da0 0x55de507241c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57728/0 (socket says 172.21.15.7:57728) 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fcad9d640 1 -- 172.21.15.7:0/4070346767 learned_addr learned my addr 172.21.15.7:0/4070346767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2880 0x7f7fb40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fbb7fe640 1 -- 172.21.15.7:0/569945919 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fc00f0580 con 0x7f7fcc109000 2024-03-26T08:56:30.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fcad9d640 1 -- 172.21.15.7:0/4070346767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50729900 msgr2=0x55de5072bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/4070346767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50729900 0x55de5072bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fcad9d640 1 -- 172.21.15.7:0/4070346767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50724790 msgr2=0x55de50729290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/4070346767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50724790 0x55de50729290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.147+0000 7f7fcad9d640 1 -- 172.21.15.7:0/4070346767 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de504677d0 con 0x55de50723da0 2024-03-26T08:56:30.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.148+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/4070346767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50723da0 0x55de507241c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4011e20 tx=0x7f7fc400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83df3a99d985cfae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.148+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2880 0x7f7fb40a4d40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400fc90 tx=0x7f7fc401e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.148+0000 7f7fb8ff9640 1 -- 172.21.15.7:0/4070346767 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc401dcb0 con 0x55de50723da0 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.148+0000 7f7fb8ff9640 1 -- 172.21.15.7:0/4070346767 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc401de50 con 0x55de50723da0 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.148+0000 7f7fb8ff9640 1 -- 172.21.15.7:0/4070346767 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4004280 con 0x55de50723da0 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.149+0000 7f7fd3a59740 1 -- 172.21.15.7:0/4070346767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50723da0 msgr2=0x55de507241c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.149+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/4070346767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50723da0 0x55de507241c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4011e20 tx=0x7f7fc400f790 comp rx=0 tx=0).stop 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.150+0000 7f7fd3a59740 1 -- 172.21.15.7:0/4070346767 shutdown_connections 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.150+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/4070346767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50729900 0x55de5072bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.150+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/4070346767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50724790 0x55de50729290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.150+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/4070346767 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50723da0 0x55de507241c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.150+0000 7f7fd3a59740 1 -- 172.21.15.7:0/4070346767 >> 172.21.15.7:0/4070346767 conn(0x55de5071f310 msgr2=0x55de5071f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.150+0000 7f7fd3a59740 1 -- 172.21.15.7:0/4070346767 shutdown_connections 2024-03-26T08:56:30.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.150+0000 7f7fd3a59740 1 -- 172.21.15.7:0/4070346767 wait complete. 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.151+0000 7f7fd3a59740 1 -- start start 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.151+0000 7f7fd3a59740 1 Processor -- start 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.151+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de5072e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.151+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de5072e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.151+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de507355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fd3a59740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5072c7f0 con 0x55de50723da0 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de5072e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de5072e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39692/0 (socket says 172.21.15.7:39692) 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fc9d9b640 1 -- 172.21.15.7:0/1644634729 learned_addr learned my addr 172.21.15.7:0/1644634729 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fca59c640 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de5072e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de50645060 con 0x55de50724790 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de507355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de5072c970 con 0x55de50729900 2024-03-26T08:56:30.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fd3a59740 10 client.0 _ll_register_callbacks cb 0x55de506276a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fc9d9b640 1 -- 172.21.15.7:0/1644634729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 msgr2=0x55de507355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fc9d9b640 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de507355c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fc9d9b640 1 -- 172.21.15.7:0/1644634729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 msgr2=0x55de5072e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fc9d9b640 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de5072e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fc9d9b640 1 -- 172.21.15.7:0/1644634729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de5039d3f0 con 0x55de50724790 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de507355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fca59c640 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de5072e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fc9d9b640 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de5072e580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00ee080 tx=0x7f7fc0005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb26542b36666c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/1644634729 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc004b040 con 0x55de50724790 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.152+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/1644634729 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fc00c5230 con 0x55de50724790 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fc401de50 con 0x55de50724790 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/1644634729 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fc00bc930 con 0x55de50724790 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fd3a59740 10 client.28880 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de5035d0f0 con 0x55de50724790 2024-03-26T08:56:30.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/1644634729 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc401de50 con 0x55de50724790 2024-03-26T08:56:30.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/1644634729 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7fc008cb60 con 0x55de50724790 2024-03-26T08:56:30.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fd3a59740 10 client.28880 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:30.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fd3a59740 10 client.28880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:30.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fd3a59740 20 client.28880 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:30.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.153+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de507399a0 0x55de5073bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.154+0000 7f7fd3a59740 4 client.28880 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:30.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.154+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de50696910 con 0x55de507399a0 2024-03-26T08:56:30.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.154+0000 7f7fca59c640 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de507399a0 0x55de5073bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.154+0000 7f7fca59c640 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de507399a0 0x55de5073bdb0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f7baadc0266930 server_cookie=e2e6542573a9cec in_seq=0 out_seq=0 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.154+0000 7f7fb1ffb640 10 client.28880 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/1644634729 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55de50696910 con 0x55de507399a0 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fb1ffb640 10 client.28880 handle_command_reply: tid=0 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 client.28880 shutdown 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 msgr2=0x55de5072e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de5072e580 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00ee080 tx=0x7f7fc0005e50 comp rx=0 tx=0).stop 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 shutdown_connections 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de507399a0 0x55de5073bdb0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fb1ffb640 0 client.28880 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de507355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de5072e580 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1644634729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de5072e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 >> 172.21.15.7:0/1644634729 conn(0x55de5071f310 msgr2=0x55de5071e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 shutdown_connections 2024-03-26T08:56:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1644634729 wait complete. 2024-03-26T08:56:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.155+0000 7f7fd3a59740 20 client.28880 trim_cache size 0 max 16384 2024-03-26T08:56:30.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fd3a59740 1 Processor -- start 2024-03-26T08:56:30.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fd3a59740 1 -- start start 2024-03-26T08:56:30.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50723da0 0x55de5073bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50724790 0x55de50735580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50729900 0x55de50725b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fd3a59740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1b1540 con 0x55de50729900 2024-03-26T08:56:30.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fd3a59740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1af040 con 0x55de50723da0 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fd3a59740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1ad0c0 con 0x55de50724790 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fcad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50729900 0x55de50725b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50723da0 0x55de5073bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fcad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50729900 0x55de50725b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36934/0 (socket says 172.21.15.7:36934) 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fc9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50724790 0x55de50735580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.177+0000 7f7fcad9d640 1 -- 172.21.15.7:0/1071032261 learned_addr learned my addr 172.21.15.7:0/1071032261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.178+0000 7f7fcad9d640 1 -- 172.21.15.7:0/1071032261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50724790 msgr2=0x55de50735580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.178+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/1071032261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50724790 0x55de50735580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.178+0000 7f7fcad9d640 1 -- 172.21.15.7:0/1071032261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50723da0 msgr2=0x55de5073bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.178+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/1071032261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50723da0 0x55de5073bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.178+0000 7f7fcad9d640 1 -- 172.21.15.7:0/1071032261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de50720080 con 0x55de50729900 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.178+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/1071032261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50729900 0x55de50725b50 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40099e0 tx=0x7f7fc4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e068c1357929b0a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.178+0000 7f7fb3fff640 1 -- 172.21.15.7:0/1071032261 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4035070 con 0x55de50729900 2024-03-26T08:56:30.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.178+0000 7f7fb3fff640 1 -- 172.21.15.7:0/1071032261 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc4005360 con 0x55de50729900 2024-03-26T08:56:30.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.178+0000 7f7fb3fff640 1 -- 172.21.15.7:0/1071032261 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc402a040 con 0x55de50729900 2024-03-26T08:56:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.179+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1071032261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50729900 msgr2=0x55de50725b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.179+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1071032261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50729900 0x55de50725b50 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40099e0 tx=0x7f7fc4005bd0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.179+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1071032261 shutdown_connections 2024-03-26T08:56:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.180+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1071032261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50729900 0x55de50725b50 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.180+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1071032261 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50724790 0x55de50735580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.180+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/1071032261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50723da0 0x55de5073bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.180+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1071032261 >> 172.21.15.7:0/1071032261 conn(0x55de5071f310 msgr2=0x55de5071f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.181+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1071032261 shutdown_connections 2024-03-26T08:56:30.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.181+0000 7f7fd3a59740 1 -- 172.21.15.7:0/1071032261 wait complete. 2024-03-26T08:56:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.181+0000 7f7fd3a59740 1 -- start start 2024-03-26T08:56:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fd3a59740 1 Processor -- start 2024-03-26T08:56:30.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de50733010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de50733550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fd3a59740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de50734560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fd3a59740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1ad6c0 con 0x55de50723da0 2024-03-26T08:56:30.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de50733010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fca59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de50733010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36948/0 (socket says 172.21.15.7:36948) 2024-03-26T08:56:30.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fca59c640 1 -- 172.21.15.7:0/2358209142 learned_addr learned my addr 172.21.15.7:0/2358209142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de50734560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1ae9e0 con 0x55de50724790 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1add20 con 0x55de50729900 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fc9d9b640 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de50733550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.182+0000 7f7fd3a59740 10 client.0 _ll_register_callbacks cb 0x7f7fc0009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fca59c640 1 -- 172.21.15.7:0/2358209142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 msgr2=0x55de50734560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fca59c640 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de50734560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fca59c640 1 -- 172.21.15.7:0/2358209142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 msgr2=0x55de50733550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fca59c640 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de50733550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fca59c640 1 -- 172.21.15.7:0/2358209142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de50734b50 con 0x55de50723da0 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fcad9d640 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de50734560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fca59c640 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de50733010 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x55de50724b90 tx=0x7f7fbc022f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93b35ce57aa0afa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/2358209142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc012ba0 con 0x55de50723da0 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fc4005360 con 0x55de50723da0 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/2358209142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7fbc012d40 con 0x55de50723da0 2024-03-26T08:56:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/2358209142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fbc032050 con 0x55de50723da0 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fd3a59740 10 client.19101 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de507956e0 con 0x55de50723da0 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.183+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/2358209142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fc4005360 con 0x55de50723da0 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.184+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/2358209142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7fbc003b90 con 0x55de50723da0 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.184+0000 7f7fd3a59740 10 client.19101 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.184+0000 7f7fd3a59740 10 client.19101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.184+0000 7f7fd3a59740 20 client.19101 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.184+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de507399a0 0x55de50797fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.184+0000 7f7fd3a59740 4 client.19101 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.184+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de50690c60 con 0x55de507399a0 2024-03-26T08:56:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.184+0000 7f7fc9d9b640 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de507399a0 0x55de50797fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fb1ffb640 10 client.19101 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fc9d9b640 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de507399a0 0x55de50797fc0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9649aa49239ac68 server_cookie=535378e64d004233 in_seq=0 out_seq=0 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fb1ffb640 1 -- 172.21.15.7:0/2358209142 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55de50690c60 con 0x55de507399a0 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fb1ffb640 10 client.19101 handle_command_reply: tid=0 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 client.19101 shutdown 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 msgr2=0x55de50733010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de50733010 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x55de50724b90 tx=0x7f7fbc022f00 comp rx=0 tx=0).stop 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 shutdown_connections 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55de507399a0 0x55de50797fc0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55de50729900 0x55de50734560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fb1ffb640 0 client.19101 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de50724790 0x55de50733550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 --2- 172.21.15.7:0/2358209142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55de50723da0 0x55de50733010 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 >> 172.21.15.7:0/2358209142 conn(0x55de5071f310 msgr2=0x55de5072df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 shutdown_connections 2024-03-26T08:56:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.185+0000 7f7fd3a59740 1 -- 172.21.15.7:0/2358209142 wait complete. 2024-03-26T08:56:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd3a59740 20 client.19101 trim_cache size 0 max 16384 2024-03-26T08:56:30.187 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:56:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:56:30.187 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:56:30.187 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 -- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2880 msgr2=0x7f7fb40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2880 0x7f7fb40a4d40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400fc90 tx=0x7f7fc401e000 comp rx=0 tx=0).stop 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 -- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 msgr2=0x7f7fcc1a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 0x7f7fcc1a0710 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009230 tx=0x7f7fc002c650 comp rx=0 tx=0).stop 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 -- 172.21.15.7:0/569945919 shutdown_connections 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7fb40a2880 0x7f7fb40a4d40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7fcc10ff20 0x7f7fcc1a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109000 0x7f7fcc1a0710 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 --2- 172.21.15.7:0/569945919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc1086a0 0x7f7fcc1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 -- 172.21.15.7:0/569945919 >> 172.21.15.7:0/569945919 conn(0x7f7fcc06e640 msgr2=0x7f7fcc070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 -- 172.21.15.7:0/569945919 shutdown_connections 2024-03-26T08:56:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.186+0000 7f7fd0e75640 1 -- 172.21.15.7:0/569945919 wait complete. 2024-03-26T08:56:30.201 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:56:30.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:56:30.257 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:30 smithi007 ceph-mon[32462]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 681 op/s 2024-03-26T08:56:30.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:30 smithi044 ceph-mon[36461]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 681 op/s 2024-03-26T08:56:30.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.369+0000 7f4cf7510640 1 -- 172.21.15.7:0/1937563911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf01005e0 msgr2=0x7f4cf0100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.369+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1937563911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf01005e0 0x7f4cf0100a00 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4cec009900 tx=0x7f4cec02bc30 comp rx=0 tx=0).stop 2024-03-26T08:56:30.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.369+0000 7f4cf7510640 1 -- 172.21.15.7:0/1937563911 shutdown_connections 2024-03-26T08:56:30.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.369+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1937563911 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cf0102190 0x7f4cf0108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.369+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1937563911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf01017d0 0x7f4cf0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.369+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1937563911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf01005e0 0x7f4cf0100a00 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.369+0000 7f4cf7510640 1 -- 172.21.15.7:0/1937563911 >> 172.21.15.7:0/1937563911 conn(0x7f4cf00fbb30 msgr2=0x7f4cf00fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 -- 172.21.15.7:0/1937563911 shutdown_connections 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 -- 172.21.15.7:0/1937563911 wait complete. 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 Processor -- start 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 -- start start 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf01005e0 0x7f4cf019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cf01017d0 0x7f4cf019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf0102190 0x7f4cf01a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf010b140 con 0x7f4cf0102190 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf010afc0 con 0x7f4cf01005e0 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf7510640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf010b2c0 con 0x7f4cf01017d0 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf5aad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf0102190 0x7f4cf01a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf5aad640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf0102190 0x7f4cf01a5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36968/0 (socket says 172.21.15.7:36968) 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf5aad640 1 -- 172.21.15.7:0/1574992516 learned_addr learned my addr 172.21.15.7:0/1574992516 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf52ac640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf01005e0 0x7f4cf019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.370+0000 7f4cf4aab640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cf01017d0 0x7f4cf019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4cf52ac640 1 -- 172.21.15.7:0/1574992516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cf01017d0 msgr2=0x7f4cf019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4cf52ac640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cf01017d0 0x7f4cf019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4cf52ac640 1 -- 172.21.15.7:0/1574992516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf0102190 msgr2=0x7f4cf01a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4cf52ac640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf0102190 0x7f4cf01a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4cf52ac640 1 -- 172.21.15.7:0/1574992516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf01a5c00 con 0x7f4cf01005e0 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4cf52ac640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf01005e0 0x7f4cf019e330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4cec036040 tx=0x7f4cec004e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4ce67fc640 1 -- 172.21.15.7:0/1574992516 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec043020 con 0x7f4cf01005e0 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4ce67fc640 1 -- 172.21.15.7:0/1574992516 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4cec02c980 con 0x7f4cf01005e0 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf01a5e90 con 0x7f4cf01005e0 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4ce67fc640 1 -- 172.21.15.7:0/1574992516 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec03e450 con 0x7f4cf01005e0 2024-03-26T08:56:30.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.371+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf01a62f0 con 0x7f4cf01005e0 2024-03-26T08:56:30.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.372+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf01a64f0 con 0x7f4cf01005e0 2024-03-26T08:56:30.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.374+0000 7f4ce67fc640 1 -- 172.21.15.7:0/1574992516 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4cec046020 con 0x7f4cf01005e0 2024-03-26T08:56:30.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.375+0000 7f4ce67fc640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cdc0a2880 0x7f4cdc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.375+0000 7f4ce67fc640 1 -- 172.21.15.7:0/1574992516 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4cec0f0ed0 con 0x7f4cf01005e0 2024-03-26T08:56:30.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.375+0000 7f4cf4aab640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cdc0a2880 0x7f4cdc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.376+0000 7f4cf4aab640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cdc0a2880 0x7f4cdc0a4d40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4cf019f4f0 tx=0x7f4ce0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.379+0000 7f4ce67fc640 1 -- 172.21.15.7:0/1574992516 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4cec04d020 con 0x7f4cf01005e0 2024-03-26T08:56:30.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:30 smithi016 ceph-mon[35851]: pgmap v782: 97 pgs: 97 active+clean; 1.6 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 681 op/s 2024-03-26T08:56:30.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.732+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cf01a66c0 con 0x7f4cf01005e0 2024-03-26T08:56:30.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.733+0000 7f4ce67fc640 1 -- 172.21.15.7:0/1574992516 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f4cec0bdf30 con 0x7f4cf01005e0 2024-03-26T08:56:30.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:56:30.731 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:56:30.732 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.735+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cdc0a2880 msgr2=0x7f4cdc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.735+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cdc0a2880 0x7f4cdc0a4d40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4cf019f4f0 tx=0x7f4ce0005f90 comp rx=0 tx=0).stop 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.735+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf01005e0 msgr2=0x7f4cf019e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.735+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf01005e0 0x7f4cf019e330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4cec036040 tx=0x7f4cec004e70 comp rx=0 tx=0).stop 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.736+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 shutdown_connections 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.736+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cdc0a2880 0x7f4cdc0a4d40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.736+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf0102190 0x7f4cf01a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.736+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4cf01017d0 0x7f4cf019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.736+0000 7f4cf7510640 1 --2- 172.21.15.7:0/1574992516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf01005e0 0x7f4cf019e330 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.736+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 >> 172.21.15.7:0/1574992516 conn(0x7f4cf00fbb30 msgr2=0x7f4cf010dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.736+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 shutdown_connections 2024-03-26T08:56:30.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.736+0000 7f4cf7510640 1 -- 172.21.15.7:0/1574992516 wait complete. 2024-03-26T08:56:30.751 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:56:30.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:56:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.897+0000 7fc688e12640 1 -- 172.21.15.7:0/4197615184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc684102140 msgr2=0x7fc684104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.897+0000 7fc688e12640 1 --2- 172.21.15.7:0/4197615184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc684102140 0x7fc684104530 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc674009860 tx=0x7fc67402bb70 comp rx=0 tx=0).stop 2024-03-26T08:56:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.898+0000 7fc688e12640 1 -- 172.21.15.7:0/4197615184 shutdown_connections 2024-03-26T08:56:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.898+0000 7fc688e12640 1 --2- 172.21.15.7:0/4197615184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6841073a0 0x7fc6841097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.898+0000 7fc688e12640 1 --2- 172.21.15.7:0/4197615184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc684104a70 0x7fc684106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.898+0000 7fc688e12640 1 --2- 172.21.15.7:0/4197615184 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc684102140 0x7fc684104530 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.898+0000 7fc688e12640 1 -- 172.21.15.7:0/4197615184 >> 172.21.15.7:0/4197615184 conn(0x7fc6840fbb30 msgr2=0x7fc6840fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.898+0000 7fc688e12640 1 -- 172.21.15.7:0/4197615184 shutdown_connections 2024-03-26T08:56:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.898+0000 7fc688e12640 1 -- 172.21.15.7:0/4197615184 wait complete. 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc688e12640 1 Processor -- start 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc688e12640 1 -- start start 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc688e12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc684102140 0x7fc6841979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc688e12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc684104a70 0x7fc684197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc688e12640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6841073a0 0x7fc68419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc688e12640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68410d350 con 0x7fc6841073a0 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc688e12640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68410d1d0 con 0x7fc684102140 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc688e12640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68410d4d0 con 0x7fc684104a70 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc68259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc684102140 0x7fc6841979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc68259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc684102140 0x7fc6841979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39742/0 (socket says 172.21.15.7:39742) 2024-03-26T08:56:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc68259c640 1 -- 172.21.15.7:0/3295662405 learned_addr learned my addr 172.21.15.7:0/3295662405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc682d9d640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6841073a0 0x7fc68419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.899+0000 7fc681d9b640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc684104a70 0x7fc684197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc68259c640 1 -- 172.21.15.7:0/3295662405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc684104a70 msgr2=0x7fc684197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc68259c640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc684104a70 0x7fc684197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc68259c640 1 -- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6841073a0 msgr2=0x7fc68419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc68259c640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6841073a0 0x7fc68419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc68259c640 1 -- 172.21.15.7:0/3295662405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68419f5f0 con 0x7fc684102140 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc682d9d640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6841073a0 0x7fc68419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc681d9b640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc684104a70 0x7fc684197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc68259c640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc684102140 0x7fc6841979d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc674005940 tx=0x7fc674032470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc6737fe640 1 -- 172.21.15.7:0/3295662405 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674041020 con 0x7fc684102140 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc688e12640 1 -- 172.21.15.7:0/3295662405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68419f880 con 0x7fc684102140 2024-03-26T08:56:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc6737fe640 1 -- 172.21.15.7:0/3295662405 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc67402c9f0 con 0x7fc684102140 2024-03-26T08:56:30.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc6737fe640 1 -- 172.21.15.7:0/3295662405 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674039040 con 0x7fc684102140 2024-03-26T08:56:30.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.900+0000 7fc688e12640 1 -- 172.21.15.7:0/3295662405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68419fd70 con 0x7fc684102140 2024-03-26T08:56:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc6737fe640 1 -- 172.21.15.7:0/3295662405 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc674044020 con 0x7fc684102140 2024-03-26T08:56:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc68b9ff740 1 Processor -- start 2024-03-26T08:56:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc68b9ff740 1 -- start start 2024-03-26T08:56:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x56258740a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 0x56258740f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740fa80 0x562587411ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625873c4610 con 0x56258740fa80 2024-03-26T08:56:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625873673c0 con 0x56258740a490 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56258730fd60 con 0x56258740ae80 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc682d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x56258740a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc68259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 0x56258740f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740fa80 0x562587411ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc682d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x56258740a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39744/0 (socket says 172.21.15.7:39744) 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc682d9d640 1 -- 172.21.15.7:0/3950562096 learned_addr learned my addr 172.21.15.7:0/3950562096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc6737fe640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc66c0a2800 0x7fc66c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc681d9b640 1 -- 172.21.15.7:0/3950562096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 msgr2=0x56258740f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc681d9b640 1 --2- 172.21.15.7:0/3950562096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 0x56258740f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc681d9b640 1 -- 172.21.15.7:0/3950562096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 msgr2=0x56258740a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc681d9b640 1 --2- 172.21.15.7:0/3950562096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x56258740a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc681d9b640 1 -- 172.21.15.7:0/3950562096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56258707b3f0 con 0x56258740fa80 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.903+0000 7fc682d9d640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc66c0a2800 0x7fc66c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.904+0000 7fc6737fe640 1 -- 172.21.15.7:0/3295662405 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc6740f0570 con 0x7fc684102140 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.904+0000 7fc681d9b640 1 --2- 172.21.15.7:0/3950562096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740fa80 0x562587411ef0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc68406a8a0 tx=0x7fc6780117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630cd69233846a0c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.904+0000 7fc670ff9640 1 -- 172.21.15.7:0/3950562096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678012450 con 0x56258740fa80 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.904+0000 7fc670ff9640 1 -- 172.21.15.7:0/3950562096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc678012ad0 con 0x56258740fa80 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.904+0000 7fc670ff9640 1 -- 172.21.15.7:0/3950562096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67800fa90 con 0x56258740fa80 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.905+0000 7fc68b9ff740 1 -- 172.21.15.7:0/3950562096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740fa80 msgr2=0x562587411ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.905+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/3950562096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740fa80 0x562587411ef0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc68406a8a0 tx=0x7fc6780117b0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.905+0000 7fc682d9d640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc66c0a2800 0x7fc66c0a4cc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009fd0 tx=0x7fc67c014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.905+0000 7fc68b9ff740 1 -- 172.21.15.7:0/3950562096 shutdown_connections 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.905+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/3950562096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740fa80 0x562587411ef0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.905+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/3950562096 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 0x56258740f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.905+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/3950562096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x56258740a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.905+0000 7fc68b9ff740 1 -- 172.21.15.7:0/3950562096 >> 172.21.15.7:0/3950562096 conn(0x5625873fcb80 msgr2=0x5625873fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.907+0000 7fc68b9ff740 1 -- 172.21.15.7:0/3950562096 shutdown_connections 2024-03-26T08:56:30.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.907+0000 7fc68b9ff740 1 -- 172.21.15.7:0/3950562096 wait complete. 2024-03-26T08:56:30.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.907+0000 7fc68b9ff740 1 -- start start 2024-03-26T08:56:30.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc68b9ff740 1 Processor -- start 2024-03-26T08:56:30.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874141f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 0x562587414730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740fa80 0x56258741b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562587323240 con 0x56258740a490 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562587363460 con 0x56258740fa80 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562587412a10 con 0x56258740ae80 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc68b9ff740 10 client.0 _ll_register_callbacks cb 0x562587322c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 0x562587414730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc682d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740fa80 0x56258741b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 0x562587414730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57814/0 (socket says 172.21.15.7:57814) 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc681d9b640 1 -- 172.21.15.7:0/4252189214 learned_addr learned my addr 172.21.15.7:0/4252189214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc68259c640 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874141f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc682d9d640 1 -- 172.21.15.7:0/4252189214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 msgr2=0x562587414730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc682d9d640 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 0x562587414730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc682d9d640 1 -- 172.21.15.7:0/4252189214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 msgr2=0x5625874141f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc682d9d640 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874141f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.908+0000 7fc682d9d640 1 -- 172.21.15.7:0/4252189214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625871457d0 con 0x56258740fa80 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc682d9d640 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740fa80 0x56258741b770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x562587400840 tx=0x7fc67c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b781b003689174ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc669ffb640 1 -- 172.21.15.7:0/4252189214 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c02b060 con 0x56258740fa80 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc669ffb640 1 -- 172.21.15.7:0/4252189214 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc67c01bd90 con 0x56258740fa80 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc68b9ff740 1 -- 172.21.15.7:0/4252189214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc678012ad0 con 0x56258740fa80 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc669ffb640 1 -- 172.21.15.7:0/4252189214 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc67c026400 con 0x56258740fa80 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc68b9ff740 10 client.28904 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc68b9ff740 1 -- 172.21.15.7:0/4252189214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56258741eb50 con 0x56258740fa80 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc669ffb640 1 -- 172.21.15.7:0/4252189214 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc678012ad0 con 0x56258740fa80 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.909+0000 7fc669ffb640 1 -- 172.21.15.7:0/4252189214 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc67c030070 con 0x56258740fa80 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.910+0000 7fc68b9ff740 10 client.28904 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.910+0000 7fc68b9ff740 10 client.28904 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.910+0000 7fc68b9ff740 20 client.28904 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.910+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562587420270 0x562587422680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.910+0000 7fc68b9ff740 4 client.28904 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.910+0000 7fc68b9ff740 1 -- 172.21.15.7:0/4252189214 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562587374900 con 0x562587420270 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.910+0000 7fc68259c640 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562587420270 0x562587422680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.910+0000 7fc68259c640 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562587420270 0x562587422680 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f4fa90050838402 server_cookie=87b1284360d24318 in_seq=0 out_seq=0 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.910+0000 7fc669ffb640 10 client.28904 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc669ffb640 1 -- 172.21.15.7:0/4252189214 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562587374900 con 0x562587420270 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc669ffb640 10 client.28904 handle_command_reply: tid=0 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc68b9ff740 1 client.28904 shutdown 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc68b9ff740 1 -- 172.21.15.7:0/4252189214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740fa80 msgr2=0x56258741b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740fa80 0x56258741b770 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x562587400840 tx=0x7fc67c004a40 comp rx=0 tx=0).stop 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc68b9ff740 1 -- 172.21.15.7:0/4252189214 shutdown_connections 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x562587420270 0x562587422680 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740fa80 0x56258741b770 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740ae80 0x562587414730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/4252189214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874141f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc68b9ff740 1 -- 172.21.15.7:0/4252189214 >> 172.21.15.7:0/4252189214 conn(0x5625873fcb80 msgr2=0x562587409b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.911+0000 7fc669ffb640 0 client.28904 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:56:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.912+0000 7fc68b9ff740 1 -- 172.21.15.7:0/4252189214 shutdown_connections 2024-03-26T08:56:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.912+0000 7fc68b9ff740 1 -- 172.21.15.7:0/4252189214 wait complete. 2024-03-26T08:56:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.912+0000 7fc68b9ff740 20 client.28904 trim_cache size 0 max 16384 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.929+0000 7fc68b9ff740 1 Processor -- start 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.929+0000 7fc68b9ff740 1 -- start start 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.929+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x5625874099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.929+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740ae80 0x56258741b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 0x562587426f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc68259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x5625874099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6841a8e10 con 0x56258740ae80 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6841a6910 con 0x56258740a490 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6841a4990 con 0x56258740fa80 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740ae80 0x56258741b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740ae80 0x56258741b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37008/0 (socket says 172.21.15.7:37008) 2024-03-26T08:56:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc681d9b640 1 -- 172.21.15.7:0/2932462556 learned_addr learned my addr 172.21.15.7:0/2932462556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc682d9d640 1 --2- 172.21.15.7:0/2932462556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 0x562587426f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc68259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x5625874099a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39766/0 (socket says 172.21.15.7:39766) 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc681d9b640 1 -- 172.21.15.7:0/2932462556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 msgr2=0x562587426f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc681d9b640 1 --2- 172.21.15.7:0/2932462556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 0x562587426f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc681d9b640 1 -- 172.21.15.7:0/2932462556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 msgr2=0x5625874099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc681d9b640 1 --2- 172.21.15.7:0/2932462556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x5625874099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.930+0000 7fc681d9b640 1 -- 172.21.15.7:0/2932462556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625873fd8f0 con 0x56258740ae80 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.931+0000 7fc681d9b640 1 --2- 172.21.15.7:0/2932462556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740ae80 0x56258741b730 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc678009b80 tx=0x7fc6780092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cffbb78bedc81f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.931+0000 7fc66bfff640 1 -- 172.21.15.7:0/2932462556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67804b070 con 0x56258740ae80 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.931+0000 7fc66bfff640 1 -- 172.21.15.7:0/2932462556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc678005040 con 0x56258740ae80 2024-03-26T08:56:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.931+0000 7fc66bfff640 1 -- 172.21.15.7:0/2932462556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6780048c0 con 0x56258740ae80 2024-03-26T08:56:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 -- 172.21.15.7:0/2932462556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740ae80 msgr2=0x56258741b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/2932462556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740ae80 0x56258741b730 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc678009b80 tx=0x7fc6780092d0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 -- 172.21.15.7:0/2932462556 shutdown_connections 2024-03-26T08:56:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/2932462556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 0x562587426f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/2932462556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740ae80 0x56258741b730 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/2932462556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740a490 0x5625874099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 -- 172.21.15.7:0/2932462556 >> 172.21.15.7:0/2932462556 conn(0x5625873fcb80 msgr2=0x562587409290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 -- 172.21.15.7:0/2932462556 shutdown_connections 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 -- 172.21.15.7:0/2932462556 wait complete. 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 -- start start 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 Processor -- start 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874193d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740ae80 0x562587419910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 0x56258741a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6841a4f90 con 0x56258740a490 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6841a62b0 con 0x56258740ae80 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.932+0000 7fc68b9ff740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6841a55f0 con 0x56258740fa80 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874193d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68259c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874193d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37022/0 (socket says 172.21.15.7:37022) 2024-03-26T08:56:30.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68259c640 1 -- 172.21.15.7:0/251622466 learned_addr learned my addr 172.21.15.7:0/251622466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68b9ff740 10 client.0 _ll_register_callbacks cb 0x7fc67c012490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc681d9b640 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740ae80 0x562587419910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc682d9d640 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 0x56258741a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68259c640 1 -- 172.21.15.7:0/251622466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 msgr2=0x56258741a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68259c640 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 0x56258741a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68259c640 1 -- 172.21.15.7:0/251622466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740ae80 msgr2=0x562587419910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68259c640 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740ae80 0x562587419910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68259c640 1 -- 172.21.15.7:0/251622466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56258741aec0 con 0x56258740a490 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68259c640 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874193d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc6740349e0 tx=0x7fc6740095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d15b28e82a5eec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc669ffb640 1 -- 172.21.15.7:0/251622466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6740bced0 con 0x56258740a490 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc669ffb640 1 -- 172.21.15.7:0/251622466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc6740ee140 con 0x56258740a490 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68b9ff740 1 -- 172.21.15.7:0/251622466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc678005040 con 0x56258740a490 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc669ffb640 1 -- 172.21.15.7:0/251622466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6740c1020 con 0x56258740a490 2024-03-26T08:56:30.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68b9ff740 10 client.19125 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.933+0000 7fc68b9ff740 1 -- 172.21.15.7:0/251622466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56258747c110 con 0x56258740a490 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.934+0000 7fc669ffb640 1 -- 172.21.15.7:0/251622466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc678005040 con 0x56258740a490 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.934+0000 7fc669ffb640 1 -- 172.21.15.7:0/251622466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc67403c030 con 0x56258740a490 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.934+0000 7fc68b9ff740 10 client.19125 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.934+0000 7fc68b9ff740 10 client.19125 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.934+0000 7fc68b9ff740 20 client.19125 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.934+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56258747ca90 0x56258747eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.934+0000 7fc68b9ff740 4 client.19125 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.934+0000 7fc68b9ff740 1 -- 172.21.15.7:0/251622466 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562587413fe0 con 0x56258747ca90 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.934+0000 7fc681d9b640 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56258747ca90 0x56258747eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc681d9b640 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56258747ca90 0x56258747eeb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ab0220bd856587f server_cookie=2d27967a4f72e087 in_seq=0 out_seq=0 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc669ffb640 10 client.19125 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:56:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc669ffb640 1 -- 172.21.15.7:0/251622466 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562587413fe0 con 0x56258747ca90 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc669ffb640 10 client.19125 handle_command_reply: tid=0 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 client.19125 shutdown 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 -- 172.21.15.7:0/251622466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 msgr2=0x5625874193d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874193d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc6740349e0 tx=0x7fc6740095b0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 -- 172.21.15.7:0/251622466 shutdown_connections 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56258747ca90 0x56258747eeb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56258740fa80 0x56258741a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56258740ae80 0x562587419910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 --2- 172.21.15.7:0/251622466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258740a490 0x5625874193d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc669ffb640 0 client.19125 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 -- 172.21.15.7:0/251622466 >> 172.21.15.7:0/251622466 conn(0x5625873fcb80 msgr2=0x562587409b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 -- 172.21.15.7:0/251622466 shutdown_connections 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 1 -- 172.21.15.7:0/251622466 wait complete. 2024-03-26T08:56:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.935+0000 7fc68b9ff740 20 client.19125 trim_cache size 0 max 16384 2024-03-26T08:56:30.939 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:56:30.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 -- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc66c0a2800 msgr2=0x7fc66c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc66c0a2800 0x7fc66c0a4cc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009fd0 tx=0x7fc67c014040 comp rx=0 tx=0).stop 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 -- 172.21.15.7:0/3295662405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc684102140 msgr2=0x7fc6841979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc684102140 0x7fc6841979d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc674005940 tx=0x7fc674032470 comp rx=0 tx=0).stop 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 -- 172.21.15.7:0/3295662405 shutdown_connections 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc66c0a2800 0x7fc66c0a4cc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6841073a0 0x7fc68419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc684104a70 0x7fc684197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 --2- 172.21.15.7:0/3295662405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc684102140 0x7fc6841979d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 -- 172.21.15.7:0/3295662405 >> 172.21.15.7:0/3295662405 conn(0x7fc6840fbb30 msgr2=0x7fc684102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 -- 172.21.15.7:0/3295662405 shutdown_connections 2024-03-26T08:56:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:30.936+0000 7fc688e12640 1 -- 172.21.15.7:0/3295662405 wait complete. 2024-03-26T08:56:30.947 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:56:30.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:56:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 840.0 s, 1384.8 tps, lat 71.626 ms stddev 83.703 2024-03-26T08:56:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.120+0000 7fcb04eff640 1 -- 172.21.15.7:0/259771580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb00108ed0 msgr2=0x7fcb00109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.120+0000 7fcb04eff640 1 --2- 172.21.15.7:0/259771580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb00108ed0 0x7fcb00109350 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009900 tx=0x7fcaf002bc80 comp rx=0 tx=0).stop 2024-03-26T08:56:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.123+0000 7fcb04eff640 1 -- 172.21.15.7:0/259771580 shutdown_connections 2024-03-26T08:56:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.123+0000 7fcb04eff640 1 --2- 172.21.15.7:0/259771580 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb00109890 0x7fcb00110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.123+0000 7fcb04eff640 1 --2- 172.21.15.7:0/259771580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb00108ed0 0x7fcb00109350 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.123+0000 7fcb04eff640 1 --2- 172.21.15.7:0/259771580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00107ce0 0x7fcb00108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.123+0000 7fcb04eff640 1 -- 172.21.15.7:0/259771580 >> 172.21.15.7:0/259771580 conn(0x7fcb00076c90 msgr2=0x7fcb000790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.125+0000 7fcb04eff640 1 -- 172.21.15.7:0/259771580 shutdown_connections 2024-03-26T08:56:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.125+0000 7fcb04eff640 1 -- 172.21.15.7:0/259771580 wait complete. 2024-03-26T08:56:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.126+0000 7fcb04eff640 1 Processor -- start 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.130+0000 7fcb04eff640 1 -- start start 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.130+0000 7fcb04eff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb00107ce0 0x7fcb0007eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.130+0000 7fcb04eff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb00108ed0 0x7fcb0007f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.130+0000 7fcb04eff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00109890 0x7fcb001ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.130+0000 7fcb04eff640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00113ac0 con 0x7fcb00108ed0 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.130+0000 7fcb04eff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00113940 con 0x7fcb00109890 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.130+0000 7fcb04eff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00113c40 con 0x7fcb00107ce0 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.131+0000 7fcafed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00109890 0x7fcb001ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.131+0000 7fcafed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00109890 0x7fcb001ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39804/0 (socket says 172.21.15.7:39804) 2024-03-26T08:56:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.131+0000 7fcafed9d640 1 -- 172.21.15.7:0/1866981303 learned_addr learned my addr 172.21.15.7:0/1866981303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.131+0000 7fcafe59c640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb00107ce0 0x7fcb0007eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.132+0000 7fcafed9d640 1 -- 172.21.15.7:0/1866981303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb00107ce0 msgr2=0x7fcb0007eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.132+0000 7fcafed9d640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb00107ce0 0x7fcb0007eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.132+0000 7fcafed9d640 1 -- 172.21.15.7:0/1866981303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb00108ed0 msgr2=0x7fcb0007f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.132+0000 7fcafed9d640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb00108ed0 0x7fcb0007f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.132+0000 7fcafed9d640 1 -- 172.21.15.7:0/1866981303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0006b0c0 con 0x7fcb00109890 2024-03-26T08:56:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.132+0000 7fcafed9d640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00109890 0x7fcb001ae1d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800b970 tx=0x7fcaf800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.132+0000 7fcaf77fe640 1 -- 172.21.15.7:0/1866981303 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8019070 con 0x7fcb00109890 2024-03-26T08:56:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.132+0000 7fcb04eff640 1 -- 172.21.15.7:0/1866981303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0006b3b0 con 0x7fcb00109890 2024-03-26T08:56:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.132+0000 7fcb04eff640 1 -- 172.21.15.7:0/1866981303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0006b950 con 0x7fcb00109890 2024-03-26T08:56:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.133+0000 7fcaf77fe640 1 -- 172.21.15.7:0/1866981303 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcaf800cb40 con 0x7fcb00109890 2024-03-26T08:56:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.133+0000 7fcaf77fe640 1 -- 172.21.15.7:0/1866981303 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8023060 con 0x7fcb00109890 2024-03-26T08:56:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.136+0000 7fcb07bc8740 1 Processor -- start 2024-03-26T08:56:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.136+0000 7fcaf77fe640 1 -- 172.21.15.7:0/1866981303 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcaf801c070 con 0x7fcb00109890 2024-03-26T08:56:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.137+0000 7fcb07bc8740 1 -- start start 2024-03-26T08:56:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.137+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee67ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.137+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee6cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.137+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 0x56049ee6f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.137+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049ee2a610 con 0x56049ee6d1f0 2024-03-26T08:56:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.137+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049edcd3c0 con 0x56049ee68080 2024-03-26T08:56:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.137+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049ed75d60 con 0x56049ee67690 2024-03-26T08:56:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.138+0000 7fcaf77fe640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcaec0a28d0 0x7fcaec0a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.138+0000 7fcaf77fe640 1 -- 172.21.15.7:0/1866981303 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcaf80cdfb0 con 0x7fcb00109890 2024-03-26T08:56:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.138+0000 7fcafed9d640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcaec0a28d0 0x7fcaec0a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.138+0000 7fcafe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee6cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.138+0000 7fcafe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee6cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39812/0 (socket says 172.21.15.7:39812) 2024-03-26T08:56:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcafe59c640 1 -- 172.21.15.7:0/2489450429 learned_addr learned my addr 172.21.15.7:0/2489450429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2489450429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 0x56049ee6f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2489450429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee67ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcafed9d640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcaec0a28d0 0x7fcaec0a4d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8096e20 tx=0x7fcaf8009450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcafe59c640 1 -- 172.21.15.7:0/2489450429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 msgr2=0x56049ee67ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcafe59c640 1 --2- 172.21.15.7:0/2489450429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee67ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcafe59c640 1 -- 172.21.15.7:0/2489450429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 msgr2=0x56049ee6f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcafe59c640 1 --2- 172.21.15.7:0/2489450429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 0x56049ee6f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcafe59c640 1 -- 172.21.15.7:0/2489450429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56049eae13f0 con 0x56049ee68080 2024-03-26T08:56:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.139+0000 7fcafe59c640 1 --2- 172.21.15.7:0/2489450429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee6cb80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009a30 tx=0x7fcae8009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec71251bbeb5911d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.140+0000 7fcaf4ff9640 1 -- 172.21.15.7:0/2489450429 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8017070 con 0x56049ee68080 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.140+0000 7fcaf4ff9640 1 -- 172.21.15.7:0/2489450429 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae8013470 con 0x56049ee68080 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.140+0000 7fcaf4ff9640 1 -- 172.21.15.7:0/2489450429 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80125f0 con 0x56049ee68080 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2489450429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 msgr2=0x56049ee6cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2489450429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee6cb80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009a30 tx=0x7fcae8009ef0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2489450429 shutdown_connections 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2489450429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 0x56049ee6f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2489450429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee6cb80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2489450429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee67ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2489450429 >> 172.21.15.7:0/2489450429 conn(0x56049ee62c00 msgr2=0x56049ee63000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2489450429 shutdown_connections 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2489450429 wait complete. 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.141+0000 7fcb07bc8740 1 -- start start 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcb07bc8740 1 Processor -- start 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee67690 0x56049ee71a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee71fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee6d1f0 0x56049ee78fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049ed89240 con 0x56049ee67690 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049edc9460 con 0x56049ee68080 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049ee701a0 con 0x56049ee6d1f0 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcaf7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee67690 0x56049ee71a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcaf7fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee67690 0x56049ee71a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37064/0 (socket says 172.21.15.7:37064) 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcaf7fff640 1 -- 172.21.15.7:0/2436465813 learned_addr learned my addr 172.21.15.7:0/2436465813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcafe59c640 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee71fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcb07bc8740 10 client.0 _ll_register_callbacks cb 0x56049ed88c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee6d1f0 0x56049ee78fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcaf7fff640 1 -- 172.21.15.7:0/2436465813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee6d1f0 msgr2=0x56049ee78fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee6d1f0 0x56049ee78fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcaf7fff640 1 -- 172.21.15.7:0/2436465813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 msgr2=0x56049ee71fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee71fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.142+0000 7fcaf7fff640 1 -- 172.21.15.7:0/2436465813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56049ebab7d0 con 0x56049ee67690 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee67690 0x56049ee71a90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fcaf002c800 tx=0x7fcaf0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f9f21ae926364a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2436465813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0049020 con 0x56049ee67690 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2436465813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcaf0004030 con 0x56049ee67690 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2436465813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcae8013470 con 0x56049ee67690 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2436465813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaf0004230 con 0x56049ee67690 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcb07bc8740 10 client.19134 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2436465813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56049ee7c360 con 0x56049ee67690 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2436465813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae8013470 con 0x56049ee67690 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2436465813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcaf00037b0 con 0x56049ee67690 2024-03-26T08:56:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.143+0000 7fcb07bc8740 10 client.19134 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.144+0000 7fcb07bc8740 10 client.19134 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:56:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.144+0000 7fcb07bc8740 20 client.19134 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.144+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56049ee7d980 0x56049ee7fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.144+0000 7fcb07bc8740 4 client.19134 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.144+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2436465813 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56049edda900 con 0x56049ee7d980 2024-03-26T08:56:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.144+0000 7fcafe59c640 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56049ee7d980 0x56049ee7fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.147+0000 7fcae5ffb640 10 client.19134 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:56:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.147+0000 7fcafe59c640 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56049ee7d980 0x56049ee7fe20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=408e8555176e5263 server_cookie=eab1762d26c8f2e1 in_seq=0 out_seq=0 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.148+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2436465813 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56049edda900 con 0x56049ee7d980 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.148+0000 7fcae5ffb640 10 client.19134 handle_command_reply: tid=0 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.148+0000 7fcb07bc8740 1 client.19134 shutdown 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2436465813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee67690 msgr2=0x56049ee71a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee67690 0x56049ee71a90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fcaf002c800 tx=0x7fcaf0005fd0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2436465813 shutdown_connections 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56049ee7d980 0x56049ee7fe20 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee6d1f0 0x56049ee78fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee71fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2436465813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee67690 0x56049ee71a90 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcae5ffb640 0 client.19134 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:56:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2436465813 >> 172.21.15.7:0/2436465813 conn(0x56049ee62c00 msgr2=0x56049ee61e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2436465813 shutdown_connections 2024-03-26T08:56:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2436465813 wait complete. 2024-03-26T08:56:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.149+0000 7fcb07bc8740 20 client.19134 trim_cache size 0 max 16384 2024-03-26T08:56:31.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.165+0000 7fcb07bc8740 1 Processor -- start 2024-03-26T08:56:31.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.165+0000 7fcb07bc8740 1 -- start start 2024-03-26T08:56:31.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.165+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee71db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee78f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 0x56049ee61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00108c0 con 0x56049ee6d1f0 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae000e3c0 con 0x56049ee68080 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae000c440 con 0x56049ee67690 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcaf7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee71db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcafe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee78f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcaf7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee71db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57860/0 (socket says 172.21.15.7:57860) 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcafe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee78f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39826/0 (socket says 172.21.15.7:39826) 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcaf7fff640 1 -- 172.21.15.7:0/3445006164 learned_addr learned my addr 172.21.15.7:0/3445006164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/3445006164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 0x56049ee61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcafe59c640 1 -- 172.21.15.7:0/3445006164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 msgr2=0x56049ee71db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcafe59c640 1 --2- 172.21.15.7:0/3445006164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee71db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcafe59c640 1 -- 172.21.15.7:0/3445006164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 msgr2=0x56049ee61590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcafe59c640 1 --2- 172.21.15.7:0/3445006164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 0x56049ee61590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcafe59c640 1 -- 172.21.15.7:0/3445006164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56049ee5e670 con 0x56049ee68080 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.166+0000 7fcafe59c640 1 --2- 172.21.15.7:0/3445006164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee78f60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x56049ee63b70 tx=0x7fcae8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcbea0e59b7409e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcae7fff640 1 -- 172.21.15.7:0/3445006164 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8017040 con 0x56049ee68080 2024-03-26T08:56:31.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcae7fff640 1 -- 172.21.15.7:0/3445006164 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae800edc0 con 0x56049ee68080 2024-03-26T08:56:31.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcae7fff640 1 -- 172.21.15.7:0/3445006164 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8003a80 con 0x56049ee68080 2024-03-26T08:56:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcb07bc8740 1 -- 172.21.15.7:0/3445006164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 msgr2=0x56049ee78f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/3445006164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee78f60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x56049ee63b70 tx=0x7fcae8008040 comp rx=0 tx=0).stop 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcb07bc8740 1 -- 172.21.15.7:0/3445006164 shutdown_connections 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/3445006164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee6d1f0 0x56049ee61590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/3445006164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee68080 0x56049ee78f60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/3445006164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee71db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcb07bc8740 1 -- 172.21.15.7:0/3445006164 >> 172.21.15.7:0/3445006164 conn(0x56049ee62c00 msgr2=0x56049ee716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcb07bc8740 1 -- 172.21.15.7:0/3445006164 shutdown_connections 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.167+0000 7fcb07bc8740 1 -- 172.21.15.7:0/3445006164 wait complete. 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcb07bc8740 1 -- start start 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcb07bc8740 1 Processor -- start 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee76e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee68080 0x56049ee77380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcb07bc8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee6d1f0 0x56049ee782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae000ca40 con 0x56049ee68080 2024-03-26T08:56:31.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae000dd60 con 0x56049ee6d1f0 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcb07bc8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae000d0a0 con 0x56049ee67690 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcb07bc8740 10 client.0 _ll_register_callbacks cb 0x7fcaf002ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcafe59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee68080 0x56049ee77380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcafe59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee68080 0x56049ee77380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37082/0 (socket says 172.21.15.7:37082) 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcaf7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee6d1f0 0x56049ee782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcafe59c640 1 -- 172.21.15.7:0/2531916819 learned_addr learned my addr 172.21.15.7:0/2531916819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.168+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee76e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcafe59c640 1 -- 172.21.15.7:0/2531916819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 msgr2=0x56049ee76e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcafe59c640 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee76e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcafe59c640 1 -- 172.21.15.7:0/2531916819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee6d1f0 msgr2=0x56049ee782d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcafe59c640 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee6d1f0 0x56049ee782d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcafe59c640 1 -- 172.21.15.7:0/2531916819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56049ee788c0 con 0x56049ee68080 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee6d1f0 0x56049ee782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcafe59c640 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee68080 0x56049ee77380 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009a30 tx=0x7fcae800eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb752b26a4f74258 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:56:31.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2531916819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8017030 con 0x56049ee68080 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2531916819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcae8004250 con 0x56049ee68080 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.169+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2531916819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcae800edc0 con 0x56049ee68080 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2531916819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcae8004450 con 0x56049ee68080 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcb07bc8740 10 client.19140 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2531916819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56049eed9990 con 0x56049ee68080 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2531916819 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae800edc0 con 0x56049ee68080 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2531916819 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fcae8003c40 con 0x56049ee68080 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcb07bc8740 10 client.19140 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcb07bc8740 10 client.19140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcb07bc8740 20 client.19140 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56049eed9f60 0x56049eedc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcb07bc8740 4 client.19140 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2531916819 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56049edd4c50 con 0x56049eed9f60 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.170+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56049eed9f60 0x56049eedc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcaf7fff640 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56049eed9f60 0x56049eedc350 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fd8dfb6cf2d0adf server_cookie=10500e57ae7b3e2d in_seq=0 out_seq=0 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcae5ffb640 10 client.19140 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcae5ffb640 1 -- 172.21.15.7:0/2531916819 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56049edd4c50 con 0x56049eed9f60 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcae5ffb640 10 client.19140 handle_command_reply: tid=0 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 client.19140 shutdown 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2531916819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee68080 msgr2=0x56049ee77380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee68080 0x56049ee77380 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009a30 tx=0x7fcae800eb40 comp rx=0 tx=0).stop 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2531916819 shutdown_connections 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56049ee6d1f0 0x56049ee782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56049ee68080 0x56049ee77380 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x56049eed9f60 0x56049eedc350 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 --2- 172.21.15.7:0/2531916819 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56049ee67690 0x56049ee76e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2531916819 >> 172.21.15.7:0/2531916819 conn(0x56049ee62c00 msgr2=0x56049ee65e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcae5ffb640 0 client.19140 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2531916819 shutdown_connections 2024-03-26T08:56:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 1 -- 172.21.15.7:0/2531916819 wait complete. 2024-03-26T08:56:31.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.171+0000 7fcb07bc8740 20 client.19140 trim_cache size 0 max 16384 2024-03-26T08:56:31.174 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:56:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.172+0000 7fcb04eff640 1 -- 172.21.15.7:0/1866981303 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcaec0a28d0 msgr2=0x7fcaec0a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.172+0000 7fcb04eff640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcaec0a28d0 0x7fcaec0a4d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8096e20 tx=0x7fcaf8009450 comp rx=0 tx=0).stop 2024-03-26T08:56:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.172+0000 7fcb04eff640 1 -- 172.21.15.7:0/1866981303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00109890 msgr2=0x7fcb001ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.172+0000 7fcb04eff640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00109890 0x7fcb001ae1d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800b970 tx=0x7fcaf800be30 comp rx=0 tx=0).stop 2024-03-26T08:56:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.173+0000 7fcb04eff640 1 -- 172.21.15.7:0/1866981303 shutdown_connections 2024-03-26T08:56:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.173+0000 7fcb04eff640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fcaec0a28d0 0x7fcaec0a4d90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.173+0000 7fcb04eff640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb00109890 0x7fcb001ae1d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.173+0000 7fcb04eff640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb00108ed0 0x7fcb0007f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.173+0000 7fcb04eff640 1 --2- 172.21.15.7:0/1866981303 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcb00107ce0 0x7fcb0007eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.173+0000 7fcb04eff640 1 -- 172.21.15.7:0/1866981303 >> 172.21.15.7:0/1866981303 conn(0x7fcb00076c90 msgr2=0x7fcb0010c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.173+0000 7fcb04eff640 1 -- 172.21.15.7:0/1866981303 shutdown_connections 2024-03-26T08:56:31.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.173+0000 7fcb04eff640 1 -- 172.21.15.7:0/1866981303 wait complete. 2024-03-26T08:56:31.185 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:56:31.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:56:31.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:31 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1574992516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:56:31.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:31 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1574992516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:56:31.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.354+0000 7fcced91b640 1 -- 172.21.15.7:0/888013175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce81086a0 msgr2=0x7fcce810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.354+0000 7fcce5ffb640 1 -- 172.21.15.7:0/888013175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8005220 con 0x7fcce81086a0 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.354+0000 7fcced91b640 1 --2- 172.21.15.7:0/888013175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce81086a0 0x7fcce810aad0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009900 tx=0x7fccd802bc60 comp rx=0 tx=0).stop 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.354+0000 7fcced91b640 1 -- 172.21.15.7:0/888013175 shutdown_connections 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.354+0000 7fcced91b640 1 --2- 172.21.15.7:0/888013175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce810d9c0 0x7fcce810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.354+0000 7fcced91b640 1 --2- 172.21.15.7:0/888013175 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce810b010 0x7fcce810d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.354+0000 7fcced91b640 1 --2- 172.21.15.7:0/888013175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce81086a0 0x7fcce810aad0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.354+0000 7fcced91b640 1 -- 172.21.15.7:0/888013175 >> 172.21.15.7:0/888013175 conn(0x7fcce806e640 msgr2=0x7fcce8070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.355+0000 7fcced91b640 1 -- 172.21.15.7:0/888013175 shutdown_connections 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.355+0000 7fcced91b640 1 -- 172.21.15.7:0/888013175 wait complete. 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.355+0000 7fcced91b640 1 Processor -- start 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.356+0000 7fcced91b640 1 -- start start 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.356+0000 7fcced91b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce81086a0 0x7fcce8114d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.356+0000 7fcced91b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce810b010 0x7fcce81152d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.356+0000 7fcced91b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce810d9c0 0x7fcce81b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.356+0000 7fcced91b640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce811c190 con 0x7fcce810b010 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.356+0000 7fcced91b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce811c010 con 0x7fcce810d9c0 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.356+0000 7fcced91b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce811c310 con 0x7fcce81086a0 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce810d9c0 0x7fcce81b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce810d9c0 0x7fcce81b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39842/0 (socket says 172.21.15.7:39842) 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce77fe640 1 -- 172.21.15.7:0/4219428916 learned_addr learned my addr 172.21.15.7:0/4219428916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce81086a0 0x7fcce8114d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce77fe640 1 -- 172.21.15.7:0/4219428916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce81086a0 msgr2=0x7fcce8114d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce77fe640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce81086a0 0x7fcce8114d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce67fc640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce810b010 0x7fcce81152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce77fe640 1 -- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce810b010 msgr2=0x7fcce81152d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce77fe640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce810b010 0x7fcce81152d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce77fe640 1 -- 172.21.15.7:0/4219428916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce81b6d80 con 0x7fcce810d9c0 2024-03-26T08:56:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce67fc640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce810b010 0x7fcce81152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:56:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.357+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce81086a0 0x7fcce8114d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:56:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.358+0000 7fcce77fe640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce810d9c0 0x7fcce81b6840 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcce000eaa0 tx=0x7fcce000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.358+0000 7fccec919640 1 -- 172.21.15.7:0/4219428916 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000cdb0 con 0x7fcce810d9c0 2024-03-26T08:56:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.358+0000 7fcced91b640 1 -- 172.21.15.7:0/4219428916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce81b6f50 con 0x7fcce810d9c0 2024-03-26T08:56:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.358+0000 7fcced91b640 1 -- 172.21.15.7:0/4219428916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce81b73b0 con 0x7fcce810d9c0 2024-03-26T08:56:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.359+0000 7fccec919640 1 -- 172.21.15.7:0/4219428916 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcce0005020 con 0x7fcce810d9c0 2024-03-26T08:56:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.359+0000 7fccec919640 1 -- 172.21.15.7:0/4219428916 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce00116b0 con 0x7fcce810d9c0 2024-03-26T08:56:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.362+0000 7fccec919640 1 -- 172.21.15.7:0/4219428916 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fcce00118f0 con 0x7fcce810d9c0 2024-03-26T08:56:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.362+0000 7fccf05e4740 1 Processor -- start 2024-03-26T08:56:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.362+0000 7fccec919640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fccd00a2a10 0x7fccd00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.363+0000 7fccf05e4740 1 -- start start 2024-03-26T08:56:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.363+0000 7fccec919640 1 -- 172.21.15.7:0/4219428916 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcce00cfcb0 con 0x7fcce810d9c0 2024-03-26T08:56:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.363+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a37cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.363+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a381b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.363+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 0x562b4a384680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.364+0000 7fcce6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a37cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.364+0000 7fcce6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a37cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37102/0 (socket says 172.21.15.7:37102) 2024-03-26T08:56:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.364+0000 7fcce6ffd640 1 -- 172.21.15.7:0/1868904027 learned_addr learned my addr 172.21.15.7:0/1868904027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.364+0000 7fcce77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 0x562b4a384680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.364+0000 7fccf05e4740 1 -- 172.21.15.7:0/1868904027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4a33f610 con 0x562b4a37c690 2024-03-26T08:56:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.364+0000 7fccf05e4740 1 -- 172.21.15.7:0/1868904027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4a2e23c0 con 0x562b4a3821f0 2024-03-26T08:56:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.364+0000 7fccf05e4740 1 -- 172.21.15.7:0/1868904027 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4a28ad60 con 0x562b4a37d080 2024-03-26T08:56:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.365+0000 7fcce6ffd640 1 -- 172.21.15.7:0/1868904027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 msgr2=0x562b4a381b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:56:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.365+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/1868904027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a381b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.365+0000 7fcce6ffd640 1 -- 172.21.15.7:0/1868904027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 msgr2=0x562b4a384680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.365+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/1868904027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 0x562b4a384680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.365+0000 7fcce6ffd640 1 -- 172.21.15.7:0/1868904027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b49ff63f0 con 0x562b4a37c690 2024-03-26T08:56:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.365+0000 7fcce67fc640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fccd00a2a10 0x7fccd00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.365+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/1868904027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a37cab0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000c00 tx=0x7fccd8009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85cdef890f50bc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.366+0000 7fcccdffb640 1 -- 172.21.15.7:0/1868904027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8044020 con 0x562b4a37c690 2024-03-26T08:56:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.366+0000 7fcccdffb640 1 -- 172.21.15.7:0/1868904027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccd8005520 con 0x562b4a37c690 2024-03-26T08:56:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.366+0000 7fcccdffb640 1 -- 172.21.15.7:0/1868904027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8004c60 con 0x562b4a37c690 2024-03-26T08:56:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 -- 172.21.15.7:0/1868904027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 msgr2=0x562b4a37cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1868904027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a37cab0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000c00 tx=0x7fccd8009400 comp rx=0 tx=0).stop 2024-03-26T08:56:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fcce67fc640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fccd00a2a10 0x7fccd00a4ed0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x562b4a37d480 tx=0x7fccdc005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 -- 172.21.15.7:0/1868904027 shutdown_connections 2024-03-26T08:56:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1868904027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 0x562b4a384680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1868904027 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a381b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1868904027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a37cab0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 -- 172.21.15.7:0/1868904027 >> 172.21.15.7:0/1868904027 conn(0x562b4a377c00 msgr2=0x562b4a378000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 -- 172.21.15.7:0/1868904027 shutdown_connections 2024-03-26T08:56:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 -- 172.21.15.7:0/1868904027 wait complete. 2024-03-26T08:56:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.367+0000 7fccf05e4740 1 -- start start 2024-03-26T08:56:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.368+0000 7fccf05e4740 1 Processor -- start 2024-03-26T08:56:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.368+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a386ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.368+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a386ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.368+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 0x562b4a38dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.368+0000 7fccf05e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4a29e240 con 0x562b4a37c690 2024-03-26T08:56:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.368+0000 7fccf05e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4a2de460 con 0x562b4a3821f0 2024-03-26T08:56:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.368+0000 7fccf05e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4a385140 con 0x562b4a37d080 2024-03-26T08:56:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.368+0000 7fccf05e4740 10 client.0 _ll_register_callbacks cb 0x562b4a29dc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.368+0000 7fcce6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a386ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce6ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a386ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37114/0 (socket says 172.21.15.7:37114) 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce6ffd640 1 -- 172.21.15.7:0/1920828713 learned_addr learned my addr 172.21.15.7:0/1920828713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce77fe640 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 0x562b4a38dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce67fc640 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a386ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce6ffd640 1 -- 172.21.15.7:0/1920828713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 msgr2=0x562b4a386ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a386ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce6ffd640 1 -- 172.21.15.7:0/1920828713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 msgr2=0x562b4a38dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 0x562b4a38dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce6ffd640 1 -- 172.21.15.7:0/1920828713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b4a0c07d0 con 0x562b4a37c690 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a386ab0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fccd8036040 tx=0x7fccd8004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a135dc152ce23f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.369+0000 7fccc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:56:31.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.370+0000 7fccc6ffd640 1 -- 172.21.15.7:0/1920828713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd804f020 con 0x562b4a37c690 2024-03-26T08:56:31.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.370+0000 7fccc6ffd640 1 -- 172.21.15.7:0/1920828713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fccd8004250 con 0x562b4a37c690 2024-03-26T08:56:31.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.370+0000 7fccf05e4740 1 -- 172.21.15.7:0/1920828713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccd8005520 con 0x562b4a37c690 2024-03-26T08:56:31.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.370+0000 7fccc6ffd640 1 -- 172.21.15.7:0/1920828713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccd8044040 con 0x562b4a37c690 2024-03-26T08:56:31.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.370+0000 7fccf05e4740 10 client.19149 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.370+0000 7fccf05e4740 1 -- 172.21.15.7:0/1920828713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b4a391330 con 0x562b4a37c690 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.371+0000 7fccc6ffd640 1 -- 172.21.15.7:0/1920828713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccd8004450 con 0x562b4a37c690 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.371+0000 7fccc6ffd640 1 -- 172.21.15.7:0/1920828713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fccd8040470 con 0x562b4a37c690 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.371+0000 7fccf05e4740 10 client.19149 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.371+0000 7fccf05e4740 10 client.19149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.371+0000 7fccf05e4740 20 client.19149 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.371+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562b4a392680 0x562b4a394a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.371+0000 7fccf05e4740 4 client.19149 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.371+0000 7fccf05e4740 1 -- 172.21.15.7:0/1920828713 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccd8005520 con 0x562b4a392680 2024-03-26T08:56:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.372+0000 7fcce67fc640 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562b4a392680 0x562b4a394a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.372+0000 7fccc6ffd640 10 client.19149 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.372+0000 7fcce67fc640 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562b4a392680 0x562b4a394a70 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f10d33416f35962 server_cookie=14275f9ef9505749 in_seq=0 out_seq=0 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccc6ffd640 1 -- 172.21.15.7:0/1920828713 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fccd8005520 con 0x562b4a392680 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccc6ffd640 10 client.19149 handle_command_reply: tid=0 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccf05e4740 1 client.19149 shutdown 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccf05e4740 1 -- 172.21.15.7:0/1920828713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 msgr2=0x562b4a386ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a386ab0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fccd8036040 tx=0x7fccd8004c10 comp rx=0 tx=0).stop 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccf05e4740 1 -- 172.21.15.7:0/1920828713 shutdown_connections 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562b4a392680 0x562b4a394a70 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccc6ffd640 0 client.19149 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a3821f0 0x562b4a38dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a386ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccf05e4740 1 --2- 172.21.15.7:0/1920828713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a37c690 0x562b4a386ab0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.373+0000 7fccf05e4740 1 -- 172.21.15.7:0/1920828713 >> 172.21.15.7:0/1920828713 conn(0x562b4a377c00 msgr2=0x562b4a376e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.374+0000 7fccf05e4740 1 -- 172.21.15.7:0/1920828713 shutdown_connections 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.374+0000 7fccf05e4740 1 -- 172.21.15.7:0/1920828713 wait complete. 2024-03-26T08:56:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.374+0000 7fccf05e4740 20 client.19149 trim_cache size 0 max 16384 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fccf05e4740 1 Processor -- start 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fccf05e4740 1 -- start start 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37c690 0x562b4a37b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37d080 0x562b4a38df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 0x562b4a3991d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fccf05e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd4010980 con 0x562b4a3821f0 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fccf05e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd400e480 con 0x562b4a37d080 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fccf05e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd400c500 con 0x562b4a37c690 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37d080 0x562b4a38df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37c690 0x562b4a37b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37d080 0x562b4a38df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39880/0 (socket says 172.21.15.7:39880) 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce67fc640 1 -- 172.21.15.7:0/314400532 learned_addr learned my addr 172.21.15.7:0/314400532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce77fe640 1 --2- 172.21.15.7:0/314400532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 0x562b4a3991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce67fc640 1 -- 172.21.15.7:0/314400532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37c690 msgr2=0x562b4a37b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce67fc640 1 --2- 172.21.15.7:0/314400532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37c690 0x562b4a37b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce67fc640 1 -- 172.21.15.7:0/314400532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 msgr2=0x562b4a3991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce67fc640 1 --2- 172.21.15.7:0/314400532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 0x562b4a3991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.389+0000 7fcce67fc640 1 -- 172.21.15.7:0/314400532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b4a378650 con 0x562b4a37d080 2024-03-26T08:56:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.390+0000 7fcce67fc640 1 --2- 172.21.15.7:0/314400532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37d080 0x562b4a38df50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x562b4a37daf0 tx=0x7fccdc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=173277fe87fe4483 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.390+0000 7fcccd7fa640 1 -- 172.21.15.7:0/314400532 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc02a070 con 0x562b4a37d080 2024-03-26T08:56:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.390+0000 7fcccd7fa640 1 -- 172.21.15.7:0/314400532 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccdc024250 con 0x562b4a37d080 2024-03-26T08:56:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.390+0000 7fcccd7fa640 1 -- 172.21.15.7:0/314400532 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00f730 con 0x562b4a37d080 2024-03-26T08:56:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.391+0000 7fccf05e4740 1 -- 172.21.15.7:0/314400532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37d080 msgr2=0x562b4a38df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.391+0000 7fccf05e4740 1 --2- 172.21.15.7:0/314400532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37d080 0x562b4a38df50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x562b4a37daf0 tx=0x7fccdc013000 comp rx=0 tx=0).stop 2024-03-26T08:56:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.391+0000 7fccf05e4740 1 -- 172.21.15.7:0/314400532 shutdown_connections 2024-03-26T08:56:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.391+0000 7fccf05e4740 1 --2- 172.21.15.7:0/314400532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 0x562b4a3991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.391+0000 7fccf05e4740 1 --2- 172.21.15.7:0/314400532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37d080 0x562b4a38df50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.391+0000 7fccf05e4740 1 --2- 172.21.15.7:0/314400532 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37c690 0x562b4a37b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.391+0000 7fccf05e4740 1 -- 172.21.15.7:0/314400532 >> 172.21.15.7:0/314400532 conn(0x562b4a377c00 msgr2=0x562b4a37ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.391+0000 7fccf05e4740 1 -- 172.21.15.7:0/314400532 shutdown_connections 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.391+0000 7fccf05e4740 1 -- 172.21.15.7:0/314400532 wait complete. 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.392+0000 7fccf05e4740 1 -- start start 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.392+0000 7fccf05e4740 1 Processor -- start 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37c690 0x562b4a38baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a38c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fccf05e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 0x562b4a38d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fccf05e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd400cb00 con 0x562b4a3821f0 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fccf05e4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd400de20 con 0x562b4a37c690 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fccf05e4740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd400d160 con 0x562b4a37d080 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37c690 0x562b4a38baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37c690 0x562b4a38baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39894/0 (socket says 172.21.15.7:39894) 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce6ffd640 1 -- 172.21.15.7:0/593252236 learned_addr learned my addr 172.21.15.7:0/593252236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce77fe640 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 0x562b4a38d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce67fc640 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a38c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fccf05e4740 10 client.0 _ll_register_callbacks cb 0x7fccd8005390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce6ffd640 1 -- 172.21.15.7:0/593252236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 msgr2=0x562b4a38c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a38c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce6ffd640 1 -- 172.21.15.7:0/593252236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 msgr2=0x562b4a38d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 0x562b4a38d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce6ffd640 1 -- 172.21.15.7:0/593252236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b4a38d630 con 0x562b4a37c690 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fcce6ffd640 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37c690 0x562b4a38baf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x562b4a37c3e0 tx=0x7fccd80406a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7070af2d2a386596 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.393+0000 7fccc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccc6ffd640 1 -- 172.21.15.7:0/593252236 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8063060 con 0x562b4a37c690 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccc6ffd640 1 -- 172.21.15.7:0/593252236 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fccd8040b80 con 0x562b4a37c690 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccf05e4740 1 -- 172.21.15.7:0/593252236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccdc024250 con 0x562b4a37c690 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccc6ffd640 1 -- 172.21.15.7:0/593252236 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccd804f070 con 0x562b4a37c690 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccf05e4740 10 client.28943 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccf05e4740 1 -- 172.21.15.7:0/593252236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b4a3ee150 con 0x562b4a37c690 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccc6ffd640 1 -- 172.21.15.7:0/593252236 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fccdc024250 con 0x562b4a37c690 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccc6ffd640 1 -- 172.21.15.7:0/593252236 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fccd805d030 con 0x562b4a37c690 2024-03-26T08:56:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccf05e4740 10 client.28943 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccf05e4740 10 client.28943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccf05e4740 20 client.28943 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccf05e4740 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562b4a3ee900 0x562b4a3f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccf05e4740 4 client.28943 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fccf05e4740 1 -- 172.21.15.7:0/593252236 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562b4a2ef900 con 0x562b4a3ee900 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.394+0000 7fcce67fc640 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562b4a3ee900 0x562b4a3f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fcce67fc640 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562b4a3ee900 0x562b4a3f0d20 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ab2d76fdc16b63 server_cookie=43de2b6ace71587f in_seq=0 out_seq=0 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccc6ffd640 10 client.28943 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccc6ffd640 1 -- 172.21.15.7:0/593252236 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562b4a2ef900 con 0x562b4a3ee900 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccc6ffd640 10 client.28943 handle_command_reply: tid=0 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 client.28943 shutdown 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 -- 172.21.15.7:0/593252236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37c690 msgr2=0x562b4a38baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37c690 0x562b4a38baf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x562b4a37c3e0 tx=0x7fccd80406a0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 -- 172.21.15.7:0/593252236 shutdown_connections 2024-03-26T08:56:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562b4a3ee900 0x562b4a3f0d20 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccc6ffd640 0 client.28943 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562b4a3821f0 0x562b4a38d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562b4a37d080 0x562b4a38c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 --2- 172.21.15.7:0/593252236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b4a37c690 0x562b4a38baf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 -- 172.21.15.7:0/593252236 >> 172.21.15.7:0/593252236 conn(0x562b4a377c00 msgr2=0x562b4a37b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 -- 172.21.15.7:0/593252236 shutdown_connections 2024-03-26T08:56:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 1 -- 172.21.15.7:0/593252236 wait complete. 2024-03-26T08:56:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.395+0000 7fccf05e4740 20 client.28943 trim_cache size 0 max 16384 2024-03-26T08:56:31.398 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:56:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 -- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fccd00a2a10 msgr2=0x7fccd00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fccd00a2a10 0x7fccd00a4ed0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x562b4a37d480 tx=0x7fccdc005d30 comp rx=0 tx=0).stop 2024-03-26T08:56:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 -- 172.21.15.7:0/4219428916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce810d9c0 msgr2=0x7fcce81b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce810d9c0 0x7fcce81b6840 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcce000eaa0 tx=0x7fcce000ef60 comp rx=0 tx=0).stop 2024-03-26T08:56:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 -- 172.21.15.7:0/4219428916 shutdown_connections 2024-03-26T08:56:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fccd00a2a10 0x7fccd00a4ed0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcce810d9c0 0x7fcce81b6840 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce810b010 0x7fcce81152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 --2- 172.21.15.7:0/4219428916 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcce81086a0 0x7fcce8114d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.398+0000 7fcced91b640 1 -- 172.21.15.7:0/4219428916 >> 172.21.15.7:0/4219428916 conn(0x7fcce806e640 msgr2=0x7fcce810b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.399+0000 7fcced91b640 1 -- 172.21.15.7:0/4219428916 shutdown_connections 2024-03-26T08:56:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:31.399+0000 7fcced91b640 1 -- 172.21.15.7:0/4219428916 wait complete. 2024-03-26T08:56:31.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:31 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1574992516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:56:32.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:32 smithi007 ceph-mon[32462]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-03-26T08:56:32.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:32 smithi044 ceph-mon[36461]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-03-26T08:56:32.409 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:56:32.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:56:32.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:56:32.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:32 smithi016 ceph-mon[35851]: pgmap v783: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.7 MiB/s wr, 614 op/s 2024-03-26T08:56:32.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 -- 172.21.15.7:0/2640193115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd89010d9c0 msgr2=0x7fd89010de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 --2- 172.21.15.7:0/2640193115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd89010d9c0 0x7fd89010de40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd88801acb0 tx=0x7fd88803d130 comp rx=0 tx=0).stop 2024-03-26T08:56:32.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 -- 172.21.15.7:0/2640193115 shutdown_connections 2024-03-26T08:56:32.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 --2- 172.21.15.7:0/2640193115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd89010d9c0 0x7fd89010de40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 --2- 172.21.15.7:0/2640193115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd89010b010 0x7fd89010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 --2- 172.21.15.7:0/2640193115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8901086a0 0x7fd89010aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 -- 172.21.15.7:0/2640193115 >> 172.21.15.7:0/2640193115 conn(0x7fd89006e640 msgr2=0x7fd890070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 -- 172.21.15.7:0/2640193115 shutdown_connections 2024-03-26T08:56:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 -- 172.21.15.7:0/2640193115 wait complete. 2024-03-26T08:56:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.574+0000 7fd8962dc640 1 Processor -- start 2024-03-26T08:56:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd8962dc640 1 -- start start 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd8962dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8901086a0 0x7fd89019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd8962dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd89010b010 0x7fd89019e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd8962dc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd89010d9c0 0x7fd8901a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd8962dc640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd890111940 con 0x7fd8901086a0 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd8962dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8901117c0 con 0x7fd89010b010 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd8962dc640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd890111ac0 con 0x7fd89010d9c0 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd88ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8901086a0 0x7fd89019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd88ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8901086a0 0x7fd89019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37136/0 (socket says 172.21.15.7:37136) 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd88ffff640 1 -- 172.21.15.7:0/3877223263 learned_addr learned my addr 172.21.15.7:0/3877223263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd88f7fe640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd89010b010 0x7fd89019e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd894879640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd89010d9c0 0x7fd8901a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd88ffff640 1 -- 172.21.15.7:0/3877223263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd89010d9c0 msgr2=0x7fd8901a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd88ffff640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd89010d9c0 0x7fd8901a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd88ffff640 1 -- 172.21.15.7:0/3877223263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd89010b010 msgr2=0x7fd89019e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd88ffff640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd89010b010 0x7fd89019e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.575+0000 7fd88ffff640 1 -- 172.21.15.7:0/3877223263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8901a5bf0 con 0x7fd8901086a0 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.576+0000 7fd88ffff640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8901086a0 0x7fd89019dfd0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd88400b4a0 tx=0x7fd88400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.576+0000 7fd88d7fa640 1 -- 172.21.15.7:0/3877223263 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884014070 con 0x7fd8901086a0 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.576+0000 7fd8962dc640 1 -- 172.21.15.7:0/3877223263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8901a5ee0 con 0x7fd8901086a0 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.576+0000 7fd88d7fa640 1 -- 172.21.15.7:0/3877223263 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd884010af0 con 0x7fd8901086a0 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.576+0000 7fd8962dc640 1 -- 172.21.15.7:0/3877223263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8901a6420 con 0x7fd8901086a0 2024-03-26T08:56:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.576+0000 7fd88d7fa640 1 -- 172.21.15.7:0/3877223263 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884017d10 con 0x7fd8901086a0 2024-03-26T08:56:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd898fa1740 1 Processor -- start 2024-03-26T08:56:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd898fa1740 1 -- start start 2024-03-26T08:56:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbefd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf0350 0x564b4cbf4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf54c0 0x564b4cbf7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd898fa1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4cb2fc90 con 0x564b4cbef960 2024-03-26T08:56:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd898fa1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4cb55070 con 0x564b4cbf0350 2024-03-26T08:56:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd898fa1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4cafdc80 con 0x564b4cbf54c0 2024-03-26T08:56:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd894879640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbefd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.579+0000 7fd88d7fa640 1 -- 172.21.15.7:0/3877223263 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd884017070 con 0x7fd8901086a0 2024-03-26T08:56:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd894879640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbefd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37142/0 (socket says 172.21.15.7:37142) 2024-03-26T08:56:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd894879640 1 -- 172.21.15.7:0/167384142 learned_addr learned my addr 172.21.15.7:0/167384142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd894879640 1 -- 172.21.15.7:0/167384142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf54c0 msgr2=0x564b4cbf7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd894879640 1 --2- 172.21.15.7:0/167384142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf54c0 0x564b4cbf7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd894879640 1 -- 172.21.15.7:0/167384142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf0350 msgr2=0x564b4cbf4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd894879640 1 --2- 172.21.15.7:0/167384142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf0350 0x564b4cbf4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd894879640 1 -- 172.21.15.7:0/167384142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b4caf5d10 con 0x564b4cbef960 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd88d7fa640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8780a2800 0x7fd8780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd88d7fa640 1 -- 172.21.15.7:0/3877223263 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd8840cdd20 con 0x7fd8901086a0 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.580+0000 7fd894879640 1 --2- 172.21.15.7:0/167384142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbefd80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd88801acb0 tx=0x7fd88801a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2efca7e4e1c06c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.581+0000 7fd88ffff640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8780a2800 0x7fd8780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.581+0000 7fd8767fc640 1 -- 172.21.15.7:0/167384142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888055030 con 0x564b4cbef960 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.581+0000 7fd8767fc640 1 -- 172.21.15.7:0/167384142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd888043600 con 0x564b4cbef960 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.581+0000 7fd8767fc640 1 -- 172.21.15.7:0/167384142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88804a040 con 0x564b4cbef960 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 -- 172.21.15.7:0/167384142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 msgr2=0x564b4cbefd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 --2- 172.21.15.7:0/167384142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbefd80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd88801acb0 tx=0x7fd88801a950 comp rx=0 tx=0).stop 2024-03-26T08:56:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd88ffff640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8780a2800 0x7fd8780a4cc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd88409a700 tx=0x7fd884009390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 -- 172.21.15.7:0/167384142 shutdown_connections 2024-03-26T08:56:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 --2- 172.21.15.7:0/167384142 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf54c0 0x564b4cbf7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 --2- 172.21.15.7:0/167384142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf0350 0x564b4cbf4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 --2- 172.21.15.7:0/167384142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbefd80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 -- 172.21.15.7:0/167384142 >> 172.21.15.7:0/167384142 conn(0x564b4cbb81a0 msgr2=0x564b4cbeb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 -- 172.21.15.7:0/167384142 shutdown_connections 2024-03-26T08:56:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 -- 172.21.15.7:0/167384142 wait complete. 2024-03-26T08:56:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.582+0000 7fd898fa1740 1 -- start start 2024-03-26T08:56:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd898fa1740 1 Processor -- start 2024-03-26T08:56:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbef960 0x564b4cbf9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 0x564b4cbfa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf54c0 0x564b4cc01290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd898fa1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4cb110d0 con 0x564b4cbf54c0 2024-03-26T08:56:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd898fa1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4cb51110 con 0x564b4cbef960 2024-03-26T08:56:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd898fa1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4cbf8460 con 0x564b4cbf0350 2024-03-26T08:56:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd894879640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf54c0 0x564b4cc01290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd88f7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 0x564b4cbfa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd894879640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf54c0 0x564b4cc01290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37146/0 (socket says 172.21.15.7:37146) 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd894879640 1 -- 172.21.15.7:0/2222728365 learned_addr learned my addr 172.21.15.7:0/2222728365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd898fa1740 10 client.0 _ll_register_callbacks cb 0x564b4cb4ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.583+0000 7fd894879640 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbef960 0x564b4cbf9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd894879640 1 -- 172.21.15.7:0/2222728365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 msgr2=0x564b4cbfa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd894879640 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 0x564b4cbfa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd894879640 1 -- 172.21.15.7:0/2222728365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbef960 msgr2=0x564b4cbf9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd894879640 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbef960 0x564b4cbf9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd894879640 1 -- 172.21.15.7:0/2222728365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b4c9317e0 con 0x564b4cbf54c0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd894879640 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf54c0 0x564b4cc01290 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd888048040 tx=0x7fd88803d130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7780e290ff3373bd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd86f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd86f7fe640 1 -- 172.21.15.7:0/2222728365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88804a030 con 0x564b4cbf54c0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd86f7fe640 1 -- 172.21.15.7:0/2222728365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd888004650 con 0x564b4cbf54c0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd898fa1740 1 -- 172.21.15.7:0/2222728365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd888043600 con 0x564b4cbf54c0 2024-03-26T08:56:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd86f7fe640 1 -- 172.21.15.7:0/2222728365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd888004850 con 0x564b4cbf54c0 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd898fa1740 10 client.19167 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd898fa1740 1 -- 172.21.15.7:0/2222728365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564b4c991f70 con 0x564b4cbf54c0 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.584+0000 7fd86f7fe640 1 -- 172.21.15.7:0/2222728365 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd888043600 con 0x564b4cbf54c0 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.585+0000 7fd86f7fe640 1 -- 172.21.15.7:0/2222728365 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd88803dce0 con 0x564b4cbf54c0 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.585+0000 7fd898fa1740 10 client.19167 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.585+0000 7fd898fa1740 10 client.19167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.585+0000 7fd898fa1740 20 client.19167 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.585+0000 7fd898fa1740 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564b4cc05bc0 0x564b4cc08000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.585+0000 7fd898fa1740 4 client.19167 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.585+0000 7fd898fa1740 1 -- 172.21.15.7:0/2222728365 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b4cb625b0 con 0x564b4cc05bc0 2024-03-26T08:56:32.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.585+0000 7fd88f7fe640 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564b4cc05bc0 0x564b4cc08000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.587+0000 7fd86f7fe640 10 client.19167 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.587+0000 7fd88f7fe640 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564b4cc05bc0 0x564b4cc08000 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6634389e444a0b3 server_cookie=7a5738ce7f49dced in_seq=0 out_seq=0 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.588+0000 7fd86f7fe640 1 -- 172.21.15.7:0/2222728365 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564b4cb625b0 con 0x564b4cc05bc0 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.588+0000 7fd86f7fe640 10 client.19167 handle_command_reply: tid=0 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 client.19167 shutdown 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 -- 172.21.15.7:0/2222728365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf54c0 msgr2=0x564b4cc01290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf54c0 0x564b4cc01290 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd888048040 tx=0x7fd88803d130 comp rx=0 tx=0).stop 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 -- 172.21.15.7:0/2222728365 shutdown_connections 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564b4cc05bc0 0x564b4cc08000 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf54c0 0x564b4cc01290 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 0x564b4cbfa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 --2- 172.21.15.7:0/2222728365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbef960 0x564b4cbf9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 -- 172.21.15.7:0/2222728365 >> 172.21.15.7:0/2222728365 conn(0x564b4cbb81a0 msgr2=0x564b4cbea530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd86f7fe640 0 client.19167 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 -- 172.21.15.7:0/2222728365 shutdown_connections 2024-03-26T08:56:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 1 -- 172.21.15.7:0/2222728365 wait complete. 2024-03-26T08:56:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.589+0000 7fd898fa1740 20 client.19167 trim_cache size 0 max 16384 2024-03-26T08:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.614+0000 7fd898fa1740 1 Processor -- start 2024-03-26T08:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.614+0000 7fd898fa1740 1 -- start start 2024-03-26T08:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.614+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.614+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 0x564b4cc01250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.614+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf54c0 0x564b4cbeea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.614+0000 7fd898fa1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8901b16a0 con 0x564b4cbef960 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.614+0000 7fd898fa1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8901af1a0 con 0x564b4cbf54c0 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.614+0000 7fd898fa1740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8901ad220 con 0x564b4cbf0350 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbee510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37160/0 (socket says 172.21.15.7:37160) 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 -- 172.21.15.7:0/1738059583 learned_addr learned my addr 172.21.15.7:0/1738059583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd88f7fe640 1 --2- 172.21.15.7:0/1738059583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 0x564b4cc01250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 --2- 172.21.15.7:0/1738059583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf54c0 0x564b4cbeea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 -- 172.21.15.7:0/1738059583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 msgr2=0x564b4cc01250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 --2- 172.21.15.7:0/1738059583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 0x564b4cc01250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 -- 172.21.15.7:0/1738059583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf54c0 msgr2=0x564b4cbeea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 --2- 172.21.15.7:0/1738059583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf54c0 0x564b4cbeea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 -- 172.21.15.7:0/1738059583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b4c829170 con 0x564b4cbef960 2024-03-26T08:56:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd894879640 1 --2- 172.21.15.7:0/1738059583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbee510 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd888047040 tx=0x7fd88801a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a084841f35ecbfcd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd875ffb640 1 -- 172.21.15.7:0/1738059583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88806c070 con 0x564b4cbef960 2024-03-26T08:56:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.615+0000 7fd875ffb640 1 -- 172.21.15.7:0/1738059583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd88806d040 con 0x564b4cbef960 2024-03-26T08:56:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.616+0000 7fd875ffb640 1 -- 172.21.15.7:0/1738059583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888055070 con 0x564b4cbef960 2024-03-26T08:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.616+0000 7fd898fa1740 1 -- 172.21.15.7:0/1738059583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 msgr2=0x564b4cbee510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.616+0000 7fd898fa1740 1 --2- 172.21.15.7:0/1738059583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbee510 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd888047040 tx=0x7fd88801a950 comp rx=0 tx=0).stop 2024-03-26T08:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.617+0000 7fd898fa1740 1 -- 172.21.15.7:0/1738059583 shutdown_connections 2024-03-26T08:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.617+0000 7fd898fa1740 1 --2- 172.21.15.7:0/1738059583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf54c0 0x564b4cbeea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.617+0000 7fd898fa1740 1 --2- 172.21.15.7:0/1738059583 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbf0350 0x564b4cc01250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.618+0000 7fd898fa1740 1 --2- 172.21.15.7:0/1738059583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbef960 0x564b4cbee510 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.618+0000 7fd898fa1740 1 -- 172.21.15.7:0/1738059583 >> 172.21.15.7:0/1738059583 conn(0x564b4cbb81a0 msgr2=0x564b4cbf1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.618+0000 7fd898fa1740 1 -- 172.21.15.7:0/1738059583 shutdown_connections 2024-03-26T08:56:32.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.618+0000 7fd898fa1740 1 -- 172.21.15.7:0/1738059583 wait complete. 2024-03-26T08:56:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.618+0000 7fd898fa1740 1 -- start start 2024-03-26T08:56:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd898fa1740 1 Processor -- start 2024-03-26T08:56:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbef960 0x564b4cbfee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd898fa1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf0350 0x564b4cbff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd894879640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbef960 0x564b4cbfee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd894879640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbef960 0x564b4cbfee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:57972/0 (socket says 172.21.15.7:57972) 2024-03-26T08:56:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd894879640 1 -- 172.21.15.7:0/3068439503 learned_addr learned my addr 172.21.15.7:0/3068439503 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd898fa1740 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf54c0 0x564b4cc00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8901ad820 con 0x564b4cbf0350 2024-03-26T08:56:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8901aeb40 con 0x564b4cbf54c0 2024-03-26T08:56:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.619+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8901ade80 con 0x564b4cbef960 2024-03-26T08:56:32.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.620+0000 7fd898fa1740 10 client.0 _ll_register_callbacks cb 0x7fd88801a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.620+0000 7fd894879640 1 -- 172.21.15.7:0/3068439503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf54c0 msgr2=0x564b4cc00350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.620+0000 7fd894879640 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf54c0 0x564b4cc00350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.620+0000 7fd894879640 1 -- 172.21.15.7:0/3068439503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf0350 msgr2=0x564b4cbff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.620+0000 7fd894879640 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf0350 0x564b4cbff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.620+0000 7fd894879640 1 -- 172.21.15.7:0/3068439503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b4cc00940 con 0x564b4cbef960 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.620+0000 7fd894879640 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbef960 0x564b4cbfee00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd88801acb0 tx=0x7fd88803d910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0049ecbfe9bb0d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.620+0000 7fd86f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.621+0000 7fd86f7fe640 1 -- 172.21.15.7:0/3068439503 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88806c070 con 0x564b4cbef960 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.621+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd88806d040 con 0x564b4cbef960 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.621+0000 7fd86f7fe640 1 -- 172.21.15.7:0/3068439503 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd888003ce0 con 0x564b4cbef960 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.621+0000 7fd86f7fe640 1 -- 172.21.15.7:0/3068439503 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd888055040 con 0x564b4cbef960 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.621+0000 7fd898fa1740 10 client.28945 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.621+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564b4cc61930 con 0x564b4cbef960 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.621+0000 7fd86f7fe640 1 -- 172.21.15.7:0/3068439503 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd88806d040 con 0x564b4cbef960 2024-03-26T08:56:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.622+0000 7fd86f7fe640 1 -- 172.21.15.7:0/3068439503 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd88803dce0 con 0x564b4cbef960 2024-03-26T08:56:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.622+0000 7fd898fa1740 10 client.28945 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.622+0000 7fd898fa1740 10 client.28945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.622+0000 7fd898fa1740 20 client.28945 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.622+0000 7fd898fa1740 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564b4cc62110 0x564b4cc64530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.622+0000 7fd898fa1740 4 client.28945 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:56:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.622+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564b4cb5c900 con 0x564b4cc62110 2024-03-26T08:56:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.622+0000 7fd88f7fe640 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564b4cc62110 0x564b4cc64530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.623+0000 7fd88f7fe640 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564b4cc62110 0x564b4cc64530 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e09aec903589e4 server_cookie=67f38dab2de7274d in_seq=0 out_seq=0 2024-03-26T08:56:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.623+0000 7fd86f7fe640 10 client.28945 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.624+0000 7fd86f7fe640 1 -- 172.21.15.7:0/3068439503 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564b4cb5c900 con 0x564b4cc62110 2024-03-26T08:56:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.624+0000 7fd86f7fe640 10 client.28945 handle_command_reply: tid=0 2024-03-26T08:56:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 client.28945 shutdown 2024-03-26T08:56:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbef960 msgr2=0x564b4cbfee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbef960 0x564b4cbfee00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd88801acb0 tx=0x7fd88803d910 comp rx=0 tx=0).stop 2024-03-26T08:56:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 shutdown_connections 2024-03-26T08:56:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x564b4cc62110 0x564b4cc64530 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd86f7fe640 0 client.28945 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b4cbf54c0 0x564b4cc00350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b4cbf0350 0x564b4cbff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 --2- 172.21.15.7:0/3068439503 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x564b4cbef960 0x564b4cbfee00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 >> 172.21.15.7:0/3068439503 conn(0x564b4cbb81a0 msgr2=0x564b4cbee510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 shutdown_connections 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 1 -- 172.21.15.7:0/3068439503 wait complete. 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.625+0000 7fd898fa1740 20 client.28945 trim_cache size 0 max 16384 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "d8fa7dc6-8b03-4303-b7db-416fd021b780", 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:56:32.624 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:56:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 -- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8780a2800 msgr2=0x7fd8780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8780a2800 0x7fd8780a4cc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd88409a700 tx=0x7fd884009390 comp rx=0 tx=0).stop 2024-03-26T08:56:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 -- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8901086a0 msgr2=0x7fd89019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8901086a0 0x7fd89019dfd0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd88400b4a0 tx=0x7fd88400b960 comp rx=0 tx=0).stop 2024-03-26T08:56:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 -- 172.21.15.7:0/3877223263 shutdown_connections 2024-03-26T08:56:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd8780a2800 0x7fd8780a4cc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd89010d9c0 0x7fd8901a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd89010b010 0x7fd89019e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 --2- 172.21.15.7:0/3877223263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8901086a0 0x7fd89019dfd0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 -- 172.21.15.7:0/3877223263 >> 172.21.15.7:0/3877223263 conn(0x7fd89006e640 msgr2=0x7fd890108db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 -- 172.21.15.7:0/3877223263 shutdown_connections 2024-03-26T08:56:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.626+0000 7fd8962dc640 1 -- 172.21.15.7:0/3877223263 wait complete. 2024-03-26T08:56:32.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:56:32.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:56:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.797+0000 7fc3b28d3640 1 -- 172.21.15.7:0/2030904312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ac109000 msgr2=0x7fc3ac10f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.797+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/2030904312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ac109000 0x7fc3ac10f9e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc394009900 tx=0x7fc39402bc40 comp rx=0 tx=0).stop 2024-03-26T08:56:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.797+0000 7fc3b28d3640 1 -- 172.21.15.7:0/2030904312 shutdown_connections 2024-03-26T08:56:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.797+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/2030904312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac10ff20 0x7fc3ac112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.797+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/2030904312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ac109000 0x7fc3ac10f9e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.798+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/2030904312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3ac1086a0 0x7fc3ac108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.798+0000 7fc3b28d3640 1 -- 172.21.15.7:0/2030904312 >> 172.21.15.7:0/2030904312 conn(0x7fc3ac06e640 msgr2=0x7fc3ac070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.798+0000 7fc3b28d3640 1 -- 172.21.15.7:0/2030904312 shutdown_connections 2024-03-26T08:56:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.798+0000 7fc3b28d3640 1 -- 172.21.15.7:0/2030904312 wait complete. 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.798+0000 7fc3b28d3640 1 Processor -- start 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.798+0000 7fc3b28d3640 1 -- start start 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3b28d3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac1086a0 0x7fc3ac1a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3b28d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3ac109000 0x7fc3ac1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3b28d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ac10ff20 0x7fc3ac1a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3b28d3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac115ae0 con 0x7fc3ac10ff20 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3b28d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac115960 con 0x7fc3ac109000 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3b28d3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac115c60 con 0x7fc3ac1086a0 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3ac109000 0x7fc3ac1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3b0e70640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ac10ff20 0x7fc3ac1a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3ac109000 0x7fc3ac1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39954/0 (socket says 172.21.15.7:39954) 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3a37fe640 1 -- 172.21.15.7:0/987818999 learned_addr learned my addr 172.21.15.7:0/987818999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3abfff640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac1086a0 0x7fc3ac1a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3a37fe640 1 -- 172.21.15.7:0/987818999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac1086a0 msgr2=0x7fc3ac1a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac1086a0 0x7fc3ac1a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3a37fe640 1 -- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ac10ff20 msgr2=0x7fc3ac1a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ac10ff20 0x7fc3ac1a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3a37fe640 1 -- 172.21.15.7:0/987818999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac1a7da0 con 0x7fc3ac109000 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.799+0000 7fc3b0e70640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ac10ff20 0x7fc3ac1a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.800+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3ac109000 0x7fc3ac1a06c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc394009230 tx=0x7fc394005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.800+0000 7fc3a9ffb640 1 -- 172.21.15.7:0/987818999 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394041070 con 0x7fc3ac109000 2024-03-26T08:56:32.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.800+0000 7fc3b28d3640 1 -- 172.21.15.7:0/987818999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ac1a8030 con 0x7fc3ac109000 2024-03-26T08:56:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.800+0000 7fc3b28d3640 1 -- 172.21.15.7:0/987818999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ac1a8570 con 0x7fc3ac109000 2024-03-26T08:56:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.801+0000 7fc3a9ffb640 1 -- 172.21.15.7:0/987818999 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3940054e0 con 0x7fc3ac109000 2024-03-26T08:56:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.801+0000 7fc3a9ffb640 1 -- 172.21.15.7:0/987818999 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39404b060 con 0x7fc3ac109000 2024-03-26T08:56:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.805+0000 7fc3b5595740 1 Processor -- start 2024-03-26T08:56:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.805+0000 7fc3a9ffb640 1 -- 172.21.15.7:0/987818999 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc394044070 con 0x7fc3ac109000 2024-03-26T08:56:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3b5595740 1 -- start start 2024-03-26T08:56:32.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e6005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e600ba0 0x55f51e6056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e605d10 0x55f51e6081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3b5595740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51e5613d0 con 0x55f51e605d10 2024-03-26T08:56:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3b5595740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51e509ad0 con 0x55f51e6001b0 2024-03-26T08:56:32.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3b5595740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51e55d470 con 0x55f51e600ba0 2024-03-26T08:56:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e600ba0 0x55f51e6056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e605d10 0x55f51e6081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e600ba0 0x55f51e6056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58006/0 (socket says 172.21.15.7:58006) 2024-03-26T08:56:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3b0e70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e6005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3abfff640 1 -- 172.21.15.7:0/2115929275 learned_addr learned my addr 172.21.15.7:0/2115929275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3a37fe640 1 -- 172.21.15.7:0/2115929275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e600ba0 msgr2=0x55f51e6056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/2115929275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e600ba0 0x55f51e6056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3a37fe640 1 -- 172.21.15.7:0/2115929275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 msgr2=0x55f51e6005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/2115929275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e6005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3a9ffb640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc3980a2880 0x7fc3980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.806+0000 7fc3a37fe640 1 -- 172.21.15.7:0/2115929275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f51e33f7d0 con 0x55f51e605d10 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.807+0000 7fc3b0e70640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc3980a2880 0x7fc3980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.807+0000 7fc3a9ffb640 1 -- 172.21.15.7:0/987818999 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc3940f02d0 con 0x7fc3ac109000 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.807+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/2115929275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e605d10 0x55f51e6081a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc3940edd70 tx=0x7fc3940b9080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdfb60bd0d68ba8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.807+0000 7fc3a27fc640 1 -- 172.21.15.7:0/2115929275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940bc860 con 0x55f51e605d10 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.807+0000 7fc3a27fc640 1 -- 172.21.15.7:0/2115929275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc394005690 con 0x55f51e605d10 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.807+0000 7fc3a27fc640 1 -- 172.21.15.7:0/2115929275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940bc680 con 0x55f51e605d10 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 -- 172.21.15.7:0/2115929275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e605d10 msgr2=0x55f51e6081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 --2- 172.21.15.7:0/2115929275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e605d10 0x55f51e6081a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc3940edd70 tx=0x7fc3940b9080 comp rx=0 tx=0).stop 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b0e70640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc3980a2880 0x7fc3980a4d40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55f51e601400 tx=0x7fc3a4009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 -- 172.21.15.7:0/2115929275 shutdown_connections 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 --2- 172.21.15.7:0/2115929275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e605d10 0x55f51e6081a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 --2- 172.21.15.7:0/2115929275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e600ba0 0x55f51e6056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 --2- 172.21.15.7:0/2115929275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e6005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 -- 172.21.15.7:0/2115929275 >> 172.21.15.7:0/2115929275 conn(0x55f51e5fb720 msgr2=0x55f51e5fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 -- 172.21.15.7:0/2115929275 shutdown_connections 2024-03-26T08:56:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 -- 172.21.15.7:0/2115929275 wait complete. 2024-03-26T08:56:32.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.808+0000 7fc3b5595740 1 -- start start 2024-03-26T08:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3b5595740 1 Processor -- start 2024-03-26T08:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e60a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e60a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e605d10 0x55f51e611920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3b5595740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51e608a80 con 0x55f51e600ba0 2024-03-26T08:56:32.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3b5595740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51e51d060 con 0x55f51e6001b0 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3b5595740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51e608c00 con 0x55f51e605d10 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3b5595740 10 client.0 _ll_register_callbacks cb 0x55f51e4ff6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e60a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e60a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39972/0 (socket says 172.21.15.7:39972) 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3abfff640 1 -- 172.21.15.7:0/668859772 learned_addr learned my addr 172.21.15.7:0/668859772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3b0e70640 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e605d10 0x55f51e611920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.809+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e60a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.810+0000 7fc3b0e70640 1 -- 172.21.15.7:0/668859772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 msgr2=0x55f51e60a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.810+0000 7fc3b0e70640 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e60a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.810+0000 7fc3b0e70640 1 -- 172.21.15.7:0/668859772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 msgr2=0x55f51e60a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.810+0000 7fc3b0e70640 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e60a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.810+0000 7fc3b0e70640 1 -- 172.21.15.7:0/668859772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f51e2753f0 con 0x55f51e605d10 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.810+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e60a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:56:32.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.810+0000 7fc3abfff640 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e60a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.810+0000 7fc3b0e70640 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e605d10 0x55f51e611920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4011690 tx=0x7fc3a4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac2031e63afdc31 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.810+0000 7fc38b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:56:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.811+0000 7fc38b7fe640 1 -- 172.21.15.7:0/668859772 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a402b060 con 0x55f51e605d10 2024-03-26T08:56:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.812+0000 7fc38b7fe640 1 -- 172.21.15.7:0/668859772 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc3a401c670 con 0x55f51e605d10 2024-03-26T08:56:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.812+0000 7fc3b5595740 1 -- 172.21.15.7:0/668859772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f51e56e910 con 0x55f51e605d10 2024-03-26T08:56:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.812+0000 7fc38b7fe640 1 -- 172.21.15.7:0/668859772 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3a401c870 con 0x55f51e605d10 2024-03-26T08:56:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.813+0000 7fc3b5595740 10 client.28957 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:32.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.813+0000 7fc3b5595740 1 -- 172.21.15.7:0/668859772 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f51e2350f0 con 0x55f51e605d10 2024-03-26T08:56:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.813+0000 7fc38b7fe640 1 -- 172.21.15.7:0/668859772 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f51e56e910 con 0x55f51e605d10 2024-03-26T08:56:32.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.813+0000 7fc38b7fe640 1 -- 172.21.15.7:0/668859772 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc3a401a040 con 0x55f51e605d10 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.813+0000 7fc3b5595740 10 client.28957 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.813+0000 7fc3b5595740 10 client.28957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.813+0000 7fc3b5595740 20 client.28957 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.813+0000 7fc3b5595740 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f51e615de0 0x55f51e618240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.813+0000 7fc3b5595740 4 client.28957 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.814+0000 7fc3b5595740 1 -- 172.21.15.7:0/668859772 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f51e568c60 con 0x55f51e615de0 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.814+0000 7fc3abfff640 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f51e615de0 0x55f51e618240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.814+0000 7fc3abfff640 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f51e615de0 0x55f51e618240 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e3790386b09bb server_cookie=a635fee32d0c8e4c in_seq=0 out_seq=0 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.814+0000 7fc38b7fe640 10 client.28957 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.815+0000 7fc38b7fe640 1 -- 172.21.15.7:0/668859772 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f51e568c60 con 0x55f51e615de0 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.815+0000 7fc38b7fe640 10 client.28957 handle_command_reply: tid=0 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.815+0000 7fc3b5595740 1 client.28957 shutdown 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 -- 172.21.15.7:0/668859772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e605d10 msgr2=0x55f51e611920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e605d10 0x55f51e611920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4011690 tx=0x7fc3a4004a40 comp rx=0 tx=0).stop 2024-03-26T08:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 -- 172.21.15.7:0/668859772 shutdown_connections 2024-03-26T08:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f51e615de0 0x55f51e618240 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e605d10 0x55f51e611920 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e60a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 --2- 172.21.15.7:0/668859772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e6001b0 0x55f51e60a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 -- 172.21.15.7:0/668859772 >> 172.21.15.7:0/668859772 conn(0x55f51e5fb720 msgr2=0x55f51e5f6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc38b7fe640 0 client.28957 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:32.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 -- 172.21.15.7:0/668859772 shutdown_connections 2024-03-26T08:56:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 1 -- 172.21.15.7:0/668859772 wait complete. 2024-03-26T08:56:32.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.816+0000 7fc3b5595740 20 client.28957 trim_cache size 0 max 16384 2024-03-26T08:56:32.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.838+0000 7fc3b5595740 1 Processor -- start 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.838+0000 7fc3b5595740 1 -- start start 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e6099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e6118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 0x55f51e5f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3b5595740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1b16b0 con 0x55f51e600ba0 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3b5595740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1af1b0 con 0x55f51e605d10 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3b5595740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1ad230 con 0x55f51e6001b0 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3a37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e6118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e6099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e6099c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58014/0 (socket says 172.21.15.7:58014) 2024-03-26T08:56:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3abfff640 1 -- 172.21.15.7:0/1556729298 learned_addr learned my addr 172.21.15.7:0/1556729298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3b0e70640 1 --2- 172.21.15.7:0/1556729298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 0x55f51e5f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3abfff640 1 -- 172.21.15.7:0/1556729298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 msgr2=0x55f51e5f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3abfff640 1 --2- 172.21.15.7:0/1556729298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 0x55f51e5f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3abfff640 1 -- 172.21.15.7:0/1556729298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 msgr2=0x55f51e6118e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3abfff640 1 --2- 172.21.15.7:0/1556729298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e6118e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.839+0000 7fc3abfff640 1 -- 172.21.15.7:0/1556729298 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f51e5fb2c0 con 0x55f51e6001b0 2024-03-26T08:56:32.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.841+0000 7fc3abfff640 1 --2- 172.21.15.7:0/1556729298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e6099c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0045e0 tx=0x7fc39c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91d8306bc979229 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.842+0000 7fc3a1ffb640 1 -- 172.21.15.7:0/1556729298 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c015020 con 0x55f51e6001b0 2024-03-26T08:56:32.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.842+0000 7fc3a1ffb640 1 -- 172.21.15.7:0/1556729298 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc39c01dde0 con 0x55f51e6001b0 2024-03-26T08:56:32.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.842+0000 7fc3a1ffb640 1 -- 172.21.15.7:0/1556729298 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0035f0 con 0x55f51e6001b0 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 -- 172.21.15.7:0/1556729298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 msgr2=0x55f51e6099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 --2- 172.21.15.7:0/1556729298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e6099c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0045e0 tx=0x7fc39c004610 comp rx=0 tx=0).stop 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 -- 172.21.15.7:0/1556729298 shutdown_connections 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 --2- 172.21.15.7:0/1556729298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 0x55f51e5f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 --2- 172.21.15.7:0/1556729298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e6118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 --2- 172.21.15.7:0/1556729298 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e6099c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 -- 172.21.15.7:0/1556729298 >> 172.21.15.7:0/1556729298 conn(0x55f51e5fb720 msgr2=0x55f51e602000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 -- 172.21.15.7:0/1556729298 shutdown_connections 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 -- 172.21.15.7:0/1556729298 wait complete. 2024-03-26T08:56:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 -- start start 2024-03-26T08:56:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.843+0000 7fc3b5595740 1 Processor -- start 2024-03-26T08:56:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e60f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e60f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3b5595740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 0x55f51e6107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3b5595740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1ad830 con 0x55f51e600ba0 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3b5595740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1aeb50 con 0x55f51e605d10 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3b5595740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1ade90 con 0x55f51e6001b0 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3b5595740 10 client.0 _ll_register_callbacks cb 0x7fc3a40115e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e60f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3abfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e60f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58024/0 (socket says 172.21.15.7:58024) 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3abfff640 1 -- 172.21.15.7:0/3165561311 learned_addr learned my addr 172.21.15.7:0/3165561311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e60f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3b0e70640 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 0x55f51e6107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3abfff640 1 -- 172.21.15.7:0/3165561311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 msgr2=0x55f51e6107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3abfff640 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 0x55f51e6107b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3abfff640 1 -- 172.21.15.7:0/3165561311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 msgr2=0x55f51e60f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3abfff640 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e60f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.844+0000 7fc3abfff640 1 -- 172.21.15.7:0/3165561311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f51e610da0 con 0x55f51e6001b0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc3abfff640 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e60f2b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc39c004b70 tx=0x7fc39c004a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f465c956bb89bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc38b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc38b7fe640 1 -- 172.21.15.7:0/3165561311 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c02c070 con 0x55f51e6001b0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc38b7fe640 1 -- 172.21.15.7:0/3165561311 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc39c005050 con 0x55f51e6001b0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc3b5595740 1 -- 172.21.15.7:0/3165561311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc39c01dde0 con 0x55f51e6001b0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc38b7fe640 1 -- 172.21.15.7:0/3165561311 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc39c038050 con 0x55f51e6001b0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc3b5595740 10 client.28963 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc3b5595740 1 -- 172.21.15.7:0/3165561311 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f51e671910 con 0x55f51e6001b0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc38b7fe640 1 -- 172.21.15.7:0/3165561311 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc39c01dde0 con 0x55f51e6001b0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc38b7fe640 1 -- 172.21.15.7:0/3165561311 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fc39c003d90 con 0x55f51e6001b0 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc3b5595740 10 client.28963 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc3b5595740 10 client.28963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.845+0000 7fc3b5595740 20 client.28963 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.846+0000 7fc3b5595740 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f51e6721f0 0x55f51e674610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.846+0000 7fc3b5595740 4 client.28963 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.846+0000 7fc3b5595740 1 -- 172.21.15.7:0/3165561311 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f51e4e64e0 con 0x55f51e6721f0 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.846+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f51e6721f0 0x55f51e674610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.846+0000 7fc38b7fe640 10 client.28963 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.846+0000 7fc3a37fe640 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f51e6721f0 0x55f51e674610 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d3faa534897a55 server_cookie=7fc0469e51b9509a in_seq=0 out_seq=0 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.846+0000 7fc38b7fe640 1 -- 172.21.15.7:0/3165561311 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f51e4e64e0 con 0x55f51e6721f0 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.846+0000 7fc38b7fe640 10 client.28963 handle_command_reply: tid=0 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.846+0000 7fc3b5595740 1 client.28963 shutdown 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 -- 172.21.15.7:0/3165561311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 msgr2=0x55f51e60f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e60f2b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc39c004b70 tx=0x7fc39c004a60 comp rx=0 tx=0).stop 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 -- 172.21.15.7:0/3165561311 shutdown_connections 2024-03-26T08:56:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f51e6721f0 0x55f51e674610 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f51e605d10 0x55f51e6107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc38b7fe640 0 client.28963 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f51e600ba0 0x55f51e60f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 --2- 172.21.15.7:0/3165561311 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f51e6001b0 0x55f51e60f2b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 -- 172.21.15.7:0/3165561311 >> 172.21.15.7:0/3165561311 conn(0x55f51e5fb720 msgr2=0x55f51e602000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 -- 172.21.15.7:0/3165561311 shutdown_connections 2024-03-26T08:56:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 1 -- 172.21.15.7:0/3165561311 wait complete. 2024-03-26T08:56:32.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.847+0000 7fc3b5595740 20 client.28963 trim_cache size 0 max 16384 2024-03-26T08:56:32.849 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:56:32.850 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T08:56:32.850 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:56:32.850 INFO:teuthology.orchestra.run.smithi007.stdout: "d8fa7dc6-8b03-4303-b7db-416fd021b780": { 2024-03-26T08:56:32.850 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:56:32.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "d8fa7dc6-8b03-4303-b7db-416fd021b780", 2024-03-26T08:56:32.850 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:56:32.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:56:32.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:56:32.850 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:56:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.848+0000 7fc3b28d3640 1 -- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc3980a2880 msgr2=0x7fc3980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.848+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc3980a2880 0x7fc3980a4d40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55f51e601400 tx=0x7fc3a4009600 comp rx=0 tx=0).stop 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.848+0000 7fc3b28d3640 1 -- 172.21.15.7:0/987818999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3ac109000 msgr2=0x7fc3ac1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.848+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3ac109000 0x7fc3ac1a06c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc394009230 tx=0x7fc394005dd0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.850+0000 7fc3b28d3640 1 -- 172.21.15.7:0/987818999 shutdown_connections 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.850+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc3980a2880 0x7fc3980a4d40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.850+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3ac10ff20 0x7fc3ac1a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.850+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3ac109000 0x7fc3ac1a06c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.850+0000 7fc3b28d3640 1 --2- 172.21.15.7:0/987818999 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3ac1086a0 0x7fc3ac1a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.850+0000 7fc3b28d3640 1 -- 172.21.15.7:0/987818999 >> 172.21.15.7:0/987818999 conn(0x7fc3ac06e640 msgr2=0x7fc3ac0702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.850+0000 7fc3b28d3640 1 -- 172.21.15.7:0/987818999 shutdown_connections 2024-03-26T08:56:32.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:56:32.850+0000 7fc3b28d3640 1 -- 172.21.15.7:0/987818999 wait complete. 2024-03-26T08:56:32.860 INFO:tasks.cephfs.filesystem:scrub status for tag:d8fa7dc6-8b03-4303-b7db-416fd021b780 - {'path': '/', 'tag': 'd8fa7dc6-8b03-4303-b7db-416fd021b780', 'options': 'recursive,force'} 2024-03-26T08:56:34.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:34 smithi007 ceph-mon[32462]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-03-26T08:56:34.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:34 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:56:34.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:34 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:34.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:34 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:56:34.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:34 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:56:34.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:34 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:34.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:34 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:34.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:34 smithi044 ceph-mon[36461]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-03-26T08:56:34.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:34 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:56:34.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:34 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:34.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:34 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:56:34.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:34 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:56:34.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:34 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:34.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:34 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:34.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:34 smithi016 ceph-mon[35851]: pgmap v784: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-03-26T08:56:34.500 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:34 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:56:34.500 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:34 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:34.500 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:34 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:56:34.500 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:34 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:56:34.500 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:34 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:34.500 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:34 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:56:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.617+0000 7f96d06f8640 1 -- 172.21.15.7:0/1200804464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 msgr2=0x7f96c81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:35.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.617+0000 7f96d06f8640 1 --2- 172.21.15.7:0/1200804464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 0x7f96c81013d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009900 tx=0x7f96c402cd40 comp rx=0 tx=0).stop 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.617+0000 7f96d06f8640 1 -- 172.21.15.7:0/1200804464 shutdown_connections 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.617+0000 7f96d06f8640 1 --2- 172.21.15.7:0/1200804464 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96c8101910 0x7f96c81081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.617+0000 7f96d06f8640 1 --2- 172.21.15.7:0/1200804464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 0x7f96c81013d0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.617+0000 7f96d06f8640 1 --2- 172.21.15.7:0/1200804464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c80ffd60 0x7f96c8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.617+0000 7f96d06f8640 1 -- 172.21.15.7:0/1200804464 >> 172.21.15.7:0/1200804464 conn(0x7f96c80fb2f0 msgr2=0x7f96c80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.617+0000 7f96d06f8640 1 -- 172.21.15.7:0/1200804464 shutdown_connections 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.617+0000 7f96d06f8640 1 -- 172.21.15.7:0/1200804464 wait complete. 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96d06f8640 1 Processor -- start 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96d06f8640 1 -- start start 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96d06f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96c80ffd60 0x7f96c8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96d06f8640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 0x7f96c8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96d06f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c8101910 0x7f96c819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96d06f8640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c810b1b0 con 0x7f96c8100f50 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96d06f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c810b030 con 0x7f96c8101910 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96d06f8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c810b330 con 0x7f96c80ffd60 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96cdc93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 0x7f96c8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96cdc93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 0x7f96c8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37224/0 (socket says 172.21.15.7:37224) 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96cdc93640 1 -- 172.21.15.7:0/291902237 learned_addr learned my addr 172.21.15.7:0/291902237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96ce494640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96c80ffd60 0x7f96c8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.618+0000 7f96cec95640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c8101910 0x7f96c819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96cdc93640 1 -- 172.21.15.7:0/291902237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96c80ffd60 msgr2=0x7f96c8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96cdc93640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96c80ffd60 0x7f96c8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96cdc93640 1 -- 172.21.15.7:0/291902237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c8101910 msgr2=0x7f96c819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96cdc93640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c8101910 0x7f96c819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96cdc93640 1 -- 172.21.15.7:0/291902237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c819f6b0 con 0x7f96c8100f50 2024-03-26T08:56:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96cdc93640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 0x7f96c8197fd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f96c4000c00 tx=0x7f96c4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96bf7fe640 1 -- 172.21.15.7:0/291902237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c4042020 con 0x7f96c8100f50 2024-03-26T08:56:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96d06f8640 1 -- 172.21.15.7:0/291902237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c81954a0 con 0x7f96c8100f50 2024-03-26T08:56:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96bf7fe640 1 -- 172.21.15.7:0/291902237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96c40054e0 con 0x7f96c8100f50 2024-03-26T08:56:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96bf7fe640 1 -- 172.21.15.7:0/291902237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40354e0 con 0x7f96c8100f50 2024-03-26T08:56:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.619+0000 7f96d06f8640 1 -- 172.21.15.7:0/291902237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c8195a40 con 0x7f96c8100f50 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.621+0000 7f96bf7fe640 1 -- 172.21.15.7:0/291902237 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f96c4045020 con 0x7f96c8100f50 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.622+0000 7f96d33bf740 1 Processor -- start 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.622+0000 7f96d33bf740 1 -- start start 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.622+0000 7f96bf7fe640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f96b40a28f0 0x7f96b40a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.622+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed83fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.622+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed844e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.622+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed845480 0x5575ed847910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.622+0000 7f96d33bf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ed7a4b10 con 0x5575ed845480 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96d33bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ed74d900 con 0x5575ed83f920 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96d33bf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ed7a0bb0 con 0x5575ed840310 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96bf7fe640 1 -- 172.21.15.7:0/291902237 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96c40f0b50 con 0x7f96c8100f50 2024-03-26T08:56:35.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96cdc93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed844e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96cdc93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed844e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:58052/0 (socket says 172.21.15.7:58052) 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96cdc93640 1 -- 172.21.15.7:0/2363302912 learned_addr learned my addr 172.21.15.7:0/2363302912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96ce494640 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed83fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96cec95640 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed845480 0x5575ed847910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96ce494640 1 -- 172.21.15.7:0/2363302912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 msgr2=0x5575ed844e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96ce494640 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed844e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96ce494640 1 -- 172.21.15.7:0/2363302912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed845480 msgr2=0x5575ed847910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96ce494640 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed845480 0x5575ed847910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96ce494640 1 -- 172.21.15.7:0/2363302912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ed4b9210 con 0x5575ed83f920 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96cec95640 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed845480 0x5575ed847910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96cec95640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f96b40a28f0 0x7f96b40a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96ce494640 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed83fd40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f96c8100db0 tx=0x7f96b800ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a078b1e3a5ed880e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.623+0000 7f96bcff9640 1 -- 172.21.15.7:0/2363302912 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8019400 con 0x5575ed83f920 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96bcff9640 1 -- 172.21.15.7:0/2363302912 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96b801e070 con 0x5575ed83f920 2024-03-26T08:56:35.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96bcff9640 1 -- 172.21.15.7:0/2363302912 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b80187a0 con 0x5575ed83f920 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96d33bf740 1 -- 172.21.15.7:0/2363302912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 msgr2=0x5575ed83fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96cec95640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f96b40a28f0 0x7f96b40a4db0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009900 tx=0x7f96c0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96d33bf740 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed83fd40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f96c8100db0 tx=0x7f96b800ff90 comp rx=0 tx=0).stop 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96d33bf740 1 -- 172.21.15.7:0/2363302912 shutdown_connections 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96d33bf740 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed845480 0x5575ed847910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96d33bf740 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed844e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96d33bf740 1 --2- 172.21.15.7:0/2363302912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed83fd40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96d33bf740 1 -- 172.21.15.7:0/2363302912 >> 172.21.15.7:0/2363302912 conn(0x5575ed83ae90 msgr2=0x5575ed83b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96d33bf740 1 -- 172.21.15.7:0/2363302912 shutdown_connections 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.624+0000 7f96d33bf740 1 -- 172.21.15.7:0/2363302912 wait complete. 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96d33bf740 1 -- start start 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96d33bf740 1 Processor -- start 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed849bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed84a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed845480 0x5575ed851150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96d33bf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ed848430 con 0x5575ed840310 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96d33bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ed760d50 con 0x5575ed83f920 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96d33bf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ed8485b0 con 0x5575ed845480 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96d33bf740 10 client.0 _ll_register_callbacks cb 0x5575ed760720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96ce494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed849bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96ce494640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed849bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40024/0 (socket says 172.21.15.7:40024) 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96cec95640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed845480 0x5575ed851150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96ce494640 1 -- 172.21.15.7:0/3672555754 learned_addr learned my addr 172.21.15.7:0/3672555754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96cdc93640 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed84a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96ce494640 1 -- 172.21.15.7:0/3672555754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed845480 msgr2=0x5575ed851150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96ce494640 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed845480 0x5575ed851150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96ce494640 1 -- 172.21.15.7:0/3672555754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 msgr2=0x5575ed84a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.625+0000 7f96ce494640 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed84a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96ce494640 1 -- 172.21.15.7:0/3672555754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575ed585620 con 0x5575ed83f920 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96ce494640 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed849bd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f96b80055f0 tx=0x7f96b8005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865a1eb2f3c4ce19 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96b1ffb640 1 -- 172.21.15.7:0/3672555754 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b80048c0 con 0x5575ed83f920 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96d33bf740 1 -- 172.21.15.7:0/3672555754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96b801e070 con 0x5575ed83f920 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96b1ffb640 1 -- 172.21.15.7:0/3672555754 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96b8004a60 con 0x5575ed83f920 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96b1ffb640 1 -- 172.21.15.7:0/3672555754 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96b8004ca0 con 0x5575ed83f920 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96d33bf740 10 client.28985 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96d33bf740 1 -- 172.21.15.7:0/3672555754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5575ed854510 con 0x5575ed83f920 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.626+0000 7f96b1ffb640 1 -- 172.21.15.7:0/3672555754 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96b801e070 con 0x5575ed83f920 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.627+0000 7f96b1ffb640 1 -- 172.21.15.7:0/3672555754 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f96b802c570 con 0x5575ed83f920 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.627+0000 7f96d33bf740 10 client.28985 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.627+0000 7f96d33bf740 10 client.28985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.627+0000 7f96d33bf740 20 client.28985 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.627+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5575ed8558c0 0x5575ed857cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.627+0000 7f96d33bf740 4 client.28985 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.627+0000 7f96d33bf740 1 -- 172.21.15.7:0/3672555754 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575ed7b2050 con 0x5575ed8558c0 2024-03-26T08:56:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.627+0000 7f96cdc93640 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5575ed8558c0 0x5575ed857cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.629+0000 7f96b1ffb640 10 client.28985 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.629+0000 7f96cdc93640 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5575ed8558c0 0x5575ed857cb0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879b57b6b0cc14e5 server_cookie=9f01b904d82f7917 in_seq=0 out_seq=0 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.630+0000 7f96b1ffb640 1 -- 172.21.15.7:0/3672555754 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5575ed7b2050 con 0x5575ed8558c0 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.630+0000 7f96b1ffb640 10 client.28985 handle_command_reply: tid=0 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.630+0000 7f96d33bf740 1 client.28985 shutdown 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.630+0000 7f96d33bf740 1 -- 172.21.15.7:0/3672555754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 msgr2=0x5575ed849bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.630+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed849bd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f96b80055f0 tx=0x7f96b8005620 comp rx=0 tx=0).stop 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96d33bf740 1 -- 172.21.15.7:0/3672555754 shutdown_connections 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5575ed8558c0 0x5575ed857cb0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96b1ffb640 0 client.28985 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed845480 0x5575ed851150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed84a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3672555754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed83f920 0x5575ed849bd0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96d33bf740 1 -- 172.21.15.7:0/3672555754 >> 172.21.15.7:0/3672555754 conn(0x5575ed83ae90 msgr2=0x5575ed83a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96d33bf740 1 -- 172.21.15.7:0/3672555754 shutdown_connections 2024-03-26T08:56:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96d33bf740 1 -- 172.21.15.7:0/3672555754 wait complete. 2024-03-26T08:56:35.634 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.631+0000 7f96d33bf740 20 client.28985 trim_cache size 0 max 16384 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.647+0000 7f96d33bf740 1 Processor -- start 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.647+0000 7f96d33bf740 1 -- start start 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed83f920 0x5575ed839790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed851110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed839cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96d33bf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c81a8ea0 con 0x5575ed840310 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96d33bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c81a69a0 con 0x5575ed845480 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96d33bf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c81a4a20 con 0x5575ed83f920 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cdc93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed851110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cec95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed839cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96ce494640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed83f920 0x5575ed839790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cdc93640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed851110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37256/0 (socket says 172.21.15.7:37256) 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cec95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed839cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40032/0 (socket says 172.21.15.7:40032) 2024-03-26T08:56:35.647 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cdc93640 1 -- 172.21.15.7:0/3126182017 learned_addr learned my addr 172.21.15.7:0/3126182017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cec95640 1 -- 172.21.15.7:0/3126182017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed83f920 msgr2=0x5575ed839790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cec95640 1 --2- 172.21.15.7:0/3126182017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed83f920 0x5575ed839790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cec95640 1 -- 172.21.15.7:0/3126182017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 msgr2=0x5575ed851110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cec95640 1 --2- 172.21.15.7:0/3126182017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed851110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cec95640 1 -- 172.21.15.7:0/3126182017 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ed8486e0 con 0x5575ed845480 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cdc93640 1 --2- 172.21.15.7:0/3126182017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed851110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.648+0000 7f96cec95640 1 --2- 172.21.15.7:0/3126182017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed839cd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f96c000ae00 tx=0x7f96c0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e1453f078d5abb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96b3fff640 1 -- 172.21.15.7:0/3126182017 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0008700 con 0x5575ed845480 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96b3fff640 1 -- 172.21.15.7:0/3126182017 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96c0029070 con 0x5575ed845480 2024-03-26T08:56:35.648 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96b3fff640 1 -- 172.21.15.7:0/3126182017 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0018c80 con 0x5575ed845480 2024-03-26T08:56:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96d33bf740 1 -- 172.21.15.7:0/3126182017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 msgr2=0x5575ed839cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3126182017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed839cd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f96c000ae00 tx=0x7f96c0016000 comp rx=0 tx=0).stop 2024-03-26T08:56:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96d33bf740 1 -- 172.21.15.7:0/3126182017 shutdown_connections 2024-03-26T08:56:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3126182017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed839cd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3126182017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed840310 0x5575ed851110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96d33bf740 1 --2- 172.21.15.7:0/3126182017 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed83f920 0x5575ed839790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96d33bf740 1 -- 172.21.15.7:0/3126182017 >> 172.21.15.7:0/3126182017 conn(0x5575ed83ae90 msgr2=0x5575ed83e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96d33bf740 1 -- 172.21.15.7:0/3126182017 shutdown_connections 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.649+0000 7f96d33bf740 1 -- 172.21.15.7:0/3126182017 wait complete. 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96d33bf740 1 -- start start 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96d33bf740 1 Processor -- start 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed83f920 0x5575ed84edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed84f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96d33bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed850340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96d33bf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c81a5020 con 0x5575ed83f920 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96d33bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c81a6340 con 0x5575ed845480 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96d33bf740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c81a5680 con 0x5575ed840310 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96d33bf740 10 client.0 _ll_register_callbacks cb 0x7f96b8005490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96ce494640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed83f920 0x5575ed84edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96cec95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed850340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.651 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96ce494640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed83f920 0x5575ed84edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37258/0 (socket says 172.21.15.7:37258) 2024-03-26T08:56:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96cdc93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed84f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96ce494640 1 -- 172.21.15.7:0/529121342 learned_addr learned my addr 172.21.15.7:0/529121342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96cec95640 1 -- 172.21.15.7:0/529121342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 msgr2=0x5575ed84f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.650+0000 7f96cec95640 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed84f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.651+0000 7f96cec95640 1 -- 172.21.15.7:0/529121342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed83f920 msgr2=0x5575ed84edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.651+0000 7f96cec95640 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed83f920 0x5575ed84edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.651+0000 7f96cec95640 1 -- 172.21.15.7:0/529121342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575ed850930 con 0x5575ed845480 2024-03-26T08:56:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.651+0000 7f96ce494640 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed83f920 0x5575ed84edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:35.652 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.651+0000 7f96cdc93640 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed84f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.652+0000 7f96cec95640 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed850340 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f96c001c010 tx=0x7f96c0019c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1741fca6fd5cd6c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.652+0000 7f96b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:56:35.653 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.653+0000 7f96b1ffb640 1 -- 172.21.15.7:0/529121342 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0008710 con 0x5575ed845480 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.653+0000 7f96b1ffb640 1 -- 172.21.15.7:0/529121342 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96c0019e70 con 0x5575ed845480 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.653+0000 7f96d33bf740 1 -- 172.21.15.7:0/529121342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96c0029070 con 0x5575ed845480 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.653+0000 7f96b1ffb640 1 -- 172.21.15.7:0/529121342 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96c00088b0 con 0x5575ed845480 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.653+0000 7f96d33bf740 10 client.28991 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.653+0000 7f96d33bf740 1 -- 172.21.15.7:0/529121342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5575ed8b15c0 con 0x5575ed845480 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.653+0000 7f96b1ffb640 1 -- 172.21.15.7:0/529121342 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96c0029070 con 0x5575ed845480 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96b1ffb640 1 -- 172.21.15.7:0/529121342 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f96c002d410 con 0x5575ed845480 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96d33bf740 10 client.28991 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96d33bf740 10 client.28991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96d33bf740 20 client.28991 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96d33bf740 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5575ed8558c0 0x5575ed8b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96d33bf740 4 client.28991 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96d33bf740 1 -- 172.21.15.7:0/529121342 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5575ed7ac3a0 con 0x5575ed8558c0 2024-03-26T08:56:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96ce494640 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5575ed8558c0 0x5575ed8b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96b1ffb640 10 client.28991 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:35.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.654+0000 7f96ce494640 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5575ed8558c0 0x5575ed8b3f70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b8d8940844ac84 server_cookie=9e0084256fc7d0f4 in_seq=0 out_seq=0 2024-03-26T08:56:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.684+0000 7f96b1ffb640 1 -- 172.21.15.7:0/529121342 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5575ed7ac3a0 con 0x5575ed8558c0 2024-03-26T08:56:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.684+0000 7f96b1ffb640 10 client.28991 handle_command_reply: tid=0 2024-03-26T08:56:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.684+0000 7f96d33bf740 1 client.28991 shutdown 2024-03-26T08:56:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.684+0000 7f96d33bf740 1 -- 172.21.15.7:0/529121342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 msgr2=0x5575ed850340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.684+0000 7f96d33bf740 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed850340 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f96c001c010 tx=0x7f96c0019c80 comp rx=0 tx=0).stop 2024-03-26T08:56:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d33bf740 1 -- 172.21.15.7:0/529121342 shutdown_connections 2024-03-26T08:56:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d33bf740 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5575ed8558c0 0x5575ed8b3f70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d33bf740 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5575ed845480 0x5575ed850340 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96b1ffb640 0 client.28991 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d33bf740 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5575ed840310 0x5575ed84f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d33bf740 1 --2- 172.21.15.7:0/529121342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5575ed83f920 0x5575ed84edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d33bf740 1 -- 172.21.15.7:0/529121342 >> 172.21.15.7:0/529121342 conn(0x5575ed83ae90 msgr2=0x5575ed839790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d33bf740 1 -- 172.21.15.7:0/529121342 shutdown_connections 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d33bf740 1 -- 172.21.15.7:0/529121342 wait complete. 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d33bf740 20 client.28991 trim_cache size 0 max 16384 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:56:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:56:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 -- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f96b40a28f0 msgr2=0x7f96b40a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f96b40a28f0 0x7f96b40a4db0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009900 tx=0x7f96c0004a40 comp rx=0 tx=0).stop 2024-03-26T08:56:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 -- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 msgr2=0x7f96c8197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 0x7f96c8197fd0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f96c4000c00 tx=0x7f96c4005d50 comp rx=0 tx=0).stop 2024-03-26T08:56:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 -- 172.21.15.7:0/291902237 shutdown_connections 2024-03-26T08:56:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f96b40a28f0 0x7f96b40a4db0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96c8101910 0x7f96c819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96c8100f50 0x7f96c8197fd0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 --2- 172.21.15.7:0/291902237 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f96c80ffd60 0x7f96c8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 -- 172.21.15.7:0/291902237 >> 172.21.15.7:0/291902237 conn(0x7f96c80fb2f0 msgr2=0x7f96c81042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 -- 172.21.15.7:0/291902237 shutdown_connections 2024-03-26T08:56:35.686 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:35.685+0000 7f96d06f8640 1 -- 172.21.15.7:0/291902237 wait complete. 2024-03-26T08:56:36.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 845.0 s, 1790.0 tps, lat 56.213 ms stddev 68.612 2024-03-26T08:56:36.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:36 smithi007 ceph-mon[32462]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.6 MiB/s wr, 622 op/s 2024-03-26T08:56:36.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:36 smithi044 ceph-mon[36461]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.6 MiB/s wr, 622 op/s 2024-03-26T08:56:36.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:36 smithi016 ceph-mon[35851]: pgmap v785: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.6 MiB/s wr, 622 op/s 2024-03-26T08:56:38.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:38 smithi007 ceph-mon[32462]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.2 MiB/s wr, 600 op/s 2024-03-26T08:56:38.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:38 smithi044 ceph-mon[36461]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.2 MiB/s wr, 600 op/s 2024-03-26T08:56:38.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:38 smithi016 ceph-mon[35851]: pgmap v786: 97 pgs: 97 active+clean; 1.6 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.2 MiB/s wr, 600 op/s 2024-03-26T08:56:40.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:40 smithi007 ceph-mon[32462]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.6 MiB/s wr, 714 op/s 2024-03-26T08:56:40.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:40 smithi044 ceph-mon[36461]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.6 MiB/s wr, 714 op/s 2024-03-26T08:56:40.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:40 smithi016 ceph-mon[35851]: pgmap v787: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.6 MiB/s wr, 714 op/s 2024-03-26T08:56:41.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 850.0 s, 1630.4 tps, lat 61.642 ms stddev 70.172 2024-03-26T08:56:42.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:42 smithi007 ceph-mon[32462]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 MiB/s wr, 738 op/s 2024-03-26T08:56:42.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:42 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:56:42.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:42 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:56:42.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:42 smithi044 ceph-mon[36461]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 MiB/s wr, 738 op/s 2024-03-26T08:56:42.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:42 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:56:42.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:42 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:56:42.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:42 smithi016 ceph-mon[35851]: pgmap v788: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 MiB/s wr, 738 op/s 2024-03-26T08:56:42.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:42 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:56:42.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:42 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:56:44.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:44 smithi007 ceph-mon[32462]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.1 MiB/s wr, 717 op/s 2024-03-26T08:56:44.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:44 smithi044 ceph-mon[36461]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.1 MiB/s wr, 717 op/s 2024-03-26T08:56:44.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:44 smithi016 ceph-mon[35851]: pgmap v789: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 7.1 MiB/s wr, 717 op/s 2024-03-26T08:56:46.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 855.0 s, 1776.2 tps, lat 56.118 ms stddev 67.321 2024-03-26T08:56:46.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:46 smithi007 ceph-mon[32462]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.6 MiB/s wr, 644 op/s 2024-03-26T08:56:46.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:46 smithi044 ceph-mon[36461]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.6 MiB/s wr, 644 op/s 2024-03-26T08:56:46.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:46 smithi016 ceph-mon[35851]: pgmap v790: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.6 MiB/s wr, 644 op/s 2024-03-26T08:56:48.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:48 smithi044 ceph-mon[36461]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.7 MiB/s wr, 638 op/s 2024-03-26T08:56:48.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:48 smithi016 ceph-mon[35851]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.7 MiB/s wr, 638 op/s 2024-03-26T08:56:48.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:48 smithi007 ceph-mon[32462]: pgmap v791: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 7.7 MiB/s wr, 638 op/s 2024-03-26T08:56:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 -- 172.21.15.7:0/1915619245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f138c1012d0 msgr2=0x7f138c101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 --2- 172.21.15.7:0/1915619245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f138c1012d0 0x7f138c101750 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f1388009900 tx=0x7f138802bc30 comp rx=0 tx=0).stop 2024-03-26T08:56:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 -- 172.21.15.7:0/1915619245 shutdown_connections 2024-03-26T08:56:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 --2- 172.21.15.7:0/1915619245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f138c1012d0 0x7f138c101750 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 --2- 172.21.15.7:0/1915619245 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c100930 0x7f138c100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 --2- 172.21.15.7:0/1915619245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c0fffd0 0x7f138c1003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 -- 172.21.15.7:0/1915619245 >> 172.21.15.7:0/1915619245 conn(0x7f138c0fbb70 msgr2=0x7f138c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 -- 172.21.15.7:0/1915619245 shutdown_connections 2024-03-26T08:56:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 -- 172.21.15.7:0/1915619245 wait complete. 2024-03-26T08:56:48.790 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.791+0000 7f139479d640 1 Processor -- start 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f139479d640 1 -- start start 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f139479d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f138c0fffd0 0x7f138c105b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f139479d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c100930 0x7f138c106090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f139479d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c1012d0 0x7f138c1026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f139479d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c10f570 con 0x7f138c0fffd0 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f139479d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c10f3f0 con 0x7f138c100930 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f139479d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c10f6f0 con 0x7f138c1012d0 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f1391d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c100930 0x7f138c106090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f1392d3a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c1012d0 0x7f138c1026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f1391d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c100930 0x7f138c106090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38844/0 (socket says 172.21.15.7:38844) 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f1391d38640 1 -- 172.21.15.7:0/569465574 learned_addr learned my addr 172.21.15.7:0/569465574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.792+0000 7f1392539640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f138c0fffd0 0x7f138c105b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f1392d3a640 1 -- 172.21.15.7:0/569465574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c100930 msgr2=0x7f138c106090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f1392d3a640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c100930 0x7f138c106090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f1392d3a640 1 -- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f138c0fffd0 msgr2=0x7f138c105b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f1392d3a640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f138c0fffd0 0x7f138c105b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f1392d3a640 1 -- 172.21.15.7:0/569465574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c102f80 con 0x7f138c1012d0 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f1391d38640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c100930 0x7f138c106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f1392d3a640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c1012d0 0x7f138c1026c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f138802c170 tx=0x7f138802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f13837fe640 1 -- 172.21.15.7:0/569465574 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388041070 con 0x7f138c1012d0 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f139479d640 1 -- 172.21.15.7:0/569465574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138c1a5fd0 con 0x7f138c1012d0 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f13837fe640 1 -- 172.21.15.7:0/569465574 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1388004450 con 0x7f138c1012d0 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f13837fe640 1 -- 172.21.15.7:0/569465574 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388034440 con 0x7f138c1012d0 2024-03-26T08:56:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.793+0000 7f139479d640 1 -- 172.21.15.7:0/569465574 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138c1a6380 con 0x7f138c1012d0 2024-03-26T08:56:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.796+0000 7f1397382740 1 Processor -- start 2024-03-26T08:56:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.796+0000 7f13837fe640 1 -- 172.21.15.7:0/569465574 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1388044070 con 0x7f138c1012d0 2024-03-26T08:56:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.796+0000 7f1397382740 1 -- start start 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1397382740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137ef920 0x55d4137efd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1397382740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f0310 0x55d4137f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1397382740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 0x55d4137f7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1397382740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d413754b10 con 0x55d4137f0310 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1397382740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4136fd900 con 0x55d4137ef920 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1397382740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d413750bb0 con 0x55d4137f5480 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137ef920 0x55d4137efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1392539640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f0310 0x55d4137f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137ef920 0x55d4137efd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38846/0 (socket says 172.21.15.7:38846) 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 -- 172.21.15.7:0/59241128 learned_addr learned my addr 172.21.15.7:0/59241128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 --2- 172.21.15.7:0/59241128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 0x55d4137f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.796 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f13837fe640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13780a27f0 0x7f13780a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 -- 172.21.15.7:0/59241128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 msgr2=0x55d4137f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 --2- 172.21.15.7:0/59241128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 0x55d4137f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 -- 172.21.15.7:0/59241128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f0310 msgr2=0x55d4137f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1392d3a640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13780a27f0 0x7f13780a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 --2- 172.21.15.7:0/59241128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f0310 0x55d4137f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 -- 172.21.15.7:0/59241128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d413469210 con 0x55d4137ef920 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1392539640 1 --2- 172.21.15.7:0/59241128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f0310 0x55d4137f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f13837fe640 1 -- 172.21.15.7:0/569465574 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f13880f0630 con 0x7f138c1012d0 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1391d38640 1 --2- 172.21.15.7:0/59241128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137ef920 0x55d4137efd40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f137c009810 tx=0x7f137c009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88ba763a70687434 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1380ff9640 1 -- 172.21.15.7:0/59241128 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00fc50 con 0x55d4137ef920 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1380ff9640 1 -- 172.21.15.7:0/59241128 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f137c00fdf0 con 0x55d4137ef920 2024-03-26T08:56:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.797+0000 7f1380ff9640 1 -- 172.21.15.7:0/59241128 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00e680 con 0x55d4137ef920 2024-03-26T08:56:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.798+0000 7f1397382740 1 -- 172.21.15.7:0/59241128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137ef920 msgr2=0x55d4137efd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.798+0000 7f1397382740 1 --2- 172.21.15.7:0/59241128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137ef920 0x55d4137efd40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f137c009810 tx=0x7f137c009c80 comp rx=0 tx=0).stop 2024-03-26T08:56:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.798+0000 7f1392d3a640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13780a27f0 0x7f13780a4cb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f138802cfd0 tx=0x7f13880b2520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.799+0000 7f1397382740 1 -- 172.21.15.7:0/59241128 shutdown_connections 2024-03-26T08:56:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.799+0000 7f1397382740 1 --2- 172.21.15.7:0/59241128 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 0x55d4137f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.799+0000 7f1397382740 1 --2- 172.21.15.7:0/59241128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f0310 0x55d4137f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.799+0000 7f1397382740 1 --2- 172.21.15.7:0/59241128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137ef920 0x55d4137efd40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.799+0000 7f1397382740 1 -- 172.21.15.7:0/59241128 >> 172.21.15.7:0/59241128 conn(0x55d4137eae90 msgr2=0x55d4137eb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 -- 172.21.15.7:0/59241128 shutdown_connections 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 -- 172.21.15.7:0/59241128 wait complete. 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 -- start start 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 Processor -- start 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137ef920 0x55d4137f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137fa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f5480 0x55d413801280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4137f8430 con 0x55d4137f5480 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d413710d50 con 0x55d4137f0310 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4137f85b0 con 0x55d4137ef920 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1397382740 10 client.0 _ll_register_callbacks cb 0x55d413710720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1391d38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f5480 0x55d413801280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.800+0000 7f1392539640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137fa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1391d38640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137ef920 0x55d4137f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.803 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1392539640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137fa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38860/0 (socket says 172.21.15.7:38860) 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1392539640 1 -- 172.21.15.7:0/1737720813 learned_addr learned my addr 172.21.15.7:0/1737720813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1392539640 1 -- 172.21.15.7:0/1737720813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137ef920 msgr2=0x55d4137f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1392539640 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137ef920 0x55d4137f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1392539640 1 -- 172.21.15.7:0/1737720813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f5480 msgr2=0x55d413801280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1392539640 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f5480 0x55d413801280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1392539640 1 -- 172.21.15.7:0/1737720813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d413535620 con 0x55d4137f0310 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1392539640 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137fa240 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1384011d10 tx=0x7f138400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d831dad8017b2c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1375ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1375ffb640 1 -- 172.21.15.7:0/1737720813 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384003970 con 0x55d4137f0310 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1375ffb640 1 -- 172.21.15.7:0/1737720813 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1384017050 con 0x55d4137f0310 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1397382740 1 -- 172.21.15.7:0/1737720813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f137c00fdf0 con 0x55d4137f0310 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1375ffb640 1 -- 172.21.15.7:0/1737720813 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1384003b10 con 0x55d4137f0310 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1397382740 10 client.29003 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.801+0000 7f1397382740 1 -- 172.21.15.7:0/1737720813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d413804430 con 0x55d4137f0310 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1375ffb640 1 -- 172.21.15.7:0/1737720813 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f137c00fdf0 con 0x55d4137f0310 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1375ffb640 1 -- 172.21.15.7:0/1737720813 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f1384019030 con 0x55d4137f0310 2024-03-26T08:56:48.804 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1397382740 10 client.29003 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1397382740 10 client.29003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1397382740 20 client.29003 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1397382740 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d413805bb0 0x55d413807fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1397382740 4 client.29003 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1397382740 1 -- 172.21.15.7:0/1737720813 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d413762050 con 0x55d413805bb0 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1391d38640 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d413805bb0 0x55d413807fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1391d38640 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d413805bb0 0x55d413807fd0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f23d601d6df0aba4 server_cookie=6bfa231f3648baf6 in_seq=0 out_seq=0 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.802+0000 7f1375ffb640 10 client.29003 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1375ffb640 1 -- 172.21.15.7:0/1737720813 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d413762050 con 0x55d413805bb0 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1375ffb640 10 client.29003 handle_command_reply: tid=0 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 client.29003 shutdown 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 -- 172.21.15.7:0/1737720813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 msgr2=0x55d4137fa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137fa240 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1384011d10 tx=0x7f138400f790 comp rx=0 tx=0).stop 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 -- 172.21.15.7:0/1737720813 shutdown_connections 2024-03-26T08:56:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d413805bb0 0x55d413807fd0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1375ffb640 0 client.29003 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137f5480 0x55d413801280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137fa240 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 --2- 172.21.15.7:0/1737720813 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137ef920 0x55d4137f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 -- 172.21.15.7:0/1737720813 >> 172.21.15.7:0/1737720813 conn(0x55d4137eae90 msgr2=0x55d4137ea020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 -- 172.21.15.7:0/1737720813 shutdown_connections 2024-03-26T08:56:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 1 -- 172.21.15.7:0/1737720813 wait complete. 2024-03-26T08:56:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.803+0000 7f1397382740 20 client.29003 trim_cache size 0 max 16384 2024-03-26T08:56:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.828+0000 7f1397382740 1 Processor -- start 2024-03-26T08:56:48.827 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.828+0000 7f1397382740 1 -- start start 2024-03-26T08:56:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.828+0000 7f1397382740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.828+0000 7f1397382740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f0310 0x55d413801240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.828+0000 7f1397382740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f5480 0x55d4137eb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.828+0000 7f1397382740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1b1570 con 0x55d4137ef920 2024-03-26T08:56:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.828+0000 7f1397382740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1af070 con 0x55d4137f5480 2024-03-26T08:56:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.829+0000 7f1397382740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1ad0f0 con 0x55d4137f0310 2024-03-26T08:56:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.832+0000 7f1391d38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f5480 0x55d4137eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.832+0000 7f1391d38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.832+0000 7f1391d38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137f0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32812/0 (socket says 172.21.15.7:32812) 2024-03-26T08:56:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.832+0000 7f1391d38640 1 -- 172.21.15.7:0/458284989 learned_addr learned my addr 172.21.15.7:0/458284989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:48.830 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.832+0000 7f1392539640 1 --2- 172.21.15.7:0/458284989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f0310 0x55d413801240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.834+0000 7f1392539640 1 -- 172.21.15.7:0/458284989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f5480 msgr2=0x55d4137eb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.834+0000 7f1392539640 1 --2- 172.21.15.7:0/458284989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f5480 0x55d4137eb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.834+0000 7f1392539640 1 -- 172.21.15.7:0/458284989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 msgr2=0x55d4137f0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.834+0000 7f1392539640 1 --2- 172.21.15.7:0/458284989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137f0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.834+0000 7f1392539640 1 -- 172.21.15.7:0/458284989 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d413808340 con 0x55d4137f0310 2024-03-26T08:56:48.832 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.834+0000 7f1392539640 1 --2- 172.21.15.7:0/458284989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f0310 0x55d413801240 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1384000f80 tx=0x7f1384005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2acc793b0124eb41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.835+0000 7f1377fff640 1 -- 172.21.15.7:0/458284989 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384029040 con 0x55d4137f0310 2024-03-26T08:56:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.835+0000 7f1377fff640 1 -- 172.21.15.7:0/458284989 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f138401a840 con 0x55d4137f0310 2024-03-26T08:56:48.833 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.835+0000 7f1377fff640 1 -- 172.21.15.7:0/458284989 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384005850 con 0x55d4137f0310 2024-03-26T08:56:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.837+0000 7f1397382740 1 -- 172.21.15.7:0/458284989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f0310 msgr2=0x55d413801240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.837+0000 7f1397382740 1 --2- 172.21.15.7:0/458284989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f0310 0x55d413801240 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1384000f80 tx=0x7f1384005f90 comp rx=0 tx=0).stop 2024-03-26T08:56:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.837+0000 7f1397382740 1 -- 172.21.15.7:0/458284989 shutdown_connections 2024-03-26T08:56:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.837+0000 7f1397382740 1 --2- 172.21.15.7:0/458284989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f5480 0x55d4137eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.837+0000 7f1397382740 1 --2- 172.21.15.7:0/458284989 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f0310 0x55d413801240 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.837+0000 7f1397382740 1 --2- 172.21.15.7:0/458284989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.835 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.837+0000 7f1397382740 1 -- 172.21.15.7:0/458284989 >> 172.21.15.7:0/458284989 conn(0x55d4137eae90 msgr2=0x55d4137e5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:48.840 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.842+0000 7f1397382740 1 -- 172.21.15.7:0/458284989 shutdown_connections 2024-03-26T08:56:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.844+0000 7f1397382740 1 -- 172.21.15.7:0/458284989 wait complete. 2024-03-26T08:56:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.844+0000 7f1397382740 1 -- start start 2024-03-26T08:56:48.842 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.844+0000 7f1397382740 1 Processor -- start 2024-03-26T08:56:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.845+0000 7f1397382740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137feae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.845+0000 7f1397382740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137ff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.845+0000 7f1397382740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 0x55d4137ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.845+0000 7f1397382740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1ad6f0 con 0x55d4137ef920 2024-03-26T08:56:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.845+0000 7f1397382740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1aea10 con 0x55d4137f0310 2024-03-26T08:56:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.845+0000 7f1397382740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1add50 con 0x55d4137f5480 2024-03-26T08:56:48.843 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.845+0000 7f1397382740 10 client.0 _ll_register_callbacks cb 0x7f1384011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:56:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.846+0000 7f1391d38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.846+0000 7f1391d38640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137feae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32824/0 (socket says 172.21.15.7:32824) 2024-03-26T08:56:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.846+0000 7f1391d38640 1 -- 172.21.15.7:0/460504337 learned_addr learned my addr 172.21.15.7:0/460504337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:56:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.846+0000 7f1391d38640 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 0x55d4137ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.847+0000 7f1392539640 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.847+0000 7f1391d38640 1 -- 172.21.15.7:0/460504337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 msgr2=0x55d4137ffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.847+0000 7f1391d38640 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 0x55d4137ffef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.847+0000 7f1391d38640 1 -- 172.21.15.7:0/460504337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 msgr2=0x55d4137ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:56:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.847+0000 7f1391d38640 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137ff020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.847+0000 7f1391d38640 1 -- 172.21.15.7:0/460504337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4138004e0 con 0x55d4137ef920 2024-03-26T08:56:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.847+0000 7f1391d38640 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137feae0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x55d4137f0b70 tx=0x7f137c009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dad4f4923ac9c60 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:56:48.846 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.848+0000 7f1375ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.848+0000 7f1375ffb640 1 -- 172.21.15.7:0/460504337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c028040 con 0x55d4137ef920 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.848+0000 7f1397382740 1 -- 172.21.15.7:0/460504337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f138401a840 con 0x55d4137ef920 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.848+0000 7f1375ffb640 1 -- 172.21.15.7:0/460504337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f137c019040 con 0x55d4137ef920 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.848+0000 7f1375ffb640 1 -- 172.21.15.7:0/460504337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f137c00fd60 con 0x55d4137ef920 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.848+0000 7f1397382740 10 client.19224 fetch_fsmap learned FSMap version 25 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.848+0000 7f1397382740 1 -- 172.21.15.7:0/460504337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d4138724f0 con 0x55d4137ef920 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.849+0000 7f1375ffb640 1 -- 172.21.15.7:0/460504337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f138401a840 con 0x55d4137ef920 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.849+0000 7f1375ffb640 1 -- 172.21.15.7:0/460504337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f137c02e040 con 0x55d4137ef920 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.849+0000 7f1397382740 10 client.19224 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.849+0000 7f1397382740 10 client.19224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:56:48.847 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.849+0000 7f1397382740 20 client.19224 populate_metadata read hostname 'smithi007' 2024-03-26T08:56:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.849+0000 7f1397382740 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d413805bb0 0x55d413874e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:56:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.849+0000 7f1397382740 4 client.19224 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:56:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.849+0000 7f1397382740 1 -- 172.21.15.7:0/460504337 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d41375c3a0 con 0x55d413805bb0 2024-03-26T08:56:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.849+0000 7f1392539640 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d413805bb0 0x55d413874e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:56:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.850+0000 7f1375ffb640 10 client.19224 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:48.848 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.850+0000 7f1392539640 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d413805bb0 0x55d413874e60 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96cc701f347aedd5 server_cookie=552b5529cb1c7a1a in_seq=0 out_seq=0 2024-03-26T08:56:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1375ffb640 1 -- 172.21.15.7:0/460504337 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d41375c3a0 con 0x55d413805bb0 2024-03-26T08:56:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1375ffb640 10 client.19224 handle_command_reply: tid=0 2024-03-26T08:56:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 client.19224 shutdown 2024-03-26T08:56:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 -- 172.21.15.7:0/460504337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 msgr2=0x55d4137feae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:48.877 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137feae0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x55d4137f0b70 tx=0x7f137c009c80 comp rx=0 tx=0).stop 2024-03-26T08:56:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 -- 172.21.15.7:0/460504337 shutdown_connections 2024-03-26T08:56:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d413805bb0 0x55d413874e60 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d4137f5480 0x55d4137ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d4137f0310 0x55d4137ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 --2- 172.21.15.7:0/460504337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4137ef920 0x55d4137feae0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1375ffb640 0 client.19224 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:56:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 -- 172.21.15.7:0/460504337 >> 172.21.15.7:0/460504337 conn(0x55d4137eae90 msgr2=0x55d4137f0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 -- 172.21.15.7:0/460504337 shutdown_connections 2024-03-26T08:56:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 1 -- 172.21.15.7:0/460504337 wait complete. 2024-03-26T08:56:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.879+0000 7f1397382740 20 client.19224 trim_cache size 0 max 16384 2024-03-26T08:56:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:56:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:56:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:56:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:56:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 -- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13780a27f0 msgr2=0x7f13780a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13780a27f0 0x7f13780a4cb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f138802cfd0 tx=0x7f13880b2520 comp rx=0 tx=0).stop 2024-03-26T08:56:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 -- 172.21.15.7:0/569465574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c1012d0 msgr2=0x7f138c1026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:56:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c1012d0 0x7f138c1026c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f138802c170 tx=0x7f138802c6c0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 -- 172.21.15.7:0/569465574 shutdown_connections 2024-03-26T08:56:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f13780a27f0 0x7f13780a4cb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f138c1012d0 0x7f138c1026c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c100930 0x7f138c106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 --2- 172.21.15.7:0/569465574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f138c0fffd0 0x7f138c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:56:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 -- 172.21.15.7:0/569465574 >> 172.21.15.7:0/569465574 conn(0x7f138c0fbb70 msgr2=0x7f138c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:56:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 -- 172.21.15.7:0/569465574 shutdown_connections 2024-03-26T08:56:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:56:48.880+0000 7f139479d640 1 -- 172.21.15.7:0/569465574 wait complete. 2024-03-26T08:56:50.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:50 smithi044 ceph-mon[36461]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.4 MiB/s wr, 699 op/s 2024-03-26T08:56:50.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:50 smithi016 ceph-mon[35851]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.4 MiB/s wr, 699 op/s 2024-03-26T08:56:50.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:50 smithi007 ceph-mon[32462]: pgmap v792: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 8.4 MiB/s wr, 699 op/s 2024-03-26T08:56:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 860.0 s, 1265.0 tps, lat 78.233 ms stddev 83.434 2024-03-26T08:56:52.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:52 smithi044 ceph-mon[36461]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8.3 MiB/s wr, 632 op/s 2024-03-26T08:56:52.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:56:52.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:52 smithi016 ceph-mon[35851]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8.3 MiB/s wr, 632 op/s 2024-03-26T08:56:52.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:56:52.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:52 smithi007 ceph-mon[32462]: pgmap v793: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8.3 MiB/s wr, 632 op/s 2024-03-26T08:56:52.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:56:53.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:56:53.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:56:53.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:56:53.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:56:53.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:56:53.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:56:53.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:56:53.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:56:53.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:56:54.322 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:54 smithi044 ceph-mon[36461]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.8 MiB/s wr, 595 op/s 2024-03-26T08:56:54.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:54 smithi016 ceph-mon[35851]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.8 MiB/s wr, 595 op/s 2024-03-26T08:56:54.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:54 smithi007 ceph-mon[32462]: pgmap v794: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.8 MiB/s wr, 595 op/s 2024-03-26T08:56:56.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 865.0 s, 1819.8 tps, lat 55.452 ms stddev 71.797 2024-03-26T08:56:56.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:56 smithi016 ceph-mon[35851]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.0 MiB/s wr, 617 op/s 2024-03-26T08:56:56.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:56 smithi007 ceph-mon[32462]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.0 MiB/s wr, 617 op/s 2024-03-26T08:56:56.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:56 smithi044 ceph-mon[36461]: pgmap v795: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 7.0 MiB/s wr, 617 op/s 2024-03-26T08:56:58.351 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:56:58 smithi007 ceph-mon[32462]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 MiB/s wr, 612 op/s 2024-03-26T08:56:58.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:56:58 smithi016 ceph-mon[35851]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 MiB/s wr, 612 op/s 2024-03-26T08:56:58.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:56:58 smithi044 ceph-mon[36461]: pgmap v796: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 MiB/s wr, 612 op/s 2024-03-26T08:57:00.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:00 smithi016 ceph-mon[35851]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 652 op/s 2024-03-26T08:57:00.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:00 smithi007 ceph-mon[32462]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 652 op/s 2024-03-26T08:57:00.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:00 smithi044 ceph-mon[36461]: pgmap v797: 97 pgs: 97 active+clean; 1.6 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.1 MiB/s wr, 652 op/s 2024-03-26T08:57:01.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 870.0 s, 1548.8 tps, lat 64.541 ms stddev 77.519 2024-03-26T08:57:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.001+0000 7fbd87e36640 1 -- 172.21.15.7:0/751870217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd80102970 msgr2=0x7fbd80102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.001+0000 7fbd87e36640 1 --2- 172.21.15.7:0/751870217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd80102970 0x7fbd80102df0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009900 tx=0x7fbd7002bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.001+0000 7fbd87e36640 1 -- 172.21.15.7:0/751870217 shutdown_connections 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.001+0000 7fbd87e36640 1 --2- 172.21.15.7:0/751870217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd80103330 0x7fbd80109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.001+0000 7fbd87e36640 1 --2- 172.21.15.7:0/751870217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd80102970 0x7fbd80102df0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.001+0000 7fbd87e36640 1 --2- 172.21.15.7:0/751870217 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd800ff9a0 0x7fbd800ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.001+0000 7fbd87e36640 1 -- 172.21.15.7:0/751870217 >> 172.21.15.7:0/751870217 conn(0x7fbd800fb2f0 msgr2=0x7fbd800fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.001+0000 7fbd87e36640 1 -- 172.21.15.7:0/751870217 shutdown_connections 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.001+0000 7fbd87e36640 1 -- 172.21.15.7:0/751870217 wait complete. 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd87e36640 1 Processor -- start 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd87e36640 1 -- start start 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd87e36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800ff9a0 0x7fbd801022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.002 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd87e36640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd80102970 0x7fbd80100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd853d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd80102970 0x7fbd80100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd853d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd80102970 0x7fbd80100920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33042/0 (socket says 172.21.15.7:33042) 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd853d1640 1 -- 172.21.15.7:0/115800716 learned_addr learned my addr 172.21.15.7:0/115800716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd85bd2640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800ff9a0 0x7fbd801022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd87e36640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd80103330 0x7fbd80100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8010d380 con 0x7fbd80102970 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8010d200 con 0x7fbd800ff9a0 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.002+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8010d500 con 0x7fbd80103330 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd863d3640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd80103330 0x7fbd80100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd85bd2640 1 -- 172.21.15.7:0/115800716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd80103330 msgr2=0x7fbd80100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd85bd2640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd80103330 0x7fbd80100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd85bd2640 1 -- 172.21.15.7:0/115800716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd80102970 msgr2=0x7fbd80100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd85bd2640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd80102970 0x7fbd80100920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd85bd2640 1 -- 172.21.15.7:0/115800716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd801016f0 con 0x7fbd800ff9a0 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd863d3640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd80103330 0x7fbd80100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:57:02.003 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd85bd2640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800ff9a0 0x7fbd801022e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00e9c0 tx=0x7fbd7c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd76ffd640 1 -- 172.21.15.7:0/115800716 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c00ccc0 con 0x7fbd800ff9a0 2024-03-26T08:57:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd76ffd640 1 -- 172.21.15.7:0/115800716 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd7c0035f0 con 0x7fbd800ff9a0 2024-03-26T08:57:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd801a1b20 con 0x7fbd800ff9a0 2024-03-26T08:57:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd76ffd640 1 -- 172.21.15.7:0/115800716 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c00ce60 con 0x7fbd800ff9a0 2024-03-26T08:57:02.004 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.003+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd801a2040 con 0x7fbd800ff9a0 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.007+0000 7fbd8aa27740 1 Processor -- start 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd8aa27740 1 -- start start 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feec920 0x55798feecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 0x55798fef1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 0x55798fef4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798fe51b10 con 0x55798fef2480 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798fdfa900 con 0x55798feec920 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798fe4dbb0 con 0x55798feed310 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd76ffd640 1 -- 172.21.15.7:0/115800716 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fbd7c0037a0 con 0x7fbd800ff9a0 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd863d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feec920 0x55798feecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd863d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feec920 0x55798feecd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45024/0 (socket says 172.21.15.7:45024) 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd863d3640 1 -- 172.21.15.7:0/3333096847 learned_addr learned my addr 172.21.15.7:0/3333096847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd85bd2640 1 --2- 172.21.15.7:0/3333096847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 0x55798fef1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd853d1640 1 --2- 172.21.15.7:0/3333096847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 0x55798fef4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd863d3640 1 -- 172.21.15.7:0/3333096847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 msgr2=0x55798fef1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd863d3640 1 --2- 172.21.15.7:0/3333096847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 0x55798fef1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd863d3640 1 -- 172.21.15.7:0/3333096847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 msgr2=0x55798fef4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd863d3640 1 --2- 172.21.15.7:0/3333096847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 0x55798fef4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.008+0000 7fbd863d3640 1 -- 172.21.15.7:0/3333096847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798fb66210 con 0x55798feec920 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd863d3640 1 --2- 172.21.15.7:0/3333096847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feec920 0x55798feecd40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009f90 tx=0x7fbd7800f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76e28a7eca5cc7e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd76ffd640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbd6c0a2870 0x7fbd6c0a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd6bfff640 1 -- 172.21.15.7:0/3333096847 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78003b70 con 0x55798feec920 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd6bfff640 1 -- 172.21.15.7:0/3333096847 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd78003d10 con 0x55798feec920 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd6bfff640 1 -- 172.21.15.7:0/3333096847 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78011a30 con 0x55798feec920 2024-03-26T08:57:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd85bd2640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbd6c0a2870 0x7fbd6c0a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3333096847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feec920 msgr2=0x55798feecd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3333096847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feec920 0x55798feecd40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009f90 tx=0x7fbd7800f7f0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3333096847 shutdown_connections 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3333096847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 0x55798fef4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3333096847 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 0x55798fef1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3333096847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feec920 0x55798feecd40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.009+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3333096847 >> 172.21.15.7:0/3333096847 conn(0x55798fee7e90 msgr2=0x55798fee8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.010+0000 7fbd76ffd640 1 -- 172.21.15.7:0/115800716 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbd7c0cf050 con 0x7fbd800ff9a0 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.010+0000 7fbd85bd2640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbd6c0a2870 0x7fbd6c0a4d30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c09b940 tx=0x7fbd7c0d2a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.010+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3333096847 shutdown_connections 2024-03-26T08:57:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.010+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3333096847 wait complete. 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.010+0000 7fbd8aa27740 1 -- start start 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.010+0000 7fbd8aa27740 1 Processor -- start 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fef6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feed310 0x55798fef7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 0x55798fefe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798fef5430 con 0x55798fef2480 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798fe0dd50 con 0x55798feed310 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798fef55b0 con 0x55798feec920 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd853d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feed310 0x55798fef7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd853d1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feed310 0x55798fef7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45034/0 (socket says 172.21.15.7:45034) 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd863d3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fef6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd853d1640 1 -- 172.21.15.7:0/3544646255 learned_addr learned my addr 172.21.15.7:0/3544646255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd8aa27740 10 client.0 _ll_register_callbacks cb 0x55798fe0d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd863d3640 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 0x55798fefe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd853d1640 1 -- 172.21.15.7:0/3544646255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 msgr2=0x55798fef6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd853d1640 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fef6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd853d1640 1 -- 172.21.15.7:0/3544646255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 msgr2=0x55798fefe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd853d1640 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 0x55798fefe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd853d1640 1 -- 172.21.15.7:0/3544646255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55798fc32620 con 0x55798feed310 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd853d1640 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feed310 0x55798fef7100 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009430 tx=0x7fbd70004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b9bf834cb3115a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd697fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd697fa640 1 -- 172.21.15.7:0/3544646255 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70005d50 con 0x55798feed310 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd697fa640 1 -- 172.21.15.7:0/3544646255 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbd700046c0 con 0x55798feed310 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.011+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3544646255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd78003d10 con 0x55798feed310 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd697fa640 1 -- 172.21.15.7:0/3544646255 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd700048c0 con 0x55798feed310 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd8aa27740 10 client.29018 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3544646255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55798ff014e0 con 0x55798feed310 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd697fa640 1 -- 172.21.15.7:0/3544646255 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd78003d10 con 0x55798feed310 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd697fa640 1 -- 172.21.15.7:0/3544646255 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbd70003790 con 0x55798feed310 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd8aa27740 10 client.29018 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd8aa27740 10 client.29018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd8aa27740 20 client.29018 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55798ff02b70 0x55798ff04f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd8aa27740 4 client.29018 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3544646255 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55798fe5f050 con 0x55798ff02b70 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.012+0000 7fbd863d3640 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55798ff02b70 0x55798ff04f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.013+0000 7fbd697fa640 10 client.29018 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.013+0000 7fbd863d3640 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55798ff02b70 0x55798ff04f80 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d766b6b5d72228 server_cookie=64afce459160e01d in_seq=0 out_seq=0 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.013+0000 7fbd697fa640 1 -- 172.21.15.7:0/3544646255 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55798fe5f050 con 0x55798ff02b70 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.013+0000 7fbd697fa640 10 client.29018 handle_command_reply: tid=0 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.013+0000 7fbd8aa27740 1 client.29018 shutdown 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3544646255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feed310 msgr2=0x55798fef7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feed310 0x55798fef7100 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009430 tx=0x7fbd70004270 comp rx=0 tx=0).stop 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3544646255 shutdown_connections 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55798ff02b70 0x55798ff04f80 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798fef2480 0x55798fefe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798feed310 0x55798fef7100 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/3544646255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fef6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3544646255 >> 172.21.15.7:0/3544646255 conn(0x55798fee7e90 msgr2=0x55798fee7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd697fa640 0 client.29018 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3544646255 shutdown_connections 2024-03-26T08:57:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 1 -- 172.21.15.7:0/3544646255 wait complete. 2024-03-26T08:57:02.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.014+0000 7fbd8aa27740 20 client.29018 trim_cache size 0 max 16384 2024-03-26T08:57:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.031+0000 7fbd8aa27740 1 Processor -- start 2024-03-26T08:57:02.030 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.031+0000 7fbd8aa27740 1 -- start start 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feec920 0x55798feee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 0x55798feeead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 0x55798fefe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd801ad230 con 0x55798feec920 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd801aad30 con 0x55798fef2480 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd801a8db0 con 0x55798feed310 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd863d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feec920 0x55798feee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd863d3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feec920 0x55798feee590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33066/0 (socket says 172.21.15.7:33066) 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd853d1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 0x55798feeead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd863d3640 1 -- 172.21.15.7:0/617306324 learned_addr learned my addr 172.21.15.7:0/617306324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd863d3640 1 --2- 172.21.15.7:0/617306324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 0x55798fefe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd853d1640 1 -- 172.21.15.7:0/617306324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 msgr2=0x55798fefe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd853d1640 1 --2- 172.21.15.7:0/617306324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 0x55798fefe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd853d1640 1 -- 172.21.15.7:0/617306324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feec920 msgr2=0x55798feee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:02.031 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd853d1640 1 --2- 172.21.15.7:0/617306324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feec920 0x55798feee590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd853d1640 1 -- 172.21.15.7:0/617306324 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798feedba0 con 0x55798feed310 2024-03-26T08:57:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd863d3640 1 --2- 172.21.15.7:0/617306324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feec920 0x55798feee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.032+0000 7fbd853d1640 1 --2- 172.21.15.7:0/617306324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 0x55798feeead0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbd70036040 tx=0x7fbd70009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcae28f7a97a963 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.033+0000 7fbd6b7fe640 1 -- 172.21.15.7:0/617306324 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70000e60 con 0x55798feed310 2024-03-26T08:57:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.034+0000 7fbd6b7fe640 1 -- 172.21.15.7:0/617306324 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd70009520 con 0x55798feed310 2024-03-26T08:57:02.032 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.034+0000 7fbd6b7fe640 1 -- 172.21.15.7:0/617306324 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70005250 con 0x55798feed310 2024-03-26T08:57:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.035+0000 7fbd8aa27740 1 -- 172.21.15.7:0/617306324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 msgr2=0x55798feeead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.035+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/617306324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 0x55798feeead0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbd70036040 tx=0x7fbd70009250 comp rx=0 tx=0).stop 2024-03-26T08:57:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.035+0000 7fbd8aa27740 1 -- 172.21.15.7:0/617306324 shutdown_connections 2024-03-26T08:57:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.035+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/617306324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 0x55798fefe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.035+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/617306324 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feed310 0x55798feeead0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.035+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/617306324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feec920 0x55798feee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.035+0000 7fbd8aa27740 1 -- 172.21.15.7:0/617306324 >> 172.21.15.7:0/617306324 conn(0x55798fee7e90 msgr2=0x55798fee8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.035+0000 7fbd8aa27740 1 -- 172.21.15.7:0/617306324 shutdown_connections 2024-03-26T08:57:02.034 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.035+0000 7fbd8aa27740 1 -- 172.21.15.7:0/617306324 wait complete. 2024-03-26T08:57:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd8aa27740 1 -- start start 2024-03-26T08:57:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd8aa27740 1 Processor -- start 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fefb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feed310 0x55798fefba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd8aa27740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 0x55798fefca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd8aa27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd801a93b0 con 0x55798feed310 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd853d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feed310 0x55798fefba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd863d3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fefb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd863d3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fefb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60134/0 (socket says 172.21.15.7:60134) 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd863d3640 1 -- 172.21.15.7:0/607734341 learned_addr learned my addr 172.21.15.7:0/607734341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd853d1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feed310 0x55798fefba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33078/0 (socket says 172.21.15.7:33078) 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd801aa6d0 con 0x55798fef2480 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd801a9a10 con 0x55798feec920 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd8aa27740 10 client.0 _ll_register_callbacks cb 0x7fbd7002cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.036+0000 7fbd863d3640 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 0x55798fefca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.037 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd863d3640 1 -- 172.21.15.7:0/607734341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 msgr2=0x55798fefca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd863d3640 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 0x55798fefca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd863d3640 1 -- 172.21.15.7:0/607734341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feed310 msgr2=0x55798fefba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd863d3640 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feed310 0x55798fefba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd863d3640 1 -- 172.21.15.7:0/607734341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55798fefd080 con 0x55798feec920 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd863d3640 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fefb540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009f90 tx=0x7fbd78009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a109b19982b1953f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd697fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd697fa640 1 -- 172.21.15.7:0/607734341 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78024040 con 0x55798feec920 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd697fa640 1 -- 172.21.15.7:0/607734341 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbd78037040 con 0x55798feec920 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd70009520 con 0x55798feec920 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd697fa640 1 -- 172.21.15.7:0/607734341 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd7802e880 con 0x55798feec920 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd8aa27740 10 client.29008 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.037+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55798fefe370 con 0x55798feec920 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd697fa640 1 -- 172.21.15.7:0/607734341 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd70009520 con 0x55798feec920 2024-03-26T08:57:02.038 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd697fa640 1 -- 172.21.15.7:0/607734341 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbd7803a020 con 0x55798feec920 2024-03-26T08:57:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd8aa27740 10 client.29008 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd8aa27740 10 client.29008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd8aa27740 20 client.29008 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55798ff5e930 0x55798ff60d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd8aa27740 4 client.29008 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:57:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55798fe593a0 con 0x55798ff5e930 2024-03-26T08:57:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd853d1640 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55798ff5e930 0x55798ff60d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd697fa640 10 client.29008 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:02.039 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.038+0000 7fbd853d1640 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55798ff5e930 0x55798ff60d50 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28532ea83921fb7 server_cookie=b92a9108ce8852ba in_seq=0 out_seq=0 2024-03-26T08:57:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd697fa640 1 -- 172.21.15.7:0/607734341 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55798fe593a0 con 0x55798ff5e930 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd697fa640 10 client.29008 handle_command_reply: tid=0 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 client.29008 shutdown 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 msgr2=0x55798fefb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fefb540 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009f90 tx=0x7fbd78009500 comp rx=0 tx=0).stop 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 shutdown_connections 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55798ff5e930 0x55798ff60d50 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798fef2480 0x55798fefca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd697fa640 0 client.29008 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55798feed310 0x55798fefba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 --2- 172.21.15.7:0/607734341 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55798feec920 0x55798fefb540 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 >> 172.21.15.7:0/607734341 conn(0x55798fee7e90 msgr2=0x55798fef6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 shutdown_connections 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 1 -- 172.21.15.7:0/607734341 wait complete. 2024-03-26T08:57:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.062+0000 7fbd8aa27740 20 client.29008 trim_cache size 0 max 16384 2024-03-26T08:57:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:57:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:57:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:57:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbd6c0a2870 msgr2=0x7fbd6c0a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbd6c0a2870 0x7fbd6c0a4d30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c09b940 tx=0x7fbd7c0d2a90 comp rx=0 tx=0).stop 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800ff9a0 msgr2=0x7fbd801022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800ff9a0 0x7fbd801022e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00e9c0 tx=0x7fbd7c00ee80 comp rx=0 tx=0).stop 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 shutdown_connections 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbd6c0a2870 0x7fbd6c0a4d30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbd80103330 0x7fbd80100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd80102970 0x7fbd80100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 --2- 172.21.15.7:0/115800716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd800ff9a0 0x7fbd801022e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 >> 172.21.15.7:0/115800716 conn(0x7fbd800fb2f0 msgr2=0x7fbd80105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 shutdown_connections 2024-03-26T08:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:02.063+0000 7fbd87e36640 1 -- 172.21.15.7:0/115800716 wait complete. 2024-03-26T08:57:02.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:02 smithi016 ceph-mon[35851]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 656 op/s 2024-03-26T08:57:02.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:02 smithi007 ceph-mon[32462]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 656 op/s 2024-03-26T08:57:02.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:02 smithi044 ceph-mon[36461]: pgmap v798: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.2 MiB/s wr, 656 op/s 2024-03-26T08:57:02.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:57:02.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:57:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.028+0000 7fa59240a640 1 -- 172.21.15.7:0/2832697031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c109680 msgr2=0x7fa58c10df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.028+0000 7fa59240a640 1 --2- 172.21.15.7:0/2832697031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c109680 0x7fa58c10df20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009ca0 tx=0x7fa57c02c140 comp rx=0 tx=0).stop 2024-03-26T08:57:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.030+0000 7fa59240a640 1 -- 172.21.15.7:0/2832697031 shutdown_connections 2024-03-26T08:57:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.030+0000 7fa59240a640 1 --2- 172.21.15.7:0/2832697031 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c109680 0x7fa58c10df20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.030+0000 7fa59240a640 1 --2- 172.21.15.7:0/2832697031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c108cc0 0x7fa58c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.030+0000 7fa59240a640 1 --2- 172.21.15.7:0/2832697031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c072a10 0x7fa58c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.030+0000 7fa59240a640 1 -- 172.21.15.7:0/2832697031 >> 172.21.15.7:0/2832697031 conn(0x7fa58c06e4e0 msgr2=0x7fa58c070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.037+0000 7fa59240a640 1 -- 172.21.15.7:0/2832697031 shutdown_connections 2024-03-26T08:57:03.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.039+0000 7fa59240a640 1 -- 172.21.15.7:0/2832697031 wait complete. 2024-03-26T08:57:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.039+0000 7fa59240a640 1 Processor -- start 2024-03-26T08:57:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.043+0000 7fa59240a640 1 -- start start 2024-03-26T08:57:03.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.043+0000 7fa59240a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c072a10 0x7fa58c116ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.043+0000 7fa59240a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c108cc0 0x7fa58c117220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.043+0000 7fa59240a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c109680 0x7fa58c1b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.044+0000 7fa59240a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c11bfe0 con 0x7fa58c109680 2024-03-26T08:57:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.044+0000 7fa59240a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c11be60 con 0x7fa58c072a10 2024-03-26T08:57:03.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.044+0000 7fa59240a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c11c160 con 0x7fa58c108cc0 2024-03-26T08:57:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.046+0000 7fa590c07640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c108cc0 0x7fa58c117220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.046+0000 7fa590c07640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c108cc0 0x7fa58c117220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60150/0 (socket says 172.21.15.7:60150) 2024-03-26T08:57:03.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.047+0000 7fa590c07640 1 -- 172.21.15.7:0/3125854000 learned_addr learned my addr 172.21.15.7:0/3125854000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.047+0000 7fa590c07640 1 -- 172.21.15.7:0/3125854000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c072a10 msgr2=0x7fa58c116ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T08:57:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.047+0000 7fa590c07640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c072a10 0x7fa58c116ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.047+0000 7fa590c07640 1 -- 172.21.15.7:0/3125854000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c109680 msgr2=0x7fa58c1b66f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:57:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.047+0000 7fa590c07640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c109680 0x7fa58c1b66f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.047+0000 7fa590c07640 1 -- 172.21.15.7:0/3125854000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c11a3a0 con 0x7fa58c108cc0 2024-03-26T08:57:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.049+0000 7fa590c07640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c108cc0 0x7fa58c117220 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c970 tx=0x7fa58400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.052+0000 7fa58a7fc640 1 -- 172.21.15.7:0/3125854000 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584007b90 con 0x7fa58c108cc0 2024-03-26T08:57:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.052+0000 7fa58a7fc640 1 -- 172.21.15.7:0/3125854000 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa584007d30 con 0x7fa58c108cc0 2024-03-26T08:57:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.052+0000 7fa58a7fc640 1 -- 172.21.15.7:0/3125854000 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58400f8a0 con 0x7fa58c108cc0 2024-03-26T08:57:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.053+0000 7fa59240a640 1 -- 172.21.15.7:0/3125854000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58c1b6cf0 con 0x7fa58c108cc0 2024-03-26T08:57:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.053+0000 7fa59240a640 1 -- 172.21.15.7:0/3125854000 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c1b71c0 con 0x7fa58c108cc0 2024-03-26T08:57:03.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.058+0000 7fa59624b740 1 Processor -- start 2024-03-26T08:57:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.058+0000 7fa59624b740 1 -- start start 2024-03-26T08:57:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.058+0000 7fa58a7fc640 1 -- 172.21.15.7:0/3125854000 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa58400fa40 con 0x7fa58c108cc0 2024-03-26T08:57:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.058+0000 7fa59624b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557370388fa0 0x5573703893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.058+0000 7fa59624b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370389990 0x55737038e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.058+0000 7fa59624b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55737038eb00 0x557370390f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.058+0000 7fa59624b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573702ee3d0 con 0x55737038eb00 2024-03-26T08:57:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.058+0000 7fa59624b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557370296ad0 con 0x557370389990 2024-03-26T08:57:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.058+0000 7fa59624b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573702ea470 con 0x557370388fa0 2024-03-26T08:57:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa590c07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55737038eb00 0x557370390f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa590c07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55737038eb00 0x557370390f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33092/0 (socket says 172.21.15.7:33092) 2024-03-26T08:57:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa590c07640 1 -- 172.21.15.7:0/1184238895 learned_addr learned my addr 172.21.15.7:0/1184238895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa591408640 1 --2- 172.21.15.7:0/1184238895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370389990 0x55737038e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa58a7fc640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5740a2a00 0x7fa5740a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa590c07640 1 -- 172.21.15.7:0/1184238895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557370388fa0 msgr2=0x5573703893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa590c07640 1 --2- 172.21.15.7:0/1184238895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557370388fa0 0x5573703893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa590c07640 1 -- 172.21.15.7:0/1184238895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370389990 msgr2=0x55737038e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa590c07640 1 --2- 172.21.15.7:0/1184238895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370389990 0x55737038e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.059+0000 7fa590c07640 1 -- 172.21.15.7:0/1184238895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573700cc7d0 con 0x55737038eb00 2024-03-26T08:57:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.060+0000 7fa58a7fc640 1 -- 172.21.15.7:0/3125854000 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa5840cece0 con 0x7fa58c108cc0 2024-03-26T08:57:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.060+0000 7fa590c07640 1 --2- 172.21.15.7:0/1184238895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55737038eb00 0x557370390f90 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa5840124f0 tx=0x7fa584021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507562ad536ea7c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.061+0000 7fa5737fe640 1 -- 172.21.15.7:0/1184238895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58409b210 con 0x55737038eb00 2024-03-26T08:57:03.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.061+0000 7fa5737fe640 1 -- 172.21.15.7:0/1184238895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5840d2070 con 0x55737038eb00 2024-03-26T08:57:03.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.061+0000 7fa5737fe640 1 -- 172.21.15.7:0/1184238895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840d23e0 con 0x55737038eb00 2024-03-26T08:57:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.063+0000 7fa591c09640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5740a2a00 0x7fa5740a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa591c09640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5740a2a00 0x7fa5740a4ec0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa57c006010 tx=0x7fa57c0323d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 -- 172.21.15.7:0/1184238895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55737038eb00 msgr2=0x557370390f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 --2- 172.21.15.7:0/1184238895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55737038eb00 0x557370390f90 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fa5840124f0 tx=0x7fa584021000 comp rx=0 tx=0).stop 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 -- 172.21.15.7:0/1184238895 shutdown_connections 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 --2- 172.21.15.7:0/1184238895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55737038eb00 0x557370390f90 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 --2- 172.21.15.7:0/1184238895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370389990 0x55737038e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 --2- 172.21.15.7:0/1184238895 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557370388fa0 0x5573703893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 -- 172.21.15.7:0/1184238895 >> 172.21.15.7:0/1184238895 conn(0x557370384510 msgr2=0x557370384910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 -- 172.21.15.7:0/1184238895 shutdown_connections 2024-03-26T08:57:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 -- 172.21.15.7:0/1184238895 wait complete. 2024-03-26T08:57:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 -- start start 2024-03-26T08:57:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.064+0000 7fa59624b740 1 Processor -- start 2024-03-26T08:57:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa59624b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370388fa0 0x557370393420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa59624b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x557370393960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa59624b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 0x55737039a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa590c07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x557370393960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa590c07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x557370393960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33096/0 (socket says 172.21.15.7:33096) 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa590c07640 1 -- 172.21.15.7:0/502276272 learned_addr learned my addr 172.21.15.7:0/502276272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557370391a50 con 0x557370389990 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573702aa060 con 0x557370388fa0 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557370391bd0 con 0x55737038eb00 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa591c09640 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 0x55737039a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.065+0000 7fa59624b740 10 client.0 _ll_register_callbacks cb 0x55737028c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa590c07640 1 -- 172.21.15.7:0/502276272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 msgr2=0x55737039a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa590c07640 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 0x55737039a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa590c07640 1 -- 172.21.15.7:0/502276272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370388fa0 msgr2=0x557370393420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa590c07640 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370388fa0 0x557370393420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa590c07640 1 -- 172.21.15.7:0/502276272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573700023f0 con 0x557370389990 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa590c07640 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x557370393960 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa584023030 tx=0x7fa584018ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57988b186181554 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa570ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa570ff9640 1 -- 172.21.15.7:0/502276272 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58409b030 con 0x557370389990 2024-03-26T08:57:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa570ff9640 1 -- 172.21.15.7:0/502276272 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa584097560 con 0x557370389990 2024-03-26T08:57:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.066+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5840d2070 con 0x557370389990 2024-03-26T08:57:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.067+0000 7fa570ff9640 1 -- 172.21.15.7:0/502276272 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa58409b210 con 0x557370389990 2024-03-26T08:57:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.067+0000 7fa59624b740 10 client.19248 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.067+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55736ffc20f0 con 0x557370389990 2024-03-26T08:57:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.068+0000 7fa570ff9640 1 -- 172.21.15.7:0/502276272 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5840d2070 con 0x557370389990 2024-03-26T08:57:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.068+0000 7fa570ff9640 1 -- 172.21.15.7:0/502276272 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa5840d5060 con 0x557370389990 2024-03-26T08:57:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.068+0000 7fa59624b740 10 client.19248 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.068+0000 7fa59624b740 10 client.19248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.068+0000 7fa59624b740 20 client.19248 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.068+0000 7fa59624b740 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55737039eae0 0x5573703a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.068+0000 7fa59624b740 4 client.19248 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.068+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573702fb910 con 0x55737039eae0 2024-03-26T08:57:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.069+0000 7fa591408640 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55737039eae0 0x5573703a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.071+0000 7fa570ff9640 10 client.19248 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.071+0000 7fa591408640 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55737039eae0 0x5573703a0ed0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d0288700ea7793 server_cookie=aabe51b13ed3ec7e in_seq=0 out_seq=0 2024-03-26T08:57:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa570ff9640 1 -- 172.21.15.7:0/502276272 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5573702fb910 con 0x55737039eae0 2024-03-26T08:57:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa570ff9640 10 client.19248 handle_command_reply: tid=0 2024-03-26T08:57:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 client.19248 shutdown 2024-03-26T08:57:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 msgr2=0x557370393960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x557370393960 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fa584023030 tx=0x7fa584018ef0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 shutdown_connections 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55737039eae0 0x5573703a0ed0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 0x55737039a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa570ff9640 0 client.19248 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x557370393960 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 --2- 172.21.15.7:0/502276272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370388fa0 0x557370393420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 >> 172.21.15.7:0/502276272 conn(0x557370384510 msgr2=0x5573703837b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 shutdown_connections 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.072+0000 7fa59624b740 1 -- 172.21.15.7:0/502276272 wait complete. 2024-03-26T08:57:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.073+0000 7fa59624b740 20 client.19248 trim_cache size 0 max 16384 2024-03-26T08:57:03.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.096+0000 7fa59624b740 1 Processor -- start 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.096+0000 7fa59624b740 1 -- start start 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa59624b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370388fa0 0x55737038ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa59624b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x55737038b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa59624b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 0x55737039a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa59624b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1c2460 con 0x557370389990 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa59624b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1bff60 con 0x557370388fa0 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa59624b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1bdfc0 con 0x55737038eb00 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa590c07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x55737038b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa591408640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370388fa0 0x55737038ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa590c07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x55737038b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33104/0 (socket says 172.21.15.7:33104) 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa590c07640 1 -- 172.21.15.7:0/3869439575 learned_addr learned my addr 172.21.15.7:0/3869439575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa591c09640 1 --2- 172.21.15.7:0/3869439575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 0x55737039a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa590c07640 1 -- 172.21.15.7:0/3869439575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 msgr2=0x55737039a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa590c07640 1 --2- 172.21.15.7:0/3869439575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 0x55737039a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa590c07640 1 -- 172.21.15.7:0/3869439575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370388fa0 msgr2=0x55737038ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa590c07640 1 --2- 172.21.15.7:0/3869439575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370388fa0 0x55737038ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa590c07640 1 -- 172.21.15.7:0/3869439575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737038a220 con 0x557370389990 2024-03-26T08:57:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.097+0000 7fa590c07640 1 --2- 172.21.15.7:0/3869439575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x55737038b3d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa584097b30 tx=0x7fa584018df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe784c4d1e0f82a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa572ffd640 1 -- 172.21.15.7:0/3869439575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840d2040 con 0x557370389990 2024-03-26T08:57:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa572ffd640 1 -- 172.21.15.7:0/3869439575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5840b8070 con 0x557370389990 2024-03-26T08:57:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa572ffd640 1 -- 172.21.15.7:0/3869439575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840cc820 con 0x557370389990 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa59624b740 1 -- 172.21.15.7:0/3869439575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 msgr2=0x55737038b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa59624b740 1 --2- 172.21.15.7:0/3869439575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x55737038b3d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fa584097b30 tx=0x7fa584018df0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa59624b740 1 -- 172.21.15.7:0/3869439575 shutdown_connections 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa59624b740 1 --2- 172.21.15.7:0/3869439575 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55737038eb00 0x55737039a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa59624b740 1 --2- 172.21.15.7:0/3869439575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x55737038b3d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa59624b740 1 --2- 172.21.15.7:0/3869439575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557370388fa0 0x55737038ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa59624b740 1 -- 172.21.15.7:0/3869439575 >> 172.21.15.7:0/3869439575 conn(0x557370384510 msgr2=0x557370393420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa59624b740 1 -- 172.21.15.7:0/3869439575 shutdown_connections 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.098+0000 7fa59624b740 1 -- 172.21.15.7:0/3869439575 wait complete. 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa59624b740 1 -- start start 2024-03-26T08:57:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa59624b740 1 Processor -- start 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa59624b740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557370388fa0 0x557370398380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa59624b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x5573703988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa59624b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55737038eb00 0x557370399880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa59624b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1be5e0 con 0x557370389990 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa59624b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1bf900 con 0x55737038eb00 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa59624b740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c1bec40 con 0x557370388fa0 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa59624b740 10 client.0 _ll_register_callbacks cb 0x7fa584097830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa590c07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x5573703988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa590c07640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x5573703988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33110/0 (socket says 172.21.15.7:33110) 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa590c07640 1 -- 172.21.15.7:0/3254020709 learned_addr learned my addr 172.21.15.7:0/3254020709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa591408640 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557370388fa0 0x557370398380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa591c09640 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55737038eb00 0x557370399880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa590c07640 1 -- 172.21.15.7:0/3254020709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557370388fa0 msgr2=0x557370398380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa590c07640 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557370388fa0 0x557370398380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa590c07640 1 -- 172.21.15.7:0/3254020709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55737038eb00 msgr2=0x557370399880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa590c07640 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55737038eb00 0x557370399880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.099+0000 7fa590c07640 1 -- 172.21.15.7:0/3254020709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557370399e70 con 0x557370389990 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa590c07640 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x5573703988c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa58409b660 tx=0x7fa58400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3aad27c5919116 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa570ff9640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa570ff9640 1 -- 172.21.15.7:0/3254020709 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840d2040 con 0x557370389990 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa570ff9640 1 -- 172.21.15.7:0/3254020709 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa5840cc820 con 0x557370389990 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa59624b740 1 -- 172.21.15.7:0/3254020709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5840b8070 con 0x557370389990 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa570ff9640 1 -- 172.21.15.7:0/3254020709 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5840d23e0 con 0x557370389990 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa59624b740 10 client.19254 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa59624b740 1 -- 172.21.15.7:0/3254020709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557370399e70 con 0x557370389990 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa570ff9640 1 -- 172.21.15.7:0/3254020709 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5840b8070 con 0x557370389990 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa570ff9640 1 -- 172.21.15.7:0/3254020709 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fa5840d5070 con 0x557370389990 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa59624b740 10 client.19254 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa59624b740 10 client.19254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa59624b740 20 client.19254 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa59624b740 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55737039eae0 0x5573703fca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa59624b740 4 client.19254 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.100+0000 7fa59624b740 1 -- 172.21.15.7:0/3254020709 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5573702f5c60 con 0x55737039eae0 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa591408640 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55737039eae0 0x5573703fca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa591408640 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55737039eae0 0x5573703fca50 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3424a1cbd34373 server_cookie=639f9e5c7a2ecaac in_seq=0 out_seq=0 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa570ff9640 10 client.19254 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa570ff9640 1 -- 172.21.15.7:0/3254020709 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5573702f5c60 con 0x55737039eae0 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa570ff9640 10 client.19254 handle_command_reply: tid=0 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 client.19254 shutdown 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 -- 172.21.15.7:0/3254020709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 msgr2=0x5573703988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x5573703988c0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa58409b660 tx=0x7fa58400efd0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 -- 172.21.15.7:0/3254020709 shutdown_connections 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55737039eae0 0x5573703fca50 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55737038eb00 0x557370399880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557370389990 0x5573703988c0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 --2- 172.21.15.7:0/3254020709 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557370388fa0 0x557370398380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 -- 172.21.15.7:0/3254020709 >> 172.21.15.7:0/3254020709 conn(0x557370384510 msgr2=0x5573703933e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa570ff9640 0 client.19254 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 -- 172.21.15.7:0/3254020709 shutdown_connections 2024-03-26T08:57:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.101+0000 7fa59624b740 1 -- 172.21.15.7:0/3254020709 wait complete. 2024-03-26T08:57:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.102+0000 7fa59624b740 20 client.19254 trim_cache size 0 max 16384 2024-03-26T08:57:03.105 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:57:03.105 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:57:03.105 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:57:03.105 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:57:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 -- 172.21.15.7:0/3125854000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5740a2a00 msgr2=0x7fa5740a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5740a2a00 0x7fa5740a4ec0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa57c006010 tx=0x7fa57c0323d0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 -- 172.21.15.7:0/3125854000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c108cc0 msgr2=0x7fa58c117220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c108cc0 0x7fa58c117220 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa58400c970 tx=0x7fa58400ce30 comp rx=0 tx=0).stop 2024-03-26T08:57:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 -- 172.21.15.7:0/3125854000 shutdown_connections 2024-03-26T08:57:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa5740a2a00 0x7fa5740a4ec0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa58c109680 0x7fa58c1b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa58c108cc0 0x7fa58c117220 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 --2- 172.21.15.7:0/3125854000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa58c072a10 0x7fa58c116ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 -- 172.21.15.7:0/3125854000 >> 172.21.15.7:0/3125854000 conn(0x7fa58c06e4e0 msgr2=0x7fa58c10bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 -- 172.21.15.7:0/3125854000 shutdown_connections 2024-03-26T08:57:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.104+0000 7fa59240a640 1 -- 172.21.15.7:0/3125854000 wait complete. 2024-03-26T08:57:03.114 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:57:03.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:57:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.250+0000 7f25e7940640 1 -- 172.21.15.7:0/173265604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01017c0 msgr2=0x7f25e0101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.250+0000 7f25e7940640 1 --2- 172.21.15.7:0/173265604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01017c0 0x7f25e0101c40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009900 tx=0x7f25dc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.250+0000 7f25e7940640 1 -- 172.21.15.7:0/173265604 shutdown_connections 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.250+0000 7f25e7940640 1 --2- 172.21.15.7:0/173265604 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25e0102180 0x7f25e0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.250+0000 7f25e7940640 1 --2- 172.21.15.7:0/173265604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01017c0 0x7f25e0101c40 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.250+0000 7f25e7940640 1 --2- 172.21.15.7:0/173265604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e01005d0 0x7f25e01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.250+0000 7f25e7940640 1 -- 172.21.15.7:0/173265604 >> 172.21.15.7:0/173265604 conn(0x7f25e00fbb40 msgr2=0x7f25e00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.250+0000 7f25e7940640 1 -- 172.21.15.7:0/173265604 shutdown_connections 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.250+0000 7f25e7940640 1 -- 172.21.15.7:0/173265604 wait complete. 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e7940640 1 Processor -- start 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e7940640 1 -- start start 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e7940640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01005d0 0x7f25e0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e7940640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e01017c0 0x7f25e0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e7940640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25e0102180 0x7f25e019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e7940640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e010c250 con 0x7f25e01005d0 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e7940640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e010c0d0 con 0x7f25e01017c0 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e7940640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e010c3d0 con 0x7f25e0102180 2024-03-26T08:57:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e56dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01005d0 0x7f25e0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e56dc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01005d0 0x7f25e0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33122/0 (socket says 172.21.15.7:33122) 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e56dc640 1 -- 172.21.15.7:0/2152004328 learned_addr learned my addr 172.21.15.7:0/2152004328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e4edb640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e01017c0 0x7f25e0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.251+0000 7f25e5edd640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25e0102180 0x7f25e019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25e56dc640 1 -- 172.21.15.7:0/2152004328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25e0102180 msgr2=0x7f25e019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25e56dc640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25e0102180 0x7f25e019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25e56dc640 1 -- 172.21.15.7:0/2152004328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e01017c0 msgr2=0x7f25e0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25e56dc640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e01017c0 0x7f25e0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25e56dc640 1 -- 172.21.15.7:0/2152004328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e019f690 con 0x7f25e01005d0 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25e56dc640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01005d0 0x7f25e0197a70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f25d000e9c0 tx=0x7f25d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25d67fc640 1 -- 172.21.15.7:0/2152004328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000cd80 con 0x7f25e01005d0 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e019f980 con 0x7f25e01005d0 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25d67fc640 1 -- 172.21.15.7:0/2152004328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25d00035f0 con 0x7f25e01005d0 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25d67fc640 1 -- 172.21.15.7:0/2152004328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0010690 con 0x7f25e01005d0 2024-03-26T08:57:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.252+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e019fea0 con 0x7f25e01005d0 2024-03-26T08:57:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.253+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e01a0130 con 0x7f25e01005d0 2024-03-26T08:57:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.255+0000 7f25d67fc640 1 -- 172.21.15.7:0/2152004328 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f25d00037a0 con 0x7f25e01005d0 2024-03-26T08:57:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.255+0000 7f25d67fc640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.255+0000 7f25e4edb640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.255+0000 7f25d67fc640 1 -- 172.21.15.7:0/2152004328 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25d00ceef0 con 0x7f25e01005d0 2024-03-26T08:57:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.256+0000 7f25e4edb640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009230 tx=0x7f25dc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.259+0000 7f25d67fc640 1 -- 172.21.15.7:0/2152004328 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f25d009c020 con 0x7f25e01005d0 2024-03-26T08:57:03.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.568+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25e01a03a0 con 0x7f25e01005d0 2024-03-26T08:57:03.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.569+0000 7f25d67fc640 1 -- 172.21.15.7:0/2152004328 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f25d009b920 con 0x7f25e01005d0 2024-03-26T08:57:03.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:03.568 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:57:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.571+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25cc0a2800 msgr2=0x7f25cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.571+0000 7f25e7940640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009230 tx=0x7f25dc005fd0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01005d0 msgr2=0x7f25e0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01005d0 0x7f25e0197a70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f25d000e9c0 tx=0x7f25d000ee80 comp rx=0 tx=0).stop 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 shutdown_connections 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25cc0a2800 0x7f25cc0a4cc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25e0102180 0x7f25e019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e01017c0 0x7f25e0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 --2- 172.21.15.7:0/2152004328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e01005d0 0x7f25e0197a70 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 >> 172.21.15.7:0/2152004328 conn(0x7f25e00fbb40 msgr2=0x7f25e00fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 shutdown_connections 2024-03-26T08:57:03.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.572+0000 7f25e7940640 1 -- 172.21.15.7:0/2152004328 wait complete. 2024-03-26T08:57:03.582 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:57:03.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:57:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.734+0000 7fdaca16f640 1 -- 172.21.15.7:0/2037442114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 msgr2=0x7fdac4104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.734+0000 7fdaca16f640 1 --2- 172.21.15.7:0/2037442114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 0x7fdac4104d10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009900 tx=0x7fdab802bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.734+0000 7fdaca16f640 1 -- 172.21.15.7:0/2037442114 shutdown_connections 2024-03-26T08:57:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.734+0000 7fdaca16f640 1 --2- 172.21.15.7:0/2037442114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac4105250 0x7fdac41056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.734+0000 7fdaca16f640 1 --2- 172.21.15.7:0/2037442114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 0x7fdac4104d10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.734+0000 7fdaca16f640 1 --2- 172.21.15.7:0/2037442114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac40fff30 0x7fdac4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.734+0000 7fdaca16f640 1 -- 172.21.15.7:0/2037442114 >> 172.21.15.7:0/2037442114 conn(0x7fdac40fbb30 msgr2=0x7fdac40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.734+0000 7fdaca16f640 1 -- 172.21.15.7:0/2037442114 shutdown_connections 2024-03-26T08:57:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.734+0000 7fdaca16f640 1 -- 172.21.15.7:0/2037442114 wait complete. 2024-03-26T08:57:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdaca16f640 1 Processor -- start 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdaca16f640 1 -- start start 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdaca16f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac40fff30 0x7fdac419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdaca16f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 0x7fdac419e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdaca16f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac4105250 0x7fdac41a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdaca16f640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4113930 con 0x7fdac41028a0 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdaca16f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41137b0 con 0x7fdac4105250 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdaca16f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4113ab0 con 0x7fdac40fff30 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdac2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 0x7fdac419e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdac37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac40fff30 0x7fdac419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdac2ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 0x7fdac419e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33136/0 (socket says 172.21.15.7:33136) 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdac2ffd640 1 -- 172.21.15.7:0/1580298115 learned_addr learned my addr 172.21.15.7:0/1580298115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.735+0000 7fdac3fff640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac4105250 0x7fdac41a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdac2ffd640 1 -- 172.21.15.7:0/1580298115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac40fff30 msgr2=0x7fdac419dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdac2ffd640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac40fff30 0x7fdac419dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdac2ffd640 1 -- 172.21.15.7:0/1580298115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac4105250 msgr2=0x7fdac41a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdac2ffd640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac4105250 0x7fdac41a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdac2ffd640 1 -- 172.21.15.7:0/1580298115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac41a5bf0 con 0x7fdac41028a0 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdac2ffd640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 0x7fdac419e510 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009230 tx=0x7fdab802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdac0ff9640 1 -- 172.21.15.7:0/1580298115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8041070 con 0x7fdac41028a0 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdac0ff9640 1 -- 172.21.15.7:0/1580298115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdab8004450 con 0x7fdac41028a0 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdaca16f640 1 -- 172.21.15.7:0/1580298115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac41a5e80 con 0x7fdac41028a0 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdac0ff9640 1 -- 172.21.15.7:0/1580298115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8003e20 con 0x7fdac41028a0 2024-03-26T08:57:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.736+0000 7fdaca16f640 1 -- 172.21.15.7:0/1580298115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac41a62e0 con 0x7fdac41028a0 2024-03-26T08:57:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.739+0000 7fdac0ff9640 1 -- 172.21.15.7:0/1580298115 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fdab8044070 con 0x7fdac41028a0 2024-03-26T08:57:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.739+0000 7fdacce30740 1 Processor -- start 2024-03-26T08:57:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdacce30740 1 -- start start 2024-03-26T08:57:03.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdacce30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbe0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdacce30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe1080 0x56250fbe5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdacce30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 0x56250fbe8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdacce30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fba3610 con 0x56250fbe1080 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdacce30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fb463c0 con 0x56250fbe0690 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdacce30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250faeed60 con 0x56250fbe61f0 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac0ff9640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe1080 0x56250fbe5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbe0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe1080 0x56250fbe5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33148/0 (socket says 172.21.15.7:33148) 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbe0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45148/0 (socket says 172.21.15.7:45148) 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac37fe640 1 -- 172.21.15.7:0/4216956458 learned_addr learned my addr 172.21.15.7:0/4216956458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac0ff9640 1 -- 172.21.15.7:0/1580298115 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdab80f05d0 con 0x7fdac41028a0 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac2ffd640 1 --2- 172.21.15.7:0/4216956458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 0x56250fbe8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac3fff640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac37fe640 1 -- 172.21.15.7:0/4216956458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 msgr2=0x56250fbe8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac37fe640 1 --2- 172.21.15.7:0/4216956458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 0x56250fbe8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac37fe640 1 -- 172.21.15.7:0/4216956458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 msgr2=0x56250fbe0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac37fe640 1 --2- 172.21.15.7:0/4216956458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbe0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.740+0000 7fdac37fe640 1 -- 172.21.15.7:0/4216956458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56250f85a3f0 con 0x56250fbe1080 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.741+0000 7fdac37fe640 1 --2- 172.21.15.7:0/4216956458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe1080 0x56250fbe5b80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009cf0 tx=0x7fdab4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da470951726204f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.741+0000 7fdac3fff640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x56250fbe18e0 tx=0x7fdabc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.741+0000 7fdaa9ffb640 1 -- 172.21.15.7:0/4216956458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab401e070 con 0x56250fbe1080 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.741+0000 7fdaa9ffb640 1 -- 172.21.15.7:0/4216956458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdab4019b10 con 0x56250fbe1080 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.741+0000 7fdaa9ffb640 1 -- 172.21.15.7:0/4216956458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab400ea60 con 0x56250fbe1080 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.741+0000 7fdacce30740 1 -- 172.21.15.7:0/4216956458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe1080 msgr2=0x56250fbe5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.741+0000 7fdacce30740 1 --2- 172.21.15.7:0/4216956458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe1080 0x56250fbe5b80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009cf0 tx=0x7fdab4011810 comp rx=0 tx=0).stop 2024-03-26T08:57:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 -- 172.21.15.7:0/4216956458 shutdown_connections 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 --2- 172.21.15.7:0/4216956458 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 0x56250fbe8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 --2- 172.21.15.7:0/4216956458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe1080 0x56250fbe5b80 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 --2- 172.21.15.7:0/4216956458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbe0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 -- 172.21.15.7:0/4216956458 >> 172.21.15.7:0/4216956458 conn(0x56250fbdbc00 msgr2=0x56250fbdc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 -- 172.21.15.7:0/4216956458 shutdown_connections 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 -- 172.21.15.7:0/4216956458 wait complete. 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 -- start start 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 Processor -- start 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbeaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 0x56250fbeafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fb02240 con 0x56250fbe61f0 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fb42460 con 0x56250fbe0690 2024-03-26T08:57:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fbe91a0 con 0x56250fbe1080 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdacce30740 10 client.0 _ll_register_callbacks cb 0x56250fb01c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.742+0000 7fdac3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbeaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33152/0 (socket says 172.21.15.7:33152) 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac3fff640 1 -- 172.21.15.7:0/3855670044 learned_addr learned my addr 172.21.15.7:0/3855670044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac2ffd640 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 0x56250fbeafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac3fff640 1 -- 172.21.15.7:0/3855670044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 msgr2=0x56250fbeafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac3fff640 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 0x56250fbeafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac3fff640 1 -- 172.21.15.7:0/3855670044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 msgr2=0x56250fbeaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac3fff640 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbeaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac3fff640 1 -- 172.21.15.7:0/3855670044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56250f9247d0 con 0x56250fbe61f0 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac37fe640 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbeaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdac3fff640 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fe0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdabc011ee0 tx=0x7fdabc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653eca9b65d591e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdaa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/3855670044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc01e410 con 0x56250fbe61f0 2024-03-26T08:57:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/3855670044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdabc01e5b0 con 0x56250fbe61f0 2024-03-26T08:57:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.743+0000 7fdacce30740 1 -- 172.21.15.7:0/3855670044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdab4019b10 con 0x56250fbe61f0 2024-03-26T08:57:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.745+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/3855670044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdabc01e7b0 con 0x56250fbe61f0 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.745+0000 7fdacce30740 10 client.19272 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.745+0000 7fdacce30740 1 -- 172.21.15.7:0/3855670044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56250fbf53c0 con 0x56250fbe61f0 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.746+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/3855670044 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdab4019b10 con 0x56250fbe61f0 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.746+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/3855670044 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdabc01d070 con 0x56250fbe61f0 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.746+0000 7fdacce30740 10 client.19272 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.746+0000 7fdacce30740 10 client.19272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.746+0000 7fdacce30740 20 client.19272 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.746+0000 7fdacce30740 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56250fbf6b80 0x56250fbf9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.746+0000 7fdacce30740 4 client.19272 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.746+0000 7fdacce30740 1 -- 172.21.15.7:0/3855670044 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56250fb53900 con 0x56250fbf6b80 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.746+0000 7fdac37fe640 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56250fbf6b80 0x56250fbf9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.747+0000 7fdac37fe640 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56250fbf6b80 0x56250fbf9000 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28e2ad02d00ef775 server_cookie=2410851e60e57cf6 in_seq=0 out_seq=0 2024-03-26T08:57:03.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.747+0000 7fdaa2ffd640 10 client.19272 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.747+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/3855670044 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56250fb53900 con 0x56250fbf6b80 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.747+0000 7fdaa2ffd640 10 client.19272 handle_command_reply: tid=0 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 client.19272 shutdown 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 -- 172.21.15.7:0/3855670044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 msgr2=0x56250fbf1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fe0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fdabc011ee0 tx=0x7fdabc004ab0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 -- 172.21.15.7:0/3855670044 shutdown_connections 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56250fbf6b80 0x56250fbf9000 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fe0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 0x56250fbeafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 --2- 172.21.15.7:0/3855670044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbeaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdaa2ffd640 0 client.19272 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 -- 172.21.15.7:0/3855670044 >> 172.21.15.7:0/3855670044 conn(0x56250fbdbc00 msgr2=0x56250fbdae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 -- 172.21.15.7:0/3855670044 shutdown_connections 2024-03-26T08:57:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 1 -- 172.21.15.7:0/3855670044 wait complete. 2024-03-26T08:57:03.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.748+0000 7fdacce30740 20 client.19272 trim_cache size 0 max 16384 2024-03-26T08:57:03.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdacce30740 1 Processor -- start 2024-03-26T08:57:03.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdacce30740 1 -- start start 2024-03-26T08:57:03.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdacce30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbe2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdacce30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 0x56250fbe2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdacce30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdacce30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41b15b0 con 0x56250fbe61f0 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdacce30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41af0b0 con 0x56250fbe0690 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdacce30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41ad130 con 0x56250fbe1080 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdac3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdac3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33160/0 (socket says 172.21.15.7:33160) 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdac3fff640 1 -- 172.21.15.7:0/1160238414 learned_addr learned my addr 172.21.15.7:0/1160238414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdac2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 0x56250fbe2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.763+0000 7fdac37fe640 1 --2- 172.21.15.7:0/1160238414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbe2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.764+0000 7fdac3fff640 1 -- 172.21.15.7:0/1160238414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 msgr2=0x56250fbe2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.764+0000 7fdac3fff640 1 --2- 172.21.15.7:0/1160238414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 0x56250fbe2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.764+0000 7fdac3fff640 1 -- 172.21.15.7:0/1160238414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 msgr2=0x56250fbe2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.764+0000 7fdac3fff640 1 --2- 172.21.15.7:0/1160238414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbe2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.764+0000 7fdac3fff640 1 -- 172.21.15.7:0/1160238414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56250fbe1910 con 0x56250fbe61f0 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.764+0000 7fdac3fff640 1 --2- 172.21.15.7:0/1160238414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fa0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009ce0 tx=0x7fdabc009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17492585298083d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.764+0000 7fdaa97fa640 1 -- 172.21.15.7:0/1160238414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc039070 con 0x56250fbe61f0 2024-03-26T08:57:03.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.764+0000 7fdaa97fa640 1 -- 172.21.15.7:0/1160238414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdabc01e410 con 0x56250fbe61f0 2024-03-26T08:57:03.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.764+0000 7fdaa97fa640 1 -- 172.21.15.7:0/1160238414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc02a6d0 con 0x56250fbe61f0 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.765+0000 7fdacce30740 1 -- 172.21.15.7:0/1160238414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 msgr2=0x56250fbf1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.765+0000 7fdacce30740 1 --2- 172.21.15.7:0/1160238414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fa0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009ce0 tx=0x7fdabc009570 comp rx=0 tx=0).stop 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.766+0000 7fdacce30740 1 -- 172.21.15.7:0/1160238414 shutdown_connections 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.766+0000 7fdacce30740 1 --2- 172.21.15.7:0/1160238414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe61f0 0x56250fbf1fa0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.766+0000 7fdacce30740 1 --2- 172.21.15.7:0/1160238414 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe1080 0x56250fbe2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.766+0000 7fdacce30740 1 --2- 172.21.15.7:0/1160238414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe0690 0x56250fbe2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.766+0000 7fdacce30740 1 -- 172.21.15.7:0/1160238414 >> 172.21.15.7:0/1160238414 conn(0x56250fbdbc00 msgr2=0x56250fbeaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.766+0000 7fdacce30740 1 -- 172.21.15.7:0/1160238414 shutdown_connections 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.767+0000 7fdacce30740 1 -- 172.21.15.7:0/1160238414 wait complete. 2024-03-26T08:57:03.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.767+0000 7fdacce30740 1 -- start start 2024-03-26T08:57:03.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.767+0000 7fdacce30740 1 Processor -- start 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdacce30740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe0690 0x56250fbef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdacce30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe1080 0x56250fbef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdacce30740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 0x56250fbf0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdacce30740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41ad730 con 0x56250fbe0690 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdacce30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41aea50 con 0x56250fbe1080 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdacce30740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41add90 con 0x56250fbe61f0 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdacce30740 10 client.0 _ll_register_callbacks cb 0x7fdabc009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe0690 0x56250fbef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe1080 0x56250fbef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac37fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe0690 0x56250fbef430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33174/0 (socket says 172.21.15.7:33174) 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe1080 0x56250fbef970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45174/0 (socket says 172.21.15.7:45174) 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac37fe640 1 -- 172.21.15.7:0/2790535854 learned_addr learned my addr 172.21.15.7:0/2790535854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac3fff640 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 0x56250fbf0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac37fe640 1 -- 172.21.15.7:0/2790535854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 msgr2=0x56250fbf0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac37fe640 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 0x56250fbf0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac37fe640 1 -- 172.21.15.7:0/2790535854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe1080 msgr2=0x56250fbef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac37fe640 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe1080 0x56250fbef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac37fe640 1 -- 172.21.15.7:0/2790535854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56250fbf0f70 con 0x56250fbe0690 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac3fff640 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 0x56250fbf0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdac37fe640 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe0690 0x56250fbef430 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009c40 tx=0x7fdab4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e5f03a60671e83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.768+0000 7fdaa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/2790535854 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab401e040 con 0x56250fbe0690 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/2790535854 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdab4005250 con 0x56250fbe0690 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdacce30740 1 -- 172.21.15.7:0/2790535854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdabc01e410 con 0x56250fbe0690 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/2790535854 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdab4005470 con 0x56250fbe0690 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdacce30740 10 client.19278 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:03.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdacce30740 1 -- 172.21.15.7:0/2790535854 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56250fbf22e0 con 0x56250fbe0690 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/2790535854 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdabc01e410 con 0x56250fbe0690 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/2790535854 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdab400e550 con 0x56250fbe0690 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdacce30740 10 client.19278 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdacce30740 10 client.19278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdacce30740 20 client.19278 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdacce30740 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56250fc52a90 0x56250fc54eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdacce30740 4 client.19278 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdacce30740 1 -- 172.21.15.7:0/2790535854 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56250fb4dc50 con 0x56250fc52a90 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.769+0000 7fdac2ffd640 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56250fc52a90 0x56250fc54eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdac2ffd640 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56250fc52a90 0x56250fc54eb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58b7a4969e503a40 server_cookie=cfdacf698d6656b1 in_seq=0 out_seq=0 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdaa2ffd640 10 client.19278 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdaa2ffd640 1 -- 172.21.15.7:0/2790535854 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56250fb4dc50 con 0x56250fc52a90 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdaa2ffd640 10 client.19278 handle_command_reply: tid=0 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 client.19278 shutdown 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 -- 172.21.15.7:0/2790535854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe0690 msgr2=0x56250fbef430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe0690 0x56250fbef430 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009c40 tx=0x7fdab4005f10 comp rx=0 tx=0).stop 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 -- 172.21.15.7:0/2790535854 shutdown_connections 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x56250fc52a90 0x56250fc54eb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56250fbe61f0 0x56250fbf0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56250fbe1080 0x56250fbef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdaa2ffd640 0 client.19278 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 --2- 172.21.15.7:0/2790535854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56250fbe0690 0x56250fbef430 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 -- 172.21.15.7:0/2790535854 >> 172.21.15.7:0/2790535854 conn(0x56250fbdbc00 msgr2=0x56250fbeaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.770+0000 7fdacce30740 1 -- 172.21.15.7:0/2790535854 shutdown_connections 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdacce30740 1 -- 172.21.15.7:0/2790535854 wait complete. 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdacce30740 20 client.19278 trim_cache size 0 max 16384 2024-03-26T08:57:03.772 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:57:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 -- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdaac0a2800 msgr2=0x7fdaac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x56250fbe18e0 tx=0x7fdabc009500 comp rx=0 tx=0).stop 2024-03-26T08:57:03.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 -- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 msgr2=0x7fdac419e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 0x7fdac419e510 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009230 tx=0x7fdab802c650 comp rx=0 tx=0).stop 2024-03-26T08:57:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 -- 172.21.15.7:0/1580298115 shutdown_connections 2024-03-26T08:57:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac4105250 0x7fdac41a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac41028a0 0x7fdac419e510 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 --2- 172.21.15.7:0/1580298115 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdac40fff30 0x7fdac419dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 -- 172.21.15.7:0/1580298115 >> 172.21.15.7:0/1580298115 conn(0x7fdac40fbb30 msgr2=0x7fdac41008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 -- 172.21.15.7:0/1580298115 shutdown_connections 2024-03-26T08:57:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.771+0000 7fdaca16f640 1 -- 172.21.15.7:0/1580298115 wait complete. 2024-03-26T08:57:03.781 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:57:03.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:57:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.924+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2430365325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 msgr2=0x7fdb94102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.924+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2430365325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 0x7fdb94102d60 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009900 tx=0x7fdb8402bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.924+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2430365325 shutdown_connections 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.924+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2430365325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb941032a0 0x7fdb94109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.924+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2430365325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 0x7fdb94102d60 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.924+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2430365325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941016f0 0x7fdb94101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.924+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2430365325 >> 172.21.15.7:0/2430365325 conn(0x7fdb940fcc60 msgr2=0x7fdb940ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2430365325 shutdown_connections 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2430365325 wait complete. 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 Processor -- start 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 -- start start 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb941016f0 0x7fdb941979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 0x7fdb94197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941032a0 0x7fdb9419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9410d3c0 con 0x7fdb941028e0 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9410d240 con 0x7fdb941032a0 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9baf1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9410d540 con 0x7fdb941016f0 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9908c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 0x7fdb94197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9908c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 0x7fdb94197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33184/0 (socket says 172.21.15.7:33184) 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.925+0000 7fdb9908c640 1 -- 172.21.15.7:0/2537135279 learned_addr learned my addr 172.21.15.7:0/2537135279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb9988d640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb941016f0 0x7fdb941979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941032a0 0x7fdb9419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb9908c640 1 -- 172.21.15.7:0/2537135279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb941016f0 msgr2=0x7fdb941979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb9908c640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb941016f0 0x7fdb941979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb9908c640 1 -- 172.21.15.7:0/2537135279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941032a0 msgr2=0x7fdb9419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb9908c640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941032a0 0x7fdb9419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb9908c640 1 -- 172.21.15.7:0/2537135279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9419f600 con 0x7fdb941028e0 2024-03-26T08:57:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb9908c640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 0x7fdb94197f20 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fdb84000c00 tx=0x7fdb8402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb8affd640 1 -- 172.21.15.7:0/2537135279 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84043020 con 0x7fdb941028e0 2024-03-26T08:57:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb8affd640 1 -- 172.21.15.7:0/2537135279 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb840054e0 con 0x7fdb941028e0 2024-03-26T08:57:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.926+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2537135279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9419f890 con 0x7fdb941028e0 2024-03-26T08:57:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.927+0000 7fdb8affd640 1 -- 172.21.15.7:0/2537135279 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84003c40 con 0x7fdb941028e0 2024-03-26T08:57:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.927+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2537135279 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9419fe30 con 0x7fdb941028e0 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.929+0000 7fdb8affd640 1 -- 172.21.15.7:0/2537135279 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fdb84046020 con 0x7fdb941028e0 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.929+0000 7fdb9e7ba740 1 Processor -- start 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.929+0000 7fdb8affd640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb800a2880 0x7fdb800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.929+0000 7fdb8affd640 1 -- 172.21.15.7:0/2537135279 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb840f0a50 con 0x7fdb941028e0 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.929+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb800a2880 0x7fdb800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9e7ba740 1 -- start start 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e61ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e6204f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 0x55f2e6207a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e61be610 con 0x55f2e6205600 2024-03-26T08:57:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e61613c0 con 0x55f2e61ffaa0 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e6109d60 con 0x55f2e6200490 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9988d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e61ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9908c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e6204f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9988d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e61ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45202/0 (socket says 172.21.15.7:45202) 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9988d640 1 -- 172.21.15.7:0/1370730286 learned_addr learned my addr 172.21.15.7:0/1370730286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb800a2880 0x7fdb800a4d40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c005fd0 tx=0x7fdb8c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/1370730286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 0x55f2e6207a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9988d640 1 -- 172.21.15.7:0/1370730286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 msgr2=0x55f2e6204f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9988d640 1 --2- 172.21.15.7:0/1370730286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e6204f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9988d640 1 -- 172.21.15.7:0/1370730286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 msgr2=0x55f2e6207a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9988d640 1 --2- 172.21.15.7:0/1370730286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 0x55f2e6207a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9988d640 1 -- 172.21.15.7:0/1370730286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2e5e753f0 con 0x55f2e61ffaa0 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.930+0000 7fdb9908c640 1 --2- 172.21.15.7:0/1370730286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e6204f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:03.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.931+0000 7fdb9988d640 1 --2- 172.21.15.7:0/1370730286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e61ffec0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009ca0 tx=0x7fdb90011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=769dbfd282c3ec43 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.931+0000 7fdb7ffff640 1 -- 172.21.15.7:0/1370730286 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90004220 con 0x55f2e61ffaa0 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.931+0000 7fdb7ffff640 1 -- 172.21.15.7:0/1370730286 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb900043c0 con 0x55f2e61ffaa0 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.931+0000 7fdb7ffff640 1 -- 172.21.15.7:0/1370730286 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90019930 con 0x55f2e61ffaa0 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.931+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1370730286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 msgr2=0x55f2e61ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.931+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/1370730286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e61ffec0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009ca0 tx=0x7fdb90011810 comp rx=0 tx=0).stop 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.932+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1370730286 shutdown_connections 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.932+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/1370730286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 0x55f2e6207a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.932+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/1370730286 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e6204f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.932+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/1370730286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e61ffec0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.932+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1370730286 >> 172.21.15.7:0/1370730286 conn(0x55f2e61f6c00 msgr2=0x55f2e61f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.932+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1370730286 shutdown_connections 2024-03-26T08:57:03.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.932+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1370730286 wait complete. 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.933+0000 7fdb9e7ba740 1 -- start start 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.933+0000 7fdb9e7ba740 1 Processor -- start 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.933+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e6209e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 0x55f2e62113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e611d240 con 0x55f2e6205600 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e615d460 con 0x55f2e61ffaa0 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e62085b0 con 0x55f2e6200490 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9908c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9908c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60302/0 (socket says 172.21.15.7:60302) 2024-03-26T08:57:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9908c640 1 -- 172.21.15.7:0/538312506 learned_addr learned my addr 172.21.15.7:0/538312506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9e7ba740 10 client.0 _ll_register_callbacks cb 0x55f2e611cc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 0x55f2e62113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9988d640 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e6209e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9908c640 1 -- 172.21.15.7:0/538312506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 msgr2=0x55f2e6209e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9908c640 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e6209e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9908c640 1 -- 172.21.15.7:0/538312506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 msgr2=0x55f2e62113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9908c640 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 0x55f2e62113b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9908c640 1 -- 172.21.15.7:0/538312506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2e5f3f7d0 con 0x55f2e6200490 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9988d640 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e6209e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.934+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 0x55f2e62113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9908c640 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620a370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdb840b9dc0 tx=0x7fdb8402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d76e224c21663b6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/538312506 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb840d6060 con 0x55f2e6200490 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/538312506 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb840d6200 con 0x55f2e6200490 2024-03-26T08:57:03.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/538312506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb900043c0 con 0x55f2e6200490 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/538312506 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb84005690 con 0x55f2e6200490 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9e7ba740 10 client.29059 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/538312506 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f2e61fd060 con 0x55f2e6200490 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/538312506 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb84004c60 con 0x55f2e6200490 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/538312506 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdb840c1170 con 0x55f2e6200490 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9e7ba740 10 client.29059 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9e7ba740 10 client.29059 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9e7ba740 20 client.29059 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55f2e61fe7e0 0x55f2e61fec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9e7ba740 4 client.29059 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.935+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/538312506 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb900043c0 con 0x55f2e61fe7e0 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.936+0000 7fdb9988d640 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55f2e61fe7e0 0x55f2e61fec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.936+0000 7fdb7d7fa640 10 client.29059 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:57:03.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.936+0000 7fdb9988d640 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55f2e61fe7e0 0x55f2e61fec10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a37e3d1ddbc677eb server_cookie=2a710e561f04d5b1 in_seq=0 out_seq=0 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/538312506 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fdb900043c0 con 0x55f2e61fe7e0 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb7d7fa640 10 client.29059 handle_command_reply: tid=0 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 client.29059 shutdown 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/538312506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 msgr2=0x55f2e620a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620a370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdb840b9dc0 tx=0x7fdb8402bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/538312506 shutdown_connections 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e6205600 0x55f2e62113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55f2e61fe7e0 0x55f2e61fec10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620a370 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/538312506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e61ffaa0 0x55f2e6209e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb7d7fa640 0 client.29059 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/538312506 >> 172.21.15.7:0/538312506 conn(0x55f2e61f6c00 msgr2=0x55f2e61f5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/538312506 shutdown_connections 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/538312506 wait complete. 2024-03-26T08:57:03.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.937+0000 7fdb9e7ba740 20 client.29059 trim_cache size 0 max 16384 2024-03-26T08:57:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.952+0000 7fdb9e7ba740 1 Processor -- start 2024-03-26T08:57:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.953+0000 7fdb9e7ba740 1 -- start start 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.953+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e62098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.953+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6200490 0x55f2e6211370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.953+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6205600 0x55f2e61f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.953+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941a8e20 con 0x55f2e61ffaa0 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.953+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941a6920 con 0x55f2e6200490 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.953+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941a49a0 con 0x55f2e6205600 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.953+0000 7fdb9a08e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6205600 0x55f2e61f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.953+0000 7fdb9a08e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6205600 0x55f2e61f5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60318/0 (socket says 172.21.15.7:60318) 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9a08e640 1 -- 172.21.15.7:0/1051060840 learned_addr learned my addr 172.21.15.7:0/1051060840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9908c640 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6200490 0x55f2e6211370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9988d640 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e62098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9a08e640 1 -- 172.21.15.7:0/1051060840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6200490 msgr2=0x55f2e6211370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6200490 0x55f2e6211370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9a08e640 1 -- 172.21.15.7:0/1051060840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 msgr2=0x55f2e62098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e62098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9a08e640 1 -- 172.21.15.7:0/1051060840 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2e61f2480 con 0x55f2e6205600 2024-03-26T08:57:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9988d640 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e62098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6205600 0x55f2e61f5590 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e790 tx=0x7fdb8c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24dfc2812936e569 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb7f7fe640 1 -- 172.21.15.7:0/1051060840 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00edd0 con 0x55f2e6205600 2024-03-26T08:57:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb7f7fe640 1 -- 172.21.15.7:0/1051060840 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb8c003970 con 0x55f2e6205600 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.954+0000 7fdb7f7fe640 1 -- 172.21.15.7:0/1051060840 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c004330 con 0x55f2e6205600 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.955+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1051060840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6205600 msgr2=0x55f2e61f5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.955+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6205600 0x55f2e61f5590 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e790 tx=0x7fdb8c014000 comp rx=0 tx=0).stop 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.955+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1051060840 shutdown_connections 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.955+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6205600 0x55f2e61f5590 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.955+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6200490 0x55f2e6211370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.955+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/1051060840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e62098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.955+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1051060840 >> 172.21.15.7:0/1051060840 conn(0x55f2e61f6c00 msgr2=0x55f2e6201990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.955+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1051060840 shutdown_connections 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.955+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/1051060840 wait complete. 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9e7ba740 1 -- start start 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9e7ba740 1 Processor -- start 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e620eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9e7ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6205600 0x55f2e6210000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941a4fa0 con 0x55f2e61ffaa0 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941a62c0 con 0x55f2e6205600 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9e7ba740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941a5600 con 0x55f2e6200490 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9e7ba740 10 client.0 _ll_register_callbacks cb 0x7fdb840bce40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9988d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e620eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9988d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e620eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33218/0 (socket says 172.21.15.7:33218) 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9988d640 1 -- 172.21.15.7:0/3320324915 learned_addr learned my addr 172.21.15.7:0/3320324915 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9908c640 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.956+0000 7fdb9a08e640 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6205600 0x55f2e6210000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9988d640 1 -- 172.21.15.7:0/3320324915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 msgr2=0x55f2e620eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9988d640 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9988d640 1 -- 172.21.15.7:0/3320324915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6205600 msgr2=0x55f2e6210000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9988d640 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6205600 0x55f2e6210000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9988d640 1 -- 172.21.15.7:0/3320324915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2e62105f0 con 0x55f2e61ffaa0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9908c640 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9988d640 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e620eab0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009250 tx=0x7fdb900096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c494fbf9135f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/3320324915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90005280 con 0x55f2e61ffaa0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/3320324915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb90005420 con 0x55f2e61ffaa0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/3320324915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb8c003970 con 0x55f2e61ffaa0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/3320324915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb90005620 con 0x55f2e61ffaa0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9e7ba740 10 client.19296 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/3320324915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f2e6218940 con 0x55f2e61ffaa0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/3320324915 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb8c003970 con 0x55f2e61ffaa0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.957+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/3320324915 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdb90019910 con 0x55f2e61ffaa0 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9e7ba740 10 client.19296 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9e7ba740 10 client.19296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:57:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9e7ba740 20 client.19296 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55f2e62190c0 0x55f2e62194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9e7ba740 4 client.19296 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/3320324915 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f2e616e900 con 0x55f2e62190c0 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9908c640 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55f2e62190c0 0x55f2e62194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9908c640 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55f2e62190c0 0x55f2e62194d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=755a23df8c6b7db9 server_cookie=2fba4d71a17c2b90 in_seq=0 out_seq=0 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb7d7fa640 10 client.19296 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb7d7fa640 1 -- 172.21.15.7:0/3320324915 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f2e616e900 con 0x55f2e62190c0 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb7d7fa640 10 client.19296 handle_command_reply: tid=0 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9e7ba740 1 client.19296 shutdown 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/3320324915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 msgr2=0x55f2e620eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.958+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e620eab0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009250 tx=0x7fdb900096f0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/3320324915 shutdown_connections 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2e6205600 0x55f2e6210000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55f2e62190c0 0x55f2e62194d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f2e6200490 0x55f2e620eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb7d7fa640 0 client.19296 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:57:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9e7ba740 1 --2- 172.21.15.7:0/3320324915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f2e61ffaa0 0x55f2e620eab0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/3320324915 >> 172.21.15.7:0/3320324915 conn(0x55f2e61f6c00 msgr2=0x55f2e6201990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/3320324915 shutdown_connections 2024-03-26T08:57:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9e7ba740 1 -- 172.21.15.7:0/3320324915 wait complete. 2024-03-26T08:57:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9e7ba740 20 client.19296 trim_cache size 0 max 16384 2024-03-26T08:57:03.963 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:57:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb800a2880 msgr2=0x7fdb800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb800a2880 0x7fdb800a4d40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c005fd0 tx=0x7fdb8c009500 comp rx=0 tx=0).stop 2024-03-26T08:57:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 msgr2=0x7fdb94197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 0x7fdb94197f20 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fdb84000c00 tx=0x7fdb8402c650 comp rx=0 tx=0).stop 2024-03-26T08:57:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2537135279 shutdown_connections 2024-03-26T08:57:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb800a2880 0x7fdb800a4d40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941032a0 0x7fdb9419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941028e0 0x7fdb94197f20 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 --2- 172.21.15.7:0/2537135279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb941016f0 0x7fdb941979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2537135279 >> 172.21.15.7:0/2537135279 conn(0x7fdb940fcc60 msgr2=0x7fdb94105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.959+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2537135279 shutdown_connections 2024-03-26T08:57:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:03.960+0000 7fdb9baf1640 1 -- 172.21.15.7:0/2537135279 wait complete. 2024-03-26T08:57:03.971 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:57:03.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:57:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.134+0000 7fd10a713640 1 -- 172.21.15.7:0/2985454153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10410b010 msgr2=0x7fd10410d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.134+0000 7fd10a713640 1 --2- 172.21.15.7:0/2985454153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10410b010 0x7fd10410d480 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009ca0 tx=0x7fd0f402c140 comp rx=0 tx=0).stop 2024-03-26T08:57:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.135+0000 7fd10a713640 1 -- 172.21.15.7:0/2985454153 shutdown_connections 2024-03-26T08:57:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.135+0000 7fd10a713640 1 --2- 172.21.15.7:0/2985454153 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd10410d9c0 0x7fd10410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.135+0000 7fd10a713640 1 --2- 172.21.15.7:0/2985454153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10410b010 0x7fd10410d480 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.135+0000 7fd10a713640 1 --2- 172.21.15.7:0/2985454153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1041086a0 0x7fd10410aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.135+0000 7fd10a713640 1 -- 172.21.15.7:0/2985454153 >> 172.21.15.7:0/2985454153 conn(0x7fd10406e640 msgr2=0x7fd104070a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.135+0000 7fd10a713640 1 -- 172.21.15.7:0/2985454153 shutdown_connections 2024-03-26T08:57:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.135+0000 7fd10a713640 1 -- 172.21.15.7:0/2985454153 wait complete. 2024-03-26T08:57:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.136+0000 7fd10a713640 1 Processor -- start 2024-03-26T08:57:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.136+0000 7fd10a713640 1 -- start start 2024-03-26T08:57:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.136+0000 7fd10a713640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1041086a0 0x7fd104118010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.136+0000 7fd10a713640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10410b010 0x7fd104118550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.137+0000 7fd10a713640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10410d9c0 0x7fd104119590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.137+0000 7fd103fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1041086a0 0x7fd104118010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.137+0000 7fd103fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1041086a0 0x7fd104118010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60340/0 (socket says 172.21.15.7:60340) 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.137+0000 7fd103fff640 1 -- 172.21.15.7:0/4003782431 learned_addr learned my addr 172.21.15.7:0/4003782431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.137+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd104110820 con 0x7fd10410b010 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.137+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041106a0 con 0x7fd10410d9c0 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.137+0000 7fd108cb0640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10410d9c0 0x7fd104119590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.137+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041109a0 con 0x7fd1041086a0 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.137+0000 7fd1037fe640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10410b010 0x7fd104118550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.138+0000 7fd103fff640 1 -- 172.21.15.7:0/4003782431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10410d9c0 msgr2=0x7fd104119590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.138+0000 7fd103fff640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10410d9c0 0x7fd104119590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.138+0000 7fd103fff640 1 -- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10410b010 msgr2=0x7fd104118550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.138+0000 7fd103fff640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10410b010 0x7fd104118550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.138+0000 7fd103fff640 1 -- 172.21.15.7:0/4003782431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd104119c00 con 0x7fd1041086a0 2024-03-26T08:57:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.138+0000 7fd103fff640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1041086a0 0x7fd104118010 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b4a0 tx=0x7fd0fc00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.138+0000 7fd1017fa640 1 -- 172.21.15.7:0/4003782431 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc014070 con 0x7fd1041086a0 2024-03-26T08:57:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.138+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd104119ef0 con 0x7fd1041086a0 2024-03-26T08:57:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.139+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1041b74d0 con 0x7fd1041086a0 2024-03-26T08:57:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.139+0000 7fd1017fa640 1 -- 172.21.15.7:0/4003782431 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0fc010af0 con 0x7fd1041086a0 2024-03-26T08:57:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.139+0000 7fd1017fa640 1 -- 172.21.15.7:0/4003782431 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc01e060 con 0x7fd1041086a0 2024-03-26T08:57:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.142+0000 7fd10d3d0740 1 Processor -- start 2024-03-26T08:57:04.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.143+0000 7fd10d3d0740 1 -- start start 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.143+0000 7fd1017fa640 1 -- 172.21.15.7:0/4003782431 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fd0fc017070 con 0x7fd1041086a0 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.143+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 0x561e9c1d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.143+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1d7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.143+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d81f0 0x561e9c1da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.143+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9c195610 con 0x561e9c1d3080 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.143+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9c1383c0 con 0x561e9c1d81f0 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.143+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9c0e0d60 con 0x561e9c1d2690 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.144+0000 7fd103fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1d7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.144+0000 7fd103fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1d7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33252/0 (socket says 172.21.15.7:33252) 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.144+0000 7fd103fff640 1 -- 172.21.15.7:0/3319830356 learned_addr learned my addr 172.21.15.7:0/3319830356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.144+0000 7fd108cb0640 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 0x561e9c1d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.144+0000 7fd1037fe640 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d81f0 0x561e9c1da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.144+0000 7fd1037fe640 1 -- 172.21.15.7:0/3319830356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 msgr2=0x561e9c1d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.144+0000 7fd1037fe640 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 0x561e9c1d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.145+0000 7fd1037fe640 1 -- 172.21.15.7:0/3319830356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 msgr2=0x561e9c1d7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.145+0000 7fd1037fe640 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1d7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.145+0000 7fd1037fe640 1 -- 172.21.15.7:0/3319830356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e9be4c3f0 con 0x561e9c1d81f0 2024-03-26T08:57:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.145+0000 7fd103fff640 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.145+0000 7fd1037fe640 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d81f0 0x561e9c1da680 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402c750 tx=0x7fd0f4009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cc5d5fc0db23b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.145+0000 7fd0fa7fc640 1 -- 172.21.15.7:0/3319830356 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4043020 con 0x561e9c1d81f0 2024-03-26T08:57:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.146+0000 7fd0fa7fc640 1 -- 172.21.15.7:0/3319830356 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4003970 con 0x561e9c1d81f0 2024-03-26T08:57:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.146+0000 7fd0fa7fc640 1 -- 172.21.15.7:0/3319830356 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4039040 con 0x561e9c1d81f0 2024-03-26T08:57:04.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.146+0000 7fd1017fa640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd0f00a2950 0x7fd0f00a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.146+0000 7fd1017fa640 1 -- 172.21.15.7:0/4003782431 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0fc0cdde0 con 0x7fd1041086a0 2024-03-26T08:57:04.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.147+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3319830356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d81f0 msgr2=0x561e9c1da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:04.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.147+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d81f0 0x561e9c1da680 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402c750 tx=0x7fd0f4009870 comp rx=0 tx=0).stop 2024-03-26T08:57:04.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.147+0000 7fd108cb0640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd0f00a2950 0x7fd0f00a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.148+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3319830356 shutdown_connections 2024-03-26T08:57:04.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.148+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d81f0 0x561e9c1da680 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.148+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1d7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.148+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3319830356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 0x561e9c1d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.148+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3319830356 >> 172.21.15.7:0/3319830356 conn(0x561e9c1cdc00 msgr2=0x561e9c1ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:04.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.149+0000 7fd108cb0640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd0f00a2950 0x7fd0f00a4e10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x561e9c1d38e0 tx=0x7fd0ec00b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:04.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.149+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3319830356 shutdown_connections 2024-03-26T08:57:04.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.149+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3319830356 wait complete. 2024-03-26T08:57:04.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.150+0000 7fd10d3d0740 1 -- start start 2024-03-26T08:57:04.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.150+0000 7fd10d3d0740 1 Processor -- start 2024-03-26T08:57:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d2690 0x561e9c1dc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 0x561e9c1e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9c0f4240 con 0x561e9c1d2690 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9c134460 con 0x561e9c1d3080 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45274/0 (socket says 172.21.15.7:45274) 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd1037fe640 1 -- 172.21.15.7:0/1001958569 learned_addr learned my addr 172.21.15.7:0/1001958569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd108cb0640 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 0x561e9c1e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd10d3d0740 1 -- 172.21.15.7:0/1001958569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9c1db140 con 0x561e9c1d81f0 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd10d3d0740 10 client.0 _ll_register_callbacks cb 0x561e9c0f3c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.151+0000 7fd103fff640 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d2690 0x561e9c1dc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd108cb0640 1 -- 172.21.15.7:0/1001958569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 msgr2=0x561e9c1dcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd108cb0640 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd108cb0640 1 -- 172.21.15.7:0/1001958569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d2690 msgr2=0x561e9c1dc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd108cb0640 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d2690 0x561e9c1dc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd108cb0640 1 -- 172.21.15.7:0/1001958569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e9bf167d0 con 0x561e9c1d81f0 2024-03-26T08:57:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd1037fe640 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd108cb0640 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 0x561e9c1e3f20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec003ca0 tx=0x7fd0ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=327325a8818ed341 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd0e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:57:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd0e37fe640 1 -- 172.21.15.7:0/1001958569 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec01b660 con 0x561e9c1d81f0 2024-03-26T08:57:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd0e37fe640 1 -- 172.21.15.7:0/1001958569 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0ec01b800 con 0x561e9c1d81f0 2024-03-26T08:57:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.152+0000 7fd10d3d0740 1 -- 172.21.15.7:0/1001958569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0f4003970 con 0x561e9c1d81f0 2024-03-26T08:57:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd0e37fe640 1 -- 172.21.15.7:0/1001958569 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0ec01ba00 con 0x561e9c1d81f0 2024-03-26T08:57:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd10d3d0740 10 client.29077 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd10d3d0740 1 -- 172.21.15.7:0/1001958569 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e9c1e7390 con 0x561e9c1d81f0 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd0e37fe640 1 -- 172.21.15.7:0/1001958569 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4003970 con 0x561e9c1d81f0 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd0e37fe640 1 -- 172.21.15.7:0/1001958569 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd0ec024070 con 0x561e9c1d81f0 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd10d3d0740 10 client.29077 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd10d3d0740 10 client.29077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd10d3d0740 20 client.29077 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561e9c1e8e10 0x561e9c1eb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd10d3d0740 4 client.29077 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.153+0000 7fd10d3d0740 1 -- 172.21.15.7:0/1001958569 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e9c145900 con 0x561e9c1e8e10 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.154+0000 7fd103fff640 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561e9c1e8e10 0x561e9c1eb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.154+0000 7fd0e37fe640 10 client.29077 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.154+0000 7fd103fff640 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561e9c1e8e10 0x561e9c1eb2d0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=790a854b5f42f56 server_cookie=51e4be07b95e60ea in_seq=0 out_seq=0 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd0e37fe640 1 -- 172.21.15.7:0/1001958569 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561e9c145900 con 0x561e9c1e8e10 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd0e37fe640 10 client.29077 handle_command_reply: tid=0 2024-03-26T08:57:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd10d3d0740 1 client.29077 shutdown 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd10d3d0740 1 -- 172.21.15.7:0/1001958569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 msgr2=0x561e9c1e3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 0x561e9c1e3f20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec003ca0 tx=0x7fd0ec0046c0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd10d3d0740 1 -- 172.21.15.7:0/1001958569 shutdown_connections 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561e9c1e8e10 0x561e9c1eb2d0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 0x561e9c1e3f20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/1001958569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d2690 0x561e9c1dc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd10d3d0740 1 -- 172.21.15.7:0/1001958569 >> 172.21.15.7:0/1001958569 conn(0x561e9c1cdc00 msgr2=0x561e9c1cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.155+0000 7fd0e37fe640 0 client.29077 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.156+0000 7fd10d3d0740 1 -- 172.21.15.7:0/1001958569 shutdown_connections 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.156+0000 7fd10d3d0740 1 -- 172.21.15.7:0/1001958569 wait complete. 2024-03-26T08:57:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.156+0000 7fd10d3d0740 20 client.29077 trim_cache size 0 max 16384 2024-03-26T08:57:04.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd10d3d0740 1 Processor -- start 2024-03-26T08:57:04.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd10d3d0740 1 -- start start 2024-03-26T08:57:04.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d2690 0x561e9c1dc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 0x561e9c1e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041c2460 con 0x561e9c1d3080 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041bff60 con 0x561e9c1d2690 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041bdfe0 con 0x561e9c1d81f0 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33280/0 (socket says 172.21.15.7:33280) 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd1037fe640 1 -- 172.21.15.7:0/430112774 learned_addr learned my addr 172.21.15.7:0/430112774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd103fff640 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d2690 0x561e9c1dc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd108cb0640 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 0x561e9c1e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd1037fe640 1 -- 172.21.15.7:0/430112774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 msgr2=0x561e9c1e3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.169+0000 7fd1037fe640 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 0x561e9c1e3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.170+0000 7fd1037fe640 1 -- 172.21.15.7:0/430112774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d2690 msgr2=0x561e9c1dc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.170+0000 7fd1037fe640 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d2690 0x561e9c1dc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.170+0000 7fd1037fe640 1 -- 172.21.15.7:0/430112774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e9c1ce970 con 0x561e9c1d3080 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.170+0000 7fd103fff640 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d2690 0x561e9c1dc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.170+0000 7fd1037fe640 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402c6d0 tx=0x7fd0f4036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f141b2c0db7a3865 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.170+0000 7fd0f9ffb640 1 -- 172.21.15.7:0/430112774 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4043070 con 0x561e9c1d3080 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.170+0000 7fd0f9ffb640 1 -- 172.21.15.7:0/430112774 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4003e50 con 0x561e9c1d3080 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.170+0000 7fd0f9ffb640 1 -- 172.21.15.7:0/430112774 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4039050 con 0x561e9c1d3080 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 -- 172.21.15.7:0/430112774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 msgr2=0x561e9c1dcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402c6d0 tx=0x7fd0f4036040 comp rx=0 tx=0).stop 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 -- 172.21.15.7:0/430112774 shutdown_connections 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d81f0 0x561e9c1e3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d3080 0x561e9c1dcee0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/430112774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d2690 0x561e9c1dc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 -- 172.21.15.7:0/430112774 >> 172.21.15.7:0/430112774 conn(0x561e9c1cdc00 msgr2=0x561e9c1eae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 -- 172.21.15.7:0/430112774 shutdown_connections 2024-03-26T08:57:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 -- 172.21.15.7:0/430112774 wait complete. 2024-03-26T08:57:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.171+0000 7fd10d3d0740 1 -- start start 2024-03-26T08:57:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd10d3d0740 1 Processor -- start 2024-03-26T08:57:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 0x561e9c1e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd10d3d0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d81f0 0x561e9c242790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041be5e0 con 0x561e9c1d81f0 2024-03-26T08:57:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041bf900 con 0x561e9c1d3080 2024-03-26T08:57:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd10d3d0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041bec40 con 0x561e9c1d2690 2024-03-26T08:57:04.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd10d3d0740 10 client.0 _ll_register_callbacks cb 0x7fd0ec003880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd103fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 0x561e9c1e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd1037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd108cb0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d81f0 0x561e9c242790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd103fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 0x561e9c1e0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60374/0 (socket says 172.21.15.7:60374) 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd103fff640 1 -- 172.21.15.7:0/3580411164 learned_addr learned my addr 172.21.15.7:0/3580411164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd108cb0640 1 -- 172.21.15.7:0/3580411164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 msgr2=0x561e9c1e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd108cb0640 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 0x561e9c1e0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd108cb0640 1 -- 172.21.15.7:0/3580411164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 msgr2=0x561e9c1e0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd108cb0640 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1e0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd108cb0640 1 -- 172.21.15.7:0/3580411164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e9c1e40a0 con 0x561e9c1d81f0 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.172+0000 7fd1037fe640 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1e0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd108cb0640 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d81f0 0x561e9c242790 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec017040 tx=0x7fd0ec01bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab36b05f7a1296eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd0e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd0e37fe640 1 -- 172.21.15.7:0/3580411164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec01b660 con 0x561e9c1d81f0 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd0e37fe640 1 -- 172.21.15.7:0/3580411164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0ec01b800 con 0x561e9c1d81f0 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3580411164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0f4003e50 con 0x561e9c1d81f0 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd0e37fe640 1 -- 172.21.15.7:0/3580411164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0ec005040 con 0x561e9c1d81f0 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd10d3d0740 10 client.19311 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3580411164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e9c243cc0 con 0x561e9c1d81f0 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd0e37fe640 1 -- 172.21.15.7:0/3580411164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4003e50 con 0x561e9c1d81f0 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd0e37fe640 1 -- 172.21.15.7:0/3580411164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fd0ec024040 con 0x561e9c1d81f0 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd10d3d0740 10 client.19311 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd10d3d0740 10 client.19311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd10d3d0740 20 client.19311 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561e9c2445e0 0x561e9c246a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd10d3d0740 4 client.19311 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.173+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3580411164 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e9c13fc50 con 0x561e9c2445e0 2024-03-26T08:57:04.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd103fff640 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561e9c2445e0 0x561e9c246a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd0e37fe640 10 client.19311 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd103fff640 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561e9c2445e0 0x561e9c246a00 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e10b20c33efefa57 server_cookie=f19ae09897b6bf8b in_seq=0 out_seq=0 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd0e37fe640 1 -- 172.21.15.7:0/3580411164 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e9c13fc50 con 0x561e9c2445e0 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd0e37fe640 10 client.19311 handle_command_reply: tid=0 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 client.19311 shutdown 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3580411164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d81f0 msgr2=0x561e9c242790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d81f0 0x561e9c242790 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec017040 tx=0x7fd0ec01bba0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3580411164 shutdown_connections 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561e9c2445e0 0x561e9c246a00 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561e9c1d81f0 0x561e9c242790 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561e9c1d3080 0x561e9c1e0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd0e37fe640 0 client.19311 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 --2- 172.21.15.7:0/3580411164 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561e9c1d2690 0x561e9c1e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3580411164 >> 172.21.15.7:0/3580411164 conn(0x561e9c1cdc00 msgr2=0x561e9c1dc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.174+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3580411164 shutdown_connections 2024-03-26T08:57:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10d3d0740 1 -- 172.21.15.7:0/3580411164 wait complete. 2024-03-26T08:57:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10d3d0740 20 client.19311 trim_cache size 0 max 16384 2024-03-26T08:57:04.179 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd0f00a2950 msgr2=0x7fd0f00a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd0f00a2950 0x7fd0f00a4e10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x561e9c1d38e0 tx=0x7fd0ec00b2e0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1041086a0 msgr2=0x7fd104118010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1041086a0 0x7fd104118010 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b4a0 tx=0x7fd0fc00b960 comp rx=0 tx=0).stop 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 shutdown_connections 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fd0f00a2950 0x7fd0f00a4e10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd10410d9c0 0x7fd104119590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10410b010 0x7fd104118550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 --2- 172.21.15.7:0/4003782431 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd1041086a0 0x7fd104118010 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 >> 172.21.15.7:0/4003782431 conn(0x7fd10406e640 msgr2=0x7fd10410b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 shutdown_connections 2024-03-26T08:57:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:04.175+0000 7fd10a713640 1 -- 172.21.15.7:0/4003782431 wait complete. 2024-03-26T08:57:04.298 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:04 smithi007 ceph-mon[32462]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 619 op/s 2024-03-26T08:57:04.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:04 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2152004328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:57:04.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:04 smithi016 ceph-mon[35851]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 619 op/s 2024-03-26T08:57:04.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:04 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2152004328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:57:04.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:04 smithi044 ceph-mon[36461]: pgmap v799: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.8 MiB/s wr, 619 op/s 2024-03-26T08:57:04.573 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:04 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2152004328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:57:05.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:57:05.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:57:05.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:57:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 -- 172.21.15.7:0/2864824938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628100850 msgr2=0x7f3628100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 --2- 172.21.15.7:0/2864824938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628100850 0x7f3628100cd0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f361c009930 tx=0x7f361c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 -- 172.21.15.7:0/2864824938 shutdown_connections 2024-03-26T08:57:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 --2- 172.21.15.7:0/2864824938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3628101210 0x7f3628108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 --2- 172.21.15.7:0/2864824938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628100850 0x7f3628100cd0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 --2- 172.21.15.7:0/2864824938 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3628101bb0 0x7f3628101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 -- 172.21.15.7:0/2864824938 >> 172.21.15.7:0/2864824938 conn(0x7f36280fb2c0 msgr2=0x7f36280fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 -- 172.21.15.7:0/2864824938 shutdown_connections 2024-03-26T08:57:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 -- 172.21.15.7:0/2864824938 wait complete. 2024-03-26T08:57:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.314+0000 7f362d685640 1 Processor -- start 2024-03-26T08:57:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f362d685640 1 -- start start 2024-03-26T08:57:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f362d685640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3628100850 0x7f3628100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f362d685640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3628101210 0x7f362819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f362d685640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628101bb0 0x7f362819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f362d685640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362810c260 con 0x7f3628101bb0 2024-03-26T08:57:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f362d685640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362810c0e0 con 0x7f3628101210 2024-03-26T08:57:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f362d685640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362810c3e0 con 0x7f3628100850 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f36277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628101bb0 0x7f362819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f36277fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628101bb0 0x7f362819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33314/0 (socket says 172.21.15.7:33314) 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f36277fe640 1 -- 172.21.15.7:0/2239911617 learned_addr learned my addr 172.21.15.7:0/2239911617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f3626ffd640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3628100850 0x7f3628100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f36267fc640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3628101210 0x7f362819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f36277fe640 1 -- 172.21.15.7:0/2239911617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3628100850 msgr2=0x7f3628100310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f36277fe640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3628100850 0x7f3628100310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f36277fe640 1 -- 172.21.15.7:0/2239911617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3628101210 msgr2=0x7f362819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f36277fe640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3628101210 0x7f362819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.315+0000 7f36277fe640 1 -- 172.21.15.7:0/2239911617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362819f620 con 0x7f3628101bb0 2024-03-26T08:57:05.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.316+0000 7f36277fe640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628101bb0 0x7f362819ed60 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f362000eb10 tx=0x7f362000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.317+0000 7f360ffff640 1 -- 172.21.15.7:0/2239911617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620004b20 con 0x7f3628101bb0 2024-03-26T08:57:05.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.317+0000 7f360ffff640 1 -- 172.21.15.7:0/2239911617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3620004cc0 con 0x7f3628101bb0 2024-03-26T08:57:05.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.317+0000 7f360ffff640 1 -- 172.21.15.7:0/2239911617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620010760 con 0x7f3628101bb0 2024-03-26T08:57:05.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.317+0000 7f362d685640 1 -- 172.21.15.7:0/2239911617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362819f910 con 0x7f3628101bb0 2024-03-26T08:57:05.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.317+0000 7f362d685640 1 -- 172.21.15.7:0/2239911617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362819fde0 con 0x7f3628101bb0 2024-03-26T08:57:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.320+0000 7f360ffff640 1 -- 172.21.15.7:0/2239911617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3620005020 con 0x7f3628101bb0 2024-03-26T08:57:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.320+0000 7f363027c740 1 Processor -- start 2024-03-26T08:57:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.320+0000 7f363027c740 1 -- start start 2024-03-26T08:57:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f363027c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bb960 0x55fc765bbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f363027c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 0x55fc765c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f363027c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f363027c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc764fbc90 con 0x55fc765c14c0 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f363027c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc76521070 con 0x55fc765bc350 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f363027c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc764c9c80 con 0x55fc765bb960 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f36267fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bb960 0x55fc765bbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f3626ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 0x55fc765c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f36267fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bb960 0x55fc765bbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60396/0 (socket says 172.21.15.7:60396) 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f36267fc640 1 -- 172.21.15.7:0/3513671593 learned_addr learned my addr 172.21.15.7:0/3513671593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f36267fc640 1 --2- 172.21.15.7:0/3513671593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f360ffff640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36100a2950 0x7f36100a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f3626ffd640 1 -- 172.21.15.7:0/3513671593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bb960 msgr2=0x55fc765bbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f3626ffd640 1 --2- 172.21.15.7:0/3513671593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bb960 0x55fc765bbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f36277fe640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36100a2950 0x7f36100a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f3626ffd640 1 -- 172.21.15.7:0/3513671593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 msgr2=0x55fc765c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f3626ffd640 1 --2- 172.21.15.7:0/3513671593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f3626ffd640 1 -- 172.21.15.7:0/3513671593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc764c1d10 con 0x55fc765bc350 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f360ffff640 1 -- 172.21.15.7:0/2239911617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f36200ceed0 con 0x7f3628101bb0 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.321+0000 7f36267fc640 1 --2- 172.21.15.7:0/3513671593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bb960 0x55fc765bbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.322+0000 7f3626ffd640 1 --2- 172.21.15.7:0/3513671593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 0x55fc765c0e50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3618009bd0 tx=0x7f3618011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b0fe06e128c504 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.322+0000 7f36277fe640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36100a2950 0x7f36100a4e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f36200cc9d0 tx=0x7f36200666b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.322+0000 7f360d7fa640 1 -- 172.21.15.7:0/3513671593 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618004230 con 0x55fc765bc350 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.322+0000 7f360d7fa640 1 -- 172.21.15.7:0/3513671593 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3618009e40 con 0x55fc765bc350 2024-03-26T08:57:05.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.322+0000 7f360d7fa640 1 -- 172.21.15.7:0/3513671593 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618019750 con 0x55fc765bc350 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 -- 172.21.15.7:0/3513671593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 msgr2=0x55fc765c0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 --2- 172.21.15.7:0/3513671593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 0x55fc765c0e50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3618009bd0 tx=0x7f3618011810 comp rx=0 tx=0).stop 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 -- 172.21.15.7:0/3513671593 shutdown_connections 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 --2- 172.21.15.7:0/3513671593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 --2- 172.21.15.7:0/3513671593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 0x55fc765c0e50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 --2- 172.21.15.7:0/3513671593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bb960 0x55fc765bbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 -- 172.21.15.7:0/3513671593 >> 172.21.15.7:0/3513671593 conn(0x55fc765841a0 msgr2=0x55fc765b72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 -- 172.21.15.7:0/3513671593 shutdown_connections 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 -- 172.21.15.7:0/3513671593 wait complete. 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 -- start start 2024-03-26T08:57:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 Processor -- start 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 0x55fc765c6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765c14c0 0x55fc765cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc764dd0d0 con 0x55fc765bb960 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc7651d110 con 0x55fc765bc350 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc765c4470 con 0x55fc765c14c0 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f363027c740 10 client.0 _ll_register_callbacks cb 0x55fc76518e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f36267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f36267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33334/0 (socket says 172.21.15.7:33334) 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f36267fc640 1 -- 172.21.15.7:0/3923588779 learned_addr learned my addr 172.21.15.7:0/3923588779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.323+0000 7f3626ffd640 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 0x55fc765c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f36267fc640 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765c14c0 0x55fc765cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f36267fc640 1 -- 172.21.15.7:0/3923588779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765c14c0 msgr2=0x55fc765cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f36267fc640 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765c14c0 0x55fc765cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f36267fc640 1 -- 172.21.15.7:0/3923588779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 msgr2=0x55fc765c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f36267fc640 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 0x55fc765c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f36267fc640 1 -- 172.21.15.7:0/3923588779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc762fd7e0 con 0x55fc765bb960 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f36267fc640 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765c5d50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f361c02c170 tx=0x7f361c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4f4109bb336a42 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f360aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f360aff5640 1 -- 172.21.15.7:0/3923588779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c003c30 con 0x55fc765bb960 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f360aff5640 1 -- 172.21.15.7:0/3923588779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f361c003dd0 con 0x55fc765bb960 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f363027c740 1 -- 172.21.15.7:0/3923588779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc7652e5b0 con 0x55fc765bb960 2024-03-26T08:57:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f360aff5640 1 -- 172.21.15.7:0/3923588779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f361c02cca0 con 0x55fc765bb960 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f363027c740 10 client.19323 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f363027c740 1 -- 172.21.15.7:0/3923588779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fc7635df70 con 0x55fc765bb960 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.324+0000 7f360aff5640 1 -- 172.21.15.7:0/3923588779 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fc7652e5b0 con 0x55fc765bb960 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f360aff5640 1 -- 172.21.15.7:0/3923588779 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f361c04d060 con 0x55fc765bb960 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f363027c740 10 client.19323 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f363027c740 10 client.19323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f363027c740 20 client.19323 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f363027c740 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55fc765d1a70 0x55fc765d3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f363027c740 4 client.19323 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f363027c740 1 -- 172.21.15.7:0/3923588779 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc76528900 con 0x55fc765d1a70 2024-03-26T08:57:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f3626ffd640 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55fc765d1a70 0x55fc765d3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f360aff5640 10 client.19323 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.325+0000 7f3626ffd640 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55fc765d1a70 0x55fc765d3ef0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad0a12262c69c00 server_cookie=62b36811d61eb70b in_seq=0 out_seq=0 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f360aff5640 1 -- 172.21.15.7:0/3923588779 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fc76528900 con 0x55fc765d1a70 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f360aff5640 10 client.19323 handle_command_reply: tid=0 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 client.19323 shutdown 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 -- 172.21.15.7:0/3923588779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 msgr2=0x55fc765c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765c5d50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f361c02c170 tx=0x7f361c02e040 comp rx=0 tx=0).stop 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 -- 172.21.15.7:0/3923588779 shutdown_connections 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55fc765d1a70 0x55fc765d3ef0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765c14c0 0x55fc765cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bc350 0x55fc765c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 --2- 172.21.15.7:0/3923588779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765c5d50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f360aff5640 0 client.19323 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:05.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 -- 172.21.15.7:0/3923588779 >> 172.21.15.7:0/3923588779 conn(0x55fc765841a0 msgr2=0x55fc765b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 -- 172.21.15.7:0/3923588779 shutdown_connections 2024-03-26T08:57:05.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 1 -- 172.21.15.7:0/3923588779 wait complete. 2024-03-26T08:57:05.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.326+0000 7f363027c740 20 client.19323 trim_cache size 0 max 16384 2024-03-26T08:57:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.351+0000 7f363027c740 1 Processor -- start 2024-03-26T08:57:05.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.353+0000 7f363027c740 1 -- start start 2024-03-26T08:57:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.353+0000 7f363027c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bb960 0x55fc765bb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.353+0000 7f363027c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bc350 0x55fc765cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.353+0000 7f363027c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.353+0000 7f363027c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281ad070 con 0x55fc765c14c0 2024-03-26T08:57:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.353+0000 7f363027c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281aab70 con 0x55fc765bb960 2024-03-26T08:57:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.353+0000 7f363027c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281a8bf0 con 0x55fc765bc350 2024-03-26T08:57:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.357+0000 7f36267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.357+0000 7f36267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33346/0 (socket says 172.21.15.7:33346) 2024-03-26T08:57:05.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.357+0000 7f36267fc640 1 -- 172.21.15.7:0/3209823917 learned_addr learned my addr 172.21.15.7:0/3209823917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.357+0000 7f36267fc640 1 --2- 172.21.15.7:0/3209823917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bb960 0x55fc765bb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.360+0000 7f36267fc640 1 -- 172.21.15.7:0/3209823917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bc350 msgr2=0x55fc765cd220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:57:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.360+0000 7f36267fc640 1 --2- 172.21.15.7:0/3209823917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bc350 0x55fc765cd220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.360+0000 7f36267fc640 1 -- 172.21.15.7:0/3209823917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bb960 msgr2=0x55fc765bb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.360+0000 7f36267fc640 1 --2- 172.21.15.7:0/3209823917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bb960 0x55fc765bb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.360+0000 7f36267fc640 1 -- 172.21.15.7:0/3209823917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc765c1210 con 0x55fc765c14c0 2024-03-26T08:57:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.362+0000 7f36267fc640 1 --2- 172.21.15.7:0/3209823917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765b6cb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f361c02c9e0 tx=0x7f361c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9651e76360ea49 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.363+0000 7f360cff9640 1 -- 172.21.15.7:0/3209823917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c060070 con 0x55fc765c14c0 2024-03-26T08:57:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.363+0000 7f360cff9640 1 -- 172.21.15.7:0/3209823917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f361c003c30 con 0x55fc765c14c0 2024-03-26T08:57:05.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.364+0000 7f363027c740 1 -- 172.21.15.7:0/3209823917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 msgr2=0x55fc765b6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.364+0000 7f363027c740 1 --2- 172.21.15.7:0/3209823917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765b6cb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f361c02c9e0 tx=0x7f361c000bc0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.369+0000 7f363027c740 1 -- 172.21.15.7:0/3209823917 shutdown_connections 2024-03-26T08:57:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.369+0000 7f363027c740 1 --2- 172.21.15.7:0/3209823917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765c14c0 0x55fc765b6cb0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.369+0000 7f363027c740 1 --2- 172.21.15.7:0/3209823917 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bc350 0x55fc765cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.369+0000 7f363027c740 1 --2- 172.21.15.7:0/3209823917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765bb960 0x55fc765bb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.369+0000 7f363027c740 1 -- 172.21.15.7:0/3209823917 >> 172.21.15.7:0/3209823917 conn(0x55fc765841a0 msgr2=0x55fc765d87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.371+0000 7f363027c740 1 -- 172.21.15.7:0/3209823917 shutdown_connections 2024-03-26T08:57:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.373+0000 7f363027c740 1 -- 172.21.15.7:0/3209823917 wait complete. 2024-03-26T08:57:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.373+0000 7f363027c740 1 -- start start 2024-03-26T08:57:05.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.373+0000 7f363027c740 1 Processor -- start 2024-03-26T08:57:05.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f363027c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f363027c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bc350 0x55fc765cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f363027c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765c14c0 0x55fc765c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f363027c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281a91f0 con 0x55fc765bb960 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f363027c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281aa510 con 0x55fc765c14c0 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f363027c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281a9850 con 0x55fc765bc350 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f363027c740 10 client.0 _ll_register_callbacks cb 0x7f361c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f36267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f36267fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33362/0 (socket says 172.21.15.7:33362) 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f36267fc640 1 -- 172.21.15.7:0/3564884623 learned_addr learned my addr 172.21.15.7:0/3564884623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.374+0000 7f36267fc640 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765c14c0 0x55fc765c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.375+0000 7f36267fc640 1 -- 172.21.15.7:0/3564884623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bc350 msgr2=0x55fc765cd170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.375+0000 7f36267fc640 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bc350 0x55fc765cd170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.375+0000 7f36267fc640 1 -- 172.21.15.7:0/3564884623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765c14c0 msgr2=0x55fc765c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.375+0000 7f36267fc640 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765c14c0 0x55fc765c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.375+0000 7f36267fc640 1 -- 172.21.15.7:0/3564884623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc761f5170 con 0x55fc765bb960 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.375+0000 7f36267fc640 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765ccc30 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f361c05e040 tx=0x7f361c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc306543c9f8eae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.376+0000 7f360aff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.376+0000 7f360aff5640 1 -- 172.21.15.7:0/3564884623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c0347b0 con 0x55fc765bb960 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.376+0000 7f360aff5640 1 -- 172.21.15.7:0/3564884623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f361c034950 con 0x55fc765bb960 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.376+0000 7f363027c740 1 -- 172.21.15.7:0/3564884623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f361c003c30 con 0x55fc765bb960 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.376+0000 7f360aff5640 1 -- 172.21.15.7:0/3564884623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f361c034b50 con 0x55fc765bb960 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.376+0000 7f363027c740 10 client.19329 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.376+0000 7f363027c740 1 -- 172.21.15.7:0/3564884623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fc76627350 con 0x55fc765bb960 2024-03-26T08:57:05.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.377+0000 7f360aff5640 1 -- 172.21.15.7:0/3564884623 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f361c003c30 con 0x55fc765bb960 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.377+0000 7f360aff5640 1 -- 172.21.15.7:0/3564884623 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f361c04d040 con 0x55fc765bb960 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.377+0000 7f363027c740 10 client.19329 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.377+0000 7f363027c740 10 client.19329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.377+0000 7f363027c740 20 client.19329 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.377+0000 7f363027c740 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55fc766279c0 0x55fc76629de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.378+0000 7f363027c740 4 client.19329 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.378+0000 7f363027c740 1 -- 172.21.15.7:0/3564884623 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fc764a6b80 con 0x55fc766279c0 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.378+0000 7f3626ffd640 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55fc766279c0 0x55fc76629de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.378+0000 7f360aff5640 10 client.19329 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.379+0000 7f3626ffd640 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55fc766279c0 0x55fc76629de0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c22ecbb41557e5 server_cookie=a4e0d8c5b7ee6fe7 in_seq=0 out_seq=0 2024-03-26T08:57:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.380+0000 7f360aff5640 1 -- 172.21.15.7:0/3564884623 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fc764a6b80 con 0x55fc766279c0 2024-03-26T08:57:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.380+0000 7f360aff5640 10 client.19329 handle_command_reply: tid=0 2024-03-26T08:57:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 client.19329 shutdown 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 -- 172.21.15.7:0/3564884623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 msgr2=0x55fc765ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765ccc30 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f361c05e040 tx=0x7f361c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 -- 172.21.15.7:0/3564884623 shutdown_connections 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55fc766279c0 0x55fc76629de0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc765c14c0 0x55fc765c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55fc765bc350 0x55fc765cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 --2- 172.21.15.7:0/3564884623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fc765bb960 0x55fc765ccc30 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f360aff5640 0 client.19329 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 -- 172.21.15.7:0/3564884623 >> 172.21.15.7:0/3564884623 conn(0x55fc765841a0 msgr2=0x55fc765d9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 -- 172.21.15.7:0/3564884623 shutdown_connections 2024-03-26T08:57:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 1 -- 172.21.15.7:0/3564884623 wait complete. 2024-03-26T08:57:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.381+0000 7f363027c740 20 client.19329 trim_cache size 0 max 16384 2024-03-26T08:57:05.382 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:57:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:57:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "7f63d58d-cdf9-4986-bfb9-a36330b0d41c", 2024-03-26T08:57:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:57:05.382 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:57:05.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 -- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36100a2950 msgr2=0x7f36100a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36100a2950 0x7f36100a4e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f36200cc9d0 tx=0x7f36200666b0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 -- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628101bb0 msgr2=0x7f362819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628101bb0 0x7f362819ed60 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f362000eb10 tx=0x7f362000c2d0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 -- 172.21.15.7:0/2239911617 shutdown_connections 2024-03-26T08:57:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f36100a2950 0x7f36100a4e10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628101bb0 0x7f362819ed60 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3628101210 0x7f362819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 --2- 172.21.15.7:0/2239911617 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3628100850 0x7f3628100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.382+0000 7f362d685640 1 -- 172.21.15.7:0/2239911617 >> 172.21.15.7:0/2239911617 conn(0x7f36280fb2c0 msgr2=0x7f3628104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.383+0000 7f362d685640 1 -- 172.21.15.7:0/2239911617 shutdown_connections 2024-03-26T08:57:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.383+0000 7f362d685640 1 -- 172.21.15.7:0/2239911617 wait complete. 2024-03-26T08:57:05.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:57:05.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:57:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.526+0000 7fb78b244640 1 -- 172.21.15.7:0/1226015904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb784105250 msgr2=0x7fb7841056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.526+0000 7fb78b244640 1 --2- 172.21.15.7:0/1226015904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb784105250 0x7fb7841056d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009be0 tx=0x7fb77c02bef0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 -- 172.21.15.7:0/1226015904 shutdown_connections 2024-03-26T08:57:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 --2- 172.21.15.7:0/1226015904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb784105250 0x7fb7841056d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 --2- 172.21.15.7:0/1226015904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7841028a0 0x7fb784104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 --2- 172.21.15.7:0/1226015904 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7840fff30 0x7fb784102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 -- 172.21.15.7:0/1226015904 >> 172.21.15.7:0/1226015904 conn(0x7fb7840fbb30 msgr2=0x7fb7840fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 -- 172.21.15.7:0/1226015904 shutdown_connections 2024-03-26T08:57:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 -- 172.21.15.7:0/1226015904 wait complete. 2024-03-26T08:57:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 Processor -- start 2024-03-26T08:57:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 -- start start 2024-03-26T08:57:05.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7840fff30 0x7fb78419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.527+0000 7fb78b244640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7841028a0 0x7fb78419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb78b244640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784105250 0x7fb7841a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb78b244640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784113910 con 0x7fb7841028a0 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb78b244640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784113790 con 0x7fb784105250 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb78b244640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784113a90 con 0x7fb7840fff30 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb783fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7841028a0 0x7fb78419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb788fe0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7840fff30 0x7fb78419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb783fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7841028a0 0x7fb78419e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33368/0 (socket says 172.21.15.7:33368) 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb783fff640 1 -- 172.21.15.7:0/3392337871 learned_addr learned my addr 172.21.15.7:0/3392337871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb7897e1640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784105250 0x7fb7841a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb783fff640 1 -- 172.21.15.7:0/3392337871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7840fff30 msgr2=0x7fb78419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb783fff640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7840fff30 0x7fb78419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb783fff640 1 -- 172.21.15.7:0/3392337871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784105250 msgr2=0x7fb7841a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb783fff640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784105250 0x7fb7841a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb783fff640 1 -- 172.21.15.7:0/3392337871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7841a5b70 con 0x7fb7841028a0 2024-03-26T08:57:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb783fff640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7841028a0 0x7fb78419e490 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb77000d950 tx=0x7fb77000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.528+0000 7fb781ffb640 1 -- 172.21.15.7:0/3392337871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770017070 con 0x7fb7841028a0 2024-03-26T08:57:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.529+0000 7fb781ffb640 1 -- 172.21.15.7:0/3392337871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb77000de80 con 0x7fb7841028a0 2024-03-26T08:57:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.529+0000 7fb78b244640 1 -- 172.21.15.7:0/3392337871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7841a5e60 con 0x7fb7841028a0 2024-03-26T08:57:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.529+0000 7fb781ffb640 1 -- 172.21.15.7:0/3392337871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77000f620 con 0x7fb7841028a0 2024-03-26T08:57:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.529+0000 7fb78b244640 1 -- 172.21.15.7:0/3392337871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7841a6320 con 0x7fb7841028a0 2024-03-26T08:57:05.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb781ffb640 1 -- 172.21.15.7:0/3392337871 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb770009c30 con 0x7fb7841028a0 2024-03-26T08:57:05.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb78de2f740 1 Processor -- start 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb78de2f740 1 -- start start 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 0x5570120191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012019790 0x55701201e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 0x557012020d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb78de2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x557011f7e3d0 con 0x55701201e900 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb78de2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557011f26ad0 con 0x557012018da0 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb78de2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557011f7a470 con 0x557012019790 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb781ffb640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb7740a2800 0x7fb7740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb788fe0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012019790 0x55701201e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb783fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 0x557012020d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb788fe0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012019790 0x55701201e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:60432/0 (socket says 172.21.15.7:60432) 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb788fe0640 1 -- 172.21.15.7:0/2776938315 learned_addr learned my addr 172.21.15.7:0/2776938315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb7897e1640 1 --2- 172.21.15.7:0/2776938315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 0x5570120191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.532+0000 7fb7897e1640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb7740a2800 0x7fb7740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb781ffb640 1 -- 172.21.15.7:0/3392337871 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb7700ce620 con 0x7fb7841028a0 2024-03-26T08:57:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb788fe0640 1 -- 172.21.15.7:0/2776938315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 msgr2=0x5570120191c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb788fe0640 1 --2- 172.21.15.7:0/2776938315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 0x5570120191c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb788fe0640 1 -- 172.21.15.7:0/2776938315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 msgr2=0x557012020d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb788fe0640 1 --2- 172.21.15.7:0/2776938315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 0x557012020d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb788fe0640 1 -- 172.21.15.7:0/2776938315 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557011d5c7d0 con 0x557012019790 2024-03-26T08:57:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb788fe0640 1 --2- 172.21.15.7:0/2776938315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012019790 0x55701201e290 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb778009cd0 tx=0x7fb778011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1a29bc06eb6f87c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb7897e1640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb7740a2800 0x7fb7740a4cc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb77c004ab0 tx=0x7fb77c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb76effd640 1 -- 172.21.15.7:0/2776938315 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77801e070 con 0x557012019790 2024-03-26T08:57:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb76effd640 1 -- 172.21.15.7:0/2776938315 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb778019b10 con 0x557012019790 2024-03-26T08:57:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.533+0000 7fb76effd640 1 -- 172.21.15.7:0/2776938315 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77800eb00 con 0x557012019790 2024-03-26T08:57:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.535+0000 7fb78de2f740 1 -- 172.21.15.7:0/2776938315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012019790 msgr2=0x55701201e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.535+0000 7fb78de2f740 1 --2- 172.21.15.7:0/2776938315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012019790 0x55701201e290 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb778009cd0 tx=0x7fb778011810 comp rx=0 tx=0).stop 2024-03-26T08:57:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.536+0000 7fb78de2f740 1 -- 172.21.15.7:0/2776938315 shutdown_connections 2024-03-26T08:57:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.536+0000 7fb78de2f740 1 --2- 172.21.15.7:0/2776938315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 0x557012020d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.536+0000 7fb78de2f740 1 --2- 172.21.15.7:0/2776938315 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012019790 0x55701201e290 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.537+0000 7fb78de2f740 1 --2- 172.21.15.7:0/2776938315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 0x5570120191c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.537+0000 7fb78de2f740 1 -- 172.21.15.7:0/2776938315 >> 172.21.15.7:0/2776938315 conn(0x557012014310 msgr2=0x557012014710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.537+0000 7fb78de2f740 1 -- 172.21.15.7:0/2776938315 shutdown_connections 2024-03-26T08:57:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.537+0000 7fb78de2f740 1 -- 172.21.15.7:0/2776938315 wait complete. 2024-03-26T08:57:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.537+0000 7fb78de2f740 1 -- start start 2024-03-26T08:57:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.537+0000 7fb78de2f740 1 Processor -- start 2024-03-26T08:57:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 0x557012023220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012019790 0x557012023760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701201e900 0x55701202a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb78de2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570120218b0 con 0x557012019790 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb78de2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557011f3a060 con 0x557012018da0 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb78de2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x557012021a30 con 0x55701201e900 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb783fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012019790 0x557012023760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb78de2f740 10 client.0 _ll_register_callbacks cb 0x557011f1c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb783fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012019790 0x557012023760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33392/0 (socket says 172.21.15.7:33392) 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb788fe0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 0x557012023220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb783fff640 1 -- 172.21.15.7:0/3576106701 learned_addr learned my addr 172.21.15.7:0/3576106701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb7897e1640 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701201e900 0x55701202a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb783fff640 1 -- 172.21.15.7:0/3576106701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701201e900 msgr2=0x55701202a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb783fff640 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701201e900 0x55701202a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb783fff640 1 -- 172.21.15.7:0/3576106701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 msgr2=0x557012023220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb783fff640 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 0x557012023220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb783fff640 1 -- 172.21.15.7:0/3576106701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557011c923f0 con 0x557012019790 2024-03-26T08:57:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb783fff640 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012019790 0x557012023760 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb770097990 tx=0x7fb77000b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6103a4a96851bf9b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.538+0000 7fb763fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb763fff640 1 -- 172.21.15.7:0/3576106701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770020040 con 0x557012019790 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb78de2f740 1 -- 172.21.15.7:0/3576106701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb778019b10 con 0x557012019790 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb763fff640 1 -- 172.21.15.7:0/3576106701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb770095b30 con 0x557012019790 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb763fff640 1 -- 172.21.15.7:0/3576106701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb77009f110 con 0x557012019790 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb78de2f740 10 client.19341 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb78de2f740 1 -- 172.21.15.7:0/3576106701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x557011c520f0 con 0x557012019790 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb763fff640 1 -- 172.21.15.7:0/3576106701 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb778019b10 con 0x557012019790 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb763fff640 1 -- 172.21.15.7:0/3576106701 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb77009d020 con 0x557012019790 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb78de2f740 10 client.19341 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb78de2f740 10 client.19341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb78de2f740 20 client.19341 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55701202ec90 0x557012031110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb78de2f740 4 client.19341 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.539+0000 7fb78de2f740 1 -- 172.21.15.7:0/3576106701 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557011f8b910 con 0x55701202ec90 2024-03-26T08:57:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.540+0000 7fb788fe0640 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55701202ec90 0x557012031110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.540+0000 7fb788fe0640 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55701202ec90 0x557012031110 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39aad31695664674 server_cookie=58dedf7e3ccbb7d3 in_seq=0 out_seq=0 2024-03-26T08:57:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.540+0000 7fb763fff640 10 client.19341 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb763fff640 1 -- 172.21.15.7:0/3576106701 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557011f8b910 con 0x55701202ec90 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb763fff640 10 client.19341 handle_command_reply: tid=0 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 client.19341 shutdown 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 -- 172.21.15.7:0/3576106701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012019790 msgr2=0x557012023760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012019790 0x557012023760 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb770097990 tx=0x7fb77000b550 comp rx=0 tx=0).stop 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 -- 172.21.15.7:0/3576106701 shutdown_connections 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55701202ec90 0x557012031110 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55701201e900 0x55701202a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012019790 0x557012023760 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3576106701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012018da0 0x557012023220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 -- 172.21.15.7:0/3576106701 >> 172.21.15.7:0/3576106701 conn(0x557012014310 msgr2=0x557012013530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb763fff640 0 client.19341 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:05.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 -- 172.21.15.7:0/3576106701 shutdown_connections 2024-03-26T08:57:05.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 1 -- 172.21.15.7:0/3576106701 wait complete. 2024-03-26T08:57:05.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.542+0000 7fb78de2f740 20 client.19341 trim_cache size 0 max 16384 2024-03-26T08:57:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.584+0000 7fb78de2f740 1 Processor -- start 2024-03-26T08:57:05.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.588+0000 7fb78de2f740 1 -- start start 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.588+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012018da0 0x557012012ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.588+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012019790 0x55701202a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.588+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 0x5570120131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.588+0000 7fb78de2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7841b1530 con 0x55701201e900 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.588+0000 7fb78de2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7841af030 con 0x557012019790 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.588+0000 7fb78de2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7841ad0b0 con 0x557012018da0 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.589+0000 7fb783fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012019790 0x55701202a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.589+0000 7fb783fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012019790 0x55701202a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45388/0 (socket says 172.21.15.7:45388) 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.589+0000 7fb783fff640 1 -- 172.21.15.7:0/3749734913 learned_addr learned my addr 172.21.15.7:0/3749734913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.591+0000 7fb788fe0640 1 --2- 172.21.15.7:0/3749734913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012018da0 0x557012012ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.593+0000 7fb7897e1640 1 --2- 172.21.15.7:0/3749734913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 0x5570120131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.596+0000 7fb783fff640 1 -- 172.21.15.7:0/3749734913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012018da0 msgr2=0x557012012ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.596+0000 7fb783fff640 1 --2- 172.21.15.7:0/3749734913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012018da0 0x557012012ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.596+0000 7fb783fff640 1 -- 172.21.15.7:0/3749734913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 msgr2=0x5570120131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.596+0000 7fb783fff640 1 --2- 172.21.15.7:0/3749734913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 0x5570120131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.596+0000 7fb783fff640 1 -- 172.21.15.7:0/3749734913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557012015080 con 0x557012019790 2024-03-26T08:57:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.609+0000 7fb783fff640 1 --2- 172.21.15.7:0/3749734913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012019790 0x55701202a760 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb7700058c0 tx=0x7fb770000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9855aefa61512bb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.614+0000 7fb76e7fc640 1 -- 172.21.15.7:0/3749734913 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7700b7070 con 0x557012019790 2024-03-26T08:57:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.615+0000 7fb76e7fc640 1 -- 172.21.15.7:0/3749734913 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb770097630 con 0x557012019790 2024-03-26T08:57:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.616+0000 7fb78de2f740 1 -- 172.21.15.7:0/3749734913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012019790 msgr2=0x55701202a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.616+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3749734913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012019790 0x55701202a760 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb7700058c0 tx=0x7fb770000bc0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.617+0000 7fb78de2f740 1 -- 172.21.15.7:0/3749734913 shutdown_connections 2024-03-26T08:57:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.617+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3749734913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55701201e900 0x5570120131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.617+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3749734913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557012019790 0x55701202a760 secure :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb7700058c0 tx=0x7fb770000bc0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.617+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3749734913 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012018da0 0x557012012ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.617+0000 7fb78de2f740 1 -- 172.21.15.7:0/3749734913 >> 172.21.15.7:0/3749734913 conn(0x557012014310 msgr2=0x5570120146f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.625+0000 7fb78de2f740 1 -- 172.21.15.7:0/3749734913 shutdown_connections 2024-03-26T08:57:05.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.626+0000 7fb78de2f740 1 -- 172.21.15.7:0/3749734913 wait complete. 2024-03-26T08:57:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.627+0000 7fb78de2f740 1 -- start start 2024-03-26T08:57:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.627+0000 7fb78de2f740 1 Processor -- start 2024-03-26T08:57:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.627+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012018da0 0x55701202a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.628+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55701201e900 0x55701202a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.628+0000 7fb78de2f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012026b10 0x557012026f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.628+0000 7fb78de2f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7841ad6b0 con 0x557012018da0 2024-03-26T08:57:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.628+0000 7fb78de2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7841ae9d0 con 0x55701201e900 2024-03-26T08:57:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.628+0000 7fb78de2f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7841add10 con 0x557012026b10 2024-03-26T08:57:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.628+0000 7fb78de2f740 10 client.0 _ll_register_callbacks cb 0x7fb770097310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.628+0000 7fb788fe0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012018da0 0x55701202a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.628+0000 7fb788fe0640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012018da0 0x55701202a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33406/0 (socket says 172.21.15.7:33406) 2024-03-26T08:57:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.628+0000 7fb788fe0640 1 -- 172.21.15.7:0/3900933954 learned_addr learned my addr 172.21.15.7:0/3900933954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.629+0000 7fb788fe0640 1 -- 172.21.15.7:0/3900933954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012026b10 msgr2=0x557012026f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:57:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.629+0000 7fb788fe0640 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012026b10 0x557012026f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.629+0000 7fb788fe0640 1 -- 172.21.15.7:0/3900933954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55701201e900 msgr2=0x55701202a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:05.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.630+0000 7fb788fe0640 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55701201e900 0x55701202a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.630+0000 7fb788fe0640 1 -- 172.21.15.7:0/3900933954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557012027580 con 0x557012018da0 2024-03-26T08:57:05.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.630+0000 7fb788fe0640 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012018da0 0x55701202a000 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb778006010 tx=0x7fb778009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff1650f427f8dfe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.632+0000 7fb763fff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.632+0000 7fb763fff640 1 -- 172.21.15.7:0/3900933954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77801e040 con 0x557012018da0 2024-03-26T08:57:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.632+0000 7fb78de2f740 1 -- 172.21.15.7:0/3900933954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb770097630 con 0x557012018da0 2024-03-26T08:57:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.632+0000 7fb763fff640 1 -- 172.21.15.7:0/3900933954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb778005040 con 0x557012018da0 2024-03-26T08:57:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.633+0000 7fb763fff640 1 -- 172.21.15.7:0/3900933954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb778039050 con 0x557012018da0 2024-03-26T08:57:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb78de2f740 10 client.19347 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb78de2f740 1 -- 172.21.15.7:0/3900933954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55701209c5f0 con 0x557012018da0 2024-03-26T08:57:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb763fff640 1 -- 172.21.15.7:0/3900933954 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb770097630 con 0x557012018da0 2024-03-26T08:57:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb763fff640 1 -- 172.21.15.7:0/3900933954 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb7780035f0 con 0x557012018da0 2024-03-26T08:57:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb78de2f740 10 client.19347 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb78de2f740 10 client.19347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb78de2f740 20 client.19347 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55701209cf80 0x55701209f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb78de2f740 4 client.19347 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:57:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.634+0000 7fb78de2f740 1 -- 172.21.15.7:0/3900933954 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557011f85c60 con 0x55701209cf80 2024-03-26T08:57:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.635+0000 7fb783fff640 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55701209cf80 0x55701209f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.646+0000 7fb783fff640 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55701209cf80 0x55701209f3a0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce0d8bf27ea6e32 server_cookie=4b17f83078efe325 in_seq=0 out_seq=0 2024-03-26T08:57:05.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.646+0000 7fb763fff640 10 client.19347 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb763fff640 1 -- 172.21.15.7:0/3900933954 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x557011f85c60 con 0x55701209cf80 2024-03-26T08:57:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb763fff640 10 client.19347 handle_command_reply: tid=0 2024-03-26T08:57:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 client.19347 shutdown 2024-03-26T08:57:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 -- 172.21.15.7:0/3900933954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012018da0 msgr2=0x55701202a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012018da0 0x55701202a000 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb778006010 tx=0x7fb778009f40 comp rx=0 tx=0).stop 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 -- 172.21.15.7:0/3900933954 shutdown_connections 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55701209cf80 0x55701209f3a0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557012026b10 0x557012026f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55701201e900 0x55701202a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 --2- 172.21.15.7:0/3900933954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557012018da0 0x55701202a000 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb763fff640 0 client.19347 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 -- 172.21.15.7:0/3900933954 >> 172.21.15.7:0/3900933954 conn(0x557012014310 msgr2=0x557012017570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 -- 172.21.15.7:0/3900933954 shutdown_connections 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 1 -- 172.21.15.7:0/3900933954 wait complete. 2024-03-26T08:57:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.650+0000 7fb78de2f740 20 client.19347 trim_cache size 0 max 16384 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (24 inodes in the stack)", 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "7f63d58d-cdf9-4986-bfb9-a36330b0d41c": { 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "7f63d58d-cdf9-4986-bfb9-a36330b0d41c", 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:57:05.650 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.651+0000 7fb78b244640 1 -- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb7740a2800 msgr2=0x7fb7740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.651+0000 7fb78b244640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb7740a2800 0x7fb7740a4cc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb77c004ab0 tx=0x7fb77c004a40 comp rx=0 tx=0).stop 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.651+0000 7fb78b244640 1 -- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7841028a0 msgr2=0x7fb78419e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.651+0000 7fb78b244640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7841028a0 0x7fb78419e490 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb77000d950 tx=0x7fb77000dd50 comp rx=0 tx=0).stop 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.651+0000 7fb78b244640 1 -- 172.21.15.7:0/3392337871 shutdown_connections 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.652+0000 7fb78b244640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb7740a2800 0x7fb7740a4cc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.652+0000 7fb78b244640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784105250 0x7fb7841a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.652+0000 7fb78b244640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7841028a0 0x7fb78419e490 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.652+0000 7fb78b244640 1 --2- 172.21.15.7:0/3392337871 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb7840fff30 0x7fb78419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.652+0000 7fb78b244640 1 -- 172.21.15.7:0/3392337871 >> 172.21.15.7:0/3392337871 conn(0x7fb7840fbb30 msgr2=0x7fb784100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.652+0000 7fb78b244640 1 -- 172.21.15.7:0/3392337871 shutdown_connections 2024-03-26T08:57:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:05.652+0000 7fb78b244640 1 -- 172.21.15.7:0/3392337871 wait complete. 2024-03-26T08:57:05.662 INFO:tasks.cephfs.filesystem:scrub status for tag:7f63d58d-cdf9-4986-bfb9-a36330b0d41c - {'path': '/', 'tag': '7f63d58d-cdf9-4986-bfb9-a36330b0d41c', 'options': 'recursive,force'} 2024-03-26T08:57:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 875.0 s, 1415.8 tps, lat 69.977 ms stddev 73.388 2024-03-26T08:57:06.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:06 smithi016 ceph-mon[35851]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-03-26T08:57:06.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:06 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:57:06.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:06 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:06.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:06 smithi007 ceph-mon[32462]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-03-26T08:57:06.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:06 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:57:06.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:06 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:06.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:06 smithi044 ceph-mon[36461]: pgmap v800: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.1 MiB/s wr, 650 op/s 2024-03-26T08:57:06.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:06 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:57:06.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:06 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:07.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:07 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:57:07.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:07 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:07.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:07 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:57:07.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:07 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:07.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:07 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:57:07.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:07 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:07.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:07 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:57:07.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:07 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:07.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:07 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:57:07.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:07 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:07.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:07 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:57:07.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:07 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:08.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:08 smithi016 ceph-mon[35851]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-03-26T08:57:08.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:08 smithi007 ceph-mon[32462]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-03-26T08:57:08.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:08 smithi044 ceph-mon[36461]: pgmap v801: 97 pgs: 97 active+clean; 1.6 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.3 MiB/s wr, 580 op/s 2024-03-26T08:57:10.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:10 smithi016 ceph-mon[35851]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.2 MiB/s wr, 721 op/s 2024-03-26T08:57:10.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:10 smithi007 ceph-mon[32462]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.2 MiB/s wr, 721 op/s 2024-03-26T08:57:10.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:10 smithi044 ceph-mon[36461]: pgmap v802: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.2 MiB/s wr, 721 op/s 2024-03-26T08:57:11.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 880.0 s, 2318.2 tps, lat 43.911 ms stddev 58.281 2024-03-26T08:57:12.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:12 smithi016 ceph-mon[35851]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-03-26T08:57:12.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:12 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:57:12.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:12 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:57:12.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:12 smithi007 ceph-mon[32462]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-03-26T08:57:12.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:12 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:57:12.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:12 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:57:12.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:12 smithi044 ceph-mon[36461]: pgmap v803: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.5 MiB/s wr, 766 op/s 2024-03-26T08:57:12.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:12 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:57:12.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:12 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:57:14.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:14 smithi016 ceph-mon[35851]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.2 MiB/s wr, 664 op/s 2024-03-26T08:57:14.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:14 smithi007 ceph-mon[32462]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.2 MiB/s wr, 664 op/s 2024-03-26T08:57:14.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:14 smithi044 ceph-mon[36461]: pgmap v804: 97 pgs: 97 active+clean; 1.6 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 5.2 MiB/s wr, 664 op/s 2024-03-26T08:57:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.203+0000 7f220959d640 1 -- 172.21.15.7:0/3143541774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041086a0 msgr2=0x7f220410aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.203+0000 7f220959d640 1 --2- 172.21.15.7:0/3143541774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041086a0 0x7f220410aad0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009900 tx=0x7f21f402bc80 comp rx=0 tx=0).stop 2024-03-26T08:57:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.204+0000 7f220959d640 1 -- 172.21.15.7:0/3143541774 shutdown_connections 2024-03-26T08:57:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.204+0000 7f220959d640 1 --2- 172.21.15.7:0/3143541774 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f220410d9c0 0x7f220410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.204+0000 7f220959d640 1 --2- 172.21.15.7:0/3143541774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220410b010 0x7f220410d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.204+0000 7f220959d640 1 --2- 172.21.15.7:0/3143541774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041086a0 0x7f220410aad0 secure :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009900 tx=0x7f21f402bc80 comp rx=0 tx=0).stop 2024-03-26T08:57:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.204+0000 7f220959d640 1 -- 172.21.15.7:0/3143541774 >> 172.21.15.7:0/3143541774 conn(0x7f220406e6b0 msgr2=0x7f2204070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.208+0000 7f220959d640 1 -- 172.21.15.7:0/3143541774 shutdown_connections 2024-03-26T08:57:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.210+0000 7f220959d640 1 -- 172.21.15.7:0/3143541774 wait complete. 2024-03-26T08:57:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.210+0000 7f220959d640 1 Processor -- start 2024-03-26T08:57:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.215+0000 7f220959d640 1 -- start start 2024-03-26T08:57:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.215+0000 7f220959d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220410b010 0x7f2204114ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.215+0000 7f220959d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f220410d9c0 0x7f2204115530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.215+0000 7f220959d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041b4950 0x7f22041b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.215+0000 7f220959d640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220411c250 con 0x7f22041b4950 2024-03-26T08:57:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.215+0000 7f220959d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220411c0d0 con 0x7f220410b010 2024-03-26T08:57:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.215+0000 7f220959d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220411c3d0 con 0x7f220410d9c0 2024-03-26T08:57:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.217+0000 7f22037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041b4950 0x7f22041b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.217+0000 7f22037fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041b4950 0x7f22041b6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33350/0 (socket says 172.21.15.7:33350) 2024-03-26T08:57:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.217+0000 7f22037fe640 1 -- 172.21.15.7:0/1380266120 learned_addr learned my addr 172.21.15.7:0/1380266120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.218+0000 7f22037fe640 1 -- 172.21.15.7:0/1380266120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f220410d9c0 msgr2=0x7f2204115530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T08:57:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.218+0000 7f22037fe640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f220410d9c0 0x7f2204115530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.218+0000 7f22037fe640 1 -- 172.21.15.7:0/1380266120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220410b010 msgr2=0x7f2204114ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.218+0000 7f22037fe640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220410b010 0x7f2204114ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.218+0000 7f22037fe640 1 -- 172.21.15.7:0/1380266120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22041b73b0 con 0x7f22041b4950 2024-03-26T08:57:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.218+0000 7f22037fe640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041b4950 0x7f22041b6d40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00b970 tx=0x7f21fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.221+0000 7f21ebfff640 1 -- 172.21.15.7:0/1380266120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc019070 con 0x7f22041b4950 2024-03-26T08:57:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.221+0000 7f21ebfff640 1 -- 172.21.15.7:0/1380266120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21fc00cb40 con 0x7f22041b4950 2024-03-26T08:57:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.221+0000 7f21ebfff640 1 -- 172.21.15.7:0/1380266120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0076c0 con 0x7f22041b4950 2024-03-26T08:57:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.221+0000 7f220959d640 1 -- 172.21.15.7:0/1380266120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22041b7640 con 0x7f22041b4950 2024-03-26T08:57:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.221+0000 7f220959d640 1 -- 172.21.15.7:0/1380266120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22041b7b40 con 0x7f22041b4950 2024-03-26T08:57:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.225+0000 7f220c181740 1 Processor -- start 2024-03-26T08:57:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.225+0000 7f220c181740 1 -- start start 2024-03-26T08:57:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.225+0000 7f220c181740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.225+0000 7f220c181740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.225+0000 7f220c181740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 0x55777b1e5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.225+0000 7f220c181740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777b142b10 con 0x55777b1e3480 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.225+0000 7f220c181740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777b0eb900 con 0x55777b1dd920 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.225+0000 7f220c181740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777b13ebb0 con 0x55777b1de310 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.226+0000 7f22027fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.226+0000 7f22027fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1ddd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47912/0 (socket says 172.21.15.7:47912) 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.226+0000 7f22027fc640 1 -- 172.21.15.7:0/556376753 learned_addr learned my addr 172.21.15.7:0/556376753 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.226+0000 7f22027fc640 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 0x55777b1e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.226+0000 7f21ebfff640 1 -- 172.21.15.7:0/1380266120 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f21fc01c070 con 0x7f22041b4950 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.227+0000 7f2202ffd640 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.227+0000 7f22027fc640 1 -- 172.21.15.7:0/556376753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 msgr2=0x55777b1e2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.227+0000 7f22027fc640 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.227+0000 7f22027fc640 1 -- 172.21.15.7:0/556376753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 msgr2=0x55777b1e5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.227+0000 7f22027fc640 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 0x55777b1e5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.227+0000 7f22027fc640 1 -- 172.21.15.7:0/556376753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55777ae57210 con 0x55777b1dd920 2024-03-26T08:57:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.227+0000 7f21ebfff640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21ec0a29c0 0x7f21ec0a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.227+0000 7f2202ffd640 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:57:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.228+0000 7f21ebfff640 1 -- 172.21.15.7:0/1380266120 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f21fc0cdf40 con 0x7f22041b4950 2024-03-26T08:57:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.228+0000 7f22027fc640 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1ddd40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f21f80363b0 tx=0x7f21f800b0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63255678c440235 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.229+0000 7f2202ffd640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21ec0a29c0 0x7f21ec0a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.229+0000 7f21e97fa640 1 -- 172.21.15.7:0/556376753 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f803e070 con 0x55777b1dd920 2024-03-26T08:57:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.229+0000 7f21e97fa640 1 -- 172.21.15.7:0/556376753 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21f8033040 con 0x55777b1dd920 2024-03-26T08:57:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.230+0000 7f220c181740 1 -- 172.21.15.7:0/556376753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 msgr2=0x55777b1ddd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.230+0000 7f220c181740 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1ddd40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f21f80363b0 tx=0x7f21f800b0a0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.231+0000 7f220c181740 1 -- 172.21.15.7:0/556376753 shutdown_connections 2024-03-26T08:57:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.231+0000 7f220c181740 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 0x55777b1e5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.231+0000 7f220c181740 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.231+0000 7f220c181740 1 --2- 172.21.15.7:0/556376753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1ddd40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.231+0000 7f220c181740 1 -- 172.21.15.7:0/556376753 >> 172.21.15.7:0/556376753 conn(0x55777b1d8e90 msgr2=0x55777b1d9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.231+0000 7f220c181740 1 -- 172.21.15.7:0/556376753 shutdown_connections 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.231+0000 7f220c181740 1 -- 172.21.15.7:0/556376753 wait complete. 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.232+0000 7f220c181740 1 -- start start 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.232+0000 7f220c181740 1 Processor -- start 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.232+0000 7f220c181740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.232+0000 7f220c181740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.232+0000 7f220c181740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 0x55777b1ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.232+0000 7f220c181740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777b1e6420 con 0x55777b1e3480 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.232+0000 7f220c181740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777b0fed50 con 0x55777b1dd920 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.232+0000 7f220c181740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55777b1e65a0 con 0x55777b1de310 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.232+0000 7f220c181740 10 client.0 _ll_register_callbacks cb 0x55777b0fe720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f2202ffd640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21ec0a29c0 0x7f21ec0a4e80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f21f402c800 tx=0x7f21f4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f22027fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f22027fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:33754/0 (socket says 172.21.15.7:33754) 2024-03-26T08:57:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f22027fc640 1 -- 172.21.15.7:0/2483217377 learned_addr learned my addr 172.21.15.7:0/2483217377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f22037fe640 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 0x55777b1ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f2202ffd640 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1e7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f22037fe640 1 -- 172.21.15.7:0/2483217377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 msgr2=0x55777b1e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f22037fe640 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f22037fe640 1 -- 172.21.15.7:0/2483217377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 msgr2=0x55777b1e7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f22037fe640 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1e7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.233+0000 7f22037fe640 1 -- 172.21.15.7:0/2483217377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55777af23620 con 0x55777b1e3480 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.234+0000 7f22037fe640 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 0x55777b1ef1b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f21fc014e90 tx=0x7f21fc09a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7317285421dccc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.234+0000 7f21e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.234+0000 7f21e6ff5640 1 -- 172.21.15.7:0/2483217377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc09bbd0 con 0x55777b1e3480 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.234+0000 7f21e6ff5640 1 -- 172.21.15.7:0/2483217377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f21fc09bd70 con 0x55777b1e3480 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.234+0000 7f220c181740 1 -- 172.21.15.7:0/2483217377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21f8033040 con 0x55777b1e3480 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.235+0000 7f21e6ff5640 1 -- 172.21.15.7:0/2483217377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21fc0b9050 con 0x55777b1e3480 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.235+0000 7f220c181740 10 client.19359 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.235+0000 7f220c181740 1 -- 172.21.15.7:0/2483217377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55777b1f2560 con 0x55777b1e3480 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.236+0000 7f21e6ff5640 1 -- 172.21.15.7:0/2483217377 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21f8033040 con 0x55777b1e3480 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.236+0000 7f21e6ff5640 1 -- 172.21.15.7:0/2483217377 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f21fc099e00 con 0x55777b1e3480 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.236+0000 7f220c181740 10 client.19359 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.236+0000 7f220c181740 10 client.19359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.236+0000 7f220c181740 20 client.19359 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.236+0000 7f220c181740 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55777b1f3ca0 0x55777b1f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.236+0000 7f220c181740 4 client.19359 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.236+0000 7f220c181740 1 -- 172.21.15.7:0/2483217377 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55777b150050 con 0x55777b1f3ca0 2024-03-26T08:57:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.236+0000 7f2202ffd640 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55777b1f3ca0 0x55777b1f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.237+0000 7f21e6ff5640 10 client.19359 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.237+0000 7f2202ffd640 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55777b1f3ca0 0x55777b1f60c0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf6249389de6b78 server_cookie=16ea8ca025c3210e in_seq=0 out_seq=0 2024-03-26T08:57:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f21e6ff5640 1 -- 172.21.15.7:0/2483217377 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55777b150050 con 0x55777b1f3ca0 2024-03-26T08:57:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f21e6ff5640 10 client.19359 handle_command_reply: tid=0 2024-03-26T08:57:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 client.19359 shutdown 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 -- 172.21.15.7:0/2483217377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 msgr2=0x55777b1ef1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 0x55777b1ef1b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f21fc014e90 tx=0x7f21fc09a960 comp rx=0 tx=0).stop 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 -- 172.21.15.7:0/2483217377 shutdown_connections 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55777b1f3ca0 0x55777b1f60c0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1e3480 0x55777b1ef1b0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1de310 0x55777b1e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 --2- 172.21.15.7:0/2483217377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1e7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 -- 172.21.15.7:0/2483217377 >> 172.21.15.7:0/2483217377 conn(0x55777b1d8e90 msgr2=0x55777b1d8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f21e6ff5640 0 client.19359 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 -- 172.21.15.7:0/2483217377 shutdown_connections 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 1 -- 172.21.15.7:0/2483217377 wait complete. 2024-03-26T08:57:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.238+0000 7f220c181740 20 client.19359 trim_cache size 0 max 16384 2024-03-26T08:57:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f220c181740 1 Processor -- start 2024-03-26T08:57:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f220c181740 1 -- start start 2024-03-26T08:57:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f220c181740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1dd920 0x55777b1d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f220c181740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1de310 0x55777b1ef170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f220c181740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 0x55777b1fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f220c181740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f0010900 con 0x55777b1dd920 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f220c181740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f000e400 con 0x55777b1de310 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f220c181740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f000c480 con 0x55777b1e3480 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f22027fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1de310 0x55777b1ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f22027fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1de310 0x55777b1ef170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47924/0 (socket says 172.21.15.7:47924) 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f22027fc640 1 -- 172.21.15.7:0/2016815792 learned_addr learned my addr 172.21.15.7:0/2016815792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f2202ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1dd920 0x55777b1d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f22037fe640 1 --2- 172.21.15.7:0/2016815792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 0x55777b1fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f22027fc640 1 -- 172.21.15.7:0/2016815792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 msgr2=0x55777b1fa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f22027fc640 1 --2- 172.21.15.7:0/2016815792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 0x55777b1fa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f22027fc640 1 -- 172.21.15.7:0/2016815792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1dd920 msgr2=0x55777b1d7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f22027fc640 1 --2- 172.21.15.7:0/2016815792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1dd920 0x55777b1d7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.254+0000 7f22027fc640 1 -- 172.21.15.7:0/2016815792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55777b1ddd00 con 0x55777b1de310 2024-03-26T08:57:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.255+0000 7f22027fc640 1 --2- 172.21.15.7:0/2016815792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1de310 0x55777b1ef170 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f21f8036d00 tx=0x7f21f8000f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3870660c54b6a24e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.255+0000 7f21e8ff9640 1 -- 172.21.15.7:0/2016815792 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8047070 con 0x55777b1de310 2024-03-26T08:57:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.255+0000 7f21e8ff9640 1 -- 172.21.15.7:0/2016815792 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21f8039da0 con 0x55777b1de310 2024-03-26T08:57:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.255+0000 7f21e8ff9640 1 -- 172.21.15.7:0/2016815792 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f803e040 con 0x55777b1de310 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.255+0000 7f220c181740 1 -- 172.21.15.7:0/2016815792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1de310 msgr2=0x55777b1ef170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.255+0000 7f220c181740 1 --2- 172.21.15.7:0/2016815792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1de310 0x55777b1ef170 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f21f8036d00 tx=0x7f21f8000f90 comp rx=0 tx=0).stop 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 -- 172.21.15.7:0/2016815792 shutdown_connections 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 --2- 172.21.15.7:0/2016815792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 0x55777b1fa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 --2- 172.21.15.7:0/2016815792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1de310 0x55777b1ef170 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 --2- 172.21.15.7:0/2016815792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1dd920 0x55777b1d7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 -- 172.21.15.7:0/2016815792 >> 172.21.15.7:0/2016815792 conn(0x55777b1d8e90 msgr2=0x55777b1d7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 -- 172.21.15.7:0/2016815792 shutdown_connections 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 -- 172.21.15.7:0/2016815792 wait complete. 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 -- start start 2024-03-26T08:57:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 Processor -- start 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1de310 0x55777b1ed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f22027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1de310 0x55777b1ed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f22027fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1de310 0x55777b1ed570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33388/0 (socket says 172.21.15.7:33388) 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f2202ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f220c181740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 0x55777b1edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f2202ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1eef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47930/0 (socket says 172.21.15.7:47930) 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.256+0000 7f22027fc640 1 -- 172.21.15.7:0/3881666090 learned_addr learned my addr 172.21.15.7:0/3881666090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f000ca80 con 0x55777b1de310 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f000dda0 con 0x55777b1dd920 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f000d0e0 con 0x55777b1e3480 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f220c181740 10 client.0 _ll_register_callbacks cb 0x7f21fc09afa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f22037fe640 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 0x55777b1edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f22027fc640 1 -- 172.21.15.7:0/3881666090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 msgr2=0x55777b1edab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f22027fc640 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 0x55777b1edab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f22027fc640 1 -- 172.21.15.7:0/3881666090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 msgr2=0x55777b1eef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f22027fc640 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1eef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f22027fc640 1 -- 172.21.15.7:0/3881666090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55777b1ee0f0 con 0x55777b1de310 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f22037fe640 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 0x55777b1edab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f2202ffd640 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f22027fc640 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1de310 0x55777b1ed570 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f21f800bf90 tx=0x7f21f8008c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7fbcf3ba9de9d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f21e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f21e6ff5640 1 -- 172.21.15.7:0/3881666090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8047030 con 0x55777b1de310 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f21e6ff5640 1 -- 172.21.15.7:0/3881666090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f21f8039ad0 con 0x55777b1de310 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21f8039da0 con 0x55777b1de310 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f21e6ff5640 1 -- 172.21.15.7:0/3881666090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21f803e040 con 0x55777b1de310 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f220c181740 10 client.19365 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.257+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55777b2503b0 con 0x55777b1de310 2024-03-26T08:57:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.258+0000 7f21e6ff5640 1 -- 172.21.15.7:0/3881666090 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21f8039da0 con 0x55777b1de310 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.258+0000 7f21e6ff5640 1 -- 172.21.15.7:0/3881666090 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f21f8056020 con 0x55777b1de310 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.258+0000 7f220c181740 10 client.19365 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.258+0000 7f220c181740 10 client.19365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.258+0000 7f220c181740 20 client.19365 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.258+0000 7f220c181740 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55777b250ad0 0x55777b252ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.258+0000 7f220c181740 4 client.19365 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.258+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55777b14a3a0 con 0x55777b250ad0 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.258+0000 7f2202ffd640 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55777b250ad0 0x55777b252ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.259+0000 7f21e6ff5640 10 client.19365 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.259+0000 7f2202ffd640 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55777b250ad0 0x55777b252ef0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59d3ee596a6fa08 server_cookie=7bb52c3577ffef1a in_seq=0 out_seq=0 2024-03-26T08:57:15.274 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.275+0000 7f21e6ff5640 1 -- 172.21.15.7:0/3881666090 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55777b14a3a0 con 0x55777b250ad0 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.275+0000 7f21e6ff5640 10 client.19365 handle_command_reply: tid=0 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.275+0000 7f220c181740 1 client.19365 shutdown 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.275+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1de310 msgr2=0x55777b1ed570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.275+0000 7f220c181740 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1de310 0x55777b1ed570 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f21f800bf90 tx=0x7f21f8008c10 comp rx=0 tx=0).stop 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 shutdown_connections 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220c181740 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55777b250ad0 0x55777b252ef0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220c181740 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55777b1e3480 0x55777b1edab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220c181740 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55777b1de310 0x55777b1ed570 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220c181740 1 --2- 172.21.15.7:0/3881666090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55777b1dd920 0x55777b1eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f21e6ff5640 0 client.19365 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 >> 172.21.15.7:0/3881666090 conn(0x55777b1d8e90 msgr2=0x55777b1d8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 shutdown_connections 2024-03-26T08:57:15.275 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220c181740 1 -- 172.21.15.7:0/3881666090 wait complete. 2024-03-26T08:57:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220c181740 20 client.19365 trim_cache size 0 max 16384 2024-03-26T08:57:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:57:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:57:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:57:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:57:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220959d640 1 -- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21ec0a29c0 msgr2=0x7f21ec0a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220959d640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21ec0a29c0 0x7f21ec0a4e80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f21f402c800 tx=0x7f21f4000bc0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220959d640 1 -- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041b4950 msgr2=0x7f22041b6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.276+0000 7f220959d640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041b4950 0x7f22041b6d40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00b970 tx=0x7f21fc00be30 comp rx=0 tx=0).stop 2024-03-26T08:57:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.277+0000 7f220959d640 1 -- 172.21.15.7:0/1380266120 shutdown_connections 2024-03-26T08:57:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.277+0000 7f220959d640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f21ec0a29c0 0x7f21ec0a4e80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.277+0000 7f220959d640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22041b4950 0x7f22041b6d40 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.277+0000 7f220959d640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f220410d9c0 0x7f2204115530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.277+0000 7f220959d640 1 --2- 172.21.15.7:0/1380266120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f220410b010 0x7f2204114ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.277+0000 7f220959d640 1 -- 172.21.15.7:0/1380266120 >> 172.21.15.7:0/1380266120 conn(0x7f220406e6b0 msgr2=0x7f220410c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.277+0000 7f220959d640 1 -- 172.21.15.7:0/1380266120 shutdown_connections 2024-03-26T08:57:15.278 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:15.277+0000 7f220959d640 1 -- 172.21.15.7:0/1380266120 wait complete. 2024-03-26T08:57:16.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 885.0 s, 1747.6 tps, lat 56.692 ms stddev 70.703 2024-03-26T08:57:16.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:16 smithi016 ceph-mon[35851]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.5 MiB/s wr, 770 op/s 2024-03-26T08:57:16.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:16 smithi007 ceph-mon[32462]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.5 MiB/s wr, 770 op/s 2024-03-26T08:57:16.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:16 smithi044 ceph-mon[36461]: pgmap v805: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.5 MiB/s wr, 770 op/s 2024-03-26T08:57:18.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:18 smithi016 ceph-mon[35851]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.6 MiB/s wr, 681 op/s 2024-03-26T08:57:18.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:18 smithi007 ceph-mon[32462]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.6 MiB/s wr, 681 op/s 2024-03-26T08:57:18.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:18 smithi044 ceph-mon[36461]: pgmap v806: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.6 MiB/s wr, 681 op/s 2024-03-26T08:57:20.502 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:20 smithi016 ceph-mon[35851]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 MiB/s wr, 749 op/s 2024-03-26T08:57:20.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:20 smithi007 ceph-mon[32462]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 MiB/s wr, 749 op/s 2024-03-26T08:57:20.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:20 smithi044 ceph-mon[36461]: pgmap v807: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 MiB/s wr, 749 op/s 2024-03-26T08:57:21.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 890.0 s, 1391.4 tps, lat 71.837 ms stddev 80.381 2024-03-26T08:57:22.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:22 smithi016 ceph-mon[35851]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 694 op/s 2024-03-26T08:57:22.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:22 smithi007 ceph-mon[32462]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 694 op/s 2024-03-26T08:57:22.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:22 smithi044 ceph-mon[36461]: pgmap v808: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.4 MiB/s wr, 694 op/s 2024-03-26T08:57:24.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:24 smithi016 ceph-mon[35851]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-03-26T08:57:24.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:24 smithi007 ceph-mon[32462]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-03-26T08:57:24.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:24 smithi044 ceph-mon[36461]: pgmap v809: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.4 MiB/s wr, 628 op/s 2024-03-26T08:57:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 895.0 s, 1337.0 tps, lat 74.301 ms stddev 78.048 2024-03-26T08:57:26.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:26 smithi016 ceph-mon[35851]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 MiB/s wr, 685 op/s 2024-03-26T08:57:26.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:26 smithi007 ceph-mon[32462]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 MiB/s wr, 685 op/s 2024-03-26T08:57:26.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:26 smithi044 ceph-mon[36461]: pgmap v810: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.9 MiB/s wr, 685 op/s 2024-03-26T08:57:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.404+0000 7fb071fb9640 1 -- 172.21.15.7:0/1338409267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c104b00 msgr2=0x7fb06c106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.404+0000 7fb071fb9640 1 --2- 172.21.15.7:0/1338409267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c104b00 0x7fb06c106ef0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009900 tx=0x7fb05c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.405+0000 7fb071fb9640 1 -- 172.21.15.7:0/1338409267 shutdown_connections 2024-03-26T08:57:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.405+0000 7fb071fb9640 1 --2- 172.21.15.7:0/1338409267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06c107430 0x7fb06c109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.405+0000 7fb071fb9640 1 --2- 172.21.15.7:0/1338409267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c104b00 0x7fb06c106ef0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.406 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.405+0000 7fb071fb9640 1 --2- 172.21.15.7:0/1338409267 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c1021d0 0x7fb06c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.405+0000 7fb071fb9640 1 -- 172.21.15.7:0/1338409267 >> 172.21.15.7:0/1338409267 conn(0x7fb06c0fbb40 msgr2=0x7fb06c0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.405+0000 7fb071fb9640 1 -- 172.21.15.7:0/1338409267 shutdown_connections 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.405+0000 7fb071fb9640 1 -- 172.21.15.7:0/1338409267 wait complete. 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.405+0000 7fb071fb9640 1 Processor -- start 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.405+0000 7fb071fb9640 1 -- start start 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb071fb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06c1021d0 0x7fb06c101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb071fb9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c104b00 0x7fb06c100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb071fb9640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c107430 0x7fb06c1006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb071fb9640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c10d370 con 0x7fb06c107430 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb071fb9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c10d1f0 con 0x7fb06c1021d0 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb071fb9640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c10d4f0 con 0x7fb06c104b00 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c107430 0x7fb06c1006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c104b00 0x7fb06c100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c107430 0x7fb06c1006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51536/0 (socket says 172.21.15.7:51536) 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06bfff640 1 -- 172.21.15.7:0/3955404490 learned_addr learned my addr 172.21.15.7:0/3955404490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06b7fe640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06c1021d0 0x7fb06c101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06bfff640 1 -- 172.21.15.7:0/3955404490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c104b00 msgr2=0x7fb06c100180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06bfff640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c104b00 0x7fb06c100180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06bfff640 1 -- 172.21.15.7:0/3955404490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06c1021d0 msgr2=0x7fb06c101b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06bfff640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06c1021d0 0x7fb06c101b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06bfff640 1 -- 172.21.15.7:0/3955404490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c100f50 con 0x7fb06c107430 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb06bfff640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c107430 0x7fb06c1006c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb06400eb10 tx=0x7fb06400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb068ff9640 1 -- 172.21.15.7:0/3955404490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064004b20 con 0x7fb06c107430 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb068ff9640 1 -- 172.21.15.7:0/3955404490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb064004cc0 con 0x7fb06c107430 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb071fb9640 1 -- 172.21.15.7:0/3955404490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06c1a1b60 con 0x7fb06c107430 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb068ff9640 1 -- 172.21.15.7:0/3955404490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064010760 con 0x7fb06c107430 2024-03-26T08:57:28.408 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.406+0000 7fb071fb9640 1 -- 172.21.15.7:0/3955404490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06c1a2080 con 0x7fb06c107430 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb074ba2740 1 Processor -- start 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb068ff9640 1 -- 172.21.15.7:0/3955404490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb064005020 con 0x7fb06c107430 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb074ba2740 1 -- start start 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f377920 0x55824f377d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f37ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f37d480 0x55824f37f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb074ba2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824f2dcb10 con 0x55824f378310 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb074ba2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824f285900 con 0x55824f377920 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb074ba2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824f2d8bb0 con 0x55824f37d480 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f37ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb06affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f377920 0x55824f377d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.411 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f37ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51544/0 (socket says 172.21.15.7:51544) 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb06b7fe640 1 -- 172.21.15.7:0/1189034536 learned_addr learned my addr 172.21.15.7:0/1189034536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb068ff9640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0540a2870 0x7fb0540a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.409+0000 7fb06affd640 1 --2- 172.21.15.7:0/1189034536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f37d480 0x55824f37f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb06bfff640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0540a2870 0x7fb0540a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb06affd640 1 -- 172.21.15.7:0/1189034536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f37d480 msgr2=0x55824f37f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb06affd640 1 --2- 172.21.15.7:0/1189034536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f37d480 0x55824f37f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb06affd640 1 -- 172.21.15.7:0/1189034536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 msgr2=0x55824f37ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb06affd640 1 --2- 172.21.15.7:0/1189034536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f37ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb06affd640 1 -- 172.21.15.7:0/1189034536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55824eff1210 con 0x55824f377920 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb068ff9640 1 -- 172.21.15.7:0/3955404490 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb0640cefc0 con 0x7fb06c107430 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb06affd640 1 --2- 172.21.15.7:0/1189034536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f377920 0x55824f377d40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009230 tx=0x7fb05c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a173dffd771fe8ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb06bfff640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0540a2870 0x7fb0540a4d30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb0640a3bd0 tx=0x7fb064086040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb051ffb640 1 -- 172.21.15.7:0/1189034536 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c004020 con 0x55824f377920 2024-03-26T08:57:28.412 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb051ffb640 1 -- 172.21.15.7:0/1189034536 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb05c0041c0 con 0x55824f377920 2024-03-26T08:57:28.413 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.410+0000 7fb051ffb640 1 -- 172.21.15.7:0/1189034536 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c043480 con 0x55824f377920 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.413+0000 7fb074ba2740 1 -- 172.21.15.7:0/1189034536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f377920 msgr2=0x55824f377d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.413+0000 7fb074ba2740 1 --2- 172.21.15.7:0/1189034536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f377920 0x55824f377d40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009230 tx=0x7fb05c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.413+0000 7fb074ba2740 1 -- 172.21.15.7:0/1189034536 shutdown_connections 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.413+0000 7fb074ba2740 1 --2- 172.21.15.7:0/1189034536 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f37d480 0x55824f37f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.413+0000 7fb074ba2740 1 --2- 172.21.15.7:0/1189034536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f37ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.413+0000 7fb074ba2740 1 --2- 172.21.15.7:0/1189034536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f377920 0x55824f377d40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.413+0000 7fb074ba2740 1 -- 172.21.15.7:0/1189034536 >> 172.21.15.7:0/1189034536 conn(0x55824f372e90 msgr2=0x55824f373290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.413+0000 7fb074ba2740 1 -- 172.21.15.7:0/1189034536 shutdown_connections 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.413+0000 7fb074ba2740 1 -- 172.21.15.7:0/1189034536 wait complete. 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb074ba2740 1 -- start start 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb074ba2740 1 Processor -- start 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f381da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f37d480 0x55824f3822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f387310 0x55824f389700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb074ba2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824f3803d0 con 0x55824f378310 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb074ba2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824f298d50 con 0x55824f37d480 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb074ba2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55824f380550 con 0x55824f387310 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb074ba2740 10 client.0 _ll_register_callbacks cb 0x55824f298720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb06affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f381da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f37d480 0x55824f3822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb06affd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f381da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51554/0 (socket says 172.21.15.7:51554) 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb06affd640 1 -- 172.21.15.7:0/256084460 learned_addr learned my addr 172.21.15.7:0/256084460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f37d480 0x55824f3822e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55340/0 (socket says 172.21.15.7:55340) 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.414+0000 7fb06affd640 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f387310 0x55824f389700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb06b7fe640 1 -- 172.21.15.7:0/256084460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f387310 msgr2=0x55824f389700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb06b7fe640 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f387310 0x55824f389700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb06b7fe640 1 -- 172.21.15.7:0/256084460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 msgr2=0x55824f381da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb06b7fe640 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f381da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb06b7fe640 1 -- 172.21.15.7:0/256084460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55824f0bd620 con 0x55824f37d480 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb06b7fe640 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f37d480 0x55824f3822e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55824f378710 tx=0x7fb06000bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a3051ec0be43134 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb04affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb04affd640 1 -- 172.21.15.7:0/256084460 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060003df0 con 0x55824f37d480 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb04affd640 1 -- 172.21.15.7:0/256084460 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb060011440 con 0x55824f37d480 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb074ba2740 1 -- 172.21.15.7:0/256084460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb05c0041c0 con 0x55824f37d480 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb04affd640 1 -- 172.21.15.7:0/256084460 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb060011640 con 0x55824f37d480 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb074ba2740 10 client.29147 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb074ba2740 1 -- 172.21.15.7:0/256084460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55824f38cae0 con 0x55824f37d480 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.415+0000 7fb04affd640 1 -- 172.21.15.7:0/256084460 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb05c0041c0 con 0x55824f37d480 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb04affd640 1 -- 172.21.15.7:0/256084460 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb06001b980 con 0x55824f37d480 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb074ba2740 10 client.29147 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb074ba2740 10 client.29147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb074ba2740 20 client.29147 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb074ba2740 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55824f38e150 0x55824f3905d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb074ba2740 4 client.29147 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb074ba2740 1 -- 172.21.15.7:0/256084460 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55824f2ea050 con 0x55824f38e150 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb06affd640 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55824f38e150 0x55824f3905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb06affd640 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55824f38e150 0x55824f3905d0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec136755ab806381 server_cookie=fc980b4ad1be385 in_seq=0 out_seq=0 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.416+0000 7fb04affd640 10 client.29147 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb04affd640 1 -- 172.21.15.7:0/256084460 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55824f2ea050 con 0x55824f38e150 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb04affd640 10 client.29147 handle_command_reply: tid=0 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 client.29147 shutdown 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 -- 172.21.15.7:0/256084460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f37d480 msgr2=0x55824f3822e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f37d480 0x55824f3822e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x55824f378710 tx=0x7fb06000bf70 comp rx=0 tx=0).stop 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 -- 172.21.15.7:0/256084460 shutdown_connections 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55824f38e150 0x55824f3905d0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f387310 0x55824f389700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f37d480 0x55824f3822e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 --2- 172.21.15.7:0/256084460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f378310 0x55824f381da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb04affd640 0 client.29147 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 -- 172.21.15.7:0/256084460 >> 172.21.15.7:0/256084460 conn(0x55824f372e90 msgr2=0x55824f372020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 -- 172.21.15.7:0/256084460 shutdown_connections 2024-03-26T08:57:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 1 -- 172.21.15.7:0/256084460 wait complete. 2024-03-26T08:57:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.417+0000 7fb074ba2740 20 client.29147 trim_cache size 0 max 16384 2024-03-26T08:57:28.433 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.434+0000 7fb074ba2740 1 Processor -- start 2024-03-26T08:57:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.434+0000 7fb074ba2740 1 -- start start 2024-03-26T08:57:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f378310 0x55824f373270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 0x55824f381da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb06affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f378310 0x55824f373270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb06affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f378310 0x55824f373270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55346/0 (socket says 172.21.15.7:55346) 2024-03-26T08:57:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f387310 0x55824f3822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb06affd640 1 -- 172.21.15.7:0/3950609336 learned_addr learned my addr 172.21.15.7:0/3950609336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb074ba2740 1 -- 172.21.15.7:0/3950609336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c1ad260 con 0x55824f37d480 2024-03-26T08:57:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb074ba2740 1 -- 172.21.15.7:0/3950609336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c1aad60 con 0x55824f378310 2024-03-26T08:57:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb074ba2740 1 -- 172.21.15.7:0/3950609336 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c1a8de0 con 0x55824f387310 2024-03-26T08:57:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb06affd640 1 -- 172.21.15.7:0/3950609336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f387310 msgr2=0x55824f3822e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:28.435 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb06affd640 1 --2- 172.21.15.7:0/3950609336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f387310 0x55824f3822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb06affd640 1 -- 172.21.15.7:0/3950609336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 msgr2=0x55824f381da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb06affd640 1 --2- 172.21.15.7:0/3950609336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 0x55824f381da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.435+0000 7fb06affd640 1 -- 172.21.15.7:0/3950609336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55824f38def0 con 0x55824f378310 2024-03-26T08:57:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.436+0000 7fb06affd640 1 --2- 172.21.15.7:0/3950609336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f378310 0x55824f373270 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb05c005ab0 tx=0x7fb05c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ff2018a72f2f3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.436+0000 7fb0517fa640 1 -- 172.21.15.7:0/3950609336 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0541c0 con 0x55824f378310 2024-03-26T08:57:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.436+0000 7fb0517fa640 1 -- 172.21.15.7:0/3950609336 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb05c04a040 con 0x55824f378310 2024-03-26T08:57:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.436+0000 7fb0517fa640 1 -- 172.21.15.7:0/3950609336 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0052a0 con 0x55824f378310 2024-03-26T08:57:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 -- 172.21.15.7:0/3950609336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f378310 msgr2=0x55824f373270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 --2- 172.21.15.7:0/3950609336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f378310 0x55824f373270 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb05c005ab0 tx=0x7fb05c000bc0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 -- 172.21.15.7:0/3950609336 shutdown_connections 2024-03-26T08:57:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 --2- 172.21.15.7:0/3950609336 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f387310 0x55824f3822e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 --2- 172.21.15.7:0/3950609336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 0x55824f381da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 --2- 172.21.15.7:0/3950609336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f378310 0x55824f373270 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 -- 172.21.15.7:0/3950609336 >> 172.21.15.7:0/3950609336 conn(0x55824f372e90 msgr2=0x55824f36deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 -- 172.21.15.7:0/3950609336 shutdown_connections 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 -- 172.21.15.7:0/3950609336 wait complete. 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.437+0000 7fb074ba2740 1 -- start start 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb074ba2740 1 Processor -- start 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f378310 0x55824f3a0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 0x55824f3a0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb074ba2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f387310 0x55824f39e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb074ba2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c1a93e0 con 0x55824f37d480 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb074ba2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c1aa700 con 0x55824f387310 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb074ba2740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c1a9a40 con 0x55824f378310 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb074ba2740 10 client.0 _ll_register_callbacks cb 0x7fb0600043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 0x55824f3a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 0x55824f3a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51568/0 (socket says 172.21.15.7:51568) 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06b7fe640 1 -- 172.21.15.7:0/4140714224 learned_addr learned my addr 172.21.15.7:0/4140714224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06affd640 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f378310 0x55824f3a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06affd640 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f387310 0x55824f39e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06b7fe640 1 -- 172.21.15.7:0/4140714224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f378310 msgr2=0x55824f3a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06b7fe640 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f378310 0x55824f3a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06b7fe640 1 -- 172.21.15.7:0/4140714224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f387310 msgr2=0x55824f39e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06b7fe640 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f387310 0x55824f39e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06b7fe640 1 -- 172.21.15.7:0/4140714224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55824f39ef80 con 0x55824f37d480 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.438+0000 7fb06affd640 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f387310 0x55824f39e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb06b7fe640 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 0x55824f3a0680 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb0600058c0 tx=0x7fb060004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcf82c921eab618 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb04affd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb04affd640 1 -- 172.21.15.7:0/4140714224 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060005250 con 0x55824f37d480 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb074ba2740 1 -- 172.21.15.7:0/4140714224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb05c04a040 con 0x55824f37d480 2024-03-26T08:57:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb04affd640 1 -- 172.21.15.7:0/4140714224 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb0600053f0 con 0x55824f37d480 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb04affd640 1 -- 172.21.15.7:0/4140714224 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0600055f0 con 0x55824f37d480 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb074ba2740 10 client.19380 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb074ba2740 1 -- 172.21.15.7:0/4140714224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55824f3e9ac0 con 0x55824f37d480 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb04affd640 1 -- 172.21.15.7:0/4140714224 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb05c04a040 con 0x55824f37d480 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb04affd640 1 -- 172.21.15.7:0/4140714224 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fb06001b910 con 0x55824f37d480 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb074ba2740 10 client.19380 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb074ba2740 10 client.19380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb074ba2740 20 client.19380 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb074ba2740 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55824f3ea420 0x55824f3ec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb074ba2740 4 client.19380 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.439+0000 7fb074ba2740 1 -- 172.21.15.7:0/4140714224 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55824f2e43a0 con 0x55824f3ea420 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.440+0000 7fb06affd640 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55824f3ea420 0x55824f3ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.440+0000 7fb04affd640 10 client.19380 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.440+0000 7fb06affd640 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55824f3ea420 0x55824f3ec840 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5dc891aec02909 server_cookie=f1987d54de3a8386 in_seq=0 out_seq=0 2024-03-26T08:57:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb04affd640 1 -- 172.21.15.7:0/4140714224 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55824f2e43a0 con 0x55824f3ea420 2024-03-26T08:57:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb04affd640 10 client.19380 handle_command_reply: tid=0 2024-03-26T08:57:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 client.19380 shutdown 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 -- 172.21.15.7:0/4140714224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 msgr2=0x55824f3a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 0x55824f3a0680 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb0600058c0 tx=0x7fb060004300 comp rx=0 tx=0).stop 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 -- 172.21.15.7:0/4140714224 shutdown_connections 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55824f3ea420 0x55824f3ec840 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55824f387310 0x55824f39e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55824f37d480 0x55824f3a0680 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb04affd640 0 client.19380 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 --2- 172.21.15.7:0/4140714224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55824f378310 0x55824f3a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 -- 172.21.15.7:0/4140714224 >> 172.21.15.7:0/4140714224 conn(0x55824f372e90 msgr2=0x55824f373270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 -- 172.21.15.7:0/4140714224 shutdown_connections 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 1 -- 172.21.15.7:0/4140714224 wait complete. 2024-03-26T08:57:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.467+0000 7fb074ba2740 20 client.19380 trim_cache size 0 max 16384 2024-03-26T08:57:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:57:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:57:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:57:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 -- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0540a2870 msgr2=0x7fb0540a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0540a2870 0x7fb0540a4d30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb0640a3bd0 tx=0x7fb064086040 comp rx=0 tx=0).stop 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 -- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c107430 msgr2=0x7fb06c1006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c107430 0x7fb06c1006c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb06400eb10 tx=0x7fb06400c2d0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 -- 172.21.15.7:0/3955404490 shutdown_connections 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb0540a2870 0x7fb0540a4d30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb06c107430 0x7fb06c1006c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb06c104b00 0x7fb06c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 --2- 172.21.15.7:0/3955404490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb06c1021d0 0x7fb06c101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 -- 172.21.15.7:0/3955404490 >> 172.21.15.7:0/3955404490 conn(0x7fb06c0fbb40 msgr2=0x7fb06c102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 -- 172.21.15.7:0/3955404490 shutdown_connections 2024-03-26T08:57:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:28.468+0000 7fb071fb9640 1 -- 172.21.15.7:0/3955404490 wait complete. 2024-03-26T08:57:28.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:28 smithi016 ceph-mon[35851]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.6 MiB/s wr, 564 op/s 2024-03-26T08:57:28.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:28 smithi007 ceph-mon[32462]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.6 MiB/s wr, 564 op/s 2024-03-26T08:57:28.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:28 smithi044 ceph-mon[36461]: pgmap v811: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.6 MiB/s wr, 564 op/s 2024-03-26T08:57:30.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:30 smithi016 ceph-mon[35851]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.3 MiB/s wr, 648 op/s 2024-03-26T08:57:30.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:30 smithi007 ceph-mon[32462]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.3 MiB/s wr, 648 op/s 2024-03-26T08:57:30.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:30 smithi044 ceph-mon[36461]: pgmap v812: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.3 MiB/s wr, 648 op/s 2024-03-26T08:57:31.077 INFO:teuthology.orchestra.run.smithi007.stderr:progress: 900.0 s, 1532.2 tps, lat 65.724 ms stddev 78.280 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:transaction type: 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:scaling factor: 32 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:query mode: simple 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:number of clients: 100 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:number of threads: 4 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:duration: 900 s 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:number of transactions actually processed: 2391452 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:latency average = 37.630 ms 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:latency stddev = 49.357 ms 2024-03-26T08:57:31.181 INFO:teuthology.orchestra.run.smithi007.stdout:tps = 2656.863607 (including connections establishing) 2024-03-26T08:57:31.182 INFO:teuthology.orchestra.run.smithi007.stdout:tps = 2656.871150 (excluding connections establishing) 2024-03-26T08:57:31.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-03-26T08:57:32.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:32 smithi016 ceph-mon[35851]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-03-26T08:57:32.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:32 smithi007 ceph-mon[32462]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-03-26T08:57:32.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:32 smithi044 ceph-mon[36461]: pgmap v813: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 673 op/s 2024-03-26T08:57:34.499 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:34 smithi016 ceph-mon[35851]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 591 op/s 2024-03-26T08:57:34.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:34 smithi007 ceph-mon[32462]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 591 op/s 2024-03-26T08:57:34.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:34 smithi044 ceph-mon[36461]: pgmap v814: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.6 MiB/s wr, 591 op/s 2024-03-26T08:57:35.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:57:35.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:57:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.828+0000 7f8d16f59640 1 -- 172.21.15.7:0/1909646601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d101028a0 msgr2=0x7f8d10102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.828+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1909646601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d101028a0 0x7f8d10102d20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009900 tx=0x7f8d0002bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.828+0000 7f8d16f59640 1 -- 172.21.15.7:0/1909646601 shutdown_connections 2024-03-26T08:57:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.828+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1909646601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d10103260 0x7f8d10109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.828+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1909646601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d101028a0 0x7f8d10102d20 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.828+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1909646601 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d101016b0 0x7f8d10101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.828+0000 7f8d16f59640 1 -- 172.21.15.7:0/1909646601 >> 172.21.15.7:0/1909646601 conn(0x7f8d100fcc20 msgr2=0x7f8d100ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.835+0000 7f8d16f59640 1 -- 172.21.15.7:0/1909646601 shutdown_connections 2024-03-26T08:57:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.835+0000 7f8d16f59640 1 -- 172.21.15.7:0/1909646601 wait complete. 2024-03-26T08:57:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.836+0000 7f8d16f59640 1 Processor -- start 2024-03-26T08:57:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.836+0000 7f8d16f59640 1 -- start start 2024-03-26T08:57:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.836+0000 7f8d16f59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d101016b0 0x7f8d10192e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.836+0000 7f8d16f59640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d101028a0 0x7f8d101933a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.836+0000 7f8d16f59640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d10103260 0x7f8d1019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.836+0000 7f8d16f59640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1010d3a0 con 0x7f8d10103260 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.836+0000 7f8d16f59640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1010d220 con 0x7f8d101016b0 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.836+0000 7f8d16f59640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1010d520 con 0x7f8d101028a0 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.836+0000 7f8d154f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d10103260 0x7f8d1019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.837+0000 7f8d154f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d10103260 0x7f8d1019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51574/0 (socket says 172.21.15.7:51574) 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.837+0000 7f8d154f6640 1 -- 172.21.15.7:0/1012860111 learned_addr learned my addr 172.21.15.7:0/1012860111 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.837+0000 7f8d0ffff640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d101028a0 0x7f8d101933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.837+0000 7f8d14cf5640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d101016b0 0x7f8d10192e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.837+0000 7f8d154f6640 1 -- 172.21.15.7:0/1012860111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d101028a0 msgr2=0x7f8d101933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.837+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d101028a0 0x7f8d101933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.837+0000 7f8d154f6640 1 -- 172.21.15.7:0/1012860111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d101016b0 msgr2=0x7f8d10192e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:35.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.837+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d101016b0 0x7f8d10192e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.837+0000 7f8d154f6640 1 -- 172.21.15.7:0/1012860111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1019aa80 con 0x7f8d10103260 2024-03-26T08:57:35.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.838+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d10103260 0x7f8d1019a410 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800eb10 tx=0x7f8d0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.838+0000 7f8d0dffb640 1 -- 172.21.15.7:0/1012860111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08004b20 con 0x7f8d10103260 2024-03-26T08:57:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.838+0000 7f8d0dffb640 1 -- 172.21.15.7:0/1012860111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d08004cc0 con 0x7f8d10103260 2024-03-26T08:57:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.838+0000 7f8d0dffb640 1 -- 172.21.15.7:0/1012860111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08010760 con 0x7f8d10103260 2024-03-26T08:57:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.838+0000 7f8d16f59640 1 -- 172.21.15.7:0/1012860111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1019ad70 con 0x7f8d10103260 2024-03-26T08:57:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.839+0000 7f8d16f59640 1 -- 172.21.15.7:0/1012860111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d101aec20 con 0x7f8d10103260 2024-03-26T08:57:35.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.843+0000 7f8d19c11740 1 Processor -- start 2024-03-26T08:57:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.845+0000 7f8d0dffb640 1 -- 172.21.15.7:0/1012860111 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8d08005020 con 0x7f8d10103260 2024-03-26T08:57:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.845+0000 7f8d0dffb640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cf80a2900 0x7f8cf80a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d0dffb640 1 -- 172.21.15.7:0/1012860111 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8d080cf230 con 0x7f8d10103260 2024-03-26T08:57:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d19c11740 1 -- start start 2024-03-26T08:57:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a88da0 0x55f9d8a891c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a89790 0x55f9d8a8e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d19c11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d89ee3d0 con 0x55f9d8a8e900 2024-03-26T08:57:35.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d19c11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d8996ad0 con 0x55f9d8a88da0 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d19c11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d89ea470 con 0x55f9d8a89790 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d0ffff640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cf80a2900 0x7f8cf80a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d154f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d154f6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a90d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51580/0 (socket says 172.21.15.7:51580) 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.846+0000 7f8d154f6640 1 -- 172.21.15.7:0/1553816736 learned_addr learned my addr 172.21.15.7:0/1553816736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.847+0000 7f8d0ffff640 1 --2- 172.21.15.7:0/1553816736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a89790 0x55f9d8a8e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.847+0000 7f8d154f6640 1 -- 172.21.15.7:0/1553816736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a89790 msgr2=0x55f9d8a8e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.847+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1553816736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a89790 0x55f9d8a8e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.847+0000 7f8d154f6640 1 -- 172.21.15.7:0/1553816736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a88da0 msgr2=0x55f9d8a891c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.847+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1553816736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a88da0 0x55f9d8a891c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.847+0000 7f8d154f6640 1 -- 172.21.15.7:0/1553816736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d87cc7d0 con 0x55f9d8a8e900 2024-03-26T08:57:35.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.848+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1553816736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a90d90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8d08098560 tx=0x7f8d0809b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4df7b8d64d68a3f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.848+0000 7f8cf6ffd640 1 -- 172.21.15.7:0/1553816736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0809bb90 con 0x55f9d8a8e900 2024-03-26T08:57:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.848+0000 7f8cf6ffd640 1 -- 172.21.15.7:0/1553816736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d080b4070 con 0x55f9d8a8e900 2024-03-26T08:57:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.848+0000 7f8cf6ffd640 1 -- 172.21.15.7:0/1553816736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08005310 con 0x55f9d8a8e900 2024-03-26T08:57:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.849+0000 7f8d19c11740 1 -- 172.21.15.7:0/1553816736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 msgr2=0x55f9d8a90d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:35.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.849+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1553816736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a90d90 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8d08098560 tx=0x7f8d0809b6d0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.849+0000 7f8d0ffff640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cf80a2900 0x7f8cf80a4dc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8d0002c240 tx=0x7f8d0002c980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.849+0000 7f8d19c11740 1 -- 172.21.15.7:0/1553816736 shutdown_connections 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.849+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1553816736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a90d90 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.849+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1553816736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a89790 0x55f9d8a8e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.849+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1553816736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a88da0 0x55f9d8a891c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.849+0000 7f8d19c11740 1 -- 172.21.15.7:0/1553816736 >> 172.21.15.7:0/1553816736 conn(0x55f9d8a84310 msgr2=0x55f9d8a84710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.850+0000 7f8d19c11740 1 -- 172.21.15.7:0/1553816736 shutdown_connections 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.850+0000 7f8d19c11740 1 -- 172.21.15.7:0/1553816736 wait complete. 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.850+0000 7f8d19c11740 1 -- start start 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.850+0000 7f8d19c11740 1 Processor -- start 2024-03-26T08:57:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 0x55f9d8a937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a9a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d19c11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d8a918a0 con 0x55f9d8a8e900 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d19c11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d89aa060 con 0x55f9d8a89790 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d19c11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d8a91a20 con 0x55f9d8a88da0 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d19c11740 10 client.0 _ll_register_callbacks cb 0x55f9d898c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d14cf5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d14cf5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:42996/0 (socket says 172.21.15.7:42996) 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d14cf5640 1 -- 172.21.15.7:0/1463960147 learned_addr learned my addr 172.21.15.7:0/1463960147 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:35.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a9a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.851+0000 7f8d0ffff640 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 0x55f9d8a937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.852+0000 7f8d154f6640 1 -- 172.21.15.7:0/1463960147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 msgr2=0x55f9d8a93270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.852+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.852+0000 7f8d154f6640 1 -- 172.21.15.7:0/1463960147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 msgr2=0x55f9d8a937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.852+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 0x55f9d8a937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.852+0000 7f8d154f6640 1 -- 172.21.15.7:0/1463960147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9d87023f0 con 0x55f9d8a8e900 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.852+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a9a7f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8d0809bb20 tx=0x7f8d080984a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7de94ca0c7d2a2f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.852+0000 7f8ceffff640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.853+0000 7f8ceffff640 1 -- 172.21.15.7:0/1463960147 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d080a4330 con 0x55f9d8a8e900 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.853+0000 7f8d19c11740 1 -- 172.21.15.7:0/1463960147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d080b4070 con 0x55f9d8a8e900 2024-03-26T08:57:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.853+0000 7f8ceffff640 1 -- 172.21.15.7:0/1463960147 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8d0809b1f0 con 0x55f9d8a8e900 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.854+0000 7f8ceffff640 1 -- 172.21.15.7:0/1463960147 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d0809bb90 con 0x55f9d8a8e900 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.854+0000 7f8d19c11740 10 client.19392 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.854+0000 7f8d19c11740 1 -- 172.21.15.7:0/1463960147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f9d86c20f0 con 0x55f9d8a8e900 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.854+0000 7f8ceffff640 1 -- 172.21.15.7:0/1463960147 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d080b4070 con 0x55f9d8a8e900 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.855+0000 7f8ceffff640 1 -- 172.21.15.7:0/1463960147 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8d080a0c00 con 0x55f9d8a8e900 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.855+0000 7f8d19c11740 10 client.19392 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.855+0000 7f8d19c11740 10 client.19392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.855+0000 7f8d19c11740 20 client.19392 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.855+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f9d8a9edd0 0x55f9d8aa11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.855+0000 7f8d19c11740 4 client.19392 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:35.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.855+0000 7f8d19c11740 1 -- 172.21.15.7:0/1463960147 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9d89fb910 con 0x55f9d8a9edd0 2024-03-26T08:57:35.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.855+0000 7f8d14cf5640 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f9d8a9edd0 0x55f9d8aa11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.856+0000 7f8d14cf5640 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f9d8a9edd0 0x55f9d8aa11c0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96998df19c080254 server_cookie=40b99bfb3079d08b in_seq=0 out_seq=0 2024-03-26T08:57:35.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.856+0000 7f8ceffff640 10 client.19392 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:35.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.857+0000 7f8ceffff640 1 -- 172.21.15.7:0/1463960147 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f9d89fb910 con 0x55f9d8a9edd0 2024-03-26T08:57:35.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.857+0000 7f8ceffff640 10 client.19392 handle_command_reply: tid=0 2024-03-26T08:57:35.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.857+0000 7f8d19c11740 1 client.19392 shutdown 2024-03-26T08:57:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8d19c11740 1 -- 172.21.15.7:0/1463960147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 msgr2=0x55f9d8a9a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a9a7f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f8d0809bb20 tx=0x7f8d080984a0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8d19c11740 1 -- 172.21.15.7:0/1463960147 shutdown_connections 2024-03-26T08:57:35.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f9d8a9edd0 0x55f9d8aa11c0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a9a7f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 0x55f9d8a937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1463960147 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8d19c11740 1 -- 172.21.15.7:0/1463960147 >> 172.21.15.7:0/1463960147 conn(0x55f9d8a84310 msgr2=0x55f9d8a83530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:35.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8ceffff640 0 client.19392 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:35.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.860+0000 7f8d19c11740 1 -- 172.21.15.7:0/1463960147 shutdown_connections 2024-03-26T08:57:35.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.861+0000 7f8d19c11740 1 -- 172.21.15.7:0/1463960147 wait complete. 2024-03-26T08:57:35.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.861+0000 7f8d19c11740 20 client.19392 trim_cache size 0 max 16384 2024-03-26T08:57:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.892+0000 7f8d19c11740 1 Processor -- start 2024-03-26T08:57:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d19c11740 1 -- start start 2024-03-26T08:57:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 0x55f9d8a937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a9a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d19c11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101b9e60 con 0x55f9d8a8e900 2024-03-26T08:57:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d19c11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101b7960 con 0x55f9d8a89790 2024-03-26T08:57:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d19c11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101b59e0 con 0x55f9d8a88da0 2024-03-26T08:57:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d14cf5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d14cf5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:43010/0 (socket says 172.21.15.7:43010) 2024-03-26T08:57:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d14cf5640 1 -- 172.21.15.7:0/4291896361 learned_addr learned my addr 172.21.15.7:0/4291896361 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d0ffff640 1 --2- 172.21.15.7:0/4291896361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 0x55f9d8a937b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d154f6640 1 --2- 172.21.15.7:0/4291896361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a9a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d14cf5640 1 -- 172.21.15.7:0/4291896361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 msgr2=0x55f9d8a937b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.893+0000 7f8d14cf5640 1 --2- 172.21.15.7:0/4291896361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 0x55f9d8a937b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.894+0000 7f8d14cf5640 1 -- 172.21.15.7:0/4291896361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 msgr2=0x55f9d8a9a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.894+0000 7f8d14cf5640 1 --2- 172.21.15.7:0/4291896361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a9a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.894+0000 7f8d14cf5640 1 -- 172.21.15.7:0/4291896361 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d8a8df20 con 0x55f9d8a88da0 2024-03-26T08:57:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.894+0000 7f8d14cf5640 1 --2- 172.21.15.7:0/4291896361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009850 tx=0x7f8d04005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b91310de4140ea1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.894+0000 7f8cf67fc640 1 -- 172.21.15.7:0/4291896361 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04004020 con 0x55f9d8a88da0 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.894+0000 7f8cf67fc640 1 -- 172.21.15.7:0/4291896361 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d040041c0 con 0x55f9d8a88da0 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8cf67fc640 1 -- 172.21.15.7:0/4291896361 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d040127d0 con 0x55f9d8a88da0 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8d19c11740 1 -- 172.21.15.7:0/4291896361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 msgr2=0x55f9d8a93270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8d19c11740 1 --2- 172.21.15.7:0/4291896361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009850 tx=0x7f8d04005f90 comp rx=0 tx=0).stop 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8d19c11740 1 -- 172.21.15.7:0/4291896361 shutdown_connections 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8d19c11740 1 --2- 172.21.15.7:0/4291896361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a8e900 0x55f9d8a9a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8d19c11740 1 --2- 172.21.15.7:0/4291896361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a89790 0x55f9d8a937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8d19c11740 1 --2- 172.21.15.7:0/4291896361 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a93270 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8d19c11740 1 -- 172.21.15.7:0/4291896361 >> 172.21.15.7:0/4291896361 conn(0x55f9d8a84310 msgr2=0x55f9d8a846f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8d19c11740 1 -- 172.21.15.7:0/4291896361 shutdown_connections 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.895+0000 7f8d19c11740 1 -- 172.21.15.7:0/4291896361 wait complete. 2024-03-26T08:57:35.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.896+0000 7f8d19c11740 1 -- start start 2024-03-26T08:57:35.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.896+0000 7f8d19c11740 1 Processor -- start 2024-03-26T08:57:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.896+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a97ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.896+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a89790 0x55f9d8a98000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.896+0000 7f8d19c11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a8e900 0x55f9d8a99010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.896+0000 7f8d19c11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101b5fe0 con 0x55f9d8a89790 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.896+0000 7f8d19c11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101b7300 con 0x55f9d8a8e900 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.896+0000 7f8d19c11740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d101b6640 con 0x55f9d8a88da0 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.897+0000 7f8d19c11740 10 client.0 _ll_register_callbacks cb 0x7f8d08097ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.897+0000 7f8d154f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a8e900 0x55f9d8a99010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.897+0000 7f8d14cf5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a97ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.897+0000 7f8d154f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a8e900 0x55f9d8a99010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55400/0 (socket says 172.21.15.7:55400) 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.897+0000 7f8d154f6640 1 -- 172.21.15.7:0/1956379992 learned_addr learned my addr 172.21.15.7:0/1956379992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.898+0000 7f8d0ffff640 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a89790 0x55f9d8a98000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.898+0000 7f8d154f6640 1 -- 172.21.15.7:0/1956379992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 msgr2=0x55f9d8a97ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.898+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a97ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.898+0000 7f8d154f6640 1 -- 172.21.15.7:0/1956379992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a89790 msgr2=0x55f9d8a98000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.898+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a89790 0x55f9d8a98000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.898+0000 7f8d154f6640 1 -- 172.21.15.7:0/1956379992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9d8a99600 con 0x55f9d8a8e900 2024-03-26T08:57:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.899+0000 7f8d154f6640 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a8e900 0x55f9d8a99010 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8d080a3e00 tx=0x7f8d08098200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d03a454602675fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.899+0000 7f8ceffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:57:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.899+0000 7f8ceffff640 1 -- 172.21.15.7:0/1956379992 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d080a4330 con 0x55f9d8a8e900 2024-03-26T08:57:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.899+0000 7f8d19c11740 1 -- 172.21.15.7:0/1956379992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d040041c0 con 0x55f9d8a8e900 2024-03-26T08:57:35.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.899+0000 7f8ceffff640 1 -- 172.21.15.7:0/1956379992 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8d080a0c00 con 0x55f9d8a8e900 2024-03-26T08:57:35.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.900+0000 7f8ceffff640 1 -- 172.21.15.7:0/1956379992 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d08086320 con 0x55f9d8a8e900 2024-03-26T08:57:35.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.900+0000 7f8d19c11740 10 client.29165 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:35.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.900+0000 7f8d19c11740 1 -- 172.21.15.7:0/1956379992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f9d8a99600 con 0x55f9d8a8e900 2024-03-26T08:57:35.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.901+0000 7f8ceffff640 1 -- 172.21.15.7:0/1956379992 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d040041c0 con 0x55f9d8a8e900 2024-03-26T08:57:35.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.901+0000 7f8ceffff640 1 -- 172.21.15.7:0/1956379992 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f8d0809aa50 con 0x55f9d8a8e900 2024-03-26T08:57:35.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.901+0000 7f8d19c11740 10 client.29165 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:35.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.902+0000 7f8d19c11740 10 client.29165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:35.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.902+0000 7f8d19c11740 20 client.29165 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:35.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.902+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f9d8afa7f0 0x55f9d8afcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:35.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.902+0000 7f8d19c11740 4 client.29165 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:57:35.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.902+0000 7f8d19c11740 1 -- 172.21.15.7:0/1956379992 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f9d89f5c60 con 0x55f9d8afa7f0 2024-03-26T08:57:35.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.902+0000 7f8d14cf5640 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f9d8afa7f0 0x55f9d8afcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:35.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.903+0000 7f8d14cf5640 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f9d8afa7f0 0x55f9d8afcbe0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c6c9e69ccd278 server_cookie=3eabd02f89338176 in_seq=0 out_seq=0 2024-03-26T08:57:35.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.904+0000 7f8ceffff640 10 client.29165 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:35.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.904+0000 7f8ceffff640 1 -- 172.21.15.7:0/1956379992 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f9d89f5c60 con 0x55f9d8afa7f0 2024-03-26T08:57:35.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.904+0000 7f8ceffff640 10 client.29165 handle_command_reply: tid=0 2024-03-26T08:57:35.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.904+0000 7f8d19c11740 1 client.29165 shutdown 2024-03-26T08:57:35.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.904+0000 7f8d19c11740 1 -- 172.21.15.7:0/1956379992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a8e900 msgr2=0x55f9d8a99010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:35.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.904+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a8e900 0x55f9d8a99010 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8d080a3e00 tx=0x7f8d08098200 comp rx=0 tx=0).stop 2024-03-26T08:57:35.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.905+0000 7f8d19c11740 1 -- 172.21.15.7:0/1956379992 shutdown_connections 2024-03-26T08:57:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.905+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55f9d8afa7f0 0x55f9d8afcbe0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.905+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f9d8a8e900 0x55f9d8a99010 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.905+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f9d8a89790 0x55f9d8a98000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.905+0000 7f8d19c11740 1 --2- 172.21.15.7:0/1956379992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55f9d8a88da0 0x55f9d8a97ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.905+0000 7f8d19c11740 1 -- 172.21.15.7:0/1956379992 >> 172.21.15.7:0/1956379992 conn(0x55f9d8a84310 msgr2=0x55f9d8a93270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:35.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.905+0000 7f8ceffff640 0 client.29165 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.906+0000 7f8d19c11740 1 -- 172.21.15.7:0/1956379992 shutdown_connections 2024-03-26T08:57:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.906+0000 7f8d19c11740 1 -- 172.21.15.7:0/1956379992 wait complete. 2024-03-26T08:57:35.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.907+0000 7f8d19c11740 20 client.29165 trim_cache size 0 max 16384 2024-03-26T08:57:35.905 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:57:35.905 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:57:35.905 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:57:35.905 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:57:35.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 -- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cf80a2900 msgr2=0x7f8cf80a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:35.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cf80a2900 0x7f8cf80a4dc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8d0002c240 tx=0x7f8d0002c980 comp rx=0 tx=0).stop 2024-03-26T08:57:35.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 -- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d10103260 msgr2=0x7f8d1019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:35.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d10103260 0x7f8d1019a410 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800eb10 tx=0x7f8d0800c2d0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 -- 172.21.15.7:0/1012860111 shutdown_connections 2024-03-26T08:57:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8cf80a2900 0x7f8cf80a4dc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d10103260 0x7f8d1019a410 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8d101028a0 0x7f8d101933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 --2- 172.21.15.7:0/1012860111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d101016b0 0x7f8d10192e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 -- 172.21.15.7:0/1012860111 >> 172.21.15.7:0/1012860111 conn(0x7f8d100fcc20 msgr2=0x7f8d10103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 -- 172.21.15.7:0/1012860111 shutdown_connections 2024-03-26T08:57:35.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:35.908+0000 7f8d16f59640 1 -- 172.21.15.7:0/1012860111 wait complete. 2024-03-26T08:57:35.924 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:57:35.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:57:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.071+0000 7fac1dbf3640 1 -- 172.21.15.7:0/3874453673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 msgr2=0x7fac18101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.071+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/3874453673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 0x7fac18101f80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fac08009900 tx=0x7fac0802bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.071+0000 7fac1dbf3640 1 -- 172.21.15.7:0/3874453673 shutdown_connections 2024-03-26T08:57:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.072+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/3874453673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac18101250 0x7fac18108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.072+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/3874453673 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac18100890 0x7fac18100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.072+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/3874453673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 0x7fac18101f80 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.072+0000 7fac1dbf3640 1 -- 172.21.15.7:0/3874453673 >> 172.21.15.7:0/3874453673 conn(0x7fac180fb2c0 msgr2=0x7fac180fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.072+0000 7fac1dbf3640 1 -- 172.21.15.7:0/3874453673 shutdown_connections 2024-03-26T08:57:36.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.072+0000 7fac1dbf3640 1 -- 172.21.15.7:0/3874453673 wait complete. 2024-03-26T08:57:36.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.073+0000 7fac1dbf3640 1 Processor -- start 2024-03-26T08:57:36.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.073+0000 7fac1dbf3640 1 -- start start 2024-03-26T08:57:36.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac1dbf3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac18100890 0x7fac1819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac1dbf3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac18101250 0x7fac1819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac1dbf3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 0x7fac181a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac1dbf3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1810b1a0 con 0x7fac18101b60 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac1dbf3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1810b020 con 0x7fac18100890 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac1dbf3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1810b320 con 0x7fac18101250 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 0x7fac181a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac18100890 0x7fac1819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac17fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 0x7fac181a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51606/0 (socket says 172.21.15.7:51606) 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac17fff640 1 -- 172.21.15.7:0/774901632 learned_addr learned my addr 172.21.15.7:0/774901632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac16ffd640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac18101250 0x7fac1819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac17fff640 1 -- 172.21.15.7:0/774901632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac18101250 msgr2=0x7fac1819e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac17fff640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac18101250 0x7fac1819e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac17fff640 1 -- 172.21.15.7:0/774901632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac18100890 msgr2=0x7fac1819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac17fff640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac18100890 0x7fac1819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.074+0000 7fac17fff640 1 -- 172.21.15.7:0/774901632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac181a5b70 con 0x7fac18101b60 2024-03-26T08:57:36.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.075+0000 7fac17fff640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 0x7fac181a5500 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fac1000cce0 tx=0x7fac1000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.075+0000 7fac14ff9640 1 -- 172.21.15.7:0/774901632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000f450 con 0x7fac18101b60 2024-03-26T08:57:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.075+0000 7fac14ff9640 1 -- 172.21.15.7:0/774901632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fac1000fad0 con 0x7fac18101b60 2024-03-26T08:57:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.075+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac181a5e60 con 0x7fac18101b60 2024-03-26T08:57:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.075+0000 7fac14ff9640 1 -- 172.21.15.7:0/774901632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10018c10 con 0x7fac18101b60 2024-03-26T08:57:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.075+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac181a62d0 con 0x7fac18101b60 2024-03-26T08:57:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.076+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac181a6590 con 0x7fac18101b60 2024-03-26T08:57:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.077+0000 7fac14ff9640 1 -- 172.21.15.7:0/774901632 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fac1000f5f0 con 0x7fac18101b60 2024-03-26T08:57:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.078+0000 7fac14ff9640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fac000a2850 0x7fac000a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.078+0000 7fac177fe640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fac000a2850 0x7fac000a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.078+0000 7fac14ff9640 1 -- 172.21.15.7:0/774901632 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fac100cf0a0 con 0x7fac18101b60 2024-03-26T08:57:36.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.079+0000 7fac177fe640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fac000a2850 0x7fac000a4d10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fac08009850 tx=0x7fac0802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.085+0000 7fac14ff9640 1 -- 172.21.15.7:0/774901632 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fac10014030 con 0x7fac18101b60 2024-03-26T08:57:36.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.415+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac181a6760 con 0x7fac18101b60 2024-03-26T08:57:36.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.416+0000 7fac14ff9640 1 -- 172.21.15.7:0/774901632 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7fac100afe20 con 0x7fac18101b60 2024-03-26T08:57:36.415 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:36.415 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:57:36.415 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:57:36.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.418+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fac000a2850 msgr2=0x7fac000a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.418+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fac000a2850 0x7fac000a4d10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fac08009850 tx=0x7fac0802df90 comp rx=0 tx=0).stop 2024-03-26T08:57:36.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 msgr2=0x7fac181a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 0x7fac181a5500 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fac1000cce0 tx=0x7fac1000a620 comp rx=0 tx=0).stop 2024-03-26T08:57:36.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 shutdown_connections 2024-03-26T08:57:36.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fac000a2850 0x7fac000a4d10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac18101b60 0x7fac181a5500 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fac18101250 0x7fac1819e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 --2- 172.21.15.7:0/774901632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac18100890 0x7fac1819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 >> 172.21.15.7:0/774901632 conn(0x7fac180fb2c0 msgr2=0x7fac1810dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 shutdown_connections 2024-03-26T08:57:36.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.419+0000 7fac1dbf3640 1 -- 172.21.15.7:0/774901632 wait complete. 2024-03-26T08:57:36.430 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:57:36.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:57:36.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:36 smithi007 ceph-mon[32462]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 669 op/s 2024-03-26T08:57:36.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.566+0000 7fdb3bea1640 1 -- 172.21.15.7:0/3918762356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34103310 msgr2=0x7fdb34109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.566+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/3918762356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34103310 0x7fdb34109ba0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009cd0 tx=0x7fdb2c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.567+0000 7fdb3bea1640 1 -- 172.21.15.7:0/3918762356 shutdown_connections 2024-03-26T08:57:36.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.567+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/3918762356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34103310 0x7fdb34109ba0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.567+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/3918762356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb34102950 0x7fdb34102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.567+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/3918762356 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb340ff980 0x7fdb340ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.567+0000 7fdb3bea1640 1 -- 172.21.15.7:0/3918762356 >> 172.21.15.7:0/3918762356 conn(0x7fdb340fb2b0 msgr2=0x7fdb340fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.568+0000 7fdb3bea1640 1 -- 172.21.15.7:0/3918762356 shutdown_connections 2024-03-26T08:57:36.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.568+0000 7fdb3bea1640 1 -- 172.21.15.7:0/3918762356 wait complete. 2024-03-26T08:57:36.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.568+0000 7fdb3bea1640 1 Processor -- start 2024-03-26T08:57:36.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.568+0000 7fdb3bea1640 1 -- start start 2024-03-26T08:57:36.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.568+0000 7fdb3bea1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ff980 0x7fdb34199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3bea1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb34102950 0x7fdb3419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3bea1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34103310 0x7fdb341a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3bea1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3410d350 con 0x7fdb34103310 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3bea1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3410d1d0 con 0x7fdb340ff980 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3bea1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3410d4d0 con 0x7fdb34102950 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3943c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb34102950 0x7fdb3419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3a43e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34103310 0x7fdb341a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3943c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb34102950 0x7fdb3419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53416/0 (socket says 172.21.15.7:53416) 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb39c3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ff980 0x7fdb34199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3943c640 1 -- 172.21.15.7:0/1910985551 learned_addr learned my addr 172.21.15.7:0/1910985551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb39c3d640 1 -- 172.21.15.7:0/1910985551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb34102950 msgr2=0x7fdb3419a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb39c3d640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb34102950 0x7fdb3419a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb39c3d640 1 -- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34103310 msgr2=0x7fdb341a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb39c3d640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34103310 0x7fdb341a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb39c3d640 1 -- 172.21.15.7:0/1910985551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb341a1830 con 0x7fdb340ff980 2024-03-26T08:57:36.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.569+0000 7fdb3a43e640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34103310 0x7fdb341a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:57:36.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.570+0000 7fdb39c3d640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ff980 0x7fdb34199c20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000e9c0 tx=0x7fdb3000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.570+0000 7fdb2affd640 1 -- 172.21.15.7:0/1910985551 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000ccc0 con 0x7fdb340ff980 2024-03-26T08:57:36.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.570+0000 7fdb3bea1640 1 -- 172.21.15.7:0/1910985551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb341a1b20 con 0x7fdb340ff980 2024-03-26T08:57:36.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.570+0000 7fdb2affd640 1 -- 172.21.15.7:0/1910985551 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb300035f0 con 0x7fdb340ff980 2024-03-26T08:57:36.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.570+0000 7fdb2affd640 1 -- 172.21.15.7:0/1910985551 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3000ce60 con 0x7fdb340ff980 2024-03-26T08:57:36.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.570+0000 7fdb3bea1640 1 -- 172.21.15.7:0/1910985551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb341a2010 con 0x7fdb340ff980 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.573+0000 7fdb2affd640 1 -- 172.21.15.7:0/1910985551 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fdb30003af0 con 0x7fdb340ff980 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.573+0000 7fdb3eb69740 1 Processor -- start 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.573+0000 7fdb3eb69740 1 -- start start 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x55689649fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 0x5568964a7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3eb69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556896462610 con 0x5568964a0080 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3eb69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568964053c0 con 0x55689649f690 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3eb69740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568963add60 con 0x5568964a51f0 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb2affd640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb200a2800 0x7fdb200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3a43e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x55689649fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3a43e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x55689649fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44246/0 (socket says 172.21.15.7:44246) 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3a43e640 1 -- 172.21.15.7:0/3533545233 learned_addr learned my addr 172.21.15.7:0/3533545233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb39c3d640 1 --2- 172.21.15.7:0/3533545233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb2affd640 1 -- 172.21.15.7:0/1910985551 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb300cef70 con 0x7fdb340ff980 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3a43e640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb200a2800 0x7fdb200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3943c640 1 --2- 172.21.15.7:0/3533545233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 0x5568964a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3a43e640 1 -- 172.21.15.7:0/3533545233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 msgr2=0x5568964a7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3a43e640 1 --2- 172.21.15.7:0/3533545233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 0x5568964a7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3a43e640 1 -- 172.21.15.7:0/3533545233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 msgr2=0x5568964a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3a43e640 1 --2- 172.21.15.7:0/3533545233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.574+0000 7fdb3a43e640 1 -- 172.21.15.7:0/3533545233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568961193f0 con 0x55689649f690 2024-03-26T08:57:36.575 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:36 smithi044 ceph-mon[36461]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 669 op/s 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.575+0000 7fdb3a43e640 1 --2- 172.21.15.7:0/3533545233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x55689649fab0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005e30 tx=0x7fdb2c032c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c5511cd18b567bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.576+0000 7fdb3a43e640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb200a2800 0x7fdb200a4cc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c004af0 tx=0x7fdb2c047000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.576+0000 7fdb1ffff640 1 -- 172.21.15.7:0/3533545233 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c047070 con 0x55689649f690 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.576+0000 7fdb1ffff640 1 -- 172.21.15.7:0/3533545233 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb2c034920 con 0x55689649f690 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.576+0000 7fdb1ffff640 1 -- 172.21.15.7:0/3533545233 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c039040 con 0x55689649f690 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3533545233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 msgr2=0x55689649fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/3533545233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x55689649fab0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005e30 tx=0x7fdb2c032c50 comp rx=0 tx=0).stop 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3533545233 shutdown_connections 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/3533545233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 0x5568964a7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/3533545233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/3533545233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x55689649fab0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3533545233 >> 172.21.15.7:0/3533545233 conn(0x55689649ac00 msgr2=0x55689649b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3533545233 shutdown_connections 2024-03-26T08:57:36.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3533545233 wait complete. 2024-03-26T08:57:36.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.577+0000 7fdb3eb69740 1 -- start start 2024-03-26T08:57:36.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.578+0000 7fdb3eb69740 1 Processor -- start 2024-03-26T08:57:36.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.578+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.578+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb39c3d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568964a51f0 0x5568964b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb39c3d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53434/0 (socket says 172.21.15.7:53434) 2024-03-26T08:57:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3eb69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568963c1240 con 0x5568964a0080 2024-03-26T08:57:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3eb69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556896401460 con 0x5568964a51f0 2024-03-26T08:57:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3eb69740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568964a81a0 con 0x55689649f690 2024-03-26T08:57:36.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb39c3d640 1 -- 172.21.15.7:0/139245521 learned_addr learned my addr 172.21.15.7:0/139245521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3eb69740 10 client.0 _ll_register_callbacks cb 0x5568963c0c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3943c640 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3943c640 1 -- 172.21.15.7:0/139245521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55689649f690 msgr2=0x5568964a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3943c640 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3943c640 1 -- 172.21.15.7:0/139245521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568964a51f0 msgr2=0x5568964b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3943c640 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568964a51f0 0x5568964b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.579+0000 7fdb3943c640 1 -- 172.21.15.7:0/139245521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568961e37d0 con 0x5568964a0080 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb3943c640 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964a9f50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fdb240126d0 tx=0x7fdb24012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73058fba9829b9a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/139245521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24004050 con 0x5568964a0080 2024-03-26T08:57:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/139245521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb240041f0 con 0x5568964a0080 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb3eb69740 1 -- 172.21.15.7:0/139245521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb2c034920 con 0x5568964a0080 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/139245521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb240043f0 con 0x5568964a0080 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb3eb69740 10 client.19416 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb3eb69740 1 -- 172.21.15.7:0/139245521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5568964bcc30 con 0x5568964a0080 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/139245521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb2c034920 con 0x5568964a0080 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.580+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/139245521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdb2400f970 con 0x5568964a0080 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.581+0000 7fdb3eb69740 10 client.19416 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.581+0000 7fdb3eb69740 10 client.19416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.581+0000 7fdb3eb69740 20 client.19416 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.581+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5568964be380 0x5568964c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.581+0000 7fdb3eb69740 4 client.19416 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.581+0000 7fdb3eb69740 1 -- 172.21.15.7:0/139245521 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556896412900 con 0x5568964be380 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.581+0000 7fdb39c3d640 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5568964be380 0x5568964c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.581+0000 7fdb39c3d640 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5568964be380 0x5568964c07c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=997042704c752ef4 server_cookie=c8e3f1bb3699ef9b in_seq=0 out_seq=0 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.581+0000 7fdb1d7fa640 10 client.19416 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.582+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/139245521 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556896412900 con 0x5568964be380 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.582+0000 7fdb1d7fa640 10 client.19416 handle_command_reply: tid=0 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.582+0000 7fdb3eb69740 1 client.19416 shutdown 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 -- 172.21.15.7:0/139245521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 msgr2=0x5568964a9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964a9f50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fdb240126d0 tx=0x7fdb24012ad0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 -- 172.21.15.7:0/139245521 shutdown_connections 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5568964be380 0x5568964c07c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb1d7fa640 0 client.19416 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:57:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568964a51f0 0x5568964b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964a9f50 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/139245521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 -- 172.21.15.7:0/139245521 >> 172.21.15.7:0/139245521 conn(0x55689649ac00 msgr2=0x556896499e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 -- 172.21.15.7:0/139245521 shutdown_connections 2024-03-26T08:57:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 1 -- 172.21.15.7:0/139245521 wait complete. 2024-03-26T08:57:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.583+0000 7fdb3eb69740 20 client.19416 trim_cache size 0 max 16384 2024-03-26T08:57:36.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3eb69740 1 Processor -- start 2024-03-26T08:57:36.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3eb69740 1 -- start start 2024-03-26T08:57:36.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a0080 0x5568964a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a51f0 0x5568964b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb39c3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb39c3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44252/0 (socket says 172.21.15.7:44252) 2024-03-26T08:57:36.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3943c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a0080 0x5568964a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3943c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a0080 0x5568964a9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53444/0 (socket says 172.21.15.7:53444) 2024-03-26T08:57:36.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3a43e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a51f0 0x5568964b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3a43e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a51f0 0x5568964b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56328/0 (socket says 172.21.15.7:56328) 2024-03-26T08:57:36.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb39c3d640 1 -- 172.21.15.7:0/3758704256 learned_addr learned my addr 172.21.15.7:0/3758704256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3758704256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341a8da0 con 0x5568964a51f0 2024-03-26T08:57:36.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3758704256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341a68a0 con 0x55689649f690 2024-03-26T08:57:36.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.602+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3758704256 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341a4960 con 0x5568964a0080 2024-03-26T08:57:36.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.603+0000 7fdb3943c640 1 -- 172.21.15.7:0/3758704256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 msgr2=0x5568964a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.603+0000 7fdb3943c640 1 --2- 172.21.15.7:0/3758704256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.603+0000 7fdb3943c640 1 -- 172.21.15.7:0/3758704256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a51f0 msgr2=0x5568964b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.603+0000 7fdb3943c640 1 --2- 172.21.15.7:0/3758704256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a51f0 0x5568964b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.603+0000 7fdb3943c640 1 -- 172.21.15.7:0/3758704256 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568964a82e0 con 0x5568964a0080 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.604+0000 7fdb3943c640 1 --2- 172.21.15.7:0/3758704256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a0080 0x5568964a9f50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400fe30 tx=0x7fdb24012380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c5c9df1d79a45c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.604+0000 7fdb1f7fe640 1 -- 172.21.15.7:0/3758704256 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2402c070 con 0x5568964a0080 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.604+0000 7fdb1f7fe640 1 -- 172.21.15.7:0/3758704256 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb2400fac0 con 0x5568964a0080 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.604+0000 7fdb1f7fe640 1 -- 172.21.15.7:0/3758704256 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24010970 con 0x5568964a0080 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.605+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3758704256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a0080 msgr2=0x5568964a9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.605+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/3758704256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a0080 0x5568964a9f50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400fe30 tx=0x7fdb24012380 comp rx=0 tx=0).stop 2024-03-26T08:57:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.606+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3758704256 shutdown_connections 2024-03-26T08:57:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.606+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/3758704256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a51f0 0x5568964b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.606+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/3758704256 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a0080 0x5568964a9f50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.606+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/3758704256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.606+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3758704256 >> 172.21.15.7:0/3758704256 conn(0x55689649ac00 msgr2=0x55689649afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.606+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3758704256 shutdown_connections 2024-03-26T08:57:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.606+0000 7fdb3eb69740 1 -- 172.21.15.7:0/3758704256 wait complete. 2024-03-26T08:57:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.607+0000 7fdb3eb69740 1 -- start start 2024-03-26T08:57:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.607+0000 7fdb3eb69740 1 Processor -- start 2024-03-26T08:57:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.607+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964ae890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.607+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.607+0000 7fdb3eb69740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 0x5568964afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.607+0000 7fdb3eb69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341a4f60 con 0x5568964a0080 2024-03-26T08:57:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.607+0000 7fdb3eb69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341a6240 con 0x55689649f690 2024-03-26T08:57:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.607+0000 7fdb3eb69740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341a5580 con 0x5568964a51f0 2024-03-26T08:57:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.607+0000 7fdb3eb69740 10 client.0 _ll_register_callbacks cb 0x7fdb24009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.608+0000 7fdb39c3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.608+0000 7fdb39c3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44266/0 (socket says 172.21.15.7:44266) 2024-03-26T08:57:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.608+0000 7fdb39c3d640 1 -- 172.21.15.7:0/1842633884 learned_addr learned my addr 172.21.15.7:0/1842633884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.608+0000 7fdb3943c640 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.608+0000 7fdb3a43e640 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 0x5568964afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.608+0000 7fdb39c3d640 1 -- 172.21.15.7:0/1842633884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 msgr2=0x5568964afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.608+0000 7fdb39c3d640 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 0x5568964afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.609+0000 7fdb39c3d640 1 -- 172.21.15.7:0/1842633884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 msgr2=0x5568964aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.609+0000 7fdb39c3d640 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.609+0000 7fdb39c3d640 1 -- 172.21.15.7:0/1842633884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568964b03d0 con 0x55689649f690 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.609+0000 7fdb39c3d640 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964ae890 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdb30097ce0 tx=0x7fdb3006a6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38eba0c707630c35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.609+0000 7fdb1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.609+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/1842633884 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb300b4060 con 0x55689649f690 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.609+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/1842633884 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb30097ef0 con 0x55689649f690 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.609+0000 7fdb3eb69740 1 -- 172.21.15.7:0/1842633884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb2400fac0 con 0x55689649f690 2024-03-26T08:57:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.610+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/1842633884 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb30065050 con 0x55689649f690 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.610+0000 7fdb3eb69740 10 client.29183 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.610+0000 7fdb3eb69740 1 -- 172.21.15.7:0/1842633884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556896519e10 con 0x55689649f690 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.610+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/1842633884 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb2400fac0 con 0x55689649f690 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.610+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/1842633884 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fdb300860c0 con 0x55689649f690 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.610+0000 7fdb3eb69740 10 client.29183 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.610+0000 7fdb3eb69740 10 client.29183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.610+0000 7fdb3eb69740 20 client.29183 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.611+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5568964be380 0x55689651c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.611+0000 7fdb3eb69740 4 client.29183 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:57:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.611+0000 7fdb3eb69740 1 -- 172.21.15.7:0/1842633884 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55689640cc50 con 0x5568964be380 2024-03-26T08:57:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.611+0000 7fdb3943c640 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5568964be380 0x55689651c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.611+0000 7fdb3943c640 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5568964be380 0x55689651c760 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13d3877879782852 server_cookie=e658232c2f7d6d21 in_seq=0 out_seq=0 2024-03-26T08:57:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.611+0000 7fdb1d7fa640 10 client.29183 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:57:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.612+0000 7fdb1d7fa640 1 -- 172.21.15.7:0/1842633884 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55689640cc50 con 0x5568964be380 2024-03-26T08:57:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.612+0000 7fdb1d7fa640 10 client.29183 handle_command_reply: tid=0 2024-03-26T08:57:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.612+0000 7fdb3eb69740 1 client.29183 shutdown 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 -- 172.21.15.7:0/1842633884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 msgr2=0x5568964ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964ae890 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdb30097ce0 tx=0x7fdb3006a6e0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 -- 172.21.15.7:0/1842633884 shutdown_connections 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x5568964be380 0x55689651c760 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5568964a51f0 0x5568964afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb1d7fa640 0 client.29183 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568964a0080 0x5568964aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 --2- 172.21.15.7:0/1842633884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55689649f690 0x5568964ae890 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 -- 172.21.15.7:0/1842633884 >> 172.21.15.7:0/1842633884 conn(0x55689649ac00 msgr2=0x5568964a9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 -- 172.21.15.7:0/1842633884 shutdown_connections 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 1 -- 172.21.15.7:0/1842633884 wait complete. 2024-03-26T08:57:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.613+0000 7fdb3eb69740 20 client.29183 trim_cache size 0 max 16384 2024-03-26T08:57:36.615 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:57:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.617+0000 7fdb3bea1640 1 -- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb200a2800 msgr2=0x7fdb200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.617+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb200a2800 0x7fdb200a4cc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c004af0 tx=0x7fdb2c047000 comp rx=0 tx=0).stop 2024-03-26T08:57:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.617+0000 7fdb3bea1640 1 -- 172.21.15.7:0/1910985551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ff980 msgr2=0x7fdb34199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.617+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ff980 0x7fdb34199c20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000e9c0 tx=0x7fdb3000ee80 comp rx=0 tx=0).stop 2024-03-26T08:57:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.618+0000 7fdb3bea1640 1 -- 172.21.15.7:0/1910985551 shutdown_connections 2024-03-26T08:57:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.618+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fdb200a2800 0x7fdb200a4cc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.618+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34103310 0x7fdb341a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.618+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdb34102950 0x7fdb3419a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.618+0000 7fdb3bea1640 1 --2- 172.21.15.7:0/1910985551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb340ff980 0x7fdb34199c20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.619+0000 7fdb3bea1640 1 -- 172.21.15.7:0/1910985551 >> 172.21.15.7:0/1910985551 conn(0x7fdb340fb2b0 msgr2=0x7fdb34100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.619+0000 7fdb3bea1640 1 -- 172.21.15.7:0/1910985551 shutdown_connections 2024-03-26T08:57:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.619+0000 7fdb3bea1640 1 -- 172.21.15.7:0/1910985551 wait complete. 2024-03-26T08:57:36.638 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:57:36.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:57:36.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:36 smithi016 ceph-mon[35851]: pgmap v815: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 669 op/s 2024-03-26T08:57:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.785+0000 7f4eea3cd640 1 -- 172.21.15.7:0/1756503362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4104a70 msgr2=0x7f4ee4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.785+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/1756503362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4104a70 0x7f4ee4106e60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009900 tx=0x7f4ed802bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.785+0000 7f4eea3cd640 1 -- 172.21.15.7:0/1756503362 shutdown_connections 2024-03-26T08:57:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.785+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/1756503362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee41073a0 0x7f4ee41097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.785+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/1756503362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4104a70 0x7f4ee4106e60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.785+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/1756503362 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee4102140 0x7f4ee4104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.785+0000 7f4eea3cd640 1 -- 172.21.15.7:0/1756503362 >> 172.21.15.7:0/1756503362 conn(0x7f4ee40fbb30 msgr2=0x7f4ee40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.785+0000 7f4eea3cd640 1 -- 172.21.15.7:0/1756503362 shutdown_connections 2024-03-26T08:57:36.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.785+0000 7f4eea3cd640 1 -- 172.21.15.7:0/1756503362 wait complete. 2024-03-26T08:57:36.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4eea3cd640 1 Processor -- start 2024-03-26T08:57:36.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4eea3cd640 1 -- start start 2024-03-26T08:57:36.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4eea3cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4102140 0x7f4ee4101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4eea3cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee4104a70 0x7f4ee41000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4eea3cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee41073a0 0x7f4ee4100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4eea3cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee410d350 con 0x7f4ee4102140 2024-03-26T08:57:36.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4eea3cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee410d1d0 con 0x7f4ee4104a70 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4eea3cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee410d4d0 con 0x7f4ee41073a0 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4102140 0x7f4ee4101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4102140 0x7f4ee4101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56358/0 (socket says 172.21.15.7:56358) 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4ee3fff640 1 -- 172.21.15.7:0/2677991902 learned_addr learned my addr 172.21.15.7:0/2677991902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4ee896a640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee41073a0 0x7f4ee4100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.786+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee4104a70 0x7f4ee41000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4ee3fff640 1 -- 172.21.15.7:0/2677991902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee41073a0 msgr2=0x7f4ee4100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee41073a0 0x7f4ee4100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4ee3fff640 1 -- 172.21.15.7:0/2677991902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee4104a70 msgr2=0x7f4ee41000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee4104a70 0x7f4ee41000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4ee3fff640 1 -- 172.21.15.7:0/2677991902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4100ec0 con 0x7f4ee4102140 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4102140 0x7f4ee4101ab0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400e9f0 tx=0x7f4ed400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4ee17fa640 1 -- 172.21.15.7:0/2677991902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400cd80 con 0x7f4ee4102140 2024-03-26T08:57:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4ee17fa640 1 -- 172.21.15.7:0/2677991902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ed4005500 con 0x7f4ee4102140 2024-03-26T08:57:36.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4eea3cd640 1 -- 172.21.15.7:0/2677991902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee41a1b50 con 0x7f4ee4102140 2024-03-26T08:57:36.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4ee17fa640 1 -- 172.21.15.7:0/2677991902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4010690 con 0x7f4ee4102140 2024-03-26T08:57:36.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.787+0000 7f4eea3cd640 1 -- 172.21.15.7:0/2677991902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee41a2070 con 0x7f4ee4102140 2024-03-26T08:57:36.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4ee17fa640 1 -- 172.21.15.7:0/2677991902 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4ed40046c0 con 0x7f4ee4102140 2024-03-26T08:57:36.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4eecfb0740 1 Processor -- start 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4eecfb0740 1 -- start start 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec132ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec133080 0x5598ec137b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 0x5598ec13a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec0f5610 con 0x5598ec132690 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec0983c0 con 0x5598ec133080 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec040d60 con 0x5598ec1381f0 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4ee17fa640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4ecc0a2900 0x7f4ecc0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.791+0000 7f4ee896a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec132ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee896a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec132ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56368/0 (socket says 172.21.15.7:56368) 2024-03-26T08:57:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee896a640 1 -- 172.21.15.7:0/952043363 learned_addr learned my addr 172.21.15.7:0/952043363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 0x5598ec13a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee17fa640 1 -- 172.21.15.7:0/2677991902 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4ed40cee50 con 0x7f4ee4102140 2024-03-26T08:57:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee896a640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4ecc0a2900 0x7f4ecc0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec133080 0x5598ec137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee37fe640 1 -- 172.21.15.7:0/952043363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec133080 msgr2=0x5598ec137b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec133080 0x5598ec137b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee37fe640 1 -- 172.21.15.7:0/952043363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 msgr2=0x5598ec132ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec132ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee37fe640 1 -- 172.21.15.7:0/952043363 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598ebdac3f0 con 0x5598ec1381f0 2024-03-26T08:57:36.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec133080 0x5598ec137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:36.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.792+0000 7f4ee896a640 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec132ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.794+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 0x5598ec13a680 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4ed802c170 tx=0x7f4ed802c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2bfb44904c6787 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.794+0000 7f4ee896a640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4ecc0a2900 0x7f4ecc0a4dc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5598ec1338e0 tx=0x7f4edc004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.794+0000 7f4eca7fc640 1 -- 172.21.15.7:0/952043363 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed802cb80 con 0x5598ec1381f0 2024-03-26T08:57:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.794+0000 7f4eca7fc640 1 -- 172.21.15.7:0/952043363 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ed8005040 con 0x5598ec1381f0 2024-03-26T08:57:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.795+0000 7f4eecfb0740 1 -- 172.21.15.7:0/952043363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 msgr2=0x5598ec13a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.795+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 0x5598ec13a680 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4ed802c170 tx=0x7f4ed802c660 comp rx=0 tx=0).stop 2024-03-26T08:57:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.796+0000 7f4eecfb0740 1 -- 172.21.15.7:0/952043363 shutdown_connections 2024-03-26T08:57:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.796+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 0x5598ec13a680 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.796+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec133080 0x5598ec137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.796+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/952043363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec132ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.796+0000 7f4eecfb0740 1 -- 172.21.15.7:0/952043363 >> 172.21.15.7:0/952043363 conn(0x5598ec12dc00 msgr2=0x5598ec12e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.796+0000 7f4eecfb0740 1 -- 172.21.15.7:0/952043363 shutdown_connections 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.796+0000 7f4eecfb0740 1 -- 172.21.15.7:0/952043363 wait complete. 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.796+0000 7f4eecfb0740 1 -- start start 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4eecfb0740 1 Processor -- start 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec132690 0x5598ec1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec133080 0x5598ec156730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 0x5598ec15d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec054240 con 0x5598ec133080 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec094460 con 0x5598ec132690 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598ec14c1a0 con 0x5598ec1381f0 2024-03-26T08:57:36.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4eecfb0740 10 client.0 _ll_register_callbacks cb 0x5598ec053c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec132690 0x5598ec1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec132690 0x5598ec1561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44286/0 (socket says 172.21.15.7:44286) 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4ee3fff640 1 -- 172.21.15.7:0/1046264247 learned_addr learned my addr 172.21.15.7:0/1046264247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec133080 0x5598ec156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.797+0000 7f4ee896a640 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 0x5598ec15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.798+0000 7f4ee3fff640 1 -- 172.21.15.7:0/1046264247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 msgr2=0x5598ec15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.798+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 0x5598ec15d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.798+0000 7f4ee3fff640 1 -- 172.21.15.7:0/1046264247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec133080 msgr2=0x5598ec156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.798+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec133080 0x5598ec156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.798+0000 7f4ee3fff640 1 -- 172.21.15.7:0/1046264247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598ebe767d0 con 0x5598ec132690 2024-03-26T08:57:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.798+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec132690 0x5598ec1561f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4098180 tx=0x7f4ed409b370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbbeb603fc480aa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.799+0000 7f4ec37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:57:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.799+0000 7f4ec37fe640 1 -- 172.21.15.7:0/1046264247 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed409b7b0 con 0x5598ec132690 2024-03-26T08:57:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.800+0000 7f4ec37fe640 1 -- 172.21.15.7:0/1046264247 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4ed4086070 con 0x5598ec132690 2024-03-26T08:57:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.800+0000 7f4eecfb0740 1 -- 172.21.15.7:0/1046264247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ed8005040 con 0x5598ec132690 2024-03-26T08:57:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.800+0000 7f4ec37fe640 1 -- 172.21.15.7:0/1046264247 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed409f900 con 0x5598ec132690 2024-03-26T08:57:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.800+0000 7f4eecfb0740 10 client.29195 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.800+0000 7f4eecfb0740 1 -- 172.21.15.7:0/1046264247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5598ec160b50 con 0x5598ec132690 2024-03-26T08:57:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.800+0000 7f4ec37fe640 1 -- 172.21.15.7:0/1046264247 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ed8005040 con 0x5598ec132690 2024-03-26T08:57:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4ec37fe640 1 -- 172.21.15.7:0/1046264247 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4ed409ac00 con 0x5598ec132690 2024-03-26T08:57:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4eecfb0740 10 client.29195 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4eecfb0740 10 client.29195 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:57:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4eecfb0740 20 client.29195 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5598ec162180 0x5598ec1645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4eecfb0740 4 client.29195 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4eecfb0740 1 -- 172.21.15.7:0/1046264247 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598ec0a5900 con 0x5598ec162180 2024-03-26T08:57:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5598ec162180 0x5598ec1645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5598ec162180 0x5598ec1645b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7de83d153c65721c server_cookie=51de4aefea9bd3f3 in_seq=0 out_seq=0 2024-03-26T08:57:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.801+0000 7f4ec37fe640 10 client.29195 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:57:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.802+0000 7f4ec37fe640 1 -- 172.21.15.7:0/1046264247 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5598ec0a5900 con 0x5598ec162180 2024-03-26T08:57:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.802+0000 7f4ec37fe640 10 client.29195 handle_command_reply: tid=0 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 client.29195 shutdown 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 -- 172.21.15.7:0/1046264247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec132690 msgr2=0x5598ec1561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec132690 0x5598ec1561f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4098180 tx=0x7f4ed409b370 comp rx=0 tx=0).stop 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 -- 172.21.15.7:0/1046264247 shutdown_connections 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5598ec162180 0x5598ec1645b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec1381f0 0x5598ec15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec133080 0x5598ec156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/1046264247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec132690 0x5598ec1561f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 -- 172.21.15.7:0/1046264247 >> 172.21.15.7:0/1046264247 conn(0x5598ec12dc00 msgr2=0x5598ec12ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4ec37fe640 0 client.29195 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:57:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 -- 172.21.15.7:0/1046264247 shutdown_connections 2024-03-26T08:57:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.803+0000 7f4eecfb0740 1 -- 172.21.15.7:0/1046264247 wait complete. 2024-03-26T08:57:36.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.804+0000 7f4eecfb0740 20 client.29195 trim_cache size 0 max 16384 2024-03-26T08:57:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.824+0000 7f4eecfb0740 1 Processor -- start 2024-03-26T08:57:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.824+0000 7f4eecfb0740 1 -- start start 2024-03-26T08:57:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec1344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 0x5598ec134a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec15d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee4071c00 con 0x5598ec132690 2024-03-26T08:57:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee406f700 con 0x5598ec1381f0 2024-03-26T08:57:36.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee406d780 con 0x5598ec133080 2024-03-26T08:57:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec1344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee896a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee896a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec15d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44288/0 (socket says 172.21.15.7:44288) 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee896a640 1 -- 172.21.15.7:0/4127349527 learned_addr learned my addr 172.21.15.7:0/4127349527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec1344e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56390/0 (socket says 172.21.15.7:56390) 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 0x5598ec134a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee896a640 1 -- 172.21.15.7:0/4127349527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 msgr2=0x5598ec134a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee896a640 1 --2- 172.21.15.7:0/4127349527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 0x5598ec134a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee896a640 1 -- 172.21.15.7:0/4127349527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 msgr2=0x5598ec1344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee896a640 1 --2- 172.21.15.7:0/4127349527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec1344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.825+0000 7f4ee896a640 1 -- 172.21.15.7:0/4127349527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598ec14c320 con 0x5598ec1381f0 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.826+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/4127349527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 0x5598ec134a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.826+0000 7f4ee896a640 1 --2- 172.21.15.7:0/4127349527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec15d730 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4edc014010 tx=0x7f4edc009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcdd2eda86eccdb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.826+0000 7f4ec9ffb640 1 -- 172.21.15.7:0/4127349527 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc013180 con 0x5598ec1381f0 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.826+0000 7f4ec9ffb640 1 -- 172.21.15.7:0/4127349527 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4edc00fcf0 con 0x5598ec1381f0 2024-03-26T08:57:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.826+0000 7f4ec9ffb640 1 -- 172.21.15.7:0/4127349527 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc022dc0 con 0x5598ec1381f0 2024-03-26T08:57:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.827+0000 7f4eecfb0740 1 -- 172.21.15.7:0/4127349527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 msgr2=0x5598ec15d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.827+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/4127349527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec15d730 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4edc014010 tx=0x7f4edc009b30 comp rx=0 tx=0).stop 2024-03-26T08:57:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.827+0000 7f4eecfb0740 1 -- 172.21.15.7:0/4127349527 shutdown_connections 2024-03-26T08:57:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.827+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/4127349527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec15d730 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.827+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/4127349527 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 0x5598ec134a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.827+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/4127349527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec1344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.827+0000 7f4eecfb0740 1 -- 172.21.15.7:0/4127349527 >> 172.21.15.7:0/4127349527 conn(0x5598ec12dc00 msgr2=0x5598ec154e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.828+0000 7f4eecfb0740 1 -- 172.21.15.7:0/4127349527 shutdown_connections 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.828+0000 7f4eecfb0740 1 -- 172.21.15.7:0/4127349527 wait complete. 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.828+0000 7f4eecfb0740 1 -- start start 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.828+0000 7f4eecfb0740 1 Processor -- start 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.828+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.828+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 0x5598ec15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4eecfb0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec1bba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee406dd80 con 0x5598ec132690 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee406f0a0 con 0x5598ec1381f0 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4eecfb0740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee406e3e0 con 0x5598ec133080 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4eecfb0740 10 client.0 _ll_register_callbacks cb 0x7f4ed409b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec15a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56402/0 (socket says 172.21.15.7:56402) 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee3fff640 1 -- 172.21.15.7:0/2876816328 learned_addr learned my addr 172.21.15.7:0/2876816328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee896a640 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec1bba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee3fff640 1 -- 172.21.15.7:0/2876816328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 msgr2=0x5598ec15ac20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 0x5598ec15ac20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee3fff640 1 -- 172.21.15.7:0/2876816328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 msgr2=0x5598ec1bba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec1bba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee3fff640 1 -- 172.21.15.7:0/2876816328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598ec1bbff0 con 0x5598ec132690 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ee3fff640 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec15a6e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4ed409b0a0 tx=0x7f4ed4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ab43c20f2e380f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.829+0000 7f4ec37fe640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.830+0000 7f4ec37fe640 1 -- 172.21.15.7:0/2876816328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed408c040 con 0x5598ec132690 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.830+0000 7f4ec37fe640 1 -- 172.21.15.7:0/2876816328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4ed40a3ac0 con 0x5598ec132690 2024-03-26T08:57:36.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.830+0000 7f4eecfb0740 1 -- 172.21.15.7:0/2876816328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4edc00fcf0 con 0x5598ec132690 2024-03-26T08:57:36.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.830+0000 7f4ec37fe640 1 -- 172.21.15.7:0/2876816328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed409ae00 con 0x5598ec132690 2024-03-26T08:57:36.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.830+0000 7f4eecfb0740 10 client.19440 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:36.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.830+0000 7f4eecfb0740 1 -- 172.21.15.7:0/2876816328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5598ec1bd360 con 0x5598ec132690 2024-03-26T08:57:36.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.831+0000 7f4ec37fe640 1 -- 172.21.15.7:0/2876816328 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4edc00fcf0 con 0x5598ec132690 2024-03-26T08:57:36.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.831+0000 7f4ec37fe640 1 -- 172.21.15.7:0/2876816328 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f4ed40afb90 con 0x5598ec132690 2024-03-26T08:57:36.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.831+0000 7f4eecfb0740 10 client.19440 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.831+0000 7f4eecfb0740 10 client.19440 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.831+0000 7f4eecfb0740 20 client.19440 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.831+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5598ec1bda00 0x5598ec1bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.831+0000 7f4eecfb0740 4 client.19440 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.831+0000 7f4eecfb0740 1 -- 172.21.15.7:0/2876816328 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5598ec09fc50 con 0x5598ec1bda00 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.832+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5598ec1bda00 0x5598ec1bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.832+0000 7f4ee37fe640 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5598ec1bda00 0x5598ec1bfe20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b97b633fc179493 server_cookie=10a251b3debb0cbf in_seq=0 out_seq=0 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.832+0000 7f4ec37fe640 10 client.19440 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.832+0000 7f4ec37fe640 1 -- 172.21.15.7:0/2876816328 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5598ec09fc50 con 0x5598ec1bda00 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.832+0000 7f4ec37fe640 10 client.19440 handle_command_reply: tid=0 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.832+0000 7f4eecfb0740 1 client.19440 shutdown 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.833+0000 7f4eecfb0740 1 -- 172.21.15.7:0/2876816328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 msgr2=0x5598ec15a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.833+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec15a6e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4ed409b0a0 tx=0x7f4ed4009310 comp rx=0 tx=0).stop 2024-03-26T08:57:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.833+0000 7f4eecfb0740 1 -- 172.21.15.7:0/2876816328 shutdown_connections 2024-03-26T08:57:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.833+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5598ec1381f0 0x5598ec1bba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.833+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x5598ec1bda00 0x5598ec1bfe20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.833+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5598ec133080 0x5598ec15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.833+0000 7f4eecfb0740 1 --2- 172.21.15.7:0/2876816328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598ec132690 0x5598ec15a6e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.833+0000 7f4eecfb0740 1 -- 172.21.15.7:0/2876816328 >> 172.21.15.7:0/2876816328 conn(0x5598ec12dc00 msgr2=0x5598ec1344e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.833+0000 7f4ec37fe640 0 client.19440 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:57:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.834+0000 7f4eecfb0740 1 -- 172.21.15.7:0/2876816328 shutdown_connections 2024-03-26T08:57:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.834+0000 7f4eecfb0740 1 -- 172.21.15.7:0/2876816328 wait complete. 2024-03-26T08:57:36.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.835+0000 7f4eecfb0740 20 client.19440 trim_cache size 0 max 16384 2024-03-26T08:57:36.835 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:57:36.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.835+0000 7f4eea3cd640 1 -- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4ecc0a2900 msgr2=0x7f4ecc0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.835+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4ecc0a2900 0x7f4ecc0a4dc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5598ec1338e0 tx=0x7f4edc004ab0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.835+0000 7f4eea3cd640 1 -- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4102140 msgr2=0x7f4ee4101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.835+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4102140 0x7f4ee4101ab0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400e9f0 tx=0x7f4ed400edf0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.836+0000 7f4eea3cd640 1 -- 172.21.15.7:0/2677991902 shutdown_connections 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.836+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4ecc0a2900 0x7f4ecc0a4dc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.836+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4ee41073a0 0x7f4ee4100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.836+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee4104a70 0x7f4ee41000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.836+0000 7f4eea3cd640 1 --2- 172.21.15.7:0/2677991902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ee4102140 0x7f4ee4101ab0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.836+0000 7f4eea3cd640 1 -- 172.21.15.7:0/2677991902 >> 172.21.15.7:0/2677991902 conn(0x7f4ee40fbb30 msgr2=0x7f4ee4103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.836+0000 7f4eea3cd640 1 -- 172.21.15.7:0/2677991902 shutdown_connections 2024-03-26T08:57:36.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:36.836+0000 7f4eea3cd640 1 -- 172.21.15.7:0/2677991902 wait complete. 2024-03-26T08:57:36.848 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:57:36.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:57:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.002+0000 7fe6097cd640 1 -- 172.21.15.7:0/584857400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 msgr2=0x7fe604100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.002+0000 7fe6097cd640 1 --2- 172.21.15.7:0/584857400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 0x7fe604100170 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400a910 tx=0x7fe5f402be40 comp rx=0 tx=0).stop 2024-03-26T08:57:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.002+0000 7fe6097cd640 1 -- 172.21.15.7:0/584857400 shutdown_connections 2024-03-26T08:57:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.002+0000 7fe6097cd640 1 --2- 172.21.15.7:0/584857400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604101900 0x7fe604108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.002+0000 7fe6097cd640 1 --2- 172.21.15.7:0/584857400 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe604100f40 0x7fe6041013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.002+0000 7fe6097cd640 1 --2- 172.21.15.7:0/584857400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 0x7fe604100170 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.002+0000 7fe6097cd640 1 -- 172.21.15.7:0/584857400 >> 172.21.15.7:0/584857400 conn(0x7fe6040fb2c0 msgr2=0x7fe6040fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.002+0000 7fe6097cd640 1 -- 172.21.15.7:0/584857400 shutdown_connections 2024-03-26T08:57:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.002+0000 7fe6097cd640 1 -- 172.21.15.7:0/584857400 wait complete. 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe6097cd640 1 Processor -- start 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe6097cd640 1 -- start start 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe6097cd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 0x7fe6041999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe6097cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604100f40 0x7fe604199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe6097cd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe604101900 0x7fe6041a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe6097cd640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60410b170 con 0x7fe6040ffd50 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe6097cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60410aff0 con 0x7fe604100f40 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe6097cd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60410b2f0 con 0x7fe604101900 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 0x7fe6041999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 0x7fe6041999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56408/0 (socket says 172.21.15.7:56408) 2024-03-26T08:57:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe602ffd640 1 -- 172.21.15.7:0/1319044234 learned_addr learned my addr 172.21.15.7:0/1319044234 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe6037fe640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe604101900 0x7fe6041a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.003+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604100f40 0x7fe604199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe602ffd640 1 -- 172.21.15.7:0/1319044234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe604101900 msgr2=0x7fe6041a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe602ffd640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe604101900 0x7fe6041a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe602ffd640 1 -- 172.21.15.7:0/1319044234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604100f40 msgr2=0x7fe604199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe602ffd640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604100f40 0x7fe604199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe602ffd640 1 -- 172.21.15.7:0/1319044234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6041a1610 con 0x7fe6040ffd50 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe602ffd640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 0x7fe6041999f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4036040 tx=0x7fe5f402ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe600ff9640 1 -- 172.21.15.7:0/1319044234 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4042020 con 0x7fe6040ffd50 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe600ff9640 1 -- 172.21.15.7:0/1319044234 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5f40035f0 con 0x7fe6040ffd50 2024-03-26T08:57:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe600ff9640 1 -- 172.21.15.7:0/1319044234 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4005000 con 0x7fe6040ffd50 2024-03-26T08:57:37.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe6097cd640 1 -- 172.21.15.7:0/1319044234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6041a18a0 con 0x7fe6040ffd50 2024-03-26T08:57:37.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.004+0000 7fe6097cd640 1 -- 172.21.15.7:0/1319044234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6041a1d40 con 0x7fe6040ffd50 2024-03-26T08:57:37.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.007+0000 7fe600ff9640 1 -- 172.21.15.7:0/1319044234 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fe5f4045020 con 0x7fe6040ffd50 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe60c3b8740 1 Processor -- start 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe60c3b8740 1 -- start start 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x560413607ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x56041360cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x56041360f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe60c3b8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604135ca610 con 0x560413608080 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe60c3b8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56041356d3c0 con 0x560413607690 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe60c3b8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x560413515d60 con 0x56041360d1f0 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x56041360cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.008+0000 7fe600ff9640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5f00a28d0 0x7fe5f00a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x56041360cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56424/0 (socket says 172.21.15.7:56424) 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe602ffd640 1 -- 172.21.15.7:0/1069548183 learned_addr learned my addr 172.21.15.7:0/1069548183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/1069548183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x56041360f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe6037fe640 1 --2- 172.21.15.7:0/1069548183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x560413607ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe6037fe640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5f00a28d0 0x7fe5f00a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe602ffd640 1 -- 172.21.15.7:0/1069548183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 msgr2=0x56041360f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe602ffd640 1 --2- 172.21.15.7:0/1069548183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x56041360f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe602ffd640 1 -- 172.21.15.7:0/1069548183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 msgr2=0x560413607ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe602ffd640 1 --2- 172.21.15.7:0/1069548183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x560413607ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe602ffd640 1 -- 172.21.15.7:0/1069548183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604132813f0 con 0x560413608080 2024-03-26T08:57:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe600ff9640 1 -- 172.21.15.7:0/1319044234 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5f40f1160 con 0x7fe6040ffd50 2024-03-26T08:57:37.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe6037fe640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5f00a28d0 0x7fe5f00a4d90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc005c40 tx=0x7fe5fc009bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:37.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.009+0000 7fe602ffd640 1 --2- 172.21.15.7:0/1069548183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x56041360cb80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40bda90 tx=0x7fe5f404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05fcefa829fcc9c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.010+0000 7fe5f97fa640 1 -- 172.21.15.7:0/1069548183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40bbc40 con 0x560413608080 2024-03-26T08:57:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.010+0000 7fe5f97fa640 1 -- 172.21.15.7:0/1069548183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5f40be200 con 0x560413608080 2024-03-26T08:57:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.010+0000 7fe5f97fa640 1 -- 172.21.15.7:0/1069548183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f40c62a0 con 0x560413608080 2024-03-26T08:57:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 -- 172.21.15.7:0/1069548183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 msgr2=0x56041360cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/1069548183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x56041360cb80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40bda90 tx=0x7fe5f404e000 comp rx=0 tx=0).stop 2024-03-26T08:57:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 -- 172.21.15.7:0/1069548183 shutdown_connections 2024-03-26T08:57:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/1069548183 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x56041360f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/1069548183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x56041360cb80 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/1069548183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x560413607ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 -- 172.21.15.7:0/1069548183 >> 172.21.15.7:0/1069548183 conn(0x560413602c00 msgr2=0x560413603000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 -- 172.21.15.7:0/1069548183 shutdown_connections 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 -- 172.21.15.7:0/1069548183 wait complete. 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 -- start start 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.011+0000 7fe60c3b8740 1 Processor -- start 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.012+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.012+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x560413611be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.012+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x560413618c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.012+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.012+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44340/0 (socket says 172.21.15.7:44340) 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.012+0000 7fe602ffd640 1 -- 172.21.15.7:0/3306076396 learned_addr learned my addr 172.21.15.7:0/3306076396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.012+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x560413529240 con 0x560413608080 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.012+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x560413569460 con 0x560413607690 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.012+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604135ca610 con 0x56041360d1f0 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x560413611be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe6037fe640 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x560413618c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe60c3b8740 10 client.0 _ll_register_callbacks cb 0x560413528c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe5fbfff640 1 -- 172.21.15.7:0/3306076396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 msgr2=0x560413618c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x560413618c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe5fbfff640 1 -- 172.21.15.7:0/3306076396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 msgr2=0x5604136116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:37.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136116a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe5fbfff640 1 -- 172.21.15.7:0/3306076396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56041334b7d0 con 0x560413608080 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x560413611be0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b3c0 tx=0x7fe5ec0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c73569dbec1da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe5ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.013+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/3306076396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec017830 con 0x560413608080 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.014+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/3306076396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5ec0179d0 con 0x560413608080 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.014+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5f40be200 con 0x560413608080 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.014+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/3306076396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5ec026050 con 0x560413608080 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.014+0000 7fe60c3b8740 10 client.19452 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.014+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56041361c040 con 0x560413608080 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.014+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/3306076396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5f40be200 con 0x560413608080 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.014+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/3306076396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe5ec01fcb0 con 0x560413608080 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.015+0000 7fe60c3b8740 10 client.19452 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:37.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.015+0000 7fe60c3b8740 10 client.19452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.015+0000 7fe60c3b8740 20 client.19452 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.015+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56041361d6f0 0x56041361fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.015+0000 7fe60c3b8740 4 client.19452 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.015+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56041357a900 con 0x56041361d6f0 2024-03-26T08:57:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.015+0000 7fe602ffd640 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56041361d6f0 0x56041361fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.016+0000 7fe602ffd640 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56041361d6f0 0x56041361fb90 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceeb3fcd31a9bcfe server_cookie=71e62fefb71326ce in_seq=0 out_seq=0 2024-03-26T08:57:37.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.016+0000 7fe5ea7fc640 10 client.19452 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:37.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/3306076396 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56041357a900 con 0x56041361d6f0 2024-03-26T08:57:37.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe5ea7fc640 10 client.19452 handle_command_reply: tid=0 2024-03-26T08:57:37.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe60c3b8740 1 client.19452 shutdown 2024-03-26T08:57:37.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 msgr2=0x560413611be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:37.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x560413611be0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b3c0 tx=0x7fe5ec0039c0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 shutdown_connections 2024-03-26T08:57:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x56041361d6f0 0x56041361fb90 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe5ea7fc640 0 client.19452 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x560413618c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413608080 0x560413611be0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/3306076396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.017+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 >> 172.21.15.7:0/3306076396 conn(0x560413602c00 msgr2=0x560413601e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.018+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 shutdown_connections 2024-03-26T08:57:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.018+0000 7fe60c3b8740 1 -- 172.21.15.7:0/3306076396 wait complete. 2024-03-26T08:57:37.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.018+0000 7fe60c3b8740 20 client.19452 trim_cache size 0 max 16384 2024-03-26T08:57:37.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.033+0000 7fe60c3b8740 1 Processor -- start 2024-03-26T08:57:37.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.033+0000 7fe60c3b8740 1 -- start start 2024-03-26T08:57:37.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.033+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413607690 0x5604136095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.033+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413608080 0x560413609b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.033+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413607690 0x5604136095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.033+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413607690 0x5604136095c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56432/0 (socket says 172.21.15.7:56432) 2024-03-26T08:57:37.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.033+0000 7fe602ffd640 1 -- 172.21.15.7:0/764786976 learned_addr learned my addr 172.21.15.7:0/764786976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:37.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.033+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x560413618be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413608080 0x560413609b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe60c3b8740 1 -- 172.21.15.7:0/764786976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041acff0 con 0x560413607690 2024-03-26T08:57:37.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe60c3b8740 1 -- 172.21.15.7:0/764786976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041aaaf0 con 0x560413608080 2024-03-26T08:57:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe6037fe640 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x560413618be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe60c3b8740 1 -- 172.21.15.7:0/764786976 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041a8b70 con 0x56041360d1f0 2024-03-26T08:57:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe5fbfff640 1 -- 172.21.15.7:0/764786976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 msgr2=0x560413618be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x560413618be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe5fbfff640 1 -- 172.21.15.7:0/764786976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413607690 msgr2=0x5604136095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413607690 0x5604136095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.034+0000 7fe5fbfff640 1 -- 172.21.15.7:0/764786976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560413612540 con 0x560413608080 2024-03-26T08:57:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.035+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413608080 0x560413609b00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0164d0 tx=0x7fe5ec005860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d78d579ba55c2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.035+0000 7fe5f8ff9640 1 -- 172.21.15.7:0/764786976 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec016c70 con 0x560413608080 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.035+0000 7fe5f8ff9640 1 -- 172.21.15.7:0/764786976 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5ec02e030 con 0x560413608080 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.035+0000 7fe5f8ff9640 1 -- 172.21.15.7:0/764786976 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec020690 con 0x560413608080 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.036+0000 7fe60c3b8740 1 -- 172.21.15.7:0/764786976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413608080 msgr2=0x560413609b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.036+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413608080 0x560413609b00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0164d0 tx=0x7fe5ec005860 comp rx=0 tx=0).stop 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.036+0000 7fe60c3b8740 1 -- 172.21.15.7:0/764786976 shutdown_connections 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.036+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56041360d1f0 0x560413618be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.036+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413608080 0x560413609b00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.036+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/764786976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560413607690 0x5604136095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.036+0000 7fe60c3b8740 1 -- 172.21.15.7:0/764786976 >> 172.21.15.7:0/764786976 conn(0x560413602c00 msgr2=0x5604136101b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.036+0000 7fe60c3b8740 1 -- 172.21.15.7:0/764786976 shutdown_connections 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.036+0000 7fe60c3b8740 1 -- 172.21.15.7:0/764786976 wait complete. 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe60c3b8740 1 -- start start 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe60c3b8740 1 Processor -- start 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560413608080 0x5604136168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe60c3b8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56041360d1f0 0x5604136178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe60c3b8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041a9170 con 0x56041360d1f0 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe60c3b8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041aa490 con 0x560413607690 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe602ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136163a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44352/0 (socket says 172.21.15.7:44352) 2024-03-26T08:57:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe60c3b8740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041a97d0 con 0x560413608080 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe602ffd640 1 -- 172.21.15.7:0/2329278005 learned_addr learned my addr 172.21.15.7:0/2329278005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe6037fe640 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56041360d1f0 0x5604136178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe5fbfff640 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560413608080 0x5604136168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.037+0000 7fe60c3b8740 10 client.0 _ll_register_callbacks cb 0x7fe5ec00b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe6037fe640 1 -- 172.21.15.7:0/2329278005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560413608080 msgr2=0x5604136168e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe6037fe640 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560413608080 0x5604136168e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe6037fe640 1 -- 172.21.15.7:0/2329278005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 msgr2=0x5604136163a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe6037fe640 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136163a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe6037fe640 1 -- 172.21.15.7:0/2329278005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560413617e90 con 0x56041360d1f0 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe6037fe640 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56041360d1f0 0x5604136178a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009270 tx=0x7fe5fc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5089863aa772159e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe5ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/2329278005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc004b20 con 0x56041360d1f0 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/2329278005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe5fc004cc0 con 0x56041360d1f0 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.038+0000 7fe60c3b8740 1 -- 172.21.15.7:0/2329278005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5ec02e030 con 0x56041360d1f0 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/2329278005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5fc020660 con 0x56041360d1f0 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe60c3b8740 10 client.19458 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe60c3b8740 1 -- 172.21.15.7:0/2329278005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560413679060 con 0x56041360d1f0 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/2329278005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe5ec02e030 con 0x56041360d1f0 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/2329278005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fe5fc033020 con 0x56041360d1f0 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe60c3b8740 10 client.19458 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe60c3b8740 10 client.19458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe60c3b8740 20 client.19458 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604136799c0 0x56041367bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe60c3b8740 4 client.19458 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:57:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe60c3b8740 1 -- 172.21.15.7:0/2329278005 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560413574c50 con 0x5604136799c0 2024-03-26T08:57:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.039+0000 7fe602ffd640 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604136799c0 0x56041367bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.040+0000 7fe602ffd640 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604136799c0 0x56041367bde0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e6fd9c2a8e856 server_cookie=906350fa8f27f78f in_seq=0 out_seq=0 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.040+0000 7fe5ea7fc640 10 client.19458 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.040+0000 7fe5ea7fc640 1 -- 172.21.15.7:0/2329278005 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560413574c50 con 0x5604136799c0 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.040+0000 7fe5ea7fc640 10 client.19458 handle_command_reply: tid=0 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.040+0000 7fe60c3b8740 1 client.19458 shutdown 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.040+0000 7fe60c3b8740 1 -- 172.21.15.7:0/2329278005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56041360d1f0 msgr2=0x5604136178a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.040+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56041360d1f0 0x5604136178a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009270 tx=0x7fe5fc01d000 comp rx=0 tx=0).stop 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe60c3b8740 1 -- 172.21.15.7:0/2329278005 shutdown_connections 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x5604136799c0 0x56041367bde0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe5ea7fc640 0 client.19458 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56041360d1f0 0x5604136178a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x560413608080 0x5604136168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe60c3b8740 1 --2- 172.21.15.7:0/2329278005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560413607690 0x5604136163a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe60c3b8740 1 -- 172.21.15.7:0/2329278005 >> 172.21.15.7:0/2329278005 conn(0x560413602c00 msgr2=0x5604136101b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe60c3b8740 1 -- 172.21.15.7:0/2329278005 shutdown_connections 2024-03-26T08:57:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe60c3b8740 1 -- 172.21.15.7:0/2329278005 wait complete. 2024-03-26T08:57:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.041+0000 7fe60c3b8740 20 client.19458 trim_cache size 0 max 16384 2024-03-26T08:57:37.045 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:57:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 -- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5f00a28d0 msgr2=0x7fe5f00a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5f00a28d0 0x7fe5f00a4d90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc005c40 tx=0x7fe5fc009bb0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 -- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 msgr2=0x7fe6041999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 0x7fe6041999f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4036040 tx=0x7fe5f402ccb0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 -- 172.21.15.7:0/1319044234 shutdown_connections 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fe5f00a28d0 0x7fe5f00a4d90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe604101900 0x7fe6041a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe604100f40 0x7fe604199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 --2- 172.21.15.7:0/1319044234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6040ffd50 0x7fe6041999f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 -- 172.21.15.7:0/1319044234 >> 172.21.15.7:0/1319044234 conn(0x7fe6040fb2c0 msgr2=0x7fe604104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 -- 172.21.15.7:0/1319044234 shutdown_connections 2024-03-26T08:57:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:37.042+0000 7fe6097cd640 1 -- 172.21.15.7:0/1319044234 wait complete. 2024-03-26T08:57:37.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:37 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/774901632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:57:37.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:37 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/774901632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:57:37.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:37 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/774901632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:57:38.057 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T08:57:38.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:57:38.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T08:57:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.203+0000 7ffad07c6640 1 -- 172.21.15.7:0/2333622964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac81005c0 msgr2=0x7ffac81009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.203+0000 7ffad07c6640 1 --2- 172.21.15.7:0/2333622964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac81005c0 0x7ffac81009e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac402bc40 comp rx=0 tx=0).stop 2024-03-26T08:57:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.203+0000 7ffad07c6640 1 -- 172.21.15.7:0/2333622964 shutdown_connections 2024-03-26T08:57:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.203+0000 7ffad07c6640 1 --2- 172.21.15.7:0/2333622964 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac8102170 0x7ffac8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.203+0000 7ffad07c6640 1 --2- 172.21.15.7:0/2333622964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac81017b0 0x7ffac8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.203+0000 7ffad07c6640 1 --2- 172.21.15.7:0/2333622964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac81005c0 0x7ffac81009e0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.203+0000 7ffad07c6640 1 -- 172.21.15.7:0/2333622964 >> 172.21.15.7:0/2333622964 conn(0x7ffac80fbb30 msgr2=0x7ffac80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.203+0000 7ffad07c6640 1 -- 172.21.15.7:0/2333622964 shutdown_connections 2024-03-26T08:57:38.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.203+0000 7ffad07c6640 1 -- 172.21.15.7:0/2333622964 wait complete. 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffad07c6640 1 Processor -- start 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffad07c6640 1 -- start start 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffad07c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac81005c0 0x7ffac8197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffad07c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac81017b0 0x7ffac8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffad07c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac8102170 0x7ffac819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffad07c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac810c210 con 0x7ffac8102170 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffad07c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac810c090 con 0x7ffac81017b0 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffad07c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac810c390 con 0x7ffac81005c0 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffaced63640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac8102170 0x7ffac819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffaced63640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac8102170 0x7ffac819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56450/0 (socket says 172.21.15.7:56450) 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7fface562640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac81005c0 0x7ffac8197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffaced63640 1 -- 172.21.15.7:0/1063594111 learned_addr learned my addr 172.21.15.7:0/1063594111 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffacdd61640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac81017b0 0x7ffac8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffaced63640 1 -- 172.21.15.7:0/1063594111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac81005c0 msgr2=0x7ffac8197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffaced63640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac81005c0 0x7ffac8197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffaced63640 1 -- 172.21.15.7:0/1063594111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac81017b0 msgr2=0x7ffac8197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffaced63640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac81017b0 0x7ffac8197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.204+0000 7ffaced63640 1 -- 172.21.15.7:0/1063594111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac819f630 con 0x7ffac8102170 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.205+0000 7ffaced63640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac8102170 0x7ffac819efc0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ffac000eb10 tx=0x7ffac000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.205+0000 7ffabf7fe640 1 -- 172.21.15.7:0/1063594111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0004b20 con 0x7ffac8102170 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.205+0000 7ffad07c6640 1 -- 172.21.15.7:0/1063594111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac819f920 con 0x7ffac8102170 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.205+0000 7ffabf7fe640 1 -- 172.21.15.7:0/1063594111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac0004cc0 con 0x7ffac8102170 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.205+0000 7ffabf7fe640 1 -- 172.21.15.7:0/1063594111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0010760 con 0x7ffac8102170 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.205+0000 7ffad07c6640 1 -- 172.21.15.7:0/1063594111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac819fe20 con 0x7ffac8102170 2024-03-26T08:57:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.207+0000 7ffabf7fe640 1 -- 172.21.15.7:0/1063594111 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7ffac0005020 con 0x7ffac8102170 2024-03-26T08:57:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffad33b5740 1 Processor -- start 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffabf7fe640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ffab40a2800 0x7ffab40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffabf7fe640 1 -- 172.21.15.7:0/1063594111 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffac00cefd0 con 0x7ffac8102170 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffad33b5740 1 -- start start 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffacdd61640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ffab40a2800 0x7ffab40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f07196d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07197350 0x556f0719be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0719c4c0 0x556f0719e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffad33b5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f070d6c90 con 0x556f07197350 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffad33b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f070fc070 con 0x556f0719c4c0 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.208+0000 7ffad33b5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f070a4c80 con 0x556f07196960 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7fface562640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f07196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffacdd61640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07197350 0x556f0719be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7fface562640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f07196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53628/0 (socket says 172.21.15.7:53628) 2024-03-26T08:57:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7fface562640 1 -- 172.21.15.7:0/965750357 learned_addr learned my addr 172.21.15.7:0/965750357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffaced63640 1 --2- 172.21.15.7:0/965750357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0719c4c0 0x556f0719e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffacdd61640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ffab40a2800 0x7ffab40a4cc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffab80095f0 tx=0x7ffab8011810 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffacdd61640 1 -- 172.21.15.7:0/965750357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 msgr2=0x556f07196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffacdd61640 1 --2- 172.21.15.7:0/965750357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f07196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffacdd61640 1 -- 172.21.15.7:0/965750357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0719c4c0 msgr2=0x556f0719e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffacdd61640 1 --2- 172.21.15.7:0/965750357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0719c4c0 0x556f0719e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffacdd61640 1 -- 172.21.15.7:0/965750357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f0709cd10 con 0x556f07197350 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7fface562640 1 --2- 172.21.15.7:0/965750357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f07196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffacdd61640 1 --2- 172.21.15.7:0/965750357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07197350 0x556f0719be50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ffab801d440 tx=0x7ffab801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a315b711c0f79408 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.209+0000 7ffabcff9640 1 -- 172.21.15.7:0/965750357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8017040 con 0x556f07197350 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffabcff9640 1 -- 172.21.15.7:0/965750357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffab8019050 con 0x556f07197350 2024-03-26T08:57:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffabcff9640 1 -- 172.21.15.7:0/965750357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab800f730 con 0x556f07197350 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffad33b5740 1 -- 172.21.15.7:0/965750357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07197350 msgr2=0x556f0719be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffad33b5740 1 --2- 172.21.15.7:0/965750357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07197350 0x556f0719be50 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ffab801d440 tx=0x7ffab801c000 comp rx=0 tx=0).stop 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffad33b5740 1 -- 172.21.15.7:0/965750357 shutdown_connections 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffad33b5740 1 --2- 172.21.15.7:0/965750357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f0719c4c0 0x556f0719e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffad33b5740 1 --2- 172.21.15.7:0/965750357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07197350 0x556f0719be50 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffad33b5740 1 --2- 172.21.15.7:0/965750357 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f07196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffad33b5740 1 -- 172.21.15.7:0/965750357 >> 172.21.15.7:0/965750357 conn(0x556f0715f1a0 msgr2=0x556f071922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffad33b5740 1 -- 172.21.15.7:0/965750357 shutdown_connections 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.210+0000 7ffad33b5740 1 -- 172.21.15.7:0/965750357 wait complete. 2024-03-26T08:57:38.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffad33b5740 1 -- start start 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffad33b5740 1 Processor -- start 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f071a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f071a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f0719c4c0 0x556f071a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffad33b5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f070b80d0 con 0x556f0719c4c0 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffad33b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f070f8110 con 0x556f07197350 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffad33b5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f0719f470 con 0x556f07196960 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffad33b5740 10 client.0 _ll_register_callbacks cb 0x556f070f3e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffaced63640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f0719c4c0 0x556f071a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffaced63640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f0719c4c0 0x556f071a82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56466/0 (socket says 172.21.15.7:56466) 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffaced63640 1 -- 172.21.15.7:0/1847921421 learned_addr learned my addr 172.21.15.7:0/1847921421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7ffacdd61640 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f071a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.211+0000 7fface562640 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f071a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffaced63640 1 -- 172.21.15.7:0/1847921421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 msgr2=0x556f071a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffaced63640 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f071a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffaced63640 1 -- 172.21.15.7:0/1847921421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 msgr2=0x556f071a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffaced63640 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f071a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffaced63640 1 -- 172.21.15.7:0/1847921421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f06ed87e0 con 0x556f0719c4c0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffaced63640 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f0719c4c0 0x556f071a82e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ffac009b670 tx=0x7ffac009b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df3f7968037f561 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffab1ffb640 1 -- 172.21.15.7:0/1847921421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00b3070 con 0x556f0719c4c0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffab1ffb640 1 -- 172.21.15.7:0/1847921421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffac00ba1f0 con 0x556f0719c4c0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffad33b5740 1 -- 172.21.15.7:0/1847921421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffab8019050 con 0x556f0719c4c0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffab1ffb640 1 -- 172.21.15.7:0/1847921421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac009fac0 con 0x556f0719c4c0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffad33b5740 10 client.19470 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffad33b5740 1 -- 172.21.15.7:0/1847921421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f06f38f70 con 0x556f0719c4c0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.212+0000 7ffab1ffb640 1 -- 172.21.15.7:0/1847921421 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac00a4110 con 0x556f0719c4c0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7ffab1ffb640 1 -- 172.21.15.7:0/1847921421 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ffac00860b0 con 0x556f0719c4c0 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7ffad33b5740 10 client.19470 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7ffad33b5740 10 client.19470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7ffad33b5740 20 client.19470 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7ffad33b5740 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f071acbd0 0x556f071aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7ffad33b5740 4 client.19470 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7ffad33b5740 1 -- 172.21.15.7:0/1847921421 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffab8019050 con 0x556f071acbd0 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7fface562640 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f071acbd0 0x556f071aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7fface562640 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f071acbd0 0x556f071aefc0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9a7d4090c294fd server_cookie=7145fd020eb4e503 in_seq=0 out_seq=0 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.213+0000 7ffab1ffb640 10 client.19470 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.214+0000 7ffab1ffb640 1 -- 172.21.15.7:0/1847921421 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7ffab8019050 con 0x556f071acbd0 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.214+0000 7ffab1ffb640 10 client.19470 handle_command_reply: tid=0 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.214+0000 7ffad33b5740 1 client.19470 shutdown 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 -- 172.21.15.7:0/1847921421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f0719c4c0 msgr2=0x556f071a82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f0719c4c0 0x556f071a82e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ffac009b670 tx=0x7ffac009b2d0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 -- 172.21.15.7:0/1847921421 shutdown_connections 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f071acbd0 0x556f071aefc0 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f0719c4c0 0x556f071a82e0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffab1ffb640 0 client.19470 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f071a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 --2- 172.21.15.7:0/1847921421 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f07196960 0x556f071a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 -- 172.21.15.7:0/1847921421 >> 172.21.15.7:0/1847921421 conn(0x556f0715f1a0 msgr2=0x556f07191530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 -- 172.21.15.7:0/1847921421 shutdown_connections 2024-03-26T08:57:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 1 -- 172.21.15.7:0/1847921421 wait complete. 2024-03-26T08:57:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.215+0000 7ffad33b5740 20 client.19470 trim_cache size 0 max 16384 2024-03-26T08:57:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.240+0000 7ffad33b5740 1 Processor -- start 2024-03-26T08:57:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.240+0000 7ffad33b5740 1 -- start start 2024-03-26T08:57:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f0719f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f0719fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 0x556f071a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7ffad33b5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81ab070 con 0x556f07196960 2024-03-26T08:57:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7ffad33b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81a8b70 con 0x556f07197350 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7ffad33b5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81a6bf0 con 0x556f0719c4c0 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7fface562640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f0719f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7fface562640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f0719f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56474/0 (socket says 172.21.15.7:56474) 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7fface562640 1 -- 172.21.15.7:0/846430900 learned_addr learned my addr 172.21.15.7:0/846430900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7ffaced63640 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 0x556f071a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7ffacdd61640 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f0719fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7fface562640 1 -- 172.21.15.7:0/846430900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 msgr2=0x556f071a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7fface562640 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 0x556f071a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7fface562640 1 -- 172.21.15.7:0/846430900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 msgr2=0x556f0719fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7fface562640 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f0719fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7fface562640 1 -- 172.21.15.7:0/846430900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f06dd0170 con 0x556f07196960 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7ffacdd61640 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f0719fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.241+0000 7fface562640 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f0719f840 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ffac4036040 tx=0x7ffac402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1006f34e30c0daa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.242+0000 7ffab3fff640 1 -- 172.21.15.7:0/846430900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac4004240 con 0x556f07196960 2024-03-26T08:57:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.242+0000 7ffab3fff640 1 -- 172.21.15.7:0/846430900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac404b070 con 0x556f07196960 2024-03-26T08:57:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.242+0000 7ffab3fff640 1 -- 172.21.15.7:0/846430900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac40049b0 con 0x556f07196960 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.242+0000 7ffad33b5740 1 -- 172.21.15.7:0/846430900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 msgr2=0x556f0719f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.242+0000 7ffad33b5740 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f0719f840 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ffac4036040 tx=0x7ffac402e040 comp rx=0 tx=0).stop 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 -- 172.21.15.7:0/846430900 shutdown_connections 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 0x556f071a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f0719fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 --2- 172.21.15.7:0/846430900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f0719f840 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 -- 172.21.15.7:0/846430900 >> 172.21.15.7:0/846430900 conn(0x556f0715f1a0 msgr2=0x556f071a0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 -- 172.21.15.7:0/846430900 shutdown_connections 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 -- 172.21.15.7:0/846430900 wait complete. 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 -- start start 2024-03-26T08:57:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 Processor -- start 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f071a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f071a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 0x556f071a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81a71f0 con 0x556f07196960 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81a8510 con 0x556f07197350 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81a7850 con 0x556f0719c4c0 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7fface562640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f071a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7ffad33b5740 10 client.0 _ll_register_callbacks cb 0x7ffac0097c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.243+0000 7fface562640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f071a5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56480/0 (socket says 172.21.15.7:56480) 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7fface562640 1 -- 172.21.15.7:0/712658429 learned_addr learned my addr 172.21.15.7:0/712658429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7ffacdd61640 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f071a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7ffaced63640 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 0x556f071a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7fface562640 1 -- 172.21.15.7:0/712658429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 msgr2=0x556f071a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7fface562640 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 0x556f071a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7fface562640 1 -- 172.21.15.7:0/712658429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 msgr2=0x556f071a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7fface562640 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f071a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7fface562640 1 -- 172.21.15.7:0/712658429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f071a7ac0 con 0x556f07196960 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7fface562640 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f071a5f80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ffac4000c00 tx=0x7ffac40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4772d75c6f3d4b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.244+0000 7ffab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.245+0000 7ffab1ffb640 1 -- 172.21.15.7:0/712658429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac40037f0 con 0x556f07196960 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.245+0000 7ffab1ffb640 1 -- 172.21.15.7:0/712658429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffac4003990 con 0x556f07196960 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.245+0000 7ffad33b5740 1 -- 172.21.15.7:0/712658429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffac404b070 con 0x556f07196960 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.245+0000 7ffab1ffb640 1 -- 172.21.15.7:0/712658429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac4003b90 con 0x556f07196960 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.245+0000 7ffad33b5740 10 client.19476 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.245+0000 7ffad33b5740 1 -- 172.21.15.7:0/712658429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f0720ce70 con 0x556f07196960 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.245+0000 7ffab1ffb640 1 -- 172.21.15.7:0/712658429 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffac404b070 con 0x556f07196960 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.246+0000 7ffab1ffb640 1 -- 172.21.15.7:0/712658429 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7ffac404f420 con 0x556f07196960 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.246+0000 7ffad33b5740 10 client.19476 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.246+0000 7ffad33b5740 10 client.19476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.246+0000 7ffad33b5740 20 client.19476 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.246+0000 7ffad33b5740 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f071acbd0 0x556f0720f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.246+0000 7ffad33b5740 4 client.19476 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T08:57:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.246+0000 7ffad33b5740 1 -- 172.21.15.7:0/712658429 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556f071095b0 con 0x556f071acbd0 2024-03-26T08:57:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.246+0000 7ffacdd61640 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f071acbd0 0x556f0720f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.247+0000 7ffab1ffb640 10 client.19476 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.247+0000 7ffacdd61640 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f071acbd0 0x556f0720f780 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7ccb93983ff3a8 server_cookie=6e2107973c702fdb in_seq=0 out_seq=0 2024-03-26T08:57:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.248+0000 7ffab1ffb640 1 -- 172.21.15.7:0/712658429 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556f071095b0 con 0x556f071acbd0 2024-03-26T08:57:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.248+0000 7ffab1ffb640 10 client.19476 handle_command_reply: tid=0 2024-03-26T08:57:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.248+0000 7ffad33b5740 1 client.19476 shutdown 2024-03-26T08:57:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 -- 172.21.15.7:0/712658429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 msgr2=0x556f071a5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f071a5f80 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ffac4000c00 tx=0x7ffac40040c0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 -- 172.21.15.7:0/712658429 shutdown_connections 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x556f071acbd0 0x556f0720f780 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556f0719c4c0 0x556f071a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556f07197350 0x556f071a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 --2- 172.21.15.7:0/712658429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556f07196960 0x556f071a5f80 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 -- 172.21.15.7:0/712658429 >> 172.21.15.7:0/712658429 conn(0x556f0715f1a0 msgr2=0x556f071a0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffab1ffb640 0 client.19476 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 -- 172.21.15.7:0/712658429 shutdown_connections 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 1 -- 172.21.15.7:0/712658429 wait complete. 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.249+0000 7ffad33b5740 20 client.19476 trim_cache size 0 max 16384 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:57:38.250 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-03-26T08:57:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "7eab4d4d-9da6-41cc-972b-9de075656620", 2024-03-26T08:57:38.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-03-26T08:57:38.251 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.251+0000 7ffad07c6640 1 -- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ffab40a2800 msgr2=0x7ffab40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.251+0000 7ffad07c6640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ffab40a2800 0x7ffab40a4cc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffab80095f0 tx=0x7ffab8011810 comp rx=0 tx=0).stop 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.251+0000 7ffad07c6640 1 -- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac8102170 msgr2=0x7ffac819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.251+0000 7ffad07c6640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac8102170 0x7ffac819efc0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ffac000eb10 tx=0x7ffac000c2d0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.252+0000 7ffad07c6640 1 -- 172.21.15.7:0/1063594111 shutdown_connections 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.252+0000 7ffad07c6640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7ffab40a2800 0x7ffab40a4cc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.252+0000 7ffad07c6640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac8102170 0x7ffac819efc0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.252+0000 7ffad07c6640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac81017b0 0x7ffac8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.252+0000 7ffad07c6640 1 --2- 172.21.15.7:0/1063594111 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffac81005c0 0x7ffac8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.252+0000 7ffad07c6640 1 -- 172.21.15.7:0/1063594111 >> 172.21.15.7:0/1063594111 conn(0x7ffac80fbb30 msgr2=0x7ffac8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.252+0000 7ffad07c6640 1 -- 172.21.15.7:0/1063594111 shutdown_connections 2024-03-26T08:57:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.252+0000 7ffad07c6640 1 -- 172.21.15.7:0/1063594111 wait complete. 2024-03-26T08:57:38.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:57:38.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:57:38.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.404+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/1021816151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc01017b0 msgr2=0x7fbfc0101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.404+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/1021816151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc01017b0 0x7fbfc0101c30 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009900 tx=0x7fbfb402bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:38.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.404+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/1021816151 shutdown_connections 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.404+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/1021816151 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc0102170 0x7fbfc0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.404+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/1021816151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc01017b0 0x7fbfc0101c30 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.404+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/1021816151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfc01005c0 0x7fbfc01009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.404+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/1021816151 >> 172.21.15.7:0/1021816151 conn(0x7fbfc00fbb30 msgr2=0x7fbfc00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.404+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/1021816151 shutdown_connections 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.404+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/1021816151 wait complete. 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.405+0000 7fbfc5ff3640 1 Processor -- start 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.405+0000 7fbfc5ff3640 1 -- start start 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.405+0000 7fbfc5ff3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01005c0 0x7fbfc0199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.405+0000 7fbfc5ff3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfc01017b0 0x7fbfc019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.405+0000 7fbfc5ff3640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc0102170 0x7fbfc01a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.405+0000 7fbfc5ff3640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc010a810 con 0x7fbfc0102170 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.405+0000 7fbfc5ff3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc010a690 con 0x7fbfc01017b0 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.405+0000 7fbfc5ff3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc010a990 con 0x7fbfc01005c0 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc0102170 0x7fbfc01a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc0102170 0x7fbfc01a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56492/0 (socket says 172.21.15.7:56492) 2024-03-26T08:57:38.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbffff640 1 -- 172.21.15.7:0/2465456462 learned_addr learned my addr 172.21.15.7:0/2465456462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01005c0 0x7fbfc0199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfc01017b0 0x7fbfc019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbffff640 1 -- 172.21.15.7:0/2465456462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01005c0 msgr2=0x7fbfc0199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbffff640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01005c0 0x7fbfc0199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbffff640 1 -- 172.21.15.7:0/2465456462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfc01017b0 msgr2=0x7fbfc019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbffff640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfc01017b0 0x7fbfc019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbffff640 1 -- 172.21.15.7:0/2465456462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc01a1810 con 0x7fbfc0102170 2024-03-26T08:57:38.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.406+0000 7fbfbffff640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc0102170 0x7fbfc01a11a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800eaa0 tx=0x7fbfb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.407+0000 7fbfbcff9640 1 -- 172.21.15.7:0/2465456462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800cdb0 con 0x7fbfc0102170 2024-03-26T08:57:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.407+0000 7fbfbcff9640 1 -- 172.21.15.7:0/2465456462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfb8005020 con 0x7fbfc0102170 2024-03-26T08:57:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.407+0000 7fbfbcff9640 1 -- 172.21.15.7:0/2465456462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80116b0 con 0x7fbfc0102170 2024-03-26T08:57:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.407+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/2465456462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc01a1b00 con 0x7fbfc0102170 2024-03-26T08:57:38.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.408+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/2465456462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc01a2070 con 0x7fbfc0102170 2024-03-26T08:57:38.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfbcff9640 1 -- 172.21.15.7:0/2465456462 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fbfb8005560 con 0x7fbfc0102170 2024-03-26T08:57:38.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfc8bd7740 1 Processor -- start 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfc8bd7740 1 -- start start 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e5b0e0 0x558df8e5b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e5bad0 0x558df8e605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e60c40 0x558df8e79240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfc8bd7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df8dc53d0 con 0x558df8e60c40 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfc8bd7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df8d6dad0 con 0x558df8e5b0e0 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfc8bd7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df8dc1470 con 0x558df8e5bad0 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e5bad0 0x558df8e605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfbf7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e5bad0 0x558df8e605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53676/0 (socket says 172.21.15.7:53676) 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.411+0000 7fbfbf7fe640 1 -- 172.21.15.7:0/1817432199 learned_addr learned my addr 172.21.15.7:0/1817432199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfbcff9640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbfa80a2900 0x7fbfa80a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfbf7fe640 1 -- 172.21.15.7:0/1817432199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e5b0e0 msgr2=0x558df8e5b500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/1817432199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e5b0e0 0x558df8e5b500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfbf7fe640 1 -- 172.21.15.7:0/1817432199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e60c40 msgr2=0x558df8e79240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/1817432199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e60c40 0x558df8e79240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfbf7fe640 1 -- 172.21.15.7:0/1817432199 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558df8ba37d0 con 0x558df8e5bad0 2024-03-26T08:57:38.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfbcff9640 1 -- 172.21.15.7:0/2465456462 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbfb80ceb80 con 0x7fbfc0102170 2024-03-26T08:57:38.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/1817432199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e5bad0 0x558df8e605d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000bcb0 tx=0x7fbfb00327b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75489d030f57ac6a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfbffff640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbfa80a2900 0x7fbfa80a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.412+0000 7fbfa5ffb640 1 -- 172.21.15.7:0/1817432199 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0004020 con 0x558df8e5bad0 2024-03-26T08:57:38.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfa5ffb640 1 -- 172.21.15.7:0/1817432199 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfb00041c0 con 0x558df8e5bad0 2024-03-26T08:57:38.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfa5ffb640 1 -- 172.21.15.7:0/1817432199 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00338f0 con 0x558df8e5bad0 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfbffff640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbfa80a2900 0x7fbfa80a4dc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbfb809b520 tx=0x7fbfb8086040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/1817432199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e5bad0 msgr2=0x558df8e605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/1817432199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e5bad0 0x558df8e605d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000bcb0 tx=0x7fbfb00327b0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/1817432199 shutdown_connections 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/1817432199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e60c40 0x558df8e79240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/1817432199 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e5bad0 0x558df8e605d0 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000bcb0 tx=0x7fbfb00327b0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/1817432199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e5b0e0 0x558df8e5b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.413+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/1817432199 >> 172.21.15.7:0/1817432199 conn(0x558df8e63b20 msgr2=0x558df8e63f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.414+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/1817432199 shutdown_connections 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.414+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/1817432199 wait complete. 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.414+0000 7fbfc8bd7740 1 -- start start 2024-03-26T08:57:38.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.415+0000 7fbfc8bd7740 1 Processor -- start 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.415+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6a570 0x558df8e6a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.415+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6aef0 0x558df8e6fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.415+0000 7fbfbeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6a570 0x558df8e6a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfbeffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6a570 0x558df8e6a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56494/0 (socket says 172.21.15.7:56494) 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfbeffd640 1 -- 172.21.15.7:0/2155612935 learned_addr learned my addr 172.21.15.7:0/2155612935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e70080 0x558df8e724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df8e683e0 con 0x558df8e6a570 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df8d81060 con 0x558df8e70080 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6aef0 0x558df8e6fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x558df8e68560 con 0x558df8e6aef0 2024-03-26T08:57:38.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfc8bd7740 10 client.0 _ll_register_callbacks cb 0x558df8d636a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfbeffd640 1 -- 172.21.15.7:0/2155612935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6aef0 msgr2=0x558df8e6fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6aef0 0x558df8e6fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfbeffd640 1 -- 172.21.15.7:0/2155612935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e70080 msgr2=0x558df8e724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e70080 0x558df8e724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.416+0000 7fbfbeffd640 1 -- 172.21.15.7:0/2155612935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558df8ad93f0 con 0x558df8e6a570 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.417+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6a570 0x558df8e6a9b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009900 tx=0x7fbfb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a1846153719241 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.417+0000 7fbf9effd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.417+0000 7fbf9effd640 1 -- 172.21.15.7:0/2155612935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4005d50 con 0x558df8e6a570 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.417+0000 7fbf9effd640 1 -- 172.21.15.7:0/2155612935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbfb4005240 con 0x558df8e6a570 2024-03-26T08:57:38.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.417+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfb00041c0 con 0x558df8e6a570 2024-03-26T08:57:38.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.417+0000 7fbf9effd640 1 -- 172.21.15.7:0/2155612935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb4005440 con 0x558df8e6a570 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.417+0000 7fbfc8bd7740 10 client.19485 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.417+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x558df8a990f0 con 0x558df8e6a570 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.418+0000 7fbf9effd640 1 -- 172.21.15.7:0/2155612935 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbfb40046c0 con 0x558df8e6a570 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.418+0000 7fbfc8bd7740 10 client.19485 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.418+0000 7fbfc8bd7740 10 client.19485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.418+0000 7fbfc8bd7740 20 client.19485 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.418+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558df8e768b0 0x558df8e78d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.418+0000 7fbfc8bd7740 4 client.19485 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.418+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfb00041c0 con 0x558df8e768b0 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.418+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558df8e768b0 0x558df8e78d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.420+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558df8e768b0 0x558df8e78d70 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b9a0d9e4f806948 server_cookie=dd0be0ba7971d8a3 in_seq=0 out_seq=0 2024-03-26T08:57:38.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.420+0000 7fbf9effd640 10 client.19485 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbf9effd640 1 -- 172.21.15.7:0/2155612935 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7fbfb00041c0 con 0x558df8e768b0 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbf9effd640 10 client.19485 handle_command_reply: tid=0 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 client.19485 shutdown 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6a570 msgr2=0x558df8e6a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6a570 0x558df8e6a9b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009900 tx=0x7fbfb4004270 comp rx=0 tx=0).stop 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 shutdown_connections 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558df8e768b0 0x558df8e78d70 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e70080 0x558df8e724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbf9effd640 0 client.19485 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6aef0 0x558df8e6fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/2155612935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6a570 0x558df8e6a9b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 >> 172.21.15.7:0/2155612935 conn(0x558df8e63b20 msgr2=0x558df8e5a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.421+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 shutdown_connections 2024-03-26T08:57:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.422+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/2155612935 wait complete. 2024-03-26T08:57:38.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.422+0000 7fbfc8bd7740 20 client.19485 trim_cache size 0 max 16384 2024-03-26T08:57:38.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.445+0000 7fbfc8bd7740 1 Processor -- start 2024-03-26T08:57:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfc8bd7740 1 -- start start 2024-03-26T08:57:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6a570 0x558df8e692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6aef0 0x558df8e59ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e70080 0x558df8e5a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfc8bd7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01a8e00 con 0x558df8e70080 2024-03-26T08:57:38.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfbeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6a570 0x558df8e692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfbeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6a570 0x558df8e692c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53692/0 (socket says 172.21.15.7:53692) 2024-03-26T08:57:38.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfbeffd640 1 -- 172.21.15.7:0/4223665851 learned_addr learned my addr 172.21.15.7:0/4223665851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/4223665851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01a6900 con 0x558df8e6aef0 2024-03-26T08:57:38.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/4223665851 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01a4980 con 0x558df8e6a570 2024-03-26T08:57:38.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e70080 0x558df8e5a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.447+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6aef0 0x558df8e59ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfbeffd640 1 -- 172.21.15.7:0/4223665851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6a570 msgr2=0x558df8e692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6a570 0x558df8e692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfbeffd640 1 -- 172.21.15.7:0/4223665851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6aef0 msgr2=0x558df8e59ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6aef0 0x558df8e59ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfbeffd640 1 -- 172.21.15.7:0/4223665851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558df8e60990 con 0x558df8e70080 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6aef0 0x558df8e59ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e70080 0x558df8e5a1e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fbfb402c170 tx=0x7fbfb402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfac634fdeaa1b0c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfa57fa640 1 -- 172.21.15.7:0/4223665851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4058050 con 0x558df8e70080 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfa57fa640 1 -- 172.21.15.7:0/4223665851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfb4056070 con 0x558df8e70080 2024-03-26T08:57:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.448+0000 7fbfa57fa640 1 -- 172.21.15.7:0/4223665851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40039f0 con 0x558df8e70080 2024-03-26T08:57:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.449+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/4223665851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e70080 msgr2=0x558df8e5a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.449+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e70080 0x558df8e5a1e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fbfb402c170 tx=0x7fbfb402e000 comp rx=0 tx=0).stop 2024-03-26T08:57:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.449+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/4223665851 shutdown_connections 2024-03-26T08:57:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.449+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e70080 0x558df8e5a1e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.449+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6aef0 0x558df8e59ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.449+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/4223665851 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e6a570 0x558df8e692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.449+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/4223665851 >> 172.21.15.7:0/4223665851 conn(0x558df8e63b20 msgr2=0x558df8e68bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.450+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/4223665851 shutdown_connections 2024-03-26T08:57:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.450+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/4223665851 wait complete. 2024-03-26T08:57:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.450+0000 7fbfc8bd7740 1 -- start start 2024-03-26T08:57:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.450+0000 7fbfc8bd7740 1 Processor -- start 2024-03-26T08:57:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6a570 0x558df8e88890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6aef0 0x558df8e88dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfc8bd7740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e70080 0x558df8e86ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfc8bd7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01a4f80 con 0x558df8e6aef0 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfc8bd7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01a62a0 con 0x558df8e6a570 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfc8bd7740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01a55e0 con 0x558df8e70080 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6a570 0x558df8e88890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6a570 0x558df8e88890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44442/0 (socket says 172.21.15.7:44442) 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfc8bd7740 10 client.0 _ll_register_callbacks cb 0x7fbfb4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfbeffd640 1 -- 172.21.15.7:0/706412898 learned_addr learned my addr 172.21.15.7:0/706412898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e70080 0x558df8e86ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfbeffd640 1 -- 172.21.15.7:0/706412898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e70080 msgr2=0x558df8e86ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e70080 0x558df8e86ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfbeffd640 1 -- 172.21.15.7:0/706412898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6aef0 msgr2=0x558df8e88dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6aef0 0x558df8e88dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.451+0000 7fbfbeffd640 1 -- 172.21.15.7:0/706412898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558df8e87680 con 0x558df8e6a570 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.452+0000 7fbfbeffd640 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6a570 0x558df8e88890 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbfb402c8e0 tx=0x7fbfb4004bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6aecc0181212b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.452+0000 7fbf9effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:57:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.452+0000 7fbf9effd640 1 -- 172.21.15.7:0/706412898 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4058050 con 0x558df8e6a570 2024-03-26T08:57:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.452+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/706412898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfb4056070 con 0x558df8e6a570 2024-03-26T08:57:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.452+0000 7fbf9effd640 1 -- 172.21.15.7:0/706412898 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbfb4003b30 con 0x558df8e6a570 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.453+0000 7fbf9effd640 1 -- 172.21.15.7:0/706412898 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb4003d30 con 0x558df8e6a570 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.453+0000 7fbfc8bd7740 10 client.29249 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.453+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/706412898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558df8ed2180 con 0x558df8e6a570 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.453+0000 7fbf9effd640 1 -- 172.21.15.7:0/706412898 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbfb4059040 con 0x558df8e6a570 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.454+0000 7fbf9effd640 1 -- 172.21.15.7:0/706412898 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7fbfb4046cb0 con 0x558df8e6a570 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.454+0000 7fbfc8bd7740 10 client.29249 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.454+0000 7fbfc8bd7740 10 client.29249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.454+0000 7fbfc8bd7740 20 client.29249 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.455+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558df8e768b0 0x558df8ed4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.455+0000 7fbfc8bd7740 4 client.29249 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:57:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.455+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/706412898 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fbfb4056070 con 0x558df8e768b0 2024-03-26T08:57:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.455+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558df8e768b0 0x558df8ed4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.455+0000 7fbfbf7fe640 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558df8e768b0 0x558df8ed4980 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63af552d89469626 server_cookie=161cb09892a96dea in_seq=0 out_seq=0 2024-03-26T08:57:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.456+0000 7fbf9effd640 10 client.29249 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.456+0000 7fbf9effd640 1 -- 172.21.15.7:0/706412898 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x7fbfb4056070 con 0x558df8e768b0 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.456+0000 7fbf9effd640 10 client.29249 handle_command_reply: tid=0 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.456+0000 7fbfc8bd7740 1 client.29249 shutdown 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.456+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/706412898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6a570 msgr2=0x558df8e88890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.456+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6a570 0x558df8e88890 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbfb402c8e0 tx=0x7fbfb4004bc0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.456+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/706412898 shutdown_connections 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.457+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x558df8e768b0 0x558df8ed4980 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.457+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x558df8e70080 0x558df8e86ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.457+0000 7fbf9effd640 0 client.29249 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.457+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558df8e6aef0 0x558df8e88dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.457+0000 7fbfc8bd7740 1 --2- 172.21.15.7:0/706412898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558df8e6a570 0x558df8e88890 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.457+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/706412898 >> 172.21.15.7:0/706412898 conn(0x558df8e63b20 msgr2=0x558df8e69440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.457+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/706412898 shutdown_connections 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.457+0000 7fbfc8bd7740 1 -- 172.21.15.7:0/706412898 wait complete. 2024-03-26T08:57:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.457+0000 7fbfc8bd7740 20 client.29249 trim_cache size 0 max 16384 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (85 inodes in the stack)", 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout: "7eab4d4d-9da6-41cc-972b-9de075656620": { 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "7eab4d4d-9da6-41cc-972b-9de075656620", 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:57:38.457 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.458+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbfa80a2900 msgr2=0x7fbfa80a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.458+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbfa80a2900 0x7fbfa80a4dc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbfb809b520 tx=0x7fbfb8086040 comp rx=0 tx=0).stop 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.458+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc0102170 msgr2=0x7fbfc01a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.458+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc0102170 0x7fbfc01a11a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800eaa0 tx=0x7fbfb800ef60 comp rx=0 tx=0).stop 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.458+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/2465456462 shutdown_connections 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.459+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fbfa80a2900 0x7fbfa80a4dc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.459+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc0102170 0x7fbfc01a11a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.459+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfc01017b0 0x7fbfc019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.459+0000 7fbfc5ff3640 1 --2- 172.21.15.7:0/2465456462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbfc01005c0 0x7fbfc0199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.459+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/2465456462 >> 172.21.15.7:0/2465456462 conn(0x7fbfc00fbb30 msgr2=0x7fbfc00fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.459+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/2465456462 shutdown_connections 2024-03-26T08:57:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:57:38.459+0000 7fbfc5ff3640 1 -- 172.21.15.7:0/2465456462 wait complete. 2024-03-26T08:57:38.470 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:38 smithi007 ceph-mon[32462]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 657 op/s 2024-03-26T08:57:38.471 INFO:tasks.cephfs.filesystem:scrub status for tag:7eab4d4d-9da6-41cc-972b-9de075656620 - {'path': '/', 'tag': '7eab4d4d-9da6-41cc-972b-9de075656620', 'options': 'recursive,force'} 2024-03-26T08:57:38.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:38 smithi044 ceph-mon[36461]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 657 op/s 2024-03-26T08:57:38.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:38 smithi016 ceph-mon[35851]: pgmap v816: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.9 MiB/s wr, 657 op/s 2024-03-26T08:57:39.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:39 smithi007 ceph-mon[32462]: scrub queued for path: / 2024-03-26T08:57:39.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:39 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:39.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:39 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:57:39.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:39 smithi044 ceph-mon[36461]: scrub queued for path: / 2024-03-26T08:57:39.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:39 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:39.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:39 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:57:39.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:39 smithi016 ceph-mon[35851]: scrub queued for path: / 2024-03-26T08:57:39.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:39 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:39.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:39 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:57:40.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:40 smithi007 ceph-mon[32462]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 919 op/s 2024-03-26T08:57:40.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:40 smithi007 ceph-mon[32462]: scrub summary: active paths [/] 2024-03-26T08:57:40.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:40 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:40.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:40 smithi044 ceph-mon[36461]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 919 op/s 2024-03-26T08:57:40.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:40 smithi044 ceph-mon[36461]: scrub summary: active paths [/] 2024-03-26T08:57:40.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:40 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:40.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:40 smithi016 ceph-mon[35851]: pgmap v817: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 919 op/s 2024-03-26T08:57:40.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:40 smithi016 ceph-mon[35851]: scrub summary: active paths [/] 2024-03-26T08:57:40.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:40 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.608+0000 7f7d74497640 1 -- 172.21.15.7:0/1058585665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c102970 msgr2=0x7f7d6c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.608+0000 7f7d74497640 1 --2- 172.21.15.7:0/1058585665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c102970 0x7f7d6c102df0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009930 tx=0x7f7d5c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.608+0000 7f7d74497640 1 -- 172.21.15.7:0/1058585665 shutdown_connections 2024-03-26T08:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.608+0000 7f7d74497640 1 --2- 172.21.15.7:0/1058585665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6c103330 0x7f7d6c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.608+0000 7f7d74497640 1 --2- 172.21.15.7:0/1058585665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c102970 0x7f7d6c102df0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.608+0000 7f7d74497640 1 --2- 172.21.15.7:0/1058585665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c0ff9a0 0x7f7d6c0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.608+0000 7f7d74497640 1 -- 172.21.15.7:0/1058585665 >> 172.21.15.7:0/1058585665 conn(0x7f7d6c0fb2f0 msgr2=0x7f7d6c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.608+0000 7f7d74497640 1 -- 172.21.15.7:0/1058585665 shutdown_connections 2024-03-26T08:57:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.608+0000 7f7d74497640 1 -- 172.21.15.7:0/1058585665 wait complete. 2024-03-26T08:57:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.609+0000 7f7d74497640 1 Processor -- start 2024-03-26T08:57:41.610 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.609+0000 7f7d74497640 1 -- start start 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d74497640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c0ff9a0 0x7f7d6c1022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d74497640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6c102970 0x7f7d6c100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d74497640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c103330 0x7f7d6c100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d72233640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c0ff9a0 0x7f7d6c1022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d72233640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c0ff9a0 0x7f7d6c1022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56522/0 (socket says 172.21.15.7:56522) 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d71a32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6c102970 0x7f7d6c100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d71a32640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6c102970 0x7f7d6c100920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53720/0 (socket says 172.21.15.7:53720) 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d71a32640 1 -- 172.21.15.7:0/2542902504 learned_addr learned my addr 172.21.15.7:0/2542902504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c10d3b0 con 0x7f7d6c0ff9a0 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c10d230 con 0x7f7d6c103330 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.610+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c10d530 con 0x7f7d6c102970 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.611+0000 7f7d72a34640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c103330 0x7f7d6c100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.611+0000 7f7d71a32640 1 -- 172.21.15.7:0/2542902504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c103330 msgr2=0x7f7d6c100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.611+0000 7f7d71a32640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c103330 0x7f7d6c100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.611+0000 7f7d71a32640 1 -- 172.21.15.7:0/2542902504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c0ff9a0 msgr2=0x7f7d6c1022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.611+0000 7f7d71a32640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c0ff9a0 0x7f7d6c1022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.611+0000 7f7d71a32640 1 -- 172.21.15.7:0/2542902504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c1016f0 con 0x7f7d6c102970 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.611+0000 7f7d72a34640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c103330 0x7f7d6c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.611+0000 7f7d71a32640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6c102970 0x7f7d6c100920 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c000c00 tx=0x7f7d5c02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.612+0000 7f7d637fe640 1 -- 172.21.15.7:0/2542902504 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c043020 con 0x7f7d6c102970 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.612+0000 7f7d637fe640 1 -- 172.21.15.7:0/2542902504 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c0054e0 con 0x7f7d6c102970 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.612+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c1a1b20 con 0x7f7d6c102970 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.612+0000 7f7d637fe640 1 -- 172.21.15.7:0/2542902504 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c003ce0 con 0x7f7d6c102970 2024-03-26T08:57:41.612 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.612+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c1a1fd0 con 0x7f7d6c102970 2024-03-26T08:57:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.616+0000 7f7d77081740 1 Processor -- start 2024-03-26T08:57:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.616+0000 7f7d77081740 1 -- start start 2024-03-26T08:57:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.616+0000 7f7d77081740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487eff130 0x55d487eff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.616+0000 7f7d77081740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487effb20 0x55d487f04620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.616+0000 7f7d77081740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f07120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.616+0000 7f7d77081740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d487e5bb10 con 0x55d487eff130 2024-03-26T08:57:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.616+0000 7f7d77081740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d487e04900 con 0x55d487effb20 2024-03-26T08:57:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.616+0000 7f7d77081740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d487e57bb0 con 0x55d487f04c90 2024-03-26T08:57:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.617+0000 7f7d72a34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487eff130 0x55d487eff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.617+0000 7f7d72a34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487eff130 0x55d487eff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56530/0 (socket says 172.21.15.7:56530) 2024-03-26T08:57:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.617+0000 7f7d72a34640 1 -- 172.21.15.7:0/3309262354 learned_addr learned my addr 172.21.15.7:0/3309262354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.617+0000 7f7d72a34640 1 -- 172.21.15.7:0/3309262354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 msgr2=0x55d487f07120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T08:57:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.617+0000 7f7d72a34640 1 --2- 172.21.15.7:0/3309262354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f07120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.617+0000 7f7d72a34640 1 -- 172.21.15.7:0/3309262354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487effb20 msgr2=0x55d487f04620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T08:57:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.617+0000 7f7d72a34640 1 --2- 172.21.15.7:0/3309262354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487effb20 0x55d487f04620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.617+0000 7f7d72a34640 1 -- 172.21.15.7:0/3309262354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d487b70210 con 0x55d487eff130 2024-03-26T08:57:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.618+0000 7f7d637fe640 1 -- 172.21.15.7:0/2542902504 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7d5c046020 con 0x7f7d6c102970 2024-03-26T08:57:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.618+0000 7f7d72a34640 1 --2- 172.21.15.7:0/3309262354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487eff130 0x55d487eff550 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009fd0 tx=0x7f7d640108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d941a2b9e341a71b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.618+0000 7f7d637fe640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d580a28f0 0x7f7d580a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.619+0000 7f7d637fe640 1 -- 172.21.15.7:0/2542902504 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7d5c0f0bc0 con 0x7f7d6c102970 2024-03-26T08:57:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.619+0000 7f7d60bfe640 1 -- 172.21.15.7:0/3309262354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64011760 con 0x55d487eff130 2024-03-26T08:57:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.619+0000 7f7d60bfe640 1 -- 172.21.15.7:0/3309262354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d64011de0 con 0x55d487eff130 2024-03-26T08:57:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.619+0000 7f7d60bfe640 1 -- 172.21.15.7:0/3309262354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400e3e0 con 0x55d487eff130 2024-03-26T08:57:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.619+0000 7f7d72233640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d580a28f0 0x7f7d580a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.618 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.620+0000 7f7d77081740 1 -- 172.21.15.7:0/3309262354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487eff130 msgr2=0x55d487eff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.620+0000 7f7d77081740 1 --2- 172.21.15.7:0/3309262354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487eff130 0x55d487eff550 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009fd0 tx=0x7f7d640108e0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.620+0000 7f7d77081740 1 -- 172.21.15.7:0/3309262354 shutdown_connections 2024-03-26T08:57:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.620+0000 7f7d77081740 1 --2- 172.21.15.7:0/3309262354 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f07120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.620+0000 7f7d77081740 1 --2- 172.21.15.7:0/3309262354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487effb20 0x55d487f04620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.620+0000 7f7d77081740 1 --2- 172.21.15.7:0/3309262354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487eff130 0x55d487eff550 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.620+0000 7f7d77081740 1 -- 172.21.15.7:0/3309262354 >> 172.21.15.7:0/3309262354 conn(0x55d487ef1e90 msgr2=0x55d487ef2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.620+0000 7f7d72233640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d580a28f0 0x7f7d580a4db0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55d487efff20 tx=0x7f7d6800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:41.619 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.621+0000 7f7d77081740 1 -- 172.21.15.7:0/3309262354 shutdown_connections 2024-03-26T08:57:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.621+0000 7f7d77081740 1 -- 172.21.15.7:0/3309262354 wait complete. 2024-03-26T08:57:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.622+0000 7f7d77081740 1 -- start start 2024-03-26T08:57:41.620 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.622+0000 7f7d77081740 1 Processor -- start 2024-03-26T08:57:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d77081740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f09560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.621 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d77081740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f09aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d77081740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f10ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d77081740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d487f07c40 con 0x55d487effb20 2024-03-26T08:57:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d77081740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d487e17d50 con 0x55d487eff130 2024-03-26T08:57:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d77081740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d487f07dc0 con 0x55d487f04c90 2024-03-26T08:57:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d77081740 10 client.0 _ll_register_callbacks cb 0x55d487e17720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d72233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f09560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.622 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d72233640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f09560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44464/0 (socket says 172.21.15.7:44464) 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d72a34640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f10ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d72a34640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f10ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:53734/0 (socket says 172.21.15.7:53734) 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d72a34640 1 -- 172.21.15.7:0/428729692 learned_addr learned my addr 172.21.15.7:0/428729692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.623+0000 7f7d71a32640 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f09aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d72a34640 1 -- 172.21.15.7:0/428729692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 msgr2=0x55d487f09560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d72a34640 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f09560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d72a34640 1 -- 172.21.15.7:0/428729692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 msgr2=0x55d487f09aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d72a34640 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f09aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d72a34640 1 -- 172.21.15.7:0/428729692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d487c3c620 con 0x55d487f04c90 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d72a34640 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f10ae0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7d64010de0 tx=0x7f7d64011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0300a370320e168 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:41.623 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-03-26T08:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d55ffb640 1 -- 172.21.15.7:0/428729692 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64029060 con 0x55d487f04c90 2024-03-26T08:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d77081740 1 -- 172.21.15.7:0/428729692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d64011de0 con 0x55d487f04c90 2024-03-26T08:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.624+0000 7f7d55ffb640 1 -- 172.21.15.7:0/428729692 <== mon.1 v2:172.21.15.44:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7d64004a40 con 0x55d487f04c90 2024-03-26T08:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d55ffb640 1 -- 172.21.15.7:0/428729692 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d64004c40 con 0x55d487f04c90 2024-03-26T08:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d77081740 10 client.29245 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:41.626 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d77081740 1 -- 172.21.15.7:0/428729692 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d487f13c40 con 0x55d487f04c90 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d55ffb640 1 -- 172.21.15.7:0/428729692 <== mon.1 v2:172.21.15.44:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d64011de0 con 0x55d487f04c90 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d55ffb640 1 -- 172.21.15.7:0/428729692 <== mon.1 v2:172.21.15.44:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7d64023070 con 0x55d487f04c90 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d77081740 10 client.29245 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d77081740 10 client.29245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d77081740 20 client.29245 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d77081740 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d487f14ff0 0x55d487f173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d77081740 4 client.29245 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.625+0000 7f7d77081740 1 -- 172.21.15.7:0/428729692 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d487e69050 con 0x55d487f14ff0 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.626+0000 7f7d72233640 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d487f14ff0 0x55d487f173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.626+0000 7f7d72233640 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d487f14ff0 0x55d487f173e0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d03dcaa5d33769 server_cookie=330cbd02c667e388 in_seq=0 out_seq=0 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.626+0000 7f7d55ffb640 10 client.29245 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d55ffb640 1 -- 172.21.15.7:0/428729692 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d487e69050 con 0x55d487f14ff0 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d55ffb640 10 client.29245 handle_command_reply: tid=0 2024-03-26T08:57:41.627 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 client.29245 shutdown 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 -- 172.21.15.7:0/428729692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 msgr2=0x55d487f10ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f10ae0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7d64010de0 tx=0x7f7d64011f90 comp rx=0 tx=0).stop 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 -- 172.21.15.7:0/428729692 shutdown_connections 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d487f14ff0 0x55d487f173e0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f10ae0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f09aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 --2- 172.21.15.7:0/428729692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f09560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 -- 172.21.15.7:0/428729692 >> 172.21.15.7:0/428729692 conn(0x55d487ef1e90 msgr2=0x55d487ef1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d55ffb640 0 client.29245 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:41.628 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 -- 172.21.15.7:0/428729692 shutdown_connections 2024-03-26T08:57:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 1 -- 172.21.15.7:0/428729692 wait complete. 2024-03-26T08:57:41.629 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.627+0000 7f7d77081740 20 client.29245 trim_cache size 0 max 16384 2024-03-26T08:57:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.654+0000 7f7d77081740 1 Processor -- start 2024-03-26T08:57:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d77081740 1 -- start start 2024-03-26T08:57:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d77081740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f00f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d77081740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487effb20 0x55d487f01480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d77081740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487f04c90 0x55d487f10aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d77081740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1a8dd0 con 0x55d487f04c90 2024-03-26T08:57:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d77081740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1a68d0 con 0x55d487eff130 2024-03-26T08:57:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d77081740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1a4950 con 0x55d487effb20 2024-03-26T08:57:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d72a34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487f04c90 0x55d487f10aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d72a34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487f04c90 0x55d487f10aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56544/0 (socket says 172.21.15.7:56544) 2024-03-26T08:57:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.655+0000 7f7d72a34640 1 -- 172.21.15.7:0/1078432225 learned_addr learned my addr 172.21.15.7:0/1078432225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.656+0000 7f7d72233640 1 --2- 172.21.15.7:0/1078432225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f00f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.655 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.656+0000 7f7d71a32640 1 --2- 172.21.15.7:0/1078432225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487effb20 0x55d487f01480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.656+0000 7f7d72a34640 1 -- 172.21.15.7:0/1078432225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487effb20 msgr2=0x55d487f01480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.656+0000 7f7d72a34640 1 --2- 172.21.15.7:0/1078432225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487effb20 0x55d487f01480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.656+0000 7f7d72a34640 1 -- 172.21.15.7:0/1078432225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 msgr2=0x55d487f00f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.656+0000 7f7d72a34640 1 --2- 172.21.15.7:0/1078432225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f00f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.656+0000 7f7d72a34640 1 -- 172.21.15.7:0/1078432225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d487ef2c00 con 0x55d487f04c90 2024-03-26T08:57:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.657+0000 7f7d72a34640 1 --2- 172.21.15.7:0/1078432225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487f04c90 0x55d487f10aa0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7d64005c70 tx=0x7f7d64024560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5e75533359bf8b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:41.656 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.657+0000 7f7d57fff640 1 -- 172.21.15.7:0/1078432225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64033080 con 0x55d487f04c90 2024-03-26T08:57:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.657+0000 7f7d57fff640 1 -- 172.21.15.7:0/1078432225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d640049c0 con 0x55d487f04c90 2024-03-26T08:57:41.657 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.657+0000 7f7d57fff640 1 -- 172.21.15.7:0/1078432225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64024ce0 con 0x55d487f04c90 2024-03-26T08:57:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.659+0000 7f7d77081740 1 -- 172.21.15.7:0/1078432225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487f04c90 msgr2=0x55d487f10aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.659+0000 7f7d77081740 1 --2- 172.21.15.7:0/1078432225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487f04c90 0x55d487f10aa0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7d64005c70 tx=0x7f7d64024560 comp rx=0 tx=0).stop 2024-03-26T08:57:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.659+0000 7f7d77081740 1 -- 172.21.15.7:0/1078432225 shutdown_connections 2024-03-26T08:57:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.659+0000 7f7d77081740 1 --2- 172.21.15.7:0/1078432225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487f04c90 0x55d487f10aa0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.659+0000 7f7d77081740 1 --2- 172.21.15.7:0/1078432225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487effb20 0x55d487f01480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.660+0000 7f7d77081740 1 --2- 172.21.15.7:0/1078432225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f00f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.658 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.660+0000 7f7d77081740 1 -- 172.21.15.7:0/1078432225 >> 172.21.15.7:0/1078432225 conn(0x55d487ef1e90 msgr2=0x55d487f17000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.661+0000 7f7d77081740 1 -- 172.21.15.7:0/1078432225 shutdown_connections 2024-03-26T08:57:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.661+0000 7f7d77081740 1 -- 172.21.15.7:0/1078432225 wait complete. 2024-03-26T08:57:41.659 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.661+0000 7f7d77081740 1 -- start start 2024-03-26T08:57:41.660 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.661+0000 7f7d77081740 1 Processor -- start 2024-03-26T08:57:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.662+0000 7f7d77081740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f0dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.662+0000 7f7d77081740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f0e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.662+0000 7f7d77081740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f0f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.662+0000 7f7d71a32640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f0e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.662+0000 7f7d71a32640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f0e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56552/0 (socket says 172.21.15.7:56552) 2024-03-26T08:57:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.662+0000 7f7d71a32640 1 -- 172.21.15.7:0/698836116 learned_addr learned my addr 172.21.15.7:0/698836116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:57:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1a4f50 con 0x55d487effb20 2024-03-26T08:57:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1a6270 con 0x55d487eff130 2024-03-26T08:57:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1a55b0 con 0x55d487f04c90 2024-03-26T08:57:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d72a34640 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f0f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d77081740 10 client.0 _ll_register_callbacks cb 0x7f7d64009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:57:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d71a32640 1 -- 172.21.15.7:0/698836116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 msgr2=0x55d487f0f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d71a32640 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f0f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d71a32640 1 -- 172.21.15.7:0/698836116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 msgr2=0x55d487f0dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:57:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d71a32640 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f0dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d71a32640 1 -- 172.21.15.7:0/698836116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d487f0f7c0 con 0x55d487effb20 2024-03-26T08:57:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.663+0000 7f7d72a34640 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f0f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:57:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.664+0000 7f7d71a32640 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f0e260 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b9980 tx=0x7f7d5c0bd400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e54f78aff3c726a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:57:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.664+0000 7f7d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:57:41.664 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.664+0000 7f7d55ffb640 1 -- 172.21.15.7:0/698836116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c004c80 con 0x55d487effb20 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.664+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d640049c0 con 0x55d487effb20 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.664+0000 7f7d55ffb640 1 -- 172.21.15.7:0/698836116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7d5c0ee790 con 0x55d487effb20 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d55ffb640 1 -- 172.21.15.7:0/698836116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d5c0044a0 con 0x55d487effb20 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d77081740 10 client.19506 fetch_fsmap learned FSMap version 25 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d487f10b90 con 0x55d487effb20 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d55ffb640 1 -- 172.21.15.7:0/698836116 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d640049c0 con 0x55d487effb20 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d55ffb640 1 -- 172.21.15.7:0/698836116 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f7d5c0d6220 con 0x55d487effb20 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d77081740 10 client.19506 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d77081740 10 client.19506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d77081740 20 client.19506 populate_metadata read hostname 'smithi007' 2024-03-26T08:57:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d77081740 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d487f70aa0 0x55d487f72ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:57:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d77081740 4 client.19506 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-26T08:57:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d487e633a0 con 0x55d487f70aa0 2024-03-26T08:57:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.665+0000 7f7d72233640 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d487f70aa0 0x55d487f72ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:57:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.672+0000 7f7d72233640 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d487f70aa0 0x55d487f72ec0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac9dab5a946d557 server_cookie=a08c707dc62b6ea6 in_seq=0 out_seq=0 2024-03-26T08:57:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:41.672+0000 7f7d55ffb640 10 client.19506 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d55ffb640 1 -- 172.21.15.7:0/698836116 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d487e633a0 con 0x55d487f70aa0 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d55ffb640 10 client.19506 handle_command_reply: tid=0 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 client.19506 shutdown 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 msgr2=0x55d487f0e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f0e260 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b9980 tx=0x7f7d5c0bd400 comp rx=0 tx=0).stop 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 shutdown_connections 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x55d487f70aa0 0x55d487f72ec0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55d487f04c90 0x55d487f0f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d487effb20 0x55d487f0e260 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 --2- 172.21.15.7:0/698836116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d487eff130 0x55d487f0dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d55ffb640 0 client.19506 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:57:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 >> 172.21.15.7:0/698836116 conn(0x55d487ef1e90 msgr2=0x55d487f00f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 shutdown_connections 2024-03-26T08:57:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 1 -- 172.21.15.7:0/698836116 wait complete. 2024-03-26T08:57:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.015+0000 7f7d77081740 20 client.19506 trim_cache size 0 max 16384 2024-03-26T08:57:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:{ 2024-03-26T08:57:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "message": "", 2024-03-26T08:57:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout: "return_code": 0 2024-03-26T08:57:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stdout:} 2024-03-26T08:57:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d580a28f0 msgr2=0x7f7d580a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d580a28f0 0x7f7d580a4db0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55d487efff20 tx=0x7f7d6800b390 comp rx=0 tx=0).stop 2024-03-26T08:57:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6c102970 msgr2=0x7f7d6c100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:57:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6c102970 0x7f7d6c100920 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c000c00 tx=0x7f7d5c02c6c0 comp rx=0 tx=0).stop 2024-03-26T08:57:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 shutdown_connections 2024-03-26T08:57:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7d580a28f0 0x7f7d580a4db0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d6c103330 0x7f7d6c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d6c102970 0x7f7d6c100920 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 --2- 172.21.15.7:0/2542902504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6c0ff9a0 0x7f7d6c1022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:57:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 >> 172.21.15.7:0/2542902504 conn(0x7f7d6c0fb2f0 msgr2=0x7f7d6c105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:57:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 shutdown_connections 2024-03-26T08:57:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi007.front.sepia.ceph.com.smithi007.stderr:2024-03-26T08:57:42.016+0000 7f7d74497640 1 -- 172.21.15.7:0/2542902504 wait complete. 2024-03-26T08:57:42.548 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:42 smithi007 ceph-mon[32462]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 1.02k op/s 2024-03-26T08:57:42.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:42 smithi044 ceph-mon[36461]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 1.02k op/s 2024-03-26T08:57:42.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:42 smithi016 ceph-mon[35851]: pgmap v818: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.6 MiB/s wr, 1.02k op/s 2024-03-26T08:57:42.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/: 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 .. 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 data 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data: 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:total 36 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 . 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 .. 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 08:42 base 2024-03-26T08:57:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Mar 26 08:42 current_logfiles 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 26 08:42 global 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 log 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_commit_ts 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_dynshmem 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Mar 26 08:42 pg_hba.conf 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Mar 26 08:42 pg_ident.conf 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:57 pg_logical 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 pg_multixact 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_notify 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_replslot 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_serial 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_snapshots 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:57 pg_stat 2024-03-26T08:57:43.005 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:57 pg_stat_tmp 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:57 pg_subtrans 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_tblspc 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pg_twophase 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 PG_VERSION 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Mar 26 08:57 pg_wal 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:55 pg_xact 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Mar 26 08:42 postgresql.auto.conf 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Mar 26 08:42 postgresql.conf 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Mar 26 08:42 postmaster.opts 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base: 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.006 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 08:42 . 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 1 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 13493 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 26 08:45 13494 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 pgsql_tmp 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/1: 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 . 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 08:42 .. 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 112 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 113 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 1247 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1247_fsm 2024-03-26T08:57:43.007 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1247_vm 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 08:42 1249 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1249_fsm 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1249_vm 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 08:42 1255 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1255_fsm 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1255_vm 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 08:42 1259 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1259_fsm 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1259_vm 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 08:42 13344 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13344_fsm 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13344_vm 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13346 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13348 2024-03-26T08:57:43.008 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13349_fsm 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349_vm 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13351 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13353 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13354_fsm 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354_vm 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13356 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13358 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13359_fsm 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359_vm 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13361 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13363 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1417 2024-03-26T08:57:43.009 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1418 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 174 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 175 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2187 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2224 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2328 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2336 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2337 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2579 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2600 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2600_fsm 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2600_vm 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2601_fsm 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601_vm 2024-03-26T08:57:43.010 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2602 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2602_fsm 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2602_vm 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2603 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2603_fsm 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2603_vm 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2604 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2605 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2605_fsm 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2605_vm 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2606_fsm 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606_vm 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2607 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2607_fsm 2024-03-26T08:57:43.011 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2607_vm 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 08:42 2608 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2608_fsm 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2608_vm 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 08:42 2609 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2609_fsm 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2609_vm 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2610 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2610_fsm 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2610_vm 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2611 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2612_fsm 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612_vm 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2613 2024-03-26T08:57:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2615_fsm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615_vm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2616 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2616_fsm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2616_vm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 08:42 2617 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2617_fsm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2617_vm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 08:42 2618 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2618_fsm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2618_vm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 08:42 2619 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2619_fsm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2619_vm 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2620 2024-03-26T08:57:43.013 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2650 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2651 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2652 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2653 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2654 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2655 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2656 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2657 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 08:42 2658 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 2659 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2660 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2661 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2662 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2663 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2664 2024-03-26T08:57:43.014 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2665 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2666 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2667 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2668 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2669 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2670 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 08:42 2673 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 08:42 2674 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 08:42 2675 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2678 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2679 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2680 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2681 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2682 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2683 2024-03-26T08:57:43.015 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2684 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2685 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2686 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2687 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2688 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2689 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 08:42 2690 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 08:42 2691 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2692 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2693 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2696 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2699 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2701 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2702 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2703 2024-03-26T08:57:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2704 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2753 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2753_fsm 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2753_vm 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2754 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2755 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2756 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2757 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2830 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2831 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2832 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2833 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2834 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2835 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2836 2024-03-26T08:57:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2837 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 08:42 2838 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2838_fsm 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2838_vm 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2839 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840_fsm 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2840_vm 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2841 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2995 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2996 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3079_fsm 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079_vm 2024-03-26T08:57:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3080 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3081 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3085 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3118 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3119 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 08:42 3164 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3256 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3257 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3258 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3350 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3351 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3379 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3380 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3381 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3394 2024-03-26T08:57:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3394_fsm 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3394_vm 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3395 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3429 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3430 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3431 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3433 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3439 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3440 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3455 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 08:42 3456 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3456_fsm 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3456_vm 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3466 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3467 2024-03-26T08:57:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3468 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3501 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3502 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3503 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3534 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3541_fsm 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541_vm 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3542 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3574 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3575 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3576 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3596 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3597 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3598 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3599 2024-03-26T08:57:43.021 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3600_fsm 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600_vm 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3601_fsm 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601_vm 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3602_fsm 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602_vm 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603_fsm 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3603_vm 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3604 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3605 2024-03-26T08:57:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3606 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3607 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3608 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 3609 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3712 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3764_fsm 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764_vm 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3766 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3767 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3997 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4143 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4144 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4145 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4146 2024-03-26T08:57:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4147 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4148 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4149 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4150 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4151 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4152 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4153 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4154 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4155 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4156 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4157 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4158 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4159 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4160 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4161 2024-03-26T08:57:43.024 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4162 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4163 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4164 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4165 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4166 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4167 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4168 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4169 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4170 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4171 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4172 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4173 2024-03-26T08:57:43.027 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4174 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 5002 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 548 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 549 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6102 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6104 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6106 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6110 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6111 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6112 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6113 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6117 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 826 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 827 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 828 2024-03-26T08:57:43.028 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 08:42 pg_filenode.map 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 PG_VERSION 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:total 7777 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Mar 26 08:42 . 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 08:42 .. 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 112 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 113 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 1247 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1247_fsm 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1247_vm 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 08:42 1249 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1249_fsm 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1249_vm 2024-03-26T08:57:43.029 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 08:42 1255 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1255_fsm 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1255_vm 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 08:42 1259 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1259_fsm 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1259_vm 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 08:42 13344 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13344_fsm 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13344_vm 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13346 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13348 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13349_fsm 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349_vm 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13351 2024-03-26T08:57:43.030 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13353 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13354_fsm 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354_vm 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13356 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13358 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13359_fsm 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359_vm 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13361 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13363 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1417 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1418 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 174 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 175 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2187 2024-03-26T08:57:43.031 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2224 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2328 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2336 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2337 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2579 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2600 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2600_fsm 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2600_vm 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2601_fsm 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601_vm 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2602 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2602_fsm 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2602_vm 2024-03-26T08:57:43.032 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2603 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2603_fsm 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2603_vm 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2604 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2605 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2605_fsm 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2605_vm 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2606_fsm 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606_vm 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2607 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2607_fsm 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2607_vm 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 08:42 2608 2024-03-26T08:57:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2608_fsm 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2608_vm 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 08:42 2609 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2609_fsm 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2609_vm 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2610 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2610_fsm 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2610_vm 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2611 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2612_fsm 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612_vm 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2613 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2615_fsm 2024-03-26T08:57:43.034 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615_vm 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2616 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2616_fsm 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2616_vm 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 08:42 2617 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2617_fsm 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2617_vm 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 08:42 2618 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2618_fsm 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2618_vm 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Mar 26 08:42 2619 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2619_fsm 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2619_vm 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2620 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2650 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2651 2024-03-26T08:57:43.035 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2652 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2653 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2654 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2655 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2656 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2657 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 08:42 2658 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 2659 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2660 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2661 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2662 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2663 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2664 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2665 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2666 2024-03-26T08:57:43.036 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2667 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2668 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2669 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2670 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 08:42 2673 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 08:42 2674 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 08:42 2675 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2678 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2679 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2680 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2681 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2682 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2683 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2684 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2685 2024-03-26T08:57:43.037 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2686 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2687 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2688 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2689 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 08:42 2690 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 08:42 2691 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2692 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2693 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2696 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2699 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2701 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2702 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2703 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2704 2024-03-26T08:57:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2753 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2753_fsm 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2753_vm 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2754 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2755 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2756 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2757 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2830 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2831 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2832 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2833 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2834 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2835 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2836 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2837 2024-03-26T08:57:43.039 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 08:42 2838 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2838_fsm 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2838_vm 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2839 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840_fsm 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2840_vm 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2841 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2995 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2996 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3079_fsm 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079_vm 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3080 2024-03-26T08:57:43.040 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3081 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3085 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3118 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3119 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 08:42 3164 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3256 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3257 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3258 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3350 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3351 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3379 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3380 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3381 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3394 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3394_fsm 2024-03-26T08:57:43.041 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3394_vm 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3395 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3429 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3430 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3431 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3433 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3439 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3440 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3455 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 08:42 3456 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3456_fsm 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3456_vm 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3466 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3467 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3468 2024-03-26T08:57:43.042 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3501 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3502 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3503 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3534 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3541_fsm 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541_vm 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3542 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3574 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3575 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3576 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3596 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3597 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3598 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3599 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600 2024-03-26T08:57:43.043 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3600_fsm 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600_vm 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3601_fsm 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601_vm 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3602_fsm 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602_vm 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603_fsm 2024-03-26T08:57:43.044 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3603_vm 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3604 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3605 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3606 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3607 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3608 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 3609 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3712 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3764_fsm 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764_vm 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3766 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3767 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3997 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4143 2024-03-26T08:57:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4144 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4145 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4146 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4147 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4148 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4149 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4150 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4151 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4152 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4153 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4154 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4155 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4156 2024-03-26T08:57:43.046 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4157 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4158 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4159 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4160 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4161 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4162 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4163 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4164 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4165 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4166 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4167 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4168 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4169 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4170 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4171 2024-03-26T08:57:43.047 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4172 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4173 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4174 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 5002 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 548 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 549 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6102 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6104 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6106 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6110 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6111 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6112 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6113 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6117 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 826 2024-03-26T08:57:43.048 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 827 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 828 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 08:42 pg_filenode.map 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 PG_VERSION 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:total 640142 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Mar 26 08:45 . 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 08:42 .. 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 112 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 113 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 1247 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1247_fsm 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1247_vm 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Mar 26 08:42 1249 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1249_fsm 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1249_vm 2024-03-26T08:57:43.049 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Mar 26 08:42 1255 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1255_fsm 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1255_vm 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Mar 26 08:57 1259 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1259_fsm 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1259_vm 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Mar 26 08:42 13344 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13344_fsm 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13344_vm 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13346 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13348 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13349_fsm 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13349_vm 2024-03-26T08:57:43.050 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13351 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13353 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13354_fsm 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13354_vm 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13356 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13358 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 13359_fsm 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13359_vm 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 13361 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 13363 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1417 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 1418 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 447537152 Mar 26 08:57 16396 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Mar 26 08:57 16396_fsm 2024-03-26T08:57:43.051 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:44 16396_vm 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 589824 Mar 26 08:57 16397 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:57 16397_fsm 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:57 16397_vm 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1794048 Mar 26 08:57 16399 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:56 16399_fsm 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:55 16399_vm 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:57 16400 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 08:57 16402 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Mar 26 08:57 16404 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 125190144 Mar 26 08:57 16406 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:57 16406_fsm 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:56 16406_vm 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 174 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 175 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2187 2024-03-26T08:57:43.052 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2224 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2328 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2336 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2337 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2579 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2600 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2600_fsm 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2600_vm 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2601_fsm 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2601_vm 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2602 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2602_fsm 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2602_vm 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2603 2024-03-26T08:57:43.053 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2603_fsm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2603_vm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2604 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2605 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2605_fsm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2605_vm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2606_fsm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2606_vm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2607 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2607_fsm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2607_vm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Mar 26 08:42 2608 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2608_fsm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2608_vm 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Mar 26 08:42 2609 2024-03-26T08:57:43.054 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2609_fsm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2609_vm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2610 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2610_fsm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2610_vm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2611 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2612_fsm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2612_vm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2613 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2615_fsm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2615_vm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2616 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2616_fsm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2616_vm 2024-03-26T08:57:43.055 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Mar 26 08:42 2617 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2617_fsm 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2617_vm 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Mar 26 08:42 2618 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2618_fsm 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2618_vm 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Mar 26 08:56 2619 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:44 2619_fsm 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2619_vm 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2620 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2650 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2651 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2652 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2653 2024-03-26T08:57:43.056 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2654 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2655 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2656 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2657 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Mar 26 08:42 2658 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Mar 26 08:42 2659 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2660 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2661 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2662 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2663 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2664 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2665 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2666 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2667 2024-03-26T08:57:43.057 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2668 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2669 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2670 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Mar 26 08:42 2673 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Mar 26 08:42 2674 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Mar 26 08:42 2675 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2678 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2679 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2680 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2681 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2682 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2683 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2684 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2685 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2686 2024-03-26T08:57:43.058 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2687 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2688 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Mar 26 08:42 2689 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Mar 26 08:42 2690 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Mar 26 08:42 2691 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2692 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2693 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:46 2696 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2699 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2701 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2702 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2703 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 2704 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2753 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2753_fsm 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2753_vm 2024-03-26T08:57:43.059 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2754 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2755 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2756 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 2757 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2830 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2831 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2832 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2833 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2834 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2835 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2836 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2837 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Mar 26 08:42 2838 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2838_fsm 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2838_vm 2024-03-26T08:57:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2839 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2840_fsm 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2840_vm 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2841 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2995 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2996 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3079_fsm 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3079_vm 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3080 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3081 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3085 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3118 2024-03-26T08:57:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3119 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Mar 26 08:42 3164 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3256 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3257 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3258 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3350 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3351 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3379 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3380 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3381 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3394 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3394_fsm 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3394_vm 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3395 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3429 2024-03-26T08:57:43.062 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3430 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3431 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3433 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3439 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3440 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Mar 26 08:42 3455 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Mar 26 08:42 3456 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3456_fsm 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3456_vm 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3466 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3467 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3468 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3501 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3502 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3503 2024-03-26T08:57:43.063 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3534 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3541_fsm 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3541_vm 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3542 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3574 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3575 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3576 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3596 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3597 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3598 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3599 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3600_fsm 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3600_vm 2024-03-26T08:57:43.064 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3601_fsm 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3601_vm 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3602_fsm 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3602_vm 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3603_fsm 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3603_vm 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3604 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3605 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3606 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3607 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3608 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Mar 26 08:42 3609 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3712 2024-03-26T08:57:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 3764_fsm 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3764_vm 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3766 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 3767 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3997 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4143 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4144 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4145 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4146 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4147 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4148 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4149 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4150 2024-03-26T08:57:43.066 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4151 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4152 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4153 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4154 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4155 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4156 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4157 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4158 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4159 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4160 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4161 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4162 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4163 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4164 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4165 2024-03-26T08:57:43.067 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4166 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4167 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4168 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4169 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4170 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4171 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4172 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4173 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4174 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 5002 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 548 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 549 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6102 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6104 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6106 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6110 2024-03-26T08:57:43.068 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6111 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6112 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6113 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6117 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 826 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 827 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 828 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 08:42 pg_filenode.map 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Mar 26 08:42 pg_internal.init 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:42 PG_VERSION 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Mar 26 08:42 .. 2024-03-26T08:57:43.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/global: 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:total 560 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Mar 26 08:42 . 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1213 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1213_fsm 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1213_vm 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1214 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1214_fsm 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1214_vm 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 1232 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 1233 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1260 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1260_fsm 2024-03-26T08:57:43.070 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1260_vm 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1261 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1261_fsm 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1261_vm 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1262 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 1262_fsm 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 1262_vm 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2396 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Mar 26 08:42 2396_fsm 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2396_vm 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2397 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2671 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2672 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2676 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2677 2024-03-26T08:57:43.071 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2694 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2695 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2697 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Mar 26 08:42 2698 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2846 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2847 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2964 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2965 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 2966 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 2967 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 3592 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 3593 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4060 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4061 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4175 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4176 2024-03-26T08:57:43.072 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4177 2024-03-26T08:57:43.073 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4178 2024-03-26T08:57:43.073 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4181 2024-03-26T08:57:43.073 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4182 2024-03-26T08:57:43.073 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4183 2024-03-26T08:57:43.073 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4184 2024-03-26T08:57:43.073 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 4185 2024-03-26T08:57:43.074 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 4186 2024-03-26T08:57:43.074 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6000 2024-03-26T08:57:43.074 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6001 2024-03-26T08:57:43.074 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6002 2024-03-26T08:57:43.074 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 6100 2024-03-26T08:57:43.074 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6114 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 6115 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:57 pg_control 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Mar 26 08:42 pg_filenode.map 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Mar 26 08:42 pg_internal.init 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/log: 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:total 2 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1223 Mar 26 08:57 postgresql-Tue.log 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.075 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:total 1 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:57 . 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 mappings 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Mar 26 08:57 replorigin_checkpoint 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 snapshots 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:57 .. 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:57 .. 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 . 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 members 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 offsets 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 .. 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:42 0000 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:total 8 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:42 . 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Mar 26 08:42 .. 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Mar 26 08:44 0000 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.078 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.079 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:total 15 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:57 . 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Mar 26 08:57 db_0.stat 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Mar 26 08:57 db_13494.stat 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Mar 26 08:57 global.stat 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:57 . 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:total 128 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Mar 26 08:57 . 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Mar 26 08:57 0024 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.081 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:total 1048576 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Mar 26 08:57 . 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:57 0000000100000000000000CE 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:51 0000000100000000000000CF 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:52 0000000100000000000000D0 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:53 0000000100000000000000D1 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000D2 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000D3 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000D4 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:51 0000000100000000000000D5 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000D6 2024-03-26T08:57:43.082 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:51 0000000100000000000000D7 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000D8 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:52 0000000100000000000000D9 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000DA 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000DB 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000DC 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000DD 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000DE 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000DF 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:51 0000000100000000000000E0 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000E1 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:51 0000000100000000000000E2 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:52 0000000100000000000000E3 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000E4 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000E5 2024-03-26T08:57:43.083 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000E6 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:50 0000000100000000000000E7 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:53 0000000100000000000000E8 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:55 0000000100000000000000E9 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000EA 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:55 0000000100000000000000EB 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:56 0000000100000000000000EC 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000ED 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000EE 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000EF 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000F0 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:55 0000000100000000000000F1 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000F2 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000F3 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000F4 2024-03-26T08:57:43.084 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000F5 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000F6 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:55 0000000100000000000000F7 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000F8 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000F9 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000FA 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000FB 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 0000000100000000000000FC 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:56 0000000100000000000000FD 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:56 0000000100000000000000FE 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:55 0000000100000000000000FF 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 000000010000000100000000 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 000000010000000100000001 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:55 000000010000000100000002 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:57 000000010000000100000003 2024-03-26T08:57:43.085 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 000000010000000100000004 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 000000010000000100000005 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 000000010000000100000006 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:55 000000010000000100000007 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 000000010000000100000008 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 000000010000000100000009 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:55 00000001000000010000000A 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:56 00000001000000010000000B 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 00000001000000010000000C 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Mar 26 08:54 00000001000000010000000D 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 archive_status 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:total 0 2024-03-26T08:57:43.086 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Mar 26 08:42 . 2024-03-26T08:57:43.087 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Mar 26 08:57 .. 2024-03-26T08:57:43.087 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:57:43.087 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-03-26T08:57:43.087 INFO:teuthology.orchestra.run.smithi007.stdout:total 584 2024-03-26T08:57:43.087 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Mar 26 08:55 . 2024-03-26T08:57:43.087 INFO:teuthology.orchestra.run.smithi007.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Mar 26 08:57 .. 2024-03-26T08:57:43.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Mar 26 08:47 0000 2024-03-26T08:57:43.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Mar 26 08:57 0001 2024-03-26T08:57:43.087 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Mar 26 08:57 0002 2024-03-26T08:57:43.087 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-26T08:57:43.099 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi007.front.sepia.ceph.com 2024-03-26T08:57:43.123 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-26T08:57:43.135 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-26T08:57:43.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:43 smithi007 ceph-mon[32462]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:43.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:43 smithi044 ceph-mon[36461]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:43.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:43 smithi016 ceph-mon[35851]: scrub summary: idle+waiting paths [/] 2024-03-26T08:57:44.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:44 smithi007 ceph-mon[32462]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-03-26T08:57:44.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:44 smithi044 ceph-mon[36461]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-03-26T08:57:44.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:44 smithi016 ceph-mon[35851]: pgmap v819: 97 pgs: 97 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2024-03-26T08:57:46.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:46 smithi007 ceph-mon[32462]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 24 MiB/s wr, 1.85k op/s 2024-03-26T08:57:46.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:46 smithi044 ceph-mon[36461]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 24 MiB/s wr, 1.85k op/s 2024-03-26T08:57:46.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:46 smithi016 ceph-mon[35851]: pgmap v820: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 24 MiB/s wr, 1.85k op/s 2024-03-26T08:57:47.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:47 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:57:47.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:47 smithi007 ceph-mon[32462]: scrub summary: idle 2024-03-26T08:57:47.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:47 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:57:47.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:47 smithi044 ceph-mon[36461]: scrub summary: idle 2024-03-26T08:57:47.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:47 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:57:47.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:47 smithi016 ceph-mon[35851]: scrub summary: idle 2024-03-26T08:57:48.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:48 smithi007 ceph-mon[32462]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 28 MiB/s wr, 1.99k op/s 2024-03-26T08:57:48.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:48 smithi044 ceph-mon[36461]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 28 MiB/s wr, 1.99k op/s 2024-03-26T08:57:48.650 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:48 smithi016 ceph-mon[35851]: pgmap v821: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 28 MiB/s wr, 1.99k op/s 2024-03-26T08:57:50.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:50 smithi007 ceph-mon[32462]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 27 MiB/s wr, 1.90k op/s 2024-03-26T08:57:50.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:50 smithi044 ceph-mon[36461]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 27 MiB/s wr, 1.90k op/s 2024-03-26T08:57:50.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:50 smithi016 ceph-mon[35851]: pgmap v822: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 27 MiB/s wr, 1.90k op/s 2024-03-26T08:57:52.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:52 smithi007 ceph-mon[32462]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2024-03-26T08:57:52.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:52 smithi044 ceph-mon[36461]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2024-03-26T08:57:52.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:52 smithi016 ceph-mon[35851]: pgmap v823: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2024-03-26T08:57:53.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:57:53.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:57:53.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:57:54.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:54 smithi007 ceph-mon[32462]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-03-26T08:57:54.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:54 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:57:54.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:54 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:57:54.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:54 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:57:54.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:54 smithi044 ceph-mon[36461]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-03-26T08:57:54.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:54 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:57:54.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:54 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:57:54.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:54 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:57:54.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:54 smithi016 ceph-mon[35851]: pgmap v824: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-03-26T08:57:54.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:54 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T08:57:54.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:54 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T08:57:54.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:54 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' 2024-03-26T08:57:56.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:56 smithi044 ceph-mon[36461]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-03-26T08:57:56.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:56 smithi016 ceph-mon[35851]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-03-26T08:57:56.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:56 smithi007 ceph-mon[32462]: pgmap v825: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 17 MiB/s wr, 1.10k op/s 2024-03-26T08:57:58.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:57:58 smithi044 ceph-mon[36461]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.2 MiB/s wr, 345 op/s 2024-03-26T08:57:58.601 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:57:58 smithi007 ceph-mon[32462]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.2 MiB/s wr, 345 op/s 2024-03-26T08:57:58.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:57:58 smithi016 ceph-mon[35851]: pgmap v826: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.2 MiB/s wr, 345 op/s 2024-03-26T08:58:00.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:00 smithi044 ceph-mon[36461]: pgmap v827: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-03-26T08:58:00.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:00 smithi016 ceph-mon[35851]: pgmap v827: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-03-26T08:58:00.801 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:00 smithi007 ceph-mon[32462]: pgmap v827: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-03-26T08:58:02.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:02 smithi044 ceph-mon[36461]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-03-26T08:58:02.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:02 smithi016 ceph-mon[35851]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-03-26T08:58:02.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:02 smithi007 ceph-mon[32462]: pgmap v828: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.5 KiB/s wr, 7 op/s 2024-03-26T08:58:04.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:04 smithi044 ceph-mon[36461]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2024-03-26T08:58:04.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:04 smithi016 ceph-mon[35851]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2024-03-26T08:58:04.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:04 smithi007 ceph-mon[32462]: pgmap v829: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2024-03-26T08:58:06.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:06 smithi044 ceph-mon[36461]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-03-26T08:58:06.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:06 smithi016 ceph-mon[35851]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-03-26T08:58:06.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:06 smithi007 ceph-mon[32462]: pgmap v830: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-03-26T08:58:08.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T08:58:08.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T08:58:08.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:08 smithi044 ceph-mon[36461]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-03-26T08:58:08.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.589+0000 7f2600e85640 1 -- 172.21.15.7:0/799650500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc102170 msgr2=0x7f25fc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.589+0000 7f2600e85640 1 --2- 172.21.15.7:0/799650500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc102170 0x7f25fc108a00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009ca0 tx=0x7f25f402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.589+0000 7f2600e85640 1 -- 172.21.15.7:0/799650500 shutdown_connections 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.589+0000 7f2600e85640 1 --2- 172.21.15.7:0/799650500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc102170 0x7f25fc108a00 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.589+0000 7f2600e85640 1 --2- 172.21.15.7:0/799650500 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25fc1017b0 0x7f25fc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.589+0000 7f2600e85640 1 --2- 172.21.15.7:0/799650500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25fc1005c0 0x7f25fc1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.589+0000 7f2600e85640 1 -- 172.21.15.7:0/799650500 >> 172.21.15.7:0/799650500 conn(0x7f25fc0fbb50 msgr2=0x7f25fc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.589+0000 7f2600e85640 1 -- 172.21.15.7:0/799650500 shutdown_connections 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.589+0000 7f2600e85640 1 -- 172.21.15.7:0/799650500 wait complete. 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f2600e85640 1 Processor -- start 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f2600e85640 1 -- start start 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f2600e85640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc1005c0 0x7f25fc06a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f2600e85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25fc1017b0 0x7f25fc19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f2600e85640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25fc102170 0x7f25fc19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f2600e85640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc10c220 con 0x7f25fc1005c0 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f2600e85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc10c0a0 con 0x7f25fc1017b0 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f2600e85640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc10c3a0 con 0x7f25fc102170 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc1005c0 0x7f25fc06a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc1005c0 0x7f25fc06a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53166/0 (socket says 172.21.15.7:53166) 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25fa59c640 1 -- 172.21.15.7:0/2495993369 learned_addr learned my addr 172.21.15.7:0/2495993369 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25f3fff640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25fc1017b0 0x7f25fc19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25fad9d640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25fc102170 0x7f25fc19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25fa59c640 1 -- 172.21.15.7:0/2495993369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25fc102170 msgr2=0x7f25fc19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25fa59c640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25fc102170 0x7f25fc19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25fa59c640 1 -- 172.21.15.7:0/2495993369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25fc1017b0 msgr2=0x7f25fc19c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25fa59c640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25fc1017b0 0x7f25fc19c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25fa59c640 1 -- 172.21.15.7:0/2495993369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc19f5e0 con 0x7f25fc1005c0 2024-03-26T08:58:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.590+0000 7f25f3fff640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25fc1017b0 0x7f25fc19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.591+0000 7f25fa59c640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc1005c0 0x7f25fc06a8b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00e9c0 tx=0x7f25ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.591+0000 7f25f37fe640 1 -- 172.21.15.7:0/2495993369 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec00cd80 con 0x7f25fc1005c0 2024-03-26T08:58:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.591+0000 7f25f37fe640 1 -- 172.21.15.7:0/2495993369 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25ec0035f0 con 0x7f25fc1005c0 2024-03-26T08:58:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.591+0000 7f2600e85640 1 -- 172.21.15.7:0/2495993369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc19f870 con 0x7f25fc1005c0 2024-03-26T08:58:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.591+0000 7f25f37fe640 1 -- 172.21.15.7:0/2495993369 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec010690 con 0x7f25fc1005c0 2024-03-26T08:58:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.591+0000 7f2600e85640 1 -- 172.21.15.7:0/2495993369 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc19fdc0 con 0x7f25fc1005c0 2024-03-26T08:58:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.593+0000 7f25f37fe640 1 -- 172.21.15.7:0/2495993369 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f25ec0037a0 con 0x7f25fc1005c0 2024-03-26T08:58:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.593+0000 7f2603a7c740 1 Processor -- start 2024-03-26T08:58:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f2603a7c740 1 -- start start 2024-03-26T08:58:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba996e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 0x562ba9973290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba9975d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f2603a7c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba98d33d0 con 0x562ba996dda0 2024-03-26T08:58:08.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f2603a7c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba987bad0 con 0x562ba996e790 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f2603a7c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba98cf470 con 0x562ba9973900 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25f37fe640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25e80a2800 0x7f25e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba996e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 0x562ba9973290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25f3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba9975d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25fad9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba996e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53176/0 (socket says 172.21.15.7:53176) 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25fad9d640 1 -- 172.21.15.7:0/1608622150 learned_addr learned my addr 172.21.15.7:0/1608622150 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25fad9d640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25e80a2800 0x7f25e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25f37fe640 1 -- 172.21.15.7:0/2495993369 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25ec0cee50 con 0x7f25fc1005c0 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25f3fff640 1 -- 172.21.15.7:0/1608622150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 msgr2=0x562ba9973290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25f3fff640 1 --2- 172.21.15.7:0/1608622150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 0x562ba9973290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25f3fff640 1 -- 172.21.15.7:0/1608622150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 msgr2=0x562ba996e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.594+0000 7f25f3fff640 1 --2- 172.21.15.7:0/1608622150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba996e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.595+0000 7f25f3fff640 1 -- 172.21.15.7:0/1608622150 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ba96b17d0 con 0x562ba9973900 2024-03-26T08:58:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.595+0000 7f25fad9d640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25e80a2800 0x7f25e80a4cc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x562ba996eff0 tx=0x7f25f4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.595+0000 7f25f3fff640 1 --2- 172.21.15.7:0/1608622150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba9975d90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009fa0 tx=0x7f25e40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c27925fc38ee05 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.595+0000 7f25f0ff9640 1 -- 172.21.15.7:0/1608622150 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4004030 con 0x562ba9973900 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.595+0000 7f25f0ff9640 1 -- 172.21.15.7:0/1608622150 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25e4005500 con 0x562ba9973900 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.595+0000 7f25f0ff9640 1 -- 172.21.15.7:0/1608622150 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4004de0 con 0x562ba9973900 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 -- 172.21.15.7:0/1608622150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 msgr2=0x562ba9975d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1608622150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba9975d90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009fa0 tx=0x7f25e40118d0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 -- 172.21.15.7:0/1608622150 shutdown_connections 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1608622150 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba9975d90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1608622150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 0x562ba9973290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1608622150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba996e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 -- 172.21.15.7:0/1608622150 >> 172.21.15.7:0/1608622150 conn(0x562ba9969310 msgr2=0x562ba9969710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 -- 172.21.15.7:0/1608622150 shutdown_connections 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 -- 172.21.15.7:0/1608622150 wait complete. 2024-03-26T08:58:08.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 -- start start 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 Processor -- start 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba99781f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 0x562ba9978730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba997f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba99768b0 con 0x562ba996dda0 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba988f060 con 0x562ba996e790 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ba9976a30 con 0x562ba9973900 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f2603a7c740 10 client.0 _ll_register_callbacks cb 0x562ba98716a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f25fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba99781f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f25fa59c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba99781f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53188/0 (socket says 172.21.15.7:53188) 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f25fa59c640 1 -- 172.21.15.7:0/356811771 learned_addr learned my addr 172.21.15.7:0/356811771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.596+0000 7f25fad9d640 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba997f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25f3fff640 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 0x562ba9978730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25fa59c640 1 -- 172.21.15.7:0/356811771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 msgr2=0x562ba997f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25fa59c640 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba997f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25fa59c640 1 -- 172.21.15.7:0/356811771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 msgr2=0x562ba9978730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25fa59c640 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 0x562ba9978730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25fa59c640 1 -- 172.21.15.7:0/356811771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ba95e73f0 con 0x562ba996dda0 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25fa59c640 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba99781f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00c6a0 tx=0x7f25ec06a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d8f014d1929dcf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25e1ffb640 1 -- 172.21.15.7:0/356811771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec09f900 con 0x562ba996dda0 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25e1ffb640 1 -- 172.21.15.7:0/356811771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25ec086070 con 0x562ba996dda0 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f2603a7c740 1 -- 172.21.15.7:0/356811771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25e4005500 con 0x562ba996dda0 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25e1ffb640 1 -- 172.21.15.7:0/356811771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ec0a3f50 con 0x562ba996dda0 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f2603a7c740 10 client.19518 fetch_fsmap learned FSMap version 25 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f2603a7c740 1 -- 172.21.15.7:0/356811771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ba95a70f0 con 0x562ba996dda0 2024-03-26T08:58:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.597+0000 7f25e1ffb640 1 -- 172.21.15.7:0/356811771 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25e4005500 con 0x562ba996dda0 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f25e1ffb640 1 -- 172.21.15.7:0/356811771 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f25ec09ac00 con 0x562ba996dda0 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f2603a7c740 10 client.19518 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f2603a7c740 10 client.19518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f2603a7c740 20 client.19518 populate_metadata read hostname 'smithi007' 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f2603a7c740 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562ba9983cd0 0x562ba9986120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f2603a7c740 4 client.19518 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f2603a7c740 1 -- 172.21.15.7:0/356811771 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ba98e0910 con 0x562ba9983cd0 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f25f3fff640 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562ba9983cd0 0x562ba9986120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f25f3fff640 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562ba9983cd0 0x562ba9986120 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5429286163c4790 server_cookie=e88495b60ef32940 in_seq=0 out_seq=0 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.598+0000 7f25e1ffb640 10 client.19518 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f25e1ffb640 1 -- 172.21.15.7:0/356811771 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562ba98e0910 con 0x562ba9983cd0 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f25e1ffb640 10 client.19518 handle_command_reply: tid=0 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 client.19518 shutdown 2024-03-26T08:58:08.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 -- 172.21.15.7:0/356811771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 msgr2=0x562ba99781f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba99781f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00c6a0 tx=0x7f25ec06a610 comp rx=0 tx=0).stop 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 -- 172.21.15.7:0/356811771 shutdown_connections 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562ba9983cd0 0x562ba9986120 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba997f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996e790 0x562ba9978730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f25e1ffb640 0 client.19518 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 --2- 172.21.15.7:0/356811771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996dda0 0x562ba99781f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 -- 172.21.15.7:0/356811771 >> 172.21.15.7:0/356811771 conn(0x562ba9969310 msgr2=0x562ba9968530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 -- 172.21.15.7:0/356811771 shutdown_connections 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.599+0000 7f2603a7c740 1 -- 172.21.15.7:0/356811771 wait complete. 2024-03-26T08:58:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.600+0000 7f2603a7c740 20 client.19518 trim_cache size 0 max 16384 2024-03-26T08:58:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.615+0000 7f2603a7c740 1 Processor -- start 2024-03-26T08:58:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.615+0000 7f2603a7c740 1 -- start start 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba996dda0 0x562ba9978690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba9973900 0x562ba996cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f2603a7c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1aaf80 con 0x562ba996e790 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f2603a7c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1a8a80 con 0x562ba9973900 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f2603a7c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1a6b00 con 0x562ba996dda0 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25f3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25f3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53198/0 (socket says 172.21.15.7:53198) 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25f3fff640 1 -- 172.21.15.7:0/676908605 learned_addr learned my addr 172.21.15.7:0/676908605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25fa59c640 1 --2- 172.21.15.7:0/676908605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba996dda0 0x562ba9978690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25fad9d640 1 --2- 172.21.15.7:0/676908605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba9973900 0x562ba996cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25f3fff640 1 -- 172.21.15.7:0/676908605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba996dda0 msgr2=0x562ba9978690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25f3fff640 1 --2- 172.21.15.7:0/676908605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba996dda0 0x562ba9978690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25f3fff640 1 -- 172.21.15.7:0/676908605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba9973900 msgr2=0x562ba996cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25f3fff640 1 --2- 172.21.15.7:0/676908605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba9973900 0x562ba996cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25f3fff640 1 -- 172.21.15.7:0/676908605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ba9983ab0 con 0x562ba996e790 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25f3fff640 1 --2- 172.21.15.7:0/676908605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997f700 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f25e40058c0 tx=0x7f25e4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3ffaea1de6f214 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25e3fff640 1 -- 172.21.15.7:0/676908605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e401d040 con 0x562ba996e790 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25e3fff640 1 -- 172.21.15.7:0/676908605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25e402f030 con 0x562ba996e790 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.616+0000 7f25e3fff640 1 -- 172.21.15.7:0/676908605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4003850 con 0x562ba996e790 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 -- 172.21.15.7:0/676908605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 msgr2=0x562ba997f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 --2- 172.21.15.7:0/676908605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997f700 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f25e40058c0 tx=0x7f25e4009310 comp rx=0 tx=0).stop 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 -- 172.21.15.7:0/676908605 shutdown_connections 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 --2- 172.21.15.7:0/676908605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba9973900 0x562ba996cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 --2- 172.21.15.7:0/676908605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997f700 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 --2- 172.21.15.7:0/676908605 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba996dda0 0x562ba9978690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 -- 172.21.15.7:0/676908605 >> 172.21.15.7:0/676908605 conn(0x562ba9969310 msgr2=0x562ba996c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 -- 172.21.15.7:0/676908605 shutdown_connections 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 -- 172.21.15.7:0/676908605 wait complete. 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 -- start start 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.617+0000 7f2603a7c740 1 Processor -- start 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996dda0 0x562ba997d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f2603a7c740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba997ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f2603a7c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1a7100 con 0x562ba996e790 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f2603a7c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1a8420 con 0x562ba996dda0 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f2603a7c740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1a7760 con 0x562ba9973900 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f2603a7c740 10 client.0 _ll_register_callbacks cb 0x7f25ec097ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:58:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25f3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25f3fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53210/0 (socket says 172.21.15.7:53210) 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25f3fff640 1 -- 172.21.15.7:0/1765253386 learned_addr learned my addr 172.21.15.7:0/1765253386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25fa59c640 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996dda0 0x562ba997d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25fad9d640 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba997ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25f3fff640 1 -- 172.21.15.7:0/1765253386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 msgr2=0x562ba997ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25f3fff640 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba997ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25f3fff640 1 -- 172.21.15.7:0/1765253386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996dda0 msgr2=0x562ba997d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25f3fff640 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996dda0 0x562ba997d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25f3fff640 1 -- 172.21.15.7:0/1765253386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ba9969d80 con 0x562ba996e790 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25fa59c640 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996dda0 0x562ba997d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25f3fff640 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997dad0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009d60 tx=0x7f25e40042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913d052b354adff3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:58:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25e1ffb640 1 -- 172.21.15.7:0/1765253386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e401d040 con 0x562ba996e790 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f2603a7c740 1 -- 172.21.15.7:0/1765253386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25e402f030 con 0x562ba996e790 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25e1ffb640 1 -- 172.21.15.7:0/1765253386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25e402b170 con 0x562ba996e790 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f25e1ffb640 1 -- 172.21.15.7:0/1765253386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25e40045d0 con 0x562ba996e790 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.618+0000 7f2603a7c740 10 client.19524 fetch_fsmap learned FSMap version 25 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f2603a7c740 1 -- 172.21.15.7:0/1765253386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ba99dfaf0 con 0x562ba996e790 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f25e1ffb640 1 -- 172.21.15.7:0/1765253386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f25e402f030 con 0x562ba996e790 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f25e1ffb640 1 -- 172.21.15.7:0/1765253386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f25e4003bc0 con 0x562ba996e790 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f2603a7c740 10 client.19524 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f2603a7c740 10 client.19524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f2603a7c740 20 client.19524 populate_metadata read hostname 'smithi007' 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562ba99e0210 0x562ba99e2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f2603a7c740 4 client.19524 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f2603a7c740 1 -- 172.21.15.7:0/1765253386 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562ba98dac60 con 0x562ba99e0210 2024-03-26T08:58:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f25fa59c640 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562ba99e0210 0x562ba99e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f25e1ffb640 10 client.19524 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.619+0000 7f25fa59c640 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562ba99e0210 0x562ba99e2600 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99cf616614cbc2cc server_cookie=f5b67dc0691da952 in_seq=0 out_seq=0 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f25e1ffb640 1 -- 172.21.15.7:0/1765253386 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562ba98dac60 con 0x562ba99e0210 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f25e1ffb640 10 client.19524 handle_command_reply: tid=0 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 client.19524 shutdown 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 -- 172.21.15.7:0/1765253386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 msgr2=0x562ba997dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997dad0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009d60 tx=0x7f25e40042b0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 -- 172.21.15.7:0/1765253386 shutdown_connections 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x562ba99e0210 0x562ba99e2600 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x562ba9973900 0x562ba997ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562ba996e790 0x562ba997dad0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 --2- 172.21.15.7:0/1765253386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ba996dda0 0x562ba997d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f25e1ffb640 0 client.19524 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:58:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 -- 172.21.15.7:0/1765253386 >> 172.21.15.7:0/1765253386 conn(0x562ba9969310 msgr2=0x562ba996ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 -- 172.21.15.7:0/1765253386 shutdown_connections 2024-03-26T08:58:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 1 -- 172.21.15.7:0/1765253386 wait complete. 2024-03-26T08:58:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2603a7c740 20 client.19524 trim_cache size 0 max 16384 2024-03-26T08:58:08.626 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:58:08.626 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-03-26T08:58:08.626 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-03-26T08:58:08.626 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:58:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 -- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25e80a2800 msgr2=0x7f25e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25e80a2800 0x7f25e80a4cc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x562ba996eff0 tx=0x7f25f4032590 comp rx=0 tx=0).stop 2024-03-26T08:58:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 -- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc1005c0 msgr2=0x7f25fc06a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc1005c0 0x7f25fc06a8b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00e9c0 tx=0x7f25ec00ee80 comp rx=0 tx=0).stop 2024-03-26T08:58:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 -- 172.21.15.7:0/2495993369 shutdown_connections 2024-03-26T08:58:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f25e80a2800 0x7f25e80a4cc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f25fc102170 0x7f25fc19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25fc1017b0 0x7f25fc19c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 --2- 172.21.15.7:0/2495993369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25fc1005c0 0x7f25fc06a8b0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 -- 172.21.15.7:0/2495993369 >> 172.21.15.7:0/2495993369 conn(0x7f25fc0fbb50 msgr2=0x7f25fc104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 -- 172.21.15.7:0/2495993369 shutdown_connections 2024-03-26T08:58:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.620+0000 7f2600e85640 1 -- 172.21.15.7:0/2495993369 wait complete. 2024-03-26T08:58:08.628 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:08 smithi007 ceph-mon[32462]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-03-26T08:58:08.630 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T08:58:08.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:58:08.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.733+0000 7fabbf29a640 1 -- 172.21.15.7:0/1844682190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb8101730 msgr2=0x7fabb8101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.733+0000 7fabbf29a640 1 --2- 172.21.15.7:0/1844682190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb8101730 0x7fabb8101b50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009900 tx=0x7fabb002bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:08.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.733+0000 7fabbf29a640 1 -- 172.21.15.7:0/1844682190 shutdown_connections 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.733+0000 7fabbf29a640 1 --2- 172.21.15.7:0/1844682190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb81032e0 0x7fabb8109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.733+0000 7fabbf29a640 1 --2- 172.21.15.7:0/1844682190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8102920 0x7fabb8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.733+0000 7fabbf29a640 1 --2- 172.21.15.7:0/1844682190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb8101730 0x7fabb8101b50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.733+0000 7fabbf29a640 1 -- 172.21.15.7:0/1844682190 >> 172.21.15.7:0/1844682190 conn(0x7fabb80fccc0 msgr2=0x7fabb80ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.733+0000 7fabbf29a640 1 -- 172.21.15.7:0/1844682190 shutdown_connections 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.733+0000 7fabbf29a640 1 -- 172.21.15.7:0/1844682190 wait complete. 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbf29a640 1 Processor -- start 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbf29a640 1 -- start start 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbf29a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8101730 0x7fabb81979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbf29a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8102920 0x7fabb8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbf29a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb81032e0 0x7fabb819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbf29a640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb810d380 con 0x7fabb8102920 2024-03-26T08:58:08.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbf29a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb810d200 con 0x7fabb8101730 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbf29a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb810d500 con 0x7fabb81032e0 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbc835640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8102920 0x7fabb8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbc835640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8102920 0x7fabb8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53226/0 (socket says 172.21.15.7:53226) 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbc835640 1 -- 172.21.15.7:0/2123340722 learned_addr learned my addr 172.21.15.7:0/2123340722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbd036640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8101730 0x7fabb81979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.734+0000 7fabbd837640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb81032e0 0x7fabb819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabbc835640 1 -- 172.21.15.7:0/2123340722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb81032e0 msgr2=0x7fabb819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabbc835640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb81032e0 0x7fabb819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabbc835640 1 -- 172.21.15.7:0/2123340722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8101730 msgr2=0x7fabb81979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabbc835640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8101730 0x7fabb81979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabbc835640 1 -- 172.21.15.7:0/2123340722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb819f5e0 con 0x7fabb8102920 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabbd036640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8101730 0x7fabb81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabbc835640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8102920 0x7fabb8197f00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9f0 tx=0x7faba800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:08.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabae7fc640 1 -- 172.21.15.7:0/2123340722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba800cd80 con 0x7fabb8102920 2024-03-26T08:58:08.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb819f8d0 con 0x7fabb8102920 2024-03-26T08:58:08.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabae7fc640 1 -- 172.21.15.7:0/2123340722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faba8005500 con 0x7fabb8102920 2024-03-26T08:58:08.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabae7fc640 1 -- 172.21.15.7:0/2123340722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8010690 con 0x7fabb8102920 2024-03-26T08:58:08.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.735+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb819fdd0 con 0x7fabb8102920 2024-03-26T08:58:08.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.736+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb8108a50 con 0x7fabb8102920 2024-03-26T08:58:08.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.737+0000 7fabae7fc640 1 -- 172.21.15.7:0/2123340722 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7faba80046c0 con 0x7fabb8102920 2024-03-26T08:58:08.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.738+0000 7fabae7fc640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faba40a2800 0x7faba40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:08.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.738+0000 7fabae7fc640 1 -- 172.21.15.7:0/2123340722 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faba80cee90 con 0x7fabb8102920 2024-03-26T08:58:08.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.738+0000 7fabbd036640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faba40a2800 0x7faba40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:08.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.738+0000 7fabbd036640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faba40a2800 0x7faba40a4cc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fabb002c170 tx=0x7fabb002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:08.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.742+0000 7fabae7fc640 1 -- 172.21.15.7:0/2123340722 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faba809bef0 con 0x7fabb8102920 2024-03-26T08:58:08.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:08 smithi016 ceph-mon[35851]: pgmap v831: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-03-26T08:58:08.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.962+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabb802d070 con 0x7fabb8102920 2024-03-26T08:58:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.962+0000 7fabae7fc640 1 -- 172.21.15.7:0/2123340722 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7faba809b7f0 con 0x7fabb8102920 2024-03-26T08:58:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:58:08.961 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:58:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:58:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faba40a2800 msgr2=0x7faba40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faba40a2800 0x7faba40a4cc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fabb002c170 tx=0x7fabb002df90 comp rx=0 tx=0).stop 2024-03-26T08:58:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8102920 msgr2=0x7fabb8197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8102920 0x7fabb8197f00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9f0 tx=0x7faba800edf0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 shutdown_connections 2024-03-26T08:58:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7faba40a2800 0x7faba40a4cc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fabb81032e0 0x7fabb819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabb8102920 0x7fabb8197f00 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 --2- 172.21.15.7:0/2123340722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8101730 0x7fabb81979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:08.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.964+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 >> 172.21.15.7:0/2123340722 conn(0x7fabb80fccc0 msgr2=0x7fabb8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:08.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.965+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 shutdown_connections 2024-03-26T08:58:08.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:08.965+0000 7fabbf29a640 1 -- 172.21.15.7:0/2123340722 wait complete. 2024-03-26T08:58:08.974 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:58:08.974 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T08:58:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.080+0000 7f563a174640 1 -- 172.21.15.7:0/1674443665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634102130 msgr2=0x7f5634104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.080+0000 7f563a174640 1 --2- 172.21.15.7:0/1674443665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634102130 0x7f5634104520 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5624009900 tx=0x7f562402bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.080+0000 7f563a174640 1 -- 172.21.15.7:0/1674443665 shutdown_connections 2024-03-26T08:58:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.080+0000 7f563a174640 1 --2- 172.21.15.7:0/1674443665 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5634107390 0x7f56341097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.080+0000 7f563a174640 1 --2- 172.21.15.7:0/1674443665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5634104a60 0x7f5634106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.080+0000 7f563a174640 1 --2- 172.21.15.7:0/1674443665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634102130 0x7f5634104520 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.080+0000 7f563a174640 1 -- 172.21.15.7:0/1674443665 >> 172.21.15.7:0/1674443665 conn(0x7f56340fbb00 msgr2=0x7f56340fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.080+0000 7f563a174640 1 -- 172.21.15.7:0/1674443665 shutdown_connections 2024-03-26T08:58:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.080+0000 7f563a174640 1 -- 172.21.15.7:0/1674443665 wait complete. 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f563a174640 1 Processor -- start 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f563a174640 1 -- start start 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f563a174640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5634102130 0x7f5634197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f563a174640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5634104a60 0x7f5634197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f563a174640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634107390 0x7f563419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f563a174640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563410d340 con 0x7f5634107390 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f563a174640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563410d1c0 con 0x7f5634104a60 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f563a174640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563410d4c0 con 0x7f5634102130 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f5633fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634107390 0x7f563419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f5633fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634107390 0x7f563419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53254/0 (socket says 172.21.15.7:53254) 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f5633fff640 1 -- 172.21.15.7:0/2554022459 learned_addr learned my addr 172.21.15.7:0/2554022459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f56337fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5634102130 0x7f5634197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.081+0000 7f5632ffd640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5634104a60 0x7f5634197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f5633fff640 1 -- 172.21.15.7:0/2554022459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5634102130 msgr2=0x7f5634197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f5633fff640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5634102130 0x7f5634197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f5633fff640 1 -- 172.21.15.7:0/2554022459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5634104a60 msgr2=0x7f5634197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f5633fff640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5634104a60 0x7f5634197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f5633fff640 1 -- 172.21.15.7:0/2554022459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563419f660 con 0x7f5634107390 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f5633fff640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634107390 0x7f563419eff0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f562c00efd0 tx=0x7f562c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f5630ff9640 1 -- 172.21.15.7:0/2554022459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c003b70 con 0x7f5634107390 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f563a174640 1 -- 172.21.15.7:0/2554022459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563419f950 con 0x7f5634107390 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f5630ff9640 1 -- 172.21.15.7:0/2554022459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f562c003d10 con 0x7f5634107390 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f5630ff9640 1 -- 172.21.15.7:0/2554022459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0109f0 con 0x7f5634107390 2024-03-26T08:58:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.082+0000 7f563a174640 1 -- 172.21.15.7:0/2554022459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563419fe50 con 0x7f5634107390 2024-03-26T08:58:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.084+0000 7f5630ff9640 1 -- 172.21.15.7:0/2554022459 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f562c004020 con 0x7f5634107390 2024-03-26T08:58:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.084+0000 7f563ce36740 1 Processor -- start 2024-03-26T08:58:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.084+0000 7f563ce36740 1 -- start start 2024-03-26T08:58:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.084+0000 7f563ce36740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e69dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.084+0000 7f563ce36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f563ce36740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f563ce36740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55944e660610 con 0x55944e6a31f0 2024-03-26T08:58:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f563ce36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55944e6033c0 con 0x55944e69e080 2024-03-26T08:58:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f563ce36740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55944e5abd60 con 0x55944e69d690 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5630ff9640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f561c0a2800 0x7f561c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e69dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f56337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f56337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34042/0 (socket says 172.21.15.7:34042) 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6a5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53270/0 (socket says 172.21.15.7:53270) 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f56337fe640 1 -- 172.21.15.7:0/742355683 learned_addr learned my addr 172.21.15.7:0/742355683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5633fff640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f561c0a2800 0x7f561c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5630ff9640 1 -- 172.21.15.7:0/2554022459 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f562c0cef10 con 0x7f5634107390 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f56337fe640 1 -- 172.21.15.7:0/742355683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 msgr2=0x55944e69dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f56337fe640 1 --2- 172.21.15.7:0/742355683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e69dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f56337fe640 1 -- 172.21.15.7:0/742355683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 msgr2=0x55944e6a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f56337fe640 1 --2- 172.21.15.7:0/742355683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f56337fe640 1 -- 172.21.15.7:0/742355683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55944e3173f0 con 0x55944e69e080 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f56337fe640 1 --2- 172.21.15.7:0/742355683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a2b80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5624009900 tx=0x7f5624009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=296d3237da439855 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5633fff640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f561c0a2800 0x7f561c0a4cc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f562c005c40 tx=0x7f562c06d4f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5619ffb640 1 -- 172.21.15.7:0/742355683 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624005d50 con 0x55944e69e080 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5619ffb640 1 -- 172.21.15.7:0/742355683 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f562402cb80 con 0x55944e69e080 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.085+0000 7f5619ffb640 1 -- 172.21.15.7:0/742355683 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624004920 con 0x55944e69e080 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 -- 172.21.15.7:0/742355683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 msgr2=0x55944e6a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 --2- 172.21.15.7:0/742355683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a2b80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5624009900 tx=0x7f5624009560 comp rx=0 tx=0).stop 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 -- 172.21.15.7:0/742355683 shutdown_connections 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 --2- 172.21.15.7:0/742355683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 --2- 172.21.15.7:0/742355683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a2b80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 --2- 172.21.15.7:0/742355683 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e69dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 -- 172.21.15.7:0/742355683 >> 172.21.15.7:0/742355683 conn(0x55944e698c00 msgr2=0x55944e699000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 -- 172.21.15.7:0/742355683 shutdown_connections 2024-03-26T08:58:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 -- 172.21.15.7:0/742355683 wait complete. 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.086+0000 7f563ce36740 1 -- start start 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f563ce36740 1 Processor -- start 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f563ce36740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e6a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f563ce36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f563ce36740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f563ce36740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55944e5bf240 con 0x55944e6a31f0 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f563ce36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55944e5ff460 con 0x55944e69e080 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f563ce36740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55944e6a6140 con 0x55944e69d690 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f563ce36740 10 client.0 _ll_register_callbacks cb 0x55944e5bec10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53282/0 (socket says 172.21.15.7:53282) 2024-03-26T08:58:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f56337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f5632ffd640 1 -- 172.21.15.7:0/2606314499 learned_addr learned my addr 172.21.15.7:0/2606314499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f5632ffd640 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e6a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f56337fe640 1 -- 172.21.15.7:0/2606314499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 msgr2=0x55944e6a7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f56337fe640 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e6a7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f56337fe640 1 -- 172.21.15.7:0/2606314499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 msgr2=0x55944e6aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f56337fe640 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f56337fe640 1 -- 172.21.15.7:0/2606314499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55944e3e17d0 con 0x55944e69e080 2024-03-26T08:58:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.087+0000 7f5632ffd640 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f56337fe640 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a7f40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55944e699b70 tx=0x7f562402c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e787409b23472dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f5612ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f5612ffd640 1 -- 172.21.15.7:0/2606314499 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624005650 con 0x55944e69e080 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f5612ffd640 1 -- 172.21.15.7:0/2606314499 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5624003800 con 0x55944e69e080 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f563ce36740 1 -- 172.21.15.7:0/2606314499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f562402cb80 con 0x55944e69e080 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f5612ffd640 1 -- 172.21.15.7:0/2606314499 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5624005240 con 0x55944e69e080 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f563ce36740 10 client.29297 fetch_fsmap learned FSMap version 25 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f563ce36740 1 -- 172.21.15.7:0/2606314499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55944e6b22f0 con 0x55944e69e080 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f5612ffd640 1 -- 172.21.15.7:0/2606314499 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f562402cb80 con 0x55944e69e080 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f5612ffd640 1 -- 172.21.15.7:0/2606314499 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f5624004e20 con 0x55944e69e080 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f563ce36740 10 client.29297 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f563ce36740 10 client.29297 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f563ce36740 20 client.29297 populate_metadata read hostname 'smithi007' 2024-03-26T08:58:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f563ce36740 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55944e6b3640 0x55944e6b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f563ce36740 4 client.29297 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f563ce36740 1 -- 172.21.15.7:0/2606314499 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55944e610900 con 0x55944e6b3640 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.088+0000 7f5632ffd640 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55944e6b3640 0x55944e6b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.089+0000 7f5612ffd640 10 client.29297 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.089+0000 7f5632ffd640 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55944e6b3640 0x55944e6b5a30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=16fb047017d4f199 server_cookie=deca3a646259c391 in_seq=0 out_seq=0 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.089+0000 7f5612ffd640 1 -- 172.21.15.7:0/2606314499 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55944e610900 con 0x55944e6b3640 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.089+0000 7f5612ffd640 10 client.29297 handle_command_reply: tid=0 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.089+0000 7f563ce36740 1 client.29297 shutdown 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 -- 172.21.15.7:0/2606314499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 msgr2=0x55944e6a7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a7f40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55944e699b70 tx=0x7f562402c330 comp rx=0 tx=0).stop 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 -- 172.21.15.7:0/2606314499 shutdown_connections 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55944e6b3640 0x55944e6b5a30 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e6aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f5612ffd640 0 client.29297 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:58:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69e080 0x55944e6a7f40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 --2- 172.21.15.7:0/2606314499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e6a7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 -- 172.21.15.7:0/2606314499 >> 172.21.15.7:0/2606314499 conn(0x55944e698c00 msgr2=0x55944e697e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 -- 172.21.15.7:0/2606314499 shutdown_connections 2024-03-26T08:58:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 1 -- 172.21.15.7:0/2606314499 wait complete. 2024-03-26T08:58:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.090+0000 7f563ce36740 20 client.29297 trim_cache size 0 max 16384 2024-03-26T08:58:09.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f563ce36740 1 Processor -- start 2024-03-26T08:58:09.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f563ce36740 1 -- start start 2024-03-26T08:58:09.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f563ce36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69d690 0x55944e697590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f563ce36740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69e080 0x55944e6aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f563ce36740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e697ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f563ce36740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341aaf30 con 0x55944e6a31f0 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f563ce36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341a8a30 con 0x55944e69d690 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f563ce36740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341a6ab0 con 0x55944e69e080 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e697ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f56337fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69e080 0x55944e6aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e697ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53284/0 (socket says 172.21.15.7:53284) 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f5632ffd640 1 -- 172.21.15.7:0/1007694582 learned_addr learned my addr 172.21.15.7:0/1007694582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.102+0000 7f5632ffd640 1 --2- 172.21.15.7:0/1007694582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69d690 0x55944e697590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f56337fe640 1 -- 172.21.15.7:0/1007694582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69d690 msgr2=0x55944e697590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f56337fe640 1 --2- 172.21.15.7:0/1007694582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69d690 0x55944e697590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f56337fe640 1 -- 172.21.15.7:0/1007694582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 msgr2=0x55944e697ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f56337fe640 1 --2- 172.21.15.7:0/1007694582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e697ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f56337fe640 1 -- 172.21.15.7:0/1007694582 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55944e6a6280 con 0x55944e69e080 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f5632ffd640 1 --2- 172.21.15.7:0/1007694582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e697ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:09.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f56337fe640 1 --2- 172.21.15.7:0/1007694582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69e080 0x55944e6aef40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f562402c7b0 tx=0x7f5624005e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fa1503a90b17f48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f56197fa640 1 -- 172.21.15.7:0/1007694582 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56240053e0 con 0x55944e69e080 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f56197fa640 1 -- 172.21.15.7:0/1007694582 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f562404d070 con 0x55944e69e080 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f56197fa640 1 -- 172.21.15.7:0/1007694582 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624003800 con 0x55944e69e080 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f563ce36740 1 -- 172.21.15.7:0/1007694582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69e080 msgr2=0x55944e6aef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f563ce36740 1 --2- 172.21.15.7:0/1007694582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69e080 0x55944e6aef40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f562402c7b0 tx=0x7f5624005e10 comp rx=0 tx=0).stop 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f563ce36740 1 -- 172.21.15.7:0/1007694582 shutdown_connections 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f563ce36740 1 --2- 172.21.15.7:0/1007694582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e6a31f0 0x55944e697ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f563ce36740 1 --2- 172.21.15.7:0/1007694582 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69e080 0x55944e6aef40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f563ce36740 1 --2- 172.21.15.7:0/1007694582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e69d690 0x55944e697590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f563ce36740 1 -- 172.21.15.7:0/1007694582 >> 172.21.15.7:0/1007694582 conn(0x55944e698c00 msgr2=0x55944e6a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f563ce36740 1 -- 172.21.15.7:0/1007694582 shutdown_connections 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.103+0000 7f563ce36740 1 -- 172.21.15.7:0/1007694582 wait complete. 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f563ce36740 1 -- start start 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f563ce36740 1 Processor -- start 2024-03-26T08:58:09.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f563ce36740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e6acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f563ce36740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e69e080 0x55944e6ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f563ce36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e6a31f0 0x55944e6ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f563ce36740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341a70b0 con 0x55944e69e080 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f563ce36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341a83d0 con 0x55944e6a31f0 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f563ce36740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341a7710 con 0x55944e69d690 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f563ce36740 10 client.0 _ll_register_callbacks cb 0x7f5624009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f56337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e69e080 0x55944e6ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f5632ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e6acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f56337fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e69e080 0x55944e6ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53298/0 (socket says 172.21.15.7:53298) 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f56337fe640 1 -- 172.21.15.7:0/3185641983 learned_addr learned my addr 172.21.15.7:0/3185641983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.104+0000 7f5632ffd640 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e6a31f0 0x55944e6ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f56337fe640 1 -- 172.21.15.7:0/3185641983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 msgr2=0x55944e6acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f56337fe640 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e6acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f56337fe640 1 -- 172.21.15.7:0/3185641983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e6a31f0 msgr2=0x55944e6ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f56337fe640 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e6a31f0 0x55944e6ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f56337fe640 1 -- 172.21.15.7:0/3185641983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55944e6ae880 con 0x55944e69e080 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f5632ffd640 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e6a31f0 0x55944e6ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f56337fe640 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e69e080 0x55944e6ad280 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f562402c420 tx=0x7f562402cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d427707f821ed2a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f5612ffd640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f5612ffd640 1 -- 172.21.15.7:0/3185641983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624034530 con 0x55944e69e080 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f5612ffd640 1 -- 172.21.15.7:0/3185641983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f56240346d0 con 0x55944e69e080 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f563ce36740 1 -- 172.21.15.7:0/3185641983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f562404d070 con 0x55944e69e080 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f5612ffd640 1 -- 172.21.15.7:0/3185641983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56240348d0 con 0x55944e69e080 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f563ce36740 10 client.19548 fetch_fsmap learned FSMap version 25 2024-03-26T08:58:09.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f563ce36740 1 -- 172.21.15.7:0/3185641983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55944e70f490 con 0x55944e69e080 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f5612ffd640 1 -- 172.21.15.7:0/3185641983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f562404d070 con 0x55944e69e080 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f5612ffd640 1 -- 172.21.15.7:0/3185641983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f562405b070 con 0x55944e69e080 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f563ce36740 10 client.19548 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f563ce36740 10 client.19548 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f563ce36740 20 client.19548 populate_metadata read hostname 'smithi007' 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.105+0000 7f563ce36740 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55944e70fc00 0x55944e712020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 4 client.19548 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 -- 172.21.15.7:0/3185641983 --> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55944e6a77f0 con 0x55944e70fc00 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f5632ffd640 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55944e70fc00 0x55944e712020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f5632ffd640 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55944e70fc00 0x55944e712020 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=595522f563811db2 server_cookie=726c433379b08500 in_seq=0 out_seq=0 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f5612ffd640 10 client.19548 ms_handle_connect on v2:172.21.15.44:6836/1604095881 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f5612ffd640 1 -- 172.21.15.7:0/3185641983 <== mds.2 v2:172.21.15.44:6836/1604095881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55944e6a77f0 con 0x55944e70fc00 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f5612ffd640 10 client.19548 handle_command_reply: tid=0 2024-03-26T08:58:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 client.19548 shutdown 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 -- 172.21.15.7:0/3185641983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e69e080 msgr2=0x55944e6ad280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e69e080 0x55944e6ad280 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f562402c420 tx=0x7f562402cc10 comp rx=0 tx=0).stop 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 -- 172.21.15.7:0/3185641983 shutdown_connections 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.44:6836/1604095881,v1:172.21.15.44:6837/1604095881] conn(0x55944e70fc00 0x55944e712020 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55944e6a31f0 0x55944e6ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55944e69e080 0x55944e6ad280 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 --2- 172.21.15.7:0/3185641983 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55944e69d690 0x55944e6acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f5612ffd640 0 client.19548 ms_handle_reset on v2:172.21.15.44:6836/1604095881 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 -- 172.21.15.7:0/3185641983 >> 172.21.15.7:0/3185641983 conn(0x55944e698c00 msgr2=0x55944e69be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 -- 172.21.15.7:0/3185641983 shutdown_connections 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.106+0000 7f563ce36740 1 -- 172.21.15.7:0/3185641983 wait complete. 2024-03-26T08:58:09.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563ce36740 20 client.19548 trim_cache size 0 max 16384 2024-03-26T08:58:09.113 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:58:09.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 -- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f561c0a2800 msgr2=0x7f561c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f561c0a2800 0x7f561c0a4cc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f562c005c40 tx=0x7f562c06d4f0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 -- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634107390 msgr2=0x7f563419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634107390 0x7f563419eff0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f562c00efd0 tx=0x7f562c00c6a0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 -- 172.21.15.7:0/2554022459 shutdown_connections 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f561c0a2800 0x7f561c0a4cc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5634107390 0x7f563419eff0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5634104a60 0x7f5634197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 --2- 172.21.15.7:0/2554022459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5634102130 0x7f5634197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 -- 172.21.15.7:0/2554022459 >> 172.21.15.7:0/2554022459 conn(0x7f56340fbb00 msgr2=0x7f5634102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 -- 172.21.15.7:0/2554022459 shutdown_connections 2024-03-26T08:58:09.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.107+0000 7f563a174640 1 -- 172.21.15.7:0/2554022459 wait complete. 2024-03-26T08:58:09.116 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:58:09.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T08:58:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.223+0000 7f3c193c4640 1 -- 172.21.15.7:0/1238142860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c141073a0 msgr2=0x7f3c141097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.223+0000 7f3c193c4640 1 --2- 172.21.15.7:0/1238142860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c141073a0 0x7f3c141097f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009be0 tx=0x7f3c0402bef0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.223+0000 7f3c193c4640 1 -- 172.21.15.7:0/1238142860 shutdown_connections 2024-03-26T08:58:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.223+0000 7f3c193c4640 1 --2- 172.21.15.7:0/1238142860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c141073a0 0x7f3c141097f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.223+0000 7f3c193c4640 1 --2- 172.21.15.7:0/1238142860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c14104a70 0x7f3c14106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.223+0000 7f3c193c4640 1 --2- 172.21.15.7:0/1238142860 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c14102140 0x7f3c14104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.223+0000 7f3c193c4640 1 -- 172.21.15.7:0/1238142860 >> 172.21.15.7:0/1238142860 conn(0x7f3c140fbb30 msgr2=0x7f3c140fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.223+0000 7f3c193c4640 1 -- 172.21.15.7:0/1238142860 shutdown_connections 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.223+0000 7f3c193c4640 1 -- 172.21.15.7:0/1238142860 wait complete. 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c193c4640 1 Processor -- start 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c193c4640 1 -- start start 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c193c4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c14102140 0x7f3c14197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c193c4640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c14104a70 0x7f3c14197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c193c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c141073a0 0x7f3c1419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c193c4640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1410d360 con 0x7f3c14104a70 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c193c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1410d1e0 con 0x7f3c141073a0 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c193c4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1410d4e0 con 0x7f3c14102140 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c14104a70 0x7f3c14197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c14104a70 0x7f3c14197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53318/0 (socket says 172.21.15.7:53318) 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c127fc640 1 -- 172.21.15.7:0/258015838 learned_addr learned my addr 172.21.15.7:0/258015838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c137fe640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c141073a0 0x7f3c1419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.224+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c14102140 0x7f3c14197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3c127fc640 1 -- 172.21.15.7:0/258015838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c14102140 msgr2=0x7f3c14197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3c127fc640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c14102140 0x7f3c14197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3c127fc640 1 -- 172.21.15.7:0/258015838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c141073a0 msgr2=0x7f3c1419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3c127fc640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c141073a0 0x7f3c1419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3c127fc640 1 -- 172.21.15.7:0/258015838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1419f5a0 con 0x7f3c14104a70 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3c127fc640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c14104a70 0x7f3c14197ec0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800e9a0 tx=0x7f3c0800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3bfbfff640 1 -- 172.21.15.7:0/258015838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800cd80 con 0x7f3c14104a70 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3c193c4640 1 -- 172.21.15.7:0/258015838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1419f890 con 0x7f3c14104a70 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3bfbfff640 1 -- 172.21.15.7:0/258015838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c080035f0 con 0x7f3c14104a70 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3bfbfff640 1 -- 172.21.15.7:0/258015838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08010670 con 0x7f3c14104a70 2024-03-26T08:58:09.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.225+0000 7f3c193c4640 1 -- 172.21.15.7:0/258015838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1419fde0 con 0x7f3c14104a70 2024-03-26T08:58:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.227+0000 7f3bfbfff640 1 -- 172.21.15.7:0/258015838 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f3c080037a0 con 0x7f3c14104a70 2024-03-26T08:58:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.227+0000 7f3c1bfaa740 1 Processor -- start 2024-03-26T08:58:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.227+0000 7f3c1bfaa740 1 -- start start 2024-03-26T08:58:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbf4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dbf9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dbfc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810dbb7610 con 0x55810dbf5080 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810db5a3c0 con 0x55810dbf4690 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810db02d60 con 0x55810dbfa1f0 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3bfbfff640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3bfc0a2800 0x7f3bfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c12ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dbf9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c12ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dbf9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53330/0 (socket says 172.21.15.7:53330) 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c12ffd640 1 -- 172.21.15.7:0/1187662202 learned_addr learned my addr 172.21.15.7:0/1187662202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c137fe640 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbf4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c127fc640 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dbfc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c137fe640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3bfc0a2800 0x7f3bfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3bfbfff640 1 -- 172.21.15.7:0/258015838 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c080cecd0 con 0x7f3c14104a70 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c12ffd640 1 -- 172.21.15.7:0/1187662202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 msgr2=0x55810dbfc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dbfc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c12ffd640 1 -- 172.21.15.7:0/1187662202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 msgr2=0x55810dbf4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbf4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c12ffd640 1 -- 172.21.15.7:0/1187662202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810d86e3f0 con 0x55810dbf5080 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c137fe640 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dbf9b80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009d20 tx=0x7f3c0c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2332ace01b1d4027 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3c137fe640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3bfc0a2800 0x7f3bfc0a4cc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3c04004ab0 tx=0x7f3c04004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3bf97fa640 1 -- 172.21.15.7:0/1187662202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c01e070 con 0x55810dbf5080 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3bf97fa640 1 -- 172.21.15.7:0/1187662202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c019b10 con 0x55810dbf5080 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.228+0000 7f3bf97fa640 1 -- 172.21.15.7:0/1187662202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c00eb30 con 0x55810dbf5080 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/1187662202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 msgr2=0x55810dbf9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dbf9b80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009d20 tx=0x7f3c0c011810 comp rx=0 tx=0).stop 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/1187662202 shutdown_connections 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dbfc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dbf9b80 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/1187662202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbf4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/1187662202 >> 172.21.15.7:0/1187662202 conn(0x55810dbefc00 msgr2=0x55810dbf0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/1187662202 shutdown_connections 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/1187662202 wait complete. 2024-03-26T08:58:09.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.229+0000 7f3c1bfaa740 1 -- start start 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c1bfaa740 1 Processor -- start 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbfea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbf5080 0x55810dbfef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbfa1f0 0x55810dc05fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810db16240 con 0x55810dbfa1f0 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810db56460 con 0x55810dbf4690 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810dbfd1a0 con 0x55810dbf5080 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c1bfaa740 10 client.0 _ll_register_callbacks cb 0x55810db15c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbfa1f0 0x55810dc05fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c127fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbf5080 0x55810dbfef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c137fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbfa1f0 0x55810dc05fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53332/0 (socket says 172.21.15.7:53332) 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c137fe640 1 -- 172.21.15.7:0/2721808185 learned_addr learned my addr 172.21.15.7:0/2721808185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbfea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c12ffd640 1 -- 172.21.15.7:0/2721808185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbf5080 msgr2=0x55810dbfef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbf5080 0x55810dbfef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c12ffd640 1 -- 172.21.15.7:0/2721808185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbfa1f0 msgr2=0x55810dc05fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbfa1f0 0x55810dc05fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c12ffd640 1 -- 172.21.15.7:0/2721808185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55810d9387d0 con 0x55810dbf4690 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c137fe640 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbfa1f0 0x55810dc05fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbfea20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55810dbf0b70 tx=0x7f3c0c009a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18fbba820934cadb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3bf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.230+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2721808185 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c026070 con 0x55810dbf4690 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2721808185 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c0c004460 con 0x55810dbf4690 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2721808185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c0c019b10 con 0x55810dbf4690 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2721808185 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c0c032050 con 0x55810dbf4690 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c1bfaa740 10 client.29315 fetch_fsmap learned FSMap version 25 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2721808185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55810dc09310 con 0x55810dbf4690 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2721808185 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c019b10 con 0x55810dbf4690 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2721808185 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3c0c0035f0 con 0x55810dbf4690 2024-03-26T08:58:09.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c1bfaa740 10 client.29315 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c1bfaa740 10 client.29315 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c1bfaa740 20 client.29315 populate_metadata read hostname 'smithi007' 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55810dc0a6b0 0x55810dc0cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c1bfaa740 4 client.29315 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2721808185 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55810db67900 con 0x55810dc0a6b0 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.231+0000 7f3c127fc640 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55810dc0a6b0 0x55810dc0cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c127fc640 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55810dc0a6b0 0x55810dc0cae0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1a945c119a03025 server_cookie=d15a3ce5b145444 in_seq=0 out_seq=0 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3bf6ff5640 10 client.29315 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2721808185 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55810db67900 con 0x55810dc0a6b0 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3bf6ff5640 10 client.29315 handle_command_reply: tid=0 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 client.29315 shutdown 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2721808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 msgr2=0x55810dbfea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbfea20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55810dbf0b70 tx=0x7f3c0c009a00 comp rx=0 tx=0).stop 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2721808185 shutdown_connections 2024-03-26T08:58:09.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbfa1f0 0x55810dc05fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55810dc0a6b0 0x55810dc0cae0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbf5080 0x55810dbfef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2721808185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dbfea20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3bf6ff5640 0 client.29315 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:58:09.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2721808185 >> 172.21.15.7:0/2721808185 conn(0x55810dbefc00 msgr2=0x55810dbeee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2721808185 shutdown_connections 2024-03-26T08:58:09.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.232+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2721808185 wait complete. 2024-03-26T08:58:09.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.233+0000 7f3c1bfaa740 20 client.29315 trim_cache size 0 max 16384 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c1bfaa740 1 Processor -- start 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c1bfaa740 1 -- start start 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf4690 0x55810dbeeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf5080 0x55810dc05f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dc113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141aafb0 con 0x55810dbf4690 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141a8ab0 con 0x55810dbf5080 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141a6b30 con 0x55810dbfa1f0 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c12ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf4690 0x55810dbeeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c12ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf4690 0x55810dbeeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53342/0 (socket says 172.21.15.7:53342) 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c12ffd640 1 -- 172.21.15.7:0/147731770 learned_addr learned my addr 172.21.15.7:0/147731770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c127fc640 1 --2- 172.21.15.7:0/147731770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf5080 0x55810dc05f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.245+0000 7f3c137fe640 1 --2- 172.21.15.7:0/147731770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dc113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c12ffd640 1 -- 172.21.15.7:0/147731770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 msgr2=0x55810dc113a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/147731770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dc113a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c12ffd640 1 -- 172.21.15.7:0/147731770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf5080 msgr2=0x55810dc05f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/147731770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf5080 0x55810dc05f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c12ffd640 1 -- 172.21.15.7:0/147731770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810dbef550 con 0x55810dbf4690 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/147731770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf4690 0x55810dbeeca0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0039c0 tx=0x7f3c0c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c043e40a5154e1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3bf8ff9640 1 -- 172.21.15.7:0/147731770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c026030 con 0x55810dbf4690 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3bf8ff9640 1 -- 172.21.15.7:0/147731770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c0035f0 con 0x55810dbf4690 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3bf8ff9640 1 -- 172.21.15.7:0/147731770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c01e040 con 0x55810dbf4690 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/147731770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf4690 msgr2=0x55810dbeeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/147731770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf4690 0x55810dbeeca0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0039c0 tx=0x7f3c0c005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/147731770 shutdown_connections 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/147731770 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dc113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/147731770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf5080 0x55810dc05f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/147731770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf4690 0x55810dbeeca0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.246+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/147731770 >> 172.21.15.7:0/147731770 conn(0x55810dbefc00 msgr2=0x55810dbee590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/147731770 shutdown_connections 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/147731770 wait complete. 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 -- start start 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 Processor -- start 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dc03ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dc040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dc050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141a7130 con 0x55810dbf5080 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141a8450 con 0x55810dbf4690 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c141a7790 con 0x55810dbfa1f0 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c1bfaa740 10 client.0 _ll_register_callbacks cb 0x7f3c080046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dc040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c127fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dc040e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53352/0 (socket says 172.21.15.7:53352) 2024-03-26T08:58:09.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c127fc640 1 -- 172.21.15.7:0/2342513903 learned_addr learned my addr 172.21.15.7:0/2342513903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dc03ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c137fe640 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dc050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c127fc640 1 -- 172.21.15.7:0/2342513903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 msgr2=0x55810dc050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c127fc640 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dc050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c127fc640 1 -- 172.21.15.7:0/2342513903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 msgr2=0x55810dc03ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c127fc640 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dc03ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.247+0000 7f3c127fc640 1 -- 172.21.15.7:0/2342513903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55810dbf0970 con 0x55810dbf5080 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c127fc640 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dc040e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3c0809b600 tx=0x7f3c0806b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be92af822645616 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3bf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2342513903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c080bb040 con 0x55810dbf5080 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2342513903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c080cc8e0 con 0x55810dbf5080 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2342513903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c0c0035f0 con 0x55810dbf5080 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2342513903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c0809b670 con 0x55810dbf5080 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c1bfaa740 10 client.19566 fetch_fsmap learned FSMap version 25 2024-03-26T08:58:09.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2342513903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55810dc66330 con 0x55810dbf5080 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2342513903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c0035f0 con 0x55810dbf5080 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2342513903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f3c08097ce0 con 0x55810dbf5080 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c1bfaa740 10 client.19566 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c1bfaa740 10 client.19566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c1bfaa740 20 client.19566 populate_metadata read hostname 'smithi007' 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55810dc66c40 0x55810dc69060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c1bfaa740 4 client.19566 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2342513903 --> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55810db61c50 con 0x55810dc66c40 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.248+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55810dc66c40 0x55810dc69060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3bf6ff5640 10 client.19566 ms_handle_connect on v2:172.21.15.44:6838/2441928071 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c12ffd640 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55810dc66c40 0x55810dc69060 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=547b8f20a5f65e26 server_cookie=6442efeabc20085f in_seq=0 out_seq=0 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3bf6ff5640 1 -- 172.21.15.7:0/2342513903 <== mds.1 v2:172.21.15.44:6838/2441928071 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55810db61c50 con 0x55810dc66c40 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3bf6ff5640 10 client.19566 handle_command_reply: tid=0 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 client.19566 shutdown 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2342513903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 msgr2=0x55810dc040e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dc040e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f3c0809b600 tx=0x7f3c0806b190 comp rx=0 tx=0).stop 2024-03-26T08:58:09.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2342513903 shutdown_connections 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.44:6838/2441928071,v1:172.21.15.44:6839/2441928071] conn(0x55810dc66c40 0x55810dc69060 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55810dbfa1f0 0x55810dc050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55810dbf5080 0x55810dc040e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3bf6ff5640 0 client.19566 ms_handle_reset on v2:172.21.15.44:6838/2441928071 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 --2- 172.21.15.7:0/2342513903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55810dbf4690 0x55810dc03ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2342513903 >> 172.21.15.7:0/2342513903 conn(0x55810dbefc00 msgr2=0x55810dbeee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2342513903 shutdown_connections 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 1 -- 172.21.15.7:0/2342513903 wait complete. 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.249+0000 7f3c1bfaa740 20 client.19566 trim_cache size 0 max 16384 2024-03-26T08:58:09.254 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:58:09.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 -- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3bfc0a2800 msgr2=0x7f3bfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3bfc0a2800 0x7f3bfc0a4cc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3c04004ab0 tx=0x7f3c04004a40 comp rx=0 tx=0).stop 2024-03-26T08:58:09.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 -- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c14104a70 msgr2=0x7f3c14197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c14104a70 0x7f3c14197ec0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3c0800e9a0 tx=0x7f3c0800ee60 comp rx=0 tx=0).stop 2024-03-26T08:58:09.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 -- 172.21.15.7:0/258015838 shutdown_connections 2024-03-26T08:58:09.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f3bfc0a2800 0x7f3bfc0a4cc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c141073a0 0x7f3c1419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c14104a70 0x7f3c14197ec0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 --2- 172.21.15.7:0/258015838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3c14102140 0x7f3c14197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 -- 172.21.15.7:0/258015838 >> 172.21.15.7:0/258015838 conn(0x7f3c140fbb30 msgr2=0x7f3c14102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 -- 172.21.15.7:0/258015838 shutdown_connections 2024-03-26T08:58:09.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.250+0000 7f3c193c4640 1 -- 172.21.15.7:0/258015838 wait complete. 2024-03-26T08:58:09.259 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T08:58:09.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T08:58:09.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 -- 172.21.15.7:0/3694865677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 msgr2=0x7f9664102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 --2- 172.21.15.7:0/3694865677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 0x7f9664102d20 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9658009900 tx=0x7f965802bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 -- 172.21.15.7:0/3694865677 shutdown_connections 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 --2- 172.21.15.7:0/3694865677 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9664103260 0x7f9664109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 --2- 172.21.15.7:0/3694865677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 0x7f9664102d20 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 --2- 172.21.15.7:0/3694865677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96641016b0 0x7f9664101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 -- 172.21.15.7:0/3694865677 >> 172.21.15.7:0/3694865677 conn(0x7f96640fcc20 msgr2=0x7f96640ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 -- 172.21.15.7:0/3694865677 shutdown_connections 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 -- 172.21.15.7:0/3694865677 wait complete. 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.368+0000 7f966c0b2640 1 Processor -- start 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966c0b2640 1 -- start start 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966c0b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96641016b0 0x7f96641979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966c0b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 0x7f9664197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966c0b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9664103260 0x7f966419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966c0b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966410d330 con 0x7f96641028a0 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966c0b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966410d1b0 con 0x7f96641016b0 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966c0b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966410d4b0 con 0x7f9664103260 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966964d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 0x7f9664197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966964d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 0x7f9664197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53376/0 (socket says 172.21.15.7:53376) 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966964d640 1 -- 172.21.15.7:0/176515560 learned_addr learned my addr 172.21.15.7:0/176515560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f9669e4e640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96641016b0 0x7f96641979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966a64f640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9664103260 0x7f966419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966964d640 1 -- 172.21.15.7:0/176515560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9664103260 msgr2=0x7f966419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966964d640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9664103260 0x7f966419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966964d640 1 -- 172.21.15.7:0/176515560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96641016b0 msgr2=0x7f96641979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966964d640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96641016b0 0x7f96641979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966964d640 1 -- 172.21.15.7:0/176515560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966419f5f0 con 0x7f96641028a0 2024-03-26T08:58:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f9669e4e640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96641016b0 0x7f96641979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966964d640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 0x7f9664197f10 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9658009230 tx=0x7f965802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f9662ffd640 1 -- 172.21.15.7:0/176515560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658041070 con 0x7f96641028a0 2024-03-26T08:58:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f9662ffd640 1 -- 172.21.15.7:0/176515560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9658004450 con 0x7f96641028a0 2024-03-26T08:58:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f966c0b2640 1 -- 172.21.15.7:0/176515560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966419f880 con 0x7f96641028a0 2024-03-26T08:58:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.369+0000 7f9662ffd640 1 -- 172.21.15.7:0/176515560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658003e20 con 0x7f96641028a0 2024-03-26T08:58:09.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.370+0000 7f966c0b2640 1 -- 172.21.15.7:0/176515560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966419fd70 con 0x7f96641028a0 2024-03-26T08:58:09.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f9662ffd640 1 -- 172.21.15.7:0/176515560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f9658044070 con 0x7f96641028a0 2024-03-26T08:58:09.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966ed7f740 1 Processor -- start 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966ed7f740 1 -- start start 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88873ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88878b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b8887b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966ed7f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b88836610 con 0x561b888791f0 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966ed7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b887d93c0 con 0x561b88873690 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966ed7f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b88781d60 con 0x561b88874080 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f9662ffd640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f964c0a2800 0x7f964c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966964d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b8887b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f9669e4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88878b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966a64f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88873ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f9669e4e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88878b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:34350/0 (socket says 172.21.15.7:34350) 2024-03-26T08:58:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f9669e4e640 1 -- 172.21.15.7:0/2690041424 learned_addr learned my addr 172.21.15.7:0/2690041424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.372+0000 7f966a64f640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f964c0a2800 0x7f964c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f9662ffd640 1 -- 172.21.15.7:0/176515560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96580f05f0 con 0x7f96641028a0 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f9669e4e640 1 -- 172.21.15.7:0/2690041424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 msgr2=0x561b88873ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f9669e4e640 1 --2- 172.21.15.7:0/2690041424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88873ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f9669e4e640 1 -- 172.21.15.7:0/2690041424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 msgr2=0x561b8887b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f9669e4e640 1 --2- 172.21.15.7:0/2690041424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b8887b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f9669e4e640 1 -- 172.21.15.7:0/2690041424 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b884ed3f0 con 0x561b88874080 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f9669e4e640 1 --2- 172.21.15.7:0/2690041424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88878b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9654011820 tx=0x7f9654011ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaee658266dddbbf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f966a64f640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f964c0a2800 0x7f964c0a4cc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x561b888748e0 tx=0x7f965c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f964bfff640 1 -- 172.21.15.7:0/2690041424 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654011220 con 0x561b88874080 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f964bfff640 1 -- 172.21.15.7:0/2690041424 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9654019430 con 0x561b88874080 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.373+0000 7f964bfff640 1 -- 172.21.15.7:0/2690041424 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654010630 con 0x561b88874080 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 -- 172.21.15.7:0/2690041424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 msgr2=0x561b88878b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 --2- 172.21.15.7:0/2690041424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88878b80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9654011820 tx=0x7f9654011ce0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 -- 172.21.15.7:0/2690041424 shutdown_connections 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 --2- 172.21.15.7:0/2690041424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b8887b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 --2- 172.21.15.7:0/2690041424 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88878b80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 --2- 172.21.15.7:0/2690041424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88873ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 -- 172.21.15.7:0/2690041424 >> 172.21.15.7:0/2690041424 conn(0x561b8886ec00 msgr2=0x561b8886f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 -- 172.21.15.7:0/2690041424 shutdown_connections 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 -- 172.21.15.7:0/2690041424 wait complete. 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 -- start start 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.374+0000 7f966ed7f740 1 Processor -- start 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88873690 0x561b8887da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b8887df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b888791f0 0x561b88884fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966ed7f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b88795240 con 0x561b88873690 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966ed7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b887d5460 con 0x561b888791f0 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966ed7f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8887c1a0 con 0x561b88874080 2024-03-26T08:58:09.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966ed7f740 10 client.0 _ll_register_callbacks cb 0x561b88794c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f9669e4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88873690 0x561b8887da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f9669e4e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88873690 0x561b8887da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53392/0 (socket says 172.21.15.7:53392) 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f9669e4e640 1 -- 172.21.15.7:0/3361727084 learned_addr learned my addr 172.21.15.7:0/3361727084 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966964d640 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b8887df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966a64f640 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b888791f0 0x561b88884fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f9669e4e640 1 -- 172.21.15.7:0/3361727084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 msgr2=0x561b8887df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f9669e4e640 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b8887df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f9669e4e640 1 -- 172.21.15.7:0/3361727084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b888791f0 msgr2=0x561b88884fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f9669e4e640 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b888791f0 0x561b88884fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f9669e4e640 1 -- 172.21.15.7:0/3361727084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b885b77d0 con 0x561b88873690 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f9669e4e640 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88873690 0x561b8887da20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9654006010 tx=0x7f965400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae74fdfe4bcd6735 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f96497fa640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f96497fa640 1 -- 172.21.15.7:0/3361727084 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654027020 con 0x561b88873690 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f96497fa640 1 -- 172.21.15.7:0/3361727084 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9654004a40 con 0x561b88873690 2024-03-26T08:58:09.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966ed7f740 1 -- 172.21.15.7:0/3361727084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9654019430 con 0x561b88873690 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f96497fa640 1 -- 172.21.15.7:0/3361727084 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9654004c60 con 0x561b88873690 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.375+0000 7f966ed7f740 10 client.19578 fetch_fsmap learned FSMap version 25 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f966ed7f740 1 -- 172.21.15.7:0/3361727084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561b88888310 con 0x561b88873690 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f96497fa640 1 -- 172.21.15.7:0/3361727084 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9654019430 con 0x561b88873690 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f96497fa640 1 -- 172.21.15.7:0/3361727084 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9654004040 con 0x561b88873690 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f966ed7f740 10 client.19578 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f966ed7f740 10 client.19578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f966ed7f740 20 client.19578 populate_metadata read hostname 'smithi007' 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f966ed7f740 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b888897d0 0x561b8888bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f966ed7f740 4 client.19578 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f966ed7f740 1 -- 172.21.15.7:0/3361727084 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b887e6900 con 0x561b888897d0 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f966964d640 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b888897d0 0x561b8888bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f96497fa640 10 client.19578 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:58:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.376+0000 7f966964d640 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b888897d0 0x561b8888bbe0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e126b38ee3158bc6 server_cookie=466519df1577e644 in_seq=0 out_seq=0 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f96497fa640 1 -- 172.21.15.7:0/3361727084 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561b887e6900 con 0x561b888897d0 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f96497fa640 10 client.19578 handle_command_reply: tid=0 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 client.19578 shutdown 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 -- 172.21.15.7:0/3361727084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88873690 msgr2=0x561b8887da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88873690 0x561b8887da20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9654006010 tx=0x7f965400fd10 comp rx=0 tx=0).stop 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 -- 172.21.15.7:0/3361727084 shutdown_connections 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b888897d0 0x561b8888bbe0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b888791f0 0x561b88884fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f96497fa640 0 client.19578 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b8887df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 --2- 172.21.15.7:0/3361727084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88873690 0x561b8887da20 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 -- 172.21.15.7:0/3361727084 >> 172.21.15.7:0/3361727084 conn(0x561b8886ec00 msgr2=0x561b8886de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 -- 172.21.15.7:0/3361727084 shutdown_connections 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 1 -- 172.21.15.7:0/3361727084 wait complete. 2024-03-26T08:58:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.377+0000 7f966ed7f740 20 client.19578 trim_cache size 0 max 16384 2024-03-26T08:58:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966ed7f740 1 Processor -- start 2024-03-26T08:58:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966ed7f740 1 -- start start 2024-03-26T08:58:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88872930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88884f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b88876520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966ed7f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96641ab020 con 0x561b888791f0 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966ed7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96641a8b20 con 0x561b88873690 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966ed7f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96641a6ba0 con 0x561b88874080 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966a64f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b88876520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966a64f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b88876520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53396/0 (socket says 172.21.15.7:53396) 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966a64f640 1 -- 172.21.15.7:0/222285388 learned_addr learned my addr 172.21.15.7:0/222285388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966964d640 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88884f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f9669e4e640 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88872930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966a64f640 1 -- 172.21.15.7:0/222285388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 msgr2=0x561b88884f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966a64f640 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88884f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966a64f640 1 -- 172.21.15.7:0/222285388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 msgr2=0x561b88872930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966a64f640 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88872930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966a64f640 1 -- 172.21.15.7:0/222285388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b8886ea30 con 0x561b888791f0 2024-03-26T08:58:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966964d640 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88884f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.390+0000 7f966a64f640 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b88876520 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f965c009b50 tx=0x7f965c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8440eaef7328560 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f964b7fe640 1 -- 172.21.15.7:0/222285388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c0035f0 con 0x561b888791f0 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f964b7fe640 1 -- 172.21.15.7:0/222285388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f965c003c70 con 0x561b888791f0 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f964b7fe640 1 -- 172.21.15.7:0/222285388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c022c90 con 0x561b888791f0 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 -- 172.21.15.7:0/222285388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 msgr2=0x561b88876520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b88876520 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f965c009b50 tx=0x7f965c00e000 comp rx=0 tx=0).stop 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 -- 172.21.15.7:0/222285388 shutdown_connections 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b888791f0 0x561b88876520 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b88874080 0x561b88884f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 --2- 172.21.15.7:0/222285388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88872930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 -- 172.21.15.7:0/222285388 >> 172.21.15.7:0/222285388 conn(0x561b8886ec00 msgr2=0x561b8886de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 -- 172.21.15.7:0/222285388 shutdown_connections 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 -- 172.21.15.7:0/222285388 wait complete. 2024-03-26T08:58:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.391+0000 7f966ed7f740 1 -- start start 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966ed7f740 1 Processor -- start 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88882b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88874080 0x561b888830b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966ed7f740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b888791f0 0x561b888840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966ed7f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96641a71a0 con 0x561b88874080 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966ed7f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96641a84c0 con 0x561b88873690 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966ed7f740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96641a7800 con 0x561b888791f0 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966ed7f740 10 client.0 _ll_register_callbacks cb 0x7f96540116a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966964d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88874080 0x561b888830b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f9669e4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88882b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966964d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88874080 0x561b888830b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53398/0 (socket says 172.21.15.7:53398) 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966964d640 1 -- 172.21.15.7:0/4048540823 learned_addr learned my addr 172.21.15.7:0/4048540823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966a64f640 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b888791f0 0x561b888840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f9669e4e640 1 -- 172.21.15.7:0/4048540823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b888791f0 msgr2=0x561b888840c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f9669e4e640 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b888791f0 0x561b888840c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f9669e4e640 1 -- 172.21.15.7:0/4048540823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88874080 msgr2=0x561b888830b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f9669e4e640 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88874080 0x561b888830b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f9669e4e640 1 -- 172.21.15.7:0/4048540823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b888846b0 con 0x561b88873690 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.392+0000 7f966964d640 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88874080 0x561b888830b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f9669e4e640 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88882b70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9654004400 tx=0x7f9654004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5de6e2cc4acd9b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f96497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f96497fa640 1 -- 172.21.15.7:0/4048540823 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965403a020 con 0x561b88873690 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f96497fa640 1 -- 172.21.15.7:0/4048540823 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9654011280 con 0x561b88873690 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966ed7f740 1 -- 172.21.15.7:0/4048540823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f965c003c70 con 0x561b88873690 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f96497fa640 1 -- 172.21.15.7:0/4048540823 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9654027040 con 0x561b88873690 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966ed7f740 10 client.29339 fetch_fsmap learned FSMap version 25 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966ed7f740 1 -- 172.21.15.7:0/4048540823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561b888e5580 con 0x561b88873690 2024-03-26T08:58:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f96497fa640 1 -- 172.21.15.7:0/4048540823 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f965c003c70 con 0x561b88873690 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f96497fa640 1 -- 172.21.15.7:0/4048540823 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) v1 ==== 7918+0+0 (secure 0 0 0) 0x7f9654019bb0 con 0x561b88873690 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966ed7f740 10 client.29339 fetch_fsmap finished waiting for FSMap version 25 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966ed7f740 10 client.29339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966ed7f740 20 client.29339 populate_metadata read hostname 'smithi007' 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966ed7f740 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b888e5d90 0x561b888e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966ed7f740 4 client.29339 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966ed7f740 1 -- 172.21.15.7:0/4048540823 --> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b88875090 con 0x561b888e5d90 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.393+0000 7f966964d640 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b888e5d90 0x561b888e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f96497fa640 10 client.29339 ms_handle_connect on v2:172.21.15.16:6838/3011473533 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966964d640 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b888e5d90 0x561b888e81b0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a6605f8d6c29d9 server_cookie=ee1df2b527de8db4 in_seq=0 out_seq=0 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f96497fa640 1 -- 172.21.15.7:0/4048540823 <== mds.0 v2:172.21.15.16:6838/3011473533 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b88875090 con 0x561b888e5d90 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f96497fa640 10 client.29339 handle_command_reply: tid=0 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 client.29339 shutdown 2024-03-26T08:58:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 -- 172.21.15.7:0/4048540823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 msgr2=0x561b88882b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88882b70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9654004400 tx=0x7f9654004be0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 -- 172.21.15.7:0/4048540823 shutdown_connections 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.16:6838/3011473533,v1:172.21.15.16:6839/3011473533] conn(0x561b888e5d90 0x561b888e81b0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x561b888791f0 0x561b888840c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b88874080 0x561b888830b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f96497fa640 0 client.29339 ms_handle_reset on v2:172.21.15.16:6838/3011473533 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 --2- 172.21.15.7:0/4048540823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b88873690 0x561b88882b70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 -- 172.21.15.7:0/4048540823 >> 172.21.15.7:0/4048540823 conn(0x561b8886ec00 msgr2=0x561b888904a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 -- 172.21.15.7:0/4048540823 shutdown_connections 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 1 -- 172.21.15.7:0/4048540823 wait complete. 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.394+0000 7f966ed7f740 20 client.29339 trim_cache size 0 max 16384 2024-03-26T08:58:09.399 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-03-26T08:58:09.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 -- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f964c0a2800 msgr2=0x7f964c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f964c0a2800 0x7f964c0a4cc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x561b888748e0 tx=0x7f965c005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 -- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 msgr2=0x7f9664197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 0x7f9664197f10 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9658009230 tx=0x7f965802c650 comp rx=0 tx=0).stop 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 -- 172.21.15.7:0/176515560 shutdown_connections 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f964c0a2800 0x7f964c0a4cc0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9664103260 0x7f966419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96641028a0 0x7f9664197f10 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 --2- 172.21.15.7:0/176515560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96641016b0 0x7f96641979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 -- 172.21.15.7:0/176515560 >> 172.21.15.7:0/176515560 conn(0x7f96640fcc20 msgr2=0x7f9664105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 -- 172.21.15.7:0/176515560 shutdown_connections 2024-03-26T08:58:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:09.395+0000 7f966c0b2640 1 -- 172.21.15.7:0/176515560 wait complete. 2024-03-26T08:58:09.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:09 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2123340722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:58:09.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:09 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2123340722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:58:09.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:09 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2123340722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:58:10.405 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-26T08:58:10.405 INFO:tasks.fwd_scrub:done joining 2024-03-26T08:58:10.405 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-26T08:58:10.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:58:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 -- 172.21.15.7:0/1993804936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f941005d0 msgr2=0x7f7f941009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 --2- 172.21.15.7:0/1993804936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f941005d0 0x7f7f941009f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009900 tx=0x7f7f8402bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 -- 172.21.15.7:0/1993804936 shutdown_connections 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 --2- 172.21.15.7:0/1993804936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f94102180 0x7f7f94108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 --2- 172.21.15.7:0/1993804936 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f941017c0 0x7f7f94101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 --2- 172.21.15.7:0/1993804936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f941005d0 0x7f7f941009f0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 -- 172.21.15.7:0/1993804936 >> 172.21.15.7:0/1993804936 conn(0x7f7f940fbb40 msgr2=0x7f7f940fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 -- 172.21.15.7:0/1993804936 shutdown_connections 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 -- 172.21.15.7:0/1993804936 wait complete. 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 Processor -- start 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.538+0000 7f7f98b77640 1 -- start start 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f98b77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f941005d0 0x7f7f941968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f98b77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941017c0 0x7f7f94196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f98b77640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f94102180 0x7f7f9419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f98b77640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9410c2a0 con 0x7f7f94102180 2024-03-26T08:58:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f98b77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9410c120 con 0x7f7f941017c0 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f98b77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9410c420 con 0x7f7f941005d0 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f92d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f94102180 0x7f7f9419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f9259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f941005d0 0x7f7f941968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f92d9d640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f94102180 0x7f7f9419de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53404/0 (socket says 172.21.15.7:53404) 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f92d9d640 1 -- 172.21.15.7:0/3230404649 learned_addr learned my addr 172.21.15.7:0/3230404649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f8bfff640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941017c0 0x7f7f94196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f92d9d640 1 -- 172.21.15.7:0/3230404649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f941005d0 msgr2=0x7f7f941968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f92d9d640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f941005d0 0x7f7f941968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f92d9d640 1 -- 172.21.15.7:0/3230404649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941017c0 msgr2=0x7f7f94196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f92d9d640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941017c0 0x7f7f94196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f92d9d640 1 -- 172.21.15.7:0/3230404649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9419e4e0 con 0x7f7f94102180 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f92d9d640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f94102180 0x7f7f9419de70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00eb10 tx=0x7f7f8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f8b7fe640 1 -- 172.21.15.7:0/3230404649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c004b20 con 0x7f7f94102180 2024-03-26T08:58:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f8b7fe640 1 -- 172.21.15.7:0/3230404649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f8c004cc0 con 0x7f7f94102180 2024-03-26T08:58:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9419e7d0 con 0x7f7f94102180 2024-03-26T08:58:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.539+0000 7f7f8b7fe640 1 -- 172.21.15.7:0/3230404649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c010760 con 0x7f7f94102180 2024-03-26T08:58:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.540+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9419ecd0 con 0x7f7f94102180 2024-03-26T08:58:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.540+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f941078f0 con 0x7f7f94102180 2024-03-26T08:58:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.542+0000 7f7f8b7fe640 1 -- 172.21.15.7:0/3230404649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f7f8c005000 con 0x7f7f94102180 2024-03-26T08:58:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.543+0000 7f7f8b7fe640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f800a2800 0x7f7f800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.543+0000 7f7f9259c640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f800a2800 0x7f7f800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.543+0000 7f7f8b7fe640 1 -- 172.21.15.7:0/3230404649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7f8c0cf200 con 0x7f7f94102180 2024-03-26T08:58:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.543+0000 7f7f9259c640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f800a2800 0x7f7f800a4cc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7f8402c170 tx=0x7f7f8402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:10.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.547+0000 7f7f8b7fe640 1 -- 172.21.15.7:0/3230404649 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f8c053740 con 0x7f7f94102180 2024-03-26T08:58:10.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:10 smithi044 ceph-mon[36461]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-03-26T08:58:10.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.732+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f940622a0 con 0x7f7f94102180 2024-03-26T08:58:10.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.733+0000 7f7f8b7fe640 1 -- 172.21.15.7:0/3230404649 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7f7f8c09c2a0 con 0x7f7f94102180 2024-03-26T08:58:10.732 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:58:10.732 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:58:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f800a2800 msgr2=0x7f7f800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f800a2800 0x7f7f800a4cc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7f8402c170 tx=0x7f7f8402df90 comp rx=0 tx=0).stop 2024-03-26T08:58:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f94102180 msgr2=0x7f7f9419de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f94102180 0x7f7f9419de70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00eb10 tx=0x7f7f8c00c2d0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 shutdown_connections 2024-03-26T08:58:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f7f800a2800 0x7f7f800a4cc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f94102180 0x7f7f9419de70 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941017c0 0x7f7f94196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 --2- 172.21.15.7:0/3230404649 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f941005d0 0x7f7f941968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 >> 172.21.15.7:0/3230404649 conn(0x7f7f940fbb40 msgr2=0x7f7f94104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 shutdown_connections 2024-03-26T08:58:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.735+0000 7f7f98b77640 1 -- 172.21.15.7:0/3230404649 wait complete. 2024-03-26T08:58:10.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:10 smithi007 ceph-mon[32462]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-03-26T08:58:10.745 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:40:08.100906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:40:08.784972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:58:10.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:58:10.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:10 smithi016 ceph-mon[35851]: pgmap v832: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 18 KiB/s wr, 10 op/s 2024-03-26T08:58:10.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 -- 172.21.15.7:0/56275230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1011f0 msgr2=0x7f994c108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 --2- 172.21.15.7:0/56275230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1011f0 0x7f994c108a90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9944009ca0 tx=0x7f994402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 -- 172.21.15.7:0/56275230 shutdown_connections 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 --2- 172.21.15.7:0/56275230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c1011f0 0x7f994c108a90 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 --2- 172.21.15.7:0/56275230 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c100830 0x7f994c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 --2- 172.21.15.7:0/56275230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f994c101be0 0x7f994c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 -- 172.21.15.7:0/56275230 >> 172.21.15.7:0/56275230 conn(0x7f994c0fb2c0 msgr2=0x7f994c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 -- 172.21.15.7:0/56275230 shutdown_connections 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 -- 172.21.15.7:0/56275230 wait complete. 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 Processor -- start 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.854+0000 7f995387e640 1 -- start start 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995387e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c100830 0x7f994c197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995387e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f994c1011f0 0x7f994c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995387e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c101be0 0x7f994c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995387e640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c10c260 con 0x7f994c100830 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995387e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c10c0e0 con 0x7f994c1011f0 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995387e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c10c3e0 con 0x7f994c101be0 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995161a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c100830 0x7f994c197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995161a640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c100830 0x7f994c197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53422/0 (socket says 172.21.15.7:53422) 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995161a640 1 -- 172.21.15.7:0/3211135958 learned_addr learned my addr 172.21.15.7:0/3211135958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f9951e1b640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c101be0 0x7f994c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f9950e19640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f994c1011f0 0x7f994c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995161a640 1 -- 172.21.15.7:0/3211135958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c101be0 msgr2=0x7f994c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:10.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995161a640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c101be0 0x7f994c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995161a640 1 -- 172.21.15.7:0/3211135958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f994c1011f0 msgr2=0x7f994c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995161a640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f994c1011f0 0x7f994c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995161a640 1 -- 172.21.15.7:0/3211135958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c19f5a0 con 0x7f994c100830 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995161a640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c100830 0x7f994c197980 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f994800e9c0 tx=0x7f994800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f99427fc640 1 -- 172.21.15.7:0/3211135958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994800cd80 con 0x7f994c100830 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f99427fc640 1 -- 172.21.15.7:0/3211135958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99480035f0 con 0x7f994c100830 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994c19f890 con 0x7f994c100830 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f99427fc640 1 -- 172.21.15.7:0/3211135958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948010690 con 0x7f994c100830 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.855+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c19fde0 con 0x7f994c100830 2024-03-26T08:58:10.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.856+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c1a0040 con 0x7f994c100830 2024-03-26T08:58:10.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.857+0000 7f99427fc640 1 -- 172.21.15.7:0/3211135958 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f99480037a0 con 0x7f994c100830 2024-03-26T08:58:10.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.858+0000 7f99427fc640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f99380a2800 0x7f99380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:10.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.858+0000 7f9950e19640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f99380a2800 0x7f99380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:10.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.858+0000 7f99427fc640 1 -- 172.21.15.7:0/3211135958 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f99480ceef0 con 0x7f994c100830 2024-03-26T08:58:10.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.858+0000 7f9950e19640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f99380a2800 0x7f99380a4cc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f994c198e90 tx=0x7f993c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:10.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:10.862+0000 7f99427fc640 1 -- 172.21.15.7:0/3211135958 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f994809c020 con 0x7f994c100830 2024-03-26T08:58:11.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.045+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f994c1a0210 con 0x7f994c100830 2024-03-26T08:58:11.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.045+0000 7f99427fc640 1 -- 172.21.15.7:0/3211135958 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f994809b920 con 0x7f994c100830 2024-03-26T08:58:11.045 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:58:11.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f99380a2800 msgr2=0x7f99380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f99380a2800 0x7f99380a4cc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f994c198e90 tx=0x7f993c005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:11.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c100830 msgr2=0x7f994c197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c100830 0x7f994c197980 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f994800e9c0 tx=0x7f994800ee80 comp rx=0 tx=0).stop 2024-03-26T08:58:11.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 shutdown_connections 2024-03-26T08:58:11.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f99380a2800 0x7f99380a4cc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f994c101be0 0x7f994c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f994c1011f0 0x7f994c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 --2- 172.21.15.7:0/3211135958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f994c100830 0x7f994c197980 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 >> 172.21.15.7:0/3211135958 conn(0x7f994c0fb2c0 msgr2=0x7f994c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:11.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 shutdown_connections 2024-03-26T08:58:11.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.047+0000 7f995387e640 1 -- 172.21.15.7:0/3211135958 wait complete. 2024-03-26T08:58:11.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:58:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.165+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/164988795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 msgr2=0x7f8c14101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.165+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/164988795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 0x7f8c14101ad0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009900 tx=0x7f8bfc02bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.165+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/164988795 shutdown_connections 2024-03-26T08:58:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.165+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/164988795 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c14103260 0x7f8c14109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.165+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/164988795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c141028a0 0x7f8c14102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.165+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/164988795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 0x7f8c14101ad0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.165+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/164988795 >> 172.21.15.7:0/164988795 conn(0x7f8c140fcc20 msgr2=0x7f8c140ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.165+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/164988795 shutdown_connections 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.165+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/164988795 wait complete. 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c1a2c6640 1 Processor -- start 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c1a2c6640 1 -- start start 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c1a2c6640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 0x7f8c14075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c1a2c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c141028a0 0x7f8c14073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c1a2c6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c14103260 0x7f8c14073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c1a2c6640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1410d330 con 0x7f8c141016b0 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c1a2c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1410d1b0 con 0x7f8c141028a0 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c1a2c6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1410d4b0 con 0x7f8c14103260 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c13fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 0x7f8c14075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c13fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 0x7f8c14075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53432/0 (socket says 172.21.15.7:53432) 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c13fff640 1 -- 172.21.15.7:0/3874269639 learned_addr learned my addr 172.21.15.7:0/3874269639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c137fe640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c141028a0 0x7f8c14073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.166+0000 7f8c18863640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c14103260 0x7f8c14073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c13fff640 1 -- 172.21.15.7:0/3874269639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c14103260 msgr2=0x7f8c14073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c13fff640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c14103260 0x7f8c14073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c13fff640 1 -- 172.21.15.7:0/3874269639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c141028a0 msgr2=0x7f8c14073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c13fff640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c141028a0 0x7f8c14073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c13fff640 1 -- 172.21.15.7:0/3874269639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c14074860 con 0x7f8c141016b0 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c13fff640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 0x7f8c14075450 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c117fa640 1 -- 172.21.15.7:0/3874269639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc041070 con 0x7f8c141016b0 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c117fa640 1 -- 172.21.15.7:0/3874269639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bfc004450 con 0x7f8c141016b0 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c141a1ac0 con 0x7f8c141016b0 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c117fa640 1 -- 172.21.15.7:0/3874269639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc034440 con 0x7f8c141016b0 2024-03-26T08:58:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.167+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c141a2010 con 0x7f8c141016b0 2024-03-26T08:58:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.168+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c141089d0 con 0x7f8c141016b0 2024-03-26T08:58:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.169+0000 7f8c117fa640 1 -- 172.21.15.7:0/3874269639 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f8bfc044070 con 0x7f8c141016b0 2024-03-26T08:58:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.170+0000 7f8c117fa640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c000a2800 0x7f8c000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.170+0000 7f8c137fe640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c000a2800 0x7f8c000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.170+0000 7f8c117fa640 1 -- 172.21.15.7:0/3874269639 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8bfc0f0560 con 0x7f8c141016b0 2024-03-26T08:58:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.170+0000 7f8c137fe640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c000a2800 0x7f8c000a4cc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8c140751c0 tx=0x7f8c04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:11.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.174+0000 7f8c117fa640 1 -- 172.21.15.7:0/3874269639 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8bfc0bd000 con 0x7f8c141016b0 2024-03-26T08:58:11.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.361+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c140622a0 con 0x7f8c141016b0 2024-03-26T08:58:11.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.362+0000 7f8c117fa640 1 -- 172.21.15.7:0/3874269639 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8bfc0c10a0 con 0x7f8c141016b0 2024-03-26T08:58:11.360 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c000a2800 msgr2=0x7f8c000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c000a2800 0x7f8c000a4cc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8c140751c0 tx=0x7f8c04005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 msgr2=0x7f8c14075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 0x7f8c14075450 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc000c00 tx=0x7f8bfc02c640 comp rx=0 tx=0).stop 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 shutdown_connections 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f8c000a2800 0x7f8c000a4cc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8c14103260 0x7f8c14073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c141028a0 0x7f8c14073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 --2- 172.21.15.7:0/3874269639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c141016b0 0x7f8c14075450 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 >> 172.21.15.7:0/3874269639 conn(0x7f8c140fcc20 msgr2=0x7f8c14105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 shutdown_connections 2024-03-26T08:58:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.364+0000 7f8c1a2c6640 1 -- 172.21.15.7:0/3874269639 wait complete. 2024-03-26T08:58:11.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:58:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.481+0000 7f647e259640 1 -- 172.21.15.7:0/3603605165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64781016a0 msgr2=0x7f6478101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.481+0000 7f647e259640 1 --2- 172.21.15.7:0/3603605165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64781016a0 0x7f6478101ac0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6468009900 tx=0x7f646802bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.481+0000 7f647e259640 1 -- 172.21.15.7:0/3603605165 shutdown_connections 2024-03-26T08:58:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.481+0000 7f647e259640 1 --2- 172.21.15.7:0/3603605165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6478103250 0x7f6478109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.481+0000 7f647e259640 1 --2- 172.21.15.7:0/3603605165 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6478102890 0x7f6478102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.481+0000 7f647e259640 1 --2- 172.21.15.7:0/3603605165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64781016a0 0x7f6478101ac0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.481+0000 7f647e259640 1 -- 172.21.15.7:0/3603605165 >> 172.21.15.7:0/3603605165 conn(0x7f64780fcbf0 msgr2=0x7f64780ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.481+0000 7f647e259640 1 -- 172.21.15.7:0/3603605165 shutdown_connections 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.481+0000 7f647e259640 1 -- 172.21.15.7:0/3603605165 wait complete. 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f647e259640 1 Processor -- start 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f647e259640 1 -- start start 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f647e259640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64781016a0 0x7f6478197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f647e259640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6478102890 0x7f6478197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f647e259640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6478103250 0x7f647819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f647e259640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647810d320 con 0x7f6478102890 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f647e259640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647810d1a0 con 0x7f6478103250 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f647e259640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647810d4a0 con 0x7f64781016a0 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f646effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6478102890 0x7f6478197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f64777fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64781016a0 0x7f6478197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f646effd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6478102890 0x7f6478197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53458/0 (socket says 172.21.15.7:53458) 2024-03-26T08:58:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f646effd640 1 -- 172.21.15.7:0/2758809565 learned_addr learned my addr 172.21.15.7:0/2758809565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.482+0000 7f6477fff640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6478103250 0x7f647819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f646effd640 1 -- 172.21.15.7:0/2758809565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64781016a0 msgr2=0x7f6478197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f646effd640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64781016a0 0x7f6478197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f646effd640 1 -- 172.21.15.7:0/2758809565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6478103250 msgr2=0x7f647819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f646effd640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6478103250 0x7f647819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f646effd640 1 -- 172.21.15.7:0/2758809565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647819f640 con 0x7f6478102890 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f646effd640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6478102890 0x7f6478197f60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f646000e9f0 tx=0x7f646000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f64757fa640 1 -- 172.21.15.7:0/2758809565 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646000cd80 con 0x7f6478102890 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647819f930 con 0x7f6478102890 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f64757fa640 1 -- 172.21.15.7:0/2758809565 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6460005500 con 0x7f6478102890 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f64757fa640 1 -- 172.21.15.7:0/2758809565 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460010690 con 0x7f6478102890 2024-03-26T08:58:11.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.483+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647819fe30 con 0x7f6478102890 2024-03-26T08:58:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.484+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6478108a90 con 0x7f6478102890 2024-03-26T08:58:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.485+0000 7f64757fa640 1 -- 172.21.15.7:0/2758809565 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f64600046c0 con 0x7f6478102890 2024-03-26T08:58:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.486+0000 7f64757fa640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f64640a2800 0x7f64640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.486+0000 7f64777fe640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f64640a2800 0x7f64640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.486+0000 7f64757fa640 1 -- 172.21.15.7:0/2758809565 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f64600cee40 con 0x7f6478102890 2024-03-26T08:58:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.486+0000 7f64777fe640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f64640a2800 0x7f64640a4cc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6468000c00 tx=0x7f6468005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.491+0000 7f64757fa640 1 -- 172.21.15.7:0/2758809565 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f646009bee0 con 0x7f6478102890 2024-03-26T08:58:11.572 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3230404649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:58:11.573 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:11 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3211135958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:58:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.674+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64780622a0 con 0x7f6478102890 2024-03-26T08:58:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.674+0000 7f64757fa640 1 -- 172.21.15.7:0/2758809565 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f646009b7e0 con 0x7f6478102890 2024-03-26T08:58:11.673 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-26T08:58:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f64640a2800 msgr2=0x7f64640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f64640a2800 0x7f64640a4cc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6468000c00 tx=0x7f6468005fd0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6478102890 msgr2=0x7f6478197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6478102890 0x7f6478197f60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f646000e9f0 tx=0x7f646000edf0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 shutdown_connections 2024-03-26T08:58:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f64640a2800 0x7f64640a4cc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6478103250 0x7f647819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6478102890 0x7f6478197f60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 --2- 172.21.15.7:0/2758809565 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f64781016a0 0x7f6478197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 >> 172.21.15.7:0/2758809565 conn(0x7f64780fcbf0 msgr2=0x7f6478105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 shutdown_connections 2024-03-26T08:58:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.676+0000 7f647e259640 1 -- 172.21.15.7:0/2758809565 wait complete. 2024-03-26T08:58:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3230404649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:58:11.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:11 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3211135958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:58:11.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-26T08:58:11.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3230404649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:58:11.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:11 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3211135958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:58:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.790+0000 7f92ed5ba640 1 -- 172.21.15.7:0/2895945450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 msgr2=0x7f92e8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.790+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/2895945450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 0x7f92e8108a10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009ca0 tx=0x7f92d802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.790+0000 7f92ed5ba640 1 -- 172.21.15.7:0/2895945450 shutdown_connections 2024-03-26T08:58:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.790+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/2895945450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 0x7f92e8108a10 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.790+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/2895945450 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e81017c0 0x7f92e8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.790+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/2895945450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e81005d0 0x7f92e81009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.790+0000 7f92ed5ba640 1 -- 172.21.15.7:0/2895945450 >> 172.21.15.7:0/2895945450 conn(0x7f92e80fbb40 msgr2=0x7f92e80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.790+0000 7f92ed5ba640 1 -- 172.21.15.7:0/2895945450 shutdown_connections 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.790+0000 7f92ed5ba640 1 -- 172.21.15.7:0/2895945450 wait complete. 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92ed5ba640 1 Processor -- start 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92ed5ba640 1 -- start start 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92ed5ba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e81005d0 0x7f92e8197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92ed5ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e81017c0 0x7f92e8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92ed5ba640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 0x7f92e819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92ed5ba640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e810c230 con 0x7f92e8102180 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92ed5ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e810c0b0 con 0x7f92e81017c0 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92ed5ba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e810c3b0 con 0x7f92e81005d0 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92e77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 0x7f92e819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92e6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e81005d0 0x7f92e8197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92e77fe640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 0x7f92e819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53474/0 (socket says 172.21.15.7:53474) 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92e77fe640 1 -- 172.21.15.7:0/1539327739 learned_addr learned my addr 172.21.15.7:0/1539327739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.791+0000 7f92e67fc640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e81017c0 0x7f92e8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92e77fe640 1 -- 172.21.15.7:0/1539327739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e81005d0 msgr2=0x7f92e8197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92e77fe640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e81005d0 0x7f92e8197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92e77fe640 1 -- 172.21.15.7:0/1539327739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e81017c0 msgr2=0x7f92e8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92e77fe640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e81017c0 0x7f92e8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92e77fe640 1 -- 172.21.15.7:0/1539327739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e819f550 con 0x7f92e8102180 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92e67fc640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e81017c0 0x7f92e8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92e6ffd640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e81005d0 0x7f92e8197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92e77fe640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 0x7f92e819eee0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f92d8037040 tx=0x7f92d8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92dffff640 1 -- 172.21.15.7:0/1539327739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8041020 con 0x7f92e8102180 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92dffff640 1 -- 172.21.15.7:0/1539327739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f92d8034cd0 con 0x7f92e8102180 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e819f7e0 con 0x7f92e8102180 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92dffff640 1 -- 172.21.15.7:0/1539327739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8039040 con 0x7f92e8102180 2024-03-26T08:58:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.792+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e819fd20 con 0x7f92e8102180 2024-03-26T08:58:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.793+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e819ff80 con 0x7f92e8102180 2024-03-26T08:58:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.794+0000 7f92dffff640 1 -- 172.21.15.7:0/1539327739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f92d8044020 con 0x7f92e8102180 2024-03-26T08:58:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.795+0000 7f92dffff640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92d40a2880 0x7f92d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.795+0000 7f92e6ffd640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92d40a2880 0x7f92d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.795+0000 7f92dffff640 1 -- 172.21.15.7:0/1539327739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f92d80f0880 con 0x7f92e8102180 2024-03-26T08:58:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.795+0000 7f92e6ffd640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92d40a2880 0x7f92d40a4d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f92e8101620 tx=0x7f92e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:11.800+0000 7f92dffff640 1 -- 172.21.15.7:0/1539327739 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f92d804b020 con 0x7f92e8102180 2024-03-26T08:58:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.037+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f92e81a0310 con 0x7f92e8102180 2024-03-26T08:58:12.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.045+0000 7f92dffff640 1 -- 172.21.15.7:0/1539327739 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+739823 (secure 0 0 0) 0x7f92d80b7e20 con 0x7f92e8102180 2024-03-26T08:58:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:58:12.054 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:58:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 20, 2024-03-26T08:58:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "active_gid": 14152, 2024-03-26T08:58:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "active_name": "x", 2024-03-26T08:58:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addrs": { 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1233671832 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1233671832 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addr": "172.21.15.7:6801/1233671832", 2024-03-26T08:58:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "active_change": "2024-03-26T08:32:58.217914+0000", 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "active_mgr_features": 4540701547738038271, 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "available": true, 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "standbys": [ 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24112, 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "z", 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T08:58:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-03-26T08:58:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T08:58:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T08:58:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-03-26T08:58:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-03-26T08:58:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T08:58:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T08:58:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T08:58:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-03-26T08:58:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T08:58:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-03-26T08:58:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-03-26T08:58:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-03-26T08:58:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T08:58:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:58:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-26T08:58:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T08:58:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:58:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-03-26T08:58:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-03-26T08:58:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-03-26T08:58:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-03-26T08:58:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-03-26T08:58:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-03-26T08:58:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-03-26T08:58:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T08:58:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-03-26T08:58:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-03-26T08:58:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T08:58:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-03-26T08:58:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-03-26T08:58:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-03-26T08:58:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T08:58:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-03-26T08:58:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-03-26T08:58:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T08:58:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-03-26T08:58:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-03-26T08:58:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-03-26T08:58:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T08:58:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-03-26T08:58:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-03-26T08:58:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-03-26T08:58:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T08:58:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-03-26T08:58:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-03-26T08:58:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T08:58:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-03-26T08:58:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T08:58:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T08:58:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-03-26T08:58:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T08:58:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T08:58:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T08:58:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "{}", 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T08:58:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T08:58:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T08:58:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T08:58:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T08:58:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T08:58:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T08:58:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T08:58:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T08:58:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T08:58:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T08:58:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-03-26T08:58:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-03-26T08:58:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T08:58:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T08:58:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T08:58:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-03-26T08:58:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-03-26T08:58:12.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.247 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.248 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.249 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T08:58:12.253 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.254 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.256 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-03-26T08:58:12.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.258 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.259 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T08:58:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-03-26T08:58:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-03-26T08:58:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:58:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:58:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-03-26T08:58:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-03-26T08:58:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-03-26T08:58:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-03-26T08:58:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-03-26T08:58:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-03-26T08:58:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-03-26T08:58:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-03-26T08:58:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-03-26T08:58:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T08:58:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-03-26T08:58:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-03-26T08:58:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-03-26T08:58:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T08:58:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-03-26T08:58:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-03-26T08:58:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-03-26T08:58:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-03-26T08:58:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-03-26T08:58:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-03-26T08:58:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-03-26T08:58:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T08:58:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-03-26T08:58:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-03-26T08:58:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-03-26T08:58:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-26T08:58:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-03-26T08:58:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-03-26T08:58:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T08:58:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.358 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.359 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.360 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.361 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.362 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.364 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.365 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.367 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.369 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.370 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-03-26T08:58:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.373 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.374 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.380 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-26T08:58:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-03-26T08:58:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-03-26T08:58:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T08:58:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.386 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.388 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.389 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.390 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.392 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.394 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.395 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.396 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.398 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.399 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.400 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.401 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.403 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.408 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.412 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.413 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.414 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.416 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.417 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.418 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.421 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.422 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.423 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.426 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.429 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.434 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.435 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.436 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-03-26T08:58:12.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.439 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.440 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.442 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.443 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.444 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.445 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-03-26T08:58:12.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-03-26T08:58:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.448 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-03-26T08:58:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-03-26T08:58:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.450 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-03-26T08:58:12.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-03-26T08:58:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.452 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-03-26T08:58:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-03-26T08:58:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.454 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.455 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-03-26T08:58:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-03-26T08:58:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-03-26T08:58:12.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.462 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.464 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T08:58:12.465 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-03-26T08:58:12.466 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.467 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.470 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.472 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.474 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.475 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-03-26T08:58:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.478 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.479 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.480 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.481 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.482 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.483 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.486 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.487 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.490 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.496 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-03-26T08:58:12.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-03-26T08:58:12.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.502 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.505 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.506 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.508 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.509 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.510 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.513 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-26T08:58:12.517 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.518 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.520 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-03-26T08:58:12.522 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T08:58:12.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.526 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.527 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.529 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.530 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.531 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.532 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.535 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 24119, 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "y", 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.537 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.541 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-03-26T08:58:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.547 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-03-26T08:58:12.549 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.551 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.552 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.554 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T08:58:12.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-03-26T08:58:12.556 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:58:12.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.559 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.560 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.564 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-03-26T08:58:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.568 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T08:58:12.569 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.572 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-03-26T08:58:12.575 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.576 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-03-26T08:58:12.577 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-03-26T08:58:12.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.579 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.580 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T08:58:12.582 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T08:58:12.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.584 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T08:58:12.587 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.588 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.589 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.590 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.591 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.592 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.593 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.594 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.595 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.597 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.599 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.600 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-03-26T08:58:12.601 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.603 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.604 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.605 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.606 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.608 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T08:58:12.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T08:58:12.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-03-26T08:58:12.612 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-03-26T08:58:12.614 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-26T08:58:12.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.618 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.619 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-03-26T08:58:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-03-26T08:58:12.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-03-26T08:58:12.622 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-03-26T08:58:12.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.626 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.628 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.629 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.630 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.631 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.633 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.634 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.635 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.636 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.637 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.640 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T08:58:12.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.642 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.644 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.646 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.647 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.648 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.650 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-03-26T08:58:12.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.653 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T08:58:12.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T08:58:12.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T08:58:12.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.658 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "{}", 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.659 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.661 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.665 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.666 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.668 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.670 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.672 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.674 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.675 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.676 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T08:58:12.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.680 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-03-26T08:58:12.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.682 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-03-26T08:58:12.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.685 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.686 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.690 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.691 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.692 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.693 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.694 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-03-26T08:58:12.696 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.697 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.698 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.699 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.701 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.704 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-03-26T08:58:12.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.713 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.715 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.716 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-03-26T08:58:12.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.719 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-03-26T08:58:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.721 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-03-26T08:58:12.723 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.727 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.728 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.732 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.733 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.736 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.737 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.738 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.739 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.740 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.741 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.742 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.746 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.747 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.750 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.756 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.758 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-03-26T08:58:12.759 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.763 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.764 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.765 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.766 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.767 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.769 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.774 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.777 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.778 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.779 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.780 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.781 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.782 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-03-26T08:58:12.786 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.788 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.789 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.790 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.791 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.792 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.793 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.794 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.796 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-03-26T08:58:12.797 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.798 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.800 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.802 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.804 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.805 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-26T08:58:12.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.808 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.809 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.810 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.811 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.812 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.815 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.816 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.817 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.818 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.819 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.820 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.822 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.824 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.825 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.826 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.828 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.830 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.833 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.834 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.835 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.838 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.839 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.840 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.844 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.845 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.847 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.848 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.851 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.852 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.853 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-03-26T08:58:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-03-26T08:58:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-03-26T08:58:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-03-26T08:58:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-03-26T08:58:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-03-26T08:58:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-03-26T08:58:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-03-26T08:58:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-03-26T08:58:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.922 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.923 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.924 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T08:58:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-03-26T08:58:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-03-26T08:58:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "modules": [ 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard", 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "iostat", 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "nfs", 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "restful" 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-03-26T08:58:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T08:58:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-03-26T08:58:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-03-26T08:58:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-03-26T08:58:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.998 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:12.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:13.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-03-26T08:58:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-03-26T08:58:13.002 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.003 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.004 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-03-26T08:58:13.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T08:58:13.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-03-26T08:58:13.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-03-26T08:58:13.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.009 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.014 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.015 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.020 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.021 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:13.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:13.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-03-26T08:58:13.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.027 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.028 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.029 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.031 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.033 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.036 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.042 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.044 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.045 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-03-26T08:58:13.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.048 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T08:58:13.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T08:58:13.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.056 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T08:58:13.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.066 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.068 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T08:58:13.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "{}", 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T08:58:13.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.071 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.076 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T08:58:13.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T08:58:13.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T08:58:13.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-03-26T08:58:13.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.082 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T08:58:13.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T08:58:13.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:13.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.089 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.091 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.092 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.094 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.098 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-03-26T08:58:13.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.102 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-26T08:58:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-26T08:58:13.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-03-26T08:58:13.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-26T08:58:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-03-26T08:58:13.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.110 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.113 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.114 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T08:58:13.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T08:58:13.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-03-26T08:58:13.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.121 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.124 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.125 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.126 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.127 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.129 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-03-26T08:58:13.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-03-26T08:58:13.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.142 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-03-26T08:58:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-03-26T08:58:13.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.146 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.148 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.149 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.151 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.155 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-03-26T08:58:13.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:13.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.159 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.160 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.166 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.176 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.179 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.180 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.182 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.184 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.186 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.187 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.189 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.190 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.193 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-03-26T08:58:13.198 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.199 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.201 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.202 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-03-26T08:58:13.204 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.205 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.207 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.210 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.212 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.213 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.219 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.221 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.224 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T08:58:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-03-26T08:58:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-03-26T08:58:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-03-26T08:58:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-03-26T08:58:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T08:58:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-03-26T08:58:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-03-26T08:58:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-03-26T08:58:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.334 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.337 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.339 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-03-26T08:58:13.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-03-26T08:58:13.343 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.345 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-03-26T08:58:13.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.347 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-03-26T08:58:13.349 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.351 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.352 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-03-26T08:58:13.353 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "services": { 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard": "https://172.21.15.7:8443/" 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "always_on_modules": { 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus": [ 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-26T08:58:13.354 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific": [ 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-26T08:58:13.355 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy": [ 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-26T08:58:13.356 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "reef": [ 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "squid": [ 2024-03-26T08:58:13.357 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "last_failure_osd_epoch": 3, 2024-03-26T08:58:13.358 INFO:teuthology.orchestra.run.smithi007.stdout: "active_clients": [ 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1348143777 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "libcephsqlite", 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-26T08:58:13.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2409591608 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2353302922 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.360 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3588192525 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-03-26T08:58:13.361 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:58:13.363 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:12 smithi044 ceph-mon[36461]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-03-26T08:58:13.363 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:12 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/3874269639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:58:13.363 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:12 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2758809565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:58:13.363 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:12 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1539327739' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T08:58:13.363 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:12 smithi016 ceph-mon[35851]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-03-26T08:58:13.363 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:12 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/3874269639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:58:13.363 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:12 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2758809565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:58:13.363 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:12 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1539327739' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T08:58:13.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92d40a2880 msgr2=0x7f92d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:13.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92d40a2880 0x7f92d40a4d40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f92e8101620 tx=0x7f92e0005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:13.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 msgr2=0x7f92e819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:13.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 0x7f92e819eee0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f92d8037040 tx=0x7f92d8032f40 comp rx=0 tx=0).stop 2024-03-26T08:58:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 shutdown_connections 2024-03-26T08:58:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f92d40a2880 0x7f92d40a4d40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92e8102180 0x7f92e819eee0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92e81017c0 0x7f92e8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 --2- 172.21.15.7:0/1539327739 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f92e81005d0 0x7f92e8197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 >> 172.21.15.7:0/1539327739 conn(0x7f92e80fbb40 msgr2=0x7f92e80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 shutdown_connections 2024-03-26T08:58:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:12.050+0000 7f92ed5ba640 1 -- 172.21.15.7:0/1539327739 wait complete. 2024-03-26T08:58:13.371 DEBUG:tasks.check_counter:Getting stats from a 2024-03-26T08:58:13.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-26T08:58:13.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:12 smithi007 ceph-mon[32462]: pgmap v833: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 18 KiB/s wr, 8 op/s 2024-03-26T08:58:13.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:12 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/3874269639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:58:13.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:12 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2758809565' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:58:13.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:12 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1539327739' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T08:58:13.626 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:58:14.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:14 smithi016 ceph-mon[35851]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-03-26T08:58:14.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:14 smithi007 ceph-mon[32462]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-03-26T08:58:14.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:14 smithi044 ceph-mon[36461]: pgmap v834: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 17 KiB/s wr, 8 op/s 2024-03-26T08:58:15.642 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 301, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 294, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 776640, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 176288, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.045011416, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.018481778, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.019778064, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.006712261, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 294, 2024-03-26T08:58:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.023538735, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000080063 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 776640, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 176288 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 223, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 320, 2024-03-26T08:58:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 194736, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.016234485, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.012566936, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001459394, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 221, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.013751965, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000062226 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 194736 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.000280496, 2024-03-26T08:58:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000220488, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T08:58:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148704, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 29, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-26T08:58:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 299, 2024-03-26T08:58:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 731629, 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 299, 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 299, 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 731629, 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:15.653 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:58:16.343 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-26T08:58:16.344 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-26T08:58:16.344 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-03-26T08:58:16.344 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-03-26T08:58:16.344 DEBUG:tasks.check_counter:Getting stats from d 2024-03-26T08:58:16.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-26T08:58:16.604 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:58:16.649 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:16 smithi007 ceph-mon[32462]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-03-26T08:58:16.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:16 smithi016 ceph-mon[35851]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-03-26T08:58:16.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:16 smithi044 ceph-mon[36461]: pgmap v835: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 KiB/s wr, 8 op/s 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.001336140, 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000290949, 2024-03-26T08:58:18.423 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2, 2024-03-26T08:58:18.424 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 228, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 320, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 195616, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.016123199, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.012355437, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001733979, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 226, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.010948118, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000048443 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 195616 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 307, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 300, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 778336, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 179568, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.043516630, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.017789584, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.019927755, 2024-03-26T08:58:18.426 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.006480891, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 300, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.013104508, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000043681 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 778336, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 179568 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-26T08:58:18.427 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:18.428 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:18.429 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148704, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 29, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-26T08:58:18.430 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-26T08:58:18.431 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 305, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 732473, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 305, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 305, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 732473, 2024-03-26T08:58:18.432 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-26T08:58:18.433 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:18.433 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:18.433 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:18.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:18.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:58:18.433 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:18 smithi007 ceph-mon[32462]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-03-26T08:58:18.625 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:18 smithi044 ceph-mon[36461]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-03-26T08:58:18.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:18 smithi016 ceph-mon[35851]: pgmap v836: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-03-26T08:58:19.076 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-26T08:58:19.077 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-26T08:58:19.077 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-26T08:58:19.077 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-26T08:58:19.077 DEBUG:tasks.check_counter:Getting stats from g 2024-03-26T08:58:19.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-26T08:58:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:58:20.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:20 smithi007 ceph-mon[32462]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-03-26T08:58:20.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:20 smithi016 ceph-mon[35851]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-03-26T08:58:20.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:20 smithi044 ceph-mon[36461]: pgmap v837: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.6 KiB/s wr, 8 op/s 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.001285489, 2024-03-26T08:58:21.037 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000200806, 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.038 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 303, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 296, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 777184, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 177536, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.044581586, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.017098096, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.019761808, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.008477949, 2024-03-26T08:58:21.039 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 296, 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.013758538, 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000046481 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 777184, 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 177536 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2, 2024-03-26T08:58:21.040 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 225, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 320, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 195088, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.015910270, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.012505736, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001291281, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 223, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.008547367, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000038329 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:21.041 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 195088 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:21.042 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:21.043 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:21.044 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148699, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:21.045 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:21.046 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 301, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 731889, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 301, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 301, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 731889, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:21.047 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:21.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:21.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:21.048 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:58:21.797 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-26T08:58:21.797 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-26T08:58:21.797 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-03-26T08:58:21.797 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-03-26T08:58:21.797 DEBUG:tasks.check_counter:Getting stats from j 2024-03-26T08:58:21.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-26T08:58:22.046 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T08:58:22.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:22 smithi007 ceph-mon[32462]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 5 op/s 2024-03-26T08:58:22.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:22 smithi016 ceph-mon[35851]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 5 op/s 2024-03-26T08:58:22.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:22 smithi044 ceph-mon[36461]: pgmap v838: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 5 op/s 2024-03-26T08:58:23.705 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-03-26T08:58:23.705 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:23.705 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-03-26T08:58:23.705 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-03-26T08:58:23.705 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-03-26T08:58:23.705 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-03-26T08:58:23.705 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.001202279, 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000274265, 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:23.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:23.707 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 6, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 231, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3392, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 196240, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 2, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.017526737, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.013440792, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.001849878, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 230, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.013407693, 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000058294 2024-03-26T08:58:23.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 196240 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 305, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 300, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 775808, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 180368, 2024-03-26T08:58:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.045655066, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.018083044, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.022643461, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.007065283, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 299, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.018129541, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000060633 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:23.710 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 775808, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 180368 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:23.711 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:23.712 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:23.712 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:23.712 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:23.712 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:23.712 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:23.712 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:23.713 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148721, 2024-03-26T08:58:23.714 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:23.715 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-03-26T08:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 307, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 732733, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 307, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 307, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 732733, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:23.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-03-26T08:58:23.718 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-03-26T08:58:24.422 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-26T08:58:24.422 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-26T08:58:24.422 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-26T08:58:24.422 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-26T08:58:24.422 DEBUG:tasks.check_counter:Getting stats from b 2024-03-26T08:58:24.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-26T08:58:24.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:24 smithi007 ceph-mon[32462]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-03-26T08:58:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:58:24.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:24 smithi016 ceph-mon[35851]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-03-26T08:58:24.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:24 smithi044 ceph-mon[36461]: pgmap v839: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-03-26T08:58:26.529 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:26 smithi016 ceph-mon[35851]: pgmap v840: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-03-26T08:58:26.658 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-26T08:58:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 17136, 2024-03-26T08:58:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 18631, 2024-03-26T08:58:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 7778170, 2024-03-26T08:58:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 23333807, 2024-03-26T08:58:26.658 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 351, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 1.262271132, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.418507843, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.400089534, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.324902403, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 18629, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2.474175070, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000132813 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5139, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001067315, 2024-03-26T08:58:26.659 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000207 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 37456, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 1456 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 29895, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 33250, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 12372231, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 23843764, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 45, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 6, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 1.659617609, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.583184980, 2024-03-26T08:58:26.660 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 1.676992506, 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.317123998, 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 33248, 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2.037553366, 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000061283 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 30213, 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.005029122, 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000166 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 60880, 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 521456 2024-03-26T08:58:26.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 12010, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 13560, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 6485432, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 8490710, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 6, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.833033253, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.298630128, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.249185734, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.236679003, 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:26.662 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 13560, 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.794827808, 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000132361 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4689, 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001199006, 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000255 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 833888, 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 219344 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-03-26T08:58:26.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 31408, 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 11.284868442, 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000359299 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 958, 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.041123875, 2024-03-26T08:58:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000042926 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 8123, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 8122, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8122, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 30.940499125, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003809468 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 1, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 1266, 2024-03-26T08:58:26.665 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 3, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 1167, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 7, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 1160, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 1157, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 1, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 1052, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 1052, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 4, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 17085, 2024-03-26T08:58:26.666 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 12609, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 540, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 1, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 1, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 29720, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 1027, 2024-03-26T08:58:26.667 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 1746536863, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 29720, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 1015, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 29, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 986, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 28705, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 1, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 1827, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 4174, 2024-03-26T08:58:26.668 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 1, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 5564, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 2071, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 228, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 2851 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 70, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-03-26T08:58:26.669 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 118, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 553, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 284, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 269, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 29, 2024-03-26T08:58:26.670 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 1, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 3, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 8370, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 8106, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 8106, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 264, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 58, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 78, 2024-03-26T08:58:26.671 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 76, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 76, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 1, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 1, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 25149698, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 25869923, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 0, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5054, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 32.213889209, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.006373939 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 0 2024-03-26T08:58:26.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1074, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 1388, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 314, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 68, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 73, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 5, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 1167, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 2351, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 1184, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 1052, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 2342, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 1290, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 72372, 2024-03-26T08:58:26.673 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 231804 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 12084, 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 1, 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 8123, 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 307, 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 2, 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1313, 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 4.112884698, 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003132433 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.674 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 91, 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.037604158, 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000413232 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 269, 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.874498411, 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003250923 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.675 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3659, 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 17.373534083, 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.004748164 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.676 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.677 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-03-26T08:58:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.678 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 37, 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.528532833, 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.014284671 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1815, 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.007543736, 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000555120 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 91, 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.391401725, 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.004301117 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 510, 2024-03-26T08:58:26.680 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 4.831969877, 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.009474450 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-03-26T08:58:26.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2, 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.015951035, 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.007975517 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-03-26T08:58:26.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.009083356, 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003027785 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8, 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.021041691, 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002630211 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 324, 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.736453522, 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.005359424 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-03-26T08:58:26.685 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 3, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 2, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 716, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 716, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 978, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:26.686 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:26.687 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:26.688 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 346004, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 1183, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 1056, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 12, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-03-26T08:58:26.689 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 7624, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 160, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 3803429, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 52983, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:26.690 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 39574, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 26266954, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 31, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 39574, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 39574, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 206.941741541, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.005229234 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.691 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 39574, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 45650 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 39574, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 29895, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 9679, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1196, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 1555, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 146, 2024-03-26T08:58:26.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 5171, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 378, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 3, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 819, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 29749, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 3110, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-03-26T08:58:26.693 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 1, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-03-26T08:58:26.694 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 87, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 3, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 12, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 12, 2024-03-26T08:58:26.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 3236, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 0, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 286 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 1, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-03-26T08:58:26.696 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 16, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 10, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 298, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 284, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 59037, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 21288108, 2024-03-26T08:58:26.697 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 59037, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 59037, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 21288108, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-26T08:58:26.698 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 39574, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 26295638, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 39574, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 26295638, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:26.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 39574, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 39574, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 39574, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 39574, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.700 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 284, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 28684, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 284, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 117, 2024-03-26T08:58:26.701 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 28684, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55d8783ce460": { 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 8370, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 21675619, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:26.702 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 8370, 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 5054, 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 21675619, 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:26.703 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-26T08:58:26.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:26 smithi007 ceph-mon[32462]: pgmap v840: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-03-26T08:58:26.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:26 smithi044 ceph-mon[36461]: pgmap v840: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-03-26T08:58:27.388 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2024-03-26T08:58:27.388 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-26T08:58:27.388 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=70 2024-03-26T08:58:27.388 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-03-26T08:58:27.388 DEBUG:tasks.check_counter:Getting stats from e 2024-03-26T08:58:27.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-26T08:58:27.652 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:58:28.661 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:28 smithi016 ceph-mon[35851]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-03-26T08:58:28.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:28 smithi007 ceph-mon[32462]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-03-26T08:58:28.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:28 smithi044 ceph-mon[36461]: pgmap v841: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-03-26T08:58:29.511 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1096, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1096, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 300921, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 347768, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.083744674, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.027868151, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.021575774, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.026972721, 2024-03-26T08:58:29.512 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1095, 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.047832129, 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000043682 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:29.513 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1280, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1272, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1039476, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 488427, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.114131932, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.031859251, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.037120371, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.039397607, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1271, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.060408162, 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000047528 2024-03-26T08:58:29.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 802064, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 181552 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1086, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1313, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 364544, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 909112, 2024-03-26T08:58:29.515 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.103987412, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.035890519, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.025398262, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.032767113, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1311, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.085108846, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000064919 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.516 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 564400 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5254, 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.128066772, 2024-03-26T08:58:29.517 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000024375 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-03-26T08:58:29.518 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 9, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-03-26T08:58:29.519 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 9, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 9, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 3, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-03-26T08:58:29.520 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-03-26T08:58:29.521 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-03-26T08:58:29.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-03-26T08:58:29.523 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-03-26T08:58:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 7, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4205490, 2024-03-26T08:58:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4204916, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 7 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 12, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 12, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 10, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 10, 2024-03-26T08:58:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 9, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 9, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 48344, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215420 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-03-26T08:58:29.527 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.531 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.532 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-03-26T08:58:29.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.532 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.532 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-03-26T08:58:29.532 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-03-26T08:58:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.535 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.536 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 1, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 0, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 1, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 2310, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 190369, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 58, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 7624, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 160, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 46386, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 241, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 3141, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 3141, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3141, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 11.698814397, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003724550 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3141, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3142 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 3141, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 3141, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 2047, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 1092, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-03-26T08:58:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-03-26T08:58:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-03-26T08:58:29.547 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 87, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 2, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 5, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-03-26T08:58:29.548 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 1, 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-03-26T08:58:29.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 3458, 2024-03-26T08:58:29.550 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1374448, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 3458, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3458, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1374448, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:29.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 3141, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 11191, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3141, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 11191, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 3141, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 3141, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3141, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 3141, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.553 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-03-26T08:58:29.554 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x5640a69826e0": { 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:29.555 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:29.556 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-03-26T08:58:29.556 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-03-26T08:58:29.556 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:29.556 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:29.556 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:29.556 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:29.556 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:29.556 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:29.556 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-26T08:58:30.321 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-26T08:58:30.321 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-26T08:58:30.321 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-26T08:58:30.321 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-26T08:58:30.321 DEBUG:tasks.check_counter:Getting stats from h 2024-03-26T08:58:30.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-26T08:58:30.500 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:30 smithi016 ceph-mon[35851]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-03-26T08:58:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:58:30.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:30 smithi007 ceph-mon[32462]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-03-26T08:58:30.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:30 smithi044 ceph-mon[36461]: pgmap v842: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-03-26T08:58:32.329 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-26T08:58:32.329 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:32.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2276, 2024-03-26T08:58:32.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2268, 2024-03-26T08:58:32.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 9779128, 2024-03-26T08:58:32.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 804674, 2024-03-26T08:58:32.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-03-26T08:58:32.329 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.211918858, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.065986663, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.064396416, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.069867198, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2267, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.115222750, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000050826 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 801952, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 181072 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1025, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1249, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 8650882, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 888780, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.081973571, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.024635065, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.026537473, 2024-03-26T08:58:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.023872004, 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1248, 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.048342290, 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000038735 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 564672 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 672, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 674, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 5072699, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 213703, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.053161520, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.012949167, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.016431052, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.019273211, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 672, 2024-03-26T08:58:32.333 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.043451755, 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000064660 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-03-26T08:58:32.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 6182, 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.174990433, 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000028306 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.335 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-03-26T08:58:32.337 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 1872, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 1191, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 681, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 180, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 3, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-03-26T08:58:32.338 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-03-26T08:58:32.339 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 1027, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 1746536863, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-03-26T08:58:32.340 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-03-26T08:58:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-03-26T08:58:32.342 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-03-26T08:58:32.343 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-03-26T08:58:32.343 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T08:58:32.343 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-03-26T08:58:32.343 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-03-26T08:58:32.345 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-03-26T08:58:32.345 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 8373, 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-03-26T08:58:32.346 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 3, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 72, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 25149698, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 25876433, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 25874412, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.347 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 8373 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 1074, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 1359, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 285, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 68, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 73, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 5, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 1872, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 1877, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 5, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-03-26T08:58:32.348 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 57136, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215420 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.349 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.350 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.352 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.353 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-03-26T08:58:32.353 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.353 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.353 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.353 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-03-26T08:58:32.353 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.353 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.354 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.354 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-03-26T08:58:32.354 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.354 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.354 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.354 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.355 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.356 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.357 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.358 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-03-26T08:58:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.360 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 1, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 3, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 2, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 2, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 3650, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-03-26T08:58:32.361 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:32.362 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:32.363 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 237902, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 1108, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:32.364 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 7624, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 160, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 3810837, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 45767, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:32.365 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 3652, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 26, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 4, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 3652, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3652, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 12.888795503, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.003529243 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-03-26T08:58:32.366 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3652, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 3653 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 3652, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 3652, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 2152, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 1497, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-03-26T08:58:32.367 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 2, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-03-26T08:58:32.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-03-26T08:58:32.369 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 87, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 2, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 6, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 6, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-03-26T08:58:32.370 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-03-26T08:58:32.371 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:32.372 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 3969, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 23126232, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 3969, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3969, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 23126232, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.373 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 3652, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 21682139, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3652, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 21682139, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.374 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 3652, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 3652, 2024-03-26T08:58:32.375 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 3652, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 3652, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:32.376 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x55fd8f9566e0": { 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:32.378 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-26T08:58:32.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:32 smithi016 ceph-mon[35851]: pgmap v843: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-03-26T08:58:32.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:32 smithi007 ceph-mon[32462]: pgmap v843: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-03-26T08:58:32.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:32 smithi044 ceph-mon[36461]: pgmap v843: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-03-26T08:58:33.105 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-26T08:58:33.105 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-26T08:58:33.105 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-26T08:58:33.105 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-03-26T08:58:33.105 DEBUG:tasks.check_counter:Getting stats from k 2024-03-26T08:58:33.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-26T08:58:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T08:58:34.707 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:34 smithi016 ceph-mon[35851]: pgmap v844: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-26T08:58:34.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:34 smithi007 ceph-mon[32462]: pgmap v844: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-26T08:58:34.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:34 smithi044 ceph-mon[36461]: pgmap v844: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-26T08:58:35.154 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3072, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001715162, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000041341, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000694219, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-03-26T08:58:35.155 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000687125, 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000229041 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 232, 2024-03-26T08:58:35.156 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 196320, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.010640651, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.007624923, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001203057, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 230, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012087667, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000052555 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:35.157 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 196320 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 309, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 304, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 776896, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 182864, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-03-26T08:58:35.158 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.038350447, 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.013537724, 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.020025434, 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.006314282, 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 303, 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016713969, 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000055161 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:35.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 776896, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 182864 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:35.160 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:35.161 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:35.162 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148738, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 31, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:35.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 311, 2024-03-26T08:58:35.164 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 733253, 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 311, 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 311, 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 733253, 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-03-26T08:58:35.165 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-03-26T08:58:35.866 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-03-26T08:58:35.866 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-03-26T08:58:35.866 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-03-26T08:58:35.866 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-03-26T08:58:35.866 DEBUG:tasks.check_counter:Getting stats from c 2024-03-26T08:58:35.867 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-26T08:58:36.129 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:58:36.513 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:36 smithi044 ceph-mon[36461]: pgmap v845: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T08:58:36.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:36 smithi016 ceph-mon[35851]: pgmap v845: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T08:58:36.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:36 smithi007 ceph-mon[32462]: pgmap v845: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T08:58:38.124 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-26T08:58:38.124 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 509, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 510, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 118237, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 750282, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 70, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 7, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.060052031, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.013174186, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.044587758, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.001903652, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 508, 2024-03-26T08:58:38.125 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.024991321, 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000049195 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 852, 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000165334, 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000194 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.126 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 417, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 407, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 825214, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 223907, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 8, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.049761501, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.018733148, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.026973700, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.006818148, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 406, 2024-03-26T08:58:38.127 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.059039902, 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000145418 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 820256, 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 209472 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 3396, 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2050, 2024-03-26T08:58:38.128 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 899806, 2024-03-26T08:58:38.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 1095092, 2024-03-26T08:58:38.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 10, 2024-03-26T08:58:38.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-03-26T08:58:38.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.171091609, 2024-03-26T08:58:38.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.066285165, 2024-03-26T08:58:38.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.132710863, 2024-03-26T08:58:38.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.005143646, 2024-03-26T08:58:38.129 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:38.129 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2049, 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.181322845, 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000088493 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4947, 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001360630, 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000275 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 569504 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.130 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-03-26T08:58:38.131 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-03-26T08:58:38.131 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-03-26T08:58:38.131 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.131 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-03-26T08:58:38.131 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-26T08:58:38.131 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-26T08:58:38.131 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 79, 2024-03-26T08:58:38.131 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.004181020, 2024-03-26T08:58:38.132 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000052924 2024-03-26T08:58:38.132 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:38.132 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.132 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-03-26T08:58:38.132 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-26T08:58:38.132 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-26T08:58:38.132 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-26T08:58:38.133 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000007811, 2024-03-26T08:58:38.133 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000003905 2024-03-26T08:58:38.133 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:38.133 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.133 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-03-26T08:58:38.133 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 381, 2024-03-26T08:58:38.133 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 3, 2024-03-26T08:58:38.133 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-03-26T08:58:38.134 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3, 2024-03-26T08:58:38.134 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001440747, 2024-03-26T08:58:38.134 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000480249 2024-03-26T08:58:38.134 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.134 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-03-26T08:58:38.134 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 378, 2024-03-26T08:58:38.134 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-03-26T08:58:38.134 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-03-26T08:58:38.134 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 11, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 35, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 10, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 25, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 20, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 4, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 4, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 4, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 4, 2024-03-26T08:58:38.135 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 437, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 36, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 378, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 23, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 1, 2024-03-26T08:58:38.136 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 1, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 28, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 1027, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 1746536863, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 28, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T08:58:38.137 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 28, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 2, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-03-26T08:58:38.138 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 1, 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 92, 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 57, 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 58, 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 1, 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-03-26T08:58:38.139 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-03-26T08:58:38.140 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 7, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 7, 2024-03-26T08:58:38.141 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-03-26T08:58:38.142 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-03-26T08:58:38.142 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 2, 2024-03-26T08:58:38.142 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-03-26T08:58:38.142 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-03-26T08:58:38.142 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-03-26T08:58:38.142 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-03-26T08:58:38.142 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-03-26T08:58:38.142 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 4205490, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 0, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.009356945, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001559490 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 0 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 39, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 41, 2024-03-26T08:58:38.143 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 2, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 58, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 61, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 3, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 35, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 39, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 4, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 4, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 4, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 50376, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 223612 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 382, 2024-03-26T08:58:38.144 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 381, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 13, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 1, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-03-26T08:58:38.145 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.146 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000149754, 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000074877 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-03-26T08:58:38.147 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.148 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-03-26T08:58:38.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.149 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.149 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.149 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-03-26T08:58:38.149 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.149 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.150 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 1, 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001290993, 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001290993 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.151 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.152 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.153 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-03-26T08:58:38.154 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.155 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.155 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.155 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-03-26T08:58:38.155 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.155 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.155 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.155 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.156 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 1, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 0, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 1, 2024-03-26T08:58:38.157 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 2, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 2, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 980, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:38.158 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:38.159 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:38.160 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 267269, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 132, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 7624, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 160, 2024-03-26T08:58:38.161 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 191552, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 992, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 74, 2024-03-26T08:58:38.162 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 22547, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 28, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 74, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 74, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.286995174, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.003878313 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 74, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 100 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 74, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 28, 2024-03-26T08:58:38.163 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 46, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 0, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 10, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 0, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 6, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 7, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 6, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-03-26T08:58:38.164 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 28, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 20, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-03-26T08:58:38.165 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-03-26T08:58:38.166 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 87, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 2, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 9, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 13, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 4, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 21, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 0, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 2 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 1, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-03-26T08:58:38.167 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-03-26T08:58:38.168 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 4318, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1431714, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 4318, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 4318, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1431714, 2024-03-26T08:58:38.169 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:38.170 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 74, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 22547, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 74, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 22547, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-03-26T08:58:38.171 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 74, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 74, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 74, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 74, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.172 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-03-26T08:58:38.173 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x561e1241a460": { 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:38.174 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 7, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 11186, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 7, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 6, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 11186, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:38.175 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-26T08:58:38.500 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:38 smithi016 ceph-mon[35851]: pgmap v846: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-26T08:58:38.642 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:38 smithi044 ceph-mon[36461]: pgmap v846: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-26T08:58:38.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:38 smithi007 ceph-mon[32462]: pgmap v846: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-26T08:58:38.926 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-26T08:58:38.926 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-03-26T08:58:38.926 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=1 2024-03-26T08:58:38.926 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=92 2024-03-26T08:58:38.926 DEBUG:tasks.check_counter:Getting stats from f 2024-03-26T08:58:38.927 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-26T08:58:39.207 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:58:40.686 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:40 smithi044 ceph-mon[36461]: pgmap v847: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T08:58:40.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:40 smithi016 ceph-mon[35851]: pgmap v847: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T08:58:40.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:40 smithi007 ceph-mon[32462]: pgmap v847: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T08:58:40.991 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-26T08:58:40.991 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:40.991 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 3241, 2024-03-26T08:58:40.991 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 3243, 2024-03-26T08:58:40.991 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 916681, 2024-03-26T08:58:40.991 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 1030645, 2024-03-26T08:58:40.991 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 6, 2024-03-26T08:58:40.991 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-03-26T08:58:40.991 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.225058574, 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.065899358, 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.056734649, 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.087491078, 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3241, 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.115349501, 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000035590 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:40.992 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 4, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 233, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 876, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 568650, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 5, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 3, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.011639390, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.008723460, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.001107081, 2024-03-26T08:58:40.993 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000063012, 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 231, 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.015633256, 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000067676 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 567952 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.994 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 323, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 313, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 806220, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 184190, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.040625460, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.014228734, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.021460083, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.006570286, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 313, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.018233811, 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000058254 2024-03-26T08:58:40.995 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 805952, 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 183872 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-03-26T08:58:40.996 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 5312, 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.124970629, 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000023526 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:40.997 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 0, 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 0, 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-03-26T08:58:40.998 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2024-03-26T08:58:40.999 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 1, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-03-26T08:58:41.000 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-03-26T08:58:41.001 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 0 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 0, 2024-03-26T08:58:41.002 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-03-26T08:58:41.003 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 7, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-03-26T08:58:41.004 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 0, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 4196264, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 4196123, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.005 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 7 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 3, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 3, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 0, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 1, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 1, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 0, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 0, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 0, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 0, 2024-03-26T08:58:41.006 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 0, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 0, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 48592, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 215420 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 0, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 0, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 0, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-03-26T08:58:41.007 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-03-26T08:58:41.008 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.009 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.010 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.011 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.012 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-03-26T08:58:41.012 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.012 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.013 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.014 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.015 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.016 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.017 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.018 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 3, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 2, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 2, 2024-03-26T08:58:41.019 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 0, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 0, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 3651, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:41.020 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:41.021 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:41.022 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 182104, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 49, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:41.023 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 7624, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 160, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 8320, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 13, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-03-26T08:58:41.024 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 3244, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 21, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 3, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 3244, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3244, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 4.683923137, 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001443872 2024-03-26T08:58:41.025 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 3244, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 3245 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 3244, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 3244, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 0, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 2131, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 0, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 1111, 2024-03-26T08:58:41.026 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 1, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-03-26T08:58:41.027 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-03-26T08:58:41.028 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 87, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 2, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 4, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 5, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 1, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 0, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 2, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 0 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.029 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 0, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-03-26T08:58:41.030 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 3564, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1383588, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 3564, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 3564, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1383588, 2024-03-26T08:58:41.031 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 3244, 2024-03-26T08:58:41.032 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 1965, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 3244, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1965, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:41.033 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 3244, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 3244, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 3244, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 3244, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-26T08:58:41.034 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:41.035 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x562b773ec6e0": { 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:41.036 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:41.037 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:41.037 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:41.037 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-26T08:58:41.823 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-26T08:58:41.823 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-26T08:58:41.823 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-03-26T08:58:41.823 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-03-26T08:58:41.824 DEBUG:tasks.check_counter:Getting stats from i 2024-03-26T08:58:41.824 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-26T08:58:42.105 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:58:42.486 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:42 smithi044 ceph-mon[36461]: pgmap v848: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-26T08:58:42.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:42 smithi016 ceph-mon[35851]: pgmap v848: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-26T08:58:42.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:42 smithi007 ceph-mon[32462]: pgmap v848: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 439, 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 445, 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 107424, 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 99719, 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 16, 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.043245023, 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.011714928, 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.030775546, 2024-03-26T08:58:43.915 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000766817, 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 439, 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.027437764, 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000062500 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 818, 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000151065, 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000184 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 1936, 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 1584 2024-03-26T08:58:43.916 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 3733, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2144, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 1712301, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 867606, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 20, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 6, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.207320237, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.074125052, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.159529095, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.011969003, 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:43.917 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2148, 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.160109444, 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000074538 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4921, 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000998870, 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000202 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 818160, 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 186624 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 64, 2024-03-26T08:58:43.918 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 292, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 11370, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 1086120, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 56, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 5, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.031399831, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.010182244, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.018316719, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000241269, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 290, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.029727778, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000102509 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:43.919 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 61, 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000008896, 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000145 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 2496, 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 574304 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 1, 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-MDSRank": { 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-26T08:58:43.920 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 50, 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001954336, 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000039086 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "finisher-PurgeQueue": { 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "queue_len": 0, 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "complete_latency": { 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000008102, 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000004051 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.921 INFO:teuthology.orchestra.run.smithi044.stdout: "mds": { 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "request": 395, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "reply": 4, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "reply_latency": { 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 4, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.001424927, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000356231 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_reply": 0, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "forward": 391, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_complete": 0, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_fetch_keys": 0, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_commit": 11, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_split": 0, 2024-03-26T08:58:43.922 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_merge": 0, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes": 34, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_top": 11, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_bottom": 23, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pin_tail": 0, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_pinned": 18, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_expired": 4, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "inodes_with_caps": 0, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "caps": 0, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "subtrees": 3, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse": 458, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_hit": 44, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_forward": 388, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_discover": 24, 2024-03-26T08:58:43.923 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_dir_fetch": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_remote_ino": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "traverse_lock": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "load_cent": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "q": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "exported": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "exported_inodes": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "imported": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "imported_inodes": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_dir_fetch": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_backtrace_fetch": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "openino_peer_discover": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rfiles": 1027, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rbytes": 1746536863, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "root_rsnaps": 0, 2024-03-26T08:58:43.924 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_set_tag": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_inotable_repaired": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_inodes": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "scrub_file_inodes": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_inode_file_caps": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_grant": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T08:58:43.925 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps": 0, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_caps_dirty": 0, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_cap_release": 0, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "process_request_cap_release": 2 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_cache": { 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update": 0, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_update_receipt": 95, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_try_discover": 59, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_send_discover": 61, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "dir_handle_discover": 0, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays": 0, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_delayed": 0, 2024-03-26T08:58:43.926 INFO:teuthology.orchestra.run.smithi044.stdout: "num_strays_enqueuing": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_created": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_enqueued": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_reintegrated": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "strays_migrated": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_processing": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_enqueued": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "num_recovering_prioritized": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_started": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "recovery_completed": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_exportdir": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_flush": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragmentdir": 0, 2024-03-26T08:58:43.927 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_fragstats": 0, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "ireq_inodestats": 0 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_log": { 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "evlrg": 0, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "evadd": 7, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "evex": 0, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "evtrm": 0, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "ev": 7, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "evexg": 0, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "evexd": 0, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "segadd": 2, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "segex": 0, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "segtrm": 0, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "seg": 2, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "segmjr": 2, 2024-03-26T08:58:43.928 INFO:teuthology.orchestra.run.smithi044.stdout: "segexg": 0, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "segexd": 0, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "expos": 4194304, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "wrpos": 4196264, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "rdpos": 0, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "jlat": { 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 6, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.010391713, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.001731952 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "replayed": 0 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_mem": { 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "ino": 38, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "ino+": 40, 2024-03-26T08:58:43.929 INFO:teuthology.orchestra.run.smithi044.stdout: "ino-": 2, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "dir": 57, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "dir+": 62, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "dir-": 5, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "dn": 34, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "dn+": 38, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "dn-": 4, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "cap": 0, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "cap+": 4, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "cap-": 4, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "rss": 51184, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "heap": 223612 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.930 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_server": { 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_client_request": 396, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "dispatch_server_request": 0, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_request": 395, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_client_session": 21, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "handle_peer_request": 0, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "req_create_latency": { 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getattr_latency": { 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000691202, 2024-03-26T08:58:43.931 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000345601 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getfilelock_latency": { 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "req_link_latency": { 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookup_latency": { 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 2, 2024-03-26T08:58:43.932 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000733725, 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000366862 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookuphash_latency": { 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupino_latency": { 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupname_latency": { 2024-03-26T08:58:43.933 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupparent_latency": { 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lookupsnap_latency": { 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.934 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "req_lssnap_latency": { 2024-03-26T08:58:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mkdir_latency": { 2024-03-26T08:58:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.935 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mknod_latency": { 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "req_mksnap_latency": { 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.936 INFO:teuthology.orchestra.run.smithi044.stdout: "req_open_latency": { 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "req_readdir_latency": { 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rename_latency": { 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.937 INFO:teuthology.orchestra.run.smithi044.stdout: "req_renamesnap_latency": { 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "req_snapdiff_latency": { 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmdir_latency": { 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.938 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmsnap_latency": { 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "req_rmxattr_latency": { 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setattr_latency": { 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.939 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setdirlayout_latency": { 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setfilelock_latency": { 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setlayout_latency": { 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.940 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "req_setxattr_latency": { 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "req_symlink_latency": { 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "req_unlink_latency": { 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.941 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_revoke_eviction": 0, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "cap_acquisition_throttle": 0, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "req_getvxattr_latency": { 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_sessions": { 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "session_count": 1, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "session_add": 3, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "session_remove": 2, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_open": 1, 2024-03-26T08:58:43.942 INFO:teuthology.orchestra.run.smithi044.stdout: "sessions_stale": 0, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "total_load": 3, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "average_load": 1, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "avg_session_uptime": 994, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "mdthresh_evicted": 0 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:43.943 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:43.944 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:43.945 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 250929, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 123, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 7624, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 160, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:43.946 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 185631, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 948, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "objecter": { 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "op_active": 0, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "op_laggy": 0, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send": 45, 2024-03-26T08:58:43.947 INFO:teuthology.orchestra.run.smithi044.stdout: "op_send_bytes": 12577, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "op_resend": 28, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "op_reply": 45, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "op_latency": { 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 45, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.234996004, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.005222133 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "op_inflight": 0, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "oplen_avg": { 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 45, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 69 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "op": 45, 2024-03-26T08:58:43.948 INFO:teuthology.orchestra.run.smithi044.stdout: "op_r": 0, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "op_w": 45, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "op_rmw": 0, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "op_pg": 0, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_stat": 0, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_create": 10, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_read": 0, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_write": 6, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writefull": 8, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_writesame": 0, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_append": 0, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_zero": 0, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_truncate": 0, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_delete": 6, 2024-03-26T08:58:43.949 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_mapext": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_sparse_read": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_clonerange": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_getxattr": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_setxattr": 20, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_cmpxattr": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_rmxattr": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_resetxattrs": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_call": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_watch": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_notify": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls": 0, 2024-03-26T08:58:43.950 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_pgls_filter": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "osdop_other": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_active": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_send": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_resend": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "linger_ping": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_active": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_send": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "poolop_resend": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_active": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_send": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "poolstat_resend": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_active": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_send": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "statfs_resend": 0, 2024-03-26T08:58:43.951 INFO:teuthology.orchestra.run.smithi044.stdout: "command_active": 0, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "command_send": 0, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "command_resend": 0, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "map_epoch": 87, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "map_full": 0, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "map_inc": 2, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_sessions": 10, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_open": 14, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_session_close": 4, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_laggy": 0, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_wr": 17, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_rd": 0, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_del": 2 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.952 INFO:teuthology.orchestra.run.smithi044.stdout: "oft": { 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_objs": 1, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_kv_pairs": 0, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_updates": 0, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "omap_total_removes": 0 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "osd-slow-ops": { 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "slow_ops_count": 0 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "purge_queue": { 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops": 0, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing": 0, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executing_high_water": 0, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed_ops": 0, 2024-03-26T08:58:43.953 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_executed": 0, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "pq_item_in_journal": 0 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 4232, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1428889, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 4232, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:43.954 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 4232, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1428889, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_bytes": { 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:43.955 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 45, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 12577, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 45, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 12577, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-objecter_ops": { 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 1024, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:43.956 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 45, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 45, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 45, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 45, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle": { 2024-03-26T08:58:43.957 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.958 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-write_buf_throttle-0x55f27f83a460": { 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 3758096384, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 7, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 1960, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 7, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 6, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 1960, 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:43.959 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:43.960 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:43.960 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-26T08:58:44.729 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:44 smithi044 ceph-mon[36461]: pgmap v849: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-03-26T08:58:44.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:44 smithi016 ceph-mon[35851]: pgmap v849: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-03-26T08:58:44.765 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-26T08:58:44.765 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-26T08:58:44.765 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-03-26T08:58:44.765 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=95 2024-03-26T08:58:44.765 DEBUG:tasks.check_counter:Getting stats from l 2024-03-26T08:58:44.765 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-26T08:58:44.799 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:44 smithi007 ceph-mon[32462]: pgmap v849: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-03-26T08:58:45.033 INFO:teuthology.orchestra.run.smithi044.stderr:Inferring config /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T08:58:46.536 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:46 smithi044 ceph-mon[36461]: pgmap v850: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-26T08:58:46.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:46 smithi016 ceph-mon[35851]: pgmap v850: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-26T08:58:46.793 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 0, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 2, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 0, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 0, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 0, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.001596555, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.000370865, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:46.794 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T08:58:46.795 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 6, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 237, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 3392, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 197296, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 4, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 2, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.010920222, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.007668418, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.001441357, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 236, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.022800176, 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000096610 2024-03-26T08:58:46.796 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 197296 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_messages": 313, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages": 308, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_bytes": 777984, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_bytes": 185360, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_created_connections": 2, 2024-03-26T08:58:46.797 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_active_connections": 1, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_total_time": 0.040397498, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_send_time": 0.014001116, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_recv_time": 0.021420949, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_running_fast_dispatch_time": 0.006626110, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 307, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.016864492, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000054933 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_handle_ack_lat": { 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:46.798 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_recv_encrypted_bytes": 777984, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "msgr_send_encrypted_bytes": 185360 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "cct": { 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "total_workers": 0, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "unhealthy_workers": 0 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "mempool": { 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_bytes": 0, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "bloom_filter_items": 0, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_alloc_items": 0, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T08:58:46.799 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_data_items": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_other_items": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_bytes": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_extent_items": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_bytes": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_blob_items": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T08:58:46.800 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_fsck_items": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_bytes": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_txc_items": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_bytes": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluestore_writing_items": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_bytes": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_items": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_reader_items": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "bluefs_file_writer_items": 0, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_bytes": 148744, 2024-03-26T08:58:46.801 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_anon_items": 30, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_bytes": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "buffer_meta_items": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_bytes": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_items": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_bytes": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_mapbl_items": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_bytes": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_pglog_items": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_bytes": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_items": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_mapping_items": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_bytes": 0, 2024-03-26T08:58:46.802 INFO:teuthology.orchestra.run.smithi044.stdout: "pgmap_items": 0, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_bytes": 928, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_co_items": 2, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_bytes": 0, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_1_items": 0, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_bytes": 0, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "unittest_2_items": 0 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "val": 0, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "max": 104857600, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "get_started": 0, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "get": 315, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "get_sum": 733773, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_fail": 0, 2024-03-26T08:58:46.803 INFO:teuthology.orchestra.run.smithi044.stdout: "get_or_fail_success": 315, 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: "take": 0, 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: "take_sum": 0, 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: "put": 315, 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: "put_sum": 733773, 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: "wait": { 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: "avgcount": 0, 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: "sum": 0.000000000, 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: "avgtime": 0.000000000 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-03-26T08:58:46.804 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-03-26T08:58:46.805 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:46 smithi007 ceph-mon[32462]: pgmap v850: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-03-26T08:58:47.491 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-26T08:58:47.491 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-26T08:58:47.492 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-03-26T08:58:47.492 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-03-26T08:58:47.492 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-26T08:58:47.504 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-26T08:58:47.505 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:58:47.505 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T08:58:47.527 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-03-26T08:58:47.527 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T08:58:47.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:58:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 -- 172.21.15.7:0/3217720980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca41021d0 msgr2=0x7efca41045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 --2- 172.21.15.7:0/3217720980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca41021d0 0x7efca41045c0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009900 tx=0x7efc9c02bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 -- 172.21.15.7:0/3217720980 shutdown_connections 2024-03-26T08:58:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 --2- 172.21.15.7:0/3217720980 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca4107430 0x7efca4109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 --2- 172.21.15.7:0/3217720980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca4104b00 0x7efca4106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 --2- 172.21.15.7:0/3217720980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca41021d0 0x7efca41045c0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 -- 172.21.15.7:0/3217720980 >> 172.21.15.7:0/3217720980 conn(0x7efca40fbb40 msgr2=0x7efca40fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 -- 172.21.15.7:0/3217720980 shutdown_connections 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 -- 172.21.15.7:0/3217720980 wait complete. 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.747+0000 7efca9836640 1 Processor -- start 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca9836640 1 -- start start 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca9836640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41021d0 0x7efca41979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca9836640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca4104b00 0x7efca4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca9836640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca4107430 0x7efca419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca9836640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca410d370 con 0x7efca4104b00 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca9836640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca410d1f0 con 0x7efca41021d0 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca9836640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca410d4f0 con 0x7efca4107430 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca4104b00 0x7efca4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca27fc640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca4104b00 0x7efca4197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34904/0 (socket says 172.21.15.7:34904) 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca27fc640 1 -- 172.21.15.7:0/1057717809 learned_addr learned my addr 172.21.15.7:0/1057717809 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca2ffd640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41021d0 0x7efca41979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca37fe640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca4107430 0x7efca419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca37fe640 1 -- 172.21.15.7:0/1057717809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41021d0 msgr2=0x7efca41979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca37fe640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41021d0 0x7efca41979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca37fe640 1 -- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca4104b00 msgr2=0x7efca4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca37fe640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca4104b00 0x7efca4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca37fe640 1 -- 172.21.15.7:0/1057717809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca419f610 con 0x7efca4107430 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.748+0000 7efca27fc640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca4104b00 0x7efca4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.749+0000 7efca37fe640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca4107430 0x7efca419efa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efc94009fd0 tx=0x7efc9400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.749+0000 7efca8834640 1 -- 172.21.15.7:0/1057717809 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9400ce20 con 0x7efca4107430 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.749+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca419f900 con 0x7efca4107430 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.749+0000 7efca8834640 1 -- 172.21.15.7:0/1057717809 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efc94005020 con 0x7efca4107430 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.749+0000 7efca8834640 1 -- 172.21.15.7:0/1057717809 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94010720 con 0x7efca4107430 2024-03-26T08:58:47.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.749+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca419fea0 con 0x7efca4107430 2024-03-26T08:58:47.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.750+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca40fd630 con 0x7efca4107430 2024-03-26T08:58:47.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.751+0000 7efca8834640 1 -- 172.21.15.7:0/1057717809 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7efc94005540 con 0x7efca4107430 2024-03-26T08:58:47.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.752+0000 7efca8834640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc900a2800 0x7efc900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:47.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.752+0000 7efca2ffd640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc900a2800 0x7efc900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:47.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.752+0000 7efca8834640 1 -- 172.21.15.7:0/1057717809 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efc940cf1a0 con 0x7efca4107430 2024-03-26T08:58:47.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.752+0000 7efca2ffd640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc900a2800 0x7efc900a4cc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efc9c02c7b0 tx=0x7efc9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:47.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.760+0000 7efca8834640 1 -- 172.21.15.7:0/1057717809 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efc9409c290 con 0x7efca4107430 2024-03-26T08:58:47.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.946+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efca40622a0 con 0x7efca4107430 2024-03-26T08:58:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.948+0000 7efca8834640 1 -- 172.21.15.7:0/1057717809 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7efc9409bb90 con 0x7efca4107430 2024-03-26T08:58:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:58:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:58:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc900a2800 msgr2=0x7efc900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc900a2800 0x7efc900a4cc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efc9c02c7b0 tx=0x7efc9c005fd0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca4107430 msgr2=0x7efca419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca4107430 0x7efca419efa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efc94009fd0 tx=0x7efc9400ef10 comp rx=0 tx=0).stop 2024-03-26T08:58:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 shutdown_connections 2024-03-26T08:58:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7efc900a2800 0x7efc900a4cc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efca4107430 0x7efca419efa0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efca4104b00 0x7efca4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 --2- 172.21.15.7:0/1057717809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41021d0 0x7efca41979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:47.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 >> 172.21.15.7:0/1057717809 conn(0x7efca40fbb40 msgr2=0x7efca4102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:47.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 shutdown_connections 2024-03-26T08:58:47.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:47.950+0000 7efca9836640 1 -- 172.21.15.7:0/1057717809 wait complete. 2024-03-26T08:58:47.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T08:35:26.563154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T08:40:08.100906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T08:40:08.784972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T08:58:47.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T08:58:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.068+0000 7f711ae87640 1 -- 172.21.15.7:0/674083056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 msgr2=0x7f71141097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.068+0000 7f711ae87640 1 --2- 172.21.15.7:0/674083056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 0x7f71141097e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f710c009ca0 tx=0x7f710c02c0f0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.068+0000 7f711ae87640 1 -- 172.21.15.7:0/674083056 shutdown_connections 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.068+0000 7f711ae87640 1 --2- 172.21.15.7:0/674083056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 0x7f71141097e0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.068+0000 7f711ae87640 1 --2- 172.21.15.7:0/674083056 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114104ac0 0x7f7114106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.068+0000 7f711ae87640 1 --2- 172.21.15.7:0/674083056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7114102190 0x7f7114104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.068+0000 7f711ae87640 1 -- 172.21.15.7:0/674083056 >> 172.21.15.7:0/674083056 conn(0x7f71140fbb00 msgr2=0x7f71140fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.068+0000 7f711ae87640 1 -- 172.21.15.7:0/674083056 shutdown_connections 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.068+0000 7f711ae87640 1 -- 172.21.15.7:0/674083056 wait complete. 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f711ae87640 1 Processor -- start 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f711ae87640 1 -- start start 2024-03-26T08:58:48.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f711ae87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7114102190 0x7f7114101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f711ae87640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114104ac0 0x7f7114100140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f711ae87640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 0x7f7114100680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f711ae87640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711410d350 con 0x7f71141073f0 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f711ae87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711410d1d0 con 0x7f7114102190 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f711ae87640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711410d4d0 con 0x7f7114104ac0 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7119424640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 0x7f7114100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7119424640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 0x7f7114100680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34912/0 (socket says 172.21.15.7:34912) 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7119424640 1 -- 172.21.15.7:0/4063518900 learned_addr learned my addr 172.21.15.7:0/4063518900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7118c23640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7114102190 0x7f7114101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7113fff640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114104ac0 0x7f7114100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7119424640 1 -- 172.21.15.7:0/4063518900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114104ac0 msgr2=0x7f7114100140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7119424640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114104ac0 0x7f7114100140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7119424640 1 -- 172.21.15.7:0/4063518900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7114102190 msgr2=0x7f7114101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7119424640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7114102190 0x7f7114101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7119424640 1 -- 172.21.15.7:0/4063518900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7114100f10 con 0x7f71141073f0 2024-03-26T08:58:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7119424640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 0x7f7114100680 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f710c037040 tx=0x7f710c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.069+0000 7f7111ffb640 1 -- 172.21.15.7:0/4063518900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c041020 con 0x7f71141073f0 2024-03-26T08:58:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.070+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71141a1a90 con 0x7f71141073f0 2024-03-26T08:58:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.070+0000 7f7111ffb640 1 -- 172.21.15.7:0/4063518900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f710c034cd0 con 0x7f71141073f0 2024-03-26T08:58:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.070+0000 7f7111ffb640 1 -- 172.21.15.7:0/4063518900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c039040 con 0x7f71141073f0 2024-03-26T08:58:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.070+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71141a1fe0 con 0x7f71141073f0 2024-03-26T08:58:48.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.071+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7114068970 con 0x7f71141073f0 2024-03-26T08:58:48.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.072+0000 7f7111ffb640 1 -- 172.21.15.7:0/4063518900 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f710c044020 con 0x7f71141073f0 2024-03-26T08:58:48.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.072+0000 7f7111ffb640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71040a2800 0x7f71040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.073+0000 7f7111ffb640 1 -- 172.21.15.7:0/4063518900 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f710c0f0940 con 0x7f71141073f0 2024-03-26T08:58:48.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.073+0000 7f7118c23640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71040a2800 0x7f71040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.073+0000 7f7118c23640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71040a2800 0x7f71040a4cc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f71080095f0 tx=0x7f7108005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.077+0000 7f7111ffb640 1 -- 172.21.15.7:0/4063518900 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f710c04b020 con 0x7f71141073f0 2024-03-26T08:58:48.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.259+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71140622a0 con 0x7f71141073f0 2024-03-26T08:58:48.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.260+0000 7f7111ffb640 1 -- 172.21.15.7:0/4063518900 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f710c0bda30 con 0x7f71141073f0 2024-03-26T08:58:48.258 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-03-26T08:58:48.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71040a2800 msgr2=0x7f71040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71040a2800 0x7f71040a4cc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f71080095f0 tx=0x7f7108005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 msgr2=0x7f7114100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 0x7f7114100680 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f710c037040 tx=0x7f710c032f40 comp rx=0 tx=0).stop 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 shutdown_connections 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f71040a2800 0x7f71040a4cc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71141073f0 0x7f7114100680 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7114104ac0 0x7f7114100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 --2- 172.21.15.7:0/4063518900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7114102190 0x7f7114101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 >> 172.21.15.7:0/4063518900 conn(0x7f71140fbb00 msgr2=0x7f7114102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 shutdown_connections 2024-03-26T08:58:48.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.262+0000 7f711ae87640 1 -- 172.21.15.7:0/4063518900 wait complete. 2024-03-26T08:58:48.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T08:58:48.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.378+0000 7fde508a1640 1 -- 172.21.15.7:0/3163898509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1005b0 msgr2=0x7fde4c1009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.378+0000 7fde508a1640 1 --2- 172.21.15.7:0/3163898509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1005b0 0x7fde4c1009d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fde44009900 tx=0x7fde4402bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:48.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.378+0000 7fde508a1640 1 -- 172.21.15.7:0/3163898509 shutdown_connections 2024-03-26T08:58:48.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.378+0000 7fde508a1640 1 --2- 172.21.15.7:0/3163898509 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde4c102160 0x7fde4c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.378+0000 7fde508a1640 1 --2- 172.21.15.7:0/3163898509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c1017a0 0x7fde4c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.378+0000 7fde508a1640 1 --2- 172.21.15.7:0/3163898509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1005b0 0x7fde4c1009d0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.378+0000 7fde508a1640 1 -- 172.21.15.7:0/3163898509 >> 172.21.15.7:0/3163898509 conn(0x7fde4c0fbb00 msgr2=0x7fde4c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:48.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.378+0000 7fde508a1640 1 -- 172.21.15.7:0/3163898509 shutdown_connections 2024-03-26T08:58:48.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.378+0000 7fde508a1640 1 -- 172.21.15.7:0/3163898509 wait complete. 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde508a1640 1 Processor -- start 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde508a1640 1 -- start start 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde508a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde4c1005b0 0x7fde4c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde508a1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1017a0 0x7fde4c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde508a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c102160 0x7fde4c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde508a1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c10c250 con 0x7fde4c1017a0 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde508a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c10c0d0 con 0x7fde4c102160 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde508a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c10c3d0 con 0x7fde4c1005b0 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1017a0 0x7fde4c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.379+0000 7fde4a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde4c1005b0 0x7fde4c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde4c1005b0 0x7fde4c197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44654/0 (socket says 172.21.15.7:44654) 2024-03-26T08:58:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde49d9b640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1017a0 0x7fde4c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34930/0 (socket says 172.21.15.7:34930) 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4a59c640 1 -- 172.21.15.7:0/2663926970 learned_addr learned my addr 172.21.15.7:0/2663926970 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4ad9d640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c102160 0x7fde4c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4ad9d640 1 -- 172.21.15.7:0/2663926970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde4c1005b0 msgr2=0x7fde4c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4ad9d640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde4c1005b0 0x7fde4c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4ad9d640 1 -- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1017a0 msgr2=0x7fde4c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4ad9d640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1017a0 0x7fde4c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4ad9d640 1 -- 172.21.15.7:0/2663926970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c19f630 con 0x7fde4c102160 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4a59c640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde4c1005b0 0x7fde4c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde49d9b640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1017a0 0x7fde4c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde4ad9d640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c102160 0x7fde4c19efc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009fd0 tx=0x7fde3c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde3b7fe640 1 -- 172.21.15.7:0/2663926970 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00ce20 con 0x7fde4c102160 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde3b7fe640 1 -- 172.21.15.7:0/2663926970 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde3c005020 con 0x7fde4c102160 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c19f920 con 0x7fde4c102160 2024-03-26T08:58:48.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde3b7fe640 1 -- 172.21.15.7:0/2663926970 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c010720 con 0x7fde4c102160 2024-03-26T08:58:48.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.380+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c19fe20 con 0x7fde4c102160 2024-03-26T08:58:48.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.381+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c1a0110 con 0x7fde4c102160 2024-03-26T08:58:48.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.382+0000 7fde3b7fe640 1 -- 172.21.15.7:0/2663926970 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fde3c005540 con 0x7fde4c102160 2024-03-26T08:58:48.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.382+0000 7fde3b7fe640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fde340a2800 0x7fde340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.383+0000 7fde4a59c640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fde340a2800 0x7fde340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.383+0000 7fde3b7fe640 1 -- 172.21.15.7:0/2663926970 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fde3c0cf140 con 0x7fde4c102160 2024-03-26T08:58:48.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.383+0000 7fde4a59c640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fde340a2800 0x7fde340a4cc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fde44006010 tx=0x7fde4402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:48.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.387+0000 7fde3b7fe640 1 -- 172.21.15.7:0/2663926970 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde3c09c1e0 con 0x7fde4c102160 2024-03-26T08:58:48.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:48 smithi007 ceph-mon[32462]: pgmap v851: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-26T08:58:48.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:48 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1057717809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:58:48.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:48 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/4063518900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:58:48.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.571+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde4c1a04a0 con 0x7fde4c102160 2024-03-26T08:58:48.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.572+0000 7fde3b7fe640 1 -- 172.21.15.7:0/2663926970 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde3c09bae0 con 0x7fde4c102160 2024-03-26T08:58:48.571 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-03-26T08:58:48.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fde340a2800 msgr2=0x7fde340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fde340a2800 0x7fde340a4cc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fde44006010 tx=0x7fde4402df90 comp rx=0 tx=0).stop 2024-03-26T08:58:48.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c102160 msgr2=0x7fde4c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c102160 0x7fde4c19efc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009fd0 tx=0x7fde3c00ef10 comp rx=0 tx=0).stop 2024-03-26T08:58:48.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 shutdown_connections 2024-03-26T08:58:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fde340a2800 0x7fde340a4cc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c102160 0x7fde4c19efc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde4c1017a0 0x7fde4c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 --2- 172.21.15.7:0/2663926970 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fde4c1005b0 0x7fde4c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 >> 172.21.15.7:0/2663926970 conn(0x7fde4c0fbb00 msgr2=0x7fde4c069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 shutdown_connections 2024-03-26T08:58:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.574+0000 7fde508a1640 1 -- 172.21.15.7:0/2663926970 wait complete. 2024-03-26T08:58:48.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T08:58:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.691+0000 7fc5f2587640 1 -- 172.21.15.7:0/2204731118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec101bb0 msgr2=0x7fc5ec101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.691+0000 7fc5f2587640 1 --2- 172.21.15.7:0/2204731118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec101bb0 0x7fc5ec101fd0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009930 tx=0x7fc5d402bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.691+0000 7fc5f2587640 1 -- 172.21.15.7:0/2204731118 shutdown_connections 2024-03-26T08:58:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.691+0000 7fc5f2587640 1 --2- 172.21.15.7:0/2204731118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec101210 0x7fc5ec108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.691+0000 7fc5f2587640 1 --2- 172.21.15.7:0/2204731118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5ec100850 0x7fc5ec100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.691+0000 7fc5f2587640 1 --2- 172.21.15.7:0/2204731118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec101bb0 0x7fc5ec101fd0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.691+0000 7fc5f2587640 1 -- 172.21.15.7:0/2204731118 >> 172.21.15.7:0/2204731118 conn(0x7fc5ec0fb2c0 msgr2=0x7fc5ec0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.691+0000 7fc5f2587640 1 -- 172.21.15.7:0/2204731118 shutdown_connections 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.691+0000 7fc5f2587640 1 -- 172.21.15.7:0/2204731118 wait complete. 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5f2587640 1 Processor -- start 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5f2587640 1 -- start start 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5f2587640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec100850 0x7fc5ec196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5f2587640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5ec101210 0x7fc5ec196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5f2587640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec101bb0 0x7fc5ec19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5f2587640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec10c260 con 0x7fc5ec100850 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5f2587640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec10c0e0 con 0x7fc5ec101bb0 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5f2587640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec10c3e0 con 0x7fc5ec101210 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec100850 0x7fc5ec196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5ebfff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec100850 0x7fc5ec196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34946/0 (socket says 172.21.15.7:34946) 2024-03-26T08:58:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5ebfff640 1 -- 172.21.15.7:0/935714444 learned_addr learned my addr 172.21.15.7:0/935714444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5eb7fe640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5ec101210 0x7fc5ec196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.692+0000 7fc5f0b24640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec101bb0 0x7fc5ec19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5ebfff640 1 -- 172.21.15.7:0/935714444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5ec101210 msgr2=0x7fc5ec196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5ebfff640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5ec101210 0x7fc5ec196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5ebfff640 1 -- 172.21.15.7:0/935714444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec101bb0 msgr2=0x7fc5ec19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5ebfff640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec101bb0 0x7fc5ec19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5ebfff640 1 -- 172.21.15.7:0/935714444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec19e580 con 0x7fc5ec100850 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5ebfff640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec100850 0x7fc5ec196960 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc5d402c7b0 tx=0x7fc5d402c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5e97fa640 1 -- 172.21.15.7:0/935714444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4041070 con 0x7fc5ec100850 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5e97fa640 1 -- 172.21.15.7:0/935714444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc5d40054e0 con 0x7fc5ec100850 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec19e810 con 0x7fc5ec100850 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5e97fa640 1 -- 172.21.15.7:0/935714444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4034440 con 0x7fc5ec100850 2024-03-26T08:58:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.693+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec19ed00 con 0x7fc5ec100850 2024-03-26T08:58:48.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.694+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec1079b0 con 0x7fc5ec100850 2024-03-26T08:58:48.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.695+0000 7fc5e97fa640 1 -- 172.21.15.7:0/935714444 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fc5d4044070 con 0x7fc5ec100850 2024-03-26T08:58:48.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.695+0000 7fc5e97fa640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5d80a2800 0x7fc5d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:48.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.696+0000 7fc5e97fa640 1 -- 172.21.15.7:0/935714444 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc5d40f0540 con 0x7fc5ec100850 2024-03-26T08:58:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.696+0000 7fc5eb7fe640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5d80a2800 0x7fc5d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.696+0000 7fc5eb7fe640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5d80a2800 0x7fc5d80a4cc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec197e70 tx=0x7fc5dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.700+0000 7fc5e97fa640 1 -- 172.21.15.7:0/935714444 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc5d40bcf90 con 0x7fc5ec100850 2024-03-26T08:58:48.723 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:48 smithi044 ceph-mon[36461]: pgmap v851: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-26T08:58:48.723 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:48 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1057717809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:58:48.723 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:48 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/4063518900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:58:48.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:48 smithi016 ceph-mon[35851]: pgmap v851: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-26T08:58:48.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:48 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1057717809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:58:48.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:48 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/4063518900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T08:58:48.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.882+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5ec0622a0 con 0x7fc5ec100850 2024-03-26T08:58:48.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.883+0000 7fc5e97fa640 1 -- 172.21.15.7:0/935714444 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5d40c1030 con 0x7fc5ec100850 2024-03-26T08:58:48.882 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-03-26T08:58:48.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5d80a2800 msgr2=0x7fc5d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5d80a2800 0x7fc5d80a4cc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec197e70 tx=0x7fc5dc005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec100850 msgr2=0x7fc5ec196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec100850 0x7fc5ec196960 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc5d402c7b0 tx=0x7fc5d402c6d0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 shutdown_connections 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fc5d80a2800 0x7fc5d80a4cc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5ec101bb0 0x7fc5ec19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc5ec101210 0x7fc5ec196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 --2- 172.21.15.7:0/935714444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec100850 0x7fc5ec196960 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 >> 172.21.15.7:0/935714444 conn(0x7fc5ec0fb2c0 msgr2=0x7fc5ec104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 shutdown_connections 2024-03-26T08:58:48.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:48.885+0000 7fc5f2587640 1 -- 172.21.15.7:0/935714444 wait complete. 2024-03-26T08:58:48.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T08:58:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.000+0000 7fa396f13640 1 -- 172.21.15.7:0/943067778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901005c0 msgr2=0x7fa3901009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.000+0000 7fa396f13640 1 --2- 172.21.15.7:0/943067778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901005c0 0x7fa3901009e0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa388009ca0 tx=0x7fa38802c0f0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.000+0000 7fa396f13640 1 -- 172.21.15.7:0/943067778 shutdown_connections 2024-03-26T08:58:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.000+0000 7fa396f13640 1 --2- 172.21.15.7:0/943067778 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa390102170 0x7fa390108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.000+0000 7fa396f13640 1 --2- 172.21.15.7:0/943067778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3901017b0 0x7fa390101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.000+0000 7fa396f13640 1 --2- 172.21.15.7:0/943067778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901005c0 0x7fa3901009e0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.000+0000 7fa396f13640 1 -- 172.21.15.7:0/943067778 >> 172.21.15.7:0/943067778 conn(0x7fa3900fbb30 msgr2=0x7fa3900fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:49.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.000+0000 7fa396f13640 1 -- 172.21.15.7:0/943067778 shutdown_connections 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.000+0000 7fa396f13640 1 -- 172.21.15.7:0/943067778 wait complete. 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa396f13640 1 Processor -- start 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa396f13640 1 -- start start 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa396f13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3901005c0 0x7fa390197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa396f13640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901017b0 0x7fa390197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa396f13640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa390102170 0x7fa39019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa396f13640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39010c210 con 0x7fa3901017b0 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa396f13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39010c090 con 0x7fa3901005c0 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa396f13640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39010c390 con 0x7fa390102170 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901017b0 0x7fa390197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901017b0 0x7fa390197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34958/0 (socket says 172.21.15.7:34958) 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38ffff640 1 -- 172.21.15.7:0/1532899901 learned_addr learned my addr 172.21.15.7:0/1532899901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa394caf640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3901005c0 0x7fa390197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:49.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa3954b0640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa390102170 0x7fa39019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38ffff640 1 -- 172.21.15.7:0/1532899901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa390102170 msgr2=0x7fa39019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38ffff640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa390102170 0x7fa39019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38ffff640 1 -- 172.21.15.7:0/1532899901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3901005c0 msgr2=0x7fa390197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38ffff640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3901005c0 0x7fa390197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38ffff640 1 -- 172.21.15.7:0/1532899901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39019f650 con 0x7fa3901017b0 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38ffff640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901017b0 0x7fa390197f70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa38000e9f0 tx=0x7fa38000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.001+0000 7fa38dffb640 1 -- 172.21.15.7:0/1532899901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000cd80 con 0x7fa3901017b0 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.002+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39019f940 con 0x7fa3901017b0 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.002+0000 7fa38dffb640 1 -- 172.21.15.7:0/1532899901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa380010910 con 0x7fa3901017b0 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.002+0000 7fa38dffb640 1 -- 172.21.15.7:0/1532899901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380018990 con 0x7fa3901017b0 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.002+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39019fe90 con 0x7fa3901017b0 2024-03-26T08:58:49.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.003+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3901a00c0 con 0x7fa3901017b0 2024-03-26T08:58:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.004+0000 7fa38dffb640 1 -- 172.21.15.7:0/1532899901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fa380010ac0 con 0x7fa3901017b0 2024-03-26T08:58:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.004+0000 7fa38dffb640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa37c0a2800 0x7fa37c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.005+0000 7fa38dffb640 1 -- 172.21.15.7:0/1532899901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3800cf180 con 0x7fa3901017b0 2024-03-26T08:58:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.005+0000 7fa394caf640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa37c0a2800 0x7fa37c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:49.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.005+0000 7fa394caf640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa37c0a2800 0x7fa37c0a4cc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa38802c700 tx=0x7fa388032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:49.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.008+0000 7fa38dffb640 1 -- 172.21.15.7:0/1532899901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa38009bbd0 con 0x7fa3901017b0 2024-03-26T08:58:49.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.207+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3901a0290 con 0x7fa3901017b0 2024-03-26T08:58:49.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.207+0000 7fa38dffb640 1 -- 172.21.15.7:0/1532899901 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+580 (secure 0 0 0) 0x7fa38009fc70 con 0x7fa3901017b0 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1981295866 2024-03-27T08:32:58.217753+0000 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/1336720039 2024-03-27T08:32:58.217753+0000 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/444923255 2024-03-27T08:32:58.217753+0000 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3914280225 2024-03-27T08:32:30.975912+0000 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2786531355 2024-03-27T08:32:30.975912+0000 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/3110430694 2024-03-27T08:32:30.975912+0000 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/3110430694 2024-03-27T08:32:30.975912+0000 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/804148713 2024-03-27T08:32:30.975912+0000 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/1336720039 2024-03-27T08:32:58.217753+0000 2024-03-26T08:58:49.206 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2665937259 2024-03-27T08:32:58.217753+0000 2024-03-26T08:58:49.207 INFO:teuthology.orchestra.run.smithi007.stderr:listed 10 entries 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa37c0a2800 msgr2=0x7fa37c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa37c0a2800 0x7fa37c0a4cc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa38802c700 tx=0x7fa388032590 comp rx=0 tx=0).stop 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901017b0 msgr2=0x7fa390197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901017b0 0x7fa390197f70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa38000e9f0 tx=0x7fa38000eeb0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 shutdown_connections 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fa37c0a2800 0x7fa37c0a4cc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa390102170 0x7fa39019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3901017b0 0x7fa390197f70 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 --2- 172.21.15.7:0/1532899901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3901005c0 0x7fa390197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 >> 172.21.15.7:0/1532899901 conn(0x7fa3900fbb30 msgr2=0x7fa3900fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 shutdown_connections 2024-03-26T08:58:49.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.209+0000 7fa396f13640 1 -- 172.21.15.7:0/1532899901 wait complete. 2024-03-26T08:58:49.218 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-26T08:58:49.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:58:49.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /tmp/cephfs 2024-03-26T08:58:49.363 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-03-26T08:58:49.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:58:49.363 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-03-26T08:58:49.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:58:49.428 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-03-26T08:58:49.479 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-03-26T08:58:49.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:58:49.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-03-26T08:58:49.528 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:49 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2663926970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:58:49.528 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:49 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/935714444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:58:49.528 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:49 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1532899901' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:58:49.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T08:58:49.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-03-26T08:58:49.600 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-26T08:58:49.612 INFO:tasks.cephadm:Teardown begin 2024-03-26T08:58:49.612 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-26T08:58:49.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:58:49.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.724+0000 7f4d09601640 1 -- 172.21.15.7:0/275709940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 msgr2=0x7f4d041019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.724+0000 7f4d09601640 1 --2- 172.21.15.7:0/275709940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 0x7f4d041019f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009ca0 tx=0x7f4cf402c0f0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.724+0000 7f4d09601640 1 -- 172.21.15.7:0/275709940 shutdown_connections 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.724+0000 7f4d09601640 1 --2- 172.21.15.7:0/275709940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d04103180 0x7f4d04109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.724+0000 7f4d09601640 1 --2- 172.21.15.7:0/275709940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d041027c0 0x7f4d04102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.724+0000 7f4d09601640 1 --2- 172.21.15.7:0/275709940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 0x7f4d041019f0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.724+0000 7f4d09601640 1 -- 172.21.15.7:0/275709940 >> 172.21.15.7:0/275709940 conn(0x7f4d040fcb60 msgr2=0x7f4d040fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.724+0000 7f4d09601640 1 -- 172.21.15.7:0/275709940 shutdown_connections 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.724+0000 7f4d09601640 1 -- 172.21.15.7:0/275709940 wait complete. 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.725+0000 7f4d09601640 1 Processor -- start 2024-03-26T08:58:49.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.725+0000 7f4d09601640 1 -- start start 2024-03-26T08:58:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.725+0000 7f4d09601640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 0x7f4d04197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.725+0000 7f4d09601640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d041027c0 0x7f4d04197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.725+0000 7f4d09601640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d04103180 0x7f4d0419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.725+0000 7f4d09601640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0410d240 con 0x7f4d041015d0 2024-03-26T08:58:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.725+0000 7f4d09601640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0410d0c0 con 0x7f4d041027c0 2024-03-26T08:58:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.725+0000 7f4d09601640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0410d3c0 con 0x7f4d04103180 2024-03-26T08:58:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 0x7f4d04197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:49.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 0x7f4d04197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34972/0 (socket says 172.21.15.7:34972) 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d037fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d04103180 0x7f4d0419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d02ffd640 1 -- 172.21.15.7:0/1052310069 learned_addr learned my addr 172.21.15.7:0/1052310069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d027fc640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d041027c0 0x7f4d04197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d02ffd640 1 -- 172.21.15.7:0/1052310069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d04103180 msgr2=0x7f4d0419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d02ffd640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d04103180 0x7f4d0419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d02ffd640 1 -- 172.21.15.7:0/1052310069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d041027c0 msgr2=0x7f4d04197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d02ffd640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d041027c0 0x7f4d04197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d02ffd640 1 -- 172.21.15.7:0/1052310069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0419f570 con 0x7f4d041015d0 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d02ffd640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 0x7f4d04197950 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40324c0 tx=0x7f4cf4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4cfbfff640 1 -- 172.21.15.7:0/1052310069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4041020 con 0x7f4d041015d0 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0419f800 con 0x7f4d041015d0 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4cfbfff640 1 -- 172.21.15.7:0/1052310069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4cf4034cd0 con 0x7f4d041015d0 2024-03-26T08:58:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4cfbfff640 1 -- 172.21.15.7:0/1052310069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4039040 con 0x7f4d041015d0 2024-03-26T08:58:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.726+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0419fcf0 con 0x7f4d041015d0 2024-03-26T08:58:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.728+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d04108940 con 0x7f4d041015d0 2024-03-26T08:58:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.728+0000 7f4cfbfff640 1 -- 172.21.15.7:0/1052310069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f4cf4044020 con 0x7f4d041015d0 2024-03-26T08:58:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.729+0000 7f4cfbfff640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cf00a27f0 0x7f4cf00a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.729+0000 7f4d027fc640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cf00a27f0 0x7f4cf00a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.729+0000 7f4cfbfff640 1 -- 172.21.15.7:0/1052310069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4cf40f08b0 con 0x7f4d041015d0 2024-03-26T08:58:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.730+0000 7f4d027fc640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cf00a27f0 0x7f4cf00a4cb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4d04198e60 tx=0x7f4cec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:49.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.734+0000 7f4cfbfff640 1 -- 172.21.15.7:0/1052310069 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4cf404b020 con 0x7f4d041015d0 2024-03-26T08:58:49.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:49 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2663926970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:58:49.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:49 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/935714444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:58:49.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:49 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1532899901' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:58:49.822 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:49 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2663926970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T08:58:49.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:49 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/935714444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T08:58:49.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:49 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1532899901' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T08:58:49.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.960+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d0402d070 con 0x7f4d041015d0 2024-03-26T08:58:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.960+0000 7f4cfbfff640 1 -- 172.21.15.7:0/1052310069 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f4cf40bd9e0 con 0x7f4d041015d0 2024-03-26T08:58:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:58:49.960 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:58:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:58:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cf00a27f0 msgr2=0x7f4cf00a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cf00a27f0 0x7f4cf00a4cb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4d04198e60 tx=0x7f4cec005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 msgr2=0x7f4d04197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:49.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 0x7f4d04197950 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4cf40324c0 tx=0x7f4cf4032f40 comp rx=0 tx=0).stop 2024-03-26T08:58:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 shutdown_connections 2024-03-26T08:58:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f4cf00a27f0 0x7f4cf00a4cb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f4d04103180 0x7f4d0419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d041027c0 0x7f4d04197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 --2- 172.21.15.7:0/1052310069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d041015d0 0x7f4d04197950 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.962+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 >> 172.21.15.7:0/1052310069 conn(0x7f4d040fcb60 msgr2=0x7f4d04105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.963+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 shutdown_connections 2024-03-26T08:58:49.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:49.963+0000 7f4d09601640 1 -- 172.21.15.7:0/1052310069 wait complete. 2024-03-26T08:58:49.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T08:58:50.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 -- 172.21.15.7:0/2983144233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa01028d0 msgr2=0x7f5aa0102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/2983144233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa01028d0 0x7f5aa0102d50 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009900 tx=0x7f5a9002bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:50.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 -- 172.21.15.7:0/2983144233 shutdown_connections 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/2983144233 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aa0103290 0x7f5aa0109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/2983144233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa01028d0 0x7f5aa0102d50 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/2983144233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa01016e0 0x7f5aa0101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 -- 172.21.15.7:0/2983144233 >> 172.21.15.7:0/2983144233 conn(0x7f5aa00fcc30 msgr2=0x7f5aa00ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 -- 172.21.15.7:0/2983144233 shutdown_connections 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 -- 172.21.15.7:0/2983144233 wait complete. 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 Processor -- start 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 -- start start 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa01016e0 0x7f5aa01979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aa01028d0 0x7f5aa0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa0103290 0x7f5aa019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa010d3b0 con 0x7f5aa0103290 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa010d230 con 0x7f5aa01016e0 2024-03-26T08:58:50.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.080+0000 7f5aa84a1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa010d530 con 0x7f5aa01028d0 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa6a3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa0103290 0x7f5aa019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa6a3e640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa0103290 0x7f5aa019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34990/0 (socket says 172.21.15.7:34990) 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa6a3e640 1 -- 172.21.15.7:0/39861010 learned_addr learned my addr 172.21.15.7:0/39861010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa623d640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa01016e0 0x7f5aa01979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa5a3c640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aa01028d0 0x7f5aa0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa6a3e640 1 -- 172.21.15.7:0/39861010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aa01028d0 msgr2=0x7f5aa0197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa6a3e640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aa01028d0 0x7f5aa0197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa6a3e640 1 -- 172.21.15.7:0/39861010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa01016e0 msgr2=0x7f5aa01979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa6a3e640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa01016e0 0x7f5aa01979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa6a3e640 1 -- 172.21.15.7:0/39861010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa019f5c0 con 0x7f5aa0103290 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa6a3e640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa0103290 0x7f5aa019ef50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800eb10 tx=0x7f5a9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5a977fe640 1 -- 172.21.15.7:0/39861010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98004b20 con 0x7f5aa0103290 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5a977fe640 1 -- 172.21.15.7:0/39861010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5a98004cc0 con 0x7f5aa0103290 2024-03-26T08:58:50.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa019f8b0 con 0x7f5aa0103290 2024-03-26T08:58:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5a977fe640 1 -- 172.21.15.7:0/39861010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98010760 con 0x7f5aa0103290 2024-03-26T08:58:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.081+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa019fdd0 con 0x7f5aa0103290 2024-03-26T08:58:50.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.082+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa01089a0 con 0x7f5aa0103290 2024-03-26T08:58:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.083+0000 7f5a977fe640 1 -- 172.21.15.7:0/39861010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f5a98005020 con 0x7f5aa0103290 2024-03-26T08:58:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.084+0000 7f5a977fe640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a8c0a2800 0x7f5a8c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.084+0000 7f5aa623d640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a8c0a2800 0x7f5a8c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.084+0000 7f5a977fe640 1 -- 172.21.15.7:0/39861010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5a980cf180 con 0x7f5aa0103290 2024-03-26T08:58:50.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.084+0000 7f5aa623d640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a8c0a2800 0x7f5a8c0a4cc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0095f0 tx=0x7f5a9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:50.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.088+0000 7f5a977fe640 1 -- 172.21.15.7:0/39861010 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a9809c1e0 con 0x7f5aa0103290 2024-03-26T08:58:50.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.313+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5aa002d070 con 0x7f5aa0103290 2024-03-26T08:58:50.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.313+0000 7f5a977fe640 1 -- 172.21.15.7:0/39861010 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11357 (secure 0 0 0) 0x7f5a9809bae0 con 0x7f5aa0103290 2024-03-26T08:58:50.312 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:58:50.312 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2764125554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2764125554},{"type":"v1","addr":"172.21.15.7:6835","nonce":2764125554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1591045564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1591045564},{"type":"v1","addr":"172.21.15.7:6837","nonce":1591045564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14667,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2041082244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2041082244},{"type":"v1","addr":"172.21.15.7:6839","nonce":2041082244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14679,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/3829840154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":3829840154},{"type":"v1","addr":"172.21.15.7:6841","nonce":3829840154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24410,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/76809036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":76809036},{"type":"v1","addr":"172.21.15.16:6833","nonce":76809036}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6833/3400459626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":3400459626},{"type":"v1","addr":"172.21.15.44:6833","nonce":3400459626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T08:40:09.583324+0000","modified":"2024-03-26T08:44:26.102445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24512,"mds_1":24502,"mds_2":24478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/1837116746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1837116746},{"type":"v1","addr":"172.21.15.16:6835","nonce":1837116746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24466":{"gid":24466,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.44:6835/2525817262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":2525817262},{"type":"v1","addr":"172.21.15.44:6835","nonce":2525817262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24476":{"gid":24476,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/7790244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":7790244},{"type":"v1","addr":"172.21.15.16:6837","nonce":7790244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24478":{"gid":24478,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6837/1604095881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1604095881},{"type":"v1","addr":"172.21.15.44:6837","nonce":1604095881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24502":{"gid":24502,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.44:6839/2441928071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6838","nonce":2441928071},{"type":"v1","addr":"172.21.15.44:6839","nonce":2441928071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/3011473533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":3011473533},{"type":"v1","addr":"172.21.15.16:6839","nonce":3011473533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24478,24502,24512]},"id":1}]} 2024-03-26T08:58:50.313 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-03-26T08:58:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a8c0a2800 msgr2=0x7f5a8c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a8c0a2800 0x7f5a8c0a4cc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0095f0 tx=0x7f5a9c005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa0103290 msgr2=0x7f5aa019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa0103290 0x7f5aa019ef50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800eb10 tx=0x7f5a9800c2d0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 shutdown_connections 2024-03-26T08:58:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f5a8c0a2800 0x7f5a8c0a4cc0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aa0103290 0x7f5aa019ef50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5aa01028d0 0x7f5aa0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 --2- 172.21.15.7:0/39861010 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa01016e0 0x7f5aa01979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 >> 172.21.15.7:0/39861010 conn(0x7f5aa00fcc30 msgr2=0x7f5aa0105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 shutdown_connections 2024-03-26T08:58:50.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.315+0000 7f5aa84a1640 1 -- 172.21.15.7:0/39861010 wait complete. 2024-03-26T08:58:50.325 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T08:58:50.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.430+0000 7fab8eb37640 1 -- 172.21.15.7:0/438295439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 msgr2=0x7fab88106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.430+0000 7fab8eb37640 1 --2- 172.21.15.7:0/438295439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 0x7fab88106ee0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fab74009900 tx=0x7fab7402bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.430+0000 7fab8eb37640 1 -- 172.21.15.7:0/438295439 shutdown_connections 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.430+0000 7fab8eb37640 1 --2- 172.21.15.7:0/438295439 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab88107420 0x7fab88109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.430+0000 7fab8eb37640 1 --2- 172.21.15.7:0/438295439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 0x7fab88106ee0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.430+0000 7fab8eb37640 1 --2- 172.21.15.7:0/438295439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab881021c0 0x7fab881045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.430+0000 7fab8eb37640 1 -- 172.21.15.7:0/438295439 >> 172.21.15.7:0/438295439 conn(0x7fab880fbbb0 msgr2=0x7fab880fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.430+0000 7fab8eb37640 1 -- 172.21.15.7:0/438295439 shutdown_connections 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.430+0000 7fab8eb37640 1 -- 172.21.15.7:0/438295439 wait complete. 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8eb37640 1 Processor -- start 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8eb37640 1 -- start start 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8eb37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab881021c0 0x7fab881979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8eb37640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 0x7fab88197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8eb37640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab88107420 0x7fab8819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8eb37640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8810d350 con 0x7fab88104af0 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8eb37640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8810d1d0 con 0x7fab881021c0 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8eb37640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8810d4d0 con 0x7fab88107420 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab87fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 0x7fab88197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab87fff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 0x7fab88197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35002/0 (socket says 172.21.15.7:35002) 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8c8d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab881021c0 0x7fab881979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab87fff640 1 -- 172.21.15.7:0/1165112416 learned_addr learned my addr 172.21.15.7:0/1165112416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab8d0d4640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab88107420 0x7fab8819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab87fff640 1 -- 172.21.15.7:0/1165112416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab88107420 msgr2=0x7fab8819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab87fff640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab88107420 0x7fab8819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab87fff640 1 -- 172.21.15.7:0/1165112416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab881021c0 msgr2=0x7fab881979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab87fff640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab881021c0 0x7fab881979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab87fff640 1 -- 172.21.15.7:0/1165112416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8819f5e0 con 0x7fab88104af0 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.431+0000 7fab87fff640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 0x7fab88197f00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fab74000c00 tx=0x7fab7402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.432+0000 7fab85ffb640 1 -- 172.21.15.7:0/1165112416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74041070 con 0x7fab88104af0 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.432+0000 7fab85ffb640 1 -- 172.21.15.7:0/1165112416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab74004450 con 0x7fab88104af0 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.432+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8819f870 con 0x7fab88104af0 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.432+0000 7fab85ffb640 1 -- 172.21.15.7:0/1165112416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74034440 con 0x7fab88104af0 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.432+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8819fdb0 con 0x7fab88104af0 2024-03-26T08:58:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.433+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab880fd6a0 con 0x7fab88104af0 2024-03-26T08:58:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.434+0000 7fab85ffb640 1 -- 172.21.15.7:0/1165112416 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fab74044070 con 0x7fab88104af0 2024-03-26T08:58:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.435+0000 7fab85ffb640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fab780a2880 0x7fab780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.435+0000 7fab8c8d3640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fab780a2880 0x7fab780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.435+0000 7fab85ffb640 1 -- 172.21.15.7:0/1165112416 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fab740f06b0 con 0x7fab88104af0 2024-03-26T08:58:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.435+0000 7fab8c8d3640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fab780a2880 0x7fab780a4d40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009bd0 tx=0x7fab7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.439+0000 7fab85ffb640 1 -- 172.21.15.7:0/1165112416 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab740bd190 con 0x7fab88104af0 2024-03-26T08:58:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.623+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab8802d240 con 0x7fab88104af0 2024-03-26T08:58:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.625+0000 7fab85ffb640 1 -- 172.21.15.7:0/1165112416 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22567 (secure 0 0 0) 0x7fab740c1230 con 0x7fab88104af0 2024-03-26T08:58:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-03-26T08:58:50.624 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"fsid":"2c753f3e-eb4b-11ee-95d0-87774f69a715","created":"2024-03-26T08:31:55.831994+0000","modified":"2024-03-26T08:40:59.413729+0000","last_up_change":"2024-03-26T08:39:18.709879+0000","last_in_change":"2024-03-26T08:38:57.887100+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T08:35:26.563154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T08:40:08.100906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T08:40:08.784972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"11d5305b-e22f-42b9-a7b6-acc96b92baab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6803","nonce":2166020682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6805","nonce":2166020682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6809","nonce":2166020682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2166020682},{"type":"v1","addr":"172.21.15.7:6807","nonce":2166020682}]},"public_addr":"172.21.15.7:6803/2166020682","cluster_addr":"172.21.15.7:6805/2166020682","heartbeat_back_addr":"172.21.15.7:6809/2166020682","heartbeat_front_addr":"172.21.15.7:6807/2166020682","state":["exists","up"]},{"osd":1,"uuid":"95b219c6-a22b-4b66-9fa8-967c59f003ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6801","nonce":1464490925}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6803","nonce":1464490925}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6807","nonce":1464490925}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1464490925},{"type":"v1","addr":"172.21.15.16:6805","nonce":1464490925}]},"public_addr":"172.21.15.16:6801/1464490925","cluster_addr":"172.21.15.16:6803/1464490925","heartbeat_back_addr":"172.21.15.16:6807/1464490925","heartbeat_front_addr":"172.21.15.16:6805/1464490925","state":["exists","up"]},{"osd":2,"uuid":"10c4a5af-45a4-4285-b6b4-0bc980ec6faa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6801","nonce":2047416654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6803","nonce":2047416654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6807","nonce":2047416654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":2047416654},{"type":"v1","addr":"172.21.15.44:6805","nonce":2047416654}]},"public_addr":"172.21.15.44:6801/2047416654","cluster_addr":"172.21.15.44:6803/2047416654","heartbeat_back_addr":"172.21.15.44:6807/2047416654","heartbeat_front_addr":"172.21.15.44:6805/2047416654","state":["exists","up"]},{"osd":3,"uuid":"b46aab34-7b13-4e83-8fbd-7c207cd246e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6811","nonce":4236464716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6813","nonce":4236464716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6817","nonce":4236464716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":4236464716},{"type":"v1","addr":"172.21.15.7:6815","nonce":4236464716}]},"public_addr":"172.21.15.7:6811/4236464716","cluster_addr":"172.21.15.7:6813/4236464716","heartbeat_back_addr":"172.21.15.7:6817/4236464716","heartbeat_front_addr":"172.21.15.7:6815/4236464716","state":["exists","up"]},{"osd":4,"uuid":"78a3ff9e-09b6-4255-96fd-9d155625811b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6809","nonce":3280306354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6811","nonce":3280306354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6815","nonce":3280306354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3280306354},{"type":"v1","addr":"172.21.15.16:6813","nonce":3280306354}]},"public_addr":"172.21.15.16:6809/3280306354","cluster_addr":"172.21.15.16:6811/3280306354","heartbeat_back_addr":"172.21.15.16:6815/3280306354","heartbeat_front_addr":"172.21.15.16:6813/3280306354","state":["exists","up"]},{"osd":5,"uuid":"ab45e6b1-3006-4af1-90f3-455a09f442f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6809","nonce":2190779473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6811","nonce":2190779473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6815","nonce":2190779473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2190779473},{"type":"v1","addr":"172.21.15.44:6813","nonce":2190779473}]},"public_addr":"172.21.15.44:6809/2190779473","cluster_addr":"172.21.15.44:6811/2190779473","heartbeat_back_addr":"172.21.15.44:6815/2190779473","heartbeat_front_addr":"172.21.15.44:6813/2190779473","state":["exists","up"]},{"osd":6,"uuid":"44d3581c-a611-48f8-92c7-4273e6639781","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6819","nonce":2859951867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6821","nonce":2859951867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6825","nonce":2859951867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2859951867},{"type":"v1","addr":"172.21.15.7:6823","nonce":2859951867}]},"public_addr":"172.21.15.7:6819/2859951867","cluster_addr":"172.21.15.7:6821/2859951867","heartbeat_back_addr":"172.21.15.7:6825/2859951867","heartbeat_front_addr":"172.21.15.7:6823/2859951867","state":["exists","up"]},{"osd":7,"uuid":"e87d365c-1fcf-40cd-93f6-5fc972c25217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6817","nonce":3902462229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6819","nonce":3902462229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6823","nonce":3902462229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3902462229},{"type":"v1","addr":"172.21.15.16:6821","nonce":3902462229}]},"public_addr":"172.21.15.16:6817/3902462229","cluster_addr":"172.21.15.16:6819/3902462229","heartbeat_back_addr":"172.21.15.16:6823/3902462229","heartbeat_front_addr":"172.21.15.16:6821/3902462229","state":["exists","up"]},{"osd":8,"uuid":"59f51bbc-b176-435d-bf69-4269bb0a6102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6817","nonce":3863516143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6819","nonce":3863516143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6823","nonce":3863516143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":3863516143},{"type":"v1","addr":"172.21.15.44:6821","nonce":3863516143}]},"public_addr":"172.21.15.44:6817/3863516143","cluster_addr":"172.21.15.44:6819/3863516143","heartbeat_back_addr":"172.21.15.44:6823/3863516143","heartbeat_front_addr":"172.21.15.44:6821/3863516143","state":["exists","up"]},{"osd":9,"uuid":"048f7ae9-18c6-4128-815b-082a9297ed68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6827","nonce":387825612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6829","nonce":387825612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6833","nonce":387825612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":387825612},{"type":"v1","addr":"172.21.15.7:6831","nonce":387825612}]},"public_addr":"172.21.15.7:6827/387825612","cluster_addr":"172.21.15.7:6829/387825612","heartbeat_back_addr":"172.21.15.7:6833/387825612","heartbeat_front_addr":"172.21.15.7:6831/387825612","state":["exists","up"]},{"osd":10,"uuid":"40e2915e-efad-4dbd-a244-cad31df0cd80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6825","nonce":1979394150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6827","nonce":1979394150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6831","nonce":1979394150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":1979394150},{"type":"v1","addr":"172.21.15.16:6829","nonce":1979394150}]},"public_addr":"172.21.15.16:6825/1979394150","cluster_addr":"172.21.15.16:6827/1979394150","heartbeat_back_addr":"172.21.15.16:6831/1979394150","heartbeat_front_addr":"172.21.15.16:6829/1979394150","state":["exists","up"]},{"osd":11,"uuid":"e3e817ae-4259-46ec-bbe3-91c24adbeb96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6825","nonce":2818540789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6827","nonce":2818540789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6831","nonce":2818540789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":2818540789},{"type":"v1","addr":"172.21.15.44:6829","nonce":2818540789}]},"public_addr":"172.21.15.44:6825/2818540789","cluster_addr":"172.21.15.44:6827/2818540789","heartbeat_back_addr":"172.21.15.44:6831/2818540789","heartbeat_front_addr":"172.21.15.44:6829/2818540789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:35.371160+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:34:59.178122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:23.357701+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:35:48.432973+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:13.872408+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:36:39.174796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:04.786687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:30.946925+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:37:56.644604+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:22.998611+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:38:49.273092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T08:39:16.643459+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/1981295866":"2024-03-27T08:32:58.217753+0000","172.21.15.7:6801/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/444923255":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/3914280225":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/2786531355":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6801/3110430694":"2024-03-27T08:32:30.975912+0000","172.21.15.7:0/804148713":"2024-03-27T08:32:30.975912+0000","172.21.15.7:6800/1336720039":"2024-03-27T08:32:58.217753+0000","172.21.15.7:0/2665937259":"2024-03-27T08:32:58.217753+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T08:58:50.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fab780a2880 msgr2=0x7fab780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fab780a2880 0x7fab780a4d40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009bd0 tx=0x7fab7c005f90 comp rx=0 tx=0).stop 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 msgr2=0x7fab88197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 0x7fab88197f00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fab74000c00 tx=0x7fab7402c640 comp rx=0 tx=0).stop 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 shutdown_connections 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fab780a2880 0x7fab780a4d40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fab88107420 0x7fab8819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab88104af0 0x7fab88197f00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 --2- 172.21.15.7:0/1165112416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab881021c0 0x7fab881979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 >> 172.21.15.7:0/1165112416 conn(0x7fab880fbbb0 msgr2=0x7fab88102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 shutdown_connections 2024-03-26T08:58:50.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.627+0000 7fab8eb37640 1 -- 172.21.15.7:0/1165112416 wait complete. 2024-03-26T08:58:50.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:50 smithi007 ceph-mon[32462]: pgmap v852: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-03-26T08:58:50.635 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:50 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1052310069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:58:50.636 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:50 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/39861010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:58:50.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 -- 172.21.15.7:0/2921533169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee580fff20 msgr2=0x7fee58100340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 --2- 172.21.15.7:0/2921533169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee580fff20 0x7fee58100340 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fee50009900 tx=0x7fee5002bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 -- 172.21.15.7:0/2921533169 shutdown_connections 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 --2- 172.21.15.7:0/2921533169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee58101220 0x7fee581016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 --2- 172.21.15.7:0/2921533169 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee58100880 0x7fee58100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 --2- 172.21.15.7:0/2921533169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee580fff20 0x7fee58100340 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 -- 172.21.15.7:0/2921533169 >> 172.21.15.7:0/2921533169 conn(0x7fee580fbb00 msgr2=0x7fee580fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 -- 172.21.15.7:0/2921533169 shutdown_connections 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 -- 172.21.15.7:0/2921533169 wait complete. 2024-03-26T08:58:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 Processor -- start 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.743+0000 7fee6006c640 1 -- start start 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee6006c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580fff20 0x7fee58199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee6006c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee58100880 0x7fee5819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee6006c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee58101220 0x7fee581a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee6006c640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5810f4f0 con 0x7fee58101220 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee6006c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5810f370 con 0x7fee580fff20 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee6006c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5810f670 con 0x7fee58100880 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5e609640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee58101220 0x7fee581a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5e609640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee58101220 0x7fee581a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35028/0 (socket says 172.21.15.7:35028) 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5e609640 1 -- 172.21.15.7:0/2574629953 learned_addr learned my addr 172.21.15.7:0/2574629953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5d607640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee58100880 0x7fee5819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5de08640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580fff20 0x7fee58199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5e609640 1 -- 172.21.15.7:0/2574629953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee58100880 msgr2=0x7fee5819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5e609640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee58100880 0x7fee5819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5e609640 1 -- 172.21.15.7:0/2574629953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580fff20 msgr2=0x7fee58199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5e609640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580fff20 0x7fee58199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5e609640 1 -- 172.21.15.7:0/2574629953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee581a17c0 con 0x7fee58101220 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee5e609640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee58101220 0x7fee581a1150 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fee4800eb10 tx=0x7fee4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee56ffd640 1 -- 172.21.15.7:0/2574629953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48004b20 con 0x7fee58101220 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee581a1ab0 con 0x7fee58101220 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee56ffd640 1 -- 172.21.15.7:0/2574629953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee48004cc0 con 0x7fee58101220 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee56ffd640 1 -- 172.21.15.7:0/2574629953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48010760 con 0x7fee58101220 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.744+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee581a1fb0 con 0x7fee58101220 2024-03-26T08:58:50.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.745+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5810adf0 con 0x7fee58101220 2024-03-26T08:58:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.747+0000 7fee56ffd640 1 -- 172.21.15.7:0/2574629953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fee48005000 con 0x7fee58101220 2024-03-26T08:58:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.747+0000 7fee56ffd640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee400a2800 0x7fee400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.748+0000 7fee5de08640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee400a2800 0x7fee400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.748+0000 7fee56ffd640 1 -- 172.21.15.7:0/2574629953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fee480cf200 con 0x7fee58101220 2024-03-26T08:58:50.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.748+0000 7fee5de08640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee400a2800 0x7fee400a4cc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee5002c7b0 tx=0x7fee50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:50.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.751+0000 7fee56ffd640 1 -- 172.21.15.7:0/2574629953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee48053740 con 0x7fee58101220 2024-03-26T08:58:50.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:50 smithi016 ceph-mon[35851]: pgmap v852: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-03-26T08:58:50.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:50 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1052310069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:58:50.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:50 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/39861010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:58:50.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:50 smithi044 ceph-mon[36461]: pgmap v852: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 7 op/s 2024-03-26T08:58:50.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:50 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1052310069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:58:50.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:50 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/39861010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T08:58:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.973+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fee5802d2c0 con 0x7fee58101220 2024-03-26T08:58:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.975+0000 7fee56ffd640 1 -- 172.21.15.7:0/2574629953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7fee4809c2a0 con 0x7fee58101220 2024-03-26T08:58:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-26T08:58:50.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee400a2800 msgr2=0x7fee400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee400a2800 0x7fee400a4cc0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fee5002c7b0 tx=0x7fee50005fd0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee58101220 msgr2=0x7fee581a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee58101220 0x7fee581a1150 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fee4800eb10 tx=0x7fee4800c2d0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 shutdown_connections 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fee400a2800 0x7fee400a4cc0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee58101220 0x7fee581a1150 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fee58100880 0x7fee5819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 --2- 172.21.15.7:0/2574629953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee580fff20 0x7fee58199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 >> 172.21.15.7:0/2574629953 conn(0x7fee580fbb00 msgr2=0x7fee580fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 shutdown_connections 2024-03-26T08:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:50.978+0000 7fee6006c640 1 -- 172.21.15.7:0/2574629953 wait complete. 2024-03-26T08:58:50.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-26T08:58:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.097+0000 7fb5504b2640 1 -- 172.21.15.7:0/3117875445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5481005c0 msgr2=0x7fb5481009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.097+0000 7fb5504b2640 1 --2- 172.21.15.7:0/3117875445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5481005c0 0x7fb5481009e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb538009900 tx=0x7fb53802bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.097+0000 7fb5504b2640 1 -- 172.21.15.7:0/3117875445 shutdown_connections 2024-03-26T08:58:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.097+0000 7fb5504b2640 1 --2- 172.21.15.7:0/3117875445 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb548102170 0x7fb548108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.097+0000 7fb5504b2640 1 --2- 172.21.15.7:0/3117875445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481017b0 0x7fb548101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.097+0000 7fb5504b2640 1 --2- 172.21.15.7:0/3117875445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5481005c0 0x7fb5481009e0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.097+0000 7fb5504b2640 1 -- 172.21.15.7:0/3117875445 >> 172.21.15.7:0/3117875445 conn(0x7fb5480fbb30 msgr2=0x7fb5480fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.097+0000 7fb5504b2640 1 -- 172.21.15.7:0/3117875445 shutdown_connections 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.097+0000 7fb5504b2640 1 -- 172.21.15.7:0/3117875445 wait complete. 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb5504b2640 1 Processor -- start 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb5504b2640 1 -- start start 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb5504b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5481005c0 0x7fb548196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb5504b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481017b0 0x7fb548196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb5504b2640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548102170 0x7fb54819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb5504b2640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54810c210 con 0x7fb548102170 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb5504b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54810c090 con 0x7fb5481017b0 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb5504b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54810c390 con 0x7fb5481005c0 2024-03-26T08:58:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54ea4f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548102170 0x7fb54819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54e24e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5481005c0 0x7fb548196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54ea4f640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548102170 0x7fb54819dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35038/0 (socket says 172.21.15.7:35038) 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54ea4f640 1 -- 172.21.15.7:0/1919786707 learned_addr learned my addr 172.21.15.7:0/1919786707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54e24e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5481005c0 0x7fb548196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.7:44782/0 (socket says 172.21.15.7:44782) 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54da4d640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481017b0 0x7fb548196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54e24e640 1 -- 172.21.15.7:0/1919786707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481017b0 msgr2=0x7fb548196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54e24e640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481017b0 0x7fb548196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54e24e640 1 -- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548102170 msgr2=0x7fb54819dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54e24e640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548102170 0x7fb54819dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54e24e640 1 -- 172.21.15.7:0/1919786707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54819e530 con 0x7fb5481005c0 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.098+0000 7fb54ea4f640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548102170 0x7fb54819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.099+0000 7fb54e24e640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5481005c0 0x7fb548196910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb53802c170 tx=0x7fb5380040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.099+0000 7fb53f7fe640 1 -- 172.21.15.7:0/1919786707 <== mon.1 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538041070 con 0x7fb5481005c0 2024-03-26T08:58:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.099+0000 7fb53f7fe640 1 -- 172.21.15.7:0/1919786707 <== mon.1 v2:172.21.15.44:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5380044c0 con 0x7fb5481005c0 2024-03-26T08:58:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.099+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54819e7c0 con 0x7fb5481005c0 2024-03-26T08:58:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.099+0000 7fb53f7fe640 1 -- 172.21.15.7:0/1919786707 <== mon.1 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380346a0 con 0x7fb5481005c0 2024-03-26T08:58:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.099+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54819ec80 con 0x7fb5481005c0 2024-03-26T08:58:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.100+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548107a50 con 0x7fb5481005c0 2024-03-26T08:58:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.101+0000 7fb53f7fe640 1 -- 172.21.15.7:0/1919786707 <== mon.1 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7fb538044070 con 0x7fb5481005c0 2024-03-26T08:58:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.101+0000 7fb53f7fe640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5340a2800 0x7fb5340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.102+0000 7fb54da4d640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5340a2800 0x7fb5340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.102+0000 7fb53f7fe640 1 -- 172.21.15.7:0/1919786707 <== mon.1 v2:172.21.15.44:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb5380f0a80 con 0x7fb5481005c0 2024-03-26T08:58:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.102+0000 7fb54da4d640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5340a2800 0x7fb5340a4cc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb548197e20 tx=0x7fb54400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.105+0000 7fb53f7fe640 1 -- 172.21.15.7:0/1919786707 <== mon.1 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb5380bd360 con 0x7fb5481005c0 2024-03-26T08:58:51.289 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:50 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a[32458]: 2024-03-26T08:58:50.974+0000 7fc46e548700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T08:58:51.323 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i[67649]: ignoring --setuser ceph since I am not root 2024-03-26T08:58:51.323 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i[67649]: ignoring --setgroup ceph since I am not root 2024-03-26T08:58:51.323 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mds[67653]: -- 172.21.15.44:0/2921841913 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aac0183a40 con 0x55aac0156800 2024-03-26T08:58:51.323 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i[67649]: starting mds.i at 2024-03-26T08:58:51.323 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c[67346]: ignoring --setuser ceph since I am not root 2024-03-26T08:58:51.324 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c[67346]: ignoring --setgroup ceph since I am not root 2024-03-26T08:58:51.324 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mds[67350]: -- 172.21.15.44:0/4254249386 <== mon.1 v2:172.21.15.44:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f272e09a40 con 0x55f272ddd000 2024-03-26T08:58:51.324 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c[67346]: starting mds.c at 2024-03-26T08:58:51.324 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f[67036]: ignoring --setuser ceph since I am not root 2024-03-26T08:58:51.324 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f[67036]: ignoring --setgroup ceph since I am not root 2024-03-26T08:58:51.324 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mds[67040]: -- 172.21.15.44:0/936804539 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556c2dd05a40 con 0x556c2dcd8c00 2024-03-26T08:58:51.324 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f[67036]: starting mds.f at 2024-03-26T08:58:51.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.332+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb5480622a0 con 0x7fb5481005c0 2024-03-26T08:58:51.383 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e[65608]: ignoring --setuser ceph since I am not root 2024-03-26T08:58:51.384 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e[65608]: ignoring --setgroup ceph since I am not root 2024-03-26T08:58:51.384 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mds[65612]: -- 172.21.15.16:0/3134856599 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579667bba40 con 0x55796678e800 2024-03-26T08:58:51.384 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e[65608]: starting mds.e at 2024-03-26T08:58:51.384 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b[66227]: ignoring --setuser ceph since I am not root 2024-03-26T08:58:51.384 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b[66227]: ignoring --setgroup ceph since I am not root 2024-03-26T08:58:51.384 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mds[66231]: -- 172.21.15.16:0/1336118389 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55808d7b7a40 con 0x55808d78ac00 2024-03-26T08:58:51.384 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b[66227]: starting mds.b at 2024-03-26T08:58:51.384 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h[65919]: ignoring --setuser ceph since I am not root 2024-03-26T08:58:51.384 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h[65919]: ignoring --setgroup ceph since I am not root 2024-03-26T08:58:51.385 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mds[65923]: -- 172.21.15.16:0/3683382855 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fcfa883a40 con 0x55fcfa857000 2024-03-26T08:58:51.385 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h[65919]: starting mds.h at 2024-03-26T08:58:51.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:51 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1165112416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:58:51.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:51 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2574629953' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T08:58:51.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:51 smithi007 ceph-mon[32462]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T08:58:51.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:51 smithi007 ceph-mon[32462]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T08:58:51.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:51 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/2574629953' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T08:58:51.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:51 smithi007 ceph-mon[32462]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T08:58:51.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:51 smithi007 ceph-mon[32462]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-26T08:58:51.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:51 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1919786707' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:58:51.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:51 smithi007 ceph-mon[32462]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:58:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1165112416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:58:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2574629953' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T08:58:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mon[35851]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T08:58:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mon[35851]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T08:58:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/2574629953' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T08:58:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mon[35851]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T08:58:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mon[35851]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-26T08:58:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1919786707' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:58:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:51 smithi016 ceph-mon[35851]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:58:51.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1165112416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T08:58:51.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2574629953' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T08:58:51.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mon[36461]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T08:58:51.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mon[36461]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T08:58:51.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/2574629953' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T08:58:51.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mon[36461]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T08:58:51.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mon[36461]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-26T08:58:51.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1919786707' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:58:51.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:51 smithi044 ceph-mon[36461]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T08:58:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.980+0000 7fb53f7fe640 1 -- 172.21.15.7:0/1919786707 <== mon.1 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb5380c1400 con 0x7fb5481005c0 2024-03-26T08:58:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5340a2800 msgr2=0x7fb5340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5340a2800 0x7fb5340a4cc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb548197e20 tx=0x7fb54400a2d0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5481005c0 msgr2=0x7fb548196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5481005c0 0x7fb548196910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb53802c170 tx=0x7fb5380040c0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 shutdown_connections 2024-03-26T08:58:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7fb5340a2800 0x7fb5340a4cc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb548102170 0x7fb54819dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5481017b0 0x7fb548196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 --2- 172.21.15.7:0/1919786707 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb5481005c0 0x7fb548196910 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 >> 172.21.15.7:0/1919786707 conn(0x7fb5480fbb30 msgr2=0x7fb548104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 shutdown_connections 2024-03-26T08:58:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:51.982+0000 7fb5504b2640 1 -- 172.21.15.7:0/1919786707 wait complete. 2024-03-26T08:58:51.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-26T08:58:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.101+0000 7f064ce34640 1 -- 172.21.15.7:0/3872358084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648102120 msgr2=0x7f0648104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.101+0000 7f064ce34640 1 --2- 172.21.15.7:0/3872358084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648102120 0x7f0648104510 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0638009900 tx=0x7f063802bc30 comp rx=0 tx=0).stop 2024-03-26T08:58:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.101+0000 7f064ce34640 1 -- 172.21.15.7:0/3872358084 shutdown_connections 2024-03-26T08:58:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.101+0000 7f064ce34640 1 --2- 172.21.15.7:0/3872358084 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0648107380 0x7f06481097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.101+0000 7f064ce34640 1 --2- 172.21.15.7:0/3872358084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0648104a50 0x7f0648106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.101+0000 7f064ce34640 1 --2- 172.21.15.7:0/3872358084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648102120 0x7f0648104510 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.101+0000 7f064ce34640 1 -- 172.21.15.7:0/3872358084 >> 172.21.15.7:0/3872358084 conn(0x7f06480fbad0 msgr2=0x7f06480fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:52.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.101+0000 7f064ce34640 1 -- 172.21.15.7:0/3872358084 shutdown_connections 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.101+0000 7f064ce34640 1 -- 172.21.15.7:0/3872358084 wait complete. 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f064ce34640 1 Processor -- start 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f064ce34640 1 -- start start 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f064ce34640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0648102120 0x7f06481979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f064ce34640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648104a50 0x7f0648197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f064ce34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0648107380 0x7f064819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f064ce34640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064810d380 con 0x7f0648104a50 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f064ce34640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064810d200 con 0x7f0648107380 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f064ce34640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064810d500 con 0x7f0648102120 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f063ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648104a50 0x7f0648197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f063ffff640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648104a50 0x7f0648197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35052/0 (socket says 172.21.15.7:35052) 2024-03-26T08:58:52.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f063ffff640 1 -- 172.21.15.7:0/1391779173 learned_addr learned my addr 172.21.15.7:0/1391779173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f0646d9d640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0648107380 0x7f064819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.102+0000 7f064659c640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0648102120 0x7f06481979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f063ffff640 1 -- 172.21.15.7:0/1391779173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0648102120 msgr2=0x7f06481979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f063ffff640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0648102120 0x7f06481979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f063ffff640 1 -- 172.21.15.7:0/1391779173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0648107380 msgr2=0x7f064819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f063ffff640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0648107380 0x7f064819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f063ffff640 1 -- 172.21.15.7:0/1391779173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064819f5f0 con 0x7f0648104a50 2024-03-26T08:58:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f063ffff640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648104a50 0x7f0648197f10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f063000c970 tx=0x7f063000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:52.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f063f7fe640 1 -- 172.21.15.7:0/1391779173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630007bb0 con 0x7f0648104a50 2024-03-26T08:58:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f063f7fe640 1 -- 172.21.15.7:0/1391779173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0630007d50 con 0x7f0648104a50 2024-03-26T08:58:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064819f8e0 con 0x7f0648104a50 2024-03-26T08:58:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f063f7fe640 1 -- 172.21.15.7:0/1391779173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000f450 con 0x7f0648104a50 2024-03-26T08:58:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.103+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064819fe30 con 0x7f0648104a50 2024-03-26T08:58:52.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.104+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06480fd5c0 con 0x7f0648104a50 2024-03-26T08:58:52.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.105+0000 7f063f7fe640 1 -- 172.21.15.7:0/1391779173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f06300046c0 con 0x7f0648104a50 2024-03-26T08:58:52.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.106+0000 7f063f7fe640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06340a2800 0x7f06340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:52.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.106+0000 7f064659c640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06340a2800 0x7f06340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:52.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.106+0000 7f063f7fe640 1 -- 172.21.15.7:0/1391779173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f06300ceaf0 con 0x7f0648104a50 2024-03-26T08:58:52.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.106+0000 7f064659c640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06340a2800 0x7f06340a4cc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f063802c240 tx=0x7f063802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:52.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.110+0000 7f063f7fe640 1 -- 172.21.15.7:0/1391779173 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0630014030 con 0x7f0648104a50 2024-03-26T08:58:52.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.314+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f064802d070 con 0x7f0648104a50 2024-03-26T08:58:52.549 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: pgmap v853: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 6 op/s 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: Cluster is now healthy 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.44:6835/3591607244,v1:172.21.15.44:6837/3591607244] up:boot 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.44:6834/4227990186,v1:172.21.15.44:6836/4227990186] up:boot 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.16:6838/2619839450,v1:172.21.15.16:6839/2619839450] up:boot 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.16:6834/1580408254,v1:172.21.15.16:6836/1580408254] up:boot 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.16:6835/4269593822,v1:172.21.15.16:6837/4269593822] up:boot 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: mds.? [v2:172.21.15.44:6838/3310744000,v1:172.21.15.44:6839/3310744000] up:boot 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: fsmap 12 up:standby 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T08:58:52.550 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:58:52.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T08:58:52.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T08:58:52.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:58:52.551 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:52 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1391779173' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:58:52.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: pgmap v853: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 6 op/s 2024-03-26T08:58:52.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T08:58:52.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T08:58:52.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: Cluster is now healthy 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.44:6835/3591607244,v1:172.21.15.44:6837/3591607244] up:boot 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.44:6834/4227990186,v1:172.21.15.44:6836/4227990186] up:boot 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.16:6838/2619839450,v1:172.21.15.16:6839/2619839450] up:boot 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.16:6834/1580408254,v1:172.21.15.16:6836/1580408254] up:boot 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.16:6835/4269593822,v1:172.21.15.16:6837/4269593822] up:boot 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: mds.? [v2:172.21.15.44:6838/3310744000,v1:172.21.15.44:6839/3310744000] up:boot 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: fsmap 12 up:standby 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T08:58:52.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T08:58:52.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:58:52.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:52 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1391779173' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:58:52.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: pgmap v853: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 6 op/s 2024-03-26T08:58:52.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T08:58:52.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T08:58:52.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: Cluster is now healthy 2024-03-26T08:58:52.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T08:58:52.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.44:6835/3591607244,v1:172.21.15.44:6837/3591607244] up:boot 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.44:6834/4227990186,v1:172.21.15.44:6836/4227990186] up:boot 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.16:6838/2619839450,v1:172.21.15.16:6839/2619839450] up:boot 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.16:6834/1580408254,v1:172.21.15.16:6836/1580408254] up:boot 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.16:6835/4269593822,v1:172.21.15.16:6837/4269593822] up:boot 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: mds.? [v2:172.21.15.44:6838/3310744000,v1:172.21.15.44:6839/3310744000] up:boot 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: fsmap 12 up:standby 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T08:58:52.824 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:52 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1391779173' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:58:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.978+0000 7f063f7fe640 1 -- 172.21.15.7:0/1391779173 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7f063009b430 con 0x7f0648104a50 2024-03-26T08:58:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-03-26T08:58:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06340a2800 msgr2=0x7f06340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06340a2800 0x7f06340a4cc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f063802c240 tx=0x7f063802df90 comp rx=0 tx=0).stop 2024-03-26T08:58:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648104a50 msgr2=0x7f0648197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648104a50 0x7f0648197f10 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f063000c970 tx=0x7f063000ce30 comp rx=0 tx=0).stop 2024-03-26T08:58:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 shutdown_connections 2024-03-26T08:58:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f06340a2800 0x7f06340a4cc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0648107380 0x7f064819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0648104a50 0x7f0648197f10 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 --2- 172.21.15.7:0/1391779173 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f0648102120 0x7f06481979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.982+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 >> 172.21.15.7:0/1391779173 conn(0x7f06480fbad0 msgr2=0x7f0648102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.983+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 shutdown_connections 2024-03-26T08:58:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:52.983+0000 7f064ce34640 1 -- 172.21.15.7:0/1391779173 wait complete. 2024-03-26T08:58:52.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-26T08:58:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.112+0000 7f1fbccb5640 1 -- 172.21.15.7:0/3657466501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb8102170 msgr2=0x7f1fb8108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.112+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/3657466501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb8102170 0x7f1fb8108a00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009ca0 tx=0x7f1fb002c0f0 comp rx=0 tx=0).stop 2024-03-26T08:58:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.113+0000 7f1fbccb5640 1 -- 172.21.15.7:0/3657466501 shutdown_connections 2024-03-26T08:58:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.113+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/3657466501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb8102170 0x7f1fb8108a00 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.113+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/3657466501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb81017b0 0x7f1fb8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.113+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/3657466501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb81005c0 0x7f1fb81009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.113+0000 7f1fbccb5640 1 -- 172.21.15.7:0/3657466501 >> 172.21.15.7:0/3657466501 conn(0x7f1fb80fbb50 msgr2=0x7f1fb80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.113+0000 7f1fbccb5640 1 -- 172.21.15.7:0/3657466501 shutdown_connections 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.113+0000 7f1fbccb5640 1 -- 172.21.15.7:0/3657466501 wait complete. 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.113+0000 7f1fbccb5640 1 Processor -- start 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.113+0000 7f1fbccb5640 1 -- start start 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fbccb5640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb81005c0 0x7f1fb81979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fbccb5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb81017b0 0x7f1fb8197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fbccb5640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb8102170 0x7f1fb819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fbccb5640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb810c2a0 con 0x7f1fb81005c0 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fbccb5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb810c120 con 0x7f1fb81017b0 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fbccb5640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb810c420 con 0x7f1fb8102170 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb81005c0 0x7f1fb81979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:53.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb659c640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb81005c0 0x7f1fb81979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35082/0 (socket says 172.21.15.7:35082) 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb659c640 1 -- 172.21.15.7:0/302896255 learned_addr learned my addr 172.21.15.7:0/302896255 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1faffff640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb81017b0 0x7f1fb8197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb6d9d640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb8102170 0x7f1fb819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb659c640 1 -- 172.21.15.7:0/302896255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb8102170 msgr2=0x7f1fb819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb659c640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb8102170 0x7f1fb819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb659c640 1 -- 172.21.15.7:0/302896255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb81017b0 msgr2=0x7f1fb8197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb659c640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb81017b0 0x7f1fb8197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb659c640 1 -- 172.21.15.7:0/302896255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb819f5f0 con 0x7f1fb81005c0 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fb659c640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb81005c0 0x7f1fb81979d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800c970 tx=0x7f1fa800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1faf7fe640 1 -- 172.21.15.7:0/302896255 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8004220 con 0x7f1fb81005c0 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1faf7fe640 1 -- 172.21.15.7:0/302896255 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fa80043c0 con 0x7f1fb81005c0 2024-03-26T08:58:53.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb819f8e0 con 0x7f1fb81005c0 2024-03-26T08:58:53.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1faf7fe640 1 -- 172.21.15.7:0/302896255 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800f450 con 0x7f1fb81005c0 2024-03-26T08:58:53.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.114+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb819fe80 con 0x7f1fb81005c0 2024-03-26T08:58:53.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.116+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb81a0110 con 0x7f1fb81005c0 2024-03-26T08:58:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.117+0000 7f1faf7fe640 1 -- 172.21.15.7:0/302896255 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 20) v1 ==== 137289+0+0 (secure 0 0 0) 0x7f1fa80079b0 con 0x7f1fb81005c0 2024-03-26T08:58:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.117+0000 7f1faf7fe640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1fa40a2880 0x7f1fa40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T08:58:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.117+0000 7f1faffff640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1fa40a2880 0x7f1fa40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T08:58:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.117+0000 7f1faf7fe640 1 -- 172.21.15.7:0/302896255 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8002+0+0 (secure 0 0 0) 0x7f1fa80ce610 con 0x7f1fb81005c0 2024-03-26T08:58:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.118+0000 7f1faffff640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1fa40a2880 0x7f1fa40a4d40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8198ee0 tx=0x7f1fa0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T08:58:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.122+0000 7f1faf7fe640 1 -- 172.21.15.7:0/302896255 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1fa8014030 con 0x7f1fb81005c0 2024-03-26T08:58:53.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.314+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1fb81a02e0 con 0x7f1fb81005c0 2024-03-26T08:58:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.979+0000 7f1faf7fe640 1 -- 172.21.15.7:0/302896255 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1fa809b7e0 con 0x7f1fb81005c0 2024-03-26T08:58:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-03-26T08:58:53.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.984+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1fa40a2880 msgr2=0x7f1fa40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:53.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.984+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1fa40a2880 0x7f1fa40a4d40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8198ee0 tx=0x7f1fa0009110 comp rx=0 tx=0).stop 2024-03-26T08:58:53.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.984+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb81005c0 msgr2=0x7f1fb81979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T08:58:53.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.984+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb81005c0 0x7f1fb81979d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800c970 tx=0x7f1fa800cd70 comp rx=0 tx=0).stop 2024-03-26T08:58:53.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.985+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 shutdown_connections 2024-03-26T08:58:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.985+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:6800/1233671832,v1:172.21.15.7:6801/1233671832] conn(0x7f1fa40a2880 0x7f1fa40a4d40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.985+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1fb8102170 0x7f1fb819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.985+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb81017b0 0x7f1fb8197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T08:58:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.985+0000 7f1fbccb5640 1 --2- 172.21.15.7:0/302896255 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb81005c0 0x7f1fb81979d0 secure :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800c970 tx=0x7f1fa800cd70 comp rx=0 tx=0).stop 2024-03-26T08:58:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.985+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 >> 172.21.15.7:0/302896255 conn(0x7f1fb80fbb50 msgr2=0x7f1fb80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T08:58:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.985+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 shutdown_connections 2024-03-26T08:58:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-03-26T08:58:53.985+0000 7f1fbccb5640 1 -- 172.21.15.7:0/302896255 wait complete. 2024-03-26T08:58:54.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:58:54.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:58:54.078 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T08:58:54.118 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-26T08:58:54.118 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T08:58:54.135 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T08:58:54.155 DEBUG:teuthology.orchestra.run.smithi044:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T08:58:54.175 INFO:tasks.cephadm:Stopping all daemons... 2024-03-26T08:58:54.175 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-26T08:58:54.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.a 2024-03-26T08:58:54.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:53 smithi016 ceph-mon[35851]: pgmap v855: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-03-26T08:58:54.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:53 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/1391779173' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:58:54.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:53 smithi016 ceph-mon[35851]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T08:58:54.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:53 smithi016 ceph-mon[35851]: from='client.? 172.21.15.7:0/302896255' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:58:54.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:53 smithi016 ceph-mon[35851]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:58:54.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:53 smithi007 ceph-mon[32462]: pgmap v855: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-03-26T08:58:54.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:53 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/1391779173' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:58:54.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:53 smithi007 ceph-mon[32462]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T08:58:54.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:53 smithi007 ceph-mon[32462]: from='client.? 172.21.15.7:0/302896255' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:58:54.299 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:53 smithi007 ceph-mon[32462]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:58:54.300 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:54 smithi007 systemd[1]: Stopping Ceph mon.a for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:58:54.323 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:53 smithi044 ceph-mon[36461]: pgmap v855: 97 pgs: 97 active+clean; 1.6 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2024-03-26T08:58:54.323 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:53 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/1391779173' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T08:58:54.323 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:53 smithi044 ceph-mon[36461]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T08:58:54.323 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:53 smithi044 ceph-mon[36461]: from='client.? 172.21.15.7:0/302896255' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T08:58:54.323 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:53 smithi044 ceph-mon[36461]: from='mgr.14152 172.21.15.7:0/562468441' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T08:58:55.049 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a[32458]: 2024-03-26T08:58:54.550+0000 7fc473d53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:58:55.049 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a[32458]: 2024-03-26T08:58:54.550+0000 7fc473d53700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-26T08:58:55.049 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:54 smithi007 podman[103561]: 2024-03-26 08:58:54.785346952 +0000 UTC m=+0.364619721 container died e370da5b95266058f482c9dd0fe004a73af9eb054dd1273cd299c27201487c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T08:58:55.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:55 smithi007 podman[103561]: 2024-03-26 08:58:55.176540047 +0000 UTC m=+0.755812812 container cleanup e370da5b95266058f482c9dd0fe004a73af9eb054dd1273cd299c27201487c52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T08:58:55.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Mar 26 08:58:55 smithi007 bash[103561]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-a 2024-03-26T08:58:55.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.a.service' 2024-03-26T08:58:55.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:58:55.610 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-26T08:58:55.610 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-26T08:58:55.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.b 2024-03-26T08:58:55.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:55 smithi016 systemd[1]: Stopping Ceph mon.b for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:58:56.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:55 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-b[35847]: 2024-03-26T08:58:55.945+0000 7f7b4b3e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:58:56.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:55 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-b[35847]: 2024-03-26T08:58:55.945+0000 7f7b4b3e3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-26T08:58:56.250 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:55 smithi016 podman[71702]: 2024-03-26 08:58:55.99831033 +0000 UTC m=+0.182909488 container died d98f5c5ac880a7b44e6be687beee07e02f3926286ac7bc8aa1977bf0d1a1734b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-26T08:58:56.688 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:56 smithi016 podman[71702]: 2024-03-26 08:58:56.355572351 +0000 UTC m=+0.540171500 container cleanup d98f5c5ac880a7b44e6be687beee07e02f3926286ac7bc8aa1977bf0d1a1734b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-26T08:58:56.689 INFO:journalctl@ceph.mon.b.smithi016.stdout:Mar 26 08:58:56 smithi016 bash[71702]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-b 2024-03-26T08:58:56.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.b.service' 2024-03-26T08:58:56.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:58:56.868 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-26T08:58:56.868 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-26T08:58:56.869 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.c 2024-03-26T08:58:57.210 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:56 smithi044 systemd[1]: Stopping Ceph mon.c for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:58:57.527 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-c[36457]: 2024-03-26T08:58:57.210+0000 7f292b148700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:58:57.527 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-c[36457]: 2024-03-26T08:58:57.210+0000 7f292b148700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-26T08:58:57.527 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:57 smithi044 podman[72644]: 2024-03-26 08:58:57.262417418 +0000 UTC m=+0.178216112 container died d20b903c79fca76a44e1453ae9b2ef93404a52ba3ed1dd066fffd913eab4260e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-c, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:58:57.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:57 smithi044 podman[72644]: 2024-03-26 08:58:57.620094955 +0000 UTC m=+0.535893639 container cleanup d20b903c79fca76a44e1453ae9b2ef93404a52ba3ed1dd066fffd913eab4260e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:58:57.823 INFO:journalctl@ceph.mon.c.smithi044.stdout:Mar 26 08:58:57 smithi044 bash[72644]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mon-c 2024-03-26T08:58:58.303 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mon.c.service' 2024-03-26T08:58:58.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:58:58.356 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-26T08:58:58.356 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-26T08:58:58.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.x 2024-03-26T08:58:58.680 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:58:58 smithi007 systemd[1]: Stopping Ceph mgr.x for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:58:59.049 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:58:58 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x[32694]: 2024-03-26T08:58:58.681+0000 7f0ee7b0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:58:59.049 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:58:58 smithi007 podman[103677]: 2024-03-26 08:58:58.712465517 +0000 UTC m=+0.153311755 container died 4a674aaa04826cb2caec6df38e7c034e1a41bb91c58ea08ced4f20ca89d7aad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-26T08:58:59.549 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:58:59 smithi007 podman[103677]: 2024-03-26 08:58:59.391548261 +0000 UTC m=+0.832394499 container cleanup 4a674aaa04826cb2caec6df38e7c034e1a41bb91c58ea08ced4f20ca89d7aad6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T08:58:59.549 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Mar 26 08:58:59 smithi007 bash[103677]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-x 2024-03-26T08:58:59.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-26T08:58:59.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:58:59.877 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-26T08:58:59.877 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-26T08:58:59.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.y 2024-03-26T08:59:00.214 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:58:59 smithi016 systemd[1]: Stopping Ceph mgr.y for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:00.500 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:59:00 smithi016 podman[71823]: 2024-03-26 08:59:00.215724562 +0000 UTC m=+0.133207572 container died f11437db8ff5066d2885e21201f6a36105250885625b01b918fefbe652116d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-26T08:59:00.861 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:59:00 smithi016 podman[71823]: 2024-03-26 08:59:00.611970756 +0000 UTC m=+0.529453773 container cleanup f11437db8ff5066d2885e21201f6a36105250885625b01b918fefbe652116d06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-26T08:59:00.861 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Mar 26 08:59:00 smithi016 bash[71823]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-y 2024-03-26T08:59:01.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-26T08:59:01.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:59:01.071 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-26T08:59:01.072 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-26T08:59:01.072 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.z 2024-03-26T08:59:01.412 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:59:01 smithi044 systemd[1]: Stopping Ceph mgr.z for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:01.701 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:59:01 smithi044 podman[72770]: 2024-03-26 08:59:01.413686469 +0000 UTC m=+0.137339635 container died 9426886c12da665b8d7a99811290ca28b6da4163220d18a88f654a1e0f449f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T08:59:02.001 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:59:01 smithi044 podman[72770]: 2024-03-26 08:59:01.801502569 +0000 UTC m=+0.525155709 container cleanup 9426886c12da665b8d7a99811290ca28b6da4163220d18a88f654a1e0f449f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-26T08:59:02.001 INFO:journalctl@ceph.mgr.z.smithi044.stdout:Mar 26 08:59:01 smithi044 bash[72770]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mgr-z 2024-03-26T08:59:02.219 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-26T08:59:02.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:59:02.261 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-26T08:59:02.261 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-26T08:59:02.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.0 2024-03-26T08:59:02.549 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:02 smithi007 systemd[1]: Stopping Ceph osd.0 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:03.049 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:02 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0[44631]: 2024-03-26T08:59:02.562+0000 7f87a933e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:59:03.049 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:02 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0[44631]: 2024-03-26T08:59:02.562+0000 7f87a933e700 -1 osd.0 90 *** Got signal Terminated *** 2024-03-26T08:59:03.050 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:02 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0[44631]: 2024-03-26T08:59:02.562+0000 7f87a933e700 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:59:08.437 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:08 smithi007 podman[103797]: 2024-03-26 08:59:08.135937617 +0000 UTC m=+5.679235716 container died 079acd1a3dd02f6540217288ce0193f4466debc6563ce6ec9323714e8b3905e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:59:08.747 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:08 smithi007 podman[103797]: 2024-03-26 08:59:08.538386621 +0000 UTC m=+6.081684705 container cleanup 079acd1a3dd02f6540217288ce0193f4466debc6563ce6ec9323714e8b3905e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-03-26T08:59:08.747 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:08 smithi007 bash[103797]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0 2024-03-26T08:59:09.049 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:08 smithi007 podman[103814]: 2024-03-26 08:59:08.932068416 +0000 UTC m=+0.798208791 container remove 079acd1a3dd02f6540217288ce0193f4466debc6563ce6ec9323714e8b3905e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-26T08:59:09.549 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:09 smithi007 podman[103911]: 2024-03-26 08:59:09.202339998 +0000 UTC m=+0.017701762 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:59:09.550 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:09 smithi007 podman[103911]: 2024-03-26 08:59:09.316672633 +0000 UTC m=+0.132034413 container create 6540165fc9af02547e917bd1f905f0dc3e42577bcff3993faa64b32048d9d7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:59:09.872 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:09 smithi007 podman[103911]: 2024-03-26 08:59:09.569330316 +0000 UTC m=+0.384692064 container init 6540165fc9af02547e917bd1f905f0dc3e42577bcff3993faa64b32048d9d7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-26T08:59:09.872 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:09 smithi007 podman[103911]: 2024-03-26 08:59:09.572531284 +0000 UTC m=+0.387893028 container start 6540165fc9af02547e917bd1f905f0dc3e42577bcff3993faa64b32048d9d7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:59:09.872 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:09 smithi007 podman[103911]: 2024-03-26 08:59:09.66113337 +0000 UTC m=+0.476495140 container attach 6540165fc9af02547e917bd1f905f0dc3e42577bcff3993faa64b32048d9d7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T08:59:10.230 INFO:journalctl@ceph.osd.0.smithi007.stdout:Mar 26 08:59:09 smithi007 podman[103911]: 2024-03-26 08:59:09.874425896 +0000 UTC m=+0.689787652 container died 6540165fc9af02547e917bd1f905f0dc3e42577bcff3993faa64b32048d9d7f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:59:10.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.0.service' 2024-03-26T08:59:10.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:59:10.567 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-26T08:59:10.567 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-26T08:59:10.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.1 2024-03-26T08:59:10.882 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:10 smithi016 systemd[1]: Stopping Ceph osd.1 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:11.250 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1[40475]: 2024-03-26T08:59:10.882+0000 7f2c8adf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:59:11.250 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1[40475]: 2024-03-26T08:59:10.882+0000 7f2c8adf4700 -1 osd.1 90 *** Got signal Terminated *** 2024-03-26T08:59:11.250 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1[40475]: 2024-03-26T08:59:10.882+0000 7f2c8adf4700 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:59:16.598 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:16 smithi016 podman[71947]: 2024-03-26 08:59:16.296670161 +0000 UTC m=+5.528988669 container died d4884ea2e9238798066b410ee825a903627d13e8c949e4d594e7ed45bf36bfa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-26T08:59:16.967 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:16 smithi016 podman[71947]: 2024-03-26 08:59:16.708439901 +0000 UTC m=+5.940758394 container cleanup d4884ea2e9238798066b410ee825a903627d13e8c949e4d594e7ed45bf36bfa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-26T08:59:16.967 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:16 smithi016 bash[71947]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1 2024-03-26T08:59:17.250 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:16 smithi016 podman[71965]: 2024-03-26 08:59:16.968618536 +0000 UTC m=+0.670109824 container remove d4884ea2e9238798066b410ee825a903627d13e8c949e4d594e7ed45bf36bfa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:59:17.695 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:17 smithi016 podman[72055]: 2024-03-26 08:59:17.230057258 +0000 UTC m=+0.017838383 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:59:17.696 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:17 smithi016 podman[72055]: 2024-03-26 08:59:17.352775207 +0000 UTC m=+0.140556304 container create 5dd86053aba6f141d238ee6f2bc722a356b04a1c8a160c0c5ac60920a9fd3a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:59:17.696 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:17 smithi016 podman[72055]: 2024-03-26 08:59:17.697251407 +0000 UTC m=+0.485032537 container init 5dd86053aba6f141d238ee6f2bc722a356b04a1c8a160c0c5ac60920a9fd3a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-26T08:59:17.999 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:17 smithi016 podman[72055]: 2024-03-26 08:59:17.700879543 +0000 UTC m=+0.488660648 container start 5dd86053aba6f141d238ee6f2bc722a356b04a1c8a160c0c5ac60920a9fd3a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T08:59:18.000 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:17 smithi016 podman[72055]: 2024-03-26 08:59:17.797294393 +0000 UTC m=+0.585075520 container attach 5dd86053aba6f141d238ee6f2bc722a356b04a1c8a160c0c5ac60920a9fd3a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-26T08:59:18.250 INFO:journalctl@ceph.osd.1.smithi016.stdout:Mar 26 08:59:18 smithi016 podman[72055]: 2024-03-26 08:59:18.002167029 +0000 UTC m=+0.789948160 container died 5dd86053aba6f141d238ee6f2bc722a356b04a1c8a160c0c5ac60920a9fd3a8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T08:59:18.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.1.service' 2024-03-26T08:59:18.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:59:18.693 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-26T08:59:18.693 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-26T08:59:18.694 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.2 2024-03-26T08:59:19.005 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:18 smithi044 systemd[1]: Stopping Ceph osd.2 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:19.282 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:19 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2[41668]: 2024-03-26T08:59:19.005+0000 7fdf703a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:59:19.282 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:19 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2[41668]: 2024-03-26T08:59:19.005+0000 7fdf703a3700 -1 osd.2 90 *** Got signal Terminated *** 2024-03-26T08:59:19.282 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:19 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2[41668]: 2024-03-26T08:59:19.005+0000 7fdf703a3700 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:59:24.706 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:24 smithi044 podman[72895]: 2024-03-26 08:59:24.41783195 +0000 UTC m=+5.518965220 container died 5d8f8d55638e6fec61970983c8877b303b90253593b89e074249bc6825dad56f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-26T08:59:25.007 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:24 smithi044 podman[72895]: 2024-03-26 08:59:24.806749465 +0000 UTC m=+5.907882725 container cleanup 5d8f8d55638e6fec61970983c8877b303b90253593b89e074249bc6825dad56f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-26T08:59:25.007 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:24 smithi044 bash[72895]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2 2024-03-26T08:59:25.323 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:25 smithi044 podman[72925]: 2024-03-26 08:59:25.008709613 +0000 UTC m=+0.587581158 container remove 5d8f8d55638e6fec61970983c8877b303b90253593b89e074249bc6825dad56f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-26T08:59:25.711 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:25 smithi044 podman[73018]: 2024-03-26 08:59:25.346204743 +0000 UTC m=+0.018092909 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:59:25.711 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:25 smithi044 podman[73018]: 2024-03-26 08:59:25.460146411 +0000 UTC m=+0.132034608 container create 64006fa5ad5b6276b31bfadce6b6439a0d53b57ab821deea432c6cdb10bfe44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:59:25.711 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:25 smithi044 podman[73018]: 2024-03-26 08:59:25.713197093 +0000 UTC m=+0.385085254 container init 64006fa5ad5b6276b31bfadce6b6439a0d53b57ab821deea432c6cdb10bfe44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-26T08:59:26.013 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:25 smithi044 podman[73018]: 2024-03-26 08:59:25.716770297 +0000 UTC m=+0.388658449 container start 64006fa5ad5b6276b31bfadce6b6439a0d53b57ab821deea432c6cdb10bfe44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T08:59:26.013 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:25 smithi044 podman[73018]: 2024-03-26 08:59:25.830043662 +0000 UTC m=+0.501931848 container attach 64006fa5ad5b6276b31bfadce6b6439a0d53b57ab821deea432c6cdb10bfe44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:59:26.323 INFO:journalctl@ceph.osd.2.smithi044.stdout:Mar 26 08:59:26 smithi044 podman[73018]: 2024-03-26 08:59:26.015111395 +0000 UTC m=+0.686999566 container died 64006fa5ad5b6276b31bfadce6b6439a0d53b57ab821deea432c6cdb10bfe44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:59:26.649 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.2.service' 2024-03-26T08:59:26.701 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:59:26.702 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-26T08:59:26.702 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-26T08:59:26.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.3 2024-03-26T08:59:26.991 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:26 smithi007 systemd[1]: Stopping Ceph osd.3 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:27.299 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:26 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3[51327]: 2024-03-26T08:59:26.992+0000 7ffb96078700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:59:27.299 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:26 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3[51327]: 2024-03-26T08:59:26.992+0000 7ffb96078700 -1 osd.3 90 *** Got signal Terminated *** 2024-03-26T08:59:27.299 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:26 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3[51327]: 2024-03-26T08:59:26.992+0000 7ffb96078700 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:59:29.826 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:29 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:29.530+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:30.549 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:30 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:30.263+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:31.073 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:30 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:30.578+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:31.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:30 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:30.724+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:31.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:31 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:31.206+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:31.549 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:31 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:31.291+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:31.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:31.260+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:32.073 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:31.590+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:32.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:31.724+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:32.268 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:31 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:31.962+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:32.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:32 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:32.209+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:32.549 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:32 smithi007 podman[104017]: 2024-03-26 08:59:32.494301851 +0000 UTC m=+5.598288221 container died 954a0ee870def665dbcecc635c9a32ff7c9d2bd9efcf92443c597eceaa9a3834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-26T08:59:32.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:32 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:32.269+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:32.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:32.235+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:33.049 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:32 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:32.921+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:33.050 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:32 smithi007 podman[104017]: 2024-03-26 08:59:32.885178102 +0000 UTC m=+5.989164471 container cleanup 954a0ee870def665dbcecc635c9a32ff7c9d2bd9efcf92443c597eceaa9a3834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2) 2024-03-26T08:59:33.050 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:32 smithi007 bash[104017]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3 2024-03-26T08:59:33.073 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:32.617+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:33.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:32.771+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:33.434 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:33 smithi007 podman[104044]: 2024-03-26 08:59:33.12007991 +0000 UTC m=+0.623428939 container remove 954a0ee870def665dbcecc635c9a32ff7c9d2bd9efcf92443c597eceaa9a3834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-26T08:59:33.434 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:33 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:33.289+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:33.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:33 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:33.237+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:33.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:33.228+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:33.797 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:33 smithi007 podman[104136]: 2024-03-26 08:59:33.338710756 +0000 UTC m=+0.018121082 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:59:33.798 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:33 smithi007 podman[104136]: 2024-03-26 08:59:33.462744598 +0000 UTC m=+0.142154962 container create aa0dd9b1865b873bb9f235559ae88be10e34318ef32ada935bdcbde3a7e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T08:59:33.798 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:33 smithi007 podman[104136]: 2024-03-26 08:59:33.673804947 +0000 UTC m=+0.353215273 container init aa0dd9b1865b873bb9f235559ae88be10e34318ef32ada935bdcbde3a7e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-26T08:59:33.798 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:33 smithi007 podman[104136]: 2024-03-26 08:59:33.676993686 +0000 UTC m=+0.356404012 container start aa0dd9b1865b873bb9f235559ae88be10e34318ef32ada935bdcbde3a7e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:59:33.798 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:33 smithi007 podman[104136]: 2024-03-26 08:59:33.798601616 +0000 UTC m=+0.478011973 container attach aa0dd9b1865b873bb9f235559ae88be10e34318ef32ada935bdcbde3a7e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-26T08:59:34.050 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:33 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:33.884+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:34.050 INFO:journalctl@ceph.osd.3.smithi007.stdout:Mar 26 08:59:33 smithi007 podman[104136]: 2024-03-26 08:59:33.982292963 +0000 UTC m=+0.661703288 container died aa0dd9b1865b873bb9f235559ae88be10e34318ef32ada935bdcbde3a7e7e692 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-03-26T08:59:34.073 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:33.664+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:34.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:33.813+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:34.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:34 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:34.191+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:34.549 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:34 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:34.272+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:34.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:34.240+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:34.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.3.service' 2024-03-26T08:59:34.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:59:34.668 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-26T08:59:34.668 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-26T08:59:34.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.4 2024-03-26T08:59:34.978 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:34 smithi016 systemd[1]: Stopping Ceph osd.4 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:35.073 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:34.622+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:35.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:34.792+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:35.239 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:34 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:34.896+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:35.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:35 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:35.166+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:35.250 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:34 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4[47315]: 2024-03-26T08:59:34.979+0000 7f0eaeee5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:59:35.250 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:34 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4[47315]: 2024-03-26T08:59:34.979+0000 7f0eaeee5700 -1 osd.4 90 *** Got signal Terminated *** 2024-03-26T08:59:35.250 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:34 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4[47315]: 2024-03-26T08:59:34.979+0000 7f0eaeee5700 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:59:35.549 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:35 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:35.239+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:35.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:35.205+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:36.073 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:35.625+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:36.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:35.783+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:36.287 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:35 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:35.877+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:36.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:36 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:36.161+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:36.549 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:36.289+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:36.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:36.214+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:37.073 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:36.652+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:37.074 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:36.830+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:37.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:36.909+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:37.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:36 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:36.909+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:37.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:37 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:37.116+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:37.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:37 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:37.244+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:37.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:37 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:37.332+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:38.073 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:37 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:37.651+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:38.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:37 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:37.835+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:38.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:37 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:37.928+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:38.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:37 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:37.928+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:38.500 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:38 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4[47315]: 2024-03-26T08:59:38.195+0000 7f0eab4d9700 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.597222+0000 front 2024-03-26T08:59:14.597251+0000 (oldest deadline 2024-03-26T08:59:38.097361+0000) 2024-03-26T08:59:38.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:38 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:38.127+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:38.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:38 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:38.230+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:38.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:38 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:38.377+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:38.882 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:38 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:38.621+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:39.260 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:38 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:38.884+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:39.260 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:38 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:38.884+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:39.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:38 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:38.902+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:39.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:38 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:38.902+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:39.500 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:39 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4[47315]: 2024-03-26T08:59:39.190+0000 7f0eab4d9700 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.597222+0000 front 2024-03-26T08:59:14.597251+0000 (oldest deadline 2024-03-26T08:59:38.097361+0000) 2024-03-26T08:59:39.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:39 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:39.126+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:39.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:39 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:39.261+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:39.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:39 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:39.366+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:39.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:39 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:39.366+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:39.865 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:39 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:39.595+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:39.865 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:39 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:39.595+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.234094+0000 front 2024-03-26T08:59:14.233991+0000 (oldest deadline 2024-03-26T08:59:39.533951+0000) 2024-03-26T08:59:40.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:39 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:39.608+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:40.263 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:39 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:39.866+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:40.263 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:39 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:39.866+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:40.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:39 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:39.918+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:40.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:39 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:39.918+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:40.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:40 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:40.079+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:40.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:40 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:40.264+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:40.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:40 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:40.347+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:40.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:40 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:40.347+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:40.854 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:40 smithi016 podman[72174]: 2024-03-26 08:59:40.566194368 +0000 UTC m=+5.692072829 container died 5f6657b803ac30577cd084417b35640e4dac44edb5fa653f5e41adfaf4f6413b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-26T08:59:40.854 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:40 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:40.584+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:40.908 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:40 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:40.616+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:40.908 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:40 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:40.616+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.234094+0000 front 2024-03-26T08:59:14.233991+0000 (oldest deadline 2024-03-26T08:59:39.533951+0000) 2024-03-26T08:59:41.204 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:40 smithi016 podman[72174]: 2024-03-26 08:59:40.955065664 +0000 UTC m=+6.080944133 container cleanup 5f6657b803ac30577cd084417b35640e4dac44edb5fa653f5e41adfaf4f6413b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:59:41.205 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:40 smithi016 bash[72174]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4 2024-03-26T08:59:41.205 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:41 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:41.046+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:41.205 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:41 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:41.046+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:41.245 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:40 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:40.910+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:41.246 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:40 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:40.910+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:41.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:40 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:40.966+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:41.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:40 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:40.966+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:41.500 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:41 smithi016 podman[72192]: 2024-03-26 08:59:41.206683692 +0000 UTC m=+0.639233215 container remove 5f6657b803ac30577cd084417b35640e4dac44edb5fa653f5e41adfaf4f6413b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-26T08:59:41.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:41 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:41.247+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:41.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:41 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:41.339+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:41.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:41 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:41.339+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:41.859 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:41 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:41.571+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:41.859 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:41 smithi016 podman[72287]: 2024-03-26 08:59:41.43911484 +0000 UTC m=+0.018333169 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:59:41.859 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:41 smithi016 podman[72287]: 2024-03-26 08:59:41.566207032 +0000 UTC m=+0.145425341 container create 2cc6e01a5846a1c84678b652ee510156bf2026d784004d2a9c62981ab622e7ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:59:41.859 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:41 smithi016 podman[72287]: 2024-03-26 08:59:41.861119325 +0000 UTC m=+0.440337658 container init 2cc6e01a5846a1c84678b652ee510156bf2026d784004d2a9c62981ab622e7ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:59:41.934 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:41 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:41.637+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:41.935 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:41 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:41.637+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.234094+0000 front 2024-03-26T08:59:14.233991+0000 (oldest deadline 2024-03-26T08:59:39.533951+0000) 2024-03-26T08:59:42.165 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:42 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:42.095+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:42.165 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:42 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:42.095+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:42.165 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:41 smithi016 podman[72287]: 2024-03-26 08:59:41.864727948 +0000 UTC m=+0.443946265 container start 2cc6e01a5846a1c84678b652ee510156bf2026d784004d2a9c62981ab622e7ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T08:59:42.165 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:41 smithi016 podman[72287]: 2024-03-26 08:59:41.968991644 +0000 UTC m=+0.548209976 container attach 2cc6e01a5846a1c84678b652ee510156bf2026d784004d2a9c62981ab622e7ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:59:42.198 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:41 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:41.936+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:42.198 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:41 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:41.936+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:42.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:42 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:42.003+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:42.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:42 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:42.003+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:42.500 INFO:journalctl@ceph.osd.4.smithi016.stdout:Mar 26 08:59:42 smithi016 podman[72287]: 2024-03-26 08:59:42.167665759 +0000 UTC m=+0.746884077 container died 2cc6e01a5846a1c84678b652ee510156bf2026d784004d2a9c62981ab622e7ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-26T08:59:42.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:42 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:42.199+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:42.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:42 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:42.300+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:42.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:42 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:42.300+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:42.807 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:42 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:42.605+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:42.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.4.service' 2024-03-26T08:59:42.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:59:42.881 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-26T08:59:42.881 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-26T08:59:42.881 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.5 2024-03-26T08:59:42.914 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:42 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:42.640+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:42.914 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:42 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:42.640+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.234094+0000 front 2024-03-26T08:59:14.233991+0000 (oldest deadline 2024-03-26T08:59:39.533951+0000) 2024-03-26T08:59:43.203 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:42 smithi044 systemd[1]: Stopping Ceph osd.5 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:43.204 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:43 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:43.158+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:43.204 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:42 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:42.952+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:43.204 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:42 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:42.952+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:43.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:42 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:42.959+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:43.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:42 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:42.959+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:43.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:43 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:43.102+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:43.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:43 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:43.102+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:43.573 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:43 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:43.204+0000 7faf5f054700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:59:43.573 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:43 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:43.204+0000 7faf5f054700 -1 osd.5 90 *** Got signal Terminated *** 2024-03-26T08:59:43.574 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:43 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:43.204+0000 7faf5f054700 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:59:43.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:43 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:43.332+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:43.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:43 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:43.332+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:43.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:43 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:43.616+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:43.972 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:43 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:43.616+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.234094+0000 front 2024-03-26T08:59:14.233991+0000 (oldest deadline 2024-03-26T08:59:39.533951+0000) 2024-03-26T08:59:44.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:43 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:43.633+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:44.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:43 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:43.960+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:44.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:43 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:43.960+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:44.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:44 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:44.127+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:44.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:43 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:43.973+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:44.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:43 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:43.973+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:44.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:44 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:44.055+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:44.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:44 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:44.055+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:44.680 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:44 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:44.354+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:44.681 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:44 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:44.354+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:44.935 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:44 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:44.594+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:44.936 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:44 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:44.594+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.234094+0000 front 2024-03-26T08:59:14.233991+0000 (oldest deadline 2024-03-26T08:59:39.533951+0000) 2024-03-26T08:59:45.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:44 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:44.641+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:45.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:45 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:45.010+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:45.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:45 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:45.010+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:45.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:45 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:45.096+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:45.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:45 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:45.096+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:45.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:44 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:44.936+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:45.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:44 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:44.936+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:45.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:44 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:44.936+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:45.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:45 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:45.010+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:45.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:45 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:45.010+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:45.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:45 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:45.320+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:45.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:45 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:45.320+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:45.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:45 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:45.320+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:45.823 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:45 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:45.566+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:45.823 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:45 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:45.566+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.234094+0000 front 2024-03-26T08:59:14.233991+0000 (oldest deadline 2024-03-26T08:59:39.533951+0000) 2024-03-26T08:59:45.823 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:45 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:45.566+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.635000+0000 front 2024-03-26T08:59:23.635033+0000 (oldest deadline 2024-03-26T08:59:44.735211+0000) 2024-03-26T08:59:45.987 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:45 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:45.682+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:46.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:45 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:45.988+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:46.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:45 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:45.988+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:46.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:46 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:46.011+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:46.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:46 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:46.011+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:46.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:46 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:46.049+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:46.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:46 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:46.049+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:46.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:45 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:45.964+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:46.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:45 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:45.964+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:46.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:45 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:45.964+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:46.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:46 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:46.365+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:46.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:46 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:46.365+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:46.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:46 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:46.365+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:46.954 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:46 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:46.586+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:46.955 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:46 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:46.586+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.234094+0000 front 2024-03-26T08:59:14.233991+0000 (oldest deadline 2024-03-26T08:59:39.533951+0000) 2024-03-26T08:59:46.955 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:46 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:46.586+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.635000+0000 front 2024-03-26T08:59:23.635033+0000 (oldest deadline 2024-03-26T08:59:44.735211+0000) 2024-03-26T08:59:46.998 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:46 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:46.639+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:47.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:47 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:47.000+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:47.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:47 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:47.000+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:47.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:47 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:47.000+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:47.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:47.032+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:47.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:47.032+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:47.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:47 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:47.062+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:47.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:47 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:47.062+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:47.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:46 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:46.956+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:47.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:46 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:46.956+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:47.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:46 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:46.956+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:47.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:47.356+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:47.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:47.356+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:47.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:47.356+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:47.823 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:47 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:47.539+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.432173+0000 front 2024-03-26T08:59:05.432198+0000 (oldest deadline 2024-03-26T08:59:28.932462+0000) 2024-03-26T08:59:47.823 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:47 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:47.539+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.234094+0000 front 2024-03-26T08:59:14.233991+0000 (oldest deadline 2024-03-26T08:59:39.533951+0000) 2024-03-26T08:59:47.823 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:47 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5[48513]: 2024-03-26T08:59:47.539+0000 7faf5b648700 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.635000+0000 front 2024-03-26T08:59:23.635033+0000 (oldest deadline 2024-03-26T08:59:44.735211+0000) 2024-03-26T08:59:47.962 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:47 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:47.621+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:48.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:47 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:47.964+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:48.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:47 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:47.964+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:48.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:47 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:47.964+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:48.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:47.983+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:48.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:47.983+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:48.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:47.983+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:48.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:48.034+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:48.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:48.034+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:48.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:48.000+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:48.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:48.000+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:48.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:48.000+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:48.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:48 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:48.404+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:48.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:48 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:48.404+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:48.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:48 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:48.404+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:48.965 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:48 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:48.599+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:48.965 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:48 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:48.599+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:49.073 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:48 smithi044 podman[73122]: 2024-03-26 08:59:48.822578454 +0000 UTC m=+5.738976551 container died cc38895a1bbe250b296a7b27ec0c065271f6c78d30d87e6b26ce5455908bbcfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:59:49.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:48.985+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:49.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:48.985+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:49.074 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:48.969+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:49.074 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:48.969+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:49.074 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:48 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:48.969+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:49.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:48 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:48.966+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:49.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:48 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:48.966+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:49.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:48 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:48.966+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:49.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:49 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:49.006+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:49.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:49 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:49.006+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:49.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:49 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:49.006+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:49.455 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:49 smithi044 podman[73122]: 2024-03-26 08:59:49.230619691 +0000 UTC m=+6.147017795 container cleanup cc38895a1bbe250b296a7b27ec0c065271f6c78d30d87e6b26ce5455908bbcfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T08:59:49.455 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:49 smithi044 bash[73122]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5 2024-03-26T08:59:49.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:49 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:49.363+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:49.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:49 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:49.363+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:49.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:49 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:49.363+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:49.823 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:49 smithi044 podman[73138]: 2024-03-26 08:59:49.457162834 +0000 UTC m=+0.630905497 container remove cc38895a1bbe250b296a7b27ec0c065271f6c78d30d87e6b26ce5455908bbcfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-26T08:59:49.930 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:49 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:49.553+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:49.930 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:49 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:49.553+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:50.134 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:49 smithi044 podman[73227]: 2024-03-26 08:59:49.763865081 +0000 UTC m=+0.018054891 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:59:50.134 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:49 smithi044 podman[73227]: 2024-03-26 08:59:49.89170794 +0000 UTC m=+0.145897777 container create 3c2d33a42fb0491beee4fc53d14e8189ab3d92bb315cbaf014b22c603a49c2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T08:59:50.134 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:50 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:50.007+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:50.135 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:50 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:50.007+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:50.135 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:49 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:49.921+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:50.135 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:49 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:49.921+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:50.135 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:49 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:49.921+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:50.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:49 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:49.931+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:50.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:49 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:49.931+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:50.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:49 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:49.931+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:50.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:50 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:50.003+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:50.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:50 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:50.003+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:50.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:50 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:50.003+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:50.442 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:50 smithi044 podman[73227]: 2024-03-26 08:59:50.135788186 +0000 UTC m=+0.389977994 container init 3c2d33a42fb0491beee4fc53d14e8189ab3d92bb315cbaf014b22c603a49c2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:59:50.442 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:50 smithi044 podman[73227]: 2024-03-26 08:59:50.139850168 +0000 UTC m=+0.394039966 container start 3c2d33a42fb0491beee4fc53d14e8189ab3d92bb315cbaf014b22c603a49c2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-26T08:59:50.442 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:50 smithi044 podman[73227]: 2024-03-26 08:59:50.235843066 +0000 UTC m=+0.490032869 container attach 3c2d33a42fb0491beee4fc53d14e8189ab3d92bb315cbaf014b22c603a49c2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:59:50.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:50 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:50.338+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:50.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:50 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:50.338+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:50.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:50 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:50.338+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:50.803 INFO:journalctl@ceph.osd.5.smithi044.stdout:Mar 26 08:59:50 smithi044 podman[73227]: 2024-03-26 08:59:50.444155853 +0000 UTC m=+0.698345649 container died 3c2d33a42fb0491beee4fc53d14e8189ab3d92bb315cbaf014b22c603a49c2c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.vendor=CentOS) 2024-03-26T08:59:50.943 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:50 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:50.587+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:50.943 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:50 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:50.587+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:51.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:51.032+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:51.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:51.032+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:51.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:50 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:50.899+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:51.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:50 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:50.899+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:51.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:50 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:50.899+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:51.149 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.5.service' 2024-03-26T08:59:51.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:59:51.237 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-26T08:59:51.237 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-26T08:59:51.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.6 2024-03-26T08:59:51.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:50 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:50.944+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:51.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:50 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:50.944+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:51.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:50 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:50.944+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:51.299 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:51 smithi007 systemd[1]: Stopping Ceph osd.6 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:51.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:51.039+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:51.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:51.039+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:51.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:51.039+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:51.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:51.327+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:51.799 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:51.327+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:51.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:51.327+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:51.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:51.529+0000 7fc036b3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T08:59:51.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:51.529+0000 7fc036b3f700 -1 osd.6 90 *** Got signal Terminated *** 2024-03-26T08:59:51.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:51 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:51.529+0000 7fc036b3f700 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T08:59:51.945 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:51.635+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:51.945 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:51.635+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:52.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:51.946+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:52.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:51.946+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:52.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:51 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:51.946+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:52.299 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:52.040+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:52.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:52.040+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:52.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:52.040+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:52.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:52 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:52.001+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:52.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:52 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:52.001+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:52.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:51.917+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:52.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:51.917+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:52.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:51 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:51.917+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:52.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:52.318+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:52.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:52.318+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:52.800 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:52 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:52.318+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:52.896 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:52 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:52.598+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:52.896 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:52 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:52.598+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:53.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:52 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:52.896+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:53.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:52 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:52.896+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:53.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:52 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:52.896+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:53.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:53 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:53.013+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:53.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:53 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:53.013+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:53.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:52 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:52.913+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:53.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:52 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:52.913+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:53.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:52 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:52.913+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:53.549 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:53.288+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:53.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:53.288+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:53.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:53.288+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:53.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:53.072+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:53.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:53.072+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:53.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:53.072+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:53.906 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:53 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:53.590+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:53.906 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:53 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:53.590+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:54.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:53 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:53.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:54.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:53 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:53.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:54.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:53 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:53.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:54.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:53 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:53.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T08:59:54.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:53 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:53.986+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:54.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:53 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:53.986+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:54.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:53 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:53.960+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:54.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:53 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:53.960+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:54.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:53 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:53.960+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:54.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:53 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:53.960+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T08:59:54.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:54.113+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:54.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:54.113+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:54.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:54.113+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:54.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:54.248+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:54.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:54.248+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:54.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:54.248+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:54.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:54 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:54.248+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.555351+0000 front 2024-03-26T08:59:30.555391+0000 (oldest deadline 2024-03-26T08:59:53.955905+0000) 2024-03-26T08:59:54.927 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:54 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:54.635+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:54.927 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:54 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:54.635+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:55.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:54 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:54.928+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:55.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:54 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:54.928+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:55.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:54 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:54.928+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:55.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:54 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:54.928+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T08:59:55.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:54 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:54.938+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:55.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:54 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:54.938+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:55.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:54 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:54.951+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:55.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:54 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:54.951+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:55.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:54 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:54.951+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:55.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:54 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:54.951+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T08:59:55.549 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:55.238+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:55.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:55.238+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:55.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:55.238+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:55.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:55.238+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.555351+0000 front 2024-03-26T08:59:30.555391+0000 (oldest deadline 2024-03-26T08:59:53.955905+0000) 2024-03-26T08:59:55.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:55.140+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:55.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:55.141+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:55.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:55.141+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:55.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:55 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:55.141+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T08:59:55.897 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:55 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:55.645+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:55.897 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:55 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:55.645+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:56.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:55 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:55.898+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:56.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:55 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:55.898+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:56.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:55 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:55.898+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:56.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:55 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:55.898+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T08:59:56.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:55 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:55.974+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:56.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:55 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:55.974+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:56.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:55 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:55.918+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:56.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:55 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:55.918+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:56.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:55 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:55.918+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:56.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:55 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:55.918+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T08:59:56.549 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:56.217+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.552071+0000 front 2024-03-26T08:59:06.552104+0000 (oldest deadline 2024-03-26T08:59:30.052238+0000) 2024-03-26T08:59:56.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:56.217+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:15.853999+0000 front 2024-03-26T08:59:15.854161+0000 (oldest deadline 2024-03-26T08:59:39.353851+0000) 2024-03-26T08:59:56.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:56.217+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:19.354316+0000 front 2024-03-26T08:59:19.354236+0000 (oldest deadline 2024-03-26T08:59:44.654352+0000) 2024-03-26T08:59:56.550 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6[58292]: 2024-03-26T08:59:56.217+0000 7fc033133700 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.555351+0000 front 2024-03-26T08:59:30.555391+0000 (oldest deadline 2024-03-26T08:59:53.955905+0000) 2024-03-26T08:59:56.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:56.112+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:56.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:56.112+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:56.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:56.112+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:56.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:56 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:56.112+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T08:59:56.895 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:56 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:56.619+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:56.895 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:56 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:56.619+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:57.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:56 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:56.896+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:57.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:56 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:56.896+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:57.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:56 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:56.896+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:57.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:56 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:56.896+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T08:59:57.299 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:57 smithi007 podman[104240]: 2024-03-26 08:59:57.01188175 +0000 UTC m=+5.576215299 container died afb491690921b8c2b9d8dbe2823cfbdc4a34f23722cc24261f10d64834203569 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:59:57.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:57 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:57.064+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:57.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:57 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:57.064+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:57.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:57 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:57.064+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:57.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:57 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:57.064+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T08:59:57.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:56 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:56.937+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:57.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:56 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:56.937+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:57.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:56 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:56.937+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:57.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:56 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:56.937+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T08:59:57.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:56 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:56.928+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:57.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:56 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:56.928+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:57.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:56 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:56.928+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T08:59:57.647 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:57 smithi007 podman[104240]: 2024-03-26 08:59:57.421817544 +0000 UTC m=+5.986151082 container cleanup afb491690921b8c2b9d8dbe2823cfbdc4a34f23722cc24261f10d64834203569 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T08:59:57.647 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:57 smithi007 bash[104240]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6 2024-03-26T08:59:57.938 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:57 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:57.644+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:57.938 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:57 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:57.644+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:58.049 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:57 smithi007 podman[104257]: 2024-03-26 08:59:57.648158569 +0000 UTC m=+0.640601909 container remove afb491690921b8c2b9d8dbe2823cfbdc4a34f23722cc24261f10d64834203569 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-26T08:59:58.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:57 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:57.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:58.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:57 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:57.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:58.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:57 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:57.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:58.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:57 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:57.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T08:59:58.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:57.960+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:58.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:57.960+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:58.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:57.960+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T08:59:58.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:57.985+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:58.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:57.985+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:58.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:57.985+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:58.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:57 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:57.985+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T08:59:58.349 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:58 smithi007 podman[104348]: 2024-03-26 08:59:57.992698046 +0000 UTC m=+0.017610336 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T08:59:58.349 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:58 smithi007 podman[104348]: 2024-03-26 08:59:58.107437909 +0000 UTC m=+0.132350172 container create 420c62d32576ee9d7ea06b855c92bd5788549b6f16dc28ee9ff3311c9da32f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-03-26T08:59:58.349 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:58 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:58.055+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:58.350 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:58 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:58.055+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:58.350 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:58 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:58.055+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:58.350 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:58 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:58.055+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T08:59:58.750 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:58 smithi007 podman[104348]: 2024-03-26 08:59:58.451100723 +0000 UTC m=+0.476012989 container init 420c62d32576ee9d7ea06b855c92bd5788549b6f16dc28ee9ff3311c9da32f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-26T08:59:58.751 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:58 smithi007 podman[104348]: 2024-03-26 08:59:58.454403904 +0000 UTC m=+0.479316168 container start 420c62d32576ee9d7ea06b855c92bd5788549b6f16dc28ee9ff3311c9da32f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-26T08:59:58.751 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:58 smithi007 podman[104348]: 2024-03-26 08:59:58.549972363 +0000 UTC m=+0.574884627 container attach 420c62d32576ee9d7ea06b855c92bd5788549b6f16dc28ee9ff3311c9da32f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T08:59:58.947 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:58 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:58.618+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:58.947 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:58 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:58.618+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:58.947 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:58 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:58.618+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:28.270124+0000 front 2024-03-26T08:59:28.270013+0000 (oldest deadline 2024-03-26T08:59:57.670563+0000) 2024-03-26T08:59:59.049 INFO:journalctl@ceph.osd.6.smithi007.stdout:Mar 26 08:59:58 smithi007 podman[104348]: 2024-03-26 08:59:58.752231098 +0000 UTC m=+0.777143374 container died 420c62d32576ee9d7ea06b855c92bd5788549b6f16dc28ee9ff3311c9da32f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-26T08:59:59.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:58 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:58.948+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T08:59:59.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:58 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:58.948+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T08:59:59.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:58 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:58.948+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T08:59:59.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:58 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:58.948+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T08:59:59.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:58 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:58.918+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T08:59:59.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:58 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:58.918+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T08:59:59.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:58 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:58.918+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T08:59:59.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:58 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:58.946+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T08:59:59.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:58 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:58.946+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T08:59:59.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:58 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:58.946+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T08:59:59.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:58 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:58.946+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T08:59:59.385 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:59 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:59.092+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T08:59:59.385 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:59 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:59.092+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T08:59:59.385 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:59 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:59.092+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T08:59:59.385 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 08:59:59 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T08:59:59.092+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T08:59:59.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.6.service' 2024-03-26T08:59:59.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T08:59:59.454 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-26T08:59:59.454 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-26T08:59:59.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.7 2024-03-26T08:59:59.750 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:59 smithi016 systemd[1]: Stopping Ceph osd.7 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T08:59:59.750 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:59.665+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T08:59:59.750 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:59.665+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T08:59:59.751 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:59.665+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:28.270124+0000 front 2024-03-26T08:59:28.270013+0000 (oldest deadline 2024-03-26T08:59:57.670563+0000) 2024-03-26T09:00:00.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:59.966+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:00.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:59.966+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:00.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:59.966+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:00.253 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T08:59:59.966+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:00.253 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:59.790+0000 7f24bd05a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:00.254 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:59.790+0000 7f24bd05a700 -1 osd.7 90 *** Got signal Terminated *** 2024-03-26T09:00:00.254 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 08:59:59 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T08:59:59.790+0000 7f24bd05a700 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T09:00:00.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:59 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:59.910+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:00.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:59 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:59.910+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:00.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 08:59:59 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T08:59:59.910+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:00.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:59 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:59.978+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:00.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:59 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:59.978+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:00.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:59 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:59.978+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:00.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 08:59:59 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T08:59:59.978+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:00.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:00 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:00.131+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:00.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:00 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:00.131+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:00.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:00 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:00.131+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:00.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:00 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:00.131+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:00.987 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:00.705+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T09:00:00.987 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:00.705+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T09:00:00.987 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:00.705+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:28.270124+0000 front 2024-03-26T08:59:28.270013+0000 (oldest deadline 2024-03-26T08:59:57.670563+0000) 2024-03-26T09:00:01.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:00.988+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:01.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:00.988+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:01.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:00.988+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:01.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:00.988+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:01.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:00.988+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:01.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:00 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:00.875+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:01.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:00 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:00.875+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:01.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:00 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:00.875+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:01.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:00 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:00.958+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:01.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:00 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:00.958+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:01.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:00 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:00.958+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:01.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:00 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:00.958+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:01.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:01 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:01.115+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:01.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:01 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:01.115+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:01.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:01 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:01.115+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:01.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:01 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:01.115+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:02.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:01 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:01.701+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T09:00:02.027 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:01 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:01.701+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T09:00:02.028 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:01 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:01.702+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:28.270124+0000 front 2024-03-26T08:59:28.270013+0000 (oldest deadline 2024-03-26T08:59:57.670563+0000) 2024-03-26T09:00:02.028 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:01 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:01.702+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:39.371572+0000 front 2024-03-26T08:59:39.371417+0000 (oldest deadline 2024-03-26T09:00:01.571821+0000) 2024-03-26T09:00:02.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:02 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:02.002+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:02.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:02 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:02.002+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:02.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:02 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:02.002+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:02.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:02 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:02.002+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:02.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:01 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:01.898+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:02.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:01 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:01.898+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:02.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:01 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:01.898+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:02.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:01 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:01.898+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:02.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:02.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:02.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:02.501 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:02.501 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:02.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:02 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:02.132+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:02.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:02 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:02.132+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:02.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:02 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:02.132+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:02.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:02 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:02.132+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:02.988 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:02.656+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T09:00:02.988 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:02.656+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T09:00:02.988 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:02.656+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:28.270124+0000 front 2024-03-26T08:59:28.270013+0000 (oldest deadline 2024-03-26T08:59:57.670563+0000) 2024-03-26T09:00:02.988 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:02.656+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:39.371572+0000 front 2024-03-26T08:59:39.371417+0000 (oldest deadline 2024-03-26T09:00:01.571821+0000) 2024-03-26T09:00:03.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.989+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:03.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.989+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:03.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.989+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:03.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.989+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:03.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:02 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:02.989+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:03.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:03 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:03.050+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:03.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:03 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:03.050+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:03.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:03 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:03.050+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:03.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:03 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:03.050+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:03.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:02 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:02.920+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:03.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:02 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:02.920+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:03.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:02 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:02.920+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:03.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:02 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:02.920+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:03.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:03 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:03.149+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:03.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:03 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:03.149+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:03.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:03 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:03.149+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:03.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:03 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:03.149+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:04.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:03 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:03.614+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T09:00:04.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:03 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:03.614+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T09:00:04.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:03 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:03.614+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:28.270124+0000 front 2024-03-26T08:59:28.270013+0000 (oldest deadline 2024-03-26T08:59:57.670563+0000) 2024-03-26T09:00:04.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:03 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:03.614+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:39.371572+0000 front 2024-03-26T08:59:39.371417+0000 (oldest deadline 2024-03-26T09:00:01.571821+0000) 2024-03-26T09:00:04.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:04 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:04.040+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:04.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:04 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:04.040+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:04.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:04 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:04.040+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:04.324 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:04 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:04.040+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:04.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:03 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:03.908+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:04.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:03 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:03.908+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:04.325 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:03 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:03.908+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:04.325 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:03 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:03.908+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:04.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:04.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:04.501 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:04.501 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:04.501 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:04.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:04.192+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:04.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:04.192+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:04.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:04.192+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:04.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:04 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:04.192+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:04.979 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:04.658+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:13.166872+0000 front 2024-03-26T08:59:13.167172+0000 (oldest deadline 2024-03-26T08:59:39.066988+0000) 2024-03-26T09:00:04.979 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:04.658+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.668919+0000 front 2024-03-26T08:59:23.668838+0000 (oldest deadline 2024-03-26T08:59:47.768953+0000) 2024-03-26T09:00:04.979 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:04.658+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:28.270124+0000 front 2024-03-26T08:59:28.270013+0000 (oldest deadline 2024-03-26T08:59:57.670563+0000) 2024-03-26T09:00:04.979 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7[53896]: 2024-03-26T09:00:04.658+0000 7f24b5646700 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:39.371572+0000 front 2024-03-26T08:59:39.371417+0000 (oldest deadline 2024-03-26T09:00:01.571821+0000) 2024-03-26T09:00:05.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:05.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:05.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:05.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:05.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:04.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:05.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:05 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:05.023+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:05.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:05 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:05.023+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:05.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:05 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:05.023+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:05.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:05 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:05.023+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:05.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:04 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:04.913+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:05.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:04 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:04.913+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:05.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:04 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:04.913+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:05.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:04 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:04.913+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:05.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:05 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:05.145+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:05.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:05 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:05.145+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:05.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:05 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:05.145+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:05.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:05 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:05.145+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:05.677 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:05 smithi016 podman[72404]: 2024-03-26 09:00:05.410887172 +0000 UTC m=+5.741101123 container died 5b29a5048f8772982a0e5bbc83d7a309d0b3c45ce2f7b7e6892dbd5776ed3c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T09:00:06.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:05 smithi016 podman[72404]: 2024-03-26 09:00:05.762103465 +0000 UTC m=+6.092317411 container cleanup 5b29a5048f8772982a0e5bbc83d7a309d0b3c45ce2f7b7e6892dbd5776ed3c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-26T09:00:06.000 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:05 smithi016 bash[72404]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7 2024-03-26T09:00:06.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:06 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:06.050+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:06.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:06 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:06.050+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:06.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:06 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:06.050+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:06.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:06 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:06.050+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:06.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:05 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:05.899+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:06.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:05 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:05.899+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:06.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:05 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:05.899+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:06.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:05 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:05.899+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:06.382 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:06 smithi016 podman[72422]: 2024-03-26 09:00:06.046882571 +0000 UTC m=+0.631042207 container remove 5b29a5048f8772982a0e5bbc83d7a309d0b3c45ce2f7b7e6892dbd5776ed3c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T09:00:06.382 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:06.382 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:06.382 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:06.382 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:06.383 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.003+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:06.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:06.141+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:06.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:06.141+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:06.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:06.141+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:06.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:06.141+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:06.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:06 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:06.141+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:06.681 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:06 smithi016 podman[72510]: 2024-03-26 09:00:06.286994528 +0000 UTC m=+0.018077287 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T09:00:06.682 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:06 smithi016 podman[72510]: 2024-03-26 09:00:06.423018014 +0000 UTC m=+0.154100791 container create 40ebd0967650f5e90b1430f99c483fd78cfbf04e28b38315266c6e159812ea00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T09:00:06.682 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:06 smithi016 podman[72510]: 2024-03-26 09:00:06.617125616 +0000 UTC m=+0.348208379 container init 40ebd0967650f5e90b1430f99c483fd78cfbf04e28b38315266c6e159812ea00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-26T09:00:06.682 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:06 smithi016 podman[72510]: 2024-03-26 09:00:06.620660834 +0000 UTC m=+0.351743586 container start 40ebd0967650f5e90b1430f99c483fd78cfbf04e28b38315266c6e159812ea00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-26T09:00:06.682 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:06 smithi016 podman[72510]: 2024-03-26 09:00:06.6834223 +0000 UTC m=+0.414505065 container attach 40ebd0967650f5e90b1430f99c483fd78cfbf04e28b38315266c6e159812ea00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319) 2024-03-26T09:00:06.980 INFO:journalctl@ceph.osd.7.smithi016.stdout:Mar 26 09:00:06 smithi016 podman[72510]: 2024-03-26 09:00:06.922190392 +0000 UTC m=+0.653273156 container died 40ebd0967650f5e90b1430f99c483fd78cfbf04e28b38315266c6e159812ea00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T09:00:06.980 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:06.980 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:06.980 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:06.980 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:06.980 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:06 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:06.980+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:07.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:07.083+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:07.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:07.083+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:07.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:07.083+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:07.323 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:07.083+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:07.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:06 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:06.879+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:07.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:06 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:06.879+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:07.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:06 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:06.879+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:07.324 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:06 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:06.879+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:07.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.7.service' 2024-03-26T09:00:07.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:07.517 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-26T09:00:07.517 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-26T09:00:07.517 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.8 2024-03-26T09:00:07.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:07 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:07.138+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:07.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:07 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:07.138+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:07.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:07 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:07.138+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:07.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:07 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:07.138+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:07.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:07 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:07.138+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:07.823 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:07 smithi044 systemd[1]: Stopping Ceph osd.8 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:00:08.000 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:07 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:07.931+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:08.000 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:07 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:07.931+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:08.000 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:07 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:07.931+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:08.000 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:07 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:07.931+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:08.000 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:07 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:07.931+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:08.124 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:07.851+0000 7f667c11a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:08.124 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:07.851+0000 7f667c11a700 -1 osd.8 90 *** Got signal Terminated *** 2024-03-26T09:00:08.124 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:07.851+0000 7f667c11a700 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T09:00:08.124 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:07.835+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:08.124 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:07.835+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:08.124 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:07.835+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:08.125 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:07 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:07.835+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:08.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:08 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:08.149+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:08.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:08 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:08.149+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:08.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:08 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:08.149+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:08.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:08 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:08.149+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:08.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:08 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:08.149+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:08.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:08 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:08.125+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:08.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:08 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:08.125+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:08.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:08 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:08.125+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:08.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:08 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:08.125+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:09.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:08 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:08.796+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:09.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:08 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:08.796+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:09.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:08 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:08.796+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:09.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:08 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:08.796+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:09.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:08 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:08.977+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:09.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:08 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:08.977+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:09.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:08 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:08.977+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:09.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:08 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:08.977+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:09.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:08 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:08.977+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:09.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:09.118+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:09.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:09.118+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:09.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:09.118+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:09.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:09.118+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:09.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:09 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:09.118+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:09.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:09.085+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:09.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:09.085+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:09.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:09.085+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:09.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:09.085+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:10.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:09.796+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:10.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:09.796+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:10.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:09.796+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:10.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:09.796+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:10.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:09 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:09.796+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:10.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:10.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:10.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:10.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:10.501 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.021+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:10.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:10 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:10.114+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:10.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:10 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:10.114+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:10.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:10 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:10.114+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:10.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:10 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:10.114+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:10.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:10 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:10.114+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:10.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:10.124+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:10.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:10.124+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:10.574 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:10.124+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:10.574 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:10.124+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:11.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:10.752+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:11.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:10.752+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:11.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:10.752+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:11.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:10.752+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:11.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:10.752+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:11.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.983+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:11.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.983+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:11.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.983+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:11.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.983+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:11.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:10 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:10.983+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:11.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:11 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:11.089+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:11.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:11 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:11.089+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:11.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:11 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:11.089+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:11.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:11 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:11.089+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:11.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:11 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:11.089+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:11.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:11 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:11.089+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:11.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:11.136+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:11.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:11.136+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:11.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:11.136+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:11.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:11.136+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:11.574 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:11.136+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.532428+0000 front 2024-03-26T08:59:47.532463+0000 (oldest deadline 2024-03-26T09:00:11.032527+0000) 2024-03-26T09:00:12.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:11.802+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:12.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:11.802+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:12.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:11.802+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:12.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:11.802+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:12.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:11 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:11.802+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:12.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:11 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:11.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:12.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:11 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:11.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:12.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:11 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:11.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:12.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:11 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:11.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:12.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:11 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:11.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:12.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:11 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:11.939+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:12.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:12 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:12.125+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:12.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:12 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:12.125+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:12.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:12 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:12.125+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:12.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:12 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:12.125+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:12.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:12 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:12.125+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:12.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:12 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:12.125+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:12.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:12.107+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:05.222939+0000 front 2024-03-26T08:59:05.222977+0000 (oldest deadline 2024-03-26T08:59:29.923130+0000) 2024-03-26T09:00:12.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:12.107+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:14.425727+0000 front 2024-03-26T08:59:14.425596+0000 (oldest deadline 2024-03-26T08:59:38.525631+0000) 2024-03-26T09:00:12.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:12.107+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.127501+0000 front 2024-03-26T08:59:23.127541+0000 (oldest deadline 2024-03-26T08:59:44.827658+0000) 2024-03-26T09:00:12.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:12.107+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.629470+0000 front 2024-03-26T08:59:30.629435+0000 (oldest deadline 2024-03-26T08:59:52.929507+0000) 2024-03-26T09:00:12.573 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8[55061]: 2024-03-26T09:00:12.107+0000 7f6674706700 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.532428+0000 front 2024-03-26T08:59:47.532463+0000 (oldest deadline 2024-03-26T09:00:11.032527+0000) 2024-03-26T09:00:13.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:12.824+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:13.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:12.824+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:13.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:12.824+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:13.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:12.824+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:13.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:12 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:12.824+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:13.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:12 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:12.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:13.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:12 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:12.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:13.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:12 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:12.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:13.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:12 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:12.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:13.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:12 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:12.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:13.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:12 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:12.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:13.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:13 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:13.158+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:13.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:13 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:13.158+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:13.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:13 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:13.158+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:13.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:13 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:13.158+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:13.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:13 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:13.158+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:13.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:13 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:13.158+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:13.734 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:13 smithi044 podman[73346]: 2024-03-26 09:00:13.464531577 +0000 UTC m=+5.742457821 container died d45e75f140b2b28981d7e666041f929b72c4e5cf85dc0cf936cbbd0a3f95ad57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T09:00:14.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:13 smithi044 podman[73346]: 2024-03-26 09:00:13.834674824 +0000 UTC m=+6.112601058 container cleanup d45e75f140b2b28981d7e666041f929b72c4e5cf85dc0cf936cbbd0a3f95ad57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-03-26T09:00:14.073 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:13 smithi044 bash[73346]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8 2024-03-26T09:00:14.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:13.823+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:14.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:13.823+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:14.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:13.823+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:14.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:13.823+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:14.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:13 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:13.823+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:14.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:13.918+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:14.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:13.918+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:14.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:13.918+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:14.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:13.918+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:14.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:13.918+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:14.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:13 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:13.918+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:14.457 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:14 smithi044 podman[73364]: 2024-03-26 09:00:14.136187442 +0000 UTC m=+0.670415188 container remove d45e75f140b2b28981d7e666041f929b72c4e5cf85dc0cf936cbbd0a3f95ad57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T09:00:14.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:14 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:14.188+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:14.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:14 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:14.188+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:14.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:14 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:14.188+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:14.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:14 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:14.188+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:14.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:14 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:14.188+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:14.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:14 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:14.188+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:14.713 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:14 smithi044 podman[73458]: 2024-03-26 09:00:14.361520046 +0000 UTC m=+0.018113074 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T09:00:14.714 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:14 smithi044 podman[73458]: 2024-03-26 09:00:14.470171042 +0000 UTC m=+0.126764076 container create 2e1436ff2bae90f52c0513a391882a57802e60cc3c9aec60b6abcece03ac7c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-03-26T09:00:14.714 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:14 smithi044 podman[73458]: 2024-03-26 09:00:14.715109182 +0000 UTC m=+0.371702206 container init 2e1436ff2bae90f52c0513a391882a57802e60cc3c9aec60b6abcece03ac7c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-26T09:00:15.016 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:14 smithi044 podman[73458]: 2024-03-26 09:00:14.718951682 +0000 UTC m=+0.375544697 container start 2e1436ff2bae90f52c0513a391882a57802e60cc3c9aec60b6abcece03ac7c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-26T09:00:15.016 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:14 smithi044 podman[73458]: 2024-03-26 09:00:14.781324248 +0000 UTC m=+0.437917293 container attach 2e1436ff2bae90f52c0513a391882a57802e60cc3c9aec60b6abcece03ac7c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-26T09:00:15.016 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:14 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:14.830+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:15.016 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:14 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:14.830+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:15.016 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:14 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:14.830+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:15.016 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:14 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:14.830+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:15.016 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:14 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:14.830+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:15.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:14.938+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:15.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:14.938+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:15.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:14.938+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:15.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:14.938+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:15.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:14.938+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:15.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:14 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:14.938+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:15.291 INFO:journalctl@ceph.osd.8.smithi044.stdout:Mar 26 09:00:15 smithi044 podman[73458]: 2024-03-26 09:00:15.017832357 +0000 UTC m=+0.674425391 container died 2e1436ff2bae90f52c0513a391882a57802e60cc3c9aec60b6abcece03ac7c8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-8-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-26T09:00:15.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:15.155+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:15.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:15.155+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:15.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:15.155+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:15.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:15.155+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:15.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:15.155+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:15.551 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:15 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:15.155+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:15.638 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.8.service' 2024-03-26T09:00:15.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:15.684 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-26T09:00:15.684 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-26T09:00:15.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.9 2024-03-26T09:00:16.013 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:15 smithi007 systemd[1]: Stopping Ceph osd.9 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:00:16.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:15 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:15.844+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:16.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:15 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:15.844+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:16.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:15 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:15.844+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:16.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:15 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:15.844+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:16.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:15 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:15.844+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:16.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:15 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:15.947+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:16.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:15 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:15.947+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:16.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:15 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:15.947+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:16.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:15 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:15.947+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:16.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:15 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:15.947+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:16.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:15 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:15.947+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:16.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:16 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:16.014+0000 7fc24bad1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:16.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:16 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:16.014+0000 7fc24bad1700 -1 osd.9 90 *** Got signal Terminated *** 2024-03-26T09:00:16.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:16 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:16.014+0000 7fc24bad1700 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T09:00:16.300 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:16 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:16.111+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:16.301 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:16 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:16.111+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:16.301 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:16 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:16.111+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:16.301 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:16 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:16.111+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:16.301 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:16 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:16.111+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:16.301 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:16 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:16.111+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:17.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:16 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:16.822+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:17.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:16 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:16.822+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:17.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:16 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:16.822+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:17.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:16 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:16.822+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:17.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:16 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:16.822+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:17.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:16.929+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:17.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:16.929+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:17.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:16.929+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:17.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:16.929+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:17.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:16.929+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:17.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:16 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:16.929+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:17.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:17 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:17.077+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:17.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:17 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:17.077+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:17.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:17 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:17.077+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:17.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:17 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:17.077+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:17.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:17 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:17.077+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:17.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:17 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:17.077+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:18.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:17 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:17.777+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:18.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:17 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:17.777+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:18.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:17 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:17.777+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:18.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:17 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:17.777+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:18.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:17 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:17.777+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:18.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:17 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:17.777+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:18.191 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:17 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:17.933+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:18.191 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:17 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:17.933+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:18.191 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:17 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:17.933+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:18.191 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:17 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:17.934+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:18.191 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:17 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:17.934+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:18.191 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:17 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:17.934+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:18.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:18 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:18.107+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:18.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:18 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:18.107+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:18.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:18 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:18.107+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:18.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:18 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:18.107+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:18.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:18 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:18.107+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:18.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:18 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:18.107+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:19.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:18 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:18.729+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:19.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:18 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:18.729+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:19.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:18 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:18.729+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:19.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:18 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:18.729+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:19.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:18 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:18.729+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:19.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:18 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:18.729+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:19.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:18.956+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:19.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:18.956+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:19.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:18.956+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:19.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:18.956+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:19.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:18.956+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:19.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:18 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:18.956+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:19.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:19 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:19.087+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:19.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:19 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:19.087+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:19.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:19 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:19.087+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:19.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:19 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:19.087+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:19.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:19 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:19.087+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:19.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:19 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:19.087+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:20.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:19 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:19.749+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:20.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:19 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:19.749+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:20.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:19 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:19.749+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:20.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:19 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:19.749+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:20.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:19 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:19.749+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:20.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:19 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:19.749+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:20.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:19 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:19.915+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:20.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:19 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:19.915+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:20.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:19 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:19.915+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:20.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:19 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:19.915+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:20.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:19 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:19.915+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:20.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:19 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:19.915+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:20.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:20 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:20.090+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.135677+0000 front 2024-03-26T08:59:07.135635+0000 (oldest deadline 2024-03-26T08:59:31.235752+0000) 2024-03-26T09:00:20.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:20 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:20.090+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:11.737047+0000 front 2024-03-26T08:59:11.737196+0000 (oldest deadline 2024-03-26T08:59:36.437011+0000) 2024-03-26T09:00:20.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:20 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:20.090+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.939244+0000 front 2024-03-26T08:59:23.939135+0000 (oldest deadline 2024-03-26T08:59:47.439234+0000) 2024-03-26T09:00:20.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:20 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:20.090+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:29.140207+0000 front 2024-03-26T08:59:29.140375+0000 (oldest deadline 2024-03-26T08:59:54.340774+0000) 2024-03-26T09:00:20.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:20 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:20.090+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:37.842113+0000 front 2024-03-26T08:59:37.842214+0000 (oldest deadline 2024-03-26T09:00:05.442447+0000) 2024-03-26T09:00:20.550 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:20 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9[65154]: 2024-03-26T09:00:20.090+0000 7fc2440bd700 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:45.943606+0000 front 2024-03-26T08:59:45.943653+0000 (oldest deadline 2024-03-26T09:00:10.643568+0000) 2024-03-26T09:00:21.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:20 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:20.701+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:21.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:20 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:20.701+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:21.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:20 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:20.701+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:21.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:20 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:20.701+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:21.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:20 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:20.701+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:21.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:20 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:20.701+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:21.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:20 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:20.885+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:21.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:20 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:20.885+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:21.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:20 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:20.885+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:21.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:20 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:20.885+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:21.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:20 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:20.885+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:21.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:20 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:20.885+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:21.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:20 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:20.885+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:54.658309+0000 front 2024-03-26T08:59:54.658292+0000 (oldest deadline 2024-03-26T09:00:20.458561+0000) 2024-03-26T09:00:21.799 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:21 smithi007 podman[104467]: 2024-03-26 09:00:21.548305269 +0000 UTC m=+5.661387041 container died c8dd7f144d0d5ba5539e84ba6d8ad9d122a2ad7a83b009b9f50793fd0c3b5b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T09:00:22.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:21 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:21.708+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:22.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:21 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:21.708+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:22.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:21 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:21.708+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:22.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:21 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:21.708+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:22.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:21 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:21.708+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:22.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:21 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:21.708+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:22.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:21 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:21.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:22.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:21 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:21.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:22.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:21 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:21.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:22.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:21 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:21.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:22.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:21 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:21.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:22.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:21 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:21.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:22.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:21 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:21.897+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:54.658309+0000 front 2024-03-26T08:59:54.658292+0000 (oldest deadline 2024-03-26T09:00:20.458561+0000) 2024-03-26T09:00:22.271 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:22 smithi007 podman[104467]: 2024-03-26 09:00:22.022066422 +0000 UTC m=+6.135148201 container cleanup c8dd7f144d0d5ba5539e84ba6d8ad9d122a2ad7a83b009b9f50793fd0c3b5b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-26T09:00:22.271 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:22 smithi007 bash[104467]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9 2024-03-26T09:00:22.549 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:22 smithi007 podman[104484]: 2024-03-26 09:00:22.273273249 +0000 UTC m=+0.717833007 container remove c8dd7f144d0d5ba5539e84ba6d8ad9d122a2ad7a83b009b9f50793fd0c3b5b39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-26T09:00:22.837 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:22 smithi007 podman[104574]: 2024-03-26 09:00:22.489956895 +0000 UTC m=+0.017273099 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T09:00:22.837 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:22 smithi007 podman[104574]: 2024-03-26 09:00:22.616004213 +0000 UTC m=+0.143320416 container create 1e1a3d1b48455ec9b683fa845f4053a0ea2119bb366c9c41cadb91e53c32b6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T09:00:22.837 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:22 smithi007 podman[104574]: 2024-03-26 09:00:22.835183657 +0000 UTC m=+0.362499861 container init 1e1a3d1b48455ec9b683fa845f4053a0ea2119bb366c9c41cadb91e53c32b6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-26T09:00:23.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:22 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:22.706+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:23.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:22 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:22.706+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:23.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:22 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:22.706+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:23.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:22 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:22.706+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:23.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:22 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:22.706+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:23.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:22 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:22.706+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:23.133 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:22 smithi007 podman[104574]: 2024-03-26 09:00:22.838173735 +0000 UTC m=+0.365489941 container start 1e1a3d1b48455ec9b683fa845f4053a0ea2119bb366c9c41cadb91e53c32b6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True) 2024-03-26T09:00:23.133 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:22 smithi007 podman[104574]: 2024-03-26 09:00:22.935464025 +0000 UTC m=+0.462780264 container attach 1e1a3d1b48455ec9b683fa845f4053a0ea2119bb366c9c41cadb91e53c32b6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-26T09:00:23.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:22 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:22.859+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:23.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:22 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:22.859+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:23.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:22 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:22.859+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:23.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:22 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:22.859+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:23.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:22 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:22.859+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:23.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:22 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:22.859+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:23.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:22 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:22.859+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:54.658309+0000 front 2024-03-26T08:59:54.658292+0000 (oldest deadline 2024-03-26T09:00:20.458561+0000) 2024-03-26T09:00:23.453 INFO:journalctl@ceph.osd.9.smithi007.stdout:Mar 26 09:00:23 smithi007 podman[104574]: 2024-03-26 09:00:23.135027158 +0000 UTC m=+0.662343363 container died 1e1a3d1b48455ec9b683fa845f4053a0ea2119bb366c9c41cadb91e53c32b6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-26T09:00:23.853 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.9.service' 2024-03-26T09:00:23.901 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:23.901 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-26T09:00:23.901 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-26T09:00:23.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.10 2024-03-26T09:00:24.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:23 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:23.670+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:24.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:23 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:23.670+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:24.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:23 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:23.670+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:24.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:23 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:23.670+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:24.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:23 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:23.670+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:24.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:23 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:23.670+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:24.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:23 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:23.894+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:24.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:23 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:23.894+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:24.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:23 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:23.894+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:24.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:23 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:23.894+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:24.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:23 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:23.894+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:24.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:23 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:23.894+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:24.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:23 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:23.894+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:54.658309+0000 front 2024-03-26T08:59:54.658292+0000 (oldest deadline 2024-03-26T09:00:20.458561+0000) 2024-03-26T09:00:24.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:23 smithi016 systemd[1]: Stopping Ceph osd.10 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:00:24.750 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.349+0000 7f071f24a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:24.750 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.349+0000 7f071f24a700 -1 osd.10 90 *** Got signal Terminated *** 2024-03-26T09:00:24.750 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.349+0000 7f071f24a700 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T09:00:25.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:24.622+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:25.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:24.622+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:25.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:24.622+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:25.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:24.622+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:25.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:24.622+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:25.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:24.622+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:25.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.889+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:25.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.889+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:25.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.889+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:25.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.889+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:25.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.889+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:25.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.889+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:25.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:24 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:24.889+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:54.658309+0000 front 2024-03-26T08:59:54.658292+0000 (oldest deadline 2024-03-26T09:00:20.458561+0000) 2024-03-26T09:00:26.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:25 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:25.653+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:26.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:25 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:25.653+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:26.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:25 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:25.653+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:26.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:25 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:25.653+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:26.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:25 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:25.653+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:26.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:25 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:25.653+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:26.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:25 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:25.907+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:26.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:25 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:25.907+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:26.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:25 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:25.907+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:26.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:25 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:25.907+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:26.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:25 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:25.907+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:26.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:25 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:25.907+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:26.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:25 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:25.907+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:54.658309+0000 front 2024-03-26T08:59:54.658292+0000 (oldest deadline 2024-03-26T09:00:20.458561+0000) 2024-03-26T09:00:27.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:26 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:26.676+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:27.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:26 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:26.676+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:27.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:26 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:26.676+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:27.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:26 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:26.676+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:27.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:26 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:26.676+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:27.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:26 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:26.676+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:27.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:26 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:26.935+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:27.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:26 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:26.935+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:27.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:26 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:26.935+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:27.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:26 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:26.935+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:27.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:26 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:26.935+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:27.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:26 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:26.935+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:27.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:26 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:26.935+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:54.658309+0000 front 2024-03-26T08:59:54.658292+0000 (oldest deadline 2024-03-26T09:00:20.458561+0000) 2024-03-26T09:00:28.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:27 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:27.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:28.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:27 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:27.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:28.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:27 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:27.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:28.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:27 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:27.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:28.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:27 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:27.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:28.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:27 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:27.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:28.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:27 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:27.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:28.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:27 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:27.940+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:28.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:27 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:27.940+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:28.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:27 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:27.940+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:28.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:27 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:27.940+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:28.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:27 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:27.940+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:28.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:27 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:27.940+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:28.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:27 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:27.940+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:54.658309+0000 front 2024-03-26T08:59:54.658292+0000 (oldest deadline 2024-03-26T09:00:20.458561+0000) 2024-03-26T09:00:29.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:28 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:28.614+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:29.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:28 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:28.614+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:29.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:28 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:28.614+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:29.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:28 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:28.614+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:29.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:28 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:28.614+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:29.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:28 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:28.614+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:29.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:28 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:28.614+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:29.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:28 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:28.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:07.450667+0000 front 2024-03-26T08:59:07.450700+0000 (oldest deadline 2024-03-26T08:59:30.950841+0000) 2024-03-26T09:00:29.250 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:28 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:28.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-03-26T08:59:12.051723+0000 front 2024-03-26T08:59:12.051701+0000 (oldest deadline 2024-03-26T08:59:40.252518+0000) 2024-03-26T09:00:29.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:28 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:28.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:20.253151+0000 front 2024-03-26T08:59:20.253061+0000 (oldest deadline 2024-03-26T08:59:46.153082+0000) 2024-03-26T09:00:29.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:28 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:28.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:27.854287+0000 front 2024-03-26T08:59:27.854408+0000 (oldest deadline 2024-03-26T08:59:53.154152+0000) 2024-03-26T09:00:29.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:28 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:28.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:38.355589+0000 front 2024-03-26T08:59:38.355551+0000 (oldest deadline 2024-03-26T09:00:00.655689+0000) 2024-03-26T09:00:29.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:28 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:28.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:47.057495+0000 front 2024-03-26T08:59:47.057421+0000 (oldest deadline 2024-03-26T09:00:11.757183+0000) 2024-03-26T09:00:29.251 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:28 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10[60521]: 2024-03-26T09:00:28.906+0000 7f0717836700 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:54.658309+0000 front 2024-03-26T08:59:54.658292+0000 (oldest deadline 2024-03-26T09:00:20.458561+0000) 2024-03-26T09:00:30.028 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:29 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:29.575+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:30.028 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:29 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:29.575+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:30.028 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:29 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:29.575+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:30.029 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:29 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:29.575+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:30.029 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:29 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:29.575+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:30.029 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:29 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:29.575+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:30.029 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:29 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:29.575+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:30.140 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:29 smithi016 podman[72627]: 2024-03-26 09:00:29.867225911 +0000 UTC m=+5.767141404 container died b16d787d9cd6a85f71aed93e942ff693b26ef783f27ce365b06dae0407cc01d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-26T09:00:30.482 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:30 smithi016 podman[72627]: 2024-03-26 09:00:30.232882679 +0000 UTC m=+6.132798170 container cleanup b16d787d9cd6a85f71aed93e942ff693b26ef783f27ce365b06dae0407cc01d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T09:00:30.482 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:30 smithi016 bash[72627]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10 2024-03-26T09:00:30.750 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:30 smithi016 podman[72646]: 2024-03-26 09:00:30.483935566 +0000 UTC m=+0.611246047 container remove b16d787d9cd6a85f71aed93e942ff693b26ef783f27ce365b06dae0407cc01d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T09:00:31.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:30 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:30.593+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:31.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:30 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:30.593+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:31.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:30 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:30.593+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:31.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:30 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:30.593+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:31.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:30 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:30.593+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:31.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:30 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:30.593+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:31.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:30 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:30.593+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:31.137 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:30 smithi016 podman[72734]: 2024-03-26 09:00:30.722684898 +0000 UTC m=+0.017927837 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T09:00:31.137 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:30 smithi016 podman[72734]: 2024-03-26 09:00:30.826238906 +0000 UTC m=+0.121481841 container create 6365713b18d27a64e0f48906fff12ad1e8edaabfd4948935e690928a3b052866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-26T09:00:31.138 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:31 smithi016 podman[72734]: 2024-03-26 09:00:31.03813531 +0000 UTC m=+0.333378226 container init 6365713b18d27a64e0f48906fff12ad1e8edaabfd4948935e690928a3b052866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-26T09:00:31.138 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:31 smithi016 podman[72734]: 2024-03-26 09:00:31.041468363 +0000 UTC m=+0.336711278 container start 6365713b18d27a64e0f48906fff12ad1e8edaabfd4948935e690928a3b052866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T09:00:31.138 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:31 smithi016 podman[72734]: 2024-03-26 09:00:31.138043039 +0000 UTC m=+0.433285962 container attach 6365713b18d27a64e0f48906fff12ad1e8edaabfd4948935e690928a3b052866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T09:00:31.500 INFO:journalctl@ceph.osd.10.smithi016.stdout:Mar 26 09:00:31 smithi016 podman[72734]: 2024-03-26 09:00:31.34603806 +0000 UTC m=+0.641280981 container died 6365713b18d27a64e0f48906fff12ad1e8edaabfd4948935e690928a3b052866 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T09:00:31.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.10.service' 2024-03-26T09:00:31.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:31.960 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-26T09:00:31.960 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-26T09:00:31.960 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.11 2024-03-26T09:00:31.993 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:31.626+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:31.993 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:31.626+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:31.993 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:31.626+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:31.993 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:31.626+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:31.994 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:31.626+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:31.994 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:31.626+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:31.994 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:31 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:31.626+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:32.280 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 systemd[1]: Stopping Ceph osd.11 for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:00:32.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.281+0000 7ff4d8d2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:32.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.281+0000 7ff4d8d2e700 -1 osd.11 90 *** Got signal Terminated *** 2024-03-26T09:00:32.573 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.281+0000 7ff4d8d2e700 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T09:00:33.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:33.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:33.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:33.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:33.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:33.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:33.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:32 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:32.661+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:34.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:33.711+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:34.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:33.711+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:34.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:33.711+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:34.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:33.711+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:34.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:33.711+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:34.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:33.711+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:34.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:33 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:33.711+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:35.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:34.714+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:35.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:34.714+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:35.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:34.714+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:35.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:34.714+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:35.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:34.714+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:35.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:34.714+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:35.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:34 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:34.714+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:36.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:35.705+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:36.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:35.705+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:36.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:35.705+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:36.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:35.705+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:36.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:35.705+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:36.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:35.705+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:36.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:35 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:35.705+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:37.073 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:36.669+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-03-26T08:59:06.950644+0000 front 2024-03-26T08:59:06.950603+0000 (oldest deadline 2024-03-26T08:59:30.450678+0000) 2024-03-26T09:00:37.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:36.669+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6804 osd.2 since back 2024-03-26T08:59:23.754089+0000 front 2024-03-26T08:59:23.753897+0000 (oldest deadline 2024-03-26T08:59:44.853994+0000) 2024-03-26T09:00:37.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:36.669+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6814 osd.3 since back 2024-03-26T08:59:30.755182+0000 front 2024-03-26T08:59:30.755251+0000 (oldest deadline 2024-03-26T08:59:56.055381+0000) 2024-03-26T09:00:37.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:36.669+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6812 osd.4 since back 2024-03-26T08:59:36.056150+0000 front 2024-03-26T08:59:36.056191+0000 (oldest deadline 2024-03-26T09:00:01.355936+0000) 2024-03-26T09:00:37.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:36.669+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6812 osd.5 since back 2024-03-26T08:59:46.557503+0000 front 2024-03-26T08:59:46.557595+0000 (oldest deadline 2024-03-26T09:00:08.857712+0000) 2024-03-26T09:00:37.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:36.669+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.7:6822 osd.6 since back 2024-03-26T08:59:52.358692+0000 front 2024-03-26T08:59:52.358725+0000 (oldest deadline 2024-03-26T09:00:17.658515+0000) 2024-03-26T09:00:37.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:36.669+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.16:6820 osd.7 since back 2024-03-26T09:00:02.359703+0000 front 2024-03-26T09:00:02.359605+0000 (oldest deadline 2024-03-26T09:00:27.659410+0000) 2024-03-26T09:00:37.074 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:36 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11[62004]: 2024-03-26T09:00:36.669+0000 7ff4d131a700 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.44:6820 osd.8 since back 2024-03-26T09:00:11.160203+0000 front 2024-03-26T09:00:11.160173+0000 (oldest deadline 2024-03-26T09:00:36.460216+0000) 2024-03-26T09:00:37.978 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:37 smithi044 podman[73574]: 2024-03-26 09:00:37.700391099 +0000 UTC m=+5.533135702 container died 1bce0b947910562611aec5c9a56cdb45bbd26be1ea564eaa82e779d8039378a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T09:00:38.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:38 smithi044 podman[73574]: 2024-03-26 09:00:38.064299477 +0000 UTC m=+5.897044082 container cleanup 1bce0b947910562611aec5c9a56cdb45bbd26be1ea564eaa82e779d8039378a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T09:00:38.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:38 smithi044 bash[73574]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11 2024-03-26T09:00:38.636 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:38 smithi044 podman[73591]: 2024-03-26 09:00:38.340750423 +0000 UTC m=+0.640116681 container remove 1bce0b947910562611aec5c9a56cdb45bbd26be1ea564eaa82e779d8039378a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-26T09:00:38.917 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:38 smithi044 podman[73687]: 2024-03-26 09:00:38.54050369 +0000 UTC m=+0.018379135 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T09:00:38.917 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:38 smithi044 podman[73687]: 2024-03-26 09:00:38.649777634 +0000 UTC m=+0.127653062 container create 2152768cfb9913e1eee8993c7d891dc0bce3c7952ebef8255c748a270b08c25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T09:00:38.917 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:38 smithi044 podman[73687]: 2024-03-26 09:00:38.835639847 +0000 UTC m=+0.313515301 container init 2152768cfb9913e1eee8993c7d891dc0bce3c7952ebef8255c748a270b08c25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-26T09:00:38.917 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:38 smithi044 podman[73687]: 2024-03-26 09:00:38.839149482 +0000 UTC m=+0.317024912 container start 2152768cfb9913e1eee8993c7d891dc0bce3c7952ebef8255c748a270b08c25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-26T09:00:38.917 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:38 smithi044 podman[73687]: 2024-03-26 09:00:38.918532382 +0000 UTC m=+0.396407820 container attach 2152768cfb9913e1eee8993c7d891dc0bce3c7952ebef8255c748a270b08c25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T09:00:39.323 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:39 smithi044 podman[73687]: 2024-03-26 09:00:39.144913688 +0000 UTC m=+0.622789130 container died 2152768cfb9913e1eee8993c7d891dc0bce3c7952ebef8255c748a270b08c25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-26T09:00:39.675 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.11.service' 2024-03-26T09:00:39.749 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:39 smithi044 podman[73713]: 2024-03-26 09:00:39.655342631 +0000 UTC m=+0.505202430 container remove 2152768cfb9913e1eee8993c7d891dc0bce3c7952ebef8255c748a270b08c25c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-osd-11-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T09:00:39.749 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:39 smithi044 systemd[1]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.11.service: Deactivated successfully. 2024-03-26T09:00:39.750 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:39 smithi044 systemd[1]: Stopped Ceph osd.11 for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T09:00:39.750 INFO:journalctl@ceph.osd.11.smithi044.stdout:Mar 26 09:00:39 smithi044 systemd[1]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@osd.11.service: Consumed 10min 11.001s CPU time. 2024-03-26T09:00:39.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:39.767 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-26T09:00:39.767 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-26T09:00:39.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.a 2024-03-26T09:00:40.050 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 26 09:00:39 smithi007 systemd[1]: Stopping Ceph mds.a for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:00:40.050 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 26 09:00:40 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-a[71703]: 2024-03-26T09:00:40.039+0000 7f91a54f5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:40.050 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 26 09:00:40 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-a[71703]: 2024-03-26T09:00:40.039+0000 7f91a54f5700 -1 mds.a *** got signal Terminated *** 2024-03-26T09:00:43.398 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 26 09:00:43 smithi007 podman[104679]: 2024-03-26 09:00:43.13067485 +0000 UTC m=+3.174120510 container died 436cb35c96197e3a2c1ebe70ec5e1f1515fb9af8b936b7a3407c7e771c73a2bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-26T09:00:43.773 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 26 09:00:43 smithi007 podman[104679]: 2024-03-26 09:00:43.481607125 +0000 UTC m=+3.525052785 container cleanup 436cb35c96197e3a2c1ebe70ec5e1f1515fb9af8b936b7a3407c7e771c73a2bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-26T09:00:43.773 INFO:journalctl@ceph.mds.a.smithi007.stdout:Mar 26 09:00:43 smithi007 bash[104679]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-a 2024-03-26T09:00:43.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.a.service' 2024-03-26T09:00:43.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:43.951 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-26T09:00:43.952 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-26T09:00:43.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.d 2024-03-26T09:00:44.300 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 09:00:44 smithi007 systemd[1]: Stopping Ceph mds.d for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:00:44.300 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 09:00:44 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d[70712]: 2024-03-26T09:00:44.277+0000 7fb05ee88700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:44.300 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 09:00:44 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d[70712]: 2024-03-26T09:00:44.277+0000 7fb05ee88700 -1 mds.d *** got signal Terminated *** 2024-03-26T09:00:47.014 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 09:00:46 smithi007 podman[104813]: 2024-03-26 09:00:46.706656853 +0000 UTC m=+2.530483341 container died eb1102f0da13a665ef7eee105a4e06b052e592df2fced68ee5079ab56d595a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T09:00:47.014 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 09:00:47 smithi007 podman[104813]: 2024-03-26 09:00:47.015998053 +0000 UTC m=+2.839824539 container cleanup eb1102f0da13a665ef7eee105a4e06b052e592df2fced68ee5079ab56d595a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T09:00:47.299 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 09:00:47 smithi007 bash[104813]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d 2024-03-26T09:00:47.300 INFO:journalctl@ceph.mds.d.smithi007.stdout:Mar 26 09:00:47 smithi007 podman[104824]: 2024-03-26 09:00:47.236774508 +0000 UTC m=+0.533975834 container remove eb1102f0da13a665ef7eee105a4e06b052e592df2fced68ee5079ab56d595a72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T09:00:47.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.d.service' 2024-03-26T09:00:47.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:47.479 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-26T09:00:47.479 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-26T09:00:47.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.g 2024-03-26T09:00:47.787 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 26 09:00:47 smithi007 systemd[1]: Stopping Ceph mds.g for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:00:48.050 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 26 09:00:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-g[71362]: 2024-03-26T09:00:47.787+0000 7ffbd37c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:48.050 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 26 09:00:47 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-g[71362]: 2024-03-26T09:00:47.787+0000 7ffbd37c3700 -1 mds.g *** got signal Terminated *** 2024-03-26T09:00:52.050 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 26 09:00:51 smithi007 podman[104934]: 2024-03-26 09:00:51.798607562 +0000 UTC m=+4.095979844 container died 6a3f25f2be0a2051853edb78469c96d9b8729b10e1b44ae286ce51111a316eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-g, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-26T09:00:52.531 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 26 09:00:52 smithi007 podman[104934]: 2024-03-26 09:00:52.272758264 +0000 UTC m=+4.570130536 container cleanup 6a3f25f2be0a2051853edb78469c96d9b8729b10e1b44ae286ce51111a316eca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-g, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T09:00:52.531 INFO:journalctl@ceph.mds.g.smithi007.stdout:Mar 26 09:00:52 smithi007 bash[104934]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-g 2024-03-26T09:00:52.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.g.service' 2024-03-26T09:00:52.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:52.736 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-26T09:00:52.736 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-26T09:00:52.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.j 2024-03-26T09:00:53.049 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:52 smithi007 systemd[1]: Stopping Ceph mds.j for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:00:53.550 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j[70364]: 2024-03-26T09:00:53.049+0000 7faacaa1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:53.550 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:53 smithi007 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j[70364]: 2024-03-26T09:00:53.049+0000 7faacaa1f700 -1 mds.j *** got signal Terminated *** 2024-03-26T09:00:56.300 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:56 smithi007 podman[105055]: 2024-03-26 09:00:56.045291225 +0000 UTC m=+3.092629815 container died 91d9c52035e81bf66c2b1f5853e34323143e67ef1ea1e35f83d8961c3734e773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T09:00:56.575 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:56 smithi007 podman[105055]: 2024-03-26 09:00:56.392977628 +0000 UTC m=+3.440316217 container cleanup 91d9c52035e81bf66c2b1f5853e34323143e67ef1ea1e35f83d8961c3734e773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-26T09:00:56.575 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:56 smithi007 bash[105055]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j 2024-03-26T09:00:56.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.j.service' 2024-03-26T09:00:56.846 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:56 smithi007 podman[105069]: 2024-03-26 09:00:56.577158349 +0000 UTC m=+0.524063136 container remove 91d9c52035e81bf66c2b1f5853e34323143e67ef1ea1e35f83d8961c3734e773 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-03-26T09:00:56.846 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:56 smithi007 systemd[1]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.j.service: Deactivated successfully. 2024-03-26T09:00:56.846 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:56 smithi007 systemd[1]: Stopped Ceph mds.j for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T09:00:56.846 INFO:journalctl@ceph.mds.j.smithi007.stdout:Mar 26 09:00:56 smithi007 systemd[1]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.j.service: Consumed 1.097s CPU time. 2024-03-26T09:00:56.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:56.856 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-26T09:00:56.856 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-26T09:00:56.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.b 2024-03-26T09:00:57.250 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:56 smithi016 systemd[1]: Stopping Ceph mds.b for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:00:57.533 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:57 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b[66227]: 2024-03-26T09:00:57.291+0000 7fb7e6b6f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:00:57.533 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:57 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b[66227]: 2024-03-26T09:00:57.291+0000 7fb7e6b6f700 -1 mds.b *** got signal Terminated *** 2024-03-26T09:00:59.350 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:59 smithi016 podman[72840]: 2024-03-26 09:00:59.055280216 +0000 UTC m=+1.995331984 container died 19cd65b0137d56ba868c99ebf378bff18efc9fa7373483d44816e9d84eb52471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-26T09:00:59.683 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:59 smithi016 podman[72840]: 2024-03-26 09:00:59.450714385 +0000 UTC m=+2.390766141 container cleanup 19cd65b0137d56ba868c99ebf378bff18efc9fa7373483d44816e9d84eb52471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-26T09:00:59.684 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:59 smithi016 bash[72840]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b 2024-03-26T09:00:59.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.b.service' 2024-03-26T09:00:59.951 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:59 smithi016 podman[72866]: 2024-03-26 09:00:59.685204353 +0000 UTC m=+0.629198188 container remove 19cd65b0137d56ba868c99ebf378bff18efc9fa7373483d44816e9d84eb52471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-26T09:00:59.951 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:59 smithi016 systemd[1]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.b.service: Deactivated successfully. 2024-03-26T09:00:59.951 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:59 smithi016 systemd[1]: Stopped Ceph mds.b for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T09:00:59.952 INFO:journalctl@ceph.mds.b.smithi016.stdout:Mar 26 09:00:59 smithi016 systemd[1]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.b.service: Consumed 9min 27.969s CPU time. 2024-03-26T09:00:59.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:00:59.965 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-26T09:00:59.965 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-26T09:00:59.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.e 2024-03-26T09:01:00.250 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 09:01:00 smithi016 systemd[1]: Stopping Ceph mds.e for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:01:00.750 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 09:01:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e[65608]: 2024-03-26T09:01:00.286+0000 7f736740c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:01:00.751 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 09:01:00 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e[65608]: 2024-03-26T09:01:00.286+0000 7f736740c700 -1 mds.e *** got signal Terminated *** 2024-03-26T09:01:03.303 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 09:01:03 smithi016 podman[72972]: 2024-03-26 09:01:03.049688325 +0000 UTC m=+2.852502517 container died 3dc18f6b035a2b7258f4ddf9229631b623623c58d90f9c887204b7701df4b965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T09:01:03.697 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 09:01:03 smithi016 podman[72972]: 2024-03-26 09:01:03.395857322 +0000 UTC m=+3.198671510 container cleanup 3dc18f6b035a2b7258f4ddf9229631b623623c58d90f9c887204b7701df4b965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-26T09:01:03.697 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 09:01:03 smithi016 bash[72972]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e 2024-03-26T09:01:03.955 INFO:journalctl@ceph.mds.e.smithi016.stdout:Mar 26 09:01:03 smithi016 podman[73001]: 2024-03-26 09:01:03.698844167 +0000 UTC m=+0.645125279 container remove 3dc18f6b035a2b7258f4ddf9229631b623623c58d90f9c887204b7701df4b965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-26T09:01:03.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.e.service' 2024-03-26T09:01:04.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:01:04.020 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-26T09:01:04.020 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-26T09:01:04.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.h 2024-03-26T09:01:04.250 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 09:01:04 smithi016 systemd[1]: Stopping Ceph mds.h for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:01:04.750 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 09:01:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h[65919]: 2024-03-26T09:01:04.483+0000 7fe2b2e3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:01:04.751 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 09:01:04 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h[65919]: 2024-03-26T09:01:04.483+0000 7fe2b2e3a700 -1 mds.h *** got signal Terminated *** 2024-03-26T09:01:07.324 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 09:01:07 smithi016 podman[73106]: 2024-03-26 09:01:07.053282162 +0000 UTC m=+2.803540843 container died 9a5e1bdb2be06e2b2cd79c7eed70bec6c6310ad35f9a600929f42008ffcd7533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T09:01:07.682 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 09:01:07 smithi016 podman[73106]: 2024-03-26 09:01:07.42496461 +0000 UTC m=+3.175223283 container cleanup 9a5e1bdb2be06e2b2cd79c7eed70bec6c6310ad35f9a600929f42008ffcd7533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-26T09:01:07.682 INFO:journalctl@ceph.mds.h.smithi016.stdout:Mar 26 09:01:07 smithi016 bash[73106]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-h 2024-03-26T09:01:07.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.h.service' 2024-03-26T09:01:07.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:01:07.851 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-26T09:01:07.851 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-26T09:01:07.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.k 2024-03-26T09:01:08.185 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 09:01:07 smithi016 systemd[1]: Stopping Ceph mds.k for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:01:08.500 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 09:01:08 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k[65283]: 2024-03-26T09:01:08.186+0000 7facfc6fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:01:08.501 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 09:01:08 smithi016 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k[65283]: 2024-03-26T09:01:08.186+0000 7facfc6fd700 -1 mds.k *** got signal Terminated *** 2024-03-26T09:01:09.460 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 09:01:09 smithi016 podman[73226]: 2024-03-26 09:01:09.159301252 +0000 UTC m=+1.079408011 container died fcec085315304577cd4263d34564370219c1ee26e41b02270a5d01bdd7caa5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-26T09:01:09.460 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 09:01:09 smithi016 podman[73226]: 2024-03-26 09:01:09.461704764 +0000 UTC m=+1.381811512 container cleanup fcec085315304577cd4263d34564370219c1ee26e41b02270a5d01bdd7caa5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T09:01:09.750 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 09:01:09 smithi016 bash[73226]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k 2024-03-26T09:01:09.750 INFO:journalctl@ceph.mds.k.smithi016.stdout:Mar 26 09:01:09 smithi016 podman[73240]: 2024-03-26 09:01:09.662629279 +0000 UTC m=+0.496605521 container remove fcec085315304577cd4263d34564370219c1ee26e41b02270a5d01bdd7caa5e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-k, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-26T09:01:09.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.k.service' 2024-03-26T09:01:09.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:01:09.841 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-26T09:01:09.841 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-26T09:01:09.841 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.c 2024-03-26T09:01:10.130 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 09:01:09 smithi044 systemd[1]: Stopping Ceph mds.c for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:01:10.573 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 09:01:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c[67346]: 2024-03-26T09:01:10.130+0000 7f76157e4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:01:10.573 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 09:01:10 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c[67346]: 2024-03-26T09:01:10.131+0000 7f76157e4700 -1 mds.c *** got signal Terminated *** 2024-03-26T09:01:14.396 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 09:01:14 smithi044 podman[73821]: 2024-03-26 09:01:14.143312346 +0000 UTC m=+4.092487621 container died 4eede49de1663048a087b74290ced867515694100dbee4e51241d1c95e8958d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-26T09:01:14.655 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 09:01:14 smithi044 podman[73821]: 2024-03-26 09:01:14.471978287 +0000 UTC m=+4.421153564 container cleanup 4eede49de1663048a087b74290ced867515694100dbee4e51241d1c95e8958d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-26T09:01:14.655 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 09:01:14 smithi044 bash[73821]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c 2024-03-26T09:01:14.907 INFO:journalctl@ceph.mds.c.smithi044.stdout:Mar 26 09:01:14 smithi044 podman[73836]: 2024-03-26 09:01:14.656470166 +0000 UTC m=+0.509543407 container remove 4eede49de1663048a087b74290ced867515694100dbee4e51241d1c95e8958d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-03-26T09:01:14.912 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.c.service' 2024-03-26T09:01:14.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:01:14.954 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-26T09:01:14.954 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-26T09:01:14.954 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.f 2024-03-26T09:01:15.291 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 09:01:15 smithi044 systemd[1]: Stopping Ceph mds.f for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:01:15.573 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 09:01:15 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f[67036]: 2024-03-26T09:01:15.291+0000 7f57cee74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:01:15.573 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 09:01:15 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f[67036]: 2024-03-26T09:01:15.291+0000 7f57cee74700 -1 mds.f *** got signal Terminated *** 2024-03-26T09:01:19.316 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 09:01:19 smithi044 podman[73945]: 2024-03-26 09:01:19.039803676 +0000 UTC m=+3.849833782 container died 8c77619fc916570c7b2091a7f43eb47f4aab06bffffbd6751c27176f48c5b834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-26T09:01:19.573 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 09:01:19 smithi044 podman[73945]: 2024-03-26 09:01:19.40061951 +0000 UTC m=+4.210649599 container cleanup 8c77619fc916570c7b2091a7f43eb47f4aab06bffffbd6751c27176f48c5b834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T09:01:19.573 INFO:journalctl@ceph.mds.f.smithi044.stdout:Mar 26 09:01:19 smithi044 bash[73945]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-f 2024-03-26T09:01:19.849 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.f.service' 2024-03-26T09:01:19.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:01:19.888 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-26T09:01:19.888 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-26T09:01:19.889 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.i 2024-03-26T09:01:20.222 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:19 smithi044 systemd[1]: Stopping Ceph mds.i for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:01:20.573 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:20 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i[67649]: 2024-03-26T09:01:20.220+0000 7effbc8e9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:01:20.573 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:20 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i[67649]: 2024-03-26T09:01:20.220+0000 7effbc8e9700 -1 mds.i *** got signal Terminated *** 2024-03-26T09:01:23.311 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:23 smithi044 podman[74064]: 2024-03-26 09:01:23.056830579 +0000 UTC m=+2.943917969 container died 6be81398e8b113889b2801054aeec2b4b77bdad47aa63ae6dc5a1b5170f0dac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T09:01:23.573 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:23 smithi044 podman[74064]: 2024-03-26 09:01:23.387390698 +0000 UTC m=+3.274478085 container cleanup 6be81398e8b113889b2801054aeec2b4b77bdad47aa63ae6dc5a1b5170f0dac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-26T09:01:23.574 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:23 smithi044 bash[74064]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i 2024-03-26T09:01:23.810 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.i.service' 2024-03-26T09:01:23.837 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:23 smithi044 podman[74079]: 2024-03-26 09:01:23.579915442 +0000 UTC m=+0.522490849 container remove 6be81398e8b113889b2801054aeec2b4b77bdad47aa63ae6dc5a1b5170f0dac7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-i, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T09:01:23.837 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:23 smithi044 systemd[1]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.i.service: Deactivated successfully. 2024-03-26T09:01:23.837 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:23 smithi044 systemd[1]: Stopped Ceph mds.i for 2c753f3e-eb4b-11ee-95d0-87774f69a715. 2024-03-26T09:01:23.837 INFO:journalctl@ceph.mds.i.smithi044.stdout:Mar 26 09:01:23 smithi044 systemd[1]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.i.service: Consumed 8min 44.586s CPU time. 2024-03-26T09:01:23.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:01:23.848 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-26T09:01:23.848 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-26T09:01:23.848 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.l 2024-03-26T09:01:24.173 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 09:01:23 smithi044 systemd[1]: Stopping Ceph mds.l for 2c753f3e-eb4b-11ee-95d0-87774f69a715... 2024-03-26T09:01:24.573 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 09:01:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l[66724]: 2024-03-26T09:01:24.174+0000 7f66affee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T09:01:24.573 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 09:01:24 smithi044 ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l[66724]: 2024-03-26T09:01:24.174+0000 7f66affee700 -1 mds.l *** got signal Terminated *** 2024-03-26T09:01:26.573 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 09:01:26 smithi044 podman[74185]: 2024-03-26 09:01:26.310254498 +0000 UTC m=+2.234690010 container died 3cc654354be32b3d3dc4c40943e75b821f7bbffb190517ccffc3d0b9f10d80b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T09:01:26.967 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 09:01:26 smithi044 podman[74185]: 2024-03-26 09:01:26.632637104 +0000 UTC m=+2.557072616 container cleanup 3cc654354be32b3d3dc4c40943e75b821f7bbffb190517ccffc3d0b9f10d80b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-26T09:01:26.967 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 09:01:26 smithi044 bash[74185]: ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l 2024-03-26T09:01:26.967 INFO:journalctl@ceph.mds.l.smithi044.stdout:Mar 26 09:01:26 smithi044 podman[74200]: 2024-03-26 09:01:26.841937542 +0000 UTC m=+0.523894761 container remove 3cc654354be32b3d3dc4c40943e75b821f7bbffb190517ccffc3d0b9f10d80b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True) 2024-03-26T09:01:26.984 DEBUG:teuthology.orchestra.run.smithi044:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2c753f3e-eb4b-11ee-95d0-87774f69a715@mds.l.service' 2024-03-26T09:01:27.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T09:01:27.023 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-26T09:01:27.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T09:01:27.220 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T09:01:29.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T09:01:29.867 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T09:01:32.496 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T09:01:32.699 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T09:01:35.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T09:01:35.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T09:01:35.359 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T09:01:35.393 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-26T09:01:35.395 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/remote/smithi007/crash 2024-03-26T09:01:35.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/crash -- . 2024-03-26T09:01:35.432 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T09:01:35.433 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-03-26T09:01:35.435 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/remote/smithi016/crash 2024-03-26T09:01:35.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/crash -- . 2024-03-26T09:01:35.485 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T09:01:35.485 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-03-26T09:01:35.487 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/remote/smithi044/crash 2024-03-26T09:01:35.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/crash -- . 2024-03-26T09:01:35.528 INFO:teuthology.orchestra.run.smithi044.stderr:tar: /var/lib/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T09:01:35.528 INFO:teuthology.orchestra.run.smithi044.stderr:tar: Error is not recoverable: exiting now 2024-03-26T09:01:35.529 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-26T09:01:35.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T09:01:35.564 INFO:teuthology.orchestra.run.smithi007.stdout:2024-03-26T08:33:57.699377+0000 mon.a (mon.0) 189 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-26T09:01:35.564 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-26T09:01:35.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[SEC\]' /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T09:01:35.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]' /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T09:01:35.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[WRN\]' /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T09:01:35.786 INFO:teuthology.orchestra.run.smithi007.stdout:2024-03-26T08:33:57.699377+0000 mon.a (mon.0) 189 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-26T09:01:35.786 INFO:tasks.cephadm:Compressing logs... 2024-03-26T09:01:35.786 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T09:01:35.828 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T09:01:35.831 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T09:01:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T09:01:35.852 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T09:01:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-26T09:01:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-26T09:01:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T09:01:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-26T09:01:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-26T09:01:35.854 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-26T09:01:35.855 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-26T09:01:35.855 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-26T09:01:35.857 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mgr.x.log: 90.2% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T09:01:35.859 INFO:teuthology.orchestra.run.smithi007.stderr: 94.6% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.j.log.gz 2024-03-26T09:01:35.859 INFO:teuthology.orchestra.run.smithi007.stderr: 94.5% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-26T09:01:35.860 INFO:teuthology.orchestra.run.smithi007.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T09:01:35.862 INFO:teuthology.orchestra.run.smithi044.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T09:01:35.862 INFO:teuthology.orchestra.run.smithi044.stderr:: No such file or directory 2024-03-26T09:01:35.862 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-26T09:01:35.863 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T09:01:35.863 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.i.log 2024-03-26T09:01:35.863 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.l.log: /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-26T09:01:35.863 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-26T09:01:35.863 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-26T09:01:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-03-26T09:01:35.872 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-03-26T09:01:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-26T09:01:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.10.log: /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.7.log: 90.3% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T09:01:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-26T09:01:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T09:01:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log 2024-03-26T09:01:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T09:01:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log: 83.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T09:01:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-26T09:01:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-volume.log: -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-26T09:01:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.9.log: 87.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T09:01:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr: /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.f.log: 90.4% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log: 94.6% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.2.log: 87.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-volume.log 93.1% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-26T09:01:35.879 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-03-26T09:01:35.880 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-26T09:01:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-26T09:01:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T09:01:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-26T09:01:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mgr.y.log: 93.1% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-26T09:01:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log 2024-03-26T09:01:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T09:01:35.884 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.a.log: 91.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T09:01:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T09:01:35.885 INFO:teuthology.orchestra.run.smithi007.stderr: 94.5% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-26T09:01:35.886 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-volume.log: 90.9% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T09:01:35.887 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log: 87.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T09:01:35.888 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-26T09:01:35.902 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T09:01:35.953 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mon.c.log: 94.8% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-26T09:01:35.963 INFO:teuthology.orchestra.run.smithi016.stderr: 94.5% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-26T09:01:36.046 INFO:teuthology.orchestra.run.smithi044.stderr: 92.9% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-26T09:01:36.065 INFO:teuthology.orchestra.run.smithi044.stderr: 92.5% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-26T09:01:36.181 INFO:teuthology.orchestra.run.smithi007.stderr: 90.3% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-26T09:01:36.339 INFO:teuthology.orchestra.run.smithi016.stderr: 91.8% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-26T09:01:36.366 INFO:teuthology.orchestra.run.smithi044.stderr: 91.8% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-26T09:01:36.651 INFO:teuthology.orchestra.run.smithi016.stderr: 94.5% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-26T09:01:37.388 INFO:teuthology.orchestra.run.smithi007.stderr: 90.3% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-26T09:01:49.772 INFO:teuthology.orchestra.run.smithi016.stderr: 94.1% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-26T09:02:06.837 INFO:teuthology.orchestra.run.smithi007.stderr: 91.4% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-26T09:02:14.885 INFO:teuthology.orchestra.run.smithi016.stderr: 91.4% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-26T09:02:19.087 INFO:teuthology.orchestra.run.smithi044.stderr: 91.2% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-26T09:02:22.042 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-26T09:02:24.962 INFO:teuthology.orchestra.run.smithi044.stderr: 91.2% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-26T09:02:25.004 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-26T09:02:26.889 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-26T09:02:32.263 INFO:teuthology.orchestra.run.smithi044.stderr: 90.9% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-26T09:02:32.264 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-03-26T09:02:32.265 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m56.418s 2024-03-26T09:02:32.265 INFO:teuthology.orchestra.run.smithi044.stderr:user 3m10.194s 2024-03-26T09:02:32.265 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m5.342s 2024-03-26T09:02:35.019 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-26T09:02:36.637 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-26T09:02:37.831 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-26T09:02:37.833 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-03-26T09:02:37.834 INFO:teuthology.orchestra.run.smithi007.stderr:real 1m1.993s 2024-03-26T09:02:37.834 INFO:teuthology.orchestra.run.smithi007.stderr:user 3m18.487s 2024-03-26T09:02:37.834 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m5.764s 2024-03-26T09:02:40.833 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/2c753f3e-eb4b-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-26T09:02:40.836 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-03-26T09:02:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:real 1m4.990s 2024-03-26T09:02:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:user 3m42.906s 2024-03-26T09:02:40.836 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m6.069s 2024-03-26T09:02:40.836 INFO:tasks.cephadm:Archiving logs... 2024-03-26T09:02:40.837 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/remote/smithi007/log 2024-03-26T09:02:40.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T09:03:15.915 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/remote/smithi016/log 2024-03-26T09:03:15.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T09:03:50.228 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/remote/smithi044/log 2024-03-26T09:03:50.230 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T09:04:23.855 INFO:tasks.cephadm:Removing cluster... 2024-03-26T09:04:23.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 --force 2024-03-26T09:04:24.053 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T09:04:24.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 --force 2024-03-26T09:04:24.842 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T09:04:25.430 DEBUG:teuthology.orchestra.run.smithi044:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 2c753f3e-eb4b-11ee-95d0-87774f69a715 --force 2024-03-26T09:04:25.635 INFO:teuthology.orchestra.run.smithi044.stdout:Deleting cluster with fsid: 2c753f3e-eb4b-11ee-95d0-87774f69a715 2024-03-26T09:04:26.209 INFO:tasks.cephadm:Removing cephadm ... 2024-03-26T09:04:26.210 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T09:04:26.227 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T09:04:26.246 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T09:04:26.262 INFO:tasks.cephadm:Teardown complete 2024-03-26T09:04:26.262 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-26T09:04:26.273 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-26T09:04:26.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T09:04:26.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T09:04:26.289 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T09:04:26.404 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-26T09:04:26.414 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-26T09:04:26.415 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T09:04:26.418 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T09:04:26.425 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T09:04:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-03-26T09:04:26.435 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T09:04:26.435 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-03-26T09:04:26.436 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 -68us[-6936ns] +/- 81ms 2024-03-26T09:04:26.436 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -1613us[-1551us] +/- 63ms 2024-03-26T09:04:26.436 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +2320us[+2381us] +/- 86ms 2024-03-26T09:04:26.436 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T09:04:26.438 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-03-26T09:04:26.442 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-03-26T09:04:26.444 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T09:04:26.444 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-03-26T09:04:26.444 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 47 -27us[ -30us] +/- 81ms 2024-03-26T09:04:26.444 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -1593us[-1597us] +/- 63ms 2024-03-26T09:04:26.444 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +2346us[+2343us] +/- 86ms 2024-03-26T09:04:26.444 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T09:04:26.448 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T09:04:26.448 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-03-26T09:04:26.448 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 -51us[ -43us] +/- 81ms 2024-03-26T09:04:26.448 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -1617us[-1609us] +/- 63ms 2024-03-26T09:04:26.448 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +2336us[+2343us] +/- 86ms 2024-03-26T09:04:26.449 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T09:04:26.449 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-26T09:04:26.469 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-26T09:04:26.470 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-26T09:04:26.486 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T09:04:26.575 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T09:04:26.663 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T09:04:26.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T09:04:26.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T09:04:26.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T09:04:26.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T09:04:26.815 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T09:04:26.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T09:04:26.849 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-26T09:04:26.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-26T09:04:26.989 INFO:teuthology.task.internal:Duration was 2412.370555 seconds 2024-03-26T09:04:26.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-26T09:04:27.039 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-26T09:04:27.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T09:04:27.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T09:04:27.053 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T09:04:27.087 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T09:04:27.106 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T09:04:27.109 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T09:04:27.607 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-26T09:04:27.608 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-03-26T09:04:27.608 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T09:04:27.656 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-03-26T09:04:27.656 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T09:04:27.686 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-03-26T09:04:27.686 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T09:04:27.715 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-26T09:04:27.716 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T09:04:27.720 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T09:04:27.730 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T09:04:27.790 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-26T09:04:27.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T09:04:27.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T09:04:27.809 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T09:04:28.185 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-26T09:04:28.197 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-26T09:04:28.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T09:04:28.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T09:04:28.255 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T09:04:28.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-26T09:04:28.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T09:04:28.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T09:04:28.303 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T09:04:28.324 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-03-26T09:04:28.329 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-03-26T09:04:28.355 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-03-26T09:04:28.386 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T09:04:28.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T09:04:28.400 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T09:04:28.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T09:04:28.414 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T09:04:28.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T09:04:28.441 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-26T09:04:28.453 INFO:teuthology.task.internal:Transferring archived files... 2024-03-26T09:04:28.455 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/remote/smithi007 2024-03-26T09:04:28.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T09:04:28.499 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/remote/smithi016 2024-03-26T09:04:28.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T09:04:28.541 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868/remote/smithi044 2024-03-26T09:04:28.541 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T09:04:28.584 INFO:teuthology.task.internal:Removing archive directory... 2024-03-26T09:04:28.584 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T09:04:28.586 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T09:04:28.589 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T09:04:28.634 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-26T09:04:28.649 INFO:teuthology.task.internal:Not uploading archives. 2024-03-26T09:04:28.649 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-26T09:04:28.661 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-26T09:04:28.661 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T09:04:28.664 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T09:04:28.666 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T09:04:28.676 INFO:teuthology.orchestra.run.smithi007.stdout: 265305 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 09:04 /home/ubuntu/cephtest 2024-03-26T09:04:28.678 INFO:teuthology.orchestra.run.smithi016.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 09:04 /home/ubuntu/cephtest 2024-03-26T09:04:28.689 INFO:teuthology.orchestra.run.smithi044.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 09:04 /home/ubuntu/cephtest 2024-03-26T09:04:28.690 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-26T09:04:28.703 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-26T09:04:28.728 INFO:teuthology.nuke:Checking targets against current locks 2024-03-26T09:04:28.754 DEBUG:teuthology.nuke:shortname: smithi007 2024-03-26T09:04:28.755 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T09:04:28.778 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:16:08.375697', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8Yj+ysP5HHtSGqUEdgYXZC0WFzC+C5FDN3ts7vCWs6KPW/SCyNIvhZoQo1RmS9vK7Q99G/3ZXSHI5yMIbFwlU='} 2024-03-26T09:04:28.783 DEBUG:teuthology.nuke:shortname: smithi016 2024-03-26T09:04:28.783 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T09:04:28.807 DEBUG:teuthology.nuke:shortname: smithi044 2024-03-26T09:04:28.807 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T09:04:28.812 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:16:08.377228', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFo3h6Erm2BsLSnUlOdEi8FvyVwBliJDXsDikSAuHaN01MyvUFkJnD3pi4pwpeHKiBULgLOpNad1W2MxmRrSoGQ='} 2024-03-26T09:04:28.832 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622868', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 08:16:08.378476', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA22+X8yiGA/b41MzM9FcrUOdzjl1oWbKgj4gUUFUoyx2SVfSkHRLc5yVXqwwscfXUSrM8EtcWcD9F2LPnBiBEk='} 2024-03-26T09:04:28.846 INFO:teuthology.orchestra.console.smithi007:Power off 2024-03-26T09:04:28.846 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T09:04:28.863 INFO:teuthology.orchestra.console.smithi016:Power off 2024-03-26T09:04:28.863 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T09:04:28.873 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-03-26T09:04:28.874 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:28.886 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2024-03-26T09:04:28.886 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:28.900 INFO:teuthology.orchestra.console.smithi044:Power off 2024-03-26T09:04:28.900 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T09:04:28.909 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-03-26T09:04:28.914 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-03-26T09:04:28.924 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2024-03-26T09:04:28.924 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:28.947 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-03-26T09:04:32.910 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:32.917 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:32.948 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:33.019 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-03-26T09:04:33.026 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-03-26T09:04:33.057 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-03-26T09:04:37.024 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:37.032 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:37.058 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:37.135 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-03-26T09:04:37.142 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2024-03-26T09:04:37.166 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-03-26T09:04:37.236 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-03-26T09:04:37.242 INFO:teuthology.orchestra.console.smithi016:Power off completed 2024-03-26T09:04:41.168 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T09:04:41.279 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-03-26T09:04:41.379 INFO:teuthology.orchestra.console.smithi044:Power off completed 2024-03-26T09:04:41.484 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} duration: 2412.3705554008484 failure_reason: '"2024-03-26T08:33:57.699377+0000 mon.a (mon.0) 189 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: mchangir success: false 2024-03-26T09:04:41.485 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T09:04:41.567 INFO:teuthology.run:FAIL